OpenQuizz
Une application de gestion des contenus pédagogiques
Lexer Class Reference
Inheritance diagram for Lexer:
Collaboration diagram for Lexer:

Public Member Functions

def __init__ (self, environment)
 
def tokenize (self, source, name=None, filename=None, state=None)
 
def wrap (self, stream, name=None, filename=None)
 
def tokeniter (self, source, name, filename=None, state=None)
 

Data Fields

 lstrip_unless_re
 
 newline_sequence
 
 keep_trailing_newline
 
 rules
 

Detailed Description

Class that implements a lexer for a given environment. Automatically
created by the environment class, usually you don't have to do that.

Note that the lexer is not automatically bound to an environment.
Multiple environments can share the same lexer.

Constructor & Destructor Documentation

◆ __init__()

def __init__ (   self,
  environment 
)

Member Function Documentation

◆ tokeniter()

def tokeniter (   self,
  source,
  name,
  filename = None,
  state = None 
)
This method tokenizes the text and returns the tokens in a
generator.  Use this method if you just want to tokenize a template.

◆ tokenize()

def tokenize (   self,
  source,
  name = None,
  filename = None,
  state = None 
)
Calls tokeniter + tokenize and wraps it in a token stream.

◆ wrap()

def wrap (   self,
  stream,
  name = None,
  filename = None 
)
This is called with the stream as returned by `tokenize` and wraps
every token in a :class:`Token` and converts the value.

Field Documentation

◆ keep_trailing_newline

keep_trailing_newline

◆ lstrip_unless_re

lstrip_unless_re

◆ newline_sequence

newline_sequence

◆ rules

rules

The documentation for this class was generated from the following file: