Class | Rouge::RegexLexer |
In: |
lib/rouge/regex_lexer.rb
|
Parent: | Lexer |
@abstract A stateful lexer that uses sets of regular expressions to tokenize a string. Most lexers are instances of RegexLexer.
MAX_NULL_SCANS | = | 5 | The number of successive scans permitted without consuming the input stream. If this is exceeded, the match fails. |
Delegate the lex to another lexer. The lex method will be called with `:continue` set to true, so that reset! will not be called. In this way, a single lexer can be repeatedly delegated to while maintaining its own internal state stack.
@param [lex] lexer
The lexer or lexer class to delegate to
@param [String] text
The text to delegate. This defaults to the last matched string.
This implements the lexer protocol, by yielding [token, value] pairs.
The process for lexing works as follows, until the stream is empty:
Yield a token.
@param tok
the token type
@param val
(optional) the string value to yield. If absent, this defaults to the entire last match.