Class Rouge::Lexer
In: lib/rouge/lexer.rb
Parent: Object

@abstract A lexer transforms text into a stream of `[token, chunk]` pairs.

Methods

Included Modules

Token::Tokens

Attributes

options  [R]  -*- instance methods -*- #

Public Class methods

Used to specify alternate names this lexer class may be found by.

@example

  class Erb < Lexer
    tag 'erb'
    aliases 'eruby', 'rhtml'
  end

  Lexer.find('eruby') # => Erb

@return a list of all lexers.

@abstract

Return a number between 0 and 1 indicating the likelihood that the text given should be lexed with this lexer. The default implementation returns 0. Values under 0.5 will only be used to disambiguate filename or mimetype matches.

@param [TextAnalyzer] text

  the text to be analyzed, with a couple of handy methods on it,
  like {TextAnalyzer#shebang?} and {TextAnalyzer#doctype?}

Specify or get a small demo string for this lexer

Specify or get the path name containing a small demo for this lexer (can be overriden by {demo}).

Specify or get this lexer‘s description.

Specify a list of filename globs associated with this lexer.

@example

  class Ruby < Lexer
    filenames '*.rb', '*.ruby', 'Gemfile', 'Rakefile'
  end

Given a string, return the correct lexer class.

Find a lexer, with fancy shiny features.

  • The string you pass can include CGI-style options
      Lexer.find_fancy('erb?parent=tex')
    
  • You can pass the special name ‘guess’ so we guess for you, and you can pass a second argument of the code to guess by
      Lexer.find_fancy('guess', "#!/bin/bash\necho Hello, world")
    

This is used in the Redcarpet plugin as well as Rouge‘s own markdown lexer for highlighting internal code blocks.

Guess which lexer to use based on a hash of info.

@option info :mimetype

  A mimetype to guess by

@option info :filename

  A filename to guess by

@option info :source

  The source itself, which, if guessing by mimetype or filename
  fails, will be searched for shebangs, <!DOCTYPE ...> tags, and
  other hints.

@see Lexer.analyze_text @see Lexer.guesses

Guess which lexer to use based on a hash of info.

This accepts the same arguments as Lexer.guess, but will never throw an error. It will return a (possibly empty) list of potential lexers to use.

Lexes `stream` with the given options. The lex is delegated to a new instance.

@see lex

Specify a list of mimetypes associated with this lexer.

@example

  class Html < Lexer
    mimetypes 'text/html', 'application/xhtml+xml'
  end

Create a new lexer with the given options. Individual lexers may specify extra options. The only current globally accepted option is `:debug`.

@option opts :debug

  Prints debug information to stdout.  The particular info depends
  on the lexer in question.  In regex lexers, this will log the
  state stack at the beginning of each step, along with each regex
  tried and each stream consumed.  Try it, it's pretty useful.

Used to specify or get the canonical name of this lexer class.

@example

  class MyLexer < Lexer
    tag 'foo'
  end

  MyLexer.tag # => 'foo'

  Lexer.find('foo') # => MyLexer

Specify or get this lexer‘s title. Meant to be human-readable.

Protected Class methods

Public Instance methods

Given a string, yield [token, chunk] pairs. If no block is given, an enumerator is returned.

@option opts :continue

  Continue the lex from the previous state (i.e. don't call #reset!)

@abstract

Called after each lex is finished. The default implementation is a noop.

@abstract

Yield `[token, chunk]` pairs, given a prepared input stream. This must be implemented.

@param [StringScanner] stream

  the stream

delegated to {Lexer.tag}

[Validate]