de.sciss.syntaxpane.lexers

Class DefaultJFlexLexer

java.lang.Object

de.sciss.syntaxpane.lexers.DefaultJFlexLexer

All Implemented Interfaces:

Lexer

Direct Known Subclasses:

BashLexer, CLexer, ClojureLexer, CppLexer, DOSBatchLexer, GroovyLexer, JavaLexer, JavaScriptLexer, JFlexLexer, LuaLexer, MarkdownLexer, PropertiesLexer, PythonLexer, RubyLexer, ScalaLexer, SqlLexer, TALLexer, XHTMLLexer, XmlLexer, XPathLexer

public abstract class DefaultJFlexLexer
extends java.lang.Object
implements Lexer

This is a default, and abstract implementation of a Lexer using JFLex with some utility methods that Lexers can implement.

Field Summary

Fields

Modifier and Type	Field and Description
protected int	offset
protected int	tokenLength
protected int	tokenStart

Constructor Summary

Constructors

Constructor and Description
DefaultJFlexLexer()

Method Summary

Modifier and Type	Method and Description
void	parse(javax.swing.text.Segment segment, int ofst, java.util.List<Token> tokens) The DefaultJFlexLexer simply calls the yylex method of a JFlex compatible Lexer and adds the tokens obtained to an ArrayList.
protected Token	token(TokenType type) Create and return a Token of given type.
protected Token	token(TokenType type, int pairValue) Create and return a Token of given type and pairValue.
protected Token	token(TokenType type, int start, int length) Create and return a Token of given type from start with length offset is added to start
protected Token	token(TokenType type, int tStart, int tLength, int newStart, int newLength) Helper method to create and return a new Token from of TokenType tokenStart and tokenLength will be modified to the newStart and newLength params
abstract int	yychar() Return the char number from beginning of input stream.
abstract char	yycharat(int pos) Returns the character at position pos from the matched text.
abstract int	yylength() Returns the length of the matched text region.
abstract Token	yylex() This is called to return the next Token from the Input Reader
abstract void	yyreset(java.io.Reader reader) This will be called to reset the the lexer.
abstract java.lang.String	yytext() Returns the text matched by the current regular expression.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

tokenStart

protected int tokenStart

tokenLength

protected int tokenLength

offset

protected int offset

Constructor Detail

DefaultJFlexLexer

public DefaultJFlexLexer()

Method Detail

token

protected Token token(TokenType type,
                      int tStart,
                      int tLength,
                      int newStart,
                      int newLength)

Helper method to create and return a new Token from of TokenType tokenStart and tokenLength will be modified to the newStart and newLength params

token

protected Token token(TokenType type,
                      int start,
                      int length)

Create and return a Token of given type from start with length offset is added to start

token

protected Token token(TokenType type)

Create and return a Token of given type. start is obtained from yychar() and length from yylength() offset is added to start

token

protected Token token(TokenType type,
                      int pairValue)

Create and return a Token of given type and pairValue. start is obtained from yychar() and length from yylength() offset is added to start

parse

public void parse(javax.swing.text.Segment segment,
                  int ofst,
                  java.util.List<Token> tokens)

The DefaultJFlexLexer simply calls the yylex method of a JFlex compatible Lexer and adds the tokens obtained to an ArrayList.

Specified by:

parse in interface Lexer

Parameters:

segment - Text to parse.

ofst - offset to add to start of each token (useful for nesting)

tokens - List of Tokens to be added. This is done so that the caller creates the appropriate List implementation and size. The parse method just adds to the list

yyreset

public abstract void yyreset(java.io.Reader reader)

This will be called to reset the the lexer. This is created automatically by JFlex.

yylex

public abstract Token yylex()
                     throws java.io.IOException

This is called to return the next Token from the Input Reader

Returns:

next token, or null if no more tokens.

Throws:

java.io.IOException

yycharat

public abstract char yycharat(int pos)

Returns the character at position pos from the matched text. It is equivalent to yytext().charAt(pos), but faster

Parameters:

pos - the position of the character to fetch. A value from 0 to yylength()-1.

Returns:

the character at position pos

yylength

public abstract int yylength()

Returns the length of the matched text region. This method is automatically implemented by JFlex lexers

yytext

public abstract java.lang.String yytext()

Returns the text matched by the current regular expression. This method is automatically implemented by JFlex lexers

yychar

public abstract int yychar()

Return the char number from beginning of input stream. This is NOT implemented by JFlex, so the code must be added to create this and return the private yychar field