Class TokenScanner

java.lang.Object

org.aspectj.org.eclipse.jdt.internal.core.dom.rewrite.TokenScanner

public class TokenScanner
extends Object

Wraps a scanner and offers convenient methods for finding tokens

Field Summary

Fields

Modifier and Type	Field	Description
static final int	DOCUMENT_ERROR
static final int	END_OF_FILE
static final int	LEXICAL_ERROR

Constructor Summary

Constructors

Constructor	Description
TokenScanner(Scanner scanner)	Creates a TokenScanner

Method Summary

Modifier and Type	Method	Description
static org.eclipse.core.runtime.IStatus	createError(int code, String message, Throwable throwable)
int	getCurrentEndOffset()
int	getCurrentLength()
int	getCurrentStartOffset()
int	getNextEndOffset(int offset, boolean ignoreComments)	Reads the next token from the given offset and returns the offset after the token.
int	getNextStartOffset(int offset, boolean ignoreComments)	Reads the next token from the given offset and returns the start offset of the token.
int	getPreviousTokenEndOffset(TerminalToken token, int startOffset)	Reads from the given offset until a token is reached and returns the offset after the previous token.
Scanner	getScanner()	Returns the wrapped scanner
int	getTokenEndOffset(TerminalToken token, int startOffset)	Reads from the given offset until a token is reached and returns the offset after the token.
int	getTokenStartOffset(TerminalToken token, int startOffset)	Reads from the given offset until a token is reached and returns the start offset of the token.
static boolean	isComment(TerminalToken token)
static boolean	isModifier(TerminalToken token)
TerminalToken	readNext(boolean ignoreComments)	Reads the next token.
TerminalToken	readNext(int offset, boolean ignoreComments)	Reads the next token from the given offset.
void	readToToken(TerminalToken tok)	Reads until a token is reached.
void	readToToken(TerminalToken tok, int offset)	Reads until a token is reached, starting from the given offset.
void	setOffset(int offset)	Sets the scanner offset to the given offset.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

END_OF_FILE

public static final int END_OF_FILE

See Also:

• Constant Field Values

LEXICAL_ERROR

public static final int LEXICAL_ERROR

See Also:

• Constant Field Values

DOCUMENT_ERROR

public static final int DOCUMENT_ERROR

See Also:

• Constant Field Values

Constructor Details

TokenScanner

public TokenScanner(Scanner scanner)

Creates a TokenScanner

Parameters:

scanner - The scanner to be wrapped

Method Details

getScanner

public Scanner getScanner()

Returns the wrapped scanner

Returns:

IScanner

setOffset

public void setOffset(int offset)

Sets the scanner offset to the given offset.

Parameters:

offset - The offset to set

getCurrentEndOffset

public int getCurrentEndOffset()

Returns:

Returns the offset after the current token

getCurrentStartOffset

public int getCurrentStartOffset()

Returns:

Returns the start offset of the current token

getCurrentLength

public int getCurrentLength()

Returns:

Returns the length of the current token

readNext

public TerminalToken readNext(boolean ignoreComments)
                       throws org.eclipse.core.runtime.CoreException

Reads the next token.

Parameters:

ignoreComments - If set, comments will be overread

Returns:

Return the token id.

Throws:

org.eclipse.core.runtime.CoreException - Thrown when the end of the file has been reached (code END_OF_FILE) or a lexical error was detected while scanning (code LEXICAL_ERROR)

readNext

public TerminalToken readNext(int offset,
 boolean ignoreComments)
                       throws org.eclipse.core.runtime.CoreException

Reads the next token from the given offset.

Parameters:

offset - The offset to start reading from.

ignoreComments - If set, comments will be overread.

Returns:

Returns the token id.

Throws:

org.eclipse.core.runtime.CoreException - Thrown when the end of the file has been reached (code END_OF_FILE) or a lexical error was detected while scanning (code LEXICAL_ERROR)

getNextStartOffset

public int getNextStartOffset(int offset,
 boolean ignoreComments)
                       throws org.eclipse.core.runtime.CoreException

Reads the next token from the given offset and returns the start offset of the token.

Parameters:

offset - The offset to start reading from.

ignoreComments - If set, comments will be overread

Returns:

Returns the start position of the next token.

Throws:

org.eclipse.core.runtime.CoreException - Thrown when the end of the file has been reached (code END_OF_FILE) or a lexical error was detected while scanning (code LEXICAL_ERROR)

getNextEndOffset

public int getNextEndOffset(int offset,
 boolean ignoreComments)
                     throws org.eclipse.core.runtime.CoreException

Reads the next token from the given offset and returns the offset after the token.

Parameters:

offset - The offset to start reading from.

ignoreComments - If set, comments will be overread

Returns:

Returns the start position of the next token.

Throws:

org.eclipse.core.runtime.CoreException - Thrown when the end of the file has been reached (code END_OF_FILE) or a lexical error was detected while scanning (code LEXICAL_ERROR)

readToToken

public void readToToken(TerminalToken tok)
                 throws org.eclipse.core.runtime.CoreException

Reads until a token is reached.

Parameters:

tok - The token to read to.

Throws:

org.eclipse.core.runtime.CoreException - Thrown when the end of the file has been reached (code END_OF_FILE) or a lexical error was detected while scanning (code LEXICAL_ERROR)

readToToken

public void readToToken(TerminalToken tok,
 int offset)
                 throws org.eclipse.core.runtime.CoreException

Reads until a token is reached, starting from the given offset.

Parameters:

tok - The token to read to.

offset - The offset to start reading from.

Throws:

org.eclipse.core.runtime.CoreException - Thrown when the end of the file has been reached (code END_OF_FILE) or a lexical error was detected while scanning (code LEXICAL_ERROR)

getTokenStartOffset

public int getTokenStartOffset(TerminalToken token,
 int startOffset)
                        throws org.eclipse.core.runtime.CoreException

Reads from the given offset until a token is reached and returns the start offset of the token.

Parameters:

token - The token to be found.

startOffset - The offset to start reading from.

Returns:

Returns the start position of the found token.

Throws:

org.eclipse.core.runtime.CoreException - Thrown when the end of the file has been reached (code END_OF_FILE) or a lexical error was detected while scanning (code LEXICAL_ERROR)

getTokenEndOffset

public int getTokenEndOffset(TerminalToken token,
 int startOffset)
                      throws org.eclipse.core.runtime.CoreException

Reads from the given offset until a token is reached and returns the offset after the token.

Parameters:

token - The token to be found.

startOffset - Offset to start reading from

Returns:

Returns the end position of the found token.

Throws:

org.eclipse.core.runtime.CoreException - Thrown when the end of the file has been reached (code END_OF_FILE) or a lexical error was detected while scanning (code LEXICAL_ERROR)

getPreviousTokenEndOffset

public int getPreviousTokenEndOffset(TerminalToken token,
 int startOffset)
                              throws org.eclipse.core.runtime.CoreException

Reads from the given offset until a token is reached and returns the offset after the previous token.

Parameters:

token - The token to be found.

startOffset - The offset to start scanning from.

Returns:

Returns the end offset of the token previous to the given token.

Throws:

org.eclipse.core.runtime.CoreException - Thrown when the end of the file has been reached (code END_OF_FILE) or a lexical error was detected while scanning (code LEXICAL_ERROR)

isComment

public static boolean isComment(TerminalToken token)

isModifier

public static boolean isModifier(TerminalToken token)

createError

public static org.eclipse.core.runtime.IStatus createError(int code,
 String message,
 Throwable throwable)