Uses of Interface
org.antlr.runtime.Token
Packages that use Token
Package
Description
-
Uses of Token in org.antlr.analysis
Fields in org.antlr.analysis with type parameters of type TokenModifier and TypeFieldDescriptionDecisionProbe.stateToIncompletelyCoveredAltsMap
Tracks alts insufficiently covered.Methods in org.antlr.analysis that return types with arguments of type TokenModifier and TypeMethodDescriptionMachineProbe.getGrammarLocationsForInputSequence
(List<Set<NFAState>> nfaStates, List<IntSet> labels) Given an alternative associated with a DFA state, return the list of tokens (from grammar) associated with path through NFA following the labels sequence.DecisionProbe.getIncompletelyCoveredAlts
(DFAState d) Return a list of alts whose predicate context was insufficient to resolve a nondeterminism for state d.Method parameters in org.antlr.analysis with type arguments of type Token -
Uses of Token in org.antlr.codegen
Methods in org.antlr.codegen with parameters of type TokenModifier and TypeMethodDescriptionvoid
CodeGenerator.issueInvalidAttributeError
(String x, String y, Rule enclosingRule, Token actionToken, int outerAltNum) void
CodeGenerator.issueInvalidAttributeError
(String x, Rule enclosingRule, Token actionToken, int outerAltNum) void
CodeGenerator.issueInvalidScopeError
(String x, String y, Rule enclosingRule, Token actionToken, int outerAltNum) Python3Target.postProcessAction
(List<Object> chunks, Token actionToken) PythonTarget.postProcessAction
(List<Object> chunks, Token actionToken) Target.postProcessAction
(List<Object> chunks, Token actionToken) Give target a chance to do some postprocessing on actions.org.stringtemplate.v4.ST
CodeGenerator.translateTemplateConstructor
(String ruleName, int outerAltNum, Token actionToken, String templateActionText) Given a template constructor action like %foo(a={...}) in an action, translate it to the appropriate template constructor from the templateLib. -
Uses of Token in org.antlr.grammar.v3
Fields in org.antlr.grammar.v3 declared as TokenModifier and TypeFieldDescription(package private) Token
ActionAnalysis.actionToken
(package private) Token
ActionTranslator.actionToken
ANTLRParser.GrammarASTErrorNode.start
ANTLRParser.GrammarASTErrorNode.stop
Methods in org.antlr.grammar.v3 that return TokenModifier and TypeMethodDescriptionprivate final Token
ANTLRParser.LT
(int k) ActionAnalysis.nextToken()
ActionTranslator.nextToken()
ANTLRLexer.nextToken()
Methods in org.antlr.grammar.v3 with parameters of type TokenModifier and TypeMethodDescriptionANTLRParser.grammar_Adaptor.errorNode
(TokenStream input, Token start, Token stop, RecognitionException e) Constructors in org.antlr.grammar.v3 with parameters of type TokenModifierConstructorDescriptionActionTranslator
(CodeGenerator generator, String ruleName, Token actionToken, int outerAltNum) GrammarASTErrorNode
(TokenStream input, Token start, Token stop, RecognitionException e) -
Uses of Token in org.antlr.runtime
Classes in org.antlr.runtime that implement TokenModifier and TypeClassDescriptionclass
A Token object like we'd use in ANTLR 2.x; has an actual string created and associated with this object.class
Fields in org.antlr.runtime declared as TokenModifier and TypeFieldDescriptionstatic final Token
Token.EOF_TOKEN
static final Token
Token.INVALID_TOKEN
static final Token
Token.SKIP_TOKEN
In an action, a lexer rule can set token to this SKIP_TOKEN and ANTLR will avoid creating a token for this symbol and try to fetch another.ParserRuleReturnScope.start
ParserRuleReturnScope.stop
RecognitionException.token
The current Token when an error occurred.RecognizerSharedState.token
The goal of all lexer rules/methods is to create a token object.Fields in org.antlr.runtime with type parameters of type TokenModifier and TypeFieldDescriptionBufferedTokenStream.tokens
Record every single token pulled from the source so we can reproduce chunks of it later.LegacyCommonTokenStream.tokens
Record every single token pulled from the source so we can reproduce chunks of it later.Methods in org.antlr.runtime that return TokenModifier and TypeMethodDescriptionLexer.emit()
The standard method called to automatically emit a token at the outermost lexical rule.BufferedTokenStream.get
(int i) LegacyCommonTokenStream.get
(int i) Return absolute token i; ignore which channel the tokens are on; that is, count all tokens not just on-channel tokens.TokenStream.get
(int i) Get a token at an absolute index i; 0..n-1.UnbufferedTokenStream.get
(int i) Lexer.getEOFToken()
Returns the EOF token (default), if you need to return a custom token instead override this method.UnwantedTokenException.getUnexpectedToken()
protected Token
BufferedTokenStream.LB
(int k) protected Token
CommonTokenStream.LB
(int k) protected Token
LegacyCommonTokenStream.LB
(int k) Look backwards k tokens on-channel tokensBufferedTokenStream.LT
(int k) CommonTokenStream.LT
(int k) LegacyCommonTokenStream.LT
(int k) Get the ith token from the current position 1..n where k=1 is the first symbol of lookahead.TokenStream.LT
(int k) Get Token at current input pointer + i ahead where i=1 is next Token.UnbufferedTokenStream.nextElement()
Lexer.nextToken()
Return a token from this source; i.e., match a token on the char stream.TokenSource.nextToken()
Return a Token object from your input stream (usually a CharStream).Methods in org.antlr.runtime that return types with arguments of type TokenModifier and TypeMethodDescriptionBufferedTokenStream.get
(int start, int stop) Get all tokens from start..stop inclusivelyLegacyCommonTokenStream.get
(int start, int stop) Get all tokens from start..stop inclusivelyBufferedTokenStream.getTokens()
BufferedTokenStream.getTokens
(int start, int stop) BufferedTokenStream.getTokens
(int start, int stop, int ttype) Given a start and stop index, return a List of all tokens in the token type BitSet.LegacyCommonTokenStream.getTokens()
LegacyCommonTokenStream.getTokens
(int start, int stop) LegacyCommonTokenStream.getTokens
(int start, int stop, int ttype) Given a start and stop index, return a List of all tokens in the token type BitSet.Methods in org.antlr.runtime with parameters of type TokenModifier and TypeMethodDescriptionvoid
void
void
void
Currently does not support multiple emits per nextToken invocation for efficiency reasons.BaseRecognizer.getTokenErrorDisplay
(Token t) How should a token be displayed in an error message? The default is to display just the text, but during development you might want to have a lot of information spit out.void
TokenRewriteStream.insertAfter
(String programName, Token t, Object text) void
TokenRewriteStream.insertAfter
(Token t, Object text) void
TokenRewriteStream.insertBefore
(String programName, Token t, Object text) void
TokenRewriteStream.insertBefore
(Token t, Object text) boolean
void
void
void
Because the user is not required to use a token with an index stored in it, we must provide a means for two token objects themselves to indicate the start/end location.Method parameters in org.antlr.runtime with type arguments of type TokenModifier and TypeMethodDescriptionA convenience method for use most often with template rewrites.Constructors in org.antlr.runtime with parameters of type Token -
Uses of Token in org.antlr.runtime.debug
Classes in org.antlr.runtime.debug that implement TokenFields in org.antlr.runtime.debug declared as TokenFields in org.antlr.runtime.debug with type parameters of type TokenMethods in org.antlr.runtime.debug that return TokenMethods in org.antlr.runtime.debug with parameters of type TokenModifier and TypeMethodDescriptionvoid
DebugTreeAdaptor.becomeRoot
(Token newRoot, Object oldRoot) void
BlankDebugEventListener.consumeHiddenToken
(Token token) void
DebugEventHub.consumeHiddenToken
(Token token) void
DebugEventListener.consumeHiddenToken
(Token t) An off-channel input token was consumed.void
DebugEventRepeater.consumeHiddenToken
(Token token) void
DebugEventSocketProxy.consumeHiddenToken
(Token t) void
ParseTreeBuilder.consumeHiddenToken
(Token token) void
Profiler.consumeHiddenToken
(Token token) void
BlankDebugEventListener.consumeToken
(Token token) void
DebugEventHub.consumeToken
(Token token) void
DebugEventListener.consumeToken
(Token t) An input token was consumed; matched by any kind of element.void
DebugEventRepeater.consumeToken
(Token token) void
DebugEventSocketProxy.consumeToken
(Token t) void
ParseTreeBuilder.consumeToken
(Token token) void
Profiler.consumeToken
(Token token) void
BlankDebugEventListener.createNode
(Object node, Token token) void
DebugEventHub.createNode
(Object node, Token token) void
DebugEventListener.createNode
(Object node, Token token) Announce a new node built from an existing token.void
DebugEventRepeater.createNode
(Object node, Token token) void
DebugEventSocketProxy.createNode
(Object node, Token token) void
TraceDebugEventListener.createNode
(Object node, Token token) DebugTreeAdaptor.errorNode
(TokenStream input, Token start, Token stop, RecognitionException e) void
void
void
Somebody (anybody) looked ahead.void
void
void
Track refs to lookahead if in a fixed/nonfixed decision.protected String
DebugEventSocketProxy.serializeToken
(Token t) void
DebugTreeAdaptor.setTokenBoundaries
(Object t, Token startToken, Token stopToken) -
Uses of Token in org.antlr.runtime.tree
Fields in org.antlr.runtime.tree declared as TokenModifier and TypeFieldDescriptionCommonErrorNode.start
CommonErrorNode.stop
CommonTree.token
A single token is the payloadFields in org.antlr.runtime.tree with type parameters of type TokenMethods in org.antlr.runtime.tree that return TokenModifier and TypeMethodDescriptionabstract Token
BaseTreeAdaptor.createToken
(int tokenType, String text) Tell me how to create a token for use with imaginary token nodes.abstract Token
BaseTreeAdaptor.createToken
(Token fromToken) Tell me how to create a token for use with imaginary token nodes.CommonTreeAdaptor.createToken
(int tokenType, String text) Tell me how to create a token for use with imaginary token nodes.CommonTreeAdaptor.createToken
(Token fromToken) Tell me how to create a token for use with imaginary token nodes.CommonTree.getToken()
What is the Token associated with this node? If you are not using CommonTree, then you must override this in your own adaptor.Return the token object from which this node was created.RewriteRuleTokenStream.nextToken()
Methods in org.antlr.runtime.tree with parameters of type TokenModifier and TypeMethodDescriptionBaseTreeAdaptor.becomeRoot
(Token newRoot, Object oldRoot) TreeAdaptor.becomeRoot
(Token newRoot, Object oldRoot) Create a node for newRoot make it the root of oldRoot.Create a new node derived from a token, with a new token type.Same as create(tokenType,fromToken) except set the text too.Create a tree node from Token object; for CommonTree type trees, then the token just becomes the payload.abstract Token
BaseTreeAdaptor.createToken
(Token fromToken) Tell me how to create a token for use with imaginary token nodes.CommonTreeAdaptor.createToken
(Token fromToken) Tell me how to create a token for use with imaginary token nodes.BaseTreeAdaptor.errorNode
(TokenStream input, Token start, Token stop, RecognitionException e) create tree node that holds the start and stop tokens associated with an error.TreeAdaptor.errorNode
(TokenStream input, Token start, Token stop, RecognitionException e) Return a tree node representing an error.void
CommonTreeAdaptor.setTokenBoundaries
(Object t, Token startToken, Token stopToken) Track start/stop token for subtree root created for a rule.void
TreeAdaptor.setTokenBoundaries
(Object t, Token startToken, Token stopToken) Where are the bounds in the input token stream for this node and all children? Each rule that creates AST nodes will call this method right before returning.Constructors in org.antlr.runtime.tree with parameters of type TokenModifierConstructorDescriptionCommonErrorNode
(TokenStream input, Token start, Token stop, RecognitionException e) CommonTree
(Token t) TreePattern
(Token payload) WildcardTreePattern
(Token payload) -
Uses of Token in org.antlr.tool
Fields in org.antlr.tool declared as TokenModifier and TypeFieldDescriptionAttributeScope.derivedFromToken
This scope is associated with which input token (for error handling)?Grammar.LabelElementPair.label
GrammarSemanticsMessage.offendingToken
Most of the time, we'll have a token such as an undefined rule ref and so this will be set.GrammarSyntaxMessage.offendingToken
Most of the time, we'll have a token and so this will be set.Fields in org.antlr.tool with type parameters of type TokenModifier and TypeFieldDescriptionGrammarInsufficientPredicatesMessage.altToLocations
Grammar.tokenIDRefs
The unique set of all token ID references in any ruleMethods in org.antlr.tool that return TokenMethods in org.antlr.tool with parameters of type TokenModifier and TypeMethodDescriptionprotected void
NameSpaceChecker.checkForLabelConflict
(Rule r, Token label) Make sure a label doesn't conflict with another symbol.boolean
NameSpaceChecker.checkForLabelTypeMismatch
(Rule r, Token label, int type) If type of previous label differs from new label's type, that's an error.Grammar.createParameterScope
(String ruleName, Token argAction) Grammar.createReturnScope
(String ruleName, Token retAction) Grammar.createRuleScope
(String ruleName, Token scopeAction) Grammar.defineGlobalScope
(String name, Token scopeAction) protected void
Grammar.defineLabel
(Rule r, Token label, GrammarAST element, int type) Define a label defined in a rule r; check the validity then ask the Rule object to actually define it.void
Rule.defineLabel
(Token label, GrammarAST elementRef, int type) void
Grammar.defineLexerRuleFoundInParser
(Token ruleToken, GrammarAST ruleAST) void
Grammar.defineRule
(Token ruleToken, String modifier, Map<String, Object> options, GrammarAST tree, GrammarAST argActionAST, int numAlts) Define a new rule.void
Grammar.defineRuleListLabel
(String ruleName, Token label, GrammarAST element) void
Grammar.defineRuleRefLabel
(String ruleName, Token label, GrammarAST ruleRef) void
Grammar.defineTokenListLabel
(String ruleName, Token label, GrammarAST element) void
Grammar.defineTokenRefLabel
(String ruleName, Token label, GrammarAST tokenRef) void
Grammar.defineWildcardTreeLabel
(String ruleName, Token label, GrammarAST tokenRef) void
Grammar.defineWildcardTreeListLabel
(String ruleName, Token label, GrammarAST tokenRef) LeftRecursiveRuleAnalyzer.getNamesFromArgAction
(Token t) static void
ErrorManager.grammarError
(int msgID, Grammar g, Token token) static void
ErrorManager.grammarError
(int msgID, Grammar g, Token token, Object arg) static void
ErrorManager.grammarError
(int msgID, Grammar g, Token token, Object arg, Object arg2) static void
ErrorManager.grammarWarning
(int msgID, Grammar g, Token token) static void
ErrorManager.grammarWarning
(int msgID, Grammar g, Token token, Object arg) static void
ErrorManager.grammarWarning
(int msgID, Grammar g, Token token, Object arg, Object arg2) void
GrammarAST.initialize
(Token token) Save the option key/value pair and process it; return the key or null if invalid option.Save the option key/value pair and process it; return the key or null if invalid option.void
Grammar.setOptions
(Map<String, Object> options, Token optionsStartToken) void
Rule.setOptions
(Map<String, Object> options, Token optionsStartToken) void
GrammarAST.setTokenBoundaries
(Token startToken, Token stopToken) Track start/stop token for subtree root created for a rule.static void
ErrorManager.syntaxError
(int msgID, Grammar grammar, Token token, Object arg, RecognitionException re) Method parameters in org.antlr.tool with type arguments of type TokenModifier and TypeMethodDescriptionstatic void
ErrorManager.insufficientPredicates
(DecisionProbe probe, DFAState d, Map<Integer, Set<Token>> altToUncoveredLocations) Constructors in org.antlr.tool with parameters of type TokenModifierConstructorDescriptionAttributeScope
(String name, Token derivedFromToken) AttributeScope
(Grammar grammar, String name, Token derivedFromToken) GrammarAST
(Token token) GrammarSemanticsMessage
(int msgID, Grammar g, Token offendingToken) GrammarSemanticsMessage
(int msgID, Grammar g, Token offendingToken, Object arg) GrammarSemanticsMessage
(int msgID, Grammar g, Token offendingToken, Object arg, Object arg2) GrammarSyntaxMessage
(int msgID, Grammar grammar, Token offendingToken, Object arg, RecognitionException exception) GrammarSyntaxMessage
(int msgID, Grammar grammar, Token offendingToken, RecognitionException exception) LabelElementPair
(Token label, GrammarAST elementRef) RuleLabelScope
(Rule referencedRule, Token actionToken) Constructor parameters in org.antlr.tool with type arguments of type TokenModifierConstructorDescriptionGrammarInsufficientPredicatesMessage
(DecisionProbe probe, DFAState problemState, Map<Integer, Set<Token>> altToLocations)