The token manager consumes the maximum number of characters from the input stream possible that match one of these regular expressions. Token matchingĪll regular expressions in the current lexical state are considered as potential match candidates. There are four types of regular expressions: SKIP, MORE, TOKEN, and SPECIAL_TOKEN.Īll regular expressions that occur as expansion units in the grammar are considered to be in the DEFAULT lexical state and their order of occurrence is determined by their position in the grammar file. The starting lexical state can also be specified as a parameter while constructing a token manager object.Įach lexical state contains an ordered list of regular expressions - the order is derived from the order of occurrence in the input file. When the token manager is initialized, it starts off in the DEFAULT state, by default. The generated token manager is at any moment in one of these lexical states. There is a standard lexical state called DEFAULT. The JavaCC lexical specification is organized into a set of lexical states, each of which is named with a unique identifier. Access to class level declarations within lexical actions.It covers lexical states, lexical actions, and the use of SPECIAL_TOKENs. This tutorial describes the JavaCC token manager.
0 Comments
Leave a Reply. |