Tokens Overview Index
Go Up to Lexical Elements Index
This section contains C++ Token topics. Tokens are word-like units recognized by a language. The compiler recognizes six classes of tokens.
Here is the formal definition of a token:
- keyword
- identifier
- constant
- operator
- punctuator (also known as separator)
As the source code is scanned, tokens are extracted in such a way that the longest possible token from the character sequence is selected. For example, external would be parsed as a single identifier, rather than as the keyword extern followed by the identifier al.
See Token Pasting with ## Operator for a description of token pasting.