Tokens Overview Index

From RAD Studio
Jump to: navigation, search

Go Up to Lexical Elements Index

This section contains C++ Token topics. Tokens are word-like units recognized by a language. The compiler recognizes six classes of tokens.

Here is the formal definition of a token:

  • keyword
  • identifier
  • constant
  • operator
  • punctuator (also known as separator)

As the source code is scanned, tokens are extracted in such a way that the longest possible token from the character sequence is selected. For example, external would be parsed as a single identifier, rather than as the keyword extern followed by the identifier al.

See Token Pasting with ## Operator for a description of token pasting.


See Also