# dmd.tokens

Defines lexical `tokens`.

Specification: Tokens

#### Authors

Walter Bright

#### License

Boost License 1.0

Source: tokens.d

Documentation: https://dlang.org/phobos/dmd_tokens.html

Coverage: https://codecov.io/gh/dlang/dmd/src/master/src/dmd/tokens.d

- `Ckeywords`

  #### Declaration

  `static immutable TOK[TOK.max + 1] `Ckeywords`;`

  This is used to pick the C keywords out of the tokens. If it's not a C keyword, then it's an identifier.

- `Token`

  #### Declaration

  `struct `Token`;`

  - `setString`

    #### Declaration

    `nothrow void `setString`(const(char)* `ptr`, size_t `length`);`

    Set to contents of `ptr`[0..`length`]

    #### Parameters

    `const(char)* `ptr`` | pointer to string
    `size_t `length`` | `length` of string

  - `setString`

    #### Declaration

    `nothrow void `setString`(const ref OutBuffer `buf`);`

    Set to contents of `buf`

    #### Parameters

    `OutBuffer `buf`` | string (not zero terminated)

  - `setString`

    #### Declaration

    `nothrow void `setString`();`

    Set to empty string