token_get_all

    (PHP 4 >= 4.2.0, PHP 5, PHP 7, PHP 8)

    token_get_all — Split given source into PHP tokens

    Description

    token_get_all(string$code, int$flags = 0): array

    token_get_all() parses the given code string into PHP language tokens using the Zend engine's lexical scanner.

    For a list of parser tokens, see List of Parser Tokens, or use token_name() to translate a token value into its string representation.

    Parameters

    code

    The PHP source to parse.

    flags

    Valid flags:

    • TOKEN_PARSE - Recognises the ability to use reserved words in specific contexts.

    Return Values

    An array of token identifiers. Each individual token identifier is either a single character (i.e.: ;, ., >, !, etc...), or a three element array containing the token index in element 0, the string content of the original token in element 1 and the line number in element 2.

    Examples

    Example #1 token_get_all() example

    <?php
    $tokens     = token_get_all('<?php echo; ?>');

    foreach (    $tokens as $token) {
    if (    is_array($token)) {
    echo     "Line {$token[2]}: ", token_name($token[0]), " ('{$token[1]}')", PHP_EOL;
    }
    }
    ?>

    The above example will output something similar to:

    Line 1: T_OPEN_TAG ('<?php ') Line 1: T_ECHO ('echo') Line 1: T_WHITESPACE (' ') Line 1: T_CLOSE_TAG ('?>')

    Example #2 token_get_all() incorrect usage example

    <?php
    $tokens     = token_get_all('');

    foreach (    $tokens as $token) {
    if (    is_array($token)) {
    echo     "Line {$token[2]}: ", token_name($token[0]), " ('{$token[1]}')", PHP_EOL;
    }
    }
    ?>

    The above example will output something similar to:

    Line 1: T_INLINE_HTML ('')
    Note in the previous example that the string is parsed as T_INLINE_HTML rather than the expected T_COMMENT. This is because no open tag was used in the code provided. This would be equivalent to putting a comment outside of the PHP tags in a normal file.

    Example #3 token_get_all() on a class using a reserved word example

    <?php

    $source     = <<<'code'
    <?php

    class A
    {
    const PUBLIC = 1;
    }
    code;

    $tokens = token_get_all($source, TOKEN_PARSE);

    foreach (    $tokens as $token) {
    if (    is_array($token)) {
    echo     token_name($token[0]) , PHP_EOL;
    }
    }
    ?>

    The above example will output something similar to:

    T_OPEN_TAG T_WHITESPACE T_CLASS T_WHITESPACE T_STRING T_CONST T_WHITESPACE T_STRING T_LNUMBER
    Without the TOKEN_PARSE flag, the penultimate token (T_STRING) would have been T_PUBLIC.

    See Also

    To Top