Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tokenizers/Comment: add tests + fix two edge case bugs #494

Merged
merged 4 commits into from
May 18, 2024
Merged

Tokenizers/Comment: add tests + fix two edge case bugs #494

merged 4 commits into from
May 18, 2024

Commits on May 18, 2024

  1. Tokenizers/Comment: add tests 

    The `Tokenizers\Comment` class did not have any tests associated with it. This commit fixes that and documents the existing behaviour.

Note: code coverage is as high as it can go, but not 100%. The reason for this, is the tokenizer debug statements, which are conditional on a verbosity flag, which is turned off for the tests.

Loosely related to 484.
    @jrfnl
    jrfnl committed May 18, 2024
    Configuration menu
    Copy the full SHA
    d96c7d6 View commit details
    Browse the repository at this point in the history

  2. Tokenizers/PHP: bug fix - empty block comment 

    This commit fixes an edge case tokenizer bug, where a - completely empty, not even whitespace - _block comment_, would be tokenized as a docblock.

Without this commit, the `/**/` code snippet was tokenized as:
```
  8 | L07 | C  1 | CC 0 | ( 0) | T_DOC_COMMENT_OPEN_TAG     | [  4]: /**/
  9 | L07 | C  5 | CC 0 | ( 0) | T_DOC_COMMENT_CLOSE_TAG    | [  0]:
```

With the fix applied, it will be tokenized as:
```
  8 | L07 | C  1 | CC 0 | ( 0) | T_COMMENT                  | [  4]: /**/
```
    @jrfnl
    jrfnl committed May 18, 2024
    Configuration menu
    Copy the full SHA
    c54cb10 View commit details
    Browse the repository at this point in the history

  3. Tokenizers/Comment: bug fix - empty docblock 

    This commit fixes an edge case tokenizer bug, where a - completely empty, not even whitespace - _DocBlock_, would not be tokenized correctly.

Without this commit, the `/***/` code snippet was tokenized as:
```
 13 | L10 | C  1 | CC 0 | ( 0) | T_DOC_COMMENT_OPEN_TAG     | [  5]: /***/
 14 | L10 | C  6 | CC 0 | ( 0) | T_DOC_COMMENT_CLOSE_TAG    | [  0]:
```

With the fix applied, it will be tokenized as:
```
 13 | L10 | C  1 | CC 0 | ( 0) | T_DOC_COMMENT_OPEN_TAG     | [  3]: /**
 14 | L10 | C  4 | CC 0 | ( 0) | T_DOC_COMMENT_CLOSE_TAG    | [  2]: */
```
    @jrfnl
    jrfnl committed May 18, 2024
    Configuration menu
    Copy the full SHA
    252675c View commit details
    Browse the repository at this point in the history

  4. Tokenizers/Comment: minor tweaks 

    Girlscouting.

* Remove an unnecessary interim variable.
* Add a comment clarifying certain code.
* Specify the array format in the `@return` tags.
    @jrfnl
    jrfnl committed May 18, 2024
    Configuration menu
    Copy the full SHA
    bef6fff View commit details
    Browse the repository at this point in the history