Skip to content

Commit

Permalink
CSS tokenizer: bug fix
Browse files Browse the repository at this point in the history
Came across this while working on some of the Squiz CSS sniffs.

This code sample:
```css
.HelpWidgetType-new-bug-title {}
```

Was tokenized as:
```
0 :: L001 :: C1 :: T_OPEN_TAG :: (0) ::
1 :: L001 :: C1 :: T_STRING_CONCAT :: (1) :: .
2 :: L001 :: C2 :: T_STRING :: (14) :: HelpWidgetType
3 :: L001 :: C16 :: T_MINUS :: (1) :: -
4 :: L001 :: C17 :: T_NEW :: (3) :: new
5 :: L001 :: C20 :: T_STRING :: (10) :: -bug-title
6 :: L001 :: C30 :: T_WHITESPACE :: (1) ::
7 :: L001 :: C31 :: T_OPEN_CURLY_BRACKET :: (1) :: {
8 :: L001 :: C32 :: T_CLOSE_CURLY_BRACKET :: (1) :: }
```

...while I would expect the class identifier to be tokenized as one T_STRING `HelpWidgetType-new-bug-title` instead.

This commit fixes that and the code sample is now tokenized as:
```
0 :: L001 :: C1 :: T_OPEN_TAG :: (0) ::
1 :: L001 :: C1 :: T_STRING_CONCAT :: (1) :: .
2 :: L001 :: C2 :: T_STRING :: (28) :: HelpWidgetType-new-bug-title
3 :: L001 :: C30 :: T_WHITESPACE :: (1) ::
4 :: L001 :: C31 :: T_OPEN_CURLY_BRACKET :: (1) :: {
5 :: L001 :: C32 :: T_CLOSE_CURLY_BRACKET :: (1) :: }
```

No additional unit tests needed as the existing unit test in the `Squiz/Tests/CSS/SemicolonspacingUnitTest.css` file already covers this (as that's how I came across it).
  • Loading branch information
jrfnl committed Nov 27, 2018
1 parent ffefbcd commit ad27eb4
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions src/Tokenizers/CSS.php
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ public function tokenize($string)
|| $token['code'] === T_FOREACH
|| $token['code'] === T_WHILE
|| $token['code'] === T_DEC
|| $token['code'] === T_NEW
) {
$token['type'] = 'T_STRING';
$token['code'] = T_STRING;
Expand Down

0 comments on commit ad27eb4

Please sign in to comment.