Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update TypeScript lexer to allow nested generics #1002

Merged
merged 2 commits into from
Sep 21, 2024

Conversation

fredrare
Copy link
Contributor

#425 shows that generics are not correctly identified in general, but they are rather being treated as JSX elements. I proposed a simple solution in the comments by adding a space between < and the word next to it, but I believe most people will either not find the solution or some of them will find it rather unappealing.

For this reason, I made the JSX rules recursive and added a "," Punctuation token inside so that there can be a number of generics used, as well as allowing nested generics. While I am not really fond of this hack, given that generics are already treated as JSX elements, I think this is a fair and easy enough solution for most cases.

Before

Screenshot 2024-09-20 at 9 28 05 PM

With spacing solution

Screenshot 2024-09-20 at 9 30 13 PM

With recursive JSX and "," Punctuation token

Screenshot 2024-09-20 at 9 55 11 PM

@alecthomas
Copy link
Owner

Mind adding some JSX samples to the test? We should make sure that still works.

@fredrare
Copy link
Contributor Author

Sure!
I have just included a JSX example with usual attributes and syntax to the test (my bad for not including them before). It should work just fine with any old JSX elements. The only thing is that JSX now also allows for recursive elements, but the highlighter should render exactly the same output for any old valid JSX.

Screenshot 2024-09-21 at 5 58 42 AM

@alecthomas
Copy link
Owner

Awesome, thanks!

@alecthomas alecthomas merged commit a56e228 into alecthomas:master Sep 21, 2024
2 checks passed
@alecthomas
Copy link
Owner

Thanks for fixing that, appreciate it.

DennisRasey pushed a commit to DennisRasey/forgejo that referenced this pull request Jan 6, 2025
…6477)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [github.com/alecthomas/chroma/v2](https://github.com/alecthomas/chroma) | require | minor | `v2.14.0` -> `v2.15.0` |

---

### Release Notes

<details>
<summary>alecthomas/chroma (github.com/alecthomas/chroma/v2)</summary>

### [`v2.15.0`](https://github.com/alecthomas/chroma/releases/tag/v2.15.0)

[Compare Source](alecthomas/chroma@v2.14.0...v2.15.0)

#### What's Changed

-   AQL: Add builtin functions introduced in v3.12 by [@&#8203;Simran-B](https://github.com/Simran-B) in alecthomas/chroma#968
-   chore(styles): add evergarden inspired style by [@&#8203;comfysage](https://github.com/comfysage) in alecthomas/chroma#970
-   chore(deps): update dependency svu to v2 by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#971
-   chore(deps): update dependency watchexec to v2 by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#962
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#955
-   styles: Fix Gleam alias by [@&#8203;jmooring](https://github.com/jmooring) in alecthomas/chroma#973
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#974
-   chore(deps): update dependency python3 to v3.12.3 by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#976
-   added the nordic theme via chroma/styles/nordic.xml by [@&#8203;avih7531](https://github.com/avih7531) in alecthomas/chroma#977
-   Update the Materialize lexer by [@&#8203;arusahni](https://github.com/arusahni) in alecthomas/chroma#978
-   feat(lexers/hare): add done keyword by [@&#8203;wackbyte](https://github.com/wackbyte) in alecthomas/chroma#979
-   pygments2chroma: escape regex.words by [@&#8203;oliverpool](https://github.com/oliverpool) in alecthomas/chroma#982
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#981
-   Add JSONata Lexer by [@&#8203;DevDimov](https://github.com/DevDimov) in alecthomas/chroma#983
-   Go lexer: single line comment without consuming endline, disable EnsureNL by [@&#8203;msorc](https://github.com/msorc) in alecthomas/chroma#984
-   Update the Materialize lexer by [@&#8203;arusahni](https://github.com/arusahni) in alecthomas/chroma#987
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#985
-   Add Gleam to README by [@&#8203;Nicd](https://github.com/Nicd) in alecthomas/chroma#990
-   Port Minecraft lexers from Pygments by [@&#8203;kofuk](https://github.com/kofuk) in alecthomas/chroma#992
-   Don't output extra whitespace in YAML multiline by [@&#8203;Gusted](https://github.com/Gusted) in alecthomas/chroma#993
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#991
-   add beef syntax and tests by [@&#8203;Booklordofthedings](https://github.com/Booklordofthedings) in alecthomas/chroma#995
-   Update GitHub light color palette by [@&#8203;konradreiche](https://github.com/konradreiche) in alecthomas/chroma#998
-   chore(deps): update module github.com/alecthomas/kong to v1 by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#999
-   Update the Materialize lexer by [@&#8203;arusahni](https://github.com/arusahni) in alecthomas/chroma#1001
-   Update TypeScript lexer to allow nested generics by [@&#8203;fredrare](https://github.com/fredrare) in alecthomas/chroma#1002
-   Pager friendly terminal formatting by [@&#8203;walles](https://github.com/walles) in alecthomas/chroma#1006
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#997
-   Add Typst Lexer by [@&#8203;oliverpool](https://github.com/oliverpool) in alecthomas/chroma#1007
-   Add Jsonnet Lexer by [@&#8203;jolheiser](https://github.com/jolheiser) in alecthomas/chroma#1011
-   fix: add underscore parsing in numbers for haskell by [@&#8203;Gusted](https://github.com/Gusted) in alecthomas/chroma#1020
-   Add CSV lexer by [@&#8203;walles](https://github.com/walles) in alecthomas/chroma#1005
-   fix(typescript): highlight string literal type parameters by [@&#8203;SKalt](https://github.com/SKalt) in alecthomas/chroma#1010
-   add any as a builtin type for go by [@&#8203;quartercastle](https://github.com/quartercastle) in alecthomas/chroma#1021
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#1013
-   Remove whitespace tokenizing rule in markdown lexer by [@&#8203;cloudchamb3r](https://github.com/cloudchamb3r) in alecthomas/chroma#1008
-   feat(JSON): support `.jsonc` extension by [@&#8203;CarterLi](https://github.com/CarterLi) in alecthomas/chroma#1022
-   Add Eclipse ATL language (https://eclipse.dev/atl/) by [@&#8203;dwagelaar](https://github.com/dwagelaar) in alecthomas/chroma#1024
-   Import NSIS Lexer from Pygments by [@&#8203;JakobDev](https://github.com/JakobDev) in alecthomas/chroma#1026
-   chore(deps): update module github.com/alecthomas/kong to v1.5.1 by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#1025
-   fix(nix): nix lexor missing '=' operator by [@&#8203;FlyingStitchman](https://github.com/FlyingStitchman) in alecthomas/chroma#1031
-   Add WebVTT lexer ([#&#8203;707](alecthomas/chroma#707)) by [@&#8203;dschuessler](https://github.com/dschuessler) in alecthomas/chroma#1032

#### New Contributors

-   [@&#8203;comfysage](https://github.com/comfysage) made their first contribution in alecthomas/chroma#970
-   [@&#8203;avih7531](https://github.com/avih7531) made their first contribution in alecthomas/chroma#977
-   [@&#8203;wackbyte](https://github.com/wackbyte) made their first contribution in alecthomas/chroma#979
-   [@&#8203;oliverpool](https://github.com/oliverpool) made their first contribution in alecthomas/chroma#982
-   [@&#8203;DevDimov](https://github.com/DevDimov) made their first contribution in alecthomas/chroma#983
-   [@&#8203;msorc](https://github.com/msorc) made their first contribution in alecthomas/chroma#984
-   [@&#8203;Nicd](https://github.com/Nicd) made their first contribution in alecthomas/chroma#990
-   [@&#8203;Booklordofthedings](https://github.com/Booklordofthedings) made their first contribution in alecthomas/chroma#995
-   [@&#8203;konradreiche](https://github.com/konradreiche) made their first contribution in alecthomas/chroma#998
-   [@&#8203;fredrare](https://github.com/fredrare) made their first contribution in alecthomas/chroma#1002
-   [@&#8203;SKalt](https://github.com/SKalt) made their first contribution in alecthomas/chroma#1010
-   [@&#8203;quartercastle](https://github.com/quartercastle) made their first contribution in alecthomas/chroma#1021
-   [@&#8203;cloudchamb3r](https://github.com/cloudchamb3r) made their first contribution in alecthomas/chroma#1008
-   [@&#8203;CarterLi](https://github.com/CarterLi) made their first contribution in alecthomas/chroma#1022
-   [@&#8203;dwagelaar](https://github.com/dwagelaar) made their first contribution in alecthomas/chroma#1024
-   [@&#8203;FlyingStitchman](https://github.com/FlyingStitchman) made their first contribution in alecthomas/chroma#1031
-   [@&#8203;dschuessler](https://github.com/dschuessler) made their first contribution in alecthomas/chroma#1032

**Full Changelog**: alecthomas/chroma@v2.14.0...v2.15.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "* 0-3 * * *" (UTC), Automerge - "* 0-3 * * *" (UTC).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS45MC4wIiwidXBkYXRlZEluVmVyIjoiMzkuOTAuMCIsInRhcmdldEJyYW5jaCI6ImZvcmdlam8iLCJsYWJlbHMiOlsiZGVwZW5kZW5jeS11cGdyYWRlIiwidGVzdC9ub3QtbmVlZGVkIl19-->

Co-authored-by: Gusted <[email protected]>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6477
Reviewed-by: Gusted <[email protected]>
Co-authored-by: Renovate Bot <[email protected]>
Co-committed-by: Renovate Bot <[email protected]>
DennisRasey pushed a commit to DennisRasey/forgejo that referenced this pull request Jan 15, 2025
…jo) (#6553)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [github.com/alecthomas/chroma/v2](https://github.com/alecthomas/chroma) | require | minor | `v2.14.0` -> `v2.15.0` |

---

### Release Notes

<details>
<summary>alecthomas/chroma (github.com/alecthomas/chroma/v2)</summary>

### [`v2.15.0`](https://github.com/alecthomas/chroma/releases/tag/v2.15.0)

[Compare Source](alecthomas/chroma@v2.14.0...v2.15.0)

#### What's Changed

-   AQL: Add builtin functions introduced in v3.12 by [@&#8203;Simran-B](https://github.com/Simran-B) in alecthomas/chroma#968
-   chore(styles): add evergarden inspired style by [@&#8203;comfysage](https://github.com/comfysage) in alecthomas/chroma#970
-   chore(deps): update dependency svu to v2 by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#971
-   chore(deps): update dependency watchexec to v2 by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#962
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#955
-   styles: Fix Gleam alias by [@&#8203;jmooring](https://github.com/jmooring) in alecthomas/chroma#973
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#974
-   chore(deps): update dependency python3 to v3.12.3 by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#976
-   added the nordic theme via chroma/styles/nordic.xml by [@&#8203;avih7531](https://github.com/avih7531) in alecthomas/chroma#977
-   Update the Materialize lexer by [@&#8203;arusahni](https://github.com/arusahni) in alecthomas/chroma#978
-   feat(lexers/hare): add done keyword by [@&#8203;wackbyte](https://github.com/wackbyte) in alecthomas/chroma#979
-   pygments2chroma: escape regex.words by [@&#8203;oliverpool](https://github.com/oliverpool) in alecthomas/chroma#982
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#981
-   Add JSONata Lexer by [@&#8203;DevDimov](https://github.com/DevDimov) in alecthomas/chroma#983
-   Go lexer: single line comment without consuming endline, disable EnsureNL by [@&#8203;msorc](https://github.com/msorc) in alecthomas/chroma#984
-   Update the Materialize lexer by [@&#8203;arusahni](https://github.com/arusahni) in alecthomas/chroma#987
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#985
-   Add Gleam to README by [@&#8203;Nicd](https://github.com/Nicd) in alecthomas/chroma#990
-   Port Minecraft lexers from Pygments by [@&#8203;kofuk](https://github.com/kofuk) in alecthomas/chroma#992
-   Don't output extra whitespace in YAML multiline by [@&#8203;Gusted](https://github.com/Gusted) in alecthomas/chroma#993
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#991
-   add beef syntax and tests by [@&#8203;Booklordofthedings](https://github.com/Booklordofthedings) in alecthomas/chroma#995
-   Update GitHub light color palette by [@&#8203;konradreiche](https://github.com/konradreiche) in alecthomas/chroma#998
-   chore(deps): update module github.com/alecthomas/kong to v1 by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#999
-   Update the Materialize lexer by [@&#8203;arusahni](https://github.com/arusahni) in alecthomas/chroma#1001
-   Update TypeScript lexer to allow nested generics by [@&#8203;fredrare](https://github.com/fredrare) in alecthomas/chroma#1002
-   Pager friendly terminal formatting by [@&#8203;walles](https://github.com/walles) in alecthomas/chroma#1006
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#997
-   Add Typst Lexer by [@&#8203;oliverpool](https://github.com/oliverpool) in alecthomas/chroma#1007
-   Add Jsonnet Lexer by [@&#8203;jolheiser](https://github.com/jolheiser) in alecthomas/chroma#1011
-   fix: add underscore parsing in numbers for haskell by [@&#8203;Gusted](https://github.com/Gusted) in alecthomas/chroma#1020
-   Add CSV lexer by [@&#8203;walles](https://github.com/walles) in alecthomas/chroma#1005
-   fix(typescript): highlight string literal type parameters by [@&#8203;SKalt](https://github.com/SKalt) in alecthomas/chroma#1010
-   add any as a builtin type for go by [@&#8203;quartercastle](https://github.com/quartercastle) in alecthomas/chroma#1021
-   chore(deps): update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#1013
-   Remove whitespace tokenizing rule in markdown lexer by [@&#8203;cloudchamb3r](https://github.com/cloudchamb3r) in alecthomas/chroma#1008
-   feat(JSON): support `.jsonc` extension by [@&#8203;CarterLi](https://github.com/CarterLi) in alecthomas/chroma#1022
-   Add Eclipse ATL language (https://eclipse.dev/atl/) by [@&#8203;dwagelaar](https://github.com/dwagelaar) in alecthomas/chroma#1024
-   Import NSIS Lexer from Pygments by [@&#8203;JakobDev](https://github.com/JakobDev) in alecthomas/chroma#1026
-   chore(deps): update module github.com/alecthomas/kong to v1.5.1 by [@&#8203;renovate](https://github.com/renovate) in alecthomas/chroma#1025
-   fix(nix): nix lexor missing '=' operator by [@&#8203;FlyingStitchman](https://github.com/FlyingStitchman) in alecthomas/chroma#1031
-   Add WebVTT lexer ([#&#8203;707](alecthomas/chroma#707)) by [@&#8203;dschuessler](https://github.com/dschuessler) in alecthomas/chroma#1032

#### New Contributors

-   [@&#8203;comfysage](https://github.com/comfysage) made their first contribution in alecthomas/chroma#970
-   [@&#8203;avih7531](https://github.com/avih7531) made their first contribution in alecthomas/chroma#977
-   [@&#8203;wackbyte](https://github.com/wackbyte) made their first contribution in alecthomas/chroma#979
-   [@&#8203;oliverpool](https://github.com/oliverpool) made their first contribution in alecthomas/chroma#982
-   [@&#8203;DevDimov](https://github.com/DevDimov) made their first contribution in alecthomas/chroma#983
-   [@&#8203;msorc](https://github.com/msorc) made their first contribution in alecthomas/chroma#984
-   [@&#8203;Nicd](https://github.com/Nicd) made their first contribution in alecthomas/chroma#990
-   [@&#8203;Booklordofthedings](https://github.com/Booklordofthedings) made their first contribution in alecthomas/chroma#995
-   [@&#8203;konradreiche](https://github.com/konradreiche) made their first contribution in alecthomas/chroma#998
-   [@&#8203;fredrare](https://github.com/fredrare) made their first contribution in alecthomas/chroma#1002
-   [@&#8203;SKalt](https://github.com/SKalt) made their first contribution in alecthomas/chroma#1010
-   [@&#8203;quartercastle](https://github.com/quartercastle) made their first contribution in alecthomas/chroma#1021
-   [@&#8203;cloudchamb3r](https://github.com/cloudchamb3r) made their first contribution in alecthomas/chroma#1008
-   [@&#8203;CarterLi](https://github.com/CarterLi) made their first contribution in alecthomas/chroma#1022
-   [@&#8203;dwagelaar](https://github.com/dwagelaar) made their first contribution in alecthomas/chroma#1024
-   [@&#8203;FlyingStitchman](https://github.com/FlyingStitchman) made their first contribution in alecthomas/chroma#1031
-   [@&#8203;dschuessler](https://github.com/dschuessler) made their first contribution in alecthomas/chroma#1032

**Full Changelog**: alecthomas/chroma@v2.14.0...v2.15.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - "* 0-3 * * *" (UTC).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDYuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNi4wIiwidGFyZ2V0QnJhbmNoIjoidjEwLjAvZm9yZ2VqbyIsImxhYmVscyI6WyJkZXBlbmRlbmN5LXVwZ3JhZGUiLCJ0ZXN0L25vdC1uZWVkZWQiXX0=-->

Co-authored-by: viceice <[email protected]>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6553
Reviewed-by: Michael Kriese <[email protected]>
Co-authored-by: Renovate Bot <[email protected]>
Co-committed-by: Renovate Bot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants