Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: port @michael-makes/shiki-colorized-brackets JSR package #831

Merged
merged 6 commits into from
Nov 14, 2024

Conversation

MichaelMakesGames
Copy link
Contributor

@MichaelMakesGames MichaelMakesGames commented Nov 8, 2024

Description

There is a long-standing feature request to add VSCode-style bracket pair colorization to Shiki. I created a transformer that accomplishes this, and after discussion in the linked issue, it sounds like there's interest in incorporating it as an official plugin.

Linked Issues

fix #303

Additional context

This is mostly a straight port of my package, https://github.com/MichaelMakesGames/shiki-colorized-brackets. I've made some efforts to align with the styles and patterns I observed elsewhere in the Shiki repo, but let me know if there are other improvements I can make.

I'd like to call out that the way this splits tokens apart feels a little hacky, mostly just doing some basic text search. I investigated another route that used the token explanations, but that had issues highlighting mismatched brackets. If that can be worked around, that would be a more robust solution.

Question regarding the docs: currently this transformer isn't enabled in the docs. Would we want this enabled there? Perhaps only for the page for this package?

Another note: I listed myself as the author of the package. I noticed all the other packages list @antfu. I'm fine changing that.

Copy link

netlify bot commented Nov 8, 2024

Deploy Preview for shiki-matsu ready!

Name Link
🔨 Latest commit b30dcbb
🔍 Latest deploy log https://app.netlify.com/sites/shiki-matsu/deploys/6733ed4a5736d00008902b87
😎 Deploy Preview https://deploy-preview-831--shiki-matsu.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

netlify bot commented Nov 8, 2024

Deploy Preview for shiki-next ready!

Name Link
🔨 Latest commit b30dcbb
🔍 Latest deploy log https://app.netlify.com/sites/shiki-next/deploys/6733ed4a387783000833662e
😎 Deploy Preview https://deploy-preview-831--shiki-next.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@antfu
Copy link
Member

antfu commented Nov 10, 2024

Awesome! Thanks a lot! Please give me some time to properly review.

splits tokens apart feels a little hacky

Do you consider using https://shiki.style/guide/decorations? It should handle the complexity of token splitting, where you just need to provide the positions.

Would we want this enabled there? Perhaps only for the page for this package?

Yeah I would vote to enable it only for the page of that package. You can refer to how we do that with theme or other transformers.

I listed myself as the author of the package.

Surely! It totally make sense to have your name on that package!

@MichaelMakesGames
Copy link
Contributor Author

Please give me some time to properly review

Certainly! No rush

splits tokens apart feels a little hacky

Do you consider using https://shiki.style/guide/decorations?

I should clarify a bit. Splitting the tokens apart feels like the right approach to me. We are dealing with actually distinct tokens here, that are just joined together by the engine because they share a color. Also this approach has a couple benefits: no additional CSS needed so it's easier to set up, and it's able to work more seamlessly with other things in the broader ecosystem, such as Expressive Code (which generally does not work Shiki transaformers). That said, if you prefer decorations, I can look into refactoring that.

The part I considered hacky was the method for finding those distinct bracket tokens. That's needs to happen regardless of whether we use decorations or stick with current method of changing the tokens themselves. Currently this just uses regular expressions, with some logic for ignoring matches in certain tokens (eg comments and strings). The method that feels better to me, is to using the token explanations, which contains info about all of the same-color tokens that have been joined together. However, that fails with mismatched tokens, which are still are joined together even in the explanations. Eg []]], those last 2 ]] are not parsed as individual tokens. So it seems some hybrid of the 2 might be needed, but I haven't done that yet.

enable it only for the page of that package. You can refer to how we do that

Great, I'll take a look sometime in the next few days

@MichaelMakesGames
Copy link
Contributor Author

Yeah I would vote to enable it only for the page of that package

This is done in the latest commit. I added an explicitTrigger option, similar to twoslash, so the plugin can be enabled only for certain code blocks based on the meta string

@antfu antfu changed the title feat: port @michael-makes/shiki-colorized-brackets JSR package feat: port @michael-makes/shiki-colorized-brackets JSR package Nov 14, 2024
@antfu antfu merged commit 94cc6d8 into shikijs:main Nov 14, 2024
11 checks passed
@antfu
Copy link
Member

antfu commented Nov 14, 2024

Thanks a lot! Let's have it and see how it works

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bracket pair colorization
2 participants