Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use @jupyterlab/rendermime for in-chat markdown rendering #564

Merged
merged 48 commits into from
Feb 7, 2024

Conversation

andrii-i
Copy link
Collaborator

@andrii-i andrii-i commented Jan 4, 2024

  • Use @jupyterlab/rendermime instead of react-markdown for markdown rendering, make code/markdown/math blocks look the same as in JupyterLab
  • Remove unused dependencies: react-markdown, react-syntax-highlighter, rehype-katex, remark-math. As pointed out by @krassowski, "if I see this right, the package size should drop by 97.7% (over 2 MB) thanks to dropping the extra dependencies (notably react-syntax-highlighter) 🎉"
  • Escape LaTeX delimeters in LLM responses where needed by adding extra backslashes for proper rendering by @jupyterlab/rendermime
  • Fixes Improve code styling in markdown #72

RendermimeMarkdown component memoization improves page scripting/rendering speed by ~10% based on performance profiling.

Before:
Screenshot 2024-01-10 at 1 42 15 PM
Screenshot 2024-02-06 at 10 56 40 AM

After:
Screenshot 2024-01-10 at 1 40 05 PM
Screenshot 2024-02-06 at 10 52 21 AM

@andrii-i andrii-i added the enhancement New feature or request label Jan 4, 2024
@JasonWeill
Copy link
Collaborator

As discussed earlier, the narrow default width of the left panel makes it hard to render code. Perhaps if we move the chat UI to the main area (#76), this will be less of a problem, but for now, Python code straddling multiple lines looks less than ideal:

image

Options to deal with this include:

@andrii-i andrii-i force-pushed the use-rendermime-markdown branch 2 times, most recently from 9eefaf8 to 4325658 Compare January 9, 2024 07:06
@andrii-i andrii-i marked this pull request as ready for review January 10, 2024 21:43
@andrii-i
Copy link
Collaborator Author

@JasonWeill, @3coins thank you for looking into this PR, updated it based on your comments and our discussions. It is ready for (re-)review.

@andrii-i
Copy link
Collaborator Author

please update playwright snapshots

Copy link
Member

@krassowski krassowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excited to see this progress! Just some minor suggestions :)

package.json Outdated Show resolved Hide resolved
packages/jupyter-ai/src/index.ts Outdated Show resolved Hide resolved
packages/jupyter-ai/src/components/rendermime-markdown.tsx Outdated Show resolved Hide resolved
Copy link
Member

@krassowski krassowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two tiny suggestions on dependencies, otherwise it looks great (tested locally).

package.json Outdated Show resolved Hide resolved
packages/jupyter-ai/package.json Outdated Show resolved Hide resolved
Copy link
Member

@krassowski krassowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lockfile may still need updating (not sure about your internal rebase practices), but otherwise it looks good to me.

PS. If I see this right, the package size should drop by 97.7% (over 2 MB) thanks to dropping the extra dependencies (notably react-syntax-highlighter) 🎉

@andrii-i andrii-i force-pushed the use-rendermime-markdown branch 2 times, most recently from 41fc575 to 9cab65c Compare February 6, 2024 19:07
@dlqqq dlqqq self-assigned this Feb 6, 2024
@dlqqq dlqqq self-requested a review February 6, 2024 19:25
@andrii-i andrii-i force-pushed the use-rendermime-markdown branch 3 times, most recently from a748e32 to fd56bbe Compare February 7, 2024 19:42
Copy link
Member

@dlqqq dlqqq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrii-i This is awesome work! The new UI is much cleaner and noticeably faster. Just a few minor concerns, but this is 95% ready to ship. 🎉

packages/jupyter-ai/src/components/rendermime-markdown.tsx Outdated Show resolved Hide resolved
packages/jupyter-ai/package.json Show resolved Hide resolved
packages/jupyter-ai/package.json Outdated Show resolved Hide resolved
@andrii-i
Copy link
Collaborator Author

andrii-i commented Feb 7, 2024

Thank you @dlqqq. All comments are implemented.

@andrii-i andrii-i enabled auto-merge (squash) February 7, 2024 23:42
@andrii-i andrii-i merged commit a6868ff into jupyterlab:main Feb 7, 2024
8 checks passed
@andrii-i andrii-i deleted the use-rendermime-markdown branch February 7, 2024 23:50
@andrii-i
Copy link
Collaborator Author

andrii-i commented Feb 8, 2024

@meeseeksdev please backport to 1.x

Copy link

lumberbot-app bot commented Feb 8, 2024

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
git checkout 1.x
git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
git cherry-pick -x -m1 a6868ffc245e098c165399e43bb8b248fa575abd
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
git commit -am 'Backport PR #564: Use @jupyterlab/rendermime for in-chat markdown rendering'
  1. Push to a named branch:
git push YOURFORK 1.x:auto-backport-of-pr-564-on-1.x
  1. Create a PR against branch 1.x, I would have named this PR:

"Backport PR #564 on branch 1.x (Use @jupyterlab/rendermime for in-chat markdown rendering)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

andrii-i added a commit to andrii-i/jupyter-ai that referenced this pull request Feb 8, 2024
…#564)

* create markdown widget and component

* add rendermime

* add rendermime registry requrement, activation to plugin

* update var naming

* use rendermime directrly in the react component

* use css to emulate code styling

* add @types/react-dom

* create CopyButton component based on chat-code-view.tsx by @dlqqq

* add CopyButton and styling

* remoev unsused code

* add min width to the button

* omit units for 0

* use RendermimeMarkdown for ChatSettings model help

* remove react-markdown dependency

* revert CopyButton styling

* remove comments

* memoize RendermimeMarkdown (10% faster)

* Update packages/jupyter-ai/src/components/copy-button.tsx

Co-authored-by: Jason Weill <[email protected]>

* remove copying status

* update yarn.lock

* detect mimetype, render latex

* set latexTypesetter

* MathJax typeset redered md

* use rmRegistry typesetter

* remove MathJaxTypesetter import

* Modify CHAT_SYSTEM_PROMPT, use $ and $$ style LaTeX delimiters only

* remove @jupyterlab/mathjax-extension dependency

* remove @jupyterlab/builder dependency

* Update packages/jupyter-ai/src/index.ts

Co-authored-by: Michał Krassowski <[email protected]>

* render content after attaching buttons per @krassowski

* remove duplicate rmRegistry per @krassowski

* update snapshots

* Remove unused react-syntax-highlighter, rehype-katex, remark-math dependencies

* move @types/react-dom dependency from root to packages/jupyter-ai per @krassowski

* Update packages/jupyter-ai/package.json

Co-authored-by: Michał Krassowski <[email protected]>

* update yarn.lock

* Escale LaTeX delimeters with regex instead of prompt engineering

* add .jp-ai-rendermime-markdown mjx-container  styling to match jlab

* update escapeLatexDelimiters comment

* update lockfile

* update lockfile

* move escapeLatexDelimiters outside of RendermimeMarkdownBase to avoid redefinitions on every rerender

* adjust escapeLatexDelimiters parameter naming

* Rename escapeLatexDelimiters  argument from str to text

* Update packages/jupyter-ai/src/components/rendermime-markdown.tsx

Co-authored-by: david qiu <[email protected]>

* Update escapeLatexDelimiters docstring per @dlqqq

* bump react-dom version per @dlqqq

* add @jupyterlab/rendermime as a direct dependenct per @dlqqq to avoid potential build errors

---------

Co-authored-by: Jason Weill <[email protected]>
Co-authored-by: Michał Krassowski <[email protected]>
Co-authored-by: david qiu <[email protected]>
andrii-i added a commit to andrii-i/jupyter-ai that referenced this pull request Feb 8, 2024
…#564)

* create markdown widget and component

* add rendermime

* add rendermime registry requrement, activation to plugin

* update var naming

* use rendermime directrly in the react component

* use css to emulate code styling

* add @types/react-dom

* create CopyButton component based on chat-code-view.tsx by @dlqqq

* add CopyButton and styling

* remoev unsused code

* add min width to the button

* omit units for 0

* use RendermimeMarkdown for ChatSettings model help

* remove react-markdown dependency

* revert CopyButton styling

* remove comments

* memoize RendermimeMarkdown (10% faster)

* Update packages/jupyter-ai/src/components/copy-button.tsx

Co-authored-by: Jason Weill <[email protected]>

* remove copying status

* update yarn.lock

* detect mimetype, render latex

* set latexTypesetter

* MathJax typeset redered md

* use rmRegistry typesetter

* remove MathJaxTypesetter import

* Modify CHAT_SYSTEM_PROMPT, use $ and $$ style LaTeX delimiters only

* remove @jupyterlab/mathjax-extension dependency

* remove @jupyterlab/builder dependency

* Update packages/jupyter-ai/src/index.ts

Co-authored-by: Michał Krassowski <[email protected]>

* render content after attaching buttons per @krassowski

* remove duplicate rmRegistry per @krassowski

* update snapshots

* Remove unused react-syntax-highlighter, rehype-katex, remark-math dependencies

* move @types/react-dom dependency from root to packages/jupyter-ai per @krassowski

* Update packages/jupyter-ai/package.json

Co-authored-by: Michał Krassowski <[email protected]>

* update yarn.lock

* Escale LaTeX delimeters with regex instead of prompt engineering

* add .jp-ai-rendermime-markdown mjx-container  styling to match jlab

* update escapeLatexDelimiters comment

* update lockfile

* update lockfile

* move escapeLatexDelimiters outside of RendermimeMarkdownBase to avoid redefinitions on every rerender

* adjust escapeLatexDelimiters parameter naming

* Rename escapeLatexDelimiters  argument from str to text

* Update packages/jupyter-ai/src/components/rendermime-markdown.tsx

Co-authored-by: david qiu <[email protected]>

* Update escapeLatexDelimiters docstring per @dlqqq

* bump react-dom version per @dlqqq

* add @jupyterlab/rendermime as a direct dependenct per @dlqqq to avoid potential build errors

---------

Co-authored-by: Jason Weill <[email protected]>
Co-authored-by: Michał Krassowski <[email protected]>
Co-authored-by: david qiu <[email protected]>
andrii-i added a commit to andrii-i/jupyter-ai that referenced this pull request Feb 8, 2024
…#564)

* create markdown widget and component

* add rendermime

* add rendermime registry requrement, activation to plugin

* update var naming

* use rendermime directrly in the react component

* use css to emulate code styling

* add @types/react-dom

* create CopyButton component based on chat-code-view.tsx by @dlqqq

* add CopyButton and styling

* remoev unsused code

* add min width to the button

* omit units for 0

* use RendermimeMarkdown for ChatSettings model help

* remove react-markdown dependency

* revert CopyButton styling

* remove comments

* memoize RendermimeMarkdown (10% faster)

* Update packages/jupyter-ai/src/components/copy-button.tsx

Co-authored-by: Jason Weill <[email protected]>

* remove copying status

* update yarn.lock

* detect mimetype, render latex

* set latexTypesetter

* MathJax typeset redered md

* use rmRegistry typesetter

* remove MathJaxTypesetter import

* Modify CHAT_SYSTEM_PROMPT, use $ and $$ style LaTeX delimiters only

* remove @jupyterlab/mathjax-extension dependency

* remove @jupyterlab/builder dependency

* Update packages/jupyter-ai/src/index.ts

Co-authored-by: Michał Krassowski <[email protected]>

* render content after attaching buttons per @krassowski

* remove duplicate rmRegistry per @krassowski

* update snapshots

* Remove unused react-syntax-highlighter, rehype-katex, remark-math dependencies

* move @types/react-dom dependency from root to packages/jupyter-ai per @krassowski

* Update packages/jupyter-ai/package.json

Co-authored-by: Michał Krassowski <[email protected]>

* update yarn.lock

* Escale LaTeX delimeters with regex instead of prompt engineering

* add .jp-ai-rendermime-markdown mjx-container  styling to match jlab

* update escapeLatexDelimiters comment

* update lockfile

* update lockfile

* move escapeLatexDelimiters outside of RendermimeMarkdownBase to avoid redefinitions on every rerender

* adjust escapeLatexDelimiters parameter naming

* Rename escapeLatexDelimiters  argument from str to text

* Update packages/jupyter-ai/src/components/rendermime-markdown.tsx

Co-authored-by: david qiu <[email protected]>

* Update escapeLatexDelimiters docstring per @dlqqq

* bump react-dom version per @dlqqq

* add @jupyterlab/rendermime as a direct dependenct per @dlqqq to avoid potential build errors

---------

Co-authored-by: Jason Weill <[email protected]>
Co-authored-by: Michał Krassowski <[email protected]>
Co-authored-by: david qiu <[email protected]>
andrii-i added a commit to andrii-i/jupyter-ai that referenced this pull request Feb 8, 2024
…#564)

* create markdown widget and component

* add rendermime

* add rendermime registry requrement, activation to plugin

* update var naming

* use rendermime directrly in the react component

* use css to emulate code styling

* add @types/react-dom

* create CopyButton component based on chat-code-view.tsx by @dlqqq

* add CopyButton and styling

* remoev unsused code

* add min width to the button

* omit units for 0

* use RendermimeMarkdown for ChatSettings model help

* remove react-markdown dependency

* revert CopyButton styling

* remove comments

* memoize RendermimeMarkdown (10% faster)

* Update packages/jupyter-ai/src/components/copy-button.tsx

Co-authored-by: Jason Weill <[email protected]>

* remove copying status

* update yarn.lock

* detect mimetype, render latex

* set latexTypesetter

* MathJax typeset redered md

* use rmRegistry typesetter

* remove MathJaxTypesetter import

* Modify CHAT_SYSTEM_PROMPT, use $ and $$ style LaTeX delimiters only

* remove @jupyterlab/mathjax-extension dependency

* remove @jupyterlab/builder dependency

* Update packages/jupyter-ai/src/index.ts

Co-authored-by: Michał Krassowski <[email protected]>

* render content after attaching buttons per @krassowski

* remove duplicate rmRegistry per @krassowski

* update snapshots

* Remove unused react-syntax-highlighter, rehype-katex, remark-math dependencies

* move @types/react-dom dependency from root to packages/jupyter-ai per @krassowski

* Update packages/jupyter-ai/package.json

Co-authored-by: Michał Krassowski <[email protected]>

* update yarn.lock

* Escale LaTeX delimeters with regex instead of prompt engineering

* add .jp-ai-rendermime-markdown mjx-container  styling to match jlab

* update escapeLatexDelimiters comment

* update lockfile

* update lockfile

* move escapeLatexDelimiters outside of RendermimeMarkdownBase to avoid redefinitions on every rerender

* adjust escapeLatexDelimiters parameter naming

* Rename escapeLatexDelimiters  argument from str to text

* Update packages/jupyter-ai/src/components/rendermime-markdown.tsx

Co-authored-by: david qiu <[email protected]>

* Update escapeLatexDelimiters docstring per @dlqqq

* bump react-dom version per @dlqqq

* add @jupyterlab/rendermime as a direct dependenct per @dlqqq to avoid potential build errors

---------

Co-authored-by: Jason Weill <[email protected]>
Co-authored-by: Michał Krassowski <[email protected]>
Co-authored-by: david qiu <[email protected]>
andrii-i added a commit that referenced this pull request Feb 8, 2024
* Use @jupyterlab/rendermime for in-chat markdown rendering (#564)

* create markdown widget and component

* add rendermime

* add rendermime registry requrement, activation to plugin

* update var naming

* use rendermime directrly in the react component

* use css to emulate code styling

* add @types/react-dom

* create CopyButton component based on chat-code-view.tsx by @dlqqq

* add CopyButton and styling

* remoev unsused code

* add min width to the button

* omit units for 0

* use RendermimeMarkdown for ChatSettings model help

* remove react-markdown dependency

* revert CopyButton styling

* remove comments

* memoize RendermimeMarkdown (10% faster)

* Update packages/jupyter-ai/src/components/copy-button.tsx

Co-authored-by: Jason Weill <[email protected]>

* remove copying status

* update yarn.lock

* detect mimetype, render latex

* set latexTypesetter

* MathJax typeset redered md

* use rmRegistry typesetter

* remove MathJaxTypesetter import

* Modify CHAT_SYSTEM_PROMPT, use $ and $$ style LaTeX delimiters only

* remove @jupyterlab/mathjax-extension dependency

* remove @jupyterlab/builder dependency

* Update packages/jupyter-ai/src/index.ts

Co-authored-by: Michał Krassowski <[email protected]>

* render content after attaching buttons per @krassowski

* remove duplicate rmRegistry per @krassowski

* update snapshots

* Remove unused react-syntax-highlighter, rehype-katex, remark-math dependencies

* move @types/react-dom dependency from root to packages/jupyter-ai per @krassowski

* Update packages/jupyter-ai/package.json

Co-authored-by: Michał Krassowski <[email protected]>

* update yarn.lock

* Escale LaTeX delimeters with regex instead of prompt engineering

* add .jp-ai-rendermime-markdown mjx-container  styling to match jlab

* update escapeLatexDelimiters comment

* update lockfile

* update lockfile

* move escapeLatexDelimiters outside of RendermimeMarkdownBase to avoid redefinitions on every rerender

* adjust escapeLatexDelimiters parameter naming

* Rename escapeLatexDelimiters  argument from str to text

* Update packages/jupyter-ai/src/components/rendermime-markdown.tsx

Co-authored-by: david qiu <[email protected]>

* Update escapeLatexDelimiters docstring per @dlqqq

* bump react-dom version per @dlqqq

* add @jupyterlab/rendermime as a direct dependenct per @dlqqq to avoid potential build errors

---------

Co-authored-by: Jason Weill <[email protected]>
Co-authored-by: Michał Krassowski <[email protected]>
Co-authored-by: david qiu <[email protected]>

* Add "@jupyterlab/rendermime": "^3.6.3" dependency

---------

Co-authored-by: Jason Weill <[email protected]>
Co-authored-by: Michał Krassowski <[email protected]>
Co-authored-by: david qiu <[email protected]>
@krassowski
Copy link
Member

Thanks to this PR the tarball size dropped by 1.5MB 🎉

Before After
Screenshot from 2024-03-06 17-20-41 Screenshot from 2024-03-06 17-20-36

dbelgrod pushed a commit to dbelgrod/jupyter-ai that referenced this pull request Jun 10, 2024
…#564)

* create markdown widget and component

* add rendermime

* add rendermime registry requrement, activation to plugin

* update var naming

* use rendermime directrly in the react component

* use css to emulate code styling

* add @types/react-dom

* create CopyButton component based on chat-code-view.tsx by @dlqqq

* add CopyButton and styling

* remoev unsused code

* add min width to the button

* omit units for 0

* use RendermimeMarkdown for ChatSettings model help

* remove react-markdown dependency

* revert CopyButton styling

* remove comments

* memoize RendermimeMarkdown (10% faster)

* Update packages/jupyter-ai/src/components/copy-button.tsx

Co-authored-by: Jason Weill <[email protected]>

* remove copying status

* update yarn.lock

* detect mimetype, render latex

* set latexTypesetter

* MathJax typeset redered md

* use rmRegistry typesetter

* remove MathJaxTypesetter import

* Modify CHAT_SYSTEM_PROMPT, use $ and $$ style LaTeX delimiters only

* remove @jupyterlab/mathjax-extension dependency

* remove @jupyterlab/builder dependency

* Update packages/jupyter-ai/src/index.ts

Co-authored-by: Michał Krassowski <[email protected]>

* render content after attaching buttons per @krassowski

* remove duplicate rmRegistry per @krassowski

* update snapshots

* Remove unused react-syntax-highlighter, rehype-katex, remark-math dependencies

* move @types/react-dom dependency from root to packages/jupyter-ai per @krassowski

* Update packages/jupyter-ai/package.json

Co-authored-by: Michał Krassowski <[email protected]>

* update yarn.lock

* Escale LaTeX delimeters with regex instead of prompt engineering

* add .jp-ai-rendermime-markdown mjx-container  styling to match jlab

* update escapeLatexDelimiters comment

* update lockfile

* update lockfile

* move escapeLatexDelimiters outside of RendermimeMarkdownBase to avoid redefinitions on every rerender

* adjust escapeLatexDelimiters parameter naming

* Rename escapeLatexDelimiters  argument from str to text

* Update packages/jupyter-ai/src/components/rendermime-markdown.tsx

Co-authored-by: david qiu <[email protected]>

* Update escapeLatexDelimiters docstring per @dlqqq

* bump react-dom version per @dlqqq

* add @jupyterlab/rendermime as a direct dependenct per @dlqqq to avoid potential build errors

---------

Co-authored-by: Jason Weill <[email protected]>
Co-authored-by: Michał Krassowski <[email protected]>
Co-authored-by: david qiu <[email protected]>
Marchlak pushed a commit to Marchlak/jupyter-ai that referenced this pull request Oct 28, 2024
…#564)

* create markdown widget and component

* add rendermime

* add rendermime registry requrement, activation to plugin

* update var naming

* use rendermime directrly in the react component

* use css to emulate code styling

* add @types/react-dom

* create CopyButton component based on chat-code-view.tsx by @dlqqq

* add CopyButton and styling

* remoev unsused code

* add min width to the button

* omit units for 0

* use RendermimeMarkdown for ChatSettings model help

* remove react-markdown dependency

* revert CopyButton styling

* remove comments

* memoize RendermimeMarkdown (10% faster)

* Update packages/jupyter-ai/src/components/copy-button.tsx

Co-authored-by: Jason Weill <[email protected]>

* remove copying status

* update yarn.lock

* detect mimetype, render latex

* set latexTypesetter

* MathJax typeset redered md

* use rmRegistry typesetter

* remove MathJaxTypesetter import

* Modify CHAT_SYSTEM_PROMPT, use $ and $$ style LaTeX delimiters only

* remove @jupyterlab/mathjax-extension dependency

* remove @jupyterlab/builder dependency

* Update packages/jupyter-ai/src/index.ts

Co-authored-by: Michał Krassowski <[email protected]>

* render content after attaching buttons per @krassowski

* remove duplicate rmRegistry per @krassowski

* update snapshots

* Remove unused react-syntax-highlighter, rehype-katex, remark-math dependencies

* move @types/react-dom dependency from root to packages/jupyter-ai per @krassowski

* Update packages/jupyter-ai/package.json

Co-authored-by: Michał Krassowski <[email protected]>

* update yarn.lock

* Escale LaTeX delimeters with regex instead of prompt engineering

* add .jp-ai-rendermime-markdown mjx-container  styling to match jlab

* update escapeLatexDelimiters comment

* update lockfile

* update lockfile

* move escapeLatexDelimiters outside of RendermimeMarkdownBase to avoid redefinitions on every rerender

* adjust escapeLatexDelimiters parameter naming

* Rename escapeLatexDelimiters  argument from str to text

* Update packages/jupyter-ai/src/components/rendermime-markdown.tsx

Co-authored-by: david qiu <[email protected]>

* Update escapeLatexDelimiters docstring per @dlqqq

* bump react-dom version per @dlqqq

* add @jupyterlab/rendermime as a direct dependenct per @dlqqq to avoid potential build errors

---------

Co-authored-by: Jason Weill <[email protected]>
Co-authored-by: Michał Krassowski <[email protected]>
Co-authored-by: david qiu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve code styling in markdown
5 participants