fix: high performance large convo rendering

[louis-menlo]

Describe Your Changes

Enhance Jan’s convo rendering by virtualizing only the visible DOM nodes.

There was a performance issue on latest release 0.5.9, where it renders only a few tokens / second for a large convo (just around ~100 messages). After investigating, we found a few issues related to app state handling and top-level components re-rendering. We attempted to address these issues to minimize the likelihood of app re-rendering.

This pull request also addressed the issue where the conversation does not follow the generated text, which significantly impacts the user experience.

0.5.9 - Very laggy while generating

0.5.9 Nightly with better state handling

However, that’s not sufficient, we constantly strive for better UX. With this fix, we apply visible DOM node virtualization for enhanced performance and consistency, ensuring a seamless experience from small convo to any size of conversations.

Latest fix:

CleanShot.2024-11-30.at.16.47.15.mp4

A big shoutout to @tanstack/react-virtual for creating such an amazing library.

Changes made

1. Dependency Update:

   • Added @tanstack/react-virtual (version ^3.10.9) in package.json.

2. Imports:

   • Removed unused imports (ErrorMessage, ListContainer) in ChatBody index file.
   • Added new imports: useMemo, useRef, and useVirtualizer.

3. Component Logic:

   • Integrated useVirtualizer to efficiently render a virtualized list of chat messages.
   • Added useRef for the scrollable parent element and use useMemo to calculate the message count.
   • Adjusted the component's rendering logic to use the virtualizer's calculated positions and sizes.

4. Effects:

   • Updated the effect dependencies to include loadModelError and adjusted scrolling logic upon data changes.

5. Layout Adjustments:

   • Replaced ListContainer with custom div elements to cater for virtualized rendering.
   • Changed styles for LoadModelError for a flexible layout.

6. Minor Fixes:

   • Added optional chaining (?.) in conditionals to prevent potential runtime errors when accessing properties of possibly undefined values.

These changes improve performance, particularly in scenarios with large datasets by implementing efficient scrolling and rendering techniques.

[github-actions]

Barecheck - Code coverage report

Total: 69.37%

Your code coverage diff: -0.17% ▾

Uncovered files and lines

File	Lines
web/screens/Thread/ThreadCenterPanel/ChatBody/index.tsx	19, 21-22, 24, 28-29, 32-33, 37, 41, 58, 60-61, 66, 68-69, 72-74, 78-79, 84, 121
web/screens/Thread/ThreadCenterPanel/ChatItem/index.tsx	23-25, 32-36, 40-41, 44-48
web/screens/Thread/ThreadCenterPanel/LoadModelError/index.tsx	17-21, 23-24, 35-37, 40, 57

[namchuai]

✅