Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add chat template renderer from llama.cpp #1289

Merged
merged 4 commits into from
Sep 23, 2024

Conversation

nguyenhoangthuan99
Copy link
Contributor

@nguyenhoangthuan99 nguyenhoangthuan99 commented Sep 22, 2024

Replace jinja2cpp with llama.cpp for chat template parsing

Summary

This PR replaces the jinja2cpp library with code derived from the llama.cpp project for parsing chat templates. This change significantly reduces our binary size and improves the reliability of template parsing.

Changes

  • Removed jinja2cpp library and related dependencies
  • Integrated chat template parsing functionality from llama.cpp
  • Updated relevant code to use the new parsing method
  • Added appropriate license and attribution for llama.cpp code

Benefits

  1. Reduced Binary Size: Eliminates more space from our binary file.
  2. Improved Reliability: The llama.cpp parsing method has shown better consistency in handling our chat templates compared to jinja2cpp, which occasionally failed to parse correctly.
  3. Simplified Dependencies: Removes a complex external library, potentially reducing build times and complexity.

Related Issues

Fix #1063

Before add custom chat template renderer

image

After

image

@nguyenhoangthuan99 nguyenhoangthuan99 marked this pull request as ready for review September 23, 2024 01:19
Copy link
Collaborator

@namchuai namchuai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ lgtm

Copy link
Contributor

@dan-menlo dan-menlo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

@nguyenhoangthuan99 nguyenhoangthuan99 merged commit 80a1a70 into dev Sep 23, 2024
4 checks passed
@nguyenhoangthuan99 nguyenhoangthuan99 deleted the feat/render-chat-template branch September 23, 2024 06:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

chore: Improve package/binary size by remove jinja2
4 participants