Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use "binary vector" (base64) encoding for vectorization results returned by embedding provider #1800

Open
tatu-at-datastax opened this issue Dec 17, 2024 · 0 comments
Labels
Enhancement Enhancement to existing feature Feature - Collections Issue impacts users of Collections Feature - Tables Issue impacts the users of Tables Performance Performance improvements

Comments

@tatu-at-datastax
Copy link
Contributor

Currently vectorization results (embeddings) are passed as JSON float arrays in responses.
But as we have learned based on performance testing, using Base64 encoding on binary encoded 32-bit floats is significantly more performance (both lower CPU and more compact transfer format). We should see if we can use binary vector encoding for embedding results.

Note: it seems that many embedding providers already include optional settings to return such encoded results so we might not need lots of work to achieve this.

@tatu-at-datastax tatu-at-datastax changed the title Use "binary vector" (base64) encoding for vectorization results of with embedding provider Use "binary vector" (base64) encoding for vectorization results returned by embedding provider Dec 17, 2024
@amorton amorton added Enhancement Enhancement to existing feature Feature - Collections Issue impacts users of Collections Feature - Tables Issue impacts the users of Tables Performance Performance improvements labels Jan 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Enhancement to existing feature Feature - Collections Issue impacts users of Collections Feature - Tables Issue impacts the users of Tables Performance Performance improvements
Projects
None yet
Development

No branches or pull requests

2 participants