Use "binary vector" (base64) encoding for vectorization results returned by embedding provider #1800
Labels
Enhancement
Enhancement to existing feature
Feature - Collections
Issue impacts users of Collections
Feature - Tables
Issue impacts the users of Tables
Performance
Performance improvements
Currently vectorization results (embeddings) are passed as JSON float arrays in responses.
But as we have learned based on performance testing, using Base64 encoding on binary encoded 32-bit
float
s is significantly more performance (both lower CPU and more compact transfer format). We should see if we can use binary vector encoding for embedding results.Note: it seems that many embedding providers already include optional settings to return such encoded results so we might not need lots of work to achieve this.
The text was updated successfully, but these errors were encountered: