/v1/inference/embeddings input and output shape mismatch #922
Assignees
Milestone
Comments
|
Good spot. I definitely agree with at least changing it to |
ashwinb
added a commit
that referenced
this issue
Feb 21, 2025
#1161) See Issue #922 The change is slightly backwards incompatible but no callsite (in our client codebases or stack-apps) every passes a depth-2 `List[List[InterleavedContentItem]]` (which is now disallowed.) ## Test Plan ```bash $ cd llama_stack/providers/tests/inference $ pytest -s -v -k fireworks test_embeddings.py \ --inference-model nomic-ai/nomic-embed-text-v1.5 --env EMBEDDING_DIMENSION=784 $ pytest -s -v -k together test_embeddings.py \ --inference-model togethercomputer/m2-bert-80M-8k-retrieval --env EMBEDDING_DIMENSION=784 $ pytest -s -v -k ollama test_embeddings.py \ --inference-model all-minilm:latest --env EMBEDDING_DIMENSION=784 ``` Also ran `tests/client-sdk/inference/test_embeddings.py`
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
/v1/inference/embeddings: model x List[InterleavedContent] -> List[List[float]]
the shape mismatch comes from InterleavedContent allowing for List[InterleavedContentItem].
example: [string, [text0, text1], image] -?-> [embedding of string, embedding of text0, embedding of text1, embedding of image]
i suggest aligning the shapes.
my preference is to change the input shape, and use an input of
array of string | array of InterleavedContentItem, which keeps string (untyped) and text / image (typed) inputs separate.a further enhancement: embedding is often done in two modes, batch and query. in batch mode many items are embedded for storage. in query mode a single item is embedded for lookup. allowing input of
string | array of string | array of InterleavedContentItemfacilitates this use case.The text was updated successfully, but these errors were encountered: