Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix retriever and reranker to process chat completion request #915

Merged
merged 2 commits into from
Nov 19, 2024

Conversation

minmin-intel
Copy link
Collaborator

Description

When testing DocIndexRetriever with user input k and top_n as ChatCompletionRequest, I identified two errors:

  1. retriever redis: previously if input is chat completion request, since the embedding is EmbeddingResponse, it cannot be processed by the vector db, so need to add additional steps to get the embedding vectors out from the EmbeddingResponse.
  2. reranker tei: previously input_type is fixed as SearchedDoc, so if the input is ChatCompletionRequest, the microservice will return error 422 unprocessable entity.

As a result of these 2 errors, the DocIndexRetriever fails when user send ChatCompletionRequest.

This PR fixes the issues above.

Issues

List the issue or RFC link this PR is working on. If there is no such link, please mark it as n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)

Dependencies

List the newly introduced 3rd party dependency if exists.

Tests

retriever and reranker unit tests, DocIndexRetriever example tests.

Copy link

codecov bot commented Nov 19, 2024

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
comps/cores/mega/gateway.py 0.00% 1 Missing ⚠️
Files with missing lines Coverage Δ
comps/cores/mega/gateway.py 28.14% <0.00%> (ø)

🚨 Try these New Features:

@chensuyue chensuyue added this to the v1.1 milestone Nov 19, 2024
@chensuyue
Copy link
Collaborator

Bug fix for DocIndexRetriever failed in v1.1 release test, https://github.com/opea-project/GenAIExamples/actions/runs/11876357287/job/33118149000

@lvliang-intel lvliang-intel merged commit 1cf2781 into opea-project:main Nov 19, 2024
15 checks passed
cameronmorin pushed a commit to opea-aws-proserve/GenAIComps that referenced this pull request Nov 22, 2024
…roject#915)

* fix retriever and reranker to process chat completion request

Signed-off-by: minmin-intel <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: minmin-intel <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
cameronmorin pushed a commit to opea-aws-proserve/GenAIComps that referenced this pull request Nov 28, 2024
…roject#915)

* fix retriever and reranker to process chat completion request

Signed-off-by: minmin-intel <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: minmin-intel <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
cameronmorin pushed a commit to opea-aws-proserve/GenAIComps that referenced this pull request Dec 2, 2024
…roject#915)

* fix retriever and reranker to process chat completion request

Signed-off-by: minmin-intel <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: minmin-intel <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Cameron Morin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants