-
Notifications
You must be signed in to change notification settings - Fork 144
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
optimize rerank with backend ref #579
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@a32543254, |
Thanks, I'm co-working with Suyue on this ut test now, will be fixed soon. |
Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
for more information, see https://pre-commit.ci Signed-off-by: Dong, Bo1 <[email protected]>
…t#568) * fix mismatched response format w/wo streaming guardrails * fix & debug * fix & rm debug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <[email protected]>
…project#571) * fix mismatched response format w/wo streaming guardrails * fix & debug * fix & rm debug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * debug * debug * debug * fix pre-space and linebreak * fix pre-space and linebreak * fix single/double quote * fix single/double quote * remove debug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <[email protected]>
* add url validator Signed-off-by: BaoHuiling <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add validation for video_url Signed-off-by: BaoHuiling <[email protected]> --------- Signed-off-by: BaoHuiling <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <[email protected]>
* Reduced output. Signed-off-by: zepan <[email protected]> * Output the location where the modified Dockerfile file is referenced. Signed-off-by: zepan <[email protected]> * for test Signed-off-by: zepan <[email protected]> * Restore test file. Signed-off-by: zepan <[email protected]> --------- Signed-off-by: zepan <[email protected]> Signed-off-by: Dong, Bo1 <[email protected]>
* add ut and refine logging code. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update microservice port. --------- Co-authored-by: root <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <[email protected]>
* adding lancedb to langchain vectorstores Signed-off-by: sharanshirodkar7 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: sharanshirodkar7 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: lvliang-intel <[email protected]> Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: letonghan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
for more information, see https://pre-commit.ci Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
for more information, see https://pre-commit.ci Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <[email protected]>
* multimodal embedding for MM RAG for videos Signed-off-by: Tiep Le <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * develop data prep first commit Signed-off-by: Tiep Le <[email protected]> * develop dataprep microservice for multimodal data Signed-off-by: Tiep Le <[email protected]> * multimodal langchain for dataprep Signed-off-by: Tiep Le <[email protected]> * update README Signed-off-by: Tiep Le <[email protected]> * update README Signed-off-by: Tiep Le <[email protected]> * update README Signed-off-by: Tiep Le <[email protected]> * update README Signed-off-by: Tiep Le <[email protected]> * cosmetic Signed-off-by: Tiep Le <[email protected]> * test for multimodal dataprep Signed-off-by: Tiep Le <[email protected]> * update test Signed-off-by: Tiep Le <[email protected]> * update test Signed-off-by: Tiep Le <[email protected]> * update test Signed-off-by: Tiep Le <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cosmetic update Signed-off-by: Tiep Le <[email protected]> * remove langsmith Signed-off-by: Tiep Le <[email protected]> * update API to remove /dataprep from API names and remove langsmith Signed-off-by: Tiep Le <[email protected]> * update test Signed-off-by: Tiep Le <[email protected]> * update the error message per PR reviewer Signed-off-by: Tiep Le <[email protected]> --------- Signed-off-by: Tiep Le <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <[email protected]>
…project#342) * nb Signed-off-by: Berke <[email protected]> * init changes Signed-off-by: Berke <[email protected]> * docker Signed-off-by: Berke <[email protected]> * example data Signed-off-by: Berke <[email protected]> * docs(readme): update, add commands Signed-off-by: Berke <[email protected]> * fix: formatting, data sources Signed-off-by: Berke <[email protected]> * docs(readme): update instructions, add comments Signed-off-by: Berke <[email protected]> * fix: rm unused parts Signed-off-by: Berke <[email protected]> * fix: image name, compose env vars Signed-off-by: Berke <[email protected]> * fix: rm unused part Signed-off-by: Berke <[email protected]> * fix: logging name Signed-off-by: Berke <[email protected]> * fix: env var Signed-off-by: Berke <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Berke <[email protected]> * fix: rename pw docker Signed-off-by: Berke <[email protected]> * docs(readme): update input sources Signed-off-by: Berke <[email protected]> * nb Signed-off-by: Berke <[email protected]> * init changes Signed-off-by: Berke <[email protected]> * fix: formatting, data sources Signed-off-by: Berke <[email protected]> * docs(readme): update instructions, add comments Signed-off-by: Berke <[email protected]> * fix: rm unused part Signed-off-by: Berke <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Berke <[email protected]> * fix: rename pw docker Signed-off-by: Berke <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Berke <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat: mv vector store, naming, clarify instructions, improve ingestion components Signed-off-by: Berke <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests: add pw retriever test fix: update docker to include libmagic Signed-off-by: Berke <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * implement suggestions from review, entrypoint, reqs, comments, https_proxy. Signed-off-by: Berke <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: update docker tags in test and readme Signed-off-by: Berke <[email protected]> * tests: add separate pathway vectorstore test Signed-off-by: Berke <[email protected]> --------- Signed-off-by: Berke <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sihan Chen <[email protected]> Signed-off-by: Dong, Bo1 <[email protected]>
* initial commit Signed-off-by: BaoHuiling <[email protected]> * save Signed-off-by: BaoHuiling <[email protected]> * add readme, test script, fix bug Signed-off-by: BaoHuiling <[email protected]> * update video URL Signed-off-by: BaoHuiling <[email protected]> * use default Signed-off-by: BaoHuiling <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update core dependency Signed-off-by: BaoHuiling <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * use p 5000 Signed-off-by: BaoHuiling <[email protected]> * use 5037 Signed-off-by: BaoHuiling <[email protected]> * update ctnr name Signed-off-by: BaoHuiling <[email protected]> * remove langsmith Signed-off-by: BaoHuiling <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add rerank algo desc in readme Signed-off-by: BaoHuiling <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: BaoHuiling <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: chen, suyue <[email protected]> Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: zepan <[email protected]> Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: BaoHuiling <[email protected]> Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: chensuyue <[email protected]> Signed-off-by: zepan <[email protected]> Signed-off-by: Dong, Bo1 <[email protected]>
* add microservice for intent detection Signed-off-by: Liangyx2 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update license copyright Signed-off-by: Liangyx2 <[email protected]> * add ut Signed-off-by: Liangyx2 <[email protected]> * refine Signed-off-by: Liangyx2 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update folder Signed-off-by: Liangyx2 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix test Signed-off-by: Liangyx2 <[email protected]> --------- Signed-off-by: Liangyx2 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: zepan <[email protected]> Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: letonghan <[email protected]> Signed-off-by: Dong, Bo1 <[email protected]>
* Check if the document exists. Signed-off-by: zepan <[email protected]> * Add flag output. Signed-off-by: zepan <[email protected]> * Modify nginx readme. Signed-off-by: zepan <[email protected]> * Modify document detection logic Signed-off-by: zepan <[email protected]> --------- Signed-off-by: zepan <[email protected]> Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
a32543254
requested review from
XinyuYe-Intel,
letonghan,
Spycsh and
lvliang-intel
as code owners
September 3, 2024 06:33
Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
for more information, see https://pre-commit.ci
Signed-off-by: Dong, Bo1 <[email protected]>
for more information, see https://pre-commit.ci
Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
lvliang-intel
approved these changes
Sep 10, 2024
Spycsh
approved these changes
Sep 10, 2024
lkk12014402
pushed a commit
that referenced
this pull request
Sep 19, 2024
Signed-off-by: chensuyue <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Optimize the rerank with backend ref module
<style> </style>Issues
List the issue or RFC link this PR is working on. If there is no such link, please mark it as
n/a
.Type of change
List the type of change like below. Please delete options that are not relevant.
Dependencies
List the newly introduced 3rd party dependency if exists.
Tests
Describe the tests that you ran to verify your changes.