Add video-llama LVM microservice under lvms #494
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BaoHuiling
force-pushed
the
huiling-lvm
branch
from
5888b41 to
9e1d604
Compare
BaoHuiling
requested review from
Spycsh,
XuhuiRen,
letonghan and
XinyuYe-Intel
as code owners
BaoHuiling
force-pushed
the
huiling-lvm
branch
2 times, most recently
from
461cfa8 to
0574132
Compare
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
There are both 'from langchain_community.vectorstores import Redis' and 'import redis' in prepare_doc_redis.py. Redis.from_texts_return_keys() is with REDIS_URL as connection, while redis.Redis() takes the combination of REDIS_HOST and REDIS_PORT as connection. When the former and the latter are not the same, it will fail to get data from vector database. Change to use REDIS_URL as the unified way. Signed-off-by: Cathy Zhang <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
* update dataprep with parameter settings Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update port Signed-off-by: letonghan <[email protected]> --------- Signed-off-by: letonghan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: BaoHuiling <[email protected]>
* support delete for milvus Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: letonghan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: zepan <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
* refactor build image build and push workflow Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: BaoHuiling <[email protected]>
* first code of business safety classifier * allow strategy options and update readme * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * disable ray for ml strategy and update test script * add log to pii test Signed-off-by: minmin-intel <[email protected]> * update logging in test gaurdrail * rm llm strategy and change url in test Signed-off-by: minmin-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * delete file check in test and update readme Signed-off-by: minmin-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: minmin-intel <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: chen, suyue <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
* refine dataprep upload_link & ut Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add requirement Signed-off-by: letonghan <[email protected]> * modify container name Signed-off-by: letonghan <[email protected]> --------- Signed-off-by: letonghan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
* prompt_registry: Unifying API endpoint port Unifying all port for API endpoints for prompt registry services with recent register_microservice wrapper function update. Signed-off-by: Yeoh, Hoong Tee <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Yeoh, Hoong Tee <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
…me. (opea-project#412) Signed-off-by: zepan <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
…ct#446) Signed-off-by: Zahidul Haque <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: chensuyue <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
No input model from UI. vLLM and OLLAMA get the model name from the envrionment TGI mode model form ModelID Signed-off-by: Wang, Xigui <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
…format (opea-project#405) Co-authored-by: sys-lpot-val <[email protected]> Co-authored-by: lvliang-intel <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: pallavi jaini <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
…ce (opea-project#417) * remrove deprecated imports and switch to langchain-huggingface Signed-off-by: rbrugaro <[email protected]> * fix typo Signed-off-by: rbrugaro <[email protected]> * set requirements to langchain latest release Signed-off-by: rbrugaro <[email protected]> * remove version for langchain in requirements Signed-off-by: rbrugaro <[email protected]> --------- Signed-off-by: rbrugaro <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
…a-project#444) Signed-off-by: Yogesh <[email protected]> Co-authored-by: Yogesh <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
* Fix validation failure without exit. Signed-off-by: zepan <[email protected]> * Change validate log. Signed-off-by: zepan <[email protected]> * Fix validation failure without exit. Signed-off-by: zepan <[email protected]> * Change validate log. Signed-off-by: zepan <[email protected]> * Change validate log. Signed-off-by: zepan <[email protected]> * Fix issue for reranking-langchain-mosec. Signed-off-by: zepan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix script issue Signed-off-by: zepan <[email protected]> --------- Signed-off-by: zepan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: Lianhao Lu <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: letonghan <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: letonghan <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
* clean test images before test Signed-off-by: chensuyue <[email protected]> * add --no-cache for image build Signed-off-by: chensuyue <[email protected]> --------- Signed-off-by: chensuyue <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: zepan <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: V, Ganesan <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: letonghan <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
…a-project#442) Signed-off-by: Wang, Kai Lawrence <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
…PIEmbedding (opea-project#466) * Fix HuggingFaceEmbedding deprecated in favor of HuggingFaceInferenceAPIEmbedding Signed-off-by: rbrugaro <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: rbrugaro <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: chensuyue <[email protected]> Signed-off-by: zepan <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
* add single query input/output guardrails Signed-off-by: Tyler Wilbers <[email protected]> * removed comment Signed-off-by: Tyler Wilbers <[email protected]> --------- Signed-off-by: Tyler Wilbers <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: letonghan <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
* fix long form asr accuracy * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix test input pad issue --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: chen, suyue <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
* Standardize image names and labels. Signed-off-by: zepan <[email protected]> * Remove test cases that are no longer in use. Signed-off-by: zepan <[email protected]> --------- Signed-off-by: zepan <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
* Change /root/ to /hmoe/user/. Signed-off-by: zepan <[email protected]> * Fix issue. Signed-off-by: zepan <[email protected]> --------- Signed-off-by: zepan <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: zepan <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: Sun, Xuehao <[email protected]> Signed-off-by: NeuralChatBot <[email protected]> Signed-off-by: BaoHuiling <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add support for video-llama LVM, which allows us to chat based on video file with LVM. The first usecase is VideoRAGQnA use case from Multimodal RAG proposal.
Issues
List the issue or RFC link this PR is working on. If there is no such link, please mark it as
n/a.The RFC is still under review. opea-project/docs#49
Type of change
List the type of change like below. Please delete options that are not relevant.
Dependencies
python:3.9-slim git git-lfs faiss-cpu sentencepiece pandas Pillow prometheus_fastapi_instrumentator ftfy decord einops iopath omegaconf opencv-python-headless timm torch torchaudio langchain-core transformers webdataset pytorchvideo DAMO-NLP-SG/Video-LLaMA(github repo) DAMO-NLP-SG/Video-LLaMA-2-7B-Finetuned (huggingface repo)
Tests
Describe the tests that you ran to verify your changes.
https://github.com/siddhivelankar23/GenAIComps/blob/huiling-lvm/tests/test_lvms_video-llama.sh: build, start, validate, clean the microservice