Add knowledge graph components #171

XinyaoWa · 2024-06-13T09:24:35Z

Description

Add Knowledge graph components, which is based on Neo4j and can work as an enhanced retriever

Issues

n/a.

Type of change

New feature (non-breaking change which adds new functionality)

Dependencies

neo4j
langchain
tiktoken
langchain_openai
langchainhub
langchain_community
sentence-transformers

Tests

n/a.

codecov · 2024-06-17T10:51:50Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Files	Coverage Δ
comps/cores/mega/constants.py	`98.07% <100.00%> (ø)`
comps/cores/proto/docarray.py	`100.00% <100.00%> (ø)`

comps/knowledgegraphs/langchain/knowledge_graph.py

xuechendi · 2024-06-18T01:38:59Z

@XinyaoWa , The PR looks good, to make it clearer for other reviewer, please also provide info related to

a workflow diagram
screenshot for both rag query and cypher query

Signed-off-by: XuhuiRen <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

* Fix RAG performance issues Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

* add statistics * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

* Add More Contents to the Table MicroService Signed-off-by: zehao-intel <[email protected]> * reorder Signed-off-by: zehao-intel <[email protected]> * Update README.md * refine structure Signed-off-by: zehao-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <[email protected]> * refine table Signed-off-by: zehao-intel <[email protected]> * put llm to the ground Signed-off-by: zehao-intel <[email protected]> --------- Signed-off-by: zehao-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

* add python coverage Signed-off-by: chensuyue <[email protected]> * docs update Signed-off-by: chensuyue <[email protected]> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

…vllm release (opea-project#137) Signed-off-by: tianyil1 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

Signed-off-by: Xinyao Wang <[email protected]>

for more information, see https://pre-commit.ci

* move file to langchain folder Signed-off-by: letonghan <[email protected]> * support dataprep with llama_index Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add e2e test script Signed-off-by: letonghan <[email protected]> * update test script name Signed-off-by: letonghan <[email protected]> --------- Signed-off-by: letonghan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

* fix stream=false doesn't work issue Signed-off-by: letonghan <[email protected]> * support embedding comp with llama_index Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <[email protected]> * reorder Signed-off-by: zehao-intel <[email protected]> * Update README.md * refine structure Signed-off-by: zehao-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <[email protected]> * refine table Signed-off-by: zehao-intel <[email protected]> * put llm to the ground Signed-off-by: zehao-intel <[email protected]> --------- Signed-off-by: zehao-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <[email protected]> * docs update Signed-off-by: chensuyue <[email protected]> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * support embedding comp with llama_index Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add test script for embedding llama_inde Signed-off-by: letonghan <[email protected]> * remove conflict requirements Signed-off-by: letonghan <[email protected]> * update test script Signed-off-by: letonghan <[email protected]> * udpate Signed-off-by: letonghan <[email protected]> * update Signed-off-by: letonghan <[email protected]> * update Signed-off-by: letonghan <[email protected]> * fix ut issue Signed-off-by: letonghan <[email protected]> --------- Signed-off-by: letonghan <[email protected]> Signed-off-by: zehao-intel <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: tianyil1 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: Tianyi Liu <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

* Add Ollama Support Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

Signed-off-by: chensuyue <[email protected]> Signed-off-by: letonghan <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

* add translation gateway Signed-off-by: zehao-intel <[email protected]> * fix import Signed-off-by: zehao-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: zehao-intel <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

* Update LLM readme Signed-off-by: lvliang-intel <[email protected]> * update readme Signed-off-by: lvliang-intel <[email protected]> * update tgi readme Signed-off-by: lvliang-intel <[email protected]> * rollback requirements.txt Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: lvliang-intel <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

* Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <[email protected]> * docs update Signed-off-by: chensuyue <[email protected]> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: jinjunzh <[email protected]> * add milvus microservice Signed-off-by: jinjunzh <[email protected]> * fix the typo Signed-off-by: jinjunzh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: jinjunzh <[email protected]> --------- Signed-off-by: chensuyue <[email protected]> Signed-off-by: jinjunzh <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

Signed-off-by: Sun, Xuehao <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

* add ray version document to redis Signed-off-by: Chendi Xue <[email protected]> * update test Signed-off-by: Chendi Xue <[email protected]> * Add test Signed-off-by: Chendi Xue <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add TIMEOUT in container environment and return status Signed-off-by: Chendi Xue <[email protected]> * rebase on new folder layout Signed-off-by: Chendi Xue <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chendi Xue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

Signed-off-by: Sun, Xuehao <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

Signed-off-by: changwangss <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

* supported bce model for rerank. Signed-off-by: Xinyu Ye <[email protected]> * change folder Signed-off-by: Xinyu Ye <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change path in test file. Signed-off-by: Xinyu Ye <[email protected]> --------- Signed-off-by: Xinyu Ye <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

* Support vLLM XFT serving Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix access vllm issue Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add permission for run.sh Signed-off-by: lvliang-intel <[email protected]> * add readme Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix proxy issue Signed-off-by: lvliang-intel <[email protected]> --------- Signed-off-by: lvliang-intel <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

* update dataprep readme Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: letonghan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed milvus port conflict issues during deployment * align port for unified retrieval microservice --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

Signed-off-by: Xinyao Wang <[email protected]>

for more information, see https://pre-commit.ci

Signed-off-by: Xinyao Wang <[email protected]>

for more information, see https://pre-commit.ci

Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

* Add a new embedding MosecEmbedding. Signed-off-by: Jincheng Miao <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jincheng Miao <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

Signed-off-by: chensuyue <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

Signed-off-by: Xinyao Wang <[email protected]>

for more information, see https://pre-commit.ci Signed-off-by: Xinyao Wang <[email protected]>

XinyaoWa · 2024-06-18T08:24:20Z

@XinyaoWa , The PR looks good, to make it clearer for other reviewer, please also provide info related to

a workflow diagram

screenshot for both rag query and cypher query

Thanks for comments, have updated readme, please have a check.

Signed-off-by: Xinyao Wang <[email protected]>

for more information, see https://pre-commit.ci

XinyaoWa · 2024-06-19T01:48:36Z

@lvliang-intel Hi Lvliang, please have a review for this PR, thanks a lot!

lvliang-intel · 2024-06-19T13:54:16Z

@XinyaoWa,
Please resolve the conflict.

for more information, see https://pre-commit.ci

XinyaoWa · 2024-06-20T01:40:54Z

@XinyaoWa, Please resolve the conflict.

Resolved.

* enable ragas (opea-project#129) Signed-off-by: XuhuiRen <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Fix RAG performance issues (opea-project#132) * Fix RAG performance issues Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * add microservice level perf statistics (opea-project#135) * add statistics * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <[email protected]> * reorder Signed-off-by: zehao-intel <[email protected]> * Update README.md * refine structure Signed-off-by: zehao-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <[email protected]> * refine table Signed-off-by: zehao-intel <[email protected]> * put llm to the ground Signed-off-by: zehao-intel <[email protected]> --------- Signed-off-by: zehao-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <[email protected]> * docs update Signed-off-by: chensuyue <[email protected]> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * add knowledge graph Signed-off-by: Xinyao Wang <[email protected]> * knowledge graph microservice update Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Support Dataprep Microservice with Llama Index (opea-project#154) * move file to langchain folder Signed-off-by: letonghan <[email protected]> * support dataprep with llama_index Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add e2e test script Signed-off-by: letonghan <[email protected]> * update test script name Signed-off-by: letonghan <[email protected]> --------- Signed-off-by: letonghan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Support Embedding Microservice with Llama Index (opea-project#150) * fix stream=false doesn't work issue Signed-off-by: letonghan <[email protected]> * support embedding comp with llama_index Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <[email protected]> * reorder Signed-off-by: zehao-intel <[email protected]> * Update README.md * refine structure Signed-off-by: zehao-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <[email protected]> * refine table Signed-off-by: zehao-intel <[email protected]> * put llm to the ground Signed-off-by: zehao-intel <[email protected]> --------- Signed-off-by: zehao-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <[email protected]> * docs update Signed-off-by: chensuyue <[email protected]> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * support embedding comp with llama_index Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add test script for embedding llama_inde Signed-off-by: letonghan <[email protected]> * remove conflict requirements Signed-off-by: letonghan <[email protected]> * update test script Signed-off-by: letonghan <[email protected]> * udpate Signed-off-by: letonghan <[email protected]> * update Signed-off-by: letonghan <[email protected]> * update Signed-off-by: letonghan <[email protected]> * fix ut issue Signed-off-by: letonghan <[email protected]> --------- Signed-off-by: letonghan <[email protected]> Signed-off-by: zehao-intel <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: tianyil1 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: Tianyi Liu <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Support Ollama microservice (opea-project#142) * Add Ollama Support Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Fix dataprep microservice path issue (opea-project#163) Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * update CI to support dataprep_redis path level change (opea-project#155) Signed-off-by: chensuyue <[email protected]> Signed-off-by: letonghan <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Add Gateway for Translation (opea-project#169) * add translation gateway Signed-off-by: zehao-intel <[email protected]> * fix import Signed-off-by: zehao-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: zehao-intel <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Update LLM readme (opea-project#172) * Update LLM readme Signed-off-by: lvliang-intel <[email protected]> * update readme Signed-off-by: lvliang-intel <[email protected]> * update tgi readme Signed-off-by: lvliang-intel <[email protected]> * rollback requirements.txt Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: lvliang-intel <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * add milvus microservice (opea-project#158) * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <[email protected]> * docs update Signed-off-by: chensuyue <[email protected]> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: jinjunzh <[email protected]> * add milvus microservice Signed-off-by: jinjunzh <[email protected]> * fix the typo Signed-off-by: jinjunzh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: jinjunzh <[email protected]> --------- Signed-off-by: chensuyue <[email protected]> Signed-off-by: jinjunzh <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * enable python coverage (opea-project#149) Signed-off-by: Sun, Xuehao <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Add Ray version for multi file process (opea-project#119) * add ray version document to redis Signed-off-by: Chendi Xue <[email protected]> * update test Signed-off-by: Chendi Xue <[email protected]> * Add test Signed-off-by: Chendi Xue <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add TIMEOUT in container environment and return status Signed-off-by: Chendi Xue <[email protected]> * rebase on new folder layout Signed-off-by: Chendi Xue <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chendi Xue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Add codecov (opea-project#178) Signed-off-by: Sun, Xuehao <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Rename lm-eval folder to utils/lm-eval (opea-project#179) Signed-off-by: changwangss <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Support rerank and retrieval of RAG OPT (opea-project#164) * supported bce model for rerank. Signed-off-by: Xinyu Ye <[email protected]> * change folder Signed-off-by: Xinyu Ye <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change path in test file. Signed-off-by: Xinyu Ye <[email protected]> --------- Signed-off-by: Xinyu Ye <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Support vLLM XFT LLM microservice (opea-project#174) * Support vLLM XFT serving Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix access vllm issue Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add permission for run.sh Signed-off-by: lvliang-intel <[email protected]> * add readme Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix proxy issue Signed-off-by: lvliang-intel <[email protected]> --------- Signed-off-by: lvliang-intel <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Update Dataprep Microservice README (opea-project#173) * update dataprep readme Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: letonghan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * fixed milvus port conflict issues during deployment (opea-project#183) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed milvus port conflict issues during deployment * align port for unified retrieval microservice --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * remove Signed-off-by: Xinyao Wang <[email protected]> * remove hard address Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update readme Signed-off-by: Xinyao Wang <[email protected]> * add example data and ingestion Signed-off-by: Xinyao Wang <[email protected]> * fix typ Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataprep timeout issue (opea-project#203) Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Add a new embedding MosecEmbedding (opea-project#182) * Add a new embedding MosecEmbedding. Signed-off-by: Jincheng Miao <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jincheng Miao <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * expand timeout for microservice test (opea-project#208) Signed-off-by: chensuyue <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * fix typo Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Xinyao Wang <[email protected]> * fix requirement Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: XuhuiRen <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: zehao-intel <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: tianyil1 <[email protected]> Signed-off-by: letonghan <[email protected]> Signed-off-by: jinjunzh <[email protected]> Signed-off-by: Sun, Xuehao <[email protected]> Signed-off-by: Chendi Xue <[email protected]> Signed-off-by: changwangss <[email protected]> Signed-off-by: Xinyu Ye <[email protected]> Signed-off-by: Jincheng Miao <[email protected]> Co-authored-by: XuhuiRen <[email protected]> Co-authored-by: lvliang-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: Tianyi Liu <[email protected]> Co-authored-by: Letong Han <[email protected]> Co-authored-by: jasperzhu <[email protected]> Co-authored-by: Chendi.Xue <[email protected]> Co-authored-by: Sun, Xuehao <[email protected]> Co-authored-by: Wang, Chang <[email protected]> Co-authored-by: XinyuYe-Intel <[email protected]> Co-authored-by: Jincheng Miao <[email protected]> Signed-off-by: sharanshirodkar7 <[email protected]>

* enable ragas (opea-project#129) Signed-off-by: XuhuiRen <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Fix RAG performance issues (opea-project#132) * Fix RAG performance issues Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * add microservice level perf statistics (opea-project#135) * add statistics * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <[email protected]> * reorder Signed-off-by: zehao-intel <[email protected]> * Update README.md * refine structure Signed-off-by: zehao-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <[email protected]> * refine table Signed-off-by: zehao-intel <[email protected]> * put llm to the ground Signed-off-by: zehao-intel <[email protected]> --------- Signed-off-by: zehao-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <[email protected]> * docs update Signed-off-by: chensuyue <[email protected]> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * add knowledge graph Signed-off-by: Xinyao Wang <[email protected]> * knowledge graph microservice update Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Support Dataprep Microservice with Llama Index (opea-project#154) * move file to langchain folder Signed-off-by: letonghan <[email protected]> * support dataprep with llama_index Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add e2e test script Signed-off-by: letonghan <[email protected]> * update test script name Signed-off-by: letonghan <[email protected]> --------- Signed-off-by: letonghan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Support Embedding Microservice with Llama Index (opea-project#150) * fix stream=false doesn't work issue Signed-off-by: letonghan <[email protected]> * support embedding comp with llama_index Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <[email protected]> * reorder Signed-off-by: zehao-intel <[email protected]> * Update README.md * refine structure Signed-off-by: zehao-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <[email protected]> * refine table Signed-off-by: zehao-intel <[email protected]> * put llm to the ground Signed-off-by: zehao-intel <[email protected]> --------- Signed-off-by: zehao-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <[email protected]> * docs update Signed-off-by: chensuyue <[email protected]> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * support embedding comp with llama_index Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add test script for embedding llama_inde Signed-off-by: letonghan <[email protected]> * remove conflict requirements Signed-off-by: letonghan <[email protected]> * update test script Signed-off-by: letonghan <[email protected]> * udpate Signed-off-by: letonghan <[email protected]> * update Signed-off-by: letonghan <[email protected]> * update Signed-off-by: letonghan <[email protected]> * fix ut issue Signed-off-by: letonghan <[email protected]> --------- Signed-off-by: letonghan <[email protected]> Signed-off-by: zehao-intel <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: tianyil1 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: Tianyi Liu <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Support Ollama microservice (opea-project#142) * Add Ollama Support Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Fix dataprep microservice path issue (opea-project#163) Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * update CI to support dataprep_redis path level change (opea-project#155) Signed-off-by: chensuyue <[email protected]> Signed-off-by: letonghan <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Add Gateway for Translation (opea-project#169) * add translation gateway Signed-off-by: zehao-intel <[email protected]> * fix import Signed-off-by: zehao-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: zehao-intel <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Update LLM readme (opea-project#172) * Update LLM readme Signed-off-by: lvliang-intel <[email protected]> * update readme Signed-off-by: lvliang-intel <[email protected]> * update tgi readme Signed-off-by: lvliang-intel <[email protected]> * rollback requirements.txt Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: lvliang-intel <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * add milvus microservice (opea-project#158) * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <[email protected]> * docs update Signed-off-by: chensuyue <[email protected]> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: jinjunzh <[email protected]> * add milvus microservice Signed-off-by: jinjunzh <[email protected]> * fix the typo Signed-off-by: jinjunzh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: jinjunzh <[email protected]> --------- Signed-off-by: chensuyue <[email protected]> Signed-off-by: jinjunzh <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * enable python coverage (opea-project#149) Signed-off-by: Sun, Xuehao <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Add Ray version for multi file process (opea-project#119) * add ray version document to redis Signed-off-by: Chendi Xue <[email protected]> * update test Signed-off-by: Chendi Xue <[email protected]> * Add test Signed-off-by: Chendi Xue <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add TIMEOUT in container environment and return status Signed-off-by: Chendi Xue <[email protected]> * rebase on new folder layout Signed-off-by: Chendi Xue <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chendi Xue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Add codecov (opea-project#178) Signed-off-by: Sun, Xuehao <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Rename lm-eval folder to utils/lm-eval (opea-project#179) Signed-off-by: changwangss <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Support rerank and retrieval of RAG OPT (opea-project#164) * supported bce model for rerank. Signed-off-by: Xinyu Ye <[email protected]> * change folder Signed-off-by: Xinyu Ye <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change path in test file. Signed-off-by: Xinyu Ye <[email protected]> --------- Signed-off-by: Xinyu Ye <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Support vLLM XFT LLM microservice (opea-project#174) * Support vLLM XFT serving Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix access vllm issue Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add permission for run.sh Signed-off-by: lvliang-intel <[email protected]> * add readme Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix proxy issue Signed-off-by: lvliang-intel <[email protected]> --------- Signed-off-by: lvliang-intel <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Update Dataprep Microservice README (opea-project#173) * update dataprep readme Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: letonghan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * fixed milvus port conflict issues during deployment (opea-project#183) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed milvus port conflict issues during deployment * align port for unified retrieval microservice --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * remove Signed-off-by: Xinyao Wang <[email protected]> * remove hard address Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update readme Signed-off-by: Xinyao Wang <[email protected]> * add example data and ingestion Signed-off-by: Xinyao Wang <[email protected]> * fix typ Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataprep timeout issue (opea-project#203) Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Add a new embedding MosecEmbedding (opea-project#182) * Add a new embedding MosecEmbedding. Signed-off-by: Jincheng Miao <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jincheng Miao <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * expand timeout for microservice test (opea-project#208) Signed-off-by: chensuyue <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * fix typo Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Xinyao Wang <[email protected]> * fix requirement Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: XuhuiRen <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: zehao-intel <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: tianyil1 <[email protected]> Signed-off-by: letonghan <[email protected]> Signed-off-by: jinjunzh <[email protected]> Signed-off-by: Sun, Xuehao <[email protected]> Signed-off-by: Chendi Xue <[email protected]> Signed-off-by: changwangss <[email protected]> Signed-off-by: Xinyu Ye <[email protected]> Signed-off-by: Jincheng Miao <[email protected]> Co-authored-by: XuhuiRen <[email protected]> Co-authored-by: lvliang-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: Tianyi Liu <[email protected]> Co-authored-by: Letong Han <[email protected]> Co-authored-by: jasperzhu <[email protected]> Co-authored-by: Chendi.Xue <[email protected]> Co-authored-by: Sun, Xuehao <[email protected]> Co-authored-by: Wang, Chang <[email protected]> Co-authored-by: XinyuYe-Intel <[email protected]> Co-authored-by: Jincheng Miao <[email protected]> Signed-off-by: Yogesh Pandey <[email protected]>

* enable ragas (opea-project#129) Signed-off-by: XuhuiRen <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Fix RAG performance issues (opea-project#132) * Fix RAG performance issues Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * add microservice level perf statistics (opea-project#135) * add statistics * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <[email protected]> * reorder Signed-off-by: zehao-intel <[email protected]> * Update README.md * refine structure Signed-off-by: zehao-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <[email protected]> * refine table Signed-off-by: zehao-intel <[email protected]> * put llm to the ground Signed-off-by: zehao-intel <[email protected]> --------- Signed-off-by: zehao-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <[email protected]> * docs update Signed-off-by: chensuyue <[email protected]> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * add knowledge graph Signed-off-by: Xinyao Wang <[email protected]> * knowledge graph microservice update Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Support Dataprep Microservice with Llama Index (opea-project#154) * move file to langchain folder Signed-off-by: letonghan <[email protected]> * support dataprep with llama_index Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add e2e test script Signed-off-by: letonghan <[email protected]> * update test script name Signed-off-by: letonghan <[email protected]> --------- Signed-off-by: letonghan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Support Embedding Microservice with Llama Index (opea-project#150) * fix stream=false doesn't work issue Signed-off-by: letonghan <[email protected]> * support embedding comp with llama_index Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add More Contents to the Table of MicroService (opea-project#141) * Add More Contents to the Table MicroService Signed-off-by: zehao-intel <[email protected]> * reorder Signed-off-by: zehao-intel <[email protected]> * Update README.md * refine structure Signed-off-by: zehao-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix model Signed-off-by: zehao-intel <[email protected]> * refine table Signed-off-by: zehao-intel <[email protected]> * put llm to the ground Signed-off-by: zehao-intel <[email protected]> --------- Signed-off-by: zehao-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <[email protected]> * docs update Signed-off-by: chensuyue <[email protected]> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Enable vLLM Gaudi support for LLM service based on officially habana vllm release (opea-project#137) Signed-off-by: tianyil1 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * support embedding comp with llama_index Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add test script for embedding llama_inde Signed-off-by: letonghan <[email protected]> * remove conflict requirements Signed-off-by: letonghan <[email protected]> * update test script Signed-off-by: letonghan <[email protected]> * udpate Signed-off-by: letonghan <[email protected]> * update Signed-off-by: letonghan <[email protected]> * update Signed-off-by: letonghan <[email protected]> * fix ut issue Signed-off-by: letonghan <[email protected]> --------- Signed-off-by: letonghan <[email protected]> Signed-off-by: zehao-intel <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: tianyil1 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: Tianyi Liu <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Support Ollama microservice (opea-project#142) * Add Ollama Support Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Fix dataprep microservice path issue (opea-project#163) Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * update CI to support dataprep_redis path level change (opea-project#155) Signed-off-by: chensuyue <[email protected]> Signed-off-by: letonghan <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Add Gateway for Translation (opea-project#169) * add translation gateway Signed-off-by: zehao-intel <[email protected]> * fix import Signed-off-by: zehao-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: zehao-intel <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Update LLM readme (opea-project#172) * Update LLM readme Signed-off-by: lvliang-intel <[email protected]> * update readme Signed-off-by: lvliang-intel <[email protected]> * update tgi readme Signed-off-by: lvliang-intel <[email protected]> * rollback requirements.txt Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: lvliang-intel <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * add milvus microservice (opea-project#158) * Use common security content for OPEA projects (opea-project#151) * add python coverage Signed-off-by: chensuyue <[email protected]> * docs update Signed-off-by: chensuyue <[email protected]> * Revert "add python coverage" This reverts commit 69615b1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: chensuyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: jinjunzh <[email protected]> * add milvus microservice Signed-off-by: jinjunzh <[email protected]> * fix the typo Signed-off-by: jinjunzh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: jinjunzh <[email protected]> --------- Signed-off-by: chensuyue <[email protected]> Signed-off-by: jinjunzh <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * enable python coverage (opea-project#149) Signed-off-by: Sun, Xuehao <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Add Ray version for multi file process (opea-project#119) * add ray version document to redis Signed-off-by: Chendi Xue <[email protected]> * update test Signed-off-by: Chendi Xue <[email protected]> * Add test Signed-off-by: Chendi Xue <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add TIMEOUT in container environment and return status Signed-off-by: Chendi Xue <[email protected]> * rebase on new folder layout Signed-off-by: Chendi Xue <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chendi Xue <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Add codecov (opea-project#178) Signed-off-by: Sun, Xuehao <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Rename lm-eval folder to utils/lm-eval (opea-project#179) Signed-off-by: changwangss <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Support rerank and retrieval of RAG OPT (opea-project#164) * supported bce model for rerank. Signed-off-by: Xinyu Ye <[email protected]> * change folder Signed-off-by: Xinyu Ye <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change path in test file. Signed-off-by: Xinyu Ye <[email protected]> --------- Signed-off-by: Xinyu Ye <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Support vLLM XFT LLM microservice (opea-project#174) * Support vLLM XFT serving Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix access vllm issue Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add permission for run.sh Signed-off-by: lvliang-intel <[email protected]> * add readme Signed-off-by: lvliang-intel <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix proxy issue Signed-off-by: lvliang-intel <[email protected]> --------- Signed-off-by: lvliang-intel <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * Update Dataprep Microservice README (opea-project#173) * update dataprep readme Signed-off-by: letonghan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: letonghan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * fixed milvus port conflict issues during deployment (opea-project#183) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed milvus port conflict issues during deployment * align port for unified retrieval microservice --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * remove Signed-off-by: Xinyao Wang <[email protected]> * remove hard address Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update readme Signed-off-by: Xinyao Wang <[email protected]> * add example data and ingestion Signed-off-by: Xinyao Wang <[email protected]> * fix typ Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataprep timeout issue (opea-project#203) Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * Add a new embedding MosecEmbedding (opea-project#182) * Add a new embedding MosecEmbedding. Signed-off-by: Jincheng Miao <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jincheng Miao <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]> * expand timeout for microservice test (opea-project#208) Signed-off-by: chensuyue <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> * fix typo Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Xinyao Wang <[email protected]> * fix requirement Signed-off-by: Xinyao Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: XuhuiRen <[email protected]> Signed-off-by: Xinyao Wang <[email protected]> Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: zehao-intel <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: tianyil1 <[email protected]> Signed-off-by: letonghan <[email protected]> Signed-off-by: jinjunzh <[email protected]> Signed-off-by: Sun, Xuehao <[email protected]> Signed-off-by: Chendi Xue <[email protected]> Signed-off-by: changwangss <[email protected]> Signed-off-by: Xinyu Ye <[email protected]> Signed-off-by: Jincheng Miao <[email protected]> Co-authored-by: XuhuiRen <[email protected]> Co-authored-by: lvliang-intel <[email protected]> Co-authored-by: Sihan Chen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zehao-intel <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: Tianyi Liu <[email protected]> Co-authored-by: Letong Han <[email protected]> Co-authored-by: jasperzhu <[email protected]> Co-authored-by: Chendi.Xue <[email protected]> Co-authored-by: Sun, Xuehao <[email protected]> Co-authored-by: Wang, Chang <[email protected]> Co-authored-by: XinyuYe-Intel <[email protected]> Co-authored-by: Jincheng Miao <[email protected]> Signed-off-by: Daniel Whitenack <[email protected]>

XinyaoWa changed the title ~~[Draft] Add knowledge graph components~~ Add knowledge graph components Jun 17, 2024

xuechendi reviewed Jun 18, 2024

View reviewed changes

comps/knowledgegraphs/langchain/knowledge_graph.py Outdated Show resolved Hide resolved

XuhuiRen and others added 26 commits June 18, 2024 16:10

enable ragas (opea-project#129)

f84588d

Signed-off-by: XuhuiRen <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

Fix RAG performance issues (opea-project#132)

986fe6b

* Fix RAG performance issues Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

add knowledge graph

d2cf359

Signed-off-by: Xinyao Wang <[email protected]>

knowledge graph microservice update

794d8b5

Signed-off-by: Xinyao Wang <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

31695be

for more information, see https://pre-commit.ci

Support Ollama microservice (opea-project#142)

6959256

* Add Ollama Support Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

Fix dataprep microservice path issue (opea-project#163)

b94aa12

Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

update CI to support dataprep_redis path level change (opea-project#155)

2516eb1

Signed-off-by: chensuyue <[email protected]> Signed-off-by: letonghan <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

enable python coverage (opea-project#149)

bec961d

Signed-off-by: Sun, Xuehao <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

Add codecov (opea-project#178)

bdc9704

Signed-off-by: Sun, Xuehao <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Xinyao Wang <[email protected]>

Rename lm-eval folder to utils/lm-eval (opea-project#179)

fef866a

Signed-off-by: changwangss <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

remove

bcba113

Signed-off-by: Xinyao Wang <[email protected]>

XinyaoWa and others added 11 commits June 18, 2024 16:15

remove hard address

0379bcf

Signed-off-by: Xinyao Wang <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

a6c47ad

for more information, see https://pre-commit.ci

update readme

58ae6a6

Signed-off-by: Xinyao Wang <[email protected]>

add example data and ingestion

afab31c

Signed-off-by: Xinyao Wang <[email protected]>

fix typ

7c6f971

Signed-off-by: Xinyao Wang <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

d9ba9d1

for more information, see https://pre-commit.ci

Fix dataprep timeout issue (opea-project#203)

4fca625

Signed-off-by: lvliang-intel <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

expand timeout for microservice test (opea-project#208)

f83ba94

Signed-off-by: chensuyue <[email protected]> Signed-off-by: Xinyao Wang <[email protected]>

fix typo

f10f99c

Signed-off-by: Xinyao Wang <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

282e9ea

for more information, see https://pre-commit.ci Signed-off-by: Xinyao Wang <[email protected]>

fix requirement

460407f

Signed-off-by: Xinyao Wang <[email protected]>

XinyaoWa force-pushed the kg branch 2 times, most recently from 6499908 to 282e9ea Compare June 18, 2024 09:32

XinyaoWa and others added 2 commits June 18, 2024 17:36

Merge branch 'main' into kg

585f51b

[pre-commit.ci] auto fixes from pre-commit.com hooks

796c480

for more information, see https://pre-commit.ci

XinyaoWa and others added 2 commits June 20, 2024 09:40

Merge branch 'main' into kg

0f906d1

[pre-commit.ci] auto fixes from pre-commit.com hooks

bd58085

for more information, see https://pre-commit.ci

Merge branch 'main' into kg

d8266ea

lvliang-intel approved these changes Jun 20, 2024

View reviewed changes

Spycsh approved these changes Jun 20, 2024

View reviewed changes

lvliang-intel merged commit 4c0afd0 into opea-project:main Jun 20, 2024
12 checks passed

XinyaoWa deleted the kg branch November 7, 2024 03:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add knowledge graph components #171

Add knowledge graph components #171

XinyaoWa commented Jun 13, 2024

codecov bot commented Jun 17, 2024 •

edited

Loading

xuechendi commented Jun 18, 2024

XinyaoWa commented Jun 18, 2024

XinyaoWa commented Jun 19, 2024

lvliang-intel commented Jun 19, 2024

XinyaoWa commented Jun 20, 2024

Add knowledge graph components #171

Add knowledge graph components #171

Conversation

XinyaoWa commented Jun 13, 2024

Description

Issues

Type of change

Dependencies

Tests

codecov bot commented Jun 17, 2024 • edited Loading

Codecov Report

xuechendi commented Jun 18, 2024

XinyaoWa commented Jun 18, 2024

XinyaoWa commented Jun 19, 2024

lvliang-intel commented Jun 19, 2024

XinyaoWa commented Jun 20, 2024

codecov bot commented Jun 17, 2024 •

edited

Loading