Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update milvus service for dataprep and retriever #255

Merged
merged 15 commits into from
Jul 4, 2024

Conversation

jinjunzh
Copy link
Contributor

@jinjunzh jinjunzh commented Jun 27, 2024

support file upload feature for milvus service to align with redis
support MOSEC embedding API

@jinjunzh jinjunzh changed the title support file upload feature for milvus service update milvus service for dataprep and retriever Jun 27, 2024
@jinjunzh jinjunzh force-pushed the main branch 2 times, most recently from ce12b3e to fd3b174 Compare June 28, 2024 01:10
pre-commit-ci bot and others added 15 commits June 28, 2024 01:20
* Add microservice for table extraction

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update license copyright

Signed-off-by: Liangyx2 <[email protected]>

* DataPrep extract info from table in the docs

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refine

Signed-off-by: Liangyx2 <[email protected]>

* refine

Signed-off-by: Liangyx2 <[email protected]>

* Update prepare_doc_redis.py

* Update prepare_doc_qdrant.py

* Update prepare_doc_milvus.py

---------

Signed-off-by: Liangyx2 <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: XuhuiRen <[email protected]>
Signed-off-by: jinjunzh <[email protected]>
Signed-off-by: Chendi Xue <[email protected]>
Signed-off-by: jinjunzh <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: jinjunzh <[email protected]>
Signed-off-by: chensuyue <[email protected]>
Signed-off-by: jinjunzh <[email protected]>
* fix duplicate test

Signed-off-by: chensuyue <[email protected]>

* for test only

Signed-off-by: chensuyue <[email protected]>

* Revert "for test only"

This reverts commit a7718aa.

---------

Signed-off-by: chensuyue <[email protected]>
Signed-off-by: jinjunzh <[email protected]>
* Add microservice for table extraction

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update license copyright

Signed-off-by: Liangyx2 <[email protected]>

* DataPrep extract info from table in the docs

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refine

Signed-off-by: Liangyx2 <[email protected]>

* refine

Signed-off-by: Liangyx2 <[email protected]>

* Update prepare_doc_redis.py

* Update prepare_doc_qdrant.py

* Update prepare_doc_milvus.py

---------

Signed-off-by: Liangyx2 <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: XuhuiRen <[email protected]>
@ftian1 ftian1 merged commit d7cdab9 into opea-project:main Jul 4, 2024
8 checks passed
yogeshmpandey pushed a commit to yogeshmpandey/GenAIComps that referenced this pull request Jul 10, 2024
* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* DataPrep extract info from table in the docs (opea-project#146)

* Add microservice for table extraction

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update license copyright

Signed-off-by: Liangyx2 <[email protected]>

* DataPrep extract info from table in the docs

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refine

Signed-off-by: Liangyx2 <[email protected]>

* refine

Signed-off-by: Liangyx2 <[email protected]>

* Update prepare_doc_redis.py

* Update prepare_doc_qdrant.py

* Update prepare_doc_milvus.py

---------

Signed-off-by: Liangyx2 <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: XuhuiRen <[email protected]>
Signed-off-by: jinjunzh <[email protected]>

* Remove sensitive info logs (opea-project#251)

Signed-off-by: Chendi Xue <[email protected]>
Signed-off-by: jinjunzh <[email protected]>

* Added support for extracting info from image in the docs (opea-project#120)

Signed-off-by: Ye, Xinyu <[email protected]>
Signed-off-by: jinjunzh <[email protected]>

* enhance statistics ut coverage (opea-project#252)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: jinjunzh <[email protected]>

* bump version (opea-project#253)

Signed-off-by: chensuyue <[email protected]>
Signed-off-by: jinjunzh <[email protected]>

* support file upload feature for milvus service

Signed-off-by: jinjunzh <[email protected]>

* update embedding with MOSEC_EMBEDDING_ENDPOINT

Signed-off-by: jinjunzh <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: jinjunzh <[email protected]>

* support file upload feature for milvus service

Signed-off-by: jinjunzh <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: jinjunzh <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: jinjunzh <[email protected]>

* fix duplicate ci test (opea-project#256)

* fix duplicate test

Signed-off-by: chensuyue <[email protected]>

* for test only

Signed-off-by: chensuyue <[email protected]>

* Revert "for test only"

This reverts commit a7718aa.

---------

Signed-off-by: chensuyue <[email protected]>
Signed-off-by: jinjunzh <[email protected]>

* DataPrep extract info from table in the docs (opea-project#146)

* Add microservice for table extraction

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update license copyright

Signed-off-by: Liangyx2 <[email protected]>

* DataPrep extract info from table in the docs

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refine

Signed-off-by: Liangyx2 <[email protected]>

* refine

Signed-off-by: Liangyx2 <[email protected]>

* Update prepare_doc_redis.py

* Update prepare_doc_qdrant.py

* Update prepare_doc_milvus.py

---------

Signed-off-by: Liangyx2 <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: XuhuiRen <[email protected]>

---------

Signed-off-by: Liangyx2 <[email protected]>
Signed-off-by: jinjunzh <[email protected]>
Signed-off-by: Chendi Xue <[email protected]>
Signed-off-by: Ye, Xinyu <[email protected]>
Signed-off-by: chensuyue <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Liangyx2 <[email protected]>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: XuhuiRen <[email protected]>
Co-authored-by: Chendi.Xue <[email protected]>
Co-authored-by: XinyuYe-Intel <[email protected]>
Co-authored-by: Sihan Chen <[email protected]>
Signed-off-by: Yogesh Pandey <[email protected]>
dwhitena pushed a commit to predictionguard/GenAIComps that referenced this pull request Jul 24, 2024
* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* DataPrep extract info from table in the docs (opea-project#146)

* Add microservice for table extraction

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update license copyright

Signed-off-by: Liangyx2 <[email protected]>

* DataPrep extract info from table in the docs

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refine

Signed-off-by: Liangyx2 <[email protected]>

* refine

Signed-off-by: Liangyx2 <[email protected]>

* Update prepare_doc_redis.py

* Update prepare_doc_qdrant.py

* Update prepare_doc_milvus.py

---------

Signed-off-by: Liangyx2 <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: XuhuiRen <[email protected]>
Signed-off-by: jinjunzh <[email protected]>

* Remove sensitive info logs (opea-project#251)

Signed-off-by: Chendi Xue <[email protected]>
Signed-off-by: jinjunzh <[email protected]>

* Added support for extracting info from image in the docs (opea-project#120)

Signed-off-by: Ye, Xinyu <[email protected]>
Signed-off-by: jinjunzh <[email protected]>

* enhance statistics ut coverage (opea-project#252)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: jinjunzh <[email protected]>

* bump version (opea-project#253)

Signed-off-by: chensuyue <[email protected]>
Signed-off-by: jinjunzh <[email protected]>

* support file upload feature for milvus service

Signed-off-by: jinjunzh <[email protected]>

* update embedding with MOSEC_EMBEDDING_ENDPOINT

Signed-off-by: jinjunzh <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: jinjunzh <[email protected]>

* support file upload feature for milvus service

Signed-off-by: jinjunzh <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: jinjunzh <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: jinjunzh <[email protected]>

* fix duplicate ci test (opea-project#256)

* fix duplicate test

Signed-off-by: chensuyue <[email protected]>

* for test only

Signed-off-by: chensuyue <[email protected]>

* Revert "for test only"

This reverts commit a7718aa.

---------

Signed-off-by: chensuyue <[email protected]>
Signed-off-by: jinjunzh <[email protected]>

* DataPrep extract info from table in the docs (opea-project#146)

* Add microservice for table extraction

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update license copyright

Signed-off-by: Liangyx2 <[email protected]>

* DataPrep extract info from table in the docs

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refine

Signed-off-by: Liangyx2 <[email protected]>

* refine

Signed-off-by: Liangyx2 <[email protected]>

* Update prepare_doc_redis.py

* Update prepare_doc_qdrant.py

* Update prepare_doc_milvus.py

---------

Signed-off-by: Liangyx2 <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: XuhuiRen <[email protected]>

---------

Signed-off-by: Liangyx2 <[email protected]>
Signed-off-by: jinjunzh <[email protected]>
Signed-off-by: Chendi Xue <[email protected]>
Signed-off-by: Ye, Xinyu <[email protected]>
Signed-off-by: chensuyue <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Liangyx2 <[email protected]>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: XuhuiRen <[email protected]>
Co-authored-by: Chendi.Xue <[email protected]>
Co-authored-by: XinyuYe-Intel <[email protected]>
Co-authored-by: Sihan Chen <[email protected]>
Signed-off-by: Daniel Whitenack <[email protected]>
lkk12014402 pushed a commit that referenced this pull request Aug 8, 2024
Signed-off-by: Huiyan2021 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants