Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Dataprep Potential Error in get_file #540

Merged
merged 2 commits into from
Aug 21, 2024
Merged

Fix Dataprep Potential Error in get_file #540

merged 2 commits into from
Aug 21, 2024

Conversation

letonghan
Copy link
Collaborator

@letonghan letonghan commented Aug 21, 2024 •
edited
Loading

Description

Fix /v1/dataprep/get_file error of getting file list from an empty index.
Check index existance before getting data from db.
Refine dataprep logs.

Issues

n/a

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)
  • Others (enhancement, documentation, validation, etc.)

Dependencies

None

Tests

Local tested

@letonghan letonghan changed the title Fix Dataprep Error in get_file Fix Dataprep Potential Error in get_file Aug 21, 2024
@lvliang-intel lvliang-intel merged commit 04ff8bf into opea-project:main Aug 21, 2024
9 checks passed
chensuyue pushed a commit that referenced this pull request Aug 21, 2024
@letonghan @chensuyue
Fix Dataprep Potential Error in get_file (#540)
8aa27bc 
* fix get file error & refine logs

Signed-off-by: letonghan <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: letonghan <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
(cherry picked from commit 04ff8bf)
tileintel pushed a commit to siddhivelankar23/GenAIComps that referenced this pull request Aug 22, 2024
@letonghan @pre-commit-ci @siddhivelankar23
Fix Dataprep Potential Error in get_file (opea-project#540)
27441ba 
* fix get file error & refine logs

Signed-off-by: letonghan <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: letonghan <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: siddhivelankar23 <[email protected]>
srinarayan-srikanthan pushed a commit to siddhivelankar23/GenAIComps that referenced this pull request Sep 1, 2024
@letonghan @pre-commit-ci @srinarayan-srikanthan
Fix Dataprep Potential Error in get_file (opea-project#540)
a44e936 
* fix get file error & refine logs

Signed-off-by: letonghan <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: letonghan <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: srinarayan-srikanthan <[email protected]>
sharanshirodkar7 pushed a commit to predictionguard/pg-GenAIComps that referenced this pull request Sep 3, 2024
@letonghan @pre-commit-ci @sharanshirodkar7
Fix Dataprep Potential Error in get_file (opea-project#540)
52fcca4 
* fix get file error & refine logs

Signed-off-by: letonghan <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: letonghan <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
yinghu5 pushed a commit that referenced this pull request Sep 4, 2024
@srinarayan-srikanthan @pre-commit-ci
@XuehaoSun @letonghan @ZailiWang @xuechendi @Spycsh @BaoHuiling @XuhuiRen @XinyuYe-Intel @lkk12014402 @chensuyue @ashahba @qgao007
adding embedding support for CLIP based models for VideoRAGQnA exampl…
2a53e25 
…e for v0.9 (#538)

* clip embedding support

Signed-off-by: srinarayan-srikanthan <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: srinarayan-srikanthan <[email protected]>

* test script for embedding

Signed-off-by: srinarayan-srikanthan <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: srinarayan-srikanthan <[email protected]>

* fix freeze workflow (#522)

Signed-off-by: Sun, Xuehao <[email protected]>
Signed-off-by: srinarayan-srikanthan <[email protected]>

* Fix Dataprep Potential Error in get_file (#540)

* fix get file error & refine logs

Signed-off-by: letonghan <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: letonghan <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: srinarayan-srikanthan <[email protected]>

* Support SearchedDoc input type in LLM for No Rerank Pipeline (#541)

Signed-off-by: letonghan <[email protected]>
Signed-off-by: srinarayan-srikanthan <[email protected]>

* Add dependency for pdf2image and OCR processing (#421)

Signed-off-by: srinarayan-srikanthan <[email protected]>

* Add local_embedding return 768 length to align with chatqna example (#313)

Signed-off-by: Chendi.Xue <[email protected]>
Signed-off-by: srinarayan-srikanthan <[email protected]>

* add telemetry doc (#536)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: srinarayan-srikanthan <[email protected]>

* Add video-llama LVM microservice under lvms  (#495)

Signed-off-by: BaoHuiling <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: srinarayan-srikanthan <[email protected]>

* Fix the data load issue for structured files (#505)

Signed-off-by: XuhuiRen <[email protected]>
Signed-off-by: srinarayan-srikanthan <[email protected]>

* Add finetuning component (#502)

Signed-off-by: Xinyu Ye <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: lkk <[email protected]>
Co-authored-by: test <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: Letong Han <[email protected]>
Signed-off-by: srinarayan-srikanthan <[email protected]>

* add torchvision into requirements (#546)

Signed-off-by: chensuyue <[email protected]>
Signed-off-by: srinarayan-srikanthan <[email protected]>

* Use Gaudi base images from Dockerhub (#526)

* Use Gaudi base images from Dockerhub

Signed-off-by: Abolfazl Shahbazi <[email protected]>

* Fixing the malformed tag

Signed-off-by: Abolfazl Shahbazi <[email protected]>

* fix another malformed tag

Signed-off-by: Abolfazl Shahbazi <[email protected]>

---------

Signed-off-by: Abolfazl Shahbazi <[email protected]>
Signed-off-by: srinarayan-srikanthan <[email protected]>

* Add toxicity detection microservice (#338)

* Add toxicity detection microservice

Signed-off-by: Qun Gao <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Modification to toxicity plugin PR  (#432)

* changed microservice to use Service.GUARDRAILS and input/output to TextDoc

Signed-off-by: Tyler Wilbers <[email protected]>

* simplify dockerfile to use langchain

Signed-off-by: Tyler Wilbers <[email protected]>

* sort requirements

Signed-off-by: Tyler Wilbers <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Tyler Wilbers <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Minor SPDX header update (#434)

Signed-off-by: Abolfazl Shahbazi <[email protected]>

* Remove 'langsmith' per code review (#534)

Signed-off-by: Abolfazl Shahbazi <[email protected]>

* Add toxicity detection microservices with E2E testing

Signed-off-by: Qun Gao <[email protected]>

---------

Signed-off-by: Qun Gao <[email protected]>
Signed-off-by: Tyler Wilbers <[email protected]>
Signed-off-by: Abolfazl Shahbazi <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Abolfazl Shahbazi <[email protected]>
Co-authored-by: Tyler W <[email protected]>
Signed-off-by: srinarayan-srikanthan <[email protected]>

* rename script and use 5xxx

Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: srinarayan-srikanthan <[email protected]>

* add proxy for build

Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: srinarayan-srikanthan <[email protected]>

* fixed commit issues

Signed-off-by: srinarayan-srikanthan <[email protected]>

* Fix docarray constraint

Signed-off-by: srinarayan-srikanthan <[email protected]>

* updated docarray

Signed-off-by: srinarayan-srikanthan <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: srinarayan-srikanthan <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rm telemetry which cause error in mega

Signed-off-by: BaoHuiling <[email protected]>

* renamed dirs

Signed-off-by: srinarayan-srikanthan <[email protected]>

* renamed test

Signed-off-by: srinarayan-srikanthan <[email protected]>

---------

Signed-off-by: srinarayan-srikanthan <[email protected]>
Signed-off-by: Sun, Xuehao <[email protected]>
Signed-off-by: letonghan <[email protected]>
Signed-off-by: Chendi.Xue <[email protected]>
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: XuhuiRen <[email protected]>
Signed-off-by: Xinyu Ye <[email protected]>
Signed-off-by: chensuyue <[email protected]>
Signed-off-by: Abolfazl Shahbazi <[email protected]>
Signed-off-by: Qun Gao <[email protected]>
Signed-off-by: Tyler Wilbers <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sun, Xuehao <[email protected]>
Co-authored-by: Letong Han <[email protected]>
Co-authored-by: Zaili Wang <[email protected]>
Co-authored-by: Chendi.Xue <[email protected]>
Co-authored-by: Sihan Chen <[email protected]>
Co-authored-by: Huiling Bao <[email protected]>
Co-authored-by: XuhuiRen <[email protected]>
Co-authored-by: XinyuYe-Intel <[email protected]>
Co-authored-by: lkk <[email protected]>
Co-authored-by: test <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: chen, suyue <[email protected]>
Co-authored-by: Abolfazl Shahbazi <[email protected]>
Co-authored-by: qgao007 <[email protected]>
Co-authored-by: Tyler W <[email protected]>
@letonghan letonghan deleted the dataprep/fix_error branch September 23, 2024 03:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ZePan110 ZePan110 ZePan110 approved these changes

@lkk12014402 lkk12014402 lkk12014402 approved these changes

@XinyuYe-Intel XinyuYe-Intel Awaiting requested review from XinyuYe-Intel

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@letonghan @lkk12014402 @ZePan110 @lvliang-intel