Add dynamic DAG #317

Spycsh · 2024-07-17T04:16:51Z

Description

For more complex data flow, we can use dynamic DAG to orchestrate. For example, guardrails may detect a violence prompt and refused to answer earlier in the flow so it will directly return the prompt, rather than keep forwarding to other microservices. In this case, the pre-defined DAG graph need to be modified at runtime, that is where a dynamic DAG is needed.

By simply specifying the black nodes downstream_black_list=['.*'], the orchestrator will know that any nodes matched the pattern .* will be in the black list: just do not forwarding to them. In that case. GuardRails will instead become the leave node in the runtime graph so its result will be returned as the final result.

Issues

n/a

Type of change

List the type of change like below. Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds new functionality)
Breaking change (fix or feature that would break existing design and interface)

Dependencies

None

Tests

add a test and all tests are refactored

tests/cores/mega/test_runtime_graph.py

for more information, see https://pre-commit.ci

codecov · 2024-07-17T05:20:05Z

Codecov Report

Attention: Patch coverage is 67.34694% with 16 lines in your changes missing coverage. Please review.

Files	Coverage Δ
comps/cores/proto/docarray.py	`100.00% <100.00%> (ø)`
comps/cores/mega/orchestrator.py	`84.53% <93.75%> (+2.10%)`	⬆️
comps/cores/mega/gateway.py	`26.48% <0.00%> (ø)`

... and 1 file with indirect coverage changes

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Chendi.Xue <[email protected]>

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Daniel Whitenack <[email protected]>

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Berke <[email protected]>

* Refactor Translation Example Signed-off-by: zehao-intel <[email protected]> * support e2s test Signed-off-by: zehao-intel <[email protected]> Signed-off-by: zehao-intel <[email protected]> Signed-off-by: zehao-intel <[email protected]> Signed-off-by: zehao-intel <[email protected]> * fix test ip_address Signed-off-by: zehao-intel <[email protected]> * update test scripts Signed-off-by: chensuyue <[email protected]> * update test scripts Signed-off-by: chensuyue <[email protected]> * for test Signed-off-by: chensuyue <[email protected]> * fix readme and dockerfile Signed-off-by: zehao-intel <[email protected]> * revert test code Signed-off-by: chensuyue <[email protected]> * remove gaudi test update Signed-off-by: chensuyue <[email protected]> * bug fix Signed-off-by: chensuyue <[email protected]> * fix test xeon Signed-off-by: zehao-intel <[email protected]> * modify mega check Signed-off-by: zehao-intel <[email protected]> Signed-off-by: zehao-intel <[email protected]> Signed-off-by: zehao-intel <[email protected]> * fix ui Signed-off-by: zehao-intel <[email protected]> * fix ut network Signed-off-by: zehao-intel <[email protected]> * fix network Signed-off-by: zehao-intel <[email protected]> * Modify the corresponding format according to the backend new structure. (#317) * Add image build job in docker compose e2e gaudi test in CI (#305) Signed-off-by: Yingchun Guo <[email protected]> * Add gpu support for ChatQnA (#308) * add gpu support for chatqna Signed-off-by: Ding, Ke <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Ding, Ke <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update ChatQnA for Xeon docker_compose.yaml to fix downloads failing (#310) * Update docker_compose * Updated docker_compose * Updated docker_compose * Add build docker image option for test scripts (#312) Signed-off-by: chensuyue <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Yingchun Guo <[email protected]> Signed-off-by: Ding, Ke <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: WenjiaoYue <[email protected]> Co-authored-by: Ying Chun Guo <[email protected]> Co-authored-by: Ke Ding <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Steve Fowler <[email protected]> Co-authored-by: chen, suyue <[email protected]> --------- Signed-off-by: zehao-intel <[email protected]> Signed-off-by: chensuyue <[email protected]> Signed-off-by: Yingchun Guo <[email protected]> Signed-off-by: Ding, Ke <[email protected]> Signed-off-by: WenjiaoYue <[email protected]> Co-authored-by: chen, suyue <[email protected]> Co-authored-by: WenjiaoYue <[email protected]> Co-authored-by: Ying Chun Guo <[email protected]> Co-authored-by: Ke Ding <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Steve Fowler <[email protected]> Co-authored-by: lvliang-intel <[email protected]>

Spycsh and others added 10 commits July 17, 2024 11:38

allow runtime delete edge

24034ae

refactor test

6502c77

Merge branch 'main' into ddag

936cc38

[pre-commit.ci] auto fixes from pre-commit.com hooks

6f26114

for more information, see https://pre-commit.ci

ruff

befb1f1

fix

4e1ea85

Merge branch 'ddag' of https://github.com/Spycsh/GenAIComps into ddag

86c7574

[pre-commit.ci] auto fixes from pre-commit.com hooks

e8caf42

for more information, see https://pre-commit.ci

fix

8faaed1

[pre-commit.ci] auto fixes from pre-commit.com hooks

55157b1

for more information, see https://pre-commit.ci

Spycsh requested review from lvliang-intel, zehao-intel, letonghan and chensuyue and removed request for zehao-intel and lvliang-intel July 17, 2024 05:22

Spycsh and others added 6 commits July 17, 2024 16:51

skip streamingresponse

c470a07

Merge branch 'ddag' of https://github.com/Spycsh/GenAIComps into ddag

197ba1f

fix dangling graph

4671c39

[pre-commit.ci] auto fixes from pre-commit.com hooks

b9f79be

for more information, see https://pre-commit.ci

fix

b4abcad

Merge branch 'ddag' of https://github.com/Spycsh/GenAIComps into ddag

8b989f3

lvliang-intel approved these changes Jul 17, 2024

View reviewed changes

chensuyue approved these changes Jul 17, 2024

View reviewed changes

chensuyue merged commit f2995ab into opea-project:main Jul 17, 2024
10 checks passed

xuechendi pushed a commit to xuechendi/GenAIComps that referenced this pull request Jul 18, 2024

Add dynamic DAG (opea-project#317)

9cfb97e

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

xuechendi pushed a commit that referenced this pull request Jul 18, 2024

Add dynamic DAG (#317)

4606028

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Chendi.Xue <[email protected]>

Spycsh mentioned this pull request Jul 19, 2024

Add FAQGen gateway in core to support FAQGen Example, Adds FAQGen LLM service #318

Merged

3 tasks

dwhitena pushed a commit to predictionguard/GenAIComps that referenced this pull request Jul 24, 2024

Add dynamic DAG (opea-project#317)

db86fb1

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Daniel Whitenack <[email protected]>

berkecanrizai pushed a commit to berkecanrizai/GenAIComps that referenced this pull request Jul 29, 2024

Add dynamic DAG (opea-project#317)

4e12e75

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Berke <[email protected]>

berkecanrizai pushed a commit to berkecanrizai/GenAIComps that referenced this pull request Jul 29, 2024

Add dynamic DAG (opea-project#317)

f3c0248

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Berke <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add dynamic DAG #317

Add dynamic DAG #317

Spycsh commented Jul 17, 2024

codecov bot commented Jul 17, 2024 •

edited

Loading

Add dynamic DAG #317

Add dynamic DAG #317

Conversation

Spycsh commented Jul 17, 2024

Description

Issues

Type of change

Dependencies

Tests

codecov bot commented Jul 17, 2024 • edited Loading

Codecov Report

codecov bot commented Jul 17, 2024 •

edited

Loading