Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: pass docs_map to Executor #5366

Merged
merged 21 commits into from
Nov 10, 2022
Merged

feat: pass docs_map to Executor #5366

merged 21 commits into from
Nov 10, 2022

Conversation

JoanFM
Copy link
Member

@JoanFM JoanFM commented Nov 8, 2022 •
edited
Loading

Goals:
Pass docs_map to Executor as a dictionary with keys as the last Executors that processed a Request, and values as the DocumentArray in that specific Request.

@github-actions github-actions bot added size/M area/core This issue/PR affects the core codebase area/testing This issue/PR affects testing component/type labels Nov 8, 2022
@codecov
Copy link

codecov bot commented Nov 8, 2022 •
edited
Loading

Codecov Report

Merging #5366 (e1eec52) into master (de53f95) will increase coverage by 0.09%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #5366      +/-   ##
==========================================
+ Coverage   86.83%   86.93%   +0.09%     
==========================================
  Files         100      100              
  Lines        6512     6528      +16     
==========================================
+ Hits         5655     5675      +20     
+ Misses        857      853       -4
Flag Coverage Δ
jina 86.93% <100.00%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
jina/parsers/deprecated.py 83.33% <ø> (ø)
jina/parsers/orchestrate/pod.py 100.00% <ø> (ø)
jina/parsers/orchestrate/runtimes/head.py 100.00% <ø> (ø)
jina/parsers/orchestrate/runtimes/remote.py 100.00% <ø> (ø)
jina/parsers/orchestrate/runtimes/runtime.py 100.00% <ø> (ø)
jina/serve/streamer.py 95.91% <ø> (ø)
jina/orchestrate/flow/base.py 90.77% <100.00%> (ø)
jina/parsers/orchestrate/runtimes/worker.py 100.00% <100.00%> (ø)
jina/serve/gateway.py 94.11% <100.00%> (ø)
...ina/serve/runtimes/gateway/graph/topology_graph.py 99.46% <100.00%> (-0.01%) ⬇️
... and 4 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@JoanFM JoanFM linked an issue Nov 8, 2022 that may be closed by this pull request
Enforce deterministic results order on 'needs' in topology graph #4084
Closed
@JoanFM JoanFM closed this Nov 8, 2022
@JoanFM JoanFM reopened this Nov 8, 2022
@github-actions github-actions bot added area/cli This issue/PR affects the command line interface area/docs This issue/PR affects the docs labels Nov 8, 2022
@JoanFM JoanFM marked this pull request as ready for review November 8, 2022 17:48
@JoanFM JoanFM force-pushed the force-deterministic-order branch from 7dbeb7b to a4ea275 Compare November 9, 2022 08:52
JohannesMessner
JohannesMessner suggested changes Nov 9, 2022
View reviewed changes
docs/fundamentals/executor/executor-methods.md Outdated Show resolved Hide resolved
docs/fundamentals/flow/executor-args.md Outdated Show resolved Hide resolved
docs/fundamentals/flow/executor-args.md Outdated Show resolved Hide resolved
jina/parsers/orchestrate/runtimes/worker.py Outdated Show resolved Hide resolved
jina/parsers/orchestrate/runtimes/worker.py Outdated Show resolved Hide resolved
jina/types/request/data.py Outdated Show resolved Hide resolved
jina/serve/runtimes/request_handlers/worker_request_handler.py Outdated Show resolved Hide resolved
@JoanFM JoanFM changed the title feat: docs_matrix passed as dictionary feat: pass docs_map to Executor Nov 9, 2022
@JoanFM JoanFM closed this Nov 9, 2022
@JoanFM JoanFM reopened this Nov 9, 2022
JoanFM
JoanFM commented Nov 9, 2022
View reviewed changes
jina/types/request/data.py Outdated Show resolved Hide resolved
@JoanFM JoanFM closed this Nov 9, 2022
@JoanFM JoanFM reopened this Nov 9, 2022
@JoanFM JoanFM linked an issue Nov 9, 2022 that may be closed by this pull request
feat: find out which docs are which in docs_matrix #5330
Closed
@JoanFM JoanFM force-pushed the force-deterministic-order branch from 44f985a to 7e3bff3 Compare November 10, 2022 09:14
@hanxiao
Copy link
Member

hanxiao commented Nov 10, 2022 •
edited
Loading

plz use --no-reduce instead of --disable-reduce everywhere, as no is more preferrable by our convention searching --no

JohannesMessner
JohannesMessner previously requested changes Nov 10, 2022
View reviewed changes
docs/fundamentals/executor/executor-methods.md Outdated Show resolved Hide resolved
docs/fundamentals/executor/executor-methods.md Outdated Show resolved Hide resolved
docs/fundamentals/executor/executor-methods.md Outdated Show resolved Hide resolved
jina/types/request/data.py Outdated Show resolved Hide resolved
alaeddine-13
alaeddine-13 reviewed Nov 10, 2022
View reviewed changes
docs/fundamentals/executor/executor-methods.md Outdated Show resolved Hide resolved
docs/fundamentals/executor/executor-methods.md Outdated Show resolved Hide resolved
jina/orchestrate/flow/base.py Outdated Show resolved Hide resolved
jina/parsers/orchestrate/runtimes/worker.py Outdated Show resolved Hide resolved
jina/serve/runtimes/request_handlers/worker_request_handler.py Outdated Show resolved Hide resolved
jina/serve/runtimes/request_handlers/worker_request_handler.py Outdated Show resolved Hide resolved
jina/serve/runtimes/request_handlers/worker_request_handler.py Outdated Show resolved Hide resolved
JoanFM and others added 2 commits November 10, 2022 15:53
@JoanFM JoanFM closed this Nov 10, 2022
@JoanFM JoanFM reopened this Nov 10, 2022
@JoanFM JoanFM force-pushed the force-deterministic-order branch from d6e0d58 to 1451769 Compare November 10, 2022 15:11
@JoanFM JoanFM force-pushed the force-deterministic-order branch from 81d8888 to 87c36ff Compare November 10, 2022 15:36
@github-actions
Copy link

📝 Docs are deployed on https://force-deterministic-order--jina-docs.netlify.app 🎉

florian-hoenicke
florian-hoenicke approved these changes Nov 10, 2022
View reviewed changes
Copy link
Member

@florian-hoenicke florian-hoenicke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, this feature helps us a lot to reduce the complexity in our code.

@JoanFM JoanFM dismissed JohannesMessner’s stale review November 10, 2022 19:57

changes applied, if changr request I will open new PR

@JoanFM JoanFM merged commit ad96553 into master Nov 10, 2022
@JoanFM JoanFM deleted the force-deterministic-order branch November 10, 2022 19:57
@alexcg1 alexcg1 mentioned this pull request Nov 24, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@florian-hoenicke florian-hoenicke florian-hoenicke approved these changes

@JohannesMessner JohannesMessner Awaiting requested review from JohannesMessner

@alaeddine-13 alaeddine-13 Awaiting requested review from alaeddine-13

Assignees
No one assigned
Labels
area/cli This issue/PR affects the command line interface area/core This issue/PR affects the core codebase area/docs This issue/PR affects the docs area/testing This issue/PR affects testing component/type size/M
Projects
None yet
Milestone
No milestone
6 participants
@JoanFM @hanxiao @florian-hoenicke @alaeddine-13 @JohannesMessner @jina-bot