support dispatching execute task; don't dispatch ML task again #279

ylwu-amzn · 2022-04-13T19:08:57Z

Signed-off-by: Yaliang Wu [email protected]

Description

support dispatching execute task;
don't dispatch ML task again
add request id to track the request easily

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Yaliang Wu <[email protected]>

Zhangxunmt · 2022-04-14T17:50:35Z

plugin/build.gradle

@@ -207,7 +207,8 @@ jacocoTestReport {
 List<String> jacocoExclusions = [
        // TODO: add more unit test to meet the minimal test coverage.
        'org.opensearch.ml.constant.CommonValue',
-        'org.opensearch.ml.plugin.MachineLearningPlugin*'
+        'org.opensearch.ml.plugin.MachineLearningPlugin*',
+        'org.opensearch.ml.task.MLPredictTaskRunner'


We might want to refactor this PredictTaskRunner later. It has a big body in predict() method which is not test friendly.

Yes, good point, we can refactor later

Signed-off-by: Yaliang Wu <[email protected]>

codecov-commenter · 2022-04-14T20:45:57Z

Codecov Report

❗ No coverage uploaded for pull request base (main@5eb38f7). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #279   +/-   ##
=======================================
  Coverage        ?   92.16%           
  Complexity      ?      360           
=======================================
  Files           ?       51           
  Lines           ?     1111           
  Branches        ?       51           
=======================================
  Hits            ?     1024           
  Misses          ?       69           
  Partials        ?       18

Flag	Coverage Δ
ml-commons	`92.16% <0.00%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5eb38f7...00eb304. Read the comment docs.

jackiehanyang · 2022-04-15T03:29:11Z

plugin/src/test/java/org/opensearch/ml/task/TaskRunnerTests.java

@@ -46,6 +49,8 @@ public class TaskRunnerTests extends OpenSearchTestCase {
    MLTaskDispatcher mlTaskDispatcher;
    @Mock
    MLCircuitBreakerService mlCircuitBreakerService;
+    @Mock


Curiosity question: this CuslterService mock has to be used together with extending OpenSearchTestCase` right?

Answering my own question here: no it doesn't have to

opensearch-trigger-bot · 2022-04-20T20:31:06Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-279-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 1e15c8804e0734015c3494a2f0bc070d59634df9
# Push it to GitHub
git push --set-upstream origin backport/backport-279-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-279-to-2.x.

opensearch-trigger-bot · 2022-04-20T20:31:07Z

The backport to 2.0 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.0 2.0
# Navigate to the new working tree
cd .worktrees/backport-2.0
# Create a new branch
git switch --create backport/backport-279-to-2.0
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 1e15c8804e0734015c3494a2f0bc070d59634df9
# Push it to GitHub
git push --set-upstream origin backport/backport-279-to-2.0
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.0

Then, create a pull request where the base branch is 2.0 and the compare/head branch is backport/backport-279-to-2.0.

…earch-project#279) * support dispatching execute task; don't dispatch ML task again Signed-off-by: Yaliang Wu <[email protected]> * remove MLPredictTaskRunner from jacoco exclusion list Signed-off-by: Yaliang Wu <[email protected]>

… task again (#298) * support dispatching execute task; don't dispatch ML task again (#279) * support dispatching execute task; don't dispatch ML task again Signed-off-by: Yaliang Wu <[email protected]> * remove MLPredictTaskRunner from jacoco exclusion list Signed-off-by: Yaliang Wu <[email protected]> * fix compiling error Signed-off-by: Yaliang Wu <[email protected]>

…earch-project#279) * support dispatching execute task; don't dispatch ML task again Signed-off-by: Yaliang Wu <[email protected]> * remove MLPredictTaskRunner from jacoco exclusion list Signed-off-by: Yaliang Wu <[email protected]>

… task again (#279) (#301) * support dispatching execute task; don't dispatch ML task again (#279) * support dispatching execute task; don't dispatch ML task again Signed-off-by: Yaliang Wu <[email protected]> * remove MLPredictTaskRunner from jacoco exclusion list Signed-off-by: Yaliang Wu <[email protected]> * fix checkstyle error Signed-off-by: Yaliang Wu <[email protected]>

support dispatching execute task; don't dispatch ML task again

8ac0b80

Signed-off-by: Yaliang Wu <[email protected]>

ylwu-amzn requested a review from a team April 13, 2022 19:08

Zhangxunmt reviewed Apr 14, 2022

View reviewed changes

Zhangxunmt previously approved these changes Apr 14, 2022

View reviewed changes

remove MLPredictTaskRunner from jacoco exclusion list

00eb304

Signed-off-by: Yaliang Wu <[email protected]>

ylwu-amzn dismissed Zhangxunmt’s stale review via 00eb304 April 14, 2022 20:39

Zhangxunmt approved these changes Apr 15, 2022

View reviewed changes

jackiehanyang reviewed Apr 15, 2022

View reviewed changes

jackiehanyang approved these changes Apr 15, 2022

View reviewed changes

ylwu-amzn merged commit 1e15c88 into opensearch-project:main Apr 15, 2022

Zhangxunmt added bug Something isn't working backport 2.x backport 2.0 labels Apr 20, 2022

Zhangxunmt removed backport 2.x backport 2.0 labels Apr 20, 2022

ylwu-amzn mentioned this pull request Apr 23, 2022

[Backport to 1.3] support dispatching execute task; don't dispatch ML task again #298

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support dispatching execute task; don't dispatch ML task again #279

support dispatching execute task; don't dispatch ML task again #279

ylwu-amzn commented Apr 13, 2022

Zhangxunmt Apr 14, 2022 •

edited

Loading

ylwu-amzn Apr 14, 2022

codecov-commenter commented Apr 14, 2022 •

edited

Loading

jackiehanyang Apr 15, 2022

jackiehanyang Apr 15, 2022

opensearch-trigger-bot bot commented Apr 20, 2022

opensearch-trigger-bot bot commented Apr 20, 2022

support dispatching execute task; don't dispatch ML task again #279

support dispatching execute task; don't dispatch ML task again #279

Conversation

ylwu-amzn commented Apr 13, 2022

Description

Check List

Zhangxunmt Apr 14, 2022 • edited Loading

Choose a reason for hiding this comment

ylwu-amzn Apr 14, 2022

Choose a reason for hiding this comment

codecov-commenter commented Apr 14, 2022 • edited Loading

Codecov Report

jackiehanyang Apr 15, 2022

Choose a reason for hiding this comment

jackiehanyang Apr 15, 2022

Choose a reason for hiding this comment

opensearch-trigger-bot bot commented Apr 20, 2022

opensearch-trigger-bot bot commented Apr 20, 2022

Zhangxunmt Apr 14, 2022 •

edited

Loading

codecov-commenter commented Apr 14, 2022 •

edited

Loading