Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ML-Commons train api functionality #310

Conversation

rawwar
Copy link
Contributor

@rawwar rawwar commented Oct 7, 2023

Description

Include ML-Commons train api functionality

Issues Resolved

closes #286

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

rawwar added 2 commits October 8, 2023 02:53
Signed-off-by: kalyanr <[email protected]>
@rawwar rawwar changed the title Kalyan/286 ml commons add train support ML-Commons train api functionality Oct 7, 2023
rawwar added 2 commits October 8, 2023 11:56
Signed-off-by: kalyanr <[email protected]>
@rawwar rawwar marked this pull request as ready for review October 8, 2023 06:28
@rawwar
Copy link
Contributor Author

rawwar commented Oct 8, 2023

@dhrubo-os , After inserting data into a dummy index, if I immediately run the test cases to train model, I keep seeing an error No document found. If I include a sleep for 2 seconds or so, its not causing any problem. I used opensearchpy.helpers.bulk to insert the data. Does OpenSearch core insert data asynchronously? Do i need to put a check to see if the data in actually finished inserting into opensearch?

rawwar added 2 commits October 8, 2023 12:03
Signed-off-by: kalyanr <[email protected]>
Signed-off-by: kalyanr <[email protected]>
conftest.py Outdated Show resolved Hide resolved
rawwar added 10 commits October 11, 2023 15:22
Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
@rawwar rawwar requested a review from dhrubo-os October 13, 2023 18:46
@dhrubo-os
Copy link
Collaborator

dhrubo-os commented Oct 27, 2023

Test is failing. I checked this is not from the model uploading, this is related to your test.

opensearch_py_ml/ml_commons/model_train.py Outdated Show resolved Hide resolved
tests/conftest.py Outdated Show resolved Hide resolved
Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
@rawwar
Copy link
Contributor Author

rawwar commented Oct 31, 2023

Test is failing. I checked this is not from the model uploading, this is related to your test.

Fixed testcases

@rawwar rawwar requested a review from dhrubo-os October 31, 2023 21:15
…ch-py-ml into kalyan/286-ml-commons-add-train-support

Signed-off-by: kalyan <[email protected]>
Copy link

codecov bot commented Nov 3, 2023

Codecov Report

Merging #310 (a6f0969) into main (da415f0) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #310      +/-   ##
==========================================
+ Coverage   91.34%   91.36%   +0.01%     
==========================================
  Files          38       38              
  Lines        4230     4237       +7     
==========================================
+ Hits         3864     3871       +7     
  Misses        366      366              
Files Coverage Δ
opensearch_py_ml/ml_commons/ml_commons_client.py 84.66% <100.00%> (+0.68%) ⬆️

Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
Signed-off-by: kalyan <[email protected]>
@rawwar rawwar requested a review from dhrubo-os November 3, 2023 18:35
@dhrubo-os dhrubo-os merged commit 4a8a3e7 into opensearch-project:main Nov 3, 2023
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] Add ML-Commons train api functionality in py-ml
3 participants