Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix KNN SpaceType.getVectorSimilarityFunction renamed issue #834

Merged
merged 3 commits into from
Jul 19, 2024

Conversation

chishui
Copy link
Contributor

@chishui chishui commented Jul 19, 2024

Description

Knn repo SpaceType.getVectorSimilarityFunction has been rename to SpaceType.getKnnVectorSimilarityFunction by this PR which caused :compileTestFixturesJava failure https://github.com/opensearch-project/neural-search/actions/runs/9904116192/job/27602462802?pr=832

Issues Resolved

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Liyun Xiu <[email protected]>
@chishui chishui reopened this Jul 19, 2024
@yuye-aws
Copy link
Member

It is nice of you to raise the PR. Can you please update the changelog and fix CI failures?

@navneet1v
Copy link
Collaborator

@chishui seems like there are changes in MLCommons that broke the build.

Import org.opensearch.ml.client.MachineLearningNodeClient;
> Task :compileJava
                               ^
  bad class file: C:\Users\runneradmin\.gradle\caches\modules-2\files-2.1\org.opensearch\opensearch-ml-client\3.0.0.0-SNAPSHOT\886ef00577f1df5e128b402fd4c9ae6f772fd28f\opensearch-ml-client-3.0.0.0-SNAPSHOT.jar(/org/opensearch/ml/client/MachineLearningNodeClient.class)
    class file has wrong version 65.0, should be 55.0
    Please remove or make sure it appears in the correct subdirectory of the classpath.
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:19: error: cannot access FunctionName
import org.opensearch.ml.common.FunctionName;
                               ^
  bad class file: C:\Users\runneradmin\.gradle\caches\modules-2\files-2.1\org.opensearch\opensearch-ml-client\3.0.0.0-SNAPSHOT\886ef00577f1df5e128b402fd4c9ae6f772fd28f\opensearch-ml-client-3.0.0.0-SNAPSHOT.jar(/org/opensearch/ml/common/FunctionName.class)
    class file has wrong version 65.0, should be 55.0
    Please remove or make sure it appears in the correct subdirectory of the classpath.
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:20: error: cannot access MLInputDataset
import org.opensearch.ml.common.dataset.MLInputDataset;
                                       ^
  bad class file: C:\Users\runneradmin\.gradle\caches\modules-2\files-2.1\org.opensearch\opensearch-ml-client\3.0.0.0-SNAPSHOT\886ef00577f1df5e128b402fd4c9ae6f772fd28f\opensearch-ml-client-3.0.0.0-SNAPSHOT.jar(/org/opensearch/ml/common/dataset/MLInputDataset.class)
    class file has wrong version 65.0, should be [55](https://github.com/opensearch-project/neural-search/actions/runs/10002888226/job/27648933210?pr=834#step:4:56).0
    Please remove or make sure it appears in the correct subdirectory of the classpath.
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:21: error: cannot access TextDocsInputDataSet
import org.opensearch.ml.common.dataset.TextDocsInputDataSet;
                                       ^
  bad class file: C:\Users\runneradmin\.gradle\caches\modules-2\files-2.1\org.opensearch\opensearch-ml-client\3.0.0.0-SNAPSHOT\886ef00[57](https://github.com/opensearch-project/neural-search/actions/runs/10002888226/job/27648933210?pr=834#step:4:58)7f1df5e128b402fd4c9ae6f772fd28f\opensearch-ml-client-3.0.0.0-SNAPSHOT.jar(/org/opensearch/ml/common/dataset/TextDocsInputDataSet.class)
    class file has wrong version 65.0, should be 55.0
    Please remove or make sure it appears in the correct subdirectory of the classpath.
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:22: error: cannot access TextSimilarityInputDataSet
import org.opensearch.ml.common.dataset.TextSimilarityInputDataSet;
                                       ^
  bad class file: C:\Users\runneradmin\.gradle\caches\modules-2\files-2.1\org.opensearch\opensearch-ml-client\3.0.0.0-SNAPSHOT\886ef00577f1df5e128b402fd4c9ae6f772fd28f\opensearch-ml-client-3.0.0.0-SNAPSHOT.jar(/org/opensearch/ml/common/dataset/TextSimilarityInputDataSet.class)
    class file has wrong version 65.0, should be 55.0
    Please remove or make sure it appears in the correct subdirectory of the classpath.
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:23: error: cannot access MLInput
import org.opensearch.ml.common.input.MLInput;
                                     ^
  bad class file: C:\Users\runneradmin\.gradle\caches\modules-2\files-2.1\org.opensearch\opensearch-ml-client\3.0.0.0-SNAPSHOT\886ef00577f1df5e128b402fd4c9ae6f772fd28f\opensearch-ml-client-3.0.0.0-SNAPSHOT.jar(/org/opensearch/ml/common/input/MLInput.class)
    class file has wrong version 65.0, should be 55.0
    Please remove or make sure it appears in the correct subdirectory of the classpath.
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:24: error: cannot access MLOutput
import org.opensearch.ml.common.output.MLOutput;
                                      ^
  bad class file: C:\Users\runneradmin\.gradle\caches\modules-2\files-2.1\org.opensearch\opensearch-ml-client\3.0.0.0-SNAPSHOT\886ef00577f1df5e128b402fd4c9ae6f772fd28f\opensearch-ml-client-3.0.0.0-SNAPSHOT.jar(/org/opensearch/ml/common/output/MLOutput.class)
    class file has wrong version 65.0, should be 55.0
    Please remove or make sure it appears in the correct subdirectory of the classpath.
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:25: error: cannot access ModelResultFilter
import org.opensearch.ml.common.output.model.ModelResultFilter;
                                            ^
  bad class file: C:\Users\runneradmin\.gradle\caches\modules-2\files-2.1\org.opensearch\opensearch-ml-client\3.0.0.0-SNAPSHOT\886ef00577f1df5e128b402fd4c9ae6f772fd28f\opensearch-ml-client-3.0.0.0-SNAPSHOT.jar(/org/opensearch/ml/common/output/model/ModelResultFilter.class)
    class file has wrong version 65.0, should be 55.0
    Please remove or make sure it appears in the correct subdirectory of the classpath.
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:26: error: cannot access ModelTensor
import org.opensearch.ml.common.output.model.ModelTensor;
                                            ^
  bad class file: C:\Users\runneradmin\.gradle\caches\modules-2\files-2.1\org.opensearch\opensearch-ml-client\3.0.0.0-SNAPSHOT\886ef00577f1df5e128b402fd4c9ae6f772fd28f\opensearch-ml-client-3.0.0.0-SNAPSHOT.jar(/org/opensearch/ml/common/output/model/ModelTensor.class)
    class file has wrong version 65.0, should be 55.0
    Please remove or make sure it appears in the correct subdirectory of the classpath.
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:27: error: cannot access ModelTensorOutput
import org.opensearch.ml.common.output.model.ModelTensorOutput;
                                            ^
  bad class file: C:\Users\runneradmin\.gradle\caches\modules-2\files-2.1\org.opensearch\opensearch-ml-client\3.0.0.0-SNAPSHOT\886ef00577f1df5e128b402fd4c9ae6f772fd28f\opensearch-ml-client-3.0.0.0-SNAPSHOT.jar(/org/opensearch/ml/common/output/model/ModelTensorOutput.class)
    class file has wrong version 65.0, should be 55.0
    Please remove or make sure it appears in the correct subdirectory of the classpath.
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:28: error: cannot access ModelTensors
import org.opensearch.ml.common.output.model.ModelTensors;
                                            ^
  bad class file: C:\Users\runneradmin\.gradle\caches\modules-2\files-2.1\org.opensearch\opensearch-ml-client\3.0.0.0-SNAPSHOT\886ef00577f1df5e128b402fd4c9ae6f772fd28f\opensearch-ml-client-3.0.0.0-SNAPSHOT.jar(/org/opensearch/ml/common/output/model/ModelTensors.class)
    class file has wrong version 65.0, should be 55.0
    Please remove or make sure it appears in the correct subdirectory of the classpath.
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:42: error: cannot find symbol
    private final MachineLearningNodeClient mlClient;
                  ^
  symbol:   class MachineLearningNodeClient
  location: class MLCommonsClientAccessor
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:216: error: cannot find symbol
    private MLInput createMLTextInput(final List<String> targetResponseFilters, List<String> inputText) {
            ^
  symbol:   class MLInput
  location: class MLCommonsClientAccessor
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:222: error: cannot find symbol
    private MLInput createMLTextPairsInput(final String query, final List<String> inputText) {
            ^
  symbol:   class MLInput
  location: class MLCommonsClientAccessor
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:227: error: cannot find symbol
    private List<List<Float>> buildVectorFromResponse(MLOutput mlOutput) {
                                                      ^
  symbol:   class MLOutput
  location: class MLCommonsClientAccessor
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:240: error: cannot find symbol
    private List<Map<String, ?>> buildMapResultFromResponse(MLOutput mlOutput) {
                                                            ^
  symbol:   class MLOutput
  location: class MLCommonsClientAccessor
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:2[58](https://github.com/opensearch-project/neural-search/actions/runs/10002888226/job/27648933210?pr=834#step:4:59): error: cannot find symbol
    private List<Float> buildSingleVectorFromResponse(final MLOutput mlOutput) {
                                                            ^
  symbol:   class MLOutput
  location: class MLCommonsClientAccessor
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:285: error: cannot find symbol
    private MLInput createMLMultimodalInput(final List<String> targetResponseFilters, final Map<String, String> input) {
            ^
  symbol:   class MLInput
  location: class MLCommonsClientAccessor
D:\a\neural-search\neural-search\src\main\java\org\opensearch\neuralsearch\ml\MLCommonsClientAccessor.java:38: error: cannot find symbol
@RequiredArgsConstructor
^
  symbol:   class MachineLearningNodeClient
  location: class MLCommonsClientAccessor
19 errors

@yuye-aws
Copy link
Member

It's due to JDK version mismatch between ml-commons and neural search. From ml-commons, the expected JDK is 21. The version is thus 65. In neural-search, the version is 55 due to lower version JDK.

@yuye-aws
Copy link
Member

This PR is ready for review: #835

@chishui
Copy link
Contributor Author

chishui commented Jul 19, 2024

covered in this PR #835

@chishui chishui closed this Jul 19, 2024
@chishui chishui reopened this Jul 19, 2024
@chishui chishui changed the base branch from main to 2.x July 19, 2024 08:09
@chishui chishui changed the base branch from 2.x to main July 19, 2024 08:12
@zhichao-aws zhichao-aws added skip-changelog backport 2.x Label will add auto workflow to backport PR to 2.x branch labels Jul 19, 2024
@zhichao-aws zhichao-aws merged commit 08d82b1 into opensearch-project:main Jul 19, 2024
36 of 37 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 19, 2024
* Fix KNN SpaceType.getVectorSimilarityFunction renamed issue.

Signed-off-by: Liyun Xiu <[email protected]>

* Fix NPE for a test

Signed-off-by: Liyun Xiu <[email protected]>

---------

Signed-off-by: Liyun Xiu <[email protected]>
(cherry picked from commit 08d82b1)
zhichao-aws pushed a commit that referenced this pull request Jul 19, 2024
)

* Fix KNN SpaceType.getVectorSimilarityFunction renamed issue.

Signed-off-by: Liyun Xiu <[email protected]>

* Fix NPE for a test

Signed-off-by: Liyun Xiu <[email protected]>

---------

Signed-off-by: Liyun Xiu <[email protected]>
(cherry picked from commit 08d82b1)

Co-authored-by: Liyun Xiu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Label will add auto workflow to backport PR to 2.x branch skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants