Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrate ML Commons lib and K-NN Plugin #4

Closed
4 tasks done
navneet1v opened this issue Sep 26, 2022 · 1 comment
Closed
4 tasks done

Integrate ML Commons lib and K-NN Plugin #4

navneet1v opened this issue Sep 26, 2022 · 1 comment

Comments

@navneet1v
Copy link
Collaborator

navneet1v commented Sep 26, 2022 •
edited
Loading

Description

As a part of Ingestion and Search we are dependent on ML Commons Lib to create the embeddings for query string and user document and K-NN plugin for doing K-NN search. This task tracks the integration of ML commons lib in Neural-Search Plugin.

K-NN Plugin: https://github.com/opensearch-project/k-NN
ML Commons: https://github.com/opensearch-project/ml-commons

Tasks

  • Integrate ML Commons Lib in Repo
  • Unit test for the Interface
  • Integrate K-NN plugin
  • Unit test for the interface
@navneet1v navneet1v self-assigned this Sep 26, 2022
@navneet1v navneet1v changed the title Integrate ML Commons lib Integrate ML Commons lib and K-NN Plugin Sep 26, 2022
@navneet1v navneet1v removed their assignment Sep 26, 2022
@navneet1v navneet1v mentioned this issue Sep 28, 2022
Merged
5 tasks
@navneet1v
Copy link
Collaborator Author

Resolving the Github issue, as all tasks completed.

zane-neo added a commit to zane-neo/neural-search that referenced this issue Oct 20, 2022
@zane-neo
# This is a combination of 14 commits.
0dab8b1 
# This is the 1st commit message:

Add text embedding processor to neural search

Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#2 will be skipped:

# Code format
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#3 will be skipped:

# Address review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#4 will be skipped:

# Add blocking text embedding method for pipeline processor
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#5 will be skipped:

# Add BaseNeuralSearchIT and address other review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#6 will be skipped:

# Add BaseNeuralSearchIT and address other review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#7 will be skipped:

# Add BaseNeuralSearchIT and address other review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#8 will be skipped:

# Fix naming convention and IT function move to base
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#9 will be skipped:

# Fix naming convention and IT function move to base
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#10 will be skipped:

# Update src/main/java/org/opensearch/neuralsearch/ml/MLCommonsClientAccessor.java
#
# Co-authored-by: Navneet Verma <[email protected]>

# The commit message opensearch-project#11 will be skipped:

# Update src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
#
# Co-authored-by: Navneet Verma <[email protected]>

# The commit message opensearch-project#12 will be skipped:

# Fix code review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#13 will be skipped:

# Fix text embedding processor NPE
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#14 will be skipped:

# Remove jackson dependencies and fix tests with XCoontent
#
# Signed-off-by: Zan Niu <[email protected]>
zane-neo added a commit that referenced this issue Oct 20, 2022
@zane-neo
Add text embedding processor to neural search (#18)
799c402 
* # This is a combination of 14 commits.
# This is the 1st commit message:

Add text embedding processor to neural search

Signed-off-by: Zan Niu <[email protected]>

# The commit message #2 will be skipped:

# Code format
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #3 will be skipped:

# Address review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #4 will be skipped:

# Add blocking text embedding method for pipeline processor
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #5 will be skipped:

# Add BaseNeuralSearchIT and address other review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #6 will be skipped:

# Add BaseNeuralSearchIT and address other review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #7 will be skipped:

# Add BaseNeuralSearchIT and address other review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #8 will be skipped:

# Fix naming convention and IT function move to base
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #9 will be skipped:

# Fix naming convention and IT function move to base
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #10 will be skipped:

# Update src/main/java/org/opensearch/neuralsearch/ml/MLCommonsClientAccessor.java
#
# Co-authored-by: Navneet Verma <[email protected]>

# The commit message #11 will be skipped:

# Update src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
#
# Co-authored-by: Navneet Verma <[email protected]>

# The commit message #12 will be skipped:

# Fix code review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #13 will be skipped:

# Fix text embedding processor NPE
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #14 will be skipped:

# Remove jackson dependencies and fix tests with XCoontent
#
# Signed-off-by: Zan Niu <[email protected]>

* Add text embedding processor to neural search

Signed-off-by: Zan Niu <[email protected]>

* Remove unnecessary parameters in TextEmbeddingProcessor method

Signed-off-by: Zan Niu <[email protected]>

* Remove unnecessary empty string checks

Signed-off-by: Zan Niu <[email protected]>

* Add field max depth limit to prevent malicious attack

Signed-off-by: Zan Niu <[email protected]>

Signed-off-by: Zan Niu <[email protected]>
@navneet1v navneet1v mentioned this issue Jul 31, 2023
Closed
xinyual added a commit to xinyual/neural-search that referenced this issue Mar 1, 2024
@xinyual
Merge pull request opensearch-project#4 from xinyual/addMoreUTForDeli…
51905df 
…miterChunker

add more UTs and logic to check whether input is empty
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@navneet1v