Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

community: update for compatibility with latest Meilisearch version #18970

Merged
merged 6 commits into from
Mar 27, 2024
Merged

community: update for compatibility with latest Meilisearch version #18970

merged 6 commits into from
Mar 27, 2024

Conversation

CaroFG
Copy link
Contributor

@CaroFG CaroFG commented Mar 12, 2024

Thank you for contributing to LangChain!

  • PR title: "package: description"

    • Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes.
    • Example: "community: add foobar LLM"
  • PR message:

    • Description: Updates Meilisearch vectorstore for compatibility with v1.6 and above. Adds embedders settings and embedder_name which are now required.
  • Add tests and docs: If you're adding a new integration, please include

    1. a test for the integration, preferably unit tests that do not rely on network access,
    2. an example notebook showing its use. It lives in docs/docs/integrations directory.
  • Lint and test: Run make format, make lint and make test from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/

Additional guidelines:

  • Make sure optional dependencies are imported within a function.
  • Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests.
  • Most PRs should not touch more than one package.
  • Changes should be backwards compatible.
  • If you are adding something to community, do not re-import it in langchain.

If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Mar 12, 2024
Copy link

vercel bot commented Mar 12, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchain ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 27, 2024 10:08pm

@CaroFG
Copy link
Contributor Author

CaroFG commented Mar 25, 2024

Hello! I don't want to be a bother, but since it's mentioned in the PR template, I'm tagging @efriis to make sure you don't forget this PR :D

@@ -67,6 +67,7 @@ class Meilisearch(VectorStore):
embeddings = OpenAIEmbeddings()
vectorstore = Meilisearch(
embedding=embeddings,
embedders=embedders,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what are embedders? could we show how you'd define embedders in this example?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Meilisearch versions greater than 1.6, to use vector search, you must first configure the embedders index setting. This setting is a dictionary structure in which each key corresponds to an embedder.

The configuration options within an embedder object vary depending on the value assigned to its source field, which specifies the third-party service used for generating embeddings. In this context, the source is userProvided.

For userProvided sources, it's required to include a dimensions field, which outlines the size of the vectors.

An example configuration for embedders could look like this:

embedders = {
    "theEmbedderName": {
        "source": "userProvided",
        "dimensions": "1536"
    }
}

The use can define multiple embedders per index to accommodate more demanding use cases. For example, a user can define one embedder to generate text from a document image, and another embedder to translate the original document text fields to let users speaking other languages search products in their native language.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gotcha! should we that snippet in the docstring example?

@@ -125,10 +134,11 @@ def add_texts(
metadata = metadatas[i]
metadata[self._text_key] = text
embedding = embedding_vectors[i]
embedder_name
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stray line?

@dosubot dosubot bot added the lgtm PR looks good. Use to confirm that a PR is ready for merging. label Mar 27, 2024
@baskaryan baskaryan enabled auto-merge (squash) March 27, 2024 22:00
@baskaryan baskaryan merged commit cf96060 into langchain-ai:master Mar 27, 2024
61 checks passed
gkorland pushed a commit to FalkorDB/langchain that referenced this pull request Mar 30, 2024
…rsion (langchain-ai#18970)

- **Description:** Updates Meilisearch vectorstore for compatibility
with v1.6 and above. Adds embedders settings and embedder_name which are
now required.

---------

Co-authored-by: Bagatur <[email protected]>
hinthornw pushed a commit that referenced this pull request Apr 26, 2024
…rsion (#18970)

- **Description:** Updates Meilisearch vectorstore for compatibility
with v1.6 and above. Adds embedders settings and embedder_name which are
now required.

---------

Co-authored-by: Bagatur <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖:improvement Medium size change to existing code to handle new use-cases lgtm PR looks good. Use to confirm that a PR is ready for merging. size:L This PR changes 100-499 lines, ignoring generated files. Ɑ: vector store Related to vector store module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants