Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

partners: AI21 Labs Semantic Text Splitter support #19510

Merged
merged 5 commits into from
Mar 26, 2024

Conversation

miri-bar
Copy link
Contributor

Description: Added support for AI21 Labs model - Segmentation, as a Text Splitter
Dependencies: ai21, langchain-text-splitter
Twitter handle: https://github.com/AI21Labs

* add segmentation - init

* refactor

* refactor

* fix poetry.lock

* progress

* progress

* feat: merge with len function

* lint

* refactor ai21 client

* refactor client, add start of unittests

* fix unittest

* add unittests, add to test_imports

* readme

* fix linting

* poetry.lock

* lint

* fix lint text

* update dependency

* code review - refactor

* add ai21 client params, remove condition in metadata

* change integration tests names, add comments

* line too long

* remove add start index integration test

* compare segments

* more spaces, add comments

* change normalization

* lint

* fix

* Add notebook with examples

* Add split documents

* clear

* remove newline

* client -> segmentation.

* add semantic text splitter to the index file

* Fix typo

* Add reference to notebook, add tests, edit notebook

* update poetry.lock

* lint
@efriis efriis added the partner label Mar 25, 2024
@efriis efriis self-assigned this Mar 25, 2024
Copy link

vercel bot commented Mar 25, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchain ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 26, 2024 1:39am

@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. Ɑ: text splitters Related to text splitters package 🔌: openai Primarily related to OpenAI integrations 🤖:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features labels Mar 25, 2024
@dosubot dosubot bot added the lgtm PR looks good. Use to confirm that a PR is ready for merging. label Mar 25, 2024
@baskaryan baskaryan enabled auto-merge (squash) March 25, 2024 23:18
@baskaryan baskaryan merged commit 55db737 into langchain-ai:master Mar 26, 2024
21 checks passed
gkorland pushed a commit to FalkorDB/langchain that referenced this pull request Mar 30, 2024
…9510)

Description: Added support for AI21 Labs model - Segmentation, as a Text
Splitter
Dependencies: ai21, langchain-text-splitter
Twitter handle: https://github.com/AI21Labs

---------

Co-authored-by: Bagatur <[email protected]>
Co-authored-by: Bagatur <[email protected]>
chrispy-snps pushed a commit to chrispy-snps/langchain that referenced this pull request Mar 30, 2024
…9510)

Description: Added support for AI21 Labs model - Segmentation, as a Text
Splitter
Dependencies: ai21, langchain-text-splitter
Twitter handle: https://github.com/AI21Labs

---------

Co-authored-by: Bagatur <[email protected]>
Co-authored-by: Bagatur <[email protected]>
hinthornw pushed a commit that referenced this pull request Apr 26, 2024
Description: Added support for AI21 Labs model - Segmentation, as a Text
Splitter
Dependencies: ai21, langchain-text-splitter
Twitter handle: https://github.com/AI21Labs

---------

Co-authored-by: Bagatur <[email protected]>
Co-authored-by: Bagatur <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features lgtm PR looks good. Use to confirm that a PR is ready for merging. 🔌: openai Primarily related to OpenAI integrations partner size:XL This PR changes 500-999 lines, ignoring generated files. Ɑ: text splitters Related to text splitters package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants