core: Add async methods to BaseExampleSelector and SemanticSimilarityExampleSelector #19399
Conversation
dosubot
bot
added
the
size:L
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
dosubot
bot
added
Ɑ: core
cbornet
force-pushed
the
async-example-selector
branch
3 times, most recently
from
9b42695 to
426dea1
Compare
cbornet
force-pushed
the
async-example-selector
branch
from
426dea1 to
859c6e9
Compare
dosubot
bot
added
the
lgtm
| else: | ||
| return " ".join(sorted_values(example)) | ||
|
|
||
| def _documents_to_examples(self, documents: List[Document]) -> List[dict]: |
There was a problem hiding this comment.
One question -- why a static method for example to text vs. _documents_to_examples being a regular method
Are implementation providers going to need to override documents to examples based on some state available in self?
There was a problem hiding this comment.
We can’t make this one static as it uses self.example_keys.
In _document_to_example, input_keys is passed as argument instead so it can be called from the classmethod from_examples.
eyurtsev
added
the
waiting-on-author
|
Ready to merge, just need confirmation from @cbornet on whether static methods vs. regular methods should be used -- do we forsee the need to use state. If so maybe a doc-string for those methods could be added to explain how to sub-class correctly and which methods are meant to be overriden? |
Those methods are internal to the class and only there to factorize between sync and async. They are not meant to be overridden. Would you prefer a double-underscore ? |
|
@cbornet i usually pull things out into functions if they're not meant to be overridden -- otherwise folks will end up overriding them, especially since we have a bunch of examples where the correct way to do the implementation is by overriding a private method :( I'll put out the code into functions in a few |
|
I will do in a separate PR |
…ExampleSelector (langchain-ai#19399) Few-Shot prompt template may use a `SemanticSimilarityExampleSelector` that in turn uses a `VectorStore` that does I/O operations. So to work correctly on the event loop, we need: * async methods for the `VectorStore` (OK) * async methods for the `SemanticSimilarityExampleSelector` (this PR) * async methods for `BasePromptTemplate` and `BaseChatPromptTemplate` (future work)
…ExampleSelector (#19399) Few-Shot prompt template may use a `SemanticSimilarityExampleSelector` that in turn uses a `VectorStore` that does I/O operations. So to work correctly on the event loop, we need: * async methods for the `VectorStore` (OK) * async methods for the `SemanticSimilarityExampleSelector` (this PR) * async methods for `BasePromptTemplate` and `BaseChatPromptTemplate` (future work)
Few-Shot prompt template may use a
SemanticSimilarityExampleSelectorthat in turn uses aVectorStorethat does I/O operations.So to work correctly on the event loop, we need:
VectorStore(OK)SemanticSimilarityExampleSelector(this PR)BasePromptTemplateandBaseChatPromptTemplate(future work)