Dedicated docs for Record Managers

[MJDeligan]

Following discussions on #4409 and #4398 and the introduction of a new integration in #4321, it might be worth to have dedicated doc pages for record managers to better highlight their usage and particularities.

[dosubot]

🤖

That sounds like a great idea! If you're interested in contributing to the documentation for record managers, feel free to create a pull request with your proposed changes. Your insights and contributions would be valuable in improving the documentation for this feature. Thank you for considering this contribution!

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[mkesper]

What I find confusing re the API is that PostgresRecordManager and PGVectorStore seem to be completely unrelated, when, in fact, they aren't (at least for my understanding).

[MJDeligan]

The record manager is what enables the Indexing function to have memory of what documents have already been vectorised and when. This way we can tell the vector store to only re-vectorise new/changed documents and delete old ones. So PGVectorStore and PostgresRecordManager work together (and can even share the same database) to achieve the indexing behaviour, but the specific implementation of vector store and record manager does not matter. You could just as well use a Pinecone vector store with the PostgresRecordManager or swap the record manager for the SQLiteRecordManager. Vectorstores are also used independently whereas the record managers are only used with the Indexing API

[jacoblee93]

I agree! I would be down to add something under https://js.langchain.com/docs/integrations/components/indexing!

I can have a look at adding some pages next week. As a heads up, we are also looking into a more unified abstraction as well (vectorstores, indexes, docstores) for this type of pipeline workflow.

[MJDeligan]

I can have a look at adding some pages next week. As a heads up, we are also looking into a more unified abstraction as well (vectorstores, indexes, docstores) for this type of pipeline workflow.

Sounds great! Let me know if I can help.

[dosubot]

Hi, @MJDeligan,

I'm helping the langchainjs team manage their backlog and am marking this issue as stale. From what I understand, the issue proposes creating dedicated documentation pages for record managers to better showcase their usage and unique features. There have been discussions and the introduction of a new integration, with users expressing interest in contributing to the documentation and discussing the relationship between PostgresRecordManager and PGVectorStore. However, it seems that the issue remains unresolved at this time.

Could you please confirm if this issue is still relevant to the latest version of the langchainjs repository? If it is, please let the langchainjs team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!