Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FAISS should save/load its index_to_id map #853

Closed
xloem opened this issue Feb 2, 2023 · 2 comments
Closed

FAISS should save/load its index_to_id map #853

xloem opened this issue Feb 2, 2023 · 2 comments

Comments

@xloem
Copy link
Contributor

xloem commented Feb 2, 2023

The save_local, load_local member functions in vectorstore.faiss seem firstly to need to read/write the index_to_id map as well as the main index file. This would then fully separate the storage concerns between the Docstore and the VectorStore.

As it is now, the index_to_id map is an additional component the user must separately serialize/deserialize to reload an FAISS index associated with a Docstore.

@ShreyJ1729
Copy link
Contributor

Check out #789 and #880.

@xloem
Copy link
Contributor Author

xloem commented Feb 4, 2023

Thanks for your work on this. It looks like the code currently in tree is broken and any change could be an improvement. Closing this as it is a duplicate.

@xloem xloem closed this as completed Feb 4, 2023
hwchase17 pushed a commit that referenced this issue Feb 7, 2023
Fixes issues #789 and #853 by saving/loading docstore and
index_to_docstore_id alongside index
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants