Fix: FAISSDocumentStore - make write_documents properly work in combination w update_embeddings

[anakin87]

Related Issues

• fixes Overwriting duplicate documents in a FAISSDocumentStore does not remove overwritten embeddings #2834
• fixes Incremental update of FAISSDocumentStore leads to incomplete query results #5071

Proposed Changes:

The bug was described in #2834.
In short:

• when the write_documents method was called with duplicate_documents="overwrite" and without providing embeddings, for each document, FAISS vector_id was not set (even if the embedding already exists in the index).
• calling update_embeddings method with update_existing_embeddings=False does not make the embeddings for existing documents be computed (this is correct), but then the overwritten documents have no vector_id (the embeddings are "lost").

Solution
When the write_documents method is called with duplicate_documents="overwrite" and without providing embeddings, the corresponding FAISS vector_ids are recovered and set.

(no changes in update_embeddings)
Then, if update_embeddings is called with update_existing_embeddings=False, the existing vector_ids are available. Otherwise, the FAISS index is regenerated.

How did you test it?

CI, new integration test

Notes for the reviewer

I also refactored the write_documents method a bit.

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[coveralls]

Pull Request Test Coverage Report for Build 5412737136

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 309 unchanged lines in 3 files lost coverage.
• Overall coverage increased (+0.05%) to 44.03%

---

Files with Coverage Reduction	New Missed Lines	%
document_stores/filter_utils.py	83	41.91%
pipelines/base.py	110	40.61%
document_stores/faiss.py	116	18.68%

Totals
Change from base Build 5391855155:	0.05%
Covered Lines:	10074
Relevant Lines:	22880

---

💛 - Coveralls

[ZanSara]

Looking good! I would fix the test, if possible, then it's ready to merge 👍

[ZanSara]

LGTM! 🚢