How to reconcile PostgreSQL documents with the VectorDB ones?

[mboret]

Hi,

After spending time understanding why some documents were missing in the vectorDB and why I could not index them, I finally found I have some orphans' references in the PostgreSQL DB. I don't know why, maybe due to past issues with database connections during operations like connector deletion.

Details in this Slack thread

It could be nice to have a task to reconcile the Postgres and vectorDB by checking that every document ID registered in PostgreSQL is still present in the vectorDB (if not, delete it).