Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seq is saved in Mapping on Index which Causes Elastic to Re-Index #17

Open
weareu opened this issue Sep 30, 2018 · 0 comments · May be fixed by #19
Open

Seq is saved in Mapping on Index which Causes Elastic to Re-Index #17

weareu opened this issue Sep 30, 2018 · 0 comments · May be fixed by #19

Comments

@weareu
Copy link

weareu commented Sep 30, 2018

When not using load or one and syncing changes, every document will have its Seq (last since) written to the Mapping in elastic. This is to ensure sync can pick up where it left off. However from documentation it became clear that updating the mapping causes elastic to re-index the entire index every time the mapping is updated. We have been having serious elastic performance issues running tens of syncs on very large indexes. So I'm assuming this could be the cause.

With this and the related issue where template urls cannot be used with Sync, we are getting stuck with this library. I'm going to fork and change to save the Seq in a specific elastic index. I'm also going to add a key and use the CouchDB database and server as the key + custom key to save the Seq. This solves this issue, the URL Template issue and allows a user to have multiple instances of sync running from the same couchdb still. I'll also make it optional to keep backwards compatibility.

I'll pull when done. Leme know if there is a better idea.

@weareu weareu linked a pull request Sep 30, 2018 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant