Skip to content
This repository has been archived by the owner on Aug 30, 2022. It is now read-only.

Add re-indexing option to the Admin API #150

Closed
westei opened this issue Nov 23, 2017 · 5 comments
Closed

Add re-indexing option to the Admin API #150

westei opened this issue Nov 23, 2017 · 5 comments
Assignees
Milestone

Comments

@westei
Copy link
Member

westei commented Nov 23, 2017

As software updates and connecting to a different Mongo DB require a full re-index an Admin needs the possibility to trigger a re-index via the Admin API

@westei westei self-assigned this Nov 23, 2017
@mrsimpson
Copy link
Collaborator

@westei will you be able to integrate this into 0.6.1? This release will make a re-index necessary, so having it there would be beneficial

@westei
Copy link
Member Author

westei commented Dec 28, 2017

I can add an Webservice that allows to trigger re-indexing. For an UI integration I would need the help of @ja-fra

@westei
Copy link
Member Author

westei commented Dec 29, 2017

@mrsimpson Noticed that adding a re-indexing option to the Admin API does not do the trick, as in a Cloud setting the Solr Indexes are only local, but the Admin UI is global. So by calling this function one would only trigger re-indexing on the local Solr index. All other cloud instances would not perform the re-indexing.

To provide this functionality we would require some kind of notification system - an infrastructure we currently do not have in place.

IMHO it is better to provide a possibility to detect an update during application startup and perform a full re-index in such cases. WDYT?

@westei westei removed this from the v0.6.1 milestone Dec 29, 2017
@westei westei added in progress and removed ready labels Dec 29, 2017
westei added a commit that referenced this issue Dec 29, 2017
…rsation admin service `POST /admin/conversation/index` (requires the user to have ADMIN permissions).

NOTE the comments on the issue regarding using this method in cloud setups with embedded Solr Servers.
westei added a commit that referenced this issue Dec 29, 2017
…lows to query on startup if all documents in the index where indexed with the current version. If not a full re-build of the index is triggered. If not a partial update (via the last_sync field) is enabled. Changed the default for the `smarti.index.rebuildOnStartup` property to `false` as with the new functionality this is no longer required under normal operational circumstances.
@westei
Copy link
Member Author

westei commented Dec 29, 2017

Implemented a feature to the conversation index that allows to detect updates to the index.

After incompatible changes to the schema.xml or the indexing logic the ConversationIndexConfiguration#CONVERSATION_INDEX_VERSION needs to be increased.

A change of this version will trigger a full re-build of the conversation index on the next startup.

This replaces an older workaround that (by default) enforced a full rebuild on every startup (now the default is false)

@westei westei added this to the v0.6.1 milestone Dec 29, 2017
@westei
Copy link
Member Author

westei commented Jan 8, 2018

for validation one can look for INFO level loggings on startup:

  • in case of a full re-index because of a version update the log will contain conversation index contains documents indexed with an outdated version - full re-build required
  • in case of a partial update (no version update) the log will note Perform partial update of conversation index (lastSync date: {date})
  • in case a fill re-index is forced on every startup (smarti.index.rebuildOnStartup=true) the log will note full re-build on startup required via configuration

@westei westei closed this as completed Jan 15, 2018
@westei westei removed the in review label Jan 15, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants