Reindex MCP Prod ATM registry index with corrected mappings

[alexdunnjpl]

💡 Description

The reindexing sweeper picked up some properties which aren't described by the DD and aren't desirable to fix (or at least, aren't desirable to map as keyword).

These will need to be removed from the mapping, and a new index created with that mapping, migrated from the existing index.

@sjoshi-jpl to discuss with @alexdunnjpl upon his return

⚔️ Parent Epic / Related Tickets

No response

[alexdunnjpl]

@jordanpadams per @sjoshi-jpl , the logstash instance was nuked in the past 24hrs, and redeploying/reconfiguring will be a significant (1wk) amount of work.

Given that fact, it seems like the best option available to us may be to manually (i.e. python script) paginate/insert into a new index, run whatever confirmation checks we deem necessary, then delete/rename.

My only concern is ensuring consistency/completeness of the copied data (as was a factor in the pubcloud->mcp migration)

Any thoughts on that?

[tloubrieu-jpl]

We will use logstash for that work.

[alexdunnjpl]

@alexdunnjpl and @sjoshi-jpl to collaborate on this task

[tloubrieu-jpl]

@sjoshi-jpl is preparing the re-indexation with logstash.

[alexdunnjpl]

Status: logstash does not support AOSS as an input so logstash migration not possible. @alexdunnjpl to develop reusable tool for this purpose

[tloubrieu-jpl]

The code goes to registry-operation

[alexdunnjpl]

Partially implemented, currently blocked pending availability of a fresh temporary index

@sjoshi-jpl see slack for details of request

[alexdunnjpl]

Status: implemented/tested consistency check.

Awaiting @sjoshi-jpl for one last new index, then I can do a final migration and we can close this out

[jordanpadams]

Status: Running running. Keeps running running.

[alexdunnjpl]

Status: No longer running - AOSS fails repeatably under load, preventing reindexing sweeper completion.

See #329

@sjoshi-jpl investigating AOSS metrics

[jordanpadams]

Blocked by #329