Improved Fault Tolerance for Registry and Registry API

[jordanpadams]

💡 Description

B14.0

• Queries for data products with lots of metadata attributes crash in browser registry-api#292
• Update Java params and Docker image to expand JVM memory beyond 50% registry-api#300
• As a developer, I want to know what version of Harvest was used to load a product harvest#119
• As a EN Operator, I want to the registry API to be fault tolerant, analysis registry-api#297
• As a PDS operator, I want to know the health of the registry API service registry-api#336
• Registry-Sweepers Error: contained no hits when hits were expected registry-sweepers#69

B14.1

• Investigate and fix intermittent Registry-API Errors registry-api#378
• Write documentation and Training for Registry monitoring #224
• As a user, I want the registry to have 99.9999999% uptime #176
• Improve Fault Tolerance of Harvest for Forbidden Access error and Timeout harvest#125
• Too many requests error to OpenSearch harvest#134
• Data too large error from very large data products harvest#133
• Enhance provenance sweeper with more graceful failure when registry contains zero documents registry-sweepers#25
• Unexpected WARNINGs related to http requests harvest#101

[jordanpadams]

📆 April Status: Task started. Multiple options discussed during breakout on 5/9. Delayed 1 week due to some delays in other task completion. No impact on delivery.

[jordanpadams]

📆 May status: PR awaiting approval for healthcheck endpoint. No impact on build anticipated.

[jordanpadams]

📆 June status: Scope has been expanded as additional improvements have been identified and scope of #185 scope has increased. No impact on build anticipated.

[jordanpadams]

📆 July status: Significant delays due to several issues with Registry / API. Will cause deferral of other downstream tasks, but expected to be completed for delivery.

[jordanpadams]

📆 August status: Deferred some sub-tasks from this ticket to B14.1, but no impact on delivery to I&T.

[jordanpadams]

📆 September status: In work. Scope continues to increase, but top priority for B14.1.

[jordanpadams]

📆 October status: Retry functionality implemented, which closed most of the tickets. A few items remain, but will evaluate priority with respect to this build or a future build. On schedule.

[jordanpadams]

📆 November status: Delayed one sprint due to 2 unresolved bugs and a couple other hot fixes required for the release. On schedule. No impact on other deliverables expected.

[jordanpadams]

All sub-tasks for B14.1 completed. Next phase created for B15.0. #259