Adding more robust reporting for PerformHarvestJobs #34
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Prior to this commit we had two emails that were sent: 1. **harvest_failed**: Regardless of `Harvester#harvest_oai_items` results, if there were sidecar items missing, we'd send an `harvest_failed` email. This email did not include any additional information on what sidecars were missing. 2. **harvest_succeeded**: If all harvest documents had existing sidecars, we'd send a `harvest_succeeded` email. *Note:* prior to and including this change, there's not handling of errors encountered elsewhere (e.g. invalid data, network issues, database failures, etc). With this commit, we consolidate the `harvest_failed` and `harvest_succeeded` into `harvest_set_completed`. That `harvest_set_completed` means that the set completed and there may or may not be issues with document's related sidecars. If there are issues, we include details of those issues in the email. Closes #185
….com:harvard-lts/spotlight-oaipmh-resources into jeremyf---harvard-lts---CURIOSity#185-part-2
phil-plencner-hl
approved these changes
Sep 14, 2022
There was a problem hiding this comment.
I tested this out and it works as described in https://github.com/harvard-lts/CURIOSity/issues/185
Approved.
phil-plencner-hl
deleted the
jeremyf---harvard-lts---CURIOSity#185-part-2
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Prior to this commit we had two emails that were sent:
Harvester#harvest_oai_itemsresults, if there were sidecar items missing, we'd send anharvest_failedemail. This email did not include any additional information on what sidecars were missing.harvest_succeededemail.Note: prior to and including this change, there's not handling of errors encountered elsewhere (e.g. invalid data, network issues, database failures, etc).
With this commit, we consolidate the
harvest_failedandharvest_succeededintoharvest_set_completed. Sending theharvest_set_completedemail is sent when we are done running the job; there may or may not be issues with document's related sidecars. If there are issues, we include details of those issues in the email.Closes #185
Testing Instructions
Pending