Make it more obvious that scsb dump step is optional #2283
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Also, organize the docs directory a bit, and remove broken link.
christinach
requested changes
Dec 19, 2023
docs/indexing.md
Outdated
|
|
||
| Takes 24-25 hours to complete. As they download and unpack they will be placed | ||
| in `/tmp/updates/` and as they are processed they will be moved to `/data/bibdata_files/scsb_update_files/`; you can follow the progress by listing the files in these directories. You can also find the most recent Full Partner ReCAP Records from [the events page](https://bibdata.princeton.edu/events), and look at the dump files in its json. Be sure not to deploy bibdata in the middle of this job, or else the job will have to start all over again from the beginning. | ||
| If needed, [pull the most recent SCSB full dump records into dump files](./scsb/dump_files.md). |
There was a problem hiding this comment.
line 98 should be 'Then' ...since the full scsb request happened
christinach
reviewed
Dec 19, 2023
docs/scsb/dump_files.md
Outdated
| ``` | ||
| This kicks off an import job which will return immediately. This can be monitored in [sidekiq busy queue](https://bibdata.princeton.edu/sidekiq/busy) or [sidekiq waiting queue](https://bibdata.princeton.edu/sidekiq/queues/default) | ||
|
|
||
| Takes 24-25 hours to complete. As they download and unpack they will be placed |
There was a problem hiding this comment.
The time it takes to complete has changed. we need to monitor it. The job I ran last week took 12 hours to complete. The previous job that ran through cron it took 22 hours.
There was a problem hiding this comment.
The job from yesterday took 11 hours. I would change line 12 to say Takes 11-12 hours to complete.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Also, organize the docs directory a bit, and remove broken link.