archive_tarball worker

[douglatornell]

Creates a tarball of a month's run results and
moves it to remote archival storage. Compression is not used for the tarball
because the netCDF files that compose most of it are already highly compressed.
A .index text file containing a list of the files in the tarball is also created
and moved to the remote storage.

[codecov]

Codecov Report

Merging #104 (a30f175) into main (33d11d7) will increase coverage by 0.21%.
The diff coverage is 89.42%.

@@            Coverage Diff             @@
##             main     #104      +/-   ##
==========================================
+ Coverage   72.99%   73.20%   +0.21%     
==========================================
  Files         122      124       +2     
  Lines       16026    16234     +208     
  Branches     1305     1319      +14     
==========================================
+ Hits        11698    11884     +186     
- Misses       4277     4299      +22     
  Partials       51       51              

Flag	Coverage Δ
unittests	73.20% <89.42%> (+0.21%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
nowcast/workers/archive_tarball.py	72.15% <72.15%> (ø)
nowcast/next_workers.py	98.62% <100.00%> (+0.01%)	⬆️
tests/test_next_workers.py	99.71% <100.00%> (+<0.01%)	⬆️
tests/workers/test_archive_tarball.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 33d11d7...a30f175. Read the comment docs.

[douglatornell]

Tarball and index file creation in dcbe5c6 was successfully tested in production.

[douglatornell]

Tarball & index creations and rsync to graham-dtn in 51f4da1 was successfully tested in production.

[douglatornell]

Automatic month-end launch of archive_tarball worker in 71ea18f was successfully tested in production,
though transfer rates to graham-dtn are extremely slow, so rsync is still in progress ~24 hours later 🤷‍♂️