Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Figure out why automated ingestion script wants to reingest a tarball #169

Open
bedroge opened this issue Dec 4, 2023 · 0 comments
Open
Labels
bug Something isn't working

Comments

@bedroge
Copy link
Collaborator

bedroge commented Dec 4, 2023

See https://github.com/EESSI/staging/pull/486. This tarball was already ingested, but for some reason it was picked up again much later. This means that the script somehow decided to mark this as a new tarball in find_state:
https://github.com/EESSI/filesystem-layer/blob/main/scripts/automated_ingestion/eessitarball.py#L67

Log entries:

2023-11-03 19:22:42,837 - INFO - 2023.06/software/linux/x86_64/amd/zen2/1687960841/eessi-2023.06-software-linux-x86_64-amd-zen2-1687960841.tar.gz has already been ingested, skipping...
2023-11-03 19:42:52,132 - INFO - 2023.06/software/linux/x86_64/amd/zen2/1687946108/eessi-2023.06-software-linux-x86_64-amd-zen2-1687946108.tar.gz has already been ingested, skipping...
2023-11-03 19:42:58,251 - INFO - Found new tarball 2023.06/software/linux/x86_64/amd/zen2/1687960841/eessi-2023.06-software-linux-x86_64-amd-zen2-1687960841.tar.gz, downloading it...
2023-11-03 19:43:00,724 - INFO - Adding tarball's metadata to the "staged" folder of the git repository.
2023-11-03 20:00:04,561 - INFO - Found new tarball 2021.12/compat/linux/aarch64/1675183498/eessi-2021.12-compat-linux-aarch64-1675183498.tar.gz, downloading it...
2023-11-03 20:00:04,810 - ERROR - Failed to download tarball 2021.12/compat/linux/aarch64/1675183498/eessi-2021.12-compat-linux-aarch64-1675183498.tar.gz from eessi-staging to /software/tarballs/eessi-2021.12-compat-linux-aarch64-1675183498.tar.gz.
2023-11-03 20:00:04,943 - ERROR - Failed to download metadata file 2021.12/compat/linux/aarch64/1675183498/eessi-2021.12-compat-linux-aarch64-1675183498.tar.gz.meta.txt from eessi-staging to /software/tarballs/eessi-2021.12-compat-linux-aarch64-1675183498.tar.gz.meta.txt.
2023-11-03 20:00:04,943 - WARNING - Skipping this tarball...
2023-11-03 20:00:07,284 - INFO - Found new tarball 2021.12/compat/linux/ppc64le/1675198951/eessi-2021.12-compat-linux-ppc64le-1675198951.tar.gz, downloading it...
2023-11-03 20:00:07,375 - ERROR - Failed to download tarball 2021.12/compat/linux/ppc64le/1675198951/eessi-2021.12-compat-linux-ppc64le-1675198951.tar.gz from eessi-staging to /software/tarballs/eessi-2021.12-compat-linux-ppc64le-1675198951.tar.gz.
2023-11-03 20:00:07,444 - ERROR - Failed to download metadata file 2021.12/compat/linux/ppc64le/1675198951/eessi-2021.12-compat-linux-ppc64le-1675198951.tar.gz.meta.txt from eessi-staging to /software/tarballs/eessi-2021.12-compat-linux-ppc64le-1675198951.tar.gz.meta.txt.
2023-11-03 20:00:07,444 - WARNING - Skipping this tarball...

So half an hour earlier things were still find, but then something happened (GitHub issue perhaps?) that caused failures in assigning the right states to tarballs.

@bedroge bedroge added the bug Something isn't working label Dec 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant