Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accumulation of blocked GitHub CI jobs #957

Open
emanuel-schmid opened this issue Oct 4, 2024 · 0 comments
Open

Accumulation of blocked GitHub CI jobs #957

emanuel-schmid opened this issue Oct 4, 2024 · 0 comments
Labels

Comments

@emanuel-schmid
Copy link
Collaborator

Starting on September 26 there was a nearly contiguous series of failing CI (Unit testing) actions.
The jobs usually failed after 6 hours stalling at

2024-09-26T08:46:55.4880563Z ##[group]micromamba list
2024-09-26T08:46:55.4882034Z [command]/home/runner/micromamba-bin/micromamba list -r /home/runner/micromamba -n climada_env_3.10
2024-09-26T14:29:15.9153370Z ##[error]The operation was canceled.

without further information.

According to actions/runner#1326 (comment) this may be related to memory limitations.
As we all know the environment creation requires a lot of memory ...

It is very unlikely that an environment change is the root cause for this burst of stallings, as there have been no changes in the environment around that time, neither in the requirement file nor in the environment itself.

For a proper investigation we would have to set a self-hosted runner in place with the very likely outcome that it won't reproduce.
As an immediate step I'll set timeouts for the whole job to 20 minutes as all succeeding jobs are below that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant