Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Latest version (v2.317.0) of github runner available is forbidden. Nothing to upgrade to. #3378

Closed
mattnakama-skytap opened this issue Jul 9, 2024 · 7 comments
Labels
bug Something isn't working

Comments

@mattnakama-skytap
Copy link

mattnakama-skytap commented Jul 9, 2024

Describe the bug
Our github runner refuses to run jobs, saying that version v2.317.0 is forbidden. I checked this repo's latest release, and it's v2.317.0. What are we supposed to upgrade to?

The last successful job completed at 20:46:14 UTC
The first "forbidden" error message shows up at 20:46:41 UTC

To Reproduce
Steps to reproduce the behavior:

  1. Install latest github-runner v2.317.0 on NixOS
  2. Start it and try to run a github action/workflow
  3. See error

Expected behavior
A clear and concise description of what you expected to happen.

Runner Version and Platform

v2.317.0

OS of the machine running the runner? OSX/Windows/Linux/...
NixOS 24.05

What's not working?

recent journalctl output:

Jul 09 20:57:25 automation2 Runner.Listener[432636]: √ Connected to GitHub
Jul 09 20:57:25 automation2 Runner.Listener[432636]: Current runner version: '2.317.0'
Jul 09 20:57:26 automation2 Runner.Listener[432636]: 2024-07-09 20:57:25Z: Listening for Jobs
Jul 09 20:57:27 automation2 Runner.Listener[432636]: An error occured: Error: Forbidden Runner version v2.317.0 is deprecated and cannot receive messages.
Jul 09 20:57:27 automation2 systemd[1]: github-runner-argo-2.service: Main process exited, code=exited, status=1/FAILURE

Job Log Output

There's no job log output, because it refuses to run jobs.

Runner and Worker's Diagnostic Logs

If applicable, add relevant diagnostic log information. Logs are located in the runner's _diag folder. The runner logs are prefixed with Runner_ and the worker logs are prefixed with Worker_. Each job run correlates to a worker log. All sensitive information should already be masked out, but please double-check before pasting here.

@mattnakama-skytap mattnakama-skytap added the bug Something isn't working label Jul 9, 2024
@ellisvalentiner
Copy link

Seeing the same problem.

2024-07-09 21:22:38.129  DEBUG --- Docker enabled runner detected and Docker daemon wait is enabled
2024-07-09 21:22:38.13  DEBUG --- Waiting until Docker is available or the timeout of 120 seconds is reached
}CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
2024-07-09 21:22:38.146  NOTICE --- WARNING LATEST TAG HAS BEEN DEPRECATED. SEE GITHUB ISSUE FOR DETAILS:
2024-07-09 21:22:38.147  NOTICE --- https://github.com/actions/actions-runner-controller/issues/2056

√ Connected to GitHub

Current runner version: '2.317.0'
2024-07-09 21:22:40Z: Listening for Jobs
An error occured: Error: Forbidden Runner version v2.317.0 is deprecated and cannot receive messages.
Runner listener exit with terminated error, stop the service, no retry needed.
Exiting runner...
2024-07-09 21:22:40.771  NOTICE --- Runner init exited. Exiting this process with code 0 so that the container and the pod is GC'ed Kubernetes soon.
Removing the .runner file

@mattnakama-skytap
Copy link
Author

mattnakama-skytap commented Jul 9, 2024

@luketomlinson apologies for the ping, but could this be a backend issue? I happened to see your comment on #3366 (comment) and am wondering if there was a recent backend change. We did not change anything about our runners today; they just suddenly stopped working.

@luketomlinson
Copy link
Contributor

👋 This was a bad backend change. This should be rolled back now.

@ellisvalentiner
Copy link

Thanks! Our runners are working again.

@mattnakama-skytap
Copy link
Author

mattnakama-skytap commented Jul 9, 2024

This should be rolled back now.

Thank you 🙂
I can confirm that our runners are back online 💪

Edit: Note for others reading this thread: You may have to restart your runners, since this is a "permanent" error, and the runners don't seem to retry on their own. We had to restart ours after the backend change was reverted to bring them back online.

@alexellis
Copy link

Also ran into this. Thanks for the swift rollback.

@jacobwgillespie
Copy link

@luketomlinson We are currently seeing this error again, here's an example session:

[2024-07-26 17:05:02Z ERR  GitHubActionsService] GET request to https://broker.actions.githubusercontent.com/message?sessionId=cfa9c468-c485-4c87-9c52-ae63a51668a6&status=Online&runnerVersion=2.317.0&os=Linux&architecture=X64&disableUpdate=true failed. HTTP Status: Forbidden

Since v2.318.0 was just released one hour ago, perhaps this has broken the logic?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants