Attendees: jkarni, zimbatm, mic92, infinisil, kenji, drig/erethon, arian, sam , hexa, jeremy, jeff
-
Mic92: POC to evaluate nixpkgs on GitHub Actions. Results looked promising. nixpkgs-review would run in 5 minutes. the ofborg-eval was heavily swapping and taking 15min.
-
Infinisil: people might have to enable GHA in their fork, which is disabled by default
- Mic92: I didn't see this behaviour?
- Kenji: I think Github changed some defaults
- Mic92: I think this is true for periodic
- Infinisil: trying now for a new user
-
Mic92: if we want to pursue GHA, we would have to evaluate nixpkgs twice because we need to get the store paths for master, and the changes of the PR, and then we can compute all packages that have been changed, and append that textfile as data. This can then be re-used by nixpkgs-review.
-
Arian: why are we not using the PR workflow?
- Mic92: concurrency issues (limit of 20 runners per org).
- TODO: check if we hit the limit
-
hexa: the idea to comment on the PR is to compensate for the visibility issue?
- Mic92: yes. it sucks a bit, but this could be mitigated by a small web service.
- A thin wrapper that receives a webhook, checks back the PR status and translates it as a comment.
-
Infinisil: wouldn't it be possible to have a workflow that polls on behalf of the user?
- It's a workflow that tries to find the workflow on the user's push, in their fork.
- It would be triggered every time you synchronize the PR.
- Mic92: can you set this up in a way that the workflow gets triggered once the workflow is finished?
- Infinisil: I think you need to poll for this.
- Mic92: Is it a 1:1 mapping, or 1:N?
- infinisil: Something like
# .github/workflows/query-pr.yml on: pull_request_target jobs: check: runs-on: ubuntu-latest steps: - run: | gh api /repos/BASE_REPO/commits/GITHUB_SHA/check-runs
- Worry: Offloading OfBorg on to GH could give us trouble, because it might not be insignificant compute.
-
Jeremy: is this confined to PRs, or running on all branches?
- Mic92: it would be on push, but checking if the branch is part of the PR.
- Mic92: actually, there might be some synchronicity issue, because the PR happens after the push.
- Mic92: Because of that we might need a webservice that can trigger actions
- Mic92: can we get an event when we open a PR?
-
Arian: Team plan gives us 60 concurrent actions by the way. (And team plan is free for non-profit orgs)
-
Pushes don't have a base branch, need a base branch to compare the out paths
- Mic92: only if not open as a PR
- infinisil: Can pre-compute the out paths on push, cache out paths on Nixpkgs master, then comparing can be done in a PR action fairly easily
-
Jonas: who is going to make this happen?
- Infinisil, Alex Balsoft, Jeremy after early December, Mic92 can write some scripts and don't want to lead (want to work on the binary cache).
-
Infinisil: not convinced if that's a good idea.
-
Jonas: What would it take to get to feature parity
- What is the minimal set?
- Mic92: Minimal - Evaluate Prs
- Mic92: 2nd Phase - We can build packages.
- Mic92: Labels for mass-rebuilds
- Silvan: Requests reviews from maintainers (maybe not needed?)
- Silvan: Don't need to build manual with OfBorg anymore (Is already built in ci)
- Silvan: Evaluating without aliases
- Jonas: Discourage IFD's
- Silvan: Maybe it really is good to split this up in two parts:
- Evaluating
- Building
-
Arian: Average job queue time is currently 9s: https://github.com/NixOS/nixpkgs/actions/metrics/performance
-
Arian: I would aim for: Lets just try with
pull_request:and only do the complicatedpush:abuse if that job queue time is gonna go up significantly -
Jonas: Looking forward to having eval failures to block merges!
-
Silvan: Who can review pr's and help out:
- kenji: +1
- balsoft: +1
- Mic92: +1
- dgrig/erethon: +1
-
Silvan: Can add GH Team to ping for this issue
-
Silvan: TODO - Mention this effort on Discourse
-
Mic92: How do we coordinate?
- Main Evalation
- Figure out parts we can parallelize:
- most parts are fairly orthogonal
-
Silvan: Somebody could lead the Building part:
- Mic92: Find someone who can help out, maybe on discourse?
- Silvan: GH doesn't have all the architectures
- Mic92: Start with the ones we have currently
- Silvan: If we don't need ealuation anymore - this could save a lot of resources, could optimize
- hexa: Yes, but it might not apply on top of staging for example
- Silvan: Yes, staging can probably be ignored
- hexa: Yes, it tries to build against the target branch, led to some problems, for example always trying to build llvm on darwin -> continuuous timeouts
- Mic92: A ton of stuff we could potentially optimize
- Silvan: Empower users to build on more architectures
- Mic92: Convenient to have logs in public
- Mic92: I would like to see a /build command, so that builds can be manually triggered
-
Silvan: Optimization of the Eval part:
- look at path that actually changed
- mic92: Aware of nix script that gives names of paths that are actually changed?
- Silvan: Yes
- Mic92: If heavy swapping, then it might speed up, else we might see a slow down
-
Silvan: Where should we report?
- discourse?
- Mic92: Discussion would be nicer on GH, because we can link issues/pr's.
- Transfer of the Macs located at Detsys to Flying Circus
- Scheduled for 2024-11-25
- Currently enrolled into Detsys MDM Account. Can we set something up to migrate that to an infra team account?
- Report back the result to the Mac Mini Logistics room on Matrix
- MDM built into macs, but need to be enrolled into an mdm vendor
- Arian: Don't have to do it, but very convenient
- Mic92: if there are no major problems, should look into MDM as well.
- Oakhost Macs are available
- Need the usual setup
- 3 new machines
- arian can set this up
- initial password
- Mic92: Mac enrollment not very automated yet, last time hexa wrote some stuff down
- Arians keys needs to be added to the repository
- Arian:
- need access to oakhost
- ssh key to infra
- Mac Issues:
- hexa: Forking issues on seqoia
- hexa: Running quite well atm
- hexa: Patched out chrooting of nix, applied patches on top of darwin builders
- hexa: Not sure exactly why that works
- hexa: Upstreaming rosetta2-gc to nix-darwin currently
- hexa: darwin 15.1 had issues -> hetzner doesn't roll back (need rescue mode)
- hexa: Rollbacks likely possible with mdm
- arian: We can likely add hetzner darwin machines to be managed by mdm, but not too sure
- arian: Looks into if we can add without physical access, look into what detsys did
- Equinix Metal Exit Plan
- https://md.darmstadt.ccc.de/eqm-exit-plan
- hexa: This is what we had, this is what we need, this is what it is going to cost.
- mic92: Should we check out what we need for the arm64 builders?
- hexa: Basically choice between: 64GB, or 256GB of memory
- hexa: Likely want the bigger memory
- Mic92: Can try the same for arm64, for x86 we can look in to funding.
- Mic92: How long do we need for set up? - a day? Shouldn't take long to set up.
- Mic92: I will set this in motion, unless someone else want's to reach out.
- Mic92: Ok, I will do it.
- hexa: Ideally we don't have 20 small machine, but 5 big machines. Which would be good for maintenance reasons. Because we likely won't get a netboot setup anymore.
- Security Tracker
- dgrig: Jonas gave me access to a Hetzner Cloud project a couple of weeks ago. A VM is up and running, I'm figuring out how to implement this in the same way as nixos-infra.
- dgrig: Do we care about having this in Terraform? I used TF to spin this up, but the state is in my computer currently, do we care to push it to S3?
- drgrig: Can I do
nixos-installer --flake nixos/nixos-infra?How can I install the nixos-infra - Mic02: Inputs :"${inputs.nixos-infra}/keys" can convert to string.
- Example:
users.users.root.openssh.authorizedKeys.keys = [] ++ (builtins.filter (l: l != [ ]) (builtins.split "\n" (builtins.readFile inputs.phaer-keys)));
- dgrig: Do we care about the Terraform state yet?
- hexa: Do we need the state yet?
- drgrid: It is the tf state
- consensus: We don't care
- Jeremy: Nit - export the keys as a module