Ingestion of evaluation results of any supported channel

[RaitoBezarius]

The security tracker acts on supported channels, and we need to ingest an evaluation of all nixpkgs for any given supported channel at any point in time.

The tracker should subscribe to channel bumps (open problem), see how https://git.qyliss.net/pr-tracker detects them and how https://git.eno.space/label-tracker.git/ tracks them.

Proposal of implementation

Every time a channel bump, repull the repository, extract a worktree of that channel (or git clone via the reference for fast checkout), run nix-eval-jobs on that commit sha and collect the result and archive it as JSON with meta results (!!!).

Run this as a background job or a cron job that can easily be managed by infrastructure people or administrators to perform any maintenance task like cancelling evaluations, restarting evaluations, configuring the number of concurrent evaluations, etc.

Ideas for the future

Expose this data of evaluations publicly and let people access it directly, it's useful in general.

[RaitoBezarius]

I should also be able to use a management command to seed my evaluation results in my database without having to go through nix-eval-jobs through the background task system as a starter.

[Tom-Hubrecht]

I believe that to track the channel bumps the easiest way is to regularly fetch https://nixos.org/channels data

[RaitoBezarius]

I believe that to track the channel bumps the easiest way is to regularly fetch nixos.org/channels data

It's not certain this API will stay available on the long term, I don't advise to use it.

[Tom-Hubrecht]

Well, the way the other tools do it is by cloning the repo and looking at the ref of the branches linked to the channels

[RaitoBezarius]

Well, the way the other tools do it is by cloning the repo and looking at the ref of the branches linked to the channels

I think that's the sure way to go, or you can listen to events of the GitHub repository of nixpkgs.

[RaitoBezarius]

So ingestion was implemented of manually evaluated nixpkgs.

All that's left is, for "perfect":

• Track channels, e.g. track GitHub repo and poll/get notified about when a channel moves
• Trigger nix-eval-jobs as a background task for this
• Reuse the importing entrypoint we have in the manual importer

In the meantime, what we can hack is:

• Perform evaluation-time configuration of channels to track, i.e. load a fixture
• Setup crons based on known delays of channels move
• Pick up channel move manually in systemd timers and run nix-eval-jobs
• Throw them at the manual importer one by one

[RaitoBezarius]

We need to add the meta attributes in the ingester:

• knownVulnerabilities
• sourceProvenance