How to combine this with Cachix?

[hraban]

Hi, I've been trying to combine this with Cachix but I'm unable to get it to not download derivations that are already present in cachix. Do I misunderstand something about how the magic cache should work? I thought it was more like a local /nix/store, which should prevent redownloading from cachix?

I'm working on the ci/magic branch on https://github.com/hraban/cl-nix-lite :

• a run that just completed, that should have cached everything: https://github.com/hraban/cl-nix-lite/actions/runs/5393240641/jobs/9800528803
• immediately re-running all jobs still redownloads tons of stuff: https://github.com/hraban/cl-nix-lite/actions/runs/5393240641/jobs/9801186313

Backstory: I am working on a pretty large nix module for Common Lisp which contains a derivation for many of the popular packages (a few hundred). I currently use cachix and a custom sed script to only rebuild those derivations that are not yet in Cachix, and I'm playing around with this magic cache to see if it can remove the need for that custom sed script. So far I'm not seeing exactly how they're supposed to work together though. Particularly the "don't rebuild existing derivations" part of the magic cache seems elusive to me.

Here is a nightly run on my master branch using just the sed script:

• https://github.com/hraban/cl-nix-lite/actions/runs/5395920196/jobs/9798945825 (nothing changed so no rebuilding / redownload)
• https://github.com/hraban/cl-nix-lite/actions/runs/5385181677/jobs/9773921628 (new package was added, so only the dependencies were fetched)

Is that a workflow which the magic cache automates?

[lucperkins]

The issue here, I think, is that the Magic Nix Cache doesn't cache what's available in your upstream cache. The goal of the MNC is essentially to build less, not necessarily to fetch less. If something is available upstream, the MNC knows that it can be fetched in the future. This saves cache space on the Actions Cache side (GitHub is generous but cache space is not unlimited). So if you prefer to cache things on the Actions side, I'd recommend un-setting the upstream-cache parameter.

[hraban]

Thanks @lucperkins :) honestly I don't really have a clear plan, I was just playing around to see what this offers.

I followed your advice and removed the upstream-cache param but I get the exact same results:

• first run: downloads everything and stores it in the cache: https://github.com/hraban/cl-nix-lite/actions/runs/5414702351/jobs/9842130520
• second run: downloads everything, stores nothing in the cache: https://github.com/hraban/cl-nix-lite/actions/runs/5414702351/jobs/9842877172

Do I understand correctly that it should have had those cached, at that point?

[lucperkins]

@hraban In that first run you linked, the MNC copies 320 paths to the cache, while in the second it copies zero. So it is indeed caching. I don't really know the details of the Cachix Action but it's possible that they're not compatible. I don't really know Haskell so I'm not 100% sure how the cachix use command works, but I suspect it also updates the Nix configuration. It may be worth seeing the contents of $HOME/.config/nix/nix.conf after both Actions have been run. In general, though, it's not clear to me what benefit it would serve to use both Cachix and the Magic Nix Cache.

[dpc]

I arrived here to ask about combining MNC with cachix as well. We're using cachix already, and it works well, but maybe there's room for improvements still.

In general, though, it's not clear to me what benefit it would serve to use both Cachix and the Magic Nix Cache.

One problem with cachix in Github Action CI is that every run starts on a fresh VM and has to fetch all the artifacts from cachix, which seems slower than expected.

I was hoping that MNC would be faster, because it might serve data from infrastructure inside same data center etc. So MNC + cachix could be used as a hybrid cache, where MNC would be fetched from first (if available), then all the new artifacts would go to both.

Though I understand this is additional complexity and might not be the goal of this project (if it even does lead to any speedups).

[hraban]

@dpc are you building one single large final derivation, or are you building many small independent ones?

[dpc]

We have a somewhat extensive CI pipeline building large Rust project natively, running tests as Nix derivations, cross-compiling to various platforms, and all sorts of things.

[Zimmi48]

In general, though, it's not clear to me what benefit it would serve to use both Cachix and the Magic Nix Cache.

Cachix has lots of advantages (cross-project availability, populates a cache that can be used locally, etc.) that may make it the preferred way to cache Nix builds. But it has a major drawback (as highlighted in #19) which is that it requires a secret to be set, which won't happen for pull requests coming from forks. Users might hope to use the Magic Nix Cache action as a fallback in this case, to still be able to share builds across jobs and across rebuilds.

[hraban]

In general, though, it's not clear to me what benefit it would serve to use both Cachix and the Magic Nix Cache.

Cachix has lots of advantages (cross-project availability, populates a cache that can be used locally, etc.) that may make it the preferred way to cache Nix builds. But it has a major drawback (as highlighted in #19) which is that it requires a secret to be set, which won't happen for pull requests coming from forks. Users might hope to use the Magic Nix Cache action as a fallback in this case, to still be able to share builds across jobs and across rebuilds.

On the other hand, it seems that GH Actions seems to be quite conservative with sharing caches between branches, so you will end up having similar issues if e.g. you have two very similar feature branches.