DISCUSS: Disable hackage-security expiry checks by default

[snoyberg]

This is a request for end user input. Please use the Github reaction 👍 and 👎 emojis to share your thoughts.

Stack uses the hackage-security mechanism to download the package index (set of available packages on Hackage). hackage-security uses expiry times as part of its security. I'm personally not sold on this in Stack's usage, but we've stuck with the default until now.

Twice now, files from Hackage have fallen out-of-date, which has left Stack users unable to initialize new installations. We have no control over fixing that, and instead need to wait for the Hackage team to update the expired files.

Given that (1) this seems to be a recurring problem without any evidence of automation to solve it that I can see, and (2) I'm unconvinced that there is actually a security benefit to this feature, I propose that, by default, we disable the expiry checks in Stack. I propose we include this in the next point release of Stack, scheduled for the near future.

Related issues:

• Verification error: <repo>/mirrors.json is expired #3731
• mirrors.json expired: stack build has suddenly stopped working in all my projects #4927
• mirrors.json file is expired haskell/hackage-server#834

If you're on this issue and you need a workaround, please see my comment with a workaround.

[istathar]

We have layers of proxies between us and the internet and things can go wrong. It doesn't make those of us advocating Haskell look very good when suddenly after a weekend long upstream proxy outage that we probably didn't know about builds start failing.

We have sensible policies about the degree of freshness we have to maintain in our build dependencies before we allow ourselves to make a release, but that's release to production and we have tools that gate that for us; inhibiting development and testing is probably not the right outcome of this otherwise well meaning mechanism.

[snoyberg]

I looked into this a bit more, since I was curious as to why in my testing cabal v2-update on a fresh installation seemed to work while stack update was failing. From what I can tell, cabal-install is using a completely different DNS-based mirror discovery mechanism implemented here:

https://github.com/haskell/cabal/blob/ed3ae1332ca7ef4d1d09b6da6494a44f3e40f5a8/cabal-install/Distribution/Client/Security/DNS.hs

From the following comment, it seems that cabal-install is already ignoring any kind of expiry information from the mirrors.json file:

-- NB: hackage-security doesn't require DNS lookups being trustworthy,
-- as the trust is established via the cryptographically signed TUF
-- meta-data that is retrieved from the resolved Hackage repository.
-- Moreover, we already have to protect against a compromised
-- @hackage.haskell.org@ DNS entry, so an the additional
-- @_mirrors.hackage.haskell.org@ DNS entry in the same SOA doesn't
-- constitute a significant new attack vector anyway.

I'm not sure why there are two different implementations of mirror discovery between the hackage-security and cabal-install codebases, and I'd prefer if we were using the same codepath as cabal-install. But this seems like a slight nudge in the direction of saying "yes" to this issue (which is what I already wanted to do anyway).

[gbaz]

If that PR is indeed disabling all checks, then I think it is unfortunate from a security standpoint. The only specific check that hackage security does not conduct is the one on mirrors, when it uses dns based discovery, since as that comment notes, there is reason to consider dns-based discovery already authenticated. Disabling all checks -- if that's what's happening here -- opens up the way to rollback-based attacks on the security infrastructure generally.

That said, I think there is low benefit from protecting against such rollback attacks (because we do not currently have infra in place to automate things enough to have only short windows between expiries and resignings, and also because rollback attacks only work in conjunction with other vectors of compromise), but nonetheless, ignoring expiry data altogether seems like it is bypassing some best practices.