Is it safe to cache output?

[kindaro]

My understanding is that Spago puts all build products into the output sub-directory. This includes both all the dependencies and the project code itself.

I would like to speed up continuous integration by caching this directory. Is it expected by design that exactly the updated code will be re-built in the presence of output filled with earlier build products?

Also, would newer versions of dependencies be downloaded and re-built automatically if such are available?

[f-f]

I feel that caching the output folder is likely to not be a safe thing to do at least whenever the compiler version changes. A very-closely-related ticket we have about this is #527, maybe reading that could help with some context.

Since the build products are owned by the compiler (i.e. when building Spago mainly takes care of getting the sources and feeding them to the compiler), I'd suggest raising an issue in there to see if enabling such caching could be something worth pursuing.

Something that you could cache though is the .spago folder, so that you wouldn't have to download all the sources on every build.

Also, would newer versions of dependencies be downloaded and re-built automatically if such are available?

No. The basic principles of package-sets (on which Spago is based) is the version of every dependency is pinned exactly, and will always be the same - as only in this way we can make sure that the project will continue building over time - until you upgrade the package-set version, which you can do with spago upgrade-set

[hdgarrood]

It should actually be safe to cache the output directory, and we do this in our CI at Lumi. The compiler version is stored in the output directory so that the compiler knows when to invalidate existing build products. In general, as long you haven’t fiddled with the contents of the output directory, incremental builds should just work: in cases where the compiler can identify that work can be reused, it should do so, and otherwise it should rebuild as necessary. If deleting the output directory and rebuilding ever causes a successful build to start failing or a failing build to start succeeding, that’s a compiler bug.

[f-f]

Wonderful, thank you @hdgarrood for chiming in! Then I think we should definitely document this somewhere - probably in a section under "how to", with the same title as this issue)

[thomashoneyman]

For what it’s worth, the PureScript Contributors libraries all cache the output directory in CI, and it appeared to work fine when I tested builds with changed compiler versions. That is, if I remember correctly, changing the compiler version essentially invalidated the output directory.