Vendor static / generated assets

[nilium]

Request

I'd like to see the static and generated assets (those built from with the Makefile) vendored in Vault source releases (i.e., not master, branches, or elsewhere -- just for tagged releases). This applies to the UI but also any generated protobuf, for example, that is not already vendored.

When building Vault from source, we can mostly rely on the vendor directory being up to date for Go dependencies. However, static assets still need to be built from scratch. In comparison, Consul and Nomad both vendor assets, making it easier to build them from source and produce the same binary each time.

Vendoring assets makes it far easier to package both Consul and Nomad for Void Linux as well, since we can't build Vault on every platform due to issues with yarn (outside the scope of anything Hashicorp) -- this makes cross-compilation necessary, and means that some users cannot build Vault. So, in places where Hashicorp's own binaries cannot be accepted (such as when support for static linking or musl is a requirement [see hashicorp/nomad#5535 and hashicorp/nomad#5643]), vendoring assets can make it much simpler to build, package, and run Vault.

Desired Solution

As the subject says, I'd like to see Vault's static assets vendored for release commits. I don't expect this would be done for untagged commits, similar to how the vendor directory is only consistent for releases.

Whether prerelease tags should have vendored assets, I don't know. Likely look to Consul and Nomad for guidance.

Alternatives

An alternative would be to break Vault's assets out into per-tag release archives, similar to how a source archive is made available by GitHub for each Vault tag.

Additional Use-Cases

Outside of Void Linux's troubles around building Vault, this ensures that each Vault build gets the same assets and moves it towards much simpler reproducible builds. With vendored assets, and assuming vendored Go dependencies are kept, every source release of Vault should be able to build regardless of the availability of the tools needed at the time the assets were generated.

[Conan-Kudo]

This is also valuable for me for packaging for Fedora and CentOS, as JS dependencies are difficult to get right...

[the-maldridge]

Just ran into this again today. We actually don't provide the vault UI in my environment because of how difficult it is to reliably build it from source.

[the-maldridge]

Bumping this with an update from some months later. Vault is now fails-to-build-from-source on Void due to being pinned to an ancient version of nodejs. The specific error is:

error [email protected]: The engine "node" is incompatible with this module. Expected version " >= 10.* <11". Got "12.18.4"

[FatmanUK]

Ticket for the nodejs issue: #8504

[Vaelatern]

Can we please have a tarball build when the releases happen with the static-assets built? Would this be accepted if I do so in the circleci config?