[Feature Request] Reproducible builds

[KurtPfeifle]

I would like to see "reproducible builds" for AppImages.

When I use appimagetool to convert an AppDir into an AppImage, the md5 or shaX hashes change after every run, even if the AppDir didn't change.

I think this has to do with some timestamps being automatically set to current by mksquashfs.

Is there a way to avoid this?

It would be very beneficial to only have changing md5/shaX hashes if and only if some "meaty" content of the AppDir changed!

[probonopd]

A similar request came up here recently:
electron-userland/electron-builder#2454

It looks like mksquashfs got patches that can do it:
https://sourceforge.net/p/squashfs/mailman/message/35503759/

[KurtPfeifle]

Helpful resources for this topic:

• https://reproducible-builds.org/docs/
• https://reproducible-builds.org/specs/source-date-epoch/
• https://github.com/bmwiedemann/theunreproduciblepackage
• https://github.com/bmwiedemann/reproducible-faketools
• https://anonscm.debian.org/git/reproducible/reproducible-utils.git/

[TheAssassin]

@probonopd according to https://sourceforge.net/p/squashfs/mailman/message/35503759/, the only real blocker for this issue is the mkfs timestamp that stores when the file system has been created. We could set a fixed value for this (we would have to think about the best strategy for determining a value for that, though, as setting this to 0 always is IMO inferior to what we have at the moment -- my suggestion would be to e.g., check all files' timestamps in the file system and use the latest one).

If we decide we don't need that value at all, though (i.e., we're not interested in when an AppImage's squashfs filesystem was created), we can just set it to some fixed value, too. .tar.* archives don't have such timestamps either, as far as I can see, and I never missed such a value.

As far as I can see, there's no side effects in using either method. Our tools don't depend on this value, neither do third parties'.

I'll check out how we can use the patches suggested in the squashfs mailing list, and am going to build a PR demonstrating this.

[probonopd]

Thanks for picking this up @TheAssassin.

Do we want this to be applied all the time or just under certain circumstances?

[TheAssassin]

I don't see any use in the timestamp, and I think we can safely overwrite it. The remaining question is, shall we continue to use 0 (the epoch) as fixed timestamp, or use the other method I suggested?

[probonopd]

Well, if I get this right, we need to set the timestamp to a fixed value that does not change, so 0 should be as good as any other, shouldn't it?

[TheAssassin]

#625 (comment)

[probonopd]

I had read this, which led my to my comment. Then let's go for 0.

[TheAssassin]

Alright. I'll merge the PR.

[TheAssassin]

Reopening as the topic is re-discussed in #695.