Consider moving to xcaddy project

[francislavoie]

Currently, this approach has some UX problems. It requires cloning this repo and copying the files to embed to inside of the repo. This is awkward for various reasons, such as fighting with git with the files directory, having to clone at all meaning you have some extra directory sitting around, having to copy your files yourself (i.e. needing to write a script/Makefile to automate it), needing to modify module.go if you want to rename the embed, etc.

My suggestion is that we could move this to the https://github.com/caddyserver/xcaddy project, where it would be a build flag, i.e. xcaddy build --embed <path>. Then xcaddy would create the module.go file itself into a temp dir, optionally change the //go:embed line to use a different base path (maybe with another flag like --embed-dir), copy the given path (files to embed) to inside of the temp dir, then invoke the build as normal.

I think this would be significantly better UX, because there would be no need for messing around with git, no copying files by hand, no need to write your own build script to make it work as a repeated command. It would just become an argument to the xcaddy build command they (probably) already use.

[mholt]

This is probably doable. We might want to generalize that implementation to support any other Caddy modules that use go:embed. But yeah, I like the idea.

First, letsee how it goes separately on its own for a while, see how popular it is, etc.

[mohammed90]

I have an idea for this. There will be 2 parts to support this:

• xcaddy is given a directory or a zip archive to embed. If it is not zipped, it'll zip it first. Then it will simply append it to the end of the generated executable.
• This module will look for the zip file inside the executable that's running it, opening it, and serving file the embedded zipped file. Something like this is the gist of it:

exec, _ := os.Executable()
reader := zip.OpenReaderr(exec)

// .... later ...

reader.Open(name)

[francislavoie]

Yeah, that was my thinking @mohammed90.

Does it need to be zipped though? Doesn't that just add friction? I thought we could embed a whole directory. Unless you mean we should zip to compress in the compiled binary (reduce binary's size)? We'd need to make sure the fs module in Caddy can decompress it once at config startup and keep it in mem I suppose, otherwise we'd be wasting a decent amount of CPU from reading in a zipped bundle every request?

[mohammed90]

The compression isn't to reduce the binary size. It's for the benefit of having the magic header to know where it starts and to conserve the file structure.

The opening of the zip can be in the Provision and it'll be closed in Cleanup (implementing the caddy.CleanerUpper interface). It won't decompress at every request. It'll only seek/search through the decompressed zip.

I wonder if github.com/mholt/archiver/v4 may be useful to make the ambivalent to the embedded archive type.