Adding commonly used Haskell libraries

[aergus]

This probably also applies to other languages in varying extents, but I'd like to draw attention to the fact that a lot of commonly used functionality in Haskell is implemented outside of the "standard library" and thus currently missing from the corresponding Docker image. Unless the submissions are supposed to use only the standard prelude, I'd like to ask whether some commonly used Haskell libraries could be added to the image. If you agree to include include a certain set of libraries, I could make a PR with an extended Dockerfile.

As a concrete proposal, here are some packages the team I'm part of (udfew) used frequently in the last years:

• aeson: Fast JSON parsing and encoding
• array: Mutable and immutable arrays
• attoparsec: Fast combinator parsing for bytestrings and text (alternatively parsec or megaparsec)
• bytestring: Fast, compact, strict and lazy byte strings with a list interface
• cereal: A binary serialization library
• containers: Assorted concrete container types
• deepseq: Deep evaluation of data structures
• directory: Platform-agnostic library for filesystem operations
• filepath: Library for manipulating FilePaths in a cross platform way.
• hashable: A class for types that can be converted to a hash value
• io-streams: Simple, composable, and easy-to-use stream I/O
• lens: Lenses, Folds and Traversals (alternatively microlens-platform)
• monad-control: Lift control operations, like exception catching, through monad transformers
• MonadRandom: Random-number generation monad.
• mtl: Monad classes, using functional dependencies
• optparse-applicative: Utilities and combinators for parsing command line options
• random: Pseudo-random number generation
• text: An efficient packed Unicode text type.
• time: A time library
• transformers: Concrete functor and monad transformers
• transformers-base: Lift computations from the bottom of a transformer stack
• unordered-containers: Efficient hashing-based container types
• vector: Efficient Arrays

[beevee]

Sounds good. Can you make a PR? I can probably do this myself, but I'll spend much more time and will make stupid mistakes, since I'm not a Haskell developer.

[aergus]

Thanks, I can make a PR, but it would be nice if a few more things could be clarified before I start working on it:

1. The submissions will be built without Internet access, but installing the aforementioned libraries would require it. How should this be addressed? For example, should there be a separate prebuilt image with the build dependencies, and if that's the case, how should the submission Dockerfile access the image with the build dependencies?
2. Should the packages be installed via apt, cabal or stack (or something else)? (I would personally vouch for stack.)
3. On that note, my impression is that Stack is nowadays commonly used for managing dependencies of Haskell projects, so allowing submissions to be built with Stack would probably make the life easier for teams using Haskell. More concretely, one could install the aforementioned packages using a long time support resolver of Stack and replace the line running GHC in the Dockerfile with a line running stack build --resolver $RESOLVER. (This would have the side effect of giving the teams the possibility of configuring build options in their project files, but if you find that problematic, those can be (AFAIK) overridden by command line flags. Also, one could add a Stack-based Haskell image in addition if you want to keep the "vanilla" image.)

[beevee]

Let's do this:

1. Create a Dockerfile.base alongside the existing Dockerfile. Build your ideal base image for Haskell in it :-) You can use Stack. You can use Internet. We can later create a separate platform called haskell-cabal if there are any teams that disagree.
2. I'll set up a Docker Hub automation to build this image. It will be available as icfpcontest2020/haskell:latest.
3. Change Dockerfile accordingly using this base image. Try maintain backward compatibility with our starter kit or send a PR there also. You still can't use Internet in this image.

[aergus]

OK, that looks like a good plan. I will start implementing it in the next days.

[beevee]

Please note that I've committed a stub base image and made sure the entire build process works on our side. You can start from there: https://github.com/icfpcontest2020/dockerfiles/tree/master/dockerfiles/haskell

[aergus]

One more question: Is there a particular reason why the submissions are compiled and run in different containers (if that's the correct Docker jargon)? Since Stack will create a dynamically linked binary by default, just copying the compiled binary won't work if additional libraries installed via Stack are used. Though it is possible to compile statically linked binaries with Stack, it's a bit tricky, so things would be a lot easier if we could run stack exec main in the same image after compilation.

[beevee]

It's a small optimization. Runtime image does not contain any build dependencies, so it's smaller and easier to distribute among many worker nodes.

Actually, not so small: haskell:8.10.1 is 342MB, while debian:buster-20200607-slim is only 25MB.

If at all possible, please try to keep separate build and run images. Note that you are free to copy any files from your build image to your runtime image, not just one executable.

If it's too much trouble, I think we can tolerate extra image weight to make things simpler for you.