JBUILDER - A composable build system for OCaml

Jbuilder is a build system designed for OCaml projects only. It focuses on providing the user with a consistent experience and takes care of most of the low-level details of OCaml compilation. All you have to do is provide a description of your project and Jbuilder will do the rest.

The scheme it implements is inspired from the one used inside Jane Street and adapted to the open source world. It has matured over a long time and is used daily by hundred of developpers, which means that it is highly tested and productive.

Jbuilder comes with a manual. If you want to get started without reading too much, you can look at the quick start guide.

Overview

Jbuilder reads project metadata from jbuild files, which are either static files in a simple S-expression syntax or OCaml scripts. It uses this information to setup build rules, generate configuration files for development tools such as merlin, handle installation, etc…

Jbuilder itself is fast, has very low-overhead and supports parallel builds on all platforms. It has no system dependencies: all you need to build jbuilder and packages using jbuilder is OCaml. You don’t need make or bash as long as the packages themselves don’t use bash explicitely.

Especially, one should be able to install OCaml on Windows with a binary installer and then use only the Windows Console to build Jbuilder and packages using Jbuilder. Although this hasn’t been tested yet.

Strengths

Composable

Take n repositories that use Jbuilder, arrange them in any way on the file system and the result is still a single repository that Jbuilder knows how to build at once.

This make simultaneous development on multiple packages trivial.

Gracefully handles multi-package repositories

Jbuilder knows how to handle repositories containing several packages. When building via opam, it is able to correctly use libraries that were previously installed even if they are already present in the source tree.

The magic invocation is:

$ jbuilder build --only-packages <package-name> @install

Building against several configurations at once

Jbuilder is able to build a given source code repository against several configurations simultaneously. This helps maintaining packages across several versions of OCaml as you can tests them all at once without hassle.

This feature should make cross-compilation easy, see details in the roadmap.

This feature requires opam.

Jenga bridge

Jenga is another build system for OCaml that has more advanced features such as polling or much better editor integration. Jenga is more powerful and more complex and as a result as much more dependencies. It is planned to implement a small bridge between the two so that a Jbuilder project can build with Jenga using this bridge.

Status

Jbuilder is now in beta testing stage. Once a bit more testing has been done, it will be released in 1.0.

Roadmap

See ROADMAP.org for the current plan. Help on any of these points is welcome!

Implementation details

This section is for people who want to work on Jbuilder itself.

Bootstrap

In order to build itself, Jbuilder uses an OCaml script (bootstrap.ml) that dumps most of the sources of Jbuilder into a single boot.ml file. This file is built using ocamlopt or ocamlc and used to build everything else.

OCaml compatibility test

Install opam switches for all the entries in the jbuild-workspace.dev file and run:

$ make all-supported-ocaml-versions

Repository organization

• vendor/ contains dependencies of Jbuilder, that have been vendored
• plugin/ contains the API given to jbuild files that are OCaml scripts
• src/ contains the core of Jbuilder, as a library so that it can be used to implement the Jenga bridge later
• bin/ contains the command line interface
• doc/ contains the manual and rules to generate the manual pages

Design

Jbuilder was initially designed to sort out the public release of Jane Street packages which became incredibly complicated over time. It is still successfully used for this purpose.

One necessary feature to achieve this is the ability to precisely report the external dependencies necesseray to build a given set of targets without running any command, just by looking at the source tree. This is used to automatically generate the <package>.opam files for all Jane Street packages.

To implement this, the build rules are described using a build arrow, which is defined in src/build. In the end it makes the development of the internal rules of Jbuilder very composable and quite pleasant.

To deal with process multi-plexing, Jbuilder uses a simplified Lwt/Async like monad, implemented in src/future.

Code flow

• src/jbuild_types contains the internal representation of jbuild files and the parsing code
• src/jbuild_load contains the code to scan a source tree and build the internal database by reading the jbuild files
• src/gen_rules contains all the build rules of Jbuilder
• src/build_system contains a trivial implementation of a Build system. This is what Jenga will provide when implementing the bridge