Proposal: offer an object-based merge command

[Thesola10]

Applications such as rpm-ostree perform essentially a differential commit, based on performing small changes to an existing commit, or applying another commit on top of it.

The current approach is to check out the commit (slow), apply changes on an overlay, then traverse and commit the overlay filesystem (very slow). Given that OSTree is a content-addressed object tree, it is possible to perform a much faster merge by creating new metadata objects directly, skipping most of the heavy work.

I have two proposals, both of which can coexist:

Proposal 1: Merge commits, like Git

This proposal involves a hypothetical operation, ostree merge A B, which traverses, and statically merges, directory trees of commit A and commit B, into a new commit.

Handling of merge conflicts

If a directory exists in A and B, and there is no collision in their child objects, the merged directory is the sum of directories A and B.

If a directory exists in A and B, and an element exists of the same name and hash in their child objects, they are considered the same and kept as-is.

If a directory exists in A and B, and an element exists of the same name, but whose type or contents differ, the user can be manually prompted for a resolution, or a Git-like conflicts file could be used. OSTree being OSTree, no attempt will be made to understand or merge file contents.

Alternatively, the last specified commit could be given priority and overwrite any conflicting file.

Unlike Git, OSTree will be able to merge commits with no common history.

Proposal 2: Incremental commits

This proposal involves a hypothetical operation, ostree add A T, which adds the contents of directory T on top of commit A, resulting in a new commit.

For the best experience, directory T is expected to only contain files which do not exist in commit A. File removal can be envisioned using overlayfs style whiteouts, allowing for native committing of an overlayfs upper directory.

Conflict management can be handled in a similar way to Proposal 1 above.

This proposal can be emulated with ostree merge A $(ostree commit T).

Use cases

Merge operations can be used to promote OSTree as a native OCI storage backend, by constructing static merged images from layers stored in individual commits.

As stated above, the incremental commit process could be sped up by only having to read and add the upper directory from a source overlay.

Overall, this proposal would allow users to leverage the OSTree object architecture more efficiently, and encourage development of more creative applications.

[cgwalters]

We are targeting composefs as the future heart of versioned immutable filesystem trees. It already has an efficient way to merge filesystem trees.

We are also creating an OCI-native operating system with http://github.com/containers/bootc - which supports the "incremental commits" idea by reusing OCI layers as they exist.