(Archived) Roadmap - SingularityCE 3.10

[dtrudg]

SingularityCE 3.10 is under development for release in May 2022. The following roadmap items are currently scheduled to be included.

3.10 Features

• SystemD as cgroups manager
  Systemd as cgroups manager #299
  systemd is the default init for all of our target distributions, and with cgroups v2 it would be useful to use it for cgroups management. This would allow cgroups applied to containers to fit neatly into the user slice / session scopes in the systemd managed v2 hierarchy. Support merged.

• Rootless cgroups v2 resource limits
  Rootless cgroups resource limits (cgroups v2) #300
  The cgroups v2 hierarchy supports delegation, and management via systemd. This can be used to implement cgroups resource limits without root permissions being required. Will benefit use of Singularity outside of a traditional batch scheduler. E.g. workflow software will be able to enforce limits directly for singularity execution of containers. Support merged.

• Replace OCI engine with runc
  Retire SingularityCE's own OCI runtime implementation, and incorporate runc. Initially wire up runc for relevant singularity oci commands only. Essentially, SingularityCE will perform a mount from SIF and initial spec (config.json) creating. Other portions of the oci lifecycle will then be implemented via runc. Merged

  This is the first step of the work discussed in the [SingularityCE 4.0 and Beyond article].

• Correct cgroup namespace support for OCI commands
  cgroups namespace does not work correctly #298
  Fix creation of the cgroup namespace that is permitted via OCI config in the OCI runtime. This will now be achieved via OCI runtime replacement. Fixed via runc.

• Begin Removal of Code Supporting Legacy Distros
  Removal of Code Supporting Legacy Distros (3.10 tasks) #82
  SingulartyCE contains various workarounds for the RHEL6 / 2.6 kernel, old versions of invoked external programs etc. Special cases supporting these distributions can be removed gradually through 3.10 and beyond. This will reduce code and testing complexity. Merged 3.10 changes.

• Fully OCI/Docker compatible arg and env handling - optional config
  Compatibility mode to match Docker/OCI arg and env escaping / quoting behaviour #487
  Due to legacy design and implementation choices dating to Singularity 2.x, SingularityCE interprets arguments to singularity run differently from docker and other runtimes, performing a single level of shell evaluation. This causes some compatibility issues. In addition, environment variables perform evaluation. Merged.

  We cannot modify this behaviour trivially to match docker, as various users exploit the current situation to blur the lines between host and container environments. We should implement a new mode that can be explicitly enabled to match the Docker / OCI behaviour exactly. This may then become the default in future versions.

• Cgroups limits via CLI flags
  Add flags for direct application of cgroups resource limits #717
  The Singularity CLI should support flags that allow applying cgroups resource limits directly, without needing to write a cgroups toml file. E.g. it should be possible to singularity run --cpus 1 mycontainer.sif to limit execution to a single CPU. Merged.

• Experimental squashfuse SIF mount support
  feat: Experimental squashfuse unprivileged SIF mount (actions) #718
  Using experimental functionality in sylabs/sif add a means to mount SIF files with squashfuse, so that they can be used without privilege, without needing to extract to a temporary sandbox directory. Implementation should be outside of the Singularity runtime engine, as it should be adaptable for mounts with the oci command group, and future plans involving using an OCI low-level runtime. Merged.