CDI GPU support in the native runtime?

[dtrudg]

Related to - #1395

The new --oci mode for SingularityCE supports setting up GPUs and other devices in containers using the Container Device Interface (CDI): https://github.com/container-orchestrated-devices/container-device-interface

CDI is a new standard for defining the container manipulations necessary to present a device, using a well specified configuration file. Runtimes then typically apply these configurations according to device names passed with the --device flag.

Currently, the non-oci 'native' runtime supports two means of binding GPUs into containers:

• --nv or --rocm alone uses a naive binding approach. Nvidia and ROCm GPU devices are present in the container under /dev, and libraries/binaries bound in according to a singularity-specific configuration file (see docs here).
• --nv with --nvccliwill setup Nvidia GPU devices inside the container, by calling out to nvidia-container-cli, which is provided by the libnvidia-container package(s).

At present, we are not planning to deprecate either of these methods of binding GPUs into a container. However, it should be noted that CDI is now Nvidia's preferred method of container setup for their GPUs.

We're interested in knowing whether users anticipate that they would make beneficial use of CDI functionality in the native singularity runtime, in 4.x releases that would take place over the next year.

Some advantages of supporting CDI with the native runtime would include:

• Support for devices other than those handled by --nv / --rocm, e.g. Intel GPUs assuming CDI configurations are available.
• No need to maintain naive lists of GPU libraries/binaries to bind into a container. Using vendor tooling to generate CDI configurations ensures that the configurations are up-to-date for current driver versions.
• Consistent application of GPU support across Singularity and other runtimes.

Some disadvantages of CDI at this time may include:

• CDI Configuration files must be maintained for the GPUs on a host.
• Not yet well integrated with HPC batch schedulers, e.g. limiting GPUs through batch job options. Need to derive correct --device strings from batch job device allocation.

If you have any questions or thoughts about CDI... and whether it would be worth integrating with SingularityCE's native mode at this time, please comment in this thread!

[dtrudg]

Given that there hasn't been much input on this here, or in the community calls... we'll definitely push it out past 4.0.

Let's re-assess whether to include for 4.1 or 4.2 after the 4.0 development cycle is finished.