exposing the mt predicate to dune libraries

[avsm]

The ctypes port to dune (yallop/ocaml-ctypes#588) is almost working, except for the need to select which subpackage to link based on the -mt predicate in ocamlfind. #96 has some discussion of this, but the specific META file in ctypes is:

package "foreign" (
 version = "0.14.0"
 description = "Dynamic linking of C functions"
 requires(-mt) = "ctypes.foreign.unthreaded"
 requires(mt) = "ctypes.foreign.threaded"

Both the unthreaded and threaded variants have different c stubs, and cannot be linked simultaneously without a module clash. I'm not sure how to make this work without some support for mt in a dune file, as the right package is selected based on whether the executable uses the threads library or not.

Suggestions for workarounds welcome to unblock the ctypes port...

[avsm]

I've pushed a workaround using META templates to yallop/ocaml-ctypes@a5a9655. The problem I'm finding is when linking an executable there are now missing dependencies, since Dune has no idea about the extra dependencies introduced by the manually written requires fields that use the -mt predicates.

This means that as a hack one of the choices is manually written in each executable at the moment; e.g.:

(executables
 (names date)
 (libraries ctypes-foreign ctypes-foreign.unthreaded))

instead of

(executables
 (names date)
 (libraries ctypes-foreign))

[bobot]

Currently dune support the mt predicate by always supposing that the mt predicate is true. So currently the unthreaded version doesn't make sense with dune.

[avsm]

Thanks @bobot -- I've hardcoded the mt version in the ongoing port of ctypes for now.

[avsm]

One observation from yallop/ocaml-ctypes@25ec72a (which shifts the use of ctypes-foreign.threaded/unthreaded to use virtual modules) is that the mt predicate seems like another special case for link-time tag selection of virtual modules.

If we decide on a namespace for virtual modules that is consistent (e.g. foo.threaded, foo.unthreaded, foo.jsoo, foo.mirage-xen, foo.mirage-freestanding), then the relevant subpackages could all be selected by a link-time specification in dune rather than have to be manually specified. This is roughly what the predicate system in ocamlfind does currently.

[ghost]

@avsm that's the idea of library variants, i.e. the part that comes after virtual library: adding a mechanism to automatically select an implementation when none is specified.

[avsm]

@diml indeed -- I just want to make sure that selecting a subpackage name in today's virtual libraries is sufficient for future compatibility with library variants. It would be quite a pain to rename releases that use virtual libraries with dune 1.7.0 otherwise

[rgrinberg]

So would having a threads variant going to be sufficient for you here? Do you also need a "default" variant tag if none are specified? For example, the user is creating an executable that uses ctypes.foreign and hasn't specified in the library.

I think variants is a good solution for this as well, but it's a bit weird how we want the use of (one of) the threads to enable this tag.

[rgrinberg]

@emillon Since you're driving the ctypes port, is this still relevant?

[emillon]

The whole threaded/unthreaded distinction got removed upstream in yallop/ocaml-ctypes#651, so I don't think it's relevant anymore.