1.0+beta18 breaks my ugly (hum) project #573

ttamttam · 2018-03-03T10:43:29Z

I have a small project in which I ab-used (mis-used?) dune in order to build a shared object instead of a library. Since revision 1.0+beta18, dune fails to compile my project.

I'm not sure if my ugly hack is necessary? Is there a better way, or the possibility to improve dune to manage shared-object target?
Is it a problem with 1.0+beta18, or is it due to a non-conventional use of dune?
The real jbuild file (see below) is generated (The first line is (* -*- tuareg -*- *)), because it must be slightly adapted, depending on the target architecture (For Windows: so -> dll, "-runtime-variant _pic" -> ""): is there a better solution to solve this kind of portability issue?

Structure of my project:

/
   grandeurs.opam: (void file)
   lib/
        *.ml
        *.mli
        jbuild: (jbuild_version 1) (library(  (name grande)   (public_name grandeurs)))
   dll/
        grande_rev.c
        grandeurs.h
        grandeurs_stub.ml
        grandeurs_stub.mli
        jbuild:  (listed below)

The non-conventional jbuild file:

 (jbuild_version 1)

(rule(
  (targets(grande_rev.o))
  (deps(
    (file grande_rev.c)
    (file grandeurs.h)))
  (action (run ${OCAMLC} -c ${<}))
))

(rule(
  (targets (grandeurs_stub.cmi))
  (deps (grandeurs_stub.mli ../lib/grande.cma))
  (action (run ${OCAMLC} -c -I ../lib grande.cma ${<}))
))

(rule(
  (targets (grandeurs_stub.cmx grandeurs_stub.o))
  (deps (grandeurs_stub.ml grandeurs_stub.cmi ../lib/grande.cmxa))
  (action(run ${OCAMLOPT} -c -I ../lib grande.cmxa ${<}))
))

(rule(
  (targets (grandeurs.so))
  (deps(grandeurs_stub.cmx grande_rev.o ../lib/grande.cmxa ))
  (action (run ${OCAMLOPT} -o ${@} -output-obj -runtime-variant _pic -I ../lib grande.cmxa grandeurs_stub.cmx grande_rev.o))
))

(alias(
      (name install)
      (deps (grandeurs.so))
))

With 1.0+beta17:

$ make
jbuilder build
    ocamldep lib/grande.dependsi.ocamldep-output
      ocamlc lib/grande__.{cmi,cmo,cmt}
    ocamldep lib/grande.depends.ocamldep-output
      ocamlc lib/grande__Arrondi.{cmi,cmti}
      ocamlc lib/grande.{cmi,cmti}
    ocamlopt lib/grande__.{cmx,o}
      ocamlc lib/grande__Arrondi.{cmo,cmt}
      ocamlc lib/grande__Interval.{cmi,cmti}
    ocamlopt lib/grande__Arrondi.{cmx,o}
      ocamlc lib/grande__Interval.{cmo,cmt}
    ocamlopt lib/grande__Interval.{cmx,o}
      ocamlc lib/grande.{cmo,cmt}
      ocamlc lib/grande.cma
      ocamlc dll/grandeurs_stub.cmi
    ocamlopt lib/grande.{cmx,o}
    ocamlopt lib/grande.{a,cmxa}
    ocamlopt lib/grande.cmxs
      ocamlc dll/grande_rev.o
    ocamlopt dll/grandeurs_stub.{cmx,o}
    ocamlopt dll/grandeurs.so

With 1.0+beta18:

$ make
jbuilder build
      ocamlc dll/grandeurs_stub.cmi (exit 2)
(cd _build/default/dll && /home/matt/.opam/4.06.0+flambda/bin/ocamlc.opt -c -I ../lib grande.cma grandeurs_stub.mli)
File "grandeurs_stub.mli", line 13, characters 32-47:
Error: Unbound module Grande
Makefile:4 : la recette pour la cible « all » a échouée
make: *** [all] Erreur 1

Thanks for this nice program, and for your time!

The text was updated successfully, but these errors were encountered:

ghost · 2018-03-05T11:21:38Z

Basically we need #23 to do this properly. I've been meaning to take a stab at finishing it, I'll try to do that soon

ttamttam · 2018-03-05T12:52:41Z

@diml I did not spot #23

Yes, I think it would help a lot (I often share my OCaml work as shared libraries) and answer my questions 1 and 3.

About Q2: What about my current work around? Is it supposed to stop working with 1.0+beta18 revision, or is there a bug?

ghost · 2018-03-05T13:06:11Z

The reason it broke with beta18 is that we moved the .cmi files to separate per-library directories. We do that as OCaml relies on include directories and it's the only way to hide files it shouldn't see. For the workaround, I think there might be a grande.cmi missing in some deps field

ttamttam · 2018-03-05T13:12:09Z

Understood and solved by adding grande.cm{i,x} dependencies in the missing places.

Thanks

ghost · 2018-03-12T11:44:11Z

FTR, #23 has just been merged. You can now define an executable stanza with (modes (shared_object))

ttamttam · 2018-03-12T19:32:37Z

This merge solved point 3, because it introduced $(ext_dll) etc.

But I do not know how to specify my C stub and .h file?

The library options (c_names and install_c_headers) do not work with executable stanza.
And (modes (shared_object)) is allowed only for executable.

I'm not sure I can build a ML + C shared object with executable in shared_object mode?

ghost · 2018-03-13T09:28:15Z

In this case, you need to split your executable into a library and an executable. Attach the c_names to the library and put (modes (shared_object)) in the executable stanza

ttamttam · 2018-03-18T13:54:11Z

Sorry: I did not succeed :-(
Here is a repository with my attempt: https://github.com/ttamttam/ocaml-dune-shared-object-test. I hope you'll have the time to have a look and show me what I'm doing wrong?

ghost · 2018-03-18T14:18:28Z

I just tried with the development version of jbuilder and it seems to work:

$ make directdll 
jbuilder build automatic/dll/direct.so
$ ls _build/default/automatic/dll/direct.so 
_build/default/automatic/dll/direct.so*

ttamttam · 2018-03-18T17:40:48Z

I just adapted the test repository https://github.com/ttamttam/ocaml-dune-shared-object-test#makefile-targets:

make directdll compiles, as you checked…
but make directtest does not compile
while make manualtest does compile and work

(see https://github.com/ttamttam/ocaml-dune-shared-object-test/blob/master/README.org#automatic)

I thank you for your patience.

ghost · 2018-03-18T19:50:04Z

So there are two problems:

the code in automatic/dll doesn't reference mylibstubed, which is the library containing the C stubs, as a result it's not being linked in the .so by the compiler. There are two solution: add a reference from automatic/dll to mylibstubed, such as: let linkme = Mylibstubed.linkme, or add (library_flags (:standard -linkall)) to automatic/mylibstubed to force it to be linked
the C stubs of mylibstubed are referenced from the OCaml code, so the stubs are never going to be linked in. You need to add a .ml file with an external declaration to reference at least one of the C function

ttamttam · 2018-03-18T20:26:14Z

Let me thank you again!

I should have spotted the first point. The second one seems less obvious to me.

The modifications for a working solution:

Callback.register calls were moved from automatic/dll/direct.ml to automatic/libstubed/mylibstubed.ml
automatic/dll/ now contains only jbuild and a void ml file
(library_flags (:standard -linkall)) was added to automatic/libstubed/jbuild
a dummy function was added in mylib_stub.c, and declared as external in automatic/libstubed/mylibstubed.ml

Despite the dummy function, this is a lot simpler. Cool.

ghost · 2018-03-18T21:37:48Z

No problem. I guess we should allow to have C stubs for executables, that would make such cases a bit simpler

ttamttam · 2018-03-19T08:22:53Z

I don't know if a lot of people turn their OCaml code as shared object? Difficult to say if it's worth it?

This morning, I did go on with my tests, and found a small problem on cygwin with shared_object.

$ uname -a
CYGWIN_NT-10.0 DESKTOP-9DIKPMF 2.9.0(0.318/5/3) 2017-09-12 10:18 x86_64 Cygwin
$ opam switch show
4.06.0+flambda+mingw64c

The DLL does not compile with "-runtime-variant _pic". I ran the command manually without it to generate it (and tested manually that it works fine):

$ jbuilder build automatic/dll/direct.dll
    ocamlopt automatic/dll/direct.dll (exit 2)
(cd _build/default && C:\OCaml64\home\MatthieuDubuget\.opam\4.06.0+flambda+mingw64c\bin\ocamlopt.opt.exe -w -40 -g -o automatic/dll/direct.dll -cclib -lws2_32 -cclib -lversion -output-complete-obj -runtime-variant _pic -I automatic/libstubed -I lib lib/mylib.cmxa automatic/libstubed/mylibstubed.cmxa automatic/dll/.direct.eobjs/direct.cmx)
File "caml_startup", line 1:
Error: Cannot find file libasmrun_pic.a

$ cd _build/default/

$ ocamlopt.opt.exe  -w -40 -g -o automatic/dll/direct.dll -cclib -lws2_32 -cclib -lversion -output-complete-obj -I automatic/libstubed -I lib lib/mylib.cmxa automatic/libstubed/mylibstubed.cmxa automatic/dll/.direct.eobjs/direct.cmx

ghost · 2018-03-19T12:02:40Z

4.06.0+flambda+mingw64c

This is mingw, not cygwin.

I'm not sure why libasmrun_pic.a is not built/installed. @dra27, isn't -fPIC necessary with mingw?

dra27 · 2018-03-19T12:14:02Z

No PIC on Windows...

ghost · 2018-03-19T15:14:49Z

#635

rgrinberg · 2018-03-22T05:48:03Z

@ttamttam since you've said that this issue is resolved [1], I'm closing this ticket.

[1] #635 (comment)

ttamttam closed this as completed Mar 5, 2018

ttamttam reopened this Mar 12, 2018

ttamttam mentioned this issue Mar 19, 2018

Do not pass -runtime-variant _pic on Windows #635

Merged

rgrinberg closed this as completed Mar 22, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

1.0+beta18 breaks my ugly (hum) project #573

1.0+beta18 breaks my ugly (hum) project #573

ttamttam commented Mar 3, 2018

ghost commented Mar 5, 2018

ttamttam commented Mar 5, 2018

ghost commented Mar 5, 2018

ttamttam commented Mar 5, 2018

ghost commented Mar 12, 2018

ttamttam commented Mar 12, 2018

ghost commented Mar 13, 2018

ttamttam commented Mar 18, 2018

ghost commented Mar 18, 2018

ttamttam commented Mar 18, 2018

ghost commented Mar 18, 2018

ttamttam commented Mar 18, 2018

ghost commented Mar 18, 2018

ttamttam commented Mar 19, 2018

ghost commented Mar 19, 2018

dra27 commented Mar 19, 2018

ghost commented Mar 19, 2018

rgrinberg commented Mar 22, 2018

1.0+beta18 breaks my ugly (hum) project #573

1.0+beta18 breaks my ugly (hum) project #573

Comments

ttamttam commented Mar 3, 2018

ghost commented Mar 5, 2018

ttamttam commented Mar 5, 2018

ghost commented Mar 5, 2018

ttamttam commented Mar 5, 2018

ghost commented Mar 12, 2018

ttamttam commented Mar 12, 2018

ghost commented Mar 13, 2018

ttamttam commented Mar 18, 2018

ghost commented Mar 18, 2018

ttamttam commented Mar 18, 2018

ghost commented Mar 18, 2018

ttamttam commented Mar 18, 2018

ghost commented Mar 18, 2018

ttamttam commented Mar 19, 2018

ghost commented Mar 19, 2018

dra27 commented Mar 19, 2018

ghost commented Mar 19, 2018

rgrinberg commented Mar 22, 2018