Dune 3 `dune exec --` on Windows _always_ rebuilds the executable #5466

MisterDA · 2022-02-21T11:10:05Z

With dune.3 on Windows, when I dune exec -- using the executable public_name, without the .exe, the executable is always rebuild. I think that's a regression.
Using the "local" name (e.g., ./main.exe) also forces recompilation each time.

Possibly related to (but I don't think so):

Fixes dune exec not adding .exe on Windows (#4371, fixes #3322)

@snowleopard wrote:

I have a suspicion that this may be related to the "artifact substitution" mechanism that changes the contents of executables. I saw some issues with executable targets being re-promoted (but not re-built) but never had a chance to look into this

The text was updated successfully, but these errors were encountered:

bobot · 2022-02-21T14:28:55Z

My two cents: artifact subtitution is not done inside _build, and not done before executing with dune exec --.

You can perhaps test with --debug-cache=workspace-local to get information about what is not considered up to date.

MisterDA · 2022-02-22T16:11:50Z

$ dune init project main && cd main
Success: initialized project component named main

$ dune exec --debug-cache=workspace-local -- main
Workspace-local cache miss: _build/default/.dune/configurator: never seen this target before
Workspace-local cache miss: _build/default/.dune/configurator.v2: never seen this target before
Workspace-local cache miss: _build/default/bin/.merlin-conf/exe-main: never seen this target before
Workspace-local cache miss: _build/default/bin/main.ml: never seen this target before
Workspace-local cache miss: _build/default/bin/main.mli: never seen this target before
Workspace-local cache miss: _build/default/lib/.merlin-conf/lib-main: never seen this target before
Workspace-local cache miss: _build/default/lib/main.ml-gen: never seen this target before
Workspace-local cache miss: _build/default/lib/.main.objs/byte/main.cmi: never seen this target before
Workspace-local cache miss: _build/default/lib/.main.objs/native/main.cmx: never seen this target before
Workspace-local cache miss: _build/default/bin/.main.eobjs/byte/dune__exe__Main.cmi: never seen this target before
Workspace-local cache miss: _build/default/bin/.main.eobjs/native/dune__exe__Main.cmx: never seen this target before
Workspace-local cache miss: _build/default/lib/main.a: never seen this target before
Workspace-local cache miss: _build/default/bin/main.exe: never seen this target before
Workspace-local cache miss: _build/install/default/bin/main.exe: never seen this target before
Hello, World!

$ dune exec --debug-cache=workspace-local -- main
Workspace-local cache miss: _build/default/.dune/configurator: never seen this target before
Workspace-local cache miss: _build/default/.dune/configurator.v2: never seen this target before
Workspace-local cache miss: _build/default/bin/.merlin-conf/exe-main: never seen this target before
Workspace-local cache miss: _build/default/bin/main.ml: never seen this target before
Workspace-local cache miss: _build/default/bin/main.mli: never seen this target before
Workspace-local cache miss: _build/default/lib/.merlin-conf/lib-main: never seen this target before
Workspace-local cache miss: _build/default/lib/main.ml-gen: never seen this target before
Workspace-local cache miss: _build/default/lib/.main.objs/byte/main.cmi: never seen this target before
Workspace-local cache miss: _build/default/lib/.main.objs/native/main.cmx: never seen this target before
Workspace-local cache miss: _build/default/bin/.main.eobjs/byte/dune__exe__Main.cmi: never seen this target before
Workspace-local cache miss: _build/default/bin/.main.eobjs/native/dune__exe__Main.cmx: never seen this target before
Workspace-local cache miss: _build/default/lib/main.a: never seen this target before
Workspace-local cache miss: _build/default/bin/main.exe: never seen this target before
Workspace-local cache miss: _build/install/default/bin/main.exe: never seen this target before
Hello, World!

$ tree -la _build
_build
├── default
│   ├── .dune
│   │   ├── configurator
│   │   └── configurator.v2
│   ├── bin
│   │   ├── .main.eobjs
│   │   │   ├── byte
│   │   │   │   ├── dune__exe__Main.cmi
│   │   │   │   └── dune__exe__Main.cmti
│   │   │   └── native
│   │   │       ├── dune__exe__Main.cmx
│   │   │       └── dune__exe__Main.o
│   │   ├── .merlin-conf
│   │   │   └── exe-main
│   │   ├── main.exe
│   │   ├── main.ml
│   │   └── main.mli
│   └── lib
│       ├── .main.objs
│       │   ├── byte
│       │   │   ├── main.cmi
│       │   │   ├── main.cmo
│       │   │   └── main.cmt
│       │   └── native
│       │       ├── main.cmx
│       │       └── main.o
│       ├── .merlin-conf
│       │   └── lib-main
│       ├── main.a
│       ├── main.cmxa
│       └── main.ml-gen
├── install
│   └── default
│       └── bin
│           └── main.exe
└── log

So even if the targets are present on the filesystem, they're ignored.

snowleopard · 2022-02-22T18:39:58Z

Oh, wow, judging by the output you get, everything gets rebuilt for you (not just the executable).

Could you do the same with dune build main.exe --debug-cache=workspace-local instead of dune exec?

MisterDA · 2022-02-23T08:50:00Z

The target main (or any variation around it) wasn't recognized.

$ dune build --debug-cache=workspace-local
Workspace-local cache miss: _build/default/.dune/configurator: never seen this target before
Workspace-local cache miss: _build/default/.dune/configurator.v2: never seen this target before
Workspace-local cache miss: _build/default/META.main: never seen this target before
Workspace-local cache miss: _build/default/main.dune-package: never seen this target before
Workspace-local cache miss: _build/default/main.opam: never seen this target before
Workspace-local cache miss: _build/install/default/lib/main/META: never seen this target before
Workspace-local cache miss: _build/install/default/lib/main/dune-package: never seen thistarget before
Workspace-local cache miss: _build/install/default/lib/main/opam: never seen this target before
Workspace-local cache miss: _build/default/bin/.merlin-conf/exe-main: never seen this target before
Workspace-local cache miss: _build/default/test/.merlin-conf/exe-main: never seen this target before
Workspace-local cache miss: _build/default/bin/main.mli: never seen this target before
Workspace-local cache miss: _build/default/test/main.mli: never seen this target before
Workspace-local cache miss: _build/default/bin/main.ml: never seen this target before
Workspace-local cache miss: _build/default/test/main.ml: never seen this target before
Workspace-local cache miss: _build/default/lib/.merlin-conf/lib-main: never seen this target before
Workspace-local cache miss: _build/default/lib/main.ml-gen: never seen this target before
Workspace-local cache miss: _build/default/lib/.main.objs/byte/main.cmi: never seen this target before
Workspace-local cache miss: _build/default/test/.main.eobjs/byte/dune__exe__Main.cmi: never seen this target before
Workspace-local cache miss: _build/default/lib/main.cma: never seen this target before
Workspace-local cache miss: _build/default/lib/.main.objs/native/main.cmx: never seen this target before
Workspace-local cache miss: _build/default/bin/.main.eobjs/byte/dune__exe__Main.cmi: never seen this target before
Workspace-local cache miss: _build/default/test/.main.eobjs/native/dune__exe__Main.cmx: never seen this target before
Workspace-local cache miss: _build/default/bin/.main.eobjs/native/dune__exe__Main.cmx: never seen this target before
Workspace-local cache miss: _build/default/lib/main.a: never seen this target before
Workspace-local cache miss: _build/default/test/main.exe: never seen this target before
Workspace-local cache miss: _build/default/bin/main.exe: never seen this target before
Workspace-local cache miss: _build/default/lib/main.cmxs: never seen this target before
Workspace-local cache miss: _build/install/default/bin/main.exe: never seen this target before
Workspace-local cache miss: _build/default/main.install: never seen this target before
$ dune build --debug-cache=workspace-local
# nothing

MisterDA · 2022-02-23T08:52:13Z

Running dune exec after dune build doesn't rebuild everything! Maybe dune exec doesn't create a root dependency?

MisterDA · 2022-02-23T15:36:26Z

The same problem seems to appear with dune utop.

MisterDA · 2022-02-28T08:47:58Z

@snowleopard any other idea? let me know if I can help.

snowleopard · 2022-03-01T19:34:09Z

@snowleopard any other idea? let me know if I can help.

Sorry, I couldn't find a chance to investigate this yet.

What needs to be done is debugging why Dune's workspace-local cache isn't effective in this case. The main module that is responsible for caching build rules is src/dune_engine/rule_cache.ml -- perhaps, you could try finding the cause?

rgrinberg · 2022-04-22T18:38:04Z

@MisterDA could you submit a cram test that reproduces this problem? That would be a good first step to address this.

emillon · 2022-05-18T15:01:02Z

FWIW, I can't reproduce this. The second run is a clean "Hello, world" without any cache message.

emillon · 2022-05-18T15:01:10Z

(dune 3.1.1)

emillon · 2022-05-18T15:07:02Z

The bug is in 3.0.2 but not in 3.1.1. Not sure by what, but I think that we can close this one.

rgrinberg added the windows label Apr 22, 2022

rgrinberg closed this as completed May 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dune 3 `dune exec --` on Windows _always_ rebuilds the executable #5466

Dune 3 `dune exec --` on Windows _always_ rebuilds the executable #5466

MisterDA commented Feb 21, 2022

bobot commented Feb 21, 2022 •

edited

Loading

MisterDA commented Feb 22, 2022

snowleopard commented Feb 22, 2022

MisterDA commented Feb 23, 2022

MisterDA commented Feb 23, 2022

MisterDA commented Feb 23, 2022

MisterDA commented Feb 28, 2022

snowleopard commented Mar 1, 2022

rgrinberg commented Apr 22, 2022

emillon commented May 18, 2022

emillon commented May 18, 2022

emillon commented May 18, 2022

Dune 3 dune exec -- on Windows _always_ rebuilds the executable #5466

Dune 3 dune exec -- on Windows _always_ rebuilds the executable #5466

Comments

MisterDA commented Feb 21, 2022

bobot commented Feb 21, 2022 • edited Loading

MisterDA commented Feb 22, 2022

snowleopard commented Feb 22, 2022

MisterDA commented Feb 23, 2022

MisterDA commented Feb 23, 2022

MisterDA commented Feb 23, 2022

MisterDA commented Feb 28, 2022

snowleopard commented Mar 1, 2022

rgrinberg commented Apr 22, 2022

emillon commented May 18, 2022

emillon commented May 18, 2022

emillon commented May 18, 2022

Dune 3 `dune exec --` on Windows _always_ rebuilds the executable #5466

Dune 3 `dune exec --` on Windows _always_ rebuilds the executable #5466

bobot commented Feb 21, 2022 •

edited

Loading