flatpak-builder sometimes set up incorrectly submodules worktree

[alexlarsson]

From @mildred on October 18, 2016 8:9

I have the same setup on two different computers, and one is not working as expected. I'm trying to package an electron-based app and I am inspiring myself from https://github.com/vrutkovs/flatpak-electron (I made almost no modifications to the json file and the Makefile patch)

The build command I have is : flatpak-builder --repo=repo --subject="Patchwork Electron" --verbose app com.github.electron.json

At the moment the make command is called from flatpak-builder, I have the repositories that are incorrectly configured. There is a recursive submodule that points to a git repository outside of /run. Running git commands in this repository gives mixed results. For example I have:

+ cd /run/build/electron-gtk3/electron/vendor/brightray/vendor/libchromiumcontent
+ python script/bootstrap -v
fatal: /usr/lib/git/git-core/git-submodule cannot be used without a working tree.

I debugged this, and could find that in that directory, git rev-parse --is-inside-work-tree results in false instead of true.

I could trace it back to the fact that in this directory, the .git says the repository is outside /run:

$ cat /run/build/electron-gtk3/electron/vendor/brightray/vendor/libchromiumcontent/.git
gitdir: /home/mildred/Projects/patchwork-electron/.flatpak-builder/build/electron-gtk3-2/.git/modules/electron/modules/vendor/brightray/modules/vendor/libchromiumcontent

And the repository config contains a relative directory to its worktree (because it's a submodule, this is normal) which resolves outside /run when joined with the repository real path:

$ cd /run/build/electron-gtk3/electron/vendor/brightray/vendor/libchromiumcontent
$ git config core.worktree
../../../../../../../../../electron/vendor/brightray/vendor/libchromiumcontent
$ git rev-parse --git-path $(git config core.worktree)
/home/mildred/Projects/patchwork-electron/.flatpak-builder/build/electron-gtk3-2/.git/modules/electron/modules/vendor/brightray/modules/vendor/libchromiumcontent/../../../../../../../../../electron/vendor/brightray/vendor/libchromiumcontent

I tried to remove the flatpak repository and the hidden .flatpak-builder directory, but it is always the same. On another computer with the same setup, I have no problem.

I have flatpak version 0.6.11 on both on Arch and Fedora

edit : Removing the submodule working directories and running git submodule update in the Makefile is a workaround. No network access should be required for that (the git repository is already there) but the worktree is recreated anew and correctly.

Copied from original issue: flatpak/flatpak#353

[alexlarsson]

This:

/home/mildred/Projects/patchwork-electron/.flatpak-builder/build/electron-gtk3-2/.git/modules/electron/modules/vendor/brightray/modules/vendor/libchromiumcontent/../../../../../../../../../electron/vendor/brightray/vendor/libchromiumcontent

Resolves to:

/home/mildred/Projects/patchwork-electron/.flatpak-builder/build/electron-gtk3-2/electron/vendor/brightray/vendor/libchromiumcontent

This is inside the build directory of the module, and it is mounted in the build. It is also bind-mounted in /run/build/ in order to produce builds that are more likely to be independent on the source location during the build. So, this should work.

However, there is a detail here. What actually gets mounted is realpath $source_dir because that is what we can bind-mount. So, probably what happens is that on one machine the prefix of your sourcepath contains a symlink, which causes things to go pear-shaped. I'll have a look at fixing this.

[alexlarsson]

I believe that should fix it.

[alexlarsson]

From @mildred on October 18, 2016 10:16

However, there is a detail here. What actually gets mounted is realpath $source_dir because that is what we can bind-mount. So, probably what happens is that on one machine the prefix of your sourcepath contains a symlink, which causes things to go pear-shaped. I'll have a look at fixing this.

By source dir, you mean the directory containing the sources? /home/mildred/Projects/patchwork-electron in my case. If so, on both machines, it is free of symlinks (pwd and realpath $PWD are the same)

[alexlarsson]

Hmm, I don't see how it shouldn't work then? Can you run "cat /proc/self/mounts" inside the build environment and see if /home/mildred/Projects/patchwork-electron is mounted?

[alexlarsson]

Did you ever figure out the exact difference between the two machines that made it not work?

[proletarius101]

Unfortunally I also encounter this when packaging standardnotes: flatpak/flatpak-builder-tools#182