Rebar3 downloads all dependencies in rebar.lock, even the disabled ones

[badlop]

Context

In rebar3, dependencies are defined in rebar.config

Since commits 1a63443 and 0407c56, we can define the minimum required dependency version, and allow rebar3 to download any newer version.

The rebar.lock file is used to indicate what exact dependency versions are used:

• If the file does not exist, rebar3 downloads the more suitable versions and then generates the file.
• If the file exists, rebar3 downloads exactly the versions indicated in it.

This file is tracked in ejabberd git repository since da8c9f3 because rebar3 documentation recommends it:

[Lock files] should always be checked into source control.

Problem

However, now that rebar.lock is tracked in git, there is a concatenation of events that lead to a problem:

1. configure.ac may disable some features (probably because the feature is not needed, or the libraries required to compile that feature are not available in the system). This decision is stored in vars.config. Those dependencies should not be downloaded, compiled, installed.

2. rebar.config disables those dependencies (thanks to rebar.config.script and vars.config)

3. rebar.lock still mentions those dependencies, because the file is tracked in git

4. rebar3 will read rebar.config and rebar.lock, and will download and attempt to compile all the dependencies, even if some dependencies are only mentioned in rebar.lock, not in rebar.config

Those downloads are useless, and in fact attempting to compile the disabled dependencies may fail.

Interestingly, Mix does not behave that way: Mix only downloads the dependencies enabled in mix.exs, and uses mix.lock only to determine what exact version to download.

Solutions

Three solutions I can see, from easier to harder:

• A) Don't track rebar.lock in git. This immediately solves this problem. And tracking the file, until now does not seem to be specially required
• B) In configure.ac, when some feature is disabled, in addition to store this decision in vars.config, unlock the corresponding dependency in rebar.lock
• C) Modify rebar3 (or at least fill an issue in its git) to behave similarly to Mix: download only the dependencies enabled in rebar.config, and use rebar.lock only to determine what exact version to download

Solution B)

Option B) can be implemented with a patch like this:

diff --git a/configure.ac b/configure.ac
index a128e6878..01afe6dcc 100644
--- a/configure.ac
+++ b/configure.ac
@@ -333,3 +333,19 @@ AC_SUBST(CPPFLAGS)
 AC_SUBST(LDFLAGS)
 
 AC_OUTPUT
+
+AS_CASE([$rebar],
+ [*rebar3], [
+  deps=""
+  AS_IF([test "x$stun" = "xfalse"], [deps="stun,$deps"])
+  AS_IF([test "x$sqlite" = "xfalse"], [deps="sqlite3,$deps"])
+  AS_IF([test "x$pgsql" = "xfalse"], [deps="p1_pgsql,$deps"])
+  AS_IF([test "x$mysql" = "xfalse"], [deps="p1_mysql,$deps"])
+  AS_IF([test "x$zlib" = "xfalse"], [deps="ezlib,$deps"])
+  AS_IF([test "x$sip" = "xfalse"], [deps="esip,$deps"])
+  AS_IF([test "x$redis" = "xfalse"], [deps="eredis,$deps"])
+  AS_IF([test "x$pam" = "xfalse"], [deps="epam,$deps"])
+  AS_IF([test "x$deps" = "x"], [],
+   [echo "unlocking disabled rebar3 dependencies: $deps"
+    $rebar unlock "$deps"])
+ ])

For example, let's enable all the features but disable PAM, then epam will get unlocked for rebar3, and will not be downloaded:

$ ./autogen.sh ; ./configure --enable-all --disable-pam --with-rebar=rebar3
...
build tool to use (change using --with-rebar): rebar3
configure: creating ./config.status
config.status: creating Makefile
config.status: creating vars.config
unlocking disabled rebar3 dependencies: epam,
===> Analyzing applications...
===> Compiling configure_deps

$ git diff rebar.lock
diff --git a/rebar.lock b/rebar.lock
index a1cc16216..601a0fd9a 100644
--- a/rebar.lock
+++ b/rebar.lock
@@ -2,7 +2,6 @@
 [{<<"base64url">>,{pkg,<<"base64url">>,<<"1.0.1">>},1},
  {<<"cache_tab">>,{pkg,<<"cache_tab">>,<<"1.0.30">>},0},
  {<<"eimp">>,{pkg,<<"eimp">>,<<"1.0.22">>},0},
- {<<"epam">>,{pkg,<<"epam">>,<<"1.0.14">>},0},
  {<<"eredis">>,{pkg,<<"eredis">>,<<"1.2.0">>},0},
  {<<"esip">>,{pkg,<<"esip">>,<<"1.0.52">>},0},
  {<<"ezlib">>,{pkg,<<"ezlib">>,<<"1.0.12">>},0},
@@ -34,7 +33,6 @@
  {<<"base64url">>, <<"F8C7F2DA04CA9A5D0F5F50258F055E1D699F0E8BF4CFDB30B750865368403CF6">>},
  {<<"cache_tab">>, <<"6D35EECFB65FBE5FC85988503A27338D32DE01243F3FC8EA3EE7161AF08725A4">>},
  {<<"eimp">>, <<"FA9B376EF0B50E8455DB15C7C11DEA4522C6902E04412288AAB436D26335F6EB">>},
- {<<"epam">>, <<"AA0B85D27F4EF3A756AE995179DF952A0721237E83C6B79D644347B75016681A">>},
  {<<"eredis">>, <<"0B8E9CFC2C00FA1374CD107EA63B49BE08D933DF2CF175E6A89B73DD9C380DE4">>},
  {<<"esip">>, <<"A2840287C493A4280E6FBA57A257706843B025C315875E38B03FD07190E22DBA">>},
  {<<"ezlib">>, <<"FFE906BA10D03AAEE7977E1E0E81D9FFC3BB8B47FB9CD8E2E453507A2E56221F">>},
@@ -61,7 +59,6 @@
  {<<"base64url">>, <<"F9B3ADD4731A02A9B0410398B475B33E7566A695365237A6BDEE1BB447719F5C">>},
  {<<"cache_tab">>, <<"6D8A5E00D8F84C42627706A6DBEDB02E34D58495F3ED61935C8475CA0531CDA0">>},
  {<<"eimp">>, <<"B3B9FFB1D9A5F4A2BA88AC418A819164932D9A9D3A2FC3D32CA338CE855C4392">>},
- {<<"epam">>, <<"2F3449E72885A72A6C2A843F561ADD0FC2F70D7A21F61456930A547473D4D989">>},
  {<<"eredis">>, <<"D9B5ABEF2C2C8ABA8F32AA018203E0B3DC8B1157773B254AB1D4C2002317F1E1">>},
  {<<"esip">>, <<"6F00165395900500AA262CE0297162D93931C78C1464D89FD0EDC6E3D6BC011F">>},
  {<<"ezlib">>, <<"30E94355FB42260AAB6E12582CB0C56BF233515E655C8AEAF48760E7561E4EBB">>},

$ make
...

$ ls -1 _build/default/lib/ | grep pam | wc -l
0