alr install: Installation of binary crates (#1302)

* Installation of binary releases

* Basic prevention of conflicting installs

* Self-review and cleanup

* Add test and related fixes

* gprbuild tests wrt library placement and artifacts

.gitattributes

@@ -10,3 +10,4 @@
 # Mark some misidentified files as always binaries
 *.pdf -text
 *.png -text
+*.tgz -text

doc/AEPs/aep-0003.md

@@ -0,0 +1,237 @@
+    AEP: 3
+    Title: Command `alr install` information
+    Author: Alejandro R. Mosteo <[email protected]>
+    Status: Draft
+    Created: 2022-01-19
+
+Abstract
+========
+
+Information gathered about the operation of `gprinstall`, relevant to our
+purposes of how to work with static/dynamic libraries and with different
+versions of the same library/compilers.
+
+Information
+===========
+
+By using different names during installation, several versions of the same lib
+can be made to coexist, with some caveats.
+
+For example, after having installed libhello 1.0.0 (static) and libhello 1.0.1
+(dynamic) and hello 1.0.2 (dynamic), we obtain this tree:
+
+```
+prefix/
+├── bin
+│   └── hello
+├── include
+│   ├── hello=1.0.2
+│   │   └── hello
+│   │       ├── hello.adb
+│   │       └── hello_config.ads
+│   ├── libhello=1.0.0
+│   │   └── libhello
+│   │       ├── libhello.adb
+│   │       └── libhello.ads
+│   └── libhello=1.0.1
+│       └── libhello
+│           ├── libhello.adb
+│           ├── libhello.ads
+│           └── libhello_config.ads
+├── lib
+│   ├── libhello=1.0.0
+│   │   └── libhello
+│   │       ├── libhello.a
+│   │       └── libhello.ali
+│   ├── libhello=1.0.1
+│   │   └── libhello
+│   │       ├── libhello.ali
+│   │       ├── libhello_config.ali
+│   │       ├── Libhello.so.1.0.1
+│   │       └── libLibhello.so -> ../libhello/Libhello.so.1.0.1
+│   ├── Libhello.so.1.0.1 -> ../lib/libhello=1.0.1/libhello/Libhello.so.1.0.1
+│   └── libLibhello.so -> ../lib/libhello=1.0.1/libhello/libLibhello.so
+└── share
+    └── gpr
+        ├── hello.gpr
+        ├── libhello.gpr
+        └── manifests
+            ├── hello=1.0.2
+            ├── libhello=1.0.0
+            └── libhello=1.0.1
+```
+
+First caveat is that `gprinstall` clobbers `prefix/share/gpr/libhello.gpr`
+without warning, even if `-f` was not used. This means that an installation
+with the purpose of development cannot have several versions installed.
+
+Still, it seems we could rely on such a prefix for executables depending on
+different versions of the same dynamic library, as ldd shows the proper
+dependency:
+
+```
+$ ldd prefix/bin/hello
+	linux-vdso.so.1 (0x00007fffa772f000)
+	Libhello.so.1.0.1 => not found
+	libgnat-12.so => /path/to/compiler/.../libgnat-12.so (0x00007fda1ca0e000)
+	libgcc_s.so.1 => /path/to/compiler/.../libgcc_s.so.1 (0x00007fda1c7ef000)
+	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fda1c5c5000)
+	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fda1c5c0000)
+	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fda1c4d9000)
+	/lib64/ld-linux-x86-64.so.2 (0x00007fda1d0b4000)
+```
+
+By using `--mode=usage`, there is no ali, gpr or source files installed. So
+going with this mode, unneeded conflicting files are not even installed.
+
+Uninstalling two versions will remove the gpr file after the first uninstall.
+However, the second uninstall will not fail, silently removing the rest of
+files.
+
+For uninstallation we need to supply the project file, which can come from the
+original build folder or from the installed share/gpr location.
+
+However, when installing in usage mode, there will be no gpr file installed,
+forcing to preserve the original project file.
+
+In usage mode, static libraries are not installed, and no manifest is created
+if nothing gets installed. Uninstalling will then complain about lack of manifest.
+
+It seems that, at least for basic code, an executable build with a compiler and
+a dynamic library build with another are compatible.
+
+Executables depend on properly versioned `.so.x.x.x` files, so the extra `.so`
+file clobbered by several installs is not important.
+
+## Test to relocate lib files
+
+Running
+
+```
+$ rm -rf ../prefix/ ; alr exec -- gprinstall --install-name=hello=1.0.2 --link-lib-subdir=bin hello.gpr --prefix=../prefix -p --mode=usage -XLIBRARY_TYPE=relocatable -r
+$ tree ../prefix/ -F
+../prefix/
+├── bin/
+│   ├── hello*
+│   ├── Libhello.so.1.0.1 -> ../lib/hello=1.0.2/libhello/Libhello.so.1.0.1*
+│   └── libLibhello.so -> ../lib/hello=1.0.2/libhello/libLibhello.so*
+├── lib/
+│   └── hello=1.0.2/
+│       └── libhello/
+│           ├── Libhello.so.1.0.1*
+│           └── libLibhello.so -> ../libhello/Libhello.so.1.0.1*
+└── share/
+    └── gpr/
+        └── manifests/
+            └── hello=1.0.2
+```
+
+we can see that the softlinks are placed with the binaries. However, this does
+not really help as
+
+```
+PATH+=:../prefix/bin hello
+hello: error while loading shared libraries: Libhello.so.1.0.1: cannot open shared object file: No such file or directory
+```
+
+still fails (going into the `bin` directory and launching from there works).
+
+Probably on Windows it will do help, as DLLs are looked for in the PATH.
+
+For Linux, we could instruct users to set `LD_LIBRARY_PATH`, or perhaps add an
+`alr install --printenv` that can be added to `.bashrc`.
+
+## Test of artifacts
+
+The following sequence shows artifacts being placed in the expected location:
+
+```
+$ alr init --bin hello
+$ cd hello
+$ touch share/hello/hello-artifact
+$ alr build
+$ rm -rf ../prefix/ ; alr exec -- gprinstall --install-name=hello=0.1.0-dev --link-lib-subdir=bin hello.gpr --prefix=../prefix -p --mode=usage -XLIBRARY_TYPE=relocatable -r
+$ tree ../prefix/ -F
+../prefix/
+├── bin/
+│   └── hello*
+└── share/
+    ├── gpr/
+    │   └── manifests/
+    │       └── hello=0.1.0-dev
+    └── hello/
+        └── hello-artifact
+```
+
+Summary of findings
+===================
+
+- Several dynamic versions of a library are possible, for executables.
+  - Development with several versions is not possible.
+  - `LD_LIBRARY_PATH` seems necessary in any case
+- Dependencies on libraries from the compiler also appear.
+  - It would then be better to use a compiler from within the prefix.
+- Uninstall of several versions requires preserving the original project file.
+  - When uninstalling only one version, the installed gpr file suffices.
+- Compiler consistency between dynamic libraries is not mandatory.
+- Executables link against the properly versioned dynamic library.
+
+Proposal
+========
+
+Given these findings, and the primary need of using `alr install` to make binaries
+available, and not to make development libraries available, we can abandon this
+latter notion for good. Sharing of large dependencies, if ever implemented will use
+a different mechanism.
+
+For installing executables, each installation can be performed on its own: we need
+not track crates (as we can check the gprinstall manifests) nor consider
+incompatibilities, as there are none. Worst case, two releases from the same crate
+would be detected by their manifests, and two different crates clobbering each other
+would be detected during `gprinstall`.
+
+For uninstallation, we may redeploy sources to have access to the original project
+files.
+
+For (un)installation of local crates, there's also no issue: they will use the local
+version, and we have the gpr file available too for both.
+
+We could consider caching build directories for faster installs of related crates
+sharing several dependencies, and faster uninstallation. This would incur some disk
+usage penalty so we may want to make this optional.
+
+We may want to track dependencies to prevent uninstallation of libraries which are
+depended upon. This would be a final enhancement. Uninstallation doesn't seem to be
+a pressing matter, as prefixes will be fast to recreate.
+
+Tracking of dependencies isn't trivial as we can't use a single root (there will be
+"incompatible" (in solution sense) crates installed quite often, if only because of
+forced compilers).
+
+Roadmap
+=======
+
+Wanted and simple to implement:
+
+- Installation of binary releases
+   - Un-uninstallable if there is no project file
+- Installation of local releases
+- Installation of indexed releases
+
+Optional or low priority:
+
+- Uninstallation relying on project file (local/indexed releases)
+- Uninstallation of binary releases without a project file
+   - Using our own-created manifest during installation
+- Tracking of dependencies
+   - To prevent uninstalls with dependents
+   - Or to also uninstall dependents
+
+Out of scope:
+
+- Installation for development (users can manually do this via alr exec -- gprinstall)
+
+Copyright
+=========
+
+This document has been placed in the public domain.

doc/user-changes.md

@@ -6,6 +6,51 @@ stay on top of `alr` new features.
 
 ## Release 1.3-dev
 
+### New subcommand `alr install
+
+PR [#1302](https://github.com/alire-project/alire/pull/1302)
+
+A new subcommand `alr install` allows the installation of binaries to a location
+intended to be added to the user's PATH. The default install location is
+`$HOME/.alire`, with binaries going into `$HOME/.alire/bin`.
+
+This is a experimental feature that will see improvements and tweaks in further
+PRs and as we gather feedback on its usage.
+
+At present, only binary releases can be installed (e.g., compilers, `gprbuild`,
+`gnatprove`, `gnatstudio`). There is no ability to uninstall releases either
+(but reinstallation can be forced).
+
+Only one version per executable can be installed, meaning that, for example,
+only one toolchain version can exist in an installation prefix. So, this
+feature is intended to make the user's preferred version of a crate generally
+available in the system for use outside of Alire, but not to replace e.g. the
+ability of Alire to use several compilers, or to reuse compiled libraries as
+dependencies in several workspaces.
+
+Examples:
+
+```
+$ alr install gnatprove
+ⓘ Installing gnatprove=12.1.1...
+ⓘ Installation complete.
+
+$ alr install
+Installation prefix found at /home/jano/.alire
+Contents:
+   gnatprove=12.1.1
+
+$ PATH+=:$HOME/.alire/bin gnatprove --version
+Usage: gnatprove -Pproj [switches] [-cargs switches]
+...
+
+$ alr install gnatprove^11
+error: Requested release gnatprove=11.2.3 has another version already
+installed: gnatprove=12.1.1. (This error can be overridden with --force.)
+
+$ alr --force install gnatprove^11  # Downgrade installation
+```
+
 ### Find dependents of a release with `alr show --dependents
 
 PR [#1170](https://github.com/alire-project/alire/pull/1170)

scripts/alr-completion.bash

@@ -46,7 +46,7 @@ function _alr_completion() {
     # Command-specific completions
     $found &&\
     case $cmd in
-        get | show | toolchain)
+        get | install | show | toolchain)
             # Suggest crate names
             COMPREPLY+=($(compgen -W "$_alr_crates" -- $curr))
             ;;