Alias support

[danhper]

A discussion about aliases has started in asdf-vm/asdf-elixir#63.
As this is not directly related to asdf-elixir but rather to asdf, let's have the discussion here.

I personally only use "regular" versions so I do not have a strong opinion about this but I get the point.

I am not exactly sure how what the --alias flag is about though?
Could somebody please propose what he thinks would be a good way for asdf to handle aliases?

[danhper]

Originally posted by @bitwalker in asdf-vm/asdf-elixir#63 (comment)

@bitwalker the goal of asdf is to make version management simple, that usually results in it being fairly easy to use too, but being easy to use is a side affect of being simple and well designed.

I'm definitely aware of the correspondence between simple/easy, my point was that as a user, those qualities mean different things than they do as a maintainer. Specifically, with my user hat on the "ease" in which I can avoid using multiple tools to do one thing (version management), and the "simplicity" of something that doesn't use any magic, just simple OS primtives, are the reasons why I chose it.

It is fundamental.

Part of being simple is only doing one thing, and for asdf that one thing is extendable version management. Adding support for an --alias flag would add a lot of complexity to the code for a feature isn't central to the purpose of asdf.

I'm not convinced that it is as complex as you expect. Supporting options with getopt would be a simple solution, and would vibe with using OS primitives as done so far, and certainly supports the basic functionality needed for the --alias flag.

There are plenty of other tools for creating and managing aliases, these can be used in combination with asdf if users have the desire. asdf provides a good API, there is nothing preventing you from writing a shell script to make it do what you want.

I will of course have to investigate to see if there are limitations here, but if that's the case, then if I need to maintain my own shell script to add this capability, I'm not opposed to that idea, but it does seem rather fundamental to asdf itself. I'm a firm supporter of providing necessary primitives and letting the community go from there though, so as long as everything is present that would be required to support my proposal, then I'm good.

Do you have suggestions for tools for alias management? The shell is plainly inadequate for this out of the box, as creating arbitrarily-long lived aliases and destroying them on the fly is not something you get for free.

Adding an --alias flag would fundamental change what a version in asdf actual is. Currently a version is exactly what it says it is. If you specify elixir 1.7.1, you can be sure you'll get the 1.7.1 release of Elixir, if you use elixir ref:abcdef1, you can be assured you will get that ref of Elixir. Using aliases would mean .tool-versions files would not contain versions, they'd contain identifiers which may be literal versions or may be aliases.

I think there are two usage patterns, distinct from one another: First is the use you are talking about, where users are installing versions and saving .tool-versions for use in a team environment, where the predictability/stability of the version identifiers is important. Secondly, you have the use of ad-hoc installs for short periods of time, where the .tool-versions is not shared outside of the local machine (i.e. it is not committed to a project). These latter uses are very ephemeral, and are often switched to and from frequently, which is why the aliases would be so pleasant.

It would be absolutely possible for the .tool-versions to have the full path:<path> representation stored in it, rather than the alias, and it would be trivial to resolve whether something is a local alias or a "proper" version. This would be able to catch potentially unsupported/undesired use cases that you are concerned about. In practice though, I doubt this is an issue that warrants much concern; I certainly never use MAJ.MIN.BUILD versions for ad-hoc installs from a local build directory, or even git refs, they get names for their purpose, like otp-init-experiment and such. I suspect that is true of anyone who would be using the aliases functionality.

There is also the issue with aliases conflicting with actual versions. So I could create an alias named 1.7.1 and point it at my local 1.7.1 build directory. If I later changed the version of Elixir in that directory all projects that specified 1.7.1 would not be using 1.7.1.

Again, I think it is absolutely possible to deal with this edge case in a sane way, probably by warning when the .tool-versions is loading a version from a an aliased path, not an asdf-managed build, thus making it crystal clear when things have been changed out from under you.

Without knowing what all the future versions will be there is no way to prevent conflicts from aliases and literal versions.

This is really the responsibility of the users managing aliases though, just like it would be if they implemented this functionality manually. The benefit of having it in asdf is that these edge cases can be dealth with sanely and given friendly UX so that even those who think they know what they are doing, can avoid accidentally making a mistake like you've outlined. In practice I don't think this is likely to occur, but others that want aliases may have different usage patterns than me.

The asdf global and asdf local commands should only be run when you need to change the version, so you shouldn't be running them very often.

I use them very frequently when switching between builds with different things I'm testing. Granted, I'm a bit of a special case as a open source maintainer with a lot of projects with a large matrix of languages and versions to work across - but that is precisely why I use asdf. Maybe I'm expecting too much trying to solve for my use case, but I know I'm not the only one in this position.

@whatyouhide I must be misunderstanding you. You won't type asdf local elixir path:/path/to/elixir but you will type ln -s /path/to/elixir ~/.asdf/installs/elixir/master && asdf reshim elixir master? How is that easier? Either way you have to specify the full path.

He does that once, and then uses the alias from then onward, which is the point, he only needs to type the verbose command the one time.

We can move this discussion to an issue if you'd like, perhaps it would be better to do that so that you can point other users there one way or the other. I don't want to pressure you on this, if you still aren't convinced, then I will find a solution one way or the other, but I had hoped we might find consensus of some kind.

[bitwalker]

Specifically, my suggestion was that the --alias flag would be for asdf install <lang> <path|ref>:<value> [--alias=<name>]. For other forms of asdf install <lang> <version>, --alias would produce a warning/error (my opinion is that it would make sense to not support aliasing non path/ref versions). After install the alias can be used with other commands which expect a version.

The alias would ultimately just be a convenience for the CLI, under the covers the alias is always resolved to the concrete path/ref, and treated normally. Aliases would need to be stored in their own metadata file under .asdf, that way no other changes need be made to other asdf infrastructure, the aliases are always read first, and resolved to their concrete values, before handling of subcommands proceeds.

This does mean (I think) that one cannot edit .tool-versions and use aliases there, but it is certainly not something I would have any use for, and I suspect could be mentioned as a caveat in the documentation (i.e. .tool-versions always requires concrete path:<path> or ref:<ref> versions)

As for handling the --aliases flag specifically, we could use getopt, or even manual parsing, since it is the only flag supported for install, but using getopt would provide an easier path to implementation and is easy to extend later.

The main issues raised by @Stratus3D were:

1. The potential for adding a lot of complexity to asdf
2. The problem of creating aliases with the same name as a "real" version, which then pollutes other projects that use the "real" version in their .tool-versions
3. The problem of storing aliases in .tool-versions which then get shared with a team (i.e. they can't be resolved)

I don't think 1 is too much of an issue, as the implementation is a very thin layer that sits in the command processing bits to resolve an alias to a concrete version (if an alias is defined). It is not necessary to do any extra work in the subcommands as far as I can see.

I believe I've addressed both 2 and 3 in my specification of behavior above. In short, aliases are never used except at the CLI, the versions stored in .tool-versions are always concrete, so it is not posible to have conflicts with aliases, as aliases don't even exist at that point.

If I've missed something in this design, definitely let me know!

[danhper]

@bitwalker Thanks for the detailed explanation.
I think this would be a reasonable addition. Not polluting ~/.asdf/installs/PLUGIN/ is I think desirable, so I like this approach.
If we were to implement this, I suppose the required changes would be

1. alias metadata file reading/writing
2. resolving alias when asdf is passed a version
3. updating install to handle --alias
4. updating uninstall to remove aliases? (not really sure anyone uses uninstall with path:, I don't even know if it works)

Point 4 makes me think that I am not sure I like the idea of having --alias as part of the install command.
I think something like asdf alias-add and asdf alias-remove (and maybe asdf alias-list) would follow the current way we manage commands better.

Overall I don't think this would add too much complexity so if we can settle on a design I would be happy to accept a PR.

@asdf-vm/core What are your thoughts about this?

[bitwalker]

RE: 4, my intuition around asdf uninstall <lang> [path:<path>|<alias_for_path>] is that it simply unlinks the symlink created by "install" (rather than deletion, as you obviously don't want that), and in the case of an alias, would also remove the alias. It doesn't make much sense to leave the alias behind, nor does it make sense to just remove the alias.

I like the idea of having porcelain commands for alias management, but I would still support asdf uninstall <lang> <alias> as a convenience.

EDIT: As an extension of my proposal, I'd suggest something like the following for the alias metadata:

# Format: <lang>|<alias>|<concrete path/ref/version>
elixir|dev|path:/absolute/path/to/my/elixir/repo

Which can be trivially processed like so:

local aliases="$(grep -v '#.*' "${aliases_path}")"
local lang="$(echo "$aliases" | cut -d'|' -f1)"
local name="$(echo "$aliases" | cut -d'|' -f2)"
local concrete_ref="$(echo "$aliases" | cut -d'|' -f3)"

[Stratus3D]

I'm still not convinced this needs to be part of asdf. I could create a 20 line shell script that does everything you described (and it would work for anything, not just asdf), and it would be simpler than making these changes to asdf itself to support this. Why add complexity? Just create (or use an existing tool) that does this.

My two main objections here are:

• Added complexity. This new feature WILL add complexity. We already have an API that does what you need, it's just a little verbose for your use case.
• I don't want to be responsible for this feature. I don't want to deal with bug reports when it breaks. As it is right now I work on asdf for probably an average of 30 minutes a day every day of the week, and even with the amount of time I'm putting in it takes us weeks to get around to fixing known bugs with existing features. The last thing I want to be responsible for is yet another implementation of an alias utility that's tied to some asdf specific features. Talk is cheap, and few are willing to put in the time to keep things running smoothly, so most things here fall to me or @danhper (not blaming you, just point out the reality of things here).

I'm not denying this would be a useful to you, it no doubt would be. But for the most users it would not be useful, and would create unnecessary clutter in the code and docs. I estimate only about 1-5% of asdf users use path: feature right now. I have never had a need for something like this and have never used the path: feature outside of testing. I'm more than happy to create a separate script for you do to everything you need to do, but I do not want to be responsible for the maintenance of it, and I don't want to see it baked into asdf.

[Stratus3D]

Related: #499

[Stratus3D]

https://github.com/ytbryan/aka

[bitwalker]

@Stratus3D I'm willing to work on asdf and in particular be responsible for this feature. I'm certainly familiar with the burden of maintaining open source projects. If that's the major obstacle here, I'm certainly willing to put my money where my mouth is.

I'm not denying that I could write this feature for myself in shell, but that's fundamentally true of basically the entirety of what I use asdf for. I used to have my own shell functions that did something very similar to asdf-elixir and asdf-erlang (the installation/build parts anyway). I ultimately ditched those in favor of supporting an open source tool though so that I could benefit from the contributions of the community (and contribute back when possible).

The argument that one should just do it themselves also kind of ignores the possibility that a lot of people using asdf probably do so because they don't know shell particularly well and just want a tool that solves these problems for them. It may be true that most users don't use path:<path> or ref:<hash>, but how much of that is potentially due to both formats getting a single mention in the .tool-versions config doc? Neither is mentioned in the documentation for install when I looked. Even if a user becomes aware of them, if they aren't particularly comfortable with shell, asking them to write their own scripts to make their usage ergonomic is not going to be a viable solution - they'll just consider it an annoying corner of the tool and try to avoid it.

Why add complexity? Just create (or use an existing tool) that does this.

I choose tools with the hope that they (ideally) encapsulate the complexity inherent in the task for which they are used. The feature proposed here is fundamentally a solution to a problem with the UX of one part of asdf, one for which the obvious first step, a shell alias, is insufficient. I personally use shell for a lot of things, and have a large number of aliases, including making use of the aliasing functionality inherent in other tools I use (such as git aliases). If shell aliases were an appropriate solution, I wouldn't be pressing the issue.

If my willingness to step in and maintain the feature is insufficient, then I'm willing to accept that this won't make it in to asdf, but so far the strongest argument I've heard is simply that nobody wants to maintain it - that's a reasonable argument, but it's not an argument on technical merits, just lack of desire to work on it, or around it.

https://github.com/ytbryan/aka

I hope you realize that a tool like that is an absurd level of overkill for the problem being solved in this proposal. Aside from the fact that it requires a Ruby runtime, it doesn't provide a way for one to run asdf local <lang> <alias>, it just provides you a way to define something like asdf_<lang>_<alias>="asdf local <lang> <alias>", which is no different than a shell alias. The closest thing to a solution I can whip up on the spot is this:

function asdf(){
    if [ -z "$1" ]; then
        command asdf
    fi
    case "$1" in
        install)
            lang="$2"
            concrete_ver="$3"
            args=`getopt a: $*`
            set -- $args
            for i do
                case "$i" in
                    -a)
                        named_ver="$2"; shift; shift;
                        echo "$lang|$named_ver|$concrete_ver" >> ~/.asdf_aliases
                        ;;
                     --)
                        shift; break
                        ;;
                 esac
             done
             command asdf install "$lang" "$concrete_ver"
             ;;
         local)
             command asdf local "$2" "$(lookup_alias "$3")"
             ;;
         *)
             # ... other commands here
             ;;
     esac
}

I'm not even sure the above works correctly, and leaves lookup_alias unimplemented, but the point is that it at least appears to solve the problem. If this is the level of complexity you are concerned about, then that surprises me.

[danhper]

@bitwalker Thanks a lot for offering to help.

I am starting to think this might be a good opportunity to give "non-language plugins" a try. #367

I understand this feature can be useful but I think the number of people needing it is probably fairly low (I don't have data to support this so I might be wrong).
This is why this would be perfect as a plugin but we do not support such plugins yet.
I think it would be worth giving it a try.

@Stratus3D @bitwalker What do you think?

[bitwalker]

I think it definitely makes sense being a plugin if the necessary hooks are there, I'll have to read up on whats planned there.

[danhper]

I don't think we have given much thought into what hooks would be necessary for the plugins yet, so I think this could be a good opportunity to do so.

[bitwalker]

I'll chime in over in that thread once I've thought about it for a bit, I have some initial thoughts, but want to check out some assumptions first.

[andrewthauer]

For anyone interested, I created a standalone shell script called asdf-alias that creates aliases via symlinks and then reshims the plugin (required to work).

It is based on nodenv-aliases which is handy to avoid installing every minor version of a runtime. I've been using it for a little while and seems to be working fine in most cases. That said, it's not integrated into asdf so uninstalling a version won't remove the alias (although the script will remove invalid symlinks when run).

I could see something like this integrated into asdf in a couple of different ways:

1. Introducing a new asdf alias core sub command which essentially does what the script does now (or some derivative). The asdf uninstall command could do a post cleanup step to remove invalid symlink alias versions (like the script does). I'm guessing this could be done without breaking plugins. I haven't thought about what other hook support might be required though.

2. Introduce a generic external sub command mechanism to asdf. This pattern is currently used by many CLI tools such as git, rbenv, etc (e.g. asdf alias executes asdf-alias if the command exists in the path). This option does not require additional modifications to core and could provide the basis for a simple non language plugin pattern as well.

If there is interest in either option, I'd be willing to attempt a PR.

[vic]

@andrewthauer Latest asdf release lets plugins enhance the CLI with additional commands (see extension-commands)

So for example, if you have an asdf-alias plugin and install it via

asdf plugin add alias https://github.com/andrewthauer/asdf-alias.git

And it contains a bin/command script, it will be executable via the CLI as asdf alias

[andrewthauer]

Thanks for pointing that out @vic. I didn't realize that extension commands had been added.

I was able to easily port over my asdf-alias script into a basic plugin using the extension feature. It works well, but still does not have any knowledge of when versions are uninstalled though.

So, the bigger question would be, does alias support belong as a core asdf feature or delegated to a plugin? I personally think it could be a common enough feature to belong as a builtin feature/command.

[andrewthauer]

FYI - I created a plugin named asdf-alias. Also created issue #641, to point out some thoughts on extension command only plugins.

[gsevla]

Don't know I'm doing something wrong @andrewthauer, but I can't use your plugin because bin are missing. So I installed asdf-alias, but can't install an alias version and so use it.

[andrewthauer]

Yeah. I think something either changed in the underlying core API, or I read the docs wrong. I’ve been meaning to fix it, but got sidetracked. I’ll try take a stab at getting it fixed and published properly.

In the mean time you could likely take the script code and run the command directly without any changes.

[jthegedus]

@andrewthauer If there's improvements to the documentation, please open another issue to share your specific thoughts and tag me 😄 I've got a WIP rewrite of the "creating plugins" section I have been meaning to finish 😅 Plugin author feedback is not only welcome but required.

[andrewthauer]

@jthegedus - I just debugged through the code of the bin/asdf command and I can't see how custom subcommands like asdf alias would ever work with the current code. The find_cmd function always searches for custom commands in the /path/to/asdf/lib/commands directory. I could have sworn this used to work, but looking through the git history I don't see anything that sticks out. I also don't see any test fixtures that would cover the subcommand feature like this. I guess I should probably open up a new issue?

[jthegedus]

@andrewthauer yes, sounds like another issue for us to tackle

[andrewthauer]

@jthegedus. Sorry I'm actually wrong, it does look for the plugin commands. I've tracked down the problem. The find_plugin_cmd function checks for the existence of a bin directory in the plugin (https://github.com/asdf-vm/asdf/blob/master/bin/asdf#L51).

In my case, I don't' have this directory, so it doesn't bother to look for the sub command. If I add a dummy bin directory it works. I think this is probably not desired behaviour, so I'll open an issue about it. Assuming this is fixed so the bin directory isn't required, then the docs would likely be correct.

[augustobmoura]

What about having an auto-generated .tool-versions.lock? Similar to package-lock.json and Gem.lock, .tool-versions would then have a non-specific version name and the lock would have the specific resolved version, we could expand this idea to store additional metadata for validation, like checksums and source URLs. Something like a file with:

plugin-name.version=1.4.5
plugin-name.checksum=$hash
plugin-name.tarball-url=https://example.com

The user could enable this feature at a plugin level or with a global setting, similar to legacy_version_file. There are some cool possibilities with this approach. Checksum and arbitrary verifications like source urls are my favorites

A common user could bump the specific version by running something like: asdf install nodejs --update-lock. We can bikeshed on the command though