System-wide installation of packages

[svillemot]

AIUI, the Julia package manager currently allows only per-user installation of packages.

From a packager perspective (I am a Debian developer), the possibility of having both system-wide and per user packages is very important.

In Debian, we already distribute large parts of Python's PIP, Perl's CPAN, R's CPAN, Octave Forge,… through the package manager.

I can give the following reasons for doing the same for Julia packages (the description below mentions Debian at several places, but you can replace it by virtually any GNU/Linux distributor):

• uniformity of experience: a Debian user expects to get all its software through the Debian package manager (APT). Having to learn various package managers (Python's, Ruby's, Perl's, Octave's, R's,
  Emacs', Julia's…) seems redundant and a waste of time
• getting dependencies easily: if a Julia package relies on some shared library, then installing it through APT would automatically pull the library; Julia's package manager obviously cannot do that
• quality assurance and security: a Debian user typically trusts Debian Developpers to make various checks on the quality and security of a package before putting it in the Debian archive, so he will be more confident on the quality of a package that he gets through APT than through another mean; also, downloads from the Debian archive are GPG-signed, which gives the user more confidence into the fact the he downloaded an untampered piece of code
• scalability: the Debian package manager makes it much easier to deploy large installations with either many users on a single machine or many similar machines (like in large educational centers or scientific organizations). The sysadmin will clearly prefer to install an extra Debian package (since this already fits into its workflow) than to install Julia packages in the home of every user. Duplicating installations of Julia packages is also a waste of storage space, and creates a greater potential for heterogeneous installs

Obviously the goal is not to package all Julia packages into Debian. Several criteria such as popularity, usefulness, maturity and stability will matter when making the decision to create a Debian package.

Of course, the system needs to be designed in such a way that the per-user and system-wide repositories co-exist nicely. For example, if a package is installed in both locations, the per-user should probably take precedence (when executing a "Using" or when resolving dependencies).

In designing the system, it is important to recall that from user perspective, the system-wide location is read-only.

[svillemot]

I realize that this has already been a little discussed on the julia-dev list.

Actually I don't want to enforce any particular technical solution. The only important thing to me is to be able to distribute Julia packages through Debian's APT. I think the only two minimal requirements are:

• to be able to change the location from ~/.julia to something in /usr
• to have the system well-behave on read-only directories

Of course, if the system can accomodate two locations (one per-user and one system-wide), it is even better.

Also, if there is a list of "official" packages validated by the Julia dev team, then all these packages can go into a single Debian package instead of multiple ones. This is still to be determined.

[ihnorton]

+1. @stevengj made several of these points in a recent thread on julia-dev: https://groups.google.com/d/topic/julia-dev/Z6GsLURuIZE/discussion

I think your shared library point is particularly important. I want to be able to type "apt-get julia-vtk" or "julia-qt" or "julia-opencv" (those will surely come) and have it just work. These are big libraries with many dependencies; they are also tremendously useful. If Julia does not play nice with distro package managers, then PKG.jl or individual pkgs are liable to develop ad-hoc, bug-ridden, underspecified interfaces to apt, homebrew, yum, etc. In order to install libraries, Pkg or an individual pkg will need to make installation requests, and then resolve the correct version of Julia code to pull in based on what is locally available. That seems like a bad recipe for adoption because it makes life more difficult for both module developers and sysadmins. Why not let distro packagers do that work instead?

[svillemot]

@ihnorton Thanks for pointing me to that thread. I had not yet read it, my email processor is lagging :)

So, to clarify my position in light of that thread, I would be happy with the solution proposed by @stevengj in https://groups.google.com/d/msg/julia-dev/Z6GsLURuIZE/AmxBLNsz60MJ

[quinnj]

This is a pretty old thread that I believe preceded much of our current package manager functionality along with BinDeps.jl magic. Is this still an issue worth pursuing? Is there something Julia should do about it or is this a project/package specific task if they so desire.

[stevengj]

Seems to be fixed by #2295, and in particular by #2550.