Option to install package globally for multi-user access #754

alextremblay · 2021-10-25T13:12:53Z

Hi All, not sure if this qualifies as a feature request, or if it's already possible and just needs to be documented?

pipx by default installs its virtualenvs in a folder inside the user's home directory (ie ~/.local/pipx/venvs/), and installs the entrypoint binaries for those packages/virtualenvs in the user's personal path (ie ~/.local/bin).

Is it possible to install packages with pipx such that they are globally executable by all users on a system (ie install entrypoint binaries in /usr/local/bin, install virtualenvs somewhere that all users can access them?

How would this feature be useful?

Having this ability would make pipx a prime candidate for people looking to deploy python software in multi-user environments. Without this ability, pipx is not usable at all in such situations

Describe the solution you'd like

Ideally, some kind of --shared or --global flag would be the most straightforward solution, but even something like an optional environment variable would be nice

Ex: sudo pipx install --global cowsay

Describe alternatives you've considered

There are currently no alternatives / workarounds that I'm aware of. I cannot find anything about this in the documentation, the github issues, or on the web

The text was updated successfully, but these errors were encountered:

cs01 · 2021-10-25T13:54:39Z

There are a couple environment variables that I think can do what you’re asking for.

See PIPX_HOME and PIPX_BIN_DIR.

https://pypa.github.io/pipx/docs/

alextremblay · 2021-10-25T17:52:35Z

ahhhh, ok.

Apologies, i completely missed that in the documentation!

so if i understood correctly, a workaround to install globally accessible python apps with pipx on linux would be:

$ sudo PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin pipx install cowsay

is that right?

in that case, would pipx create /opt/pipx, or does it need to exist first?

alextremblay · 2021-10-26T05:55:07Z

I just tested the above command, and it works exactly as intended!

didn't need to pre-create the directory or futz with directory permissions or anything :)

Is there any interest in me creating a PR to add this to the documentation?

uranusjr · 2021-10-27T00:19:26Z

Yes, that would be awesome 🙂

Jackenmen · 2022-07-11T17:42:16Z

It appears that this is already mentioned in the documentation:
https://pypa.github.io/pipx/installation/#installation-options

Is there any interest in exposing this in pipx for easier use or should users just be expected to create a shell alias if they want to use this regularly?

Jackenmen · 2022-07-11T23:33:44Z

Something I observed while trying out this solution today - it is impossible to run pretty much all commands (e.g. pipx list) without being the owner of PIPX_HOME because pipx uses the logs subdirectory of PIPX_HOME for its logs and that directory isn't accessible to regular users when set to /opt/pipx. A workaround for this is giving access to the /opt/pipx/logs for everyone but that doesn't seem ideal.

Jackenmen · 2022-07-12T20:24:08Z

Presumably, it would be safer to take additional pre-cautions such as only using the elevated permissions for installing the package, while building it with a normal user account. I imagine that this, however, might be out of scope for pipx, even if it would be incredibly cool if it could do that 😄

uranusjr · 2022-07-13T07:02:39Z

We can probably detect the log directory’s permission, and log to somewhere the user is likely to own (e.g. use platformdirs) instead. This probably involves some tinkering with logging configuration, which is always tedious, so I’m probably not going to do that…

Jackenmen · 2022-07-13T09:23:19Z

This probably involves some tinkering with logging configuration, which is always tedious, so I’m probably not going to do that…

What I hear is PRs welcome? :-) I might tinker with it later today.

carlolars · 2022-07-14T11:16:22Z

I think it would be nice to have a switch for global install. For example when using doas instead of sudo I had to first do doas su to run the pipx command with the environment variables as root. It would be a lot easier to just do doas pipx install --global cowsay.

eode · 2022-07-15T20:54:52Z

I definitely would like to have a --global option for pipx. I've put some thought into how this could be done (fairly) reasonably:

New environment vars:
- PIPX_GLOBAL_HOME
- PIPX_GLOBAL_BIN_DIR
Normal 'local' usage is treated just as it is now (including when using as root) except:
- If the global directory is present, global info is included in output
- --local can be specified to exclude global info
- install output changes to "These apps are now locally available"
With --global:
- Use PIPX_GLOBAL_* env vars.
  - Use PIPX_GLOBAL_HOME, or distro-specific global install dir, or /opt/pipx/ (failover)
  - Use PIPX_GLOBAL_BIN_DIR, the distro-specific global bin dir, or /usr/local/bin/ (failover)
- Treat the global dir as though we have write permission -- user should have elevated permissions using sudo
Encourage proper usage via cli
- Wherever a command looks like it might be mistaken, issue a warning / suggestion
  - "Writing into /home/someuser/.local/pipx, owned by someuser, as root. Did you mean to use --global?
  - "Installing foopkg locally for the user root. Did you mean to use --global?
JSON output includes "global venvs" key as a sibling to the "venvs" key.
Release on a major version change, because it introduces a new concept and changes the output.

Example usage:

$ pipx list
global venvs are in /opt/pipx/venvs
apps are exposed on your $PATH at /usr/local/bin
   package poetry 1.1.0, installed using Python 3.10.4
    - poetry
local venvs are in /home/someuser/.local/pipx/venvs
apps are exposed on your $PATH at /home/someuser/.local/bin
   package doit 0.36.0, installed using Python 3.10.4
    - doit
   package poetry 1.1.14, installed using Python 3.10.4
    - poetry
$ pipx list --local
local venvs are in /home/someuser/.local/pipx/venvs
apps are exposed on your $PATH at /home/someuser/.local/bin
   package doit 0.36.0, installed using Python 3.10.4
    - doit
   package poetry 1.1.14, installed using Python 3.10.4
    - poetry
$ sudo pipx install --global foocmd
  installed package foocmd 0.9.0, installed using Python 3.10.4
  These apps are now globally available
    - foo
    - foolog
done! ✨ 🌟 ✨
$ pipx install foocmd
  installed package foocmd 0.9.0, installed using Python 3.10.4
  These apps are now locally available
    - foo
    - foolog
done! ✨ 🌟 ✨

All-in-all, that's a fair amount of work, but not a ton of work to make the change, but I think that all of that work would be necessary to fully implement this change.

Alternately, --global could just do an override of the existing PIPX_HOME and PIPX_BIN_DIR env-vars, using distro-specific settings. This just means that sudo pipx list --global is required to read global items. I just like the UX of pipx list showing global and local installs.

silverwind · 2023-03-10T12:17:55Z

This would be very useful to install binaries globally on a server for consumption by all users.

The workaround with PIPX_BIN_DIR sounds usable, but I would not want to alter PIPX_HOME as the assumption would be that pipx is already installed into a system directory (I think this is already the case when pipx is installed via apt).

bulletmark · 2023-09-05T04:41:34Z

I don't understand why this discussion assumes we need a --global option? Surely sudo pipx install cowsay should just install to the appropriate system directories? (It should keep private root managed pipx venvs in a system area of course). I notice that currently that command installs to /root/.local/bin/cowsay which I doubt anybody wants.

PS: To be clear, I guess I should point out that running code can trivially determine that is is running with root/sudo privilege so I am suggesting to exploit that.

alextremblay · 2023-09-05T13:05:45Z

I don't understand why this discussion assumes we need a --global option? Surely sudo pipx install cowsay should just install to the appropriate system directories? (It should keep private root managed pipx venvs in a system area of course). I notice that currently that command installs to /root/.local/bin/cowsay which I doubt anybody wants.

PS: To be clear, I guess I should point out that running code can trivially determine that is is running with root/sudo privilege so I am suggesting to exploit that.

Though that sounds good in theory, I worry that it violates the principle of least surprise. Most package managers do not behave differently when run like that, and even pip, which operates globally by default, has an explicit ‘—user’ flag for when the default mode isn’t wanted

I would prefer to see an explicit ‘—global’ flag than implicit behaviour

bulletmark · 2023-09-05T13:21:52Z

@alextremblay I was actually surprised it didn't work like this! I tried it and found it installed to /root/.local/bin/cowsay which seems non-sensical to me so I went looking and found this issue. I wouldn't use pip as a guide on how to do things as it is has all sorts of legacy obligations and I doubt anybody would design pip the way it is if we could start again.

bulletmark · 2023-09-06T04:05:08Z

I created pipxx as a proof of concept and demonstration for what I am suggesting. It adds automatic root/global install as discussed here, and an enhanced list command output.

Just type pipx install git+https://github.com/bulletmark/pipxx to try it out.

uranusjr · 2023-09-06T04:46:01Z

Personally I also feel making sudo pipx install automatically install to system directories (instead of in /root/.local) is too implicit, for what it’s worth. A --system flag seems more in line. Maybe we should even block sudo pipx install to avoid the confusion (but still allow running pipx install as root, of course).

bulletmark · 2023-09-06T05:02:22Z

Maybe we should even block sudo pipx install to avoid the confusion (but still allow running pipx install as root, of course)

Frankly, this statement horrifies me. That would break the honored orthogonal behavior of sudo and is inconsistent with what we expect and how we do things on Linux.

uranusjr · 2023-09-06T05:43:32Z

To clarify:

pipx install .. will work like it does now.
pipx install --system ... will not work unless you have write access to system locations (and if you do, installs to system locations).
sudo pipx install --system ... installs to system locations.
sudo pipx install ... will be ambiguous. We can either keep it working as-is (installs to /root/.local), but this can be surprising judging from the above discussion. So the idea is to maybe make this fail and suggest adding --system.

bulletmark · 2023-09-06T23:58:33Z

@uranusjr after writing those rules out don't they seem convoluted, and certainly redundant? Compared to simply:

# Install for my user personally:
$ pipx install cowsay

# Or, install for all users globally:
$ sudo pipx install cowsay

pip has essentially been precluded as a global or user application installation tool nowadays (i.e. since the "externally-managed-environment" block most distros now apply to it) so pipx will step in to that role and thus global install capability needs to be improved and done asap. The pip UI is a mess so hopefully pipx designers will make better decisions.

ketozhang · 2023-09-07T01:00:10Z

I disagree with sudo or --system as the UI.

Relying on sudo solely for the behavior has historically been bad UX. It's too implicits---my only intentions of using sudo pipx is to bypass permissions (often write permission as @uranusjr pointed out).

--system implies this mechanism is limited only to installing into conventional system directories (e.g., /usr/local/, /opt).

Taking a page off of NPM (analogously pipx was inspired by NPX), only one addition to @eode's suggestion:

--global behavior:
- PIPX_HOME=${prefix:-/opt/}/pipx
- PIPX_BIN_DIR=${prefix:-/usr/local}/bin
Default (or --local) behavior:
- PIPX_HOME=${prefix:-~/.local}/pipx
- PIPX_BIN_DIR=${prefix:-~/.local}/bin

these rules flexible using, for example, --prefix to set non-standard paths.

haxwithaxe · 2023-10-02T20:10:25Z

I'm not advocating for this but given pipx is sort of an improved pip from a user perspective it would make sense to mimic the behavior of pip and install globally by default and fall back to installing per user.
Is anyone actively working on this? If not I will add it to my short list of PRs to make.

strk · 2023-12-14T10:45:46Z

didn't need to pre-create the directory or futz with directory permissions or anything :)

It actually failed for me:

FileNotFoundError: [Errno 2] No such file or directory: '/home/opt/pipx/logs'

On the second run it worked, so some directories were created and some others weren't.
This was with pipx 1.0.0 as packaged in Ubuntu-22.04 (needs a sepearate ticket?)

cyberfeign · 2023-12-29T01:21:53Z

A bit late to the party since this seems to be natively implemented soon, but here is my quick function to add a --global option:

# Global pipx option
pipx() {
  if [[ "$@" =~ '--global' ]]; then
    args=()
    for arg in "$@"; do
      # Ignore bad argument
      [[ $arg != '--global' ]] && args+=("$arg")
    done
    command sudo PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin PIPX_MAN_DIR=/usr/local/share/man pipx "${args[@]}"
  else
    command pipx "$@"
  fi
}

Hope it works OK for someone else as well, not duly tested.

) (#1281) * The bare minimum implementation of --global * Added --global switch that sets the relevant paths to point to locations that should be accessable to all users. * Added --global specific tests * The implementation is crude. * Fixing failing windows tests ...by not having --global in windows The behavior intended by --global doesn't have a precedent in windows when using pip (as in running pip with sudo). Leaving windows support to another time after some discussion. * Cleaned up the changes made to implement --global * Moved path management to `pipx.paths` * Path "constants" available via a context object `pipx.paths.ctx` * Fixed some stuff missed in the merge * Fix rebase issues * Add missing mkdir calls, fix test_upgrade * Add feature note to changelog.d * Use decorator for test skips on windows * Global tests update, PR cleanup * Add note about --global switch in installation docs * Revert unnecessary formatter changes * Satisfy mypy import * Add note about --global argument to how-pipx-works * --global documentation improvements * Update home_exists in make_global * Run test_fetch_missing_python with setting local * Add make_local at the end of all global tests * Fix ensurepath * make_local on fixture cleanup * Revert "Fix ensurepath" This reverts commit fe24c8f. * Remove explicit make_local from tests * Inform user about sudo pipx ensurepath --global * Make test_cli test less tolerant * Improve grammar --------- Co-authored-by: haxwithaxe <[email protected]>

chrysle · 2024-04-21T11:14:28Z

This feature was added in #1281 and released as part of pipx 1.5.0!

eode · 2024-05-03T14:33:58Z

Awesome, thanks!

…

On Sun, Apr 21, 2024, 7:14 AM chrysle ***@***.***> wrote: Closed #754 <#754> as completed. — Reply to this email directly, view it on GitHub <#754 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAS4PT5K2KFK6MY6YYSTGN3Y6ONSXAVCNFSM5GVICI7KU5DIOJSWCZC7NNSXTWQAEJEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW4OZRGI2TKMBWGM2DOOJS> . You are receiving this because you were mentioned.Message ID: ***@***.***>

djperrefort mentioned this issue Jul 1, 2022

Wrapper scripts break when other versions of python are loaded first pitt-crc/wrappers#15

Closed

Jackenmen mentioned this issue Jul 13, 2022

Fallback to user's log path if default log path is not writable #870

Merged

1 task

Jackenmen mentioned this issue Jul 15, 2022

Consider creating Snap package bneijt/autotrash#47

Open

dukecat0 added the enhancement New feature or request label Aug 8, 2022

haxwithaxe mentioned this issue Oct 15, 2023

Added --global option to perform actions for all users (#754) #1088

Closed

1 task

gaborbernat added the help wanted Extra attention is needed label Dec 2, 2023

strk mentioned this issue Dec 14, 2023

Add install make target ihabunek/toot#437

Closed

Botspot mentioned this issue Jan 2, 2024

issues with missing magick dependency l-koehler/FF-converter#1

Closed

xavierog mentioned this issue Feb 13, 2024

Debian 12 - Pip issue xavierog/moulti#1

Closed

mairas mentioned this issue Mar 15, 2024

Updated PR: Added --global option to perform actions for all users (#754) #1281

Merged

1 task

gaby mentioned this issue Mar 17, 2024

uv pip install --system cannot create directory astral-sh/uv#2059

Open

chrysle closed this as completed Apr 21, 2024

yp05327 mentioned this issue Jul 23, 2024

Please help about Jupyter file support go-gitea/gitea#31654

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Option to install package globally for multi-user access #754

Option to install package globally for multi-user access #754

alextremblay commented Oct 25, 2021

cs01 commented Oct 25, 2021 •

edited

Loading

alextremblay commented Oct 25, 2021

alextremblay commented Oct 26, 2021

uranusjr commented Oct 27, 2021

Jackenmen commented Jul 11, 2022

Jackenmen commented Jul 11, 2022

Jackenmen commented Jul 12, 2022

uranusjr commented Jul 13, 2022

Jackenmen commented Jul 13, 2022 •

edited

Loading

carlolars commented Jul 14, 2022

eode commented Jul 15, 2022 •

edited

Loading

silverwind commented Mar 10, 2023 •

edited

Loading

bulletmark commented Sep 5, 2023 •

edited

Loading

alextremblay commented Sep 5, 2023

bulletmark commented Sep 5, 2023

bulletmark commented Sep 6, 2023

uranusjr commented Sep 6, 2023

bulletmark commented Sep 6, 2023

uranusjr commented Sep 6, 2023 •

edited

Loading

bulletmark commented Sep 6, 2023

ketozhang commented Sep 7, 2023 •

edited

Loading

haxwithaxe commented Oct 2, 2023

strk commented Dec 14, 2023

cyberfeign commented Dec 29, 2023

chrysle commented Apr 21, 2024

eode commented May 3, 2024 via email

Option to install package globally for multi-user access #754

Option to install package globally for multi-user access #754

Comments

alextremblay commented Oct 25, 2021

cs01 commented Oct 25, 2021 • edited Loading

alextremblay commented Oct 25, 2021

alextremblay commented Oct 26, 2021

uranusjr commented Oct 27, 2021

Jackenmen commented Jul 11, 2022

Jackenmen commented Jul 11, 2022

Jackenmen commented Jul 12, 2022

uranusjr commented Jul 13, 2022

Jackenmen commented Jul 13, 2022 • edited Loading

carlolars commented Jul 14, 2022

eode commented Jul 15, 2022 • edited Loading

silverwind commented Mar 10, 2023 • edited Loading

bulletmark commented Sep 5, 2023 • edited Loading

alextremblay commented Sep 5, 2023

bulletmark commented Sep 5, 2023

bulletmark commented Sep 6, 2023

uranusjr commented Sep 6, 2023

bulletmark commented Sep 6, 2023

uranusjr commented Sep 6, 2023 • edited Loading

bulletmark commented Sep 6, 2023

ketozhang commented Sep 7, 2023 • edited Loading

haxwithaxe commented Oct 2, 2023

strk commented Dec 14, 2023

cyberfeign commented Dec 29, 2023

chrysle commented Apr 21, 2024

eode commented May 3, 2024 via email

cs01 commented Oct 25, 2021 •

edited

Loading

Jackenmen commented Jul 13, 2022 •

edited

Loading

eode commented Jul 15, 2022 •

edited

Loading

silverwind commented Mar 10, 2023 •

edited

Loading

bulletmark commented Sep 5, 2023 •

edited

Loading

uranusjr commented Sep 6, 2023 •

edited

Loading

ketozhang commented Sep 7, 2023 •

edited

Loading