Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inject additional packages from text file #1252

Merged
merged 52 commits into from
May 14, 2024
Merged
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
4a8a3ce
Naive `pipx inject <package> -r requirements.txt`
jamesmyatt May 8, 2024
9a338c0
Fix imports
jamesmyatt May 8, 2024
1e80bfe
Better combination of packages
jamesmyatt May 8, 2024
500d6dd
Better help text for `inject -r`
jamesmyatt May 8, 2024
988bbe8
Add unit test
jamesmyatt May 8, 2024
daa5c80
Add changelog
jamesmyatt May 8, 2024
fe898e5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 8, 2024
2f525a0
Rename changelog file
jamesmyatt May 8, 2024
a6aee67
Fix pylint and mypy errors
jamesmyatt May 8, 2024
0d6f4e2
Fix default for requirements
jamesmyatt May 8, 2024
ccb787c
Use assignment operator since Python >= 3.8
jamesmyatt May 8, 2024
305d2ef
Update src/pipx/commands/inject.py
jamesmyatt May 8, 2024
d331d2a
Update src/pipx/main.py
jamesmyatt May 8, 2024
47a1ae5
Update tests/test_inject.py
jamesmyatt May 8, 2024
c1e1688
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 8, 2024
1a26839
Update changelog.d/1252.feature.md
jamesmyatt May 8, 2024
d6b3c28
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 8, 2024
4ac9fff
Update tests/test_inject.py
jamesmyatt May 8, 2024
5bf3566
Update test_inject.py
jamesmyatt May 8, 2024
f0144ab
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 8, 2024
04b82e5
Logging at INFO level
jamesmyatt May 8, 2024
e3afb40
Discard duplicated package specifications
jamesmyatt May 8, 2024
5509d0f
Update 1252.feature.md
jamesmyatt May 8, 2024
7358cfa
Update install-all command
jamesmyatt May 8, 2024
9b4b4dd
Expand pipx inject example
jamesmyatt May 8, 2024
6833592
Clarify changelog entry
jamesmyatt May 8, 2024
0615b08
Mention in main README
jamesmyatt May 8, 2024
9967262
Check stdout and logs in test
jamesmyatt May 8, 2024
48a88c7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 8, 2024
e078ed0
Better test for injected packages
jamesmyatt May 8, 2024
776bcac
Clarify ignoring of comments in example
jamesmyatt May 8, 2024
e2f6d3a
Clarify use of "requirement" file
jamesmyatt May 8, 2024
9792a21
Update README.md
jamesmyatt May 8, 2024
8f015c5
Update README.md
jamesmyatt May 8, 2024
8a0acbe
Check can inject each package independently
jamesmyatt May 8, 2024
90394e4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 8, 2024
9d4a7e3
Fix handling of tricky characters
jamesmyatt May 8, 2024
3a58c79
Use logger where possible
jamesmyatt May 8, 2024
80f6651
More messages in logs
jamesmyatt May 8, 2024
1aa0b0b
More debugging messages
jamesmyatt May 9, 2024
2c6201d
Inject additional package that isn't already installed
jamesmyatt May 9, 2024
959323a
Make inject order deterministic
jamesmyatt May 9, 2024
3762d65
Fix mypy error
jamesmyatt May 9, 2024
6d62bcc
tidy test_inject_single_package cases
jamesmyatt May 10, 2024
f8b9719
Better comments on tests
jamesmyatt May 10, 2024
c36056e
Update 1252.feature.md
jamesmyatt May 14, 2024
fbee0e2
Update 1252.feature.md
jamesmyatt May 14, 2024
49832dc
Update examples.md
jamesmyatt May 14, 2024
41ace41
Fix examples.md
jamesmyatt May 14, 2024
9dc46f6
Update 1252.feature.md
jamesmyatt May 14, 2024
25adc77
Update 1252.feature.md
jamesmyatt May 14, 2024
92e1660
Merge branch 'main' into inject-requirements
huxuan May 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,16 @@ If an application installed by pipx requires additional packages, you can add th
pipx inject ipython matplotlib
```

You can inject multiple packages by specifying them all on the command line,
or by listing them in a text file, with one package per line,
or a combination. For example:

```
pipx inject ipython matplotlib pandas
# or:
pipx inject ipython -r useful-packages.txt
```

### Walkthrough: Running an Application in a Temporary Virtual Environment

This is an alternative to `pipx install`.
Expand Down
16 changes: 16 additions & 0 deletions changelog.d/1252.feature.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
Add `--requirement` option to `inject` command.

This reads the list of packages to inject from a text file,
can be used multiple times,
and can be used in parallel with dependencies specified
through the command line.
The option can be abbreviated to `-r`.

Each line is a separate package specifier with the same syntax as the command line.
Comments are supported with a `#` prefix.
Hence, the syntax is a strict subset of the pip [requirements file format][pip-requirements] syntax.

If you require full pip functionality, then use the `runpip` command instead;
however, the installed packages won't be recognised as "injected".

[pip-requirements]: https://pip.pypa.io/en/stable/reference/requirements-file-format/
jamesmyatt marked this conversation as resolved.
Show resolved Hide resolved
25 changes: 23 additions & 2 deletions docs/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,13 +98,34 @@ Then you can run it as follows:
One use of the inject command is setting up a REPL with some useful extra packages.

```
pipx install ptpython
pipx inject ptpython requests pendulum
> pipx install ptpython
> pipx inject ptpython requests pendulum
```

After running the above commands, you will be able to import and use the `requests` and `pendulum` packages inside a
`ptpython` repl.

Equivalently, the extra packages can be listed in a text file (e.g. `useful-packages.txt`) with one on each line (comments ignored):

```
# Additional packages
requests

pendulum # for easier datetimes
```

This file can then be given to `pipx inject` on the command line:

```
> pipx inject ptpython -r useful-packages.txt
```

Note that these options can be repeated and used together, e.g.

```
> pipx inject ptpython package-1 -r extra-packages-1.txt -r extra-packages-2.txt package-2
```

## `pipx list` example

```
Expand Down
35 changes: 33 additions & 2 deletions src/pipx/commands/inject.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
import logging
import os
import re
import sys
from pathlib import Path
from typing import List, Optional
from typing import Generator, List, Optional, Union

from pipx import paths
from pipx.colors import bold
Expand All @@ -11,6 +13,10 @@
from pipx.util import PipxError, pipx_wrap
from pipx.venv import Venv

logger = logging.getLogger(__name__)

COMMENT_RE = re.compile(r"(^|\s+)#.*$")
chrysle marked this conversation as resolved.
Show resolved Hide resolved


def inject_dep(
venv_dir: Path,
Expand Down Expand Up @@ -103,6 +109,7 @@ def inject(
venv_dir: Path,
package_name: Optional[str],
package_specs: List[str],
requirement_files: List[str],
pip_args: List[str],
*,
verbose: bool,
Expand All @@ -112,10 +119,20 @@ def inject(
suffix: bool = False,
) -> ExitCode:
"""Returns pipx exit code."""
# Combined collection of package specifications
packages = set(package_specs)
for filename in requirement_files:
packages.update(parse_requirements(filename))

if not packages:
raise PipxError("No packages have been specified.")
chrysle marked this conversation as resolved.
Show resolved Hide resolved
logger.info("Injecting packages: %r", sorted(packages))

# Inject packages
if not include_apps and include_dependencies:
include_apps = True
all_success = True
for dep in package_specs:
for dep in packages:
all_success &= inject_dep(
venv_dir,
None,
Expand All @@ -130,3 +147,17 @@ def inject(

# Any failure to install will raise PipxError, otherwise success
return EXIT_CODE_OK if all_success else EXIT_CODE_INJECT_ERROR


def parse_requirements(filename: Union[str, os.PathLike]) -> Generator[str, None, None]:
"""
Extract package specifications from requirements file.

Return all of the non-empty lines with comments removed.
"""
# Based on https://github.com/pypa/pip/blob/main/src/pip/_internal/req/req_file.py
with open(filename) as f:
for line in f:
# Strip comments and filter empty lines
if pkgspec := COMMENT_RE.sub("", line).strip():
yield pkgspec
9 changes: 5 additions & 4 deletions src/pipx/commands/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -215,10 +215,11 @@ def install_all(
# Install the injected packages
for inject_package in venv_metadata.injected_packages.values():
commands.inject(
venv_dir,
None,
[generate_package_spec(inject_package)],
pip_args,
venv_dir=venv_dir,
package_name=None,
package_specs=[generate_package_spec(inject_package)],
requirement_files=[],
pip_args=pip_args,
verbose=verbose,
include_apps=inject_package.include_apps,
include_dependencies=inject_package.include_dependencies,
Expand Down
16 changes: 15 additions & 1 deletion src/pipx/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,7 @@ def run_pipx_command(args: argparse.Namespace, subparsers: Dict[str, argparse.Ar
venv_dir,
None,
args.dependencies,
args.requirements,
pip_args,
verbose=verbose,
include_apps=args.include_apps,
Expand Down Expand Up @@ -515,9 +516,22 @@ def _add_inject(subparsers, venv_completer: VenvCompleter, shared_parser: argpar
).completer = venv_completer
p.add_argument(
"dependencies",
nargs="+",
nargs="*",
help="the packages to inject into the Virtual Environment--either package name or pip package spec",
)
p.add_argument(
"-r",
"--requirement",
dest="requirements",
action="append",
default=[],
metavar="file",
help=(
"file containing the packages to inject into the Virtual Environment--"
"one package name or pip package spec per line. "
"May be specified multiple times."
),
)
p.add_argument(
"--include-apps",
action="store_true",
Expand Down
84 changes: 72 additions & 12 deletions tests/test_inject.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,36 @@
import logging
import re
import textwrap

import pytest # type: ignore

from helpers import PIPX_METADATA_LEGACY_VERSIONS, mock_legacy_venv, run_pipx_cli, skip_if_windows
from package_info import PKG


def test_inject_simple(pipx_temp_env, capsys):
@pytest.mark.parametrize(
"pkg_spec,",
[
PKG["black"]["spec"], # was test_inject_simple
"jaraco.clipboard==2.0.1", # was test_inject_tricky_character
"pylint==3.0.4", # was test_spec
jamesmyatt marked this conversation as resolved.
Show resolved Hide resolved
PKG["nox"]["spec"], # used in test_inject_with_req_file
PKG["pylint"]["spec"], # used in test_inject_with_req_file
PKG["isort"]["spec"], # used in test_inject_with_req_file
],
)
def test_inject_single_package(pipx_temp_env, capsys, caplog, pkg_spec):
assert not run_pipx_cli(["install", "pycowsay"])
assert not run_pipx_cli(["inject", "pycowsay", PKG["black"]["spec"]])
assert not run_pipx_cli(["inject", "pycowsay", pkg_spec])

# Check arguments have been parsed correctly
assert f"Injecting packages: {[pkg_spec]!r}" in caplog.text

# Check it's actually being installed and into correct venv
captured = capsys.readouterr()
injected = re.findall(r"injected package (.+?) into venv pycowsay", captured.out)
pkg_name, *_ = pkg_spec.split("=", 1) # assuming spec is always of the form <name>==<version>
assert set(injected) == {pkg_name}


@skip_if_windows
Expand All @@ -27,16 +51,6 @@ def test_inject_simple_legacy_venv(pipx_temp_env, capsys, metadata_version):
assert "Please uninstall and install" in capsys.readouterr().err


def test_inject_tricky_character(pipx_temp_env, capsys):
assert not run_pipx_cli(["install", "pycowsay"])
assert not run_pipx_cli(["inject", "pycowsay", "jaraco.clipboard==2.0.1"])


def test_spec(pipx_temp_env, capsys):
assert not run_pipx_cli(["install", "pycowsay"])
assert not run_pipx_cli(["inject", "pycowsay", "pylint==3.0.4"])


@pytest.mark.parametrize("with_suffix,", [(False,), (True,)])
def test_inject_include_apps(pipx_temp_env, capsys, with_suffix):
jamesmyatt marked this conversation as resolved.
Show resolved Hide resolved
install_args = []
Expand All @@ -53,3 +67,49 @@ def test_inject_include_apps(pipx_temp_env, capsys, with_suffix):
assert run_pipx_cli(["inject", "pycowsay", PKG["black"]["spec"], "--include-deps"])

assert not run_pipx_cli(["inject", f"pycowsay{suffix}", PKG["black"]["spec"], "--include-deps"])


@pytest.mark.parametrize(
"with_packages,",
[
(), # no extra packages
("black",), # duplicate from requirements file
("isort",), # additional package
],
)
def test_inject_with_req_file(pipx_temp_env, capsys, caplog, tmp_path, with_packages):
caplog.set_level(logging.INFO)

req_file = tmp_path / "inject-requirements.txt"
req_file.write_text(
textwrap.dedent(
f"""
{PKG["black"]["spec"]} # a comment inline
{PKG["nox"]["spec"]}

{PKG["pylint"]["spec"]}
# comment on separate line
"""
).strip()
)
assert not run_pipx_cli(["install", "pycowsay"])

assert not run_pipx_cli(
["inject", "pycowsay", *(PKG[pkg]["spec"] for pkg in with_packages), "--requirement", str(req_file)]
)

packages = [
("black", PKG["black"]["spec"]),
("nox", PKG["nox"]["spec"]),
("pylint", PKG["pylint"]["spec"]),
]
packages.extend((pkg, PKG[pkg]["spec"]) for pkg in with_packages)
packages = sorted(set(packages))

# Check arguments and files have been parsed correctly
assert f"Injecting packages: {[p for _, p in packages]!r}" in caplog.text

# Check they're actually being installed and into correct venv
captured = capsys.readouterr()
injected = re.findall(r"injected package (.+?) into venv pycowsay", captured.out)
assert set(injected) == {pkg for pkg, _ in packages}
Loading