✨ Add --install flag (#11)

* ✨ Add --install flag * 🐛 Remove **kwargs as it's invalid syntax on py27 * 🐛 Increase stability for --install -e . * 💚 Run CD only when tags are pushed to master * 📝 Remove useless sh syntax highlighting * 🔊 Add environment info in -vvv * 🎨 Clean up .pre-commit-config.yaml
ddelange · Jan 26, 2020 · 9fa9425 · 9fa9425
1 parent 3c9b1bb
commit 9fa9425
Show file tree

Hide file tree

Showing 6 changed files with 235 additions and 83 deletions.
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -4,6 +4,7 @@ on:
   pull_request:
     branches: "*"
   push:
+    branches: "master"
     tags: "*"
 
 jobs:

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -12,21 +12,10 @@ repos:
     -   id: black
         language_version: python3.7
 
--   repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v2.4.0
-    hooks:
-    -   id: trailing-whitespace
-
--   repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v2.4.0
-    hooks:
-    -   id: end-of-file-fixer
-
--   repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v2.4.0
+-   repo: https://github.com/pre-commit/mirrors-isort
+    rev: v4.3.21
     hooks:
-    -   id: mixed-line-ending
-        args: ['--fix=lf']
+    -   id: isort
 
 -   repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v2.4.0
@@ -44,13 +33,8 @@ repos:
             flake8-quotes,
             flake8-tuple,
         ]
-
--   repo: https://github.com/pre-commit/mirrors-isort
-    rev: v4.3.21
-    hooks:
-    -   id: isort
-
--   repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v2.4.0
-    hooks:
+    -   id: mixed-line-ending
+        args: ['--fix=lf']
+    -   id: trailing-whitespace
+    -   id: end-of-file-fixer
     -   id: check-merge-conflict
diff --git a/README.md b/README.md
@@ -1,9 +1,9 @@
 # pipgrip
 
-[![build](https://img.shields.io/github/workflow/status/ddelange/pipgrip/GH/master?logo=github&cacheSeconds=86400)](https://github.com/ddelange/pipgrip/actions)
+[![build](https://img.shields.io/github/workflow/status/ddelange/pipgrip/GH/master?logo=github&cacheSeconds=86400)](https://github.com/ddelange/pipgrip/actions?query=branch%3Amaster)
 [![codecov](https://img.shields.io/codecov/c/github/ddelange/pipgrip/master?logo=codecov&logoColor=white)](https://codecov.io/gh/ddelange/pipgrip)
 [![pypi Version](https://img.shields.io/pypi/v/pipgrip.svg?logo=pypi&logoColor=white)](https://pypi.org/project/pipgrip/)
-[![python](https://img.shields.io/pypi/pyversions/pipgrip.svg?logo=python&logoColor=white)](https://github.com/ddelange/pipgrip/releases/latest)
+[![python](https://img.shields.io/pypi/pyversions/pipgrip.svg?logo=python&logoColor=white)](https://pypi.org/project/pipgrip/)
 [![downloads](https://pepy.tech/badge/pipgrip)](https://pypistats.org/packages/pipgrip)
 [![black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/python/black)
 
@@ -24,36 +24,45 @@
 
 For offline usage, [pipdeptree](https://github.com/naiquevin/pipdeptree) can inspect the current environment and show how the currently installed packages relate to each other. This however requires the packages to be pip-installed, and (despite warnings about e.g. cyclic dependencies) offers no form of dependency resolution since it's only based on the (single) package versions installed in the environment. Such shortcomings are avoided when using pipgrip, since packages don't need to be installed and all versions available to pip are considered.
 
+
 ## Installation
 
 This pure-Python, OS independent package is available on [PyPI](https://pypi.org/project/pipgrip/):
 
-```sh
+```
 pip install pipgrip
 ```
 
 
 ## Usage
 
 This package can be used to:
-- Alleviate [Python dependency hell](https://medium.com/knerd/the-nine-circles-of-python-dependency-hell-481d53e3e025) by resolving the latest viable combination of required packages
-- Render an exhaustive dependency tree for any given pip-compatible package(s) with `--tree`
-- Detect version conflicts for given constraints and give human readable feedback about it
-- Find dependency conflicts or cyclic dependencies in local projects:
-  - `pipgrip -v --tree .`
-- Avoid bugs by running pipgrip as a stage in CI pipelines
-- Install complex packages without worries using:
+- **Render** an exhaustive dependency tree for any given pip-compatible package(s) with `--tree`
+- **Alleviate** [Python dependency hell](https://medium.com/knerd/the-nine-circles-of-python-dependency-hell-481d53e3e025) by resolving the latest viable combination of required packages
+- **Avoid** bugs by running pipgrip as a stage in CI pipelines
+- **Detect** version conflicts for given constraints and give human readable feedback about it
+- **Warn** for cyclic dependencies in local projects [and install them anyway]:
+  - `pipgrip -v --tree . [--install -e]`
+- **Install** complex packages without worries using:
+  - `pipgrip --install aiobotocore[awscli]`
   - ``pip install -U --no-deps `pipgrip --pipe aiobotocore[awscli]` ``
-- Generate a lockfile with a complete working set of dependencies for worriless installs (see [known caveats](#known-caveats)):
+- **Generate** a lockfile with a complete working set of dependencies for worriless installs:
+  - `pipgrip --lock --install --tree -v aiobotocore[awscli]`
   - `pipgrip --lock -tree aiobotocore[awscli] && pip install -U --no-deps -r ./pipgrip.lock`
   - `pipgrip aiobotocore[awscli] | pip install -U --no-deps -r /dev/stdin`
+- **Combine** dependency trees of multiple packages into one unified set of pinned packages:
+  - `pipgrip --lock --install --tree -v .[boto3] s3transfer==0.2.1`
+
+See also [known caveats](#known-caveats).
 
-```sh
+```
 $ pipgrip --help
 
 Usage: pipgrip [OPTIONS] [DEPENDENCIES]...
 
 Options:
+  --install               Install full dependency tree after resolving.
+  -e, --editable          Install a project in editable mode.
   --lock                  Write out pins to './pipgrip.lock'.
   --pipe                  Output space-separated pins instead of newline-
                           separated pins.
@@ -80,41 +89,67 @@ Options:
 #### Dependency trees
 
 Exhaustive dependency trees without the need to install any packages (at most build some wheels).
-```sh
+```
 $ pipgrip --tree pipgrip
 
-pipgrip (0.0.3)
-├── anytree (2.7.3)
-│   └── six (1.13.0)
+pipgrip (0.1.0)
+├── anytree (2.8.0)
+│   └── six>=1.9.0 (1.14.0)
 ├── click (7.0)
-├── packaging>=17 (20.0)
+├── packaging>=17 (20.1)
 │   ├── pyparsing>=2.0.2 (2.4.6)
-│   └── six (1.13.0)
+│   └── six (1.14.0)
 ├── pkginfo>=1.4.2 (1.5.0.1)
-├── setuptools>=38.3 (44.0.0)
+├── setuptools>=38.3 (45.1.0)
 └── wheel (0.33.6)
 ```
 
 #### Lockfile generation
 
 Using the `--lock` option, resolved (pinned) dependencies are additionally written to `./pipgrip.lock`.
-```sh
-$ pipgrip --lock boto3
-
-boto3==1.10.46
-botocore==1.13.46
+```
+$ pipgrip --tree --lock botocore==1.13.48 'boto3>=1.10'
+
+botocore==1.13.48 (1.13.48)
+├── docutils<0.16,>=0.10 (0.15.2)
+├── jmespath<1.0.0,>=0.7.1 (0.9.4)
+├── python-dateutil<3.0.0,>=2.1 (2.8.1)
+│   └── six>=1.5 (1.14.0)
+└── urllib3<1.26,>=1.20 (1.25.8)
+boto3 (1.10.48)
+├── botocore<1.14.0,>=1.13.48 (1.13.48)
+│   ├── docutils<0.16,>=0.10 (0.15.2)
+│   ├── jmespath<1.0.0,>=0.7.1 (0.9.4)
+│   ├── python-dateutil<3.0.0,>=2.1 (2.8.1)
+│   │   └── six>=1.5 (1.14.0)
+│   └── urllib3<1.26,>=1.20 (1.25.8)
+├── jmespath<1.0.0,>=0.7.1 (0.9.4)
+└── s3transfer<0.3.0,>=0.2.0 (0.2.1)
+    └── botocore<2.0.0,>=1.12.36 (1.13.48)
+        ├── docutils<0.16,>=0.10 (0.15.2)
+        ├── jmespath<1.0.0,>=0.7.1 (0.9.4)
+        ├── python-dateutil<3.0.0,>=2.1 (2.8.1)
+        │   └── six>=1.5 (1.14.0)
+        └── urllib3<1.26,>=1.20 (1.25.8)
+
+$ cat ./pipgrip.lock
+
+botocore==1.13.48
 docutils==0.15.2
 jmespath==0.9.4
 python-dateutil==2.8.1
-six==1.13.0
-urllib3==1.25.7
+six==1.14.0
+urllib3==1.25.8
+boto3==1.10.48
 s3transfer==0.2.1
 ```
+NOTE:
+Since the selected botocore version is older than the one required by the recent versions of boto3, all boto3 versions will be checked for compatibility with botocore==1.12.42.
 
 #### Version conflicts
 
 If version conflicts exist for the given (ranges of) package version(s), a verbose explanation is raised.
-```sh
+```
 $ pipgrip auto-sklearn~=0.6 dragnet==2.0.4
 
 Error: Because dragnet (2.0.4) depends on scikit-learn (>=0.15.2,<0.21.0)
@@ -128,54 +163,61 @@ If older versions of auto-sklearn are allowed, PubGrub will try all acceptable v
 #### Cyclic dependencies
 
 If cyclic dependencies are found, it is noted in the resulting tree.
-```sh
-$ pipgrip --tree keras==2.2.2
+```
+$ pipgrip --tree -v keras==2.2.2
 
+WARNING: Cyclic dependency found: keras depends on keras-applications and vice versa.
+WARNING: Cyclic dependency found: keras depends on keras-preprocessing and vice versa.
 keras==2.2.2 (2.2.2)
 ├── h5py (2.10.0)
-│   ├── numpy>=1.9.1 (1.18.0)
-│   └── six>=1.9.0 (1.13.0)
+│   ├── numpy>=1.7 (1.18.1)
+│   └── six (1.14.0)
 ├── keras-applications==1.0.4 (1.0.4)
 │   ├── h5py (2.10.0)
-│   │   ├── numpy>=1.9.1 (1.18.0)
-│   │   └── six>=1.9.0 (1.13.0)
-│   ├── keras==2.2.2 (2.2.2, cyclic)
-│   └── numpy>=1.9.1 (1.18.0)
+│   │   ├── numpy>=1.7 (1.18.1)
+│   │   └── six (1.14.0)
+│   ├── keras>=2.1.6 (2.2.2, cyclic)
+│   └── numpy>=1.9.1 (1.18.1)
 ├── keras-preprocessing==1.0.2 (1.0.2)
-│   ├── keras==2.2.2 (2.2.2, cyclic)
-│   ├── numpy>=1.9.1 (1.18.0)
+│   ├── keras>=2.1.6 (2.2.2, cyclic)
+│   ├── numpy>=1.9.1 (1.18.1)
 │   ├── scipy>=0.14 (1.4.1)
-│   │   └── numpy>=1.9.1 (1.18.0)
-│   └── six>=1.9.0 (1.13.0)
-├── numpy>=1.9.1 (1.18.0)
-├── pyyaml (5.2)
+│   │   └── numpy>=1.13.3 (1.18.1)
+│   └── six>=1.9.0 (1.14.0)
+├── numpy>=1.9.1 (1.18.1)
+├── pyyaml (5.3)
 ├── scipy>=0.14 (1.4.1)
-│   └── numpy>=1.9.1 (1.18.0)
-└── six>=1.9.0 (1.13.0)
+│   └── numpy>=1.13.3 (1.18.1)
+└── six>=1.9.0 (1.14.0)
 ```
 
+
 ## Known caveats
 
-- ``pip install -U `pipgrip package` `` without `--no-deps` is unsafe while pip doesn't [yet](https://twitter.com/di_codes/status/1193980331004743680) have a built-in dependency resolver, and leaves room for interpretation by pip
+- ``pip install -U `pipgrip --pipe package` `` without `--no-deps` is unsafe while pip doesn't [yet](https://twitter.com/di_codes/status/1193980331004743680) have a built-in dependency resolver, and leaves room for interpretation by pip
 - Package names are canonicalised in wheel metadata, resulting in e.g. `path.py -> path-py` and `keras_preprocessing -> keras-preprocessing` in output
 - [VCS Support](https://pip.pypa.io/en/stable/reference/pip_install/#vcs-support) isn't implemented yet
 - `--reversed-tree` isn't implemented yet
-- Since `pip install -r` does not accept `.` as requirement, it is omitted from lockfiles, so `--pipe` should be used when installing local projects
-- Installing packages using pipgrip is not very intuitive, so maybe pipgrip needs a stable `--install` flag
+- Since `pip install -r` does not accept `.` as requirement, it is omitted from lockfiles, so `--install` or `--pipe` should be used when installing local projects
+- The equivalent of e.g. `pip install ../aiobotocore[boto3]` is not yet implemented. However, e.g. `pipgrip --install .[boto3]` is allowed.
+
 
 ## Development
 
+[![gitmoji](https://img.shields.io/badge/gitmoji-%20%F0%9F%98%9C%20%F0%9F%98%8D-ffdd67)](https://github.com/carloscuesta/gitmoji-cli)
+[![pre-commit](https://img.shields.io/badge/pre--commit-available-green?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)
+
 Create a virtual environment and get ready to develop:
 
-```sh
+```
 make install
 ```
 
 This [make-command](Makefile) is equivalent to the following steps:
 
 Install pre-commit and other continous integration dependencies in order to make commits and run tests.
 
-```sh
+```
 pip install -r requirements/ci.txt
 pre-commit install
 ```
@@ -184,10 +226,11 @@ With requirements installed, `make lint` and `make test` can now be run. There i
 
 To import the package in the python environment, install the package (`-e` for editable installation, upon import, python will read directly from the repository).
 
-```sh
+```
 pip install -e .
 ```
 
+
 ## See also
 
 - [PubGrub spec](https://github.com/dart-lang/pub/blob/SDK-2.2.1-dev.3.0/doc/solver.md)