Skip to content

Commit

Permalink
Use latest Pip version when building image.
Browse files Browse the repository at this point in the history 
The initial problem with the new PIP resolved in 20.3 version have
been successfully solved. Seems that version 21.* is much more
stable and actually works in all cases, so we are switching back
to it.

Also changed pip and wheel dependencies to ~= (compatible) version
hoping that the experience of backwards incomptible release in
major version update have been adopted by the PIP team with
21* series release.

Closes apache#12838
  • Loading branch information
@potiuk
potiuk committed Mar 26, 2021
1 parent 614be87 commit f380e6a
Show file tree
Hide file tree
Showing 89 changed files with 37 additions and 689 deletions.
24 changes: 4 additions & 20 deletions CONTRIBUTING.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -553,17 +553,9 @@ Airflow dependencies

.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
might work with Apache Airflow as of 20.3.3, but it might lead to errors in installation. It might
depend on your choice of extras. In order to install Airflow you might need to either downgrade
pip to version 20.2.4 ``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3,
you need to add option ``--use-deprecated legacy-resolver`` to your pip install command.
The official way of installing Airflow is with the ``pip`` tool.

While ``pip 20.3.3`` solved most of the ``teething`` problems of 20.3, this note will remain here until we
set ``pip 20.3`` as official version in our CI pipeline where we are testing the installation as well.
Due to those constraints, only ``pip`` installation is currently officially supported.

While they are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
While there are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
`pip-tools <https://pypi.org/project/pip-tools/>`_, they do not share the same workflow as
``pip`` - especially when it comes to constraint vs. requirements management.
Installing via ``Poetry`` or ``pip-tools`` is not currently supported.
Expand Down Expand Up @@ -788,17 +780,9 @@ Pinned constraint files

.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
might work with Apache Airflow as of 20.3.3, but it might lead to errors in installation. It might
depend on your choice of extras. In order to install Airflow you might need to either downgrade
pip to version 20.2.4 ``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3,
you need to add option ``--use-deprecated legacy-resolver`` to your pip install command.

While ``pip 20.3.3`` solved most of the ``teething`` problems of 20.3, this note will remain here until we
set ``pip 20.3`` as official version in our CI pipeline where we are testing the installation as well.
Due to those constraints, only ``pip`` installation is currently officially supported.
The official way of installing Airflow is with the ``pip`` tool.

While they are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
While there are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
`pip-tools <https://pypi.org/project/pip-tools/>`_, they do not share the same workflow as
``pip`` - especially when it comes to constraint vs. requirements management.
Installing via ``Poetry`` or ``pip-tools`` is not currently supported.
Expand Down
12 changes: 2 additions & 10 deletions CONTRIBUTORS_QUICK_START.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -167,17 +167,9 @@ Setup Airflow with Breeze and PyCharm

.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
might work with Apache Airflow as of 20.3.3, but it might lead to errors in installation. It might
depend on your choice of extras. In order to install Airflow you might need to either downgrade
pip to version 20.2.4 ``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3,
you need to add option ``--use-deprecated legacy-resolver`` to your pip install command.
The official way of installing Airflow is with the ``pip`` tool.

While ``pip 20.3.3`` solved most of the ``teething`` problems of 20.3, this note will remain here until we
set ``pip 20.3`` as official version in our CI pipeline where we are testing the installation as well.
Due to those constraints, only ``pip`` installation is currently officially supported.

While they are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
While there are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
`pip-tools <https://pypi.org/project/pip-tools/>`_, they do not share the same workflow as
``pip`` - especially when it comes to constraint vs. requirements management.
Installing via ``Poetry`` or ``pip-tools`` is not currently supported.
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ ARG CASS_DRIVER_BUILD_CONCURRENCY="8"

ARG PYTHON_BASE_IMAGE="python:3.6-slim-buster"

ARG AIRFLOW_PIP_VERSION=20.2.4
ARG AIRFLOW_PIP_VERSION=21.0.1

# By default PIP has progress bar but you can disable it.
ARG PIP_PROGRESS_BAR="on"
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile.ci
View file
Original file line number Diff line number Diff line change
Expand Up @@ -266,7 +266,7 @@ ENV INSTALL_FROM_DOCKER_CONTEXT_FILES=${INSTALL_FROM_DOCKER_CONTEXT_FILES}
ARG INSTALL_FROM_PYPI="true"
ENV INSTALL_FROM_PYPI=${INSTALL_FROM_PYPI}

ARG AIRFLOW_PIP_VERSION=20.2.4
ARG AIRFLOW_PIP_VERSION=21.0.1
ENV AIRFLOW_PIP_VERSION=${AIRFLOW_PIP_VERSION}

# In the CI image we always:
Expand Down
14 changes: 3 additions & 11 deletions IMAGES.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -122,17 +122,9 @@ parameter to Breeze:
.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
might work with Apache Airflow as of 20.3.3, but it might lead to errors in installation. It might
depend on your choice of extras. In order to install Airflow you might need to either downgrade
pip to version 20.2.4 ``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3,
you need to add option ``--use-deprecated legacy-resolver`` to your pip install command.
The official way of installing Airflow is with the ``pip`` tool.

While ``pip 20.3.3`` solved most of the ``teething`` problems of 20.3, this note will remain here until we
set ``pip 20.3`` as official version in our CI pipeline where we are testing the installation as well.
Due to those constraints, only ``pip`` installation is currently officially supported.

While they are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
While there are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
`pip-tools <https://pypi.org/project/pip-tools/>`_, they do not share the same workflow as
``pip`` - especially when it comes to constraint vs. requirements management.
Installing via ``Poetry`` or ``pip-tools`` is not currently supported.
Expand Down Expand Up @@ -673,7 +665,7 @@ The following build arguments (``--build-arg`` in docker build command) can be u
| ``ADDITIONAL_RUNTIME_APT_ENV`` | | Additional env variables defined |
| | | when installing runtime deps |
+------------------------------------------+------------------------------------------+------------------------------------------+
| ``AIRFLOW_PIP_VERSION`` | ``20.2.4`` | PIP version used. |
| ``AIRFLOW_PIP_VERSION`` | ``21.0.1`` | PIP version used. |
+------------------------------------------+------------------------------------------+------------------------------------------+
| ``PIP_PROGRESS_BAR`` | ``on`` | Progress bar for PIP installation |
+------------------------------------------+------------------------------------------+------------------------------------------+
Expand Down
8 changes: 0 additions & 8 deletions INSTALL
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,14 +28,6 @@ java -jar apache-rat.jar -E ./.rat-excludes -d .
python3 -m venv PATH_TO_YOUR_VENV
source PATH_TO_YOUR_VENV/bin/activate

NOTE!!

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
``--use-deprecated legacy-resolver`` to your pip install command.

# [required] building and installing by pip (preferred)
pip install .

Expand Down
24 changes: 4 additions & 20 deletions LOCAL_VIRTUALENV.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,17 +63,9 @@ Extra Packages

.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
might work with Apache Airflow as of 20.3.3, but it might lead to errors in installation. It might
depend on your choice of extras. In order to install Airflow you might need to either downgrade
pip to version 20.2.4 ``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3,
you need to add option ``--use-deprecated legacy-resolver`` to your pip install command.
The official way of installing Airflow is with the ``pip`` tool.

While ``pip 20.3.3`` solved most of the ``teething`` problems of 20.3, this note will remain here until we
set ``pip 20.3`` as official version in our CI pipeline where we are testing the installation as well.
Due to those constraints, only ``pip`` installation is currently officially supported.

While they are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
While there are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
`pip-tools <https://pypi.org/project/pip-tools/>`_, they do not share the same workflow as
``pip`` - especially when it comes to constraint vs. requirements management.
Installing via ``Poetry`` or ``pip-tools`` is not currently supported.
Expand Down Expand Up @@ -137,17 +129,9 @@ To create and initialize the local virtualenv:

.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
might work with Apache Airflow as of 20.3.3, but it might lead to errors in installation. It might
depend on your choice of extras. In order to install Airflow you might need to either downgrade
pip to version 20.2.4 ``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3,
you need to add option ``--use-deprecated legacy-resolver`` to your pip install command.

While ``pip 20.3.3`` solved most of the ``teething`` problems of 20.3, this note will remain here until we
set ``pip 20.3`` as official version in our CI pipeline where we are testing the installation as well.
Due to those constraints, only ``pip`` installation is currently officially supported.
The official way of installing Airflow is with the ``pip`` tool.

While they are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
While there are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
`pip-tools <https://pypi.org/project/pip-tools/>`_, they do not share the same workflow as
``pip`` - especially when it comes to constraint vs. requirements management.
Installing via ``Poetry`` or ``pip-tools`` is not currently supported.
Expand Down
15 changes: 3 additions & 12 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -147,18 +147,9 @@ correct Airflow tag/version/branch and Python versions in the URL.

1. Installing just Airflow:

NOTE!!!

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
might work with Apache Airflow as of 20.3.3, but it might lead to errors in installation. It might
depend on your choice of extras. In order to install Airflow reliably, you might need to either downgrade
pip to version 20.2.4 `pip install --upgrade pip==20.2.4` or, in case you use Pip 20.3,
you might need to add option] `--use-deprecated legacy-resolver` to your pip install command.
While `pip 20.3.3` solved most of the `teething` problems of 20.3, this note will remain here until we
set `pip 20.3` as official version in our CI pipeline where we are testing the installation as well.
Due to those constraints, only `pip` installation is currently officially supported.

While they are some successes with using other tools like [poetry](https://python-poetry.org) or
The official way of installing Airflow is with the `pip` tool.

While there are some successes with using other tools like [poetry](https://python-poetry.org) or
[pip-tools](https://pypi.org/project/pip-tools), they do not share the same workflow as
`pip` - especially when it comes to constraint vs. requirements management.
Installing via `Poetry` or `pip-tools` is not currently supported.
Expand Down
9 changes: 0 additions & 9 deletions UPDATING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1762,15 +1762,6 @@ If you want to install integration for Microsoft Azure, then instead of `pip ins
you should use `pip install apache-airflow[apache.atlas]`.


NOTE!

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
`pip install --upgrade pip==20.2.4` or, in case you use Pip 20.3, you need to add option
`--use-deprecated legacy-resolver` to your pip install command.


If you want to install integration for Microsoft Azure, then instead of

```
Expand Down
8 changes: 0 additions & 8 deletions airflow/providers/apache/beam/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,6 @@ are in `airflow.providers.apache.beam` python package.

## Installation

NOTE!

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
`pip install --upgrade pip==20.2.4` or, in case you use Pip 20.3, you need to add option
`--use-deprecated legacy-resolver` to your pip install command.

You can install this package on top of an existing airflow 2.* installation via
`pip install apache-airflow-providers-apache-beam`

Expand Down
9 changes: 0 additions & 9 deletions dev/provider_packages/PROVIDER_INDEX_TEMPLATE.rst.jinja2
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,15 +47,6 @@ are in ``{{FULL_PACKAGE_NAME}}`` python package.
Installation
------------

.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
``--use-deprecated legacy-resolver`` to your pip install command.


You can install this package on top of an existing airflow 2.* installation via
``pip install {{PACKAGE_PIP_NAME}}``
{%- if PIP_REQUIREMENTS %}
Expand Down
8 changes: 0 additions & 8 deletions dev/provider_packages/PROVIDER_README_TEMPLATE.rst.jinja2
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,6 @@ in the `documentation <https://airflow.apache.org/docs/{{ PACKAGE_PIP_NAME }}/{{
Installation
============

NOTE!

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
``--use-deprecated legacy-resolver`` to your pip install command.

You can install this package on top of an existing airflow 2.* installation via
``pip install {{PACKAGE_PIP_NAME}}``

Expand Down
18 changes: 0 additions & 18 deletions docs/apache-airflow-providers-airbyte/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -64,15 +64,6 @@ are in ``airflow.providers.airbyte`` python package.
Installation
------------

.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
``--use-deprecated legacy-resolver`` to your pip install command.


You can install this package on top of an existing airflow 2.* installation via
``pip install apache-airflow-providers-airbyte``

Expand Down Expand Up @@ -147,15 +138,6 @@ are in ``airflow.providers.airbyte`` python package.
Installation
------------

.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
``--use-deprecated legacy-resolver`` to your pip install command.


You can install this package on top of an existing airflow 2.* installation via
``pip install apache-airflow-providers-airbyte``

Expand Down
9 changes: 0 additions & 9 deletions docs/apache-airflow-providers-amazon/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -70,15 +70,6 @@ are in ``airflow.providers.amazon`` python package.
Installation
------------

.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
``--use-deprecated legacy-resolver`` to your pip install command.


You can install this package on top of an existing airflow 2.* installation via
``pip install apache-airflow-providers-amazon``

Expand Down
8 changes: 0 additions & 8 deletions docs/apache-airflow-providers-apache-beam/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,14 +62,6 @@ are in ``airflow.providers.apache.beam`` python package.
Installation
------------

.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
``--use-deprecated legacy-resolver`` to your pip install command.


You can install this package on top of an existing airflow 2.* installation via
``pip install apache-airflow-providers-apache-beam``
Expand Down
8 changes: 0 additions & 8 deletions docs/apache-airflow-providers-apache-cassandra/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -68,14 +68,6 @@ are in ``airflow.providers.apache.cassandra`` python package.
Installation
------------

.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
``--use-deprecated legacy-resolver`` to your pip install command.


You can install this package on top of an existing airflow 2.* installation via
``pip install apache-airflow-providers-apache-cassandra``
Expand Down
8 changes: 0 additions & 8 deletions docs/apache-airflow-providers-apache-druid/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,14 +55,6 @@ are in ``airflow.providers.apache.druid`` python package.
Installation
------------

.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
``--use-deprecated legacy-resolver`` to your pip install command.


You can install this package on top of an existing airflow 2.* installation via
``pip install apache-airflow-providers-apache-druid``
Expand Down
8 changes: 0 additions & 8 deletions docs/apache-airflow-providers-apache-hdfs/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -64,14 +64,6 @@ are in ``airflow.providers.apache.hdfs`` python package.
Installation
------------

.. note::

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
``--use-deprecated legacy-resolver`` to your pip install command.


You can install this package on top of an existing airflow 2.* installation via
``pip install apache-airflow-providers-apache-hdfs``
Expand Down
Loading

0 comments on commit f380e6a

Please sign in to comment.