1.10.12 attrs package dependency breaks pipenv locking

[aidar-ms]

Apache Airflow version: 1.10.12

Environment:

• OS (e.g. from /etc/os-release): Any OS: Mac OS, Linux
• Install tools: Pipenv
• Others:

What happened:

Trying to lock pip dependencies with pipenv is not working, I'm 90% sure because Apache Airflow specifies attrs package dependency at "~=19.3" somewhere in the project. Here's the error message:
ERROR: ERROR: Could not find a version that matches attrs>=17.3.0,>=17.4.0,>=20.1.0,~=19.3 Tried: 15.0.0, 15.0.0, 15.1.0, 15.1.0, 15.2.0, 15.2.0, 16.0.0, 16.0.0, 16.1.0, 16.1.0, 16.2.0, 16.2.0, 16.3.0, 16.3.0, 17.1.0, 17.1.0, 17.2.0, 17.2.0, 17.3.0, 17.3.0, 17.4.0, 17.4.0, 18.1.0, 18.1.0, 18.2.0, 18.2.0, 19.1.0, 19.1.0, 19.2.0, 19.2.0, 19.3.0, 19.3.0, 20.1.0, 20.1.0, 20.2.0, 20.2.0

What you expected to happen:

Pipenv dependencies lock without errors

How to reproduce it:
Try to add apache-airflow = {extras = ["postgres", "redis", "s3", "ssh", "crypto", "kubernetes"],version = "==1.10.12"} to Pipfile and run pipenv install

Anything else we need to know:
Here's how pipenv graph lists apache-airflow dependencies:

apache-airflow==1.10.12
  ...
  - attrs [required: ~=19.3, installed: 19.3.0]
  - cached-property [required: ~=1.5, installed: 1.5.2]

Whereas in setup.py file, https://github.com/apache/airflow/blob/master/setup.py, attrs are locked at 'attrs>=20.0, <21.0'.

Question:
I know it might pipenv-related question, but did you pin attrs package version at 19.3 anywhere in the project? And do you have any idea on how can I alter the version number so that pipenv stops complaining

In fact, I face quite a lot of issues with pipenv due to Airflow subdependencies. Is it a known issue that Airflow is not working well with Pipenv?

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[kaxil]

Duplicate of #11965

[kaxil]

This has been fixed in #11969 and will be available in Airflow 1.10.13

Thanks for reporting

[aidar-ms]

@kaxil Ok, thanks. Does the due date on November 03, 2020 here https://github.com/apache/airflow/milestone/7 mean that we should be expecting the release of 1.10.13 on November 3rd 2020? If not, when do you think the new version will be shipped?

[potiuk]

We had a planning meeting - I just moved this to the end of next week. We still have some work to do to complete it. You might expect it in 1-3 weeks to finish. Provisional date is set for 24 Nov but it might be earlier as well.

In the meantime if you follow the installation mechanism that is described in INSTALL and https://airflow.apache.org/docs/stable/installation.html everything should work just fine.

The proper way of installing airflow is with using constraints for example:

pip install \
 apache-airflow[postgres,gcp]==1.10.12 \
 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt"

[potiuk]

And if you want to read more why it is like that you can read more here: https://github.com/apache/airflow/blob/master/README.md#installing-from-pypi