Airflow fails to initdb with cattrs 1.1.0

[tdbabas]

Apache Airflow version:

1.10.12

Kubernetes version (if you are using kubernetes) (use kubectl version):

N/A

Environment:

• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release): Centos 7
• Kernel (e.g. uname -a): 3.10.0-229.el7.x86_64
• Install tools: pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6)
• Others:

What happened:

Following the instructions here, I encountered an issue at the airflow initdb stage:

[xxx@xxx ~]# airflow initdb
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 26, in <module>
    from airflow.bin.cli import CLIFactory
  File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 94, in <module>
    api_module = import_module(conf.get('cli', 'api_client'))  # type: Any
  File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/local/lib/python3.6/site-packages/airflow/api/client/local_client.py", line 24, in <module>
    from airflow.api.common.experimental import delete_dag
  File "/usr/local/lib/python3.6/site-packages/airflow/api/common/experimental/delete_dag.py", line 26, in <module>
    from airflow.models.serialized_dag import SerializedDagModel
  File "/usr/local/lib/python3.6/site-packages/airflow/models/serialized_dag.py", line 35, in <module>
    from airflow.serialization.serialized_objects import SerializedDAG
  File "/usr/local/lib/python3.6/site-packages/airflow/serialization/serialized_objects.py", line 28, in <module>
    import cattr
  File "/usr/local/lib/python3.6/site-packages/cattr/__init__.py", line 1, in <module>
    from .converters import Converter, GenConverter, UnstructureStrategy
  File "/usr/local/lib/python3.6/site-packages/cattr/converters.py", line 16, in <module>
    from attr import fields, resolve_types
ImportError: cannot import name 'resolve_types'

What you expected to happen:

I expected the Airflow DB to be initialised as per the instructions.

How to reproduce it:

On a fresh installation, follow the Quick Start guide until the apache initdb stage.

Anything else we need to know:

Investigation suggests that this was caused by a Python dependency; specifically cattrs==1.1.0, which was released yesterday (2020-10-29). Downgrading cattrs manually to 1.0.0 does fix the issue and allows the Airflow database to be initialised:

[xxx@xxx ~]# pip3 install cattrs==1.0.0
Collecting cattrs==1.0.0
  Downloading https://files.pythonhosted.org/packages/17/5b/6afbdaeb066ecf8ca28d85851048103ac80bb169491a54a14bd39823c422/cattrs-1.0.0-py2.py3-none-any.whl
Requirement already satisfied: attrs>=17.3 in /usr/local/lib/python3.6/site-packages (from cattrs==1.0.0)
Installing collected packages: cattrs
  Found existing installation: cattrs 1.1.0
    Uninstalling cattrs-1.1.0:
      Successfully uninstalled cattrs-1.1.0
Successfully installed cattrs-1.0.0
[xxx@xxx ~]# airflow initdb            
DB: sqlite:////path/to/airflow.db
[2020-10-30 09:36:05,431] {db.py:378} INFO - Creating tables
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> e3a246e0dc1, current schema
INFO  [alembic.runtime.migration] Running upgrade e3a246e0dc1 -> 1507a7289a2f, create is_encrypted
/usr/local/lib/python3.6/site-packages/alembic/ddl/sqlite.py:44: UserWarning: Skipping unsupported ALTER for creation of implicit constraintPlease refer to the batch mode feature which allows for SQLite migrations using a copy-and-move strategy.
...
Done.

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[cgddrd]

Can confirm we're experiencing this as part of our production stack (auto scaling actions are now failing when trying to install 1.10.12) - a hotfix would be greatly appreciated.

[kaxil]

You can use the following for now, this will be fixed in 1.10.13

pip install apache-airflow==1.10.12 \
 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt"

[cgddrd]

Many thanks @kaxil

[kaxil]

#11969 -- This PR will be included in 1.10.13 and should fix this issue too

[jodmoreira]

You can use the following for now, this will be fixed in 1.10.13

pip install apache-airflow==1.10.12 \
 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt"

It worked! Thanks!

[zachliu]

@jodmoreira oh how i envy you, the constraint file constraints-3.8.txt doesn't work for my 1.10.12 😿 #8467 (comment)

[kaxil]

@jodmoreira oh how i envy you, the constraint file constraints-3.8.txt doesn't work for my 1.10.12 😿 #8467 (comment)

Just update both attrs and cattrs that too should work.

or just downgrade cattrs

[potiuk]

Yeah. Constraints file does not work if you already have higher version installed.

[zachliu]

@kaxil @potiuk ah, i'm sorry i didn't describe it clearly 😅
the constraint file constraints-3.8.txt did solve the attr and/or cattrs issue, but in order to use it i had to do the following changes due to #8467 (comment):

original:

oauthlib==3.1.0
requests-oauthlib==1.3.0

my updates:

oauthlib==2.1.0
requests-oauthlib==1.1.0

[dark-spark2]

@kaxil can I change attrs to higher version( attrs==20.1.0 ) for airflow 1.10.12?

[0xCakin]

You can use the following for now, this will be fixed in 1.10.13

pip install apache-airflow==1.10.12 \
 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt"

Thanks a lot!

[mmgithub123]

great!
I have the some question，
when i use pip3 install cattrs==1.0.0.
fix my problem

[gaj995]

You can use the following for now, this will be fixed in 1.10.13

pip install apache-airflow==1.10.12 \
 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt"

Didn't work for me :(

[potiuk]

Didn't work for me :(

What do you mean by 'didn't work"? Can you please provide some specifics on what you tried and how it did not work @gaj995 ?

[potiuk]

I'd recommend Airflow adopts the same policy because this philosophy works great for me.

Unfortunately, It is beyond of our control when it comes to our users. We have no physical capacity to force our users to use our preferred installation method. So we better try it ourselves and . We have to assume our users installed latest PIP (which the cannot for now as stated in #12838 which is currently. And in many toolchains where Airflow is installed people build Airflow Images automatically and running the PyPI pip. Including our image which is currently pinned to 20.2.4.

Beware of cattrs 1.n.n hell with Python 3.9 (i.e. archlinux).

Yep. This is one of the reasons why Python 3.9 is not officially supported yet by Airflow

[sephethus]

Nothing works in this entire thread, nothing. I'm using docker-compose up to create a local instance and it all fails. 1.10.13, adding cattrs and attrs and upping the version of oauthlib and requests-oauthlib, nothing works, period. What now?

There is a known bug with the latest PiP that was released two days ago. You probably hit the problem described here: pypa/pip#9203

The new version of PIP has MANY problems and broke a lot of other projects. For now the best course of action is to downgrade Pip to 20.2.4 version (this is what we did in official docker images and this is what is recommended in 1.10.14 documentation (release candidate will be there shortly soon).

According to my pip3 --version I'm on 20.0.2 so that can't be the issue.

[potiuk]

Are you trying to develop Airflow or run it for "production" on MacOS? We do not support MacOS as "execution" environment" - only Linux systems are supported for 'running' Airflow', MacOS in our case is only supported as development environment.

It looks like a Mac-only thing, so if you are using MacOS to develop airflow and you can use dockerised development environment instead, while we manage to pin-point it? (.

On MacOS we also have Breeze that provides a fully-featured docker-based dev environment that works flawlessly with MacOS as well. We fully support it and there is plenty of documentation and movies on how to use Breeze for development purpose including 'start-airflow' command that automatically spins of running airflow scheduler + webserver + command line from your local sources.

BTW. If you have migrated to BigSur, this might be the root cause of the problem. Usually the "POSIX" development environment has lots of problems with the new releases of MacOS. It usually takes couple of months and few releases of patches + some releases of brew and other dev environments to get it working. Again Breeze is your best bet there as it is "stable" environment. As on as you have Docker and Docker-compose, it will work on MacOS/Linux and even WSL2 for Windows in similar way.