[CT-2225] [Bug] Suddenly getting ModuleNotFoundError: No module named 'pytz'

[spacecowboy]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

I am installing dbt-bigquery with meltano (which installs it in a isolated venv).

Today when invoking dbt deps using meltano invoke dbt-bigquery:deps I am getting a stacktrace with

ModuleNotFoundError: No module named 'pytz'

Expected Behavior

pytz should be found. I have noted that it is not included in the requirements. So while it's strange that it suddenly started failing, maybe it was more of an accident that it ever worked in the first place?

Steps To Reproduce

With versions specified as

dbt-core~=1.3.0
dbt-bigquery~=1.3.0

invoking dbt deps should not throw a ModuleNotFoundError

Relevant log output

Traceback (most recent call last):
  File "/workspaces/elt/.meltano/transformers/dbt-bigquery/venv/bin/dbt", line 5, in <module>
    from dbt.main import main
  File "/workspaces/elt/.meltano/transformers/dbt-bigquery/venv/lib/python3.9/site-packages/dbt/main.py", line 24, in <module>
    import dbt.task.build as build_task
  File "/workspaces/elt/.meltano/transformers/dbt-bigquery/venv/lib/python3.9/site-packages/dbt/task/build.py", line 1, in <module>
    from .run import RunTask, ModelRunner as run_model_runner
  File "/workspaces/elt/.meltano/transformers/dbt-bigquery/venv/lib/python3.9/site-packages/dbt/task/run.py", line 8, in <module>
    from .compile import CompileRunner, CompileTask
  File "/workspaces/elt/.meltano/transformers/dbt-bigquery/venv/lib/python3.9/site-packages/dbt/task/compile.py", line 4, in <module>
    from .runnable import GraphRunnableTask
  File "/workspaces/elt/.meltano/transformers/dbt-bigquery/venv/lib/python3.9/site-packages/dbt/task/runnable.py", line 11, in <module>
    from .printer import (
  File "/workspaces/elt/.meltano/transformers/dbt-bigquery/venv/lib/python3.9/site-packages/dbt/task/printer.py", line 22, in <module>
    from dbt.tracking import InvocationProcessor
  File "/workspaces/elt/.meltano/transformers/dbt-bigquery/venv/lib/python3.9/site-packages/dbt/tracking.py", line 25, in <module>
    import pytz
ModuleNotFoundError: No module named 'pytz'

Environment

- OS: Linux (fresh docker container inside virtual environment)
- Python: 3.9
- dbt: 1.3.1 (~=1.3.0)

Which database adapter are you using with dbt?

other (mention it in "Additional Context")

Additional Context

No response

[spacecowboy]

Workaround I've found is to specify pytz explicitly in the meltano config as so:

  transformers:
    - name: dbt-bigquery
      variant: dbt-labs
      pip_url: dbt-core~=1.3.0 dbt-bigquery~=1.3.0 pytz==2021.1

but ideally dbt should define it's dependencies properly

[lrousset]

Facing this as well went with the same fix.

[wojciech-mazur-mlg]

If I see correctly it's because of the Babel package (included indirectly by agate). They've just released version 2.12 which incudes pytz only if you have python < 3.9.
https://github.com/python-babel/babel/blob/v2.12.0/setup.py

# Python 3.9 and later include zoneinfo which replaces pytz
'pytz>=2015.7; python_version<"3.9"',

I think if dbt uses pytz directly it should specify it explicitly in reqs.
Pinning Babel to ver 2.11.0 should also work.

[jtcohen6]

Thanks for the report, all. It sounds like the fix here is to add pytz as an explicit dependency to setup.py. We'll need to fix this and backport it to all currently supported versions.

Separately, we should look into whether we ought to replace pytz with zoneinfo for Python 3.9+. I added a new thread in this disucssion: #6915 (comment)

[WittierDinosaur]

Don't suppose you could also backport to 1.0?

[joellabes]

Don't suppose you could also backport to 1.0?

@WittierDinosaur we provide critical fixes for minor versions of Core for 1 year from their release date, and v1.0 came out in December 2021 so is no longer covered: https://docs.getdbt.com/docs/dbt-versions/core

[kaleynguyen]

I also had this bug too when trying to install DBT using the Docker file from the DBT site and docker-compose. I found a way to fix it for the moment but I think the bug comes from the Dockerfile.

First, you need to use docker with the build kit like this ‘DOCKER_BUILDKIT=1 docker build .’ Then alter the Dockerfile with a simple run command ‘RUN python -m pip install dbt-core’ or any other pip install that you need.