Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

database connection isn't set to UTC #1293

Closed
plup opened this issue Jun 16, 2021 · 28 comments
Closed

database connection isn't set to UTC #1293

plup opened this issue Jun 16, 2021 · 28 comments

Comments

@plup
Copy link

plup commented Jun 16, 2021

tl; dr: if you are using Django 2.2 you must use psycopg < 2.9 in your requirements file.


I just add this error after an upgrade. It's probably right about my database not being in UTC but I don't see the problem.

venv/bin/pip install -U psycopg2
Collecting psycopg2
  Downloading psycopg2-2.9.tar.gz (379 kB)
     |████████████████████████████████| 379 kB 7.2 MB/s 
Building wheels for collected packages: psycopg2
  Building wheel for psycopg2 (setup.py) ... done
  Created wheel for psycopg2: filename=psycopg2-2.9-cp36-cp36m-linux_x86_64.whl size=473570 sha256=c01f7407f6cc56a729d8693eb62133f284c8612b0ae0ef109d8bfa01ba5441d6
  Stored in directory: /home/test2/.cache/pip/wheels/70/01/46/39007954b343d1d72463a755e6a74e6106b10748fe22e2058b
Successfully built psycopg2
Installing collected packages: psycopg2
  Attempting uninstall: psycopg2
    Found existing installation: psycopg2 2.8.6
    Uninstalling psycopg2-2.8.6:
      Successfully uninstalled psycopg2-2.8.6
Successfully installed psycopg2-2.9
@dvarrazzo
Copy link
Member

Sorry, where is the error?

@coderanger
Copy link

Just spent a while debugging this, Psycopg 2.9 changed the value passed to tzinfo_factory from an int to a timedelta. Django 2.2 (possibly newer but I'm on 2.2) has a check for offset == 0 and since timedelta(0) != 0 it goes boom. Not sure what the path forward here is but that's the cause I think.

@dvarrazzo
Copy link
Member

Uhm, I checked and I thought that the change wouldn't have affected django. Looking into it...

@coderanger
Copy link

Looks like Django removed the check in 3.1 when it added support for non-UTC DB timezones. Django 3.0 is no longer supported but 2.2 is for another year-ish so if not fixing this on this side we'll need a 2.2 patch release :)

@coderanger
Copy link

@coderanger
Copy link

Arguably this is outside the purview of Django LTS support so I think you could just say "not a conflict" and maybe Django notes the compat issue in a blog post or something, since the fix of pinning to 2.8.6 until getting to Django 3 is easy :D

@dvarrazzo
Copy link
Member

I see. I can take a look. However I don't see exactly how to fix the problem.

The change was needed to support timezone with fractional minutes, otherwise timestamps in the past have an error of up to 30 sec. (#1272). The integer passed to FixedOffsetTimezone is a number of minutes so it didn't allow to use seconds, as the datetime.timezone module of Python 3.7 and following allows (of course FixedOffsetTimezone predates (datetime.timezone).

One option to fix the problem would have been to pass a number of seconds, but that would have been dangerous as it could have been misinterpreted. But because now the stdlib has a datetime object it seems right to offer the same interface.

If there is a way to fix it on our side (suggestions welcome) I'm happy to introduce a change. However if only Django 2.2 is affected I am tempted to suggest people to just stick to psycopg2 >= 2.8, < 2.9 in their requirements, as semver suggest to do, and have the issue fixed in a Django 2.2.x release, which might accommodate for the new value. Opening them a ticket and sending a MR...

@dvarrazzo
Copy link
Member

@coderanger
Copy link

Sounds good to me. Just one of the hazards of being on old versions of stuff :)

@dvarrazzo
Copy link
Member

Sounds good to me. Just one of the hazards of being on old versions of stuff :)

More an hazard of not making use of semver, IMO. This is not a change that would have been pushed in a bugfix release: people who constrained the library to the major version shouldn't have found the problem. I bet the OP has just psycopg2 in their requirements file.

@coderanger
Copy link

2.8 to 2.9 is a feature release according to semver, not a major. For semver to catch this it would have had to be 3.0 instead. My pin was ~=2.8.

@dvarrazzo
Copy link
Member

That sounds true, and this change seems indeed a semver violation, but relying on things not changing in versions which are more than a bugfix release is pretty fragile. I have had more breakages than I would like to remember assuming that nothing bad would have happened going from x.y to x.(y+1). If you don't need new features it's safer to specify >= x.y, < x.(y+1). Even more safer is to use Poetry or something similar (because breakages do happen in patch versions too).

I am looking forward to release psycopg 3, but that's a major rewrite, not a change in what I supposed to be an internal interface of an object predating the python datetime module (I don't think it was even documented that the object took an int number of minutes as constructor parameter).

@dvarrazzo
Copy link
Member

MR ready in django/django#14530. Tests pass ok, see https://djangoci.com/job/pull-requests-bionic/database=postgres,label=bionic-pr,python=python3.8/12017/console

@MotasemAghbar
Copy link

MotasemAghbar commented Jun 17, 2021

Same issue with django 2.2.x, fixed by downgrade psycopg2 to 2.8.6.

Django Error: "database connection isn't set to UTC"

@elliottb303
Copy link

@plup Was on Django 3.0.5, and AWS RDS pg DB was set to UTC.. So a strange error.

Bumping Django to 3.2 and using psyopgg2 2.9.1 is looking good so far.

Django==3.2
psycopg2-binary==2.9.1

Django trace I was having for more info..
/usr/local/lib/python3.9/site-packages/django/db/backends/postgresql/utils.py in utc_tzinfo_factory
raise AssertionError("database connection isn't set to UTC")

kofrezo added a commit to innogames/serveradmin that referenced this issue Jun 18, 2021
jeromecc added a commit to DocTocToc/doctoctocbot that referenced this issue Jun 19, 2021
terceiro added a commit to terceiro/wafer that referenced this issue Jun 20, 2021
This is needed for the CI to work before
psycopg/psycopg2#1293 is fixed somehow.
terceiro added a commit to terceiro/wafer that referenced this issue Jun 20, 2021
This is needed for the CI to work before
psycopg/psycopg2#1293 is fixed somehow.
terceiro added a commit to terceiro/wafer that referenced this issue Jun 20, 2021
This is needed for the CI to work before
psycopg/psycopg2#1293 is fixed somehow.
terceiro added a commit to terceiro/wafer that referenced this issue Jun 20, 2021
This is needed for the CI to work before
psycopg/psycopg2#1293 is fixed somehow.
bruvellu added a commit to bruvellu/cifonauta that referenced this issue Sep 26, 2021
A bug in psycopg2 newest version was causing a database connection
issue. See psycopg/psycopg2#1293 (comment)
for details.
armenzg added a commit to getsentry/sentry that referenced this issue Oct 1, 2021
For the incompatibility issue with Django visit [this issue][incompatible],

In this PR, we're going to downgrade to the compatible 2.8.6 version (thus, matching
the Python 3.6 version), removing the need to install `postgresql` (this means partially
reverting #28607) and instead installing a wheel I built on my machine that I have stored
in GC storage.

Fixes #28958

[incompatible]: psycopg/psycopg2#1293
longhotsummer added a commit to laws-africa/indigo that referenced this issue Oct 4, 2021
armenzg added a commit to getsentry/sentry that referenced this issue Oct 6, 2021
For the incompatibility issue with Django visit [this issue][incompatible],

In this PR, we're going to downgrade to the compatible 2.8.6 version (thus, matching
the Python 3.6 version), removing the need to install `postgresql` (this means partially
reverting #28607) and instead installing a wheel I built on my machine that I have stored
in GC storage.

Fixes #28958

[incompatible]: psycopg/psycopg2#1293
armenzg added a commit to getsentry/sentry that referenced this issue Oct 7, 2021
For the incompatibility issue with Django visit [this issue][incompatible],

In this PR, we're going to downgrade to the compatible 2.8.6 version (thus, matching
the Python 3.6 version), removing the need to install `postgresql` (this means partially
reverting #28607) and instead installing a wheel I built on my machine that I have stored
in GC storage.

Fixes #28958

[incompatible]: psycopg/psycopg2#1293
robhudson added a commit to mozilla/nucleus that referenced this issue Oct 8, 2021
pmac pushed a commit to mozilla/nucleus that referenced this issue Oct 13, 2021
* Bump pytest-django from 3.8.0 to 4.4.0 (from PR #449)

Bumps [pytest-django](https://github.com/pytest-dev/pytest-django) from 3.8.0 to 4.4.0.
- [Release notes](https://github.com/pytest-dev/pytest-django/releases)
- [Changelog](https://github.com/pytest-dev/pytest-django/blob/master/docs/changelog.rst)
- [Commits](pytest-dev/pytest-django@v3.8.0...v4.4.0)

* Bump psycopg2-binary from 2.8.3 to 2.9.1 (from PR #453)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.8.3 to 2.9.1.
- [Release notes](https://github.com/psycopg/psycopg2/releases)
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/commits)

* Bump pytest from 5.4.1 to 6.2.5 (from PR #479)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 5.4.1 to 6.2.5.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@5.4.1...6.2.5)

* Bump packaging from 20.1 to 21.0 (from PR #458)

Bumps [packaging](https://github.com/pypa/packaging) from 20.1 to 21.0.
- [Release notes](https://github.com/pypa/packaging/releases)
- [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst)
- [Commits](pypa/packaging@20.1...21.0)

* Bump cffi from 1.12.3 to 1.14.6 (from PR #466)

Bumps [cffi](http://cffi.readthedocs.org) from 1.12.3 to 1.14.6.

* Bump whitenoise from 5.0.1 to 5.3.0 (from PR #467)

Bumps [whitenoise](https://github.com/evansd/whitenoise) from 5.0.1 to 5.3.0.
- [Release notes](https://github.com/evansd/whitenoise/releases)
- [Changelog](https://github.com/evansd/whitenoise/blob/master/docs/changelog.rst)
- [Commits](evansd/whitenoise@v5.0.1...v5.3.0)

* Bump mozilla-django-oidc from 1.2.2 to 2.0.0 (from PR #470)

Bumps [mozilla-django-oidc](https://github.com/mozilla/mozilla-django-oidc) from 1.2.2 to 2.0.0.
- [Release notes](https://github.com/mozilla/mozilla-django-oidc/releases)
- [Changelog](https://github.com/mozilla/mozilla-django-oidc/blob/master/HISTORY.rst)
- [Commits](mozilla/mozilla-django-oidc@1.2.2...2.0.0)

* Bump django-pagedown from 2.1.3 to 2.2.1 (from PR #473)

Bumps [django-pagedown](https://github.com/timmyomahony/django-pagedown) from 2.1.3 to 2.2.1.
- [Release notes](https://github.com/timmyomahony/django-pagedown/releases)
- [Commits](timmyomahony/django-pagedown@2.1.3...2.2.1)

* Bump pluggy from 0.13.1 to 1.0.0 (from PR #477)

Bumps [pluggy](https://github.com/pytest-dev/pluggy) from 0.13.1 to 1.0.0.
- [Release notes](https://github.com/pytest-dev/pluggy/releases)
- [Changelog](https://github.com/pytest-dev/pluggy/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pluggy@0.13.1...1.0.0)

* Bump deprecated from 1.2.7 to 1.2.13 (from PR #482)

Bumps [deprecated](https://github.com/tantale/deprecated) from 1.2.7 to 1.2.13.
- [Release notes](https://github.com/tantale/deprecated/releases)
- [Changelog](https://github.com/tantale/deprecated/blob/master/CHANGELOG.rst)
- [Commits](laurent-laporte-pro/deprecated@v1.2.7...v1.2.13)

* Bump pillow from 8.0.1 to 8.3.2 (from PR #484)

Bumps [pillow](https://github.com/python-pillow/Pillow) from 8.0.1 to 8.3.2.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/master/CHANGES.rst)
- [Commits](python-pillow/Pillow@8.0.1...8.3.2)

* Bump django-jinja from 2.4.1 to 2.9.1 (from PR #485)

Bumps [django-jinja](https://github.com/niwinz/django-jinja) from 2.4.1 to 2.9.1.
- [Release notes](https://github.com/niwinz/django-jinja/releases)
- [Changelog](https://github.com/niwinz/django-jinja/blob/master/CHANGES.adoc)
- [Commits](niwinz/django-jinja@2.4.1...2.9.1)

* Bump sqlparse from 0.3.0 to 0.4.2 (from PR #487)

Bumps [sqlparse](https://github.com/andialbrecht/sqlparse) from 0.3.0 to 0.4.2.
- [Release notes](https://github.com/andialbrecht/sqlparse/releases)
- [Changelog](https://github.com/andialbrecht/sqlparse/blob/master/CHANGELOG)
- [Commits](andialbrecht/sqlparse@0.3.0...0.4.2)

* Bump more-itertools from 8.2.0 to 8.10.0 (from PR #489)

Bumps [more-itertools](https://github.com/more-itertools/more-itertools) from 8.2.0 to 8.10.0.
- [Release notes](https://github.com/more-itertools/more-itertools/releases)
- [Commits](more-itertools/more-itertools@v8.2.0...v8.10.0)

* Bump django-filter from 2.2.0 to 21.1 (from PR #490)

Bumps [django-filter](https://github.com/carltongibson/django-filter) from 2.2.0 to 21.1.
- [Release notes](https://github.com/carltongibson/django-filter/releases)
- [Changelog](https://github.com/carltongibson/django-filter/blob/main/CHANGES.rst)
- [Commits](carltongibson/django-filter@2.2.0...21.1)

* Bump pytz from 2019.3 to 2021.3 (from PR #498)

Bumps [pytz](https://github.com/stub42/pytz) from 2019.3 to 2021.3.
- [Release notes](https://github.com/stub42/pytz/releases)
- [Commits](stub42/pytz@release_2019.3...release_2021.3)

* Bump python-decouple from 3.1 to 3.5 (from PR #499)

Bumps [python-decouple](https://github.com/henriquebastos/python-decouple) from 3.1 to 3.5.
- [Release notes](https://github.com/henriquebastos/python-decouple/releases)
- [Changelog](https://github.com/henriquebastos/python-decouple/blob/master/CHANGELOG.md)
- [Commits](HBNetwork/python-decouple@3.1...3.5)

* Add toml and iniconfig for updated pytest

* Bump Jinja2 and dep after django-jinja update

* Upgrade to latest Django 2.2.x

* Downgrade psycopg2-binary

See psycopg/psycopg2#1293

* Bump py from 1.8.1 to 1.10.0 (from PR #352)

Bumps [py](https://github.com/pytest-dev/py) from 1.8.1 to 1.10.0.
- [Release notes](https://github.com/pytest-dev/py/releases)
- [Changelog](https://github.com/pytest-dev/py/blob/master/CHANGELOG.rst)
- [Commits](pytest-dev/py@1.8.1...1.10.0)

Signed-off-by: dependabot-preview[bot] <[email protected]>

* Bump contextlib2 from 0.5.5 to 21.6.0 (from PR #455)

Bumps [contextlib2](https://github.com/jazzband/contextlib2) from 0.5.5 to 21.6.0.
- [Release notes](https://github.com/jazzband/contextlib2/releases)
- [Changelog](https://github.com/jazzband/contextlib2/blob/master/NEWS.rst)
- [Commits](jazzband/contextlib2@v0.5.5...21.6.0)

* Bump six from 1.12.0 to 1.16.0

* Bump typing from 3.7.4 to 3.7.4.3 (from PR #246)

Bumps [typing](https://github.com/python/typing) from 3.7.4 to 3.7.4.3.
- [Release notes](https://github.com/python/typing/releases)
- [Commits](python/typing@3.7.4...3.7.4.3)

Signed-off-by: dependabot-preview[bot] <[email protected]>

* Bump flake8 from 3.7.9 to 3.9.2 (from PR #429)

Bumps [flake8](https://gitlab.com/pycqa/flake8) from 3.7.9 to 3.9.2.
- [Release notes](https://gitlab.com/pycqa/flake8/tags)
- [Commits](https://gitlab.com/pycqa/flake8/compare/3.7.9...3.9.2)

Signed-off-by: dependabot[bot] <[email protected]>

* Bump spinach from 0.0.11 to 0.0.14 (from PR #428)

Bumps [spinach](https://github.com/NicolasLM/spinach) from 0.0.11 to 0.0.14.
- [Release notes](https://github.com/NicolasLM/spinach/releases)
- [Commits](NicolasLM/spinach@v0.0.11...v0.0.14)

Signed-off-by: dependabot[bot] <[email protected]>

* Bump pbr from 5.4.4 to 5.6.0 (from PR #423)

Bumps [pbr](https://docs.openstack.org/pbr/latest/) from 5.4.4 to 5.6.0.

Signed-off-by: dependabot[bot] <[email protected]>

* Bump gunicorn from 19.9.0 to 20.1.0 (from PR #408)

Bumps [gunicorn](https://github.com/benoitc/gunicorn) from 19.9.0 to 20.1.0.
- [Release notes](https://github.com/benoitc/gunicorn/releases)
- [Commits](benoitc/gunicorn@19.9.0...20.1.0)

Signed-off-by: dependabot-preview[bot] <[email protected]>

* Bump pycodestyle from 2.5.0 to 2.7.0

* Bump attrs from 19.3.0 to 21.2.0

* Bump django-cache-url from 3.0.0 to 3.2.3

* Bump django-crum from 0.7.5 to 0.7.9

* Bump django-csp from 3.5 to 3.7

* Bump django-extensions from 2.2.8 to 3.1.3

* Bump django-watchman from 0.17.0 to 1.2.0

* Bump djangorestframework from 3.11.0 to 3.12.4

* Bump zipp from 3.1.0 to 3.6.0

* Bump wrapt from 1.12.1 to 1.13.1

* Bump josepy from 1.2.0 to 1.10.0, plus dependencies

* Bump PyGithub from 1.46 to 1.55, including dependencies

* Bump importlib-metadata from 1.5.0 to 4.8.1

* Bump pyparsing from 2.4.6. to 2.4.7

* Bump redis from 3.4.1 to 3.5.3

* Bump urlwait from 0.4 to 1.0

* Remove unused wcwidth dependency

* Add charset-normalizer 2.0.6 as a new dependency of requests

* Add PyNaCl 1.4.0 as a new dependency of PyGithub

* Add typing-extensions 3.10.0.2 as a new dependency of importlib-metadata

* Bump pyflakes from 2.1.1 to 2.4.0

* Remove unused enum34 dependency

* Bump netaddr from 0.7.19 to 0.8.0

* Remove unused configparser dependency

* Bump meinheld from 1.0.1 to 1.0.2, plus dependencies

* Remove unused ipaddress dependency

* Downgrade pyflakes to satisfy flake8 dependency requirements

* Downgrade greenlet to satisfy meinheld requirements

* Fix github tests by passing `headers` positional arg

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
camallen added a commit to zooniverse/hamlet that referenced this issue Oct 14, 2021
ensure we can still access the DB in Django 2.X LTS releases psycopg/psycopg2#1293
camallen added a commit to zooniverse/hamlet that referenced this issue Oct 14, 2021
ensure we can still access the DB in Django 2.X LTS releases psycopg/psycopg2#1293
jtriley added a commit to nerc-project/coldfront-nerc that referenced this issue Nov 1, 2021
ushkarev added a commit to ministryofjustice/money-to-prisoners-api that referenced this issue Nov 10, 2021
@elkd
Copy link

elkd commented Nov 20, 2021

Same issue with django 2.2.x, fixed by downgrade psycopg2 to 2.8.6.

Django Error: "database connection isn't set to UTC"

Just to highlight the conclusion for this issue, if you're on Django >=2.2.x, <3, you SHOULD PIN to Psycopg2 == 2.8.6.
Nah fix for ya!

@psycopg psycopg locked as resolved and limited conversation to collaborators Nov 20, 2021
paulhfischer pushed a commit to paulhfischer/skriptentool that referenced this issue Feb 13, 2022
ipadla added a commit to ipadla/foodgram that referenced this issue Aug 2, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants