[MAINTENANCE] Pydantic 2.0 support #8604
Conversation
✅ Deploy Preview for niobium-lead-7998 canceled.
|
| - run: invoke fmt --check | ||
| - name: Marker-coverage-check | ||
| run: | | ||
| invoke marker-coverage | ||
| - name: Type-check | ||
| run: | | ||
| invoke type-check --ci --pretty | ||
| invoke type-check --ci --pretty --check-stub-sources | ||
| - name: Marker-coverage-check | ||
| run: invoke marker-coverage |
There was a problem hiding this comment.
I swapped the order of these to get faster feedback on typing issues
| @@ -405,6 +404,24 @@ jobs: | |||
| - name: Run the tests | |||
| run: invoke ci-tests -m unit --xdist --slowest=10 --timeout=2.0 | |||
|
|
|||
| pydantic-v1: | |||
There was a problem hiding this comment.
Added a test for our min version of pydantic now that the other tests will be installing v2
| # this is higher than our min version but is the earliest version that actually works | ||
| # TODO: update our min version or fix gx for our current min version | ||
| pydantic==1.10.8 |
There was a problem hiding this comment.
Our min version of pydantic in our requirements file is 1.9 but gx actually breaks at import time on this version, and it looks like we were never testing against this version in our CI.
This does not appear related to moving to pydantic v2.
I will create followup tickets for taking the action here.
| from pydantic.v1 import ( | ||
| AnyUrl, | ||
| UrlError, | ||
| error_wrappers, | ||
| errors, | ||
| fields, | ||
| generics, | ||
| json, | ||
| networks, | ||
| schema, | ||
| typing, | ||
| ) |
There was a problem hiding this comment.
How does it work if I'm creating a V2 Base Model and I want to use one of these classes? I guess we can cross that bridge when we get there.
There was a problem hiding this comment.
That won't be possible yet, without some conditional imports.
We would need some kind of pydantic_v2 compatibility module that works like our other optional dependencies (sqlalchemy).
Unless we just decide to drop pydantic v1, but I think we need to let the ecosystem catch up before we can do that.
| @@ -2,7 +2,7 @@ black[jupyter]==23.3.0 # Linting / code style | |||
| Click>=7.1.2 # CLI tooling | |||
| cookiecutter==2.1.1 # Project templating | |||
| mypy==1.5.1 # Type checker | |||
| pydantic>=1.0,<2.0 # Needed for mypy plugin | |||
| pydantic>=1.0 # Needed for mypy plugin | |||
There was a problem hiding this comment.
Actually, I don't understand why we have this entry in cli/requirements at all.
It being in our core requirements should be enough.
Pull in changes from great-expectations#8604 MR to enable pydantic v2 compat for 0.15.50 I dropped the code changes that corresponded to net-new changes since 0.15.50, and also added the re-pointed imports to other locations that imported from pydantic. There was one import of typing_extensions.override that I also pulled in the compatibility change for, as compatibility/pydantic.py imports compatibility/not_imported.py, which imports compatibility/typing_extensions.py. Co-authored-by: Gabriel <[email protected]>
Pull in changes from great-expectations#8604 MR to enable pydantic v2 compat for 0.15.50 I dropped the code changes that corresponded to net-new changes since 0.15.50, and also added the re-pointed imports to other locations that imported from pydantic. There was one import of typing_extensions.override that I also pulled in the compatibility change for, as compatibility/pydantic.py imports compatibility/not_imported.py, which imports compatibility/typing_extensions.py. Co-authored-by: Gabriel <[email protected]>
Add pydantic compatibility module and update imports
Migration guide for reference:
https://docs.pydantic.dev/dev-v2/migration/
Note: this PR does not aim to fully migrate to pydantic v2 patterns but simply to remove the upper bound on v2 so that downstream dependencies and users can install pydantic v2 along with
great_expectations.invoke lint(usesblack+ruff)