chore: update python deps to breaking change breakpoints #14420
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## chore_release-7.2.0 #14420 +/- ##
=======================================================
- Coverage 68.12% 67.77% -0.36%
=======================================================
Files 2518 2518
Lines 72031 71972 -59
Branches 9243 9245 +2
=======================================================
- Hits 49073 48779 -294
- Misses 20755 20991 +236
+ Partials 2203 2202 -1
Flags with carried forward coverage won't be shown. Click here to find out more.
|
In 4.18 which you may note is not a major version change jsonschema split off all of its reference management stuff into an independent library and kept some backwards compatibility shims which... aren't. So limit it to <4.18 and set 4.17.3 in the pipfile to prevent having to change a bunch of stuff about how we do internal referencing.
Some typing changes, a couple test changes. protocol api old tests seemed broken. The command schema weirdly needs to be updated.
The difference here is the switch from anyOf to oneOf for the command create, which seems to make no difference since only one element can be present.
- pydantic 1.10 (last before 2's breaking changes) - fastapi 0.99 (last before it focused on pydantic 2) - mypy 1.8 - pytest 7.2 (tavern can't tolerate >7.3, even latest tavern) Fixes from all the above. Pretty minor, but there were two annoying ones and a weird one. Annoying: - Fastapi started trying to shove all responses into pydantic fields if you didn't provide a response= value to tis route method. pydantic doesn't want to put our PydanticResponses in fields. Fix: specify the response model _contained_ in the PydanticResponse in the route handler. This is annoying, and luckily we now have paramspec, so make a decorator decorator that encompasses both the actual endpoint method and the route method call so you don't have to remember. - Fastapi's openapi integration got more specific about requiring examples to be exactly jsonschema example keywords, which are lists of objects that don't have associated docs when you're putting examples in a schema component Weird: Fastapi started properly noting that filenames are optional in file upload bodies, and also specifies BytesIO, and mypy finally noticed that mutable attributes of Protocols must be invariant per the PEP, so we needed to change some api-side type descriptors in the protocol file readers. No functinoality change, just wanted to mention because that's why this commit touches api.
Mypy 1.8, pytest 7.4.4. Minimal changes.
aiohttp 3.9, mypy 1.8, pytest 7.4
mypy 1.8, diff match 2023, pytest 7.4.4
sfoster1
force-pushed
the
bump-py-deps
branch
from
9aa3062 to
87a886e
Compare
|
Works on flex with Opentrons/oe-core#132 (build: https://builds.opentrons.com/ot3-oe/7790572076/ot3-system.zip ) |
|
Works on ot-2 with Opentrons/buildroot#220 (build: https://builds.opentrons.com/ot2-br/7807327676/ot2-system.zip ) |
CaseyBatten
reviewed
Feb 7, 2024
vegano1
approved these changes
Feb 7, 2024
Comment on lines
-77
to
+79
| if self._state == ExecutionState.CANCELLED: | ||
| # type-ignore needed because this is a reentrant function and narrowing cannot | ||
| # apply | ||
| if self._state == ExecutionState.CANCELLED: # type: ignore[comparison-overlap] |
There was a problem hiding this comment.
Oof, this seems like bad mypy behavior.
This was referenced Feb 7, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a PRs to update python dependencies to modern versions. This one stops at the major breaking change point of pydantic 2.0. While Pydantic 2.0 is apparently much much faster (it's written in rust if you can believe it) it also makes a bunch of breaking api changes and in theory provides back-compat patches. Similarly, fastapi 0.100.0 switches internally to depend on pydantic 2, with similarly theoretical back compat patches. So, we've updated to,
Dependency Version Changes
In prod, major deps (full list is essentially entirely in the robot-server pipfile, see below for why):
In dev tooling,
Version Definition Changes
Some people want to use our python libraries that are published on pypi. The problem with this is that our libraries -
opentronsandopentrons_shared_datahave explicit version pinning in their setup.pyinstall_requires, which makes it incredibly hard for them to coexist with other python packages that aren't comaintained by us (see #11912 , #12839 ). One way to fix this would be version ranges in the setup.py and explicit versions (that are contained within those ranges, and that match or define what's present on the robot) in the pipfiles. We couldn't do this because pipenv had problems with it.Now, however, we've upgraded pipenv, and that strategy works! And since I was going around bumping all the deps anyway, I could figure out what the actual functional dependency version boundaries were. So as part of this,
opentrons(api/setup.py) andopentrons_shared_data(shared-data/python/setup.py) now have version ranges for all of theirinstall_requiresthat aren't other opentrons packages, and I'm pretty sure about those version ranges. They may be smaller than would be ideal, but they're real.Testing (and how to get this out of draft)
This PR should stay in draft until the system builders are known to have the right dependency versions.