Improve mypy typing performance

[cdce8p]

Ref: python/mypy#14627

Trying to run mypy with pydantic 2.0.0.dev.0 hangs when trying to lint

# test.py
from pydantic import BaseModel

The likely reason for that is the CoreSchema type. Mypy can't handle unions with a lot of items that well. Especially for TypedDicts, it tries to find a common base type but when there is none, checks the compatibility of each keys individually.

That's even more of an issue as there are also some recursive annotations with CoreSchema.

pydantic-core/pydantic_core/core_schema.py

Lines 2547 to 2588 in bfb373c

	CoreSchema = Union[
	AnySchema,
	NoneSchema,
	BoolSchema,
	IntSchema,
	FloatSchema,
	StringSchema,
	BytesSchema,
	DateSchema,
	TimeSchema,
	DatetimeSchema,
	TimedeltaSchema,
	LiteralSchema,
	IsInstanceSchema,
	IsSubclassSchema,
	CallableSchema,
	ListSchema,
	TuplePositionalSchema,
	TupleVariableSchema,
	SetSchema,
	FrozenSetSchema,
	GeneratorSchema,
	DictSchema,
	FunctionSchema,
	FunctionWrapSchema,
	FunctionPlainSchema,
	WithDefaultSchema,
	NullableSchema,
	UnionSchema,
	TaggedUnionSchema,
	ChainSchema,
	LaxOrStrictSchema,
	TypedDictSchema,
	ModelSchema,
	ArgumentsSchema,
	CallSchema,
	RecursiveReferenceSchema,
	CustomErrorSchema,
	JsonSchema,
	UrlSchema,
	MultiHostUrlSchema,
	]

pydantic-core/pydantic_core/core_schema.py

Lines 1724 to 1726 in bfb373c

	class UnionSchema(TypedDict, total=False):
	type: Required[Literal['union']]
	choices: Required[List[CoreSchema]]

To improve mypy performance, I added an empty CoreSchemaBase TypedDict type from which all CoreSchema types can inherit and replaced the recursive CoreSchema types. With that a mypy run is back down to 15-20s.

time mypy --no-incremental test.py

[cdce8p]

CI issues seems to be related to cargo. The branch probably needs to be changed to just release-0.18.

pydantic-core/Cargo.toml

Line 30 in bfb373c

pyo3 = { git = "https://github.com/PyO3/pyo3", branch = "release-0.18.1" }

pydantic-core/Cargo.toml

Line 71 in bfb373c

pyo3-build-config = { git = "https://github.com/PyO3/pyo3", branch = "release-0.18.1" }

[samuelcolvin]

See also python/mypy#14034.

V0.18.1 of pyo3 was released late last night, so we need to.update cargo.

But the bigger question is: does this worth generate_core_schema.py - remember CoreSchema is consumed and sure to build a validator.

[codecov-commenter]

Codecov Report

Merging #387 (54f4a48) into main (bfb373c) will increase coverage by 0.00%.
The diff coverage is 100.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #387   +/-   ##
=======================================
  Coverage   95.31%   95.31%           
=======================================
  Files          90       90           
  Lines       10510    10512    +2     
  Branches        8        8           
=======================================
+ Hits        10018    10020    +2     
  Misses        492      492           

Impacted Files	Coverage Δ
pydantic_core/core_schema.py	98.78% <100.00%> (+<0.01%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bfb373c...54f4a48. Read the comment docs.

[codspeed-hq]

CodSpeed Performance Report

Merging #387 improve-typing-performance (54f4a48) will not alter performances.

Summary

🔥 0 improvements
❌ 0 regressions
✅ 85 untouched benchmarks

🆕 0 new benchmarks
⁉️ 0 dropped benchmarks

[cdce8p]

Seems like this won't type check, unfortunately 😕
It was worth a shot.