Skip to content

Commit

Permalink
PEP 675.
Browse files Browse the repository at this point in the history
This commit adds shallow type-checking support for **PEP 675 – Arbitrary
Literal String Type** (i.e., `typing.LiteralString`). As PEP 675 advises
for runtime type-checkers, @beartype now reduces (i.e., aliases) all
`typing.LiteralString` type hints to the standard `str` type. Sadly,
deeply type-checking literal strings is intractable for runtime
type-checkers and is the textbook example of validation that can be
performed *only* by static type-checkers. Type-checking literal strings
requires:

* Efficiently parsing abstract syntax trees (ASTs) at runtime (which is
  technically feasible, given sufficiently aggressive caching).
* Correctly transitively inferring literal strings across operations and
  calls (which requires parsing the entire codebase of the app stack and
  constructing an in-memory graph of all type relations, which is
  pragmatically infeasible).

This isn't to say that PEP 675 is bad, however. PEP 675 is actually
*incredible*, reducing the attack surface of database injection attacks
at static analysis time with *no* harmful tradeoffs. And... who knows?
With enough intestinal fortitude, handlebar mustaches, and monocled
eyepieces, perhaps even @beartype can surmount this Hill of Thorns.
(*Nuanced dance of the prancing nougat!*)
  • Loading branch information
leycec committed Apr 22, 2023
1 parent f57b001 commit 2003998
Show file tree
Hide file tree
Showing 7 changed files with 172 additions and 18 deletions.
20 changes: 17 additions & 3 deletions beartype/_check/conv/convreduce.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
HintSignFinal,
HintSignGeneric,
# HintSignIO,
HintSignLiteralString,
HintSignNewType,
HintSignNone,
HintSignNumpyArray,
Expand Down Expand Up @@ -63,6 +64,7 @@
from beartype._util.hint.pep.proposal.utilpep591 import reduce_hint_pep591
from beartype._util.hint.pep.proposal.utilpep593 import reduce_hint_pep593
from beartype._util.hint.pep.proposal.utilpep647 import reduce_hint_pep647
from beartype._util.hint.pep.proposal.utilpep675 import reduce_hint_pep675
from beartype._util.hint.pep.utilpepget import get_hint_pep_sign_or_none
from beartype._util.hint.pep.utilpepreduce import reduce_hint_pep_unsigned
from collections.abc import Callable
Expand Down Expand Up @@ -229,9 +231,7 @@ def reduce_hint(
HintSignTypedDict: reduce_hint_pep589,

# ..................{ PEP 591 }..................
#FIXME: Remove *AFTER* deeply type-checking typed dictionaries. For now,
#shallowly type-checking such hints by reduction to untyped dictionaries
#remains the sanest temporary work-around.
#FIXME: Remove *AFTER* deeply type-checking final type hints.

# If this hint is a PEP 591-compliant "typing.Final[...]" type hint,
# silently reduce this hint to its subscripted argument (e.g., from
Expand All @@ -252,6 +252,20 @@ def reduce_hint(
# * Else, raise an exception.
HintSignTypeGuard: reduce_hint_pep647,

# ..................{ PEP 675 }..................
#FIXME: Remove *AFTER* deeply type-checking literal strings. Note that doing
#so will prove extremely non-trivial or possibly even infeasible, suggesting
#we will probably *NEVER* deeply type-check literal strings. It's *NOT*
#simply a matter of efficiently parsing ASTs at runtime; it's that as well
#as correctly transitively inferring literal strings across operations and
#calls, which effectively requires parsing the entire codebase and
#constructing an in-memory graph of all type relations. See also:
# https://peps.python.org/pep-0675/#inferring-literalstring

# If this hint is a PEP 675-compliant "typing.LiteralString" type hint,
# reduce this hint to the standard "str" type.
HintSignLiteralString: reduce_hint_pep675,

# ..................{ NON-PEP ~ numpy }..................
# If this hint is a PEP-noncompliant typed NumPy array (e.g.,
# "numpy.typing.NDArray[np.float64]"), reduce this hint to the equivalent
Expand Down
4 changes: 4 additions & 0 deletions beartype/_data/hint/pep/sign/datapepsignset.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
HintSignKeysView,
HintSignList,
HintSignLiteral,
HintSignLiteralString,
HintSignMapping,
HintSignMappingView,
HintSignMatch,
Expand Down Expand Up @@ -487,6 +488,9 @@
# ..................{ PEP 647 }..................
HintSignTypeGuard,

# ..................{ PEP 675 }..................
HintSignLiteralString,

#FIXME: Excise us up, please. This should *NOT* be required.
# ..................{ NON-PEP ~ lib : pandera }..................
# All PEP-noncompliant pandera type hints are subsequently reduced to
Expand Down
39 changes: 39 additions & 0 deletions beartype/_util/hint/pep/proposal/utilpep675.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#!/usr/bin/env python3
# --------------------( LICENSE )--------------------
# Copyright (c) 2014-2023 Beartype authors.
# See "LICENSE" for further details.

'''
Project-wide :pep:`647`-compliant **type hint** (i.e., objects created by
subscripting the :obj:`typing.Final` type hint factory) utilities.
This private submodule is *not* intended for importation by downstream callers.
'''

# ....................{ IMPORTS }....................
from beartype.typing import Type

# ....................{ REDUCERS }....................
#FIXME: Unit test us up, please.
def reduce_hint_pep675(*args, **kwargs) -> Type[str]:
'''
Reduce the passed :pep:`675`-compliant **literal string type hint** (i.e.,
the :obj:`typing.LiteralString` singleton) to the builtin :class:`str` class
as advised by :pep:`675` when performing runtime type-checking.
This reducer is intentionally *not* memoized (e.g., by the
:func:`callable_cached` decorator), as the implementation trivially reduces
to an efficient one-liner.
Parameters
----------
All passed arguments are silently ignored.
Returns
----------
Type[str]
Builtin :class:`str` class.
'''

# Unconditionally reduce this hint to the builtin "str" class.
return str
17 changes: 15 additions & 2 deletions beartype_test/a00_unit/data/hint/pep/data_pep.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,18 @@
# ....................{ TODO }....................
#FIXME: In hindsight, the structure of both this submodule and subsidiary
#submodules imported below by the _init() method is simply *ABYSMAL.* Instead:
#* Do everything below first for low-hanging fruit *NOT* widely used throughout
# our test suite. This means (in order):
# * "HINTS_PEP_IGNORABLE_DEEP".
# * "HINTS_PEP_IGNORABLE_SHALLOW".
# * "HINTS_IGNORABLE".
# * "HINTS_PEP_HASHABLE".
# * "NOT_HINTS_NONPEP".
# * "HINTS_PEP_META".
#
# In particular, avoid attempting to refactor "HINTS_PEP_META" until *AFTER*
# refactoring everything else. Refactoring "HINTS_PEP_META" will prove
# extremely time-consuming and thus non-trivial, sadly. *sigh*
#* Define one new @pytest.fixture-decorated session-scoped function for each
# public global variable currently defined below: e.g.,
# # Instead of this...
Expand All @@ -40,7 +52,7 @@
# hints_pep_ignorable_shallow_pep593 |
# ...
# )
#* In "beartype_test.a00_unit.conftest" submodule, import those fixtures to
#* In the "beartype_test.a00_unit.conftest" submodule, import those fixtures to
# implicitly expose those fixtures to all unit tests: e.g.,
# from beartype_test.a00_util.data.hint.pep.data_pep import (
# hints_pep_ignorable_shallow,
Expand Down Expand Up @@ -144,11 +156,11 @@ def _init() -> None:
_data_pep589,
_data_pep593,
_data_pep604,
_data_pep675,
)

# Submodule globals to be redefined below.
global \
HINTS_META, \
HINTS_PEP_HASHABLE, \
HINTS_PEP_IGNORABLE_DEEP, \
HINTS_PEP_IGNORABLE_SHALLOW, \
Expand All @@ -168,6 +180,7 @@ def _init() -> None:
_data_pep589,
_data_pep593,
_data_pep604,
_data_pep675,
)

# Initialize all private submodules of this subpackage.
Expand Down
22 changes: 10 additions & 12 deletions beartype_test/a00_unit/data/hint/pep/proposal/_data_pep604.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
#!/usr/bin/env python3
# --------------------( LICENSE )--------------------
# --------------------( LICENSE )--------------------
# Copyright (c) 2014-2023 Beartype authors.
# See "LICENSE" for further details.

'''
Project-wide :pep:`604`-compliant **type hint test data.**
'''

# ....................{ IMPORTS }....................
# ....................{ IMPORTS }....................
from beartype._util.py.utilpyversion import IS_PYTHON_AT_LEAST_3_10

# ....................{ ADDERS }....................
# ....................{ ADDERS }....................
def add_data(data_module: 'ModuleType') -> None:
'''
Add :pep:`604`-compliant type hint test data to various global containers
Expand All @@ -28,7 +28,7 @@ def add_data(data_module: 'ModuleType') -> None:
return
# Else, this interpreter supports PEP 604.

# ..................{ IMPORTS }..................
# ..................{ IMPORTS }..................
# Defer attribute-dependent imports.
from beartype._data.hint.pep.sign.datapepsigns import (
HintSignList,
Expand All @@ -39,11 +39,9 @@ def add_data(data_module: 'ModuleType') -> None:
HintPithSatisfiedMetadata,
HintPithUnsatisfiedMetadata,
)
from typing import (
Any,
)
from beartype.typing import Any

# ..................{ SETS }..................
# ..................{ SETS }..................
# Add PEP 604-specific deeply ignorable test type hints to this set global.
data_module.HINTS_PEP_IGNORABLE_DEEP.update((
# "|"-style unions containing any ignorable type hint.
Expand All @@ -58,10 +56,10 @@ def add_data(data_module: 'ModuleType') -> None:
complex | int | object,
))

# ..................{ TUPLES }..................
# ..................{ TUPLES }..................
# Add PEP 604-specific test type hints to this tuple global.
data_module.HINTS_PEP_META.extend((
# ................{ |-UNION }................
# ................{ |-UNION }................
# Union of one non-"typing" type and an originative "typing" type,
# exercising a prominent edge case when raising human-readable
# exceptions describing the failure of passed parameters or returned
Expand Down Expand Up @@ -118,7 +116,7 @@ def add_data(data_module: 'ModuleType') -> None:
),
),

# ................{ UNION ~ nested }................
# ................{ UNION ~ nested }................
# Nested unions exercising edge cases induced by Python >= 3.8
# optimizations leveraging PEP 572-style assignment expressions.

Expand Down Expand Up @@ -172,7 +170,7 @@ def add_data(data_module: 'ModuleType') -> None:
),
),

# ................{ UNION ~ optional }................
# ................{ UNION ~ optional }................
# Optional isinstance()-able "typing" type.
HintPepMetadata(
hint=tuple[str, ...] | None,
Expand Down
84 changes: 84 additions & 0 deletions beartype_test/a00_unit/data/hint/pep/proposal/_data_pep675.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
#!/usr/bin/env python3
# --------------------( LICENSE )--------------------
# Copyright (c) 2014-2023 Beartype authors.
# See "LICENSE" for further details.

'''
Project-wide :pep:`604`-compliant **type hint test data.**
'''

# ....................{ IMPORTS }....................
from beartype._util.py.utilpyversion import IS_PYTHON_AT_LEAST_3_11

# ....................{ ADDERS }....................
def add_data(data_module: 'ModuleType') -> None:
'''
Add :pep:`604`-compliant type hint test data to various global containers
declared by the passed module.
Parameters
----------
data_module : ModuleType
Module to be added to.
'''

# If the active Python interpreter targets Python < 3.11, this interpreter
# fails to support PEP 675. In this case, reduce to a noop.
if not IS_PYTHON_AT_LEAST_3_11:
return
# Else, this interpreter supports PEP 675.

# ..................{ IMPORTS }..................
# Defer attribute-dependent imports.
from beartype._data.hint.pep.sign.datapepsigns import (
HintSignLiteralString)
from beartype_test.a00_unit.data.hint.util.data_hintmetacls import (
HintPepMetadata,
HintPithSatisfiedMetadata,
HintPithUnsatisfiedMetadata,
)
from beartype.typing import LiteralString

# ..................{ TUPLES }..................
# Add PEP 675-specific test type hints to this tuple global.
data_module.HINTS_PEP_META.extend((
# ................{ PEP 675 }................
# Literal string type hint.
HintPepMetadata(
hint=LiteralString,
pep_sign=HintSignLiteralString,
piths_meta=(
# Literal string statically defined from a literal string.
HintPithSatisfiedMetadata(
'All overgrown with azure moss and flowers'),
# Literal string dynamically defined from the concatenation of
# multiple literal strings, inducing type inference in PEP 675.
HintPithSatisfiedMetadata(
'So sweet, ' + 'the sense faints picturing them! ' + 'Thou'
),
# Non-literal string dynamically defined from a literal integer.
# Ideally, @beartype should reject this non-literal;
# pragmatically, runtime type-checkers are incapable of doing
# so in the general case. For example, str() could have been
# shadowed by a user-defined function of the same name returning
# a valid literal string.
HintPithSatisfiedMetadata(str(0xFEEDFACE)),

# Literal byte string.
HintPithUnsatisfiedMetadata(
pith=b"For whose path the Atlantic's level powers",
# Match that the exception message raised for this object
# declares the types *NOT* satisfied by this object.
exception_str_match_regexes=(
r'\bstr\b',
),
# Match that the exception message raised for this object
# does *NOT* contain a newline or bullet delimiter.
exception_str_not_match_regexes=(
r'\n',
r'\*',
),
),
),
),
))
4 changes: 3 additions & 1 deletion doc/src/pep.rst
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ you into stunned disbelief that somebody typed all this. [#rsi]_
+------------------------+-----------------------------------------------+---------------------------+---------------------------+
| | :pep:`673 <673>` | *none* | *none* |
+------------------------+-----------------------------------------------+---------------------------+---------------------------+
| | :pep:`675 <675>` | *none* | *none* |
| | :pep:`675 <675>` | **0.13.2**\ \ *current* | *none* |
+------------------------+-----------------------------------------------+---------------------------+---------------------------+
| | :pep:`681 <681>` | *none* | *none* |
+------------------------+-----------------------------------------------+---------------------------+---------------------------+
Expand Down Expand Up @@ -362,6 +362,8 @@ you into stunned disbelief that somebody typed all this. [#rsi]_
+------------------------+-----------------------------------------------+---------------------------+---------------------------+
| | :obj:`typing.Literal` | **0.7.0**\ \ *current* | **0.7.0**\ \ *current* |
+------------------------+-----------------------------------------------+---------------------------+---------------------------+
| | :obj:`typing.LiteralString` | **0.13.2**\ \ *current* | *none* |
+------------------------+-----------------------------------------------+---------------------------+---------------------------+
| | :obj:`typing.Mapping` | **0.2.0**\ \ *current* | *none* |
+------------------------+-----------------------------------------------+---------------------------+---------------------------+
| | :obj:`typing.MappingView` | **0.2.0**\ \ *current* | *none* |
Expand Down

0 comments on commit 2003998

Please sign in to comment.