[FR] Updated typing-extensions dependency

[eric-forte-elastic]

Related Issues

Summary

This PR is to address python dependency issues with typing-extensions. It appears that this issue is specific to Python 3.12. This was discovered by @ajurjevi in: #3164.

Python 3.12 Test Dockerfiles

The following Dockerfile will replicate the issue.

FROM python:3.12-bookworm
RUN apt-get update && apt-get install -y git
RUN pip install --upgrade pip
RUN git clone https://github.com/elastic/detection-rules.git siem-rules-git-sync/detection-rules/
# set directory 
WORKDIR /siem-rules-git-sync/detection-rules
RUN git pull
RUN pip install ".[dev]"
RUN python -m detection_rules dev build-release --generate-navigator

The following Dockerfile uses the fix from this PR to address the issue.

FROM python:3.12-bookworm
RUN apt-get update && apt-get install -y git
RUN pip install --upgrade pip
RUN git clone https://github.com/elastic/detection-rules.git siem-rules-git-sync/detection-rules/
# set directory 
WORKDIR /siem-rules-git-sync/detection-rules
RUN git pull && git checkout update_python_deps
RUN pip install ".[dev]"
RUN python -m detection_rules dev build-release --generate-navigator

NOTE: Known issue with typing-extensions on Python 3.12 and that upgrading breaks building packages on Python 3.8 and Python 3.9. It appears to work correctly in 3.10 and 3.11, but as the original error does not occur in those versions this PR only updates the dependency for 3.12.

Python 3.8 Error

./env/detection-rules-build/bin/python -m detection_rules dev build-release --generate-navigator
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 185, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/local/lib/python3.8/runpy.py", line 144, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/usr/local/lib/python3.8/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/siem-rules-git-sync/detection-rules/detection_rules/__init__.py", line 13, in <module>
    from . import (  # noqa: E402
  File "/siem-rules-git-sync/detection-rules/detection_rules/devtools.py", line 37, in <module>
    from .docs import IntegrationSecurityDocs, IntegrationSecurityDocsMDX
  File "/siem-rules-git-sync/detection-rules/detection_rules/docs.py", line 22, in <module>
    from .packaging import Package
  File "/siem-rules-git-sync/detection-rules/detection_rules/packaging.py", line 27, in <module>
    from .version_lock import default_version_lock
  File "/siem-rules-git-sync/detection-rules/detection_rules/version_lock.py", line 334, in <module>
    default_version_lock = VersionLock(ETC_VERSION_LOCK_PATH, ETC_DEPRECATED_RULES_PATH, name='default')
  File "/siem-rules-git-sync/detection-rules/detection_rules/version_lock.py", line 136, in __init__
    self.version_lock = VersionLockFile.load_from_file(version_lock_file)
  File "/siem-rules-git-sync/detection-rules/detection_rules/mixins.py", line 161, in load_from_file
    loaded = cls.from_dict(dict(data=contents))
  File "/siem-rules-git-sync/detection-rules/detection_rules/mixins.py", line 138, in from_dict
    schema = cls.__schema()
  File "/siem-rules-git-sync/detection-rules/detection_rules/utils.py", line 296, in wrapped
    _cache[func_key][cache_key] = f(*args, **kwargs)
  File "/siem-rules-git-sync/detection-rules/detection_rules/mixins.py", line 129, in __schema
    return marshmallow_dataclass.class_schema(cls)()
  File "/siem-rules-git-sync/detection-rules/env/detection-rules-build/lib/python3.8/site-packages/marshmallow_dataclass/__init__.py", line 369, in class_schema
    return _internal_class_schema(clazz, base_schema, clazz_frame)
  File "/siem-rules-git-sync/detection-rules/env/detection-rules-build/lib/python3.8/site-packages/marshmallow_dataclass/__init__.py", line 415, in _internal_class_schema
    attributes.update(
  File "/siem-rules-git-sync/detection-rules/env/detection-rules-build/lib/python3.8/site-packages/marshmallow_dataclass/__init__.py", line 418, in <genexpr>
    field_for_schema(
  File "/siem-rules-git-sync/detection-rules/env/detection-rules-build/lib/python3.8/site-packages/marshmallow_dataclass/__init__.py", line 712, in field_for_schema
    generic_field = _field_for_generic_type(typ, base_schema, typ_frame, **metadata)
  File "/siem-rules-git-sync/detection-rules/env/detection-rules-build/lib/python3.8/site-packages/marshmallow_dataclass/__init__.py", line 568, in _field_for_generic_type
    keys=field_for_schema(
  File "/siem-rules-git-sync/detection-rules/env/detection-rules-build/lib/python3.8/site-packages/marshmallow_dataclass/__init__.py", line 712, in field_for_schema
    generic_field = _field_for_generic_type(typ, base_schema, typ_frame, **metadata)
  File "/siem-rules-git-sync/detection-rules/env/detection-rules-build/lib/python3.8/site-packages/marshmallow_dataclass/__init__.py", line 594, in _field_for_generic_type
    [
  File "/siem-rules-git-sync/detection-rules/env/detection-rules-build/lib/python3.8/site-packages/marshmallow_dataclass/__init__.py", line 597, in <listcomp>
    field_for_schema(
  File "/siem-rules-git-sync/detection-rules/env/detection-rules-build/lib/python3.8/site-packages/marshmallow_dataclass/__init__.py", line 730, in field_for_schema
    if issubclass(typ, Enum):
TypeError: issubclass() arg 1 must be a class
make: *** [Makefile:51: release] Error 1

Testing

Use either the above Dockerfiles or your local Python 3.12 environment to duplicate the error and check the fix.

[Mikaayenson]

LGTM, note your dockerfiles in the summary are duplicates.

[eric-forte-elastic]

LGTM, note your dockerfiles in the summary are duplicates.

They are close to duplicates:
First one to replicate the issue is: RUN git pull
Second one pulls from this branch to see the issue closed: RUN git pull && git checkout update_python_deps RUN pip install ".[dev]"

[terrancedejesus]

LGTM.