Update pre-commit hooks and drop support for Python 3.6 & 3.7 (#1109)

* Update pre-commit hooks

* Update dev requirements to recent enough versions of code check tools

* Update mypy Python version to match recommended dev version (3.8)

* Drop support for Python 3.6 and Python 3.7

* Enforce that dev linters match their pre-commit version

* Bump dev version (just in case we would want to distinguish before vs. after dropping
support for Python 3.6 / 3.7)

* Add news fragment

* Remove useless `# pragma: no cover`

.circleci/config.yml

@@ -39,4 +39,4 @@ workflows:
       - test_linux:
           matrix:
             parameters:
-              py_version: ["3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]
+              py_version: ["3.8", "3.9", "3.10", "3.11"]

.pre-commit-config.yaml

@@ -1,24 +1,26 @@
+default_language_version:
+      python: python3.8
+
+# Hook versions should match those in requirements/dev.txt
 repos:
-  - repo: https://github.com/timothycrosley/isort
-    rev: 5.0.9
+  - repo: https://github.com/PyCQA/isort
+    rev: 5.12.0
     hooks:
       - id: isort
 
   - repo: https://github.com/psf/black
-    rev: 20.8b1
+    rev: 23.7.0
     hooks:
       - id: black
-        language_version: python3.8
 
-  - repo: https://gitlab.com/pycqa/flake8
-    rev: 3.7.9
+  - repo: https://github.com/PyCQA/flake8
+    rev: 6.0.0
     hooks:
       - id: flake8
-        additional_dependencies: [-e, 'git+https://github.com/pycqa/pyflakes.git@1911c20#egg=pyflakes']
 
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v0.931
+    rev: v1.4.1
     hooks:
       - id: mypy
         args: [--strict]
-        additional_dependencies: ['pytest']
+        additional_dependencies: ['attrs', 'pytest']

.readthedocs.yaml

@@ -16,7 +16,7 @@ formats:
 
 # Optionally set the version of Python and requirements required to build your docs
 python:
-  version: 3.7
+  version: 3.8
   install:
     - method: pip
       path: .

CONTRIBUTING.md

@@ -15,11 +15,11 @@ Optionally install commit hooks: `pre-commit install`
 pre-commit will verify your code lints cleanly when you commit. You can use `git commit -n` to skip the pre-commit hook for a specific commit.
 
 ##### Mac
-OmegaConf is compatible with Python 3.6.4 and newer. Unfortunately Mac comes with older versions.
+OmegaConf is compatible with Python 3.8 and newer.
 
 One way to install multiple Python versions on Mac to to use pyenv.
 The instructions [here](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/MAC_SETUP.md)
-will provide full details. It shows how to use pyenv on mac to install multiple versions of Python and have
+will provide full details. It shows how to use pyenv on Mac to install multiple versions of Python and have
 pyenv make specific versions available in specific directories automatically.
 This plays well with Conda, which supports a single Python version. Pyenv will provide the versions not installed by Conda (which are used when running nox).
 
@@ -31,24 +31,16 @@ Run all CI tests with nox:
 ```
 $ nox -l
 Sessions defined in /home/omry/dev/omegaconf/noxfile.py:
-* omegaconf-3.6
-* omegaconf-3.7
 * omegaconf-3.8
 * omegaconf-3.9
 * omegaconf-3.10
 * docs
-* coverage-3.6
-* coverage-3.7
 * coverage-3.8
 * coverage-3.9
 * coverage-3.10
-* lint-3.6
-* lint-3.7
 * lint-3.8
 * lint-3.9
 * lint-3.10
-* test_jupyter_notebook-3.6
-* test_jupyter_notebook-3.7
 * test_jupyter_notebook-3.8
 * test_jupyter_notebook-3.9
 * test_jupyter_notebook-3.10

docs/source/structured_config.rst

@@ -22,10 +22,8 @@ Structured Configs
 Structured configs are used to create OmegaConf configuration object with runtime type safety.
 In addition, they can be used with tools like mypy or your IDE for static type checking.
 
-Two types of structures classes are supported: dataclasses and attr classes.
-
-- `dataclasses <https://docs.python.org/3.7/library/dataclasses.html>`_ are standard as of Python 3.7 or newer and are available in Python 3.6 via the `dataclasses` pip package.
-- `attrs <https://github.com/python-attrs/attrs>`_  Offset slightly cleaner syntax in some cases but depends on the attrs pip package.
+Two types of structures classes are supported: `dataclasses <https://docs.python.org/3/library/dataclasses.html>`_ and `attrs <https://github.com/python-attrs/attrs>`_ classes
+(that offers slightly cleaner syntax in some cases but depends on the attrs pip package).
 
 This documentation will use dataclasses, but you can use the annotation ``@attr.s(auto_attribs=True)`` from attrs instead of ``@dataclass``.
 
@@ -440,10 +438,6 @@ OmegaConf supports field modifiers such as ``MISSING`` and ``Optional``.
 
     >>> conf: Modifiers = OmegaConf.structured(Modifiers)
 
-Note for Python3.6 users: :ref:`pickling <save_and_load_pickle_file>`
-structured configs with complex type annotations, such as dict-of-list or
-list-of-optional, is not supported.
-
 Mandatory missing values
 ++++++++++++++++++++++++
 

docs/source/usage.rst

@@ -27,7 +27,7 @@ Just pip install::
 
     pip install omegaconf
 
-OmegaConf requires Python 3.6 and newer.
+OmegaConf requires Python 3.8 or newer.
 
 .. _creating:
 
@@ -316,12 +316,6 @@ Note that the saved file may be incompatible across different versions of OmegaC
     ...     loaded = pickle.load(fp)
     ...     assert conf == loaded
 
-Note for Python3.6 users: due to limitations in pickling support,
-:ref:`structured configs <structured_configs>` with complex type hints (such as
-:ref:`nested container types <nested_dict_and_list_annotations>` or
-:ref:`containers with optional element types <other_special_features>`) cannot
-be pickled using Python3.6.
-
 
 .. _interpolation:
 

news/1109.api_change

@@ -0,0 +1 @@
+Python 3.6 and 3.7 are not supported anymore: OmegaConf now requires Python 3.8+

noxfile.py

@@ -4,7 +4,7 @@
 import nox
 from nox import Session
 
-DEFAULT_PYTHON_VERSIONS = ["3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]
+DEFAULT_PYTHON_VERSIONS = ["3.8", "3.9", "3.10", "3.11"]
 
 PYTHON_VERSIONS = os.environ.get(
     "NOX_PYTHON_VERSIONS", ",".join(DEFAULT_PYTHON_VERSIONS)
@@ -63,7 +63,7 @@ def version_string_to_tuple(version: str) -> Tuple[int, ...]:
     return tuple(map(int, version.split(".")))
 
 
-@nox.session(python=[v for v in PYTHON_VERSIONS if version_string_to_tuple(v) >= (3, 7)])  # type: ignore
+@nox.session(python=PYTHON_VERSIONS)  # type: ignore
 def lint(session: Session) -> None:
     deps(session, editable_install=True)
     session.run(

omegaconf/_utils.py

@@ -6,20 +6,9 @@
 import sys
 import types
 import warnings
-from contextlib import contextmanager
 from enum import Enum
 from textwrap import dedent
-from typing import (
-    Any,
-    Dict,
-    Iterator,
-    List,
-    Optional,
-    Tuple,
-    Type,
-    Union,
-    get_type_hints,
-)
+from typing import Any, Dict, List, Optional, Tuple, Type, Union, get_type_hints
 
 import yaml
 
@@ -635,32 +624,22 @@ def is_dict_annotation(type_: Any) -> bool:
     origin = getattr(type_, "__origin__", None)
     # type_dict is a bit hard to detect.
     # this support is tentative, if it eventually causes issues in other areas it may be dropped.
-    if sys.version_info < (3, 7, 0):  # pragma: no cover
-        typed_dict = hasattr(type_, "__base__") and type_.__base__ == Dict
-        return origin is Dict or type_ is Dict or typed_dict
-    else:  # pragma: no cover
-        typed_dict = hasattr(type_, "__base__") and type_.__base__ == dict
-        return origin is dict or typed_dict
+    typed_dict = hasattr(type_, "__base__") and type_.__base__ == dict
+    return origin is dict or typed_dict
 
 
 def is_list_annotation(type_: Any) -> bool:
     if type_ in (list, List):
         return True
     origin = getattr(type_, "__origin__", None)
-    if sys.version_info < (3, 7, 0):
-        return origin is List or type_ is List  # pragma: no cover
-    else:
-        return origin is list  # pragma: no cover
+    return origin is list
 
 
 def is_tuple_annotation(type_: Any) -> bool:
     if type_ in (tuple, Tuple):
         return True
     origin = getattr(type_, "__origin__", None)
-    if sys.version_info < (3, 7, 0):
-        return origin is Tuple or type_ is Tuple  # pragma: no cover
-    else:
-        return origin is tuple  # pragma: no cover
+    return origin is tuple
 
 
 def is_supported_union_annotation(obj: Any) -> bool:
@@ -1032,10 +1011,3 @@ def split_key(key: str) -> List[str]:
     tokens += [dot_key if dot_key else bracket_key for dot_key, bracket_key in others]
 
     return tokens
-
-
-# Similar to Python 3.7+'s `contextlib.nullcontext` (which should be used instead,
-# once support for Python 3.6 is dropped).
-@contextmanager
-def nullcontext(enter_result: Any = None) -> Iterator[Any]:
-    yield enter_result

omegaconf/basecontainer.py

@@ -45,7 +45,6 @@
     InterpolationResolutionError,
     KeyValidationError,
     MissingMandatoryValue,
-    OmegaConfBaseException,
     ReadonlyConfigError,
     ValidationError,
 )
@@ -133,12 +132,6 @@ def __getstate__(self) -> Dict[str, Any]:
                 dict_copy["_metadata"].ref_type = List
             else:
                 assert False
-        if sys.version_info < (3, 7):  # pragma: no cover
-            element_type = self._metadata.element_type
-            if is_union_annotation(element_type):
-                raise OmegaConfBaseException(
-                    "Serializing structured configs with `Union` element type requires python >= 3.7"
-                )
         return dict_copy
 
     # Support pickle

omegaconf/omegaconf.py

@@ -7,7 +7,7 @@
 import sys
 import warnings
 from collections import defaultdict
-from contextlib import contextmanager
+from contextlib import contextmanager, nullcontext
 from enum import Enum
 from textwrap import dedent
 from typing import (
@@ -49,7 +49,6 @@
     is_structured_config,
     is_tuple_annotation,
     is_union_annotation,
-    nullcontext,
     split_key,
     type_str,
 )

omegaconf/version.py

@@ -2,12 +2,13 @@
 
 __version__ = "2.4.0.dev1"
 
-msg = """OmegaConf 2.0 and above is compatible with Python 3.6 and newer.
+msg = """OmegaConf 2.4 and above is compatible with Python 3.8 and newer.
 You have the following options:
-1. Upgrade to Python 3.6 or newer.
-   This is highly recommended. new features will not be added to OmegaConf 1.4.
-2. Continue using OmegaConf 1.4:
-    You can pip install 'OmegaConf<1.5' to do that.
+1. Upgrade to Python 3.8 or newer.
+   This is highly recommended. new features will not be added to OmegaConf 2.3.
+2. Continue using OmegaConf 2.3:
+    You can pip install 'OmegaConf<2.4' to do that.
 """
-if sys.version_info < (3, 6):
+
+if sys.version_info < (3, 8):
     raise ImportError(msg)  # pragma: no cover

requirements/base.txt

@@ -1,4 +1,2 @@
 antlr4-python3-runtime==4.9.*
 PyYAML>=5.1.0
-# Use dataclasses backport for Python 3.6.
-dataclasses;python_version=='3.6'

requirements/dev.txt

@@ -1,12 +1,12 @@
 -r base.txt
 -r docs.txt
 attrs
-black
+black==23.7.0
 build
 coveralls
-flake8>=4
-isort~=5.0
-mypy
+flake8==6.0.0
+isort==5.12.0
+mypy==1.4.1
 nox
 pre-commit
 pyflakes
@@ -15,6 +15,6 @@ pytest-benchmark
 pytest-lazy-fixture
 pytest-mock
 towncrier
+types-setuptools  # makes mypy happy
 twine
 pydevd
-

setup.cfg

@@ -2,7 +2,7 @@
 test=pytest
 
 [mypy]
-python_version = 3.7
+python_version = 3.8
 mypy_path=.stubs
 exclude = build/
 

setup.py

@@ -57,10 +57,8 @@
             "pydevd_plugins",
             "pydevd_plugins.extensions",
         ],
-        python_requires=">=3.6",
+        python_requires=">=3.8",
         classifiers=[
-            "Programming Language :: Python :: 3.6",
-            "Programming Language :: Python :: 3.7",
             "Programming Language :: Python :: 3.8",
             "Programming Language :: Python :: 3.9",
             "Programming Language :: Python :: 3.10",

tests/examples/dataclass_postponed_annotations.py

@@ -1,6 +1,6 @@
 # `from __future__` has to be the very first thing in a module
 # otherwise a syntax error is raised
-from __future__ import annotations  # noqa  # Python 3.6 linters complain
+from __future__ import annotations
 
 from dataclasses import dataclass, fields
 from enum import Enum

tests/examples/test_postponed_annotations.py

@@ -1,17 +1,10 @@
-import sys
-
-from pytest import mark
-
-
-@mark.skipif(sys.version_info < (3, 7), reason="requires Python 3.7")
 def test_simple_types_class_postponed() -> None:
     # import from a module which has `from __future__ import annotations`
     from tests.examples.dataclass_postponed_annotations import simple_types_class
 
     simple_types_class()
 
 
-@mark.skipif(sys.version_info < (3, 7), reason="requires Python 3.7")
 def test_conversions_postponed() -> None:
     # import from a module which has `from __future__ import annotations`
     from tests.examples.dataclass_postponed_annotations import conversions

tests/test_base_config.py

@@ -1,4 +1,5 @@
 import copy
+from contextlib import nullcontext
 from typing import Any, Dict, List, Optional, Union
 
 from pytest import mark, param, raises
@@ -19,7 +20,7 @@
     open_dict,
     read_write,
 )
-from omegaconf._utils import _ensure_container, nullcontext
+from omegaconf._utils import _ensure_container
 from omegaconf.errors import ConfigAttributeError, ConfigKeyError, MissingMandatoryValue
 from tests import (
     ConcretePlugin,

tests/test_basic_ops_list.py

@@ -1,5 +1,6 @@
 # -*- coding: utf-8 -*-
 import re
+from contextlib import nullcontext
 from pathlib import Path
 from textwrap import dedent
 from typing import Any, Callable, List, MutableSequence, Optional, Union
@@ -8,7 +9,7 @@
 from pytest import mark, param, raises
 
 from omegaconf import MISSING, AnyNode, DictConfig, ListConfig, OmegaConf, flag_override
-from omegaconf._utils import _ensure_container, nullcontext
+from omegaconf._utils import _ensure_container
 from omegaconf.base import Node
 from omegaconf.errors import (
     ConfigTypeError,