Optimise import qiskit with lazy imports (#7525)

* Unify lazy handling of optional dependencies

This introduces new `HAS_X` variables for each of Qiskit's optional
dependencies, and provides a simple unified interface to them from
`qiskit.utils.optionals`.  These objects lazily test for their
dependency when evaluated in a Boolean context, and have two `require_`
methods to unify the exception handling.  `require_now` tests
immediately for the dependency and raises `MissingOptionalLibraryError`
if it is not present, and `require_in_call` is a decorator that lazily
tests for the dependencies when the function is called.

These remove the burden of raising nice exceptions from the usage
points; a function marked `HAS_MATPLOTLIB.require_in_call` can now
safely `import matplotlib` without special handling, for example.  This
also provides a unified way for consumers of `qiskit` (such as the test
suite) to query the presence of libraries.

All tests are now lazy, and imports are moved to the point of usage, not
the point of import of the module.  This means that `import qiskit` is
significantly faster for people who have many of the optional
dependencies installed; rather than them all being loaded at initial
import just to test their presence, they will now be loaded on demand.

* Optimise time taken for `import qiskit`

This makes several imports lazy, only being imported when they are
actually called and used.  In particular, no component of `scipy` is
imported during `import qiskit` now, nor is `pkg_resources` (which is
surprisingly heavy).

No changes were made to algorithms or opflow, since these are not
immediately imported during `import qiskit`, and likely require more
significant work than the rest of the library.

* Import missing to-be-deprecated names

* Convert straggler tests to require_now

* Correct requirements in test cases

* Add `require_in_instance` class decorator

Effectively this is just a wrapper around `__init__`, except that this
class-decorator form will do the right thing even if `__init__` isn't
explicitly defined on the given class.

The implementation of `wrap_method` is a replacement for the older
`test.decorators._wrap_method`, which didn't handle all the possible
special cases as well, and messed up the documentation of its wrapped
functions.  That wasn't so important when it was just a private
function, but now it has become public (so that
`test.decorators.enforce_subclasses_call` can still use it from
`qiskit.utils`), it needed reworking to be more polished.

* Privatise non-public names rather than del

* Add tests of `require_in_instance`

* Fix typos in documentation

* Add section on requirements to CONTRIBUTING

* Update documentation on HoareOptimizer error

* Remove UK localisation

* Mention more uses of PIL in documentation

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

CONTRIBUTING.md

@@ -23,6 +23,10 @@ contributing to terra, these are documented below.
   * [Branches](#branches)
   * [Release Cycle](#release-cycle)
 * [Adding deprecation warnings](#adding-deprecation-warnings)
+* [Using dependencies](#using-dependencies)
+  * [Adding a requirement](#adding-a-requirement)
+  * [Adding an optional dependency](#adding-an-optional-dependency)
+  * [Checking for optionals](#checking-for-optionals)
 * [Dealing with git blame ignore list](#dealing-with-the-git-blame-ignore-list)
 
 ### Choose an issue to work on
@@ -478,6 +482,37 @@ def test_method2(self):
 
 `test_method1_deprecated` can be removed after `Obj.method1` is removed (following the [Qiskit Deprecation Policy](https://qiskit.org/documentation/contributing_to_qiskit.html#deprecation-policy)).
 
+## Using dependencies
+
+We distinguish between "requirements" and "optional dependencies" in qiskit-terra.
+A requirement is a package that is absolutely necessary for core functionality in qiskit-terra, such as Numpy or Scipy.
+An optional dependency is a package that is used for specialized functionality, which might not be needed by all users.
+If a new feature has a new dependency, it is almost certainly optional.
+
+### Adding a requirement
+
+Any new requirement must have broad system support; it needs to be supported on all the Python versions and operating systems that qiskit-terra supports.
+It also cannot impose many version restrictions on other packages.
+Users often install qiskit-terra into virtual environments with many different packages in, and we need to ensure that neither we, nor any of our requirements, conflict with their other packages.
+When adding a new requirement, you must add it to [`requirements.txt`](requirements.txt) with as loose a constraint on the allowed versions as possible.
+
+### Adding an optional dependency
+
+New features can also use optional dependencies, which might be used only in very limited parts of qiskit-terra.
+These are not required to use the rest of the package, and so should not be added to `requirements.txt`.
+Instead, if several optional dependencies are grouped together to provide one feature, you can consider adding an "extra" to the package metadata, such as the `visualization` extra that installs Matplotlib and Seaborn (amongst others).
+To do this, modify the [`setup.py`](setup.py) file, adding another entry in the `extras_require` keyword argument to `setup()` at the bottom of the file.
+You do not need to be quite as accepting of all versions here, but it is still a good idea to be as permissive as you possibly can be.
+You should also add a new "tester" to [`qiskit.utils.optionals`](qiskit/utils/optionals.py), for use in the next section.
+
+### Checking for optionals
+
+You cannot `import` an optional dependency at the top of a file, because if it is not installed, it will raise an error and qiskit-terra will be unusable.
+We also largely want to avoid importing packages until they are actually used; if we import a lot of packages during `import qiskit`, it becomes sluggish for the user if they have a large environment.
+Instead, you should use [one of the "lazy testers" for optional dependencies](https://qiskit.org/documentation/apidoc/utils.html#module-qiskit.utils.optionals), and import your optional dependency inside the function or class that uses it, as in the examples within that link.
+Very lightweight _requirements_ can be imported at the tops of files, but even this should be limited; it's always ok to `import numpy`, but Scipy modules are relatively heavy, so only import them within functions that use them.
+
+
 ## Dealing with the git blame ignore list
 
 In the qiskit-terra repository we maintain a list of commits for git blame

qiskit/algorithms/optimizers/bobyqa.py

@@ -15,17 +15,11 @@
 from typing import Any, Dict, Tuple, List, Callable, Optional
 
 import numpy as np
-from qiskit.exceptions import MissingOptionalLibraryError
+from qiskit.utils import optionals as _optionals
 from .optimizer import Optimizer, OptimizerSupportLevel, OptimizerResult, POINT
 
-try:
-    import skquant.opt as skq
-
-    _HAS_SKQUANT = True
-except ImportError:
-    _HAS_SKQUANT = False
-
 
+@_optionals.HAS_SKQUANT.require_in_instance
 class BOBYQA(Optimizer):
     """Bound Optimization BY Quadratic Approximation algorithm.
 
@@ -48,10 +42,6 @@ def __init__(
         Raises:
             MissingOptionalLibraryError: scikit-quant not installed
         """
-        if not _HAS_SKQUANT:
-            raise MissingOptionalLibraryError(
-                libname="scikit-quant", name="BOBYQA", pip_install="pip install scikit-quant"
-            )
         super().__init__()
         self._maxiter = maxiter
 
@@ -74,6 +64,8 @@ def minimize(
         jac: Optional[Callable[[POINT], POINT]] = None,
         bounds: Optional[List[Tuple[float, float]]] = None,
     ) -> OptimizerResult:
+        from skquant import opt as skq
+
         res, history = skq.minimize(
             func=fun,
             x0=np.asarray(x0),

qiskit/algorithms/optimizers/imfil.py

@@ -14,17 +14,11 @@
 
 from typing import Any, Dict, Callable, Optional, List, Tuple
 
-from qiskit.exceptions import MissingOptionalLibraryError
+from qiskit.utils import optionals as _optionals
 from .optimizer import Optimizer, OptimizerSupportLevel, OptimizerResult, POINT
 
-try:
-    import skquant.opt as skq
-
-    _HAS_SKQUANT = True
-except ImportError:
-    _HAS_SKQUANT = False
-
 
+@_optionals.HAS_SKQUANT.require_in_instance
 class IMFIL(Optimizer):
     """IMplicit FILtering algorithm.
 
@@ -49,10 +43,6 @@ def __init__(
         Raises:
             MissingOptionalLibraryError: scikit-quant not installed
         """
-        if not _HAS_SKQUANT:
-            raise MissingOptionalLibraryError(
-                libname="scikit-quant", name="IMFIL", pip_install="pip install scikit-quant"
-            )
         super().__init__()
         self._maxiter = maxiter
 
@@ -77,6 +67,8 @@ def minimize(
         jac: Optional[Callable[[POINT], POINT]] = None,
         bounds: Optional[List[Tuple[float, float]]] = None,
     ) -> OptimizerResult:
+        from skquant import opt as skq
+
         res, history = skq.minimize(
             func=fun,
             x0=x0,

qiskit/algorithms/optimizers/nlopts/nloptimizer.py

@@ -17,24 +17,12 @@
 from abc import abstractmethod
 import logging
 import numpy as np
-from qiskit.exceptions import MissingOptionalLibraryError
+
+from qiskit.utils import optionals as _optionals
 from ..optimizer import Optimizer, OptimizerSupportLevel, OptimizerResult, POINT
 
 logger = logging.getLogger(__name__)
 
-try:
-    import nlopt
-
-    logger.info(
-        "NLopt version: %s.%s.%s",
-        nlopt.version_major(),
-        nlopt.version_minor(),
-        nlopt.version_bugfix(),
-    )
-    _HAS_NLOPT = True
-except ImportError:
-    _HAS_NLOPT = False
-
 
 class NLoptOptimizerType(Enum):
     """NLopt Valid Optimizer"""
@@ -46,11 +34,14 @@ class NLoptOptimizerType(Enum):
     GN_ISRES = 5
 
 
+@_optionals.HAS_NLOPT.require_in_instance
 class NLoptOptimizer(Optimizer):
     """
     NLopt global optimizer base class
     """
 
+    # pylint: disable=import-error
+
     _OPTIONS = ["max_evals"]
 
     def __init__(self, max_evals: int = 1000) -> None:  # pylint: disable=unused-argument
@@ -61,15 +52,7 @@ def __init__(self, max_evals: int = 1000) -> None:  # pylint: disable=unused-arg
         Raises:
             MissingOptionalLibraryError: NLopt library not installed.
         """
-        if not _HAS_NLOPT:
-            raise MissingOptionalLibraryError(
-                libname="nlopt",
-                name="NLoptOptimizer",
-                msg=(
-                    "See https://qiskit.org/documentation/apidoc/"
-                    "qiskit.algorithms.optimizers.nlopts.html for installation information"
-                ),
-            )
+        import nlopt
 
         super().__init__()
         for k, v in list(locals().items()):
@@ -124,6 +107,8 @@ def minimize(
         jac: Optional[Callable[[POINT], POINT]] = None,
         bounds: Optional[List[Tuple[float, float]]] = None,
     ) -> OptimizerResult:
+        import nlopt
+
         x0 = np.asarray(x0)
 
         if bounds is None:

qiskit/algorithms/optimizers/snobfit.py

@@ -15,25 +15,12 @@
 from typing import Any, Dict, Optional, Callable, Tuple, List
 
 import numpy as np
-from qiskit.exceptions import MissingOptionalLibraryError
+from qiskit.utils import optionals as _optionals
 from .optimizer import Optimizer, OptimizerSupportLevel, OptimizerResult, POINT
 
 
-try:
-    import skquant.opt as skq
-
-    _HAS_SKQUANT = True
-except ImportError:
-    _HAS_SKQUANT = False
-
-try:
-    from SQSnobFit import optset
-
-    _HAS_SKSNOBFIT = True
-except ImportError:
-    _HAS_SKSNOBFIT = False
-
-
+@_optionals.HAS_SKQUANT.require_in_instance
+@_optionals.HAS_SQSNOBFIT.require_in_instance
 class SNOBFIT(Optimizer):
     """Stable Noisy Optimization by Branch and FIT algorithm.
 
@@ -64,14 +51,6 @@ def __init__(
         Raises:
             MissingOptionalLibraryError: scikit-quant or SQSnobFit not installed
         """
-        if not _HAS_SKQUANT:
-            raise MissingOptionalLibraryError(
-                libname="scikit-quant", name="SNOBFIT", pip_install="pip install scikit-quant"
-            )
-        if not _HAS_SKSNOBFIT:
-            raise MissingOptionalLibraryError(
-                libname="SQSnobFit", name="SNOBFIT", pip_install="pip install SQSnobFit"
-            )
         super().__init__()
         self._maxiter = maxiter
         self._maxfail = maxfail
@@ -102,6 +81,9 @@ def minimize(
         jac: Optional[Callable[[POINT], POINT]] = None,
         bounds: Optional[List[Tuple[float, float]]] = None,
     ) -> OptimizerResult:
+        import skquant.opt as skq
+        from SQSnobFit import optset
+
         snobfit_settings = {
             "maxmp": self._maxmp,
             "maxfail": self._maxfail,

qiskit/circuit/gate.py

@@ -15,7 +15,6 @@
 from warnings import warn
 from typing import List, Optional, Union, Tuple
 import numpy as np
-from scipy.linalg import schur
 
 from qiskit.circuit.parameterexpression import ParameterExpression
 from qiskit.circuit.exceptions import CircuitError
@@ -72,6 +71,7 @@ def power(self, exponent: float):
         """
         from qiskit.quantum_info.operators import Operator  # pylint: disable=cyclic-import
         from qiskit.extensions.unitary import UnitaryGate  # pylint: disable=cyclic-import
+        from scipy.linalg import schur
 
         # Should be diagonalized because it's a unitary.
         decomposition, unitary = schur(Operator(self).data, output="complex")

qiskit/circuit/parameter.py

@@ -15,15 +15,9 @@
 
 from uuid import uuid4
 
+from qiskit.utils import optionals as _optionals
 from .parameterexpression import ParameterExpression
 
-try:
-    import symengine
-
-    HAS_SYMENGINE = True
-except ImportError:
-    HAS_SYMENGINE = False
-
 
 class Parameter(ParameterExpression):
     """Parameter Class for variable parameters.
@@ -81,11 +75,13 @@ def __init__(self, name: str):
                 be any unicode string, e.g. "ϕ".
         """
         self._name = name
-        if not HAS_SYMENGINE:
+        if not _optionals.HAS_SYMENGINE:
             from sympy import Symbol
 
             symbol = Symbol(name)
         else:
+            import symengine
+
             symbol = symengine.Symbol(name)
         super().__init__(symbol_map={self: symbol}, expr=symbol)
 
@@ -126,10 +122,12 @@ def __getstate__(self):
 
     def __setstate__(self, state):
         self._name = state["name"]
-        if not HAS_SYMENGINE:
+        if not _optionals.HAS_SYMENGINE:
             from sympy import Symbol
 
             symbol = Symbol(self._name)
         else:
+            import symengine
+
             symbol = symengine.Symbol(self._name)
         super().__init__(symbol_map={self: symbol}, expr=symbol)