Add no-optionals and full-optionals test runs

[jakelishman]

Summary

This restructures the CI slightly to perform a complete "no-optionals" run, and a complete "all optionals" run (for the optionals that are readily accessible as Python packages, without complex additional setup). Previously, we only did a partial test with some of the oldest optional components, which could have allowed for behaviour to accidentally require optional components without us noticing.

This splits the requirements-dev.txt file into two; lines that remain in the file are what is actually required to run the test suite, run the style checks, and do the documentation build. The rest (and everything that was missing) is added to a new
requirements-optional.txt file, which can be used to pull in (almost) all of the packages that Terra can use to provide additional / accelerated functionality.

Several tests needed to gain additional skips to account for this change. There is a good chance that some tests are missing skips for some libraries that are not the first point of failure, but it's hard to test explicitly for these in one go.

Details and comments

[qiskit-bot]

Thank you for opening a new pull request.

Before your PR can be merged it will first need to pass continuous integration tests and be reviewed. Sometimes the review process can be slow, so please be patient.

While you're waiting, please feel free to review other open PRs. While only a subset of people are authorized to approve pull requests for merging, everyone is encouraged to review open pull requests. Doing reviews helps reduce the burden on the core team and helps make the project's code better for everyone.

One or more of the the following people are requested to review this:

• @Qiskit/terra-core
• @enavarro51
• @ikkoham

[coveralls]

Pull Request Test Coverage Report for Build 5600511995

• 1 of 1 (100.0%) changed or added relevant line in 1 file are covered.
• 222 unchanged lines in 21 files lost coverage.
• Overall coverage increased (+1.4%) to 87.444%

---

Files with Coverage Reduction	New Missed Lines	%
crates/accelerate/src/sabre_layout.rs	1	99.43%
qiskit/transpiler/passes/layout/dense_layout.py	1	95.1%
qiskit/visualization/circuit/_utils.py	1	94.35%
crates/qasm2/src/lex.rs	2	91.39%
qiskit/circuit/controlflow/switch_case.py	2	94.08%
qiskit/qasm3/ast.py	2	97.79%
qiskit/circuit/classical/expr/visitors.py	3	95.59%
qiskit/circuit/library/standard_gates/x.py	3	99.12%
qiskit/transpiler/preset_passmanagers/level1.py	3	96.67%
qiskit/transpiler/passes/routing/sabre_swap.py	5	96.89%

Totals
Change from base Build 5592160564:	1.4%
Covered Lines:	73869
Relevant Lines:	84476

---

💛 - Coveralls

[jakelishman]

Tests should pass after #8973.

[jakelishman]

Blocked by #9021.

[jakelishman]

Ready for review.

[jakelishman]

The failure on Windows is due to #10415.

[jakelishman]

With #10441 merged, this is unblocked.

[Eric-Arellano]

Will re-review tomorrow. Let's make sure to land this before #10443.

[Eric-Arellano]

Thanks! Great cleanup.

Feel free to merge without applying my changes - they're somewhat nitty

[mtreinish]

This LGTM, I'm a still mildly concerned about the upper version runs not including the optionals as I just know we've had issues with some optional dependencies lagging on python version support. The nightly jobs provide the coverage, but we've been bad historically about catching failures in the nightly runs. That being said this is a good cleanup and restructuring of how we deal with the optionals and testing. We can always revisit the exact testing matrix and strategy in a followup pr if we have issues for 3.12.