Inconsistencies between FakeBackend{,V2} and IBMBackend

[kdk]

What should we add?

The FakeBackends defined in qiskit.provider.fake_provider differ in how closely they mirror the Backends returned by the qiskit-ibm{,q}-provider depending on if they are instances of BackendV1 or BackendV2. For example,

>>> fake_mumbai_v1 = FakeMumbai()
>>> fake_mumbai_v2 = FakeMumbaiV2()
>>> mumbai_v1 = provider_v1.get_backend('ibmq_mumbai')
>>> mumbai_v2 = provider_v2.get_backend('ibmq_mumbai')

>>> mumbai_v1.configuration().processor_type
{'family': 'Falcon', 'revision': '5.10'}
>>> fake_mumbai_v1.configuration().processor_type
{'family': 'Falcon', 'revision': 5.1}
>>> # ^ Seems like a JSON encoding bug, separate issue
>>> mumbai_v2.processor_type
{'family': 'Falcon', 'revision': '5.10'}
>>> fake_mumbai_v2.processor_type
AttributeError: 'FakeMumbaiV2' object has no attribute 'processor_type'

From discussion at the last terra meeting, this was an intentional change as processor_type and related fields were part of the Qiskit's BackendV1 definition but are not included as part of BackendV2 (the qiskit_ibm_provider adds them back as custom fields). This however causes issues for users who use the fake backends as stand-ins for IBM backends for running noisy simulations, testing compilation, etc. and assume that the fake backends represent static replicas of their IBM provider counterparts.

From the discussion, there arose two use cases for the fake backends:

1. For terra developers who use as the fake backends to test Qiskit specific functionality in the transpiler, visualization, pulse, etc. modules (e.g. using FakeSherbrooke to exercise compilation to ECR gates)
2. For end users who, following the model of how the fake backends are used in release notes, documentation, Qiskit textbook, want to use the fake backends as as snapshots of IBM devices

These differ mainly in that the first use case doesn't want to couple strongly to provider-specific behavior like the use of custom properties (to avoid mixing vendor- or provider-specific behavior with base terra testing and implementations) while the second explicitly does want a strong coupling between fake and real backends (to avoid having to write special case handling for the fake backends, or complicate using them to test and validate code before sending to real devices).

The proposed solution involves the following steps:

Tasks

Preview Give feedback

1. 1) Update existing test usage to remove IBM-derived fake backends in favor of case-specific fake backend construction (e.g. test-local fake backends to exercise specific cases) and a collection of vendor-agnostic fake backends for common cases, named for what property they help exercise (e.g. Fake500Qubits, FakeWithISwap, ...)
2. 2) Migrate the existing IBM-derived fake backends (and the associated tooling) from their location in qiskit.providers.fake_provider to the qiskit_ibm_provider package
3. 3) Update fake backend usage in documentation to reference new location
4. 4) Deprecate IBM-derived fake backends from qiskit.providers.fake_provider

Opening this issue to consolidate discussion on the above, and track the progress of implementation.

[jakelishman]

To add a little bit more on point 2 of the plan: if end users want "mock" versions of IBM devices that use simulators behind the scenes, but otherwise are identical in interface to the "real" devices, then those are going to need to live in qiskit-ibm-provider along with the real versions. It's not feasible from a maintenance perspective, nor from separation-of-concerns nor Terra's backend-agnosticism, if Terra contains ibm-provider specific code. The term "migrate" probably masks a little bit of complexity, in that if they do move, then ibm-provider will probably want to rewrite the fake backends themselves using roughly the same code as the real ones, rather than actually copying Terra's snapshot-version code and trying to modify it to match.

Another user story we've heard is that they like that the fake backends are accessible even for devices that they don't have access do - not everyone can run stuff on Kolkata, for example, but people like testing all the pipelines and then simulating it.

[nonhermitian]

If getting the fake backends to match the actual device requires moving them to the IBM provider, then I am all for that. The current inconsistencies, e.g. no backend.configuration() for fake V2 backends, is a bit annoying to work with. Their inclusion with Terra is probably more legacy than any practical limitation at this point.

they like that the fake backends are accessible even for devices that they don't have access do

This is absolutely something that is important.

[HuangJunye]

I migrated the fake backends last year so I think can help here. The plan laid out by @kdk is very reasonable. So the first step is to introduce generic backends to replace IBM-specific backends for terra testing. Should I add these testing backends to qiskit.test like it used to be before the migration to make it clear that they are for testing purposes?

[kdk]

I migrated the fake backends last year so I think can help here. The plan laid out by @kdk is very reasonable. So the first step is to introduce generic backends to replace IBM-specific backends for terra testing. Should I add these testing backends to qiskit.test like it used to be before the migration to make it clear that they are for testing purposes?

That sounds good to me.

[jakelishman]

I think whatever fake backends Terra has should still live in fake_provider for the same reasons that we originally moved the testing ones there - people want access to simple backend representations to test their code against.

[mtreinish]

Superseded by: #10954