Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove zero-probability states from Sampler's result #9248

Merged
merged 14 commits into from
Jan 19, 2023
Merged

Remove zero-probability states from Sampler's result #9248

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
a43b0e2
Remove zero probability from Sampler's result
t-imamichi Dec 6, 2022
bd0c7d4
fix TestGrover
t-imamichi Dec 7, 2022
539ba8e
add reno
t-imamichi Dec 7, 2022
dc9676c
Merge branch 'main' into sampler-zero-filter
t-imamichi Dec 8, 2022
30a9311
fix a test
t-imamichi Dec 8, 2022
3ec27fd
Merge branch 'main' into sampler-zero-filter
t-imamichi Dec 9, 2022
1362a17
Merge branch 'main' into sampler-zero-filter
Cryoris Dec 9, 2022
6ffe3bf
Merge branch 'main' into sampler-zero-filter
t-imamichi Dec 11, 2022
8492b7b
Merge branch 'main' into sampler-zero-filter
t-imamichi Dec 12, 2022
6c1ccf4
Merge branch 'main' into sampler-zero-filter
Cryoris Dec 20, 2022
32a90c8
Merge branch 'main' into sampler-zero-filter
t-imamichi Jan 4, 2023
9325e17
Merge branch 'main' into sampler-zero-filter
ikkoham Jan 18, 2023
35dbd3b
round probabilities
ikkoham Jan 19, 2023
cac2cee
Merge branch 'main' into sampler-zero-filter
mergify[bot] Jan 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions qiskit/primitives/sampler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,16 +118,18 @@ def _call(
)
qargs_list.append(self._qargs_list[i])
probabilities = [
Statevector(bound_circuit_to_instruction(circ)).probabilities(qargs=qargs)
Statevector(bound_circuit_to_instruction(circ)).probabilities_dict(qargs=qargs)
for circ, qargs in zip(bound_circuits, qargs_list)
]
if shots is not None:
probabilities = [
rng.multinomial(shots, probability) / shots for probability in probabilities
]
for i, prob_dict in enumerate(probabilities):
counts = rng.multinomial(shots, np.fromiter(prob_dict.values(), dtype=float))
probabilities[i] = {
key: count / shots for key, count in zip(prob_dict.keys(), counts) if count > 0
}
for metadatum in metadata:
metadatum["shots"] = shots
quasis = [QuasiDistribution(dict(enumerate(p)), shots=shots) for p in probabilities]
quasis = [QuasiDistribution(p, shots=shots) for p in probabilities]

return SamplerResult(quasis, metadata)

Expand Down
5 changes: 5 additions & 0 deletions releasenotes/notes/update-sampler-zero-filter-8bf0d721af4fbd17.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
upgrade:
- |
Updated the reference implementation :meth:`~.Sampler.run` to return
only states with non-zero probabilities.
9 changes: 7 additions & 2 deletions test/python/algorithms/test_grover.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -257,10 +257,15 @@ def test_circuit_result(self, use_sampler):
if use_sampler:
for i, dist in enumerate(result.circuit_results):
keys, values = zip(*sorted(dist.items()))
self.assertTupleEqual(keys, ("00", "01", "10", "11"))
if i in (0, 3):
np.testing.assert_allclose(values, [0, 0, 0, 1], atol=0.2)
if use_sampler == "ideal":
self.assertTupleEqual(keys, ("00", "01", "10", "11"))
np.testing.assert_allclose(values, [0, 0, 0, 1], atol=0.2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that 0 probabilities are not included, shouldn't this just show the "11" key here? That also seems to be what the error is about 🙂

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideal case should include "00", "01", and "10" with very small probabilities. The test on Linux passes but that on Mac fails. I'm investigating the difference.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot reproduce the CI failure locally on MacOS with Python 3.7 🤔

The values with which "00", "01" and "10" are included are about 1e-34 maybe we don't really need to check that this is recognized as non-zero? This seems a bit volatile to me... since this is about algorithm performance rather than the technicality which element is populated and from the sampler tests we know that the zero-probability states are removed, how about just checking if the values are correct in the Grover test? E.g. like

if i in (0, 3):
    values = [dist.get(key, 0) for key in ["00", "01", "10", "11"]]
    np.testing.assert_allclose(values, [0, 0, 0, 1], atol=0.2)

Copy link
Contributor

@jlapeyre jlapeyre Jan 19, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I makes sense to me to filter out probabilities like this:

For the exact distribution: Remove probabilities satisfying $p_i \ll 1/|p| $. The reasoning is that the sum of any set of $p_i$ s of this magnitude is negligible. On the other hand we may want to just remove what we think is in some sense a numerical artifact. This might not be easy to determine. For example 1e-34 is probably safe to remove. But the former at least has a reason behind the choice. You still have to decide what is much less than one. Maybe 1e-10

For the distribution from samples, you should remove a probability if $p_i / n_\text{shots} \ll 1$. This will make sampling efficient.

But you want to do something to remove small probabilities in both cases. The dicts could otherwise consume a lot of memory and be populated mostly with garbage.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will use decimals of Statevector.probabilities to round the results.

else:
self.assertTupleEqual(keys, ("11",))
np.testing.assert_allclose(values, [1], atol=0.2)
else:
self.assertTupleEqual(keys, ("00", "01", "10", "11"))
np.testing.assert_allclose(values, [0.25, 0.25, 0.25, 0.25], atol=0.2)
else:
expected_results = [
Expand Down
117 changes: 51 additions & 66 deletions test/python/primitives/test_sampler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,8 +135,8 @@ def test_sampler_example(self):
self.assertIsInstance(result, SamplerResult)
self.assertEqual(len(result.quasi_dists), 1)
keys, values = zip(*sorted(result.quasi_dists[0].items()))
self.assertTupleEqual(keys, tuple(range(4)))
np.testing.assert_allclose(values, [0.5, 0, 0, 0.5])
self.assertTupleEqual(keys, (0, 3))
np.testing.assert_allclose(values, [0.5, 0.5])
self.assertEqual(len(result.metadata), 1)

# executes three Bell circuits
Expand All @@ -148,8 +148,8 @@ def test_sampler_example(self):
self.assertEqual(len(result.metadata), 3)
for dist in result.quasi_dists:
keys, values = zip(*sorted(dist.items()))
self.assertTupleEqual(keys, tuple(range(4)))
np.testing.assert_allclose(values, [0.5, 0, 0, 0.5])
self.assertTupleEqual(keys, (0, 3))
np.testing.assert_allclose(values, [0.5, 0.5])

with self.assertWarns(DeprecationWarning):
sampler = Sampler([bell])
Expand All @@ -159,8 +159,8 @@ def test_sampler_example(self):
self.assertEqual(len(result.metadata), 3)
for dist in result.quasi_dists:
keys, values = zip(*sorted(dist.items()))
self.assertTupleEqual(keys, tuple(range(4)))
np.testing.assert_allclose(values, [0.5, 0, 0, 0.5])
self.assertTupleEqual(keys, (0, 3))
np.testing.assert_allclose(values, [0.5, 0.5])

# parametrized circuit
pqc = RealAmplitudes(num_qubits=2, reps=2)
Expand Down Expand Up @@ -226,23 +226,23 @@ def test_sampler_param_order(self):

# qc({x: 0, y: 0})
keys, values = zip(*sorted(result.quasi_dists[0].items()))
self.assertTupleEqual(keys, tuple(range(8)))
np.testing.assert_allclose(values, [0, 0, 0, 0, 1, 0, 0, 0])
self.assertTupleEqual(keys, (4,))
np.testing.assert_allclose(values, [1])

# qc({x: 0, y: 0})
keys, values = zip(*sorted(result.quasi_dists[1].items()))
self.assertTupleEqual(keys, tuple(range(8)))
np.testing.assert_allclose(values, [0, 0, 0, 0, 1, 0, 0, 0])
self.assertTupleEqual(keys, (4,))
np.testing.assert_allclose(values, [1])

# qc({x: pi/2, y: 0})
keys, values = zip(*sorted(result.quasi_dists[2].items()))
self.assertTupleEqual(keys, tuple(range(8)))
np.testing.assert_allclose(values, [0, 0, 0, 0, 0.5, 0.5, 0, 0])
self.assertTupleEqual(keys, (4, 5))
np.testing.assert_allclose(values, [0.5, 0.5])

# qc({x: 0, y: pi/2})
keys, values = zip(*sorted(result.quasi_dists[3].items()))
self.assertTupleEqual(keys, tuple(range(8)))
np.testing.assert_allclose(values, [0, 0, 0, 0, 0.5, 0, 0.5, 0])
self.assertTupleEqual(keys, (4, 6))
np.testing.assert_allclose(values, [0.5, 0.5])

def test_sampler_reverse_meas_order(self):
"""test for sampler with reverse measurement order"""
Expand All @@ -265,23 +265,23 @@ def test_sampler_reverse_meas_order(self):

# qc({x: 0, y: 0})
keys, values = zip(*sorted(result.quasi_dists[0].items()))
self.assertTupleEqual(keys, tuple(range(8)))
np.testing.assert_allclose(values, [0, 1, 0, 0, 0, 0, 0, 0])
self.assertTupleEqual(keys, (1,))
np.testing.assert_allclose(values, [1])

# qc({x: 0, y: 0})
keys, values = zip(*sorted(result.quasi_dists[1].items()))
self.assertTupleEqual(keys, tuple(range(8)))
np.testing.assert_allclose(values, [0, 1, 0, 0, 0, 0, 0, 0])
self.assertTupleEqual(keys, (1,))
np.testing.assert_allclose(values, [1])

# qc({x: pi/2, y: 0})
keys, values = zip(*sorted(result.quasi_dists[2].items()))
self.assertTupleEqual(keys, tuple(range(8)))
np.testing.assert_allclose(values, [0, 0.5, 0, 0, 0, 0.5, 0, 0])
self.assertTupleEqual(keys, (1, 5))
np.testing.assert_allclose(values, [0.5, 0.5])

# qc({x: 0, y: pi/2})
keys, values = zip(*sorted(result.quasi_dists[3].items()))
self.assertTupleEqual(keys, tuple(range(8)))
np.testing.assert_allclose(values, [0, 0.5, 0, 0.5, 0, 0, 0, 0])
self.assertTupleEqual(keys, (1, 3))
np.testing.assert_allclose(values, [0.5, 0.5])

def test_1qubit(self):
"""test for 1-qubit cases"""
Expand All @@ -297,13 +297,10 @@ def test_1qubit(self):
self.assertIsInstance(result, SamplerResult)
self.assertEqual(len(result.quasi_dists), 2)

keys, values = zip(*sorted(result.quasi_dists[0].items()))
self.assertTupleEqual(keys, tuple(range(2)))
np.testing.assert_allclose(values, [1, 0])

keys, values = zip(*sorted(result.quasi_dists[1].items()))
self.assertTupleEqual(keys, tuple(range(2)))
np.testing.assert_allclose(values, [0, 1])
for i in range(2):
keys, values = zip(*sorted(result.quasi_dists[i].items()))
self.assertTupleEqual(keys, (i,))
np.testing.assert_allclose(values, [1])

def test_2qubit(self):
"""test for 2-qubit cases"""
Expand All @@ -328,21 +325,10 @@ def test_2qubit(self):
self.assertIsInstance(result, SamplerResult)
self.assertEqual(len(result.quasi_dists), 4)

keys, values = zip(*sorted(result.quasi_dists[0].items()))
self.assertTupleEqual(keys, tuple(range(4)))
np.testing.assert_allclose(values, [1, 0, 0, 0])

keys, values = zip(*sorted(result.quasi_dists[1].items()))
self.assertTupleEqual(keys, tuple(range(4)))
np.testing.assert_allclose(values, [0, 1, 0, 0])

keys, values = zip(*sorted(result.quasi_dists[2].items()))
self.assertTupleEqual(keys, tuple(range(4)))
np.testing.assert_allclose(values, [0, 0, 1, 0])

keys, values = zip(*sorted(result.quasi_dists[3].items()))
self.assertTupleEqual(keys, tuple(range(4)))
np.testing.assert_allclose(values, [0, 0, 0, 1])
for i in range(4):
keys, values = zip(*sorted(result.quasi_dists[i].items()))
self.assertTupleEqual(keys, (i,))
np.testing.assert_allclose(values, [1])

def test_errors(self):
"""Test for errors"""
Expand Down Expand Up @@ -532,13 +518,10 @@ def test_run_1qubit(self):
self.assertIsInstance(result, SamplerResult)
self.assertEqual(len(result.quasi_dists), 2)

keys, values = zip(*sorted(result.quasi_dists[0].items()))
self.assertTupleEqual(keys, tuple(range(2)))
np.testing.assert_allclose(values, [1, 0])

keys, values = zip(*sorted(result.quasi_dists[1].items()))
self.assertTupleEqual(keys, tuple(range(2)))
np.testing.assert_allclose(values, [0, 1])
for i in range(2):
keys, values = zip(*sorted(result.quasi_dists[i].items()))
self.assertTupleEqual(keys, (i,))
np.testing.assert_allclose(values, [1])

def test_run_2qubit(self):
"""test for 2-qubit cases"""
Expand All @@ -559,21 +542,10 @@ def test_run_2qubit(self):
self.assertIsInstance(result, SamplerResult)
self.assertEqual(len(result.quasi_dists), 4)

keys, values = zip(*sorted(result.quasi_dists[0].items()))
self.assertTupleEqual(keys, tuple(range(4)))
np.testing.assert_allclose(values, [1, 0, 0, 0])

keys, values = zip(*sorted(result.quasi_dists[1].items()))
self.assertTupleEqual(keys, tuple(range(4)))
np.testing.assert_allclose(values, [0, 1, 0, 0])

keys, values = zip(*sorted(result.quasi_dists[2].items()))
self.assertTupleEqual(keys, tuple(range(4)))
np.testing.assert_allclose(values, [0, 0, 1, 0])

keys, values = zip(*sorted(result.quasi_dists[3].items()))
self.assertTupleEqual(keys, tuple(range(4)))
np.testing.assert_allclose(values, [0, 0, 0, 1])
for i in range(4):
keys, values = zip(*sorted(result.quasi_dists[i].items()))
self.assertTupleEqual(keys, (i,))
np.testing.assert_allclose(values, [1])

def test_run_single_circuit(self):
"""Test for single circuit case."""
Expand Down Expand Up @@ -721,6 +693,19 @@ def test_run_with_shots_option_none(self):
).result()
self.assertDictAlmostEqual(result_42.quasi_dists, result_15.quasi_dists)

def test_run_shots_result_size(self):
Cryoris marked this conversation as resolved.
Show resolved Hide resolved
"""test with shots option to validate the result size"""
n = 10
shots = 100
qc = QuantumCircuit(n)
qc.h(range(n))
qc.measure_all()
sampler = Sampler()
result = sampler.run(qc, [], shots=shots, seed=42).result()
self.assertEqual(len(result.quasi_dists), 1)
self.assertLessEqual(len(result.quasi_dists[0]), shots)
self.assertAlmostEqual(sum(result.quasi_dists[0].values()), 1.0)

def test_primitive_job_status_done(self):
"""test primitive job's status"""
bell = self._circuit[1]
Expand Down