Algorithmic benchmarking results

An overview of the latest benchmarking results achieved using Q-CTRL performance management on IBM Quantum services

Using Q-CTRL's performance-management option, you can run larger and more complex algorithms on top of the IBM Quantum platform while still maintaining quality execution results. At Q-CTRL, benchmarking is performed continuously to test the performance of new error suppression methods. This document reveals some of the latest benchmarking results achieved across different algorithms, which are adapted from the QED-C Application-Oriented Performance Benchmarks for Quantum Computing.

These results push past many of the known thresholds of what can be run successfully on quantum hardware. It's highly encouraged that you try reproducing these results on your own using Q-CTRL's performance management or try outperforming them using alternative methods. These results are continuously improving and will be updated periodically as the size of experiments grows closer to utility-scale.

Success metrics

  1. Success probability: The probability that a single execution (shot) of the circuit will yield the correct solution. The ideal value is 1.
  2. Selectivity: A way to measure the signal-to-noise ratio or how much the right answer stands out relative to the other (wrong) choices. More precisely, selectivity is a ratio of the success probability to the probability of the most likely incorrect bitstring. Greater than 1 is ideal.
  3. Confidence level: The odds of running the circuit N times (shots) and picking a correct solution instead of the incorrect alternatives. The ideal value is 100%.
  4. Hellinger fidelity: A function of the Hellinger distance, which is used to quantify the similarity between two probability distributions. The ideal value is 1.

Multiple success metrics are used because together they combine to tell a complete story. As problem sizes get larger, the number of possible outcomes increases vastly, meaning that the success probability naturally decreases. Selectivity provides an indication that even a low success probability can be meaningful.

A selectivity greater than one implies that the solution bitstring is much more probable than any other, and the signal can be amplified by simply performing further executions (averaging). As you repeat the experiment, you will keep getting the correct solution as the most probable answer, and this boosts the confidence that the algorithm is working as expected, as opposed to obtaining this result by chance.

With several measures of success probability and selectivity greater than one, the confidence level can be measured and shown to converge to 100%. Together, these three metrics provide a more complete view of how most algorithms are performing. Some algorithms also have specific metrics that can be used to quantify performance.

For algorithms where the ideal distribution is known, Hellinger fidelity can be used to compare the generated and ideal distributions. In the example of QAOA, metrics specific to the problem are defined.

Configuration information

The results were collected using 127-qubit IBM devices: IBM Brisbane and IBM Sherbrooke. Daily benchmark tests are run and these represent mean values.

The "Q-CTRL + IBM label" represents results using Q-CTRL's performance management option, whereas "Without Q-CTRL" measurements were taken using the built-in error suppression and mitigation in Qiskit Runtime (optimization_level=1 and resilience_level=1).

Bernstein–Vazirani algorithm

In the following figure, the success probability of a Bernstein–Vazirani algorithm is shown as a function of the number of qubits, ranging from 10 to 45. Up to and including 45 qubits, the algorithm is still able to deliver good results with selectivity greater than one.

Success probability

Screenshot 2023-12-01 at 12.46.43 PM.png-1

Selectivity

Screenshot 2023-12-01 at 12.47.25 PM.png-2

Because the selectivity is greater than one, the confidence level can be increased by running more iterations (shots). In the plot of "Confidence level," it's clear that greater than 99% confidence can be reached across all three circuit sizes—arriving there just takes more shots and averaging as the number of qubits increases. Still, achieving 99% confidence only takes about 1000 shots with Q-CTRL.

Achieving this level of confidence isn't possible above 10 qubits without Q-CTRL. Because selectivity is negative, increasing the number of shots will continue to generate a random spread of results.

Confidence level

Screenshot 2023-12-01 at 5.18.03 PM.png-3

Quantum Fourier transform

The following set of figures show the success probability and selectivity of running the Quantum Fourier transform algorithm up to 14 qubits. Since this algorithm is more complex than the previous, scaling to higher qubit counts poses a greater challenge. However, Q-CTRL performance management still increases the size of the circuit you can run from 8 up to 14, while still achieving meaningful results.

Screenshot 2023-12-01 at 12.47.37 PM.png-4

Screenshot 2023-12-01 at 12.47.48 PM.png-5

Quantum phase estimation

The following figures show the success probability and selectivity for the quantum phase estimation algorithm. Using the Q-CTRL performance management strategy, you can achieve meaningful results running this algorithm up to 16 qubits, compared to only 8 qubits without Q-CTRL.

Screenshot 2023-12-01 at 1.00.56 PM.png-6

Screenshot 2023-12-01 at 1.02.27 PM.png-7

Greenberger–Horne–Zeilinger (GHZ) state

Entanglement is a resource used in quantum computing that links qubits together in a truly “non-classical” way. Generating a large entagled GHZ state is notoriously difficult.

The current record is a 32-qubit GHZ state, realized earlier this year. The combination of Q-CTRL and IBM Quantum has surpassed that record. It's possible to generate GHZ states up to about 60 qubits using Q-CTRL's error suppression.

Here, Hellinger fidelity is a metric that compares the expected and actual outputs of the machine following state generation—higher is better. The horizontal dashed line near 0.5 is a typical threshold used to identify whether a state passes a test as verifiably entangled. At 60 qubits, the value is 0.501.

Screenshot 2023-12-01 at 6.22.38 PM.png-8

The following table provides the numerical data for previous figure.

Number of qubitsQ-CTRL + IBM Hellinger fidelityHellinger fidelity without Q-CTRL
300.7820.451
400.6940.380
500.5700.224
600.5010.020
800.3120
1000.1110

Quantum approximate optimization algorithm (QAOA)

The algorithmic performance of the quantum approximate optimization algorithm (QAOA) was benchmarked using randomly generated MaxCut problems of 3-regular graphs (where each node is connected to exactly three other nodes) with a unique solution. In the QAOA implementation benchmarked, p=1, where p is an integer parameter which dictates the depth of the ansatz, and thus affects the approximation quality.

Since QAOA is an iterative algorithm, the individual metrics generated after each circuit execution (the quantum piece of the quantum-classical hybrid algorithm) are not as useful as a measure of the overall performance of the algorithm. Rather, it's more indicative to look at the quality of the final solution.

In the case of the MaxCut problem, two metrics are relevant:

  1. Measured-to-ideal max cut ratio: The ratio of $C_\text{true}/C_\text{measured}$, where $C_\text{true}$ is the true max cut value as found using classical methods, and $C_\text{measured}$ is the highest max cut value determined by the QAOA algorithm. The ideal value is 1, indicating that the correct solution was found.
  2. Probability gain over random sampling: The ratio of the probability of measuring the true solution from a randomly sampled cut compared to a sample from the distribution generated by the optimal QAOA circuit.

The following table indicates that for QAOA problems up to and including 40 qubits, the correct solution was found using Q-CTRL performance management, as indicated by the measured-to-ideal max cut ratio value of 1. In the case of 50 qubits, the measured-to-ideal max cut ratio was 0.97, indicating that while the ideal solution was not found, quality alternatives were found that were not far from ideal.

At 50 qubits, the probability gain over random sampling value indicates that you are 229,300× more likely to select a correct max cut value from executing QAOA using Q-CTRL performance management than if you were to randomly choose a cut.

Number of qubitsMeasured-to-ideal max cut ratioProbability gain over random sampling
28156,600
321520,000
40129,800
500.97229,300

Conclusion

This document provides an indication of the performance improvement you can expect to achieve across various algorithms using Q-CTRL performance management on IBM Quantum services. The reported metrics represent current mean values across multiple benchmarking experiments, but they will continue to improve further over subsequent weeks and months as Q-CTRL methods are continuously evolving.

Validating and reproducing these metrics is welcomed and encouraged. Learn how to get started with Q-CTRL performance management, and run the Bernstein–Vazirani algorithm.

Was this useful?

cta background

Ready to use Q-CTRL Embedded?

Maximize the utility of IBM Quantum services with Q-CTRL performance management.

Go to IBM Cloud