iqm.benchmarks.benchmark_definition.Benchmark

class iqm.benchmarks.benchmark_definition.Benchmark(backend: str | IQMBackendBase, configuration: BenchmarkConfigurationBase, **kwargs: dict[str, Any])

Bases: ABC

A base class for running cortex-based Benchmark experiments.

In order to write a new benchmark, it is recommended to derive a class from this baseclass. A new benchmark is defined by deriving from this base class and implementing the execute method. Additionally, a custom analysis for the benchmark can be implemented by giving the pointer to the analysis method in analysis_function field. The given analysis_function should accept AnalysisResult as its input and return the final result.

Attributes

default_options
options

Methods

analysis_function(result)	The default analysis that only pass the result through.
analyze([run_index])	The default analysis for the benchmark.
execute(backend)	Executes the benchmark.
name()	Return the name of the benchmark.
run()	Runs the benchmark using the given backend.

Parameters:

• backend (str | IQMBackendBase)

• configuration (BenchmarkConfigurationBase)

• kwargs (dict[str, Any])

static analysis_function(result: BenchmarkRunResult) → BenchmarkAnalysisResult

The default analysis that only pass the result through.

Parameters:

result (BenchmarkRunResult)

Return type:

BenchmarkAnalysisResult

abstractmethod classmethod name() → str

Return the name of the benchmark.

Return type:

str

abstractmethod execute(backend: IQMBackend | IQMFacadeBackend) → Dataset

Executes the benchmark.

This method should be overridden by deriving classes.

Parameters:

backend (IQMBackend | IQMFacadeBackend) – Qiskit backend used to execute benchmarks.

Returns:

An xarray dataset which contains to results of the benchmark execution. The dataset should contain all the information necessary for analysing the benchmark results.

Return type:

Dataset

run() → BenchmarkRunResult

Runs the benchmark using the given backend.

Returns:

The result of the benchmark run.

Return type:

RunResult

analyze(run_index: int = -1) → BenchmarkAnalysisResult

The default analysis for the benchmark.

Internally uses the function defined by the attribute analysis_function to perform the analysis. This function makes a shallow copy of the dataset produced by run. Therefore, it is recommended to not make changes to the data of the dataset but just the structure of the array.

Parameters:

run_index (int) – Index for the run to analyze.

Returns:

An analysis result constructed from the run and updated by the analysis method defined by the analysis_function field.

Return type:

BenchmarkAnalysisResult