# How to use the standard error function of benchmark functions¶

This section explains how to use the standard error function added to the benchmark function.

## Install Libraries and model preparation¶

We can branch an additional feature/error_bar repository into own repository, which we forked from OpenJij’s repository.

Antiferomagnetic one-dimensional Ising model preparation,

[ ]:

import random
import numpy as np
import matplotlib.pyplot as plt
import openjij as oj

# set antiferomagnetic one-dimensional Ising model
N = 30
h = {0: -10}
J = {(i, i+1): 1 for i in range(N-1)}


## Benchmark function¶

The usage is the same as before.

Auguments are unchanged. The following calculations are now stored in a new dictionary type as the return value.

• se_success_prob: a list of standard errors for success probability
standard deviation of the expected value of success probability when we anneal iteration times. the values are stored for each step_num
• se_residual_energy: a list of standard errors for residual energy
standard deviation of the average value of residual energy when we anneal iteration times. the values are stored for each step_num
• se_lower_tts: a list of lower errors in TTS The lower error of TTS computed based on the upper error of success probability

• se_upper_tts: a list of upper errors in TTS The upper error of TTS computed based on the lower error of success probability

[2]:

# set optimal solution
correct_state = [(-1)**i for i in range(N)]

# set step_num and iteration
step_num_list = list(range(10, 101, 10))  # [10, 20,,,, 100]
iteration = 100

# compute TTS, residual energy, success probability, specimen average & standard error
result = oj.solver_benchmark(
solver=lambda time, **args: sampler.sample_ising(h,J,num_sweeps=time),
time_list=step_num_list, solutions=[correct_state], p_r=0.99
)

[1]:

# option: seaborn is good at visualization
# !pip install seaborn
import seaborn as sns
sns.set()


### TTS, residual energy and success probability¶

We can visulalize TTS, residual energy, success probability as before.

[8]:

fig, (axL,axC,axR) = plt.subplots(ncols=3, figsize=(15,3))

fontsize = 10
axL.plot(result['time'], result['tts'], color = 'black')
axL.set_xlabel('annealing time', fontsize=fontsize)
axL.set_ylabel('TTS', fontsize=fontsize)

axC.plot(result['time'], result['residual_energy'], color = 'black')
axC.set_xlabel('annealing time', fontsize=fontsize)
axC.set_ylabel('Residual energy', fontsize=fontsize)

axR.plot(result['time'], result['success_prob'], color = 'black')
axR.set_xlabel('annealing time', fontsize=fontsize)
axR.set_ylabel('Success probability', fontsize=fontsize)

[8]:

Text(0, 0.5, 'Success probability')


### Standard error¶

The value of standard error is stored in se_success_prob & se_residual_energy. These allow us to display standard error bars.

[9]:

# check for standard errors
print(result['se_upper_tts'])
print(result['se_lower_tts'])
print(result['se_residual_energy'])
print(result['se_success_prob'])

[13388.196321142404, 9911.102719950344, 7710.473222454254, 11779.439837899115, 14848.876789318652, 16224.713930223505, 23163.040767149592, 34171.31829135721, 13500.864960678293, 33035.50440761748]
[10060.731171089232, 5437.479680351055, 3893.4360537639604, 5714.889476543427, 5937.451546843923, 6243.406315991658, 6459.91006968379, 9529.994155075381, 4097.1222783392495, 12231.748228729884]
[2.7689475811345194, 2.280882338575193, 2.0175993335414915, 2.060106891411318, 1.9508092079074686, 1.8214213564070665, 1.8063859784194247, 1.6497321484459715, 1.639290312077982, 1.50137983673067]
[0.14070529413628965, 0.27265992434429076, 0.30151134457776363, 0.31446603773522014, 0.3684529491774706, 0.3775251680686369, 0.4351941398892446, 0.4351941398892446, 0.42295258468165076, 0.38612291966536916]

[10]:

# visualization
fig, (axL,axC,axR) = plt.subplots(ncols=3, figsize=(15,3))

fontsize = 10
axL.plot(result['time'], result['tts'])
axL.errorbar(result['time'], result['tts'], yerr = (result['se_lower_tts'],result['se_upper_tts']), capsize=5, fmt='o', markersize=5, ecolor='black', markeredgecolor = "black", color='w')
axL.set_xlabel('annealing time', fontsize=fontsize)
axL.set_ylabel('TTS', fontsize=fontsize)

axC.plot(result['time'], result['residual_energy'])
axC.errorbar(result['time'], result['residual_energy'], yerr = result['se_residual_energy'], capsize=5, fmt='o', markersize=5, ecolor='black', markeredgecolor = "black", color='w')
axC.set_xlabel('annealing time', fontsize=fontsize)
axC.set_ylabel('Residual energy', fontsize=fontsize)

axR.plot(result['time'], result['success_prob'])
axR.errorbar(result['time'], result['success_prob'], yerr = result['se_success_prob'], capsize=5, fmt='o', markersize=5, ecolor='black', markeredgecolor = "black", color='w')
axR.set_xlabel('annealing time', fontsize=fontsize)
axR.set_ylabel('Success probability', fontsize=fontsize)

[10]:

Text(0, 0.5, 'Success probability')


As annealing time per session increased, error bars of the standard errors for success probability and residual energy become shorter and shorter.