{ "cells": [ { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "V_NR6GiIBDP2" }, "source": [ "# 2-An Evaluation of Annealing Algorithms" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "view-in-github" }, "source": [ "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/OpenJij/OpenJijTutorial/blob/master/source/en/002-Evaluation.ipynb)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "FbP_6wGgBDP6" }, "source": [ "Annealing algorithms are heuristics, so it may not be able to give an optimal solution every time. These are algorithms for approximate solutions. In addition, these are probabilistic algorithm and solutions are also different each time. Therefore, when we evaluate them, we use various averages to evaluate these solution.\n", "\n", "The three following indicators are often used.\n", "\n", "- Success probability\n", "- TTS : Time to solution\n", "- Resudial energy\n", "\n", "In particular, **TTS** is a measure of computation time and is often used in various evaluations. A Residual energy is an average value of how close to an optimal solution we got." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "av11BMPjBDP8" }, "source": [ "## Time to solution\n", "\n", "An annealing algorithm can produce some kind of solution at any computation time. However even if the calculation is fast, it is useless if it gives wrong answers. We set an index (e.g., the time it takes to get the optimal solution with 90% probability) for the computation time it takes for the optimal solution to be calculated with the probability we need.\n", "\n", "As shown in the previous chapter, an annealing algorithm looks for the optimal solution from among multiple runs, therefore multiple runs should be taken into account to evaluate the computation time.\n", "\n", "A Time to Solution (TTS) is computed by taking into account the multiple annealing process.\n", "\n", "We can easily lead TTS as follows." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "kqwBOHdVBDP9" }, "source": [ "One annealing time is defined as $\\tau$. Let the probability of calculating the optimal solution in one annealing session be $p_s(\\tau)$. $p_s(\\tau)$ is the probability of success used to evaluate an algorithm.\n", "\n", "From these definition, a failure probability that the optimal solution is not calculated in one annealing session is\n", "\n", "$$1-p_s(\\tau)$$\n", "\n", "We repeat $R$ times. Then the probability that the optimal solution is not calculated in all these $R$ times is \n", "\n", "$$\\{ 1-p_s(\\tau) \\}^R$$\n", "\n", "Therefore the probability of obtaining the optimal solution at leaset once out of $R$ times $p_R$ is found to be\n", "\n", "$$p_R = 1-\\{ 1-p_s(\\tau)\\}^R$$\n", "\n", "We solve this equation for $R$, and we get immediately\n", "\n", "$$R = \\frac{\\ln(1-p_R)}{\\ln\\{1-p_s(\\tau)\\}}$$\n", "\n", "To get the total computation time, we multiply the time per one annealing calculation by this formula.\n", "\n", "$${\\rm TTS}(\\tau, p_R) = \\tau R = \\tau \\frac{\\ln(1-p_R)}{\\ln\\{1-p_s(\\tau)\\}}$$\n", "\n", "This value means the total computation time for one annealing session, taking into account the computation time of $p_R$ and the number of iterations until the optimal solution is found with probability $p_s(p_R)$ when the algorithm with probability $p_s(\\tau)$ is used to find the optimal solution." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "UH5K7TpfBDP-" }, "source": [ "In an evaluation of the actual computation, $p_R$ is given as a constant. The most common value used in research and other studies is $p_R = 0.99$. Then calculate $p_s(\\tau)$ in various annealing time $\\tau$. We use them to compute ${\\rm TTS}(\\tau, p_R)$." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Y_aOax-OBDP_" }, "source": [ "### Measuring TTS with OpenJij\n", "\n", "In this section, we measure TTS with OpenJij. In the following, We consider a one-dimensional antiferromagnetic Ising model. This is the physical model represented by the following Hamiltonian\n", "\n", "$$H(\\{\\sigma_i\\}) = \\sum_{i=0}^{N-1} J_{i, i+1}\\sigma_i \\sigma_{i+1} + \\sum_{i=0}^{N-1} h_i \\sigma_i$$\n", "\n", "where $J_{ij} \\in [0.1, 1.0]$\u3001$h_0 = -1$ respectively, and other longitudinal fields consider zero.\n", "\n", "From antiferromagnetic condition, $J_{ij} > 0$, each spin faces a differenct direction and its energy is lowered. Therefore, optimal solution looks like $\\{\\sigma_i\\}$\u306f$1, -1, 1, -1, \\cdots$. In addition, from $h_0=-1$ condition, we get the 0-th spin is $\\sigma_0 = 1$. Finally, the optimal solution is $1, -1, 1, -1, \\cdots$.\n", "\n", "In short, the problem of \"compute TTS of this problem\" means total computation time it takes to obtain $1, -1, 1, \\cdots$.\n", "\n", "Let's solve the Ising model above and see how TTS chanegs as the time per calculation is extended." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import random\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "# import OpenJij\n", "import openjij as oj " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# make one-dimensional antiferromagnetic Ising model\n", "N = 30\n", "h = {0: -10}\n", "J = {(i, i+1): 1 for i in range(N-1)}" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "63j1bHf1BDQR" }, "source": [ "## Compute TTS\n", "\n", "The response class returned by sample_ising or sample_qubo of openjij has member variable info. This is a dictionary of information for each sampler. Most sampler have an execution_time key, which is the execution time of each algorithm (in $\\mu$s). \n", "\n", "In the case of SASampler, the computation time per one cycle of Simulated Annealing is stored." ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "29.743169725406915\n", "40.29410003568046\n", "55.23409017769154\n", "80.4729093943024\n", "88.36826986225788\n" ] } ], "source": [ "# set optimal solution\n", "correct_state = [(-1)**i for i in range(N)]\n", "\n", "# define pR for TTS\n", "pR = 0.99\n", "\n", "# define num_sweeps_list, which is argument of Sampler \n", "# num_sweeps means the number of splits for decraesing parameters (temperature, transverse field) during annealing.\n", "# therefore, the more we increase this parameter, the more slowly it is equivalent to annealing.\n", "\n", "num_sweeps_list = list(range(10, 51, 10)) \n", "\n", "TTS_list = [] # define empty list for TTS of each computation\n", "tau_list = [] # define empty list for computation time\n", "\n", "# define empty list for success probability\n", "ps_list = []\n", "\n", "# set the number of times of annealing for computing probability\n", "num_reads = 100\n", "\n", "for num_sweeps in num_sweeps_list:\n", " sampler = oj.SASampler(num_sweeps=num_sweeps, num_reads=num_reads) \n", " response = sampler.sample_ising(h, J)\n", "\n", " # get execution time of each annealing\n", " tau = response.info['execution_time']\n", " \n", " # get ps. \n", " # state is ndarray, and we can compare this list with .all()\n", " ps = sum([1 if (state == correct_state).all() else 0 for state in response.states])/num_reads\n", " \n", " \n", " # When ps = 0, TTS diverges to infinity. We avoid this situation\n", " if ps == 0.0:\n", " continue\n", " \n", " # compute TTS\n", " TTS_list.append(np.log(1-pR)/np.log(1-ps)*tau)\n", " tau_list.append(tau)\n", "\n", " # compute success probability\n", " ps_list.append(ps)" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Text(0, 0.5, 'Success probability')" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4gAAADQCAYAAABFhU/JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de7glVXnn8e+PRsQb98YgDWkc28yAUQbOIJk40YBig4bGC7FJDESZ9GDAKE4u8MQk48yYQZNIZKJgjyBgIoiKoVWUMCiaKA10y0VaIXTAS4cOFzEIIQIN7/xR6wybw7nsbs45+5zd38/z7GdXrVqr6q16mrN4d1WtlapCkiRJkqRtBh2AJEmSJGluMEGUJEmSJAEmiJIkSZKkxgRRkiRJkgSYIEqSJEmSGhNESZIkSRIA2w46gNm222671eLFiwcdhiRtFdauXXtPVS0cdBya++yfJWn2TNY/b3UJ4uLFi1mzZs2gw5CkrUKS7w06Bs0P9s+SNHsm6599xFSSJEmSBJggSpIkSZIaE0RJkiRJEmCCKEmSJElqTBAlSZIkSYAJoiRJkiSpMUGUJEmSJAEmiJIkSZKkxgRRkiRJkgSYIEqSJEmSGhNESZIkSRJggihJkiRJakwQJUmSJEmACaIkSZIkqTFBlCRJkiQBJoiSJEmSpMYEUZKkrUCSpUluSbI+ySnjbE+SM9r2G5Mc0LPtnCR3JblpTJtdklye5Nb2vfOY7XsneSDJb8/cmUmSppMJoiRJQy7JAuBDwOHAvsAxSfYdU+1wYEn7rADO7Nl2LrB0nF2fAlxRVUuAK9p6r9OBLz7V+CVJs8cEUZKk4XcQsL6qbquqh4ELgWVj6iwDzq/OamCnJHsAVNXXgHvH2e8y4Ly2fB5w1OiGJEcBtwHrpvVMJEkzygRRkqThtyfwg571Da1sc+uM9dyq2gjQvncHSPIs4PeA9zyFmCVJA2CCKEnS8Ms4ZbUFdfr1HuD0qnpg0qCSFUnWJFlz9913b+GhJEnTadtBByBJkmbcBmCvnvVFwB1bUGesO5PsUVUb2+Ood7XylwJvTPJ+YCfgsSQ/qaq/6G1cVSuBlQAjIyNbmoxKkqaRdxAlSRp+1wJLkuyTZDtgObBqTJ1VwLFtNNODgftGHx+dxCrguLZ8HHAJQFX9p6paXFWLgT8H/nhscihJmptmPEFMsiDJdUk+39aPTrIuyWNJRsbUPbUNr31Lklf3lB+Y5Ftt2xlJ0sqfnuSTrfzqJItn+nwkSZpvqmoTcBJwGfAd4KKqWpfkhCQntGqX0g0qsx74P8BvjrZPcgFwFfAzSTYkOb5tOg14VZJbgVe1dUnSPDYbj5i+g64z2qGt3wS8HvhIb6U23PZyYD/gecD/TfLCqnqUbqjtFcBqug5sKd2w2ccDP6qqFyRZDrwPeNOMn5EkSfNMVV1K14f2lp3Vs1zAiRO0PWaC8h8Ch05x3P+2ubFKkgZnRu8gJlkEvAb46GhZVX2nqm4Zp/oy4MKqeqiqbqf7BfOg9k7DDlV1Veu8zufxYbR7h9f+NHDo6N1FSZIkSdLmmelHTP8c+F3gsT7qTjS89p5teWz5E9q0x2fuA3Ydu2NHSZMkSZKkqc1YgpjktcBdVbW23ybjlNUk5ZO1eWJB1cqqGqmqkYULF/YZjiRJkiRtXWbyDuLPA0cm+S5wIXBIkr+cpP5Ew2tvaMtjy5/QJsm2wI7AvdMRvCRJkiRtbWYsQayqU6tqURviejnw5ap68yRNVgHL28ik+wBLgGvaENv3Jzm4vV94LG0YbZ44vPYb2zGcR0mSJEmStsBsjGL6BEleB/xvYCHwhSTXV9Wr23DbFwHfBjYBJ7YRTAHeBpwLPINu9NIvtvKzgY8nWU9353D57J2JJEmSJA2XWUkQq+pK4Mq2/FngsxPUey/w3nHK1wAvGqf8J8DR0xiqJEmSJG21ZnoUU0mSNE2S7DLoGCRJw80EUZKk+ePqJJ9KcoTz/kqSZoIJoiRJ88cLgZXArwHrk/xxkhcOOCZJ0hAxQZQkaZ6ozuVVdQzwn+lG8r4myVeT/NyAw5MkDYFZH8VUkiRtmSS7Am+mu4N4J/B2uimf9gc+BewzuOgkScPABFGSpPnjKuDjwFFVtaGnfE2SswYUkyRpiPiIqSRJ88e7q+p/9CaHSY4GqKr3DS4sSdKwMEGUJGn+OGWcslNnPQpJ0tDyEVNJkua4JIcDRwB7JjmjZ9MOwKbBRCVJGkYmiJIkzX13AGuAI4G1PeX3AycPJCJJ0lAyQZQkaY6rqhuAG5L8VVV5x1CSNGNMECVJmuOSXFRVvwxcl6TGbq+qFw8gLEnSEDJBlCRp7ntH+37tQKOQJA09E0RJkua4qtrYvr836FgkScPNaS4kSZrjktyf5MfjfO5P8uM+97E0yS1J1id50nQZ6ZzRtt+Y5ICebeckuSvJTWPa7JLk8iS3tu+dW/mrkqxN8q32fchTvQaSpNlhgihJ0hxXVc+pqh3G+TynqnaYqn2SBcCHgMOBfYFjkuw7ptrhwJL2WQGc2bPtXGDpOLs+BbiiqpYAV/D4PI33AL9UVT8LHAd8vN9zlSQNlgmiJElzXJId2vcu43362MVBwPqquq2qHgYuBJaNqbMMOL86q4GdkuwBUFVfA+4dZ7/LgPPa8nnAUa3+dVV1RytfB2yf5On9n7EkaVB8B1GSpLnvE3QD1KwFCkjPtgKeP0X7PYEf9KxvAF7aR509gY2T7Pe5Pe9Hbkyy+zh13gBcV1UPTRGjJGkOMEGUJGmOq6rXtu99tnAXGads7HQZ/dTZvIMm+wHvAw6bYPsKusdZ2XvvvZ/KoSRJ08RHTCVJmkeSvD7JB5L8WZKj+my2AdirZ30RcMcW1BnrztHHUNv3XT1xLgI+CxxbVf8wXuOqWllVI1U1snDhwr5ORJI0s0wQJUmaJ5J8GDgB+BZwE3BCkg/10fRaYEmSfZJsBywHVo2pswo4to1mejBw3+jjo5NYRTcIDe37khbnTsAXgFOr6ut9xCdJmiN8xFSSpPnj5cCLqqoAkpxHlyxOqqo2JTkJuAxYAJxTVeuSnNC2nwVcChwBrAceBN4y2j7JBcArgN2SbAD+qKrOBk4DLkpyPPB94OjW5CTgBcAfJPmDVnZYVf3/O4ySpLnJBFGSpPnjFmBv4HttfS/gxn4aVtWldElgb9lZPcsFnDhB22MmKP8hcOg45f8T+J/9xCVJmltMECVJmuOSfI5uwJgdge8kuaatvxT4xiBjkyQNFxNESZLmvj8ddACSpK2DCaIkSXNcVX110DFIkrYOjmIqSdI8keTgJNcmeSDJw0keTfLjQcclSRoeJoiSJM0ffwEcA9wKPAP4z61MkqRpMeMJYpIFSa5L8vm2vkuSy5Pc2r537ql7apL1SW5J8uqe8gOTfKttOyNJWvnTk3yylV+dZPFMn48kSYNUVeuBBVX1aFV9jG76CUmSpsVs3EF8B/CdnvVTgCuqaglwRVsnyb50E/fuBywFPpxkQWtzJrACWNI+S1v58cCPquoFwOnA+2b2VCRJGqgH20T31yd5f5KTgWcNOihJ0vCY0QQxySLgNcBHe4qXAee15fOAo3rKL6yqh6rqdrqJeg9KsgewQ1Vd1eZoOn9Mm9F9fRo4dPTuoiRJQ+jX6Pruk4B/oZsH8Q0DjUiSNFRmehTTPwd+F3hOT9lzq2ojQFVtTLJ7K98TWN1Tb0Mre6Qtjy0fbfODtq9NSe4DdgXumebzkCRp4Krqe+0O4mLgYuCWqnp4sFFJkobJjN1BTPJa4K6qWttvk3HKapLyydqMjWVFkjVJ1tx99919hiNJ0tyS5DXAPwBn0A1Osz7J4YONSpI0TGbyDuLPA0cmOQLYHtghyV8CdybZo9093AO4q9XfQPeozKhFwB2tfNE45b1tNiTZFtgRuHdsIFW1ElgJMDIy8qQEUpKkeeLPgF9sA9WQ5N8AXwC+ONCoJElDY8buIFbVqVW1qKoW0w0+8+WqejOwCjiuVTsOuKQtrwKWt5FJ96EbjOaa9jjq/W3upwDHjmkzuq83tmOYAEqShtVdo8lhcxuP/9AqSdJTNtPvII7nNOCiJMcD3weOBqiqdUkuAr4NbAJOrKpHW5u3AefSzfn0RR7/pfRs4ONJ1tPdOVw+WychSdJsSfL6trguyaXARXSvVBwNXDuwwCRJQ2dWEsSquhK4si3/EDh0gnrvBd47Tvka4EXjlP+ElmBKkjTEfqln+U7g5W35bmDnJ1eXJGnLDOIOoiRJ2gxV9ZZBxyBJ2jrM6DyIkiRp+iRZlOSzSe5KcmeSz7Q5hyVJmhYmiJIkzR8foxug7Xl0cwF/rpVJkjQtJkwQk/x0kh171n8xyQeTvKtN0itJkmbXwqr6WFVtap9zgYWDDkqSNDwmu4N4EfAsgCT7A5+iG3X0JcCHZz40SZI0xj1J3pxkQfu8GfjhoIOSJA2PyQapeUZVjU5I/2bgnKr6syTbANfPfGiSJGmMtwJ/AZxON83FN1qZJEnTYrIEMT3LhwCnAlTVY9189ZIkabYkWQD8cVUdOehYJEnDa7JHTL+S5KIkH6SbY+nLAEn2AB6ejeAkSVKnqh4FFm7pOABJlia5Jcn6JKeMsz1Jzmjbb0xyQM+2c9rIqTeNabNLksuT3Nq+d+7Zdmrb1y1JXr0lMUuSZt9kCeI7gIuB7wIvq6pHWvlPAb8/w3FJkqQn+y7w9SR/0AaNe1eSd03VqN19/BBwOLAvcEySfcdUOxxY0j4rgDN7tp0LLB1n16cAV1TVEuCKtk7b93Jgv9buwy0GSdIcN9kjppdV1WFjC6vquhmMR5IkTeyO9tkGeM5mtDsIWF9VtwEkuRBYBny7p84y4PyqKmB1kp2S7FFVG6vqa0kWj7PfZcAr2vJ5wJXA77XyC6vqIeD2JOtbDFdtRsySpAGYLEF02GxJkuaQqnoPQJIdutW6v8+mewI/6FnfALy0jzp7Ahsn2e9zq2pji21jkt179rV6nH09QZIVdHcr2Xvvvac+C0nSjJssQdwxyesn2lhVF89APJIkaQJJRoCP0e4eJrkPeGtVrZ2q6ThltQV1+tXXvqpqJbASYGRkZEuPJUmaRpMmiMBrmfiPvAmiJEmz6xzgN6vqbwGSvIwuYXzxFO02AHv1rC+ie1R1c+uMdefoY6htELu7nsK+JElzwGQJ4j9VlXMrSZI0d9w/mhwCVNXfJennMdNrgSVJ9gH+kW4AmV8ZU2cVcFJ7P/GlwH2jj49OYhVwHHBa+76kp/wTST4API9u4Jtr+ohTkjRgkyWID81aFJIkqR/XJPkIcAHd0zxvAq4cnZKiqr45XqOq2pTkJOAyYAFwTlWtS3JC234WcClwBLAeeBB4y2j7JBfQDUazW5INwB9V1dl0ieFFSY4Hvg8c3fa3LslFdIPgbAJObNN0SJLmuHSDlY2zIXmkqp42y/HMuJGRkVqzZs2gw5CkrUKStVU1Mug4hkWSr0yyuarqkFkLZprZP0vS7Jmsf57sDuKdk82tVFUfeMqRSZKkvlXVLw46BknScJssQVwAPJvxB6mRJEmSJA2ZyRLEjVX132ctEkmSJEnSQG0zyTbvHEqSJEnSVmSyBPHQWYtCkiRNKcnRSZ7Tlt+d5OLREUwlSZoOEyaIVXXvbAYiSZKm9AdVdX+SlwGvBs4DzhxwTJKkITLZHURJkjS3jM4l+BrgzKq6BNhugPFIkoaMCaIkSfPHPyb5CPDLwKVJno59uSRpGtmpSJI0f/wycBmwtKr+GdgF+J3BhiRJGiaTTXMhSZLmlj2AL1TVQ0leAbwYOH+wIUmShol3ECVJmj8+Azya5AXA2cA+wCcGG5IkaZiYIEqSNH88VlWbgNcDf15VJ9PdVZQkaVrMWIKYZPsk1yS5Icm6JO9p5S9JclWSbyX5XJIdetqcmmR9kluSvLqn/MBWf32SM5KklT89ySdb+dVJFs/U+UiSNAc8kuQY4Fjg863saQOMR5I0ZGbyDuJDwCFV9RJgf2BpkoOBjwKnVNXPAp+lvVyfZF9gObAfsBT4cJIFbV9nAiuAJe2ztJUfD/yoql4AnA68bwbPR5KkQXsL8HPAe6vq9iT7AH854JgkSUNkxhLE6jzQVp/WPgX8DPC1Vn458Ia2vAy4sKoeqqrbgfXAQUn2AHaoqquqquhexj+qp815bfnTwKGjdxclSRo2VfVt4PeAb7b126vqtMFGJUkaJjP6DmKSBUmuB+4CLq+qq4GbgCNblaOBvdrynsAPeppvaGV7tuWx5U9o097JuA/YdfrPRJKkwUvyS8D1wJfa+v5JVg02KknSMJnRBLGqHq2q/YFFdHcDXwS8FTgxyVrgOcDDrfp4d/5qkvLJ2jxBkhVJ1iRZc/fdd2/uaUiSNFf8N+Ag4J8Bqup6upFMJUmaFrMyimmbzPdKuol9b66qw6rqQOAC4B9atQ08fjcRuqTyjla+aJzyJ7RJsi2wI3DvOMdfWVUjVTWycOHCaTsvSZJm2aaqum9M2ZN+GJUkaUvN5CimC5Ps1JafAbwSuDnJ7q1sG+DdwFmtySpgeRuZdB+6wWiuqaqNwP1JDm7vFx4LXNLT5ri2/Ebgy+09RUmShtFNSX4FWJBkSZL/DXxj0EFJkobHTN5B3AP4SpIbgWvp3kH8PHBMkr8Hbqa7E/gxgKpaB1wEfJvu3YoTq+rRtq+30Y1+up7ujuMXW/nZwK5J1gPvAk6ZwfORJGnQ3k432vdDwCfo3r1/50AjkiQNlWxtN9xGRkZqzZo1gw5DkrYKSdZW1cig4xAkWQp8EFgAfHTs6KftKZ0PAkcADwK/XlXfnKxtkpfQPQn0bOC7wK9W1Y+TPI3uh90DgG2B86vqf00Wn/2zJM2eyfrnWXkHUZIkPXVJLh99faOt75zksj7aLQA+BBwO7Ev3NM++Y6odzuPzDa+gm4N4qrbjzm1MN0r501v5gcB/SbJ4s09YkjTrTBAlSZo/dmsDvwFQVT8Cdu+j3UHA+qq6raoeBi6km0u41zK6O31VVauBndpcxJO1nWhu4wKe1QaQewbdiOU/3sxzlSQNgAmiJEnzx2NJ9h5dSfLT9DeK6URzDfdTZ7K2E81t/GngX4CNwPeBP62qJ40yLkmae0wQJUmaP34f+LskH0/ycbq7d6f20a6feYO3ZD7iieY2Pgh4FHge3TyN/zXJ858UlPMUS9Kcs+2gA5AkSf2pqi8lOQA4mC5xO7mq7umj6URzDfdTZ7uJ2lbVzcBhAEleCLym1fkV4EtV9QhwV5KvAyPAbWPOZyWwErpBavo4D0nSDPMOoiRJ80SS1wGPVNXnq+pzwKYkR/XR9FpgSZJ9kmwHLKebS7jXKuDYdA4G7mtzEU/YdpK5jb8PHNL29Sy6hPbmp3DqkqRZYoIoSdL88UdVdd/oShuw5o+malRVm4CTgMuA7wAXVdW6JCckOaFVu5TuDt964P8AvzlZ29Zm3LmN6UY9fTbdO4rXAh+rqhu3+KwlSbPGR0wlSZo/xvtht6++vKoupUsCe8vO6lku4MR+27byD9LNjzi2/AG6QWskSfOMdxAlSZo/1iT5QJJ/k+T5SU4H1g46KEnS8DBBlCRp/ng73UihnwQ+BfyECe76SZK0JXzEVJKkeaKq/gU4ZdBxSJKGlwmiJEnzRJKv8OT5C6mqQwYQjiRpCJkgSpI0f/x2z/L2wBuATQOKRZI0hEwQJUmaJ6pq7IA0X0/y1YEEI0kaSiaIkiTNE0l26VndBjgQ+KkBhSNJGkImiJIkzR9r6d5BDN2jpbcDxw80IknSUDFBlCRpnqiqfQYdgyRpuDkPoiRJc1yS/5Dkp3rWj01ySZIzxjx2KknSU2KCKEnS3PcR4GGAJL8AnAacD9wHrBxgXJKkIeMjppIkzX0LquretvwmYGVVfQb4TJLrBxiXJGnIeAdRkqS5b0GS0R91DwW+3LPNH3slSdPGTkWSpLnvAuCrSe4B/hX4W4AkL6B7zFSSpGlhgihJ0hxXVe9NcgWwB/A3VVVt0zbA2wcXmSRp2JggSpI0D1TV6nHK/n4QsUiShpfvIEqSJEmSABNESZIkSVJjgihJkiRJAkwQJUmSJEmNCaIkSZIkCZjBBDHJ9kmuSXJDknVJ3tPK90+yOsn1SdYkOainzalJ1ie5Jcmre8oPTPKttu2MJGnlT0/yyVZ+dZLFM3U+kiTNZ0mWtv51fZJTxtme1seuT3JjkgOmapvkJUmuan3055Ls0LPtxW3burZ9+5k/S0nSUzWTdxAfAg6pqpcA+wNLkxwMvB94T1XtD/xhWyfJvsByYD9gKfDhJAvavs4EVgBL2mdpKz8e+FFVvQA4HXjfDJ6PJEnzUutPPwQcDuwLHNP63V6H83g/u4Ku752q7UeBU6rqZ4HPAr/T2mwL/CVwQlXtB7wCeGSmzk+SNH1mLEGszgNt9WntU+0z+gvjjsAdbXkZcGFVPVRVtwPrgYOS7AHsUFVXtYmBzweO6mlzXlv+NHDo6N1FSZL0/x0ErK+q26rqYeBCuj601zLg/NZ/rwZ2an3wZG1/BvhaW74ceENbPgy4sapuAKiqH1bVozN1cpKk6TOj7yAmWZDkeuAu4PKquhp4J/AnSX4A/Clwaqu+J/CDnuYbWtmebXls+RPaVNUm4D5g13HiWNEeZ11z9913T9fpSZI0X0zUx/ZTZ7K2NwFHtuWjgb3a8guBSnJZkm8m+d2nfAaSpFkxowliVT3aHiVdRHc38EXA24CTq2ov4GTg7FZ9vDt/NUn5ZG3GxrGyqkaqamThwoWbexqSJM13/fSXW9IPvxU4Mcla4DnAw618W+BlwK+279clOfRJQfkDriTNObMyimlV/TNwJd27g8cBF7dNn6J7dAW6XyT36mm2iO7x0w1teWz5E9q09x12BO6d9hOQJGl+m6iP7afOhG2r6uaqOqyqDgQuAP6hZ19frap7qupB4FLgAMbwB1xJmntmchTThUl2asvPAF4J3EzXqby8VTsEuLUtrwKWt5FJ96F7Sf6aqtoI3J/k4PZ+4bHAJT1tjmvLbwS+3N5TlCRJj7sWWJJknyTb0Q0Kt2pMnVXAsW0004OB+1ofPGHbJLu3722AdwNntX1dBrw4yTPbD7gvB749s6coSZoO287gvvcAzmujn20DXFRVn0/yz8AHW4fxE7qR0qiqdUkuoutANgEn9rzQ/jbgXOAZwBfbB7rHUz+eZD3dncPlM3g+kiTNS1W1KclJdInbAuCc1u+e0LafRXeX7wi6QeIeBN4yWdu262OSnNiWLwY+1tr8KMkH6JLLAi6tqi/MwqlKkp6ibG033EZGRmrNmjWDDkOStgpJ1lbVyKDj0Nxn/yxJs2ey/nlW3kGUJEmSJM19JoiSJEmSJMAEUZIkSZLUmCBKkiRJkgATREmSJElSY4IoSZIkSQJMECVJkiRJjQmiJEmSJAkwQZQkSZIkNSaIkiRJkiTABFGSJEmS1JggSpIkSZIAE0RJkiRJUmOCKEmSJEkCTBAlSZIkSY0JoiRJkiQJMEGUJEmSJDWpqkHHMKuS3A18bwua7gbcM83hDBuvUX+8TlPzGk1tvlyjn66qhYMOQnOf/fOM8hr1x+s0Na9Rf+bDdZqwf97qEsQtlWRNVY0MOo65zGvUH6/T1LxGU/MaSR3/W5ia16g/XqepeY36M9+vk4+YSpIkSZIAE0RJkiRJUmOC2L+Vgw5gHvAa9cfrNDWv0dS8RlLH/xam5jXqj9dpal6j/szr6+Q7iJIkSZIkwDuIkiRJkqTGBHEcSU5Osi7JTUkuSLJ9kj9JcnOSG5N8NslOg45zkMa7Rj3bfjtJJdltkDHOBRNdpyRvT3JL2/b+Qcc5SBP897Z/ktVJrk+yJslBg45z0JK8o12jdUne2cp2SXJ5klvb986DjlOaSfbP/bGPnpr989Tsn/szjP2zCeIYSfYEfgsYqaoXAQuA5cDlwIuq6sXA3wOnDi7KwZrkGpFkL+BVwPcHF+HcMNF1SvKLwDLgxVW1H/CnAwxzoCb5t/R+4D1VtT/wh219q5XkRcBvAAcBLwFem2QJcApwRVUtAa5o69JQsn/uj3301Oyfp2b/3J9h7Z9NEMe3LfCMJNsCzwTuqKq/qapNbftqYNHAopsbnnSNWvnpwO8CvtzaGe86vQ04raoeAqiquwYY31ww3jUqYIe2fUce//e1tfp3wOqqerD9Hfoq8Dq6/5E5r9U5DzhqQPFJs8X+uT/20VOzf56a/fPUhrJ/NkEco6r+ke4Xo+8DG4H7qupvxlR7K/DF2Y5trpjoGiU5EvjHqrphoAHOEZP8W3oh8J+SXJ3kq0n+wyDjHKRJrtE7gT9J8oO2fau+IwDcBPxCkl2TPBM4AtgLeG5VbQRo37sPMEZpRtk/98c+emr2z1Ozf+7bUPbPJohjtGeElwH7AM8DnpXkzT3bfx/YBPzVYCIcvAmu0bHA79M9biAm/be0LbAzcDDwO8BFSTKwQAdokmv0NuDkqtoLOBk4e3BRDl5VfQd4H92jdF8CbqD7OyRtNeyf+2MfPTX756nZP/dnWPtnE8QneyVwe1XdXVWPABcD/xEgyXHAa4Ffra17fpDxrtFb6P6I3JDku3SP+HwzyU8NLsyBm+jf0gbg4upcAzwGbK2DBUx0jY5rywCfonu2f6tWVWdX1QFV9QvAvcCtwJ1J9gBo31v741AabvbP/bGPnpr989Tsn/s0jP2zCeKTfR84OMkz269GhwLfSbIU+D3gyKp6cKARDt541+jiqtq9qhZX1WK6P7IHVNU/DTLQARv33xLw18AhAEleCGwH3DOwKAdromt0B/DyVucQuj+2W7Uku7fvvYHXAxcAq+g6a9r3JYOJTpoV9s/9sY+emv3z1Oyf+zSM/fO2gw5grqmqq5N8Gvgm3S3i64CVwDrg6cDl7WmD1VV1wsACHaBJrpF6THKdCjgnyU3Aw8BxW+sv3pNco+uAD7YX438CrBhclHPGZ5LsCjwCnFhVP0pyGt0jUMfTdeZHDzRCaQbZP/fHPnpq9s9Ts3/eLEPXP2cr/XcvSZIkSRrDR0wlSZIkSYAJoiRJkiSpMUGUJEmSJAEmiJIkSZKkxgRRkiRJkgSYIEpzTlEJtuQAAAMySURBVJLvJtmtLX9jmva5f5IjetaPTHLKdOxbkqStgf2zthZOcyHNMUm+C4xU1bRNzpvk19s+T5qufUqStDWxf9bWwjuI0jiS/HWStUnWJVnRU/5AkvcmuSHJ6iTPbeXnJjkjyTeS3JbkjT1tfifJtUluTPKeqY4xJo4H2vcrklyZ5NNJbk7yV2kzQic5opX9XYvh82P2sR3w34E3Jbk+yZuS/HqSv+iJ/cwkX2mxvzzJOUm+k+Tcnv0cluSqJN9M8qkkz56GSy1JUt/sn+2fNfNMEKXxvbWqDgRGgN9KsmsrfxawuqpeAnwN+I2eNnsALwNeC5wG3R9tYAlwELA/cGCSX5jiGBP598A7gX2B5wM/n2R74CPA4VX1MmDh2EZV9TDwh8Anq2r/qvrkOPveGTgEOBn4HHA6sB/ws+3xl92AdwOvrKoDgDXAu6aIV5Kk6Wb/bP+sGbbtoAOQ5qjfSvK6trwXXSfyQ+BhYPQXwLXAq3ra/HVVPQZ8e/SXS+Cw9rmurT+77etrkxxjItdU1QaAJNcDi4EHgNuq6vZW5wJg3F87p/C5qqok3wLurKpvteOsa8dZRNfxfb39MLodcNUWHEeSpKfC/hn7Z80sE0RpjCSvAF4J/FxVPZjkSmD7tvmRevzF3Ud54n9DD/Xupuf7f1XVRzbjGBPp3f/osTNB3c01uu/HxhznsXacR4HLq+qYaTqeJEmbxf7Z/lmzw0dMpSfbEfhR6xj+LXDwU9jXZcBbR98HSLJnkt2n8Rg3A89Psritv2mCevcDz9nCYwCspntk5gUASZ6Z5IVPYX+SJG0u++cns3/WtDNBlJ7sS8C2SW4E/gfdH98tUlV/A3wCuKo9HvJpuo5gWo5RVf8K/CbwpSR/B9wJ3DdO1a8A+46+BL8Fx7kb+HXgghbzauDfbknMkiRtIfvnJx/H/lnTzmkupHkuybOr6oE2atqHgFur6vRBxyVJ0tbM/lnzlXcQpfnvN9pL8evoHo35yBT1JUnSzLN/1rzkHURJkiRJEuAdREmSJElSY4IoSZIkSQJMECVJkiRJjQmiJEmSJAkwQZQkSZIkNSaIkiRJkiQA/h8ONGgDErXyZAAAAABJRU5ErkJggg==\n", "text/plain": [ "