{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 6-Machine Learning (QBoost) with Quantum Annealing" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "view-in-github" }, "source": [ "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/OpenJij/OpenJijTutorial/blob/master/source/en/006-Machine_Learning_by_QA.ipynb)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this section, we describe machine lerning (ML) as an example of an application of quantum annealing (QA) optimization.\n", "\n", "In the first, we show clustering using PyQUBO and OpenJij. \n", "In the seconde, we execute an ensamble study called QBoost with PyQUBO and D-Wave sampler." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Clustering\n", "\n", "Clustering is the task of deviding given set of data into $n$ clusters ($n$ is our input). For the sake of simplicity, let us consider the number of cluster is 2 in this time.\n", "\n", "### Importing the required libraries\n", "\n", "We import scikit-learn library for ML." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# import libraries\n", "import numpy as np\n", "from matplotlib import pyplot as plt\n", "from sklearn import cluster\n", "import pandas as pd\n", "from scipy.spatial import distance_matrix \n", "from pyqubo import Array, Constraint, Placeholder, solve_qubo\n", "import openjij as oj\n", "from sklearn.model_selection import train_test_split" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Make artificial data\n", "\n", "In this case, let us generate linearly separable data in a two-dimensional plane artificially." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "data = []\n", "label = []\n", "for i in range(100):\n", " # generate 0 to 1 random number\n", " p = np.random.uniform(0, 1)\n", " # set class 1 when certain condition are met, and -1 when it are not met\n", " cls =1 if p>0.5 else -1\n", " # create random numbers following a normal distribution\n", " data.append(np.random.normal(0, 0.5, 2) + np.array([cls, cls]))\n", " label.append(cls)\n", "# formatted as a DataFrame\n", "df1 = pd.DataFrame(data, columns=[\"x\", \"y\"], index=range(len(data)))\n", "df1[\"label\"] = label" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEGCAYAAABsLkJ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAbyElEQVR4nO3df5BdZXkH8O9z91difhjcjSEkwUgXq8EKwi1al7EYnJZGiFWU0RaVak1phw62tQkOVdvaqbPo+Ac/OjQDTKWlagRtUsFR6OJoqKRsmE2ARGtw1OyKkmwDsiG5u3v36R/33uTu3XPvPeee8573fc/5fmYYN/fe3Pvck/V93h/P+x5RVRARUf4UbAdARER2MAEQEeUUEwARUU4xARAR5RQTABFRTnXbDiCKgYEBXb9+ve0wiIi8snfv3qOqurLxca8SwPr16zE6Omo7DCIir4jIT4Me5xQQEVFOMQEQEeUUEwARUU4xARAR5RQTABFRTllLACKyTkQeEZEDIvK0iNxgKxYi6szkVAn7Dj+PyamS7VCoAzbLQGcB/JWqPiEiywDsFZGHVPWAxZiIKKSdYxPYdv9+9BQKmJmbw81XvQGbL1hjOyyKwNoIQFWfVdUnqj+/COAgAP72EHlgcqqEbffvx8mZObxYmsXJmTlsvX9/YiMBjizS4cRGMBFZD+CNAPbYjYSIwhg/dgI9hQJOYu7UYz2FAsaPnUD/0r5Y782RRXqsLwKLyFIA9wP4mKr+KuD5LSIyKiKjR44cST9AIlpg7RmLMTM3N++xmbk5rD1jcaz3NT2yoPmsJgAR6UGl8b9XVb8W9BpV3a6qRVUtrly54CgLIrKgf2kfbr7qDVjUU8Cyvm4s6ing5qveELv3XxtZ1KuNLCh51qaAREQA3AXgoKp+wVYcRNSZzReswdDgAMaPncDaMxbHbvwBcyMLCmZzBDAE4AMANorIWPW/TRbjIaKI+pf24fx1KxJp/GvvZ2JkQcGsjQBUdTcAsfX5ROQmEyMLCuZEFRARUb3+pX1s+FNgvQqIiIjsYAIgIsopJgAiopxiAiAiyikmACLP8dwc6hSrgIg8xnNzKA6OAIg8xXNzKC4mACJP8dwciosJgMhxzeb40zo3h2sM2cU1ACKHtZrjr52bs7Xh+SR30HKNIdtEVW3HEFqxWNTR0VHbYRClYnKqhKHhEZycOd3LX9RTwKPbNs5r5CenSkbOzQn7+eQ+EdmrqsXGxzkFROSosHP8SZ/IGfXzyV9MAESOsn02vu3PJ/OYAIgcZftsfNufT+ZxDYDIcabm+H35/DTY/o6mP7/ZGgCrgIgcZ/tsfNufb5rtSiebn88pICLKLZO7qcPsn7C9m5sjACLKrVql00mcXuyuVTrFGfWE7dWb+vywOAIgotwyUekUpVdvu9KKCYCIcstEpVOU/RO2K604BUREXjBVKbP5gjUYGhxI7L2j9uqT/vwomACIKHVRG3PTlTJJVjp1ckaTrUorJgAiSlW7xrwxOdTPqdcWS7fevx9DgwPOlqfa7NVHwQRARKlp15gHJYdX9S+xWinTKR/2T3ARmDIjj+fW+/adWy2QNqueWdLbxTOJDOEIgDLB9m5OG3z8zq0WSMePnYDOzT+aRucUx6fLxu97kFdMAOQ9H+eI4/L1O7daID12fBql8vwEUCorlvR2eTOn7hsmAPKe7d2UNvj8nZs15seny1jUU5h3A5rergJ+/sIJDK5a5sWcum+4BkDes72b0oZm33lJb5cXawJBN7EJ+veaLs/ho/eMYtfYRJrh5QYTAHnP9m5KG4K+89UXrcUVt+3GNXfuwdDwiHeNZv/SPnzyHRvQ0yXzHi/NaqoHpOUJp4AoE/I4R1z/nZf0duGK23ZbXROIu1N359gEPvPAAXQXCpgpl+c958v0lm+YACgz8jhHXPvO+w4/b3VNIG5FUv2idpCsT+nZwikgogywuQ6SxJn2QfsDAOBlPV1tp/R82wvhEo4AiNqwfbvAMDo5fyYpSVQkBSWwvu4C7vjARTjvrOVN38fHvRAuYQIgasGnBsbWOkgSo49mCeytr1nZ9O/4thfCxY4EEwBRE741MICddZCkRh9REtjkVAmP/OA5dMn8iiFXF4td7UgwARA14fNmq7QlNfoIk8BqjWl3QXB8en61kIuLxS53JKwuAovI3SLynIg8ZTMOoiAubjBzecEzaHNX0uob06nS6cZ/SW/7xWJbotwhLG22RwD/AuA2APdYjoNoAZsLq0FsTCO4Nm8dNCpb0teFv7vyPLztta90IsZGLnYkaqwmAFX9roistxkDUSuubDCzMY1ga966VdIJakzLc+ps4w+415GoZ3sE0JaIbAGwBQDOPvtsy9FQHrmwwSzt9Qhb89btko7LjWkrrnQkGjmfAFR1O4DtAFAsFrXNy4kyKe1pBBsL4EFJ5+Nf3YcNq5djcNWyU68z2ZianPJyoSPRiDuBiTzQ7sC7pBeHbcxbBy2WTpcVm27dveBguzALzlGvyc6xCQwNj3h7mF4nnB8BEFFFs56vibl6G1MtQUkHAKZn5yJPP0W9Ji6XappkNQGIyJcAXApgQETGAXxaVe+yGRORyxqnEZJuuOqnQOJOtUSdTqklnY9/dR+mG+4MFmX6qZNrktc9H7argN5v8/OJfJdkw9Ws19zp0c6djEo2X7AGG1Yvx6Zbd2N69vR3ijL9FPaa1CcoF0s16+MDYGRtglNARB4Larimy+XIDdfkVAlb79uH0qzGHknEHZUMrlqGz7+ns+mnyakSXjgxjely6x3CQQnKpeqi+vhOzpahqljc0514OS4TAJHHatMmf7ljDLUO85wCjx46GqmRuHfPz1Ca7XzapV4So5JOpp/qG805BboLmNdo1i+YByWoR7dtxKPbNlov1QyKDwBeLM2eijWptQkmACLPDQ0OoKtQwGx1JDBT1kiNxORUCbc/cmjB452MJID4FUT1Ux/nr1sR+u80Npp93QXc/ocXLjhOulWCMn2URRhB8dVLcm2CZaBEnmksbxw/dgK9XZ2fNRP09wHg+red21EjE+cezZ2WYgaVkPZ2FfDyxT2hdhPbnu+v16waqibJWDkCIPJI0Nz10OBArAat2c1Y/uBNne+872QKJ87aQZRG3fXdxI3xBa0BJBUrEwCRJ1rNXcdp0Ew1iFF3vsZZO4j6HVw9mqGmMT6AVUBEudaqgWzWoIWtxXehQYw7NRP1O7h4NEO9xvhMxMoEQOSJdg1kY4MRtRbfdoOYxEjE9nfwDRMAkSeiNJC+Hm3gwkgkT5gAiDwStoH0+WgD9uLTwwRA5JkwDaTrpY7kBu4DIMqgOLX4lB8cARBlFOfTqR0mAKIMy/N8ums3tHcREwAROSOpRtvWDe19wwRARE5IqtG2WQLr26iDCYDIc741OkGSbLRtlcD6OOpgAiDymI+NTpAkG20bJbC+brxjGSiRp+obnRdLszg5U7l5eu2YaJ8k2WjbKIENOo46ypHctnAEQOQpn3f7Nkr6RNK0S2B93XjHBEDkKV8bnWaSbrTTLIF1/R4DzTABEHnK10anFZ/3Lfi48Y4JgMhjPjY6WeZbAmMCIPKcb40OuYNVQEREOcUEQESxTE6VsO/w8x2Xn8b9+9Q5TgERUcdabUQLs0M5KxvZfMUEQEQdabX7dfeho20b9iR2z2bhGAybmACIqCNBG9EKIvj+M0dDNexxN7Jx9BAf1wDIS5w3ti9oI9pL02X8xY6xBa8NOhYhzka2LB2DYRMTAHln59gEhoZHcM2dezA0PIJdYxO2Q8ql2ka0vu75zchMGTg5075hj3Nmj69n77iGU0DkFV9PXcyqzReswYqX9eK6f92Ll2bKpx7v6xKoCPq6Wu9Q7nQjW9aOwbCFCYC8kqUD0LLivLOWYw467zEpCB64/hIcny63bdg72ciWxWMwbGACIK+w5+eeZo3x4KplRj+Xx2DExwRAXmHPL7w0SyRtNcY8BiMeJgDyDnt+7dkokWRj7B8mAPISG5vmuFBOYVktAxWRy0XkhyJySERutBkLUVawRJLCspYARKQLwO0Afg/ABgDvF5ENtuIhygoulFNYNkcAFwM4pKo/VtVpAF8G8E6L8RBlgo2bopOfbK4BrAFwuO7P4wDeZCkWokzhQjmF4fwisIhsAbAFAM4++2zL0RD5gwvl1I7NKaAJAOvq/ry2+tg8qrpdVYuqWly5cmVqwRERZZ3NBPA4gHNF5NUi0gvgfQB2WYyHiChXrE0BqeqsiFwP4FsAugDcrapP24qHiKhR1m840zYBiMifA/g3VT2W9Ier6oMAHkz6fYmI4srDDWfCTAGtAvC4iOyobtwS00EREdmUlxvOtE0Aqvo3AM4FcBeAawH8SET+UUR+zXBsRKHw7mCUtLzspg61BqCqKiK/APALALMAzgBwn4g8pKpbTQZI1EoehumUvrzspm47AhCRG0RkL4CbATwK4DdU9U8BXATgKsPxkafS6JXnZZhO6cvLbuowI4BXAHi3qv60/kFVnRORK8yERT5Lq1fOu4ORSXnYTd02Aajqp1s8dzDZcChNJkrc0jyKOC/DdLIn67uprR4HTfbsHJvA0PAIrrlzD4aGR7BrbMEm7I6kuXiWl2E6kSnOnwVEyTPZS0+7V56HYTqRKRwB5JDJXrqNXnn/0j6cv24FG3+iiDgC8FDcuXvTvXT2yon8wATgmSQqbGq99K0N75NkQ531xTOiLGAC8EjQ3P3H79uPDauXY3DVskjvxV46EXENwCNBc/fTs3PYdMv3Oqri8XHunMc+ECWHIwCPBM3dA8B0WY3V2ruExz4QJYsjAI/U5u57uxf+s7lyUJWpHjqPfSBKHkcAntl8wRpsWL0cm275HqbLeupxF3bAmuyh89gHouRxBOChwVXL8Pn3nu/UDljTPXQe+0CUPI4APOVaFY/pHnoapatEecMEkBAb9w51qdY+jR66a0mPyHdMAAnwpTrFZJJKq4fuUtIj8h0TQExpHn8cRxpJKskeuo0RFVHeMAHE5EN1SppJKokeui8jKiLfsQooJh+qU0ye/pl03T/r/YnSwwQQkw83JTGVpEzcVCbNG8oQ5R2ngBLgenWKiQVaU9NKPoyoiLKCCSAhrlenJJ2kTK19sN6fKD1MADmSZJIy2VN3fUQVBquYyAdMANQR0z31MMnK1UaWVUzkCyYAiqS+0bXZU3e1kfVlXwgRwARAETRrdNNu2FxuZH3YF0JUwzJQCsWl+vynf/4CCiLzHnOlVJRVTOQTJgAKxZX6/J1jE/joPaN4abo873FXGlkf9oUQ1XAKiEIJ07M1vShbG4WUZnXe433dbjWyWahionxgAqBQ2lX9pLEoGzS//rKeLtzxgYvw1tesTPSz4nJ9XwgRwARAETTr2aa1KBs0CpmD4ryzlif2GUR5wjUAiqR/aR/OX7diXsOe1voA59eJksURAMWWZuUL59eJksMRQAYkfSRzVGn3zINGIUQUnZURgIi8F8DfAngdgItVddRGHFngyo7YuD1zV491IMoyW1NATwF4N4B/tvT5meDajthOK19cSWJRMWmR76wkAFU9CADSsJuTosnCsQOuJbGwfE1aRPWcXwMQkS0iMioio0eOHLEdjlNsHDsQtN4QZw3ClR3GUbh0LAZRHMZGACLyMIAzA566SVV3hn0fVd0OYDsAFItFbfPyXOlf2oeri2txz/d/duqxq4trjfWcg3q9CsTqCft4dk4WRl5EgMEEoKpvN/XeVDE5VcKO0fF5j+0YHccNl70msYaoNs+9pLdrwVTNX9+3H4CiNKsdT9/4eAcwH5MWURDuA/CY6Z5ofY+/NFtGoTB/zaarIIAKgNMHs3Xy+b7V9vuYtIiC2CoDfReAWwGsBPCAiIyp6u/aiMVnJnuiQYuzKM+fgSvPKcoNn39yttzR5/t2do5vSYsoiJVFYFX9uqquVdU+VV1luvG3vVHKFJMbsIIWZ/u6BL3dpz/rU1duWPD3VPOzTMMNaeS7zE8BZb1cL0xPtJN69aDRhRQED1x/CY5PV3r548dOYHFPN14szZ56zeKe7rZTQDbq51mzT7RQphOArzXmUbWaPuk0ATab5x5ctWze66JOQdlIyFnvBBB1KtMJIO/lenETYLvRRdTFUBsJOS+dAKJOZDoB5L1cL4kE2G5xNspiaLt4TEzT5L0TQNRKphNA3sv10kqAYSt4WsVjapom7DXgGgHlUaYTAJBeuZ6LDYhrCbBZPACMTdOEuQZprRG4+DtC+Zb5BACYrzF3bZGxvqFxrV49KJ59h583Ok3T6hqktUbg2u8IEZCTBGCSa4uMzRqapI+GiJNMGhNyGlNVzToBaawRuPY7QlTj/GmgrnPpNEvTp1TuHJvA0PAIrrlzD4aGR7BrbCKR9222oQ2A8Q18aSQfl35HiOpxBBCTS5VGJnuzpnuxjdM0uw8dxdDwiPEpkzTWSVz6HSGqxwQQkwsLrfUndppqaNKYKqlN06Q9ZWJ6ncSF3xGiIEwACbC50No45391cS12jI4n3tCY7sXWry3YqN03XSjg2mI8EcAEkJgkGpCoC6xBPeUdo+P4Rt15PbX3CfvezV5nshfbmMQ++Y4NmZwy8e3EU8o+JgBHdFIm2KynfHy6jPPXrYj83u1eZ6IXG5TEPvPAAXzyig34zDcOcMqEyCAmAAd0OucdZlom7HuHfV3SvdhmSez1Z70cj27byCkTIoNYBuqATssEw9wPIOx72ypVbJXEeN4+kVkcATggzgJru2mZsO9tq1SRFTJE9jABOCBuI9hqWibse9tsiFkhQ2SH+HQLv2KxqKOjo7bDMMbkYWFxq4BMfy4RmSMie1W12Pg4RwAOMVkmGPa9k4yBB6ARuY2LwGSE6XOJiCg+JgAyggegEbmPCYCM4AFoRO5jAohocqpk/IjiLAizR4GI7OIicARc1Iymk/JOVg0RpYcJICQf7urkYuMZpaqICZYoXUwAIdk4ojgK3xtPHxIsUdZwDSAklxc1s1ByyaohovQxAYTk8qJmFhpPlxMsUVZxCigCV8+syULjyUPhiNLHBBCRi3d1ykrj6WqCJcoqJoCMqG88l/R24fh0GZNTJe8aURcTLFFWMQFkSP/SPuw+dNTraiDAzXJWoixiAsiQLJRS+l7OSuQTVgFliO/VQFkoZyXyCRNAhvheDeR7AiPyDRNAhri8VyEM3xMYkW+4BpAxPpdSZqWclcgXVhKAiHwOwJUApgE8A+CPVPV5G7H4qlWljM+llD4nMCLf2BoBPATgE6o6KyLDAD4BYJulWLyT9UoZnxMYkU+srAGo6rdVdbb6x8cArLURh49YKUNESXFhEfjDAL7Z7EkR2SIioyIyeuTIkRTDchMrZYgoKcamgETkYQBnBjx1k6rurL7mJgCzAO5t9j6quh3AdgAoFotqIFSvsFKGiJJiLAGo6ttbPS8i1wK4AsBlqpr7hj0sVsoQUVJsVQFdDmArgN9W1ZdsxOAzVsoQURJsVQHdBqAPwEMiAgCPqep1lmLxEitliCguKwlAVQdtfC4REZ3mQhUQERFZwARARJRTTABk3eRUCfsOP8/NbEQp42FwZFXWj7UgchlHAGQNj7UgsosJgKzhsRZEdjEBkDU81oLILiYAssb3O5gR+Y6LwGQVj7UgsocJgKzjsRZEdnAKiIgop5gAiIhyigmAiCinmACIiHKKCYCIKKfEp7sxisgRAD81+BEDAI4afP9OuBgT4GZcjCkcF2MC3IwrKzG9SlVXNj7oVQIwTURGVbVoO456LsYEuBkXYwrHxZgAN+PKekycAiIiyikmACKinGICmG+77QACuBgT4GZcjCkcF2MC3Iwr0zFxDYCIKKc4AiAiyikmACKinMp1AhCRz4nID0Rkv4h8XURWNHnd5SLyQxE5JCI3Go7pvSLytIjMiUjTUi8R+YmIPCkiYyIyajKmiHGlea1eISIPiciPqv97RpPXlavXaUxEdhmKpeX3FpE+EflK9fk9IrLeRBwRY7pWRI7UXZs/TiGmu0XkORF5qsnzIiK3VGPeLyIXOhDTpSLyQt11+lQKMa0TkUdE5ED1/3c3BLwm/rVS1dz+B+B3AHRXfx4GMBzwmi4AzwA4B0AvgH0ANhiM6XUAfh3AdwAUW7zuJwAGUrxWbeOycK1uBnBj9ecbg/79qs9NGb42bb83gD8DcEf15/cB+IoDMV0L4La0foeqn/lWABcCeKrJ85sAfBOAAHgzgD0OxHQpgG+kfJ1WA7iw+vMyAP8b8O8X+1rlegSgqt9W1dnqHx8DsDbgZRcDOKSqP1bVaQBfBvBOgzEdVNUfmnr/ToWMK9VrVX3vL1Z//iKA3zf4Wa2E+d71sd4H4DIREcsxpU5Vvwvg/1q85J0A7tGKxwCsEJHVlmNKnao+q6pPVH9+EcBBAGsaXhb7WuU6ATT4MCrZtNEaAIfr/jyOhf8QNiiAb4vIXhHZYjuYqrSv1SpVfbb68y8ArGryukUiMioij4mIiSQR5nufek210/ECgH4DsUSJCQCuqk4f3Cci6wzGE5ar/3/7LRHZJyLfFJHz0vzg6nThGwHsaXgq9rXK/B3BRORhAGcGPHWTqu6svuYmALMA7nUlphAuUdUJEXklgIdE5AfVnoztuBLVKqb6P6iqikizmuZXVa/VOQBGRORJVX0m6Vg99J8AvqSqJRH5E1RGKBstx+SiJ1D5HZoSkU0A/gPAuWl8sIgsBXA/gI+p6q+Sfv/MJwBVfXur50XkWgBXALhMqxNrDSYA1PeM1lYfMxZTyPeYqP7vcyLydVSG/LESQAJxpXqtROSXIrJaVZ+tDn2fa/IetWv1YxH5Diq9qSQTQJjvXXvNuIh0A3g5gMkEY4gck6rWf/6dqKyp2Jb471Bc9Q2vqj4oIv8kIgOqavSQOBHpQaXxv1dVvxbwktjXKtdTQCJyOYCtADar6ktNXvY4gHNF5NUi0ovKAp6RSpKwRGSJiCyr/YzKYnZgBUPK0r5WuwB8qPrzhwAsGKWIyBki0lf9eQDAEIADCccR5nvXx/oeACNNOhypxdQwX7wZlXlm23YB+GC1wuXNAF6om+azQkTOrK3XiMjFqLSbJpM3qp93F4CDqvqFJi+Lf63SXNl27T8Ah1CZQxur/ler0jgLwIN1r9uEyir8M6hMh5iM6V2ozOWVAPwSwLcaY0KlsmNf9b+nTccUNi4L16ofwH8B+BGAhwG8ovp4EcCd1Z/fAuDJ6rV6EsBHDMWy4HsD+HtUOhcAsAjAV6u/c/8D4JwU/s3axfTZ6u/PPgCPAHhtCjF9CcCzAGaqv08fAXAdgOuqzwuA26sxP4kWlXApxnR93XV6DMBbUojpElTW+fbXtU+bkr5WPAqCiCincj0FRESUZ0wAREQ5xQRARJRTTABERDnFBEBElFNMAEREOcUEQESUU0wARDGIyG9WD1NbVN2h/bSIvN52XERhcCMYUUwi8g+o7PRdDGBcVT9rOSSiUJgAiGKqnrXzOICTqBwTULYcElEonAIiiq8fwFJU7ty0yHIsRKFxBEAUk1TuM/xlAK8GsFpVr7ccElEomb8fAJFJIvJBADOq+u8i0gXgv0Vko6qO2I6NqB2OAIiIcoprAEREOcUEQESUU0wAREQ5xQRARJRTTABERDnFBEBElFNMAEREOfX/I7paso5tG/sAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# visualize dataset\n", "df1.plot(kind='scatter', x=\"x\", y=\"y\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We demonstrate clustering by minimizing the following Hamiltonians.\n", "\n", "$$\n", "H = - \\sum_{i, j} \\frac{1}{2}d_{i,j} (1 - \\sigma _i \\sigma_j)\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Where $i, j$ is sample No., $d_{i, j}$ is a distance between $i$ and $j$, $\\sigma_i=\\{-1,1\\}$ is spin variable that indicates whether $i$ belong to one of the two clusters.\n", "\n", "Each term of this Hamiltonian sum behaves as follows.\n", "\n", "- 0 for $\\sigma_i = \\sigma_j $\n", "- $d_{i,j}$ for $\\sigma_i \\neq \\sigma_j $ \n", "\n", "Note that minus of R.H.S., Hamiltonian means the problem is \"Choosing pairs of $\\{\\sigma _1, \\sigma _2 \\ldots \\}$ that maximizes the distance between the samples of different classes\"." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Clustering with PyQUBO\n", "\n", "At first, we formulate the Hamiltonian in PyQUBO. Second, we execute simulated annealing (SA) with `solve_qubo`." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "def clustering_pyqubo(df):\n", " # set distance matrix\n", " d_ij = distance_matrix(df, df)\n", " # set spin variables\n", " spin = Array.create(\"spin\", shape= len(df), vartype=\"SPIN\")\n", " # set the total Hamiltonian\n", " H = - 0.5* sum(\n", " [d_ij[i,j]* (1 - spin[i]* spin[j]) for i in range(len(df)) for j in range(len(df))]\n", " )\n", " # compile\n", " model = H.compile()\n", " # convert to QUBO\n", " qubo, offset = model.to_qubo()\n", " # solve with SA\n", " raw_solution = solve_qubo(qubo, num_reads=10)\n", " # decode for easier analysis\n", " decoded_solution, broken, energy= model.decode_solution(raw_solution, vartype=\"SPIN\")\n", " # extract label\n", " labels = [decoded_solution[\"spin\"][idx] for idx in range(len(df))]\n", " return labels, energy" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We execute and check a solution." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "label [0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 1.0, 1.0, 0.0]\n", "energy -15262.268540013521\n" ] } ], "source": [ "labels, energy =clustering_pyqubo(df1[[\"x\", \"y\"]])\n", "print(\"label\", labels)\n", "print(\"energy\", energy)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us visualize the result." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAVqElEQVR4nO3df4hlZ33H8c93N7uQwYhksiAk2TvSWqmEgt1BKkILaksaSsWCYLkGJcKWoUIKFmuZv/ePIggBBVlQq86gFFQiNRITqvWfKk4kiEmMBMmuK4Kb7B9dWGGzmW//OHM7s3fOOff8es5znnPeL7hs5s6dc8+c7H7Oud/n+zzH3F0AgHSdiL0DAIB2CHIASBxBDgCJI8gBIHEEOQAk7o4Yb3rPPff4xsZGjLcGgGQ988wzr7j7meXnowT5xsaG9vb2Yrw1ACTLzC7lPU9pBQASR5ADQOIIcgBIHEEOAIkjyAEgcQQ5gOHa3ZU2NqQTJ7I/d3dj79EgRWk/BICVdnel8+elGzeyry9dyr6WpPk83n4NEFfkAIZpe/swxBdu3Miex20IcgDDdPlyvecnjCAHMExnz9Z7fsIIcgDDdOGCtLZ2+3Nra9nzuA1BDmCY5nPp4kVpNpPMsj8vXmSgMwddKwCGaz4nuCvgihwAEkeQA0DiCHIASBxBDgCJI8gBIHEEOQAkjiAHgMS1DnIzu9/Mvm9mz5vZc2b2aBc7BgCoposJQbckfcLdf2pmd0l6xsyecvfnO9g2AGCF1lfk7v5bd//pwX9fl/SCpHvbbhcAghrRTSs6naJvZhuS3iHpxznfOy/pvCSdZfUyADGN7KYV5u7dbMjsDZL+W9IFd/9m2Ws3Nzd9b2+vk/cFgNo2NrLwXjabSS+/3PfeVGZmz7j75vLznXStmNkpSd+QtLsqxAEgupHdtKKLrhWT9AVJL7j7Z9rvEgAEFuOmFQFr8l1ckb9b0sOS3mNmzx48HupguwAQRt83rVjU5C9dktwPa/IdhXlnNfI6qJEDiG53N7uR8+XL2ZX4hQvhBjo7qskX1cgJcgAI7cSJ7Ep8mZm0v195M0EHO4HejagHGBMQuCZPkCM9geuNQOcC1+QJcqRne/twIsfCjRvZ8wiDT0DtBL6RNDVypKejeiMqWp4FKWVXk9zRvnfUyDEeMXqAp4xPQINHkCM9ffcAT93IZkGOEUGO9ASuN2LJ3XfXex6963T1Q6A38znBDRzgihxAuWvX6j2P4wJ3/RDkAMoxuNxOD/MeCHIA5RhcbqeHrh+CHEA5Bpfb6aHrhyAHsNp8nq3St7+f/dlFiE9ltmgPpSmCHED/prReTg+lKYIcQP+mNFu0h9IUa60A6B/r5TTCWisAhoOWxk4R5AD6R0tjpwhyAP2jpbFTrLUCIA7Wy+kMV+QAUNfAeuAJciA1AwuRyRlgDzxBDqSkLER2d6V77slqzmbZf08h5Ps+sQ2wB54+ciAlGxtZeC9bX5euX5du3rz9+VOnpC99aby16Bj3E43YA08fOTAGRQstvfrq8RCXpNdeC3+lGLPUE+PqeIA98AQ5kJImYdFmlb1VIR27XhzjfqID7IEnyIGUPPRQ9hH+qLW1rLRSpOmVYpWQjl0vrnJ13PUnhiH2wLt7749z5845gAM7O+6zmbtZ9ufOTvHr1tbcs1jNHmbuW1vZ906fvv17kvupU8XbW2U2O749KXt+wSz/NWbN3rOuomOy2M+trePfX1trfkwik7TnOZlKkAMx5QVRUdCsCtadHff19cPn19fbBVaVkK4S9qEtToRHQ3w51GPuX4eKgpyuFSCmoi6U2Sy7gcNRfXdLVNm3GF0jRYr2N0+iqyzStQIMUZ3Bur67JaoM6i3qxUdr9HfeGWZ/VqkzwDmyVRYJciCmOuHcd7dEnUG93//+8L9ffTXOTMeiY5k3ODy2VRbz6i11H5K+KOl3kn5e5fXUyIEDdWrki9dXGRjtU4g6eZPfs+hYbm0N75g1pJCDnZL+XNKfEuSYtKYhO8RwrqPrzpW6J7flnx3Ssex4f4qCvLPBTjPbkPSf7v7Aqtcy2InRGdKgX9/qDNjG2F4sAf5ORB/sNLPzZrZnZntXr17t622BfsSeGBNT17X7GLM1Q+jx70RvQe7uF9190903z5w509fbAv0YUvh0MZOxzjbqznRcte0BrmXSSJ9/J/LqLU0ekjZEjRxTFWpiTN0aa5v6cpfbaLPtkO/fpwB/JxR6ZidBjkkLET5NttlFeIScrVl1213PUo0hwN+JoEEu6WuSfivpNUlXJH2s7PUEOUap646JKlPyl9+viw6SkOunVN321tbx16Z4VZ5a10oddK0AFZRNyf/qV/M7Iu68M5uQs6xOx0fIrpGibZ84IX3lK1ldfXdXevjh/N89tc6VjkXvWgFQU9mgX1FHhNS+gyTkDNILF6TTp48/v78vPfJIFuLb2/khLqXXudITghwIoYvOkbJALQq0a9far5Udcr3t+Vy666787928mYV4WVin1rnSl7x6S+gHNXKMQlH9s8tBrqL36GP52FCzJIvq5ItaedHvZpZejbxjYj1yoENlYd1XyIZs0Qu5/aLjszhGZTfQmDiCHOhSWVj3ddeckOuKhDwZ7exkdy5a3vbp07d/qhnSmikDURTkd0Su7ABpKpu1d/ZsfmdG1/Xd+TzcOi4hZyUu9vnRRw87bNbXpcceG/+6NIEw2Ak0UdZR0ve64V3fXFgKP01+PpdeeeXwevyVVw5DvMpNn3Ebghxooiys+7zLeqjQ6/tkdNSUFyBrKq/eEvpBjRyjMIQ6blEt++TJ9vsT6/drOsZQtr9D+H/VATGzExihotmfUrrroTeZWVq29rc0mrXimdkJpKZK7busZp1qOaJJWaesHDOBUg1BDgxR1dp3XugdleKU9jpjDIuTXd4VvJT9/kNaKz4QSivAENUpL+zuSh/5iPT669VePxZ55ZRls1n25xhuHafi0gp95MAQ1bmKXFyp5tWB++gyiSWvZHLU0d9/5MeG0gowRHX7uEO3PIboVW+rrDRy9Pfvsx00EoIcGKImA37zeVYq2N/P/qwSVFUCeqgTdIpOaouSyYiCeqW8nsTQD/rIgQpC9z5XXRirj0XAmqi6/2O5B6jTRw5gWdUB1bI7Fe3vh9q7ahY3oliscbOYWXtUyDse9axosJMgB6aqakCnHoRDPhHVxIQgALerOqAac92VLoReAGwACHJgqqoGdOpdH6mfiCogyIGpqhPQTTpihiL1E1EF1MgBIBHUyAFgpAhyAEgcQQ4AiSPIASBxBDmANAxx4a6BYBlbAMO3vPb4YuEuaVRthE1xRQ5g+CZwu7Y2CHIAwzeB27W1QZADGL4JrJfSBkEOYPgmsF5KGwQ5gOGbwHopbXQS5Gb2oJm9aGYvmdmnutgmMFm02eVLeeGuwFq3H5rZSUmfk/SXkq5I+omZfdvdn2+7bWByaLNDA11ckb9T0kvu/it3vynp65Le38F2gemhzQ4NdBHk90r69ZGvrxw8dxszO29me2a2d/Xq1Q7eFhgh2uzQQG+Dne5+0d033X3zzJkzfb0tkBba7NBAF0H+G0n3H/n6voPnANRFmx0a6CLIfyLprWb2FjM7LelDkr7dwXaB6aHNDg207lpx91tm9nFJT0o6KemL7v5c6z0Dpmo+J7hRSyc1cnd/wt3/yN3/wN35DBgI7cUA8rCMbSJoLwZQhCn6iaC9GEARgjwRtBcDKEKQJ4L2YgBFCPJE0F4MoAhBngjaiwEUoWslIbQXA8jDFTnQJyYDIACuyIG+MBkAgXBFDvSFyQAIhCAH+sJkAARCkAN9YTIAAiHIgb4wGQCBEOQdSKURIZX9HC0mAyAQgrylRSPCpUuS+2EjQsiQbBLIMfYzlKRPSPO59PLL0v5+9ichji64e++Pc+fOeep2dtxnM/csFo8/ZrNw77u2dvt7ra1lz5cp2tfZ7PB3MTv8eqia/v7AGEja85xMJcgbyAuT5YdZmPcuC+Sj+7cczGbF+5pSMFb5/YGxKgpySisN5LUDLwvViLCqg62ohHL33fk/d/JkcWvzEEsYdPABxxHkDawKjZCNCKs62IrmnCz266i1Nen11/O3tzgB9FVTr3rSoIMPOI4gb6AsNEI3IqzqYCs6yVy7lt8wMZvlv77sSr2porCuMxBLBx+QI6/eEvoxxhp5n3XlssHJujXkot+l69p/2TGru89bW+4nT2avOXky+xqYAjHY2a2hdno0Ocnk/S5NTghlx6Nse0UDsXknjdgnUSAmgnxCujjJ1AnMKq8tC+s6Jw26VjBlRUFOjXyEuphzUmcSYpVF/coGKevUvelaAY4jyFGo6gmhSriWhXWdkwZdK8BxBPlI9dkDXiVcV4V11ZMGXSvAcQR5BU3XNok1mabvdVWqhmvfJR9gMvIK56EfKQ12Nu0CidlZEWNAcKhdPMCYiMHOZprcnavqz4S4at/dza7A83QxIFi0zyzqB8RDkK/QpEuiys+EKH8stlnk7Nl2J48xLYULjAlBvkKTLokqPxPiPrxli3mtrUkPPdQuiIv2+cMfHs6iWsAUEeQrNOmSqPIzXfVDH73CLiqpSNmA4BNPtDt5lO0bV+dAPAT5Ck26JKr8TBf90MuljiKzWfbebU8eq/at7ScKAM0Q5BVUGchbrj1L5T/TRT90lXXRj26z7ckjb5+XMcMS6F+rIDezD5rZc2a2b2abXe1UapoMAnbRD10WmnnbbHvyOLrPRaqcFIZ4wwogaXk9iVUfkv5Y0tsk/UDSZtWfS6mPvIpYCzlVvQ/n+nr2WP7vNv3eTXvlY/fYL/aBnnekSCFXP5x6kNdZhrVLRaG4tVW+pnhXwdkkEGOvXjiEEwnQVPQgl3Re0p6kvbNnz/byS/elTjh1fTVYZy3xISz7GuuktxD7RAK0URTkln2vmJk9LenNOd/advfHD17zA0n/7O57Vco5m5ubvrdX6aVJWNTIjw48rq0d1qd3d7OByUuXstr18iFfX5cee6y72ZAnTpR3sUjZfuzvd/N+dWxs5LdJzmbZoHBoRccm1vEA6jCzZ9z92HjkysFOd3+fuz+Q83g8zK6mp2zg8uhAqJQfIq++2m0PdpUBx1jLvsZevZBlcDFGtB92pKhFsUqLoNRtD/aqNsGYy77GXr0w9okECKFt++EHzOyKpHdJ+o6ZPdnNbo1Hnb7qVa+t2ra3HJbr69ljKMu+xlxgK/aJBAhhZY08hLHVyMsU1YTzlNWJV9XhAYxf4xo52qkyG1Ja/fE+xCJbAMaBIA8s76P8zk72qPPxftU6KcyWBKaL0koiytr2LlwIW3ZZtE9evnx413vKOUD/KK0krqzbIuQ64dxMAhg+gjwRZd0WIdcJ76I2H+qWdpSSgAyllRGo0hnTdOZk25mQIbpt6ODBVFFaGbGQ64TffXf+81VnQvZ1Szs6eDBlBPkIdLVO+LLdXen69ePPnzpVfSZkV7e0C71NIGUE+UgsZkvu7HQ3BX17W7p58/jzb3xj9RJGiLVNWC8FuN3ognzqg2BdTkEvusK9dq36NkKsbcJ6KcCSvLVtQz9C3VgilZsGpHKHmq7W7g7x+6ZyDIEuqel65CGE6lqJvdZ1FSl1XKS0r8AUTKJrJYVBsJQ6LvpcKXDqJTGgjWSCvMo/9BQGwYZ2sll1XPtYcpbZo0A7SQR51X/oKQyCDelkM5QATelTCjBESQR51X/oKdw0oKuTTReliKEE6NA+pQDJyRsBDf2o27US+87rXVvVcVHl+1105wzluHJne6AaFXStJHFFPqRyRGhVyh1dXUkP5bimUBIDBi0v3UM/6l6Rp9IfXsWq36XK1WlXV9JDOq70hQOrqeCKPIkgdx/PP/RVQV0lpNuUIpaP49bWOI4rMAVFQZ5EaUWKe+f1Lq0a2KtS7mhaisgr23z5y9nPdX1c6QsH+pNMkI/FqqCuEtJNu3P66lIZSlsjMBUEec9WBXXVkG7yCaWvNr8htDXyiQBTMqq1VlIR62bGfa1F0/auQm2xRgzGahJrraQiVr2/rza/2G2NQ/hEAPSJIJ+Qvma+xu4LZ6YopoYgn5g+Pg3EXioh9icCoG8EeQJSHLiL2S4a+xMB0DeCfOBo5asv9icCoG90rQxcCnc9AtAPulYSxcAdgFUI8oFj4A7AKgT5wDFwB2CVVkFuZp82s1+Y2c/M7Ftm9qaudgwZBu4ArNJqsNPM/krSf7n7LTP7N0ly939Z9XMMdgJAfUEGO939e+5+6+DLH0m6r832AAD1dVkjf0TSd4u+aWbnzWzPzPauXr3a4dtiiFKcxASk6o5VLzCzpyW9Oedb2+7++MFrtiXdklT4z9XdL0q6KGWllUZ7iyQsrz64mMQkUdsHQmg9IcjMPirpHyS9191vrHi5JGrkY8ckJiCMohr5yivyFRt9UNInJf1F1RDH+DGJCehX2xr5ZyXdJekpM3vWzD7fwT4hcUxiAvrV6orc3f+wqx3BeFy4kH+HHiYxAWEwsxOdYxIT0K9WV+RAkfmc4Ab6whV5D+ipBhASV+SB0VMNIDSuyAMbwx3d+UQBDBtX5IGl3lPNJwpg+LgiDyz1nuoxfKIAxo4gDyz1G0Ok/okCmAKCPLDUe6pT/0QBTAFB3oP5PFssan8/+zOVEJfS/0QBTAFBjlKpf6IApoCuFazELE1g2LgiB4DEEeQAkDiCHAASR5ADQOIIcgBIXOubLzd6U7OrknJuz9ureyS9EnkfhopjU4xjU4xjk6/L4zJz9zPLT0YJ8iEws728u1GDY1OGY1OMY5Ovj+NCaQUAEkeQA0DiphzkF2PvwIBxbIpxbIpxbPIFPy6TrZEDwFhM+YocAEaBIAeAxE06yM3s02b2CzP7mZl9y8zeFHufhsLMPmhmz5nZvplNvqXMzB40sxfN7CUz+1Ts/RkSM/uimf3OzH4ee1+GxMzuN7Pvm9nzB/+WHg31XpMOcklPSXrA3f9E0i8l/Wvk/RmSn0v6O0k/jL0jsZnZSUmfk/TXkt4u6e/N7O1x92pQ/l3Sg7F3YoBuSfqEu79d0p9J+sdQf28mHeTu/j13v3Xw5Y8k3Rdzf4bE3V9w9xdj78dAvFPSS+7+K3e/Kenrkt4feZ8Gw91/KOla7P0YGnf/rbv/9OC/r0t6QdK9Id5r0kG+5BFJ3429ExikeyX9+sjXVxToHyTGycw2JL1D0o9DbH/0dwgys6clvTnnW9vu/vjBa7aVfQza7XPfYqtybAC0Y2ZvkPQNSf/k7v8b4j1GH+Tu/r6y75vZRyX9jaT3+sSa6lcdG/y/30i6/8jX9x08B5Qys1PKQnzX3b8Z6n0mXVoxswclfVLS37r7jdj7g8H6iaS3mtlbzOy0pA9J+nbkfcLAmZlJ+oKkF9z9MyHfa9JBLumzku6S9JSZPWtmn4+9Q0NhZh8wsyuS3iXpO2b2ZOx9iuVgQPzjkp5UNmD1H+7+XNy9Gg4z+5qk/5H0NjO7YmYfi71PA/FuSQ9Les9BvjxrZg+FeCOm6ANA4qZ+RQ4AySPIASBxBDkAJI4gB4DEEeQAkDiCHAASR5ADQOL+DyeKOQtPdrK7AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "for idx, label in enumerate(labels):\n", " if label:\n", " plt.scatter(df1.loc[idx][\"x\"], df1.loc[idx][\"y\"], color=\"b\") \n", " else:\n", " plt.scatter(df1.loc[idx][\"x\"], df1.loc[idx][\"y\"], color=\"r\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Clustering with OpenJij solver\n", "\n", "Next, we introduce clustering with OpenJij solver and use PyQUBO to formulate QUBO." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "def clustering_openjij(df):\n", " # set distance matrix\n", " d_ij = distance_matrix(df, df)\n", " # set spin variables\n", " spin = Array.create(\"spin\", shape= len(df), vartype=\"SPIN\")\n", " # set total Hamiltonian\n", " H = - 0.5* sum(\n", " [d_ij[i,j]* (1 - spin[i]* spin[j]) for i in range(len(df)) for j in range(len(df))]\n", " )\n", " # compile\n", " model = H.compile()\n", " # convert to QUBO\n", " qubo, offset = model.to_qubo()\n", " # set OpenJij SA sampler\n", " sampler = oj.SASampler(num_reads=10, num_sweeps=100)\n", " # solve with above sampler\n", " response = sampler.sample_qubo(qubo)\n", " # extract raw data\n", " raw_solution = dict(zip(response.indices, response.states[np.argmin(response.energies)]))\n", " # decode for easier analysis\n", " decoded_solution, broken, energy= model.decode_solution(raw_solution, vartype=\"SPIN\")\n", " # extract labels\n", " labels = [int(decoded_solution[\"spin\"][idx] ) for idx in range(len(df))]\n", " return labels, sum(response.energies)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We execute and check a solution." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "label [0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 0, 1, 1, 0]\n", "energy -152622.68540013512\n" ] } ], "source": [ "labels, energy =clustering_openjij(df1[[\"x\", \"y\"]])\n", "print(\"label\", labels)\n", "print(\"energy\", energy)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And also we visualize the result." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAVqElEQVR4nO3df4hlZ33H8c93N7uQwYhksiAk2TvSWqmEgt1BKkILaksaSsWCYLkGJcKWoUIKFmuZv/ePIggBBVlQq86gFFQiNRITqvWfKk4kiEmMBMmuK4Kb7B9dWGGzmW//OHM7s3fOOff8es5znnPeL7hs5s6dc8+c7H7Oud/n+zzH3F0AgHSdiL0DAIB2CHIASBxBDgCJI8gBIHEEOQAk7o4Yb3rPPff4xsZGjLcGgGQ988wzr7j7meXnowT5xsaG9vb2Yrw1ACTLzC7lPU9pBQASR5ADQOIIcgBIHEEOAIkjyAEgcQQ5gOHa3ZU2NqQTJ7I/d3dj79EgRWk/BICVdnel8+elGzeyry9dyr6WpPk83n4NEFfkAIZpe/swxBdu3Miex20IcgDDdPlyvecnjCAHMExnz9Z7fsIIcgDDdOGCtLZ2+3Nra9nzuA1BDmCY5nPp4kVpNpPMsj8vXmSgMwddKwCGaz4nuCvgihwAEkeQA0DiCHIASBxBDgCJI8gBIHEEOQAkjiAHgMS1DnIzu9/Mvm9mz5vZc2b2aBc7BgCoposJQbckfcLdf2pmd0l6xsyecvfnO9g2AGCF1lfk7v5bd//pwX9fl/SCpHvbbhcAghrRTSs6naJvZhuS3iHpxznfOy/pvCSdZfUyADGN7KYV5u7dbMjsDZL+W9IFd/9m2Ws3Nzd9b2+vk/cFgNo2NrLwXjabSS+/3PfeVGZmz7j75vLznXStmNkpSd+QtLsqxAEgupHdtKKLrhWT9AVJL7j7Z9rvEgAEFuOmFQFr8l1ckb9b0sOS3mNmzx48HupguwAQRt83rVjU5C9dktwPa/IdhXlnNfI6qJEDiG53N7uR8+XL2ZX4hQvhBjo7qskX1cgJcgAI7cSJ7Ep8mZm0v195M0EHO4HejagHGBMQuCZPkCM9geuNQOcC1+QJcqRne/twIsfCjRvZ8wiDT0DtBL6RNDVypKejeiMqWp4FKWVXk9zRvnfUyDEeMXqAp4xPQINHkCM9ffcAT93IZkGOEUGO9ASuN2LJ3XfXex6963T1Q6A38znBDRzgihxAuWvX6j2P4wJ3/RDkAMoxuNxOD/MeCHIA5RhcbqeHrh+CHEA5Bpfb6aHrhyAHsNp8nq3St7+f/dlFiE9ltmgPpSmCHED/prReTg+lKYIcQP+mNFu0h9IUa60A6B/r5TTCWisAhoOWxk4R5AD6R0tjpwhyAP2jpbFTrLUCIA7Wy+kMV+QAUNfAeuAJciA1AwuRyRlgDzxBDqSkLER2d6V77slqzmbZf08h5Ps+sQ2wB54+ciAlGxtZeC9bX5euX5du3rz9+VOnpC99aby16Bj3E43YA08fOTAGRQstvfrq8RCXpNdeC3+lGLPUE+PqeIA98AQ5kJImYdFmlb1VIR27XhzjfqID7IEnyIGUPPRQ9hH+qLW1rLRSpOmVYpWQjl0vrnJ13PUnhiH2wLt7749z5845gAM7O+6zmbtZ9ufOTvHr1tbcs1jNHmbuW1vZ906fvv17kvupU8XbW2U2O749KXt+wSz/NWbN3rOuomOy2M+trePfX1trfkwik7TnOZlKkAMx5QVRUdCsCtadHff19cPn19fbBVaVkK4S9qEtToRHQ3w51GPuX4eKgpyuFSCmoi6U2Sy7gcNRfXdLVNm3GF0jRYr2N0+iqyzStQIMUZ3Bur67JaoM6i3qxUdr9HfeGWZ/VqkzwDmyVRYJciCmOuHcd7dEnUG93//+8L9ffTXOTMeiY5k3ODy2VRbz6i11H5K+KOl3kn5e5fXUyIEDdWrki9dXGRjtU4g6eZPfs+hYbm0N75g1pJCDnZL+XNKfEuSYtKYhO8RwrqPrzpW6J7flnx3Ssex4f4qCvLPBTjPbkPSf7v7Aqtcy2InRGdKgX9/qDNjG2F4sAf5ORB/sNLPzZrZnZntXr17t622BfsSeGBNT17X7GLM1Q+jx70RvQe7uF9190903z5w509fbAv0YUvh0MZOxzjbqznRcte0BrmXSSJ9/J/LqLU0ekjZEjRxTFWpiTN0aa5v6cpfbaLPtkO/fpwB/JxR6ZidBjkkLET5NttlFeIScrVl1213PUo0hwN+JoEEu6WuSfivpNUlXJH2s7PUEOUap646JKlPyl9+viw6SkOunVN321tbx16Z4VZ5a10oddK0AFZRNyf/qV/M7Iu68M5uQs6xOx0fIrpGibZ84IX3lK1ldfXdXevjh/N89tc6VjkXvWgFQU9mgX1FHhNS+gyTkDNILF6TTp48/v78vPfJIFuLb2/khLqXXudITghwIoYvOkbJALQq0a9far5Udcr3t+Vy666787928mYV4WVin1rnSl7x6S+gHNXKMQlH9s8tBrqL36GP52FCzJIvq5ItaedHvZpZejbxjYj1yoENlYd1XyIZs0Qu5/aLjszhGZTfQmDiCHOhSWVj3ddeckOuKhDwZ7exkdy5a3vbp07d/qhnSmikDURTkd0Su7ABpKpu1d/ZsfmdG1/Xd+TzcOi4hZyUu9vnRRw87bNbXpcceG/+6NIEw2Ak0UdZR0ve64V3fXFgKP01+PpdeeeXwevyVVw5DvMpNn3Ebghxooiys+7zLeqjQ6/tkdNSUFyBrKq/eEvpBjRyjMIQ6blEt++TJ9vsT6/drOsZQtr9D+H/VATGzExihotmfUrrroTeZWVq29rc0mrXimdkJpKZK7busZp1qOaJJWaesHDOBUg1BDgxR1dp3XugdleKU9jpjDIuTXd4VvJT9/kNaKz4QSivAENUpL+zuSh/5iPT669VePxZ55ZRls1n25xhuHafi0gp95MAQ1bmKXFyp5tWB++gyiSWvZHLU0d9/5MeG0gowRHX7uEO3PIboVW+rrDRy9Pfvsx00EoIcGKImA37zeVYq2N/P/qwSVFUCeqgTdIpOaouSyYiCeqW8nsTQD/rIgQpC9z5XXRirj0XAmqi6/2O5B6jTRw5gWdUB1bI7Fe3vh9q7ahY3oliscbOYWXtUyDse9axosJMgB6aqakCnHoRDPhHVxIQgALerOqAac92VLoReAGwACHJgqqoGdOpdH6mfiCogyIGpqhPQTTpihiL1E1EF1MgBIBHUyAFgpAhyAEgcQQ4AiSPIASBxBDmANAxx4a6BYBlbAMO3vPb4YuEuaVRthE1xRQ5g+CZwu7Y2CHIAwzeB27W1QZADGL4JrJfSBkEOYPgmsF5KGwQ5gOGbwHopbXQS5Gb2oJm9aGYvmdmnutgmMFm02eVLeeGuwFq3H5rZSUmfk/SXkq5I+omZfdvdn2+7bWByaLNDA11ckb9T0kvu/it3vynp65Le38F2gemhzQ4NdBHk90r69ZGvrxw8dxszO29me2a2d/Xq1Q7eFhgh2uzQQG+Dne5+0d033X3zzJkzfb0tkBba7NBAF0H+G0n3H/n6voPnANRFmx0a6CLIfyLprWb2FjM7LelDkr7dwXaB6aHNDg207lpx91tm9nFJT0o6KemL7v5c6z0Dpmo+J7hRSyc1cnd/wt3/yN3/wN35DBgI7cUA8rCMbSJoLwZQhCn6iaC9GEARgjwRtBcDKEKQJ4L2YgBFCPJE0F4MoAhBngjaiwEUoWslIbQXA8jDFTnQJyYDIACuyIG+MBkAgXBFDvSFyQAIhCAH+sJkAARCkAN9YTIAAiHIgb4wGQCBEOQdSKURIZX9HC0mAyAQgrylRSPCpUuS+2EjQsiQbBLIMfYzlKRPSPO59PLL0v5+9ichji64e++Pc+fOeep2dtxnM/csFo8/ZrNw77u2dvt7ra1lz5cp2tfZ7PB3MTv8eqia/v7AGEja85xMJcgbyAuT5YdZmPcuC+Sj+7cczGbF+5pSMFb5/YGxKgpySisN5LUDLwvViLCqg62ohHL33fk/d/JkcWvzEEsYdPABxxHkDawKjZCNCKs62IrmnCz266i1Nen11/O3tzgB9FVTr3rSoIMPOI4gb6AsNEI3IqzqYCs6yVy7lt8wMZvlv77sSr2porCuMxBLBx+QI6/eEvoxxhp5n3XlssHJujXkot+l69p/2TGru89bW+4nT2avOXky+xqYAjHY2a2hdno0Ocnk/S5NTghlx6Nse0UDsXknjdgnUSAmgnxCujjJ1AnMKq8tC+s6Jw26VjBlRUFOjXyEuphzUmcSYpVF/coGKevUvelaAY4jyFGo6gmhSriWhXWdkwZdK8BxBPlI9dkDXiVcV4V11ZMGXSvAcQR5BU3XNok1mabvdVWqhmvfJR9gMvIK56EfKQ12Nu0CidlZEWNAcKhdPMCYiMHOZprcnavqz4S4at/dza7A83QxIFi0zyzqB8RDkK/QpEuiys+EKH8stlnk7Nl2J48xLYULjAlBvkKTLokqPxPiPrxli3mtrUkPPdQuiIv2+cMfHs6iWsAUEeQrNOmSqPIzXfVDH73CLiqpSNmA4BNPtDt5lO0bV+dAPAT5Ck26JKr8TBf90MuljiKzWfbebU8eq/at7ScKAM0Q5BVUGchbrj1L5T/TRT90lXXRj26z7ckjb5+XMcMS6F+rIDezD5rZc2a2b2abXe1UapoMAnbRD10WmnnbbHvyOLrPRaqcFIZ4wwogaXk9iVUfkv5Y0tsk/UDSZtWfS6mPvIpYCzlVvQ/n+nr2WP7vNv3eTXvlY/fYL/aBnnekSCFXP5x6kNdZhrVLRaG4tVW+pnhXwdkkEGOvXjiEEwnQVPQgl3Re0p6kvbNnz/byS/elTjh1fTVYZy3xISz7GuuktxD7RAK0URTkln2vmJk9LenNOd/advfHD17zA0n/7O57Vco5m5ubvrdX6aVJWNTIjw48rq0d1qd3d7OByUuXstr18iFfX5cee6y72ZAnTpR3sUjZfuzvd/N+dWxs5LdJzmbZoHBoRccm1vEA6jCzZ9z92HjkysFOd3+fuz+Q83g8zK6mp2zg8uhAqJQfIq++2m0PdpUBx1jLvsZevZBlcDFGtB92pKhFsUqLoNRtD/aqNsGYy77GXr0w9okECKFt++EHzOyKpHdJ+o6ZPdnNbo1Hnb7qVa+t2ra3HJbr69ljKMu+xlxgK/aJBAhhZY08hLHVyMsU1YTzlNWJV9XhAYxf4xo52qkyG1Ja/fE+xCJbAMaBIA8s76P8zk72qPPxftU6KcyWBKaL0koiytr2LlwIW3ZZtE9evnx413vKOUD/KK0krqzbIuQ64dxMAhg+gjwRZd0WIdcJ76I2H+qWdpSSgAyllRGo0hnTdOZk25mQIbpt6ODBVFFaGbGQ64TffXf+81VnQvZ1Szs6eDBlBPkIdLVO+LLdXen69ePPnzpVfSZkV7e0C71NIGUE+UgsZkvu7HQ3BX17W7p58/jzb3xj9RJGiLVNWC8FuN3ognzqg2BdTkEvusK9dq36NkKsbcJ6KcCSvLVtQz9C3VgilZsGpHKHmq7W7g7x+6ZyDIEuqel65CGE6lqJvdZ1FSl1XKS0r8AUTKJrJYVBsJQ6LvpcKXDqJTGgjWSCvMo/9BQGwYZ2sll1XPtYcpbZo0A7SQR51X/oKQyCDelkM5QATelTCjBESQR51X/oKdw0oKuTTReliKEE6NA+pQDJyRsBDf2o27US+87rXVvVcVHl+1105wzluHJne6AaFXStJHFFPqRyRGhVyh1dXUkP5bimUBIDBi0v3UM/6l6Rp9IfXsWq36XK1WlXV9JDOq70hQOrqeCKPIkgdx/PP/RVQV0lpNuUIpaP49bWOI4rMAVFQZ5EaUWKe+f1Lq0a2KtS7mhaisgr23z5y9nPdX1c6QsH+pNMkI/FqqCuEtJNu3P66lIZSlsjMBUEec9WBXXVkG7yCaWvNr8htDXyiQBTMqq1VlIR62bGfa1F0/auQm2xRgzGahJrraQiVr2/rza/2G2NQ/hEAPSJIJ+Qvma+xu4LZ6YopoYgn5g+Pg3EXioh9icCoG8EeQJSHLiL2S4a+xMB0DeCfOBo5asv9icCoG90rQxcCnc9AtAPulYSxcAdgFUI8oFj4A7AKgT5wDFwB2CVVkFuZp82s1+Y2c/M7Ftm9qaudgwZBu4ArNJqsNPM/krSf7n7LTP7N0ly939Z9XMMdgJAfUEGO939e+5+6+DLH0m6r832AAD1dVkjf0TSd4u+aWbnzWzPzPauXr3a4dtiiFKcxASk6o5VLzCzpyW9Oedb2+7++MFrtiXdklT4z9XdL0q6KGWllUZ7iyQsrz64mMQkUdsHQmg9IcjMPirpHyS9191vrHi5JGrkY8ckJiCMohr5yivyFRt9UNInJf1F1RDH+DGJCehX2xr5ZyXdJekpM3vWzD7fwT4hcUxiAvrV6orc3f+wqx3BeFy4kH+HHiYxAWEwsxOdYxIT0K9WV+RAkfmc4Ab6whV5D+ipBhASV+SB0VMNIDSuyAMbwx3d+UQBDBtX5IGl3lPNJwpg+LgiDyz1nuoxfKIAxo4gDyz1G0Ok/okCmAKCPLDUe6pT/0QBTAFB3oP5PFssan8/+zOVEJfS/0QBTAFBjlKpf6IApoCuFazELE1g2LgiB4DEEeQAkDiCHAASR5ADQOIIcgBIXOubLzd6U7OrknJuz9ureyS9EnkfhopjU4xjU4xjk6/L4zJz9zPLT0YJ8iEws728u1GDY1OGY1OMY5Ovj+NCaQUAEkeQA0DiphzkF2PvwIBxbIpxbIpxbPIFPy6TrZEDwFhM+YocAEaBIAeAxE06yM3s02b2CzP7mZl9y8zeFHufhsLMPmhmz5nZvplNvqXMzB40sxfN7CUz+1Ts/RkSM/uimf3OzH4ee1+GxMzuN7Pvm9nzB/+WHg31XpMOcklPSXrA3f9E0i8l/Wvk/RmSn0v6O0k/jL0jsZnZSUmfk/TXkt4u6e/N7O1x92pQ/l3Sg7F3YoBuSfqEu79d0p9J+sdQf28mHeTu/j13v3Xw5Y8k3Rdzf4bE3V9w9xdj78dAvFPSS+7+K3e/Kenrkt4feZ8Gw91/KOla7P0YGnf/rbv/9OC/r0t6QdK9Id5r0kG+5BFJ3429ExikeyX9+sjXVxToHyTGycw2JL1D0o9DbH/0dwgys6clvTnnW9vu/vjBa7aVfQza7XPfYqtybAC0Y2ZvkPQNSf/k7v8b4j1GH+Tu/r6y75vZRyX9jaT3+sSa6lcdG/y/30i6/8jX9x08B5Qys1PKQnzX3b8Z6n0mXVoxswclfVLS37r7jdj7g8H6iaS3mtlbzOy0pA9J+nbkfcLAmZlJ+oKkF9z9MyHfa9JBLumzku6S9JSZPWtmn4+9Q0NhZh8wsyuS3iXpO2b2ZOx9iuVgQPzjkp5UNmD1H+7+XNy9Gg4z+5qk/5H0NjO7YmYfi71PA/FuSQ9Les9BvjxrZg+FeCOm6ANA4qZ+RQ4AySPIASBxBDkAJI4gB4DEEeQAkDiCHAASR5ADQOL+DyeKOQtPdrK7AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "for idx, label in enumerate(labels):\n", " if label:\n", " plt.scatter(df1.loc[idx][\"x\"], df1.loc[idx][\"y\"], color=\"b\") \n", " else:\n", " plt.scatter(df1.loc[idx][\"x\"], df1.loc[idx][\"y\"], color=\"r\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can get a same figure as using PyQUBO SA." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## QBoost\n", "\n", "QBoost is a one of the ensamble learning using QA. Ensamble learning involves preparing a number of weak predictors and combining the results of each of these predictors to obtain the final prediction result.\n", "\n", "QBoost uses QA to optimize the best combination of learners for a given training data. We handle classification problem in this time.\n", "\n", "We define that the set of $D$ training data are $\\{\\vec x^{(d)}\\}(d=1, ..., D)$, corresponding label are $\\{y^{(d)}\\}(d=1, ..., D), y^{(d)}\\in \\{-1, 1\\}$ and the (function) set of $N$ weak learner is $\\{C_i\\}(i=1, ..., N)$. For some data $\\vec x^{(d)}$, $C_i(\\vec x^{(d)})\\in \\{-1, 1\\}$. \n", "\n", "Based on the definitions above, the classification labels are as follows.\n", "\n", "$${\\rm sgn}\\left( \\sum_{i=1}^{N} w_i C_i({\\vec x}^{(d)})\\right)$$\n", "\n", "Where $w_i\\in\\{0, 1\\} (i=1, ..., N)$, is a weight of each predictor (bool value to adopt or not adopt the predictor for the final prediction).QBoost optimizes the combination of $w_i$ so that prediction matches the training data while erasing the number of weak learners.\n", "\n", "Hamiltonian in this problem is as follows.\n", "\n", "$$H(\\vec w) = \\sum_{d=1}^{D} \\left( \\frac{1}{N}\\sum_{i=1}^{N} w_i C_i(\\vec x^{(d)})-y^{(d)} \\right)^2 + \\lambda \\sum _i^N w_i$$\n", "\n", "The first term represents the difference between weak classifier and the correct label. The second term represents a degree of the number of weak classifier to be employed in the final classifier. $\\lambda$ is the regularization parameter that adjust how much the number of weak classifiers affects the total Hamiltonian.\n", "\n", "We optimize this Hamiltonian by recognizing the first term as a cost (objective function) and the second term as a constraint.Minimizing with QA allows us to obtain a combination of weak classifiers that best fits the training data." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Scripts\n", "\n", "Let us try QBoost. We use the cancer identification dataset from scikit-learn for training data. For simplicity, we will only use two character types for training: \"0\" and \"1\"." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "# import libraries\n", "import pandas as pd \n", "from scipy import stats \n", "from sklearn import datasets\n", "from sklearn import metrics" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "# load data\n", "cancerdata = datasets.load_breast_cancer()\n", "# set the number of training data & test data\n", "num_train = 450" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this time, we consider that feature of noise exists." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(569, 60)\n" ] } ], "source": [ "data_noisy = np.concatenate((cancerdata.data, np.random.rand(cancerdata.data.shape[0], 30)), axis=1)\n", "print(data_noisy.shape)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "# convert from label {0, 1} to {-1, 1}\n", "labels = (cancerdata.target-0.5) * 2" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "# divide dataset to training and test\n", "X_train = data_noisy[:num_train, :]\n", "X_test = data_noisy[num_train:, :]\n", "y_train = labels[:num_train]\n", "y_test = labels[num_train:]" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "# from the result of weak learnor\n", "def aggre_mean(Y_list):\n", " return ((np.mean(Y_list, axis=0)>0)-0.5) * 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Set of Weak Learner\n", "\n", "We make weak learner with scikit-learn. In this time, we choose decision stump. Desision stump is a single-layer decision tree. As it will be used as a weak classifier, the features to be used for segmentation are selected randomly (it's a good understanding that we execute single-layer of random forest)." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "# import required libraries\n", "from sklearn.tree import DecisionTreeClassifier as DTC\n", "\n", "# set the number of weak classifier\n", "num_clf = 32\n", "# set the number of ensembles to be taken out for one sample in bootstrap sampling\n", "sample_train = 40\n", "# set model\n", "models = [DTC(splitter=\"random\",max_depth=1) for i in range(num_clf)]\n", "for model in models:\n", " # extract randomly\n", " train_idx = np.random.choice(np.arange(X_train.shape[0]), sample_train)\n", " # make decision tree with variables\n", " model.fit(X=X_train[train_idx], y=y_train[train_idx])\n", "y_pred_list_train = []\n", "for model in models:\n", " # execute prediction with model\n", " y_pred_list_train.append(model.predict(X_train))\n", "y_pred_list_train = np.asanyarray(y_pred_list_train)\n", "y_pred_train =np.sign(y_pred_list_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We look accuracy of all weak learner as the final classifier. Henceforth, we refer to this combination as baseline." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0.9495798319327731\n" ] } ], "source": [ "y_pred_list_test = []\n", "for model in models:\n", " # execute with test data\n", " y_pred_list_test.append(model.predict(X_test))\n", " \n", "y_pred_list_test = np.array(y_pred_list_test)\n", "y_pred_test = np.sign(np.sum(y_pred_list_test,axis=0))\n", "# compute score of prediction accuracy\n", "acc_test_base = metrics.accuracy_score(y_true=y_test, y_pred=y_pred_test)\n", "print(acc_test_base)" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "# set class of QBoost\n", "class QBoost():\n", " def __init__(self, y_train, ys_pred):\n", " self.num_clf = ys_pred.shape[0]\n", " # set binary variables\n", " self.Ws = Array.create(\"weight\", shape = self.num_clf, vartype=\"BINARY\")\n", " # set hyperparameter with PyQUBO Placeholder\n", " self.param_lamda = Placeholder(\"norm\")\n", " # set combination of weak classifier Hamiltonian\n", " self.H_clf = sum( [ (1/self.num_clf * sum([W*C for W, C in zip(self.Ws, y_clf)])- y_true)**2 for y_true, y_clf in zip(y_train, ys_pred.T)\n", " ])\n", " # set normalization term as a constraint\n", " self.H_norm = Constraint(sum([W for W in self.Ws]), \"norm\")\n", " # set total Hamiltonian\n", " self.H = self.H_clf + self.H_norm * self.param_lamda\n", " # compile\n", " self.model = self.H.compile()\n", " # set function for converting to QUBO\n", " def to_qubo(self, norm_param=1):\n", " # set value of hyperparameter\n", " self.feed_dict = {'norm': norm_param}\n", " return self.model.to_qubo(feed_dict=self.feed_dict)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "qboost = QBoost(y_train=y_train, ys_pred=y_pred_list_train)\n", "# make QUBO with lambda=3\n", "qubo = qboost.to_qubo(3)[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Execute QBoost with D-Wave Sampler" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "# import required libraries\n", "from dwave.system.samplers import DWaveSampler\n", "from dwave.system.composites import EmbeddingComposite" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "ename": "SolverAuthenticationError", "evalue": "Token not accepted for that action.", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mSolverAuthenticationError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 1\u001b[0m dw = DWaveSampler(endpoint='https://cloud.dwavesys.com/sapi/', \n\u001b[1;32m 2\u001b[0m \u001b[0mtoken\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'xxxx'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m solver='DW_2000Q_VFYC_6')\n\u001b[0m\u001b[1;32m 4\u001b[0m \u001b[0msampler\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mEmbeddingComposite\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdw\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/.local/share/virtualenvs/OpenJijTutorial-bCQ9CWHW/lib/python3.7/site-packages/dwave/system/samplers/dwave_sampler.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, failover, retry_interval, **config)\u001b[0m\n\u001b[1;32m 163\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 164\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclient\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mClient\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfrom_config\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mconfig\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 165\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msolver\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclient\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_solver\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 166\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 167\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfailover\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfailover\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/.local/share/virtualenvs/OpenJijTutorial-bCQ9CWHW/lib/python3.7/site-packages/dwave/cloud/client.py\u001b[0m in \u001b[0;36mget_solver\u001b[0;34m(self, name, refresh, **filters)\u001b[0m\n\u001b[1;32m 1077\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1078\u001b[0m \u001b[0mlogger\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdebug\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Fetching solvers according to filters=%r\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfilters\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1079\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_solvers\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrefresh\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mrefresh\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mfilters\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1080\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mIndexError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1081\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mSolverNotFoundError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Solver with the requested features not available\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/.local/share/virtualenvs/OpenJijTutorial-bCQ9CWHW/lib/python3.7/site-packages/dwave/cloud/client.py\u001b[0m in \u001b[0;36mget_solvers\u001b[0;34m(self, refresh, order_by, **filters)\u001b[0m\n\u001b[1;32m 990\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 991\u001b[0m \u001b[0;31m# filter\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 992\u001b[0;31m \u001b[0msolvers\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_fetch_solvers\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mquery\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 993\u001b[0m \u001b[0msolvers\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0ms\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ms\u001b[0m \u001b[0;32min\u001b[0m \u001b[0msolvers\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mp\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mp\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mpredicates\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 994\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/.local/share/virtualenvs/OpenJijTutorial-bCQ9CWHW/lib/python3.7/site-packages/dwave/cloud/utils.py\u001b[0m in \u001b[0;36mwrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 404\u001b[0m \u001b[0mval\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'val'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 405\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 406\u001b[0;31m \u001b[0mval\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 407\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcache\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexpires\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mnow\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmaxage\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mval\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 408\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/.local/share/virtualenvs/OpenJijTutorial-bCQ9CWHW/lib/python3.7/site-packages/dwave/cloud/client.py\u001b[0m in \u001b[0;36m_fetch_solvers\u001b[0;34m(self, name)\u001b[0m\n\u001b[1;32m 622\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 623\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mresponse\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstatus_code\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m401\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 624\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mSolverAuthenticationError\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 625\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 626\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mresponse\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstatus_code\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m404\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;31mSolverAuthenticationError\u001b[0m: Token not accepted for that action." ] } ], "source": [ "# set DwaveSampler with token yourself\n", "dw = DWaveSampler(endpoint='https://cloud.dwavesys.com/sapi/', \n", " token='xxxx', \n", " solver='DW_2000Q_VFYC_6')\n", "# embed on chimeragraph\n", "sampler = EmbeddingComposite(dw)" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "ename": "NameError", "evalue": "name 'sampler' is not defined", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;31m# D-Wave\u30b5\u30f3\u30d7\u30e9\u30fc\u3067\u8a08\u7b97\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0msampleset\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msampler\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msample_qubo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mqubo\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_reads\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m100\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[0;31mNameError\u001b[0m: name 'sampler' is not defined" ] } ], "source": [ "# compute DWaveSampler\n", "sampleset = sampler.sample_qubo(qubo, num_reads=100)" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "ename": "NameError", "evalue": "name 'sampleset' is not defined", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;31m# \u7d50\u679c\u306e\u78ba\u8a8d\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msampleset\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[0;31mNameError\u001b[0m: name 'sampleset' is not defined" ] } ], "source": [ "# check the result\n", "print(sampleset)" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "ename": "NameError", "evalue": "name 'sampleset' is not defined", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0menergies\u001b[0m \u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m \u001b[0mdecoded_sol\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mqboost\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdecode_dimod_response\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msampleset\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mqboost\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfeed_dict\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 7\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0md_sol\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbroken\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0menergy\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdecoded_sol\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[0mdecoded_solutions\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0md_sol\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;31mNameError\u001b[0m: name 'sampleset' is not defined" ] } ], "source": [ "# decode each computation result with PyQUBO\n", "decoded_solutions = []\n", "brokens = []\n", "energies =[]\n", "\n", "decoded_sol = qboost.model.decode_dimod_response(sampleset, feed_dict=qboost.feed_dict)\n", "for d_sol, broken, energy in decoded_sol:\n", " decoded_solutions.append(d_sol)\n", " brokens.append(broken)\n", " energies.append(energy)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us check the accuracy in the training/validation data when using a combination of weak classifiers obtained by D-Wave." ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [], "source": [ "accs_train_Dwaves = []\n", "accs_test_Dwaves = []\n", "for decoded_solution in decoded_solutions:\n", " idx_clf_DWave=[]\n", " for key, val in decoded_solution[\"weight\"].items():\n", " if val == 1:\n", " idx_clf_DWave.append(int(key))\n", " y_pred_train_DWave = np.sign(np.sum(y_pred_list_train[idx_clf_DWave, :], axis=0))\n", " y_pred_test_DWave = np.sign(np.sum(y_pred_list_test[idx_clf_DWave, :], axis=0))\n", " acc_train_DWave = metrics.accuracy_score(y_true=y_train, y_pred=y_pred_train_DWave)\n", " acc_test_DWave= metrics.accuracy_score(y_true=y_test, y_pred=y_pred_test_DWave)\n", " accs_train_Dwaves.append(acc_train_DWave)\n", " accs_test_Dwaves.append(acc_test_DWave)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We make a graph with energy on the horizontal axis and accuracy on the vertical axis." ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAt0AAAHwCAYAAAB67dOHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdf5xdVX3v/9eHYSADgQw/NCWT8KOUBilBUlLAL36/DUUNUNQh/coVxfoDRG8L0gc1lVhUUHzAbXpR/F6Vi7dUKQJyW4iIaEB0pCo/TExIgJLLD5FkQgHRCQQHCMP6/nH2JCcnZzLnzJw1M+fM6/l4nMecvfbae6291z5n3rNnn30ipYQkSZKkfHYa7w5IkiRJrc7QLUmSJGVm6JYkSZIyM3RLkiRJmRm6JUmSpMwM3ZIkSVJmhm5JDRMRPRFx1giX3T8iNkVEW6P7VdbGRRFx7Q7mPxgR80e47hQRfzDizmnCc4wljYahW9K4iIgnIuItg9MppSdTSlNTSgPj1aeU0h+llHrGul3DnCS1PkO3pJpExM7j3Qe1tijx99II5PwPkaTG8M1N0pCKs9GfiIjVwIsRsXNEHBsRP4uIvoi4f6jLMSLi4Ij4YUQ8FxG/johvRkRnMe9fgP2B7xSXlPxdRBxYnPHduagzIyJuiYjfRMSjEfHhsnVfFBE3RsQ1EfFCcVnIvLL5n4iI3mLe2og4oaxru+xguS1n34s2/jUivlXU/UVEvHGYXXZyRDxebO+S8gAZER+KiP+IiN9GxLKIOKAov6uocn+xL/5LRPw4Iv6imP/mYr+cXEy/JSJWDbfeYt6hEXFHsQ/XRsRpZfO+HhFfjojvFtt3b0QcPNSG7Wjci8uKPhcRPy3WdXtE7FvHsp+PiJ8CvwN+PyLeVvR3Y0R8pdgfZ0XErsW2zClb/vUR0R8Rr6vS5yGPwWL+ExHx8YhYXbT1rYiYUjZ/UUQ8FREbIuJDQ+2bou4Hi3F4oTgGPlIx/50RsSoino+IxyLixKJ874j456KN30bE0qL8AxHxk4p1bPmPSDF+X42I2yLiReD4iPjziFhZtLEuIi6qWP7NZeOwrmjjTyLi6Sj7ozoi/qL8GJPUICklHz58+Kj6AJ4AVgGzgA6gC3gOOJnSH+1vLaZfV9TvAc4qnv9BMX9X4HXAXcAXK9b9lrLpA4EE7FxM/xj4CjAFOBJ4FjihmHcR8FLRjzbgUuCeYt5sYB0wo2y9Bw+3XGWfirqbgf8XaAc+DvwSaB9iXyXgR8DelP6g+D9l+6IbeBR4A7AzcCHws4pl/6Bs+rPA/1c8/yTwGPDfyuZdMdx6gd2L/fDBYt4fA78G/qiY/3XgN8DRxfxvAjcMsW21jPtjwB9SOk56gMvqWPZJ4I+KfrwOeB5YWEyfV4zD4L78yuC+KKbPA74zRL9rOQbvA2YU4/YfwEeLeScCTwOHF/vyuspxqmjrz4GDgQD+lNIfEH9czDsa2Fj0ZadinxxazPsu8C1gL0rH2Z8W5R8AflLlGPuDsvHbCBxXrHMKMB+YU0wfUfS/u6i/P/ACcHrRzj7AkcW8h4CTytq5Gfjb8X7/8eGj1R7j3gEfPnxM3EcRSj5UNv0J4F8q6iwD3l887xkMR1XW1Q2srFh31dBNKeQPAHuUzb8U+Hrx/CLgB2XzDgP6i+d/ADwDvIWKgLyj5Sr7VNQtD+Q7AU8B//cQ25eAE8um/wq4s3j+PeDMinX9DjigbNny0H0CsLp4/n3gLLb+UfFjYOFw6wX+C/DvFX38n8BniudfB/5X2byTgYeH2LZaxv3Cim3/fh3LfrZs3l8Cd5dNB6U/HgZD9zHF9E7F9HLgtBqP52rH4Bll0/8AXFk8v5riD4di+g8rx2mYtpYC55Xt9y9UqbMf8BqwV5V5H2D40H3NMH344mC7wGLg5h2M7zeL53sXx9B+tWynDx8+an94eYmk4awre34A8K7i39N9EdEHvJlSeNhG8W//G6J0mcfzwLXAvpX1hjAD+E1K6YWysl9ROkM46D/Lnv8OmBIRO6eUHgX+hlJofqbow4zhlhuiH1u2PaX0GrC+6NtQyvfVr8rqHgBcUbbPfkMpTHZR3d3AH0bEdEpn+a8BZhWXbBxN6YztcOs9ADimYqzeC/xeWTuV+2LqEP2pZdyHWlcty5bvtxlsu98Tpf0+OH0v8CLwpxFxKKU/sm6p1ukaj8Gh+r1NPyiN55Ai4qSIuKe4/KWP0h8xg23NovSfgEqzKB3nv93RunegvH9ExDER8aOIeDYiNgIfraEPUNovb4+IqcBplP5Ye2qEfZI0BEO3pOGksufrKJ217Cx77J5SuqzKcpcWyx6RUtoTOINSIKy23kobgL0jYo+ysv2B3po6nNJ1KaU3Uwp8CfhvtSxXxazBJ1G6Pntm0bdh61Pq72DddcBHKvZbR0rpZ0P0/3fACkqXTjyQUnoF+BlwPvBYSunXNax3HfDjinlTU0r/te69UN+4j2TZ8mPhKUr7GSh9uLJ8uvANSsfT+4B/TSm9NETbwx2DO/IU249nVRGxK/BvwD8C01NKncBtZW2to3TpSaV1lI7zzirzXgR2K2vj96rUqXwNXUfpD5BZKaVpwJU19IGUUi+lP/ROpbRP/6VaPUmjY+iWVI/BM2ILIqItIqZExPyIqAxFAHsAm4C+iOgCFlXMfxr4/WqNpJTWUQqZlxZtHAGcSem64x2KiNkR8WdFEHoJ6Kd0qcpIHBURC4sz4X8DvAzcs4P6iyJir4iYRSkwf6sovxJYHBF/VPRxWkS8q2y5avvix8A5xU8oXYZRPj3cem+ldLb8fRHRXjz+JCLeUPPWb1XPuI922e8CcyKiu9jvf822Z+ehFApPpRSir9lB28MdgztyI/CBiDgsInYDPrODurtQum78WeDViDgJeFvZ/H8CPhgRJ0TEThHRFRGHFmeTvwd8pThu2iPi/ymWuR/4o4g4Mkof7ryohj7vQenM+UsRcTTwnrJ53wTeEhGnRekD0ftExJFl868B/o7SNeE319CWpDoZuiXVrAjD76T04b5nKZ09W0T195KLKX14byOlIHVTxfxLgQuLSw4+XmX50yld572BUgj4TErpjhq6uStwGaUPDf4n8PqivyPxbUrXRv+W0hnAhSmlzcPUX0Hpw6ffpRS2SCndTOls+w3FZQ4PACeVLXcR8I1iXwzeYeTHlELUXUNM73C9xaU5bwPeTWkf/mdRd9d6d0Kd4z6qZYuz+O+idH31c5Suu19O6Q+ewTrrgV9QOtP77ztofrhjcEf9/h6la6J/SOnDqj/cQd0XgI9RCuq/pRR2bymbfx+lD7R+oejLjyn9FwZKx9Vm4GFKn0X4m2KZ/0PpQ7M/AB4BtrmTyRD+CvhsRLwAfLroz2AfnqR0ycvfUroMaRVQfjeem4s+3ZxSerGGtiTVKUqXy0mSyhW3W/uDlNIZ492Xyay4rGc98N6U0o/Kyq8GNqSULhy3zrWYiHiM0uVKPxjvvkityC+7kCRNKBGxALiX0qVBiyhdl3xP2fwDKd1ScO44dK8lRem+8IkdnNGXNDpeXiJJmmjeROlOG78G3k7pXtP9ABHxOUqX0SxJKf1y/LrYOiKiB/gq8NfFXXokZeDlJZIkSVJmnumWJEmSMjN0S5IkSZlNig9S7rvvvunAAw8cs/ZefPFFdt999zFrT6PnmDUfx6z5OGbNxzFrPo7Z+FuxYsWvU0qvqyyfFKH7wAMPZPny5WPWXk9PD/Pnzx+z9jR6jlnzccyaj2PWfByz5uOYjb+I+FW1ci8vkSRJkjIzdEuSJEmZGbolSZKkzCbFNd2SJEnKb/Pmzaxfv56XXnppvLuS3ZQpU5g5cybt7e011Td0S5IkqSHWr1/PHnvswYEHHkhEjHd3skkp8dxzz7F+/XoOOuigmpbx8hJJkiQ1xEsvvcQ+++zT0oEbICLYZ5996jqjb+iWJElSw7R64B5U73YauiVJktQS+vr6+MpXvlL3cieffDJ9fX0ZerSVoVuSJEktYajQPTAwsMPlbrvtNjo7O3N1C/CDlJIkSRonS1f2smTZWjb09TOjs4NFC2bTPbdrxOu74IILeOyxxzjyyCNpb29n6tSp7LfffqxatYqHHnqI7u5u1q1bx0svvcR5553H2WefDWz99vJNmzZx0kkn8eY3v5mf/exndHV18e1vf5uOjo5Rb6tnuiVJkjTmlq7sZfFNa+jt6ycBvX39LL5pDUtX9o54nZdddhkHH3wwq1atYsmSJdx33318/vOf56GHHgLg6quvZsWKFSxfvpwvfelLPPfcc9ut45FHHuGv//qvefDBB+ns7OTf/u3fRtyfcoZuSZIkjbkly9bSv3nbyz76Nw+wZNnahrVx9NFHb3NLvy996Uu88Y1v5Nhjj2XdunU88sgj2y1z0EEHceSRRwJw1FFH8cQTTzSkL15eIkmSpDG3oa+/rvKR2H333bc87+np4Qc/+AF33303u+22G/Pnz696y79dd911y/O2tjb6+xvTH890S5IkaczN6Kx+nfRQ5bXYY489eOGFF6rO27hxI3vttRe77bYbDz/8MPfcc8+I2xkJQ7ckSZLG3KIFs+lob9umrKO9jUULZo94nfvssw/HHXcchx9+OIsWLdpm3oknnsirr77KEUccwac+9SmOPfbYEbczEl5eIrWC1TfCnZ+Fjeth2kw44dNwxGnj3Svl5JirWd16Pqz4OqQBiDY46gNwyuXj3SuNg8G7lDTy7iUA1113XdXyXXfdle9973tV5w1et73vvvvywAMPbCn/+Mc/Pqq+lDN0S81u9Y3wnY/B5uKas43rStNgCGtVjrma1a3nw/J/2jqdBrZOG7wnpe65XaMO2c3Cy0ukZnfnZ7eGr0Gb+0vlak2OuZrViq/XVy61EEO31Ow2rq+vXM3PMVezSkN8K+BQ5VILMXRLzW7azPrK1fwcczWraKuvXGohhm6p2Z3waWivuL1Se0epXK3JMVezOuoD9ZVLLcTQLTW7I06Dt38Jps0CovTz7V/yA3WtzDFXszrlcph35tYz29FWmvZDlJoEvHuJ1AqOOM3ANdk45mpWp1xuyFY2fX19XHfddfzVX/1V3ct+8Ytf5Oyzz2a33XbL0DPPdEuSJKlF9PX18ZWvfGVEy37xi1/kd7/7XYN7tJVnuiVJkjQ+GvxFXxdccAGPPfYYRx55JG9961t5/etfz4033sjLL7/MqaeeysUXX8yLL77Iaaedxvr16xkYGOBTn/oUTz/9NBs2bOD4449n33335Uc/+lEDN7LE0C1JkqSxl+GLvi677DIeeOABVq1axe23386//uu/ct9995FS4h3veAd33XUXzz77LDNmzOC73/1uqdmNG5k2bRqXX345P/rRj9h3330bsXXb8fISSZIkjb3MX/R1++23c/vttzN37lz++I//mIcffphHHnmEOXPm8IMf/IBPfOIT/Pu//zvTpk1rSHvD8Uy3JEmSxl7mL/pKKbF48WI+8pGPbDdvxYoV3HbbbSxevJi3ve1tfPrT+W+56pluSZIkjb0MX/S1xx578MILLwCwYMECrr76ajZt2gRAb28vzzzzDBs2bGC33XbjjDPO4OMf/zi/+MUvtls2B890S5Ikaeyd8Oltr+mGUX/R1z777MNxxx3H4YcfzkknncR73vMe3vSmNwEwdepUrr32Wh599FEWLVrETjvtRHt7O1/96lcBOPvssznppJPYb7/9/CClJEmSWsTghyUbePcSgOuuu26b6fPOO2+b6YMPPpgFCxZst9y5557LueeeO6q2d8TQLUmSpPExib7oy2u6JUmSpMwM3ZIkSVJmhm5JkiQ1TEppvLswJurdTkO3JEmSGmLKlCk899xzLR+8U0o899xzTJkypeZl/CClJEmSGmLmzJmsX7+eZ599dry7kt2UKVOYObP2e4obuiVJktQQ7e3tHHTQQePdjQnJy0skSZKkzAzdkiRJUmaGbkmSJCkzQ7ckSZKUmaFbkiRJyszQLUmSJGVm6JYkSZIyM3RLkiRJmRm6JUmSpMwM3ZIkSVJmhm5JkiQps6yhOyJOjIi1EfFoRFxQZf5eEXFzRKyOiPsi4vCyeU9ExJqIWBURy8vKL4qI3qJ8VUScnHMbJEmSpNHaOdeKI6IN+DLwVmA98POIuCWl9FBZtU8Cq1JKp0bEoUX9E8rmH59S+nWV1X8hpfSPufouSZIkNVLOM91HA4+mlB5PKb0C3AC8s6LOYcCdACmlh4EDI2J6xj5JkiRJYy5n6O4C1pVNry/Kyt0PLASIiKOBA4CZxbwE3B4RKyLi7IrlzikuSbk6IvZqfNclSZKkxomUUp4VR7wLWJBSOquYfh9wdErp3LI6ewJXAHOBNcChwFkppfsjYkZKaUNEvB64Azg3pXRXcSb815RC+eeA/VJKH6rS/tnA2QDTp08/6oYbbsiyndVs2rSJqVOnjll7Gj3HrPk4Zs3HMWs+jlnzcczG3/HHH78ipTSvsjzbNd2UzmzPKpueCWwor5BSeh74IEBEBPDL4kFKaUPx85mIuJnS5Sp3pZSeHlw+Ir4G3Fqt8ZTSVcBVAPPmzUvz589vyEbVoqenh7FsT6PnmDUfx6z5OGbNxzFrPo7ZxJXz8pKfA4dExEERsQvwbuCW8goR0VnMAziLUqh+PiJ2j4g9ijq7A28DHiim9ytbxamD5ZIkSdJEle1Md0rp1Yg4B1gGtAFXp5QejIiPFvOvBN4AXBMRA8BDwJnF4tOBm0snv9kZuC6l9P1i3j9ExJGULi95AvhIrm2QJEmSGiHn5SWklG4Dbqsou7Ls+d3AIVWWexx44xDrfF+DuylJkiRl5TdSSpIkSZkZuiVJkqTMDN2SJElSZoZuSZIkKTNDtyRJkpSZoVuSJEnKzNAtSZIkZWboliRJkjIzdEuSJEmZGbolSZKkzAzdkiRJUmaGbkmSJCkzQ7ckSZKUmaFbkiRJyszQLUmSJGVm6JYkSZIyM3RLkiRJmRm6JUmSpMwM3ZIkSVJmhm5JkiQpM0O3JEmSlJmhW5IkScrM0C1JkiRlZuiWJEmSMjN0S5IkSZkZuiVJkqTMDN2SJElSZoZuSZIkKTNDtyRJkpSZoVuSJEnKzNAtSZIkZWboliRJkjIzdEuSJEmZGbolSZKkzAzdkiRJUmaGbkmSJCkzQ7ckSZKUmaFbkiRJyszQLUmSJGVm6JYkSZIyM3RLkiRJmRm6JUmSpMwM3ZIkSVJmhm5JkiQpM0O3JEmSlJmhW5IkScrM0C1JkiRlZuiWJEmSMjN0S5IkSZkZuiVJkqTMsobuiDgxItZGxKMRcUGV+XtFxM0RsToi7ouIw8vmPRERayJiVUQsLyvfOyLuiIhHip975dwGSZIkabSyhe6IaAO+DJwEHAacHhGHVVT7JLAqpXQE8JfAFRXzj08pHZlSmldWdgFwZ0rpEODOYlqSJEmasHKe6T4aeDSl9HhK6RXgBuCdFXUOoxScSSk9DBwYEdOHWe87gW8Uz78BdDeuy5IkSVLj5QzdXcC6sun1RVm5+4GFABFxNHAAMLOYl4DbI2JFRJxdtsz0lNJTAMXP12fouyRJktQwO2dcd1QpSxXTlwFXRMQqYA2wEni1mHdcSmlDRLweuCMiHk4p3VVz46WgfjbA9OnT6enpqbf/I7Zp06YxbU+j55g1H8es+Thmzccxaz6O2cSVM3SvB2aVTc8ENpRXSCk9D3wQICIC+GXxIKW0ofj5TETcTOlylbuApyNiv5TSUxGxH/BMtcZTSlcBVwHMmzcvzZ8/v3FbNoyenh7Gsj2NnmPWfByz5uOYNR/HrPk4ZhNXztD9c+CQiDgI6AXeDbynvEJEdAK/K675Pgu4K6X0fETsDuyUUnqheP424LPFYrcA76d0lvz9wLczbkNzWX0j3PlZ2LgOog3SAEybBSd8Go44bYzaXg/TZtbX5miWnUhaZTsawX0h5VXra6xZXou3ng8rvl76vRVtcNQH4JTLx7tXUkNlC90ppVcj4hxgGdAGXJ1SejAiPlrMvxJ4A3BNRAwADwFnFotPB24unfxmZ+C6lNL3i3mXATdGxJnAk8C7cm1DU1l9I3znY7C5vzSdBko/N64rlUO+N9rKtutpczTLTiStsh2N4L6Q8qr1NdYsr8Vbz4fl/7R1Og1snTZ4q4VkvU93Sum2lNIfppQOTil9vii7sgjcpJTuTikdklI6NKW0MKX026L88ZTSG4vHHw0uW8x7LqV0QrHcCSml3+TchqZx52e3vrFW2txfmj+Wbdfa5miWnUhaZTsawX0h5VXra6xZXosrvl5fudSk/EbKVrFx/ejm52i7ljZHs+xE0irb0QjuCymvWl9jzfJaHPzPbK3lUpMydLeKaTNHNz9H27W0OZplJ5JW2Y5GcF9IedX6GmuW12K01VcuNSlDd6s44dPQ3lF9XntHaf5Ytl1rm6NZdiJple1oBPeFlFetr7FmeS0e9YH6yqUmZehuFUecBm//UuluJbD1DMG0WaXynB+a2abtqK/N0Sw7kbTKdjSC+0LKq9bXWLO8Fk+5HOadufX3VrSVpv0QpVpMzlsGaqwdcdr4vZmOpu3x7Hcjtcp2NIL7Qsqr1tdYs7wWT7nckK2WZ+iWJElqkKUre1mybC0b+vqZ0dnBogWz6Z7bNd7d0gRg6JYkSWqApSt7WXzTGvo3l+680tvXz+Kb1gAYvOU13ZIkSY2wZNnaLYF7UP/mAZYsWztOPdJEYuiWJElqgA191b+kbqhyTS6GbkmSpAaY0Vn91r1DlWtyMXRLkiQ1wKIFs+lo3/ZLfTra21i0YPY49UgTiR+klCRJaoDBD0t69xJVY+iWJElqkO65XYZsVeXlJZIkSVJmhm5JkiQpM0O3JEmSlJmhW5IkScrM0C1JkiRlZuiWJEmSMjN0S5IkSZkZuiVJkqTMDN2SJElSZoZuSZIkKTNDtyRJkpSZoVuSJEnKzNAtSZIkZWboliRJkjIzdEuSJEmZGbolSZKkzAzdkiRJUmaGbkmSJCmznce7A61oQ18/By++jYGUaIvg9GNmcUn3nC3zj/jM93n+5YEt03vu2sbqi0+suq4Ll67h+nvXDbkuSdLks3RlL0uWrWVDXz8zOjtYtGA23XO7RlxvovF3n1qRZ7ob7MKla3juxVcYSAmAgZS49p4nuXDpGmD7wA3w/MsDHPGZ71dd17X3PDnkuiRJk8/Slb0svmkNvX39JKC3r5/FN61h6creEdWbaPzdp1Zl6G6w6+9dt8PyysA9qFr5cOuSJE0+S5atpX/ztr8z+jcPsGTZ2hHVm2j83adWZehusMG/zGstH6t1SZJaw4a+/prKa6030fi7T63K0N1gbRF1lY/VuiRJrWFGZ0dN5bXWm2j83adWZehusNOPmbXD8j13bas6v1r5cOuSJE0+ixbMpqN9298ZHe1tLFowe0T1Jhp/96lVGbob7JLuOeyz+y5b/iJvi+CMY/ff8qnr1RefuF3AHuruJZd0z+GMY/cfcl2SpMmne24Xly6cQ1dnBwF0dXZw6cI5292VpNZ6E42/+9SqvGVgBjM6O3js0vlDzh/q9oDVXNI9xzcaSdI2uud21RSea6030fi7T63IM92SJElSZoZuSZIkKTNDtyRJkpSZoVuSJEnKzNAtSZIkZWboliRJkjIzdEuSJEmZGbolSZKkzAzdkiRJUmaGbkmSJCkzQ7ckSZKUWdbQHREnRsTaiHg0Ii6oMn+viLg5IlZHxH0RcXjF/LaIWBkRt5aVXRQRvRGxqnicnHMbJEmSpNHKFrojog34MnAScBhwekQcVlHtk8CqlNIRwF8CV1TMPw/4jyqr/0JK6cjicVuDuy5JkiQ1VM4z3UcDj6aUHk8pvQLcALyzos5hwJ0AKaWHgQMjYjpARMwE/hz4Xxn7KEmSJGW3c8Z1dwHryqbXA8dU1LkfWAj8JCKOBg4AZgJPA18E/g7Yo8q6z4mIvwSWA3+bUvptZYWIOBs4G2D69On09PSMamPqsWnTpjFtT6PnmDUfx6z5OGbNxzFrPo7ZxJUzdEeVslQxfRlwRUSsAtYAK4FXI+IU4JmU0oqImF+xzFeBzxXr+hzw34EPbddQSlcBVwHMmzcvzZ9fuZp8enp6GMv2NHqOWfNxzJqPY9Z8HLPm45hNXDlD93pgVtn0TGBDeYWU0vPABwEiIoBfFo93A+8oPiQ5BdgzIq5NKZ2RUnp6cPmI+BpwK5IkSdIElvOa7p8Dh0TEQRGxC6UgfUt5hYjoLOYBnAXclVJ6PqW0OKU0M6V0YLHcD1NKZxTL7Fe2ilOBBzJugyRJkjRq2c50p5RejYhzgGVAG3B1SunBiPhoMf9K4A3ANRExADwEnFnDqv8hIo6kdHnJE8BHcvR/tJau7GXJsrVs6OtnRmcHixbMpntu13h3a1gXLl3D9feuYyAl2iI4/ZhZXNI9p2H1Bx3z+Tt4+oVXtkxP32MX7v37tzZkGwaNdAyadexyaPS+GOnxIklSs8t5eQnF7fxuqyi7suz53cAhw6yjB+gpm35fQzuZQV//ZhbfuYb+zQMA9Pb1s/imNQATOrxduHQN197z5JbpgZS2TFcLRvXWH1QZuAGefuEVjvn8HQ0L3ktX9rL4pvrHYKTLtaJG74uRHi+SJLUCv5Eyg6c3vrQlqAzq3zzAkmVrx6lHtbn+3nVZywdVBu7hykdiybK1IxqDkS7Xihq9L0Z6vEiS1AoM3Rm8MvBa1fINff1j3JP6DKTKm8s0tnwsDbWvhxuDkS7Xihq9Lyby8SJJUm6G7gx2aau+W2d0doxxT+rTFtXu8ti48rE01L4ebgxGulwravS+mMjHiyRJuRm6M5g+bQod7W3blHW0t7Fowexx6lFtTj9mVtbyQdP32KWu8pFYtGD2iMZgpMu1okbvi5EeL5IktQJDdwadHe1cunAOXZ0dBNDV2cGlC+dM+A/iXdI9hzOO3X/Lmce2CM44dv8hP+RWb/1B9/79W7cL2I2+e0n33K4RjcFIl2tFjd4XIz1eJElqBZEmwfWU8+bNS8uXLx+z9vw2qObjmDUfx6z5OGbNxzFrPo7Z+IuIFSmleZXlnumWJEmSMjN0S5IkSZkZuiVJkqmkh3IAACAASURBVKTMDN2SJElSZoZuSZIkKTNDtyRJkpSZoVuSJEnKzNAtSZIkZWboliRJkjIzdEuSJEmZGbolSZKkzAzdkiRJUmaGbkmSJCmzmkJ3RPxbRPx5RBjSJUmSpDrVGqK/CrwHeCQiLouIQzP2SZIkSWopNYXulNIPUkrvBf4YeAK4IyJ+FhEfjIj2nB2UJEmSml3Nl4tExD7AB4CzgJXAFZRC+B1ZeiZJkiS1iJ1rqRQRNwGHAv8CvD2l9FQx61sRsTxX5yRJkqRWUFPoBv5HSumH1WaklOY1sD8ahaUre1mybC0b+vqZ0dnBogWz6Z7bNe7ryqlZ+tloFy5dw/X3rmMgJdoiOP2YWVzSPWe8u6UxNFmPfTU/3780WdUaut8QEb9IKfUBRMRewOkppa/k65rqsXRlL4tvWkP/5gEAevv6WXzTGoC6fxE3cl05NUs/G+3CpWu49p4nt0wPpLRl2l9ck8NkPfbV/Hz/0mRW6zXdHx4M3AAppd8CH87TJY3EkmVrt/wCHtS/eYAly9aO67pyapZ+Ntr1966rq1ytZ7Ie+2p+vn9pMqs1dO8UETE4ERFtwC55uqSR2NDXX1f5WK0rp2bpZ6MNpFRXuVrPZD321fx8/9JkVmvoXgbcGBEnRMSfAdcD38/XLdVrRmdHXeVjta6cmqWfjda29e/fmsrVeibrsa/m5/uXJrNaQ/cngB8C/xX4a+BO4O9ydUr1W7RgNh3tbduUdbS3sWjB7HFdV07N0s9GO/2YWXWVq/VM1mNfzc/3L01mNX2QMqX0GqVvpfxq3u5opAY/PNWIuxk0cl05NUs/G23ww0Z++n/ymqzHvpqf71+azGq9T/chwKXAYcCUwfKU0u9n6pdGoHtuV8N+6TZyXTk1Sz8b7ZLuOf6SmuQm67Gv5uf7lyarWi8v+WdKZ7lfBY4HrqH0RTmSJEmShlFr6O5IKd0JRErpVymli4A/y9ctSZIkqXXU+uU4L0XETsAjEXEO0Au8Pl+3JEmSpNZR65nuvwF2Az4GHAWcAbw/V6ckSZKkVjLsme7ii3BOSyktAjYBH8zeq2a3cT1c1AkUN/vfZXc45YtwxGml6Uv3h5c3bq3f1gFT9y0tN20mnPDprXVvPR9WfB3SAEQbHPWBUnll2SmXb9uHyuX2OQSee2To6cF1VGuvct07aufAN8NvHoeN60rTaWDoZctN3Q8+/nDp+eob4c7PVt8fg/7HMfDrh7ctmzZr27rV1vPkPcNvXy3tN0rlsbDrNFj85ND1Bw03To3ahkbvi3qPr4kg9/EwlsebpPx8TY+vCbz/hw3dKaWBiDgqIiIlvzJqWLeeD7/bny2BG+CVF2HpR0vPv/vxbUMWwEB/KaRC6ed3PlZ6/uQ9sPyfttZLA9tOV5YNhpdbz99+ufKAWm16+T/BEz+tXl6+7sptrWznlz/edrpWm56CfzwU3vbZ0vZvLr5Zr3x/DL5oqgXuyrqw/Xpu+gjwWvXtm/qO0s/VNw7ffqNUBm4oTV+6/46Dd7X9Xj5OjdqGRu+L4fo9EeU+HsbyeJOUn6/p8TXB93+tl5esBL4dEe+LiIWDj5wda1orvl69/LWB0l9elSGrms39pbpDrWu4dutZrly1ILuj9Y20naFseqq03Zsrvsp6cH8MGqqf5XWrrac8cJcr345a2m+UoY6F4Y6R4cajUdvQ6H0xVsdRI+U+HsbyeJOUn6/p8TXB93+tH6TcG3iObe9YkoCbGt6jZrejs7sb19e+no3r2eZseT3t1nOGud5111I+GkPto7r3XR3Kt6MR7ec23Hg0ahsavS/G8jhqlNzHQzMcb5Jq52t6fE3w/V/rN1J6HXetom3oedNmbr2MZDjTZsLzG2oPJOXt1nMtdb3rrixvdGAaah9Nm1nfOqD2fV2+fY1oP7eh9vvgdjRqGxq9L4br90SU+3hohuNNUu18TY+vCb7/a7q8JCL+OSKurnzk7lxTGvygY6Wd2koX8+86bfh1tHeU6g61ruHarWe5cvseOvy6aykfqan7lba7vWPb8sH9MWiofpbXrbaeoQ738u2opf1GGepYGO4YGW48GrUNjd4XY3UcNVLu42EsjzdJ+fmaHl8TfP/Xek33rcB3i8edwJ6U7mSiSqdcDrvtC8TWsl12h+4rSxfxL35y+1DV1lG68wZR+vn2L5XqnnI5zDtz65nAaCtNVysr/yBateX2PXTH0/POhHPuHX7dldtaWf+gPy22hfrOYA7eveSI00rbX21/DDrn3urBu7xutfUs/J/Db18t7TdKtWOhlruXDHVcDG5Ho7ah0ftiuH5PRLmPh7E83iTl52t6fE3w/R8juSFJ8UU5P0gpNcW3Us6bNy8tX758zNrr6elh/vz5Y9aeRs8xaz6OWfNxzJqPY9Z8HLPxFxErUkrzKstrPdNd6RBg/9F1SZIkSZocavogZUS8wLa30vhP4BNZeiRJkiS1mFrvXrJH7o5IkiRJrarWu5ecGhHTyqY7I6I7X7ckSZKk1lHrNd2fSSlt+Zq8lFIf8Jk8XZIkSZJaS62hu1q9YS9NiYgTI2JtRDwaERdUmb9XRNwcEasj4r6IOLxifltErIyIW8vK9o6IOyLikeLnXjVugyRJkjQuav0a+OURcTnwZUofqDwXWLGjBSKiraj/VmA98POIuCWl9FBZtU8Cq1JKp0bEoUX9E8rmnwf8B6X7gg+6ALgzpXRZEeQvwA91bmPpyl6WLFvLhr5+ZnR2sGjBbLrndo1Jm719/bRFMJASXWVtX7h0Ddffu46BlGiL4PRjZnFJ95xx628O47kdlW0ff+jr+NHDzzb9PpU0Oezod4TUKmoN3ecCnwK+VUzfDlw4zDJHA4+mlB4HiIgbgHcC5aH7MOBSgJTSwxFxYERMTyk9HREzgT8HPg+cX7bMO4H5xfNvAD0YurdYurKXxTetoX9z6eu2e/v6WXzTGoBsoauyzYHi3u+Dbf/v5U/y08d+s6X+QEpce0/pC2DmHbD3mPc3h/HY7ztqe3D/jnVfJKleFy5ds817VvnvCIO3WklNl5eklF5MKV2QUppXPD6ZUnpxmMW6gHVl0+uLsnL3AwsBIuJo4ABgZjHvi8DfAa9VLDM9pfRU0a+ngNfXsg2TxZJla7eEr0H9mwdYsmztmLZZ3nZ54C53/b3rxqW/OYznduxo/491XySpXtffu66ucqlZ1fSNlBFxB/Cu4gOUFNdR35BSWrCDZd4FLEgpnVVMvw84OqV0blmdPYErgLnAGuBQ4CxgFnBySumvImI+8PGU0inFMn0ppc6ydfw2pbTddd0RcTZwNsD06dOPuuGGG4bdzkbZtGkTU6dOHbP2yq3p3TjkvDld04acl6vN0cjV32pGO2bjsd9rabvSWO7T3MbzdaaRccyaz1iM2Xi+f7YiX2fj7/jjj6/6jZS1Xl6y72DgBkgp/TYihjvDvJ5SeB40E9hQXiGl9DzwQYCICOCXxePdwDsi4mRgCrBnRFybUjoDeDoi9kspPRUR+wHPVGs8pXQVcBWUvgZ+LL8SdTy/gvXvL/shvX3925V3dXZw7nvnj2mbw2mL4PemTRnz/lYz2jEbj/0+XNvj0Zex5FcdNx/HrPmMxZidufi2LZcllmuL4LEWes8aK77OJq5a717yWkRs+dr3iDiQbb+hspqfA4dExEERsQulIH1LeYXift+7FJNnAXellJ5PKS1OKc1MKR1YLPfDInBTrOP9xfP3A9+ucRsmhUULZtPR3rZNWUd7G4sWzB7TNsvbPu7gvavOO/2YWePS3xzGczt2tP/Hui+SVK/Tj5lVV7nUrGo90/33wE8i4sfF9P9DcenGUFJKr0bEOcAyoA24OqX0YER8tJh/JfAG4JqIGKD0Acsza+jLZcCNEXEm8CTwrhq3YVIY/KDcWN5Fo7zNkdy9ZKz7m8N47Pcdte3dSyQ1i8HfBd69RK2upmu6AYrLSc4GVlG65OOZlNJdGfvWMPPmzUvLly8fs/b8107zccyaj2PWfByz5uOYNR/HbPxFxMiv6Y6IsyjdM3smpdB9LHA38GeN7KQkSZLUimq9pvs84E+AX6WUjqd0t5Fns/VKkiRJaiG1hu6XUkovAUTErimlhwE/lSVJkiTVoNYPUq6PiE5gKXBHRPyWitv/SZIkSaquptCdUjq1eHpRRPwImAZ8P1uvJEmSpBZS65nuLVJKPx6+liRJkqRBtV7TLUmSJGmEDN2SJElSZoZuSZIkKTNDtyRJkpSZoVuSJEnKzNAtSZIkZWboliRJkjIzdEuSJEmZGbolSZKkzAzdkiRJUmaGbkmSJCkzQ7ckSZKU2c7j3YFWtXRlL0uWrWVDXz8zOjtYtGA23XO7Rl13JPVzr2eitzmRHfGZ7/P8ywNbpvfctY3VF584jj1qDMe59Vy4dA3X37uOgZRoi+D0Y2ZxSfecUdeVpMnC0J1BX/9mFt+5hv7NpTDV29fP4pvWAGwXPJau7GXxTbXVHUn9oTRqPfUYjzYnssrADfD8ywMc8ZnvN3Xwdpxbz4VL13DtPU9umR5Iact0ZZjeUd23dI5BZyVpgvLykgye3vjSlsAxqH/zAEuWrd2u7pJla2uuO5L6Q2nUeiZ6mxNZZeAerrxZOM6t5/p719VcXk9dSZpMDN0ZvDLwWtXyDX39NZU1snwojVrPRG9TY89xbj0DKdVcXk9dSZpMDN0Z7NJWfbfO6OyoqayR5UNp1Homepsae45z62mLqLm8nrqSNJkYujOYPm0KHe1t25R1tLexaMHs7eouWjC75rojqT+URq1norc5ke25a1td5c3CcW49px8zq+byeupK0mRi6M6gs6OdSxfOoauzgwC6Oju4dOGcqh8i657bVXPdkdQfSqPWM9HbnMhWX3zidgG7Fe5e4ji3nku653DGsftvOVvdFsEZx+5f9Y4k9dSVpMkk0iS4zm7evHlp+fLlY9ZeT08P8+fPH7P2NHqOWfNxzJqPY9Z8HLPm45iNv4hYkVKaV1numW5JkiQpM0O3JEmSlJmhW5IkScrM0C1JkiRlZuiWJEmSMjN0S5IkSZkZuiVJkqTMDN2SJElSZoZuSZIkKTNDtyRJkpSZoVuSJEnKzNAtSZIkZWboliRJkjIzdEuSJEmZGbolSZKkzAzdkiRJUmaGbkmSJCkzQ7ckSZKU2c7j3QHVb+nKXpYsW8uGvn5mdHawaMFsuud2bVPnwqVruP7edQykRFsEv/+63Xj82d8NOX36MbO4pHvOOG1R/WrZB/Ws44IjX6NvZW/d61Dra8SxJkkaGxP5PdvQ3WSWruxl8U1r6N88AEBvXz+Lb1oDsOWgunDpGq6958ktywykxCPPvLjD6cH6zRC8a9kH9a7jlYHX6l6HWl8jjjVJ0tiY6O/ZXl7SZJYsW7vlYBrUv3mAJcvWbpm+/t51I1r3SJcba7Xsg7FYh1qfx4kkNY+J/p5t6G4yG/r6hy0fSGlE6x7pcmOtln0wFutQ6/M4kaTmMdHfsw3dTWZGZ8ew5W0RI1r3SJcba7Xsg7FYh1qfx4kkNY+J/p5t6G4yixbMpqO9bZuyjvY2Fi2YvWX69GNmjWjdI11urNWyD8ZiHWp9HieS1Dwm+nt21tAdESdGxNqIeDQiLqgyf6+IuDkiVkfEfRFxeFE+pZi+PyIejIiLy5a5KCJ6I2JV8Tg55zZMNN1zu7h04Ry6OjsIoKuzg0sXztnmAwKXdM/hjGP333Lmui2CQ16/+w6nzzh2/6b4ECXUtg/qXccubTvVvQ61vkYca5KksTHR37Oz3b0kItqALwNvBdYDP4+IW1JKD5VV+ySwKqV0akQcWtQ/AXgZ+LOU0qaIaAd+EhHfSyndUyz3hZTSP+bq+0TXPbdr2APoku45TROiR6KWfVDPOnp6epg/QV6UmlgacaxJksbGRH7Pznmm+2jg0ZTS4ymlV4AbgHdW1DkMuBMgpfQwcGBETE8lm4o67cWjOT7lJ0mSJFXIGbq7gPJ70K0vysrdDywEiIijgQOAmcV0W0SsAp4B7kgp3Vu23DnFJSlXR8ReuTZAkiRJaoRImW4TFxHvAhaklM4qpt8HHJ1SOreszp7AFcBcYA1wKHBWSun+sjqdwM3AuSmlByJiOvBrSme+Pwfsl1L6UJX2zwbOBpg+ffpRN9xwQ5btrGbTpk1MnTp1zNrT6Dlmzccxaz6OWfNxzJqPYzb+jj/++BUppXmV5Tm/kXI9UH47jJnAhvIKKaXngQ8CREQAvywe5XX6IqIHOBF4IKX09OC8iPgacGu1xlNKVwFXAcybNy/Nnz9/dFtTh56eHsayPY2eY9Z8HLPm45g1H8es+ThmE1fOy0t+DhwSEQdFxC7Au4FbyitERGcxD+As4K6U0vMR8briDDcR0QG8BXi4mN6vbBWnAg9k3AZJkiRp1LKd6U4pvRoR5wDLgDbg6pTSgxHx0WL+lcAbgGsiYgB4CDizWHw/4BvFHVB2Am5MKQ2e0f6HiDiS0uUlTwAfybUNkiRJUiPkvLyElNJtwG0VZVeWPb8bOKTKcqspXeddbZ3va3A3JUmSpKz8RkpJkiQpM0O3JEmSlJmhW5IkScrM0C1JkiRlZuiWJEmSMjN0S5IkSZkZuiVJkqTMDN2SJElSZoZuSZIkKTNDtyRJkpSZoVuSJEnKzNAtSZIkZbbzeHdArWnpyl6WLFtLb19/1fldnR0sWjCb7rldQy67oa+fGUPUe+/X7uanj/1my/RxB+/NNz/8pmH7deHSNVx/7zoGUqItgtOPmcUl3XPq3Dqp9VR73X166Rqef3lgS509d21j9cUnjni9Fxz5Gn0re6u+7iWp1XmmWw23dGUvi29aM2TgBujt62fxTWtYurJ3yGXTEPUqAzfATx/7De/92t077NeFS9dw7T1PMpASAAMpce09T3Lh0jV1bqHUWqq97v7mW6u2CdwAz788wBGf+f6I1/vKwGtVX/eSNBkYutVwS5atpX/zwLD1+jcPsGTZ2mGXraxXGbiHKx90/b3r6iqXJotaX7PAdkG83vVWe91L0mRg6FbDbdjBGe7h6g61bD3rHMrgGe5ay6XJohGvr3rWm6s9SZrIDN1quBmdHSOuO9Sy9axzKG0RdZVLk0UjXl/1rDdXe5I0kRm61XCLFsymo71t2Hod7W0sWjB72GUr6x138N5V1zdU+aDTj5lVV7k0WdT6moXShylHs95qr3tJmgwM3ZPZ6hvhC4fDRZ2ln6tvrG3eMLrndnHpwjl0VZzNesdOP+Enu3yMx3d9D/dMOY9r/uRX293FoHzZoHSXk0sXztmm3jc//KbtAnYtdy+5pHsOZxy7/5Yz220RnHHs/t69RJNe99wurvmTX3HPlPO2vD7/9/+1fpuAffHOV/PormewOv4LXLw33Hr+0Cu89Xy4eG+6v30YD+58Ov+94xoC2KVtp+1ez1JVxTHERdOGP96kJuEtAyer1TfCdz4Gm4trKzeuK00PGmreEafVtPruuV3b/mJdfSN855+3rPP3eJbfW/MZOHCv7da53bJV1HJ7wGou6Z5jyJYqrb6RP1nzGaAfYuvrc/VffKn0+rz1fFj+g6310wAs/6fS81Mu33Zdt56/dR6wU3qNv+D7/MWbu+iZ+g7mG7g1nIpjaIfHm9REPNM9Wd352a2hetDm/lL5jublaE/S+Bru9bni69WXq1ZeT12pGo8htSjPdE9WG9fXVz7cvBztSRobw70+0xC3CaxWXk9dqRqPIbUoz3RPVtNmDl2+o3k52pM0voZ7fcYQH56sVl5PXakajyG1KEP3ZHXCp6G94rZd7R2l8h3Ny9GepPE13OvzqA9UX65aeT11pWo8htSiDN2T1RGnwdu/BNNmAVH6+fbiQ1M7mpejPUnja7jX5ymXw7wzt55pjLbSdLUPtdVTV6rGY0gtymu6J7PBgF3vvBztSRpfw70+T7m89tBTT12pGo8htSDPdEuSJEmZGbolSZKkzAzdkiRJUmaGbkmSJCkzQ7ckSZKUmaFbkiRJyszQLUmSJGVm6JYkSZIyM3RLkiRJmRm6JUmSpMwM3ZIkSVJmhm5JkiQps53HuwOSNJktXdnLkmVr2dDXz4zODhYtmE333K7x7lZW7/3a3fz0sd9smT7u4L355offNI49kqT8PNMtSeNk6cpeFt+0ht6+fhLQ29fP4pvWsHRl73h3LZvKwA3w08d+w3u/dvc49UiSxoahW5LGyZJla+nfPLBNWf/mAZYsWztOPcqvMnAPVy5JrcLQLUnjZENff13lkqTmZeiWpHEyo7OjrnJJUvMydEvSOFm0YDYd7W3blHW0t7Fowexx6lF+xx28d13lktQqDN2SNE6653Zx6cI5dHV2EEBXZweXLpzT0ncv+eaH37RdwPbuJZImA28ZKEnjqHtuV0uH7GoM2JImI890S5IkSZkZuiVJkqTMDN2SJElSZoZuSZIkKbOsoTsiToyItRHxaERcUGX+XhFxc0Ssjoj7IuLwonxKMX1/RDwYEReXLbN3RNwREY8UP/fKuQ2SJEnSaGUL3RHRBnwZOAk4DDg9Ig6rqPZJYFVK6QjgL4ErivKXgT9LKb0ROBI4MSKOLeZdANyZUjoEuLOYliRJkiasnGe6jwYeTSk9nlJ6BbgBeGdFncMoBWdSSg8DB0bE9FSyqajTXjxSMf1O4BvF828A3Rm3QZIkSRq1nKG7C1hXNr2+KCt3P7AQICKOBg4AZhbTbRGxCngGuCOldG+xzPSU0lMAxc/XZ9sCSZIkqQFyfjlOVClLFdOXAVcU4XoNsBJ4FSClNAAcGRGdwM0RcXhK6YGaG484GzgbYPr06fT09NS/BSO0adOmMW1Po+eYNR/HrPk4Zs3HMWs+jtnElTN0rwdmlU3PBDaUV0gpPQ98ECAiAvhl8Siv0xcRPcCJwAPA0xGxX0rpqYjYj9KZ8O2klK4CrgKYN29emj9/fgM2qTY9PT2MZXsaPces+Thmzccxaz6OWfNxzCaunJeX/Bw4JCIOiohdgHcDt5RXiIjOYh7AWcBdKaXnI+J1xRluIqIDeAvwcFHvFuD9xfP3A9/OuA2SJEnSqGU7051SejUizgGWAW3A1SmlByPio8X8K4E3ANdExADwEHBmsfh+wDeKO6DsBNyYUrq1mHcZcGNEnAk8Cbwr1zZIkiRJjZDz8hJSSrcBt1WUXVn2/G7gkCrLrQbmDrHO54ATGttTSZIkKR+/kVKSJEnKzNAtSZIkZWboliRJkjIzdEuSJEmZGbolSZKkzAzdkiRJUmaGbkmSJCkzQ7ckSZKUmaFbkiRJyszQLUmSJGVm6JYkSZIyM3RLkiRJmRm6JUmSpMwM3ZIkSVJmhm5JkiQpM0O3JEmSlJmhW5IkScrM0C1JkiRlZuiWJEmSMjN0S5IkSZkZuiVJkqTMDN2SJElSZoZuSZIkKTNDtyRJkpSZoVuSJEnKzNAtSZIkZWboliRJkjIzdEuSJEmZGbolSZKkzAzdkiRJUmaGbkmSJCkzQ7ckSZKUmaFbkiRJyszQLUmSJGVm6JYkSZIyM3RLkiRJmRm6JUmSpMwM3ZIkSVJmhm5JkiQpM0O3JEmSlJmhW5IkScrM0C1JkiRlZuiWJEmSMjN0S5IkSZkZuiVJkqTMDN2SJElSZoZuSZIkKTNDtyRJkpSZoVuSJEnKzNAtSZIkZZY1dEfEiRGxNiIejYgLqszfKyJujojVEXFfRBxelM+KiB9FxH9ExIMRcV7ZMhdFRG9ErCoeJ+fcBkmSJGm0ds614ohoA74MvBVYD/w8Im5JKT1UVu2TwKqU0qkRcWhR/wTgVeBvU0q/iIg9gBURcUfZsl9IKf1jrr5LkiRJjZTzTPfRwKMppcdTSq8ANwDvrKhzGHAnQErpYeDAiJieUnoqpfSLovwF4D+Arox9lSRJkrLJGbq7gHVl0+vZPjjfDywEiIijgQOAmeUVIuJAYC5wb1nxOcUlKVdHxF6N7bYkSZLUWJFSyrPiiHcBC1JKZxXT7wOOTimdW1ZnT+AKSqF6DXAocFZK6f5i/lTgx8DnU0o3FWXTgV8DCfgcsF9K6UNV2j8bOBtg+vTpR91www1ZtrOaTZs2MXXq1DFrT6PnmDUfx6z5OGbNxzFrPo7Z+Dv++ONXpJTmVZZnu6ab0pntWWXTM4EN5RVSSs8DHwSIiAB+WTyIiHbg34BvDgbuYpmnB59HxNeAW6s1nlK6CrgKYN68eWn+/Pmj3qBa9fT0MJbtafQcs+bjmDWf/7+9+4+9q67vOP582UKpRCsIcygUatZN2VKLosKAQUC6ucQfwFYRdZj5O5IOMpKlZjJ/TM3IptP98gdgMFtYq9KlKqE1OAQJHb+ptLKAWlmVDGUNwmQUmvf+OKf09su9LRe/53vv7ff5SE6+n/s55577vvedc77v7+f7ueeYs8ljziaPORtfXU4vuRlYnGRRkv2Bs4G1vRskeV67DuCdwHVV9fO2AL8U+F5VfXLKcw7reXgGcFdn70CSJEmaBp2NdFfVE0nOA9YBc4DLqmpTkve26z8LvBT4UpIdwGbgHe3TTwDeBnw3yR1t3weq6irg4iRLaaaXbAHe09V7kCRJkqZDl9NLaIvkq6b0fbanfSOwuM/zvgNkwD7fNs1hdmPjarjmI/DQVlhwOJx2ESxZPuqo1Ks3R0d/HDY+YI6kXp9YCI89tOvxvAWw8r7h9tHvOLv9n+GH3961zaKT4dy1g/chSfsA70jZhUe3wddWwEP/BVTz82srml8+Gg8bV++eox3bzZHUa2rBDc3jTyx8+vvod5xd+a7dC25oHl/++l86ZEkaZxbdXXj4fnj80d37Hn+0Ge3ReLjmI+ZI2pOpBffe+vvpd5wNMrUQl6R9jEV3F3Zs79//0NaZjUODDcqFOZKmj8eTJD3JorsLc/bv37/g8P79mnmDcmGOpOnj8SRJT7Lo7sJzDoP95u/et9/85suUGg+nEkVFswAACI5JREFUXWSOpD2Zt2C4/n76HWeDLDr56e9XkiaQRXcX5h8Er/sMLDgCSPPzdZ/xyhjjZMny3XM0Z39zJPVaed9TC+xhr17S7zg78wtPLbC9eomkWaDTSwbOakuWW8CNu94cXXstLDlllNFI42fYywP20+8489woaRZypFuSJEnqmEW3JEmS1DGLbkmSJKljFt2SJElSxyy6JUmSpI5ZdEuSJEkds+iWJEmSOmbRLUmSJHXMoluSJEnqmEW3JEmS1DGLbkmSJKljFt2SJElSxyy6JUmSpI5ZdEuSJEkds+iWJEmSOpaqGnUMnUvyU+BHM/iShwA/m8HX0y/PnE0eczZ5zNnkMWeTx5yN3pFVdejUzllRdM+0JLdU1bGjjkNPnzmbPOZs8pizyWPOJo85G19OL5EkSZI6ZtEtSZIkdcyiuxufH3UAGpo5mzzmbPKYs8ljziaPORtTzumWJEmSOuZItyRJktQxi+4hJbksyQNJ7urpW5XkjnbZkuSOtn+/JJcn+W6S7yVZObrIZ68BOVuaZEObs1uSvKpn3cok9yb5zyS/O5qoNUzekpye5Nb2WLs1yamji3z2GvZYa9cvTPJIkgtnPmI9g/PjkiQ3JtnUHm8HjCby2WvIc6N1yDipKpchFuB3gJcDdw1Y/zfARW37HOBf2/azgS3AUaN+D7Nt6ZczYD3w2rb9+8C1bfto4E5gHrAI+D4wZ9TvYTYuQ+btGOCFbfu3gB+POv7ZuAyTs571XwW+DFw46vhn4zLkcTYX2Ai8rH38fM+PY58z65AxWhzpHlJVXQf8T791SQIsB67YuTlwYJK5wHxgO/DzmYhTuwzIWQHPbdsLgJ+07TfQnKAeq6ofAvcCr0Izbpi8VdXtVbUzh5uAA5LMm5FA9aQhjzWSvBH4AU3ONAJD5mwZsLGq7myf+2BV7ZiRQPWkIXNmHTJG5o46gH3MScB/V9U97eOv0BRx99P8hXlBVfUt2DXjzgfWJflrmmlWv932vwjY0LPd1rZP42FQ3nqdBdxeVY/NaGQapG/OkhwI/BlwOuDUkvEy6Dj7daCSrAMOpRmguHhEMWp3g3JmHTJGHOmeXm9m1yg3NCOkO4AX0kxV+NMkLx5FYHqK99GcfI4ALgAubfvTZ1sv8TM+BuUNgCS/CfwV8J4RxKb+BuXsw8CnquqRkUWmQQblbC5wIvCW9ucZSU4bTYiaYlDOrEPGiEX3NGn/dXMmsKqn+xzg6qp6vKoeAG4AvDXreDgXuLJtf5ldU0i2Akf0bHc4Pf8O18gNyhtJDgfWAH9UVd8fQWzqb1DOXg1cnGQLzSjdB5KcN/PhqY89nR+/XVU/q6pfAFfRzC3W6A3KmXXIGLHonj6vAe6uqq09ffcBp6ZxIHAccPdIotNUPwFObtunAjunBK0Fzk4yL8kiYDFw0wjiU39985bkecA3gJVVdcOIYlN/fXNWVSdV1VFVdRTwt8DHq+rvRxOiphh0flwHLEny7Hag6WRg8wji01MNypl1yBhxTveQklwBnAIckmQr8BdVdSlwNrtPLQH4B+CLwF000xa+WFUbZzBc0T9nwLuAT7e/OP4PeDdAVW1KsprmF8kTwPv9otBoDJM34Dzg14APJvlg27esHdnRDBkyZxoDQ54ftyX5JHAzzbS7q6rqGyMJfBYb8jizDhkj3pFSkiRJ6pjTSyRJkqSOWXRLkiRJHbPoliRJkjpm0S1JkiR1zKJbkiRJ6phFtyRpj5J8KMmPk9yR5J4kVyY5etRxSdIkseiWpH1Qe73e6fSpqlpaVYtp7rz7rSSHTvNrSNI+y6JbkkYsyVuT3NSOJH8uyZy2/5EkH0tyZ5INSV7Q9h+a5KtJbm6XE9r+DyX5fJL1wJfa7b6Z5LZ2vz9KckiSjyb5k57X/1iSFU833qpaBaynucX01Pfyj0le37bXJLmsbb8jyV+27X9LcmuSTUne3fa9L8nFPft5e5K/29PnI0mTxKJbkkYoyUuBNwEnVNVSYAfwlnb1gcCGqnoZcB3NXecAPk0z8vxK4Czgkp5dvgJ4Q1WdQ3Onum9V1cuBNcDCdptLgXPb138WzR11/yXJJUmOfZqh3wa8pE//dcBJbftFwM5pKCcC17ftP66qVwDHAiuSPB/4CnBmz37eBKzay+cjSRPD28BL0midRlMo35wEYD6w8/b124Gvt+1bgdPb9muAo9vtAZ6b5Dlte21VPdq2TwTOAKiqq5Nsa9tbkjyY5BjgBcDtVfUg8M4h4s6A/uuB89s535uBg5IcBhwP7BxNX5HkjLZ9BLC4qjYk+UGS44B7gN8AbgDez+DPR5ImhkW3JI1WgMuramWfdY9XVbXtHew6Zz8LOL6nuG521BSl/ztl34NcArwd+FXgsuHD5hjgliSvBj7X9l1UVWuTHAT8Hs2o98HAcuCRqno4ySk0fzQcX1W/SHItcED7/FXttncDa6qq0rypQZ+PJE0Mp5dI0mhdA/xBkl8BSHJwkiP38pz1wHk7HyRZOmC779AUsSRZBhzUs24NTWH8SmDdMAEnOQtYBlxRVf/RfsFyaVWtbTe5ETifpui+HriQXVNLFgDb2oL7JcBxPbu+Engj8GaaAhye2ecjSWPHoluSRqiqNgN/DqxPshH4JnDYXp62Ajg2ycYkm4H3Dtjuw8CyJLcBrwXuBx5uX3c78O/A6qraAbCXOd0X7LxkIPBW4NSq+umAba8H5lbVvTRzvw9mV9F9NTC3fa8fBTb0fBbbaKakHFlVN7V9z+TzkaSxk13/uZQk7UuSzAN2VNUTSY4H/qn9MuLOL1DeBvxhVd0zyjglaTZwTrck7bsWAqvbAns77dVP2i85fp1m3rQFtyTNAEe6JUmSpI45p1uSJEnqmEW3JEmS1DGLbkmSJKljFt2SJElSxyy6JUmSpI5ZdEuSJEkd+3/EqMwCafdt4AAAAABJRU5ErkJggg==\n", "image/svg+xml": [ "\r\n", "\r\n", "\r\n", "\r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", " \r\n", "\r\n" ], "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.figure(figsize=(12, 8))\n", "plt.scatter(energies, accs_train_Dwaves, label=\"train\" )\n", "plt.scatter(energies, accs_test_Dwaves, label=\"test\")\n", "plt.xlabel(\"energy: D-wave\")\n", "plt.ylabel(\"accuracy\")\n", "plt.title(\"relationship between energy and accuracy\")\n", "plt.grid()\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "base accuracy is 0.9411764705882353\n", "max accuracy\u3000of QBoost is 0.957983193277311\n", "average accuracy\u3000of QBoost is 0.9398183515830576\n" ] } ], "source": [ "print(\"base accuracy is {}\".format(acc_test_base))\n", "print(\"max accuracy\u3000of QBoost is {}\".format(max(accs_test_Dwaves)))\n", "print(\"average accuracy\u3000of QBoost is {}\".format(np.mean(np.asarray(accs_test_Dwaves))))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "D-Wave samplig can perform hundreds or more sampling in a short period. It is possible to create a more accurate classifier than a baseling if you use the results that maximiza accuracy." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 2 }