{ "cells": [ { "cell_type": "markdown", "id": "0d6fecdf-48c0-4745-b802-2117fb3137cf", "metadata": {}, "source": [ "# Basics of CuPy" ] }, { "cell_type": "markdown", "id": "15a05d43-0bf5-48d3-9c88-6074eed82a04", "metadata": {}, "source": [ "## Overview\n", "### In this tutorial, you learn:\n", "\n", "* Basics of Cupy and GPU computing\n", "* Data Transfer Between Host and Device\n", "* Compare speeds to NumPy\n", "\n", "## Prerequisites\n", "\n", "| Concepts | Importance | Notes |\n", "| --- | --- | --- |\n", "| [Familiarity with NumPy](https://foundations.projectpythia.org/core/numpy.html) | Necessary | |\n", "\n", "- **Time to learn**: 30 minutes\n", "\n", "## Introduction to CuPy\n", "CuPy is an open-source GPU-accelerated array library for Python that is compatible with NumPy/SciPy. \n", "\n", "\n", "\n", "CuPy uses NVIDIA CUDA to run operations on the GPU, which can provide significant performance improvements for numerical computations compared to running on the CPU, especially at larger data sizes. CuPy provides a NumPy-like interface for array manipulation and supports a wide range of mathematical operations, making it a powerful tool for scientific computing on GPUs.\n", "\n", "
\n", " In simple terms, CuPy can be described as the GPU equivalent of NumPy.\n", "
\n", "\n", "CuPy is a library that has similar capabilities as NumPy, but with important distinctions that make it ideal for GPU computing. CuPy provides:\n", "\n", "* An object similar to NumPy's multidimensional array, except that it resides in the memory of the GPU, allowing for faster computations involving large data sets.\n", "\n", "* A system for applying \"universal functions\" (`ufuncs`) that adhere to broadcasting rules. This system leverages the parallel computing power of GPUs for better performance.\n", "\n", "* CuPy provides an extensive collection of CUDA-ready array functions. CUDA is NVIDIA's parallel computing platform and API model, which allows software developers to use a CUDA-enabled GPU for general purpose processing. CuPy's extensive set of pre-implemented mathematical functions can be used on arrays right off the bat, taking full advantage of GPU acceleration.\n", "\n", "For more information about CuPy, please visit:\n", "\n", "[CuPy Homepage](https://docs.cupy.dev/en/stable/index.html#)\n", "\n", "[CuPy Github](https://github.com/cupy/cupy)\n", "\n", "In this tutorial, we will explore the distinctive features of CuPy and show their differences from NumPy. Let's get started!" ] }, { "cell_type": "markdown", "id": "77343efb-de6d-423c-b1cd-934c5d6d68e1", "metadata": {}, "source": [ "## Getting Started with CuPy" ] }, { "cell_type": "markdown", "id": "1c0a8fe5-0923-464e-8ea0-77e8d46b7977", "metadata": {}, "source": [ "Once CuPy is installed, we can import it in the same way as NumPy:\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "55c72b7d-8899-4e2f-9432-e9cf1531cbdf", "metadata": {}, "outputs": [], "source": [ "## Import NumPy and CuPy\n", "import cupy as cp\n", "import numpy as np" ] }, { "cell_type": "markdown", "id": "62af1f7c-0ac2-4bad-ab92-8f1bcfbaffe3", "metadata": {}, "source": [ "### Arrays in CuPy vs. NumPy\n", "\n", "CuPy arrays can be declared using the `cupy.ndarray` class, much like NumPy arrays using `numpy.ndarrays`. However, it is important to note that while NumPy arrays are generated on the CPU (referred to as the \"host\"), CuPy arrays are generated on the GPU (known as the \"device\").\n", "\n", "CuPy arrays look just like NumPy arrays:" ] }, { "cell_type": "code", "execution_count": 2, "id": "c98d68a4-3b43-4a7d-91e2-53afdb121273", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "On the CPU: [1 2 3 4 5]\n", "\n" ] } ], "source": [ "# create a 1D array with 5 elements on CPU\n", "arr_cpu = np.array([1, 2, 3, 4, 5])\n", "print(\"On the CPU: \", arr_cpu)\n", "print(type(arr_cpu))" ] }, { "cell_type": "code", "execution_count": 3, "id": "7f09bd38-67fd-465f-a3f7-547b2b989b62", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "On the GPU: [1 2 3 4 5]\n", "\n" ] } ], "source": [ "# create a 1D array with 5 elements on GPU\n", "arr_gpu = cp.array([1, 2, 3, 4, 5])\n", "print(\"On the GPU: \", arr_gpu)\n", "print(type(arr_gpu))" ] }, { "cell_type": "markdown", "id": "e4d08c51-65a1-471f-841d-418ad0df592c", "metadata": {}, "source": [ " You can also create multi-dimensional arrays:" ] }, { "cell_type": "code", "execution_count": 4, "id": "693b52b5-0b94-464d-b3bd-1c4a53b4f17d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "On the CPU: [[0. 0. 0. 0.]\n", " [0. 0. 0. 0.]\n", " [0. 0. 0. 0.]]\n", "\n" ] } ], "source": [ "# create a 2D array of zeros with 3 rows and 4 columns\n", "arr_cpu = np.zeros((3, 4))\n", "print(\"On the CPU: \", arr_cpu)\n", "print(type(arr_cpu))" ] }, { "cell_type": "code", "execution_count": 5, "id": "9845d93b-0d04-450b-ae68-47fc911f339d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "On the GPU: [[0. 0. 0. 0.]\n", " [0. 0. 0. 0.]\n", " [0. 0. 0. 0.]]\n", "\n" ] } ], "source": [ "arr_gpu = cp.zeros((3, 4))\n", "print(\"On the GPU: \", arr_gpu)\n", "print(type(arr_gpu))" ] }, { "cell_type": "markdown", "id": "266ab29b-d11f-419d-b52b-9b6be5638945", "metadata": {}, "source": [ "As we can see in the above examples, CuPy arrays look just like NumPy arrays, except that Cupy arrays are stored on GPUs vs. Numpy arrays are stored on CPUs." ] }, { "cell_type": "markdown", "id": "5398e305-063a-4b15-b259-5eddf29c8cf9", "metadata": {}, "source": [ "### Basic Operations \n", "CuPy provides equivalents for many common NumPy functions, although not all. Most of CuPy's functions have the same function call as their NumPy counterparts. See the reference for the supported subset of NumPy API.\n", "| | |\n", "| :--- | :--- |\n", "| **NumPy** | **CuPy** |\n", "| numpy.identity | cupy.identity |\n", "| numpy.matmul | cupy.matmul |\n", "| numpy.nan_to_num | cupy.nan_to_num |\n", "| numpy.zeros | cupy.zeros |\n", "| numpy.ones | cupy.ones |\n", "| numpy.shape | cupy.shape |\n", "| numpy.reshape | cupy.reshape |\n", "| numpy.tensordot | cupy.tensordot |\n", "| numpy.transpose | cupy.transpose |\n", "| numpy.fft.fft | cupy.fft.fft |\n", "\n", "Cupy also provides equivalant functions for some SciPy functions, but its implementation is not as extensive as NumPy's.\n", "\n", "See [here](https://docs.cupy.dev/en/stable/reference/comparison.html) for a full list of CuPy's Numpy and Scipy equivalent functions.\n", "\n", "\n", "[CuPy API Reference](https://docs.cupy.dev/en/stable/reference/index.html)" ] }, { "cell_type": "code", "execution_count": 6, "id": "0850adf9-0c24-4687-b8de-1b7da734347e", "metadata": {}, "outputs": [], "source": [ "# NumPy: Create an array\n", "numpy_a = np.array([1, 2, 3, 4, 5])\n", "\n", "# CuPy: Create an array\n", "cupy_a = cp.array([1, 2, 3, 4, 5])" ] }, { "cell_type": "markdown", "id": "45fe9f2a-e00f-4cfa-b0a5-eb5a5682e743", "metadata": {}, "source": [ "Basic arithmetic operations is exactly identical between numpy and cupy. " ] }, { "cell_type": "code", "execution_count": 7, "id": "f6e7880b-2238-4c3b-a431-157e6c5389dc", "metadata": {}, "outputs": [], "source": [ "# Basic arithmetic operations\n", "numpy_b = numpy_a + 2\n", "cupy_b = cupy_a + 2\n", "\n", "numpy_c = numpy_a * 2\n", "cupy_c = cupy_a * 2\n", "\n", "numpy_d = numpy_a.dot(numpy_a)\n", "cupy_d = cupy_a.dot(cupy_a)\n", "\n", "# Reshaping arrays\n", "numpy_e = numpy_a.reshape(5, 1)\n", "cupy_e = cupy_a.reshape(5, 1)\n", "\n", "# Transposing arrays\n", "numpy_f = numpy_e.T\n", "cupy_f = cupy_e.T\n", "\n", "# Complex example: element-wise exponential and sum\n", "numpy_g = np.exp(numpy_a) / np.sum(np.exp(numpy_a))\n", "cupy_g = cp.exp(cupy_a) / cp.sum(cp.exp(cupy_a))" ] }, { "cell_type": "markdown", "id": "9f25ee88-1adf-45fd-8b24-04e30fe4488f", "metadata": {}, "source": [ "### Data Transfer\n", "\n", "#### Data Transfer to a Device\n", "`cupy.asarray()` can be used to move a numpy array to a device (GPU)." ] }, { "cell_type": "code", "execution_count": 8, "id": "20bd69c4-5a8b-4147-9169-efc65a49b5e4", "metadata": {}, "outputs": [], "source": [ "# Move data to GPU\n", "arr_gpu = cp.asarray(arr_cpu)" ] }, { "cell_type": "markdown", "id": "39ccf012-f467-49f4-99cd-0eea489e21a0", "metadata": {}, "source": [ "#### Move array from GPU to the CPU\n", "\n", "Moving a device array to the host (i.e. CPU) can be done by `cupy.asnumpy()` as follows:" ] }, { "cell_type": "code", "execution_count": 9, "id": "2e557105-755f-48ec-977e-bedee81b99c9", "metadata": {}, "outputs": [], "source": [ "# Move data back to host\n", "arr_cpu = cp.asnumpy(arr_gpu)" ] }, { "cell_type": "markdown", "id": "30386bbc-26b0-4afd-904b-30bb34d80d6a", "metadata": {}, "source": [ "We can also use `cupy.ndarray.get()`:" ] }, { "cell_type": "code", "execution_count": 10, "id": "bc09a284-dbd1-4f30-b262-0761b2832bfa", "metadata": {}, "outputs": [], "source": [ "arr_cpu = arr_gpu.get()" ] }, { "cell_type": "markdown", "id": "46dfb920-eb81-4cd1-b407-00099b76f633", "metadata": { "tags": [] }, "source": [ "### Device Information \n", "CuPy introduces the concept of a *current* device, which represents the default GPU device for array allocation, manipulation, calculations, and other operations. \n", "\n", "`cupy.ndarray.device` attribute can be used to determine the device allocated to a CUPY array: " ] }, { "cell_type": "code", "execution_count": 11, "id": "114120c2-99c1-4f0f-9ad8-40486dfff4e5", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cupy_g.device" ] }, { "cell_type": "markdown", "id": "e6310585-8dbe-4cc4-a235-6694db49d44a", "metadata": {}, "source": [ "To obtain the total number of accessible devices, you can utilize the getDeviceCount function." ] }, { "cell_type": "code", "execution_count": 12, "id": "e808fa97-7360-4f4a-b239-12d6a3cacbaf", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cp.cuda.runtime.getDeviceCount()" ] }, { "cell_type": "markdown", "id": "983b94ba-8127-461c-89cb-651fe123ccbb", "metadata": {}, "source": [ "The default behavior runs code on Device 0, but we can transfer arrays other devices with CuPy using `cp.cuda.Device()`. This capability becomes significantly important when your code is designed to harness the power of multiple GPUs.\n", "\n", "If you want to change to a different GPU device, you can do so by utilizing the \"device\" context manager. For example the following create an array on the GPU 2. \n", "\n", "``` python \n", "with cp.cuda.Device(2):\n", " x_on_gpu2 = cp.array([1, 2, 3, 4, 5])\n", "```\n", "\n", "There is no need for explicit device switching when only one device is available." ] }, { "cell_type": "markdown", "id": "747151e6-dc5f-4444-a906-528d1066a1dd", "metadata": {}, "source": [ "## CuPy vs NumPy: Speed Comparison\n", "\n", "Now that we are familar with CuPy, let's explore the performance improvements that CuPy can provide in comparison to NumPy for different data sizes. \n", "\n", "First, we are looking at matrix multiplication for array size of 3000x3000." ] }, { "cell_type": "code", "execution_count": 13, "id": "1545d1e5-3ae8-422b-95b5-cd88e7eb64e7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "NumPy time: 0.7095739841461182 seconds\n", "CuPy time: 0.6216685771942139 seconds\n", "CuPy provides a 1.14 x speedup over NumPy.\n" ] } ], "source": [ "import time\n", "\n", "# create two 3000x3000 matrices\n", "n = 3000\n", "\n", "a_np = np.random.rand(n, n)\n", "b_np = np.random.rand(n, n)\n", "\n", "a_cp = cp.asarray(a_np)\n", "b_cp = cp.asarray(b_np)\n", "\n", "# perform matrix multiplication with NumPy and time it\n", "start_time = time.time()\n", "c_np = np.matmul(a_np, b_np)\n", "end_time = time.time()\n", "\n", "numpy_time = end_time - start_time\n", "print(\"NumPy time:\", numpy_time, \"seconds\")\n", "\n", "# perform matrix multiplication with CuPy and time it\n", "start_time = time.time()\n", "c_cp = cp.matmul(a_cp, b_cp)\n", "cp.cuda.Stream.null.synchronize() # wait for GPU computation to finish\n", "end_time = time.time()\n", "\n", "cupy_time = end_time - start_time\n", "\n", "print(\"CuPy time:\", cupy_time, \"seconds\")\n", "print(\"CuPy provides a\", round(numpy_time / cupy_time, 2), \"x speedup over NumPy.\")" ] }, { "cell_type": "markdown", "id": "a1cb881d-9ef1-4fbb-8044-d5bd4c8cb8b6", "metadata": {}, "source": [ "Now, let's run the same CuPy operation again:" ] }, { "cell_type": "code", "execution_count": 14, "id": "a83b1fd5-7896-49ed-9e64-74bcd1417c2c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CuPy time: 0.01408529281616211 seconds\n", "CuPy provides a 50.38 x speedup over NumPy.\n" ] } ], "source": [ "# perform matrix multiplication with CuPy and time it\n", "start_time = time.time()\n", "c_cp = cp.matmul(a_cp, b_cp)\n", "cp.cuda.Stream.null.synchronize() # wait for GPU computation to finish\n", "end_time = time.time()\n", "\n", "cupy_time = end_time - start_time\n", "\n", "print(\"CuPy time:\", cupy_time, \"seconds\")\n", "print(\"CuPy provides a\", round(numpy_time / cupy_time, 2), \"x speedup over NumPy.\")" ] }, { "cell_type": "markdown", "id": "ca229603-89a0-49ca-8920-b40d29a2b703", "metadata": {}, "source": [ "### What happened? Why CuPy is faster the second time?\n", "When running these functions for the first time, you may experience a brief pause. This occurs as CuPy compiles the CUDA functions for the first time and cached them on disk for future use.\n" ] }, { "cell_type": "markdown", "id": "662bb0c3-4051-4125-b801-173b8b3c30b5", "metadata": {}, "source": [ "Now, let's make the same comparison with different array sizes." ] }, { "cell_type": "markdown", "id": "29b798e6-4c8b-44c6-86df-b92abdb0a683", "metadata": {}, "source": [ "We can use the following function to find the size of a variable on memory. " ] }, { "cell_type": "code", "execution_count": 15, "id": "e4fdad11-9e3b-4f65-9dce-9ffbd33dc419", "metadata": {}, "outputs": [], "source": [ "# Define function to display variable size in MB\n", "import sys\n", "\n", "\n", "def var_size(in_var):\n", " result = sys.getsizeof(in_var) / 1e6\n", " print(f\"Size of variable: {result:.2f} MB\")" ] }, { "cell_type": "code", "execution_count": 33, "id": "bba9681e-ca7b-486c-92c8-1a79434ba0da", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "n = 100\n", "Size of variable: 0.08 MB\n", "CuPy provides a 6.45 x speedup over NumPy.\n", "\n", "n = 200\n", "Size of variable: 0.32 MB\n", "CuPy provides a 1.28 x speedup over NumPy.\n", "\n", "n = 500\n", "Size of variable: 2.00 MB\n", "CuPy provides a 9.83 x speedup over NumPy.\n", "\n", "n = 1000\n", "Size of variable: 8.00 MB\n", "CuPy provides a 41.17 x speedup over NumPy.\n", "\n", "n = 2000\n", "Size of variable: 32.00 MB\n", "CuPy provides a 72.55 x speedup over NumPy.\n", "\n", "n = 5000\n", "Size of variable: 200.00 MB\n", "CuPy provides a 77.9 x speedup over NumPy.\n", "\n", "n = 10000\n", "Size of variable: 800.00 MB\n", "CuPy provides a 80.68 x speedup over NumPy.\n", "\n" ] } ], "source": [ "speed_ups = []\n", "arr_sizes = []\n", "sizes = [100, 200, 500, 1000, 2000, 5000, 10000]\n", "for n in sizes:\n", " print(\"n =\", n)\n", "\n", " # create two nxn matrices\n", " a_np = np.random.rand(n, n)\n", " b_np = np.random.rand(n, n)\n", "\n", " a_cp = cp.asarray(a_np)\n", " b_cp = cp.asarray(b_np)\n", "\n", " arr_size = a_cp.nbytes / 1e6\n", " print(f\"Size of variable: {arr_size:.2f} MB\")\n", "\n", " # perform matrix multiplication with NumPy and time it\n", " start_time = time.time()\n", " c_np = np.matmul(a_np, b_np)\n", " end_time = time.time()\n", " numpy_time = end_time - start_time\n", "\n", " # perform matrix multiplication with CuPy and time it\n", " start_time = time.time()\n", " c_cp = cp.matmul(a_cp, b_cp)\n", " cp.cuda.Stream.null.synchronize() # wait for GPU computation to finish\n", " end_time = time.time()\n", " cupy_time = end_time - start_time\n", "\n", " speed_up = round(numpy_time / cupy_time, 2)\n", "\n", " speed_ups.append(speed_up)\n", " arr_sizes.append(arr_size)\n", " # print the speedup\n", " print(\"CuPy provides a\", speed_up, \"x speedup over NumPy.\\n\")" ] }, { "cell_type": "markdown", "id": "fc3dd3c8-032f-437b-a6d0-7dae7e55b73c", "metadata": {}, "source": [ "We can also create a plot of data size vs. speed-ups:" ] }, { "cell_type": "code", "execution_count": 36, "id": "fd5dbadf-8286-464d-880c-c25297ed310d", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcsAAAHACAYAAADNxUOEAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAABSPElEQVR4nO3de1xUdf4/8NdwGy7CgAoDKCIa5AUveNdcxbymubXuupVpmm1rXkq0VnO1FVsDdR+Zv9qyclu1dc22TffbRREqNVs0L4A3XEtFQAVRgRkUhoGZz+8PnCPjgMyBGWZgXs/HYx7BOYeZN6fy5ed8bgohhAARERHVy83RBRARETk7hiUREVEDGJZEREQNYFgSERE1gGFJRETUAIYlERFRAxiWREREDWBYEhERNcDD0QXYm9FoxNWrV+Hv7w+FQuHocoiIyEGEECgrK0N4eDjc3OS1FVt9WF69ehURERGOLoOIiJxEfn4+OnbsKOtnWn1Y+vv7A6i5OQEBAQ6uhoiIHEWr1SIiIkLKBTlafViaHr0GBAQwLImIqFFdchzgQ0RE1ACGJRERUQMYlkRERA1gWBIRETWAYUlERNQAhiUREVEDGJZEREQNYFgSERE1gGFJRETUgFa/gg8REbVcBqPAkZxiFJXpEOLvjUFRbeHu1vybYjAsiYjIKaWcLsCqL7NRoNFJx8JU3lg5uQcmxIY1ay18DEtERE4n5XQB5m7LMAtKACjU6DB3WwZSThc0az0MSyIicioGo8CqL7Mh6jhnOrbqy2wYjHVdYR8ODcvq6mqsWLECUVFR8PHxQZcuXfD666/DaDRK1wghkJiYiPDwcPj4+CA+Ph5nzpxxYNVERNRUuioDCjU6/K9Qix8v3sTeM4X419F8bPr+Il7+V5ZFi7I2AaBAo8ORnOJmq9ehfZZr167F+++/j61bt6Jnz544duwYnn32WahUKixcuBAAsG7dOqxfvx5btmxBTEwMVq9ejbFjx+LcuXON2pOMiIhsw2AU0FZUQVNRhVLTP8v10FRUQVNec6y0vOa4pkIvfV1aUQV9tbHhD2hAUVn9gWprDg3LQ4cO4bHHHsOkSZMAAJ07d8Ynn3yCY8eOAahpVW7YsAHLly/HlClTAABbt26FWq3G9u3bMWfOHIfVTkTUGgghoKsyorRCfyfsal7aiiqU3hNwWtP5Cj005VXQ6qqb9NnubgoE+nhC5eMJla+n9HVFlQF7z1xr8OdD/L2b9PlyODQshw8fjvfffx8//fQTYmJicOLECfzwww/YsGEDACAnJweFhYUYN26c9DNKpRIjR45Eenp6nWFZWVmJyspK6XutVmv334OIyNFMrbzS2q07qbVXfwtPY4NWnp+XOwJ9vWpCz8cTgb61A9BLOhbo44kA09e+XvDzcq9zI2aDUWD42u9QqNHV2W+pABCqqplG0lwcGpZLly6FRqNBt27d4O7uDoPBgDfeeANPPfUUAKCwsBAAoFarzX5OrVYjNze3zvdMTk7GqlWr7Fs4EZEdCCFQUWW4G2b1BZz09d3WYFkTW3kebgqLFt69ASiFYK0AVPl4wtPdtsNf3N0UWDm5B+Zuy4ACMAtMU7SunNyjWedbOjQsP/30U2zbtg3bt29Hz549kZWVhYSEBISHh2PmzJnSdff+zUMIUeffRgBg2bJlWLx4sfS9VqtFRESEfX4BIqI6VBuM0OqqpRaeWcDVDrpaAWh69Kk3NK2V10bpYdHCq/mneQtPCsY7gVhfK89RJsSGYeP0fhbzLEMdNM/SoWH5hz/8Aa+++iqefPJJAECvXr2Qm5uL5ORkzJw5E6GhoQBqWphhYXdvTFFRkUVr00SpVEKpVNq/eCJq1YQQKNcb7j7GrNBLfXb3Blztvj1NeRXKKpveyrvbiqsJNLNHmLUecQbc84jT1q08R5oQG4axPUK5gk95eTnc3Mz/xbq7u0tTR6KiohAaGoq0tDTExcUBAPR6PQ4cOIC1a9c2e71E1PJUG4x3++7MWnh6qc/OvIWnh6aiGpoKPaoMTZvH56/0qNVHZ/kI03xwi5f0CNTXyVp5juTupsDQru0cXYZjw3Ly5Ml444030KlTJ/Ts2ROZmZlYv349Zs+eDaDm8WtCQgKSkpIQHR2N6OhoJCUlwdfXF9OmTXNk6UROz1nW1LQFUyvPbPDKPS08i769O/+81cRWnqe74k7AeUgtPPPBKx53+/ZqBWBra+W5OoeG5TvvvIPXXnsN8+bNQ1FREcLDwzFnzhz86U9/kq5ZsmQJKioqMG/ePJSUlGDw4MFITU3lHEui+3CmNTVrq6rdyqtreoIUeuYjOkvLq1DdxNVa/JUed/ro7jzavOcRZp19e76e8PFkK48AhRCi+dYLcgCtVguVSgWNRoOAgABHl0Nkd6Y1Ne/9H9v0x/3G6f2aFJhCCNzWGywmn9fbt1crAG3VyqvzEabFlIW7/X0B3h7wYCvP5TUlD7jrCFEr0tCamgrUrKk5tkcojELUCjO9WbDd28K7t2+vya08bw+zFp5lwFm28FQ+bOWR4zAsiVqRIznFVq2p2fNPKdA1cSK6l7ubVQF374hOf7byqAViWBK1cEajwMUbt5GRV4JdmVes+pnaQRng7SE9ygz0vTM9ofZE9Np9e7Vag96ebmzlkctgWBK1MFpdFbLySpGZV4qMvBJk5ZdCU1El6z3e+m0fxD8YggAfzxY7QpaoOTEsiZyY0Shw/votZOSWSOF4/vot3Dssz9vTDb07BKJPhAqfHb+M0vK6w9O0puYv+3ZgSBLJwLAkciKl5Xpk5pciM7cEmfmlyMorrXM1mE5tfdGvUyDiOgWhX6cgdAvzl+b09Y8MwtxtGQCcY01NotaAYUnkIAajwE/XypCRd7fVePH6bYvrfDzd0SdCJQVjXKdAtG9T/5KOzramJlFrwLAkaibFt/XIrBWMJ/JLcVtvsLguqr0f4qRWYyAeVPvLHj3qTGtqErUGDEsiO6g2GPG/wjKzcLx0s9ziOj8vd/TtFCi1GPtGBKGtn5dNanCWNTWJWgOGJZENXC+rRGZeCTLySpGZV4KTlzWoqLJsNXYN9rsTjEHoFxmI6BB/tvaIWgCGJZFMVQYjzhZoa0ao5te0GvOLKyyu8/f2QN+Iu63GuIggqHw9HVAxETUVw5KoAde0OotWY+U9q98oFEB0SBspGPt1CkLX4DZwY6uRqFVgWBLVUlltQPZVrRSMmXmluFJq2WpU+XhKoRjXKRB9IgIR4M1WI1FrxbAkl1agqUBGbumd6RslOH1VC/09rUY3BRCj9ke/yLtTN7q09+NSb0QuhGFJLkNXZcCZqxpk5JYiM78EGbmlKNRaLjre1s8LcRGB6BcZhLiIQPSOCEQbJf9XIXJl/BOAWiUhBK6UViAjr1QaiJN9VYMqg/k6ce5uCnQP80dcRM3o1LiIIES282WrkYjMMCypVajQG3DqikZ6nJqRV4rrZZUW17Vv42W2Ek7vjir4evF/AyK6P/4pQS2OEAJ5xeXSZP/MvFKcLdBabEjs4aZAz/AAxNUaodoxyIetRiKSjWFJTu92ZTVOXtZIwZiVX4Ibt/QW14X4K9HvzmT/fp2CENtBBW9PdwdUTEStDcOSnIoQApduliMjt0QKx/8VanFPoxFe7m7o2SHgbl9jpyCEq7zZaiQiu2BYkkOV6apqWo13BuFk5pWgpI69GMNV3tLj1LhOQegZHsBWIxE1G4YlyWIwikbvZGE0Cly8cctswv+5a2UWGxl7ebihdwdVrUn/QQhVedvhtyEisg7DkqyWcrrAYo/EsPvskaipqMKJ/LuDcDLzSqDVWW5k3DHIR9qOql+nIHQPC4CXh7wtqYiI7IlhSVZJOV2AudsycE8jEIUaHeZuy8C70/qha0ibO9M2asLx/PVbFq1Gb0839O4YaLZUXIg/W41E5NwYltQgg1Fg1ZfZFkEJQDo2f7tlkAJAZDtfs8XFHwz1h6fMjYyJiByNYUkNOpJTbPbotS4CgNLDzSwY+3YKRPs2yuYpkojIjhiW1KCisvsHpcmaKb3wq34d7VwNEVHz4/MwapC1fYqhKh87V0JE5BgMS2rQoKi2CFN5o74JIgrUjIodFNW2OcsiImo2DEtqkLubAisn96jznClAV07uYfV8SyKiloZhSVaZEBuGd6bFWbQuQ1Xe2Di9X53zLImIWgsO8CGrham8IQD4e3vgz4/FQh0gbwUfIqKWimFJVks/fxMAMCI6GI/HdXBwNUREzcehj2E7d+4MhUJh8Zo/fz6Amh0oEhMTER4eDh8fH8THx+PMmTOOLNmlpV+oCcuhXds5uBIioubl0LA8evQoCgoKpFdaWhoAYOrUqQCAdevWYf369fjrX/+Ko0ePIjQ0FGPHjkVZWZkjy3ZJuioDjueVAGBYEpHrcWhYBgcHIzQ0VHp99dVX6Nq1K0aOHAkhBDZs2IDly5djypQpiI2NxdatW1FeXo7t27c7smyXlJFXAn21EeoAJbq093N0OUREzcppRsPq9Xps27YNs2fPhkKhQE5ODgoLCzFu3DjpGqVSiZEjRyI9Pb3e96msrIRWqzV7UdMduvMIdljX9txgmYhcjtOE5X/+8x+UlpZi1qxZAIDCwkIAgFqtNrtOrVZL5+qSnJwMlUolvSIiIuxWsythfyURuTKnCcuPPvoIjzzyCMLDw82O39uKEULct2WzbNkyaDQa6ZWfn2+Xel3JrcpqnMgvBQAMY1gSkQtyiqkjubm5+Oabb7Bz507pWGhoKICaFmZY2N0J70VFRRatzdqUSiWUSu50YUtHLxWj2ijQqa0vOgb5OrocIqJm5xQty82bNyMkJASTJk2SjkVFRSE0NFQaIQvU9GseOHAAw4YNc0SZLutufyVblUTkmhzesjQajdi8eTNmzpwJD4+75SgUCiQkJCApKQnR0dGIjo5GUlISfH19MW3aNAdW7HrSL9wAwP5KInJdDg/Lb775Bnl5eZg9e7bFuSVLlqCiogLz5s1DSUkJBg8ejNTUVPj7+zugUtekKa/Cmas1I4qHdmFYEpFrUgghhKOLsCetVguVSgWNRoOAgABHl9Pi7D1TiDn/OI4HQtrgm8UjHV0OEVGjNSUPZLUshRA4cOAADh48iEuXLqG8vBzBwcGIi4vDmDFjOE2jFWJ/JRGRlQN8KioqkJSUhIiICDzyyCP4+uuvUVpaCnd3d5w/fx4rV65EVFQUJk6ciMOHD9u7ZmpGpv5KhiURuTKrWpYxMTEYPHgw3n//fYwfPx6enp4W1+Tm5mL79u144oknsGLFCjz//PM2L5aa1/WySvx07RYUCmBwFMOSiFyXVWG5Z88exMbG3veayMhILFu2DC+//DJyc3NtUhw51qGLNY9gu4cGIMjPy8HVEBE5jlWPYRsKytq8vLwQHR3d6ILIeRziI1giIgCNXJTg4MGDmD59OoYOHYorV64AAP7xj3/ghx9+sGlx5FjS4J4HGJZE5Npkh+Xnn3+O8ePHw8fHB5mZmaisrAQAlJWVISkpyeYFkmNcKa3ApZvlcHdTYGDnto4uh4jIoWSH5erVq/H+++9j06ZNZgN9hg0bhoyMDJsWR45jalX27qiCv7flgC4iIlciOyzPnTuHESNGWBwPCAhAaWmpLWoiJ8ApI0REd8kOy7CwMJw/f97i+A8//IAuXbrYpChyLCGE2WbPRESuTnZYzpkzBwsXLsSPP/4IhUKBq1ev4p///CdeeeUVzJs3zx41UjO7dLMcBRodvNzd0D8yyNHlEBE5nOyF1JcsWQKNRoNRo0ZBp9NhxIgRUCqVeOWVV7BgwQJ71EjNzPQINq5TILw93R1cDRGR4zVq15E33ngDy5cvR3Z2NoxGI3r06IE2bdrYujZyED6CJSIy1+gtunx9fTFgwABb1kJOwKy/kvMriYgANCIsdTod3nnnHezbtw9FRUUwGo1m5zl9pGX76dot3Lyth4+nO/p0DHR0OURETkF2WM6ePRtpaWn4zW9+g0GDBkGhUNijLnIQU3/lwKi28PJo1AJPREStjuyw/Prrr7F792489NBD9qiHHCyd+1cSEVmQ3XTo0KED/P397VELOZjBKHD4zk4jQ7swLImITGSH5ZtvvomlS5dyG65W6MxVDcp01fD39kDP8ABHl0NE5DRkP4YdMGAAdDodunTpAl9fX4uNoIuLi21WHDUv0yjYwVHt4OHO/koiIhPZYfnUU0/hypUrSEpKglqt5gCfVoT9lUREdZMdlunp6Th06BD69Oljj3rIQfTVRhy9VPNUgPMriYjMyX7W1q1bN1RUVNijFnKgk5dLUa43oJ2fF2JCOICLiKg22WG5Zs0avPzyy9i/fz9u3rwJrVZr9qKWyfQIdkiXdnBz46N1IqLaZD+GnTBhAgBg9OjRZseFEFAoFDAYDLapjJqVaTGCoeyvJCKyIDss9+3bZ486yIF0VQZk5JYC4OAeIqK6yA7LkSNH2qMOcqCM3BLoDUaEBngjqr2fo8shInI6VoXlyZMnERsbCzc3N5w8efK+1/bu3dsmhVHzqT1lhFOBiIgsWRWWffv2RWFhIUJCQtC3b18oFAoIISyuY59ly8T+SiKi+7MqLHNychAcHCx9Ta3HrcpqnLisAcCwJCKqj1VhGRkZKX2dm5uLYcOGwcPD/Eerq6uRnp5udi05v6M5xTAYBTq19UXHIF9Hl0NE5JRkz7McNWpUneu/ajQajBo1yiZFUfMxPYLlKFgiovrJDkvTfMp73bx5E35+HEnZ0pgG9/ARLBFR/ayeOjJlyhQANYN4Zs2aBaVSKZ0zGAw4efIkhg0bJruAK1euYOnSpdizZw8qKioQExODjz76CP379wdQE86rVq3Chx9+iJKSEgwePBjvvvsuevbsKfuzyFxpuR7ZBTWrLjEsiYjqZ3VYqlQqADXh5e/vDx8fH+mcl5cXhgwZgueff17Wh5eUlOChhx7CqFGjsGfPHoSEhODChQsIDAyUrlm3bh3Wr1+PLVu2ICYmBqtXr8bYsWNx7tw5bkLdRIcvFkMIIDqkDUL8vR1dDhGR07I6LDdv3gwA6Ny5M1555RWbPHJdu3YtIiIipPc2vb+JEAIbNmzA8uXLpZbt1q1boVarsX37dsyZM6fJNbiyQ+yvJCKyiuw+y5UrV9qsb/KLL77AgAEDMHXqVISEhCAuLg6bNm2Szufk5KCwsBDjxo2TjimVSowcORLp6el1vmdlZSUXd7fS3f7K9g6uhIjIuckOS1u6ePEiNm7ciOjoaOzduxcvvPACXnrpJXz88ccAgMLCQgCAWq02+zm1Wi2du1dycjJUKpX0ioiIsO8v0UIVlenwc9EtKBTAkC5tHV0OEZFTc2hYGo1G9OvXD0lJSYiLi8OcOXPw/PPPY+PGjWbX3Tv6tr4RuQCwbNkyaDQa6ZWfn2+3+luyQ3dalT3CAhDo6+XgaoiInJtDwzIsLAw9evQwO9a9e3fk5eUBAEJDQwHAohVZVFRk0do0USqVCAgIMHuRpUO11oMlIqL7kx2Wtlzu7qGHHsK5c+fMjv3000/SKkBRUVEIDQ1FWlqadF6v1+PAgQONmqZCdx26aApL9lcSETVEdlg+8MADGDVqFLZt2wadTtekD1+0aBEOHz6MpKQknD9/Htu3b8eHH36I+fPnA6h5/JqQkICkpCTs2rULp0+fxqxZs+Dr64tp06Y16bNd2eWScuTeLIe7mwIDo9hfSUTUENlheeLECcTFxeHll19GaGgo5syZgyNHjjTqwwcOHIhdu3bhk08+QWxsLP785z9jw4YNePrpp6VrlixZgoSEBMybNw8DBgzAlStXkJqayjmWTWB6BNunowptlLK3NCUicjkKUddeW1aorq7Gl19+iS1btmDPnj2Ijo7Gc889hxkzZkg7lDgDrVYLlUoFjUbD/ss7Fn+ahZ2ZV7Bg1AN4ZfyDji6HiKhZNCUPGj3Ax8PDA7/61a/wr3/9C2vXrsWFCxfwyiuvoGPHjnjmmWdQUFDQ2LcmOxJCcD1YIiKZGh2Wx44dw7x58xAWFob169fjlVdewYULF/Ddd9/hypUreOyxx2xZJ9lIzo3bKNTq4OXuhv6RQY4uh4ioRZDdYbV+/Xps3rwZ586dw8SJE/Hxxx9j4sSJcHOryd2oqCh88MEH6Natm82LpaYztSr7RQbC29PdwdUQEbUMssNy48aNmD17Np599llpHuS9OnXqhI8++qjJxZHt3Z1fySkjRETWkh2WP//8c4PXeHl5YebMmY0qiOzHaBQ4fJGLERARyWV1n2V5eTnmz5+PDh06ICQkBNOmTcONGzfsWRvZ2E9FZbh5Ww9fL3f07hjo6HKIiFoMq8Ny5cqV2LJlCyZNmoQnn3wSaWlpmDt3rj1rIxtLP1/TqhzQuS28PBy60iERUYti9WPYnTt34qOPPsKTTz4JAJg+fToeeughGAwGuLtzoEhLkM71YImIGsXq5kV+fj5+8YtfSN8PGjQIHh4euHr1ql0KI9uqNhjxI/sriYgaxeqwNBgM8PIy38rJw8MD1dXVNi+KbO/MVS3KKqvh7+2BnuEqR5dDRNSiWP0YVgiBWbNmQalUSsd0Oh1eeOEF+Pn5Scd27txp2wrJJkyPYId0aQd3t7r3AiUiorpZHZZ1TQWZPn26TYsh+znER7BERI1mdVhu3rzZnnWQHemrjTiaUwyAixEQETUG5w+4gBOXS1FRZUA7Py/EqNs4uhwiohZH9go+Op0O77zzDvbt24eioiIYjUaz8xkZGTYrjmzDNL9ySNd2UCjYX0lEJJfssJw9ezbS0tLwm9/8BoMGDeIfvi1A+oWalZbYX0lE1Diyw/Lrr7/G7t278dBDD9mjHrKxCr0BmXmlANhfSUTUWLL7LDt06AB/f3971EJ2cDy3BHqDEWEqb3Ru5+vocoiIWiTZYfnmm29i6dKlyM3NtUc9ZGOHLtY8gh3K/koiokaT/Rh2wIAB0Ol06NKlC3x9feHp6Wl2vri42GbFUdOlc/9KIqImkx2WTz31FK5cuYKkpCSo1Wq2VpxYma4KJy9rANS0LImIqHFkh2V6ejoOHTqEPn362KMesqGjl4phMApEtvNFh0AfR5dDRNRiye6z7NatGyoqKuxRC9mYaX4lp4wQETWN7LBcs2YNXn75Zezfvx83b96EVqs1e5HzMPVXDmV/JRFRk8h+DDthwgQAwOjRo82OCyGgUChgMBhsUxk1ScltPbILav7yMrQLW5ZERE0hOyz37dtnjzrIxn7MqWlVxqjbINhf2cDVRER0P7LDcuTIkfaog2xMegTLViURUZPJDsvvv//+vudHjBjR6GLIdthfSURkO7LDMj4+3uJY7bmW7LN0vCKtDueLbkGhAIZ0aevocoiIWjzZo2FLSkrMXkVFRUhJScHAgQORmppqjxpJpkMXa1qVPcMDEOjr5eBqiIhaPtktS5VKZXFs7NixUCqVWLRoEY4fP26Twqjx7s6v5CNYIiJbkN2yrE9wcDDOnTtnq7ejJkivtXg6ERE1neyW5cmTJ82+F0KgoKAAa9as4RJ4TiC/uBz5xRXwcFNgYGf2VxIR2YLslmXfvn0RFxeHvn37Sl9PnDgRer0eH330kaz3SkxMhEKhMHuFhoZK54UQSExMRHh4OHx8fBAfH48zZ87ILdmlmPore3dUoY1S9t+FiIioDrL/NM3JyTH73s3NDcHBwfD29m5UAT179sQ333wjfe/u7i59vW7dOqxfvx5btmxBTEwMVq9ejbFjx+LcuXPcgLoeh7glFxGRzckOy8jISNsW4OFh1po0EUJgw4YNWL58OaZMmQIA2Lp1K9RqNbZv3445c+bYtI7WQAiB9As1/ZVcPJ2IyHasDsuPP/7YquueeeYZWQX8/PPPCA8Ph1KpxODBg5GUlIQuXbogJycHhYWFGDdunHStUqnEyJEjkZ6eXm9YVlZWorKyUvrelRZ3v3jjNq5pK+Hl4YZ+kUGOLoeIqNWwOiwXLlxY7zmFQoHbt2+jurpaVlgOHjwYH3/8MWJiYnDt2jWsXr0aw4YNw5kzZ1BYWAgAUKvVZj+jVquRm5tb73smJydj1apVVtfQmphW7enfKQjenu4NXE1ERNayeoDPvYsRmF7Z2dn47W9/CyEExo4dK+vDH3nkEfz6179Gr169MGbMGHz99dcAah63mtReHQi4u7tJfZYtWwaNRiO98vPzZdXUkh3iI1giIrto9DzLsrIyrFixAjExMcjKysLevXuRkpLSpGL8/PzQq1cv/Pzzz1I/pqmFaVJUVGTR2qxNqVQiICDA7OUKjEaBwxeLAQDDHmBYEhHZkuyw1Ov1WL9+PaKiovDvf/8bmzdvxuHDhzFq1KgmF1NZWYmzZ88iLCwMUVFRCA0NRVpamtlnHzhwAMOGDWvyZ7U2566Vofi2Hr5e7ujdMdDR5RARtSpW91kKIfDxxx/jT3/6E6qrq5GUlITnnnvObKqHXK+88gomT56MTp06oaioCKtXr4ZWq8XMmTOhUCiQkJCApKQkREdHIzo6GklJSfD19cW0adMa/Zmtlam/cmDntvB0t9nCTEREBBlh2adPH1y4cAEvvvgiEhIS4Ovri9u3b1tcJ+ex5+XLl/HUU0/hxo0bCA4OxpAhQ3D48GFpesqSJUtQUVGBefPmoaSkBIMHD0ZqairnWNaB/ZVERPajEEIIay50c7vbWqlrgI1p4I2zbdGl1WqhUqmg0Whabf9ltcGIuNfTUFZZjS8XDEevjpaL3RMRubqm5IHVLct9+/bJLoyax+mrWpRVViPA2wM9wlvnXwiIiBzJ6rAcOXKkPeugJjCt2jOkSzu4u9U/rYaIiBqHI0FagbvrwbK/kojIHhiWLZy+2oijl2rmVw7l4ulERHbBsGzhsvJLoasyop2fF2LUbRxdDhFRq8SwbOFM/ZVDu7a77zKARETUeFaHZXh4OObOnYs9e/ZAr9fbsyaSIZ37VxIR2Z3VYbl9+3b4+vripZdeQvv27TF16lT84x//QHFxsT3ro/uo0BuQmVcCgIN7iIjsyeqwjI+Px5tvvomff/4Zhw4dQr9+/fDuu+8iLCwM8fHxeOutt3DhwgV71kr3OJZbjCqDQLjKG5HtfB1dDhFRq9WoPsuePXti2bJlOHz4MPLy8vD000/ju+++Q69evRAbGytttUX2ZZoyMoT9lUREdmX1ogT1UavVeP755/H888+jvLwce/fuhVKptEVt1AD2VxIRNY8mh2Vtvr6++NWvfmXLt6R6aHVVOHm5FEDNSFgiIrIfTh1poY7mFMMogM7tfNEh0MfR5RARtWoMyxbK9AiWq/YQEdkfw7KFSud6sEREzabRYXn+/Hns3bsXFRUVAGr2s6TmUXxbj7MFWgA1O40QEZF9yQ7LmzdvYsyYMYiJicHEiRNRUFAAAPjd736Hl19+2eYFkqXDF2talTHqNgj258hjIiJ7kx2WixYtgoeHB/Ly8uDre3ci/BNPPIGUlBSbFkd1O8QpI0REzUr21JHU1FTs3bsXHTt2NDseHR2N3NxcmxVG9au9eDoREdmf7Jbl7du3zVqUJjdu3OBiBM3gmlaHC9dvQ6EAhkQxLImImoPssBwxYgQ+/vhj6XuFQgGj0Yi//OUvGDVqlE2LI0umR7Cx4SqofD0dXA0RkWuQ/Rj2L3/5C+Lj43Hs2DHo9XosWbIEZ86cQXFxMf773//ao0aqxfQIllNGiIiaj+yWZY8ePXDy5EkMGjQIY8eOxe3btzFlyhRkZmaia9eu9qiRarm7GAHDkoiouTRqbdjQ0FCsWrXK1rVQA/KLy3G5pAIebgoM7NzW0eUQEbmMRoWlTqfDyZMnUVRUBKPRaHbul7/8pU0KI0um/so+EYHwU9p0DXwiIroP2X/ipqSk4JlnnsGNGzcszikUChgMBpsURpbYX0lE5Biy+ywXLFiAqVOnoqCgAEaj0ezFoLQfIQT7K4mIHER2y7KoqAiLFy+GWq22Rz1UB4NRYFfmFRSVVcLDTYE+HQMdXRIRkUuR3bL8zW9+g/3799uhFKpLyukCDF/7HV757AQAoNooMGb9AaScLnBwZURErkMhZG4XUl5ejqlTpyI4OBi9evWCp6f5xPiXXnrJpgU2lVarhUqlgkajQUBAgKPLkSXldAHmbsvAvf+CFHf+uXF6P0yIDWvusoiIWqSm5IHsx7Dbt2/H3r174ePjg/3790OhUEjnFAqF04VlS2UwCqz6MtsiKAFAoCYwV32ZjbE9QuHupqjjKiIishXZYblixQq8/vrrePXVV+Hmxr2j7eVITjEKNLp6zwsABRodjuQUc8APEZGdyU47vV6PJ554gkFpZ0Vl9QdlY64jIqLGk514M2fOxKeffmrzQpKTk6FQKJCQkCAdE0IgMTER4eHh8PHxQXx8PM6cOWPzz3ZGIf7eNr2OiIgaT/ZjWIPBgHXr1mHv3r3o3bu3xQCf9evXyy7i6NGj+PDDD9G7d2+z4+vWrcP69euxZcsWxMTEYPXq1Rg7dizOnTsHf39/2Z/TkgyKaoswlTcKNbo6+y0VAEJV3hgUxWXviIjsTXbL8tSpU4iLi4ObmxtOnz6NzMxM6ZWVlSW7gFu3buHpp5/Gpk2bEBQUJB0XQmDDhg1Yvnw5pkyZgtjYWGzduhXl5eXYvn277M9padzdFFg5uUed50zDeVZO7sHBPUREzUB2y3Lfvn02LWD+/PmYNGkSxowZg9WrV0vHc3JyUFhYiHHjxknHlEolRo4cifT0dMyZM8emdTijCbFh2Di9H+b+MwO1J/iEqryxcnIPThshImomDl2Ne8eOHcjIyMDRo0ctzhUWFgKAxUpBarUaubm59b5nZWUlKisrpe+1Wq2NqnWMX0QHS0G55te9ENnWD4Oi2rJFSUTUjKwKyylTpmDLli0ICAjAlClT7nvtzp07rfrg/Px8LFy4EKmpqfD2rn+QSu15nEDN49l7j9WWnJzcqrYPyy8pBwCofDzx5MBODq6GiMg1WdVnqVKppIAKCAiASqWq92Wt48ePo6ioCP3794eHhwc8PDxw4MABvP322/Dw8JBalKYWpklRUdF916VdtmwZNBqN9MrPz7e6JmeUX1wBAIho6+PgSoiIXJdVLcvNmzdLX2/ZssUmHzx69GicOnXK7Nizzz6Lbt26YenSpejSpQtCQ0ORlpaGuLg4ADVzPA8cOIC1a9fW+75KpRJKpdImNTqD/OKalmVEkK+DKyEicl2yR8M+/PDDKC0ttTiu1Wrx8MMPW/0+/v7+iI2NNXv5+fmhXbt2iI2NleZcJiUlYdeuXTh9+jRmzZoFX19fTJs2TW7ZLZbpMWxEW4YlEZGjyB7gs3//fuj1eovjOp0OBw8etElRJkuWLEFFRQXmzZuHkpISDB48GKmpqa1+jmVt0mPYID6GJSJyFKvD8uTJk9LX2dnZZn2JBoMBKSkp6NChQ5OKuXfrL4VCgcTERCQmJjbpfVuyy3dalh3ZsiQichirw7Jv375QKBRQKBR1Pm718fHBO++8Y9PiXJ0Qgn2WREROwOqwzMnJgRACXbp0wZEjRxAcHCyd8/LyQkhICNzd3e1SpKsqKa/Cbb0BANCRj2GJiBzG6rCMjIwEABiNRrsVQ+ZMrcoQfyW8PfkXESIiR+E+W06MI2GJiJwDw9KJcSQsEZFzYFg6MbYsiYicA8PSiXEkLBGRc5AdlrNmzcL3339vj1roHpdLah7DduS6sEREDiU7LMvKyjBu3DhER0cjKSkJV65csUddLs9oFLhSYuqzZMuSiMiRZIfl559/jitXrmDBggX47LPP0LlzZzzyyCP497//jaqqKnvU6JKulemgNxjh7qZAmKr+LcyIiMj+GtVn2a5dOyxcuBCZmZk4cuQIHnjgAcyYMQPh4eFYtGgRfv75Z1vX6XJMI2HDA73h4c6uZSIiR2rSn8IFBQVITU1Famoq3N3dMXHiRJw5cwY9evTAW2+9ZasaXZJpTVg+giUicjzZYVlVVYXPP/8cjz76KCIjI/HZZ59h0aJFKCgowNatW5Gamop//OMfeP311+1Rr8u4O8eSYUlE5Giyt+gKCwuD0WjEU089hSNHjqBv374W14wfPx6BgYE2KM913Z1jyZGwRESOJjss33rrLUydOhXe3vUPOgkKCkJOTk6TCnN10hxLLkhARORwsh7D5ubmQqfTYfPmzcjOzrZXTYRacyz5GJaIyOGsbll+//33mDhxIsrLa1o8Hh4e2Lp1K5566im7FeeqqgxGFGju9FnyMSwRkcNZ3bJ87bXXMGrUKFy+fBk3b97E7NmzsWTJEnvW5rKullbAKAClhxuC2ygdXQ4RkcuzOixPnTqF5ORkhIeHIygoCG+++SauXr2KkpISe9bnkkwjYTsG+UChUDi4GiIisjosS0tLERISIn3v5+cHX19flJaW2qMul8bdRoiInIus0bDZ2dkoLCyUvhdC4OzZsygrK5OO9e7d23bVuSjuNkJE5FxkheXo0aMhhDA79uijj0KhUEAIAYVCAYPBYNMCXVF+CQf3EBE5E6vDkvMmmw9blkREzsXqsIyMjLRnHVTLZfZZEhE5Fdkr+DS08fOIESMaXQwB5fpq3LilB8CWJRGRs5AdlvHx8RbHak9vYJ9l05hW7vH39oDK19PB1RAREdCIXUdKSkrMXkVFRUhJScHAgQORmppqjxpdCvsriYicj+yWpUqlsjg2duxYKJVKLFq0CMePH7dJYa7q7gLqHAlLROQsmrT5c23BwcE4d+6crd7OZUnTRtiyJCJyGrJblidPnjT7XgiBgoICrFmzBn369LFZYa6KW3MRETkf2WHZt29faRGC2oYMGYK///3vNivMVXFBAiIi5yM7LO9dnMDNzQ3BwcH33QyarCOEwGUO8CEicjqyw5KLE9iPpqIKZZXVALjpMxGRM7F6gM93332HHj16QKvVWpzTaDTo2bMnDh48KOvDN27ciN69eyMgIAABAQEYOnQo9uzZI50XQiAxMRHh4eHw8fFBfHw8zpw5I+szWhLT1lzt2yjh4+Xu4GqIiMjE6rDcsGEDnn/+eQQEBFicU6lUmDNnDtavXy/rwzt27Ig1a9bg2LFjOHbsGB5++GE89thjUiCuW7cO69evx1//+lccPXoUoaGhGDt2rNkuJ63J3a252F9JRORMrA7LEydOYMKECfWeHzdunOw5lpMnT8bEiRMRExODmJgYvPHGG2jTpg0OHz4MIQQ2bNiA5cuXY8qUKYiNjcXWrVtRXl6O7du3y/qcloILEhAROSerw/LatWvw9Kx/+TUPDw9cv3690YUYDAbs2LEDt2/fxtChQ5GTk4PCwkKMGzdOukapVGLkyJFIT09v9Oc4M7YsiYick9UDfDp06IBTp07hgQceqPP8yZMnERYWJruAU6dOYejQodDpdGjTpg127dqFHj16SIGoVqvNrler1cjNza33/SorK1FZWSl9X1cfq7My9VmyZUlE5FysbllOnDgRf/rTn6DT6SzOVVRUYOXKlXj00UdlF/Dggw8iKysLhw8fxty5czFz5kxkZ2dL52sv0g5A2mS6PsnJyVCpVNIrIiJCdk2Oks+tuYiInJJC3Lu6QD2uXbuGfv36wd3dHQsWLMCDDz4IhUKBs2fP4t1334XBYEBGRoZFS1CuMWPGoGvXrli6dCm6du2KjIwMxMXFSecfe+wxBAYGYuvWrXX+fF0ty4iICGg0mjoHJzkLo1Gg259SoK824vs/jEKndgxMIiJb0mq1UKlUjcoDqx/DqtVqpKenY+7cuVi2bJm0go9CocD48ePx3nvvNTkogZqWY2VlJaKiohAaGoq0tDQpLPV6PQ4cOIC1a9fW+/NKpRJKpbLJdTS367cqoa82wk0BhAVygQciImcia1GCyMhI7N69GyUlJTh//jyEEIiOjkZQUFCjPvyPf/wjHnnkEURERKCsrAw7duzA/v37kZKSAoVCgYSEBCQlJSE6OhrR0dFISkqCr68vpk2b1qjPc2amkbBhKh94uttsfXsiIrIB2Sv4AEBQUBAGDhzY5A+/du0aZsyYgYKCAqhUKvTu3RspKSkYO3YsAGDJkiWoqKjAvHnzUFJSgsGDByM1NRX+/v5N/mxnc5lrwhIROS2rwvKFF17A8uXLrRos8+mnn6K6uhpPP/10g9d+9NFH9z2vUCiQmJiIxMREa8ps0TjHkojIeVkVlsHBwYiNjcWwYcPwy1/+EgMGDEB4eDi8vb1RUlKC7Oxs/PDDD9ixYwc6dOiADz/80N51tzocCUtE5LysCss///nPePHFF/HRRx/h/fffx+nTp83O+/v7Y8yYMfjb3/5mtogAWU+aY8nHsERETsfqPsuQkBAsW7YMy5YtQ2lpKXJzc1FRUYH27duja9eu9537SA2TWpZ8DEtE5HQaNcAnMDAQgYGBNi7FdVUbjCjQ1Cz2wMewRETOh3MUnECBRgeDUcDLww3BbVreHFEiotaOYekETCNhOwb5wM2Nj7OJiJwNw9IJsL+SiMi5MSydAEfCEhE5t0YN8AGAoqIinDt3DgqFAjExMQgJCbFlXS6FLUsiIucmu2Wp1WoxY8YMdOjQASNHjsSIESPQoUMHTJ8+HRqNxh41tnrS6j0cCUtE5JRkh+Xvfvc7/Pjjj/jqq69QWloKjUaDr776CseOHcPzzz9vjxpbvfw768J2DOJjWCIiZyT7MezXX3+NvXv3Yvjw4dKx8ePHY9OmTZgwYYJNi3MFuioDrpfV7L/Jx7BERM5JdsuyXbt2UKlUFsdVKlWjt+pyZZfv9Fe2UXog0NfTwdUQEVFdZIflihUrsHjxYhQUFEjHCgsL8Yc//AGvvfaaTYtzBaaRsB2DfLhkIBGRk5L9GHbjxo04f/48IiMj0alTJwBAXl4elEolrl+/jg8++EC6NiMjw3aVtlLcbYSIyPnJDsvHH3/cDmW4Lu5jSUTk/GSH5cqVK+1Rh8viggRERM6PK/g4GBckICJyfrJblm5ubvcdiGIwGJpUkKvhggRERM5Pdlju2rXL7PuqqipkZmZi69atWLVqlc0KcwWaiipoddUAuCABEZEzkx2Wjz32mMWx3/zmN+jZsyc+/fRTPPfcczYpzBWYWpXt/Lzgp2z0Mr1ERGRnNuuzHDx4ML755htbvZ1LMC1I0JGPYImInJpNwrKiogLvvPMOOnbsaIu3cxnSSFg+giUicmqyn/0FBQWZDfARQqCsrAy+vr7Ytm2bTYtr7bggARFRyyA7LN966y2zsHRzc0NwcDAGDx7MtWFl4oIEREQtg+ywnDVrlh3KcE2mrbm4IAERkXOzKixPnjxp9Rv27t270cW4EiGENMCHLUsiIudmVVj27dsXCoUCQggA4KIENnD9ViV0VUYoFEB4IFuWRETOzKrRsDk5Obh48SJycnKwc+dOREVF4b333kNmZiYyMzPx3nvvoWvXrvj888/tXW+rYRoJGxbgDS8PrjpIROTMrGpZRkZGSl9PnToVb7/9NiZOnCgd6927NyIiIvDaa69xVxIrcY4lEVHLIbtJc+rUKURFRVkcj4qKQnZ2tk2KcgWXTYN72F9JROT0ZIdl9+7dsXr1auh0OulYZWUlVq9eje7du9u0uNbs7gLq7K8kInJ2sqeOvP/++5g8eTIiIiLQp08fAMCJEyegUCjw1Vdf2bzA1opbcxERtRyyW5aDBg1CTk4O3njjDfTu3Ru9evVCUlIScnJyMGjQIFnvlZycjIEDB8Lf3x8hISF4/PHHce7cObNrhBBITExEeHg4fHx8EB8fjzNnzsgt2+nc3fSZYUlE5OwatdWFr68vfv/73zf5ww8cOID58+dj4MCBqK6uxvLlyzFu3DhkZ2fDz88PALBu3TqsX78eW7ZsQUxMDFavXo2xY8fi3Llz8Pf3b3INjmAwClwt5YIEREQtRaPmLPzjH//A8OHDER4ejtzcXAA1y+D93//9n6z3SUlJwaxZs9CzZ0/06dMHmzdvRl5eHo4fPw6gplW5YcMGLF++HFOmTEFsbCy2bt2K8vJybN++vTGlO4UCTQWqjQJe7m5Q+3s7uhwiImqA7LDcuHEjFi9ejEceeQQlJSXSIgRBQUHYsGFDk4rRaDQAgLZt2wKomd9ZWFiIcePGSdcolUqMHDkS6enpTfosRzI9gu0Q5AM3t/oXeCAiIucgOyzfeecdbNq0CcuXL4eHx92nuAMGDMCpU6caXYgQAosXL8bw4cMRGxsLACgsLAQAqNVqs2vVarV07l6VlZXQarVmL2djGtzTkVtzERG1CLLDMicnB3FxcRbHlUolbt++3ehCFixYgJMnT+KTTz6xOHfv8npCiHqX3EtOToZKpZJeERERja7JXi4Xc2suIqKWRHZYRkVFISsry+L4nj170KNHj0YV8eKLL+KLL77Avn37zDaQDg0NBQCLVmRRUZFFa9Nk2bJl0Gg00is/P79RNdlTPhckICJqUWSPhv3DH/6A+fPnQ6fTQQiBI0eO4JNPPkFycjL+9re/yXovIQRefPFF7Nq1C/v377dYGSgqKgqhoaFIS0uTWrN6vR4HDhzA2rVr63xPpVIJpVIp99dqVlyQgIioZZEdls8++yyqq6uxZMkSlJeXY9q0aejQoQP+3//7f3jyySdlvdf8+fOxfft2/N///R/8/f2lFqRKpYKPjw8UCgUSEhKQlJSE6OhoREdHIykpCb6+vpg2bZrc0p0GFyQgImpZFMK071Yj3LhxA0ajESEhIY378Hr6HTdv3ixtMi2EwKpVq/DBBx+gpKQEgwcPxrvvvisNAmqIVquFSqWCRqNBQEBAo+q0JV2VAd1eSwEAZLw2Fm39vBxcERGRa2hKHjRqUYLq6mrs378fFy5ckFp4V69eRUBAANq0aWP1+1iT0wqFAomJiUhMTGxMqU7nyp3FCPy83BHk6+ngaoiIyBqywzI3NxcTJkxAXl4eKisrMXbsWPj7+2PdunXQ6XR4//337VFnq5FfayTs/TbRJiIi5yF7NOzChQsxYMAAlJSUwMfn7gCVX/3qV/j2229tWlxrZBoJ25H9lURELYbsluUPP/yA//73v/DyMu9ri4yMxJUrV2xWWGtlmmPJBQmIiFoO2S1Lo9EoLXFX2+XLl1vswubNSRoJywUJiIhaDNlhOXbsWLM1YBUKBW7duoWVK1di4sSJtqytVZK25mLLkoioxZD9GPatt97CqFGj0KNHD+h0OkybNg0///wz2rdvX+dSdWSOLUsiopZHdliGh4cjKysLn3zyCTIyMmA0GvHcc8/h6aefNhvwQ5bKdFUoLa8CwLAkImpJGjXP0sfHB7Nnz8bs2bNtXU+rZnoEG+TriTbKRt16IiJygEb9iX3u3Dm88847OHv2LBQKBbp164YFCxagW7dutq6vVeEjWCKilkn2AJ9///vfiI2NxfHjx9GnTx/07t0bGRkZ6NWrFz777DN71NhqSAsScI4lEVGLIrtluWTJEixbtgyvv/662fGVK1di6dKlmDp1qs2Ka20umxYk4G4jREQtiuyWZWFhIZ555hmL49OnT7fYd5LMsWVJRNQyyQ7L+Ph4HDx40OL4Dz/8gF/84hc2Kaq1Yp8lEVHLJPsx7C9/+UssXboUx48fx5AhQwAAhw8fxmeffYZVq1bhiy++MLuWagghuCABEVELJXs/Szc36xqjCoWizmXxmpuz7Gd541YlBqz+BgoF8L8/T4DSw91htRARuaJm3c/SaDTK/RHC3f5Ktb83g5KIqIWR3WdJjWMaCRvBkbBERC2O1WH5448/Ys+ePWbHPv74Y0RFRSEkJAS///3vUVlZafMCWwtpcA9HwhIRtThWh2ViYiJOnjwpfX/q1Ck899xzGDNmDF599VV8+eWXSE5OtkuRrYFpcE9HjoQlImpxrA7LrKwsjB49Wvp+x44dGDx4MDZt2oTFixfj7bffxr/+9S+7FNkaXJZalnwMS0TU0lgdliUlJVCr1dL3Bw4cwIQJE6TvBw4ciPz8fNtW14pICxKwZUlE1OJYHZZqtRo5OTkAAL1ej4yMDAwdOlQ6X1ZWBk9PT9tX2AoYjAJXSk0DfBiWREQtjdVhOWHCBLz66qs4ePAgli1bBl9fX7MVe06ePImuXbvapciW7ppWhyqDgKe7AqEB3o4uh4iIZLJ6nuXq1asxZcoUjBw5Em3atMHWrVvh5eUlnf/73/+OcePG2aXIls70CDY80AfubgoHV0NERHJZHZbBwcE4ePAgNBoN2rRpA3d384n1n332Gdq0aWPzAluDfNMcS04bISJqkWSv4KNSqeo83rZt2yYX01rdHdzDkbBERC0RV/BpBqYFCTqyZUlE1CIxLJvB5WKOhCUiaskYls0gnwsSEBG1aAxLO6usNqBQqwPAliURUUvFsLSzq6U6CAH4eLqjnZ9Xwz9AREROh2FpZ7VHwioUnGNJRNQSMSztjFtzERG1fAxLO8vnSFgiohbPoWH5/fffY/LkyQgPD4dCocB//vMfs/NCCCQmJiI8PBw+Pj6Ij4/HmTNnHFNsIxiMAln5JQCAaqMRBqNwcEVERNQYDg3L27dvo0+fPvjrX/9a5/l169Zh/fr1+Otf/4qjR48iNDQUY8eORVlZWTNXKl/K6QIMX/sdDl8sBgBsO5yH4Wu/Q8rpAgdXRkREcimEEE7R3FEoFNi1axcef/xxADWtyvDwcCQkJGDp0qUAgMrKSqjVaqxduxZz5syx6n21Wi1UKhU0Gg0CAgLsVb6ZlNMFmLstA/feWNPwno3T+2FCbFiz1EJERDWakgdO22eZk5ODwsJCs51MlEolRo4cifT09Hp/rrKyElqt1uzVnAxGgVVfZlsEJQDp2Kovs/lIloioBXHasCwsLARQs+l0bWq1WjpXl+TkZKhUKukVERFh1zrvdSSnGAUaXb3nBYACjQ5HcoqbrygiImoSpw1Lk3vnJgoh7jtfcdmyZdBoNNIrPz/f3iWaKSqrPygbcx0RETme7C26mktoaCiAmhZmWNjd/r2ioiKL1mZtSqUSSqXS7vXVJ8Tf26bXERGR4zltyzIqKgqhoaFIS0uTjun1ehw4cADDhg1zYGX3NyiqLcJU3qiv7asAEKbyxqAo7v9JRNRSODQsb926haysLGRlZQGoGdSTlZWFvLw8KBQKJCQkICkpCbt27cLp06cxa9Ys+Pr6Ytq0aY4s+77c3RRYOblHnQN8TAG6cnIPuLtx6TsiopbCoY9hjx07hlGjRknfL168GAAwc+ZMbNmyBUuWLEFFRQXmzZuHkpISDB48GKmpqfD393dUyVaZEBuGoV3a4dDFm2bHQ1XeWDm5B6eNEBG1ME4zz9JeHDHPUgiBwUnfoqisEn+c2B3qACVC/GsevbJFSUTkGE3JA6cd4NOSnS0oQ1FZJXw83TFzWCSUHu6OLomIiJrAaQf4tGQHfroOABjWtR2DkoioFWBY2sH+c0UAgPgHgx1cCRER2QIfw1rBYBQ4klOMojJdg32PZboqHM+t2WlkZExIc5ZJRER2wrBsQMrpAqz6MttsCbuw+4xq/e/5m6g2CnRp74dO7biHJRFRa8DHsPdh2j3k3rVeCzU6zN2WUed2W6b+yhExfARLRNRaMCzr0ZjdQ4QQOMD+SiKiVodhWY/G7B5yvugWrmp0UHq4YUiXds1QJRERNQeGZT0as3vI/nM1j2CHdGkHb09OGSEiai0YlvVozO4hpv7KkeyvJCJqVRiW9ZC7e8jtymrpkSz7K4mIWheGZT1Mu4cAsAjMunYPOXzxJvQGIyLa+iCqvV/zFUpERHbHsLyPCbFh2Di9H0JV5o9kQ1Xe2Di9n9k8S1N/ZXxMCBQKLpZORNSaMCwbMCE2DD8sfRgjYtoDAKb274gflj5sFpRCCOz/qWbKCPsriYhaH4alFdzdFBjapSYs9QajxVJ3OTduI7+4Al7ubhjalVNGiIhaG4allboG1/RDXrh+y+KcaRTswKgg+Cm5giARUWvDsLRS15A2AICL12/j3v2yTf2VfARLRNQ6MSyt1KmtLzzcFCjXG1CovbsQga7KgMMXbwIA4h/kLiNERK0Rw9JKnu5uiLyzi8iFotvS8cMXb6Ky2ogwlTei77Q+iYiodWFYytA1uCYMa/dbmvor4x8M5pQRIqJWimEpg6nf0iws2V9JRNTqMSxluLdlmXezHBdv3IaHmwLDHmjvyNKIiMiOGJYydDFNH7nTZ3ngzkIE/SKDEODt6bC6iIjIvhiWMnRtX9OyLNTqcKuy2qy/koiIWi+GpQwqX0+0b6MEAPyvQIv0CzVTRthfSUTUujEsZTKt5LPjaD7K9QYE+yvRIyzAwVUREZE9MSxliroTlrsyrgAARkS355QRIqJWjmEpQ8rpAnx9sgAAYLiz5N23/ytCyukCR5ZFRER2xrC0UsrpAszdloEyXbXZcU15FeZuy2BgEhG1YgxLKxiMAqu+zIao45zp2Kovs2Ew1nUFERG1dAxLKxzJKUaBRlfveQGgQKPDkZzi5iuKiIiaDcPSCkVl9QdlY64jIqKWhWFphRB/b5teR0RELUuLCMv33nsPUVFR8Pb2Rv/+/XHw4MFm/fxBUW0RpvJGfRNEFADCVN4YFNW2OcsiIqJm4vRh+emnnyIhIQHLly9HZmYmfvGLX+CRRx5BXl5es9Xg7qbAysk9AMAiME3fr5zcA+5unG9JRNQaKYQQTj2Ec/DgwejXrx82btwoHevevTsef/xxJCcnN/jzWq0WKpUKGo0GAQFNW2kn5XQBVn2ZbTbYJ0zljZWTe2BCbFiT3puIiOyrKXngYaeabEKv1+P48eN49dVXzY6PGzcO6enpdf5MZWUlKisrpe+1Wq3N6pkQG4axPUJxJKcYRWU6hPjXPHpli5KIqHVz6rC8ceMGDAYD1Gq12XG1Wo3CwsI6fyY5ORmrVq2yW03ubgoM7drObu9PRETOx+n7LAFYrL0qhKh3PdZly5ZBo9FIr/z8/OYokYiIWjGnblm2b98e7u7uFq3IoqIii9amiVKphFKpbI7yiIjIRTh1y9LLywv9+/dHWlqa2fG0tDQMGzbMQVUREZGrceqWJQAsXrwYM2bMwIABAzB06FB8+OGHyMvLwwsvvODo0oiIyEU4fVg+8cQTuHnzJl5//XUUFBQgNjYWu3fvRmRkpKNLIyIiF+H08yybypbzLImIqOVqSh44dZ8lERGRM2BYEhERNYBhSURE1ACGJRERUQOcfjRsU5nGL9lyjVgiImp5TDnQmHGtrT4sy8rKAAAREREOroSIiJxBWVkZVCqVrJ9p9VNHjEYjrl69Cn9//3rXk62LVqtFREQE8vPzOeXkHrw39eO9uT/en/rx3tTPVvdGCIGysjKEh4fDzU1eL2Srb1m6ubmhY8eOjf75gIAA/odbD96b+vHe3B/vT/14b+pni3sjt0VpwgE+REREDWBYEhERNYBhWQ+lUomVK1dyu6868N7Uj/fm/nh/6sd7Uz9nuDetfoAPERFRU7FlSURE1ACGJRERUQMYlkRERA1gWBIRETWAYVmP9957D1FRUfD29kb//v1x8OBBR5dkM8nJyRg4cCD8/f0REhKCxx9/HOfOnTO7RgiBxMREhIeHw8fHB/Hx8Thz5ozZNZWVlXjxxRfRvn17+Pn54Ze//CUuX75sdk1JSQlmzJgBlUoFlUqFGTNmoLS01N6/os0kJydDoVAgISFBOubq9+bKlSuYPn062rVrB19fX/Tt2xfHjx+Xzrvq/amursaKFSsQFRUFHx8fdOnSBa+//jqMRqN0javcm++//x6TJ09GeHg4FAoF/vOf/5idb877kJeXh8mTJ8PPzw/t27fHSy+9BL1eL/+XEmRhx44dwtPTU2zatElkZ2eLhQsXCj8/P5Gbm+vo0mxi/PjxYvPmzeL06dMiKytLTJo0SXTq1EncunVLumbNmjXC399ffP755+LUqVPiiSeeEGFhYUKr1UrXvPDCC6JDhw4iLS1NZGRkiFGjRok+ffqI6upq6ZoJEyaI2NhYkZ6eLtLT00VsbKx49NFHm/X3bawjR46Izp07i969e4uFCxdKx1353hQXF4vIyEgxa9Ys8eOPP4qcnBzxzTffiPPnz0vXuOr9Wb16tWjXrp346quvRE5Ojvjss89EmzZtxIYNG6RrXOXe7N69Wyxfvlx8/vnnAoDYtWuX2fnmug/V1dUiNjZWjBo1SmRkZIi0tDQRHh4uFixYIPt3YljWYdCgQeKFF14wO9atWzfx6quvOqgi+yoqKhIAxIEDB4QQQhiNRhEaGirWrFkjXaPT6YRKpRLvv/++EEKI0tJS4enpKXbs2CFdc+XKFeHm5iZSUlKEEEJkZ2cLAOLw4cPSNYcOHRIAxP/+97/m+NUaraysTERHR4u0tDQxcuRIKSxd/d4sXbpUDB8+vN7zrnx/Jk2aJGbPnm12bMqUKWL69OlCCNe9N/eGZXPeh927dws3Nzdx5coV6ZpPPvlEKJVKodFoZP0efAx7D71ej+PHj2PcuHFmx8eNG4f09HQHVWVfGo0GANC2bVsAQE5ODgoLC83ugVKpxMiRI6V7cPz4cVRVVZldEx4ejtjYWOmaQ4cOQaVSYfDgwdI1Q4YMgUqlcvp7OX/+fEyaNAljxowxO+7q9+aLL77AgAEDMHXqVISEhCAuLg6bNm2Szrvy/Rk+fDi+/fZb/PTTTwCAEydO4IcffsDEiRMBuPa9qa0578OhQ4cQGxuL8PBw6Zrx48ejsrLSrOvAGq1+IXW5bty4AYPBALVabXZcrVajsLDQQVXZjxACixcvxvDhwxEbGwsA0u9Z1z3Izc2VrvHy8kJQUJDFNaafLywsREhIiMVnhoSEOPW93LFjBzIyMnD06FGLc65+by5evIiNGzdi8eLF+OMf/4gjR47gpZdeglKpxDPPPOPS92fp0qXQaDTo1q0b3N3dYTAY8MYbb+Cpp54CwP92TJrzPhQWFlp8TlBQELy8vGTfK4ZlPe7dzksIIWuLr5ZiwYIFOHnyJH744QeLc425B/deU9f1znwv8/PzsXDhQqSmpsLb27ve61zx3gA1W94NGDAASUlJAIC4uDicOXMGGzduxDPPPCNd54r359NPP8W2bduwfft29OzZE1lZWUhISEB4eDhmzpwpXeeK96YuzXUfbHWv+Bj2Hu3bt4e7u7vF3zqKioos/obS0r344ov44osvsG/fPrNtzEJDQwHgvvcgNDQUer0eJSUl973m2rVrFp97/fp1p72Xx48fR1FREfr37w8PDw94eHjgwIEDePvtt+Hh4SHV7Yr3BgDCwsLQo0cPs2Pdu3dHXl4eANf+b+cPf/gDXn31VTz55JPo1asXZsyYgUWLFiE5ORmAa9+b2przPoSGhlp8TklJCaqqqmTfK4blPby8vNC/f3+kpaWZHU9LS8OwYcMcVJVtCSGwYMEC7Ny5E9999x2ioqLMzkdFRSE0NNTsHuj1ehw4cEC6B/3794enp6fZNQUFBTh9+rR0zdChQ6HRaHDkyBHpmh9//BEajcZp7+Xo0aNx6tQpZGVlSa8BAwbg6aefRlZWFrp06eKy9wYAHnroIYtpRj/99BMiIyMBuPZ/O+Xl5RYbCru7u0tTR1z53tTWnPdh6NChOH36NAoKCqRrUlNToVQq0b9/f3mFyxoO5CJMU0c++ugjkZ2dLRISEoSfn5+4dOmSo0uziblz5wqVSiX2798vCgoKpFd5ebl0zZo1a4RKpRI7d+4Up06dEk899VSdQ7s7duwovvnmG5GRkSEefvjhOod29+7dWxw6dEgcOnRI9OrVy6mGuFuj9mhYIVz73hw5ckR4eHiIN954Q/z888/in//8p/D19RXbtm2TrnHV+zNz5kzRoUMHaerIzp07Rfv27cWSJUuka1zl3pSVlYnMzEyRmZkpAIj169eLzMxMafpdc90H09SR0aNHi4yMDPHNN9+Ijh07cuqILb377rsiMjJSeHl5iX79+knTKloDAHW+Nm/eLF1jNBrFypUrRWhoqFAqlWLEiBHi1KlTZu9TUVEhFixYINq2bSt8fHzEo48+KvLy8syuuXnzpnj66aeFv7+/8Pf3F08//bQoKSlpht/Sdu4NS1e/N19++aWIjY0VSqVSdOvWTXz44Ydm5131/mi1WrFw4ULRqVMn4e3tLbp06SKWL18uKisrpWtc5d7s27evzj9jZs6cKYRo3vuQm5srJk2aJHx8fETbtm3FggULhE6nk/07cYsuIiKiBrDPkoiIqAEMSyIiogYwLImIiBrAsCQiImoAw5KIiKgBDEsiIqIGMCyJiIgawLAkaoU6d+6MDRs22OW99+/fD4VCYbEjPVFrxrAksoNZs2ZBoVDghRdesDg3b948KBQKzJo1y+r3u3TpEhQKBbKysqy6/ujRo/j9739v9fvLMWzYMBQUFEClUtnl/YmcEcOSyE4iIiKwY8cOVFRUSMd0Oh0++eQTdOrUyS6fqdfrAQDBwcHw9fW1y2d4eXkhNDS0RW0HRdRUDEsiO+nXrx86deqEnTt3Ssd27tyJiIgIxMXFmV2bkpKC4cOHIzAwEO3atcOjjz6KCxcuSOdNO8PExcVBoVAgPj4eQE0L9vHHH0dycjLCw8MRExMDwPwx7P79++Hl5YWDBw9K7/fmm2+iffv2Zrsx1Jabm4vJkycjKCgIfn5+6NmzJ3bv3i29X+3HsPHx8VAoFBavS5cuAQA0Gg1+//vfIyQkBAEBAXj44Ydx4sSJxt1UIgdhWBLZ0bPPPovNmzdL3//973/H7NmzLa67ffs2Fi9ejKNHj+Lbb7+Fm5sbfvWrX0nbO5m2Ifrmm29QUFBgFsDffvstzp49i7S0NHz11VcW7x0fH4+EhATMmDEDGo0GJ06cwPLly7Fp0yaEhYXVWff8+fNRWVmJ77//HqdOncLatWvRpk2bOq/duXMnCgoKpNeUKVPw4IMPQq1WQwiBSZMmobCwELt378bx48fRr18/jB49GsXFxdbfSCJHk730OhE1aObMmeKxxx4T169fF0qlUuTk5IhLly4Jb29vcf36dfHYY49JOzDUpaioSACQdmLIyckRAERmZqbF56jVarOdLYQQIjIyUrz11lvS95WVlSIuLk789re/FT179hS/+93v7lt/r169RGJiYp3nTDtK1LXLxfr160VgYKA4d+6cEEKIb7/9VgQEBFjs8tC1a1fxwQcf3LcGImfi4eCsJmrV2rdvj0mTJmHr1q1SK6t9+/YW1124cAGvvfYaDh8+jBs3bkgtyry8PMTGxt73M3r16gUvL6/7XuPl5YVt27ahd+/eiIyMbHCk7EsvvYS5c+ciNTUVY8aMwa9//Wv07t37vj+zZ88evPrqq/jyyy+lx8HHjx/HrVu30K5dO7NrKyoqzB4zEzk7hiWRnc2ePRsLFiwAALz77rt1XjN58mRERERg06ZNCA8Ph9FoRGxsrDRg5378/PysqiM9PR0AUFxcjOLi4vv+3O9+9zuMHz8eX3/9NVJTU5GcnIw333wTL774Yp3XZ2dn48knn8SaNWswbtw46bjRaERYWBj2799v8TOBgYFW1U3kDNhnSWRnEyZMgF6vh16vx/jx4y3O37x5E2fPnsWKFSswevRodO/eHSUlJWbXmFqOBoOhUTVcuHABixYtwqZNmzBkyBA888wzUuu1PhEREXjhhRewc+dOvPzyy9i0aVOd1928eROTJ0/GlClTsGjRIrNz/fr1Q2FhITw8PPDAAw+YvepqYRM5K4YlkZ25u7vj7NmzOHv2LNzd3S3OBwUFoV27dvjwww9x/vx5fPfdd1i8eLHZNSEhIfDx8UFKSgquXbsGjUZj9ecbDAbMmDED48aNkwYcnT59Gm+++Wa9P5OQkIC9e/ciJycHGRkZ+O6779C9e/c6r50yZQp8fHyQmJiIwsJC6WUwGDBmzBgMHToUjz/+OPbu3YtLly4hPT0dK1aswLFjx6z+HYgcjWFJ1AwCAgIQEBBQ5zk3Nzfs2LEDx48fR2xsLBYtWoS//OUvZtd4eHjg7bffxgcffIDw8HA89thjVn/2G2+8gUuXLuHDDz8EAISGhuJvf/sbVqxYUe8iBwaDAfPnz0f37t0xYcIEPPjgg3jvvffqvPb777/HmTNn0LlzZ4SFhUmv/Px8KBQK7N69GyNGjMDs2bMRExODJ598EpcuXYJarbb6dyByNIUQQji6CCIiImfGliUREVEDGJZEREQNYFgSERE1gGFJRETUAIYlERFRAxiWREREDWBYEhERNYBhSURE1ACGJRERUQMYlkRERA1gWBIRETWAYUlERNSA/w/GRxIzaI49hgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "\n", "plt.figure(figsize=(5, 5))\n", "plt.plot(sizes, speed_ups, marker=\"o\")\n", "plt.xlabel(\"Matrix size\")\n", "plt.ylabel(\"Speedup (CuPy time / NumPy time)\")\n", "# plt.xticks(sizes)\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "93da5acf-ba39-4617-8246-ac7f7b3fd8df", "metadata": {}, "source": [ "```{note}\n", "As we can see above, GPUs computations can be slower than CPUs!\n", "```\n", "\n", "There are several reasons for this: \n", " \n", "* The size of our arrays: The GPU's performance relies on parallelism, processing thousands of values simultaneously. To fully leverage the GPU's capabilities, we require a significantly larger array. As we see in the above example, for bigger matrix size we see more speed-ups. \n", "\n", "* The simplicity of our calculation: Transferring a calculation to the GPU involves considerable overhead compared to executing a function on the CPU. If our calculation lacks a sufficient number of mathematical operations (known as \"arithmetic intensity\"), the GPU will spend most of its time waiting for data movement.\n", "\n", "* Data copying to and from the GPU impacts performance: While including copy time can be realistic for a single function, there are instances where we need to execute multiple GPU operations sequentially. In such cases, it is advantageous to transfer data to the GPU and keep it there until all processing is complete." ] }, { "cell_type": "markdown", "id": "1fc57cc2-d237-49e9-bf4f-ef74511673d7", "metadata": {}, "source": [ "Congratulations! You have now uncovered the capabilities of CuPy. It's time to unleash its power and accelerate your own code by replacing NumPy with CuPy wherever applicable and appropriate. In the next chapters we will delve into Cupy Xarray capabilities. \n", "\n", "## Summary\n", "\n", "In this notebook, we have learned about:\n", "\n", "* Cupy Basics\n", "* Data Transfer between Device and Host\n", "* Performance of Cupy vs. Numpy on different array sizes. \n", "\n", "```{seealso}\n", "\n", "[CuPy Homepage](https://cupy.dev/) \n", "[CuPy Github](https://github.com/cupy/cupy) \n", "[CuPy User Guide](https://docs.cupy.dev/en/stable/user_guide/index.html)\n", "```" ] } ], "metadata": { "kernelspec": { "display_name": "Python [conda env:gpu-xdev]", "language": "python", "name": "conda-env-gpu-xdev-py" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.15" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": {}, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 5 }