{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Transformer Model\n", "In this notebook, we show an example of how Transformer can be used with darts.\n", "If you are new to darts, we recommend you first follow the [quick start](https://unit8co.github.io/darts/quickstart/00-quickstart.html) notebook." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# fix python path if working locally\n", "from utils import fix_pythonpath_if_working_locally\n", "\n", "fix_pythonpath_if_working_locally()" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "%load_ext autoreload\n", "%autoreload 2\n", "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "tags": [] }, "outputs": [], "source": [ "import warnings\n", "\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import pandas as pd\n", "\n", "from darts.dataprocessing.transformers import Scaler\n", "from darts.datasets import AirPassengersDataset, SunspotsDataset\n", "from darts.metrics import mape\n", "from darts.models import ExponentialSmoothing, TransformerModel\n", "from darts.utils.statistics import check_seasonality\n", "\n", "warnings.filterwarnings(\"ignore\")\n", "import logging\n", "\n", "logging.disable(logging.CRITICAL)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Air Passengers Example" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, we will test the performance of the transformer architecture on the 'air passengers' dataset." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "# Read data:\n", "series = AirPassengersDataset().load().astype(np.float32)\n", "\n", "# Create training and validation sets:\n", "train, val = series.split_after(pd.Timestamp(\"19590101\"))\n", "\n", "# Normalize the time series (note: we avoid fitting the transformer on the validation set)\n", "# Change name\n", "scaler = Scaler()\n", "train_scaled = scaler.fit_transform(train)\n", "val_scaled = scaler.transform(val)\n", "series_scaled = scaler.transform(series)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"the 'air passengers' dataset has 144 data points\"" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "f\"the 'air passengers' dataset has {len(series)} data points\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We train a standard transformer architecture with default hyperparameters, tweaking only two of them:\n", "\n", "* _d\\_model_, the input dimensionality of the transformer architecture (*after* performing time series embedding). Its default value is 512. We lower the value from 512 to 64, since it is hard to learn such an high-dimensional representation from an univariate time series\n", "* _nhead_, the number of heads in the multi-head attention mechanism. We increase the value from 8 to 32. This means that we compute multi-head attention with 32 heads of size _d\\_model_/_nhead_=64/32=2 each. This way, we obtain low-dimensional heads that are hopefully suitable to learn from univariate time series\n", "\n", "The goal is to perform one-step forecasting." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "scrolled": true }, "outputs": [], "source": [ "my_model = TransformerModel(\n", " input_chunk_length=12,\n", " output_chunk_length=1,\n", " batch_size=32,\n", " n_epochs=200,\n", " model_name=\"air_transformer\",\n", " nr_epochs_val_period=10,\n", " d_model=16,\n", " nhead=8,\n", " num_encoder_layers=2,\n", " num_decoder_layers=2,\n", " dim_feedforward=128,\n", " dropout=0.1,\n", " activation=\"relu\",\n", " random_state=42,\n", " save_checkpoints=True,\n", " force_reset=True,\n", ")" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "scrolled": false, "tags": [] }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "2f9a2a1d3b8c4b33a99279fb2373d78b", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/200 [00:00" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# this function evaluates a model on a given validation set for n time-steps\n", "def eval_model(model, n, series, val_series):\n", " pred_series = model.predict(n=n)\n", " plt.figure(figsize=(8, 5))\n", " series.plot(label=\"actual\")\n", " pred_series.plot(label=\"forecast\")\n", " plt.title(f\"MAPE: {mape(pred_series, val_series):.2f}%\")\n", " plt.legend()\n", "\n", "\n", "eval_model(my_model, 26, series_scaled, val_scaled)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then, by using the best model obtained over training, according to validation loss:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "tags": [] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAAFVCAYAAADYEVdtAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABhp0lEQVR4nO3dd3wUdf7H8dek9wIkhITeqwEZkI4KiAUVsWIXFdRDzzvL+TvPw9PTE+88u6d4elb0rFhPKYJ0dOgKoUMghJBGCmmb3fn9MTubTd9NtqR8no+HD5KZzcx3vwTf+y3z/Sq6riOEEEII3wvwdwGEEEKI9kpCWAghhPATCWEhhBDCTySEhRBCCD+REBZCCCH8REJYCCGE8BMJYSGEEMJPJISF8CBFUQ4rilKhKEqnGse3KoqiK4rSs8bxR+3Hz6px/GZFUayKohQrilKoKMo2RVFm2M+drSiKzX7O+b+xLpZxkKIoPyiKUqAoyn5FUS5z8edW2MsaVMe5yfZzf3U6NkVRlEOKopxQFOUap+NxiqJsURQl2pX7CtGWSQgL4XmHgNnmN4qiDAMiar5IURQFuBHIs/9Z0wZd16OAOOAN4CNFUeLt547ruh5V478NjRXMHqBfAF8DHYC5wHuKovRv5OeuA4LrORcMPA9sqnHqOeBiYDrwiqIogfbjfwOe0nW9qLHyCtHWSQgL4XnvUj1UbwLeqeN1E4EuwD3ANYqihNR1MV3XbcCbQDjQp5llGwgkA8/qum7Vdf0HYB1wQ30/oChKLLAAeLCel9wHLAXSahyP1HX9F13XtwMVQEdFUUYDvXRd/6iZ70OINkFCWAjP2wjE2Lt9A4FrgPfqeN1NwFeAGUgX13Uxe+v1NqAY2NfYzRVFeUhRlK/dKK8CDG3g/JPAv4ATddyrBzAHeKyOnzupKEqqoiipgA3Ix2gx3+NG2YRo0ySEhfAOszU8DdgNZDifVBQlArgSWKzrugX4hNpd0mMURTmFEX6zgct0XS+wn0tWFOVUjf8iAXRdf0rX9Rn1lGsPcBJ4QFGUYEVRzgMmU0d3ub2cKjAeeLGe670APKLrenEd5+7ACN1FGC3tO4HlQJiiKN8rirJSUZTJ9VxXiHah1gQLIYRHvAusBnpRd1f0ZUAl8K39+/eB5YqiJOi6nm0/tlHX9Qn1XP+4rutd3S2UrusWRVFmYoTqHwANoyVeXvO1iqIEAK8Av9V1vdIYwq52/mIgWtf1/9Zzr23A2fbXdgGeAcYCPwL3AseB1Yqi9NBlJxnRTkkIC+EFuq4fURTlEHAhcGsdL7kJiALS7eGmYEx8uhaj9ejNsu3AaP0CoCjKeuDtOl4aA6jAf+1lNCdWHVMU5UpgCqAqimJ2U8cCVkVRhum6fmmNaz0L/EnX9VL7RDVN1/UK+6SuBIzWuRDtjoSwEN5zKxCv6/pp58d6FEVJwQiwC4AdTq+/F6NL2qshrCjKGcBejOGouzAmh71Vx0sLMCZxmboBPwEjgWxgG/CU0/nnMVq3j9e43zQgTNd1c5z6EHCuoihHgVAgt1lvSIhWTEJYCC/Rdf1APaduALbpur7U+aCiKC8A9ymK0tAkKVOyoig1x2Fv0nX9U0VR/ghM1HX9ggbufxtGy3sNME3X9XJ7GboDu4DBuq6n4zQZS1GUMPuXWbquV2LMeC5yOl8KnNZ1Pc/pWCjwd8C5ZXw3xiNXocBduq5bXXi/QrRJigzFCCGEEP4hs6OFEEIIP5EQFkIIIfxEQlgIIYTwEwlhIYQQwk8khIUQQgg/8fUjSm1yKvaJEydISkrydzFaBKmLKlIXBqmHKlIXVdpZXSj1nZCWsAdYrfKYo0nqoorUhUHqoYrURRWpC4OEsBBCCOEnEsJCCCGEn0gICyGEEH4iISyEEEL4iYSwEEII4ScSwkIIIYSfSAgLIYQQfiIh7KZVq1axfv36Zl0jKirKQ6URQgjRmkkIu8kTISyEEEKAhLDDzJkzGTlyJEOGDGHRokUAfPfdd5x55pmkpqYyZcoUDh8+zKuvvsqzzz7L8OHDWbNmDTfffDNff/214zpmK7e4uJgpU6Zw5plnMmzYML744gu/vC8hhGhr3n77bXr16sUvv/zi76I0m6/Xjm6QotS7vGaz6HrjS1a/+eabdOjQgdLSUkaNGsWll17K7bffzurVq+nVqxd5eXl06NCBO+64g6ioKO6//34A3njjjTqvFxYWxueff05MTAw5OTmMGTOGSy65xGvvUQgh2oslS5Zw+PBhnn322Xr/H9xaSEvY7oUXXiA1NZUxY8Zw9OhRFi1axKRJk+jVqxcAHTp0cOt6uq7zxz/+kTPOOIOpU6eSkZFBVlaWN4ouhBDtyqlTpwD4+OOPKSkp8W9hmqlFhbCu6175rzGrVq1i+fLlbNiwge3btzNixAiGDx/uUpmDgoIc97DZbFRUVADw/vvvk52dzebNm9m2bRudO3emrKysyXUjhBDCkJ+fD0BRURGff/65n0vTPC0qhP2loKCA+Ph4IiIiSEtLY+PGjZSVlbF69WoOHToEQF5eHgDR0dEUFRU5frZnz57s2LEDgC+//BKLxeK4ZmJiIsHBwaxcuZIjR474+F0JIUTbZIYwGOPDrZmEMHD++edTWVnJoEGDeOihhxgzZgwJCQksWrSIWbNmkZqaytVXXw3AxRdfzOeff+6YmHX77bezceNGUlNT2bBhA5GRkQBcd911aJrGsGHDeOeddxg4cKA/36IQQrQZZggHBgayfPlyjh075ucSNZ3iSnetB/n0Zr6SkZFBSkqKv4vRIkhdVJG6MEg9VJG6qNLUuqisrCQ4OBhFUbjsssv47LPP+Nvf/sZDDz3khVJ6TL0zcqUlLIQQotUwJ2XFxcVxyy23AEaXtI8blB4jISyEEKLVMLui4+PjmT59OgkJCaSlpbFnzx4/l6xpJISFEEK0Gs4hHBwczIABAwA4efKkP4vVZBLCQgghWg3nEAajWxqquqlbGwlhIYQQrYYZtmYIx8bGAsZjoa2RhLAQQohWQ1rCbdQLL7zAoEGDuO666/xdFJYsWcKuXbv8XQwhhGhxzBA2w9f8U1rCrdwrr7zCsmXLeP/99xt9bWVlpVfLIiEshBB1q9kSNrujpSXcit1xxx0cPHiQCy64gGeeeYaZM2dyxhlnMGbMGMeSlI8++ig33HAD48eP54YbbiA7O5vLL7+cUaNGcdFFF7Fu3TrA2MLwlltuYdiwYZxxxhl8+umnANx5552oqsqQIUNYsGCB494PPfQQgwcP5owzzuD+++9n/fr1fPnllzzwwAMMHz6cAwcO+L5ChBCihaqvO7q1toQb3cpQVdVYYBkwGBijadovTucuBv4EWIDNmqb9tjmFUSbZmvPj9dJXN/xZ49VXX+W7775j5cqV/OUvf2HEiBEsWbKEH374gRtvvJFt27YBsGvXLtauXUt4eDjXXnstv/vd75gwYQI//fQTN910E7t37+bxxx8nNjaWnTt3AlW/ME888QQdOnTAarUyZcoUduzYQUpKCp9//jlpaWkoisKpU6eIi4vjkksuYcaMGVxxxRVeqQ8hhGit2lpL2JX9hEuAi4C/13FuOzBe07RKVVU/UFVV1TRN82gJfWzt2rWO1uu5555Lbm4uhYWFAFxyySWEh4cDsHz5ckeXscViobCwkOLiYpYvX86HH37ouJ75i/LRRx+xaNEiKisryczMZNeuXQwePJiwsDBuvfVWZsyYwYwZM3z5VoUQotVpdy1hTdMsQLaqqnWdS3f6tgJoVlO2sRarv5mbM4CxbeHGjRsJCwtrdA3UQ4cO8Y9//IOff/6Z+Ph4br75ZsrKyggKCuKnn35ixYoVfPLJJ7z00kv88MMPvngrQgjRKrXHlnCjVFUdBSRqmraljnNzgbkA8+fPZ9q0aZ64pcdZrVYyMzMZMWIE//rXv7j33ntZv349sbGxFBUVUVhYiNVqJSMjA4AJEybw17/+lTvvvBOLxcLSpUsZMmQIY8aMYeHChfzlL38BjF+M48ePExISQnFxMUeOHOGbb74hNTWVvXv3UlpaSmpqKr169WLcuHFkZGSgKArp6emOe7UmFoulVZbbG6QuDFIPVaQuqjS1LnJycgAoLy8nIyOD8vJyAHJzc1ts3TbUSGt2CKuq2hV4DrisrvOapi0CFtm/bbErbAcGBtKlSxf+8Y9/MGfOHC644AIiIiJYvHgxKSkpxMTEEBUV5ajMf//73/zmN7/hggsuoKysjHPPPZfzzjuPhQsX8pvf/Ibp06cTGBjIggULmDVrFqNHj2bKlCl069aNiRMnEh8fT3R0NNdffz1lZWXous5zzz1HSkoKt912G7fffjvvvvsun3zyCX369PFz7bhOdompInVhkHqoInVRpal1YQ4PDho0iA4dOhAYGAgYk2JbY926vJWhqqpvAf+oMTErGvgfcIfz8Qa02BBuDvmHVUXqoorUhUHqoYrURZWm1IXNZiMoKAhd16msrCQwMJCysjLCw8MJDQ2lrKzMS6VttuZtZaiq6rfAecDrqqrerKrqa/ZT9wK9gJdUVV2lqurk5pZUCCGEqEtBQQG6rhMbG+toAYeFhRESEkJ5eXlLDuF6udQdrWnahTUOvWU//jjwuIfLJIQQQtRSc7UsU1xcHCdPnqSgoICwsDA/lKzpWvZ0ZCGEEMKu5sxoU2ueIS0hLIQQolWoL4Rb87PCEsJCCCFaBWkJCyGEEH7SWEtYQlgIIYTwksZawtIdLYQQQniJtISFEEII4NixY6iqyvPPP++ze0pLWAghhAAefvhhNm/ezAcffOCze5otXWkJCyGEaLd27NjBu+++C/g2+KQlLIQQot37v//7P8x9B3wZfDImLIQQol1btWoV3377rWN/dX+0hGsuWyktYSGEEG2erus89NBDADz44IOEhIRQVlbms40TpCUshBCi3crKymLTpk1ER0fz+9//3qctUJvN5ghZaQkLIYRod7KzswHo1q0bUVFRPm2BFhUVYbPZiIqKIjg4uNo5aQkLIYRo83JzcwHo2LEj4Nvwq68rGiA6OhqoCurWREJYCCGES8wQ7tSpE+Db3YsaCuHAwEBiYmLQdZ3CwkKvl8WTJISFEEK4JCcnB2h5LWFovePCEsJCCCFc4s/u6PpWyzK11nFhCWEhhBAuqdkd7ct9fPPy8gBpCQshhGin/NkdfeLECQCSkpLqPC8tYSGEEG2aP7ujGwthaQkLIYRo08yWsD9mR0tLWAghRLsmLWHPkxAWQgjhkpYcwtISFkII0WZVVlaSn5+PoiiOGcq+nB0tLWEhhBDtlvNiGYGBgYDvWp/FxcWcPn2a8PBwxxKVNUlLWAghRJtV8xlh8F3wZWZmAkYrWFGUOl8jLWEhhBBtVs1nhAEiIyMJDAykpKQEi8XitXubXdFdunQBQEvTeeo9nbU7dGw2HWi9LeEgfxdACCFEy2e2hG0xU1ixWWfKSAVFUYiLiyM3N5eCgoJqrWRPMkM4oXN3HnjFxj8/AnOzpK4JMPcSmDXKd+PTniQtYSGEEI3KycmB0O78VLKAqb/TuWqBjZP5uk9aoCdOnICgjvxY+E/+8aFx7MpzoFsiHMuGP7+hs3yHMWGrtXVHN9oSVlU1FlgGDAbGaJr2i9O5QOB1oB+wWdO0e71UTiGEEH6Um5sL8Reg29tuH6+EH7bodIg/Bzjg/RBOuJpTFZ3p1xXef0Rh1CAFm03nX0tg/nM6//g4GpTQNtkSLgEuAj6p49wM4LimaROBSFVVx3qycEIIIWrTdZ3s7Gyf3jM3Nxc6nA/An2+GKSMhtwCyIu4FfNASDusBwC0XGgEMEBCgcMelMKw3HMtWCOp+DxUVFRQVFXmtLJ7WaAhrmmbRNK2+v+1xwFL7198B4z1VMCGEEHX761//SmJiIitXrvTZPbNOnoK4cwGYe7HCBwuMICzBCEevh3CocZ8enaufCwxU+Ns8oyy2rg9BYAzp6eleK4unNXdiVjxQaP+6AOhQ8wWqqs4F5gLMnz+fadOmNfOWLY/FYiEjI8PfxWgRpC6qSF0YpB6qeKIuSkpKeOaZZwBYuXIl/fv390TRGvXrsTgIjCIlLg8qyqmogPiozuQXR0BICocPH3brvblTF0ePHoXQ7gBEBGaTkVF9JnZqNzhrYAc2pcVB1/vYvHmzY6y6JUhJSan3XHND+BQQY/86Fsir+QJN0xYBi+zf6s28X4uUkZHRYCW3J1IXVaQuDFIPVTxRF4sWLao2+chXdZtZlgoKTBp6mpQUo1U6pLeNtTuAiEEoiuJWWdypi5ycHOhuhPDIIQmkdK79rPCz9+iMu0uH5N9SWPRhq/mda+7s6PXAVPvX04F1zbyeEEKIeui6zosvvuj43pczgbOtKgDTRtkcxwb1sH8RMdBr3dFWq5Ws7AII6UJQoE5yPU9BjR2qEBNaCEHR/LK3VnuwxXIphFVV/RY4D3hdVdWbVVV9zX7qa6C7qqprgDJN0zZ4qZxCCNHurV69ml9+cTyg4rMQTs/SqQjqC5WFTB0V7jg+qIe9RRoxyGshnJubiy0oGYCuCQqBgXWvmAWQFFcGwN70Cq+UxRtc6o7WNO3CGofesh+vBG72bJGEEELUxWwFDxs2jJ07d/oshL/ZYB9JPLWCzokXO447WsLhAzl1yjttMOeZ0d07N/za7kkKe7MgPav+oG5pZLEOIYRoBY4ePcqSJUsICgriwQcfBHzXEv56nTERKqx0FSEhIY7jvuiONmZGG+PBPereQMlhQPdQALIKQr1SFm+QEBZCiFbgiy++wGq1MnPmTAYOHAj4LoR3HTZawh2D91Y73i0RwoKtEJJEzqlKr9y7Wgg30hIe0i8KgGJLJ8rKyrxSHk+TEBZCiFbg6NGjAIwYMcKn+/gC5BcZ3bsJcdUfcAkIUOiVZIy/niz24rrR9u7oHkkNdzP3Tja2WCSsu6O+WjoJYSGEaAWOHz8OGI8k+XLbPptNp6jUmD6U2CG41vkB3YzZ0vnljTRTm8i5Jdw9seHXOlrKoT04cuSIV8rjaRLCQgjRCpgLWyQnJ1cLYV337vILBafBpitQWUBCp/ha54f2NgK62NbNK/fPzMx0eUzYMXErtDsHDx32Snk8TUJYCCFaAbMlnJycTGhoKKGhoVRWVlJaWurV++aZayJW5tW5VWFqf2MSlCWoD5WVnh8XzszMglAj4BubHR0RphAZXAwBIfy6r3U8KywhLIQQrYBzCAM+65J2hLAll44dO9Y6P6SnfZw2fCCFhYW1zjfXsWwrBITQIaqS8NDGHz1KjDU+lOw5LBOzhBBCeEBRURFFRUWEh4c71kT2VQjnOlrC+XWGcN+ugG6BsJ6cOOn5smSdMh6J6t5IV7SpW6LRPX/4ROtYJVlCWAghWjjnVrCiGK1Bn7eE6+mODg5SCLUdAyWAHfs82/osKyuj2GIEf+9k17Y66N/dCO0T+a3jWWEJYSGEaOFqdkVDy+mOBogOPAbAL4esHr13VlaWY1JWTxdbwkP6RgNQWB7vlTFqT5MQFkKIFs6fIZzbSEsYID4sC4C9RwM9eu+ffvrJsY9w9zp2TqpLnxSjxayHdG8VW2hKCAshhBu8/UhQXZyfETb5riVsf7+WvHq3B0yIKgLgeK7nQthqtfLoo4+6/HiSyfG6sO4cPnzYY+XxFglhIYRohNVq5YcffmDevHkkJiZy5pln+rSr0/kZYZOvQjgrz1g3Olgprrc7umOMfcGOYs9FyuLFi9m1axfB0X2BxpesNDkv2HH4cMtfsMO1kW4hhGjH5s2bxxtvvOH4Picnh+PHj9PdvtG8t/mzO/r4yXIgmE7xAY5JYTUlxBvHC0pC6jzvroqKChYsWABAYGQvLJbGnxE2xUYphAaWUE4Eu/Zne6Q83iQtYSGEaMTKlSsB+P3vf0+/fv0A+0pOPuLPED6Zb7T4kxPC6n1NUkejG7q4zDMzkt98800OHTpE/8GjKbOEEBkOHWJc//mE6NMApB0q8Uh5vElCWAghGlBZWcmRI0dQFIUnnniC/v37A/Y1jX3En2PC5uYN3bpE1vuaHsnGuRJL/a9xx1NPPQXA7fP/ChhrRtfXCq9Lt0Sje/zgcZtHyuNNEsJCCNGA9PR0rFYrKSkphIWFkZRkzPzxVQjruu4I4S5dujiO+yqEi0qNTRt6d4ut9zX9eiWCXolFj6DC0ryJa3l5eRw5coTIyEgGpk4BjC0T3dGzixFtOYURzSqLL0gICyFEAw4ePAhA7969AXwewnl5eZSXlxMbG0tkZFVL0xchbLPplFUa3dD9etW/VWFKSjJYcgDIbWZxDhw4AEDfvn3JyjNav13qng9Wr372BTsKymtvONHSSAgLIUQDzBDu06cP4PsQrms8GHwTwgWnQScAKgvo1bP+XZK6dOniCOGTp5rXEt6/fz9ghPDxXONYsptbFQ/uZXxYKdU7Y7O17C5pCWEhhGiAv1vC5uNJNZ/R9UUIOy9Z2a1b/SEcHR1NoC0fgMMZxc265759+wAjhDNzjUDv0tH18WCAPl3tD/6EdCU/P79Z5fE2CWEhhGiA2T3qrxD2Z0s4p8BcqCO3wRAGiAgxZiLvO3yqWfes1hI2Gtcku9kd7ViwI7Q7J0+ebFZ5vE1CWAghGlCzJWxOjmpJIeytVbwOHzNWwgqiiOjo6AZfGxNeYfxMM1vCZgj369ePTHt3tLtjwp1iIUAvh+B4Dh9t2fsKSwgLIUQDaoZw587GqhEnTpzwyRKW9YVwWFgYISEhWCwWysq8s3fu/iNGCkaGljf62g7RxuYNx7KaV5Y6W8JujgkrikJEoPHDaYea96HA2ySEhRCiHvn5+Zw6dYqoqCgSEhIAiIqKIioqirKyMq9sYl9TfWPCgGNvYW91SR/JMFrCsZGN746UGG8s2HEit+k7KRUUFJCdnU14eDidO3fhhL0Rm9TB/WvFhht/NweOWZpcHl+QEBZCiHo4t4KdF4vw5bhwfS1h8P648LGsUgA6xTY+MSol0XgsqDmPKJmt4D59+pBXpFBpNVbKCgt1b2IWQKcoY4z6SFbTy+MLEsJCCFGPmpOyTGYI+2LpSn+G8MlcoxXZuWPja0J3t6+aVVAS3OT7OXdFN3U82NSlo7HcZmaeZ7dX9DQJYSGEqEfN8WCTr1rCVqvVcQ/zns68HcK59m0Mu3ZufOWpPt2MxZ2Ly8ObfD/nSVlNnRltMvcfzilq2atmSQgLIUQ9/B3CWVlZ2Gw2EhMTCQmp3Rr1dggXnDaet+3ZteGZ0QAD+hhpWW5r/LX18WRLuE+K0SI/VerGzg9+ICEshBD1qLlalslXjyk11AqGqhA+deqUV+5/usLYFalfz8aTcFBfY4FnW0AHKiubNjnLkyE80L5qVom1amr1Dz/8wHvvvceRIy1nn2EJYSGEqEdjY8LeDuHsbGM/3MTEuncw8GZL2Gq1UmGLAmBAn4RGXx8XEwq20xAQwqH0nCbd03m1rOM5Rld4cif3J2UBDO1v1I0lIBGr1bjWa6+9xg033MCPP/7YpGt6Q5ArL1JVdSEwDjgMzNE0zWI/Hg58BMQAlcC1mqa18LloQgjROIvFQnp6Ooqi0KNHj2rnfBXC5mpP5uNRNXkzhE+cOAFBxrNBXRIan5gFEMwpLESye382/Xp3dut+RUVFZGVlERoaSteuXZvdEu6W3AkqTkJIIpm5Ol0TlWqbQ7QUjbaEVVVNBVI0TZsIpAFXOJ2+APhF07TJwFvArd4opBBC+Fp6ejo2m42uXbsSGlp9s3pft4T9EcJHjhyFIGMXovgo134mPOg0APsOu79esxmQffr0ISAgoMmbN5iCg4MJtBozy3/ZV1DrHi2FK93R44Cl9q+/A8Y7ndsPmHtrxQNN64MQQogWpr7xYGgfIZy2PxOUAIKU0wQFudYlHBNmLF15yL7cpTucx4OBZreEAcIVo/52HSgmLy+PU6dOERkZWW/3vj+40h0dD5gPwxUAzmuX7AMGq6r6K6AAo2v+sKqqc4G5APPnz2fatGnNKnBLZLFYHKvatHdSF1WkLgyttR42b94MGMtU1iy/xWJBURSys7NJT08nMNC1Z1HdrYvDhw8DRquurp+rrDSehT1x4oTH63jLTuPeYYGlZGS4tjJYVFgplMDeQ6caLU/NujDrOykpiWPHMsjMSQIUbOXHaepbiwzKoxjYlpbHho47AOjevbvj2WtfqWu1M5MrIXwKY8wXIBZwXg37JmCtpmmPqqp6BfAI8AfnH9Y0bRGwyP6t9xda9YOMjIwGK7k9kbqoInVhaK31YLYuhwwZUmf5O3XqRHZ2NiEhIfXOXq7J3booKTFWferXr1+dP2dOGKuoqPB4HefY9wWOi6x0+dopCdmk5UFBSVCjP1OzLsxW//DhwwmLSsZi1YmLgr69mv6+EmJLyToN2YXhFBUZrfOBAwe2qN9HV7qj1wNT7V9PB9Y5nVOo6oLOwQhpIYRo9bKyjDmm5uNINfmiS9qf3dH7Dhv9wR1cWLLSlJxoPJtrBnh9MjMzKS+vvinE3r17AfvM6GaOB5uS4ozu8eO5gS1yPBhcCGFN07YBWaqqrgGGAJ+qqvqa/fRiYIaqqquAx4F/eqmcQgjhU2YI1zd+aIazN5eu9FcIW61Wdu0x3lfPFNcXu+jexZgidOp0/UtXrlu3jh49evDHP/7Rccxms7F9+3bA6HnwxHgwQDf7X93JwrAWG8IuPaKkadoDNQ7Nsx8vAM73dKGEEMK0d+9e4uPj6w0ibzFD2Ny6sCZftoR9/Zzw9u3bKbMayz127ez6MpS9HUtXhtX7mkceeQSLxcKyZcvQdR1FUTh48CCFhYUkJSWRnJzM99vMZ4Sb8SaA3ilBsBVOlUS32BCWxTqEEC1Wbm4uw4cPJzU11fHMrK+Y9/NXCJeXl1NQUEBgYKBjy8KanEPYk3sbr1692vGMcAc3Vn3s08MoT4UeQ0VFRa3za9asYeXKlQDk5eU5FucwJ2WNHDkScJoZ3YQtDJ317hYFtnLKrFHsO2hMxpIQFkIIF+3cuZPS0lIyMzO54YYbsNlsPrmvruuNdkd7O4RzcozpNp06dSIgoO7/VYeFhRESEoLFYqGsrMxj916zZo1TCLs+Jtw53l7OoE51dtM/9thjxukgoxN23TpjilHNEG7ualmO8iQmQPlRADJzAwkKCqJ79+7NuqanSQgLIVoss6UEsHTpUp5++mmf3LegoICKigqioqKIiKh7Fx5vh3Bj48EmT3dJ67puhHCwcd8ObuzH0MmcmhvcqdZjQOvXr2f58uXExMTwhz/8wXEMYMuWLUAdLeFmjgknJCRAebrxTWh3evTo4fgA0FJICAshWixzxuw555wDwJ/+9CdH68mbGuuKhrYTwgUFBZxxxhk8/PDDAOzZs4fs7GyCYlMB6N/N9WvFRwNYIbgDh49Uf7jXbAXfc889zJgxAzBCWNf1WiHsqdnRiYmJjpYwod1bXFc0SAgLIVowM4TvvPNO7r//fqxWK0899ZTX79tYVzRUhbC3Zkf7KoQ3bdrEzp07efLJJ9mwYYPRClZCsYb2R1HgDDdyKzBQITyoFIDd+6u2ETh+/Djff/89ERER3HvvvYwYMYLQ0FB27drFli1byM/PJzExkeTkZMBzLeGOHTtWtYTDukkICyGEO8wQ7t+/PzfeeCNQtf6vNzU2Mxq8v52hqyFslrGpq0CZ9wFjVcNVq1ZB5FB0ghjQDSLD3RuXjQk3JmTtOZDrOGYOK6SmptKxY0dCQ0NJTTVa2i+//DJgtIIVRaGgWOdYNgQGNr8lHBwcTGSA/UNS1CgJYSGEcJXVaq226405oSY9Pd2jM4Hr4kp3dGxsLOHhxkpM5mpMnuRqCPfq1QuoWuKyqfcBY2z2ww8/hKgRAIzo7/71zHHhg0erlro01+F23hLS7HpevHhxte+//wmsVhg/FMJDmzcxCyApdDvoVog7l84pA5t9PU+TEBZCtEhHjhzBYrGQkpJCZGQksbGxxMTEcPr0afLy8hq/QDO40h2tKEpV96kXuqQb28bQ1LNnTwAOHTrUrPuceeaZgLFwRki8sQ3AiH7uh2BygjHx6WhWpeNYXZthjBo1CsCxcpYZwl9vMD5gzRjX/AAG6NIpCApWQUAIR4qHeeSaniQhLIRokZy7ok3OrWFvcqU7GnCEsDc2BGhsoQ6T2RJuagib97n99ts599xzAYhINDbLO7MJLeFh/YzZ5NnFcVitVqBqCKGulrDz91arzrcbje9njHX/3nVJTEyEnM8AWL+n7iVITcdO6tz4hI0dB3y3zYGEsBCiRaorhHv06AF4P4Rd6Y6GqnFhb4awt7ujzfeamJjI66+/zsUXz6Q0sB8AI/q5f72hvY0lK22h/Ry7JNXVHd2xY0fH322nTp3o2rUrm3ZBbgH0SYGBPZr0dmpJSEiAnM9Bt7F8SyCFp+sP2Oc+1nn3e3jiHQlhIUQ7Z07m8WdLuLFWqDe7o10NYefu6KaMlTvfp3fv3jz1wmeUWwLp3tm9hTpMA821MCIGOMK3rhAGGDduHFA1KcvRFT3W6O73hMTERLBkEcMOKizw9fq6X5dfpPPal8bXf7jWM/d2hYSwEKJFMlvC/fpVNcfaY3d0YyEcHx9PTEwMxcXF5ObmNvjauji3hAG2GtXepFYwwAAzhMMHsP/AQYqKisjOziY0NLTWjlSXX345AJdccgkAX9kfAffUeDBUPUo2KOFXAD5ZVfcHlVe/gOJSmKrCmQN8F8Ita+kQIYSwa2hM+MiRI169t6vd0d4KYYvFQn5+Poqi0KFDwwsoK4pCr1692L59O4cOHaJTJ/ee66kZ9lv3GSHVlElZYLSeI4KLKSGSHbtzGG0fq+7du3et5TdnzJhBdnY2HTt25HCmzi+HICocJqU26dZ1uvrqq9mzZw8zrzqLc/8P/rcJikt0oiKq3l9Zuc7znxjv+8HZvgtgkJawEKIFKisr48iRIwQGBjrGPME3LeHS0lKKiooICQlxLIRRH2+FsNmi7dixI4GBgY2+vqnjwuXl5RQWFhIUFOTYJGKrfaXQpkzKMnXtUAzAL4esdU7KctapUycUReGbDcb300dDSLDngrBjx448//zznDO+P+OGQlkFXPCgzr6jVS3it7+DrDyj9T9V9ditXSItYSFEi3Pw4EF0XadXr16EhIQ4jvtiYpbzeHBj45Lempjlale0qamPKZn3MTeJ0HXdEcJN7Y4G6N9NZ28WHDoRXO94cE1frDVC8WIPdkXX9MxvFGY+rLN2B6TO0bn+PJ3DmbDB6KnmwWsVj41Fu0pawkKIFqeu8WAwQi8wMJDMzEzH86We5mpXNFSfmOXJBURcfUbY1NTHlGqOB6dnQX6RseBGSjO2bx4+wNiDOKswzqUQ3n9MZ/lmCA2BGeOaft/GjBmisOsdhevPg9JyeP0rWKYZY8GjB8EVk7137/pIS1gI0eLUNR4MxhZ4KSkppKenk5GR0WjrqilcnRkNEB0dTWRkJKdPn6aoqIiYGDc2322Auy3hpnZH17zP2h3G8RH9mjc7efRQoxu/lO7s2PE20HAI/2uJjq7DtVOhY6x3W6IdYhTe/ZPCTefr/JwGg3vA8H7QvbPnZmS7Q1rCQogWp74QBu9PznJ1ZjRUXzXLk13Sri7UYWpud3RCQgIncnXue8VozV84pnlhNLin/ecjBrBp0yaAetdtPl2q8+a3xtfzL/NdCE5VFf7veoVLJyr0SPJ9N7RJQlgI0eLU9YywyduTs9zpjgbvTM5qTkvYZrO5fB/zvXZK6My1j+lk5cE5I+Duy90scA09k0DBAqHdsFhDqpWxpveXwaliGDvEt48GtRQSwkKIFsecUVtX68nbk7Pc6Y4G70zOcjeEo6Ki6NSpE+Xl5W7t6mTe59eimazcCp07wOI/KwQGNi8MAwMV4kPtG0OE9ycpKYmIiIhar9N1nZc+M1rf82e1vwAGCWEhRAtjtVodQZKSklLrvLdbwu50R4N3Vs1yN4ShqkvanXHhkydPQnh/Vh2YiKLA+48oJHX0TBimdDhtfBExsNZ4cHqWzkc/hnPjEzo7Dxrhf8XZHrltqyMhLIRoUbKysrBarSQkJFR7PMnk7THh1tgdDU2bIZ2dnQ0dLkZH4bppMGWk51qj/bvZZ4tHDKgWwp+s0ulzjc7vX43jvaXGsYeuVTz6bHBrIrOjhRAtirnof9euXes876uWsKvd0a05hE+ePAnx8wG4aKxnQ3DEgHA+/QkIH0CfPkYgr9qqc93jOpVWOGd4GTPGhzN5OIxsh2PBJmkJCyHqtGLFCoYNG8bWrVt9el8zhOvqiobqIezJZ3NN7nZHe2NM2N3nhKFpjyll5RRB7EQURWeah1eKGjeio/FFuNES3r5f59I/6lRYYP4seOfBfH5/tdKuAxgkhIUQ9Vi8eDG//PILCxcu9Ol9jx07BtQfwjExMcTFxVFaWtqkDQsaUllZSW5uLoqiuLwGc3Nawl9++SW9e/dm48aNjmPZ2dnk5eURGBhIx44dXb5WUx5TOlHaHwJCGdHX5vHnc0cOijS+iBjA25vOZ+J8ncLTcOU58NzdCn56IqjFkRAWQtTJ7O798ssvKS4u9tl9G2sJg/e6pJ2XcXRlzWaoagk3ZdWsjz76iEOHDvHoo486jr3//vvous706dMJDg52+VqudEf/5z//ITExkS1btlBWVkZZ2CQALhjr2nt1R0ykQkTQKQgI44ednSgqMVbDevfh5s++bkskhIUQdTIDrrS0lC+++MJn93UnhJszOctms9Va+tLdrmgwVs2Kjo6mtLSUgoICt8pglv/7779nz5496LrOm2++CcCcOXPcupbzo1tWq7XO17z66qtkZ2fzyiuvGB844s8DYPoo74Tii/fFc+kEeP4ehf0fKHz1VAChIRLAziSEhRC16LperZX5wQcf+OzevmoJn3feefTp06faNZoSwtD0LmnnDxGvvPIKmzdvZufOnXTq1ImLL77YrWuFhYWRlJSE1WqtsxwFBQVomgbA559/zpZfcyBiEAF6MWOGuHUrl825SGHJkwHcc4VCnxQJ37pICAshasnJyaGsrIyIiAgCAwP5/vvvPT7+Wh9fhLDNZuPHH38kIyOD2bNnY7FYsFgsjvFvd9ekbsrkLIvF4nivYHQVP//88wDccMMNdT6e1RizNVxXD8Hq1asdq2nl5eXx8mJjadCOgdsJDpKA9BcJYSFELWa49evXj6lTp1JZWcknn3zik3u7EsLNXTUrJyeHyspKANavX8+f//xnfve737Fy5Uo6d+7Mn//8Z7eu15SWcEZGBjabjZSUFCZMmEBRURHvvfce4H5XtKmhEP7hhx8AHPsGr9puhHzP2L1NupfwDJdCWFXVhaqqrlFV9V1VVYNrnLtGVdUfVFVdparqWO8UUwjhS2a4de/endmzZwPGbGlvKywspLi4mIiICEdY1KW5LWFzdavY2FgCAwN56qmnePnllwkJCWHJkiX1PqNcn6asmmU+StSjRw/uvvtux/FRo0YxdOhQt+5vaiiEV6xYAcBjjz0GgCXIaO0PSc6o9VrhO42GsKqqqUCKpmkTgTTgCqdzycClwBRN087WNG2D10oqhPAZ5xC+7LLLCA0NZc2aNY7Hh7zF+fGkhna1ae7ELDMsR44c6QglgEWLFjFmzBi3r9eUlrBZ9p49e3LZZZc5rtHUVjDUH8InT55k586dhIWFcfvttxsbY2wZDtoQ+nWTNZv8yZWW8DjAvrgY3wHjnc6dD5QDy+yt5CgPl08I4QfOIRwTE8PUqVPRdZ01a9Z49b6udEWDMQYbGBhIVlYWZWVlbt/HDOEuXbrw0EMP8fjjj/PGG29w0003uV9oqlb3cmehDDMoe/ToQXBwMG+99Rb33nsvN998c5PKYF7L+dqmVatWATB+/HjCwsK48sorjROlaW4tCCI8z5WPQPGA2cdSAHRwOtcZ6ARMA+4E5gNPOf+wqqpzgbkA8+fPZ9q0ac0scstTc4JFeyZ1UaU118WePXsAY3eejIwMx25GGzduZNKkSW5dy516+OWXXwCIj49v9Ge6dOnCsWPH0DSt3m3y6uP8/jIzM7nlllsAmvz3FRMT47huQ9dwrotdu3YBRpd4RkYGgwcPZvDgwc2aABcWFgYYu1A5l+Orr74CQFVVMjIymDRpEk888QQAgYGBfvk9bc3/PtzV0IdKV0L4FBBj/zoWyKtxbqWmabqqqiuAP9X8YU3TFgGL7N96fo25FiAjI6PRT+7thdRFldZcF+aiFcOHDyclJYUxY8bwwgsvkJ6e7vZ7cqceSkpKAGNCWGM/07t3b44dO0ZFRYXbZTp92tjhp3///h75O4qOjgaMFmiXLl0ICKjdyfjDDz9w6tQpZs2aBVTVcWpqqsd+T8xyZGRkkJyc7OjS37RpEwCXXXYZKSkpJCcn069fP/bt2+f4O/a11vzvw5Nc6Y5eD0y1fz0dWOd0bh0w3P71cOCgpwomhPAfszvTHHs1JwqZLVVvcbU7Gpo3OcvcKtF8tKi5YmJiSExMpKysrNa4cE5ODldffTVTpkxh9uzZ5OTkANW7oz3FeUlP8z5Hjx5l3759REdHM3LkSAAUReHjjz/m3//+N2eeeabH7i/c12gIa5q2DchSVXUNMAT4VFXV1+zndgBHVVVdBcwBXvReUYUQvlBWVkZWVhaBgYGOkOrfvz+BgYEcOHCA0tJSr927KSHclMlZzmPCntKvXz8A9u3b5zi2evVqhgwZwkcffQRARUUFK1aswGazcfToUcCzIex8PbNeVq5cCcDkyZMJCqrq/ExNTeXWW2/16L2F+1yaFqdp2gM1Ds1zOvdHj5ZICOFX5gzlrl27OtZPDg0NpX///uzevZu0tDRGjBjhlXv7qiXsjRDu27cv69atY//+/ZxzzjkA3HPPPZw8eZLJkyczZMgQXnnlFZYtW8bEiROpqKggISGBiIgIj5UBjBDevn07R44cQVVV1q9fD8DEiRM9eh/hGbJYhxCiGueZ0c6GDDHWNvRml7TzB4DGNDWEdV33WggD7N+/HzB6FH755RcCAgL49ttvHa3OZcuWVXtG2NNqtoR/+uknAM466yyP30s0n4SwEKKaxkL4119/9cp9LRYLJ0+eJCAggKSkpEZf39RVswoLCyktLSUiIsIxkckTaobwr7/+itVqZcCAAURERDB8+HA6dOhAeno6y5Ytq/YePMk5hEtLS9mxYwcBAQGO8WDRskgICyGqqS+EvT05y9wKsHPnztXGLuvTrVs3wCivO1sIOreCG1oQxF01Q3j79u2AMfYKEBAQwIQJEwAcOyV5O4S3bNmC1Wpl6NChREXJMg4tkYSwEKIaf7WE3RkPBuNxnPj4eMrKyhyP+7jCG13RUD2EdV1n27ZtgPGYl8kclzXruGfPnh4tA1QPYbMrevTo0R6/j/AMCWEhRDX1hXDfvn0JCQnh8OHDFBcXe/y+7oYwNG1c2FshHBcXR6dOnSgpKSEzM7NWSxiotdCJt1vC5vPBMh7cckkICyGqqS+Eg4ODGTBgAFC12pMntfYQhqrW8L59++oM4ZSUFEcdgndCOCEhgfDwcPLz8x2PJ0lLuOWSEBZCOOi6Xm8Ig3fHhZsSwk2ZnOWLEF6xYgUFBQUkJibWmmTmvHSvN0JYURTH393JkyeJjIx0DCWIlkdCWAjhkJubS2lpKbGxsY71kJ15YlzYZrPx2muvoWma45jFYnF0nTalJezOgh2+CGFz7+XU1NRak7/MEDZXt/IG53AfOXKk43lv0fJICAshHMwWZX0tNE88K7xq1SruuOMOzjrrLP7whz9w8uRJLrroIlavXk14eDiTJ092+VqudEeXl5fzz3/+k927dwPeDWFz1SzzXs6TskxTpkxh1KhR3HDDDR6/v8n570/Gg1s22UhSCOFgtijNx39qMrujm9MSNgPKZrPx9NNP8+yzz2KxWEhMTOSrr75ya8awKyH88MMP88wzz/Dee++xefNmn7SETc7jwabIyEjHrGVvcQ5hGQ9u2aQlLIRwOHjQ2IOld+/edZ7v1asXYWFhZGRkcOrUqSbd48CBAwDMnj2bAQMGYLFYGDBgABs3bnQ7MBoL4Y0bN/Lss88CsHXrVjZs2ODTEK6rJewL0hJuPSSEhRAOhw4dAuoP4cDAQMfsXnNPXneZi1nMmjWLrVu3smTJEjZt2uT2nsBgBGlwcDAnT56stbFEWVkZt9xyCzabzTHO/PTTT1NQUEBwcDAdO3ZsUvkb0qFDB+Lj4wFjvW3nmdC+ZPYmJCUlubQEqPAfCWEhhENjLWGg2SFstoT79u1LeHg4l156KbGxsU26VkBAgCNkzF2JTI899hhpaWkMHDiQlStXEhAQwBdffAEY4eTJ1bKcma3hoUOHurTylzeMGTOGG2+8kSeeeMJr71N4hoSwEMLBDOGGWqXNCWGbzeYI4T59+jShhLWZXdLmpggAWVlZPP300yiKwptvvkm/fv2YOXOm47w3uqJN5uSsusaDfSUoKIi3336bOXPm+K0MwjUSwkK0QEeOHGHHjh0+vafNZnN0R3srhI8fP055eTmJiYke2zyhf//+AKSlpTmObd26FavVyqRJkxg7diwAd999t+O8N0P43HPPBeCiiy7y2j1E2yEhLEQLdP755zNixAg2btzos3tmZmZSUVFBYmJig4v9m6G3d+9et+9hjgd7qhUMdS8gYq7o5bxIxeTJkx2v9WYIz5kzh+PHjzNr1iyv3UO0HRLCQrQwRUVFpKWlYbPZmDNnDmVlZT65ryvjwVDVEt63bx82m82teziPB3tKXQuImCE8ePBgxzFFUViwYAEBAQGcc845Hrt/TYqieDXkRdsiISxEC+Pcwty9ezePPfaYT+7ryngwGCs9JSUlUVZW5vZevt5uCZtbGtYVwgBXXHEFpaWlXHXVVR67vxDNISEsRAtjhnD//v1RFIWnn36azZs3e/2+rraEoao17G6XtDdawomJiXTs2JHCwkIyMjLQdb3eEAYICQnx2L2FaC4JYSFaGHPC06xZs/jtb3+L1Wrlzjvv9Pp9G3tG2Jk5Luzu5CyzJezJEFYUpdpympmZmRQUFNChQwcSExM9dh8hvEFCWIgWxgy2AQMG8Ne//pWQkBB+/vlnioqKvHrfprSE3QlhXdc9/niSyXk5TedWsDwjK1o6CWEhWhgz2Pr3709kZCQDBw4EmrdesytcHROGpoVwTk4OhYWFxMbGeny1KueWcENd0UK0NBLCQrQguq47xlnNoBs2bBgAO3fu9Np9S0pKyMzMJCgoyKVlDpvymJJzK9jTLdT6WsJCtHQSwkK0IMePH+f06dN07NjR0Vr0RQibq0317NnTpb1ne/XqRVBQEOnp6ZSUlNT7urfeeot77rmH7Oxsr4wHm5wfUzKfF5YQFq2BbGUoRAviPB5sMkO4OXv4NsadSVkAwcHB9OnThz179rBv3746l2h84YUX+O1vfwvAiRMnmDRpEuD58WCAjh07kpSUxIkTJ9i0aRMgISxaB2kJC9GCOD+eZDK7Wnfu3Ol4DtbT3BkPNjX0mNLLL7/sCOC4uDjWr1/PP/7xD8A7LWGoag1XVlYSGxtLcnKyV+4jhCdJCAvRgtTVEu7WrRuxsbHk5OSQlZXllfu6MzPaVN9jSkuWLGH+/PmAEcYfffQRUVFRVFZWAt5pCUP1JSplZrRoLSSEhWhB6gphRVHqXB/Zk5oSwvXNkP74448BeOSRR7jrrrsYPHgwH3zwAQEBxv9unFv5nmTWEUhXtGg9JISFaIC3un/rU1cIg/cnZ7k7Jgz1h7A5Aeu8885zHJsxYwZff/017777rtfWVa7ZEhaiNZAQFqIeS5YsIS4ujrffftsn9ysvL+fw4cMEBATU6rJ1Hhf2NF3Xm9USTktLq/Zhpb5Z0BdccAHXX399c4tbLwlh0RpJCAtRh4MHD3LTTTdRWFjIZ5995pN7HjhwAJvNRs+ePQkNDa12ztMzpA8ePMjChQu5+OKL6dWrF6dPnyYuLo64uDiXr5GQkECHDh0oKiri+PHjAOTl5ZGXl0dkZCSdO3f2SFldFRsby8CBAwkODq5ztrYQLZFLjyipqroQGAccBuZommapcf4h4ApN01SPl1AIH6uoqODqq6+msLAQ8P5KVab6uqKhKoR//fVXbDabY3zVXatXr+b+++/n559/rnY8JCSEuXPnunUtRVEYNGgQ69atY/fu3aSkpFTboMEfE6O+/PJLcnJyZCtB0Wo0+i9ZVdVUIEXTtIlAGnBFjfPRwDDvFE8I33vooYfQNI0ePXoQFBTEwYMHG1yQwlOcl6usKT4+npSUFEpKShxdx03xl7/8hZ9//pnIyEhmz57N4sWL2bVrF6dPn2bhwoVuX8/s9t29ezfgnQ0a3NGvXz/Gjh3rl3sL0RSufJweByy1f/0dML7G+d8CL3myUEL4y7Zt23j22WcJCgriww8/ZMCAAei67ggZb2qoJQx4ZIa0+T62bdvG4sWLmT17NoMGDSIoqGnr9gwaNKjadb2xVaEQbZkr//LigUz71wVAB/OEqqqxwDBN0/6qqnX3RKuqOheYCzB//nymTZvWrAK3RBaLhYyMDH8Xo0Vo7XWxePFiAK666iq6detG7969+fXXX1mzZg1JSUluXcvduvjpp58A6NKlS50/17NnTwDWrVvHqFGj3CoLQGFhIZmZmYSFhREaGuqRv6eEhATACPWMjAx27NgBGCtYmddv7b8TniR1UaU91UVKSkq951wJ4VNAjP3rWCDP6dy9wIsN/bCmaYuARfZvffu8h49kZGQ0WMntSWuvi40bNwJw5ZVXkpKSgqqqfPXVVxw/ftzt9+VOXZSVlbF3714URWHq1KlERUXVes24ceN47bXXOHLkSJPq+NixYwAMHDiQbt26uf3zdZk4cSJgTPRKSUlxTNAaNWqUo4yt/XfCk6QuqkhdGFzpjl4PTLV/PR1Y53SuL/AnVVW/A/qpqvqwh8snhM8UFhayYcMGAgMDOeecc4Dqu/N4086dO6msrGTgwIF1BjDAGWecAcD27dubdI+0tDQAx9aIntCtWzciIiLIysoiLy/PMSbsrVWxhGhrGg1hTdO2AVmqqq4BhgCfqqr6mv3cDZqmna9p2vnAPk3TnvBqaYXwolWrVlFZWcmYMWOIjY0Fqu9T602bN28GYOTIkfW+ZvDgwQQHB7Nv3z6Ki4vdvoc3QjggIMBxvZ9++omsrCxCQ0OlhSOEi1yajaFp2gM1Ds2r4zXyeJJo1ZYuNeYfOq/01KdPH0JCQkhPT6ewsJCYmJj6frxZtmzZAsCZZ55Z72tCQkIYMmQI27ZtY/v27YwfX3OOZMPMyVPmZCpPGTRoEFu2bOHrr78GjDpr6iNUQrQ38i9FCLu6QjgoKMgRWuZm8d7gSksYYMSIEQBs3brV7Xt4oyUMVaFuhrDMjBbCdRLCQmCsnbxv3z7i4uKoOdPfecN4b6ioqHAsR2mGbH2aGsIVFRXs378fRVHo169f0wpaDzOEjxw5AkgIC+EOCWEhgGXLlgEwZcqUWs/MensHo19++QWLxUL//v2Jjo5u8LVNDeEDBw5gtVrp1asX4eHhTS5rXWp2b0sIC+E6CWEhqLsr2uTtlrCrXdEAqampKIrCL7/8QkVFhcv38FZXNBih6/zBRUJYCNdJCIsW6YsvvuCOO+5wK2iaqrS0lOXLlwPUuZiMt2dIuzIpyxQdHU3fvn2xWCwNjlHbbDYefPBBnn/+eaBqUpY3Qjg4OLha8EoIC+G6pq1VJ4QXFRQUcNNNN1FQUMCMGTOYMWOGV+/34YcfUlBQwMiRI+nVq1et82YXbmZmJvn5+cTHx3v0/u60hMHokt63bx9bt25l+PDhdb7mm2++4e9//ztgfIgwW8KenhltGjRoEGlpaQQFBXlsIRAh2gNpCYsW56WXXqKgoADAsQyit+i6zosvGou+3X333XW+JiAgwLFRgae7pC0Wi+M9NjYpy+TKuLAZwAC33XabI+i90RKGqnDv1atXk9ehFqI9khAWLUpxcTHPPvus43tPhPChQ4cYPXo055xzDvPnz+c///kP5eXlAGzYsIGtW7fSqVMnrr766nqvYU7Oak55tmzZUqsLedeuXZSXl9OnTx+X9/I1Q9jsxq5p06ZNrFmzhtjYWIYPH86RI0cc9/VWCJv1U9/mE0KIukkIixbl1VdfJTc317FZgidC+NNPP+Xnn39m1apVvPzyy8yZM4dZs2ZRUVHhaAXffvvthIWF1XsN87GlTZs2NakMJSUlTJo0CVVVq7Wm16xZA7jeFQ1VIbx9+3ZsNlut82Yr+I477uDtt98mODgYMDZV6NSpU5PK35grrriCBQsW8MQTsmieEG7Rdd2X/7VJx44d83cRWozm1EVJSYneuXNnHdA/++wzPSAgQA8ICNBLS0ubVaY777xTB/TbbrtNX7hwod6xY0cd0C+88EI9KChIDwgI0NPT0xu8xs8//6wDer9+/Vy+r3Nd7N69W8fYwEQfMmSIfvr0aX3nzp16VFSUDuhvvvmmW+8pOTlZB/Q9e/ZUO75v3z5dURQ9ODhYz8jI0HVd1x999FEd0M8++2y37uEp8u+jitRFlXZWF/XmorSERYvx73//m6ysLEaOHMnMmTMZMGAANput2StVmXvcXnrppTz44IMsXbqU2NhYvv32WyorK5k5c2ajk4lSU1MJDw9n37595Obmul2G9PR0x9e//vort912G5dccgnFxcVcc8013HzzzW5dr75x4X/+85/ous71119PcnIyAH/84x95+eWXHa1+IUTLISEsWoTy8nIWLlwIwJ/+9CcURXHsGtTcLumDBw8CVTv7nHnmmfzvf/8jMjISgHvuuafRawQHBzu6pM3tDt1x9OhRAEaPHk1YWBgffPABhw4dYuTIkbzxxhsoiuLW9czHmTRNcxyzWq289957ANx3333Vyn7XXXc5xm2FEC2HhLBoEd566y0yMjIYOnQol1xyCYBHQthqtXL48GEAevbs6Tg+duxYNm7cyGeffcbkyZNdutaYMWOA5oXwtGnTeO655wBISkpiyZIlREREuH09sywbNmxwHPv1118pKiqiZ8+ejmebhRAtmzxLIPzOYrHw1FNPAUYr2NyBxxMhfPToUSorK0lOTq61XOPQoUPdah2OHTsWqB587pQDjP13586dS//+/RkwYICjy9hdZghrmkZ5eTmhoaGODwfmOSFEyyctYeF377//PocPH2bAgAFcccUVjuPOm9jrut6ka9fsim4OM9x++uknrFarWz9rjgl3794dRVE455xzmhzAAB06dGDw4MGUl5c7HlUyPxyYHxaEEC2fhLDwK6vVypNPPgkYE4gCAwMd57p160ZsbCw5OTlkZWU16fpmCPfu3bvZZe3SpQs9evSgqKjI7clizi1hTzH3E16/fj0gISxEayQhLPxq6dKl7Nu3j169enHttddWO+eJyVnmzGhPhDBUBZw748K6rnslhMeNGwfAunXryMvLY8+ePYSFhZGamuqxewghvEtCWPiVuZzirFmz6lzusLkh7MnuaKh7QlRj8vPzKSkpITo6mtjYWI+UA6q3hM1FREaOHElISIjH7iGE8C4JYeFX5s5E9U2Q8lQIe7ol7E4Im+PBnt7YoG/fviQkJJCVlcX7778PyKQsIVobCWHhV+YSjvU9UtPSuqOHDx9OaGgoaWlp5Ofnu/QzZld09+7dPVIGk6Ioji7p//73v4CMBwvR2kgIC7+xWCzs2bMHqH+LPbOFbG504I78/Hzy8/OJjIwkMTGxeYW1CwkJaXQDhZq8MR5sMkO4srISkBAWorWREBZ+s2/fPiwWC7169SIqKqrO10RFRTFs2DAsFgurVq1y6/qHDh0CjFawuytSNcSVrQSdeTOEzXFh8/rNeexJCOF7EsLCb8zx4MZWd5o5cyYAn3/+uVvX93RXtKmxEN68eTMPP/wwx48fB7w3JgzVJ2JJK1iI1kdCWPiNOR7c2KpVl112GQBffPFFnVv31cfTM6NN9YXwtm3buPTSS1FVlSeffJJHH30U8G5LOCwszLENokzKEqL1kWUrBQAnT55k//79QNXzueYGB97S2KQs0/Dhw+nRowdHjhxh06ZNLrf4PD0z2jR06FACAwPZs2cPJSUlRERE8Ouvv3LWWWdRUVFBeHg4ZWVlLF++nLy8PK+GMMBDDz3ECy+8UOs5ayFEyyctYUFpaSlDhw5l/PjxjB8/nnHjxnHppZd6/b6NPZ5kUhSlSV3S3uqODgsLY/DgwdhsNses7U8++YSKigqmT5/OoUOHmDZtGhUVFSxevJiMjAzAeyF8ySWXsHz5cjp37uyV6wshvEdCWLB27Vqys7OJi4tj7NixhIWFsWLFCkdL1RvKysrYv38/AQEBDBw4sNHXm13Sn3/+eZ3rSFssFpYuXcptt92Gqqqoqsq6desAz3dHA7VmSC9duhSAu+66i86dO3PTTTcBxv6+FouFTp061dpAQgghJISFI0DmzZvH+vXrHQHy73//u9nXzs7OZu3atbWO79mzB6vVSt++fQkLC2v0OhMmTKBTp07s37+/1oeDr7/+mi5dujB9+nTeeOMNNm/ezObNmykrK6Nz587VtjD0FOdx4YKCAjZt2kRQUBBnn302YEwmi4qKcszQ9lYrWAjRukkIC0cIn3feeQDcfvvtALzzzjuUlZU169qXX345EydO5Kuvvqp23NXxYFNgYKBjn+GaXdL//Oc/yc3NZcCAASxYsIC1a9fy888/8/PPP7N7926vLOPoHMIrV67EarUyduxYYmJiAIiIiGDGjBmO10sICyHqIiHczmVmZrJjxw7Cw8Mdz5yOHDmSESNGkJeXx2effdbka2uaxpo1awBYsGBBtW5kVx9PcubcJW0qLS1l/fr1KIrCunXrePTRRxk/fryjSzo+Pr7J5W/I8OHDAdi5cyfffPMNUPUhxnTllVc6vpYQFkLUxaUQVlV1oaqqa1RVfVdV1WCn4xerqrpJVdW1qqo+771iCm9Zvnw5AGeffTahoaGO43PnzgXg9ddfb/K1X3zxRcfXW7dudYQVuP54krOpU6cSGRnJ1q1bOXz4MGBsXlBeXs7w4cPp2LFjk8vqrtjYWHr37k1FRYVj3eaaITx69GjHpDBPL1kphGgbGg1hVVVTgRRN0yYCacAVTqe3A+M1TZsAJKqqqnqnmMJbzK7o6dOnVzt+7bXXEhERwapVq9i7d6/b1z158iQffvghiqIwf/58AP7yl784WsNNaQmHhYVxwQUXAMYzwwArVqwA4Nxzz3W7jM1ldkmXlpYSHx/veF7XpCgKCxYsIDEx0VFuIYRw5kpLeByw1P71d4BjnTxN09I1Tau0f1sBuL6SgvA7m83GsmXLgNqtuJiYGK6++moA3nzzTbev/frrr1NRUcGMGTNYuHAhnTt3RtM0/ve//7F582YOHTpEUFAQ/fv3d+u6NbukzRCeMmWK22VsLjOEwWilBwYG1nrNjTfeSFZWFsOGDfNl0YQQrYQri3XEA5n2rwuADjVfoKrqKCBR07RaK9qrqjoXmAswf/58pk2b1vTStlAWi8XxLGhrsmvXLrKysujSpQvR0dG13sP555/Pf/7zH7788kvuvvtul65psVg4fPiwoyt69uzZ5OfnM3fuXB5//HFmzpyJxWIBYODAgWRnZ7tV5hEjRhAcHMyaNWtYu3YtmqYRFBREnz59fP534DzOO2rUqFr3b62/F54m9VBF6qJKe6qLlJSU+k/qut7gfyNHjrxr5MiRN9q/Hjly5MiXapzvOnLkyHUjR45MbOxaeht17NgxfxehTtnZ2XpJSUm9559++mkd0OfMmVPn+dLSUj0sLEwH9KysLJfueezYMf2NN97QAX3gwIG6zWbTdV3Xi4uL9aSkJB3QExIS9N/85jd6Wlqa+29K1/Xp06frgH7ppZfqgD5hwoQmXae5jh8/rgM6oB8+fLjW+Zb6e+FrUg9VpC6qtLO6qDcXXemOXg9MtX89HVhnnlBVNRr4EJinadpJ9z8fCG/JzMykZ8+e9a58lZOT43gOuGZXtCksLMwxY3rlypUu3ffEiRPcd999ADz88MOO3YsiIyNZv349q1at4vjx47z00ksMGDDArfdkcl5LGvzTFQ3QpUsXHnjgAR588EF69OjhlzIIIVq3RkNY07RtQJaqqmuAIcCnqqq+Zj99L9ALeElV1VWqqk72VkGFe9asWcPp06dZtmwZ27dvr3YuNzeXKVOmsHfvXgYOHFjtedaazID74YcfGr2nruv88Y9/5NSpU1xwwQVcd9111c736tWLyZMnExTUvCXLL7300mpbE/pjUpbp6aefZuHChX67vxCidXPp/4aapj1Q49A8+/HHgcc9XSjRfM47/Lz++uu89NJLAOTl5TF16lR27NhB//79+eGHHxrcqMEMYXMCVEM+/PBDli5dSkxMDIsWLfLoHr7OkpKSGDt2LOvXryciIkJ2DxJCtFqyWIefWa1WPvnkE15//XVef/113n//fSoqKpp9XecQfu+99ygpKUHXdW6++Wa2bdtGv379WLlyJV26dGnwOiNHjiQ2NpYDBw5w5MiRel+3a9cux+StZ555hq5duzb7PTTE7JKeOHGiV1bEEkIIX5CtDP1s8eLF3HjjjdWOpaWl8fjjTe9g0HXdEcLdu3cnPT2dTz75hPDwcL766itiYmJYvnw5ycnJjV4rMDCQs88+my+++IIVK1YwZ86cWq/5/vvvueqqqygsLOTss8/m1ltvbXLZXXXnnXeSk5PD9ddf7/V7CSGEt0hL2M++/vprwBjXNAPl+eefJy8vr8nXzMzM5OTJk8TGxvKnP/3JcU2zpbpw4UK3VnBqqEv6tdde48ILL6SwsJArr7yS119/3Wvd0M4iIyN56qmn3FpxSwghWhoJYT+yWq2OZSNfe+013n33XaZNm0ZRURHPPfdck69rtoKHDx/O7NmziYqKYsuWLWRlZTFhwgTHkpSucp6cpTut/5yZmcldd92FzWbjkUce4cMPP5Tt+oQQwg0Swn60ZcsW8vLy6NWrl2PP2wULFgBGy/XUqVNNuq4ZwiNGjCAqKoprr70WgJCQEF5//XUCAtz7ax80aBBJSUmcOHGC3bt3O45/99132Gw2LrzwQh577DG3ryuEEO2d/F/Tj5y3EDS7cMePH8+UKVMoLCzk+eebtieGGcJnnnkmAPfddx8DBgzgueeeY+DAgW5fT1EUpk41HhX/8ssvHce//fZbAC688MImlVMIIdo7CWE/qrmPr+nPf/4zAM899xwFBQVuX9e5JQzQv39/0tLSuPPOO5tcVnMd6bfffhtd17FYLI51p2VzAiGEaBoJYT8pKipi/fr1BAQE1FpsYtKkSUyYMIFTp05V2/7PFadOneLQoUOEhYU1qdVbn+nTp5OYmEhaWho///wzGzZsoKCggIEDBzq26xNCCOEeCWE/WbVqFZWVlZx11lnExcXVOm928a5bt67WuYZs27YNgGHDhjV7ZSpnwcHBjhWw3n77bf73v/8B0goWQojmkBD2k/q6ok3mms3uhnDNrmhPuummmwD44IMPHOs2y3iwEEI0nYSwnzQWwqNGjSI4OJidO3dSWFjo0jVtNpvjWV5vhHBqaiqpqank5+eze/duIiMjmThxosfvI4QQ7UW7DuGXXnqJrl27kpycTHJyMlOnTqWsrMzr901PT2fv3r3ExMQwevToOl8THh7OmWeeic1mY9OmTY1es6SkhKuuuopvvvmGoKAgr21qYLaGwXh+ODQ01Cv3EUKI9qDdhnBRUREPP/wwGRkZZGZmkpmZyYoVKxzb+3nTqlWrABrdUWjcuHFA413Sx48fZ9KkSXz66afExsby7bff0r9/f4+V19m1115LYGAgIOPBQgjRXO02hN9++20KCwsZP348GRkZvPPOOwA89dRTlJeXe/XeziHcEFfGhbds2cLo0aPZvHkzvXv3ZsOGDUybNs1jZa2pc+fOzJ07l65duzo2URBCCNE07TKEbTYbL7zwAgD33nsvycnJXHfddQwbNoyMjAzefPPNZl0/KyuLJ598khMnTtR5/scffwTg7LPPbvA6Zghv3LiRysrKWuc///xzJk6cSEZGBhMnTmTTpk0MGjSoWWV3xSuvvMLRo0fp3Lmz1+8lhBBtWbsM4e+//559+/bRvXt3Zs6cCUBAQIBjkYy//e1vzdpO8OGHH+bhhx9m8uTJZGZmVjt39OhRDh48SExMDMOHD2/wOklJSfTu3Zvi4mJ27txZ7dzq1auZNWsWJSUl3HTTTSxbtoxOnTo1ucxCCCF8r12GsLkc5G9+85tqY7KzZs1i6NChHD16lLfeeqtJ1y4rK+Pjjz8GYO/evZxzzjnVWsRmK3jChAmOsdWGmK3h9evXVzu+ZMkSAObOnct//vMfmSAlhBCtULsL4bS0NL7//nvCw8O57bbbqp0LCAjgkUceAYyxYZvN5vb1v/nmGwoLCxkyZAjDhg1jz549nHvuueTn5wOud0Wb6hsX3rBhAwCXX365T7YOFEII4XltLoTz8/M5dOhQvedffPFFAG644QY6dOhQ6/zll19Ot27dOHTokEuPBtX03nvvAXDrrbeyYsUKhg4dyu7dux1d3a5OyjLVNUO6vLycLVu2ANT7iJMQQoiWr02FsK7rnHvuufTu3ZuLL76YzZs3Vzt/6tQp3n77bQDuueeeOq8RGBjIlVdeCcB///tft+6fn5/Pt99+S0BAANdccw0JCQksXryYgIAA/vWvf7Fs2TL2799PVFSUY4ejxgwZMoQOHTqQnp7Onj17AGNVrIqKCgYPHlznkpdCCCFahzYVwuvWrXOsnfz111+jqiq33nqro1v5jTfe4PTp00ydOpUhQ4bUex1zx6CPP/7YrS7pTz75hIqKCqZMmUKXLl0AYw3nefPmYbVaHdedMGGCy+s6BwQEcNFFFwE4lorcuHEjAGPHjnW5bEIIIVqeNhXC5qNFd9xxB/fffz9hYWG8+eabvPjii1itVl566SUAfvvb3zZ4nVGjRtGzZ0+OHz/u1trNZle0udGB6bHHHiMuLs4xLuxqV7TJnMH9+eefA1XjwWPGjHHrOkIIIVqWNhPCRUVFfPTRRwD8/ve/5+9//zsffPABAA8++CB//etfOXz4MH369Gl00wFFUbjqqqsAHNdszI4dO1i9ejVhYWG1FrHo1KkTjz76qON7VydlmaZPn05YWBgbN24kMzNTWsJCCNFGtJkQ/vjjjzl9+jQTJ06kX79+gNGCnDdvHhUVFY4QvPvuuwkIaPxtmyH8ySefYLVaG339//3f/wFw++23ExMTU+v8XXfdxdixYxk6dCgjR4509W0BEBkZ6VgF69VXXyU9PZ2YmBifLMwhhBDCe9pMCL/xxhsAzJkzp9rxf/7zn47N7aOjo7nllltcut6ZZ55Jnz59OHHiBGvWrGnwtevXr+fbb78lOjra8YhTTcHBwaxdu5YdO3YQHBzsUhmcmV3SzzzzDABnnXWWSx8mhBBCtFxt4v/iu3fvZv369URFRXHFFVdUOxcREcF///tfevfuzSOPPFJnK7Uuzl3SZre2acuWLTz55JMcPXoUXdd58sknAXjggQdISEio95oBAQFNfqb34osvJiAggNOnTwMyHiyEEG1Bqw/hrKwsZs2aBcA111xDVFRUrdecccYZHDhwgAceeMCta1977bUAfPjhh47ws9lsXHXVVTz88MP07duXiy++mG3bttG5c2d+97vfNfPd1C8hIcGxcAfIeLAQQrQFrTqET548ybnnnktaWhpDhw7lb3/7m0evP3ToUMaMGUNhYaFjKcply5Zx4MABIiIisFgsfPPNNwA8+uijdX4A8CSzSxqM7mghhBCtW6sN4ezsbKZMmcKuXbsYPHgwK1as8MoGBnPnzgVg0aJFAPzrX/8CjE0adu7cyfXXX8+sWbO49dZbPX7vmmbNmkVYWBijR4+uc7UvIYQQrYui67ov7+exm7311lvccsstDBo0iJUrV3ptW73Tp0+TnJxMYWEh//vf/7jooosIDAystpVfRkYGKSkpXrl/TXv37iUuLo7ExESf3M9dvqyLlk7qwiD1UEXqoko7q4t6JwO5tGyTqqoLgXHAYWCOpmkW+/FA4HWgH7BZ07R7m1tSV918883YbDYuvPBCr+5rGxkZybXXXsurr77K7NmzsdlsXHnllX7bS7d///5+ua8QQgjPa7Q7WlXVVCBF07SJQBrgPP14BnDcfi5SVVWfzhaaM2cOSUlJXr+P2SV96tQpAO68806v31MIIUTb58qY8Dhgqf3r74DxLp5rM0aMGOFYYGPw4MFMmjTJzyUSQgjRFrjSHR0PZNq/LgA61DhXWM85AFRVnQvMBZg/f75j5afWZt68ecybN4+77rqL48ePVztnsVjIyMjwU8laFqmLKlIXBqmHKlIXVdpTXTQ09u1KCJ8CzBUuYoE8F88BoGnaImCR/VufzgLzpNtvv505c+YQGBhY61w7m2DQIKmLKlIXBqmHKlIXVaQuDK50R68Hptq/ng6sc/Fcm1NXAAshhBBN1WgIa5q2DchSVXUNMAT4VFXV1+ynvwa628+VaZq2wWslFUIIIdoYlx5R0jSt5nqP8+zHK4GbPVwmIYQQol1otStmCSGEEK2dhLAQQgjhJxLCQgghhJ9ICAshhBB+IiEshBBC+ImEsBBCCOEnEsJCCCGEn/h6P2EhhBBC2ElLWAghhPATCWEhhBDCTySEhRBCCD+REBZCCCH8REJYCCGE8BMJYSGEEMJPJISFEEIIP5EQFkIIIfxEQtgNqqpG2v9U/F0Wf1JVNcL+Z7uuBwBVVXvY/5S6UNWzpB4Mqqp293cZWgJVVeP9XYaWTlbMcoGqqucBtwPHgYWaph33c5H8QlXVmcD1wFHg7+21HsDxQeRpoBtwhaZpFj8XyW9UVU0Fngc2An/WNK3Cz0XyG1VVzwfmA+XAB8B3mqYV+7dUvqeq6mTgPiAHeBn4VdO0Mv+WqmWSlrBrrgX+DfwC3KGq6kQ/l8fnVFWdAdwCLAROAX+wH2+XLR9N00qACiAao17abV0AE4EnNU17COjt78L4i6qqgcAdwCLgL4AKRLbT34urgf9gfBC5ELjcv8VpuYL8XYCWyN7KuRpYC2QB6cBPwEr78ZGqqh5o6y1Bez3MBv4HbAFu0zQtW1XVvcCHqqomapp20q+F9BGn34nVmqYdsP+PdT/wGXCPqqrfaZqW7tdC+ojzvw9N0/YBJcD5qqo+BGSqqvoz8JWmaQf8WU5fsNfFNcCPQDGwE6PH7AgwHAgHgjE+sLVZqqqGA3/GaPn/CBwCMjH+v1kGXKSq6kBN09L8WMwWSVrCNaiqOhtYBUQABzVNKwSSgLH2bratQBgQ67dC+oBTPYQBJzVNO24P4ACM1t+hdhTAZl2EY3wgQ9M0HRiM8XvwGTBPVdVu/iqjr9Soi8P2wxFAF+B+4C6MrtiL/FA8n6pZF5qmZQErMIZstmJ0xd4O/MZfZfQF++/9BxgNlg32wwrQC9CBXRj/bvr6pYAtnISwE1VVY4CrgMcx/jFNVVW1E/Av4DZVVSM1TfsF6AH09FtBvayOejhbVdWBAJqm2TD+p1tpf233ttzdVqMufgAmq6o6xH76R4wegtMY/+O9x/4zbfLfVR11cY6qqsnApxitvW6aphVghLP5+9Emfzfq+DcyRVXVfpqmrQKWAy9rmnY98DUQoqpqQFutC4we1S8xegrvVlV1HPA9MA4YomlaLsaH+XBou78TTdXuJ2bZZzHeD3wDrAMmAb8DQoCvgBuBycBcjF+2NRhjgJ9qmva1P8rsDY3Uw5cY9XCppmmHVVW9FeMfWAHQEfhNW5p84mJdnAfMA84GTmJ0QZ7WNO0RPxTZa1z89zEFox7OwGj1XAjs1zTtL34oste4+HtxAUYvQBeMIJoP5Guado8/yuwNTvXwJcY8ma727zMwPpTeDDwFDANigDRgBsZQzr/9UOQWrU1+YneVqqpdgWcwxm+SgHc0TfsW+DtwjqZp/wDeAZ7WNG0hxj+8ecCONhbAjdXDMxiTLBbaf6Q7Rgjv0zTtpjYWwK7UxTvAo8A/gDc1TbtG07Tft8EAduXfx9sYTwx8jNEleRawvg0GsDu/F28C++xf/9TGAti5HlKAVzRN0zA+jFdomva+/fx5wLsYQzWTgZ8lgOvWLkNYVdVJTl0icZqmPaNp2ttAtKqq/6dp2lKMMQ2A54AIVVWj7V1NN2ma9qzvS+15btbDS9i7GDG628ZqmvYvHxfZa9ysi+cxPuGjadp79p9vM/+WmlAXIaqqxmiatgu4r53/XkQCYZqmfYDRc/SiH4rtcQ3UQ6yqqrcBTwCjATRN+w4YaH/dL8A9baUevKHN/I/DFaqqRqmqugxjDOdCjIkTa1VVnWd/yRrgElVV4zRNs6qqOglYgjELthhA07TK2lduXZpRDwcBNE1bo2naKd+X3POa8zthf0wJcIyVt2rNqIsD9gmMaJpm9UPRPa6ZvxenAdrC89Iu1MNqYI79z7Wqqi6wv/64/bVt5nfCW9rdmLCqqiMxFlgYjfEQeZz9z8MYQXsao6X3K/A6Rnfjp/4oqzdJPVSRuqgidVFF6sLgQj2UY3wA2QB0xpiMtdQPRW2V2l0Im1RVfQFjvOY9VVW7YHQv7gfuBd7XNO2EP8vnK1IPVaQuqkhdVJG6MDRSD++2l0cWPa1ddUdDtenx72M8YpGoaVomxvOeH2M8flTUlsb46iL1UEXqoorURRWpC4OL9VAsjx41TbttCQOoqno30AfIBw4AezVN+8m/pfI9qYcqUhdVpC6qSF0YpB48r01/gquP0yfXMzCe6zuoadp77e2XSeqhitRFFamLKlIXBqkH72nvLeHLga81TSv3d1n8SeqhitRFFamLKlIXBqkHz2vXISyEEEL4U7vsjhZCCCFaAglhIYQQwk8khIUQQgg/kRAWQggh/CTI3wUQQrhPVdUI4EGMzeTfUlX1Zoydrh6w724khGgFpCUsROsUASzA2LsV4EdgNsYev0KIVkJawkK0Tpr9z8mqquoY+7v2AB4A9qiqehjohLHf7/XAWoztKBdh/Lu/RdO071RVDQGexAjwSGAZcJemadk+fC9CtFvSEhaidfqj/c/dGAFaVxd0pP3PDRjb0P0LYxP6ROAp+7n/A+7DaEE/h7Ea0qteKbEQohYJYSFaJ3OruJOapn2Ifb/rGmzA7wBze713NU17AWOv1172YzPsf87D6N6OBKZ5pcRCiFqkO1qI1smVpe5KNU2rUFXVYv++wP6nFQh0el0lRhibm6/Lh3MhfET+sQnROhVitHT7qqp6HcZ4cFN8jfFh/CagO3A+RqtYCOEDEsJCtEKaplkwxnfjgPeoasW662/260zEmLh1AcZMayGED8gGDkIIIYSfSEtYCCGE8BMJYSGEEMJPJISFEEIIP5EQFkIIIfxEQlgIIYTwEwlhIYQQwk8khIUQQgg/kRAWQggh/OT/AXT4f8XKuoH0AAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "best_model = TransformerModel.load_from_checkpoint(\n", " model_name=\"air_transformer\", best=True\n", ")\n", "eval_model(best_model, 26, series_scaled, val_scaled)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's backtest our `Transformer` model to evaluates its performance at a forecast horizon of 6 months:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "tags": [] }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "2fb9d97d905246dba0903dfa6b2e19b9", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/19 [00:00" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.figure(figsize=(8, 5))\n", "series_scaled.plot(label=\"actual\", lw=2)\n", "backtest_series.plot(label=\"backtest\", lw=2)\n", "plt.legend()\n", "plt.title(\"Backtest, starting Jan 1959, with a 6-months horizon\")\n", "print(\n", " \"MAPE: {:.2f}%\".format(\n", " mape(\n", " scaler.inverse_transform(series_scaled),\n", " scaler.inverse_transform(backtest_series),\n", " )\n", " )\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Monthly Sun spots Example" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's test the transformer architecture on a more complex dataset, the 'monthly sunspots'. " ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAEPCAYAAACwWiQoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABM3ElEQVR4nO2dd5gVRdbG3yFLkDTEQUFRSQqulBFBRBHTB4YFxWVFQEGR4KooGFBRYMVFBVRWXJWgK6wJEBFd2QUUMJQIwqKSROSSmWGAGZgZZu73R9++9u1b1V2dbprzex4e5nZ3dVV1V799+lTVqaxwOAyCIAgivamQ7AIQBEEQ3iExJwiCyABIzAmCIDIAEnOCIIgMgMScIAgiAyAxJwiCyACSKebhRP/bs2dPwvOk+lB9MuUf1Sdl/gkpV5Z5aWlpsovgK1Sf1Ibqk9pkWn3KlZgTBEFkKiTmBEEQGQCJOUEQRAZAYk4QBJEBkJgTBEFkACTmBEEQGQCJOUEQ5YIDBw4gk0N+k5gT5ZLx48ejXbt2aN++Pc4991x8/fXXSSvL/PnzsXHjxqTlXx74+OOP0aBBA9x7773JLkpgVLI7gDF2AYApAEoAhADcDmBj5G8AGM85/zdjrDWAGZFzPs45XxpMkQnCG6tXr8aiRYuwZs0aVK1aFQcOHEBxcXHSyjN//nxcf/31aNu2bdLKkOn87W9/AwBMnz4dr7zySpJLEwwqlvlvALpxzrsA2A6gF4B8znnXyL9/R46bAGAQgKsBjAuisAThB7t370Z2djaqVq0KAMjOzkbTpk3RokULHDhwAADAOUfXrl0BAE8++SQGDhyIrl274vTTT8fUqVMBAAUFBbjuuuvQoUMHnH322Zg3bx4AoEWLFnjooYdwxRVX4IILLsCWLVsAANu3b0e3bt3Qvn17XHHFFdixYwdWrVqFhQsXYtSoUTj33HOxdetWTJ06FW3btkX79u1x6623JvjqEOmKrWXOOd9t+FkMoAxATcbYcmjW+TDOeS6AppzzzQDAGMtljGVzzg8EUWgic8jKygrkvFa+0auuugrjxo3DWWedhSuvvBK33HILLrvsMsvz/fTTT/jvf/+LI0eOoFWrVrjnnnuwZMkSNG3aFB9//DEAID8/P3p87dq1sXTpUixduhT33XcfFi1ahOHDh6N///7o378/3njjDYwYMQLz589Hz549cf311+OPf/wjAOCvf/0rfvnlF1StWhWHDh3yfjGIcoGtmOswxpoDuArAMwCWc84PMsZuB/AUgOGItfLzAdQDcMB0jsEABgPAsGHD0L17d2+ld0hJSQlCoZD9gWlCptXHT+yuy8KFC/H1119j1apV6N27N8aMGYPS0lLs3r0bRUVF2LdvH4qKihAKhXD48GF07tw5arXXq1cPa9euRcOGDbFkyRIMHToUV155JS688EIcPXoUpaWluPzyy1FSUoIuXbpg5MiRCIVCWLlyJaZNm4ZQKIRu3brhwQcfRCgUQmFhIQ4ePBgt81lnnYWbbroJPXr0wNVXX42CgoLAr5cK6dzeioqKon/rdUjX+uTk5Ai3K4k5Y+xkAHMA3ME5LwFwMLLrPQB3Rv4uMySpDSDXfB7O+QxofnXAIvpXUIRCIemFSEcyoT5GCzrR9Tn11FPRu3dvdOrUCbNmzULVqlXRqFEjNGzYMGoZ5+Tk4OSTT0bNmjWjZatatSoaNGiAFi1aYN26dVi8eDGmTJmCK664AmPHjkXFihXRpEkTVK5cGQ0bNkTFihWRk5ODChUqICcnB5UrV0ZJSUn0d/Xq1VG/fv3o+ZcuXYoVK1bgo48+Qq9evbB+/XpUqqRsdwVGOrc33aUG/C6G6VwfEbY+c8ZYJQBzATzFOf+ZMVaFMaZfmc4AtkT+3s0Ya8kYqwWgHrlYiFTl559/xubNm6O/165di+bNm6NFixb47rvvAADvv/++7Xl27dqF6tWro1+/fhg1ahTWrFkT3af7z+fNm4eLL74YAHDJJZdg7ty5AIC3334bnTt3BgDUqlULR44cAQCUlZXht99+w+WXX45nn30W+fn5OHr0qA+1Lt8E5c5LJVRe930BXAjgccbY4wCmA3iIMVYAoAjAwMhxjwKYCaAigCf8LypB+MPRo0cxfPhwHDp0CJUqVcIZZ5yBGTNm4Mcff8SgQYPw+OOPRzs/rVi/fj1GjRqFChUqoHLlypg+fXp0X15eHq688krUrFkT77zzDgBg2rRpGDBgAJ577jk0aNAAb775JgDg1ltvxV133YWpU6di7ty5GDRoEPLz8xEOhzFixAjUqVMniMtAZBhZSRxET24Wj1B9UpMWLVqAc46ioqKMqI9OOt+fyy+/HMuWLQPwu2svjesj/MygSUMEQRAZAIk5QfjM9u3bkZ2dnexiEAbKg8+cxNwnVq5cib///e/JLgZBEAKMQxMzleSPd8oQLr30UgBA+/btcckllyS5NARBGFm1alWyixA4ZJn7zM6dO5NdBIIgyiEk5j6TySE2CYJIXUjMCYIgMgASc4IgiAyAxJwgCCIDIDH3GfKZEwSRDEjMfYbEnCCIZEBiThAEkQGQmBMEQWQAJOY+Q24WgiCSAYk5QRBEBkBi7jNkmRMEkQxIzAmCIDIAEnOCIIgMgMTcZ8jNQhBEMiAxJwiiXNGzZ08UFhYmuxi+Q2LuM2SZE0Rq89FHH2H69OnJLobvkJj7DIk5QaQ+R48eTXYRfIfEnCCIckcmLvBMYk4QRLmDxJywhdwsBJH6kJgTBEGkIV27dk12EQKHxNxnyDIniNTj/PPPj/lNljlB+ERZWRlKSkqSXQyinJCJ4m2GxNxnyDJX47zzzkOdOnVQXFyc7KIQ5ZBMFHcScyIprFu3DoWFhdi+fXuyi0IQGUEluwMYYxcAmAKgBEAIwO0AbgDwFwDHAPTnnO9kjLUGMCNyzsc550uDKjSROZSVlSW7CEQ5pLxa5r8B6MY57wJgO4BeAO4H0BXAWACPR46bAGAQgKsBjPO7oOkCuVmcQWJOOCUvLw/btm1TPv7EiRNYt25dgCVKDWzFnHO+m3N+LPKzGEArAD9yzos55ysBtI/sa8o538w5PwwglzGWHUyRUxsSc2eQmBNOqVevHlq2bIk9e/YoHT906FB8+umnMdvKq2UOAGCMNQdwFYAvARw27KooOFc+gHqeS0dkPCTmhFs2btyodNxrr70Wt23MmDF+Fyfp2PrMAYAxdjKAOQDugCbeJxt2l0b+Nz6VtQHkCs4zGMBgABg2bBi6d+/uvMQeKCkpQSgUCjSPvLy8wPPQSUR9gmbPnj2oX78+gMyojxGqT7Ds37/fU3mKiopSqj6q5OTkCLerdIBWAjAXwFOc858ZY5UBtGGMVQHAAPwQOXQ3Y6wlgH0A6nHOD5jPxTmfAa2TFAAS7o8IhULSC+EXdevWDTwPnUTUJ2iys7OjdciE+hih+gRL/fr1PZWncuXKKVUfr6hY5n0BXAjgccbY4wCmA3gRwDIAxwH0jxz3KICZ0Cz3J3wuZ9pAPnNnkJuFcIvXZy3TnlVbMeecz4HmYjEzz3TcRgCdfSpX2pJpDSRoSMwJt5CYx0KThoikQmJOuCXTxNgrJOY+Qw3MGSTmhFvIMo+FxJxIKiTmhFu2bNniKT2JOUH4CIk54Zbhw4fjk08+cZ0+aDHPy8vDgQNxg/oCg8TcZzLtbR80JOaEF95++23XaYN+VuvVq4cGDRokLNQzibnPkJg7g8Sc8EJpaan9QUkmNzdu/mQgkJgTSYVefoQXvBgDiWp7hw8ftj/IB0jMfYbEyRnpYFkRqUs6PG9HjhxJSD4k5kRSITcL4YV3330Xx44dsz9QQKJeBCTmRLnAzQOVDtYYkTgee+wxV+kS1Y6oAzRNIaFxhlPLfODAgfjDH/6AEydOBFQiIt1YujS1FzVbsGBBQvIhMSeSilMxf/PNN7Fu3TqsWbMmoBIR6YZbAypRhtdLL72UkHxIzH3A2IlHlrkz3PrMM3GlGMIdqS7miYLE3Admz56d7CKkLekg5lu2bMHEiRNRWFiYsDwJdVJRzJPxolBaaYiwZtOmTa7ShcPhcm9huh2amMjrds455+D48ePIzc3Fc889l7B8iWBJpOAWFBSgRo0ageZBlrkPGBuFagP55ptvULduXbz11ltBFSujSaSYHz9+HADKxQrv6UgqukvMZUrEEpkk5j5gvHGqboMBAwYgPz8ff/7zn4MqVkZT3r9oiN9JBzfL6tWrA8tLh8TcB4wCPnLkyCSWhAgSeoGkJukg5omAxNwH3Nw4EgYNt40+GdeP7hmhCol5mpKKPrt0IZ3EnEhNUtEyTwYk5j5AlnnioetH6KSiKB89ejRu2/LlywPNk8TcB0jMEw+5WTKLbdu2uY5hkoqWeU5OTty2rl27BpYfQGLuC+YRLN9//32SSkIECYl5MHz66ado2bIlrrnmGlfpU1HM3UZy9AKJuQ+YG8WuXbts05AwaOTl5blKR5Z55qDPtUh0wKxkuGeKi4sDOzeJuQ+Qm8U9d999t6t0ybh+ixcvxqJFixKeb6bj9V6momUu4+OPPw7s3CTmPpCKHTCZiJuZtn7zf//3f0nJl5CTTs9fkLHN01bMU+kGugkWRZa5N8455xxMnz492cUgfKA8WeZBrqyVlmL+2GOPITs7G3v37g00n+nTp2PChAm2x6XSiyWTMV/noUOHJqkkRCqRTmIe5Jq3aSnm48ePR25uLmbMmBFoPkOHDsWjjz6KAwcOWB7n5m27du1al6Uqv9BLMzNJ1lcqiXkKkaibUVRUlBLlIAgiHrLMNdJazBNF0CvI0wr1atBLM3Fs27YN3bt3xxdffJHsotiSTu0iSDG3XZyCMVYbwL8BtAVwEed8A2NsM4BQ5JDxnPN/M8ZaA5gROefjnPPAB40m6iZu2LABW7dulc7g8lqOefPmoW/fvp7OQRB+MmDAAKxYsQKff/55yotlqpfPSFLFHEAhgOsAGJdYyeecdzUdNwHAIAB7AXwCILWXzHbAtddeC0CbDNSkSZO4/V4b07Zt2zylLy+k00Ob7hw8eDBheZWn0SxBYutm4ZyXcM73mzbXZIwtZ4z9kzFWL7KtKed8M+f8MIBcxli276U1keibsXv37kDKEeTbOpPItIeP8AcScw23PvNOnPPLACwB8JTgXPkA6sWlSiNEN1p2883bnVoaJOZEqpHIESblaTRLkHm6WtCZc65/g70H4M7I38ZevNoAcs3pGGODAQwGgGHDhnleF+/w4cMIhUL2B0YoKSlRPl4ksPv27ROmLygoiPl94MABR+U6dOiQo+N1nNQnldHrYFcf0aiiZNRfNc90vj/GmYqq98cthYWFcXk54cSJEwlN5wW3z7oRUURGwIWYM8aqAMjinBcB6AxgS2TXbsZYSwD7ANTjnMcNzuacz4DWSQoAnl9RtWrVklZMRCgUUj5eNO22QYMGwvQnnXRSzO/69es7Klf16tUdHa/jpD6pjF4Hu/roCyuL0iYS1TzT+f5UqvS7NKjeH7cYV613c/4KFSq4SldYWJjw+1OnTp3A8lQSc8bYYgDnAmgFYD6APoyxAgBFAAZGDnsUwEwAFQE84XM5E45ouKDqJ5JTt8mJEyccHZ/umK9jcXExqlSpkqTSECIS5foIhUJ44403lI8XPYO7du1COBx2XOYhQ4bgl19+cZTGK0l3s3DOrzVtelZwzEZolnrCCPLCOBFz8/YpU6agZ8+eynmVd5/5qFGjMGXKFNvjMq3DigA6d3YmGbI28OWXXzo+1/bt2x0dn+rQpCEJXsT8P//5j6O8ypuY6/GrdV5//XWldCTmiSNRlrlTy1jWBl588UUfSpPekJhLcDIr06vITJ061VP6dOP222+P+U0RJAlVZM/lBx98oJRe5QswXSExl+DFMiecIVr8VgRd58SRqi9Yr6EvRowY4VNJ3BFkG05rMQ/ywjhxfZDIJAa6zgTFMZJDYi6BLHOCSD3SQcw3btyYlHzTWsyDJNFiTi8Ee+gaJY5McbMcO3bMc56lpaX43//+p9z+Zs6cKd1HbhYJ6WqZHzp0KG5b48aNcfjwYUfnKW+QmBNOxdw8csoNAwcOxNlnn42XX35Z6fhkvQjTWsyDJEgxb9CgQdy2ffv24aOPPnJ0HoIIikyxzIuLiy33y4LnGZk9ezYAKK85S2KeYjhpNE4bWHmb8alC165dbcfnk2WeWdgJrQinz5qdsD799NO+nUunQgW5rJKbJQk4scz9mvSTqtZQIli+fHnc+HMzJOaJIxFt0cq3LMPvDlC7JSGNqF4TssxdkCo+8/I2g7O80aJFi6TlbY7ImUnYLZQuQibmgwYNclUGJy8HEvMASbSYyyDLPDEkyzK3+mwOkhkzZqBmzZqOAlEFQVDGipvgarI20KhRI+F24zPVr1+/uP1BDHUkMU8xRA04aMv8t99+8+U8mUqyxDxZ+Q4ZMiTm/2ThxretQuXKlR2n8SK+ekemk/Nt2rQp+vf69evRsmVL23ysXv6LFy+2Te+WtBbzTHOzPPzww76ch/AX8tUHg5vr6lTMjXmILGar8xUUFKBVq1Yx21TW67WyzD/55BN8/vnntudwQ1qLeZCQzzz1KG+WeTJJRJ3dWNlO09g9m1ZW9LvvvusoL5VzAsC3337r6ry2+QZy1gzAiZh7WciCUCdZ1zQdppD7TaqK+fLly4XbZcN9Z82aZXk+K+F121dCPvMUw2sHKFnrmUOyX8zJEAdjnYOqv5vzyoav1qpVS7j9u+++szyflWBXrVpVvWAG7O5XUPczrcU8VXzmomNFa4gS3iA3S+JIhJiLnpt169YFkpeMihUrSve5XcowWaOfSMwleLXMScz9J1XEvDwsJmJs/4kUc7fxidy6wqyE17iotRPIzZJieO0AJTHPHMxtYeTIkUkqSeIwtumg+gxE5/3iiy9cncvtC4d85ilCqrhZRGIe1Njc8owf9/vEiRNYvHixMHJlkPl6IRnicPz48ejfifSZP/roo67OFYRlbuWCsYLEPMVwMmlozZo1cdvIMvcfP0Rl2rRpuO6669C9e/eE5ptuJELM/bT43Z7LSnjdirmdRU8doAnG63BDEvPUIzc3F/fffz8AgHOunE5031VCp/pFMl4m6SbmbstolS4oNwuJeYLx2tBIzP3Hq6iMGTPGVTpRWxg6dKinsqQ6xvabDmI+YcIE38tAbpYEEqTF8s4773jKj8Tcf7zeb7exb0T5zp8/31NZnJAMcTCKXHFxcSDzJmT385dffvHtXHbIxDwcDuOVV14R7svLy1M+v9sRMW5IazEPimPHjglvpB9iTiKfPNwKUnn0mRuvVZMmTXDmmWf6nodMSO0WKfETWZuYPXu2dDr/6tWrlc8vCiZGbhYBQT1kTgLWyyAx9x+v95vEXB2z0Lqxlp3moePWvSG6T6KX0Pnnnx/9u3r16sJz3XHHHdJ8ktXBaUdai3lQ+LHWp0w4ymOcD79IhpiPHz/e0TDGIPAqDk888QSeeeYZR2m8ulUOHjyIxx57zPIlIHsW3LommjVrhu3bt8dsu/baawEAEydOjG7r2bNn9O/mzZs7zsfuZWO8X4k0BMqdmI8YMQLt27d3ZX07uTHjx48Xbi8PYv7ZZ5+hQ4cOWL9+fdw+t417//79YIx5Op+ba//YY485TuM3boV11qxZaNiwIcaNG4fHH3/cUVqvIjRkyBCMHz8eXbp0kR7jt2W+a9cuPPHEEzHb9HpUq1ZNmMZNPd2WL2jSWszd3Ihp06Zh/fr1ljGF/bDMFy9eLGys5UHMe/TogR9++AF9+/aN2+dWmMaPH4+9e/fGbd++fTvGjRun1CmVrsHPTpw44diyBjRXwf79+wMokT16gKudO3dKj5E9Cw888AByc3Nd5Wu+x/ozK/u6caMhsqBeqpDP3GesfNdOxbxx48bC7W3atIlrXOVBzHWOHTsWt81t/WVfUp07d8YTTzyBe++91/YcsjCp6YBTyzodkD1Pu3fvjs4HcIrshe2nmLsNwBU0tmLOGKvNGPuGMXaUMXZ2ZFtvxtgqxthSxlizyLbWjLEVke1XBF1wAHjhhRdcP6BW0+2d3mBdzI2+OEBbcioUCsVsK09iLsJt/WX3RL++KgH/09UyT0dUniGrtmBcrs0JMstcti2I1Y5SuQO0EMB1AN4DAMZYJQD3A+gKYCwA3WSYAGAQgKsBjPO7oDLmzZvnKp2VmMse+oMHD1oe/9RTT8XtM79srBpC69atpfvSEVGjTqagppOY//TTT76fM9VG5QRh2Ki4WdzGKddJVYPMVsw55yWcc6Pj7UwAP3LOiznnKwG0j2xvyjnfzDk/DCCXMZYdQHnjOHDggKt0CxYskO6T3awBAwYIt+sNSNQL70TMu3XrJt2XKXzwwQeu0vlh7ZgfdNmkEFWCFMc2bdr4fs5EirlXy9xtWVXcLPfcc4+nfOzE3Li/d+/elmXxEzc+87oAjEGH9a5d47nyAdRzWygnuL3p7733nnSf23UGRb3cTsS8PCBbKSYRmB90FT+7k/OlOkG1veLiYnz//feOn8UgXi4qbpZatWph9OjRrsvwv//9z3L/Aw88EP17+vTpcfvff/99x3mq4GZA5yEAJxt+61fP2FJqA4jrjmaMDQYwGACGDRvmKHKdjLy8vDi/tAxzp6cs3a5du6TnEKXRO+dEXwmhUAi1a9dWOvfRo0eV6wJo9XFyfNBMnjw5ZvREaWmp4/LJji8oKIj+XbNmTRw9ejRmv0pexuBRdvmpsH37dpx00knS/X7fH6/n2rlzp3BGoiqy+txxxx34/PPP8eyzz6JXr14YOnQoduzYEd0vK/eRI0ekeRUVFbmq7/Hjx2PS6e0kPz9fabsKw4cPV9Yu0RyF1atXe7qXOTk5wu1uxHwzgDaMsSoAGIAfItt3M8ZaAtgHoB7nPE7ZOOczAMyI/PTltVyjRg1p5cyYL6AsnVWnatOmTeM+k/TfTZs2jTu+Xr16MflYnbt69erKdQG0+jg5XoWjR49iyZIluPbaa6Wz42S88MILMb8rVarkuHyy42vUqBH9Ozs7O07MVfISfd56uX5NmjSJKZcZ/f4cO3YM1apV8/x57fVeN23a1NNIjMqVKwvLoA/zXbhwIYqKiuKm48vKbfUirFKliqv6VqtWLSad3obr1q0bs/3kkzV7tFatWo7zycrKUk4jO87v5xZQdLMwxhYDuArAawD+BOBFAMsAPBP5BwCPApgJ4FMAT/paSguC+NS1+hw151dQUBCd5SZys5g/44LwE/pJ//790bt3bwwbNizZRZHiVhT9djOonG/v3r2oXr06rr/+el/zdkPQ7ausrAz5+fmOjk8UMgPMDVZfFMlEyTLnnF8r2DzPdMxGAJ39KJQV5lmFQYwdthNzY0fna6+9Fv3bq5ingj9d76CcN28e3njjDU/n2rJlC0pLS1NmxpzfL36V+/XRRx8B0CaRJZug21coFLL1JxuxernYDSn0Iw+350xV0m7S0FdffRXzO9GWufnlYZzMoiLm5ok0N910k3RfJtCvXz/pvjp16rg6p1urKhlinkpiEXRZduzYgSVLligf7/TlIiq/+SVpPkY2A1T/LbsmbocvJjOQXtqJubkBJFrMFy1aFPPbGEFNJObmc/3lL3+J+T1jxozo36Lp6unO3LlzY34br1ezZs0SVo7t27dbTi13g1MxmjJlCh588EFfy+AEr2Lu98vAqctRJMyq/TpOxfzOO+9UOq+ZZK79m3Zibr74TtwsP//8s9JxVi+IhQsXxvw2NhIVy9wcE8a4f9++fUrlS2f8EAQ3lnn79u3tD3KIU8v8vvvuw+TJk7Ft2zbfy6JCp06d0L9/f9fpEynmVvkb7785HK3MMjejn0MWJsJt5MZkzf4E0lDMzQ3ASYP48ccfXeVhxNw47CxzuwfA+DmXSrFDgmqUXqdSA+Ky2ZU3iE4rtz5o4zDLRLJu3TrMnj07KXmLsDKaVC1zu9jiep+W2f0xZ84cALGhcZ1gdvfqiNphgwYNXOXhlLQX80S7WfwW81q1auG+++4D4H0ccTLxy2pLpReaHW595sleoCSIxY/dcPjwYek+q7yciLnO0qVLY357XWzj4osvxqBBg5SO3bBhg6e8VEl7MXfy8BsbiFUHhxMxt3OzWJ2rY8eOAIDBgwcD0CYY+LHKUTJw86CLrBhzn4RqumSQymJu1ZnuNn+/xVzlXhvRJ9wZn3nVthCE0Sca7SW6Rg0bNvQ9bxFpJ+Z79uyJ+e32U9fq08fJOZ36zI38+9//BhBrXSR7VRsdp4Lp14MumqVpZvPmzb7k5RW3bS8RYn7zzTdL96VDGAJRe7rlllvitpk70WXtMFHDfpM5eintxNzs4+rQoYPvefjpZrF6cOrWrQsgVjgT9UnmN24eFn1JLyOqn81+0b17d9fi5tYyf+qpp6I+26D45JNPpPvcjr1OpFCJvri//vrr6N8bNmzA0qVL48Rc9qL0WvYePXrgwgsvtD2OxNwB5unTbj+5rS66n2Ku8sAbz+F1ok6ycCPm48bFR0qWibndQ+K2HXz++efRVXGc4tba+/zzz5MacCwdxPyHH36w3N+uXTthlFHZCmJeLfMrr7wSnTvbz4kkMXfAwIEDY34nugPUCpEQORVzc8yRZOFUHN1cM1G/RaItc8B9p2u6TRpSRVavVK3LRRddZHuMuU7mtUJV0quMPZddo6uvvtpRfm5IOzE3P+xuxdyqYTodMmWFygNvFM508GeK8MsnKXuJBNnp6VakVOqcCiEazGTaFPfTTz/d9hhznXSrXmZtm4/v0aOHp4iT77zzjuu0qqSdmJspKirC5MmTlcaQGwXhxIkTUovM6gE0x0G3a0gq4kxi/jtu3SxeCFLM3ZDsDt5UcLM4QeVFL3OPqn6FdOjQQSnGkOwa1alTJ6ZtB9F20k7MzTdu2rRpePDBB9G2bVvLdDNnzoxZ5Wbfvn2oXLmycAKHk8Vz9Rt85ZVXCvc7vWlOYlukEkGLeZColv3AgQMYP358NHyqSroVK1Y4Lo9sJrDbFevN2IlyItwsfg7BVRFzc530drZy5Urs3r3b9nhjGitUg4cFYbSlnZib2bp1q+0xxcXFGDBgQFycZQBYtWpV3Lbly5cr5+/2wbA6x4cffqicf6qgKoiyl56O7IFJBTdL/fr18cgjj0RjUdvVedOmTXj33Xcdl0c2lVylA06FVHCzTJkyxbdzeRFzABgxYkTc8aJr4KeYBzE5Lu3FXAWrC+dVJGRR2XTcWKw33XQTvvjiC0/lSjSq9TzllFMAxIYOVjlPkALj9B7ZfaLruF1h/plnnhFu37hxo6vzOcVqfVy/8HMIrhs3izGNnWWuL/3mVcyNkJi7xOoC+xVrW9ag3H5OOYkL7ReTJk2K/h3UaBb9XsgejJ49ezrKV8fLS9mtmAc1XNLpzEinWJX76NGj6Nu3r+N0TnETYvbWW28VbvdqmYvuv3Gb/iXmxWduhsQczhvUK6+8gttuu02636uYB+FmUTlvEDz88MOu0zoVRP0BHDBggKPjZXiJBe/0Wqta5qmKVX3XrVvnKp1TjEbO/fffr5RGNi1eRczNediJucgI89MyJ585nF+Ee++9Ny5srRG/xNxPN4vxvOmCannNx73++uu+nP/XX3/FzJkzlc7l9NxmVMU8CD9/kDHvDx48iEsvvVS6389OS+NzfNddd8Usei5Dt+ZHjx4ds93qOuvDCa+55pqY7ckW8yBCOqSdmBsXc/ADvx44v90s6YZTN4t+vVSv/6+//mp7jKqVbyYon7ld3dz2p3hFJjg7duywTHfuued6zlvHHCzLiatED4NhTG+Xxmy0OXGziNJ4JQhjLe3E3O+VPFQtQxnmm/KPf/wj5rfKA6sPdbM6byrz5ZdfKscasfuSkRHkkE2n11oPhvbBBx9YLmAchJiLRl85RVbfmjVrWqbz061UWFgY/VtVzCdPngwg3qpVEXOzEBvTiOolWtLQT8ucxpkHwBtvvCENNC9DdMP0xmGOcawyzjg7Oxu1atWK27569Wrk5OREFwVOVTp37hz36StDRcydjPM342bh5F27dgkXr5g1a5bweH047MSJE9G6dWvpee0EKllfbXar7yQCFct8+/btwrTmWOSycofDYWl7s7PMq1WrFrfNzw5QEvOAMH5eqvQy/+tf/4qG4rW7ebIheGZuvPHGmN/hcBg333wzdu3a5XqERyqiIuayoXkqXHfddY7TDBo0CCeffHI0JLGOeb1WEeaQzDpHjhyxXbwgWR2od999t3B7Ir8Gzda1yOo97bTThGll63maMVrlTsXcWL7LLrtMWkYzbvuO/CCjxVxlQhEQe2HXrFlje/ytt94aXVjCrdvAjDl9ot0sZndB0MvGpcoCE0bMLxEvZfz73/9ue0yyLHNzSAqdRL5cjKNmwuGwp2utIuZmVMV88uTJUTcouVmSyJAhQxynUb0Z+qonOn6LeWlpaSDhfWU4dTV5JRXFXNXiU0GlbyfVhjYmsjzGL5pwOIwJEya4PpdXMRfN6dDFvHr16sI0MkjMA0I1nKyXTlW/LGhzg3QzqQIAZs+ejezsbMcxuq1myPnJ6tWrAzlvEBw8eDDQ86fasNVkvVzC4TDuvPNONGrUSOl4c9uUlVv/8rETcxG6JhgjJZLPPImoCtKoUaNc5+GX28A8rK569equztm/f3/k5uaCMeYoXSLcOvv27Yt2Xnm9Xk2bNvWjSDH4+QJTOZdbN8vTTz/tKp0dyRRzQDwpaO3atXHbzNfWLjSESITt7o9umRvF3M/2QT7zgPBjIobXG20OonTHHXd4Ol8qYvy0/u2335TTie6PbEUZFe655x7h9kS7ftyKZ6aJuRV656MbVN0sIlTF/L777ovpLCXLPI0J0qJNRb+yF4z1sZo2bub555+P29amTRvl9OZ7JLtn//3vf6XnqFKlinJ+qohmATq5LjJU3IvffPNN3LZkzW3Q8xXlf/jwYUfnateuXfRvP9wsdvd9ypQpMXNLSMzTGJGb5a9//av0+D/+8Y8AtEV9g0AWv8IpQbxIjA/QXXfdpZSmrKwsJgCYG1RGlujIBN2pqKhcP9GMymXLljnKR8S0adNsjxHNJE22m0U1f6traxRTKzeLnZjrYahVVhcyDoYw5i8LDGYsm5+QmJtwK2LGdGeddZb0OP1Nr7LU1c6dOx2Xw20jEcV09gv9wTA+QCeddJJS2pUrV3rO3zwrt3fv3tJjv/zyS+F2v6JrGtm/f3/cNj8ecn2GqhWimauqefsZowX4XQBV+xCsntFQKBRdxMOtm8X4ZaMi5s8880x0ropRzN966y1pGiduRlXSTsz1B/PBBx8M5PxOfbEqEQ+ND61dCFgvvPDCCzhw4ICrtFu2bPG5NL+jW4HGOqu+NI8fP+45f3NeolXdRcfK/g4SP1wdKhPfRK4YFTGfO3cuqlWr5jkMhhGnlrkV+fn5qF+/PgBrN4vV/TS+VFSf00ceeQTA73U59dRTLQ0ApwtKq+BKURhjLRhj+xljyyL/GjDGejPGVjHGljLGmvldUJ1BgwahsLAQf/7znwM5v5WLxApj4zA/kA899FD0bytrwSuqoURVyMvLw8iRI22PczKs0404+i2iql8EQOyKP07L4bbcfgia2+XlVPLWR12prFRvhb5ICeCvZW7ErZvFWA7V9q2nUR3dlmqjWZZzzrtyzrsCyANwP4CuAMYCcB9cQ4GTTjpJSQxVIu2Zuf322x0dL7opF198ccxv45qOegPzU6Ryc3Ntl2Nzw9SpU22PceL7N94z49+JXFVJ/2weNmyY7bFGEUiUZe6HmLv9mkmkz9x4PfU5FSIx92PRERU3izGAmfHLxm2oWrtyy5YG9IIXMe/EGPuCMTYBwJkAfuScF3POVwJo70/x5KjcZNFyUHboq4oAwBVXXGF7vOhNbB4DbXxIgnCzjB07FkuXLvXtfEbsHvB//vOfyueSWeaXXnopLrroIueFk2DuRDSKhB7dsUOHDrbnceMW8opZ0ObPn49evXo5Oodfk5H69Onj6jwq6ILZp08ftGrVCoB6DHHzvZDNqXAi5p06dYrW3yjmTicUWhlrxqiUQfTBuFWU3QDOANAFQEMANwEwdvf7X1ITQT1cxgblZDyv0x52P8sfRGeKjt2nryyynQgrcfTzs/Pyyy+P+W0UNz1omUzwjFatlxeu2/trvt69evWKi97ox+pWKulEnX9+9GEAv9fzxRdflOYPqN2Djz76SDh81anPXM/fWMe2bdva5i/KU2R5GzvXrRYBcYsrW59zXgSgCAAYYx8AuAOAsUdFqACMscEABgPaZ2737t3dZA9APBIgFAq5Pp+e1uhvLCwsxN133y0d2rZz585oh+OxY8ek+R8/fjy678cffwSg+aSdlFd0bElJCdauXStdScnL9dD57bffXIcWMJfFOPln//79MeUzT+XW98k6dO3qZtxvHH0hus9Gxo8fH51UZBQBq/xE+6zinFulNZarU6dOCIVCccMiS0tL8fDDD0tHIKmGsTDnbZ6cZbcUXygUUnpxyNouoLkg9XOIXBoVKlSIS19QUBC3rU+fPjF9RqFQKGbIoPn4vLy8uLx+/fVXVK1aFePHj49uq1+/fkzaDz/8EPfee29cbKbCwkKEQqHoseFwOC7P7OxsXHbZZVi+fDlOOeUU18+n0XtgxJWYM8Zqcc71ANCdAXwM4G7GWBUADMAPonSc8xkA9KWCPJliovjTskoaqVq1qnBolZ62Ro0a0W0NGjSw7DBr0qQJ6tWrB0Cbfi+9yJUqIScnB+vXr4+KeYMGDZTKay6fkVAohO+//95RGqc0btw4JtiQW3JycmI+Wc31f+2112JeSvq+7Oxs6fns8tMxirK+XbQgiPkY46ewVX6ifSrLoInSGtvfeeedh5ycHOHcgUmTJuHZZ58VnlNlOJ0ob30UiE7NmjUxZ84c6WCDnJwczJs3z3E+wO8v2GbNmsXla6RixYpx6WvWrKl0//UFMCpXrhx3vOgaNWrUCDVq1IixoM3pcnJycMMNN8RZ9gsXLsSCBQuiL+OqVasKy6hvq1Onji/PpxG335GXMsa+Y4x9ASAHwNsAXgSwDMAzkX+BIvr8/9vf/mabzs63bDxvWVmZpZuhrKws6pe3mlSiWxzGYY+qD5wdoiD6fuJnmFaRu0nHr8lOIlSXBTOTCDeL2WVibn+As7Zy/Phx14uZmK9JVlYW+vXrhzPPPFOa5ocfhHabJRs2bEBBQQGA2BemW5+5DCs3i+hrU/eVewm8p+cp84nrZU+ZSUOc80845x05550557dzzk9wzudxzi/hnHfjnAfnxI0g8t3ZBcxq3bo1OnXqJNwn6vywE/PS0tLo8D3zwgZG9NWGjI3QLzG3Yvr06Z7PEZSYJ3Las+gcKvnLxNzPzitz2URiDqivv/npp5/6Vha9vVrVV+VamNuQ0b9tTK/qM1cR83A4bDk0sXbt2hg6dKiwnDfccIPt+WXo0UrNqyHp6PVJtaGJScWNRWo1zfnQoUPYuXMnnnvuueg2FcvcCapift555zk6rwxzY3VDssXcPAGmcuXKjmfGbt68OW5by5YtbdPJhEoU/9ot5voZ25Txb9WXv5cOSvOqSXp7tfpCMQvrjh074vzJTZs2jbF2jeczdhRaWeYqE6GMlJaW2s7pMI/W0Z+Xs88+GwBczWUZM2YMALEbGEhByzwVaNeunXDRVSusgjOFw+G4lWbOOOMMW8vcCXZi/uqrr6Jp06YxLxQ7gl6txmmjO/PMMzFq1CjMnj07bp8TMdfzNT/ExcXFUV+jHufGClkn3jXXXGObVlZGfSidFU5dATrmL0MdVZePF9eQeY6Ffi6rupjzO+WUU9CkSZOYbfv27YuZYWx8SRr/Fl1X/fxOZ5waDTHZNTFv1/3/+n13E1xNNDBDlCdZ5iYeeOAB385VVlYW02inTp2KOnXqBGaZixrK4MGDsXPnTsvYLkY2btwY2ExYHScvi7Fjx+Lnn3/GpEmT0L597FSD7777zpGY165dG4888ohji8yMLH1WVhbGjh1rmdaPUTx2mMsnc7OoinQQMdmt8lYtl/F+G9MYxfy9996Lm1mq73c6AbCsrMzSzQLYr1BkdS1Fbi+V0Sl63ckyNyEay+n2jVdSUiIc9eCXmBcWFiq5WbKyspR9stdff71y/m5xIuaVKlWK1tH8IJiHAtrdp6NHj2LixImevzys8pGJuW5F6mIuGsPsBKvJZ+b6ydwsItFU7TCUYTd80k8xl9XF+Hfz5s3jFpoQuVlUnnGjmDvp+ygsLFSa2KcHjzOiEjyP3CwSRG9OfWUSq2m4okBLJSUlMTdXf1H45WaZOHGiss9cVcxVet3dTkfWcSrmVvjhMzcybtw42/RW+ciusz56Q792TmdgArFtc+LEidLjVC1zUVsXueOcWObmryfZuazO6WbUjvFv1fROxby0tNTWzSLKe9KkSUqWuajtqDyP5GZxgH5BzZZgly5don/XqlUrLt3s2bNjOkj1m+WXZb5169aYYVx+iLkKXt0UTsRc9vkMaI3Xq5jrszd12rRpY+ujdGsBbd26NTq71WscDSsLz4ubRdQv4QQ9bKsMPU+r8hsne4mWeNNxc++B36+HU6NExTIXlePXX39VCobn9hkly1yC6GboN98sQsZ4HaJ0jz8eGxtMf4BlQxnNeags8zZjxozo31Zi7mfcFq9i3rt3b+UYN8Zym+sXDodjOiNVG7Ox/I0bN47bb4x3IcKtBaSHNAW8DyM1PvhmP7y5nRp/21mw+gQ0I362HRU3i34f77zzTst4N27FXDfOvIi5THhFbbCoqEjJzeJWzMkyd4BMzK1C1IrQb9aQIUPwzjvvCI8pKyuLBogyd9yIhMdIuljm3333He6++26lY40dt2ZrNhwOx6wu5MYyF6WpVq0a1q5di59++kmY3vjAGmdXOsFOzOfPnx+3bebMmdG/jZ3dkydPxiWXXBL9bTU0UeWryLzMnJ8doPfddx8Aa0NFL79dm3UrXvq1N7owVM713nvv2VrYsslkbt0sKuj3VDYO3QsZJ+Z647ISMZU43XqDqVSpknT5J+PwJ/MDb56ivHz58pjfVsOeRA3F7aLTKotV2D0cqsG0jG4QPcyBMQ99QgUA1K1bV+mcRkGTjaHu0KEDWrVqhbfffjtun7Fubjsy7dwsN954Y9w241h086LAxt9mwTa6AN3GPfGDPXv2RF/OsgWwAXlgKfPkJbeWue7XN4aRVmHq1Km2PnPR9c3KygrUMteHU6uEQXBKxom5yDI3x/p2EtrWLi/9pWFuzOa3unkihVPLvHHjxsIedDvMowNEfPzxx5b7N2zYINy+fv36mN/GOpvnAJgfnHPOOce2XEDsS1kPXyujb9++cfkZ76Mozo5sgpYxnehenXbaaTG/ZXFSzOlbt24dc3/NRkeLFi2if5v7CESY4wx5+Xy/8MILAWhLCBp94So+f3P7v+qqq2J+qyxlJ2L16tUIh8O2bdRMaWkp3n33XQDyePnGBTKMBGmZjxgxAhMnTsRXX33lKr0VaS3mqj5z0epBss9yHfONFAVOKisrk35m2n3uuvGZO5lMpKNiAdgtGVdWViYcxmY3GsL40nTb4ePETZSVlRU3CsmYr2hUill0dIwvMNG9Mi8vOHr0aGm5KlWqhI0bN+Ltt99Gt27dYoTN7CbRy3vxxRfHvJxkmEdQeBnKqX8tXX311cppVN0sxtASTl44x48fj4uZr5L+xIkTePnlly2PqVq1qtBFpmKZu+0Ub9asGUaPHh03scoP0lrMReiN2SgCImG1m8VnTiOKGe7EMjeTKJ+53YiF4uJipXCtbobnGa1GP8T8sccesz1evw9z5sxBWVkZJkyYEN0nipQoEwajS0f04DZrpr4yYo0aNdCmTRvcdtttAIA1a9ZE902ePDnmWL39XnrppTFtSHW5uyVLlkT/Puuss6RRJ0U4XdKwuLg4GtTLTtyM/QR2McLNX3XmiXEqYq4aLMscFiQcDkefdas2G8TiEl5JazEX3VTdV2ns/fajU6hWrVoxwgDE+sz9FHPZw6SyCEXVqlWxadMm2+N0TjnlFDz55JO2x5l9/ioYh3raBUGToYt5ixYtHIn5oEGD8NZbb1nG4wHkwmC8n14i9wHW/QPmdiAbgSGzMs3lMLrV/vnPfzpa89SpmI8ZMyYaRM5O3IxRMfVj7733XuGx69ati/licuM6Up0xaq5rWVlZdMEMKxelXX0vuOACpfz9JK3FXDRc6aGHHsKmTZuEMcudIHpYb7755pjfRjeLWcz/8pe/WJ7fqjHIhEIlwFR2dnZcyFIrC8Npx5KOygNmnPIsCnalgv6yvO2225Sm1xsXZli9erWrPAH7Dlqrl7GTrxDzQy/rtJPNLrSb0GO8T3ad6E7F3Dhix84yF42flz0Dp556qmWQONXFpFU62UXzIXT0eOgq6cxcd911tnn7TcaJOQCsXLlSSczHjBkTt8KNFeYbWFpaim3btgn39e/fP5AeayN6gzGO0tAbo/FT9eDBg77nrTpc0Sv62HQ3PkoVf7vspfTNN98AcBaf5bPPPgPgzG9tXr/SKh62KGiYk1Xg7QLTORVz48Q8N2LuJBqjkY4dOyqVTx+1pnfsijCXQXU8u911T4YbJiPFfODAgRg0aJBt+gkTJkjjP4tulrnBGq1aUWM2jkzwm8OHD0d7+OvWrYt3330XtWvXxty5cwHEdnzaBZRyg3EClBWqQxBl6Et4uXk4vI6xB+JHi1jRo0ePmGnkMozt4v7778fGjRujv62sVlHYZ6sFP8wLi9uJtJf1ae2+RozXRHcXWgmnrAwqLkEdfcUg85qwRszXZMGCBdG/ZaNdVCAxd4g+YUd0440jNKwasd2IDCNmwTb2hIvE3MuyUE899ZR03y+//BLzGVqhQgX88Y9/RF5eHjp37gwAMeurytYwdeqL1I/XV4lRwUmjtvqScSMwKlaW3zPxjJ3iKnnu3bs3pg3qw+lUXyJWk4waN24ck5fdvdA7fd18Bb3yyiuW+++66y5kZWXh008/xQsvvABA3g8AyJ9ZJytS6SOOROt92uUDWK9/YAeJuUNuuOEGLFiwQDnGhAiZSIisanMjF8VyMZKTk+N4fKyOlTV9+umnx0yQEQVEUhE/p8PY9I4hlXH6OiqTlnSsQieIpq7boeKn14OuOXG3WbFq1Srb62r+7C8tLcWmTZswZcqUaD1fffVVpfyMK1zZWcdZWVn4wx/+IN2vL07hxgixi5GjozrsUdZ+rV6+snZp1Vdg/noxIorhZMQqgJrVGrOBoQdASsI/X+nXr18Y2iLRcf/Wr18fDofD4Z07dwrTmo/v1auX8Lj9+/dL88jNzZWWrWLFinHHqyDKR7R9zpw5tulFHD9+PO5cixcvDp922mnCvGvUqCEtlywPJ8fu3LlTenx2drbja9axY0fbPMvKysIrVqwIHzhwwFFZZXVr0qRJzLmeeuqpuHQHDx6U5qX/q1KlinKehw8fDofD4XBBQUFcuZs0aRLz+9ixY9K66ceKnhO78oqu05o1axyn0fnoo4+Ex7/00kvSNEeOHBGmueaaa4TH6/WUlW3ZsmXSvMLhcLi0tDTcrl07YdrCwkLLtB4RampaW+ZGzNPHjdj5Cs1BtmTWg9Xnp9U+88gHN5N/rHA79FK0CHVOTg62bdsmnExTUFCADz74wFVeqsg+bd3UUeXLIysrC507d5auEO8mX6Ob5dJLL43bb9VWdZyscqMvUSaqb9hkyVp9/utuKb/Wp7Wyeu2QrRBlro+RmjVrCr+w3E7wscoL0HRFNmIq6IXWRWSMmFvdMLsH0hxbQ9bg3Yq5ed+1115rWR6nuJ3198QTT0j3ya6ZvsahGa/hWHWGDRvmqDxWWIVkVcVpvocPH465H6LY+So4EXNddER+dj1cge42FI3IArQRUbpLzI2Ym8MbAN4iOMr6O+wEVuRqshPzHj16OCqDEVnUTj8DnqmSMWLuZty2jrnxuhFzJ0F53KwtaIUbMd+zZ0/MFGsd/WGRXTPZhKSgl69LxsMBOO/IKigoiFrmXqZsO2kjF1xwAYqKitCgQYO4fW+++SZGjx6N//znPwDi26neBoxLMLppn6IJXV7EXNYB7GTUjI7dPZTNCVFpc8lqlyIyRszdCq0orezmWzUKqwfAPF3ey2esedENQE3MzRbNgAEDLI9LpUbqBX10jyi2jhnRF1P//v0d5ymbSGbELoKjkzaya9cuaUd7w4YNMXHiRKHlDAD/+te/POWtI2r/XkZ0qE7HN+PWMhctDm6eA5DqZIyYe7HMzZ9KspvvJQ8jXsR8yJAhcdtUxPzMM8+MiTdiDvCkY/cZm+qYIzzqEfPchhOwmsmrD401o68oZRV+wS62ulPrWB9T7RTR/Ra1zz/96U+47LLLHJ3HbjSIlZEls8yDcLMA4pEpdpOsUo1yIeZ2N8UcNEl2Lr9WcfEi5u+9917cNpWVgLZu3RqzeIQsjd+WuV/XTLU87dq1E25XGaYomikoi6EOaKEjRIiidJrROy1lOG0j+thtp4jEUXSt33rrLWG8eKvzVKxY0bJcVrNrZZa53UQeUcejShuUfbmkExkj5lZvbJEv0QqrF4MfU/T9Gi2gowc7skO3FI1rkcpwYqFbuTDciLnu3zXi9eWiUp+HH344bpvVBKBTTz1VuP3rr7+2zcvuHsgscyeTZlTw60tM5su+4YYbpGmMER7NyMa6ixYCMdK6deu4bSptJxWjIDolY8Rcht2nngirG9unTx8vxQGg/gm9YMECpdEQKisnGZEF6wd+r7uTVXmMq+qYcSPCxsW3dYYPH66cfvHixXHbVF4qIkvRaiamaowQEXZhDmSipRq9cuXKlUrHrVq1Sjh8UobT2cRWURtF91mnT58+GDduXNx2u/soKofXBbnThYwRc1lj+tvf/ub4XEHffFUx79mzJ5YuXap0nAiRqB06dMjyZaVPLRdZODKs1jt1cy1FD6zekamC6BPd7T11ch2cYOczHzFihHC7XRx+HWP8cDtUhR+Ql1v2/DVq1MhyCKyMihUrxs3/sItECrgbzZIpZLyYDxw40PG5gr75fr8sZNbvNddcE7dt+PDhQpEHNL+oG0vaylrSA385QVQGJ+UShS5VdfeYY/U4ddGpYlef6tWrS9N5/Tq0sojtkD1nVu4aJ8GxrFC5hyIxd/O8paMPPWPEXIabG2kn5k5dN25jees4+Qy2Y+HChdHVYcy49eVbCZPoq8HKzSOjQ4cOyseK7p+qb/j1119XzscLVtesT58+ljMI582b58nX/dJLL7lOK3vJuF1Jyg6nVr2oj8ONBsjcSSqY1xxOFBkj5l47coxT7O2CLi1btszRuc8444zo3z///LOjtAAwePBgx2lkWPmA/Z7MJEPVVWBENtNOhMg6swriZaRly5bK+djhtrNcFGbBT5xEvTQjW3TBSfA1Jxi/QlS+zkRi3rx5c8f5ytaGDTqtF3z93meMPQvgEgDbAQzknKtFevcB4wozbjDecLuV452sq6jj5WXj59jvVBBzO3+xV0STf1RnqDpZjMIOt/52q1EefmC1WAPn3DJttWrVUK9evZjJa4sXL5auhCRDFgvHCpWJX1dffXXcHArZ8nR+sXnz5pjVvVR8+0Hgm2XOGOsAIIdz3hnATwDip1QFyKxZs+K2OfHXXnDBBahcuTJuueUW28+yRPeOJ2oij/kB+/DDD/Hqq6/GxIb3A9mnupHnn38ef/rTnzB27FjHbhnzQ//oo48q+8z9FHOrevoxjv/iiy92lc4qb9lwSyPmfiiVwGHmyVyqIY2NZVUxokT+edV7qn+dO11u8IwzzoipX7JGz/iZ6yUAPov8vQTAAADv+Hh+S9q1axfTK3/22WfjlltuUU7fvHlz5ObmKglNonvHzz///EDP379/fzRt2jRuPUp9jLCXz3JA83fr1pKVVWjET+vGiUAb7+3ZZ59te/w333wjXbzXyWLKRm6//Xal45YtW4bt27e7clvt2LFDKNwqHb7du3ePGSWmssCLeTKXaseyUcxVrPlq1arhoosuwldffaV0fiMPPvggRo4c6arvqHXr1ujYsWPc+ruJxE8xrwtAn1aYDyDudc0YGwxgMKBFxzOuhuOVLl26xIj5ZZddhlAoFHNMSUlJ3DYz5jgqIoyjJbp164Z+/frZntcLtWvXxmeffYbGjRujY8eOMdHcOnToYJn3ueeeaxs9cOTIkWjYsCF27dolPWbcuHHCBTNuueUW27rPmDEDkyZNQps2bdC3b1/p8Sr3xw3Nmzd3dd7XXnvNNl3Tpk2xdetW7N27F5988gmefvrp6D6r+ugvtVatWmH69Okx8wnGjh2rXF6R8bFixQrb9BUqVMCFF14YN8HJKp1en7Zt28YIpihekB2q9TO+iE877TSldJMmTYqO2HnjjTcS1t7mz5+PrKysQLUAsFg8RBbo3Om/jh07Du3YsePtkb87duzY8SWbNL5SVFQU/sc//hH++uuvwy+//HI4Pz8/7hjZ4hRuWLJkSXj58uW+nU+V1atXh6dPnx5+/vnnwwMHDgzv27fP8vi8vLxwly5dwp999ll4zJgx4Ycffjh82223hdu2bRsePXp0eOHChUr5Hj9+PPzSSy+F69WrFw3A3717d1+D8Pt5f3755ZfwTTfdFH7xxRfDZWVljtIuX748/MknnzjOs6SkJDxq1KjwVVddFf7hhx9s67N58+bwsWPHwuGwttDBihUrwkePHnWc77fffhu+5JJLwgDC3377rXK6AwcOhCdNmhSeM2dOeOTIkeETJ05YHm+uT35+fri0tFQ5v/z8/PDMmTMdt5n9+/eHt23b5ijNf/7zn/CiRYss772f7S3BCDU1K+yTP5Yxdi6A+znntzPGHgHwC+fcys2S8IhOoVDI07qcqQbVJ7Wh+qQ2aVwfYaeHbx2gnPO1APYyxr4A0A7A+36dmyAIgrDG125Xzrm7OKMEQRCEJzJm0hBBEER5hsScIAgiAyAxJwiCyABIzAmCIDIAEnOCIIgMwLdx5gRBEETyIMucIAgiAyAxJwiCyABIzAmCIDIAEnOCIIgMgMScIAgiAyAxJwiCyABIzAmCIDKAjBRzxlijZJfBTxhjdZNdBj/JpPpkYFurEfnf+yKlKUAmtTU7MmrSEGPsAgBPA9gP4F8AVnDODyW1UB5gjF0G4AEABwC8DOB/nPPjyS2VezKpPhnY1q4CcBeAXQCe5ZzL1xBMAzKpramSaZb5jQBmARgLoA2Ae5JbHM/cAuBNaAtjXwvg5uQWxzOZVJ8bAcxG5rS12wD8A8AGAHczxjonuTxeyaS2pkRaizlj7CTG2BTG2HWRTTsAVOGcbwPwNoBWEQsqLWCMVWeMdWGM6cum/wZgD4ClAJYBOIcx1jpZ5XMKY6wKY2wAY6xDZNMOaIt+p119IvdmGGOsS2TTVqR/WzuPMVaPMVYR2n35BpoxtBlAR8ZY06QW0gERLZgYscgB4BekaVtzS9qKOWOsMYDp0G7a55HN+wFUY4w14ZzvjGy/OElFdARjrC2A1dAsvjcjD1IWgGaRQzZCE8MzklNCV1wE4CkA10d+nwBwOrT1X9OmPoyx8wCsBNAKwB0RQT8Cra01TsO21g/AlwDuBPBPznkpgJoALuacFwP4HkA1ALWTV0p1GGOnQLPA90J7hgDt2TkNadbWvJB2Yh4RPQAoAfATgK8APMYYux6asNcDwCLH5AIoiKRLyQ4dxpi+dF8tAG9wzv8C4GNoAvgbgE4A2nLOD0J7wE6KpEv1+gDAJmhrwVZjjF0BYBGALkiT+hjq0gTALM75cACroIn6fwE0AnB+5JiUb2sAwBirDeBcAP0550MBhBhjN0ETw4GMseqc8w0AmgNokbSCOqMSgIXQ7slwxtglAD4FcAmAdunQ1vzA1zVAgyTy9n0YwM2MsZ4A8gCcCqAvNCvjRgBfQBPA8xljp0F70DYAAOc8pXp6GWPNAYwHsIUxthBAdWi+V0Dz9d0HYDmAGgB6R15i5wI4DKR8fZZyzr+Adn/qQXvQenDORzPGVgO4iTF2NlK0Poa6bGWMfQggH0ADxtgtAK4BcBzAWgD/0w5np0MzIFK1rZ0K4EFoRsL3nPMHDYK2AUAR53xV5LkaHrlH1QBUTE6JrTHUZyG08tcHcBW0heTXQPsa/Cs0V9hNqf7s+EVaWOaMsVsBLID2qfQigCzO+RYAdQBU45y/G9l+KoDvAPwdQF0AKznnzyahyJZEhrNNgvYS+gbADM75fwG0ZoxdxDnPA/ADgJs5538H8C6AywB8yzn/R7LKLcNUn6+gub8AYB2A9dBEr5AxNhnAFmjWehekYH1Mdfka2tfSlwBmALgdwI8AZgLoDeAgtLZWB6nb1poBmAzgVwANAbwBxAjaafjdqHsW2v0bAuAHzvmixJbWHlN9cgC8wjnn0AS9mHP+dmT/VQDmAPgAKfzs+ElKi7mhs2k5gCs555Og3cCzI9vfBlCLMXYG53w9NMu2QmRY1dMRIUwZIp2bFaD58c7inP+dc74Y2qc7ADwP4BEA4Jy/D6ARY6xR5LN3BOd8WlIKLkFSnyUAPmOM1YHmhjgFQEcA3QH8H4BjnPONSLH6WNyb5YyxBtA6ovdzzh/jnP8bmrFQxDnfjdRta7r1XYdzPplzPgdAZcbYY5FjTgZwDMDHjLE+AJpxzpdDc8G8kJySi5HUZxaA2oyxO6F9SV0AAJE22DpyXEo+O0GQkuPMGWM1AXwIoCuAXgA+jXTSgDF2NYBhnPPrI7/vAdAY2o3cC+AhaA9dylTMVJ8bOeeLGGPjoVlENwMIQRvfexeAvwCoDM2a+BrA45zzomSUW4ZCfXZCG0lwF7ThblUBTIH22b4WQG6q3B8HdRkMTTByAVwIzc03mnO+IxnllmGqzw3QRnM8DGAP5/xVxthYaK6ia6G5HVZG/i8CcD/nfHMSii1FoT5PAOgB4FIATwAoi/y9Bdr9yU9CsZNCSoo5ADDGOkIbyXExgBc553si288A8Gdon1d7I2/r2gA6RKyKlMRQn0sBjOecH2KMDYBmDT0dEZAKnPMxEZ9g24iFkZIo1GcCgFIAY1NFuGUo3pswgMehfWU045zPT1qBbYjU5xRoBs7L0NxALwPYDuAogEIAS6C5LT8GMDIN2ppVfYoAzIc2kqURtE7Pz5JQ1KSSsh2gnPPvAHwXGQVxBWPsfcMMrg6IdGZA858fguaKSVlM9fk/aP686vh9NMQxAMWMscoRay+lLD4zCvUpBFAMrY2VMMYqcM7LklJYGxTvTQmAihH/LE9KQRUx1KcbgMs5528xxvoCOBmaxXofgB8jLqJWySupGor1+ZlzfgLaV24oaYVNIinrMzf4x94G0A3ajUOk47MIwBWR3ykpEGZM9enKGMvmnL8MIDcyYqIDtM62kqQV0gFO65PK90mxLq9HxCLlMdXncsZYw4hw14bWmd4cwOFIH0HKo1ifo5k65FCVlHWzGGGMDQfQEpo/bBaATZzzY8ktlXsYYyOgTWDIhfZpuIFznrbWRCbVJ5PqAsQ8O3nQhupt4px/k9xSuSfT6uMnKf1mNlgO7aF12qzhnK9LVyE31OccaJ02Wzjnn6arWGRSfTKpLoDw2dnGOX8rXYUv0+oTBOlimd8MYFGqjepwC9UndcmkugBUn/JEWog5QRAEYU1Ku1kIgiAINUjMCYIgMgASc4IgiAyAxJwgCCIDIDEnCILIAFJ2Oj9BuIEx1gLaIiWAFhfm6cj21wEMBADOuauZgpG42H0ALOOcL4tsmwmgP4DzI1P9CSIpkJgTmcwdjLFnoC3w0ceH87WFFpkP0NaVJIiUgcaZExmFwTLfBm290W7QFmB4BcABaPHwKwB4FFqI3nrQAmcN45z/jzH2JDTB/ge0sKt1AAwF8C1+t/h1LgdwBzTLfDK0BSsqALiNaystEUTCIJ85kan8CC0e/MDIv/kADkX2DQDwNLTVnB6FFh1xAWOssiF9ZwAvQQvm9Fdoi4W/GNn3PrTlCjcajr8EwGvQQuk+6W9VCMIeEnMik3kDmrXcCdq6qjrXRv6/n3M+FdqShC0BnGU45nnO+RRoFn4LznkBtIUcAC341lzO+T7D8U9yzp+BFtGzhe81IQgbSMyJTGYutAUydgL4t2B/2PS/kdzI/yfw+3Ni5ZM0Hp+SCyETmQ2JOZGxcM4PQ3OxDDHFU/848v/zkZCqvRAJp2pzyrzI/50ZY7cyxk7ytcAE4QEazUJkNJzzeYLNM6F1hN4FrYP0W2gdoCWMMavTfQltDcoukXSn+FpYgvAAjWYhCILIAMjNQhAEkQGQmBMEQWQAJOYEQRAZAIk5QRBEBkBiThAEkQGQmBMEQWQAJOYEQRAZAIk5QRBEBvD/Ne3r8kkR1M0AAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "series_sunspot = SunspotsDataset().load().astype(np.float32)\n", "\n", "series_sunspot.plot()\n", "check_seasonality(series_sunspot, max_lag=240)\n", "\n", "train_sp, val_sp = series_sunspot.split_after(pd.Timestamp(\"19401001\"))\n", "\n", "scaler_sunspot = Scaler()\n", "train_sp_scaled = scaler_sunspot.fit_transform(train_sp)\n", "val_sp_scaled = scaler_sunspot.transform(val_sp)\n", "series_sp_scaled = scaler_sunspot.transform(series_sunspot)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"the 'monthly sun spots' dataset has 2820 data points\"" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "f\"the 'monthly sun spots' dataset has {len(series_sunspot)} data points\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, let's perform one-step ahead forecasting." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "my_model_sp = TransformerModel(\n", " batch_size=32,\n", " input_chunk_length=125,\n", " output_chunk_length=36,\n", " n_epochs=20,\n", " model_name=\"sun_spots_transformer\",\n", " nr_epochs_val_period=5,\n", " d_model=16,\n", " nhead=4,\n", " num_encoder_layers=2,\n", " num_decoder_layers=2,\n", " dim_feedforward=128,\n", " dropout=0.1,\n", " random_state=42,\n", " optimizer_kwargs={\"lr\": 1e-3},\n", " save_checkpoints=True,\n", " force_reset=True,\n", ")" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "ccd2a30ded334bc2a68338f7a77bc6d0", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/20 [00:00