TF4AML_theory.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ou5yCudRrIxU"
   },
   "source": [
    "# Initial setup\n",
    "Only if you run from Google Colab run those 2 cells to sync with Google Drive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Yq6D6FflrIxX",
    "outputId": "572af3d1-b98b-46ac-925b-04c044a27621"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mounted at /content/drive\n"
     ]
    }
   ],
   "source": [
    "from google.colab import drive\n",
    "drive.mount('/content/drive', force_remount=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "QdAwj0gqrIxY",
    "outputId": "cf23cfd8-05ab-41b1-fd50-3ce6b53aa5ff"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/content/drive/MyDrive/TF4AML\n"
     ]
    }
   ],
   "source": [
    "%cd /content/drive/MyDrive/TF4AML/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Gco0dDNkMa8p"
   },
   "source": [
    "# The Transformer Architecture\n",
    "\n",
    "This notebook will guide you through the building blocks of the Transformer model, one of the last two years' most influential architecture. The Transformer design has continued to outperform benchmarks since the publication of the Vaswani et al. paper [Attention Is All You Need](https://arxiv.org/abs/1706.03762) in 2017, particularly in the field of natural language processing. Transformers with an enormous number of parameters may generate long, compelling writings, opening up new AI application sectors. It is crucial to understand how the Transformer architecture works and to have developed it yourself, which we will do in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "y3nLcU1iSAey"
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch.nn.functional import softmax\n",
    "import torch.nn as nn\n",
    "from torch.autograd import Variable\n",
    "import torch.nn.functional as F\n",
    "\n",
    "import copy\n",
    "from copy import deepcopy\n",
    "import numpy as np \n",
    "import math\n",
    "import scipy.io\n",
    "import os\n",
    "import random\n",
    "\n",
    "\n",
    "## Imports for plotting\n",
    "import matplotlib.pyplot as plt\n",
    "plt.set_cmap('cividis')\n",
    "%matplotlib inline \n",
    "from IPython.display import set_matplotlib_formats\n",
    "from matplotlib.colors import to_rgb\n",
    "import matplotlib\n",
    "matplotlib.rcParams['lines.linewidth'] = 2.0\n",
    "import seaborn as sns\n",
    "sns.reset_orig()\n",
    "\n",
    "torch.manual_seed(0)\n",
    "torch.cuda.manual_seed(0)\n",
    "np.random.seed(0)\n",
    "random.seed(0)\n",
    "\n",
    "torch.backends.cudnn.deterministic=True\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9mDczNrerIxa"
   },
   "source": [
    "### The Attention Mechanism\n",
    "\n",
    "In the last several years, particularly in sequence tasks, a new collection of layers in neural networks known as the attention mechanism has garnered a lot of attention. The attention mechanism defines a weighted average of (sequence) items, with the weights dynamically computed based on an input query and the elements' keys. So what precisely does this mean? The objective is to average out the features of multiple elements. We want to weight the elements according to their actual values rather than giving each equal weight.\n",
    "\n",
    "An attention mechanism, in particular, usually has four components that must be specified:\n",
    "\n",
    "* **Query**: The query is a feature vector that identifies the elements in the sequence that we are searching for, or perhaps want to pay attention to.\n",
    "* **Keys**: We have a key, which is also a feature vector, for each input element. This feature vector essentially explains what the element \"offers\" or when it might be significant. The keys should be created so that, based on the query, we can identify the elements we wish to pay attention to.\n",
    "* **Values**: There is also a value vector available for each input element. Specifically, we want to average over this feature vector.\n",
    "* **Score function**: A scoring function called $f_{attn}$ must be specified in order to rate the items that we want to pay attention to. The query and a key are entered into the score function, which returns the attention weight and score for the query-key pair. It is typically used using straightforward similarity metrics, such as a dot product or a small MLP.\n",
    "\n",
    "A softmax is used to calculate the average's weights across all scoring function outputs. As a result, we give greater weight to value vectors whose corresponding key is most comparable to the query.\n",
    "\n",
    "\n",
    "$$\n",
    "\\alpha_i = \\frac{\\exp\\left(f_{attn}\\left(\\text{key}_i, \\text{query}\\right)\\right)}{\\sum_j \\exp\\left(f_{attn}\\left(\\text{key}_j, \\text{query}\\right)\\right)}, \\hspace{5mm} \\text{out} = \\sum_i \\alpha_i \\cdot \\text{value}_i\n",
    "$$\n",
    "\n",
    "Here is an example of attention over a sequence:\n",
    "\n",
    "<center width=\"100%\" style=\"padding:25px\"><img src=\"https://github.com/phlippe/uvadlc_notebooks/blob/master/docs/tutorial_notebooks/tutorial6/attention_example.svg?raw=1\" width=\"750px\"></center>\n",
    "\n",
    "Every word has a key and a value associated. The score function compares the query to all the keys to compute the weights. The values of the words are averaged by the attention weights. \n",
    "\n",
    "Self-attention is the term for the attention applied inside the Transformer architecture. Each sequence element in self-attention gives a key, value, and query. We execute an attention layer for each element, which checks the similarity of the keys of all the sequence members based on the element's query and returns a unique, averaged value vector for each element. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "F8OMIwoXrIxc"
   },
   "source": [
    "### Scaled Dot Product Attention  (***1 POINT***)\n",
    "\n",
    "Scaled dot product attention is the central idea of self-attention. Our objective is to develop an attention mechanism that allows each element in a sequence to pay attention to any other while maintaining compute efficiency.The input to the dot product are the queries $Q\\in\\mathbb{R}^{T\\times d_k}$, keys $K\\in\\mathbb{R}^{T\\times d_k}$ and values $V\\in\\mathbb{R}^{T\\times d_v}$ with $T$ as the sequence length, $d_k$ and $d_v$ are the hidden dimensionality for $Q$, $K$ and $V$.\n",
    "\n",
    "We compute the dot product attention as follows:\n",
    "\n",
    "$$\\text{Attention}(Q,K,V)=\\text{softmax}\\left(\\frac{QK^T}{\\sqrt{d_k}}\\right)V$$\n",
    "\n",
    "The matrix multiplication $QK^T$ produces a matrix with dimension $T\\times T$ by doing the dot product for every distinct pair of queries and keys. The attention logits for a particular element $i$ to every other element in the sequence are shown in each row. We use a softmax on these and multiply by the value vector to get a weighted mean (the weights being determined by the attention). The computation graph below provides another viewpoint on this attention technique.\n",
    "\n",
    "<center width=\"100%\"><img src=\"https://github.com/phlippe/uvadlc_notebooks/blob/master/docs/tutorial_notebooks/tutorial6/scaled_dot_product_attn.svg?raw=1\" width=\"210px\"></center>\n",
    "\n",
    "$1/\\sqrt{d_k}$, the scaling factor, is crucial to maintain an appropriate variance of attention values after initialization. As a result, $Q$ and $K$ may also have a variance of close to $1$. Keep in mind that we initialize our layers with the purpose of having equal variance across the model. Dot products over two vectors with variances $\\sigma^2$, however, produce scalars with $d_k$-times larger variance: \n",
    "\n",
    "$$q_i \\sim \\mathcal{N}(0,\\sigma^2), k_i \\sim \\mathcal{N}(0,\\sigma^2) \\to \\text{Var}\\left(\\sum_{i=1}^{d_k} q_i\\cdot k_i\\right) = \\sigma^4\\cdot d_k$$\n",
    "\n",
    "\n",
    "The optional masking of particular entries in the attention matrix is shown by the block labeled \"Mask (opt.)\" in the diagram above. When calculating the attention values, we pad the sentences to the same length and mask out the padding tokens.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LkmdPP_crIxc"
   },
   "source": [
    "Following the description we just made about scale-dot product, complete the Attention class below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "ZgYad7IHKEkY"
   },
   "outputs": [],
   "source": [
    "class Attention(nn.Module):\n",
    "    ''' Scaled Dot-Product Attention '''\n",
    "\n",
    "    def __init__(self, attn_dropout=0.1):\n",
    "        super().__init__()\n",
    "        self.dropout = nn.Dropout(attn_dropout)\n",
    "\n",
    "    def forward(self, query, key, value, mask=None):\n",
    "\n",
    "        '''\n",
    "        add here the code regarding the argument of the softmax function as defined above\n",
    "        '''\n",
    "        d=query.size(-1)\n",
    "        attn=torch.matmul(query, key.transpose(-2, -1))/(math.sqrt(d))\n",
    "        \n",
    "        if mask is not None:\n",
    "            attn = attn.masked_fill(mask == 0, -1e9)\n",
    "\n",
    "        attn = self.dropout(F.softmax(attn, dim=-1))\n",
    "        \n",
    "        '''\n",
    "        Computed attn, calculate the final output of the attention layer\n",
    "        '''\n",
    "        output=torch.matmul(attn, value)\n",
    "        return output, attn\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KjPYD5CNKEkZ"
   },
   "source": [
    "Do not modify the code below. \n",
    "\n",
    "The following example about the Attention function you just implemented does not include the mask, which instead will be used later in the MultiHeadAttention class. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "epHmxXrorIxe"
   },
   "source": [
    "We generate here some random $Q$ $K$ and $V$ to procude some attention outputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PuV0ZKfKKEkZ",
    "outputId": "0e77d74e-a92b-474f-c93e-d2569a746ef5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Q\n",
      " tensor([[ 1.5410, -0.2934],\n",
      "        [-2.1788,  0.5684],\n",
      "        [-1.0845, -1.3986]])\n",
      "K\n",
      " tensor([[ 0.4033,  0.8380],\n",
      "        [-0.7193, -0.4033],\n",
      "        [-0.5966,  0.1820]])\n",
      "V\n",
      " tensor([[-0.8567,  1.1006],\n",
      "        [-1.0712,  0.1227],\n",
      "        [-0.5663,  0.3731]])\n",
      "Values\n",
      " tensor([[-0.9328,  0.8123],\n",
      "        [-0.9093,  0.3966],\n",
      "        [-0.9970,  0.3056]])\n",
      "Attention\n",
      " tensor([[0.6291, 0.2395, 0.2425],\n",
      "        [0.1387, 0.4749, 0.4975],\n",
      "        [0.0842, 0.6800, 0.3469]])\n"
     ]
    }
   ],
   "source": [
    "torch.manual_seed(0)\n",
    "\n",
    "seq_len, d_k = 3, 2\n",
    "q = torch.randn(seq_len, d_k)\n",
    "k = torch.randn(seq_len, d_k)\n",
    "v = torch.randn(seq_len, d_k)\n",
    "attention = Attention()\n",
    "values, attn = attention(q, k, v)\n",
    "print(\"Q\\n\", q)\n",
    "print(\"K\\n\", k)\n",
    "print(\"V\\n\", v)\n",
    "print(\"Values\\n\", values)\n",
    "print(\"Attention\\n\", attn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Vu3ZM3RPrIxf"
   },
   "source": [
    "### Multi-Head Attention  (***2 POINTS***)\n",
    "\n",
    "A network can pay attention to a sequence thanks to scaled dot product attention. A single weighted average, however, is not a viable choice for a sequence element because it frequently wishes to pay attention to several separate characteristics. For this reason, we expand the attention methods to include numerous heads, or several query-key-value triplets on the same features. We specifically change a query, key, and value matrix into $h$ sub-queries, sub-keys, and sub-values, which we then independently pass through the scaled dot product attention. The heads are then combined with a final weight matrix by concatenating them. \n",
    "\n",
    "$$\n",
    "\\begin{split}\n",
    "    \\text{Multihead}(Q,K,V) & = \\text{Concat}(\\text{head}_1,...,\\text{head}_h)W^{O}\\\\\n",
    "    \\text{where } \\text{head}_i & = \\text{Attention}(QW_i^Q,KW_i^K, VW_i^V)\n",
    "\\end{split}\n",
    "$$\n",
    "\n",
    "We refer to this as Multi-Head Attention layer. We can visually see it here:\n",
    "\n",
    "<center width=\"100%\"><img src=\"https://github.com/phlippe/uvadlc_notebooks/blob/master/docs/tutorial_notebooks/tutorial6/multihead_attention.svg?raw=1\" width=\"230px\"></center>\n",
    "\n",
    "Set the feature map, $X\\in\\mathbb{R}^{B\\times T\\times d_{\\text{model}}}$, as $Q$, $K$ and $V$ (with $B$ as the batch size, $T$ the sequence length, $d_{\\text{model}}$ the hidden dimensionality of $X$). The weights $W^{Q}$, $W^{K}$, and $W^{V}$ can transform $X$ to the corresponding queries, keys, and values of the input. The final result is produced by multiplying the concatenated output by the weight matrix $W^{0}$\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "H21t3FU3rIxf"
   },
   "source": [
    "Complete the MultiHeadAttention class below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "kN18u_LLrIxg"
   },
   "outputs": [],
   "source": [
    "class MultiHeadAttention(nn.Module):\n",
    "\n",
    "    def __init__(self, num_heads, d_model, dropout=0.1):\n",
    "        \"\"\"\n",
    "        Take in model size and number of heads.\n",
    "        \"\"\"\n",
    "        super(MultiHeadAttention, self).__init__()\n",
    "        assert d_model % num_heads == 0\n",
    "        #  We assume d_v always equals d_k\n",
    "        self.d_k = d_model // num_heads\n",
    "        self.num_heads = num_heads\n",
    "        self.query_ff = nn.Linear(d_model, d_model)\n",
    "        self.key_ff = nn.Linear(d_model, d_model)\n",
    "        self.value_ff = nn.Linear(d_model, d_model)\n",
    "        self.attn_ff = nn.Linear(d_model, d_model)\n",
    "        self.attn = None\n",
    "        self.dropout = nn.Dropout(p=dropout)\n",
    "        self.attention = Attention(attn_dropout=dropout)\n",
    "\n",
    "    def forward(self, query, key, value, mask=None, return_attention=False):\n",
    "\n",
    "        if mask is not None:\n",
    "            # Same mask applied to all h heads.\n",
    "            mask = mask.unsqueeze(1)\n",
    "        nbatches = query.size(0)\n",
    "        \n",
    "        # 1) Do all the linear projections in batch from d_model => h x d_k. \n",
    "        # The query is given as example, you should do the same for key and value  \n",
    "        query = self.query_ff(query).view(nbatches, -1, self.num_heads, self.d_k).transpose(1, 2)\n",
    "        '''\n",
    "        add your code here\n",
    "        '''\n",
    "        key = self.key_ff(key).view(nbatches, -1, self.num_heads, self.d_k).transpose(1, 2)\n",
    "        value = self.value_ff(value).view(nbatches, -1, self.num_heads, self.d_k).transpose(1, 2)\n",
    "        # 2) Apply attention on all the projected vectors in batch.\n",
    "        '''\n",
    "        add your code here\n",
    "        '''        \n",
    "        x, attn = self.attention(query, key, value, mask)\n",
    "        # 3) \"Concat\" using a view and apply a final linear.\n",
    "        x = x.transpose(1, 2).contiguous().view(nbatches, -1, self.num_heads * self.d_k)\n",
    "\n",
    "        if return_attention:\n",
    "            return self.attn_ff(x), self.attn\n",
    "\n",
    "        return self.attn_ff(x)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ckj-EqTSrIxg"
   },
   "source": [
    "Do not change the following code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "pFw-_-bVrIxg",
    "outputId": "0fe57e56-d6d6-45b3-9a5a-7cc18a0e4647"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 0.1963, -0.0791,  0.0207,  ..., -0.1688,  0.1161,  0.1932],\n",
       "         [ 0.1876, -0.0731,  0.0135,  ..., -0.1666,  0.1163,  0.1807],\n",
       "         [ 0.1647, -0.0816,  0.0357,  ..., -0.1614,  0.0965,  0.2349],\n",
       "         ...,\n",
       "         [ 0.2222, -0.1297,  0.0181,  ..., -0.1854,  0.0964,  0.2300],\n",
       "         [ 0.1962, -0.1470,  0.0295,  ..., -0.2274,  0.0817,  0.2286],\n",
       "         [ 0.2229, -0.1208,  0.0031,  ..., -0.2206,  0.1585,  0.2397]]],\n",
       "       grad_fn=<ViewBackward0>)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.manual_seed(0)\n",
    "np.random.seed(0)\n",
    "\n",
    "num_heads = 8\n",
    "d_model = 512\n",
    "\n",
    "\n",
    "self_attn = MultiHeadAttention(num_heads, d_model)\n",
    "\n",
    "\n",
    "x = torch.tensor(np.random.rand(1, 7,512)).float()\n",
    "\n",
    "attn_out = self_attn(x, x, x)\n",
    "attn_out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "06ntDsDvrIxh"
   },
   "source": [
    "The fact that the multi-head attention is permutation-equivariant with respect to its inputs is an important aspect of the framework. This means that if we switch the first and second items of the input sequence, the output will still be precisely the same. As a result, the multi-head attention really considers the input as a collection of items rather than a sequence. This characteristic is what makes the Transformer architecture and the multi-head attention block so potent and versatile."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DmOVPvlOrIxh"
   },
   "source": [
    "### The Encoder-Decoder Blocks (4 POINTS)\n",
    "\n",
    "The paper's original transformer is a neural machine translation model. We could teach it to translate a sentence, for instance, into French. Encoder-decoder architecture is used by the transformer. An input sentence is processed by the encoder to extract features, which are then used by the decoder to create an output sentence (translation). The full Transformer architecture looks as follows (figure credit - [Vaswani et al., 2017](https://arxiv.org/abs/1706.03762)).:\n",
    "\n",
    "<center width=\"100%\"><img src=\"https://github.com/phlippe/uvadlc_notebooks/blob/master/docs/tutorial_notebooks/tutorial6/transformer_architecture.svg?raw=1\" width=\"400px\"></center>\n",
    "\n",
    "Let's examine the Encoder block in more depth. The Decoder is easy to use once you comprehend this.\n",
    "\n",
    "The encoder is made up by $N$ sequentially applied identical blocks. Using $x$ as the input, a Multi-Head Attention block is first applied. Using a residual connection, the output is added to the original input, and the sum is then subjected to a subsequent layer normalization. Formally, it computes:\n",
    "\n",
    " $\\text{LayerNorm}(x+\\text{Multihead}(x,x,x))$ ($x$ being $Q$, $K$ and $V$ input to the attention layer). \n",
    " \n",
    "To enable a smooth gradient flow across the model, the residual connections are essential. Furthermore, the information about the original sequence is lost in the absence of residual connections.\n",
    "\n",
    "Faster training is made possible by the layer normalization, which also offers minor regularization. Additionally, it makes sure that each element in the sequence has features that are of a similar magnitude.\n",
    "\n",
    "A small fully connected feed-forward network is also included to the model in addition to the Multi-Head Attention, and it is applied to each position individually and identically. One way to describe the complete transformation, including the residual connection, is as follows:\n",
    "\n",
    "$$\n",
    "\\begin{split}\n",
    "    \\text{FFN}(x) & = \\max(0, xW_1+b_1)W_2 + b_2\\\\\n",
    "    x & = \\text{LayerNorm}(x + \\text{FFN}(x))\n",
    "\\end{split}\n",
    "$$\n",
    "\n",
    "In addition to the layers we just discussed, a dropout layer is added in the MLP, on the output of the MLP and Multi-Head Attention as regularization. \n",
    "\n",
    "Regarding the masking in the decoder block, the target mask is needed since the transformer decoder has all the target tokens as input during training. However, it should produce output related to what happened before the current timestep, and that's why the target mask is introduced. \n",
    "\n",
    "Knowing what the target mask is used for, you should now be able to add it in the correct attention of the decoder. (hint: the two masks are used separately and there is no need to change them.) \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OSZUPctarIxh"
   },
   "source": [
    "Add your solution to the EncoderBlock and DecoderBlock classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "m88t1YecrIxh"
   },
   "outputs": [],
   "source": [
    "class EncoderBlock(nn.Module):\n",
    "\n",
    "    def __init__(self, input_dim, num_heads, dim_feedforward, dropout=0.0):\n",
    "        \"\"\"\n",
    "        Inputs:\n",
    "            input_dim - Dimensionality of the input\n",
    "            num_heads - Number of heads to use in the attention block\n",
    "            dim_feedforward - Dimensionality of the hidden layer in the MLP\n",
    "            dropout - Dropout probability to use in the dropout layers\n",
    "        \"\"\"\n",
    "        super().__init__()\n",
    "\n",
    "        # Attention layer\n",
    "        self.self_attn = MultiHeadAttention(num_heads, input_dim)\n",
    "\n",
    "        # Two-layer MLP\n",
    "        self.linear_net = nn.Sequential(\n",
    "            nn.Linear(input_dim, dim_feedforward),\n",
    "            nn.Dropout(dropout),\n",
    "            nn.ReLU(inplace=True),\n",
    "            nn.Linear(dim_feedforward, input_dim)\n",
    "        )\n",
    "\n",
    "        # Layers to apply in between the main layers\n",
    "        self.norm1 = nn.LayerNorm(input_dim)\n",
    "        self.norm2 = nn.LayerNorm(input_dim)\n",
    "        self.dropout = nn.Dropout(dropout)\n",
    "\n",
    "    def forward(self, x, mask=None): \n",
    "        # Self_attention part (use self.norm1)\n",
    "        '''\n",
    "        Add your code here\n",
    "        '''\n",
    "        attn_1 = self.self_attn(x,x,x)\n",
    "        norm_1_out = self.norm1(x+self.dropout(attn_1))\n",
    "        # MLP part (use self.norm2)\n",
    "        '''\n",
    "        Add your code here\n",
    "        '''\n",
    "        mlp_out = self.linear_net(norm_1_out)\n",
    "        norm_2_out = self.norm2(norm_1_out+self.dropout(mlp_out))\n",
    "        x = norm_2_out\n",
    "        return x\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "-0wu8WXHrIxi"
   },
   "outputs": [],
   "source": [
    "class DecoderBlock(nn.Module):\n",
    "\n",
    "    def __init__(self, input_dim, num_heads, dim_feedforward, dropout=0.0):\n",
    "        \"\"\"\n",
    "        Inputs:\n",
    "            input_dim - Dimensionality of the input\n",
    "            num_heads - Number of heads to use in the attention block\n",
    "            dim_feedforward - Dimensionality of the hidden layer in the MLP\n",
    "            dropout - Dropout probability to use in the dropout layers\n",
    "        \"\"\"\n",
    "        super().__init__()\n",
    "\n",
    "        # Self Attention layer\n",
    "        self.self_attn = MultiHeadAttention(num_heads, input_dim)\n",
    "        # Attention Layer\n",
    "        self.src_attn = MultiHeadAttention(num_heads, input_dim)\n",
    "        \n",
    "        # Two-layer MLP\n",
    "        self.linear_net = nn.Sequential(\n",
    "            nn.Linear(input_dim, dim_feedforward),\n",
    "            nn.Dropout(dropout),\n",
    "            nn.ReLU(inplace=True),\n",
    "            nn.Linear(dim_feedforward, input_dim)\n",
    "        )\n",
    "\n",
    "        # Layers to apply in between the main layers\n",
    "        self.norm1 = nn.LayerNorm(input_dim)\n",
    "        self.norm2 = nn.LayerNorm(input_dim)\n",
    "        self.norm3 = nn.LayerNorm(input_dim)\n",
    "        self.dropout = nn.Dropout(dropout)\n",
    "\n",
    "    def forward(self, x, memory, src_mask, tgt_mask): \n",
    "        # Self-Attention part (use self.norm1)\n",
    "        '''\n",
    "        Add your code here\n",
    "        '''\n",
    "        attn_1 = self.self_attn(x,x,x,tgt_mask)\n",
    "        norm_1_out = self.norm1(x + attn_1)\n",
    "        # Attention part (use self.norm2)\n",
    "        # Recall that memory is the output of the encoder and replaces x as\n",
    "        # the key and value in the attention layer\n",
    "        '''\n",
    "        Add your code here\n",
    "        '''\n",
    "        attn_2 = self.src_attn(norm_1_out,memory,memory)\n",
    "        norm_2_out = self.norm2(self.dropout(attn_2)+norm_1_out)\n",
    "        # MLP part (use self.norm3)\n",
    "        '''\n",
    "        Add your code here\n",
    "        '''\n",
    "        mlp = self.linear_net(norm_2_out)\n",
    "        norm_3_out = self.norm3(self.dropout(mlp)+norm_2_out)\n",
    "        x = norm_3_out\n",
    "        return x\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ASWMjJ-erIxi"
   },
   "source": [
    "### Positional Encoding  (***2 POINTS***)\n",
    "\n",
    "Position is crucial for deciphering the input words in activities like language understanding. Therefore, the input characteristics can be used to add the positional information. Even if we learned an embedding for every possible place, this wouldn't apply to input sequence lengths with dynamic input. Therefore, using feature patterns that the network can recognize from the features and maybe generalize to longer sequences is a better alternative.\n",
    "\n",
    "Following the solution of Vaswani et al., the positional encoding is defined as: \n",
    "\n",
    "$$\n",
    "PE_{(pos,i)} = \\begin{cases}\n",
    "    \\sin\\left(\\frac{pos}{10000^{i/d_{\\text{model}}}}\\right) & \\text{if}\\hspace{3mm} i \\text{ mod } 2=0\\\\\n",
    "    \\cos\\left(\\frac{pos}{10000^{(i-1)/d_{\\text{model}}}}\\right) & \\text{otherwise}\\\\\n",
    "\\end{cases}\n",
    "$$\n",
    "\n",
    "$PE_{(pos,i)}$ represents the position encoding at position $pos$ in the sequence, and hidden dimensionality $i$. The position information is made up of these values, which are added to the initial input features and concatenated for all hidden dimensions.\n",
    "\n",
    "Complete the initialization of the PositionalEncoding class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "wK9ueR5gKEkb"
   },
   "outputs": [],
   "source": [
    "class PositionalEncoding(nn.Module):\n",
    "    \"\"\"\n",
    "    Implement the PE function.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, d_model, dropout, max_len=5000):\n",
    "        super(PositionalEncoding, self).__init__()\n",
    "        self.dropout = nn.Dropout(p=dropout)\n",
    "\n",
    "        # Compute the positional encodings once in log space.\n",
    "        pe = torch.zeros(max_len, d_model)\n",
    "        '''\n",
    "        Add your code here\n",
    "        '''\n",
    "        for row in range(max_len):\n",
    "          for col in range(d_model):\n",
    "            if col%2 == 0:\n",
    "              pe[row][col] = math.sin(row/(10000**(col/d_model)))\n",
    "            else:\n",
    "              pe[row][col] = math.cos(row/(10000**((col-1)/d_model)))\n",
    "        pe = pe.unsqueeze(0) # the final dimension is (1, max_len, d_model)\n",
    "        self.register_buffer('pe', pe)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = x + Variable(self.pe[:, :x.size(1)], requires_grad=False)\n",
    "        return self.dropout(x)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KVGi5FZoKEkb"
   },
   "source": [
    "We may display the positional encoding below to better comprehend it. We will create a sequence-based image of the positional encoding over concealed dimensions. As a result, each pixel indicates the modification of the input feature that we do in order to encode a certain position. Here's how to do it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 241
    },
    "id": "ey8R8ei5KEkb",
    "outputId": "93745ed6-4b1a-4f46-f770-684c40679a69"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAboAAADgCAYAAAB1laOgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABMMUlEQVR4nO2dd5gdVfn4P282vZFKSCGFEBJ66CWAoUOkK9IFQUAEAX+CNAUEVMQGfkUwhKqhqiDSQwDpEAIkBAJICaQ30rNJNrvv74+Ze885w97Z2XL37m7ez/Pc556575kzZ+q587YjqophGIZhtFRalboDhmEYhlFMbKAzDMMwWjQ20BmGYRgtGhvoDMMwjBaNDXSGYRhGi8YGOsMwDKNFYwNdCRCRy0VkXIr8JBF5phH6cZeIXFfs7dQVETlNRF72lleKyGal7FNDIyIzROSAArK9ReSjlHVTz5+IqIhs3hD9TENErhaRv8flgfF5Kiv2drMiIreKyM9L3Q+jdLQudQeaAyIyA+gDVAKrgCeAH6nqyrq0p6q/8toeDHwOtFHV9bF8PDC+fr1ueahq51L3oTFR1ZeA4aXuR21Q1S+BJnWeVPUHpe6DUVrsjS47h8cP2h2BXYCflbg/RjNCIux+M4wSYDdeLVHV2cCTwDYAInKEiLwvIktF5AUR2TJXV0QuEZHZIrJCRD4Skf3j3/OqHuDF+HtprPLZoxqV3Z4iMklElsXfe3qyF0TkWhF5Jd7OMyLSy5M/JCLz4nVfFJGts+6riJwuItNFZImIPC0igzyZisgPROR/sfxmERFPfma87goR+UBEdox/3zLu89L4uB3hrdNTRB4VkeUi8iYwNNGfvCouVtvdLCKPx9t4Q0SGenUPio/5MhH5i4j8V0S+X2A/24nIjSIyJ/7cKCLtYtl0ETnMq9taRBZ5+7O7iLwa788UERmdODe/FJFXgNVAIbXrSBGZGvf1ARFpH68/WkRmee3tICJvx/v7ANA+sR8Xi8jceB9Or2YffyciX4rI/Fid18Hfjoj8REQWxG18r0BfEZEh8fFcISITAP96Gxyfp9beMbguPkYrReQ/8XkeH5/nSRJpNXLrjxCRCSLyVXz+vuPJCp5zifhj3P9l8fHcxlvvOq+dM0Xkk3gbj4pIP09W8LoWkc3j/V4WXwMPFDpGRhNDVe1TwweYARwQlzcF3geuBbYgUmUeCLQBfgp8ArQlUjnNBPrF6w0Ghsblq4G/e78r0Nrb3mnAy3G5B7AEOIVI1XxCvNwzlr8AfBr3pUO8fL3X1ulAF6AdcCPwrie7C7iuwD4fFe/LlvF2fwa86skVeAzoBgwEFgKHxLJjgdlEb74CbA4Mio/RJ8Dl8THaD1gBDI/Xux94EOhE9Edidu44eNvc3Ov7V8Cucf/GA/fHsl7AcuCYWHYBUAF8v8C+XgO8DmwM9AZeBa6NZVcC47263wQ+jMv9gcXAGKI/jQfGy729c/MlsHXcjzYFrq03gX7xuZ4O/CCWjQZmxeW2wBfAj+Pj+O14n66L5YcA8+Pj1gm4N3G8bgQejbfRBfgP8GtvO+vj49Am3p/VQPcCx+s14A9E19Q+8Tms9nqOj8EnRH9aNgI+AD4GDoiPyT3AnXHdTkT3zPdi2Y7AImDrDOf8YGAy0fUoRNdt3+R1TnTNLYrbbgf8H/Bixuv6PuCK+Fy3B/Yq9bPJPhmf4aXuQHP4xA+jlcDS+GHzF6JB5efAg169VkQP59FED/cF8Q3dJtHe1YUeDPFvp+EGulOANxPrvwacFpdfAH7myX4IPFVgP7rF29ooXs4/AKqp+yRwRmLfVgOD4mX1b3SiAerSuPw0cEE1be4NzANaeb/dFx+PMqIH9whP9ivSB7pxnmwMbgD6LvCaJxOiB2ihge5TYIy3fDAwIy5vTvQg7xgvjweujMuXAH9LtPU0cKp3bq7JcG2d7C3fANwal0fjBrp9gDmAeHVfxT3A7yD8g7NF7njF+7+K+I9WLN8D+NzbTjnhNbgA2L2a/g4kGhQ7eb/dS/pAd4VX9/fAk97y4cR/voDjgJcS2/srcFWGc74f0QC6O971lbzOgduBGzxZZ6LrbnCG6/oeYCwwIMtzwz5N52Oqy+wcpardVHWQqv5QVcuJ/oV/kaugqlVED9T+qvoJcCHRQ3yBiNzvq0hqQbCNmC+I3iZyzPPKq4mdAUSkTESuF5FPRWQ50UMVPFVTCoOAm2KV3FKif9KSZbtEb72fFtiXmfFxSu5Lb6J/6TMTsjQKbb+f345GT6lZFCZ5jL+IfyM+j9OBw0WkI3AE0YMdomN0bO4YxcdpL6Cv15a/P7Xdj2QfZ8f74vfTlxc6dr2BjsBkr59Pxb/nWKyxM1SGfixR1VUFtlUd871yeTXLue0MAnZLHM+TgE28+tUeK1V9DvgzcDMwX0TGikjXAv3379mVRG/hWa7rnxLdA29KpHYP1MNG08UGuvoxh+jmBCI7AdFDfjaAqt6rqnvFdRT4TTVt1DR9RLCNmIG5bdTAicCRRG+VGxH924boZq2JmcDZ8eCe+3RQ1Vczrju0mt/nAJtK6JSR25eFRG8KmyZkdWEuMCC3EJ+XAYWrf+0YD4x/y3Efkcr4SOCDePCDaD//ljhGnVT1em/dms5vVuYC/XP2Iq+fvrzQsVtENKBs7fVzI62bF+tcoLuIdCqwrfowE/hv4nh2VtVzsqysqn9S1Z2IVMVbABdXUy15z3YCepLhflLVeap6pqr2A84G/iKNEL5h1B8b6OrHg8A3RWR/EWkD/ARYC7wqIsNFZD+JnBrWED1oKqtpYyFQRWFHhSeALUTkRIkcIY4DtiKyI9REl7g/i4n+0f8qvXrArcBlEjuviMhGInJsxnXHAReJyE6xk8DmEjmyvEGkQvupiLSRyHHjcCI7SyXwL+BqEekoIlsBp9aivz6PA9uKyFGxU8S5hG8FSe4DfiYivSVy5LkS+Lsnvx84CDgH9zZHXOdwETk4fntuL5FjR9qgWldeI/ojcH58HRxDZKvK8SBwmohsFb95XpUTxG/QtwF/FJGNAUSkv4gcXNtOqOoXwFvAL0SkrYjsRXQOG4LHiK71U+Lro42I7CKeg1ch4nq7xffhKqJ7rrr77V7geyIyMr43fwW8oaozMmzjWO/cLiH6E1PdNowmhg109UBVPwJOJjJoLyK64Q9X1XVEhu7r49/nETk6XF5NG6uBXwKvxOqa3RPyxcBhRIPoYiL1yWGquihDF+8hUtPMJnICeL0W+/Yw0Rvo/bHacxpwaMZ1HyLap3uJ7FuPAD3i43JE3M4iIlvnd1X1w3jV84jURPOI7Cp3Zu1vYvuLiBxibiA6ZlsRPZzXFljlulg+FXgPeDv+LdfeXKKBZk/gAe/3mURveZcT/WGZSfQW0eD3VXzsjiGy3y4hsmf9y5M/SeRw8hyR88dziSYuiX9/PT6fz1L3GL0Tgd2I1NlXEV1n9UZVVxD9oTie6M1rHtE12C7D6l2JBvMlRNf8YuB31WxjIpFt/Z9Eb6dD4+1lYRfgDRFZSeTYc4Gqfp5xXaOESKjyN4yWR6wqnQWcpKrPl7o/hmE0LvZGZ7RIYnVit1g9dTmRXTLzG61hGC2Hkgx0InJHHNg5rRTbNzYI9iDy/MyplI+KPWUNwygyNT3jY9v9n+LA/akSJ2CIZYdIlCzgExG5tEH6UwrVpYjsQxSXdo+qbtPoHTAMwzCKRk3PeBEZA/yIKBZyN+AmVd1NomTgHxMlX5gFTAJOUNUP6tOfkrzRqeqLRIZswzAMo4WR4Rl/JNEgqKr6OtBNRPoSeRJ/oqqfxQ5Y98d164XZ6AzDMIzGpj9hgoNZ8W+Ffq8XTXaaHhE5CzgLoFOnTjuNGDECgI8//jiot369S+YwbNiwQFb+0Sf58vK1rl7/4WGY0+ylLsnDV1+Ff0IGDx6cL7dbvjhfnjc/nKGnbx8Xe1vepUcg++ILlziiZ8+e+XK/jTqG/fjIJe/o2i48NR2Gu7jU//3vf4GsdWtXd4st3DFY9O77Qb31npZ6k23CY/XZTBcvu3r16nx56NAw7rtqlrsGFy8PvfUHDHH7tmi969O8efOCen37usQhPWRdIJv9hTv+vTbyvMr7hefs009d4pXOncO45yH9Xcjc3GmuXrtWYZx89+23ypeT11VlpQuP2mKLLfLlVdPDeivWuXr9R2wayGYvcddI6nW1zEWKzFuwKqjXd5Mu+fLqTt0C2ZdffpkvB9dV1yDXM7M/drHvG3UIr6v2w6q/rvxrCmAL795aNCW8riq966rPtu5YffpFmBSmvNyZSDffPIyzrvzS3SOLV7hrYsBmvYN6C9e5czh//vxA1q+fSzzUTdfky7O/XBLU27i7Oz5VfcJkRZ999lm+3KVLl0A2uN/G+fLcaa5eu7LEdbVd4euqqsolBfKvq5UfhNMOrqxw11W/EWE8/qzFywFYvnw55eXlWZI/1Ir27dur38/qqKioeJ8oVjHHWFUdW4vNVNdvTfm9XjTZgS4+aGMBdt55Z5006U0A9t8/nKNyyRJ3ET/11FOBbMo3xuTLz37q6l13R5ig5IqH38iX77///kD2pz/9KV8ePsHFEP/mDy8H9S797j758gejw7Ccs88+O18+6aST8uVrDtsxqHfZ6Evy5YMHh4Pldk8/ni8femgYzta9e/d8eeIEN1/rXb1D1fgi76H8k6f/FchOvNDNSzl58uR8OXk81lxyYb5855Nhlq8brnEJ7+9a6h68v/71r4N6V1xxRb78nVZh9qjLz3TT8H1/tHu4ylU3BPW+/e1v58t77LFHILvn1z912x5+TL48uGOboN5xb7jzfuCBBwayZcuW5csTJkzIl9/cZf+g3vMzl+fLv7znD4Hs0vv/my8/+OCDgezmm2/Ol4f85/Z8+Xf/FzqGXn7Gfvnyu7seFcjOOcclDPne99yx//l+WwT1Lj8oHzvOmM3D7G8jnnjEyca4+6VPnz5BvacffzRfvn3jkYFs+Xr3ULxwgqt37A9+EtSbNs35JSSPx/Lzf5gv3zNxRr7821+dHdS7bW4HJ/vtbwPZlVdemS8ftdYNHJf/MJxk4Jz9R+TLay++JpCdcMIJ+fI+++wTyG6/6kf58nVb5idVYIvObYN6x7z2Wr68//7h9bJmjRsbnnnG3auv7TA6qPfSnBX58jXj/xTIfnLnkwA88EBxJk9Q1eDPaHV8+eWXa1R153psZhZhJp8BRLGTbQv8Xi+a7EDnM3XqVAYMiPbdf0gCnNxhbr586Sa7BbIzDnP/Gk+83z1Mtj42TPCxyy675MtfvB5O7H3DFvlZZFjQ3h2uW6rCONH99nMPpKUTwj76A8c7ow7Jl3/2+/Bm//WL7sb1B1+AM7w+3nbbbYFsi6ddvO45rd0+X3pxeKNO28fdnIN22S+QnXLKKfny3T90KvHLdzgiqHfocDeAXTk3fCiPOtgl2th4Y/fvd9aX4WDmD8C3eYMvwJ/nukElGHy/852g3iOPPJIvl190QSA7b6Dr8w1/cwPAHV91D+oNGuSyfl111VWB7FgvDviyXjvly2cfHcZYH/eQu65GJPq411575ctfvPp4IPMH4AXeAPznik+Cev51tfKJqwPZ1KlT82V/AL76huVBvetfuylf/um9YRjhabu7/AR33HFHvuwPvgDntHWDw+VXhNfO2zu7473pyFH5sj/4Atx7rnddbffNQDZmKzcAXzHrlXx5T2/whXAAnjUj/KPlD8B3eYPvXxa8ENTzB+Bp3nUP8Pjj7jwtOSccZM8b4s7Zb+93Mn/wBRgyZEi+fO211wayI1dPz5cv67FDvnzOt0cE9Y6+2N3jWxxzXCAbPXo0EGodGppWrYpu1XoUOE9E7idyRlmmqnNFZCEwTESGECW6OJ4oQUG9KMlAJyL3EWVM7yXRfFtXqert6WsZhmEYxUZEvqa6rkMbX3vGE00BhareSpTacAxRtp7VRFMzoarrReQ8ollAyoA7VPX9r22glpRkoFPVE2quZRiGYZSC+r7R1fSMj2fhOLeA7AmigbDBaBaqS8MwDKNxaIg3uqZGs9ib7bbbjpwzylVXXR3Itr3b2af67BIa0I/6kXNK2LfCeaj9ZnloPH7mPjfzzPPTvhvIfvrRI/nyPc9Oype32SZ09PC9P3/84x8Hsl7ivBjbd3eehMs/CT2bHjrC2aSu+0do5/vmN51N4/LLw9zQM2c677bjL3KOMP2u+2VQr/xs58DxrTnhqe9600P58qpNne7/nA8nBPUuvtjNfPLmTjsFsr333jtfvuEG5zyy4I+hU8KcNe5YLasIj8Gn17r9HjfO2YyS++w7Tvg2EYDfvHpfvtx6qbNjdfzRH4N6Ry51trFt7xsfyLr9+x/58ohyd119x3MiAWh/2mn58i9/GR7v7+zq7KUT9jstkM0sd8dgu26el+RHLwX1/t//+3/58i9+8YtAtvPOzhfAd3j61Y/PD+q9dbSbXGBtwoHocs/zd9TyKfny9FPDGW4+ne4SVOxx59OBbMz8snz5vffey5eXX39hUO/qQ9zED51bh28MI77l7FVlZa69kSNHBvWeffbZfPmbRx4TyH778j/z5UHvuPP3f0MPCur1W+m8Oq/8VmgbGz7cOfJctfc3Atnjn92dL0/6nXPmSp73GY/cki//57DLAtkVnlftsds6j9Jt//j7oJ7/rOnYMfTOfvXV6Hm1cmXo+d2QNIKNrlFpFgOdYRiG0TjYG51hGIbRorGBrkT44QW7e+7QAC+88EK+3OXRPweyvx7lYnOe9tzYzzp9h6DeITe6+Lhk+MJpe7hwAD/I99Zbbw3q7bLWuYU/eFwY8/XDc9yE0yePcoHPtywOY/FuvveRfHmbM0M1Vfv2Tr11ySWXBLIT93Zq1GcPPiNfPv93YTzVoX3cpNC/fesvgWzCXBeTebDnar/ohjAE4vTTT8+X//VQGMfzzredevW3g11ow/Au4XRiF97p+vjplqGb+bkXXZQvvz/cufIfdFCofnr77bfz5fKbwuMxdvTJ+bLv1H/WxaEq6sgLXDzlpZeGuWO/veXIfHnrrbfOl//5z38G9YZOd1u486wwROGCFS6g/ntHhAH6t0x3oRnX/dr14/zDTg/q+TGSSVf1g/s7Fd8Th7rjds5l4bV5jBcS8ucZjwSye19ysW27Xe36sWbNX4N6F1zgQjieefDuQPbK4S6s4vKeLjZ0t+5h4PpVTzjV/Fvtw+Nx/GVOxTdjnAul8eMlIQxqn3nhyYFs7PaH5cs92rpjc/avwwxSK444L1++yLveAF4f4EK4sj5rxh0d+lQ87SWnOPPU7QPZIf/n1Lc//7k7Hv5zBrI9a0b9MHxGNCSmujQMwzBaLPZGZxiGYbRoRMTe6AzDMIyWjb3RlQA/vODhhx8JZOef71yp/ZRIAG37uzCCvfYanS8v/O4ZQb1hq11S3fNbLQxkW650dq23X3Zu/M8fFKY36njA4Hz5hGfDnId7beRsTXfddVe+/KP9Dw/qLViwIF8eODBM5HrccS4NUDIv47r3XGxlB88u0r1N+K/s6fnOtbn8oNDlf/RFLoXUmy+7HI1PTnwhqHf77S6BzeChoZ3Fvzl2O37PfHmr738/qNdx521dH38Xul+fPNkd41e/cu7/fe57Maj3yQxnu9nplz8KZGef7WwX48e7sIFvJexrc+4cmS/7CYEBfvQj1+bJJztb0MYzXg3qTb7V5Ved54VNAFR4qWinvTQrkPX5pbMT/ewnLhfonnvuGdTz0735fUqy4yhnGzvzzDMD2YGjXVqu+TeFYRqtfzcxX957kQuD2Shx7Ww/3oVsrNuqayAb5h3XLbzjPfahh4J6Pzv1Z/mynyIO4KijjsqXv/tdF+Iz4KtwGrJ3v+3umSefD1PLrfKyS2/pJUVfOTu8pwd1cfa7Cy+8MJCNGzcuX8658efIpd6C6JmU44zxYc7NS450KdG+ujm0H0/YzNkfy7378YTWYR7WQ3u5EIidl04JZKtHRRmxKjveRDGwNzrDMAyjRWM2OsMwDKPF09Le6CRKOda0adu2rfbuHWURSKqYvvWtb+XLvooJQjXT5J+7jBZPvRaqkeavdaEHe/YIM5Hve65T+wy4yKmYnpsUqkl9ld5r3jQdEM5B5WezOOOMUIV66P6jXZ9uDFUeL3gqphc9FROEaqYxO7npNXa5NlQZLtvWZRPxVXoAD3lqJn+es6SK6fDDneoomZ1+wNIP8+V3r/hdvpxUMfmZUXbuFrqgH3DmrvnywJ9ely+/Mj1sw8+0//LLYZjGunVO7ROomBLH+8gUFdPL1zl1sK/yTWb0OHQbl91i92vCTPjlo1zS9eR0R/fd51SB/hxoPXqE0zP5GXFO87KwAAxTp+qeduWv8uUnnwizn8xYXZEvb9M1DPU4+GTn/j70Zy584a2Zy4J6/vF+/vlwBgR//kI/Y9Cpp54a1DvmGJfJZM34MFTipSv/nS8/NdvNvtA2MYfgwZu74zPq2nC2CD3Uufn70wD97W9/C+r5c8R169YtkPlhLMnre5vO7rqafvXV+fLTD4dzzn3sZV5JTuFzqDdLwXAvy9PUpeF15R9vf5ooiOahA1i4cCHr1q0LD1AD0K1bN/WzHFXHY489Nrme0/Q0KvZGZxiGYeRpoNkLDgFuIpqBYJyqXp+QXwzkJudsDWwJ9FbVr0RkBrACqATWN8SAagOdYRiGEeDnG60tIlIG3AwcSDTB6iQReVRV855Fqvpb4Ldx/cOBH6vqV14z+6rqIhqIZqG69GcYnzz57UD23HPP5ctJlconn7hsJb56pXPnzkE9f0r7fffdN5D5yyO3dRky1k64M6j36X3/yZffe/KzQPb2UjersD8bc6+24cW0x8bOw3P40VsFsoEnOq9L3SGcjHLSJJcAduJEp+L873//G9T74gun/vNnOoZQhTNihFOvJGdI9j3PthwUzkJc/pRT33403qlb3n0hVDtOWeYyhqypCq+/ft7ktrsNcN59Wx43Mqx3nJupvXzwroHszTffzJd9tU/Si85Phu2rOwF69XITgW67rfMSPeCAcIZ7fxbqwV3D/42rHnfHY/q9YbLmyW/Mzpc/WO6OR0XidvRnRd9ti1CtOeJY90e3z3FOTbik29Cgnq9K92e1hvDamTPHTeTsq9sBcqYDgB12CDMLHexNuDtqlFP19yWcAHbpo3flyx+MfyWQTZ7q1LAfrQjPhc/mnipwF09tDLDVSc5jtftRp+XLc6VbUO+VV9y2k2rBd999N1+eN29ewX4MGOAyHO2USG7uqz/32GOPQNZzhbsXFjzkMsxMv//NoN4bH7tnvq96BmgTKysf0Lks0LUNrrrs0aOHJjMRJXnggQcKqi5FZA/galU9OF6+DEBVf12g/r3A86p6W7w8A9i5IQe6lmVxNAzDMOqFiFBWVpb6IZpQ9S3vc5bXRH9gprc8K/6tum11BA4B/NgfBZ4RkcmJduuMqS4NwzCMPBltdItSbGfVvWUWUh0eDrySUFuOUtU5IrIxMEFEPlTVFwusnwl7ozMMwzACWrVqlfqpgVnApt7yAGBOgbrHA/f5P6jqnPh7AfAwsGs169WKZmGj88MLkva1oUOdPWK//fYLZL59bceRzo167cS7gnqf3/dovjzt8U8C2aQl2exru/V0YQkjEva1QSe4LB6y29Gubc8+AqF97cUXwz8wvgt6mn3Ntzcm7Wv+8dhqyIBAVv60c2f+371uYs0pzyXta27bfiYKCO1ru/R152mrhH2t//HO7X7N0DATiG9f8yfZTIYQZLWv+TMPpNnXhvYMJ7dc/ZQ7Hh/8zdl+33ltdlDvPe94pNnXdhnaLZBt+R33Z3iT4519bWmPcMaJ1193sxwk7UlvvPFGvjx7tuuXPwkwQJ8+buLY7bcPs+n7WXZ8l/L+rcMQlmWefW36veG5mPTO/Hw5q31tpy17BrKtPftaj2Pc8VjQJgxv8a8D//oAmDx5cr6cZl/r29fZlv1wHwjta8ksNb1Wu7Ckhf/w7Gv3vRHUm/TR4nz581Whfa3Me9fxZ/XYacdNgnpbnrRXvtz1iHBGi1lrouvqsMMOY+rUqQ1uo+vdu7ceffTRqXVuu+22NBtda+BjYH9gNjAJOFFV30/U2wj4HNhUVVfFv3UCWqnqirg8AbhGVZ+iHpjq0jAMw8hT3/ACVV0vIucBTxOFF9yhqu+LyA9ieW7eoaOBZ3KDXEwf4GERgWh8ure+g1yuIcMwDMPIU5/wAgBVfQJ4IvHbrYnlu4C7Er99BoRqhwagWagu/fCCzz+fEcg+/NBl43jnnXcC2XvvvZcv+671fvJkCFWBSf1z167Oxd3PyjJsWJjQeOTIkfmy744OsPnmm+fLvVo7Vca6Kc8F9Ra/7NyeZ730USD74h2nivEzL0CYTNhXn7VPZJUY6KnShicykgzY3e3bgG+4/ncdFaqDW2/lJi+dvfCrQPbRR67PU6ZMqbYMoRo2eS5WrlxJdfjnAUJ39+S58NVzfmaU4d5ErgCbdHXHoOK9FwLZsled6njmi25y0tlvzg3qfeiFBswuD1WGfuhEm4SCqX8Hdy627OJUev0SKqxNv+HU4D1Ghdkq2mznVNELyp1a3c/8AeF9kLxH/BAcX92XPA9+uEHHjqGa1z8Xm222Wb7sH3sI7xE/hAVg035uvyunu1CMFa+G6smZz7trafabodnnYy9j0JeeS35Sxe7f4Zu0D//r+5lMBiXCFwbs7a6z3vu4MIq2I0MTwbKyLvmyP1EswLRp7lryQxn85xjA3LnuOlu6dGkgy52LYmVG6dOnj5544ompdW688cZmlRmlaM4oInKHiCwQkWnebz1EZIKI/C/+7p7WhmEYhtG4ZAwvaFYU0+vyLqL4CJ9LgYmqOgyYGC8bhmEYTYScjS7t09wo2kAXxz18lfj5SCDnrnQ3cFSxtm8YhmHUjXqGFzQ5imqjE5HBwGOquk28vFRVu3nyJaparfoyjog/C6CsrGynnIt08t+En+3dT8sDoWu5bx/YaqvQ/d8PUeiwZEYgWzPJpUya96ILB5j5yudBvc8+WZIvf5pwKf5qnZsdwU+s1DWRCX9IJ2e32WJgaJMasLsLSxmw746BrMNu7sW5clNnn/JtYQDTp0/Pl5N2M3/SWn/2gq++Cv+r+PbM5Lnw7Wj+uUjaY3wbmp/tHkJ7ZteKpfnyundCW82CF106r1kvhXaQz993mYN8e+ZCb5YKCM9Fp7LC9swRfVyoxIA9wtkz+u/j9qXzHmH4QtkIZ1Ob8eXMQObbM31bTXLy4M8/d9fZokVhRiQ/rZ3/8EnaM/3wguS58O1oftk/DwC927mjtW5KmGrvK8/lf+Z/3TU26935Qb3pK5w9M22SWt+23L9DeI1t1d2F8fTbJUxBt+k33LW00V7Ottx6632CenMWu9Rk/nmA8Pgn75FPP3WzQsyf7/atkF0Zvh4O5Z8L/7mTDPvwn1dJG3S/ntH53WXUN3hr8tsNbqPr16+fJifvTXLNNdc0Kxtdk30HVdWxwFiI4uhK3B3DMIwNApt4tf7MF5G+qjpXRPoCC2pcwzAMw2hUmqPDSRqNrbr8LbBYVa8XkUuBHqr605ra8cMLFixYGMj8jOt+dgiAGTNm5Mu+CshXzUGohliyZEkg89USyQwcPv4/oKT7dZcuzt3Yz9rRv3+Y53Tw4MH58pAhQwLZwIEDC67nq0M6rF2aL1fOmh7UW/excy1fMi10QV883WV9WOxldpg/a0VQz580deHaUP20rMKpt3y1YFKj38FTE/ZuF/7X8rOrbNLbHceew0INd69tnGq059bhsWo33GXXLxvk1NeV3QcG9fzz7l9HALNmuePhX0dJdbB/zfntASxb5iYv9dWMUPhaats2nKjTv5Y22mijQOa79W+6qVNt+9dRctm/jiAMmfGvo7Kl4b1UOcOp9NZ9EqpXF011IQqLp7v1Fn0Yqr3nLXTHYE5Cdemr91eur/46gvBaSk6C27udezhv6oVv9BnQJajXY3N3LfXaJjwe3bdxmWnabh6GR5QNdKrR8nbd8uWFC8Nnkn9N+NcOhM8eX5Z8Jvlq6hUrwnswdy3NmzePtWsbfvaCAQMG6Pnnn59a55JLLjHVJYCI3AeMJspyPQu4CrgeeFBEzgC+BI4t1vYNwzCM2iMizdLhJI2iDXSqekIB0f4FfjcMwzCaAGajMwzDMFosLfGNrlmkAPNnL0iegPbtXRon3xYG0LOny5Du2x/SbBhJ25hvD/Pb6JZwe9bZzh7m28IAVkx36X0Wv++5i38Uuosv9kIUZibSSfnu2MsqQjf5ZNb8HMkUYD28GReSqY8GbOQyqXcf0i1f7rVVn6Bez23c8em0ZZjqrM1QZ9OQfi6EY9FXod3TT2+UtI35NjDfrurPVgChPSwZAuHbVZMzPfj411LSruq76Pt2Vd+mBeG1M2jQoEDmLyftqpts4lJedaoqz5erZod21bXetbTs/YRd9f0Z+fKiD51ddeHMcGZv/1pK2lV9e1ih6wjC8IseiZk7fLtqP28WiB5Ju+qWLhzAv44A2g937vWth3jXVZ/QtT7NrupfI779y0//B6E9LJmCzk+3lQwbKGRXTb79+NeSP7MIFLbR+6nTILyufPurv16xZi8YNGiQXnLJJal1zj333FQbnYgcAtxElNR5nKpen5CPBv5NNHsBwL9U9Zos69aFGt/oRGQL4GJgkF9fVfcruJJhGIbRbKmP6lJEyoCbgQOJ5qabJCKPquoHiaovqephdVy3VmTZm4eAW4HbgMoa6hqGYRjNmAZQXe4KfBLPRICI3E+UFSvLYFWfdQuSZaBbr6q31Gcj9WW77bYjF16wbFmolvFduJMqLH/ZV3kk1RW+usyfzBJCN19/W4VcfuHr6jJf5eFngU/Sqo27uFonVKO+23nHjmHmCz/7gq++9bPGAIjnjt62T6iSbL2xm+CyyivTM5wgs8prsyKhlvHVNG3XuOPTc02Y8b9bhVMxDasIM8zsKk62orVz215RFh7v5VVOBbdybXislq1wathF65zMd2GHUG23cv3aQFahvsu4Xw5Viws8xdGKsvDhMMNzf9+oTSjzJ+7t0dX1t3OfTkG9rp5rfJcB4fnss5Nzhd/sGKcGa913cFCv9SZuWXqG2YMqOzlV2vLl7t5KZsz376XkfebPevCed28lwy38e+mrR8IQheXLXXYV/z4rLy8P6vn3UlKVWOjeSj60/bcV3/QB0KGDy7ySVBn64R3+vbWxf78Qmjh8VSWEKmu/jeS92r27U/smM920Ll8KQNuy4tjRMgaM9xKRt7zlsXGSD4D+gG9rmAXsVk0be4jIFKLZxy+KJ2bNum6tyDLQ/UdEfkg0pXn+aaCqyTyWhmEYRgsgwxvdohQbXXV2w6QF+G1gkKquFJExwCPAsIzr1posA11uXvuLExverJq6hmEYRjOmAVKAzQL81+EBRG9teVR1uVd+QkT+IiK9sqxbF2rcG1UdUlOdYjN16lQGDNi0Wpl/QtKySvjlpHdmmgrBT67qZ6JIqit8WbINf9lXf3RKZAXRxe6NvWphmClh/Vyn4ls3O/QiW/HlfK/sVEfLZ4XXx4qpLoHtygVhpg4/4fF0z6vTz3YCobpvXVX4R6uQUjYtM0oyu8VGbZxKr7un7uuV8BLt0tepa5NZUwYPcJdsl02dGqnLwPCctenn6rXuPzSQSY/+Xtmp+1asCo+br2ZbvHhxIPNVfMmEzL76fLqn+kvW87NuLJ8WZuBY9orLSOKr+NJU5+vXh16XhUg+6HwVX1Ld16mTU7f617evfoNQjfe1pNEF7q2k6i/tXvVV5+1widV18aygXuWCGfny+jkzAtmaWe6+WzEzVL3699YK7x5c+db7Qb3li9y5WJRQl7+7zr+3XLk8MTnsmqqaX2K+YG6NdepKPW10k4BhIjIEmA0cDwQzuYrIJsB8VVUR2ZXoMbEYWFrTunUhi9dlG+AcIJcC/AXgr6paUXAlwzAMo1lS3zc6VV0vIucBTxOFCNyhqu+LyA9i+a3At4FzRGQ9UA4cr1GsW7Xr1m+PsqkubwHaAH+Jl0+Jf/t+fTduGIZhNC0aYvYCVX0CeCLx261e+c/An7OuW1+y7M0uqupPlvRc7CljGIZhtEBaWmaULANdpYgMVdVPAURkMxo5ns4PL1i9OnQ3XrVqVb6cdEX23aX9LAfJjAe+nSXpVu234btRf/jhh0E9P9zAXyfqs7Pr+P1N2lLWrnUu7knX6TQ7i+9WnRa+4NMq4e7eqp0X2pBi9/SX27VrF8h8241vt0lmHfHDISoSGfkrPVfqKs/mUpFwsS73ZOsSYQ5rvfbX+OWEbdbvl9/fry2vXur6vjq0oXVe6+xym1SEYSuV691ypYb2tTU4+93qVi5zzJqy8Nopb+Ou6dUJa8GaSnfOyleVeeXwnC2rcLLl68Prw7cNpdlfyyvdtVmpYShGFcu8pcJ+A/4VPS/hW/eVl8Xnc6/cIeFC79t0k/ZdfyLjLp5Nt3338Drt0N2zN3YP7Y0de7rz3r5neM312s7ZcQfu3y1fbt2jd1CvrLuzMZb13CSQSZee1ZbpHNoi11a4e9x/ZvjLrx4WxFo3GBvqfHQXA8+LyGdErp+DgO8VtVeGYRhGSWiJuS6zeF1OFJFhwHCige5D1cRfOsMwDKPFsMG80YnIfqr6nIgckxANFRFU9V9F7luetPAC/59HWgYEX+WWVMf5Krg012lf1ZXMVuCHLCQzKvh1fRfoZBu+zFfvJdtPygr1MamOY5VTl+nKRLz/SqeCq1ziqdwWhS7MlUudCq58QZisuXyxl9Fi4VJXXhSqistnu22VTw3VzWuWuf9QK1c6ZdfKhMptrrf8vxR1XHllYXWc79GdTeH79VAJL1KCtokk2r7azQ+pSMq6pqjjsqrgNhrkVMAde4Xn3VfBte8Zqoo79u6WL/squKTKzVfHSddQVSedvTACTwVXvjZUv/umhaQ6zlf919XMMM8zGXyU0RyRNDOsmuuZQj4LE4mvWfM/r+zMDrUxM/jLWc0MSXLPNd+U0pCICCINniu6pKQN298AngMOr0amQKMNdIZhGEbjUVZWVnOlZkTBgU5Vr4q/zR5nGIaxgSAiLW6gq9HiKCIXiEhXiRgnIm+LyEGN0TnDMAyj8WnVqlXqp7mRxeJ4uqreJCIHAxsTeVzeCTxT1J55+OEF69aFLta+S36au74vS4YhpMn8Nnx7gR8ykFxOynx7hD9TQnJb/nJSljY7QqH9rKgIj5VvH0jaFXxZoTKkhzLUJcwhDT/koVWHRDhEq+rDIaLlNtXKvl7PLbdp0yaQFbLbJu27frZ7v1yTzLefrvdsrqsTNuKVns01aZv17bFp2/L7n9bHNFu1L0segzZ+CIA3a0X78nDGifaerPu6UFa5ztl7tcLJqtaFduDKyqWuLImJUcXdI2txbaxrFd6P68rcckXb0K9u7RpvdoSE5XbtOmfUXVfu9nn9mvC6WuPduuWVoWxVRvuxb2dOynLL92TL5lZrWuIbXZaBLmeVHAPcqapTpKVZKg3DMIw8Le0Rn2WgmywizwBDgMtEpAvZndQMwzCMZsSG+kZ3BjAS+ExVV4tITxo5YDwtvMAnqTsuFHqQFoaQVG/5apqsqq6kzFcPhROodixYL5m13ZelqbAKlWuSFVJ9JbOf+LI09ZYvS7bRpo13jBPqLV/1pb5sTVivauVSV16xNJStXl6trHJl2MbapU71tW5FQr21fFW1snXLExOBrnKhEmvnJ9zMV7nlilUVCZlbXlPudFC+OitadmqrRV9Tb1Wv+iqk6qpO5odY+LJi/JP177pEtAVl3huEH6aRDNlIk/khHH74RvuUeh0TM4i06eTu3TaJyY/bdXX3bidvgtx2XcPru20nL5Spa3iftenS0ZN5bXTrnKjnllt17hbIWnWKQo0mnn0lxaK+djgROQS4iSgx8zhVvT4hPwm4JF5cCZyjqlNi2QxgBVEGrvUp895lJkvAeJWIzAe2EpGWFUVoGIZhBNT3jU5EyoCbgQOJ5pebJCKPquoHXrXPgW+o6hIRORQYSziT+L6qGubbqwdZpun5DXAc8AEux6UCLzZUJwzDMIymQz1Vl7sCn6jqZwAicj9wJNEYAoCqvurVf51ogtWikeUN7ShgeCnTfvlelxUVoauR71mY9DL0l33vwWQ93wMxzRsxrV5Wme8VmTZBZrKNtEwMfrYIP1uC742ZXE56U/qyQuXkenX13Mwq8z0307w/s3qGpnmCpnmQFptWbQuridJU7oVV8+FDyle5p6ntfQeENA/VNJmvpk4zA6Sp/rPWS6rE13pq9fIUL1F/Oal+r4ssrf1k/wtlaEprI3kcc21WdvgdxaKezij9AT+tzCzCt7UkZwBPessKPCMiSjT36dj6dAayDXSfEc1HV6uBTkQ2Be4BNiFS+Y+NwxR6AA8Ag4EZwHdUdUmhdgzDMIzGI6PqspeIvOUtj/UGpOpGyWqnTBeRfYkGur28n0ep6hwR2RiYICIfqmq9NIhZBrrVwLsiMhFvsFPV82tYbz3wE1V9O/bUnCwiE4DTgImqer2IXApcijNKGoZhGCUmgzPKohQnkVmA7z04gGrmbxKR7YBxwKGqmvfqUtU58fcCEXmYSBVa9IHu0fhTK1R1LjA3Lq8QkelEr7RHAqPjancDL2ADnWEYRpOgAcILJgHDRGQIMBs4HjgxsY2BRPmST1HVj73fOwGt4jGjE3AQcE19OgPZvC7vFpEOwEBV/aguGxGRwcAOwBtAn3gQRFXnxq+n1a1zFnAWRIbRLOEFSQr9K8lq60iTpdk6kjL/oslq66iNrJBdJGnDSAuB8GdH8EMb0sILkrKsNoyssqz2njT7Rlo9/xgkj0chWbKe36+kyzwVXijCutAeq+vKC9QLwxeocOtVlYcZ/3XNqmpl/u8AutZrIyGr8jLuVKx29SrLQ/urL1u/JmG3XeXL3HqVaxJteNl+KlYtC2WLPLvtGleuXBfO8ezL1peHssoKt1zh1atN1pFV6paXpIRpFArLiGTZwjn8epUJxV6aLMcc5lYvaADqY6NT1fUich7wNFF4wR2q+r6I/CCW3wpcCfQE/hJvKxdG0Ad4OP6tNXCvqj5Vn33JNZSKiBwO/A5oCwwRkZHANap6RJYNiEhn4J/Ahaq6POsBjPW9YwHatm1b4FQbhmEYDUlDBIyr6hPAE4nfbvXK3we+X816nwHb12vj1ZAlKvBqIh3p0rgj7xJlSakREWlDNMiN9+avmy8ifWN5X2BBofUNwzCMxmdDTOq8XlWXJd7EanzDivNh3g5MV9U/eKJHgVOB6+Pvf9fUlh9eUJnIHFFZ6dQVSTdzX1aXemnrJUMU0tr362btR1r7WZM1J9uviyzN/T8ZeuAnry4U2pGUpe1L1jbSjmPavqSdC39ZPTVSWht+vWTd5HXl161rouxihlHUJryiMUMxstKqTbaHcZqpIq1uWE8S9coK1MtuCvGXk1qwnLp81fsLKQYi0iwHszSyDHTTROREoExEhgHnA6/WsA7AKOAU4D0ReTf+7XKiAe5BETkD+BI4tta9NgzDMIrGhjjQ/Qi4gii04D4iA+O1Na2kqi9TfTwFwP5ZO2gYhmE0Lhvc7AWquppooLui+N0xDMMwSskGOXuBiOxMpHIc7NdX1e2K162QrLMXZKU2r+V1CVFIq1uXUIaaZIVSNyUv1rQUT4VCIArZB6prv1AIRNIl3+9/mru+L0tOGJrV5d+XJcMhsoZzpKVjKlSvNjK/j8ljmibL2kbauSh03tPayDr7R1q9ZPtl/uSt672whMrQhhvIKkIbsRZab30Y5sB6b73K0OaqXhiIJkNC1rs2dW15xnrJsBKvfa9fyXpVFX64Rdj/qnWR7IDff0mx2ODe6IDxwMXAe9g8dIZhGC2eDdFGt1BVa50ZxTAMw2h+bJCqS+AqERkHJHNd/qvwKg2LH16gyQwCntt20s25kCytXtJFvC5tZG0/LZShNjJ/e1nbSNvPrNuq68wDheolZVnr1bUNPzyivDzMSOKHL6Sd90L1apL5y3Wpl9av2pyzQuulXR9ZZWkhFVllyW0V2m6ybjFmrcgailGXNmpTLyebP/+rTG3VhQ1Rdfk9YATRDAa5o69EecoMwzCMFsSGGke3vapuW/SeGIZhGE2CljbQZdmb10Vkq6L3xDAMwyg5uTe6DS0F2F7AqSLyOZGNTgBtzuEFtaEuJzVrqEFDtZE1BKIh2khLTVTIgJ1WL6ssLVQiqyxtW1lno0huKy3sw3flTzuOdanX2G2khWL4desShpDWRtZ6adv+eihDWUFZ2nVVaL202UqytlGbfuS2N2bMGIpFfW10InIIcBPR7AXjVPX6hFxi+RiiOU9PU9W3s6xbF7IMdIfUdyOGYRhG86E+XpciUgbcDBxINAnrJBF5VFU/8KodCgyLP7sBtwC7ZVy31hQc6ESkq6ouB1bUZwOGYRhG86EBnFF2BT6Jp9xBRO4nmnDbH6yOBO7RyFX2dRHpFs9mMzjDurUm7Y3uXuAwYDKRl6X/LqvAZvXZcG1ICy/w3XCTrsiFZEnX3brIGqKNrP2tqyzNzTytjTT37rQ2CoVYNJU2GmJmgDT3/GLMPFCXbTdEP5KhDH74RV3DeLKGQNSlXnXL9W2jNusV2l5tzkXWfuSu9y+++KLg+vUlg+qyl4i85S2PjecQBegPzPRks4je2nyqq9M/47q1puBAp6qHxd+Z5p4zDMMwWgaiNcb7LYpnBK929Wp+SwZEFqqTZd1ak6a63DFtxZzh0DAMw2hBqEL95hicBfjegwOAORnrtM2wbq1JU13+Pv5uD+wMTCEabbcD3iDyxmwUSul1WRcawv22rm0Ue73G9iit73bTZGnqmbp4kNa1/WLXy9rHNE9Cn6wJx9O2m9ZGWj/SvFwLbS/N0zStjWL3sb79mDZtGsVBoaqi5mqFmQQME5EhwGzgeODERJ1HgfNiG9xuwDJVnSsiCzOsW2vSVJf7Qt4YeJaqvhcvbwNcVN8NG4ZhGE2UerzRqep6ETmPaO7SMuAOVX1fRH4Qy28FniAKLfiEKLzge2nr1mdXIFt4wYjcIBd3ZJqIjKzvhg3DMIwmiCqamL6o9k3oE0SDmf/brV5ZgXOzrltfsgx00+Okzn8nMgqeDExvyE4YhmEYTYV6qy6bHFmTOp8DXBAvv0gU3Ndo+OEFSaecrG79hcrVLfsUcmNPa6M2rvAN0UbaeoXqNeaxqmsoRqG269N+fdvIuv+l7GNt2qiLu3tWt/66hjnUtY36bqsYfczafm1mPMiFfmSdCaHW1N8ZpclR40CnqmuAP8YfwzAMoyWjfG3m9eZOljc6wzAMY4NBQStrrtaMaBYDXXMLLygGxc4Y3tDtlzLEoiHaaIhQicZsv6Hr1SX0oim10dy2XZd1Vq1aVXCd+qH2RmcYhmG0YBSoOTNKs6LGgU5EtgAuBgb59VV1vyL2yzAMwygJCpUbntflQ8CtwG1Ay1LcGoZhGCENEEfX1Mgy0K1X1VqHE4hIe6JQhHbxdv6hqleJSA/gAaLpGGYA31HVJWltZQ0vSFLI/bY2LvOFZGnu/3Vtoy79SG67rm3Uxa0/axt1DVGoaxtZ+5G23azHI22dulwTDdF+XY5vcr269L2h1mvo9pvSOSu0Xm3CC3J8+OGHBdepNy1MdZnFMv0fEfmhiPQVkR65T4b11gL7qer2wEjgEBHZHbgUmKiqw4CJ8bJhGIbRFNBYdZn2aWZkeaM7Nf6+2Putxvno4hQvK+PFNvFHiSbRGx3/fjfwAnBJpt4ahmEYRWbDDBiv83x08bTok4HNgZtV9Q0R6aOqc+O254rIxgXWPQs4CyL32g09vKAxKXYoQ0PQmH1s7tuy0JGGb6Ox1/PJhSLMnj273m1VS+6NrkhkMV+JyKbAPcAmQBXRxK43xbKrgTOBhXH1y+P8mAWp8aiLSEcR+ZmIjI2Xh4nIYVl2SFUrVXUk0ZxCu8YzH2RCVceq6s6qunNzePAahmG0FLSqKvVTT7KYr9YDP1HVLYHdgXNFZCtP/kdVHRl/akwAnWUEuRNYB+wZL88CrsuwXh5VXUqkojwEmC8ifQHi7wW1acswDMMoIqqwfl36p34cSWS2Iv4+6utd0Lm5yb1VdQXRRAL967rBLDa6oap6nIicEG+0XNLC+GNEpDdQoapLRaQDcADwG6IJ904Fro+//11TW2lelz519dbK2kaarKHbqKsnV0P3ozkc06bSj6xtQN08JrPWbYg2apMweENoozZtNlYbJ510Uqbt1BZF0cqiRpJlMl/lEJHBwA5EE37nOE9Evgu8RfTml+q5n2WgWxcPVBpvdCiRR2VN9AXuju10rYAHVfUxEXkNeFBEzgC+BI7N0JZhGIbRGKjC+hptdL1E5C1veayqjs0tiMizRPa1JFfUpisi0hn4J3Chqi6Pf74FuJZoTLoW+D1welo7WQa6q4CngE1FZDwwCjitppVUdSrRKJz8fTGwf4btGoZhGI2NKlrzQLdIVXcu3IQeUEgmIvNFpG/8NlfQfCUibYgGufGq+i+v7flenduAx2rqbBavywki8jaRQVCAC1R1UU3rGYZhGM0RRauKqrqs0XwVm8duB6ar6h8Ssr451SdwNDCtpg0WHOhEZMfET7mGB4rIwJyhsDGw2QuMNFqyV25z27em2N9S9qmY2545c2ZxGs6muqwP11ON+UpE+gHjVHUMkebwFOA9EXk3Xi8XRnCDiIwkUl3OAM6uaYNpb3S/j7/bAzsDU4je6LYjMgruVYsdMwzDMJoDSlGdUQqZr1R1DjAmLr9MAc9DVT2lttssONCp6r4AInI/cJaqvhcvbwNcVNsNGYZhGM0ArWqIEIImRRZnlBG5QQ5AVafFr42NRtbwgiS1cbPO0RCu6qVcr67t18YFu7bb2hD3ubZ167MONM3rttjHqi7nrzbbaqrHKrffRxxxRK23k5UGCApvUmQZ6KaLyDjg70Q60ZOJgvcMwzCMlkYuYLwFkWWg+x5wDnBBvPwiURyDYRiG0cLQbOEFzYos4QVrgD/GH8MwDKNFo1Dc8IJGJy284D3ibCjVoarbFaVH1WDhBYaRTlN062+ONKfjOG/evOI0rIpWbDhvdLkZCs6Nv/8Wf58ErC5ajwzDMIzSsqG80anqFwAiMkpVR3miS0XkFeCaYnfOMAzDaGRUqdrQbHRAJxHZKw7gQ0T2BDoVt1shdQ0vKERd3YZ96ura3ND9KEabTaUNn6ZyvFvyOWuIY+zTVParGG02hevxoIMOqncfqkUVrdzwwgvOAO4QkY3i5aXUkCnaMAzDaJ6oKlUV60vdjQYli9flZGB7EekKiKouK363DMMwjJKgULVuAxnoRORkVf27iPy/xO8AJDNKG4ZhGM0fVaWquBOvNjppb3Q5O1yXxuhIGhZeYBhGU6IphCHMnz+/5kp1ociqSxHpATwADCaafeA71c0QLiIzgBVAJbA+N/9d1vV90rwu/xp//6K2O2IYhmE0X4rsjHIpMFFVrxeRS+PlSwrU3bea+U9rsz6Qrrr8U9qKqnp+mtwwDMNofqgqlcV1RjkSGB2X7wZeoIaBqr7rp6kuJ3vlXwBX1aIjDUpDhxekUQx35kI0tDt3TTTmvrWkbdk1Ydsq5bYKXRP77rtvcTaomd7oeonIW97yWFUdm3ELfXIzhKvqXBHZuHBPeEZEFPir137W9fOkqS7vzpVF5EJ/2TAMw2iZZAwvWJSzmVWHiDwLbFKN6IpadGWUqs6JB7IJIvKhqr5Yi/XzZImjg5Scl4ZhGEYLQrXe4QWqekAhmYjMF5G+8dtYX2BBgTbmxN8LRORhYFei2XMyre+TdaArKeZ1aRiGEbJw4cLiNKxFV6E/CpwKXB9//ztZQUQ6Aa1UdUVcPgiXdrLG9ZOkOaOswL3JdRSR5TkRoKraNcseGYZhGM0HVaVqXVFzXV4PPCgiZwBfAscCiEg/YJyqjgH6AA/HcdutgXtV9am09dNIs9GVPH7OMAzDaGwULeIbnaouBvav5vc5wJi4/BmwfW3WT6NZqC4NwzCMRqIBbHRNjaIPdCJSBrwFzFbVw+oS1d6Y4QV1pTHdmbPS2K7qdaEpHjdouv3yaYp9bIp9StIU74u6HLe99967CD0BVahqYbMXNEYemwuA6d5yLqp9GDAxXjYMwzCaAnF4QdqnuVHUgU5EBgDfBMZ5Px9JFM1O/H1UMftgGIZhZEdVqVxXmfppbhRbdXkj8FPCxNCZotpF5CzgLICysjILLzAMw/AoZniBVjV9FXRtKNpAJyKHAQtUdbKIjK7t+nG6l7EAbdu2bVlH3TAMo4miSrN8a0ujmG90o4AjRGQM0B7oKiJ/pw5R7YZhGEYjoYpWtqx3i6LZ6FT1MlUdoKqDgeOB51T1ZFxUO2SMajcMwzAaifiNzmx09aPWUe3NIbygIWgOrtkNQVN07y4GG8r5tP0sDXvuuWdR2lWgymx0tUdVXyCaM6hOUe2GYRhGIxF7XbYkLDOKYRiGkUeroGpdy9K6NIuBzmYvMAzDCClaeAFa1MwoWbJjicjwuE6OzYArVfVGEbkaOBPIHYDLVfWJtG02RmYUwzAMo5mgCpUVVamfelJjdixV/UhVR6rqSGAnYDXwsFfljzl5TYMc2EBnGIZh+KiilVWpn3pS2+xY+wOfquoXdd1gs1Bdbihel41JU/MgawlsKN6kjYldp4XZfffdi9JuFDBe47XcS0Te8pbHxkk+spApO5bH8cB9id/OE5HvEk0Y8JOaJgZoFgOdYRiG0XhkmI9ukaruXEgoIs8Cm1QjuqI2/RCRtsARwGXez7cA1xJFQlwL/B44Pa0dG+gMwzAMR5VmeaNLRVUPKCQTkdpkxzoUeFtV53tt58sichvwWE39MRudYRiGkUeLnxmlNtmxTiChtowHxxxHA9Nq2mCzeKOz8ALDMIyQYoYXFDnXZbXZsUSkHzBOVcfEyx2BA4GzE+vfICIjiVSXM6qRf41mMdAZhmEYjYMqVBTRsapQdixVnQOM8ZZXAz2rqXdKbbdpA51hGIaRR4EWNnlB8xjoLLxgw8RcyzccLDSj9uy2225FaVeBdZbU2TAMw2ipqEJlC/uTaQOdYRiGkUdRe6MzDMMwWi6R6rLUvWhYmsVAZ+EFhmEYIcUKLzDVpWEYhtGiMWcUwzAMo0Vj4QUlwsILjJaMhVEYdWHXXXctSruq9kZnGIZhtHDMRmcYhmG0WBSlwgY6wzAMo6VShakuS4KFFxiGYYQUbfYCNWcUwzAMowVj4QWGYRhGiyYKL2hZA500B9dmEVkBfFSizfcCFpVo2z5NoR9NoQ9g/Uhi/QjZUPoxSFV7N3SjIvIUUd/TWKSqhzT0totFcxno3lLVnTe0bTe1fjSFPlg/rB/WD6O2tCp1BwzDMAyjmNhAZxiGYbRomstAN3YD3bZPU+hHU+gDWD+SWD9CrB9GQLOw0RmGYRhGXWkub3SGYRiGUSea9EAnIneIyAIRmVaKbYlIDxGZICL/i7+7N0I/NhWR50Vkuoi8LyIXlKIvItJeRN4UkSlxP35Rin54/SkTkXdE5LFS9UNEZojIeyLyroi8VcJ+dBORf4jIh/F1skcJro/h8XHIfZaLyIUlOh4/jq/RaSJyX3ztlqIfF8R9eF9ELox/K8n9YoQ06YEOuAtorFiN6rZ1KTBRVYcBE+PlYrMe+ImqbgnsDpwrIluVoC9rgf1UdXtgJHCIiOxegn7kuACY7i2Xqh/7qupIz228FP24CXhKVUcA2xMdl0bth6p+FB+HkcBOwGrg4cbuh4j0B84HdlbVbYAy4PgS9GMb4ExgV6JzcpiIDGvsfhgFUNUm/QEGA9NKsS2iIPW+cbkv8FEJ9v/fwIGl7AvQEXgb2K0U/QAGED0k9gMeK9W5AWYAvRK/NWo/gK7A58T29VL1I7Htg4BXSnQ8+gMzgR5EmZ4ei/vT2P04FhjnLf8c+GlTeIbYR5v8G12p6aOqcwHi740bc+MiMhjYAXijFH2J1YXvAguACapakn4ANxI9NKq830rRDwWeEZHJInJWifqxGbAQuDNW5Y4TkU4l6IfP8cB9cblR+6Gqs4HfAV8Cc4FlqvpMY/cDmAbsIyI9RaQjMAbYtAT9MKrBBromioh0Bv4JXKiqy0vRB1Wt1Eg1NQDYNVbPNCoichiwQFUnN/a2q2GUqu4IHEqkUt6nBH1oDewI3KKqOwCrKKE6TETaAkcAD5Vo+92BI4EhQD+gk4ic3Nj9UNXpwG+ACcBTwBQiM4TRBLCBLp35ItIXIP5e0BgbFZE2RIPceFX9Vyn7AqCqS4EXiGyYjd2PUcARIjIDuB/YT0T+XoJ+oKpz4u8FRPaoXUvQj1nArPjtGuAfRANfqa6PQ4G3VXV+vNzY/TgA+FxVF6pqBfAvYM8S9ANVvV1Vd1TVfYCvgP+Voh/G17GBLp1HgVPj8qlE9rKiIiIC3A5MV9U/lKovItJbRLrF5Q5ED5QPG7sfqnqZqg5Q1cFEKrLnVPXkxu6HiHQSkS65MpEdaFpj90NV5wEzRWR4/NP+wAeN3Q+PE3BqS0rQjy+B3UWkY3zv7E/knFOKe3fj+HsgcAzRcSnVeTF8Sm0kTPsQXShzgQqif7JnNOa2gJ5EThD/i797NMI+70VkC5oKvBt/xjR2X4DtgHfifkwDrox/b/Rj4vVpNM4ZpbGPx2ZE6qgpwPvAFaU6HkResG/F5+YRoHuJ+tERWAxs5P1Win78guhP2DTgb0C7EvXjJaI/HVOA/Ut1POzz9Y9lRjEMwzBaNKa6NAzDMFo0NtAZhmEYLRob6AzDMIwWjQ10hmEYRovGBjrDMAyjRWMDndEoiEhlnOV+mog8FKdJqs36/UTkH3F5pIiM8WRHiEiDZAcRkVcboh3DMJoOFl5gNAoislJVO8fl8cBkDQPia9PWaUTZ6s9rwC4ahtFCsTc6oxS8BGwez9X1iIhMFZHXRWQ7ABH5hjfP2Tsi0kVEBsdvg22Ba4DjYvlxInKaiPw5XneQiEyM25wYZ6lARO4SkT+JyKsi8pmIfLu6jonIyvh7tIi8IG7et/Fx5o1k/fNF5IN4e/fHv3WSaH7DSXH/j4x/7yAi98d1HxCRN0RkZ3+7cfnbInJXXO4tIv+M25okIqPi36+Ot/FCvD/ne+t/N97GFBH5W1o7hrEh0LrUHTA2LESkNVF+xKeIMlq8o6pHich+wD1EWT8uAs5V1VckSm69Jre+qq4TkSvx3ujiN7wcfwbuUdW7ReR04E/AUbGsL1HmmRFEqZn+UUN3dwC2BuYArxDl3Xw5UedSYIiqrs2lTAOuIEpVdnr825si8ixwNrBaVbeLB/W3a9g+RHPP/VFVX44H7aeBLWPZCGBfoAvwkYjcAmwRb3+Uqi4SkR4Z2jGMFo0NdEZj0UGiKX8geqO7nWj6oW8BqOpzEk1xshHRoPKHWMX5L1WdVc3LVCH2IMozCFE6qBs82SOqWgV8ICJ9MrT1pqrOAoj7PpivD3RTgfEi8ghROi6I8mAeISIXxcvtgYHAPkQDL6o6VUSmZujDAcBW3v53zeXcBB5X1bXAWhFZAPQhmrPvH6q6KN7OV2ntqOqKDH0wjGaNDXRGY1Gu0ZQ/eapTBQKqqteLyONEOT5fF5ED8N7qaolvhF7rbz7Dun79Sqq/X75JNIAdAfxcRLaO2/6Wqn7kV4x3t5BR3P+9vVduBeyhquXVtFVd/6TANqptxzA2BMxGZ5SSF4GTILKJAYtUdbmIDFXV91T1N0TJi0ck1ltBpK6rjleJZjkgbjv5BtZgiEgrYFNVfZ5oYthuQGciteCPcgO5iOwQr+Lv7zZEibNzzBeRLeM2j/Z+fwbIO92IyMgaujUR+I6I9Izr51SXtW3HMFoMNtAZpeRqYOdYhXc9bjqTC2PHkylAOfBkYr3nidRw74rIcQnZ+cD34jZPAS4oWu+hDPi7iLxHNNPDHzWau+9aoA0wVUSmxcsAtwCd4779FHjTa+tS4DHgOaJZNHKcT3yMROQD4AdpHVLV94FfAv+Nj1/Os7VW7RhGS8LCCwyjRIjIC8BFqvpWqftiGC0Ze6MzDMMwWjT2RmcYhmG0aOyNzjAMw2jR2EBnGIZhtGhsoDMMwzBaNDbQGYZhGC0aG+gMwzCMFo0NdIZhGEaL5v8DVAmzmrSSVVEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "encod_block = PositionalEncoding(d_model=48, dropout=0.1, max_len=96)\n",
    "pe = encod_block.pe.squeeze().T.cpu().numpy()\n",
    "\n",
    "fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(8,3))\n",
    "pos = ax.imshow(pe, cmap=\"RdGy\", extent=(1,pe.shape[1]+1,pe.shape[0]+1,1))\n",
    "fig.colorbar(pos, ax=ax)\n",
    "ax.set_xlabel(\"Position in sequence\")\n",
    "ax.set_ylabel(\"Hidden dimension\")\n",
    "ax.set_title(\"Positional encoding over hidden dimensions\")\n",
    "ax.set_xticks([1]+[i*10 for i in range(1,1+pe.shape[1]//10)])\n",
    "ax.set_yticks([1]+[i*10 for i in range(1,1+pe.shape[0]//10)])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1-OhCmxYKEkb"
   },
   "source": [
    "The sine and cosine waves with various wavelengths that encode the position in the hidden dimensions are easily visible. To better understand the pattern, we can examine the sine/cosine wave for each hidden dimension separately. The positional encoding for the hidden dimensions is shown in the image below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 298
    },
    "id": "FnLQ-p4-KEkb",
    "outputId": "dcb37df0-b8ae-4209-e013-6b0e791dbdeb"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtIAAAEZCAYAAAC6rcJAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAACojklEQVR4nOzdd3hT5dvA8W9G94RSKNCWvWehTMsqW6QIFUQFQURA2UgLLTLKKnsKKgoKir4WyxAQRTbIXrL36oAChe6Z5Lx/IP1RaElH0pO2z+e6uGiTnPvcJ0nvPDnnGQpJkiQEQRAEQRAEQcgVpdwJCIIgCIIgCEJhJBrSgiAIgiAIgpAHoiEtCIIgCIIgCHkgGtKCIAiCIAiCkAeiIS0IgiAIgiAIeSAa0oIgCIIgCIKQB6IhLWSrf//+bNiwAYDff/+dQYMGGWU/3bp149ixY3natkaNGty9ezfL+/Tl/OLxvSw8PJwaNWqg0WjylFduTJw4kcWLFwNw8uRJOnfubPR95sbgwYPZtGmT3GkIgqCHqNmiZoOo2QVNLXcCQta8vb15/PgxKpUq47aePXsyZcoUWfLx8fHBx8fHKLG3b99ulLjGzNlYPD09+euvv+ROI5PvvvvOKHGPHj3KihUruHTpEg4ODuzZs8co+xGEgiBqdv6Jmm0YxqrZ3333HZs3byYiIoISJUrw/vvvM3jwYKPsqzARDWkT9vXXX9OyZUu50xAEo7C2tsbX15e33nqLb775Ru50BCHfRM0WijJJkpg7dy41atTg3r17fPzxx5QtW5Zu3brJnZqsRNeOQmjjxo289957zJ07lyZNmuDt7c3+/fsz7o+JiSEgIAAvLy+aNGnCZ599lnFfSEgIHTt2pGnTpgwbNoyoqKiM+/755x+6dOlC48aNmT59Oi8uevl8n8/VqFGDX375hU6dOtGkSROCgoIyHq/VapkzZw7NmjXD29ubn3766bWX3Ly9vTl8+DAAy5cvZ/To0fj7++Ph4UG3bt04f/78a5+Pw4cPZ5nHyzm/7vi0Wi1z586lWbNmtG/fPtPzCRAfH09gYCBeXl60atWKxYsXo9Vqc/R6vOzSpUv07NkTDw8PxowZQ2pqasZ9x44do3Xr1pmem++++47u3bvTsGFDAgMDefz4MYMHD8bDw4OBAwcSGxub8fizZ8/St29fPD098fHxyXT5tX///ixZsoS+ffvi4eHBoEGDePLkCQCpqamMHz+eZs2a4enpia+vL48fP87Y7vnlVJ1Ox8qVK2nXrh0tWrTA39+f+Ph44H+XVjdt2kTbtm1p1qwZX331VbbPQ/369Xn77bdxc3PL9jGCUBSImp2ZqNmFs2Z/8skn1KlTB7VaTeXKlWnfvj2nT5/O9vHFhWhIF1Lnzp2jUqVKHD16lMGDBzNp0qSMIuPv709ycjLbt2/n8OHDDBw4EIAjR46wcOFClixZwqFDhyhfvjzjxo0D4MmTJ4wcOZIxY8Zw9OhR3N3d9f6B7Nu3j99++40tW7awY8cODh48CDwr/AcOHGDLli1s2rSJXbt25erY9uzZQ7du3Th58iTe3t7MmDEjT3m8SN/xhYSEsHfvXjZv3kxoaCh//vlnpu0nTJiAWq1m586dbN68mX/++SdTX73XvR4vSktLY/jw4fTo0YPjx4/TpUsXdu7c+drj27lzJ99//z1//fUXe/fu5ZNPPmHcuHEcO3YMnU7Hjz/+CEBUVBRDhw7l008/5fjx40yYMIFRo0ZlFF6Abdu2ERwczJEjR0hPT2fNmjUAbNq0iYSEBPbt28exY8cICgrC0tLylVw2btzIpk2bWLduHbt27SIpKYnp06dnesypU6f4888/Wbt2LStWrODmzZuvPT5BKA5Ezdafx4tEzX7GVGu2JEmcPHmSqlWr6n1sUSca0iZs+PDheHp6ZvwLCQnJuK9cuXL06dMHlUpFz549efToEY8fP+bhw4ccOHCAoKAgHBwcMDMzo2nTpgBs3boVX19f6tSpg7m5OePGjePs2bOEh4dz4MABqlatSpcuXTAzM2PAgAGUKlXqtfl98skn2NvbU65cOZo1a8aVK1cA2LFjBx9++CEuLi44ODgwZMiQXB1348aNadOmDSqVih49emTEzW0eL9J3fDt27GDAgAGULVsWR0dHhg4dmnHf48ePOXDgAIGBgVhbW+Pk5MTAgQMz9RPM7vV42b///kt6ejoDBgzAzMyMLl26UK9evdceX79+/ShVqhRlypTB09OT+vXrU7t2bczNzenYsSOXLl0CYMuWLbRu3Zo2bdqgVCp54403qFu3bqYzLb169aJSpUpYWlrSpUsXLl++DIBarSYmJoa7d++iUqmoW7cutra2r+SydetWBg4ciJubGzY2NowbN44//vgj05mrESNGYGlpSc2aNalZs6be108QigpRs0XNhuJRs5cvX45Op8PX11fvY4s60UfahK1YsSLb/nYvFhQrKysAkpKSiI2NxcHBAQcHh1e2efjwIXXq1Mn43cbGBkdHR6Kionj48CEuLi4Z9ykUCsqWLfva/JydnTPlkJiYmLGfF7d9MW5OvHhslpaWpKamotFoUKuzfrtml8eL9B3fyzmXK1cu4+fIyEg0Gg1eXl4Zt+l0ukyPz+71yCqPMmXKoFAostxXVl6MbWFh8crz83w/kZGR/Pnnn+zduzfjfo1GQ7NmzTJ+f/m5er5tjx49ePDgAePGjSMuLg4fHx/Gjh2LmZnZK/mXL18+4/fy5cuj0WiIjo7O9rnI6nkQhKJI1GxRs1+OXRRr9k8//cTmzZv5+eefMTc3f+1jiwPRkC5iXFxciI2NJS4uDnt7+0z3lS5dmoiIiIzfk5KSiImJoUyZMjg7O/PgwYOM+yRJ4v79+3nK4eVYL/4sF33H5+zsnOn3F392cXHB3Nyco0ePZvvBkJs8oqKikCQpozBHRkYapJ9w2bJl6dGjBzNnzsz1tmZmZowYMYIRI0YQHh7OkCFDqFSpEr179870uJffQ5GRkajVapycnEzidRaEwkbU7KyJmv16ctXs3377jVWrVrF+/fpcf+EqqvR27YiJiXnlX3p6ekHkJuRB6dKlad26NUFBQcTGxpKens6JEycA6N69Oxs3buTy5cukpaWxaNEi6tevj6urK23atOH69evs3LkTjUbDunXrsrzMlRNdu3Zl3bp1REVFERcXx7fffmvIQ8wTfcfXtWtXfvzxRx48eEBsbCyrVq3KuK906dK88cYbzJkzh4SEBHQ6Hffu3eP48eO5zqNhw4ao1WrWrVuHRqNh586degfm5JSPjw979+7l4MGDaLVaUlNTOXbsWI6K5dGjR7l69SparRZbW1vUanWmabyee+utt1i7di1hYWEkJiayePFiunbtmqcPK51OR2pqKunp6UiSRGpqKmlpabmOI2QmanbhImp21kTNfj05avbvv//O4sWL+f7778Ug8RfobUj36tWLFi1a0LlzZzp37kyLFi1o3749PXv25MKFCwWRY7E1bNgwPDw8Mv4NHz48R9vNmzcPtVpN165dadmyJWvXrgWgRYsWjB49mpEjR+Ll5UVYWFjGpPIlS5Zk6dKlLFy4kGbNmnH37l0aNWqUp7z79OnDG2+8gY+PD2+//TZt2rTJ9o+8oOg7vj59+uDl5UWPHj3o2bMnnTp1yrT9vHnzSE9P580336RJkyaMGjWKR48e5ToPc3Nzli9fzqZNm2jSpAl//PEHHTt2zPfxwbOzGytXruSbb76hRYsWtGnThtWrV6PT6fRu+/jxY0aNGkXjxo158803adq0aZbzufr6+uLj40O/fv1o37495ubmTJ48OU/5njhxgvr16zNkyBAiIyOpX78+H3/8cZ5iCf8jarZ8RM02HFGzX0+Omr1kyRJiYmJ45513Mt7jcs2TbkoUUlbDVF8wZcoUOnbsSKtWrQA4dOgQBw8epGvXrsyaNSvbVYYE4bn9+/czbdq0TP3ABEEwDlGzhfwSNVsQck7vGekLFy5kFGQALy8vTpw4QcOGDcVlWCFLKSkp7N+/H41GQ1RUFCtWrKBDhw5ypyUIxYKo2UJuiZotCHmntyHt6OjIqlWriIiIICIigm+//RYHBwe0Wi1KpZg9T3iVJEksW7aMJk2a8Pbbb1OlShVGjx4td1qCUCyImi3klqjZgpB3ert2PHnyhBUrVnDq1CkkSaJx48aMGDECW1tb7t+/T4UKFQoqV0EQBEEPUbMFQRAKjt6GtCAIgiAIgiAIr9I7/8nt27dZs2YNERERmVbCWbdunVETEwRBEHJP1GxBEISCo/eMtI+PD3379qVu3bqZ+tfVrVvX6MkJgiAIuSNqtiAIQsHRe0ZarVbz/vvvF0QuufL0aSI6Xe56pTg52RIdnWCQ/RsqlsipcMYxZCyRU8HGMWSsvMRRKhWUKGGT731nx1RrNshbt00tjiFjiZwKNo4hY5laHEPGKio56avZehvS7dq1Y/369XTs2DHTmuqOjo65SsTQdDop1wX5+XaGzMGU4hgyVlHOqSgfmyFjiZwKNo6hmGrNBvnrtqnFMWQskVPBxjFkLFOLY8hYRT0nyEFDetOmTQCsXr064zaFQsHu3bvzvfOAgAD27duHk5MT27Zty3c8QRCE4q6o1OyNoSEsnR/E1Tvh1Kjoymi/qfTy7WPUfQqCIOSW3ob0nj17jLbzXr160a9fPyZMmGC0fQiCoYWGbmD2/LmE37mBa8WqBPpNwNe3t9xpCTlQHF67olCzN4aGMGfKaNZ0lfB635ZD96IZNOXZvMaiMS0IginJtiF95MgRWrRowc6dO7O8/+V17fOiSZMmhIeH5zuOIBSU0NAN+E2ejE2HEbj1qk1q+CX8Jk8GKHINsqKmqL92RalmL50fxJquEu0qPfuIaldJzZquGobPDxINaRMnriQIxU22s3YsW7aMUaNGERAQkOWGwcHBBkkgPDycYcOGFbuuHevX/8wX06Zz99Z1KlSuxsxpU/jgA9McICT8T6VqNUn2HIBlhfoZt6XcPYfVybXcvn5FxswEfYr6a1eUarZKpSQl0BYzlSLjtnSthOXsBLRandH2K+TPL+vXM2nsUFZ3lfByV3HonpaPdyiYtfgb3vvgA7nTEwrIL+vXMytoEpdv3qNWFXcmTZ1VpF//bM9Ijxo1CjBc8TW06OiEXHcYd3a249GjeIPsPz+xMp0Ze7s2yeGXGDbGj/j4lHydGTPU8ZnK82RqcU6Hx3Dn5nXc366d6XYL19rcDbme57hF+fk2ZKz8xrl78zpuJvDaKZUKnJxs87S/1zH1mg05r9s1Krpy6F50xhlpgEP3tNSo6Cr731lR/NswVKwZUwJY/dKVhNVdNQyfMpEOnXxkyckYcQwZy9Ti5DdWpm5Z79py6N4jBo0dSlx8Sr6uTMj5POmr2dk2pL///vvXBv7oo49ylUhRkd8+lgmpGqbPmYNNhxEZZ8YsK9SHDiOYPX9ukbjEXNQ8iEth6f7b7Lr2CMtSbqSGX8p0VjM1/BKuFarKmKGgz8Zz91E7uWb52lmUcuPonSc0r1hSxgzzryjV7NF+Uxk0ZTRrumr+d2bz92S++LQNSBIoFPqDCAXu6p1wvN7P3ODwcldx9XY4drvHklLdl/TyLUCpkilD4XXy3S1Hp2HpvCnFrltWtg3pxMRE4NkqWefPn8fb2xuAvXv34unpWTDZmRh9fSx1kkR0Yhr341J5EJfCg7hUHsT/93N8Kg/iUolP1XD/3k3ce796Ziws5IYchyVkIyVdy48nw1l7PAyAT1q4Y+f2BZOmTYUOI7BwffYeiN6xlBa9P0Wj1aFWKfVEFQraxnP3Cf77Ok16DOLcli8zvXZxfy/HvcNARoZeoF21UoxtW5my9pZyp5wnRalmP//AHf78Q71Ceab1KceH5r+TvMeKhDazQF04X6eirEb5khy6l/zKlYSaZW0xv/Unllc2oLVxIbX626TU8EXrVEvGbIUX6R3gq9OgTHyIMiESVcJ9lAmRKBPvo0qIRPn896RHXL0Tg9cHdplie7mruHqn6I6Hy7YhPWLECAAGDRrExo0bsbW1zbh99OjRBtn5uHHjOH78OE+fPqV169aMHDmS3r1N94zs7PlzszyTPH7aDNY9rcDD+FQ0L122tLNQ42JvgYudBQ3LO1DW3oI5P1fK+qxmRXFW0xRIksSe649Zuv8W9+NS6VC9FKPaPG9gVcRcpWT2/LmEhTy7KvHBsPHsoxZf/HGFmW/WFI1pE/K8Ee1VuSRzR49hq6dbptdu0cyZdO/hy/pT4aw+eo/Dt5/wUTM3+nm6YaEuXK9jUavZvXz70Mu3z/8uxUo6Eo8vwubkEtRPrhLX9Vt0tmWNsm8h98wij/FF0xQ+3prG6u5kXEkYtEPBxOlLiO7RHYvbu7C4ForVv99hfeZrNE61SanRi9Tqb6OzcZH7EIq17Ab4jgwaweD4eSiTHqKQMo9PkNTWaO3KobMpS5pbW3S2LtRwXZnll6kaFV0L9HgKkt7p7yIjIzNN6m9ubk5ERIRBdr5o0SKDxCko4Xdu4Nbr1TPJUQ/vUa+sHS41nHGxs6CsvSVl/ms821q8+hRbBwY+O5P9wpmxp38uY9HsmQV1KEI2bjxKZOHeG5wMi6VqKRu+7lODxm6OmR7j69sbX9/emfpa/XwqnMX7bqHgKjO61UStFJee5ZapEd29NuZqZZavHcBHzdzpWqs0i/fd4ut/7rLtYhTj21XljcqFr7tHka3ZCiVJzcajca6D3a4xOG7oRlyXVWjKFq6z7UWRMvYO9jsG08erOgltBjJ8yYKM7gETp/+ve0Bqte6kVuuOIjkai+u/Y3k1FNvDM7E5PIt0Vy9SaviSWrkrmD9bRU7MAFJwsuuWczkqiXS31mhty6KzLYvOpuyzxrNtOSRz+1e6WY0OrPBqt6ytqUyYkfUg6KJAb0O6R48evPPOO3Ts2BGAXbt20bNnT6MnZorKV6iS5Zlkt0pVmdkt55eonveDfn5mrFT5Sti36k902aYGz1nImdjkdL45fJfQfyOxs1Dj374qPeuXzXGD+P3GrugkWLr/FiolTOsqGtNyyqoRrY+LvSVzfWpz7M5T5u+5wZhNF2hVuSTj2lXB1dGqALI2jKJes9MqdyXGtzIOfwzCcXNvElrPJKVO0Z0RwNQpUmNx2D4QJInYbj/Q07ESPft+9NpBXZKVEyn1PyKl/keont7E4tpGLK9twn73GKT9AaRW6sxPt0sxZ9nXrOmKmEu8AGQ7wLeSG/Htc/4F+pVuWW7OzGon8bbzOQyzWLjp0fvp8umnnxIcHIy9vT0ODg4EBwczdOjQgsjN5DR86yOidywl5e45JK2GlLvnSNz1JYF+uV+cwNe3N6eOnkSn1XLx1Bn69u7D90fvcfzuUyNkLmRHq5P47WwkvmtOEPpvJL3ql+W3QU3o3bBcrhvC/TxdGdGqEn9deUTQn1fRmtjS0cXF80b0G5Vy3oh+UbOKJfhlQGNGtqrEybAY3v3hJKsO3yElXWukjA2rONRsrVMNnvbeTnr5ltjtm4Dt/kDQpsmdVvGj02D/16eoYu8+62rjWCnXIbQlqpDUzI8n/f7haa9NpNTwxfzeXpYsW8Kars+6GJipFP91NZBYOj/ICAci+L3Tgo9/T2bvbQ3pWom9tzUM2qFgtN/UXMfq5duH/UcvotXq2H/iBj0GjMHq0s9YXN9ihMzll6NPGJVKhVKpzPhXHJ0Ki+G8dX069R+F+fEfCFvUC/PjPzB/xgyDzLTh174qFUtaM/mPK0Qnig8EQwsN3UDj5p4oVSoaN/ckNHQDp8Ji6P/TaebuvkFVZxt+6t+ICR2q4Whlluf9DGjqxmdeFfnz8kNm/CUa0wXtxUb0PJ/cN6KfM1Mp+bCpGxs+akKbqqX49sg93l17iv03oslm6n2TUhxqtmTpSOxb60jy+BSrC+tw3NIXRdIjudMqPiQJ24NTMA87QHzbOc9m48gPhQJN2SYktJ1D9Eenufz42VzULyrqg9bkooq+zECb3Uzt25zhh5ywnJ3A8ENOTJy+1CBn/xObjifdpTG2eyegjLltgIxNi94Ku3btWsaPH8/Tp0+Jjo7Gz8+PH3/8sSByMxmJaRqm/3UNV0dLvgr4LONM8qmjJw02XZ2VmYrZ3WuRmKZlyh9X0BWCD+vC4vlsK2lNB+I2biNpTQcyOnAS/actJz5Fw5zutfiqd32qORtmbt+PmrkztGUFtl96yKyd18RrWUAM1Yh+URk7C2a/9ez9YaFWMn7LRcZuusiqdetf+WJmKopVzVaqSGw5ibiOX6J+dI4SG95E/fBfubMqFizPf4/VhXUkeQwjtda7hg2usqBGJVcO3ct8FaioD1qThTYNu11jkCwc6Bbwy//OJB+9aLguNCoz4jquAKUK+53DQZtqmLgmQu8nzW+//UZISAijRo1i9OjR/Prrr4SEhBREbiZj2f7b3I9NYVqXGliZGW/+y6qlbBjfrgrH78VkTLkm5N+Ls60oVGosK9SnROdRaE+HsuEjT9pXd0Zh4HlpB7eowCct3Nl6MYrZf18XjWkj22SERvSLPN0d+bl/I8a0qcy+Pzczbca0TF/M/CZPNpnGdHGs2anV3yam12ZAiePGXlhc/U3ulIo0s7t7sT00jdRKnUlsbpxBZKP9pjJohyJzV4M/yFNXAyF71ieXYvb4IvFt5yBZGW9wtc7elXjvhZg9OofN4dlG248ccty1I6ufi4Mjd56w8dx9PvB0pUF5B6Pvr0c9FzrXdObrf+5wJjzW6PsrDsLv3MDC9dXZVmLu38HSiF+MPmlRgUHN3dly/gFzd90QjWkj2XTuPrOf94k2QiP6ObVKyQeerijObsKp6+hMX8xs/ltQyVQUx5qtca7L0z5/kO7SCPtdY7A5FAQ6jdxpFTmq6CvY//UpGqdaxHVYZrTFVXr59mHi9KUZXQ1G7khhar/WYqChAamjzmJ96ktSarxDWuUuRt9fWuUuJNX7COtzqzG/vdPo+ysoej9xevXqRe/evVm+fDnLly+nT58++Pr6FkRusotP0TDzr2tUKmnNsDcqFsg+FQoFEztUo7yDJV9sv0xMUnqB7Lcoc61YldTwS5luK4h5uxUKBcNaVmBAUzc2nrvPvN03CkX/2sLk5UZ0Qcz9HHnvZpZfzMLvmMaCSsW5ZktWTsR2//nZh/W/3+KwtR+b/u972jSvg0qlpE3zOmwMLdpn541JkfQYh+0fIZnZENft+4xp6ozlxUFrJ1YOZ4DDEdRRZ4y6z2JDk4Ld7rHorJ1JaFVwAzgT3/iCdOd62O0eizLeMNNyyk3vp85HH31EcHAwDg4O2NvbExwczMCBAwsgNfkt3HeT6MQ0pnatUaCLM9haqJn9Vi2eJqcT9NdV0fjKpwA/f57+ucwgs63klkKhYLhXRfp7uhL6730W7LkpXk8DkaMRDfJ9Mcup4lyzAVCZkdh6BnHeC/ntzwPMmTqWFV7RpATassIrmjlTRovGdF5oUnDYMRhl0kPi3lyNzrZcge4+scVEdDZlsNs7AbTiBFN+2Rybj/rpdeK9FyJZGP9qewaVBfGdVjyb8eXvEUXiqpHeT56zZ89SoUIFPvzwQwYMGIC7uzv//lv0B3PsvxHN9otRDGjmTh0XO/0bGFjNMnaMaVOZQ7ee8POpovGtTS72ddpi36o/mkOrDT7bSk4oFApGtq7E+43LE3I2kkX7bonGdD7J1YgGCPSbQOKuL2X5YpYTxbVmvyy11rvMPGnD6u6WYgq1/JIk7Pb6Y/bgJHEdlqIp07DgUzC3I6H1TNTRl7D6d1WB778oUUcex+rsKpLr9CfdvU2B71/rWJmEtnMxu38C6+MLC3z/hqb302fatGnY2Pzv8o21tTXTpk0zZk6yi0lOZ/bf16jmbMPg5u6y5dG7YTnaVnVi+cHbXLwfJ1sehVlCqobF+27h2a4bF06eNvhsKzmlUCgY06YyfRuV5/9OR7Bkv2hM59XzRnTLSiUKvBENz+aAnz9jhlGmwTSE4lizs3M17KGYQs0ArE8tx/LaRhKb+ZNW9S3Z8kir3IXUyl2wOb4IZewd2fIo1NISsd89Fp29Gwktv5AtjdTqb5Ncqy/Wp77ELOyAbHkYgt5PIEmSMs1ooFQq0WgK/6n415m3+wZxKRqmdamBmUq+OVgVCgWTO1entK05gdsuE59StJ93Y1h1+C7RiWlMaF8VlcwrDSoUCsa1rUyfhuX4+VQEQ2evMNkp1EzJi3OA127kwaSl39GyUgnm+dQp8Eb0cy8uqCTHF7PXKY41OzvPVmsTU6jlh/mNbdgcm0dK9V4kNR4pdzoktJqBpDLHbl8AiJMRuWZ7ZDbKuHvPVis0ch93fRJazUBbohr2f49CkfhQ1lzyQ++nkJubG+vWrSM9PZ309HTWrl2Lm5tbQeQmi7+vPuLvq4/4pEUFqpc2zLzC+WFvacasbrWISkhj5s5r4ixmLlx7mEDImQh6NShLnbL2cqcDPGtMj/euQu2Es2z7YanJTqFmKl6eA1zl9TFxh36khfaSbI1oU1fcavbrZD2FmiSmUMshddRZ7HePId3Fk/h288DA04Tmhc62LIktAjAPP4jFtVC50ylUzMIOYHVhLckNBpNerrnc6YCZFXGdv0KRnoD9rtEg6eTOKE/0fhIFBQVx5swZWrduTZs2bTh37hwzZswoiNwKXHRiGnN3Xae2ix0fNjWdD5565ewZ7lWRPdcf89u/9+VOp1DQSRJzd9/A3tKMz7wqyp1OJgqFggs71pn8FGqmILs5wOcvnC93aiarONVsfV6ZQu3PdKZ3c6NXz+Ixi0l+KOMjsf9jEDorZ2K7fgdqS7lTypBSpx/pLp7YHgpCkRwtdzqFgiI1Drs9n6NxrEJic3+508mgdapBQqvpmIcfxPrUCrnTyRO1vgc4OTmxePHigshFVpIkMfvv6ySna5nWpQZqmbsBvOwDT1dOhcWyeN9N6pezp4YJnC03ZdsuRHEuMo6pXapjb5n3Jb+NJfzODdx6vTqFWliIaUyhZirE85R7xaVm51Qv3z708u2Ds7Mdcf+sw/7vkcRf+oWUuv3kTs10pSVi/8dHKNKTiPH5Bcm6lNwZZaZQEt92LiVCumD7z3TiOyyVOyOTZ3MoCGViFDG+W0BtJXc6maTUeg+z8H+wPj6ftHLN0JRrKndKuaK3If3kyRNCQkKIiIjI1M8uODjYqIkVtD8uPeTAzWhGt6lMJSdrudN5hVKhYFqXGrz/4ykCt11mXT8PbMz1vnzFUkxyOssO3MKjvD3dapeRO50sPZ9CzbJC/YzbTGkKNVMhnqfcKy41Oy9Sq71N2sWfsDk6h9Sq3ZAsS8idkunR6bDfNQp19GViu61F61RD7oyypHWqQVKjz7A5uZSUGr6ku7WWOyWTZX77b6yu/Epi41FoynjInc6rFAoS2s7BLOos9n8P5+m7OwvV36berh2fffYZ8fHxtGjRgrZt22b8K0qi4lNZsPcGDcvb816j8nKnky1HazNmdqtJeEwywX9fF/2ls7Hi4G0SUjX4t69m8KW/DSWrKdRidy43mSnUTMUHg0cSvWOpyU41Z4qKQ83OM4WChNYzUaTFY3NUdKN60cbQkGcL16hVNPYL5fvUt0iv0E7utF4rqfFINI6Vnw08TE+WOx2TpEh5it1efzROtUhqMkbudLIlmdsR1/krlEmPsds9rlANJNV7SjM5ORk/P7+CyEUWkiQx869raLQSU7vUkH1mB30auToypGUFvv7nLk3dS+BTz0XulEzK+cg4Np9/wAeNXanqLO+I5Nd5PsvD7PlzCQu5gU1pd0q27k/rzj4yZ2Y6tDqJU+Z1cev4EenHvics5CauFasyzYSmmjNFRb1m55fWqRbJ9QZidW4NKbXfR1O6vv6NiriNoSHMmTKaNV0lvN6349A9LYO+20i6e1vTXpJbbUlC27k4bu6NzYlFJLacJHdGJsd2/yQUqTHE+awHlbnc6byWpnR9EltOwvbQNKzOrSa5wWC5U8oRvWek27Zty/79+wsiF1lsOv+Ao3efMqpNZVwdTavfUHYGNnWnibsj8/bc4ObjRLnTMRkancScXdcpbWvOJy3lm/87p16cQu3AgWPY1G7D0v235E7LZGw8d59rjxKZOfoTTh87ZZJTzZmiol6zDSGp6TgkKydsD0wqtDMFGNLS+UGs6Sq9tHANhWLhmvTyLUiu1Rers6tQPboodzomxeL671je+J2kJuPQlqqtfwMTkFz/Y1IrdsLm8CzUDwvHQlJ6G9Lr1q1j6NCh1KtXj0aNGuHh4UGjRo0KIjeji4hNZum+WzRxd8S3QVm508kxlVLB9DdrYmOuImDbZVLStfo3KgZCz0Zy7VEi49pVKXT9x10drejfxI2/rjziVFiM3OnI7mlSGl8duoOnuyMdqpvYQCcTV5RrtqFIFg4ktJyEWdQZLC+L5cKv3gkv1AvXJLb8AsmyJHb7/EEnPg8BFIkPsd0fSHrphiQ1+lTudHJOoSC+/UJ01s7Y//UZilTTX4xOb0P6zJkzXLlyhfPnz3P69GnOnDnD6dOnCyI3o9JJEtP/vIZCAVM6V0dpon1ps1PKxpzpb9bkTnQSn8wUC3s8Tkjlq3/u0LxiCbyrFc6G18CmbpS1t2D+nhtotMX7LNmKg3dIStfi513FZPu5m6qiWrMNLbWGL+kuntgcDUaREiN3OrKq4epUqBeukSwdSWgVhNnDf7E6/73c6chPkrDb549Ck0x8hyWgLFwnliTLEsR1WoEyPpzts9991ndfpaRN8zpsDDW9L745Wtlwy5YtrFjxbH6/+/fvc+7cOaMnZmwhZyI5HR7LuLZVcLE3nfkxc6NZhRI0Tj3P7p+XF/uFPZbsv0W6Voe/d9VC2/CyNFMxtm0Vbj5OYkMxni/84v04tlx4QF+P8lR2Mt1+7qaqqNZsg1Monw08THmKzfEFcmcjG0VaPJNaqvh4a2rmhWt2KArVwjWpVbuTWsEbm6PzUMYVjjPpxmJxZQMWd3aR2Hwi2hKFc4YjTdkm/JDUkaBfDrPCK5qUQFtWeEUzZ8pok2tM621IT5s2jbNnz7Jt2zYArK2tCQoy/X5Tr3P3SRJfHrzNG5VK0r2uaU6PllPHf/++2C/sceLeU/668ogBTd1wK1E4+rlnp21VJ5pXLME3/9zhcWKa3OkUOK3u2UI6pWzMGdzC9Pu5m6KiWLONReNcl5Q6/bG8sA7V40typyML6+OLeL9GCoETAjIWrhl+yImJ05ea9kDDlykUJLSeDfBf3/fCM+uDISnjI7E9NJW0cs1IbvCx3Onky/zfjrDax+qlvvuSyfXd19uQPnfuHFOnTsXCwgIABwcH0tPTjZ6YsWh1EkF/XsVCrWRSJ9OdHi2nwu/cwML11QUrwu8UjwUr0rU65u66QXkHSz5sYjqrUeaVQqFgfLsqpGp0fHmg+A08/P3CAy5HJTCqTSVsLQrX5UhTUdRqtrElNhuPZOGA3YEvil3jS/X40rPZS+p8QI9B/uw/ehGtVsf+oxcLVyP6Pzp7VxKb+2NxdzcWN7bKnU7BkyTs9nyOQqcl3nsRKPQ28UxaYem7r/dZVqvVaLXajAbnkydPUCoN8+IcOHCAzp0707FjR1atWmWQmPr8dDKc8/fj8fOuirOtRYHs05ieL1jxouK0YMVPJ8O5+zQZv/ZVsTRT6d+gEKhQ0poPPF3Zfukh/0bEyp1OgYlNTmfFwdt4lLenS83ScqdTaBW1mm1skmUJElsEYHb/OBbXNsqdTsGRdNjtD0SycCCxedGZlz253kekl26A7cEpxa7vu+XFHzEPP0jCG1PQOVSQO518q1HRtVD03ddbXfv378/w4cOJjo5m8eLFvPfeewwdOjTfO9ZqtUyfPp3vvvuO7du3s23bNm7cMM5Z1NDQDRmD8QIHdqVC9Ck613Q2yr4KWlYLeyT8XTwWrIiMTWH10Xu0q1aKNyqVlDsdg/q4uTulbc2Zt/sGWl3xOEv21T93SEjV4Ne+8PZzNwVFoWYXtJRafUkv3QCbw7NQpMXLnU6BsLiyAbMHJ0loOalQrSKnl1JFfNt5z/q+H5kldzZGl7GQjkpJi76j+TGiCil1PpA7LYMY7TeVQTsUmfvu/yGZXN99vddOfXx8qFOnDkePHkWSJFauXEmVKlXyveNz585RoUIF3NyeXY7v1q0bu3fvpmpVw55JDQ3dgN/kydh0GIFbr9qkhl/i1KYv2djYtUjMR/vywh7qkq684TusSBybPov23kSpgHFtK8udisFZmakY07YKgdsus/HcfXo3LCd3SkZ1JSqejf/ep49HOao528qdTqFW2Gu2LBRKElrPwvG37lgfX0Sil2l9UBuaIuUptodnkV62Cak1i95nhda5DskNh2B95itSq/civXwLuVMyiswL6dg+W0hn81WSGm8olF1zXvb8GIbPD+LqnXBqlVIyvWcDuvQyrfdsjq73ValShQ8++IB+/foZpCADREVF4eLyv1X5ypQpQ1RUlEFiv2j2/LnYdBiRaTCebceiNRjvxYU9glZv53aJRhy6FS13WkZ14GY0+29G80mLCoV21hV9OlQvhae7I18dusPTpKI78FAnSczbfZMS1mYMbVlR7nSKhMJcs+WiKdOQlNrvYXVuDaroq3KnY1Q2R+eiSI0lvvWsQt+PNjuJTcahtXfHdt8E0KTInY5RZLmQzpuFYyGdnOrl2yej7/7R9QvoX+4GFtc3y51WJrKN5pGyGNSRm8u5Tk45O2sVfucGbr1eHYwXFnIDZ2e7HO8vK/nd3tBxAD5/szZ7bkSzYO8tOjV0xSaPA7YMmZOhn6fkNC2L95+gWmlbRnauiZkqdx8EpnxsLwv2rU/XpQdZfSKCue/kbCljU3xfvi7WhpNhnL8fx/x36lPZ7fWXmAvTa1fU5LdmQ87r9ssK7LXqNhNu/UHJo9NgwFbI5vgK9fsw/CRcXA8thlOyVjPTyMkoseygx1L4sSfOl1eB9yQTyMmwca7eCcfr/cx/U88H4xXF9o2t90i4/Tv2/wSBx1tgnbcunYau2bI1pF1cXHjw4EHG71FRUZQunfMBRtHRCehy0Hf0+WA8ywr/a4Q8H4z36FHe+8I5O9vla3tDx3keK/ZpIhO8qzD4//5l5pYLjGuX+7NRhs7J0M/TV4duE/40mW/erU/Mk9wtkW7qx/YyRyW861Ge9SfD6FrdiTpl7WXPyZCx4lM0zN5+mXpl7Wnt7vDafRa21y47SqUizw1KOeW3ZkPO6/aLCva1MseymT92+wOJO/IzqdV8jJaPIWPlOI5Oi+OWMShtSvO07gikLLYpbDXkteybYFfDl40rg5n58Qqu3ntAjYqujPabmu+uD6bwPD0bjBdNu0r/a8o9H4xXFNs3j6KTUHkFU2LDm6T+PoH49osKJCd9NVu2azr16tXjzp07hIWFkZaWxvbt2/H29jb4frIajJe4q2gPxmtQ3gHfBmX59UwEFx8UrYEzd54kse5EOG/WLk0jV0e50ykQn7R0p5SNOXOL4MDDbw7fISY5Hf/2VQrd6qLFTUHVbLml1P6A9FJ1sflnOqTl7ou6qbO8+CNmj86T+MY0JPPC92UuL9Y9qs8Xe5JZ0TrWpBf1yAs/3+Z8/HtyoV5IJ7e0pWqT3HAYlldCMAs7JHc6wGvOSHt4eGR52U6SJBQKRb6XnFWr1UyZMoXBgwej1Wrx9fWlWrVq+YqZlZcH47lWrMq0GTOK/GC8Ea0qceBmNLN2XmPdBx6oc9n9wRRJksS83TewNFMyqnXRG2CYHRtzNaPbVGbyH1fYcuEBveqXlTslg7j+KIENZyPp1aAsNcsUj+4RxlRUarbslCoS2syiRGgPbE4tJbFFoNwZGYQi8SE2R+eR5taa1KpvyZ1OgVmydBGru1tmnLV9tqiHhuHzgwr1gDzV40sMtN2D8r0WDP8rjKt3wqlR0ZWJ0/N/tt3UJTYZjfnNbdjtm8CT93aBWt6F2LJtSJ85c8boO2/Tpg1t2rQx+n58fXvj69vboJcZTJ2thRp/76r4/X6J9aciGNC08C9W8vfVR5y4F4N/+6o42ZjLnU6B6lzTmY3n7rPy4G28q5XC0cpM7pTyRZIk5u++gZ2Fmk/fqCh3OkVCUarZctO4NCalZh+szn5LSs130ZYwzIBNOdkenolCk0JC65nZ9v0uil7Xj7jQ0qZht3sskoUD3Sb+zJtBJYtV+wa1FQlt5+K45V1sTiyW/ctujk9TRkdHExkZmfFPMH1tq5WibVUnvj1yl/CYZLnTyZf4lHQW77tFrTK2ReaMbG4oFAr8vKuQkKrh63/uyJ1Ovv155SFnIuIY3qoSDoX8S4GpEjU7fxJaBCCprbA9OLnQr3hoFnEYy2sbSWr0KVrH4nM1DwrPoh65YX1yGWaPLxLfdi6SVdFaQyGn0l3fILnWu1id+QbVo4uy5qK3Ib179246depE+/bt6devH97e3nzyyScFkZtgAH7eVVErFcz++3qWo+5N3fPFdOytLTm3bDCN0y6gUhafsykvquZsyzsNy7Hx3/tciSq8Zx4SUjUs3X+bWmVs8anron8DIVdEzTYMydqZpKafYx52APNbO+ROJ++0adju/wKtvTtJjUfInU2By2pRj4+3pRfafsTqh+ewPrWclBq+pFXuLHc6skps+QWSZQns9vmDTqt/AyPR25BeunQpv/76KxUrVmTPnj388MMPNGrUqCByEwygtJ0FI1pV4sS9GP649FDudHLl+WI6aU0H4v75Rkp2GMbSBbMJDd0gd2qyGdqyIiWszZi3+wa6QvjFCOC7I/eITkxjQvuqxfZLkTGJmm04yfUGoHGqie2hIEgvnFf1rP79DvXTayS0mi57X1I59PLtw8TpSxl+yAnL2QmM2G3GrLZq3qtfCNcf0KZit3ssOutSJHgVnbmi80qyLEFCqyDMHv6L1fnvZctDb0NarVZTokQJdDodOp2O5s2bc/ny5YLITTCQXg3KUr+cPYv33SxUC3tktZiOTYeitZhObtlZqhnZuhLn78ez7WLhWwzjVnQi/3cmgh51XfRO5SfkjajZBqRUk9B6JqqECKxPfyl3NrmmjI/A5sRiUit1Jq1iB7nTkc2Li3rsO3mLdzo2x3avP8qY23Knlis2xxeifnKV+HYLkCwd5U7HJKRW9SG1gjc2R+ehjJOn37vehrS9vT2JiYk0adKE8ePHM3PmTNRq2aafFvJAqVAwqVM1EtO0LN53S+50ciz8zg0sXF9dTCf8zg2ZMjINb9YuQ72y9nx54DbxKRq508kxSZKYv+cm1mYqhreqKHc6RZao2YaVXq45KdV7Yn36q0LX8LI9NA2QxNnLF6nMiOu0EpQq7Hd+BtpUuTPKEfWDU1id+Zrk2u+RXqGd3OmYDoWChDbBANjuD5BlPIPehvTKlSuxtLQkICCAVq1a4e7uzldffVUQuQkGVNnJhoFN3dhx+SFH7jyRO50ccSxbkdTwS5lue76YTnGmVCiY0L4qsSnpfHP4jtzp5Niua485eS+GT70qUsK6eM26UpBEzTa8xJaT+PlCOq3btEClUtKmeR2Tn4fY/M5uLG7tINFzDDr7wjuwzhh0duWJb78Ys0fnsflnptzp6Jee/KxLh01ZEt+YInc2JkdnV57E5v5Y3NuLxfUtBb5/vQ1pa2trVCoVarWanj178uGHH1KixOuX8RVM08Bm7lQoYcWcv6+TnC5fx/yc+OvyQ1SNfIn5a1mxWkwnp2r8N3vJhrORXHuYIHc6eiWlaVmy7ybVnW2K5awrBUnUbMP77c8DTN6TzJcdKRyLemiSsT04BU2JqiQ3HCJ3NiYprVInkhp8gvX57zG/+Yfc6byWzbG5qGNuEd9+EZK5mHM/K8n1PiK9dANsD01FkfK0QPettyG9c+dOOnXqROPGjWnUqBEeHh5i4EohZaFWEtipGpFxqaw6fFfudLJ14X4c0/+6ilcnHxbMnIH58R8IW9QL8+M/ML8YLKaTU8PeqIidhZr5e26Y/Iwsa47d42FCGv5igKHRiZpteEvnB7G6m5p2ldSYqRT/LeohsXS+aXaZsD61AlXcXRJazwKVuPqTncQWAaSXboDdnvEo4+7JnU6WzCKPYvXvapLrDSDd9Q250zFdShXx7eajSIkp8KsMejvOzZ8/n6+//poqVQr/hPQCNHJ15O16Lvx8KpzONZ1NbkW5B3EpfL75Is62FszrXhtH6wa82/vd4jXZfA45WJkxolUlZv19nR2XH/Jm7TJyp5SlW48SWH8ynG61S9OgvIPc6RR5omYbXmFa1EMVcwvrM1+RUu1t0fDSR2VOXOevKPFrF+z/+oyYXhtN64tHWiJ2u8ehs3cnocUkubMxedpStUn2GIb16RWk1uhVYO9/vWeknZycREEuYka1rkwJa3Nm7byORmc6ZzKT0rSM23yRNK2OxT3r4mgtFurQx6eeC7Vd7Jj65WoaNWuMUqWicXNPk5gi8Pkc4FXKOBD23WdUfGr8lfcEUbONIdtFPSqUlymjbEgStgcnI6nMSXxjstzZFAo6e3fivedj9vAsNkfmyJ1OJrZHZqGMCyOu/WIws5Y7nUIhsckYNA4VsdvrD5qCmbJSb0O6bt26jBkzhm3btrFz586Mf0LhZWepxs+7ClceJvB/pyPkTgcArU5i8h9XuPU4keC3alHJSRSNnFAqFDRKPU/E39+T3uwj3MZtJK3pQPwmT5a1Mf3yHOAl2g9l+swgk2jgF3WiZhtelot6/J5MQDtHSEuUO70M5je3Y35vP4nN/NDZmOYVKlOUVqUbyfUGYP3vKsxv/y13OgCYhR3E6sI6kht8gqZcU7nTKTz+Wz5cFXcXmxNLCmSXehvSiYmJWFlZ8c8//7B3796Mf0Lh5l2tFK0ql+Sbf+4QESv/QgMrDt7mwM1oxrWrQvOKxXPJ07xat2oZTl1Hm9R822IOcPmImm14Ly/qMfyQE1+M/pR+7hE4bP8Q0pPkThFFWgK2h6aRXqoOKXU/lDudQieh5WTSS9XFbvdYlPGRsuaiSI3Dbs/naEpUJbG5n6y5FEbprm+QXPNdrM58jerxJf0b5JPePtLBwcFGT0IoeAqFAv/2VXn3h1PM2XWDZb3qolDIMwjs9wsP+PFkOO80KEsfDxO7VFoIhN+5gVuvV+fbDguRb77t8Numl1NxIWq2cfTy7UMv3z6ZxmvEX2+B3d8jcdg+gNhua2W9/G59YjGqxAfEdfkGlGLe8FxTWxLfeSWOIV2x3/kZMW9vAJU83Qtt/glCmfiAmF6bi+VqlIaQ+MYXWNzdhd1eP2J8fwelymj70ntG+sGDBwwfPpwWLVrQsmVLRo4cyYMHD4yWkFBwXOwt+cyrIkfvPOWvK49kyeF0eAzBf1+nWQVHPvcu3vND55VrxaomNd/2tosPUDu5mlROxYmo2QUntVoP4jsswyzyGA7bBxb4MuIbQ0No07wOKpWSpkMWsvZJEzQujQs0h6JE61iZhHbzMHtwEpvjC2TJwfzObqwu/0qyx2doXMRsO3klWZYgwatglg/X25AOCAjA29ubgwcPcuDAAdq1a0dAQIBRkxIKzjsNy1G3rB2L9t4kJjm9QPcdHpOM/5ZLlHewJPit2qjFtGh5Eug3gcRdX2aabzt6x1Iqt/+ApLSCmy9co5NYtPcmQX9eo2G3gST8/aWYA1wGomYXrNTqbxPfYSlmkUcLtDG9MTSEOVNGs8IrmpRAW5Z3tWTaL0dMd27rQiK1Wg+Sa3+A9ekVmN0t2C5RipSn2O71R+NUk8SmYwt030VRarUepLq3M/ry4Xob0k+ePMHX1xe1Wo1araZXr148eVI4VsYT9FMpFUzqWJ24VA1L9xfc8uEJqRrGbboIwOKedbGzFJci88rXtzfzZ2Seb/vdIZ9zz6kxg345Q3iM8T/YY5PTGR16nl9OR/CuRzm2zBkn5gCXiajZBS+1es9nK+VFHMbhj48KZLaApfODWNNVyjy39ZuY7NzWhUlCq2lonGpiv2s0ysSCu5pje3AKypRo4tsvAZVFge23yHpx+fADgUZbPlxvQ7pEiRJs2bIFrVaLVqtly5YtODo6GiUZQR5VnW34sIkrv24IoZ5nI6NPoabRSQRsu8y9mGTm+tTGrYToA5Zfvr69OXX0JDqtllNHT7LYbxjLetXjUUIaH/50hsO3jdeQuvk4kQHrz3AmIpbJnaoz3rsqapXylZxEI7pgiJotj9QavsR3WIxZ+D84bB9k3Ma0pOPq7XC83DP3+zTVua0LHbUVcZ2/RqFJxm7nCNBpjL5L81s7sLy2iaTGo9A41zX6/ooLnb0ric392bjtL9o0qYJKpaRN8zoGvXKjtyE9e/ZsduzYwRtvvIGXlxd//fUXs2fPNlgCgmkoEXmc2IM/QstBRp9Cbcm+mxy985SADlVp7OZo8PjCM80qlmDtBx642FswZuMFfjh2z+ArIO67/phBP58lRaPj6z4N8KnnYtD4Qu6Jmi2f1BrvEN9+EWbhh3D442OjNKbVUWdw/M2HWqUUWc9tXdHV4PssjrQlqhLfJhjzyKNYn1hs1H0pkqOx2zeRdOd6JDUeadR9FUfrr9syaW8aK9omkRJoywqvaOZMGW2wxrTe6+nlypXj66+/NsjOBNO1YNF8SnYZhWWF+gDP/v9vujJDnkn87Wwkv56J5IPGrvSoV9ZgcYWsuTpasfq9hsz46xorDt3hysMEpnSugbV5/kYw6ySJ1UfvserwXWq72DHfpzal7cSlSFMgara8Umv2BknCbs/nOOwYTGzX1aC2zHdcRdJjbI7Owery/6G1Ls24Tz9m0FfrWdNVg5e7ikP3tAzaoWDi9KkGOAoBILXmO6REHMb65DLSy7Ug3c3L8DuRJOz2B6JIjSe+x2LZZgopypYumMHq7ha0q/Ssyduukpo1XTUMnx9EL98++Y6fbUP622+/5ZNPPmHGjBlZTov2xRdf5HvngunIdgq1X28Qk5yOo1X+/7iP3XnKgj038KpckpGtK+U7npAzVmYqZnWrSa0ytnx58DZ3niQx36dOnrvUJKVpmfbnVfZef0zXWqUJ7FgNSzPjTS0k5Iyo2aYjtVYfFJIO271+/zWmv8t7Y1qnwer8WqyPL0ShSSKp4VCSmoyhu7kd6WWbMXx+EFfvhFOjoisTp081SMNA+J/41jNRR53B/u+RPHn3LySb0gaJuzE0hKXzg7h6O5xapRR8/tE7vOVU0yCxhcyu3gnH633bTLcZshtUtg3p50vM1q0r+uoUB8+nUHt+RhqeTVemdnKly9dHaVGxBF1rlaZ1Fac8NZruRCcxcdslKjnZMLNbTVRiho4CpVAo6N/EjerOtgRuv8yA9WeY2a0mLSvlbvGbiNhkxm++xK3oRMa0qcz7jcvLNv+4kJmo2aYlpXZfQMJurx/2fw4hruu3uR5AZhZxGNsDk1E/uUqaWxsSWgWhLfG/KSSzmttaMDAza+I6f0WJ397CftcoYruvz/ecxM9nXFnTVcLrfdtnVxO+30ZapRDxRcgIalR05dC96Iwz0mDYblDZNqS9vb0BsLS0pGvXrpnu27Fjh0F2LpiOQL8J+E2eDB1GYOFam9TwSyTu+pKJfpPQVS7PzisPOXTrCdZmKtpWc6JLrdI0cS+RoynrYpLTGbv5AuYqJYt61sHGXMzQIZfn/ab9f7/EmI0X+NSrIgObuuWoMXzi3lMCtl5GJ8HSXnXFCpQmRtRs05NS+z2QdNjtm4D9jiHEdV2Vo8a0Mj4Sm8MzsLyxFa2dG7FdvyOtUmcQX1ploXWqSUKrGdjt9WPbwk+YH3os4yrAaD89VwEkHcqkhyjjwlHF3UMVH86yGXMzZlwBw3c1EDIb7TeVQVNGG60blN4WzapVq14pylndJhRuz/tBz54/l7CQG7hWrMq0F6YrG9GqEmcjYtlx+SF7rj3mj0sPKWltRscaznSpVZo6LnZZNsbSNDombr1EVHwqX/dpQFn7/PcVFPLnxX7TKw/d4aqeftOSJBFyJpLF+27iXsKahW/nvVuIYHyiZpuWlDofABJ2+yZi/+fQZysPZteY1qRgfXYV1qeWg6QjsennJHkME6vbmYCUWn3ZGPITM34OZbWP1X9nkqMZNGUUirQ4erdrhCo+DGVcGKr4cFRxYSjjw1DFR6DQpmaKdeV+PF7udpluEzOuGM/zLyfG6gaVbUN6//79HDhwgKioKGbOnJlxe0JCAipV/i5r7Nixgy+//JKbN2+yYcMG6tWrl694gmH4+vbG17d3lpcJVUoFjd0caezmiL93Vf65/YS/rjxk07n7/HomEldHSzrXLE2XWqWpWNKa0NANzxrlt29g5uTKoE/HUL9cK5mOTHhZTvtNp2l0zN19nd8vRNG6ihNBXWtgayGuKJgiY9ZsEHU7P1Lq9AMJ7PZPxP7PYf81ps3/9wBJwvzOLmwPTUMVd5fUKm+S0HIKOnsxA4fJUCiYu+M2q32sXjmTPHLmeIY8+V8fXJ1lSbT2bmicapNWqRNae3d0dq5o7dzQ2rlS40dPo3Y1EF5lzG5Q2X4ililThrp167Jnzx7q1KmTcbuNjU2+V8mqXr06y5cvZ+pUMbq4MDJXK2lXrRTtqpUiIVXDnuuP+evyQ74/do/VR+/hEH6UaztW49BpJO69nnUTWf/VAhqWcxBzCZuQF/tNT/qv33RH9VV++2EF4XduUM69Cs6t+vK0XDM+bu7OkJYVUIpLyybLmDUbRN3Or5S6/QAddvsD2THtTeb8ff/Z2TF3FyZ2LMuH5W6gKVGNGJ9fSHcTJx1M0dW7EXh98OqgtcuPdcR2+wHtf41lzG1eG8fYXQ2EgpVtQ7pmzZrUrFmT7t27o1Yb9gzU80ExQuFna6HGp64LPnVdeJyQys6rjwgcOAKHTiONPpWeYBjNKpZgbT8P+k1dxvLt3+HUdTRu/30BuvD7UoaOKcmwN1rLnaaghzFrNoi6bQgpdT9kw65jzPz1pxe6B8Ty8YYotMP68ta7y8X0ZyYs20FrldxIq9ghx3GM3dVAKFjZVtvRo0ezdOlSevbsmeX9W7duNVpSQuFUytaC9xu7MvbRPUq5ZjGVXsgNmTIT9CnvYEXskRCcuo7O9AXIqetofv/xK6YO/1jmDAV9RM0uHBb8375Xuges9rFkeMhB3honGtGmzJBnksWMK0WHQspmqbOHDx9SunRpIiIistywfPnyrw08cOBAHj9+/MrtY8aMoUOHZ9/c+vfvj7+/v+hrV8RUqlaTZM8BmabSS7l7DquTa7l9/YqMmQmvo1SpcBu3EYXqf9+vJa2GsEW90Gm1r9lSMAX5rdkg6nZBUKmUpATaYqb6XzepdK2E5ewEtFqdjJkJOfHL+vXMCprE5Zv3qFXFnUlTZ/HeBx/InZYgo2zPSJcu/WzS8RIlSmBpaYlSqeT27dvcunWL1q31X+b94YcfDJZkVqKjE9DpcrfcsSG/+RkqVlHMacLn/llOpTdtxow8xzWVYzNGLFPJKbu5xF0rVs1XfkXtecpvHKVSgZOTrf4H5lJ+azYU7bptKnFeN6et3PXRkLGKak4dOvnQoZNPpliiPho3jiFjGaNmK/UF6NevH6mpqURFRTFw4EA2btzIxIkTc5WEULz4+vZm/owZmB//gbBFvTA//gPzX5hKTzBNgX4TSNz1JSl3zyFpNaTcPUfiri8J9Jsgd2pCLoiabdpG+01l0A4Fe29rSNdK7L2tYdAOBaP9xEAzQSiM9I5IkSQJKysrfvvtN/r168cnn3zC22+/na+d/v3338yYMYMnT54wdOhQatWqxerVq/MVUzAtr5tKTzBN+uYSFwoHY9RsEHXbUMRAM0EoWnLUkD5z5gxbt25l1qxZAGjz2V+yY8eOdOzYMV8xBEEwPPEFqPAzRs0GUbcNSQw0E4SiQ2/XjsDAQL755hs6dOhAtWrVCAsLo1mzZgWRmyAIgpBLomYLgiAUHL1npJs2bUrTpk1JSEggMTERNzc3vvjii4LI7bWUyrwtDJHX7YwZS+RUOOMYMpbIqWDjGDJWbuMY8hiyYqo1G+Sv26YWx5CxRE4FG8eQsUwtjiFjFYWc9D0+2+nvnrt69SoTJkwgNjYWSZIoWbIkc+fOpVq1arlKRBAEQTA+UbMFQRAKjt6GdN++fRkzZgzNmzcH4NixYyxevJj/+7//K5AEBUEQhJwTNVsQBKHg6O0jnZSUlFGQAZo1a0ZSUpJRkxIEQRDyRtRsQRCEgqO3j7SbmxsrVqygR48eAPz++++4uroaPTFBEAQh90TNFgRBKDh6u3bExsayfPlyTp06BYCnpycjRozAwcGhQBIUBEEQck7UbEEQhIKTbUM6NTWVX375hXv37lG9enV8fX0xMzMr6PwEQRCEHBA1WxAEoeBl25AeM2YMarUaT09PDhw4QPny5Zk0aVJB5ycIgiDkgKjZgiAIBS/bPtI3b95k69atALzzzjv07i2WCRYEQTBVomYLgiAUvGxn7VCr1Vn+LAiCIJgeUbMFQRAKXrZdO2rVqoWVlRUAkiSRmpqKpaUlkiShUCg4ffp0gSaaHwEBAezbtw8nJye2bduW5zj379/H39+fx48fo1Qq6dOnDwMGDMhTrNTUVD744APS0tLQarV07tyZUaNG5Tk3rVaLr68vZcqU4ZtvvslzHG9vb2xsbFAqlahUKjZu3JinOHFxcXzxxRdcu3YNhULB7Nmz8fDwyHWcW7duMXbs2Izfw8LCGDVqFAMHDsx1rB9++IENGzagUCioXr06wcHBWFhY5DrO2rVr2bBhA5Ik0bt371zlktV7MSYmhrFjxxIREUH58uVZsmSJ3oFhWcXZsWMHX375JTdv3mTDhg3Uq1cvzznNnTuXvXv3YmZmhru7O8HBwdjb2+c6zpIlS9i9ezdKpRInJyeCg4MpU6ZMnnJ6bvXq1cybN48jR45QsmTJXMdZvnw5ISEhGduOGzeONm3a5DmnH3/8kZ9++gm1Wk2bNm3w9/fXG8sYRM3OmqHqtqjZOWOKNRvyXrcNVbOzi5WXum2omp1drLzUbUPV7Oxi5aVuF1jNloqB48ePSxcuXJC6deuWrzhRUVHShQsXJEmSpPj4eKlTp07S9evX8xRLp9NJCQkJkiRJUlpamvTOO+9IZ86cyXNua9askcaNGycNGTIkzzEkSZLatWsnRUdH5yuGJEmSv7+/FBISIkmSJKWmpkqxsbH5jqnRaKSWLVtK4eHhud72wYMHUrt27aTk5GRJkiRp1KhRUmhoaK7jXL16VerWrZuUlJQkpaenSwMGDJBu376d4+2zei/OnTtX+uabbyRJkqRvvvlGmjdvXp7i3LhxQ7p586bUr18/6dy5c/nK6eDBg1J6erokSZI0b968POcUHx+f8fPatWulyZMn5zknSZKkyMhIadCgQVLbtm1z9D7NKs6yZcuk7777Lkd56It15MgRacCAAVJqaqokSZL0+PHjXMcVXmWomi1JhqvbombnninUbEnKX902VM3OLlZe6rahanZ2sfJStw1Vs7OLlZe6XVA1W++CLEVBkyZNDDL1U+nSpalTpw4Atra2VK5cmaioqDzFUigU2NjYAKDRaNBoNCgUeVtH/sGDB+zbt4933nknT9sbWkJCAidOnMjIx9zcPEffjPU5cuQIbm5ulC9fPk/ba7VaUlJS0Gg0pKSkULp06VzHuHnzJg0aNMDKygq1Wk2TJk34+++/c7x9Vu/F3bt38/bbbwPw9ttvs2vXrjzFqVKlCpUrV85xLq+L5eXlldE9oGHDhjx48CBPcWxtbTN+Tk5OzvF7PLu/2eDgYPz8/PIdJy+yivXLL78wZMgQzM3NAXBycjLIvoo7Q75uhqrbombnninUbMhf3TZUzc4uVl7qtqFqdnax8lK3DVWzXxcrtwqqZheLhrQxhIeHc/nyZRo0aJDnGFqtlh49etCyZUtatmyZ51izZ8/Gz88PpdIwL+fHH39Mr169+PXXX/O0fVhYGCVLliQgIIC3336bSZMmGWRlte3bt/PWW2/ladsyZcowaNAg2rVrh5eXF7a2tnh5eeU6TvXq1Tl58iRPnz4lOTmZAwcO5LhgZSc6OjrjA6J06dI8efIkX/EMLTQ0lNatW+d5+8WLF9OmTRu2bt3K6NGj8xxn9+7dlC5dmpo1a+Y5xnPr16+ne/fuBAQEEBsbm+c4d+7c4eTJk/Tu3Zt+/fpx7ty5fOcmGE9+67ao2bljCjUbDF+3i3rNBsPUbUPWbDBM3TZGzRYN6TxITExk1KhRBAYGZvrmllsqlYotW7awf/9+zp07x7Vr13IdY+/evZQsWZK6devmOY8X/fLLL2zatIlvv/2W9evXc+LEiVzH0Gg0XLp0iffee4/NmzdjZWXFqlWr8pVXWloae/bsoUuXLnnaPjY2lt27d7N7924OHjxIcnIyW7ZsyXWcKlWqMHjwYAYNGsTgwYOpUaMGKpUqTzkVBl999RUqlQofH588xxg7diz79++ne/fu/PTTT3mKkZyczNdff52vhvhz7733Hn///TdbtmyhdOnSzJkzJ8+xtFotcXFxhISE4O/vz5gxY5Bev8aVIBND1G1Rs3POVGo2FK+6bYiaDfmv24as2WC4um2Mmi0a0rmUnp7OqFGj6N69O506dTJITHt7e5o1a8bBgwdzve3p06fZs2cP3t7ejBs3jqNHjzJ+/Pg85/J8QIGTkxMdO3bM07c1FxcXXFxcMs7WdOnShUuXLuU5J4ADBw5Qp04dSpUqlaftDx8+jKurKyVLlsTMzIxOnTpx5syZPMXq3bs3mzZtYv369Tg6OlKhQoU8xXnOycmJhw8fAvDw4cMcDcYoCJs2bWLfvn0sWLAgz5ewX/TWW2+xc+fOPG177949wsPD6dGjB97e3jx48IBevXrx6NGjXMcqVaoUKpUKpVJJ7969OX/+fJ5ygmd/Lx07dkShUFC/fn2USiVPnz7NczzBOAxdt0XN1s+UajYYtm4Xl5oNea/bhqzZYLi6bYyaLRrSuSBJEpMmTaJy5cp89NFH+Yr15MkT4uLiAEhJSeHw4cN56t/6+eefc+DAAfbs2cOiRYto3rw5CxYsyFNOSUlJJCQkZPz8zz//UK1atVzHcXZ2xsXFhVu3bgHP+slVqVIlTzk9t337drp165bn7cuVK8e///5LcnIykiTlK6fo6GgAIiMj2blzZ54vXT7n7e3N5s2bAdi8eTPt27fPVzxDOHDgAN9++y1fffVVxkwQeXHnzp2Mn/fs2ZOn9zhAjRo1OHLkCHv27GHPnj24uLiwceNGnJ2dcx3r+QcgwK5du/L0Hn+uQ4cOHD16FIDbt2+Tnp5OiRIl8hxPMDxD1W1Rs3PHlGo2GLZuF+WaDYap24as2WC4um2Mmp3t9HdFybhx4zh+/DhPnz7FycmJkSNH5mmxgpMnT/LBBx9QvXr1jL5tOZ0662VXrlxh4sSJaLVaJEmiS5cujBgxItdxXnTs2DHWrFmT56mUwsLCGD58OPDs8sdbb73Fp59+mqdYly9fZtKkSaSnp+Pm5kZwcHCeBw8kJyfTtm1bdu3ahZ2dXZ5iACxbtow//vgDtVpNrVq1mDVrVsaAg9x4//33iYmJQa1WExAQQIsWLXK8bVbvxQ4dOjBmzBju379P2bJlWbp0KY6OjrmO4+joyIwZM3jy5An29vbUqlWL1atX5ymnVatWkZaWlpFHgwYNmD59eq7jHDhwgNu3b6NQKChfvjxBQUE5mv5O39+st7c3v/32m94zQVnFOX78OFeuXAGgfPnyTJ8+PUeDmLKK1aNHDwIDA7ly5QpmZmb4+/vn6v0gZM1QNRsMV7dFzc45U6vZkPe6baianV2svNRtQ9Xs7GLlpW4bqmZnFysvdbuganaxaEgLgiAIgiAIgqGJrh2CIAiCIAiCkAeiIS0IgiAIgiAIeSAa0oIgCIIgCIKQB6IhLQiCIAiCIAh5IBrSgiAIgiAIgpAHoiEtGEytWrXo0aMHb731FqNGjSI5OTlX20dFRTFq1Cjg2VRM+/fvz7hv9+7d+V5p67m+ffsaJI4gCEJhJ+q2IOSPmP5OMBgPD4+Mlac+//xz6tatm+cFEDZu3MiFCxeYMmWKIVMUBEEQXiDqtiDkj1ruBISiydPTk6tXrxITE0NgYCBhYWFYWVkxffp0atasyfHjx5k1axYACoWCn376iZiYGIYNG8bGjRtZtmwZKSkpnDp1iqFDh5KSkpJRoCMiIggMDOTJkyeULFmS4OBgypUrx8SJE7G1teXChQs8evQIPz8/unTp8kpuzz84jh07xpdffkmJEiW4du0aderUyXJp1XXr1vF///d/qFQqqlatyuLFi0lKSmLGjBlcu3YNrVbLiBEj6NChAykpKQQEBHDjxg2qVKlCREQEU6ZMoV69epk+sP7880/27dvHnDlzePLkCVOnTiUyMhKAwMBAGjduzPLly4mMjCQ8PJzIyEgGDBjAhx9+CDxbTWv16tUoFApq1KjB/Pnzs40jCIKQE6Jui7ot5IEkCAbSsGFDSZIkKT09XRo2bJi0fv16afr06dLy5cslSZKkw4cPSz4+PpIkSdLQoUOlkydPSpIkSQkJCVJ6eroUFhYmdevWTZIkSQoNDZWCgoIyYr/4+9ChQ6WNGzdKkiRJGzZskD799FNJkiRpwoQJ0siRIyWtVitdv35d6tChw2vzPHr0qNSoUSPp/v37klarlfr06SOdOHHilce/8cYbUmpqqiRJkhQbGytJkiQtXLhQ2rx5c8ZtnTp1khITE6U1a9ZIEydOlCRJki5fvizVqlVLOnfuXKb9SpIk7dixQ5owYYIkSZI0bty4jP1GRERIXbp0kSRJkpYtWya9++67UmpqqhQdHS01bdpUSktLk65duyZ16tRJio6OliRJkp4+ffraOIIgCNkRdVvUbSF/xBlpwWBSUlLo0aMH8OzMxjvvvEOfPn1Yvnw5AC1atCAmJob4+HgaNWrEnDlz6N69O506dcLGxibH+zlz5kxGzB49ejB//vyM+zp06IBSqaRq1ao8fvxYb6z69evj4uICQM2aNYmIiMDT0zPTY2rUqMH48eNp3749HTp0AODQoUPs2bOHNWvWAJCamsr9+/c5ceIE/fv3z4hXo0YNvTkcPnyYGzduZPyekJBAQkICAG3atMHc3JySJUtSsmRJoqOjOXr0KF26dMlYavX5krDZxbG1tdWbgyAIxZOo26JuC/kjGtKCwVhaWrJly5ZMt0lZdMFXKBQMGTKENm3asH//fvr06cP333+PhYVFnvb74iU9c3PzXG374uNVKhVarfaVx6xatYoTJ06wZ88eVq5cyfbt2wFYtmwZlStXfm0+2UlNTc34WafT8euvv2Jpaak3P41Gk+Vzqi+OIAhCVkTdfjWf7Ii6LWRFzNohGFWTJk34/fffATh27BglSpTA1taWe/fuUaNGDYYMGULdunW5fft2pu1sbGxITEzMMqaHh0dGUdy6datR+5PpdDru379P8+bN8fPzIz4+nqSkJLy8vPjpp58yiuOlS5eAZ8e7detWAK5du8bVq1czYpUqVYqbN2+i0+nYtWtXxu3PYz13+fLl1+bUokUL/vzzT54+fQpATExMnuIIgiBkRdRtUbeFnBMNacGoRowYwYULF+jevTsLFy5kzpw5AKxdu5a33noLHx8fLC0tad26dabtmjVrxo0bN+jRowd//PFHpvu++OILNm7cSPfu3dmyZQuTJk0yWv5arRY/Pz+6d+9Oz549GThwIPb29nz22WdoNBp8fHx46623WLp0KQDvvfceSUlJdO/ene+++4769etnxPr8888ZOnQoAwYMwNnZOeP2SZMmZTxHb775Jr/88strc6pWrRrDhg2jf//++Pj4ZDynuY0jCIKQFVG3Rd0Wck5MfycIRtS/f3/8/f2pV6+e3KkIgiAIOSDqtpAb4oy0IAiCIAiCIOSBOCMtCIIgCIIgCHkgzkgLgiAIgiAIQh6IhrQgCIIgCIIg5IFoSAuCIAiCIAhCHoiGtCAIgiAIgiDkgWhIC4IgCIIgCEIeiIa0IAiCIAiCIOSBaEgLgiAIgiAIQh6IhrQgCIIgCIIg5IFoSAuCIAiCIAhCHoiGtCAIgiAIgiDkgWhIC4IgCIIgCEIeiIa0IAiCIAiCIOSBaEgL2erfvz8bNmwA4Pfff2fQoEFG2U+3bt04duxYnratUaMGd+/ezfI+fTm/eHwvCw8Pp0aNGmg0mjzllRsTJ05k8eLFAJw8eZLOnTsbfZ+5MXjwYDZt2iR3GoIg6CFqtqjZIGp2QVPLnYCQNW9vbx4/foxKpcq4rWfPnkyZMkWWfHx8fPDx8TFK7O3btxslrjFzNhZPT0/++usvudPI5LvvvjNK3B9++IEff/yRp0+fYm1tzZtvvom/vz9qtShLQuEjanb+iZptGMaq2c+lpaXh4+NDUlISBw4cMOq+CgPxiWXCvv76a1q2bCl3GoJgFN7e3vTq1Qt7e3tiYmIYNWoUP/74Ix999JHcqQlCnoiaLRQHq1evxsnJiaSkJLlTMQmia0chtHHjRt577z3mzp1LkyZN8Pb2Zv/+/Rn3x8TEEBAQgJeXF02aNOGzzz7LuC8kJISOHTvStGlThg0bRlRUVMZ9//zzD126dKFx48ZMnz4dSZJe2edzNWrU4JdffqFTp040adKEoKCgjMdrtVrmzJlDs2bN8Pb25qeffnrtJTdvb28OHz4MwPLlyxk9ejT+/v54eHjQrVs3zp8//9rn4/Dhw1nm8XLOrzs+rVbL3LlzadasGe3bt8/0fALEx8cTGBiIl5cXrVq1YvHixWi12hy9Hi+7dOkSPXv2xMPDgzFjxpCamppx37Fjx2jdunWm5+a7776je/fuNGzYkMDAQB4/fszgwYPx8PBg4MCBxMbGZjz+7Nmz9O3bF09PT3x8fDJdfu3fvz9Lliyhb9++eHh4MGjQIJ48eQJAamoq48ePp1mzZnh6euLr68vjx48ztnt+OVWn07Fy5UratWtHixYt8Pf3Jz4+HvjfpdVNmzbRtm1bmjVrxldffZXt8+Du7o69vT0AkiShVCqzveQrCIWZqNmZiZpdOGs2QFhYGL///jtDhgx57eOKE9GQLqTOnTtHpUqVOHr0KIMHD2bSpEkZRcbf35/k5GS2b9/O4cOHGThwIABHjhxh4cKFLFmyhEOHDlG+fHnGjRsHwJMnTxg5ciRjxozh6NGjuLu7c/r06dfmsG/fPn777Te2bNnCjh07OHjwIPCs8B84cIAtW7awadMmdu3alatj27NnD926dePkyZN4e3szY8aMPOXxIn3HFxISwt69e9m8eTOhoaH8+eefmbafMGECarWanTt3snnzZv75559MffVe93q8KC0tjeHDh9OjRw+OHz9Oly5d2Llz52uPb+fOnXz//ff89ddf7N27l08++YRx48Zx7NgxdDodP/74IwBRUVEMHTqUTz/9lOPHjzNhwgRGjRqVUXgBtm3bRnBwMEeOHCE9PZ01a9YAsGnTJhISEti3bx/Hjh0jKCgIS0vLV3LZuHEjmzZtYt26dezatYukpCSmT5+e6TGnTp3izz//ZO3ataxYsYKbN29me2xbt26lUaNGNG/enCtXrtC3b9/XPheCUFiJmq0/jxeJmv2MqdXsmTNnMm7cuCz3VVyJhrQJGz58OJ6enhn/QkJCMu4rV64cffr0QaVS0bNnTx49esTjx495+PAhBw4cICgoCAcHB8zMzGjatCnwrNHi6+tLnTp1MDc3Z9y4cZw9e5bw8HAOHDhA1apV6dKlC2ZmZgwYMIBSpUq9Nr9PPvkEe3t7ypUrR7Nmzbhy5QoAO3bs4MMPP8TFxQUHB4dcf3Nt3Lgxbdq0QaVS0aNHj4y4uc3jRfqOb8eOHQwYMICyZcvi6OjI0KFDM+57/PgxBw4cIDAwEGtra5ycnBg4cGCmfoLZvR4v+/fff0lPT2fAgAGYmZnRpUsX6tWr99rj69evH6VKlaJMmTJ4enpSv359ateujbm5OR07duTSpUsAbNmyhdatW9OmTRuUSiVvvPEGdevWzXSmpVevXlSqVAlLS0u6dOnC5cuXAVCr1cTExHD37l1UKhV169bF1tb2lVy2bt3KwIEDcXNzw8bGhnHjxvHHH39kOnM1YsQILC0tqVmzJjVr1nzt69e9e3dOnz7NX3/9Rd++fXFycnrtcyEIpkzUbFGzoejW7L///huNRkPHjh1fe/zFjegjbcJWrFiRbX+7FwuKlZUVAElJScTGxuLg4ICDg8Mr2zx8+JA6depk/G5jY4OjoyNRUVE8fPgQFxeXjPsUCgVly5Z9bX7Ozs6ZckhMTMzYz4vbvhg3J148NktLS1JTU9FoNNkOQssujxfpO76Xcy5XrlzGz5GRkWg0Gry8vDJu0+l0mR6f3euRVR5lypRBoVBkua+svBjbwsLilefn+X4iIyP5888/2bt3b8b9Go2GZs2aZfz+8nP1fNsePXrw4MEDxo0bR1xcHD4+PowdOxYzM7NX8i9fvnzG7+XLl0ej0RAdHZ3tc5GTfnQVK1akWrVqBAUF8eWXX+p9vCCYIlGzRc1+OXZRqdlJSUnMnz+fVatWvfbYiyO9DemYmJhXbrOxsXnlxRJMg4uLC7GxscTFxWX0P32udOnSREREZPyelJRETEwMZcqUwdnZmQcPHmTcJ0kS9+/fz1MOL8d68We56Ds+Z2fnTL+/+LOLiwvm5uYcPXo03zNKODs7ExUVhSRJGYU5MjISNze3fMUFKFu2LD169GDmzJm53tbMzIwRI0YwYsQIwsPDGTJkCJUqVaJ3796ZHvfyeygyMhK1Wo2Tk1O+X2eNRsO9e/fyFUMQNbuwETU7a6Jmv15B1+y7d+8SERHBBx98AEB6ejrx8fG88cYb/Prrr7i6uub6GIoKvV07evXqRYsWLejcuTOdO3emRYsWtG/fnp49e3LhwoWCyFHIhdKlS9O6dWuCgoKIjY0lPT2dEydOAM8uo2/cuJHLly+TlpbGokWLqF+/Pq6urrRp04br16+zc+dONBoN69aty/IyV0507dqVdevWERUVRVxcHN9++60hDzFP9B1f165d+fHHH3nw4AGxsbGZvnWXLl2aN954gzlz5pCQkIBOp+PevXscP34813k0bNgQtVrNunXr0Gg07Ny5U+/AnJzy8fFh7969HDx4EK1WS2pqKseOHctRsTx69ChXr15Fq9Via2uLWq3ONI3Xc2+99RZr164lLCyMxMREFi9eTNeuXfP0YbVhw4aMsyI3btxg1apVtGjRItdxhMxEzS5cRM3OmqjZr1fQNbtatWrs27ePzZs3s3nzZmbOnImTkxObN2/WeyWkqNPbkPby8mLVqlUcO3aMY8eO8e2339K1a1emTp1KUFBQQeRYbA0bNgwPD4+Mf8OHD8/RdvPmzUOtVtO1a1datmzJ2rVrAWjRogWjR49m5MiReHl5ERYWljGpfMmSJVm6dCkLFy6kWbNm3L17l0aNGuUp7z59+vDGG2/g4+PD22+/TZs2bbL9Iy8o+o6vT58+eHl50aNHD3r27EmnTp0ybT9v3jzS09N58803adKkCaNGjeLRo0e5zsPc3Jzly5ezadMmmjRpwh9//GGw/mZly5Zl5cqVfPPNN7Ro0YI2bdqwevVqdDqd3m0fP37MqFGjaNy4MW+++SZNmzbNcj5XX19ffHx86NevH+3bt8fc3JzJkyfnKd/Tp09njGwfMmQIrVu3zhhIJeSdqNnyETXbcETNfr2CrtlqtRpnZ+eMfw4ODiiVSpydnWV9n5gChZTVMNUX9OrVi40bN2Z5W48ePdiyZYtRExQKv/379zNt2rRM/cAEQTAOUbOF/BI1WxByTu8ZaUdHR1atWkVERAQRERF8++23ODg4oNVqUSrFpB/Cq1JSUti/fz8ajYaoqChWrFhBhw4d5E5LEIoFUbOF3BI1WxDyTu8Z6SdPnrBixQpOnTqFJEk0btyYESNGYGtry/3796lQoUJB5SoUEsnJyfTr149bt25haWlJ27ZtmTRpUpZT8wiCYFiiZgu5JWq2IOSd3oa0IAiCIAiCIAiv0jts8/bt26xZs4aIiIhME3ivW7fOqIkJgiAIuSdqtiAIQsHRe0bax8eHvn37Urdu3Uz96+rWrWv05ARBEITcETVbEASh4Og9I61Wq3n//fcLIhdBEAQhn0TNFgRBKDh6z0gvX76ckiVL0rFjR8zNzTNud3R0NHZur/X0aSI6Xe66dzs52RIdnWCQ/RsqlsipcMYxZCyRU8HGMWSsvMRRKhWUKGGT731nx1RrNshbt00tjiFjiZwKNo4hY5laHEPGKio56avZes9Ib9q0CYDVq1dn3KZQKNi9e3euEslKQEAA+/btw8nJiW3btuVqW51OynVBfr6doRgqlsipcMYxZCyRU8HGMWQsQ+ZkCKZas0H+um1qcQwZS+RUsHEMGcvU4hgyVlHPCXLQkN6zZ49Bd/iiXr160a9fPyZMmGC0fQiCIBQnRaVmh4aGsCR4BtfD7lHNzZ0xAZPx9e1j9P0KwuuI96Xwsmwb0keOHKFFixbs3Lkzy/tfXo4zL5o0aUJ4eHi+4xRGoaEhBC+cQdite7hVdifgc/HHKAhC3hWlmh0aGsJs/7FMd3SkUbXqnE5OYor/WABRJwXZiPelkJVsl7k6ceIEAHv37s3yn5B3oaEh+AeNRemroPaq2ih9FfgHjSU0NETu1IRiLjQ0BM+W9VCqlHi2rJev96QhYwn6FaWavSR4BtMdHWlmbYOZQkEzaxumOzqyJHiG3KkJxZh4XwpZkX1BlvDwcIYNG5an/naFVeUalZF6StjW+t+qUQmXE0gMSWTjwY1UcqhEWZuyKBViOV+h4Kz/eT2fjf+Mkh+WxKaaDYnXE3my7gkrF6zkg/c/kC2WYFoKomarlErOVKuOmUKRcVu6JOFx/Rpanc5o+xWE13nd+zLxwgUsqldHodbbY1YoYrJ9xb///vvXbvjRRx8ZPJnciI5OyHWHcWdnOx49ijfI/vMaS6vTcOfGHWpXq53pdptqNty5c4dPd30KgLnSHFcbd9xtK+Bm4467TQXcbN1xs3HHSm2daVtDdxMxhefJ1OMYMpap5BQ4dRIlPyyZ8QXPtpYtfAjDA4ZztNQJ0nXpaHTpaHQaNJLmv981aKRn/6f/979Gl85J/5OU+bDMK7ECp06iU0efAj82Y8XKSxylUoGTk+GXXjb1mg05r9vV3Nw5nZxEM+v/jZQ/nZxEZTMzbrzfD4u3emDRuh0KC4sc77uo/b0aI44hYxWlnDR375D09ZdUNjPL9n15u5cvWFqirlkbszp1Udeph1mduihLOhklJ2PFMWSsopKTvpqdbUM6MTEReLZK1vnz5/H29gaeXTb09PTMVRLCM5eeXmDJxfmYlzUn8XpipjPSidcTca3sxuLmKwhLuEdY4l3CEu5xLfYKB+7vRcf/zsI4W5bGzcYdN9sKPDh4n/9bsR7ngc7UrlabxOuJ+AeJPltC7oXdupf1F7ywO9yOv4lKocZMqUatNMNMYYaV2gozhRlqpRlqpRq1Qo2Z8tnvB+8fwqaazSuxLt26VJCHVKwUpZo9JmAyU/zHMh1oZGXN6eQkJj99it8776KNjCBh+hQS7eyx6NQFy+5vo65SVe6UhSJI9/QJSWu+JWXrZhSWlozs/R6Tt2xkBi++L2MImDYLu4qVSL94Hs3FCyT/+jP8t6qosmxZzOrUQ12nHuradVFXq47CzEwMWixCsm1IjxgxAoBBgwaxceNGbG1tM24fPXq0QXY+btw4jh8/ztOnT2ndujUjR46kd+/eBoltSuLS4vju6ldsD/udkhZODB3xGd8v/Q4GkHHZO3ptNPOmLqZBSQ8alPTItH2aNo2IpHDCEu4SlniPe/81sndF/MnZb85SdmDZzGf+BkDwwhnij1LIsdi0GGzL22X5Bc+9SgW+b/1zruKtr7wuy1il3J2RJAnFC5dGBcMoSjX7ee2aEzyD69evUc3NnUnzl+Dr2wdJpyP99ElStv1Oyu+bSAkNQV2rDpZv9cC8Q0eU1sabo1soHqSUFJJDfiF5/Tqk1BQse/TCeuBg3itRAnOvVi+9LxdnvF8tOjwb0CulpqK5dhXNxfOkXzxP+rmzpO76bxCwuTk7LK1YfP4cM5xLiUGLRYDezjyRkZGZJvU3NzcnIiLCIDtftGiRQeKYKkmS+CviD765soL49HjeqfQuA6p9jHV7G2o51iF44Qwu3bqEW2V35k1dnO0fkLnKnEp2lalkV/mV+C4fOoozf0K+XHhyjulnJ1OyWwkefv8QPnr1C15uBXw++dmVkRe+LN5fcx9nX2dmnp3K2Lr+2JoZvnuDUHRqtq9vH3x9+7xyKVahVGLu2RRzz6boYmNI/WsHKVs3kzB/Nny5GAvvjlh274G6dl0UCoU48yfkmKTTkfrXHyR9+zW6Rw8xb9UG62EjULtXyHhMdu/LFyksLDCrVx+zevWx+u827cMoNBcvoLl0ga8XzGWGc6mMLiLNrG2YzrMvjuK9WfjobUj36NGDd955h44dOwKwa9cuevbsafTECrvb8TdZcmEB55/+S23Huoyt60cV+2oZ9+fkj1EfhUKBW2X3LM/8latUPt/HIBRtOklHyO1f+O7q17hYufDz579xrsG/Of6C9zrPt3kx1qLpy0lrkMr317/jcsxFJjWcRp0S9Qx9WMVecarZSgdHrPq8h2XvvmguXSBl62ZSd+8kdfvvqCpX4c8yLsz/7VdmlCghzvwJr5V28jiJK5ehvX4Nda3a2E2dgVkDD/0b5pCqdBlUpctg0a49N6dPoVH5zJ/RjaysuX79msH2JxQcvQ3pTz/9lNatW3Py5EkUCgXBwcHUrl1b32bFVrImibXX1/DbnV+xVdswvl4AXVy7GW0GjqzO/EWuiaTSu5W5GnOZGo61jLJfoXCLS4tjzrkZHH34D61d2jG+XgC2ZrZU862R7y94z2X3ZdHDqTGzzk5j9NHPGFB1EO9X/RCVQpXfQxL+UxxrtkKhwKxOPczq1MNm5FjSdu8iZdsWlv/8IzPKlBFn/oRsaW7fJHHll6Qf/QelS1nsps7E3LsDCqXxZs3KbjBtVefSRtunYDw5eqeoVCqUSmXGP+FVkiRx4ME+Bh54n5DbP9Ol/JusbfN/vOnW3ajT2Pn69mHe1MXoQiUuDbmELlTii8BpuLVyY8zRzzjwYJ/R9i0UTpdjLjLk0ABOPjrGiNpjmeoxs0C7WdQuUZdvvH6gXdn2fH/9W8YdHUFU8oMC239xUJxrttLGFkuft3Fc9T230tNpZJV5lqNGVtZcD7snU3aCqdBFPyZ+3mxiBn6A5sK/WH82ihI/hWDRoZNRG9Hw32DamBiOJSWSLkkcS0pkctRDPlGpSN70m1H3LRie3nfL2rVrGT9+PE+fPiU6Oho/Pz9+/PHHgsit0IhMiiDw5HimnQ7EzsyeZS2+YXz9ABzMHQtk/76+fTh5+Dw6rY6Th88z7IMRrGj5LVXsqzLtdCD/d/MnZJ4uXDABkiQRevtXRh/5FKVCybIWX9OrYm9ZBv7ZmtkyqeE0JjaYzI2463xycAD77xtvaeviRNTs/3l+5u9Fp5OTqObmLlNGghxCQ0No5VkPlVKJV+O6/DLqU56850vqjm1Y+vahxP9txPq9frmaTjE/fH37EDhvMXMUSjyuX2OOQsmk+Ut4u8ubJC6aR+K3X4nP7EJEb9eO3377jZCQEKytn32r/+STT3j33Xfp37+/0ZMzRS/O2exayY3W/dtwufJFVAo1n9YaRa8K76BSyj8hewmLkixs9iXzzs1k1dWVhCeFMaaOH2oTyE0oeAnp8cw/F8zBqH28UaYV/vUnYWdmL3dadCrflTqO9Zh1dhpBZ77gzcfdGV5rDFZqK/0bC1kSNft/spxG78EDJn4+Qe7UhAKS1bLek0ND0LZqw3sLlqJydZMlr6y6vkm93yVh4VyS132PLvoxtuMDxAIvhUCOu3Zk9XNx8/LS3qp3lISs+D+cLjrzQ+uf6V2pr0k0op+zUFkwqWEQ/at+xB9hW5lwYizx6XFypyUUsGuxVxn6z0ccfniQT2uOZHqjOSbRiH6uvI0ry1p8zftVPmRH2DaG/fMR12Ovyp1WoSZq9jMvn/kL1kmMqVEL7z+3k/7vGbnTEwpAVst6z3Bx4Zsb12VrRGdHoVZj6x+I1UeDSd2+lbgAP6TkZLnTEvTQ25Du1asXvXv3Zvny5Sxfvpw+ffrg6+tbELmZnOCFM3Aa4IRtLVsUagW2tWwpP6g8V0Ov4GxlmoMElAolH1X/hIn1J3P+yb+MPDyUyCTDTIUlmDZJkthyN5SRR4ag0WlY0nwlvSu/Z5JzOKuVagbXGMb8pktJ0iQx4sgQNtz+P3SSWA46t0TNzszXtw8HT55Hq9Nx6Mwl+oVsRlW6DLF+Y0Rjuhi4HnavUPWTVygU2Awags34iaQfP0Ls6M/QPX0qd1rCa+htSH/00UcEBwfj4OCAvb09wcHBDBw4sABSMz1ht+5lOWdz2C3T/IN8USfXrsxvtpSYtKcMP/wJF56ckzslwYgS0xOZeXYKSy8uxMPJk1VePxSKqeYalfLku1Y/0tS5OV9dXkbAic9Z++tqPFvWQ6lS4tmyHqGhIXKnadJEzX49ZalSOCxZKRrTxYAkSVSxdyiU/eStevTCbuZcNDdvEPPZYLSR4gSYqdLbkD579iwVKlTgww8/ZMCAAbi7u/Pvv/8WRG4mRZIkHN0cSbyemOn2xOuJuFU27T/I5xqU9ODLlt9iZ2bP58dHsjtip9wpCQYSGhqS0dhs2LwWPeZ2Zv+DfQyuMYzZnvMLbOCrITiYOzC90RzG1PFjz7ZdTJo+IaM7ldJXgX/QWNGYfg1Rs/UTjeniIWnVVwyxtmbyo8eZZsiYEhPDmIDJcqenl0WrNjgs/hIpNpaYTz9Gc010eTNFehvS06ZNw8bmf2dhra2tmTZtmjFzMkm/3lqPTVcbor6PIuFyApJGIuFyAtFrown43PT/IJ9ztXHjyxarng3w+ncaa6+vFqODC7mX++6b9zbjyi9XePNRd96v8qFRp180FoVCgU+FnqTv1FBuULlM3amcBjgRvHCG3CmaLFGzc0Y0pou2pP9bT/JPP+DbfwCTFi3LNENG4Ly8LTQlB7P6DXBY+S0KM3NiRwwl7cQxuVMSXqL3E1aSpEx9KpVKJRqNxqhJmZq/wv9g1dWV9OrZmwVByzLN2ZzXld/kZG9uz9ymi+lc/k3WXl9N8L9BpGnT5E5LyKOs+u6XG1SOn78u/FOe3b8TWWi7U8lF1OycE43poill+1aSVizF3LsDNmP98X3n3Yx+8gdPni90n9nqipVw+Oo7lGXLEec3hpSdf8qdkvACvQ1pNzc31q1bR3p6Ounp6axduxY3N9Ma6WpMRx/+w/zzwTR2asKEBl/Q2/fdTHM2F7Y/yOfMlGb415/E4OrD2BW5E7/jo/nx1+9FX9RCqDD33dfHrbJ7oe5OJYfiXrNzSzSmi5bUA3tJmDcLsybNsPsiCEURmbVG5Vwahy+/QV2vAQkzppD0f+vlTkn4j96GdFBQEGfOnKF169a0adOGc+fOMWNG8biseunpBYJOf0FVu2oENZ6NmdJM7pQMSqFQ8H7VD5niMZPDOw4RMN1P9EUthJzcnYpsYzPg88lEr43O1J0qYk0E7w8tfnMi51Rxrtl5JRrTRUPaqRPET/sCda3a2M+ci8KsaH1mK+3scFiwFPN27UlasZSE5YuRdGJmI7npnfTYycmJxYsXF0QuJuVuwh0CT46nlKUzwU0WYK220b9RIdW2rDdpO9Mz+qICz/4f8KzbQGE9614c7L+/B8suljz4/gEuH7lgU82GxOuJRK+NZt7Uwv93+/y9F7xwBpduXaJ8JVdqv1eHfWV345vQhwq2FeVN0AQV15qdX88b07FjPiPWbwwO85dg1sBD7rSEHEq/con4AD9Urm7Yz1uMwtpa/0aFkMLCArtps0gs6URKyC9sPnKYr69e5kZ4GNXc3BkTMFl8ZhcwvQ3pJ0+eEBISQkRERKZ+dsHBwUZNTE6Pkh8y4fhYVAo1c5supoRFSblTMroHd+5Tu1rtTLfZVLPh0q1LMmUk6HPx6XmC/51Oyy5eeNVrw4LFc7h06xJuld0LZd/97Ly8Atj9pEhGHB7CxBPjWNHyW0paOMmdokkpjjXbUERjunDS3L1D3PgxKBwdsF+0HKW9g9wpGZVCqcRm9OdsvXeXRb/+zAwXl4xVG6f4jwUoMvW/MNDbkP7ss89o3LgxLVq0KBYrZMWnxzHhxFgSNPEsbraCctbl5U6pQDzvi/r8jDQUne4BRVFEYjhfnJqAs2VpZnnOw6GlI317f5Bpudmiqqx1OWZ7zmfsseEEnBjPkuYrsFIXzbNPeVHcarahicZ04aKNekDcuBGgUuKw6EtUpZzlTqlAKBQKVh7YxwwXF5pZP7ti3szahunAnGBxJbkg6W1IJycn4+fnVxC5yC5Vm8qkk/5EJIUT7LmQag415E6pwAR8Phn/oLEwgIzuAffX3GfO1EVypya8JDYtloATnyNJEsFNFhaqOaINpYZjLSY3nMHkUxOYfmYyMxvPRaXUW86KheJUs41FNKYLB93Tp8SNG4mUlITD8q9NbslvY7sedo9G1apnuq2RlTXXr1+TKaPiSe9gw7Zt27J///6CyEVWWp2GGWcmc/HpeQIaTKVRKU+5UypQvr59mDd1ccbUfikhqZT2Lc35SmfQ6sTUWaYiTZvKlFMTiUqJYmbjubjaFK8Pjhe1KPMGo+uO59ijIyy9uFDMh/6f4lKzjU0MQDRtusQEYsePRvvgAfZzF6GuWl3/RkVMNTf3LFdtrGriqzYWNXob0uvWrWPo0KHUq1ePRo0a4eHhQaNGjQoitwIjSRKLL87n8MNDjKw9jrZlveVOSRa+vn0ypvY7d+wqQYNncfzRUb68tEQ0UkyATtIx99xMzj/9l4AGk6lbsr7cKcmuu/vbvFe5P9vCtvDLrcI/b7YhFIeaXVBebEz//PGHeNWvgUqppJWnmB5UTlJqKvEBfmhvXsd+RjBm9RvKnZIsxgRMZkpMTKZVGyc/eMBwr9Zyp1as6L0WeuZM0f8W/v31b/kjbCv9qg7k7Yq+cqdjMt5yf5vwxHBCbv9MeRs33qn0rtwpFWurr37D3vu7GVLjM9qWbS93Oibj4xpDiUp+wHdXv6aMpQvty3eSOyVZFYeaXZCUpUqxu0Nnlu7bw4wyZcSgLplJGg3x0yaRfuYUtpOnY97SS+6UZPP8vTcneAbXr1+jqps7n7drT8dTJ0jd/TcW7TvKnGHxkKOVDbds2cKKFSsAuH//PufOnTN6YgVl053f+OnGD7zp1p2Pqn0idzomZ0jNz/Aq04avLi/jn6iDcqdTbG27t5lfbv1Id7e3ebfyB3KnY1KUCiX+9SfRoKQH887P4t/o4t2QLOo1Ww5Lv1zCjDJlaGZtg5lC8WxQl6MjS4LF/NwFSdLpSJg3m7RDB7AZMx7LTl3kTkl2vr59MlZtPHTyPO+v+Ql1/QbEzw4i/YL4uy8IehvS06ZN4+zZs2zbtg0Aa2trgoKCjJ5YQdh3fw9fXlrMG2VaMbaOX6ZldYVnlAolgQ2nUt2hBrPOTuVa7FW5Uyp2jj08wpKLC2nq3IJRdcaJ92kWzFXmTG8cTDnr8kw+NZE78bflTkk2Rblmy+V62D0aWWWeGaaRlTXXwwr/6qGFhSRJJK5cRuqObVgP+gQrcSUgSwpzc+xnzUfpXJq4AD+0kRFyp1Tk6W1Inzt3jqlTp2JhYQGAg4MD6enpRk/M2E4/Pknwv0HULVGfLxpOFyP+X8NSZcnMxvOwN3Ng0kk/HiU/lDulYuNG3DWmn5lMZbvKTPEQ79PXsTOzJ9hzIWZKMyaeGEd0ymO5U5JFUa3Zcsp2UFd5V5kyKn6Sf/yBlF9/xtK3D1YDB8udjklTOjpiP28xaDTE+Y9DF1+0p0SVm96GtFqtRqvVZpwFe/LkCUql3s1M2vXYq0w5PZHy1q7M9JyLhcpC7pRMnpNlKWZ7LiBZm0TgST+SNIn6NxLy5WFyFAEnxmNrZstsz6K9uqahuFiXJbjJAuLS4wg4OZ5kTZL+jYqYoliz5ZbdoK5hFSoiacSsRsYSGhpCK896qJRKOgT68VeFitiMElflckLtXgG7WXPRht8jfkqAeJ8akd7q2r9/f4YPH050dDSLFy/mvffeY+jQoQbZ+YEDB+jcuTMdO3Zk1apVBomZldDQEDxb1kOpUuLRojaDlvTDVm3H3KZLsDOzN9p+i5rK9lWY4jGT2wm3mHlmqpgWz4gS0xMJPDmeZG0SwZ4LKWVZPBYZMITqDjWZ4jGDW3E3CDozudi9T4tCzTY1vr59CJy3mDkKJR7XrzFHocT/o0/o8vgRCQvnilmNjCA0NITZ/mOZKOk4U606k8qUYeHhf9i46Te5Uys0zBt5YusXSPrJ4yQsmifep0ai9zqxj48PderU4ejRo0iSxMqVK6lSpUq+d6zVapk+fTrff/89ZcqU4Z133sHb25uqVavmO/aLQkND8A8ai9MAJ2pXq03i9URur7lNv6oDcRaNk1xr6tyckbXHsvTiAlZeXsbIOuPkTqnI0eg0BJ2ZxN2EOwR7LqSyff7/3oqb5qVbMqbueBZdmMeSiwsYV3dCsTmLVdhrtql6eal6gMRVK0n+8QdUrm5Yf/ChzBkWLUuCZzDd0THTqn0zEKv25ZZlt+5ow8NI/ukHVG7uWL/XT+6UipwcdbisUqWKQQrxi86dO0eFChVwc3u2oES3bt3YvXu3wYty8MIZOA1wylj62raWLeUGlePbFV8z9IPhBt1XcdGjQi8iEsP47c6vlLdxo1fF3nKnVGRIksSSC/M5+fg4fvUC8XRuKndKhdZb7m/zIPkBP99cR1mrcrxftfg0dApzzS5MrAcPQxsZSdLXX6IqVw6Ldh3kTqnIEKv2GY71J8PQRoSR9NVyVOXLY9G6ndwpFSmyjVyKiorCxcUl4/cyZcrkaoomJyfbHD0u7NY9alernek2m2o2XLp1CWdnuxzvLyv53d7QcQwZS1+cL1oF8FgbxcrLS6lVtiqtXbOfAN7UnidTfr6/Pfctf4RvZUj9IXzo8Z5J5GRKsXIbZ0Kpz4nRPea7a19TpUwF3qr8luw5FVb5rdmQ87r9MlN7rV6MU2rRPO4NfETCzGmUrFYRa4/cLSVuasdmyFh5jZMeEUEVS0tOJydlnJGGZwM8a1SokK/8itLzlJs4pRYv4O6AASTMmIrTj5WxqldX9pzkimXomi1bQzqrvjq5ufQaHZ2ATqe/v49bZXcSrydmnJEGSLyeiFtl94zLc3nx4uW9/DBUHEPGymkcv1qTiYj9jM/3jWdZi6+oav/qEq2m9jyZ8vO9O2Iny/5dRvtynXi3/IA8xS4Oz1NujaruT0TsfSYfmsyhjUf55ZufCLt1D7fK7gR8Pjlfl4nzkpNSqchzg1JO+a3ZkPO6/SK53z85iWM1fS6pwwZx79PhOH6zBlW58rLnJHesvMbRxcUS+9kQhpRxYfKjR8zg2Zno08lJTImJIXDe4jznV5Sep7zEsZ4+l7Shg7g7dBiOq35AVaaM7DkVdCxj1GzZhnK7uLjw4MGDjN+joqIoXbq0wfcT8PlkotdGk3A5AUkjkXA5gei10QR8Ptng+ypurNRWzPKch52ZHYEn/XiU8kjulAqVFwfB1m9eg4nffE6Dkh741QssNv15C8LzOaY5rWDxvPkofRXUXlUbpa8C/6CxYqnnHCqoml0YKR0dcZi3GLRa4vzHoouPkzulQklKTSUuYDzayHDe/+Z7Ji1YkmmAZ+C8xaJ/dD4oSzphP28RpKYQN2EsuiQx+5YhZHtG2sPDI8sPc0mSUCgUnD59Ol87rlevHnfu3CEsLIwyZcqwfft2Fi5cmK+YWXn+Rxe8cAaXbl3CrbI786aKP0ZDKWXpzGzP+Yw68imTTvqxtPlKrNTW+jcs5rIaBHt/zX0a122KeXNzudMrcuzM7Hm07RHlBpXLNF6CAc9qQ1GoB0WlZhdWKvcK2M+eR+y4kcRPmoD9wmUozMzkTqvQkLRa4mdMRXPuX+yCZmHm0Qhfj0avDPAU8kddqQp204OJ8x9L/NRJ2AcvQKEW6xPkR7bP3pkzxl1mV61WM2XKFAYPHoxWq8XX15dq1aoZZV9ZjbYWDKeKfTUme0zni5P+zDw7jemNg1EpVHKnZdKyGgRbdlBZlixZQL8+A2TOrmiKvB2R7XiJoqAo1ezCyqxhI2wnTiZhxhQS5s3GNnCKuLqUA5Ikkbh8MWn792AzciwW3h3lTqlIM2/aHJuxfiQumEPi8sXYjvWTO6VCLcdfQ6Kjo0lNTc34vVy5cvneeZs2bWjTpk2+4wjya166JcNrj2H5pUV8fflLhtceLXdKJu11g2AF43jdeImiSNRseVh26oIuIpykNatQlXfFeuDHcqdk8pJ/+YmU0BCs+n6AVZ+8DbIWcseqRy904WEk/996VG7uWL3zrtwpFVp6G9K7d+9m7ty5PHz4kJIlSxIZGUmVKlXYvn17QeQnFCI9K75DRFIYoXd+JeJgGDvX/mWwQV1FjWslt2LVqDMFAZ9Pxj9oLAx49qXleXeaedMWyZ2aQYmaLT+rgR+jjQwnafU3KMuVx7JTF7lTMlkpf/9J0lfLMW/fCetPR8qdTrFiPWwE2vAwEpcvRlWuPOYtveROqVDSO9hw6dKl/Prrr1SsWJE9e/bwww8/0KhRo4LITSiEPq01ilIXS7N22fdiUFc2kjSJlOtRjog1EWIQbAHy9e3DvKmL0YVKXBpyiaSQZEr7luak+3HStKn6AxQSombLT6FQYOs/CTOPxiTMmUH6v8btdlNYpZ08TsLs6Zh5NMYucAoKsZR9gVKoVNhNmYGqanXipk1Cc0PM0Z0Xet+1arWaEiVKoNPp0Ol0NG/enMuXLxdEbkIhpFKouLnxBuUHlce2li0KtQLbWrY4DXAieOEMudOTXYo2hcCTfiQ3SObTz0dmNOp0oZIYBFsAfH37cPLweXRaHReOXSd4yALORJ8i6MxkNEVkKXFRs02DwswMu5lzUJUtR1ygH9p7d+VOyaRoblwjftIEVO4VsJs1D4W5GGQtB4WVFfZzF6K0tePnjz7Ay6M2KqWSVp71xMmvHNLbtcPe3p7ExESaNGnC+PHjKVmyJGoxwlN4jfDbYaL/bxbStKlMPjmBC0/OEdhwKt5dO/LFoGliEKyMOrl2JVmbxNKLCwn+N4jAhtMK/UBZUbNNh9LeAft5i4kZOojYCeNw/Go1SkdHudOSnfbBfeLGj0Fha4P9/CUo7YrHokamSlXKmd3tO7Jk9nRmuLjQqFr1Z/N2+48FECd49NB7RnrlypVYWloSEBBAq1atcHd356uvviqI3IRC6vmgrhcV9/6/6bp0pp2exKnoE/jVD8S7nBiVbip6VPBlSI3P2Ht/N4vPz0Mn6eROKV9EzTYtqvKu2M9ZgO5hFHGBfkipRacbUV7o4mKJGz8GKTUF+wVLUZXOelEQoWAt+/47Zri40MzaBjOFgmbWNkx3dGRJsLiSrI/ehrS1tTUqlQq1Wk3Pnj358MMPKVGiREHkJhRSWS2CE7Emgnc+KZ6jgjU6DTPOTOHoo8OMretPZ9c35U5JeEnfKv3oV3Ugf4RvZeXlZVmu4ldYiJpteszq1sdu0lQ05/8lYc4MJF3h/rKWVy8uuGI/ewHqSlXkTkn4z/WwezSyyrwGRCMra66H3ZMpo8JD7/W+nTt3smDBAqKjo5EkyWCT+wtF18uL4JSv5EqDDxqyx2UnbzxoRSuX4jN9llbSEvzvdA5F7Wd4rdF0d39b7pSEbHxU7ROSNElsvBOCtdqaQdWHyJ1SnoiabZosvDuijYwk6ZsVbH30iK+OHuZ62D2qubkzJqDoz2qU1YIrgumo5ubO6eQkmlnbZNx2OjmJqjlc7r4409uQnj9/Pl9//TVVqohvjkLOvbwITkJ6PBNPjCPozBcENJhM+3Kd5E7R6HSSjgXngtl7fxdDanyGb6XieUa+sFAoFAyvNZoUTTI/3fgBa5U1fav0kzutXBM123RZffAhm3b9xcJNvxWrvqhiwRXTNyZgMlP8xzKdZ2eiTycnMfnBA8ZUqYo27B4qt+LbNVMfvV07nJycREEW8s3WzI55TZdQr0R9Zp8NYkfYNrlTMipJklhyYT5/RfzBwGqDC2WDrDhSKBSMredPu7IdWHV1JVvuhsqdUq6Jmm26FAoFX50/V+z6oooFV0yfr28fAuctZo5Cicf1a8xRKAmY+AXd7B2IGT4EzbWrcqdosvSeka5bty5jxoyhQ4cOmL8wPU2nTkX/jKJgWNZqG4KbLGTKqYnMPz+bNF0qPSr4yp2WwUmSxIrLS9gWtoX3q3xI/6ofyZ2SkAsqhYqABlNI0aaw9OJCLFVWhapfu6jZpu1GeBiNqlXPdFsjK2uuXy+ac/iKBVcKj5evJANoevQibuwIYkcNw37OQswaii45L9N7RjoxMRErKyv++ecf9u7dm/FPEPLCUmXJzMZzaVHai6UXF7Lh1i9yp2RQkiSx6upKNt7ZgG/Fd/m4+lAUCoXcaQm5pFaqmeoxg0ZOnsw/N5v99/fInVKOiZpt2p73RX3R6eQkqrq6yZSRYYWGhtDKsx4qpRKvetX51X+cWHClEFO7V8Bh5XcoSzkT+/lo0v45KHdKJkfvGeng4OCCyEMoRsxVFkxrNItZZ6fx1ZXlpOpS6Vd1oNxpGcTa66v59dZ6fNx78lmtUaIRXYiZqyyY0XgO/ifGMuvsNCxUljQv3VLutPQSNdu0ZdcXdWz9BmgfRhXq6eBCQ0OY7T+W6Y6OGf2/J0dFYdO0Oe+IBVcKLVWZMjh8uYo4v9HETfLHNmAKlp27yp2WydD79fDBgwcMHz6cFi1a0LJlS0aOHMmDBw8KIjehCDNTmjG5YRAdynVmzbVVrL76TaGecgzg5xvrWHdjDV1cuzGqzueiEV0EWKmtCfZcQEW7ykw7HcjZaNOf+ULUbNOWVV/UCZ+N4k2tjpjBA0g/d1buFPNsSfAMpjs6Zur/PaNMGZYuWSB3akI+KR0dsV+6ErMGDUmYOZXk336VOyWTobchHRAQgLe3NwcPHuTAgQO0a9eOgICAgshNKOJUSjUTG0ymm5sP62+u5esrywttY3rD7f/ju2tf075cJz6vNxGlQlzCLCpszeyY12QxZa3LMemkP5djLsqd0muJmm36fH37cPDkebQ6HQdPnufdSVNx+GYNChsbYkd9SvLmwjfIFcRcxEWd0toG+3lLMG/VhsSlC0lcvarQfmYbkt5P+ydPnuDr64tarUatVtOrVy+ePHlSELkJxYBSoWRc3Qn0rPAOG27/H0svLih0K8ttuRvKV5eX0dqlLRPrf1Hol5gWXuVoUYJ5TZfiaO7IxBPjuBl3Q+6UsiVqduGkrlQZx1U/YNakGYkL55IwPxgpLU3utHJM++ghVWxts+z/XU1MnVZkKCwssJsejEW37iT/8B2JSxYUigWGXuy738qzHqGhIQaLrbchXaJECbZs2YJWq0Wr1bJlyxYcHR0NloAgKBQKRtQeS9/KH/D7vU3MPzcbraSVO61shYaG4NmyHkqVkjpNqxK0ejItSnsxqWEQKqXeYQdCIeVs6cyCZsuwUFkycPF7NGxeC6VKiWdLwxbl/BI1u/BS2tlhP2chVv0HkvL7JmJHf4bu8WO503otSZJI2b6VmA/7MsSxBJMfP+ZYUiLpksSxpESmxMQwJmCy3GkKBqRQq/+/vXsPrOlM+z7+3TsHiUQqCRHSMIOIs1KnlidKI+MZkrSIMpiotmirqUOFJOhLWkGrSrR1KFMzjJaR0hQdJYgZx+AZNVSIdMSZRJDzzs79/mHsaaY57ZWVA67PP5Kdva51JVZ+ufe917oXztNn4jh8FHlxm8iKno0ymWq6rVI9OHd/hirihE8rZqgi5oVP1i23yx1Iz5s3jx07dtCrVy969+7NX//6V+bNm6fLzoV4wGAw8JrvG4T6vMJfL29n3v/NobCosKbb+oXNmzcSPmcyxiEG2q5si9NLdbm5+SYdUzthZ7Sr6fZEFWtctwnPXw0gdWMq9iF2tF3ZFuMQA+Fz9AvlypLMfrgZbGxwGvcG9ebOo/B8MpmvhWL656mabqtE5ps3uBs+haz50di0aMmob74j6qPYYud/Ry5c/MjeaOZxZjAYcHozjLoT3iR/107uRk5D5eXVdFslKuncfT3Xbi93+qxJkyYsX75cl50JURaDwUCozyvUMdZh5dlPMRWZmPnUHOxtas/V3jGLonEPdce5jTMAzm2caTy2MR8sns9LISNruDtRHVZ9spwmY5sUOwYIvX9s1IYBg2T2o6FOX39svJtxN3Iad94aj/PUGTgMDKzptoD7s9D5O7aRHfsRymTC6e2pOAwOwWA0lrgWsXh01R0ZirHeE2Qtms+dyRNxWfgRxnouNd2WhVLq/rn7Vbh2e6kD6VWrVvHaa68RHR1d4uoDM2fO1KUBIf7b8BajsLepw7LTixn5UQgX4lK4lJqGd/OmREydVWODlcKiQtIuXKStT9tijzv5OHH6wuka6UlUv9p6DEhmP3psW/pQf9UX3Pt/M8maH01h8o84vTUZg23NnUJmvnmDrIUxmA79HdtOT1FvxixsHpE1sIU2DkEvYKhXj3vRs7kzcQJPLFqKsUGDGu1JKYXp8EFyVq+guZ0dx3Nz6FHXyfJ1Pc/dL/W38cEtZtu3b6/LjoSwxuBfhXB8ZxJ/+OJzvMZ60danLdnnsgmfMxmgWgfTOYXZbEuLZ3PqV9g3tif7XLZlNhIg+1w23s3lYprHhXfzprXyGJDMfjQZn6iPywcfk7PiE3K/XE9hynlcomMwurpVax9lzUILUafv8xjq1eNu5DTWDRnEyrt3OX/lMj7eTZkUUb0TYAXHk8hZtZzCUycxNm5M2JhXmfXnPxHNf9Zun52ZSeTCxbrsr9SBdL9+/QBwcHDgf/+3+MLbO3bs0GXnQpTluy+24zXWq8beQr+Zd5Ovf9pI/MWtZBdm0dHtKd54K4zPP14OofdnIbPPZZO+Np2F7+rzCylqv4ips+6/oKtlx4Bk9qPLYGuL05tvY+PjS9aC98l8NRSXeQux9W1TLfuXWWhREfZdu5MwMIiPF84j2tPTclOe2eHVMwFmOvkPclYvx3T8GMaGHjhNnY7DwCCG29lh17kL82OiOXcuGR/vprqeu1/u+0MrV678RSiX9JgQeivtLfR/pvyTJac+xK9xXzq6dtJ9pYyUu+fZlLqB3Vd2opTCr3FfQn49gjb120JPaPWELzGLojl94TTezZuy8F25mOZx8uD/urYeA5LZjy6HgAHYNmvG3chwMt8Yx67/8eOTb7/hXNrFKpn5U0qR/902spf+exY6bAoOQ4bJLLQoVeyfviDa09NyGkWPuk7MBWLmzmbwi0Or5NgpPHuG7M9XYDp0AIOr2/3jNOhFDHXqWJ5TlefulzoC2bdvH4mJiVy/fp333nvP8nhWVhY2NrJOrqh6pb2F/oT3E3x3aRtbL8ZR374+vRv1wc+zL0+5d8FW46BaKcXx9CQ2XvgzR28dxsHGgeBmgxnyq5doXLdJsefKxTSiNh4DktmPB1vfNtT/fC3rfz+CxauWV9nMn/nmDbI+iMF0UGahRcWVdmHf+eSzZAT9BrsuXbHv2h27rt2xaeJVqX0VppwnZ/UKCvbvw+DiQt0JE3EcHILB0bFSda1V6qijUaNGtG/fnoSEBNq1a2d53MnJqdJ3ydqxYwfLli0jJSWFTZs20aFDh0rVE4+mst5C/61/IEdvHmLftT3surKTb9O24mLnQq9Gfvh59qVLg64VWo6usKiQPVd3sfHCBlLuncPV3o1XWo0nsOmLuNjXniuPhShPVWY2SG7XJkZXN1ZcuVzizN+8WREMatAQo6sbRldXDG5uGOo6lXgB6s9t3ryRj2OiOZd2kZYNGvKakxMDXVxkFlpYxce7aYkX9rVs6IH9M70xHTtKwZ7dABgbN8bu6e73B9ZdumJ0da3QPgr/9RM5a1ZRsGcXhrp1qTt2HA7DhmN0ci5/4ypQ6kC6devWtG7dmsDAQGx1vkK4VatWxMbG8u677+paVzxaynsL3a9xX/wa9yXfnE/SrcPsu7qHxGt72HHpW5xt69Gr0f/g59mXpxt0I37LFmIWRZN24SLezZsyedI0bLvaEvfTRm7m3aCZ86+Y1iGS55sE1Krl9oSoqKrMbJDcrm3OX0orceYvJS2NezOnF3+yfR2Mrq4Y3dwwuLr9e5DthtHt/r9bThxjwaexRLu5Wma3Z129Qt1XJxASMrwavyvxsJsUMYvZ4ZOZyy8v7Ks3ZBhKKcwX/4Xp2FFMSUco2Lub/G+3AmDT0uf+oPrpbth16ozB0bH4C7wmXrzevgO/ufgvqFMHx1GhOA4fidHliRr9nktN27fffpslS5bw4osvlvj1+Ph4zTt9cHW5EOWpyFvodWzq0KuRH70a+VFgLuB4+lH2Xd3D36/v56+Xt5N7JJfLf7mM58ueltU/ps+ZgscQD54b2I/J7cPp3rAnRoPMuIiHV1VmNkhu1zalzvx5PUn9Nesoup1BUUYGRbczUBkZFGX++/ObNzAnn6XodgaY799BdknqBaIbNSo2ux3t4cH8lZ8SMu71Gvn+xMPpwURXaRf2GQwGbJv9Cttmv8JxcAiqsJDC5LOYko5gOnaE3M0byf1yPdjassO5HotP/h/RDRr85wVewi4MgcEMj1lU4RnsqlbqQDoqKgpAFvYXDxV7G3t6evSip0cvTEUmTqQfY9TsYXi+7Fls9Y8mY5uQv6mAj+Yuq+GOhdCHZPbjpayZP9v/mqkuiSoqQt27S1FGBhd6dqaLY91iX9fzhhXi8WLNNSQGW1vs2rbDrm07+P3LqLw8TD/8A1PSEZYvnEd0gwbFX+B5ejI/6Si/qyWDaACDUkqV9YScnBwcHBwwGo2kpqZy4cIF/Pz8sLMr+/zTMWPGcOvWrV88PmnSJPz9/QEYPXo04eHhcq6dqFJGGyNtV7bFYPufcwRVoeL0uNMUmYtqsDMh9Kc1s0Fy+2Hz5/XreW/mTM7+61/4NmvGzPfe43cjrb/Dattf/5pws7nY7PbhnGwW2thwOjVVz5aFqDAbo5ETPq2w+9n5/Sal6HwuGXNR7fnbXe6JdKNGjWL9+vXcvXuXMWPG0L59e7Zv386iRYvK3O6LL77Qq8cSpadnUVRU5muAX9DzCnu9aklPVV+nrBtoVKa32vC9VVUt6anq6hiNBtzdq+6iGK2ZDY92bte2OnrU6h8QRP+AoGJ1tNQLmz6z1Nltrf3Vpp+T3nX0rFXb6uhZq7J1Sjt9ycdb+9/uqsjsck8KVUrh6OjIzp07GTVqFJ988gkpKSlWNSFETYqYOov0telknclCFSqyzmSRvjadiKmzaro1IXQnmS2sNWTIMCIXLma+wUjnc8nMNxh1vWGFEFpMipjF7MxMDudkY1KKwznZzM7MZFJE7frbXaGB9IkTJ4iPj+e5554DwPzvCxS0+v777/Hz8+PEiROMHz+eV155pVL1hCjLkCHDWPjuYoo2//t0js2qVt1AQwg9VUVmg+T2o27IkGHsT/oBc1ER+5N+kHwUNe5heYFX7qkdkZGRrFixAn9/f3x8fEhLS6NHjx6V2mn//v3p379/pWoIYY3aeAMNIapCVWQ2SG4LIarfw/C3u9yBdPfu3enevTtZWVlkZ2fj7e3NzJkzq6O3MhmNZS8ur/d2VVlLeno46+hZS3qq3jp61rK2jp7fQ0lqa2ZDzed2baujZy3pqXrr6FmrttXRs9aj0FN5zy931Y6zZ88yffp07ty5g1IKNzc3FixYgI+Pj1WNCCGEqHqS2UIIUX3KHUgPHz6cSZMm0bNnTwAOHz7M4sWL+fLLL6ulQSGEEBUnmS2EENWn3IsNc3JyLIEM0KNHD3Jycqq0KSGEENpIZgshRPUp9xxpb29vPvnkE4KDgwH45ptvePLJJ6u8MSGEENaTzBZCiOpT7qkdd+7cITY2lmPHjgHQtWtXJk6cyBNPPFEtDQohhKg4yWwhhKg+pQ6k8/Pz2bBhAxcvXqRVq1YMGTKkQreYFUIIUf0ks4UQovqVOpCeNGkStra2dO3alcTERLy8vIiKiqru/oQQQlSAZLYQQlS/Us+RTklJIT4+HoChQ4cSEhJSbU0JIYSwjmS2EEJUv1JX7bC1tS3xYyGEELWPZLYQQlS/Uk/taNOmDY6OjgAopcjPz8fBwQGlFAaDgePHj1dro0IIIUonmS2EENWv3FU7HgURERHs3bsXd3d3vv32W811rl69Snh4OLdu3cJoNDJs2DBCQ0M11crPz2fkyJEUFBRgNpv5zW9+Q1hYmObezGYzQ4YMoVGjRqxYsUJznX79+uHk5ITRaMTGxoa4uDhNde7evcvMmTNJTk7GYDAwb948OnfubHWdCxcuMHnyZMvnaWlphIWFMWbMGKtrffHFF2zatAmDwUCrVq2IiYmhTp06VtdZu3YtmzZtQilFSEiIVb2UdCxmZmYyefJkLl++jJeXFx9//HG5KyyUVGfHjh0sW7aMlJQUNm3aRIcOHTT3tGDBAvbs2YOdnR1NmzYlJiYGFxcXq+t8/PHH7N69G6PRiLu7OzExMTRq1EhTTw+sXr2ahQsXcvDgQdzc3KyuExsby8aNGy3bTpkyhT59+mju6U9/+hPr1q3D1taWPn36EB4eXm4tUTa9Mhv0y23J7IqpjZkN2nNbr8wurZaW3NYrs0urpSW39crs0mppye1qy2z1GDhy5Ig6deqUGjhwYKXqXL9+XZ06dUoppdS9e/dUQECAOnfunKZaRUVFKisrSymlVEFBgRo6dKg6ceKE5t7WrFmjpkyZosaNG6e5hlJK9e3bV6Wnp1eqhlJKhYeHq40bNyqllMrPz1d37typdM3CwkL17LPPqkuXLlm97bVr11Tfvn1Vbm6uUkqpsLAwtXnzZqvrnD17Vg0cOFDl5OQok8mkQkNDVWpqaoW3L+lYXLBggVqxYoVSSqkVK1aohQsXaqpz/vx5lZKSokaNGqVOnjxZqZ7279+vTCaTUkqphQsXau7p3r17lo/Xrl2rZs2apbknpZS6cuWKGjt2rHruuecqdJyWVGfp0qXq888/r1Af5dU6ePCgCg0NVfn5+UoppW7dumV1XfFLemW2UvrltmS29WpDZitVudzWK7NLq6Ult/XK7NJqacltvTK7tFpacru6MrvcOxs+Crp166bLGqoeHh60a9cOAGdnZ5o3b87169c11TIYDDg5OQFQWFhIYWEhBoNBU61r166xd+9ehg4dqml7vWVlZXH06FFLP/b29hV6ZVyegwcP4u3tjZeXl6btzWYzeXl5FBYWkpeXh4eHh9U1UlJS6NSpE46Ojtja2tKtWze+//77Cm9f0rG4e/duXnjhBQBeeOEFdu3apalOixYtaN68eYV7KatW7969LefZPvXUU1y7dk1THWdnZ8vHubm5FT7GS/udjYmJYdq0aZWuo0VJtTZs2MC4ceOwt7cHwN3dXZd9Pe70/H/TK7cls61XGzIbKpfbemV2abW05LZemV1aLS25rVdml1XLWtWV2Y/FQLoqXLp0iTNnztCpUyfNNcxmM8HBwTz77LM8++yzmmvNmzePadOmYTTq89/5yiuvMHjwYL766itN26elpeHm5kZERAQvvPACUVFRutyieNu2bQwaNEjTto0aNWLs2LH07duX3r174+zsTO/eva2u06pVK5KSkrh9+za5ubkkJiZWOLBKk56ebvkD4eHhQUZGRqXq6W3z5s34+flp3n7x4sX06dOH+Ph43n77bc11du/ejYeHB61bt9Zc44H169cTGBhIREQEd+7c0Vznp59+IikpiZCQEEaNGsXJkycr3ZuoOpXNbcls69SGzAb9c/tRz2zQJ7f1zGzQJ7erIrNlIK1BdnY2YWFhREZGFnvlZi0bGxu2bt3Kvn37OHnyJMnJyVbX2LNnD25ubrRv315zHz+3YcMGvv76a1atWsX69es5evSo1TUKCws5ffo0I0aMYMuWLTg6OrJy5cpK9VVQUEBCQgIDBgzQtP2dO3fYvXs3u3fvZv/+/eTm5rJ161ar67Ro0YJXX32VsWPH8uqrr+Lr64uNjY2mnh4Gn332GTY2NgQFBWmuMXnyZPbt20dgYCDr1q3TVCM3N5fly5dXaiD+wIgRI/j+++/ZunUrHh4ezJ8/X3Mts9nM3bt32bhxI+Hh4UyaNAn16F928lDSI7clsyuutmQ2PF65rUdmQ+VzW8/MBv1yuyoyWwbSVjKZTISFhREYGEhAQIAuNV1cXOjRowf79++3etvjx4+TkJBAv379mDJlCocOHeKdd97R3MuDCwrc3d3p37+/pldrnp6eeHp6WmZrBgwYwOnTpzX3BJCYmEi7du1o0KCBpu0PHDjAk08+iZubG3Z2dgQEBHDixAlNtUJCQvj6669Zv3499evXp1mzZprqPODu7s6NGzcAuHHjRoUuxqgOX3/9NXv37uXDDz/U/Bb2zw0aNIidO3dq2vbixYtcunSJ4OBg+vXrx7Vr1xg8eDA3b960ulaDBg2wsbHBaDQSEhLCDz/8oKknuP/70r9/fwwGAx07dsRoNHL79m3N9UTV0Du3JbPLV5syG/TN7ccls0F7buuZ2aBfbldFZstA2gpKKaKiomjevDkvv/xypWplZGRw9+5dAPLy8jhw4ICm81unTp1KYmIiCQkJfPTRR/Ts2ZMPP/xQU085OTlkZWVZPv773/+Oj4+P1XUaNmyIp6cnFy5cAO6fJ9eiRQtNPT2wbds2Bg4cqHn7Jk2a8I9//IPc3FyUUpXqKT09HYArV66wc+dOzW9dPtCvXz+2bNkCwJYtW3j++ecrVU8PiYmJrFq1is8++8yypJoWP/30k+XjhIQETcc4gK+vLwcPHiQhIYGEhAQ8PT2Ji4ujYcOGVtd68AcQYNeuXZqO8Qf8/f05dOgQAKmpqZhMJlxdXTXXE/rTK7cls61TmzIb9M3tRzmzQZ/c1jOzQb/crorMfiyWv5syZQpHjhzh9u3buLu789Zbb2m661dSUhIjR46kVatWlnPbKrp01n/78ccfmTFjBmazGaUUAwYMYOLEiVbX+bnDhw+zZs0azUsppaWl8eabbwL33/4YNGgQr7/+uqZaZ86cISoqCpPJhLe3NzExMZovHsjNzeW5555j165d1KtXT1MNgKVLl7J9+3ZsbW1p06YN77//vuWCA2v87ne/IzMzE1tbWyIiInjmmWcqvG1Jx6K/vz+TJk3i6tWrNG7cmCVLllC/fn2r69SvX5/o6GgyMjJwcXGhTZs2rF69WlNPK1eupKCgwNJHp06dmDt3rtV1EhMTSU1NxWAw4OXlxZw5cyq0/F15v7P9+vXjL3/5S7kzQSXVOXLkCD/++CMAXl5ezJ07t0IXMZVUKzg4mMjISH788Ufs7OwIDw+36ngQJdMrs0G/3JbMrrjaltmgPbf1yuzSamnJbb0yu7RaWnJbr8wurZaW3K6uzH4sBtJCCCGEEELoTU7tEEIIIYQQQgMZSAshhBBCCKGBDKSFEEIIIYTQQAbSQgghhBBCaCADaSGEEEIIITSQgbTQTZs2bQgODmbQoEGEhYWRm5tr1fbXr18nLCwMuL8U0759+yxf2717d6XvtPXA8OHDdakjhBAPO8ltISpHlr8TuuncubPlzlNTp06lffv2mm+AEBcXx6lTp5g9e7aeLQohhPgZyW0hKse2phsQj6auXbty9uxZMjMziYyMJC0tDUdHR+bOnUvr1q05cuQI77//PgAGg4F169aRmZnJhAkTiIuLY+nSpeTl5XHs2DHGjx9PXl6eJaAvX75MZGQkGRkZuLm5ERMTQ5MmTZgxYwbOzs6cOnWKmzdvMm3aNAYMGPCL3h784Th8+DDLli3D1dWV5ORk2rVrV+KtVf/4xz/y5ZdfYmNjQ8uWLVm8eDE5OTlER0eTnJyM2Wxm4sSJ+Pv7k5eXR0REBOfPn6dFixZcvnyZ2bNn06FDh2J/sL777jv27t3L/PnzycjI4N133+XKlSsAREZG8vTTTxMbG8uVK1e4dOkSV65cITQ0lN///vfA/btprV69GoPBgK+vLx988EGpdYQQoiIktyW3hQZKCJ089dRTSimlTCaTmjBhglq/fr2aO3euio2NVUopdeDAARUUFKSUUmr8+PEqKSlJKaVUVlaWMplMKi0tTQ0cOFAppdTmzZvVnDlzLLV//vn48eNVXFycUkqpTZs2qddff10ppdT06dPVW2+9pcxmszp37pzy9/cvs89Dhw6pLl26qKtXryqz2ayGDRumjh49+ovn9+rVS+Xn5yullLpz545SSqlFixapLVu2WB4LCAhQ2dnZas2aNWrGjBlKKaXOnDmj2rRpo06ePFlsv0optWPHDjV9+nSllFJTpkyx7Pfy5ctqwIABSimlli5dql566SWVn5+v0tPTVffu3VVBQYFKTk5WAQEBKj09XSml1O3bt8usI4QQpZHcltwWlSMz0kI3eXl5BAcHA/dnNoYOHcqwYcOIjY0F4JlnniEzM5N79+7RpUsX5s+fT2BgIAEBATg5OVV4PydOnLDUDA4O5oMPPrB8zd/fH6PRSMuWLbl161a5tTp27IinpycArVu35vLly3Tt2rXYc3x9fXnnnXd4/vnn8ff3B+Bvf/sbCQkJrFmzBoD8/HyuXr3K0aNHGT16tKWer69vuT0cOHCA8+fPWz7PysoiKysLgD59+mBvb4+bmxtubm6kp6dz6NAhBgwYYLnV6oNbwpZWx9nZudwehBCPJ8ltyW1ROTKQFrpxcHBg69atxR5TJZyCbzAYGDduHH369GHfvn0MGzaMP/zhD9SpU0fTfn/+lp69vb1V2/78+TY2NpjN5l88Z+XKlRw9epSEhAQ+/fRTtm3bBsDSpUtp3rx5mf2UJj8/3/JxUVERX331FQ4ODuX2V1hYWOLPtLw6QghREsntX/ZTGsltURJZtUNUqW7duvHNN98AcPjwYVxdXXF2dubixYv4+voybtw42rdvT2pqarHtnJycyM7OLrFm586dLaEYHx9fpeeTFRUVcfXqVXr27Mm0adO4d+8eOTk59O7dm3Xr1lnC8fTp08D97zc+Ph6A5ORkzp49a6nVoEEDUlJSKCoqYteuXZbHH9R64MyZM2X29Mwzz/Ddd99x+/ZtADIzMzXVEUKIkkhuS26LipOBtKhSEydO5NSpUwQGBrJo0SLmz58PwNq1axk0aBBBQUE4ODjg5+dXbLsePXpw/vx5goOD2b59e7GvzZw5k7i4OAIDA9m6dStRUVFV1r/ZbGbatGkEBgby4osvMmbMGFxcXHjjjTcoLCwkKCiIQYMGsWTJEgBGjBhBTk4OgYGBfP7553Ts2NFSa+rUqYwfP57Q0FAaNmxoeTwqKsryM/rtb3/Lhg0byuzJx8eHCRMmMHr0aIKCgiw/U2vrCCFESSS3JbdFxcnyd0JUodGjRxMeHk6HDh1quhUhhBAVILktrCEz0kIIIYQQQmggM9JCCCGEEEJoIDPSQgghhBBCaCADaSGEEEIIITSQgbQQQgghhBAayEBaCCGEEEIIDWQgLYQQQgghhAYykBZCCCGEEEKD/w/c7iqvVQ5h1wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x288 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.set_theme()\n",
    "fig, ax = plt.subplots(2, 2, figsize=(12,4))\n",
    "ax = [a for a_list in ax for a in a_list]\n",
    "for i in range(len(ax)):\n",
    "    ax[i].plot(np.arange(1,17), pe[i,:16], color=f'C{i}', marker=\"o\", markersize=6, markeredgecolor=\"black\")\n",
    "    ax[i].set_title(f\"Encoding in hidden dimension {i+1}\")\n",
    "    ax[i].set_xlabel(\"Position in sequence\", fontsize=10)\n",
    "    ax[i].set_ylabel(\"Positional encoding\", fontsize=10)\n",
    "    ax[i].set_xticks(np.arange(1,17))\n",
    "    ax[i].tick_params(axis='both', which='major', labelsize=10)\n",
    "    ax[i].tick_params(axis='both', which='minor', labelsize=8)\n",
    "    ax[i].set_ylim(-1.2, 1.2)\n",
    "fig.subplots_adjust(hspace=0.8)\n",
    "sns.reset_orig()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7RQKqW3WrIxj"
   },
   "source": [
    "### Transformer Network  (***2 POINTS***)\n",
    "Everything we've talked about up to this point is summarized in the class Transformer below. In this case, initial input processing positional encoding and the encoder-decoder blocks are all necessary for you to formulate your answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "sH4V6kWErIxk"
   },
   "outputs": [],
   "source": [
    "class Transformer(nn.Module):\n",
    "    def __init__(self, enc_inp_size, dec_inp_size, dec_out_size, N=6,\n",
    "                   d_model=512, dim_feedforward=2048, num_heads=8, dropout=0.1,\n",
    "                   mean=[0,0],std=[0,0]):\n",
    "        super(Transformer, self).__init__()\n",
    "        \n",
    "        self.d_model = d_model\n",
    "        self.num_heads = num_heads\n",
    "        self.dim_feedforward = dim_feedforward\n",
    "        self.dropout = dropout\n",
    "        self.N = N\n",
    "        self.mean = mean\n",
    "        self.std = std\n",
    "        self.enc_inp_size = enc_inp_size\n",
    "        self.dec_inp_size = dec_inp_size\n",
    "        self.dec_out_size = dec_out_size\n",
    "        \n",
    "        self.encoder = nn.ModuleList([deepcopy(\n",
    "            EncoderBlock(d_model, num_heads, dim_feedforward, dropout)) for _ in range(N)])\n",
    "        self.decoder = nn.ModuleList([deepcopy(\n",
    "            DecoderBlock(d_model, num_heads, dim_feedforward, dropout)) for _ in range(N)])\n",
    "        self.pos_enc = PositionalEncoding(d_model, dropout)\n",
    "        self.pos_dec = PositionalEncoding(d_model, dropout)\n",
    "        self.src_embed = nn.Linear(enc_inp_size, d_model)\n",
    "        self.tgt_embed = nn.Linear(dec_inp_size, d_model)\n",
    "        self.out = nn.Linear(d_model, dec_out_size)\n",
    "        \n",
    "        self.init_weights()\n",
    "\n",
    "\n",
    "    def forward(self, src, trg, src_mask, trg_mask):\n",
    "        \n",
    "        # First part of the forward pass: embedding and positional encoding \n",
    "        # for both the source and target\n",
    "        '''\n",
    "        Add here your code\n",
    "        '''\n",
    "        src_embedding = self.src_embed(src)\n",
    "        trg_embedding = self.tgt_embed(trg)\n",
    "        src_pos_encoding = self.pos_enc(src_embedding)\n",
    "        trg_pos_encoding = self.pos_dec(trg_embedding)\n",
    "\n",
    "        # Second part of the forward pass: the encoder and decoder layers. \n",
    "        # Look at the arguments of the forward pass of the encoder and decoder \n",
    "        # and recall that the encoder output is used as the memory in the decoder.\n",
    "        '''\n",
    "        Add here your code\n",
    "        '''\n",
    "        encoder_in = src_pos_encoding\n",
    "        decoder_in = trg_pos_encoding\n",
    "        \n",
    "        encoder_input = encoder_in \n",
    "        for layer in self.encoder:\n",
    "          encoder_out = layer(encoder_input, src_mask) \n",
    "          encoder_input = encoder_out\n",
    "          \n",
    "        decoder_input = decoder_in\n",
    "        for layer in self.decoder:\n",
    "          decoder_out = layer(decoder_input,encoder_out,src_mask,trg_mask)\n",
    "          decoder_input = decoder_out\n",
    "      \n",
    "        output = self.out(decoder_out)\n",
    "\n",
    "        return output\n",
    "    \n",
    "    \n",
    "    # Initialize parameters with Glorot / fan_avg. \n",
    "    def init_weights(self):\n",
    "        for p in self.encoder.parameters():\n",
    "            if p.dim() > 1: nn.init.xavier_uniform_(p)\n",
    "        for p in self.decoder.parameters():\n",
    "            if p.dim() > 1: nn.init.xavier_uniform_(p)\n",
    "        for p in self.pos_enc.parameters():\n",
    "            if p.dim() > 1: nn.init.xavier_uniform_(p)\n",
    "        for p in self.pos_dec.parameters():\n",
    "            if p.dim() > 1: nn.init.xavier_uniform_(p)\n",
    "        for p in self.src_embed.parameters():\n",
    "            if p.dim() > 1: nn.init.xavier_uniform_(p)\n",
    "        for p in self.tgt_embed.parameters():\n",
    "            if p.dim() > 1: nn.init.xavier_uniform_(p)\n",
    "        for p in self.out.parameters():\n",
    "            if p.dim() > 1: nn.init.xavier_uniform_(p)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hy7U3kArrIxk"
   },
   "source": [
    "### Optimizer  (***1 POINT***)\n",
    "\n",
    "Here we select same **optimizer** proposed into original Transformer paper. \n",
    "\n",
    "It uses some initial warmup epochs, where the learning rate is increased. Then it slowly decreases according to a number of epoch and the chosen embedding size. The resulting formula is:\n",
    "\n",
    "LR = $\\frac{F}{\\sqrt{D}} min( \\frac{1}{\\sqrt{epoch}},\\ epoch \\cdot W^{-\\frac{3}{2}}) $\n",
    "\n",
    "where F is a scaling factor, D is the model embedding size, W is the number of warmup epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "6kbnP33vKEkc"
   },
   "outputs": [],
   "source": [
    "class SqrtScheduler(torch.optim.lr_scheduler._LRScheduler):\n",
    "    \n",
    "    def __init__(self, optimizer, factor, model_size, warmup, max_iters):\n",
    "        self.warmup = warmup\n",
    "        self.max_num_iters = max_iters\n",
    "        self.factor = factor\n",
    "        self.model_size = model_size\n",
    "        super().__init__(optimizer)\n",
    "        \n",
    "    def get_lr(self):\n",
    "        lr_factor = self.get_lr_factor(epoch=self.last_epoch+1)\n",
    "        return [base_lr + lr_factor for base_lr in self.base_lrs]\n",
    "    \n",
    "    def get_lr_factor(self, epoch):\n",
    "        '''\n",
    "        Add here your code\n",
    "        '''\n",
    "        a=self.factor/(np.sqrt(self.model_size))\n",
    "        b_1=1/np.sqrt(epoch)\n",
    "        b_2=epoch*pow(self.warmup,-3/2)\n",
    "        b=min(b_1,b_2)\n",
    "        lr_factor=a*b\n",
    "        return lr_factor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 246
    },
    "id": "IjGdp7OSKEkc",
    "outputId": "b0c45805-262a-4fe4-8199-95d606c0e83b"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgYAAADhCAYAAACz4X2zAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABHtUlEQVR4nO3dd3hUVfrA8e+UVJIQAilIkQ4CIk2lJqBITQjVRRBQNOLqDxSlSpEqiqEIq4iKuiIqCEoWF0JRUVc6ooRdioB0CCEB0pOZuef3R8g1AwmTQMpMfD/Pk2fmlrn3vPfOZN4599xzDEophRBCCCEEYCzrAgghhBDCeUhiIIQQQgidJAZCCCGE0EliIIQQQgidJAZCCCGE0EliIIQQQgidJAbCpdlsNj766CP69etHZGQkPXv25M033yQ7O/u2txkZGUlycvIdl+3SpUs0btyYq1ev6vPmz59Pw4YNOXHihD7v/fff58UXX7zj/ZWGXbt2ER4eXib7njx5Mtu3by+Wbe3atYtmzZoRGRmp/3Xp0oVnn32WK1euOHz9lClTOHjwYJH3+/777xMZGUnv3r0JDw/njTfecPhenThxIsuXLy/yvvKKjY1l6NChDtd76KGHiIuLu6N9CdcniYFwadOnT2f//v3885//JCYmhjVr1vDHH38wefLk295mTEwMfn5+d1y2oKAgGjRowN69e/V527Zto3Pnznz33Xf6vJ07d9KpU6c73l95N2fOHNq1a1ds26tZsyYxMTH636ZNmzAajXz44YcOX7t9+3aK2gXMxo0b2bp1K6tWreJf//oXa9eu5cSJE/zjH/+43RCEKBHmsi6AELfr7NmzrF+/nv/85z/4+PgA4O3tzYwZM/jll18ASElJYcaMGRw+fBiDwUDHjh156aWXMJvNLF68mC1btuDm5kalSpWYO3cuQUFBNGzYkB07drBt2za2bNmC0Wjk1KlTeHp68sYbb1C3bl1SUlKYM2cOR48exWKx0LZtW8aPH4/ZbP+RCg0NZdeuXXTp0oWzZ8+SnZ3N8OHDWbJkCU8//TTZ2dns37+fefPm8euvv+q1HQkJCbRr147XXnuNs2fPMmTIEOrWrcu5c+d4/fXXGTt2LO3bt+fgwYPYbDZGjx7NqlWrOHHiBE2bNmXBggUYjcabjldERAT79++/afqrr74iNjYWTdM4f/48wcHBvP766wQHBxf6fNzqmKxZs4ZVq1ZhsVi4du0aUVFRDB48mK+++oo1a9aQkZGBj48Pffv2LfCYDx06lCFDhtC0aVOeeOIJwsLC+O2330hOTmbcuHE88sgjZGRk8Oqrr/Lbb7/h6+tLvXr1AHj99dcdlj81NZWkpCRatmwJUOD5WLhwIZcuXWLs2LHMmzePOnXqFOq9kJCQgM1mIzMzE09PTzw8PJg6dSpJSUkApKWlMXv2bH755RdMJhNdunRhzJgxAOzfv59BgwZx+fJl6tevz/z58/H29ub48ePMmTOHq1evYrPZGDp0KAMGDADgrbfeYv369fj7+3P33Xfr5Zg4cSL169fnqaeeync613fffcfSpUuxWCx4enoyYcIEWrRowZIlS/j111+5dOkSDRs2JDo6utDvEeEilBAuKjY2VvXv3/+W64wfP17NmjVLaZqmsrKy1IgRI9SyZcvU+fPnVcuWLVVWVpZSSqnly5erLVu2KKWUatCggUpMTFRr165VrVq1UhcuXFBKKTVz5kw1fvx4pZRSEydOVJ988olSSimr1arGjh2r3nvvvZv2v2fPHtW7d2+llFKffPKJeu2111R2dra6//77VWJiotq9e7caOHCgUkqpMWPGqJ07dyqllEpNTVUPPvigiouLU2fOnFENGjRQe/bsUUopfXrr1q1KKaWmTZumOnfurFJSUlRmZqZq37692rdv301lOXPmjGrevHm+02vXrlXNmzdXJ06cUEop9eabb6pRo0bdtI2dO3eqXr165XusCzomqamp6tFHH1VJSUlKKaX2799vt9/7779fpaSk6NMFHfPHH39cbdy4UY//u+++U0rlvA86deqklFIqOjpavfTSS8pms6mUlBQVERGhJkyYkG8c9957r+rdu7fq2bOnatOmjerTp49atmyZys7OvuX5UEqpzp07qwMHDtwy7hslJyerJ598UjVp0kQ9+uijau7cuWr37t368tdee02NGTNGWa1WlZWVpYYMGaJ27typJkyYoAYMGKDS09OV1WpVffv2VV9//bWyWCyqZ8+e6uDBg/r2e/Toofbv36+2bNmievbsqVJSUpTFYlHPPPOMevzxx5VSSk2YMEF98MEH+n7zTufG9ccff6jw8HD9nB09elS1b99epaWlqcWLF6tu3bopi8WS7/tAuD6pMRAuy2g0omnaLdf58ccf+fzzzzEYDLi7uzNo0CD++c9/8vTTT9OoUSP69u1LaGgooaGhtG3b9qbXN2nShJCQEAAaN27Mli1bgJxLAnFxcaxZswaAzMzMfPffvHlzLly4wNWrV/n++++JiorCzc2NNm3asHPnTo4fP05YWBiQ86v2xx9/5N133+XEiRNkZWWRnp6Ov78/ZrOZ5s2b69t1c3PjoYceAnKqxFu0aKHXmgQFBXHt2rUiHMkc7du3p3bt2gA8+uijREZGFun1BR2TChUq8O677/LDDz9w8uRJDh8+THp6uv66hg0b6mWHgo95Xm5ubvpxy9uO44cffmDSpEkYjUa9BuLIkSP5ljf3UgLA2rVrWbhwIT169MDNzQ0o+HwUNu4b+fr68uGHH3LmzBl27tzJ7t27eeaZZxg8eDDjxo1j+/btTJo0CZPJhMlk4tNPPwXg66+/pkuXLnh5eQFQv359kpKSOHnyJKdPn+aVV17R95GZmcn//vc/jh8/ziOPPKIf1/79+7NixYp8y5Wfn3/+mUuXLvHEE0/o8wwGA6dPnwZy3tc31oiI8kPOrHBZzZo148SJE6Smptp9scTHxzN16lQWL16MpmkYDAZ9maZpWK1WjEYjn376KXFxcezYsYPXXnuNjh07Mn78eLt9eHp66s8NBoN+XVnTNN566y3q1q0LQHJyst1+cpnNZtq0acOPP/7IoUOHaN26NQBhYWHs27ePw4cP6//YH3/8cRo2bEjHjh3p0aMHv/32m74/d3d3u3/Ebm5udvvL/TLL6/PPP+eLL74AoGnTpjz33HN218UtFovd+iaTye445Z0ujIKOycWLF/nb3/7Go48+SqtWrejevTvff/+9/jpvb2+77RR0zPNyc3PTL5XkPQ5ms9lu/RsvpxSkf//+/Pbbb7zwwgusXr0as9l8y/NRmLhv9P7779OqVStatmxJjRo1GDhwIHv37iUqKopx48ZhNpvtXnfhwgX9WOQ997nHxGaz4evrqyc3AJcvX8bX15d58+bZlTXvubzxmN74PsiNqW3btixatMiuPEFBQWzZsuWmcybKF2l8KFxWcHAwERERvPLKK6SmpgI514mnT5+Ov78/np6edOjQgU8//RSlFNnZ2axevZp27dpx+PBhwsPDqVu3LiNHjuSJJ54oUmvsDh068PHHH+vb/fvf/67/wrtRaGgoH3zwAQ888ID+BR4WFsaOHTu4cOECjRs3Jjk5mbi4OMaOHUvXrl25ePEip0+fdlgjciuPPfaY3rBuzpw5+Pn5YbFYOHbsGAD//ve/7dbfuXMn8fHxAHzxxRd07ty5SPsr6JgcPHiQgIAAnnvuOTp06KAnBTab7bZjK0hYWBhr165F0zQyMjL45ptv8v2Szs/YsWO5cOECK1eudHg+TCYTVqv1lnHfKDMzk/nz59vdpXL06FEaN24MQNu2bfn666/RNI3s7GxGjx7Nnj17Cixv7dq18fT01BODCxcuEB4ezsGDBwkNDSU2Npbk5GQ0TbNLHipVqqTfUREfH8/u3btv2nbbtm35+eefOX78OJBTE9O7d+8Ca0NE+SI1BsKlvfrqq7zzzjsMGjQIk8lEdnY2Xbp0YdSoUUDObWWzZ88mIiICi8VCx44defbZZ3F3d6dHjx70798fb29vPD09mTJlSqH3O3nyZObMmaNvt127djz99NP5rhsaGsrkyZMZMWKEPq9KlSp4e3vTvHlzDAYDfn5+PPPMM/Tt2xdvb2+Cg4Np2bIlp06dokaNGnd2kK7z9fVl3LhxREVFERAQQPfu3e2WBwcHM27cOBISEqhXrx4zZ87MdzvHjx+nRYsWdvN+/PHHAo+J1WplzZo1dO/eHYPBwAMPPEBAQACnTp0qlrjyGjlyJDNnziQiIgJfX18qV65sVwNxK35+fowdO5a5c+fSq1evAs9H27ZteeSRRxg3bhzTp08v9Hvhueeew2AwMGjQIAwGA5qm0bRpU/1X+f/93/8xZ84cIiMjsdls9OzZk65du9rdwZKXu7s777zzDnPmzOGDDz7AarXywgsv0KpVKwCOHDlC//798fPzo1GjRvptmEOHDmXs2LF069aN6tWr06ZNm5u2nXv+X3rpJZRSmM1mli5dSoUKFQp1LIVrM6j86saEEH8pX331FZs2bWLZsmVlXZQ78u9//xsfHx/CwsLQNI1Ro0bRvn17Bg8eXNZFE8JlyKUEIUS5Ub9+fZYuXUpkZCTh4eEEBQUxcODAsi6WEC5FagyEEEIIoZMaAyGEEELoJDEQQgghhE4SAyGEEELoJDEQQgghhE76MQCuXElD04qvDWblyj4kJqYW2/bKUnmJpbzEARKLsyovsZSXOEBiKYjRaKBSpYL7pJDEANA0VayJQe42y4vyEkt5iQMkFmdVXmIpL3GAxHI75FKCEEIIIXSSGAghhBBCJ4mBEEIIIXSSGJSQjCxrvkO0CiGEEM5MEoMScDExjecX/sh3v5wr66IIIYQQRSKJQQmIT0oHYN+RS2VcEiGEEKJoJDEoASajAQBbObpNRgghxF+DJAYlwGzKOazl6f5ZIYQQfw2SGJQA4/UaA6skBkIIIVyMJAYlwJCTF2CzSWIghBDCtUhiUAJyLyHYNK2MSyKEEEIUjSQGJSA3H5A2BkIIIVyNJAYlILemQO5KEEII4WokMSgBmsq9lCCJgRBCCNciiUEJ0NsY2KSNgRBCCNciiUEJyG1jIDUGQgghXI0kBiUgt42BJoMoCSGEcDGSGJSAPy8lSGIghBDCtZRoYrB+/Xp69uxJ165dWbly5U3LDx06RL9+/ejWrRuTJ0/GarXaLV+0aBFLlizRp48fP86QIUOIjIzkb3/7G4cOHQLg3LlztGjRgsjISCIjI3nqqadKMiyHpPGhEEIIV1ViiUF8fDwLFy7ks88+Y926daxatYpjx47ZrTNu3DimTZvGpk2bUEqxevVqAFJSUnjllVf46KOP7NafMmUKUVFRxMTE8OKLLzJhwgQADh48SEREBDExMcTExLB8+fKSCqtQpI2BEEIIV1ViicH27dtp06YN/v7+eHt7061bN2JjY/Xl586dIzMzk+bNmwPQr18/ffm3335LrVq1ePLJJ+22OXDgQDp27AhAw4YNuXDhAgBxcXEcPXqUyMhIhg0bxpEjR0oqrEKRHg+FEEK4KnNJbfjSpUsEBgbq00FBQRw4cKDA5YGBgcTHxwPQp08fALvLCJCTPORavHgxXbp0AcDDw4PevXszaNAgfvrpJ55//nk2bNiAu7t7ocpaubJP0YJz4L+nr+rPAwN9i3XbZaE8xADlJw6QWJxVeYmlvMQBEsvtKLHEQNM0DLmjCQFKKbtpR8sLopRi3rx5/Pbbb3zyyScAjBo1Sl8eFhbG/PnzOXHiBI0aNSpUWRMTU4u1++K8dyMkJKQU23bLQmCgr8vHAOUnDpBYnFV5iaW8xAESS0GMRsMtfxCX2KWEkJAQEhIS9OmEhASCgoIKXH758mW75fmxWq2MHTuWuLg4PvnkE3x9c7KnFStWcOXKFX09pRRmc4nlPA7JGAlCCCFcVYklBu3atWPHjh0kJSWRkZHB5s2bCQ0N1ZdXq1YNDw8P9u3bB0BMTIzd8vy88cYbpKam8uGHH+pJAcCePXtYs2YNALt370bTNOrUqVMCURVO3kaHSvoyEEII4UJK7Gd1cHAwY8aMYdiwYVgsFgYMGECzZs2Iiopi9OjR3HvvvURHRzNlyhRSU1Np0qQJw4YNK3B7SUlJrFy5kurVqzNw4EB9fkxMDJMnT2bixInExMTg4eHB/PnzMRrLrouGvDUGVpuGm9lUZmURQgghisKg5Cdtsbcx2H0kgXe/jgNg8Qsd8fFyK7Ztl7byco2uvMQBEouzKi+xlJc4QGIpyB23MbBYLMVSkL8SW55cKzPbeos1hRBCCOfiMDEYMGBAaZSjXMlb+5BlkT4NhBBCuA6HiYGXlxcXL14sjbKUG3kTg2yLrQxLIoQQQhSNw8aHGRkZPPzww4SEhODt7a3PX79+fYkWzJXlvSshK1sSAyGEEK7DYWIwefLk0ihHuZK3g6NMqTEQQgjhQhxeSnjggQfw8PBg9+7d/Pzzz/o8UTDNJpcShBBCuCaHicG6desYPXo0165dIy0tjZdfflkfBVHkL+9dCXIpQQghhCtxeCnh448/5ssvv9S7K46KiuKpp57i0UcfLfHCuaq8jQ/lUoIQQghX4rDGQNM0uzEMgoODy7RXQVcgdyUIIYRwVQ6/4f39/dm6das+vXXrVipWrFiihXJ1Nk3h7mbEYIAsSQyEEEK4EIeXEqZOncpzzz3HrFmzMBgMmM1m3n777dIom8vSlMJoMODhZiJT2hgIIYRwIQ4TAz8/P2JjYzl58iQ2m406depw8uTJUiia69K064mBuyQGQgghXEuBlxKuXr3K1atXiYqKIjU1lcqVKxMUFMTVq1f5v//7v9Iso8vRNIXRaMDLw0xGpoyVIIQQwnUUWGPw8ssv6/0WPPjgg/p8k8lE9+7dS75kLsx2PTHw9jCTniWJgRBCCNdRYGKwfPlyACZNmsTcuXNLrUDlQc6lBPD2NHMtLbusiyOEEEIUmsO7El544QWmT58OwIkTJ3juuee4fPlySZfLpWlKLiUIIYRwTQ4Tg4kTJ1KnTh0AqlWrxgMPPMCkSZNKvGCuLLfxoVxKEEII4WocJgZXrlxh2LBhAHh4ePDEE0+QkJBQ4gVzZXobA08zGVlWVJ4ukoUQQghn5jAxsNlsxMfH69OXL1+WLzoHcmsMvDzM2DRFtkUr6yIJIYQQheKwH4MnnniCPn360LFjRwwGA9u3b2f8+PGF2vj69etZunQpVquV4cOHM2TIELvlhw4dYvLkyaSlpdG6dWtmzJiB2fxnkRYtWoTJZGLUqFEAJCcnM3bsWM6cOUNAQACLFi0iMDCQ7OxsJk+ezMGDB/H09CQ6Opq6desW5TgUq9w2Bt4eObGkZ1nxcDeVWXmEEEKIwnJYYzBgwAA++ugjGjduTNOmTVm+fDkREREONxwfH8/ChQv57LPPWLduHatWreLYsWN264wbN45p06axadMmlFL6qI0pKSm88sorfPTRR3brL1q0iNatW7Nx40YGDhzInDlzAFixYgVeXl5s3LiRV155pczbQOhtDDz/TAyEEEIIV1Co0ZBCQkLo1q0bDz/8MF5eXnr/Breyfft22rRpg7+/P97e3nTr1o3Y2Fh9+blz58jMzKR58+YA9OvXT1/+7bffUqtWLZ588km7bW7btk1PSsLDw/nxxx+xWCxs27aN3r17A3D//feTlJTE+fPnCxNaichpYwBe12sM5M4EIYQQrsLhpYS33nqL9957L2dls5ns7Gzq1avH+vXrb/m6S5cuERgYqE8HBQVx4MCBApcHBgbqbRn69OkDwJIlSwrcptlsxsfHh6SkpHy3dfHiRe666y5H4ZWIvHclgNQYCCGEcB0OE4OYmBi+//57Xn/9dcaPH8/OnTv54YcfHG5Y0zQMBoM+rZSym3a0vDCUUhiNxptemzu/sCpX9inSfh3RNIWHu5lqVXNGoTS7mwkM9C3WfZQmVy57XuUlDpBYnFV5iaW8xAESy+1wmBgEBAQQFBREnTp1OHz4MH369OH99993uOGQkBD27t2rTyckJBAUFGS3PO9tj5cvX7Zbnp+goCAuX75MSEgIVquVtLQ0/P39CQ4O5tKlS9SsWbPQ28orMTEVTSu+Oy00pbBpGlkZOb0eXkxIISEhpdi2X5oCA31dtux5lZc4QGJxVuUllvISB0gsBTEaDbf8QezwZ7XZbOb06dPUqVOHvXv3YrVaycrKcrjjdu3asWPHDpKSksjIyGDz5s2Ehobqy6tVq4aHhwf79u0Dcmom8i7PT1hYGOvWrQNgw4YNtG7dGjc3N8LCwoiJiQFg7969eHh4lNllBLjexsBgoML1xocp6ZYyK4sQQghRFA4Tg5EjRzJ16lQ6derEli1b6NSpE23atHG44eDgYMaMGcOwYcPo06cP4eHhNGvWjKioKOLi4gCIjo5m7ty5dO/enfT0dL0jpYK88MIL/Prrr/Tq1YvPPvuMadOmATB06FCys7Pp1asXc+bMYd68eYWJvcTkjpVgNhmp4GkmJV3GSxBCCOEaDKqA3ooWLVrEiy++yL59+2jVqhUAGRkZnDp1ioYNGxa5PYAzK+5LCfM+34/RAGMHtWDSezupGeTD3/s0Lbbtl6byUhVXXuIAicVZlZdYykscILEU5LYvJXzzzTfEx8czY8YMrl27xtWrV8nKyiIkJIRr164VS+HKq9wukQF8vd2kxkAIIYTLKLDxYfv27enUqRMADz74oN0yg8HAoUOHSrRgrkxTOW0MAHy93Lh0JaOMSySEEEIUToE1BjNmzODQoUO0bNmSw4cP2/1JUnBruf0YAPhVcJcaAyGEEC7DYePDlStXlkY5yhXthksJqRlWNBl4SgghhAsofC9AotDs2hh4uaMpRbp0iyyEEMIFSGJQAnJvV4ScGgNALicIIYRwCYVODJKTk0uyHOVK7rDLkNPGAOBaqiQGQgghnJ/DxODEiRP07NmTXr16ER8fT48ePTh+/HhplM1l5W18WMnXA4ArqY57ixRCCCHKmsPEYPbs2UyePJnKlSsTHBzM448/rvc4KPKXt42BnhikSGIghBDC+TlMDK5evUr79u316SFDhpCamlqihXJ1eWsMPN3NeHuYuZIsiYEQQgjnV6g2BllZWXoXyAkJCWiaVqKFcnV52xgAVPLzICklswxLJIQQQhSOw2GXH3vsMZ566ikSExOZP38+//73v3n66adLo2wuK+9dCZBzOUEuJQghhHAFDhODgQMHUqtWLbZt24bVamXWrFl2lxbEzfK2MQAI8PXgTLxcfhFCCOH8HCYGuaMs3n///fq82bNnM2XKlBItmCvL28YAoJKvJ8lp2VhtGmaTdB0hhBDCeRWYGCxevJjk5GQ2bNhg19jQYrHwn//8RxKDW9BuqDGo5OuBAq6mZlGlolfZFUwIIYRwoMDE4L777iMuLg6j0Yi/v78+32QyER0dXRplc1l5R1cECPDLuWUxKVkSAyGEEM6twMQgLCyMsLAwQkNDadasWWmWyeXd2MYgyD8nGYi/kk6DGv5lVCohhBDCMYdtDPz8/Jg9ezbp6ekopdA0jVOnTvHFF1+URvlcjlIKpbC7K6FyRU9MRgOXrmSUXcGEEEKIQnDYEu7ll1/GYrGwf/9+qlWrxrFjx2jQoEFplM0l5Q6vnLfGwGQ0UrmipyQGQgghnJ7DxCAtLY0ZM2bQoUMHQkND+eijj/j1119LoWiuKbfvp7xtDACCKnlJYiCEEMLpObyUkNvw8O677+b333+nWbNmei+Ijqxfv56lS5ditVoZPnw4Q4YMsVt+6NAhJk+eTFpaGq1bt2bGjBmYzWbOnz/PuHHjSExMpHbt2kRHR1OhQgX69euHzWYDIDMzkzNnzvDjjz+SlZVFeHg4NWvWBKBKlSosX768KMeh2GhaTo2ByWh/jIIreXPs7AWUUoU+fkIIIURpc1hjcPfddzNnzhxatmzJp59+yooVK7BarQ43HB8fz8KFC/nss89Yt24dq1at4tixY3brjBs3jmnTprFp0yaUUqxevRqAGTNmMHjwYGJjY2natCnvvPMOAF999RUxMTHExMRw3333MXr0aKpUqcLBgweJiIjQl5VVUgB/Xkq48cs/qJIXmdk2UtItZVEsIYQQolAcJgbTp0+ndevWNG7cmIEDB7Jz505mzpzpcMPbt2+nTZs2+Pv74+3tTbdu3YiNjdWXnzt3jszMTJo3bw5Av379iI2NxWKxsGfPHrp162Y3P68dO3Zw+PBhoqKiAIiLi+Po0aNERkYybNgwjhw5UugDUNzya2MAEFwp586Ei0nppV4mIYQQorAcJgbPPvus/iU9ePBg3n77bVq1auVww5cuXSIwMFCfDgoKIj4+vsDlgYGBxMfHc+XKFXx8fDCbzXbz81q8eDFjxozBZDIB4OHhQe/evfn666956qmneP7558nOznZYxpJQ0KWEalV8ADh3Oa3UyySEEEIUlsM2BikpKaSnp+Pt7V2kDWuaZledfuO19YKW53cNPu/077//zpUrV+jcubM+b9SoUfrzsLAw5s+fz4kTJ2jUqFGhylq5sk/hA3PAnJwziqKfnyeBgb76/CpVfKjgaSYxJctuvitwtfIWpLzEARKLsyovsZSXOEBiuR0OEwMvLy86d+5Mw4YN7ZKDd99995avCwkJYe/evfp0QkICQUFBdssTEhL06cuXLxMUFERAQAApKSnYbDZMJtNNr9u6dSs9e/a029eKFSsIDw+nUqVKQE6SkVvjUBiJian6L/07lTuKYlpaFgkJKXbLqlapwO+nr9w035kFBvq6VHkLUl7iAInFWZWXWMpLHCCxFMRoNNzyB7HDb88BAwbc1o7btWvHkiVLSEpKwsvLi82bNzNr1ix9ebVq1fDw8GDfvn20atWKmJgYQkNDcXNzo3Xr1mzYsIGIiAjWrVtHaGio/rpff/2V4cOH2+1rz549ZGZmEhUVxe7du9E0jTp16txWue9UboJx4+2KANUDfdj1v3i5M0EIIYTTcpgY9O3b97Y2HBwczJgxYxg2bBgWi4UBAwbQrFkzoqKiGD16NPfeey/R0dFMmTKF1NRUmjRpwrBhwwB49dVXmThxIkuXLqVq1aosWLBA3+6ZM2cIDg6229fkyZOZOHEiMTExeHh4MH/+fIzGshnFMLfx4Y1tDABqBFZgW5aVKylZBPh5lnbRhBBCCIcKX99+GyIiIoiIiLCb9/777+vPGzVqxJo1a256XbVq1VixYkW+29ywYcNN84KDg/noo4/usLTF45Y1BkE5VTenLqZIYiCEEMIplc3P6nJM78cgnyNbM9gXk9HAiQvJpVwqIYQQonAkMShmt6ox8HAzUT3Ih+PnrpV2sYQQQohCcXgpYejQoXYN5QwGA15eXtSvX5+RI0fi41N8t/qVB7k3N+TXxgCg7l1+/HzwItoNQzMLIYQQzsBhjUG9evVwc3Nj6NChDB8+HF9fX7y9vcnMzGT69OmlUETXcqsaA4C6d1UkK9vGeenoSAghhBNyWGNw4MABVq1apfcLEBYWxuDBg1mwYAHh4eElXkBX82cbg/wTgzrV/AA4dv6a3hhRCCGEcBYOawxSUlJQ6s/OfzRNIz09p7//srol0Jk5qjEI8veiYgV3Dp+6UprFEkIIIQrFYY1B586dGTFiBH369EEpxb/+9S86derEv/71L6pUqVIaZXQpt+rHAHLaaDSuVYmDfyShKVVgAiGEEEKUBYc/+SdMmECvXr349ttv+fHHH4mMjOTll1+mcuXKzJ07tzTK6FL+rDEoeJ3GtQJISbdw9lJqKZVKCCGEKByHNQZGo5F+/frRo0cP/ZLCtWvXaN++fYkXzhXpicEtMoPGtQIA+N/JK9QMLj8DfAghhHB9DhODzz//nLlz52KxWIA/R0E8dOhQiRfOFeXernirsRAq+XpwV5UKxJ1IpPuDNUupZEIIIYRjDhOD5cuX8/nnn9OkSZPSKI/Lc9TGIFeL+lXYuPM0qRkWfLzcSqNoQgghhEMO2xhUqVJFkoIisBXiUgJAq4aBaEqx//eEW64nhBBClCaHiUGHDh347LPPiI+P5+rVq/qfyJ9ycLtirruDfans58m+I5IYCCGEcB4OLyW89957ZGdnM3PmTH2etDEomN7BkYO7EA0GA60aBvLdL2dJz7Tg7SmXE4QQQpS9QvV8KArP0VgJeT3YOJjNe86w69AlOreoVsIlE0IIIRwrMDGIiYkhMjKSjz76KN/lTz75ZIkVypXZNA1w3MYAoFaIL9UDK/DTb+clMRBCCOEUCkwMTp06BcDRo0dLrTDlgcrJCwrVo6HBYKBjs7v4/NvfOXMplRoydoIQQogyVmBiMHr0aADp3bCIHA2idKO2TUP4ctsxvt9/jmHdGpZk0YQQQgiHHLYx2L9/PwsWLODatWt2gymtX7++RAvmqnJ7PjQVcgwEHy832jYJ4ee4C/TpWBs/b/eSLJ4QQghxSw4Tg2nTptGvXz8aN258y9788rN+/XqWLl2K1Wpl+PDhDBkyxG75oUOHmDx5MmlpabRu3ZoZM2ZgNps5f/4848aNIzExkdq1axMdHU2FChXYvXs3o0aNIiQkBIDGjRszd+5csrOzmTx5MgcPHsTT05Po6Gjq1q1bpLIWF5sqXD8GeXV/sCY/HbjAd/vO0qdjnZIqmhBCCOGQw34MzGYzTz75JA8++CAPPPCA/udIfHw8Cxcu5LPPPmPdunWsWrWKY8eO2a0zbtw4pk2bxqZNm1BKsXr1agBmzJjB4MGDiY2NpWnTprzzzjsAHDx4kBEjRhATE0NMTIx+mWPFihV4eXmxceNGXnnlFSZNmlTkA1FcVCEGUbpR1coVaFG/Ct/uO0tmtrWESiaEEEI45jAxqF+/PkeOHCnyhrdv306bNm3w9/fH29ubbt26ERsbqy8/d+4cmZmZNG/eHIB+/foRGxuLxWJhz549dOvWzW4+QFxcHP/5z3+IiIjg2Wef5cKFCwBs27aN3r17A3D//feTlJTE+fPni1zm4qCPlVCUzADo2fZu0jKtbN5zpgRKJYQQQhSOw0sJZ86coX///tx11114eHjo8x21Mbh06RKBgYH6dFBQkF2fCDcuDwwMJD4+nitXruDj44PZbLabD+Dr60uPHj3o2rUrn3/+OWPGjOGLL77Id1sXL17krrvuchResdPbGBQxMah7V0VaNQhk467TdGpeDb8K0tZACCFE6XOYGDz//PO4uxf9S0rTNLs2CbmjMjpafuN68OdIhXl7X3zssceYP38+KSkpN71GKYXR6LAyRFe5cvHdJuh1vfFgUKBvkXszfLrvvTz/5vds+eUcz/ZrVmxlulOBgeVjaOjyEgdILM6qvMRSXuIAieV2OEwM3nzzTWJiYoq84ZCQEPbu3atPJyQkEBQUZLc8IeHPcQIuX75MUFAQAQEBpKSkYLPZMJlM+us0TWPZsmU888wzmEwm/XUmk4ng4GAuXbpEzZo17bZVWImJqfov/TuVkpIJQFJiGmnuJgdr2/MwQOfm1diw/Q9a1qtM7ap+xVKmOxEY6EtCQkpZF+OOlZc4QGJxVuUllvISB0gsBTEaDbf8QezwZ7WXlxcXL14s8o7btWvHjh07SEpKIiMjg82bNxMaGqovr1atGh4eHuzbtw/I6WkxNDQUNzc3WrduzYYNGwBYt24doaGhGI1GtmzZwqZNm/T59913H97e3oSFhenJy969e/Hw8CiTywjwZz8GRaiwsNM3tA4VK7jz8cbDWG1aMZZMCCGEcMxhjUFGRgYPP/wwISEheHt76/MdtTEIDg5mzJgxDBs2DIvFwoABA2jWrBlRUVGMHj2ae++9l+joaKZMmUJqaipNmjRh2LBhALz66qtMnDiRpUuXUrVqVRYsWADAG2+8wdSpU3n77bcJCAhg3rx5AAwdOpRp06bRq1cv3N3d9fllQSvksMsF8fY0M+SRhrz9dRyxu04T3q5WMZZOCCGEuDWDyttrUT52796d7/zC3LLoKorzUsK6n07wr59PsnxC5yL3+5DX0nUH+eVoAhOHtKRutYrFUrbbUV6q4spLHCCxOKvyEkt5iQMkloLc8aWEBx54gAYNGlCjRg2qV69O1apVsVgsxVK48khTOUMu30lSADC8e0Mq+Xrwbsx/ScuU4y2EEKJ0OLyU8NZbb/Hee+8BOQ39LBYL9erVky6RC6CUKtQASo54e7rxbGRT5n66j3fXHeSFgfdhNt1mwwUhhBCikBx+08TExPD999/TrVs3Nm/ezNy5c6lXr15plM0laZoqch8GBalzlx/DujXkvyev8NmWozi46iOEEELcMYeJQUBAAEFBQdSpU4fDhw/Tp08fGYr5Fmyauu2Gh/npeN9d9GxzN9t+Pc+/d5wqtu0KIYQQ+SnUWAmnT5+mTp067N27F6vVSlZWVmmUzSVpqngTA4B+YXVo0ySYr348wcZdkhwIIYQoOQ4Tg5EjRzJ16lQ6derEli1b6NSpE23atCmNsrkkpVEsbQzyMhoMPNXrHh64J4gvvz8uyYEQQogS47DxYefOnencuTOQ06nQqVOnaNiwYYkXzFVpSmEyFW9iAGAyGomKaAzAl98f50pKFoMeql/stRNCCCH+2hzWGKSlpTFjxgyGDx9OVlYWn332Genp6aVRNpdk04rnroT8mIxGnundhK7312Dr3rO8s+6gDNMshBCiWDlMDGbPno2fnx+JiYl4eHiQmprKtGnTSqNsLqkk2hjkZTQYGPRwfR57uD77f09g1j/3cu5yWontTwghxF+Lw8Tg0KFDjBkzBrPZjJeXF9HR0Rw6dKg0yuaSVDHflVCQR+6vwdi/NSctw8Ksf+7h57gLcjujEEKIO+YwMbhx+GKbzVakIY3/ajRVfP0YOHJPrQBeffIBaoX4sfzfh1iyNo6rqXLHiBBCiNvn8Bv+/vvv58033yQzM5OffvqJUaNGlatxEopbSbYxyE8lXw/GP9aCQQ/V478nk5jy/i5+/O28PsqjEEIIURQOE4OxY8fi7e2Nr68vCxcupGHDhkyYMKE0yuaSNHX7IyveLqPRQNcHajJjxANUD6zAxxsPM+ufe/n97NVSLYcQQgjX5/B2RTc3N55//nmef/55fd7vv/9O/fr1S7Rgrqq02hjkJyTAmwlDWrLrUDxffn+cuZ/+QquGgUR2qE31wIJH0hJCCCFyOUwM8vO3v/2NX375pbjLUi6UZhuD/BgMBto0DqFFvUA27jrF5j1n+OVIAq0bBdG7Q22qValQZmUTQgjh/G4rMZDW7wUr7rESbpeHu4k+HevQpXUNNu0+zdZ9Z9lz+BLN6lbmkftr0PjuSnc8NLQQQojy57YSA/lCKZhWTMMuFxcfLzf6h9Wl6/01+HbfWbbtP8f8L36lWpUKPNy6Og/eE4yXx229DYQQQpRD8o1QzMqyjcGt+Hq706djHXq1vZtd/7vE5j1n+CT2CF98+zutGwbR4d6qNKjp71RJjRBCiNJXYGLQokWLfGsGlFJkZmaWaKFcmaYo0zYGjriZTXRoVpX294Zw4nwy/4m7wK7/xbP94EUC/T25v1Ew9zcKomawj9QMCSHEX1CBicE333xTmuUoN2yawt3s/B1AGQwG6larSN1qFRn0cH1+OZLA9v9eJHbXaTbsPEWVip60bhTEww/eTSVPs1PWggghhCh+BSYG1apVu+ONr1+/nqVLl2K1Whk+fDhDhgyxW37o0CEmT55MWloarVu3ZsaMGZjNZs6fP8+4ceNITEykdu3aREdHU6FCBY4fP860adNITU3F09OT6dOnc88993Du3DnCw8OpWbMmAFWqVGH58uV3XP7b4WxtDArDw81E26YhtG0aQmqGhf1HE9h7JIEte84Qu+s0Pl5uNK0dwL11KtOkdgB+FdzLushCCCFKSIm1MYiPj2fhwoV89dVXuLu7M2jQIB588EHq1aunrzNu3Dhmz55N8+bNeeWVV1i9ejWDBw9mxowZDB48mF69evH222/zzjvvMG7cOKZMmcLIkSPp1KkTO3bsYMKECfzrX//i4MGDREREMHPmzJIKp9CctY1BYfl4udHxvrvoeN9dpGVaOJWQzs+/nuPgH4ns/F88BuDuEF8a1axEg5r+NKheEW9Pt7IuthBCiGJSYnXe27dvp02bNvj7++Pt7U23bt2IjY3Vl587d47MzEyaN28OQL9+/YiNjcVisbBnzx66detmNx9g4MCBdOzYEYCGDRty4cIFAOLi4jh69CiRkZEMGzaMI0eOlFRYDpV1PwbFqYKnG2EtqxMV0ZiFozowdXhr+nSsjZvZyNZ9Z1i85gCj3vqJ6R/t5vOtv/PL0QSuyVgNQgjh0kqsxuDSpUsEBgbq00FBQRw4cKDA5YGBgcTHx3PlyhV8fHwwm8128yEnSci1ePFiunTpAoCHhwe9e/dm0KBB/PTTTzz//PNs2LABd/fSr/J2ln4MipvRYKB2VT9qV/Ujon1tsi02jp9P5sjpKxw9c5Vtv55jy94zAAT4eVC7qh91rq9/d4iv3BIphBAuosT+W2uaZteqXSllN13Q8hvXA25ab968efz222988sknAIwaNUpfHhYWxvz58zlx4gSNGjUqVFkrVy6+7oKNRiNGg4HAQN9i22ZZKyiWanf5E9o6p12HxWrj9zNXOXr6Kr+fvsLRM1fYdyQBAIMBqgf5ULtqRWrd5UftuypSq6oflSt6luqdD3+Fc+KKJBbnU17iAInldpRYYhASEsLevXv16YSEBIKCguyWJyQk6NOXL18mKCiIgIAAUlJSsNlsmEwmu9dZrVYmTJhAfHw8n3zyCb6+OQdpxYoVhIeHU6lSJSAnecitcSiMxMRUNK14enO0WGwYjQYSElKKZXtlLTDQt9CxBPq4E9g4iPaNc85XSno2f1xI4Y8LyZy6mMJ/TyTy46/n9PUreJqpHuhDjSAf7gqsQNUAb0IqV8DP263YE4aixOHsJBbnVF5iKS9xgMRSEKPRcMsfxCWWGLRr144lS5aQlJSEl5cXmzdvZtasWfryatWq4eHhwb59+2jVqhUxMTGEhobi5uZG69at2bBhAxEREaxbt47Q0FAA3njjDVJTU/nwww/tLhPs2bOHzMxMoqKi2L17N5qmUadOnZIK7ZbKUxuDO+Xr7U6zupVpVreyPi8908rZhFTOXErlbEIqZy+l8tOBC2RZbPo6Xh5mqlb2JiTAW38MCfAm0N8LdzdTWYQihBB/GQZVggMfrF+/nmXLlmGxWBgwYABRUVFERUUxevRo7r33Xg4fPsyUKVNITU2lSZMmzJ07F3d3d86dO8fEiRNJTEykatWqLFiwAJvNRocOHahevTpeXl76PmJiYoiPj2fixIkkJCTg4eHBnDlzCn0ZAYq3xmDish00qhXAE90aFsv2ylppZNyaUiQlZ3IxMZ0LSelcTErnYmLO45UU+8aMFX3cCfT3IrCiF4H+ngT6e1GlYs6jv69HgbeKyi8H5ySxOJ/yEgdILAVxVGNQoomBqyjOxGD80u00rVeFYY80KJbtlbWy/mBlZFmJv5LOhcR0Ll/NIOFqJglXM7h8LYOk5CzynjWzyUBlP08C/DwJ8POgkq8nAb4eBPh5UKdmAAabDW8Ps8v36FjW56Q4SSzOp7zEARJLQcrsUsJflXLBDo6cmZeHmVohftQK8btpmdWmkZh8PVG4njAkXMvkSnIm/zt5haupWdyY9rq7GQnw9aTS9YTB38eDihXcqZj7WMEdvwrueLqbXD6BEEKI2yGJQTFz9rESyhOzyUhwJW+CK3nnu9ymaVxLzSYpJQubwcCps1dJSskiKSVLTx6upWaj5VNp5u5mvJ4o5CQMfj7ufyYO3u74ervj4+2Gj5cb3p5mSQaFEOWGJAbFrLz2Y+CKTEbj9csKngQG+tLwrptrHTSlSM2wkJyazbW0bK6lZXEtLZvktOvTqdlcTErnyJmrpGZY8t2PwZDTY6SPlxu+Xm74eLvnPL+eOPz5PCeZqOBpxstdxp8QQjgnSQyKmSaJgUsxGgz4eefUAlR3sK7VppGclk1yejapGRZS0i2kpltIybCQmmEh9fr8+CvpHD+XM89WQNsVA+DpYaaCpxlvTzMVPN3w9sjz/Pp8u2mPnOeVArRiPw5CCJFLEoNippTCJNXK5ZLZ9GcNRGEopcjIspGakU1KbgKRbiE9y0p6poW0zDyPWVYuJKWTlmkhPdOKxXrrL393NyNe7mY8Pcx4uZvw8jDjef0xZ37ucxOeudPu5pz19OcmzCajtKUQQtiRxKCYaUphNMk/WpHTY2fur/6gSkV7rcVqIz3Tej15sOoJQ3qWFUxGEhLTyMy2kZltJSPLRka2lZSr2WRk/Tkvv7YTNzIZDXpS4eluwsPNhLvbn889ch9veO7pfqv1jJiMzj/0uBAif5IYFDObJncliDvnZjZR0cdERR+Pm5YV5rYlpRQWq0ZGto3MLCsZ15MFu+fZVjKzbWRkWcnIspJl0ciy2MjKtpGaYSEr20aWxUamxUZ2to2i3NBrNhluShjc3Uy4uxlxN+c+GvHz9cRmteWZZ8LdbMTNbr2cRzezCQ+zETe3nHXc3SQBEaIkSGJQzJrVrcI9tQLKuhjiL85gMFz/IjZRscKdDyamlCLb+mficGPSkGmx/bksn8dMiw2LRSMtw8IVaxbZFhvZVg3r9W1abbfXj4jJaNCThpxkwYSb2agnEG4mI2azETeTEbc8jznzDLiZc9Y3mww5y825rzFcX9eUZ1me7ZiNchlGlFuSGBSz5/o0LVedaggBOYlG7i9/8r879LbkflY0TZFtzUkWsi02LFaNbIt207ysG5bZz8tdV8NizakJSbFqWGwaluuPVuv151atSDUgBclNGswmIx7uZkwG7BIRc55Hs8mAyZjzmDN9fd71RzeTUX9uNhkxGXO2bf+aP19rym+e/hqDJC3itkliIIQoc0ajIaeRZCmNlK6UwqblXG6x3pg42P5MHqx5ntslFnavUVhsGiazkdS0bLvXZlttpGdZsdk0rDaF1aZd/1PYtOvziilJuZHJeGPikJuEGDEbDZjNOY+m6wmF6fpzby83rFbb9Xk5CYjJaMCc57kpn9f9+fz6utefmwp8fr08BWzDaJDkpqxIYiCE+MsxGAz6F2Zxud2aQqUUmlI5yUI+CUTeR5tNw6qpG5Zp2Gw5yYkt77zriY/NprBqBa9nsWlkZuckKjZNwRUD2RYrNk1hs+UkUDZNy/O89HrRzy+hMN+QsJiMRozXE4obH7083bBeH/E2d77dOoabX3PjckMR1zfmU6ai7NMZ2qhJYiCEEGXIYMj5sjAZAScYPdRRgpNb2/Jn4qAV8DwnISkoubBdT15yn1sL2IatgG1Y82xDUwpNy/mz2jQ0S846qRlWsixWtOv7yfcxz2tLM+kpiAFuTnA8zMwc2Q7PUmprK4mBEEKIQvuztgVwK+vS3FpRa3GUUiiFffKg8iYT2k3Jhf36ml2y4SgZyZ2+aR1lP+1mMlLRx4Os9CzHQRQDSQyEEEIIcpIegwGn7L3Wr4I7CaWUGMhNwEIIIYTQSWIghBBCCJ0kBkIIIYTQSWIghBBCCJ00PqRkGpo4Y+OV21VeYikvcYDE4qzKSyzlJQ6QWG5nOwalCjEEmxBCCCH+EuRSghBCCCF0khgIIYQQQieJgRBCCCF0khgIIYQQQieJgRBCCCF0khgIIYQQQieJgRBCCCF0khgIIYQQQieJgRBCCCF0khgUo/Xr19OzZ0+6du3KypUry7o4hfKPf/yDXr160atXL+bNmwfApEmT6Nq1K5GRkURGRrJlyxYADh06RL9+/ejWrRuTJ0/GarWWZdFvMnToUHr16qWX+7fffmP79u1ERETQtWtXFi5cqK/rrLF8+eWXevkjIyNp1aoVM2fOdLlzkpqaSnh4OGfPngUo8nk4f/48Q4YMoXv37vz9738nLS3NKeJYtWoV4eHhREREMGnSJLKzs4Gcz1Hnzp3185P7+XeWOPKLpajvKWeN5YcffrD7zLRp04aRI0cCzn9e8vv/6xSfFSWKxcWLF1Xnzp3VlStXVFpamoqIiFC///57WRfrln7++Wf1t7/9TWVlZans7Gw1bNgwtXnzZhUeHq7i4+NvWr9Xr15q//79SimlJk2apFauXFnKJS6YpmmqQ4cOymKx6PMyMjJUWFiYOn36tLJYLGrEiBFq27ZtSinnjiXX0aNH1SOPPKISExNd6pz8+uuvKjw8XDVp0kSdOXPmts7DM888o7755hullFL/+Mc/1Lx588o8jhMnTqhHHnlEpaSkKE3T1Pjx49VHH32klFJq5MiR6pdffrlpG84Qh1I3x6KUKvJ7ypljyXXp0iX18MMPqz/++EMp5dznJb//v+vXr3eKz4rUGBST7du306ZNG/z9/fH29qZbt27ExsaWdbFuKTAwkIkTJ+Lu7o6bmxt169bl/PnznD9/nldeeYWIiAgWL16MpmmcO3eOzMxMmjdvDkC/fv2cKr4TJ04AMGLECHr37s2nn37KgQMHuPvuu6lRowZms5mIiAhiY2OdPpZc06dPZ8yYMXh5ebnUOVm9ejWvvvoqQUFBAEU+DxaLhT179tCtWze7+WUdh7u7O6+++io+Pj4YDAYaNGjA+fPnATh48CDLli0jIiKCmTNnkpWV5TRx5BdLRkZGkd5TzhxLXvPmzWPQoEHUqlULcO7zkt//35MnTzrFZ0USg2Jy6dIlAgMD9emgoCDi4+PLsESO1a9fX3+jnTx5ko0bN9KxY0fatGnDa6+9xurVq9m7dy9r1qy5Kb7AwECnii85OZm2bdvy9ttv8/HHH/PFF19w/vz5fM+Js8cCOYlmZmYmPXr04PLlyy51TubMmUPr1q316YI+GwWV/8qVK/j4+GA2m+3ml7Yb46hWrRrt27cHICkpiZUrV/Lwww+TlpbGPffcw7hx4/j6669JTk7mnXfecZo48oulqO8pZ44l18mTJ9m9ezfDhg0DcPrzkt//X4PB4BSfFUkMiommaRgMfw5lqZSym3Zmv//+OyNGjGD8+PHUqVOHt99+m6CgILy8vBg6dCg//PCD08fXokUL5s2bh6+vLwEBAQwYMIDFixfnW2ZnjwXgiy++4MknnwSgRo0aLnlOchVUzoLm5xeHM8UVHx/P8OHD6d+/Pw8++CAVKlTg/fffp27dupjNZkaMGMEPP/zg1HEU9T3lzLHkWrVqFYMHD8bd3R3AZc5L3v+/NWrUcIrPiiQGxSQkJISEhAR9OiEhId+qLmezb98+nnjiCV5++WX69u3LkSNH2LRpk75cKYXZbL4pvsuXLztVfHv37mXHjh36tFKKatWq5XtOnD2W7Oxs9uzZw0MPPQTgsuckV0GfjYLKHxAQQEpKCjabzW59Z3D8+HEGDRpE3759ef7554Gcxl9r1qzR18k9P84cR1HfU84cS65vv/2Wnj176tOucF5u/P/rLJ8VSQyKSbt27dixYwdJSUlkZGSwefNmQkNDy7pYt3ThwgWef/55oqOj6dWrF5Dz4Xnttde4du0aFouFVatW8cgjj1CtWjU8PDzYt28fADExMU4VX0pKCvPmzSMrK4vU1FS+/vprXnrpJf744w9OnTqFzWbjm2++ITQ01OljOXLkCLVq1cLb2xtw3XOS67777ivSeXBzc6N169Zs2LABgHXr1jlFXKmpqTz11FO88MILjBgxQp/v6enJm2++yZkzZ1BKsXLlSh555BGnjQOK/p5y5lgg59JOZmYmNWrU0Oc5+3nJ7/+vs3xWDEopdcdbEUDO7YrLli3DYrEwYMAAoqKiyrpItzR79mzWrl1LzZo19XmDBg1C0zRWrlyJ1Wqla9eujB07FoDDhw8zZcoUUlNTadKkCXPnztWr7ZzBokWL2LRpE5qmMXjwYIYPH86OHTuYO3cuWVlZhIWFMWnSJAwGg1PHsmHDBrZs2WJ3q9LKlStd7pw89NBDfPLJJ1SvXr3I5+HcuXNMnDiRxMREqlatyoIFC6hYsWKZxrF161aio6OpW7eu3bIXXniBTZs2sWTJEiwWCy1btmTGjBlOF0feWKpXr17k95Qzx3LgwAFmz57N6tWr7dZx5vNS0P/fWrVqlflnRRIDIYQQQujkUoIQQgghdJIYCCGEEEIniYEQQgghdJIYCCGEEEIniYEQQgghdJIYCOGkHnroIeLi4oCcUdi2bt1arNsfMWIESUlJAERFRXHs2LFi3X5+cvsCyMzM5K233mLdunVFen3Dhg31MhfWtm3beOutt265zldffaWPyFccxo4dy/Hjx4tte0KUJnNZF0AI4diuXbuoV69esW7z559/1p+///77xbrtgkRHRzNw4EA8PT154YUXSmWfcXFxXLt2rVT2leuFF17g5ZdfZtWqVU7XdbAQjkhiIISTW7lyJQcPHmTevHmYTCbCwsKIjo5mz5492Gw2GjduzJQpU/Dx8eGhhx6iWbNmHDlyhJdeegmz2cyyZcvIzs4mKSmJPn368OKLLzJp0iQAhg8fznvvvceQIUN46623uPfee1m1ahUrVqzAaDRSpUoVpk6dSu3atZk4cSI+Pj4cOXKEixcv0rBhQ9544w0qVKjA4sWL2bJlC25ublSqVIm5c+fe1DXrhQsX+P7775kyZQoAEydOpH79+jz11FPce++9PPPMM/z8889cunSJp59+msGDB+d7PBYtWkRcXByapvHiiy/SuXNn0tPTmT59OqdOneLq1atUqFCB6OhoUlJS+OKLL7DZbPj6+jJmzBiWLVvG119/jdls5u677+b1118HcrqTfeaZZ7hw4QImk4n58+dTt25dUlJSmDNnDkePHsVisdC2bVvGjx+P2WwuMO4aNWrg6+vLt99+S5cuXUrw3SFECbjjgZuFECWic+fO6sCBA0oppR5//HG1ceNGpZRSS5YsUa+//rrSNE0ppdT8+fPVq6++qr/mH//4h1JKKU3T1OOPP66PTX/x4kV1zz33qMTERKWUUg0aNNCf5+5r+/btqkuXLvr8tWvXqh49eihN09SECRPsxo/v06ePWrNmjTp//rxq2bKlysrKUkoptXz5crVly5ab4lmxYoWaMGGCPj1hwgT1wQcf6GVZsWKFUkqpuLg41bRpU5WZmXnTNho0aKCWLVumlFLqyJEj6oEHHlCJiYlq48aNatasWfp6U6dOVTNnzlRKKbV48WI1Y8YMpZRSW7duVV27dlVXr15VSin12muvqXfeeUetXbtWtW7dWp08eVIppdSsWbPUpEmTlFJKTZw4UX3yySdKKaWsVqsaO3aseu+99xzG/eGHH6rx48ffFIMQzk5qDIRwMdu2bSMlJYXt27cDYLFYqFy5sr48d0hag8HAu+++y7Zt2/jmm284fvw4SikyMjIK3PZPP/1Ez549CQgIAHLGd58zZw5nz54FoGPHjnqXyw0aNODatWsEBwfTqFEj+vbtS2hoKKGhobRt2/ambZ84ccKu+9cbPfzwwwA0adKE7Oxs0tPT8fDwuGm9xx57TN9/3bp12b9/P927d6dGjRqsWLGCU6dOsXv3blq0aHHTa3fs2EH37t31LmNza06++uormjVrxt133w3APffcw5YtW4Cc4x0XF6cPyJOZmQngMO7q1auzcePGAuMVwllJYiCEi9E0jVdeeYWwsDAgZ9z5rKwsfXnu4Evp6en07duXLl260Lp1a/r378/WrVtRt+gFXdO0m+YppbBarUDOwDS5cod9NRqNfPrpp8TFxbFjxw5ee+01OnbsyPjx4+22kzt8bEFyk4Dca/IFldNo/LPNtKZpmM1mPvvsM1avXs2QIUOIiIjA399fT2byMplMdtf8k5OTSU5OBtDHtM8bW+4+3nrrLX2MhOTkZAwGg8O4zWazXVmFcBXyrhXCBZhMJv3LuUOHDqxcuZLs7Gw0TWPq1KksWLDgptecOnWK1NRUXnzxRR566CF27dqlv+bGbebq2LEjGzZs0Fv+r127Fn9/f/2XdH4OHz5MeHg4devWZeTIkTzxxBP63RR51a5dmzNnztz2Mcj19ddfA/Df//6X06dPc9999/Gf//yHvn37MnDgQGrXrs13332nD0WbN8527dqxZcsWUlNTAViyZAkff/zxLffXoUMHPv74Y5RSZGdn8/e//51PP/3UYdxnz56lTp06dxyvEKVNagyEcAEPPfQQCxYswGKx8Nxzz/HGG2/Qt29fbDYb99xzDxMnTrzpNQ0bNqRTp0706NEDd3d3GjRoQL169Th16hQ1a9ake/fuDB06lCVLluivad++PU888QTDhw9H0zQCAgJYtmzZLX/5NmrUiB49etC/f3+8vb3x9PTUGxjm1aVLFz744ANsNhsmk+m2j8WZM2fo06cPBoOBBQsW4O/vz4gRI5g2bZpe3d+8eXOOHj0KQJs2bRg7diyzZs1i6tSpHDt2TL8cUa9ePWbNmsXmzZsL3N/kyZOZM2cOERERWCwW2rVrx9NPP42bm9st4/7pp594/PHHbztOIcqKjK4ohCg1U6dOpW3btvTs2bOsi1KiTp8+zdixY+V2ReGS5FKCEKLUjBs3ji+//FJvwFdeLVq0iNmzZ0tSIFyS1BgIIYQQQic1BkIIIYTQSWIghBBCCJ0kBkIIIYTQSWIghBBCCJ0kBkIIIYTQSWIghBBCCN3/A9nRR/4Q8nfwAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Needed for initializing the lr scheduler\n",
    "p = nn.Parameter(torch.empty(4,4))\n",
    "optimizer = torch.optim.Adam([p], lr=0)\n",
    "\n",
    "lr_scheduler = SqrtScheduler(optimizer=optimizer, factor = 1.0, model_size = 512, warmup=10, max_iters=2000)\n",
    "\n",
    "# Plotting\n",
    "epochs = list(range(2000))\n",
    "sns.set()\n",
    "plt.figure(figsize=(8,3))\n",
    "plt.plot(epochs, [lr_scheduler.get_lr_factor(e+1) for e in epochs])\n",
    "plt.ylabel(\"Learning rate factor\")\n",
    "plt.xlabel(\"Iterations (in batches)\")\n",
    "plt.title(\"Cosine Warm-up Learning Rate Scheduler\")\n",
    "plt.show()\n",
    "sns.reset_orig()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DVg0w41BrIxl"
   },
   "source": [
    "The code below is used as a sanity check to verify the good implementation of your code. Do not change what follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "KE6p1MaArIxl",
    "outputId": "bcf0c756-8a23-4439-df22-4fac53f1b123"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current device: cuda - Type: NVIDIA GeForce RTX 2070\n"
     ]
    }
   ],
   "source": [
    "# Select GPU device for the training if available\n",
    "if not torch.cuda.is_available():\n",
    "    device=torch.device(\"cpu\")\n",
    "    print(\"Current device:\", device)\n",
    "else:\n",
    "    device=torch.device(\"cuda\")\n",
    "    print(\"Current device:\", device, \"- Type:\", torch.cuda.get_device_name(0))\n",
    "\n",
    "\n",
    "enc_input_size = 2\n",
    "dec_input_size = 3\n",
    "dec_output_size = 3\n",
    "\n",
    "\n",
    "num_heads = 8\n",
    "d_model = 512\n",
    "dim_feedforward = 2048\n",
    "dropout = 0.1\n",
    "preds_num = 8\n",
    "\n",
    "def subsequent_mask(size):\n",
    "    \"\"\"\n",
    "    Mask out subsequent positions.\n",
    "    \"\"\"\n",
    "    attn_shape = (1, size, size)\n",
    "    mask = np.triu(np.ones(attn_shape), k=1).astype('uint8')\n",
    "    return torch.from_numpy(mask) == 0\n",
    "\n",
    "torch.manual_seed(0)\n",
    "tf = Transformer(enc_input_size, dec_input_size, dec_output_size, N=6, \n",
    "            d_model=d_model, dim_feedforward=dim_feedforward, \n",
    "            num_heads=num_heads, dropout=dropout).to(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_FySOGyfrIxl",
    "outputId": "0db640f9-ec56-4c16-85e5-c17ddfa4d49b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[1.4025, 0.1318, 2.2073],\n",
       "         [1.4107, 0.1384, 2.2111],\n",
       "         [1.4233, 0.1524, 2.2139],\n",
       "         [1.4406, 0.1682, 2.2192],\n",
       "         [1.4553, 0.1758, 2.2197],\n",
       "         [1.4646, 0.1773, 2.2245],\n",
       "         [1.4616, 0.1686, 2.2236],\n",
       "         [1.4464, 0.1582, 2.2220]]], device='cuda:0', grad_fn=<ViewBackward0>)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.seed(0)\n",
    "\n",
    "batch = torch.tensor(np.random.rand(1, 8,4)).float().to(device)\n",
    "inp = batch[:,1:,0:2].to(device) \n",
    "target = batch[:,:-1,2:].to(device) \n",
    "\n",
    "# We create a third mask channel to append to the 2 speeds. \n",
    "# This helps the decoder differentiating between start of sequence token (with mask token 1) and target speeds (with mask token 0)\n",
    "# Summarizing: start_of_seq token is (0,0) and the mask token is 1 ---> [0, 0, 1]\n",
    "#              target inputs are (u_i, v_i) and the mask token is 0 ---> [u_i, v_i, 0]\n",
    "start_of_seq = torch.Tensor([0, 0, 1]).unsqueeze(0).unsqueeze(1).repeat(target.shape[0], 1, 1).to(device)\n",
    "target_c = torch.zeros((target.shape[0], target.shape[1], 1)).to(device)\n",
    "target = torch.cat((target, target_c), -1)\n",
    "# Final decoder input is the concatenation of them along temporal dimension\n",
    "dec_inp = torch.cat((start_of_seq, target), 1)\n",
    "\n",
    "# Source attention is enabled between all the observed input (mask elements are setted to 1)\n",
    "src_att = torch.ones((inp.shape[0], 1, inp.shape[1])).to(device)\n",
    "# For the target attention we mask future elements to prevent model cheating (corresponding future mask elements are setted to False)\n",
    "# The mask is changed dinamically to use teacher forcing learning\n",
    "trg_att = subsequent_mask(dec_inp.shape[1]).repeat(dec_inp.shape[0], 1, 1).to(device)\n",
    "\n",
    "# Source, target and corresponding attention mask are passed to the model for the forward step\n",
    "tf.eval()\n",
    "pred = tf(inp.float(), dec_inp.float(), src_att, trg_att)\n",
    "pred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xP-B2SU_rIxl"
   },
   "source": [
    "---\n",
    "\n",
    "This notebook was created by Luca Franco and Alessandro Flaborea."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "provenance": []
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "vscode": {
   "interpreter": {
    "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}