RNN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "seven-peter",
   "metadata": {},
   "source": [
    "# 序列建模：循环神经网络\n",
    "\n",
    "时间序列和文本分析都涉及到序列的分析：在时间序列中，指的是$\\left\\{x_t, t=1,2,...,T\\right\\}$，其中$x_t$可以是$k$维向量，而在文本分析中，如果我们把每个词都看成是一个向量，那么一个句子、文章就可以看成是词向量的序列。\n",
    "\n",
    "就像在时间序列中的分析一样，序列建模的关键点在于时间上的相依性：在时间维度上，不同时间点的$x_t$显然应该是相关的。特别是对于一些长记忆的模型，可能间隔时间比较长的过去也会对现在造成影响。\n",
    "\n",
    "而**循环神经网络**（**recurrent neural networks**，**RNN**）通过引入神经元之间的“循环”（cycles），来对序列的不同期之间相依性进行建模。\n",
    "\n",
    "在循环神经网络的建模中，有一个“隐藏的状态”（hidden state），包含了将观察到的时间序列进行运算并体现最终标签的信息。RNN的建模方法与状态空间模型、隐马尔科夫链等有非常类似的地方，但是不同于这些模型的地方是，在RNN中，并没有对隐藏状态的更新函数做任何假设。\n",
    "\n",
    "如果我们记$h_t$为隐藏的状态，那么循环神经网络的结构可以写为如下的形式：$$h_t=f\\left(h_{t-1},x_t\\right)$$其中$x_t$为当期的可观测变量，而$h_{t-1}$为上一期的隐藏状态。\n",
    "\n",
    "以上过程可以使用如下图表示：\n",
    "\n",
    "![](pic/RNN.gv.png \"RNN\")\n",
    "\n",
    "其中上图为RNN表示的一般简单表示，而下面的图是上面的图的展开形式。在上面的图中，我们使用了一个$h$指向$h$本身的箭头，说明隐藏状态$h$是自我更新的。\n",
    "\n",
    "经过迭代，第$t$期的状态可以写为过去所有可观测变量的函数：$$h_t=g\\left(x_1,x_2,...,x_t\\right)=f\\left(h_{t-1},x_t\\right)=f\\left(f\\left(h_{t-2},x_{t-1}\\right),x_t\\right)=\\cdots $$从而$f$可以看作是一个将过去的信息整合的过程，而由于$f$函数随时间的不变性，我们就可以根据当期的$h_t$预测下一期的$h_{t+1}$，进而对标签进行预测。所以这里的关键点在于$f$：$f$在不同期的函数形式相同，从而在不同期之间共享参数，可以看做具有马尔科夫性质。\n",
    "\n",
    "在PyTorch中，可以使用torch.nn.RNN()来创建RNN层，其函数形式为：$$h_t=tanh\\left(W_{ih}x_t+b_{ih}+W_{hh}h_{t-1}+b_{hh}\\right)$$我们一般使用tanh激活函数，不过也可以使用ReLU激活函数。\n",
    "\n",
    "下面我们使用一个简单的债券收益率预测展示RNN的用法。首先载入数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "statewide-transportation",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>IB010007</th>\n",
       "      <th>IB010011</th>\n",
       "      <th>SH010107</th>\n",
       "      <th>SH010303</th>\n",
       "      <th>SH010504</th>\n",
       "      <th>SH010512</th>\n",
       "      <th>SH010609</th>\n",
       "      <th>SH010619</th>\n",
       "      <th>SH010706</th>\n",
       "      <th>...</th>\n",
       "      <th>IB200003</th>\n",
       "      <th>IB209901</th>\n",
       "      <th>IB209902</th>\n",
       "      <th>IB209903</th>\n",
       "      <th>IB209904</th>\n",
       "      <th>IB209905</th>\n",
       "      <th>IB209906</th>\n",
       "      <th>IB209907</th>\n",
       "      <th>IB209908</th>\n",
       "      <th>IB209909</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2018-01-02</td>\n",
       "      <td>4.1168</td>\n",
       "      <td>3.8014</td>\n",
       "      <td>4.1168</td>\n",
       "      <td>4.2580</td>\n",
       "      <td>3.9000</td>\n",
       "      <td>3.7726</td>\n",
       "      <td>3.9000</td>\n",
       "      <td>3.8026</td>\n",
       "      <td>4.2265</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2018-01-03</td>\n",
       "      <td>4.0911</td>\n",
       "      <td>3.7837</td>\n",
       "      <td>4.0911</td>\n",
       "      <td>4.2822</td>\n",
       "      <td>3.9052</td>\n",
       "      <td>3.7370</td>\n",
       "      <td>3.9122</td>\n",
       "      <td>3.7873</td>\n",
       "      <td>4.2339</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2018-01-04</td>\n",
       "      <td>4.1428</td>\n",
       "      <td>3.8092</td>\n",
       "      <td>4.1428</td>\n",
       "      <td>4.3109</td>\n",
       "      <td>3.9137</td>\n",
       "      <td>3.7391</td>\n",
       "      <td>3.9216</td>\n",
       "      <td>3.8136</td>\n",
       "      <td>4.2414</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2018-01-05</td>\n",
       "      <td>4.1080</td>\n",
       "      <td>3.7726</td>\n",
       "      <td>4.1080</td>\n",
       "      <td>4.2829</td>\n",
       "      <td>3.9103</td>\n",
       "      <td>3.7046</td>\n",
       "      <td>3.9174</td>\n",
       "      <td>3.7765</td>\n",
       "      <td>4.2314</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2018-01-08</td>\n",
       "      <td>4.0987</td>\n",
       "      <td>3.7356</td>\n",
       "      <td>4.0987</td>\n",
       "      <td>4.2629</td>\n",
       "      <td>3.9036</td>\n",
       "      <td>3.6663</td>\n",
       "      <td>3.9086</td>\n",
       "      <td>3.7403</td>\n",
       "      <td>4.2214</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>521</th>\n",
       "      <td>2020-02-27</td>\n",
       "      <td>2.0304</td>\n",
       "      <td>2.1411</td>\n",
       "      <td>2.0304</td>\n",
       "      <td>2.5027</td>\n",
       "      <td>2.6275</td>\n",
       "      <td>1.8926</td>\n",
       "      <td>2.7841</td>\n",
       "      <td>2.1709</td>\n",
       "      <td>3.1601</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.7054</td>\n",
       "      <td>1.8110</td>\n",
       "      <td>1.7243</td>\n",
       "      <td>1.7327</td>\n",
       "      <td>1.7477</td>\n",
       "      <td>1.8384</td>\n",
       "      <td>1.7489</td>\n",
       "      <td>1.7499</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>522</th>\n",
       "      <td>2020-02-28</td>\n",
       "      <td>2.0280</td>\n",
       "      <td>2.1188</td>\n",
       "      <td>2.0280</td>\n",
       "      <td>2.5090</td>\n",
       "      <td>2.5566</td>\n",
       "      <td>1.9142</td>\n",
       "      <td>2.7240</td>\n",
       "      <td>2.1443</td>\n",
       "      <td>3.1405</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.7111</td>\n",
       "      <td>1.8361</td>\n",
       "      <td>1.7267</td>\n",
       "      <td>1.7334</td>\n",
       "      <td>1.7556</td>\n",
       "      <td>1.8314</td>\n",
       "      <td>1.7681</td>\n",
       "      <td>1.7800</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>523</th>\n",
       "      <td>2020-03-02</td>\n",
       "      <td>2.0221</td>\n",
       "      <td>2.1136</td>\n",
       "      <td>2.0221</td>\n",
       "      <td>2.5213</td>\n",
       "      <td>2.5658</td>\n",
       "      <td>1.9103</td>\n",
       "      <td>2.7314</td>\n",
       "      <td>2.1395</td>\n",
       "      <td>3.1599</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.6720</td>\n",
       "      <td>1.8412</td>\n",
       "      <td>1.6854</td>\n",
       "      <td>1.7498</td>\n",
       "      <td>1.7190</td>\n",
       "      <td>1.8299</td>\n",
       "      <td>1.7418</td>\n",
       "      <td>1.7638</td>\n",
       "      <td>1.7819</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>524</th>\n",
       "      <td>2020-03-03</td>\n",
       "      <td>2.0536</td>\n",
       "      <td>2.1320</td>\n",
       "      <td>2.0536</td>\n",
       "      <td>2.5319</td>\n",
       "      <td>2.5607</td>\n",
       "      <td>1.9187</td>\n",
       "      <td>2.7390</td>\n",
       "      <td>2.1540</td>\n",
       "      <td>3.1682</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.6581</td>\n",
       "      <td>1.8349</td>\n",
       "      <td>1.6691</td>\n",
       "      <td>1.7323</td>\n",
       "      <td>1.7003</td>\n",
       "      <td>1.8301</td>\n",
       "      <td>1.7318</td>\n",
       "      <td>1.7601</td>\n",
       "      <td>1.7799</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>525</th>\n",
       "      <td>2020-03-04</td>\n",
       "      <td>2.0549</td>\n",
       "      <td>2.1329</td>\n",
       "      <td>2.0549</td>\n",
       "      <td>2.4106</td>\n",
       "      <td>2.5134</td>\n",
       "      <td>1.8903</td>\n",
       "      <td>2.6732</td>\n",
       "      <td>2.1530</td>\n",
       "      <td>3.1490</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.6305</td>\n",
       "      <td>1.7935</td>\n",
       "      <td>1.6689</td>\n",
       "      <td>1.7446</td>\n",
       "      <td>1.7180</td>\n",
       "      <td>1.8071</td>\n",
       "      <td>1.7289</td>\n",
       "      <td>1.7393</td>\n",
       "      <td>1.7499</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>526 rows × 1699 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           Date  IB010007  IB010011  SH010107  SH010303  SH010504  SH010512  \\\n",
       "0    2018-01-02    4.1168    3.8014    4.1168    4.2580    3.9000    3.7726   \n",
       "1    2018-01-03    4.0911    3.7837    4.0911    4.2822    3.9052    3.7370   \n",
       "2    2018-01-04    4.1428    3.8092    4.1428    4.3109    3.9137    3.7391   \n",
       "3    2018-01-05    4.1080    3.7726    4.1080    4.2829    3.9103    3.7046   \n",
       "4    2018-01-08    4.0987    3.7356    4.0987    4.2629    3.9036    3.6663   \n",
       "..          ...       ...       ...       ...       ...       ...       ...   \n",
       "521  2020-02-27    2.0304    2.1411    2.0304    2.5027    2.6275    1.8926   \n",
       "522  2020-02-28    2.0280    2.1188    2.0280    2.5090    2.5566    1.9142   \n",
       "523  2020-03-02    2.0221    2.1136    2.0221    2.5213    2.5658    1.9103   \n",
       "524  2020-03-03    2.0536    2.1320    2.0536    2.5319    2.5607    1.9187   \n",
       "525  2020-03-04    2.0549    2.1329    2.0549    2.4106    2.5134    1.8903   \n",
       "\n",
       "     SH010609  SH010619  SH010706  ...  IB200003  IB209901  IB209902  \\\n",
       "0      3.9000    3.8026    4.2265  ...       NaN       NaN       NaN   \n",
       "1      3.9122    3.7873    4.2339  ...       NaN       NaN       NaN   \n",
       "2      3.9216    3.8136    4.2414  ...       NaN       NaN       NaN   \n",
       "3      3.9174    3.7765    4.2314  ...       NaN       NaN       NaN   \n",
       "4      3.9086    3.7403    4.2214  ...       NaN       NaN       NaN   \n",
       "..        ...       ...       ...  ...       ...       ...       ...   \n",
       "521    2.7841    2.1709    3.1601  ...       NaN    1.7054    1.8110   \n",
       "522    2.7240    2.1443    3.1405  ...       NaN    1.7111    1.8361   \n",
       "523    2.7314    2.1395    3.1599  ...       NaN    1.6720    1.8412   \n",
       "524    2.7390    2.1540    3.1682  ...       NaN    1.6581    1.8349   \n",
       "525    2.6732    2.1530    3.1490  ...       NaN    1.6305    1.7935   \n",
       "\n",
       "     IB209903  IB209904  IB209905  IB209906  IB209907  IB209908  IB209909  \n",
       "0         NaN       NaN       NaN       NaN       NaN       NaN       NaN  \n",
       "1         NaN       NaN       NaN       NaN       NaN       NaN       NaN  \n",
       "2         NaN       NaN       NaN       NaN       NaN       NaN       NaN  \n",
       "3         NaN       NaN       NaN       NaN       NaN       NaN       NaN  \n",
       "4         NaN       NaN       NaN       NaN       NaN       NaN       NaN  \n",
       "..        ...       ...       ...       ...       ...       ...       ...  \n",
       "521    1.7243    1.7327    1.7477    1.8384    1.7489    1.7499       NaN  \n",
       "522    1.7267    1.7334    1.7556    1.8314    1.7681    1.7800       NaN  \n",
       "523    1.6854    1.7498    1.7190    1.8299    1.7418    1.7638    1.7819  \n",
       "524    1.6691    1.7323    1.7003    1.8301    1.7318    1.7601    1.7799  \n",
       "525    1.6689    1.7446    1.7180    1.8071    1.7289    1.7393    1.7499  \n",
       "\n",
       "[526 rows x 1699 columns]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "IR = pd.read_csv(\"csv/bond_interest.csv\")\n",
    "IR"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "joined-binding",
   "metadata": {},
   "source": [
    "接下来，我们希望使用前29天的数据预测明天的收益率，所以我们要将数据整理为时间序列的格式，并产生滞后项："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "permanent-daily",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>SZ101510</th>\n",
       "      <th>SH019510</th>\n",
       "      <th>SZ101727</th>\n",
       "      <th>IB080002</th>\n",
       "      <th>SZ101614</th>\n",
       "      <th>BC170018</th>\n",
       "      <th>SZ101808</th>\n",
       "      <th>SH019112</th>\n",
       "      <th>BC140005</th>\n",
       "      <th>BC180009</th>\n",
       "      <th>...</th>\n",
       "      <th>IB080013</th>\n",
       "      <th>SH019580</th>\n",
       "      <th>SZ101034</th>\n",
       "      <th>SH019603</th>\n",
       "      <th>IB100007</th>\n",
       "      <th>IB160023</th>\n",
       "      <th>SZ101034</th>\n",
       "      <th>SH019406</th>\n",
       "      <th>SH019806</th>\n",
       "      <th>SZ101716</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>271</th>\n",
       "      <th>321</th>\n",
       "      <th>327</th>\n",
       "      <th>458</th>\n",
       "      <th>385</th>\n",
       "      <th>120</th>\n",
       "      <th>497</th>\n",
       "      <th>291</th>\n",
       "      <th>328</th>\n",
       "      <th>258</th>\n",
       "      <th>...</th>\n",
       "      <th>136</th>\n",
       "      <th>474</th>\n",
       "      <th>143</th>\n",
       "      <th>350</th>\n",
       "      <th>358</th>\n",
       "      <th>454</th>\n",
       "      <th>449</th>\n",
       "      <th>281</th>\n",
       "      <th>129</th>\n",
       "      <th>434</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>r_28</th>\n",
       "      <td>3.6907</td>\n",
       "      <td>3.7654</td>\n",
       "      <td>3.0504</td>\n",
       "      <td>2.8275</td>\n",
       "      <td>2.9866</td>\n",
       "      <td>3.6909</td>\n",
       "      <td>2.5050</td>\n",
       "      <td>3.7371</td>\n",
       "      <td>2.9545</td>\n",
       "      <td>2.9667</td>\n",
       "      <td>...</td>\n",
       "      <td>3.6241</td>\n",
       "      <td>3.2393</td>\n",
       "      <td>3.3610</td>\n",
       "      <td>2.9179</td>\n",
       "      <td>2.6573</td>\n",
       "      <td>3.1678</td>\n",
       "      <td>2.5716</td>\n",
       "      <td>2.5761</td>\n",
       "      <td>3.9707</td>\n",
       "      <td>2.3550</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_27</th>\n",
       "      <td>3.6957</td>\n",
       "      <td>3.7555</td>\n",
       "      <td>3.0499</td>\n",
       "      <td>2.8325</td>\n",
       "      <td>2.9928</td>\n",
       "      <td>3.6974</td>\n",
       "      <td>2.5093</td>\n",
       "      <td>3.7303</td>\n",
       "      <td>3.0079</td>\n",
       "      <td>3.0007</td>\n",
       "      <td>...</td>\n",
       "      <td>3.6043</td>\n",
       "      <td>3.2484</td>\n",
       "      <td>3.3611</td>\n",
       "      <td>2.9695</td>\n",
       "      <td>2.6417</td>\n",
       "      <td>3.1606</td>\n",
       "      <td>2.5556</td>\n",
       "      <td>2.5373</td>\n",
       "      <td>3.9769</td>\n",
       "      <td>2.3577</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_26</th>\n",
       "      <td>3.7158</td>\n",
       "      <td>3.7330</td>\n",
       "      <td>3.1128</td>\n",
       "      <td>2.8314</td>\n",
       "      <td>3.0114</td>\n",
       "      <td>3.6622</td>\n",
       "      <td>2.5350</td>\n",
       "      <td>3.7327</td>\n",
       "      <td>3.0323</td>\n",
       "      <td>3.0124</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5617</td>\n",
       "      <td>3.2627</td>\n",
       "      <td>3.3157</td>\n",
       "      <td>2.9353</td>\n",
       "      <td>2.6266</td>\n",
       "      <td>3.1682</td>\n",
       "      <td>2.5521</td>\n",
       "      <td>2.5470</td>\n",
       "      <td>3.9819</td>\n",
       "      <td>2.3616</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_25</th>\n",
       "      <td>3.7158</td>\n",
       "      <td>3.7140</td>\n",
       "      <td>3.1464</td>\n",
       "      <td>2.8194</td>\n",
       "      <td>2.9995</td>\n",
       "      <td>3.6582</td>\n",
       "      <td>2.5256</td>\n",
       "      <td>3.7185</td>\n",
       "      <td>3.0752</td>\n",
       "      <td>3.0238</td>\n",
       "      <td>...</td>\n",
       "      <td>3.6068</td>\n",
       "      <td>3.2646</td>\n",
       "      <td>3.2749</td>\n",
       "      <td>2.9401</td>\n",
       "      <td>2.6278</td>\n",
       "      <td>3.1839</td>\n",
       "      <td>2.5488</td>\n",
       "      <td>2.5957</td>\n",
       "      <td>3.9744</td>\n",
       "      <td>2.3326</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_24</th>\n",
       "      <td>3.6908</td>\n",
       "      <td>3.7116</td>\n",
       "      <td>3.1962</td>\n",
       "      <td>2.8224</td>\n",
       "      <td>2.9960</td>\n",
       "      <td>3.6499</td>\n",
       "      <td>2.5128</td>\n",
       "      <td>3.7185</td>\n",
       "      <td>3.1265</td>\n",
       "      <td>3.0230</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5767</td>\n",
       "      <td>3.2366</td>\n",
       "      <td>3.2486</td>\n",
       "      <td>2.9485</td>\n",
       "      <td>2.6736</td>\n",
       "      <td>3.1914</td>\n",
       "      <td>2.5707</td>\n",
       "      <td>2.5776</td>\n",
       "      <td>3.9745</td>\n",
       "      <td>2.3129</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_23</th>\n",
       "      <td>3.7158</td>\n",
       "      <td>3.7191</td>\n",
       "      <td>3.2219</td>\n",
       "      <td>2.8483</td>\n",
       "      <td>2.9544</td>\n",
       "      <td>3.6471</td>\n",
       "      <td>2.5112</td>\n",
       "      <td>3.7092</td>\n",
       "      <td>3.1558</td>\n",
       "      <td>3.0137</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5618</td>\n",
       "      <td>3.2400</td>\n",
       "      <td>3.2162</td>\n",
       "      <td>2.9324</td>\n",
       "      <td>2.6486</td>\n",
       "      <td>3.1936</td>\n",
       "      <td>2.5547</td>\n",
       "      <td>2.5898</td>\n",
       "      <td>3.9945</td>\n",
       "      <td>2.3360</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_22</th>\n",
       "      <td>3.7333</td>\n",
       "      <td>3.7141</td>\n",
       "      <td>3.2378</td>\n",
       "      <td>2.8564</td>\n",
       "      <td>2.9367</td>\n",
       "      <td>3.6596</td>\n",
       "      <td>2.4541</td>\n",
       "      <td>3.7190</td>\n",
       "      <td>3.2197</td>\n",
       "      <td>2.9852</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5816</td>\n",
       "      <td>3.2250</td>\n",
       "      <td>3.2015</td>\n",
       "      <td>2.9428</td>\n",
       "      <td>2.6566</td>\n",
       "      <td>3.1890</td>\n",
       "      <td>2.5691</td>\n",
       "      <td>2.5865</td>\n",
       "      <td>3.9782</td>\n",
       "      <td>2.3175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_21</th>\n",
       "      <td>3.7308</td>\n",
       "      <td>3.6944</td>\n",
       "      <td>3.2845</td>\n",
       "      <td>2.8554</td>\n",
       "      <td>2.9291</td>\n",
       "      <td>3.6687</td>\n",
       "      <td>2.4435</td>\n",
       "      <td>3.7314</td>\n",
       "      <td>3.2025</td>\n",
       "      <td>2.9879</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5817</td>\n",
       "      <td>3.2402</td>\n",
       "      <td>3.1957</td>\n",
       "      <td>2.9484</td>\n",
       "      <td>2.7121</td>\n",
       "      <td>3.1765</td>\n",
       "      <td>2.5699</td>\n",
       "      <td>2.5822</td>\n",
       "      <td>3.9558</td>\n",
       "      <td>2.2974</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_20</th>\n",
       "      <td>3.7408</td>\n",
       "      <td>3.7392</td>\n",
       "      <td>3.2747</td>\n",
       "      <td>2.8448</td>\n",
       "      <td>2.9585</td>\n",
       "      <td>3.6994</td>\n",
       "      <td>2.4419</td>\n",
       "      <td>3.7481</td>\n",
       "      <td>3.1727</td>\n",
       "      <td>3.0096</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5542</td>\n",
       "      <td>3.2284</td>\n",
       "      <td>3.1836</td>\n",
       "      <td>2.9367</td>\n",
       "      <td>2.6452</td>\n",
       "      <td>3.1994</td>\n",
       "      <td>2.5597</td>\n",
       "      <td>2.6620</td>\n",
       "      <td>3.9407</td>\n",
       "      <td>2.2834</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_19</th>\n",
       "      <td>3.7507</td>\n",
       "      <td>3.7487</td>\n",
       "      <td>3.2472</td>\n",
       "      <td>2.8495</td>\n",
       "      <td>2.9677</td>\n",
       "      <td>3.6992</td>\n",
       "      <td>2.4130</td>\n",
       "      <td>3.7382</td>\n",
       "      <td>3.1706</td>\n",
       "      <td>3.0080</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5316</td>\n",
       "      <td>3.1886</td>\n",
       "      <td>3.2063</td>\n",
       "      <td>2.9421</td>\n",
       "      <td>2.6356</td>\n",
       "      <td>3.2248</td>\n",
       "      <td>2.5309</td>\n",
       "      <td>2.6477</td>\n",
       "      <td>3.9057</td>\n",
       "      <td>2.2815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_18</th>\n",
       "      <td>3.6882</td>\n",
       "      <td>3.8055</td>\n",
       "      <td>3.2564</td>\n",
       "      <td>2.8827</td>\n",
       "      <td>2.9650</td>\n",
       "      <td>3.6991</td>\n",
       "      <td>2.3795</td>\n",
       "      <td>3.7431</td>\n",
       "      <td>3.2325</td>\n",
       "      <td>3.0080</td>\n",
       "      <td>...</td>\n",
       "      <td>3.4768</td>\n",
       "      <td>3.1636</td>\n",
       "      <td>3.2037</td>\n",
       "      <td>2.9429</td>\n",
       "      <td>2.6085</td>\n",
       "      <td>3.2498</td>\n",
       "      <td>2.5551</td>\n",
       "      <td>2.6236</td>\n",
       "      <td>3.9356</td>\n",
       "      <td>2.3000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_17</th>\n",
       "      <td>3.7007</td>\n",
       "      <td>3.8131</td>\n",
       "      <td>3.3186</td>\n",
       "      <td>2.8793</td>\n",
       "      <td>2.9762</td>\n",
       "      <td>3.7086</td>\n",
       "      <td>2.3397</td>\n",
       "      <td>3.7455</td>\n",
       "      <td>3.2487</td>\n",
       "      <td>2.9672</td>\n",
       "      <td>...</td>\n",
       "      <td>3.4679</td>\n",
       "      <td>3.1626</td>\n",
       "      <td>3.1819</td>\n",
       "      <td>2.9446</td>\n",
       "      <td>2.5940</td>\n",
       "      <td>3.2446</td>\n",
       "      <td>2.5637</td>\n",
       "      <td>2.6132</td>\n",
       "      <td>3.9182</td>\n",
       "      <td>2.2889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_16</th>\n",
       "      <td>3.7208</td>\n",
       "      <td>3.8105</td>\n",
       "      <td>3.3354</td>\n",
       "      <td>2.9530</td>\n",
       "      <td>2.9585</td>\n",
       "      <td>3.7038</td>\n",
       "      <td>2.3315</td>\n",
       "      <td>3.7216</td>\n",
       "      <td>3.2489</td>\n",
       "      <td>2.9308</td>\n",
       "      <td>...</td>\n",
       "      <td>3.4966</td>\n",
       "      <td>3.1616</td>\n",
       "      <td>3.1748</td>\n",
       "      <td>2.9393</td>\n",
       "      <td>2.6078</td>\n",
       "      <td>3.2474</td>\n",
       "      <td>2.5565</td>\n",
       "      <td>2.6014</td>\n",
       "      <td>3.9083</td>\n",
       "      <td>2.3192</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_15</th>\n",
       "      <td>3.7208</td>\n",
       "      <td>3.8455</td>\n",
       "      <td>3.3362</td>\n",
       "      <td>2.9452</td>\n",
       "      <td>2.9456</td>\n",
       "      <td>3.7100</td>\n",
       "      <td>2.2044</td>\n",
       "      <td>3.7639</td>\n",
       "      <td>3.1947</td>\n",
       "      <td>2.8955</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5119</td>\n",
       "      <td>3.1622</td>\n",
       "      <td>3.1797</td>\n",
       "      <td>2.9126</td>\n",
       "      <td>2.5823</td>\n",
       "      <td>3.2609</td>\n",
       "      <td>2.5712</td>\n",
       "      <td>2.5738</td>\n",
       "      <td>3.9182</td>\n",
       "      <td>2.3404</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_14</th>\n",
       "      <td>3.7257</td>\n",
       "      <td>3.8455</td>\n",
       "      <td>3.2880</td>\n",
       "      <td>2.9136</td>\n",
       "      <td>2.9709</td>\n",
       "      <td>3.7435</td>\n",
       "      <td>2.1890</td>\n",
       "      <td>3.7789</td>\n",
       "      <td>3.1938</td>\n",
       "      <td>2.8545</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5015</td>\n",
       "      <td>3.1611</td>\n",
       "      <td>3.1468</td>\n",
       "      <td>2.8987</td>\n",
       "      <td>2.5715</td>\n",
       "      <td>3.3111</td>\n",
       "      <td>2.5886</td>\n",
       "      <td>2.5681</td>\n",
       "      <td>3.9133</td>\n",
       "      <td>2.3315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_13</th>\n",
       "      <td>3.7282</td>\n",
       "      <td>3.8555</td>\n",
       "      <td>3.2942</td>\n",
       "      <td>2.9437</td>\n",
       "      <td>2.9689</td>\n",
       "      <td>3.7225</td>\n",
       "      <td>2.0666</td>\n",
       "      <td>3.7589</td>\n",
       "      <td>3.2140</td>\n",
       "      <td>2.8540</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5272</td>\n",
       "      <td>3.1436</td>\n",
       "      <td>3.1428</td>\n",
       "      <td>2.9310</td>\n",
       "      <td>2.5643</td>\n",
       "      <td>3.3130</td>\n",
       "      <td>2.5803</td>\n",
       "      <td>2.5596</td>\n",
       "      <td>3.9033</td>\n",
       "      <td>2.3314</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_12</th>\n",
       "      <td>3.7512</td>\n",
       "      <td>3.8855</td>\n",
       "      <td>3.3236</td>\n",
       "      <td>2.9163</td>\n",
       "      <td>2.9625</td>\n",
       "      <td>3.6756</td>\n",
       "      <td>2.0127</td>\n",
       "      <td>3.7689</td>\n",
       "      <td>3.2336</td>\n",
       "      <td>2.8818</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5442</td>\n",
       "      <td>3.1455</td>\n",
       "      <td>3.1569</td>\n",
       "      <td>2.9030</td>\n",
       "      <td>2.5663</td>\n",
       "      <td>3.3295</td>\n",
       "      <td>2.6276</td>\n",
       "      <td>2.5670</td>\n",
       "      <td>3.8747</td>\n",
       "      <td>2.3174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_11</th>\n",
       "      <td>3.7637</td>\n",
       "      <td>3.9554</td>\n",
       "      <td>3.3378</td>\n",
       "      <td>2.9040</td>\n",
       "      <td>2.9495</td>\n",
       "      <td>3.6531</td>\n",
       "      <td>2.0252</td>\n",
       "      <td>3.7692</td>\n",
       "      <td>3.2581</td>\n",
       "      <td>2.8353</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5440</td>\n",
       "      <td>3.1392</td>\n",
       "      <td>3.1417</td>\n",
       "      <td>2.8726</td>\n",
       "      <td>2.5783</td>\n",
       "      <td>3.2980</td>\n",
       "      <td>2.6405</td>\n",
       "      <td>2.5665</td>\n",
       "      <td>3.8123</td>\n",
       "      <td>2.3108</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_10</th>\n",
       "      <td>3.7763</td>\n",
       "      <td>3.9767</td>\n",
       "      <td>3.3630</td>\n",
       "      <td>2.9232</td>\n",
       "      <td>2.9504</td>\n",
       "      <td>3.6022</td>\n",
       "      <td>2.0231</td>\n",
       "      <td>3.7780</td>\n",
       "      <td>3.2337</td>\n",
       "      <td>2.8339</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5117</td>\n",
       "      <td>3.1393</td>\n",
       "      <td>3.1663</td>\n",
       "      <td>2.8510</td>\n",
       "      <td>2.5816</td>\n",
       "      <td>3.2539</td>\n",
       "      <td>2.6213</td>\n",
       "      <td>2.5690</td>\n",
       "      <td>3.8054</td>\n",
       "      <td>2.3056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_9</th>\n",
       "      <td>3.7450</td>\n",
       "      <td>3.9767</td>\n",
       "      <td>3.3409</td>\n",
       "      <td>2.9281</td>\n",
       "      <td>2.9404</td>\n",
       "      <td>3.6442</td>\n",
       "      <td>2.0048</td>\n",
       "      <td>3.7839</td>\n",
       "      <td>3.2045</td>\n",
       "      <td>2.8523</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5168</td>\n",
       "      <td>3.1703</td>\n",
       "      <td>3.1582</td>\n",
       "      <td>2.8626</td>\n",
       "      <td>2.6185</td>\n",
       "      <td>3.2804</td>\n",
       "      <td>2.6318</td>\n",
       "      <td>2.5698</td>\n",
       "      <td>3.8197</td>\n",
       "      <td>2.3225</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_8</th>\n",
       "      <td>3.7411</td>\n",
       "      <td>3.9566</td>\n",
       "      <td>3.2999</td>\n",
       "      <td>2.9207</td>\n",
       "      <td>2.9337</td>\n",
       "      <td>3.6149</td>\n",
       "      <td>2.0536</td>\n",
       "      <td>3.7843</td>\n",
       "      <td>3.2095</td>\n",
       "      <td>2.8677</td>\n",
       "      <td>...</td>\n",
       "      <td>3.4867</td>\n",
       "      <td>3.1701</td>\n",
       "      <td>3.1526</td>\n",
       "      <td>2.8434</td>\n",
       "      <td>2.6420</td>\n",
       "      <td>3.2368</td>\n",
       "      <td>2.6366</td>\n",
       "      <td>2.5159</td>\n",
       "      <td>3.8336</td>\n",
       "      <td>2.3184</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_7</th>\n",
       "      <td>3.7344</td>\n",
       "      <td>3.9909</td>\n",
       "      <td>3.3226</td>\n",
       "      <td>2.9305</td>\n",
       "      <td>2.9525</td>\n",
       "      <td>3.6019</td>\n",
       "      <td>2.0574</td>\n",
       "      <td>3.7642</td>\n",
       "      <td>3.1852</td>\n",
       "      <td>2.8927</td>\n",
       "      <td>...</td>\n",
       "      <td>3.4816</td>\n",
       "      <td>3.1680</td>\n",
       "      <td>3.1182</td>\n",
       "      <td>2.8385</td>\n",
       "      <td>2.6277</td>\n",
       "      <td>3.2436</td>\n",
       "      <td>2.6637</td>\n",
       "      <td>2.5540</td>\n",
       "      <td>3.8199</td>\n",
       "      <td>2.3214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_6</th>\n",
       "      <td>3.7367</td>\n",
       "      <td>4.0080</td>\n",
       "      <td>3.3012</td>\n",
       "      <td>2.9187</td>\n",
       "      <td>2.9461</td>\n",
       "      <td>3.6198</td>\n",
       "      <td>2.0223</td>\n",
       "      <td>3.7491</td>\n",
       "      <td>3.1350</td>\n",
       "      <td>2.8909</td>\n",
       "      <td>...</td>\n",
       "      <td>3.4816</td>\n",
       "      <td>3.1705</td>\n",
       "      <td>3.0925</td>\n",
       "      <td>2.8458</td>\n",
       "      <td>2.6332</td>\n",
       "      <td>3.2483</td>\n",
       "      <td>2.6638</td>\n",
       "      <td>2.5452</td>\n",
       "      <td>3.8474</td>\n",
       "      <td>2.3209</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_5</th>\n",
       "      <td>3.7225</td>\n",
       "      <td>4.0080</td>\n",
       "      <td>3.2565</td>\n",
       "      <td>2.9340</td>\n",
       "      <td>2.9506</td>\n",
       "      <td>3.6096</td>\n",
       "      <td>2.0332</td>\n",
       "      <td>3.7441</td>\n",
       "      <td>3.1541</td>\n",
       "      <td>2.8458</td>\n",
       "      <td>...</td>\n",
       "      <td>3.4716</td>\n",
       "      <td>3.1703</td>\n",
       "      <td>3.0231</td>\n",
       "      <td>2.8407</td>\n",
       "      <td>2.6284</td>\n",
       "      <td>3.2506</td>\n",
       "      <td>2.6568</td>\n",
       "      <td>2.5764</td>\n",
       "      <td>3.8611</td>\n",
       "      <td>2.3118</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_4</th>\n",
       "      <td>3.7225</td>\n",
       "      <td>4.0380</td>\n",
       "      <td>3.2662</td>\n",
       "      <td>2.9149</td>\n",
       "      <td>2.9623</td>\n",
       "      <td>3.5937</td>\n",
       "      <td>2.0259</td>\n",
       "      <td>3.7450</td>\n",
       "      <td>3.1434</td>\n",
       "      <td>2.8202</td>\n",
       "      <td>...</td>\n",
       "      <td>3.4416</td>\n",
       "      <td>3.1500</td>\n",
       "      <td>2.9935</td>\n",
       "      <td>2.8508</td>\n",
       "      <td>2.5938</td>\n",
       "      <td>3.2271</td>\n",
       "      <td>2.6629</td>\n",
       "      <td>2.5928</td>\n",
       "      <td>3.8611</td>\n",
       "      <td>2.2768</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_3</th>\n",
       "      <td>3.7132</td>\n",
       "      <td>4.0180</td>\n",
       "      <td>3.2566</td>\n",
       "      <td>2.8888</td>\n",
       "      <td>2.9598</td>\n",
       "      <td>3.5709</td>\n",
       "      <td>2.0082</td>\n",
       "      <td>3.7450</td>\n",
       "      <td>3.1303</td>\n",
       "      <td>2.8543</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5141</td>\n",
       "      <td>3.1439</td>\n",
       "      <td>2.9588</td>\n",
       "      <td>2.8761</td>\n",
       "      <td>2.5651</td>\n",
       "      <td>3.2191</td>\n",
       "      <td>2.6628</td>\n",
       "      <td>2.5966</td>\n",
       "      <td>3.8461</td>\n",
       "      <td>2.2875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_2</th>\n",
       "      <td>3.7230</td>\n",
       "      <td>4.0055</td>\n",
       "      <td>3.2398</td>\n",
       "      <td>2.8747</td>\n",
       "      <td>2.9422</td>\n",
       "      <td>3.5169</td>\n",
       "      <td>2.0300</td>\n",
       "      <td>3.7550</td>\n",
       "      <td>3.1401</td>\n",
       "      <td>2.8639</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5192</td>\n",
       "      <td>3.1355</td>\n",
       "      <td>2.9494</td>\n",
       "      <td>2.9399</td>\n",
       "      <td>2.5634</td>\n",
       "      <td>3.2122</td>\n",
       "      <td>2.6622</td>\n",
       "      <td>2.6206</td>\n",
       "      <td>3.8486</td>\n",
       "      <td>2.3034</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r_1</th>\n",
       "      <td>3.7383</td>\n",
       "      <td>4.0095</td>\n",
       "      <td>3.2444</td>\n",
       "      <td>2.8686</td>\n",
       "      <td>2.9326</td>\n",
       "      <td>3.5287</td>\n",
       "      <td>2.0641</td>\n",
       "      <td>3.7504</td>\n",
       "      <td>3.1178</td>\n",
       "      <td>2.8612</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5622</td>\n",
       "      <td>3.1296</td>\n",
       "      <td>2.9284</td>\n",
       "      <td>2.9370</td>\n",
       "      <td>2.5579</td>\n",
       "      <td>3.2232</td>\n",
       "      <td>2.6559</td>\n",
       "      <td>2.6203</td>\n",
       "      <td>3.8237</td>\n",
       "      <td>2.3010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>r</th>\n",
       "      <td>3.7549</td>\n",
       "      <td>3.9945</td>\n",
       "      <td>3.2185</td>\n",
       "      <td>2.8448</td>\n",
       "      <td>2.8953</td>\n",
       "      <td>3.5549</td>\n",
       "      <td>2.1525</td>\n",
       "      <td>3.7504</td>\n",
       "      <td>3.1461</td>\n",
       "      <td>2.8538</td>\n",
       "      <td>...</td>\n",
       "      <td>3.5620</td>\n",
       "      <td>3.1343</td>\n",
       "      <td>2.8936</td>\n",
       "      <td>2.9197</td>\n",
       "      <td>2.5575</td>\n",
       "      <td>3.2133</td>\n",
       "      <td>2.6571</td>\n",
       "      <td>2.5939</td>\n",
       "      <td>3.8162</td>\n",
       "      <td>2.3097</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>29 rows × 278003 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     SZ101510 SH019510 SZ101727 IB080002 SZ101614 BC170018 SZ101808 SH019112  \\\n",
       "          271      321      327      458      385      120      497      291   \n",
       "r_28   3.6907   3.7654   3.0504   2.8275   2.9866   3.6909   2.5050   3.7371   \n",
       "r_27   3.6957   3.7555   3.0499   2.8325   2.9928   3.6974   2.5093   3.7303   \n",
       "r_26   3.7158   3.7330   3.1128   2.8314   3.0114   3.6622   2.5350   3.7327   \n",
       "r_25   3.7158   3.7140   3.1464   2.8194   2.9995   3.6582   2.5256   3.7185   \n",
       "r_24   3.6908   3.7116   3.1962   2.8224   2.9960   3.6499   2.5128   3.7185   \n",
       "r_23   3.7158   3.7191   3.2219   2.8483   2.9544   3.6471   2.5112   3.7092   \n",
       "r_22   3.7333   3.7141   3.2378   2.8564   2.9367   3.6596   2.4541   3.7190   \n",
       "r_21   3.7308   3.6944   3.2845   2.8554   2.9291   3.6687   2.4435   3.7314   \n",
       "r_20   3.7408   3.7392   3.2747   2.8448   2.9585   3.6994   2.4419   3.7481   \n",
       "r_19   3.7507   3.7487   3.2472   2.8495   2.9677   3.6992   2.4130   3.7382   \n",
       "r_18   3.6882   3.8055   3.2564   2.8827   2.9650   3.6991   2.3795   3.7431   \n",
       "r_17   3.7007   3.8131   3.3186   2.8793   2.9762   3.7086   2.3397   3.7455   \n",
       "r_16   3.7208   3.8105   3.3354   2.9530   2.9585   3.7038   2.3315   3.7216   \n",
       "r_15   3.7208   3.8455   3.3362   2.9452   2.9456   3.7100   2.2044   3.7639   \n",
       "r_14   3.7257   3.8455   3.2880   2.9136   2.9709   3.7435   2.1890   3.7789   \n",
       "r_13   3.7282   3.8555   3.2942   2.9437   2.9689   3.7225   2.0666   3.7589   \n",
       "r_12   3.7512   3.8855   3.3236   2.9163   2.9625   3.6756   2.0127   3.7689   \n",
       "r_11   3.7637   3.9554   3.3378   2.9040   2.9495   3.6531   2.0252   3.7692   \n",
       "r_10   3.7763   3.9767   3.3630   2.9232   2.9504   3.6022   2.0231   3.7780   \n",
       "r_9    3.7450   3.9767   3.3409   2.9281   2.9404   3.6442   2.0048   3.7839   \n",
       "r_8    3.7411   3.9566   3.2999   2.9207   2.9337   3.6149   2.0536   3.7843   \n",
       "r_7    3.7344   3.9909   3.3226   2.9305   2.9525   3.6019   2.0574   3.7642   \n",
       "r_6    3.7367   4.0080   3.3012   2.9187   2.9461   3.6198   2.0223   3.7491   \n",
       "r_5    3.7225   4.0080   3.2565   2.9340   2.9506   3.6096   2.0332   3.7441   \n",
       "r_4    3.7225   4.0380   3.2662   2.9149   2.9623   3.5937   2.0259   3.7450   \n",
       "r_3    3.7132   4.0180   3.2566   2.8888   2.9598   3.5709   2.0082   3.7450   \n",
       "r_2    3.7230   4.0055   3.2398   2.8747   2.9422   3.5169   2.0300   3.7550   \n",
       "r_1    3.7383   4.0095   3.2444   2.8686   2.9326   3.5287   2.0641   3.7504   \n",
       "r      3.7549   3.9945   3.2185   2.8448   2.8953   3.5549   2.1525   3.7504   \n",
       "\n",
       "     BC140005 BC180009  ... IB080013 SH019580 SZ101034 SH019603 IB100007  \\\n",
       "          328      258  ...      136      474      143      350      358   \n",
       "r_28   2.9545   2.9667  ...   3.6241   3.2393   3.3610   2.9179   2.6573   \n",
       "r_27   3.0079   3.0007  ...   3.6043   3.2484   3.3611   2.9695   2.6417   \n",
       "r_26   3.0323   3.0124  ...   3.5617   3.2627   3.3157   2.9353   2.6266   \n",
       "r_25   3.0752   3.0238  ...   3.6068   3.2646   3.2749   2.9401   2.6278   \n",
       "r_24   3.1265   3.0230  ...   3.5767   3.2366   3.2486   2.9485   2.6736   \n",
       "r_23   3.1558   3.0137  ...   3.5618   3.2400   3.2162   2.9324   2.6486   \n",
       "r_22   3.2197   2.9852  ...   3.5816   3.2250   3.2015   2.9428   2.6566   \n",
       "r_21   3.2025   2.9879  ...   3.5817   3.2402   3.1957   2.9484   2.7121   \n",
       "r_20   3.1727   3.0096  ...   3.5542   3.2284   3.1836   2.9367   2.6452   \n",
       "r_19   3.1706   3.0080  ...   3.5316   3.1886   3.2063   2.9421   2.6356   \n",
       "r_18   3.2325   3.0080  ...   3.4768   3.1636   3.2037   2.9429   2.6085   \n",
       "r_17   3.2487   2.9672  ...   3.4679   3.1626   3.1819   2.9446   2.5940   \n",
       "r_16   3.2489   2.9308  ...   3.4966   3.1616   3.1748   2.9393   2.6078   \n",
       "r_15   3.1947   2.8955  ...   3.5119   3.1622   3.1797   2.9126   2.5823   \n",
       "r_14   3.1938   2.8545  ...   3.5015   3.1611   3.1468   2.8987   2.5715   \n",
       "r_13   3.2140   2.8540  ...   3.5272   3.1436   3.1428   2.9310   2.5643   \n",
       "r_12   3.2336   2.8818  ...   3.5442   3.1455   3.1569   2.9030   2.5663   \n",
       "r_11   3.2581   2.8353  ...   3.5440   3.1392   3.1417   2.8726   2.5783   \n",
       "r_10   3.2337   2.8339  ...   3.5117   3.1393   3.1663   2.8510   2.5816   \n",
       "r_9    3.2045   2.8523  ...   3.5168   3.1703   3.1582   2.8626   2.6185   \n",
       "r_8    3.2095   2.8677  ...   3.4867   3.1701   3.1526   2.8434   2.6420   \n",
       "r_7    3.1852   2.8927  ...   3.4816   3.1680   3.1182   2.8385   2.6277   \n",
       "r_6    3.1350   2.8909  ...   3.4816   3.1705   3.0925   2.8458   2.6332   \n",
       "r_5    3.1541   2.8458  ...   3.4716   3.1703   3.0231   2.8407   2.6284   \n",
       "r_4    3.1434   2.8202  ...   3.4416   3.1500   2.9935   2.8508   2.5938   \n",
       "r_3    3.1303   2.8543  ...   3.5141   3.1439   2.9588   2.8761   2.5651   \n",
       "r_2    3.1401   2.8639  ...   3.5192   3.1355   2.9494   2.9399   2.5634   \n",
       "r_1    3.1178   2.8612  ...   3.5622   3.1296   2.9284   2.9370   2.5579   \n",
       "r      3.1461   2.8538  ...   3.5620   3.1343   2.8936   2.9197   2.5575   \n",
       "\n",
       "     IB160023 SZ101034 SH019406 SH019806 SZ101716  \n",
       "          454      449      281      129      434  \n",
       "r_28   3.1678   2.5716   2.5761   3.9707   2.3550  \n",
       "r_27   3.1606   2.5556   2.5373   3.9769   2.3577  \n",
       "r_26   3.1682   2.5521   2.5470   3.9819   2.3616  \n",
       "r_25   3.1839   2.5488   2.5957   3.9744   2.3326  \n",
       "r_24   3.1914   2.5707   2.5776   3.9745   2.3129  \n",
       "r_23   3.1936   2.5547   2.5898   3.9945   2.3360  \n",
       "r_22   3.1890   2.5691   2.5865   3.9782   2.3175  \n",
       "r_21   3.1765   2.5699   2.5822   3.9558   2.2974  \n",
       "r_20   3.1994   2.5597   2.6620   3.9407   2.2834  \n",
       "r_19   3.2248   2.5309   2.6477   3.9057   2.2815  \n",
       "r_18   3.2498   2.5551   2.6236   3.9356   2.3000  \n",
       "r_17   3.2446   2.5637   2.6132   3.9182   2.2889  \n",
       "r_16   3.2474   2.5565   2.6014   3.9083   2.3192  \n",
       "r_15   3.2609   2.5712   2.5738   3.9182   2.3404  \n",
       "r_14   3.3111   2.5886   2.5681   3.9133   2.3315  \n",
       "r_13   3.3130   2.5803   2.5596   3.9033   2.3314  \n",
       "r_12   3.3295   2.6276   2.5670   3.8747   2.3174  \n",
       "r_11   3.2980   2.6405   2.5665   3.8123   2.3108  \n",
       "r_10   3.2539   2.6213   2.5690   3.8054   2.3056  \n",
       "r_9    3.2804   2.6318   2.5698   3.8197   2.3225  \n",
       "r_8    3.2368   2.6366   2.5159   3.8336   2.3184  \n",
       "r_7    3.2436   2.6637   2.5540   3.8199   2.3214  \n",
       "r_6    3.2483   2.6638   2.5452   3.8474   2.3209  \n",
       "r_5    3.2506   2.6568   2.5764   3.8611   2.3118  \n",
       "r_4    3.2271   2.6629   2.5928   3.8611   2.2768  \n",
       "r_3    3.2191   2.6628   2.5966   3.8461   2.2875  \n",
       "r_2    3.2122   2.6622   2.6206   3.8486   2.3034  \n",
       "r_1    3.2232   2.6559   2.6203   3.8237   2.3010  \n",
       "r      3.2133   2.6571   2.5939   3.8162   2.3097  \n",
       "\n",
       "[29 rows x 278003 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "IR_long = IR.iloc[:, 1:]\n",
    "IR_long = IR_long.T.stack()\n",
    "IR_long = pd.DataFrame({'r': IR_long})\n",
    "IR_long['label'] = IR_long['r'].groupby(level=0).shift(-1)\n",
    "lag_length = 29\n",
    "for i in range(lag_length - 1):\n",
    "    IR_long['r_' + str(i + 1)] = IR_long['r'].groupby(level=0).shift(i + 1)\n",
    "IR_long = IR_long.dropna()\n",
    "IR_long['random'] = np.random.random(IR_long.shape[0])\n",
    "IR_long = IR_long.sort_values('random')\n",
    "del IR_long['random']\n",
    "Y = IR_long['label']\n",
    "X = IR_long.drop(['label'], axis=1)\n",
    "X = X.T\n",
    "X['n'] = np.arange(X.shape[0])\n",
    "X = X.sort_values('n', ascending=False)\n",
    "del X['n']\n",
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "available-insight",
   "metadata": {},
   "source": [
    "然后定义数据集和数据加载器："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "floating-vitamin",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch.utils.data import Dataset, DataLoader\n",
    "\n",
    "Y_train = Y.iloc[:270000]\n",
    "X_train = X.iloc[:, :270000]\n",
    "Y_test = Y.iloc[270000:]\n",
    "X_test = X.iloc[:, 270000:]\n",
    "\n",
    "\n",
    "class IR_data(Dataset):\n",
    "\n",
    "    def __init__(self, X, Y):\n",
    "        self.X = X\n",
    "        self.Y = Y\n",
    "\n",
    "    def __len__(self):\n",
    "        return self.X.shape[1]\n",
    "\n",
    "    def __getitem__(self, i):\n",
    "        x = torch.tensor(self.X.iloc[:, i])\n",
    "        label = torch.tensor(self.Y.iloc[i])\n",
    "        return x, label\n",
    "\n",
    "\n",
    "batch_size = 500\n",
    "\n",
    "train_data = IR_data(X_train, Y_train)\n",
    "dl = DataLoader(train_data,\n",
    "                shuffle=True,\n",
    "                batch_size=batch_size,\n",
    "                pin_memory=True,\n",
    "                num_workers=10)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "laughing-symposium",
   "metadata": {},
   "source": [
    "接下来定义RNN模型。我们可以使用nn.RNN来创建RNN层，创建该层时需要的参数有：\n",
    "\n",
    "* input_size：输入的大小，也就是特征数量，在上面的例子中，我们并没有使用其他的时间序列帮助预测，仅仅是收益率的滞后，那么input_size=1；如果我们加入了其他时间序列共同预测，比如加入了SHIBOR等，那么input_size=2\n",
    "* hidden_size：隐藏状态的数量\n",
    "* num_layers：嵌套的RNN的层数，默认为1，即只有一个RNN，可以多个RNN套在一起使用（多个RNN串联起来）\n",
    "* nonlinearity： relu或者tanh，用于拟合$f$函数的激活函数\n",
    "* dropout：用于拟合$f$函数的网络的dropout概率，默认为0，即不做dropout\n",
    "\n",
    "经过初始化之后，返回一个计算RNN的函数（比如起名为rnn），该函数需要的输入为：\n",
    "\n",
    "* x：可观测的$x$，注意形状应该为（序列长度$T$×批大小$B$×input_size）\n",
    "* h_0：隐藏状态矩阵，形状应该为（层数num_layers×批大小×hidden_size），一般初始化为0\n",
    "\n",
    "所以在下面的定义中，我们还构造了一个初始化h矩阵的函数\n",
    "\n",
    "该函数得到的输出也有两个：\n",
    "\n",
    "* output：输出的状态，形状为（序列长度$T$×批大小$B$×hidden_size），即返回了每个时间$t$的隐藏状态\n",
    "* h_n：$t=T$时的隐藏状态，形状为（层数num_layers×批大小$B$×hidden_size），即最终时刻不同层的RNN的隐藏状态\n",
    "\n",
    "比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "strategic-kelly",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "初始的隐藏状态：\n",
      "tensor([[[0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.]]])\n",
      "数据：\n",
      "tensor([[[0.7329],\n",
      "         [0.0712],\n",
      "         [0.2676],\n",
      "         [0.6008],\n",
      "         [0.5957],\n",
      "         [0.0201],\n",
      "         [0.1616],\n",
      "         [0.6980],\n",
      "         [0.5511],\n",
      "         [0.3454]],\n",
      "\n",
      "        [[0.2540],\n",
      "         [0.6135],\n",
      "         [0.2185],\n",
      "         [0.7870],\n",
      "         [0.5979],\n",
      "         [0.1410],\n",
      "         [0.1753],\n",
      "         [0.9912],\n",
      "         [0.2242],\n",
      "         [0.8234]],\n",
      "\n",
      "        [[0.9572],\n",
      "         [0.4853],\n",
      "         [0.3195],\n",
      "         [0.1507],\n",
      "         [0.1896],\n",
      "         [0.2773],\n",
      "         [0.2488],\n",
      "         [0.3728],\n",
      "         [0.3903],\n",
      "         [0.5662]],\n",
      "\n",
      "        [[0.6967],\n",
      "         [0.3983],\n",
      "         [0.5251],\n",
      "         [0.5580],\n",
      "         [0.4968],\n",
      "         [0.0174],\n",
      "         [0.6082],\n",
      "         [0.2834],\n",
      "         [0.0396],\n",
      "         [0.8965]],\n",
      "\n",
      "        [[0.2374],\n",
      "         [0.2211],\n",
      "         [0.8555],\n",
      "         [0.3312],\n",
      "         [0.7122],\n",
      "         [0.4921],\n",
      "         [0.1051],\n",
      "         [0.6706],\n",
      "         [0.4136],\n",
      "         [0.7118]]])\n",
      " 形状：torch.Size([5, 10, 1])\n",
      "output=\n",
      "tensor([[[-0.5215, -0.1266],\n",
      "         [-0.5203,  0.1395],\n",
      "         [-0.5207,  0.0609],\n",
      "         [-0.5212, -0.0737],\n",
      "         [-0.5212, -0.0717],\n",
      "         [-0.5202,  0.1597],\n",
      "         [-0.5205,  0.1035],\n",
      "         [-0.5214, -0.1127],\n",
      "         [-0.5212, -0.0537],\n",
      "         [-0.5208,  0.0295]],\n",
      "\n",
      "        [[-0.6389,  0.3607],\n",
      "         [-0.5509,  0.0906],\n",
      "         [-0.5779,  0.2843],\n",
      "         [-0.6232,  0.1332],\n",
      "         [-0.6223,  0.2064],\n",
      "         [-0.5428,  0.2647],\n",
      "         [-0.5630,  0.2796],\n",
      "         [-0.6356,  0.0720],\n",
      "         [-0.6160,  0.3373],\n",
      "         [-0.5896,  0.0643]],\n",
      "\n",
      "        [[-0.4871, -0.1094],\n",
      "         [-0.5725,  0.1809],\n",
      "         [-0.5056,  0.1586],\n",
      "         [-0.5678,  0.3181],\n",
      "         [-0.5416,  0.2679],\n",
      "         [-0.5074,  0.1693],\n",
      "         [-0.5049,  0.1821],\n",
      "         [-0.5909,  0.2710],\n",
      "         [-0.4915,  0.1206],\n",
      "         [-0.5872,  0.1802]],\n",
      "\n",
      "        [[-0.6297,  0.1717],\n",
      "         [-0.5436,  0.1784],\n",
      "         [-0.5416,  0.1091],\n",
      "         [-0.4913,  0.0409],\n",
      "         [-0.5064,  0.0798],\n",
      "         [-0.5371,  0.3005],\n",
      "         [-0.5330,  0.0631],\n",
      "         [-0.5128,  0.1856],\n",
      "         [-0.5524,  0.3091],\n",
      "         [-0.5469, -0.0139]],\n",
      "\n",
      "        [[-0.5553,  0.2702],\n",
      "         [-0.5397,  0.2353],\n",
      "         [-0.5653,  0.0189],\n",
      "         [-0.5807,  0.2390],\n",
      "         [-0.5701,  0.0758],\n",
      "         [-0.4930,  0.0624],\n",
      "         [-0.5788,  0.3300],\n",
      "         [-0.5331,  0.0398],\n",
      "         [-0.4920,  0.0966],\n",
      "         [-0.6074,  0.1437]]], grad_fn=<StackBackward0>)\n",
      "h_n=\n",
      "tensor([[[-0.5553,  0.2702],\n",
      "         [-0.5397,  0.2353],\n",
      "         [-0.5653,  0.0189],\n",
      "         [-0.5807,  0.2390],\n",
      "         [-0.5701,  0.0758],\n",
      "         [-0.4930,  0.0624],\n",
      "         [-0.5788,  0.3300],\n",
      "         [-0.5331,  0.0398],\n",
      "         [-0.4920,  0.0966],\n",
      "         [-0.6074,  0.1437]]], grad_fn=<StackBackward0>)\n",
      "output[-1]=\n",
      "tensor([[-0.5553,  0.2702],\n",
      "        [-0.5397,  0.2353],\n",
      "        [-0.5653,  0.0189],\n",
      "        [-0.5807,  0.2390],\n",
      "        [-0.5701,  0.0758],\n",
      "        [-0.4930,  0.0624],\n",
      "        [-0.5788,  0.3300],\n",
      "        [-0.5331,  0.0398],\n",
      "        [-0.4920,  0.0966],\n",
      "        [-0.6074,  0.1437]], grad_fn=<SelectBackward0>)\n"
     ]
    }
   ],
   "source": [
    "from torch import nn\n",
    "import numpy as np\n",
    "# 序列长度为5，每个batch有10个样本，1层（单个）RNN，隐藏状态数量为2个，变量个数为1\n",
    "batch_len = 10\n",
    "seq_len = 5\n",
    "hidden_size = 2\n",
    "rnn = nn.RNN(input_size=1, hidden_size=hidden_size)\n",
    "\n",
    "hidden = torch.zeros(1, batch_len, hidden_size)\n",
    "x = torch.tensor(np.random.random((seq_len, batch_len, 1))).float()\n",
    "\n",
    "print(\"初始的隐藏状态：\\n%s\" % hidden)\n",
    "print(\"数据：\\n%s\\n 形状：%s\" % (x, x.shape))\n",
    "\n",
    "out, hn = rnn(x, hidden)\n",
    "\n",
    "print(\"output=\\n%s\" % out)\n",
    "print(\"h_n=\\n%s\" % hn)\n",
    "print(\"output[-1]=\\n%s\" % out[-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "returning-sword",
   "metadata": {},
   "source": [
    "可以看到，h_n返回的结果和output\\[-1\\]返回的结果一样，都是最后一期batch中每个样本的隐藏状态。\n",
    "\n",
    "以上是只有一层RNN的情况，如果有两层RNN，那么返回的hn实际上包含了两层RNN最终时间的序列状态，而output\\[-1\\]则只包含了最后一层、最后一期的隐藏状态："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "coordinated-blast",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "初始的隐藏状态：\n",
      "tensor([[[0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.]],\n",
      "\n",
      "        [[0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.],\n",
      "         [0., 0.]]])\n",
      "数据：\n",
      "tensor([[[7.7206e-01],\n",
      "         [5.9411e-01],\n",
      "         [6.0247e-01],\n",
      "         [8.5634e-02],\n",
      "         [6.2373e-01],\n",
      "         [2.4784e-01],\n",
      "         [2.2806e-01],\n",
      "         [1.3007e-02],\n",
      "         [3.5345e-01],\n",
      "         [3.5433e-02]],\n",
      "\n",
      "        [[8.9365e-01],\n",
      "         [7.2431e-02],\n",
      "         [3.8674e-01],\n",
      "         [4.2065e-01],\n",
      "         [8.3363e-01],\n",
      "         [8.1836e-01],\n",
      "         [7.3934e-01],\n",
      "         [5.4868e-01],\n",
      "         [2.5100e-01],\n",
      "         [2.4561e-01]],\n",
      "\n",
      "        [[2.5593e-01],\n",
      "         [6.7334e-01],\n",
      "         [1.2492e-01],\n",
      "         [7.3122e-01],\n",
      "         [2.5928e-01],\n",
      "         [8.7730e-02],\n",
      "         [7.8177e-02],\n",
      "         [3.3072e-02],\n",
      "         [3.2049e-01],\n",
      "         [5.6795e-01]],\n",
      "\n",
      "        [[3.6363e-02],\n",
      "         [6.1135e-01],\n",
      "         [8.4880e-01],\n",
      "         [5.6099e-01],\n",
      "         [3.9529e-01],\n",
      "         [9.6294e-01],\n",
      "         [2.0709e-01],\n",
      "         [1.6185e-01],\n",
      "         [9.1721e-01],\n",
      "         [6.6540e-01]],\n",
      "\n",
      "        [[1.5119e-01],\n",
      "         [7.5817e-01],\n",
      "         [8.2086e-04],\n",
      "         [8.1395e-01],\n",
      "         [3.9526e-01],\n",
      "         [1.8533e-01],\n",
      "         [4.1863e-01],\n",
      "         [3.8584e-01],\n",
      "         [8.3460e-01],\n",
      "         [3.9009e-01]]])\n",
      " 形状：torch.Size([5, 10, 1])\n",
      "output=\n",
      "tensor([[[0.8086, 0.1144],\n",
      "         [0.8096, 0.1212],\n",
      "         [0.8096, 0.1209],\n",
      "         [0.8125, 0.1401],\n",
      "         [0.8094, 0.1201],\n",
      "         [0.8116, 0.1342],\n",
      "         [0.8117, 0.1349],\n",
      "         [0.8129, 0.1427],\n",
      "         [0.8110, 0.1303],\n",
      "         [0.8128, 0.1419]],\n",
      "\n",
      "        [[0.9057, 0.4828],\n",
      "         [0.9085, 0.5030],\n",
      "         [0.9079, 0.4968],\n",
      "         [0.9110, 0.5055],\n",
      "         [0.9069, 0.4873],\n",
      "         [0.9093, 0.4951],\n",
      "         [0.9096, 0.4970],\n",
      "         [0.9112, 0.5043],\n",
      "         [0.9097, 0.5042],\n",
      "         [0.9116, 0.5096]],\n",
      "\n",
      "        [[0.9421, 0.5264],\n",
      "         [0.9441, 0.5322],\n",
      "         [0.9439, 0.5372],\n",
      "         [0.9437, 0.5251],\n",
      "         [0.9425, 0.5273],\n",
      "         [0.9432, 0.5301],\n",
      "         [0.9434, 0.5317],\n",
      "         [0.9442, 0.5355],\n",
      "         [0.9443, 0.5356],\n",
      "         [0.9444, 0.5310]],\n",
      "\n",
      "        [[0.9474, 0.5525],\n",
      "         [0.9465, 0.5351],\n",
      "         [0.9473, 0.5398],\n",
      "         [0.9462, 0.5369],\n",
      "         [0.9471, 0.5467],\n",
      "         [0.9469, 0.5395],\n",
      "         [0.9478, 0.5524],\n",
      "         [0.9481, 0.5537],\n",
      "         [0.9469, 0.5359],\n",
      "         [0.9467, 0.5373]],\n",
      "\n",
      "        [[0.9490, 0.5526],\n",
      "         [0.9469, 0.5361],\n",
      "         [0.9475, 0.5437],\n",
      "         [0.9469, 0.5352],\n",
      "         [0.9480, 0.5436],\n",
      "         [0.9470, 0.5372],\n",
      "         [0.9485, 0.5455],\n",
      "         [0.9487, 0.5468],\n",
      "         [0.9463, 0.5279],\n",
      "         [0.9473, 0.5411]]], grad_fn=<StackBackward0>)\n",
      "h_n=\n",
      "tensor([[[ 0.3498, -0.8678],\n",
      "         [ 0.2564, -0.8443],\n",
      "         [ 0.4245, -0.8425],\n",
      "         [ 0.2463, -0.8435],\n",
      "         [ 0.3280, -0.8519],\n",
      "         [ 0.4065, -0.8315],\n",
      "         [ 0.3164, -0.8565],\n",
      "         [ 0.3195, -0.8587],\n",
      "         [ 0.2751, -0.8259],\n",
      "         [ 0.3349, -0.8464]],\n",
      "\n",
      "        [[ 0.9490,  0.5526],\n",
      "         [ 0.9469,  0.5361],\n",
      "         [ 0.9475,  0.5437],\n",
      "         [ 0.9469,  0.5352],\n",
      "         [ 0.9480,  0.5436],\n",
      "         [ 0.9470,  0.5372],\n",
      "         [ 0.9485,  0.5455],\n",
      "         [ 0.9487,  0.5468],\n",
      "         [ 0.9463,  0.5279],\n",
      "         [ 0.9473,  0.5411]]], grad_fn=<StackBackward0>)\n",
      "output[-1]=\n",
      "tensor([[0.9490, 0.5526],\n",
      "        [0.9469, 0.5361],\n",
      "        [0.9475, 0.5437],\n",
      "        [0.9469, 0.5352],\n",
      "        [0.9480, 0.5436],\n",
      "        [0.9470, 0.5372],\n",
      "        [0.9485, 0.5455],\n",
      "        [0.9487, 0.5468],\n",
      "        [0.9463, 0.5279],\n",
      "        [0.9473, 0.5411]], grad_fn=<SelectBackward0>)\n"
     ]
    }
   ],
   "source": [
    "from torch import nn\n",
    "import numpy as np\n",
    "# 序列长度为5，每个batch有10个样本，1层（单个）RNN，隐藏状态数量为2个，变量个数为1\n",
    "batch_len = 10\n",
    "seq_len = 5\n",
    "hidden_size = 2\n",
    "rnn = nn.RNN(input_size=1, hidden_size=hidden_size, num_layers=2)\n",
    "\n",
    "hidden = torch.zeros(2, batch_len, hidden_size)\n",
    "x = torch.tensor(np.random.random((seq_len, batch_len, 1))).float()\n",
    "\n",
    "print(\"初始的隐藏状态：\\n%s\" % hidden)\n",
    "print(\"数据：\\n%s\\n 形状：%s\" % (x, x.shape))\n",
    "\n",
    "out, hn = rnn(x, hidden)\n",
    "\n",
    "print(\"output=\\n%s\" % out)\n",
    "print(\"h_n=\\n%s\" % hn)\n",
    "print(\"output[-1]=\\n%s\" % out[-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "previous-detroit",
   "metadata": {},
   "source": [
    "有了如上的基础知识后，我们就可以定义RNN模型了，比如，我们使用两层的RNN模型，为了防止过拟合，加入dropout，然后将第二层的隐藏状态带入到一个前馈网络中，进行预测："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "adequate-fusion",
   "metadata": {},
   "outputs": [],
   "source": [
    "class RNN(nn.Module):\n",
    "\n",
    "    def __init__(self, batch_size, feature_size, hidden_size=30):\n",
    "        super(RNN, self).__init__()\n",
    "        self.batch_size = batch_size  #批大小\n",
    "        self.feature_size = feature_size  #特征数量\n",
    "        self.hidden_size = hidden_size  #隐藏状态大小\n",
    "\n",
    "        self.rnn = nn.RNN(input_size=feature_size,\n",
    "                          nonlinearity='tanh',\n",
    "                          hidden_size=hidden_size,\n",
    "                          num_layers=2,\n",
    "                          dropout=0.5)\n",
    "        self.layer2 = nn.Sequential(nn.Linear(hidden_size, 50),\n",
    "                                    nn.ReLU(inplace=True), nn.LayerNorm(50),\n",
    "                                    nn.Linear(50, 50), nn.ReLU(inplace=True),\n",
    "                                    nn.Dropout(p=0.5), nn.Linear(50, 1))\n",
    "        self.hidden = self.init_hidden()\n",
    "\n",
    "    def init_hidden(self):\n",
    "        h = torch.zeros(2, self.batch_size, self.hidden_size).to(device)\n",
    "        return h\n",
    "\n",
    "    def forward(self, x):\n",
    "        out, h = self.rnn(x, self.hidden)\n",
    "        y = self.layer2(out[-1])\n",
    "        return y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "international-budapest",
   "metadata": {},
   "source": [
    "然后求最优化："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "variable-deviation",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "第0次epoch，Smoothed Loss=0.12222139537334442，LR=[0.009000000000000001]\n",
      "第20次epoch，Smoothed Loss=0.013956679763871957，LR=[0.0010941898913151245]\n",
      "第40次epoch，Smoothed Loss=0.013510473361483477，LR=[0.00013302794647291155]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABLkAAAGsCAYAAAAxGHVCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAnrklEQVR4nO3de5CcdZno8acnl86FyUAIM5MhkwuCgAmCBIQAKheJxARFqjyoyKJ7ypKVANlwVslRF9SFQbfkUCuCC+UiHpfLbinKHhAMQgIsCUJINAQJt0BikiEQw0xuTEjmPX9gGiczkwv0Jb+ez6dqyvTb7/T79NC/6fjN2925LMuyAAAAAICE1VR6AAAAAAB4t0QuAAAAAJIncgEAAACQPJELAAAAgOSJXAAAAAAkT+QCAAAAIHkiFwAAAADJ61/pAXbU2dkZq1atitra2sjlcpUeBwAAAIAKyrIs1q9fH01NTVFT0/v5Wntd5Fq1alU0NzdXegwAAAAA9iIrVqyIUaNG9Xr9Xhe5amtrI+KtwYcNG1bhaQAAAACopPb29mhubi40o97sdZFr+0sUhw0bJnIBAAAAEBGxy7e18sbzAAAAACRP5AIAAAAgeSIXAAAAAMkTuQAAAABInsgFAAAAQPJELgAAAACSJ3IBAAAAkDyRCwAAAIDkiVwAAAAAJE/kAgAAACB5IhcAAAAAyRO5AAAAAEieyAUAAABA8vpXeoC+4KmVbfH6pjfjvQ37RP2wQZUeBwAAAKDqOJOrDK7+9TPx+R8/Fo++sLbSowAAAABUJZELAAAAgOSJXAAAAAAkT+QCAAAAIHkiFwAAAADJE7kAAAAASJ7IBQAAAEDyRC4AAAAAkidyAQAAAJA8kQsAAACA5IlcZZDLvfW/WWSVHQQAAACgSolcAAAAACRP5AIAAAAgeXscuR566KE488wzo6mpKXK5XPzyl7/scn2WZXHFFVdEU1NTDB48OE4++eRYsmRJseYFAAAAgG72OHJt3LgxjjzyyLjuuut6vP573/teXHPNNXHdddfF448/Ho2NjXH66afH+vXr3/WwAAAAANCT/nv6DVOmTIkpU6b0eF2WZXHttdfG17/+9Tj77LMjIuKWW26JhoaGuPXWW+PLX/5yt+/p6OiIjo6OwuX29vY9HQkAAACAPq6o78m1bNmyaG1tjcmTJxe25fP5+MhHPhKPPvpoj9/T0tISdXV1ha/m5uZijgQAAABAH1DUyNXa2hoREQ0NDV22NzQ0FK7b0axZs6Ktra3wtWLFimKOBAAAAEAfsMcvV9wduVyuy+Usy7pt2y6fz0c+ny/FGAAAAAD0EUU9k6uxsTEiottZW2vWrOl2dhcAAAAAFEtRI9e4ceOisbExZs+eXdi2ZcuWmDt3bpxwwgnFPFSSsqzSEwAAAABUpz1+ueKGDRvi+eefL1xetmxZLFq0KIYPHx6jR4+OGTNmxFVXXRWHHHJIHHLIIXHVVVfFkCFD4nOf+1xRBwcAAACA7fY4cj3xxBNxyimnFC7PnDkzIiLOP//8+MlPfhJf/epXY/PmzfGVr3wl1q1bF8cdd1z85je/idra2uJNDQAAAAB/ZY8j18knnxzZTl53l8vl4oorrogrrrji3cwFAAAAALutqO/JBQAAAACVIHIBAAAAkDyRCwAAAIDkiVwAAAAAJE/kAgAAACB5IhcAAAAAyRO5yiCXy0VERJZVeBAAAACAKiVyAQAAAJA8kQsAAACA5IlcAAAAACRP5AIAAAAgeSIXAAAAAMkTuQAAAABInsgFAAAAQPJELgAAAACSJ3IBAAAAkDyRqwxylR4AAAAAoMqJXGWUVXoAAAAAgColcgEAAACQPJELAAAAgOSJXAAAAAAkT+QCAAAAIHkiFwAAAADJE7kAAAAASJ7IBQAAAEDyRC4AAAAAkidylUEuV+kJAAAAAKqbyFVGWZZVegQAAACAqiRyAQAAAJA8kQsAAACA5IlcAAAAACRP5AIAAAAgeSIXAAAAAMkTuQAAAABInsgFAAAAQPJELgAAAACSJ3KVQa7SAwAAAABUOZGrjLJKDwAAAABQpUQuAAAAAJIncgEAAACQPJELAAAAgOSJXAAAAAAkT+QCAAAAIHkiFwAAAADJE7kAAAAASJ7IBQAAAEDyRK4yyOVylR4BAAAAoKqJXOWUVXoAAAAAgOokcgEAAACQPJELAAAAgOSJXAAAAAAkT+QCAAAAIHkiFwAAAADJE7kAAAAASJ7IBQAAAEDyRC4AAAAAklf0yLV169b4xje+EePGjYvBgwfHQQcdFN/+9rejs7Oz2IdKRq7SAwAAAABUuf7FvsHvfve78aMf/ShuueWWGD9+fDzxxBPxxS9+Merq6uKSSy4p9uEAAAAAoPiRa968efHJT34ypk6dGhERY8eOjdtuuy2eeOKJHvfv6OiIjo6OwuX29vZij7TXyCKr9AgAAAAAVanoL1c86aST4re//W08++yzERHx+9//Ph555JH4+Mc/3uP+LS0tUVdXV/hqbm4u9kgAAAAAVLmin8n1ta99Ldra2uKwww6Lfv36xbZt2+LKK6+Mz372sz3uP2vWrJg5c2bhcnt7u9AFAAAAwB4peuS644474mc/+1nceuutMX78+Fi0aFHMmDEjmpqa4vzzz++2fz6fj3w+X+wxAAAAAOhDih65/uEf/iEuu+yy+MxnPhMREUcccUS8/PLL0dLS0mPkAgAAAIB3q+jvybVp06aoqel6s/369YvOzs5iHwoAAAAAIqIEZ3KdeeaZceWVV8bo0aNj/PjxsXDhwrjmmmvib//2b4t9KAAAAACIiBJErh/84AfxzW9+M77yla/EmjVroqmpKb785S/HP/7jPxb7UAAAAAAQESWIXLW1tXHttdfGtddeW+ybTlYuV+kJAAAAAKpb0d+TCwAAAADKTeQqoyyr9AQAAAAA1UnkAgAAACB5IhcAAAAAyRO5AAAAAEieyAUAAABA8kQuAAAAAJIncgEAAACQPJELAAAAgOSJXGWRq/QAAAAAAFVN5AIAAAAgeSJXGWWVHgAAAACgSolcAAAAACRP5AIAAAAgeSIXAAAAAMkTuQAAAABInsgFAAAAQPJELgAAAACSJ3IBAAAAkDyRqwxyuUpPAAAAAFDdRC4AAAAAkidyAQAAAJA8kauMsqzSEwAAAABUJ5ELAAAAgOSJXAAAAAAkT+QCAAAAIHkiFwAAAADJE7kAAAAASJ7IBQAAAEDyRK4yyFV6AAAAAIAqJ3IBAAAAkDyRCwAAAIDkiVxllEVW6REAAAAAqpLIBQAAAEDyRC4AAAAAkidyAQAAAJA8kQsAAACA5IlcAAAAACRP5AIAAAAgeSJXGeRylZ4AAAAAoLqJXAAAAAAkT+QCAAAAIHkiVxllWaUnAAAAAKhOIhcAAAAAyRO5AAAAAEieyAUAAABA8kQuAAAAAJIncgEAAACQPJELAAAAgOSJXGWQi1ylRwAAAACoaiIXAAAAAMkTuQAAAABInsgFAAAAQPJErjLKKj0AAAAAQJUSuQAAAABInsgFAAAAQPJKErlWrlwZn//852P//fePIUOGxFFHHRULFiwoxaEAAAAAIPoX+wbXrVsXJ554Ypxyyinx61//Ourr6+OFF16Ifffdt9iHAgAAAICIKEHk+u53vxvNzc1x8803F7aNHTu21/07Ojqio6OjcLm9vb3YIwEAAABQ5Yr+csW77rorjjnmmPj0pz8d9fX18YEPfCBuuummXvdvaWmJurq6wldzc3OxR6q4XK7SEwAAAABUt6JHrhdffDFuuOGGOOSQQ+K+++6LCy64IC6++OL46U9/2uP+s2bNira2tsLXihUrij0SAAAAAFWu6C9X7OzsjGOOOSauuuqqiIj4wAc+EEuWLIkbbrgh/uZv/qbb/vl8PvL5fLHHAAAAAKAPKfqZXCNHjoz3ve99XbYdfvjhsXz58mIfCgAAAAAiogSR68QTT4ylS5d22fbss8/GmDFjin2o9GRZpScAAAAAqEpFj1x///d/H/Pnz4+rrroqnn/++bj11lvjxhtvjAsvvLDYhwIAAACAiChB5Dr22GPjzjvvjNtuuy0mTJgQ3/nOd+Laa6+Nc889t9iHAgAAAICIKMEbz0dETJs2LaZNm1aKmwYAAACAbop+JhcAAAAAlJvIBQAAAEDyRK4yyOUqPQEAAABAdRO5AAAAAEieyAUAAABA8kQuAAAAAJIncpVRVukBAAAAAKqUyAUAAABA8kQuAAAAAJIncgEAAACQPJELAAAAgOSJXAAAAAAkT+Qqg1zkKj0CAAAAQFUTuQAAAABInsgFAAAAQPJELgAAAACSJ3IBAAAAkDyRq4yyrNITAAAAAFQnkQsAAACA5IlcAAAAACRP5AIAAAAgeSIXAAAAAMkTucohV+kBAAAAAKqbyAUAAABA8kQuAAAAAJIncgEAAACQPJELAAAAgOSJXGWUZVmlRwAAAACoSiIXAAAAAMkTuQAAAABInsgFAAAAQPJELgAAAACSJ3KVQa7SAwAAAABUOZELAAAAgOSJXAAAAAAkT+QCAAAAIHkiFwAAAADJE7nKKKv0AAAAAABVSuQCAAAAIHkiFwAAAADJE7kAAAAASJ7IBQAAAEDyRK4yyOVylR4BAAAAoKqJXAAAAAAkT+QCAAAAIHkiFwAAAADJE7kAAAAASJ7IBQAAAEDyRK4yyrJKTwAAAABQnUQuAAAAAJIncgEAAACQPJELAAAAgOSJXGWQq/QAAAAAAFVO5AIAAAAgeSIXAAAAAMkreeRqaWmJXC4XM2bMKPWhAAAAAOijShq5Hn/88bjxxhvj/e9/fykPAwAAAEAfV7LItWHDhjj33HPjpptuiv32269UhwEAAACA0kWuCy+8MKZOnRof/ehHd7pfR0dHtLe3d/mqVlmlBwAAAACoUv1LcaO33357PPnkk/H444/vct+Wlpb41re+VYoxAAAAAOgjin4m14oVK+KSSy6Jn/3sZzFo0KBd7j9r1qxoa2srfK1YsaLYIwEAAABQ5Yp+JteCBQtizZo1MXHixMK2bdu2xUMPPRTXXXdddHR0RL9+/QrX5fP5yOfzxR4DAAAAgD6k6JHrtNNOi8WLF3fZ9sUvfjEOO+yw+NrXvtYlcPUVuVylJwAAAACobkWPXLW1tTFhwoQu24YOHRr7779/t+0AAAAAUAwl+3RFAAAAACiXkny64o7mzJlTjsMAAAAA0Ec5kwsAAACA5IlcAAAAACRP5CqjLMsqPQIAAABAVRK5AAAAAEieyAUAAABA8kQuAAAAAJIncpVBrtIDAAAAAFQ5kQsAAACA5IlcAAAAACRP5AIAAAAgeSIXAAAAAMkTuQAAAABInsgFAAAAQPJELgAAAACSJ3IBAAAAkDyRCwAAAIDkiVxlkMvlKj0CAAAAQFUTuQAAAABInsgFAAAAQPJELgAAAACSJ3IBAAAAkDyRCwAAAIDkiVwAAAAAJE/kKqMsq/QEAAAAANVJ5AIAAAAgeSIXAAAAAMkTucogV+kBAAAAAKqcyAUAAABA8kQuAAAAAJIncgEAAACQPJELAAAAgOSJXAAAAAAkT+QCAAAAIHkiVxllkVV6BAAAAICqJHIBAAAAkDyRCwAAAIDkiVzlkKv0AAAAAADVTeQCAAAAIHkiFwAAAADJE7kAAAAASJ7IBQAAAEDyRC4AAAAAkidyAQAAAJA8kQsAAACA5IlcZZRllZ4AAAAAoDqJXAAAAAAkT+Qqg1zkKj0CAAAAQFUTuQAAAABInsgFAAAAQPJELgAAAACSJ3IBAAAAkDyRCwAAAIDkiVwAAAAAJE/kAgAAACB5IlcZZZUeAAAAAKBKiVwAAAAAJK/okaulpSWOPfbYqK2tjfr6+jjrrLNi6dKlxT5MUnK5Sk8AAAAAUN2KHrnmzp0bF154YcyfPz9mz54dW7dujcmTJ8fGjRuLfSgAAAAAiIiI/sW+wXvvvbfL5Ztvvjnq6+tjwYIF8eEPf7jYhwMAAACA4keuHbW1tUVExPDhw3u8vqOjIzo6OgqX29vbSz0SAAAAAFWmpG88n2VZzJw5M0466aSYMGFCj/u0tLREXV1d4au5ubmUIwEAAABQhUoauaZPnx5/+MMf4rbbbut1n1mzZkVbW1vha8WKFaUcCQAAAIAqVLKXK1500UVx1113xUMPPRSjRo3qdb98Ph/5fL5UYwAAAADQBxQ9cmVZFhdddFHceeedMWfOnBg3blyxDwEAAAAAXRQ9cl144YVx6623xq9+9auora2N1tbWiIioq6uLwYMHF/twSdiytTMiIrZu66zwJAAAAADVKZdlWVbUG8zletx+8803xxe+8IVdfn97e3vU1dVFW1tbDBs2rJijVczYy+4u/Pmlq6dWcBIAAACAtOxuKyrJyxUBAAAAoJxK+umKAAAAAFAOIhcAAAAAyRO5AAAAAEieyAUAAABA8kQuAAAAAJIncgEAAACQPJELAAAAgOSJXAAAAAAkT+QCAAAAIHkiFwAAAADJE7kAAAAASJ7IBQAAAEDyRC4AAAAAkidyAQAAAJA8kQsAAACA5IlcAAAAACRP5AIAAAAgeSIXAAAAAMkTuQAAAABInshVBl8949CIiDhjfGOFJwEAAACoTiJXGeyT7x8REf1qchWeBAAAAKA6iVxlkMu9Fbc6s6zCkwAAAABUJ5GrDLafwCVyAQAAAJSGyFUGNX85k2vJqvYKTwIAAABQnUSuMliyqi0iIv60bnOFJwEAAACoTiJXGTz7yoZKjwAAAABQ1UQuAAAAAJIncgEAAACQPJGrDDZt2VrpEQAAAACqmshVBk+t9KmKAAAAAKUkcpXBoQ21hT+3tr0RWZZVcBoAAACA6iNylcF76ocW/nx8y2/jW//1dAWnAQAAAKg+IlcZXHLae7tc/smjL0Vnp7O5AAAAAIpF5CqDfj38lP/p7j+WfxAAAACAKiVylcGWrd3P2vq3/14Wr7S/UYFpAAAAAKqPyFUGhzbW9rj9hKsfKPMkAAAAANVJ5CqDfjW5Hrdv875cAAAAAEUhcgEAAACQPJGrTObPOq3SIwAAAABULZGrTBrrBsWCb3y02/YFL6+rwDQAAAAA1UXkKqP998l32/b06vYKTAIAAABQXUSuMvu///ODlR4BAAAAoOqIXGX2oUMO6HL5m798Kn61aGWFpgEAAACoDiLXXuCS2xdVegQAAACApIlcFXDgvoMrPQIAAABAVRG5KuDCUw7uti3LsgpMAgAAAFAdRK4K+Myxzd22jZt1T/zX71dVYBoAAACA9IlcFVBTk4uLTzuk2/aLblsYr23oqMBEAAAAAGkTuSrkolO7v2QxIuKYf7o/Lv2P33v5IgAAAMAeELkqZEC/mnjp6qnxf845stt1P3/yTzFu1j0VmAroyZatnTHrF3+If77vGQEaAABgLyVyVdhZRx3Y63UTLr+vjJMAPbnmN0vjvd/4ddz2uxXxwwdfEKABAAD2UiJXheVyuXjsf5/W43UbOrbG2MvujhOvfqDMUwHb/csDz1d6BAAAAHaDyLUXaBg2KJa1fDwG9u/5P8fK1zfH2Mvu9qb0UGabtmyt9AgAAADsJpFrL5HL5WLpd86IGR/t/qmL2x3zT/fH2MvujqWt68s4GfRdy/+8qdu2IQP7VWASAAAAdkXk2ovkcrmY8dH3xktXT40nvvHRXvf72LUPxdjL7o7zfvxYvLre2V1QKm2b3oyIiP41uZjzv06OiIhNW7ZVcCIAAAB6I3LtpUbsk4+Xrp4aXz3j0F73efi51+LYK986u+u4q+6P//r9qjJOCNVv05tvBa3DRtbG4L86g2vdxi2VGgkAAIBelCxyXX/99TFu3LgYNGhQTJw4MR5++OFSHaqqfeXkg+Olq6fG81dOiZMPPaDX/V5p74iLblsYYy+7O8ZedndM/ZeH496nVsfLazfGxg7vKwTvxOa/nLU1ZED/aBg2qLD9Rw+9UKmRAAAA6EX/UtzoHXfcETNmzIjrr78+TjzxxPjXf/3XmDJlSjz99NMxevToUhyy6vXvVxM/+eIHIyLi+79ZGj/YxSe+LVnVHhf87Mmd7nPlpyZE3eABcfKh9TGwX02vb3wPfdXvlv05IiJe29j1ZcF3PrkyZk05vBIjAQAA0ItclmVZsW/0uOOOi6OPPjpuuOGGwrbDDz88zjrrrGhpadnp97a3t0ddXV20tbXFsGHDij1a1cmyLMbNuqfSY1Aiwwb1j/Y3SnMm3qSD9o95L64tyW0Xy6ENtbH0lfUxoF8u3tzW/VfVkIH9urxHVk0uorPov9He8tLVU2PsZXeX5sZL6IgD62LxyrZerx+13+D407rNsU++f2zo4azPA/cdHCtf31zKEZNUX5uPNbvxnogD+9XElm2dRTnmXz/ex+4/JF5a2/2DEbbbvnaKYcz+Q+LltZti/6EDY+0uXqr7/lF1sd+QgfH06vbdfs/Igf1rYsvW4vyMdqVu8IAYWTcolv95U9HfX2/7/TigNr/L+z6+aVgsWdUeB9fvE8OHDIzXN2+JZ1/ZsEfHGzygX2x+893dh8vPfF90bO2Mq3/9TETs+nG1K6X8HfxOXHHm+2JbFvGd//d0YVtPa+PwkcPiA6P3jVsfW77L2zy4fp94fs3O/1t9euKo+GNrezy1sr2wrbefzcfGN8R9S17p8XYOOmBo7Dt4QDy5/PUu22vz/eOrUw6LJ176c/x6cWvhd8wh9fvEc73M1lQ3KFa1vbHTuT9+RGPcs7i12/aRdYPiqOZ949dPdb9uR7X5/rG+Y2ucdVRT/HLR22+j8bHxDTE03z9+8eTKHr/v4tMOiRfWbIi7F6/e5TF25pRDD4hnWtfH6rY3dnmfd5xxd+z4M37fyGHx9Or2bvuNHj4kzjm2OWpyufjuvc/s9Db71eRi27tcOLvz+/miUw+OjR3b4t/+e1kMGlATo/YbEiP2GRhLVrZHY92geG7NhsLfCc45pjkWrli3W7+XTj70gJiz9NVu29/p/Zpw4LBYt/HNWPn65jjogKHx4qsbe9xv+ikHR75/TXx/9rPdrjtw38ExbsTQeOT51wrbDh85LJ5pbY/e/p/viH3y8dqGjjhoxNB48bXux9x3yIA4qnnfOKR+n3hqZXvMX7a2cFvfmHp4/PN9S6NjD5/L8v1rCt/zvz9+WFx1zzNx3Ljh8dhf/pG1N43DBkVr+9uP7dpB/WP9G1tj/6EDo27wgML8nzyqKX61aFV8cNzwwj/c/o9jRsV/PPGnXc62/Xnqrx3VvG+0tr0Rre1vxGGNtXFAbT4WLn89NnRsjVwuCj+P3h4TOzpjfGOsfH1zLF7ZFoc11sYz7/LD1Xb8ufy1Efvk48D9BsfvV7xe2Db9lIOjad/B8c1fPbXTx2pvf5d+t8/D094/Mk48eETc8fiKWPSXufYbMiDW/eU9gSMi3tuwT7d1+Ikjm+Kuv3qbou3rtifXn3t0fOiQEVE7aMA7nnNvsrutqOiRa8uWLTFkyJD4z//8z/jUpz5V2H7JJZfEokWLYu7cuV327+joiI6Ot/8y2N7eHs3NzSLXO/Tmts6Ys/TV+Om8l+Lh517b9TcAuyXVyAUAAPRNt33p+Jj0nv0rPUZR7G7kKvrLFV977bXYtm1bNDQ0dNne0NAQra3d/wWopaUlvvWtbxV7jD5rQL+aOP19DXH6+xp6vL6zM4tXN3TEytc3x++W/TkeevbVaG17I9o2v9ml/LN3KOZZIDs64T37x6Mv7N1ncu0t5s86LSIiXrzq43HK9+fEy+/iTIdy29lZAsVSyjMOU1eqM1t25196i6W3MxVS9Z4DhsYLvZwZ0NfsP3Rg5HIRr22ozg/TOPHg/WPzlm3dzoTqSf+aXGwt0mIdPXxILP9zaZ8nTjp4RJezVEpt3IihsWwP/o64499fjj9oeMx/sfffWc3DB8eKP1fXWcNnHtkUW7ZuK/lzcCkc1bxv4cyScvvrM4J25uD6faK+Nl/2v8v2dDZoMX5eOzsb550aOrBfbCzRp4Lv7O9+O56NtDPbz/jr7VUbpVSb7x9jRwzd6SseSunkQw+I/jU1cf8fS/c7YtCAmjh6zL4lu/29VUnekysiIpfLdbmcZVm3bRERs2bNipkzZxYubz+Ti9KoqclFw7BB0TBsUBw9er+44CPvqfRIkIyamlzM/YdTKj0GAAAAPSh65BoxYkT069ev21lba9as6XZ2V0REPp+PfD5f7DEAAAAA6EOK/nF6AwcOjIkTJ8bs2bO7bJ89e3accMIJxT4cAAAAAJTm5YozZ86M8847L4455piYNGlS3HjjjbF8+fK44IILSnE4AAAAAPq4kkSuc845J9auXRvf/va3Y/Xq1TFhwoS45557YsyYMaU4HAAAAAB9XC7LdufzK8pndz8WEgAAAIDqt7utqOjvyQUAAAAA5SZyAQAAAJA8kQsAAACA5IlcAAAAACRP5AIAAAAgeSIXAAAAAMkTuQAAAABInsgFAAAAQPJELgAAAACS17/SA+woy7KIiGhvb6/wJAAAAABU2vZGtL0Z9Wavi1zr16+PiIjm5uYKTwIAAADA3mL9+vVRV1fX6/W5bFcZrMw6Oztj1apVUVtbG7lcrtLjFEV7e3s0NzfHihUrYtiwYZUeByrKeoC3WQ/wFmsB3mY9wNusB7bLsizWr18fTU1NUVPT+ztv7XVnctXU1MSoUaMqPUZJDBs2zMKEv7Ae4G3WA7zFWoC3WQ/wNuuBiNjpGVzbeeN5AAAAAJIncgEAAACQPJGrDPL5fFx++eWRz+crPQpUnPUAb7Me4C3WArzNeoC3WQ/sqb3ujecBAAAAYE85kwsAAACA5IlcAAAAACRP5AIAAAAgeSIXAAAAAMkTuQAAAABInshVBtdff32MGzcuBg0aFBMnToyHH3640iPBO3bFFVdELpfr8tXY2Fi4PsuyuOKKK6KpqSkGDx4cJ598cixZsqTLbXR0dMRFF10UI0aMiKFDh8YnPvGJ+NOf/tRln3Xr1sV5550XdXV1UVdXF+edd168/vrr5biL0KuHHnoozjzzzGhqaopcLhe//OUvu1xfzsf/8uXL48wzz4yhQ4fGiBEj4uKLL44tW7aU4m5Dj3a1Hr7whS90e744/vjju+xjPVANWlpa4thjj43a2tqor6+Ps846K5YuXdplH88P9BW7sx48P1BKIleJ3XHHHTFjxoz4+te/HgsXLowPfehDMWXKlFi+fHmlR4N3bPz48bF69erC1+LFiwvXfe9734trrrkmrrvuunj88cejsbExTj/99Fi/fn1hnxkzZsSdd94Zt99+ezzyyCOxYcOGmDZtWmzbtq2wz+c+97lYtGhR3HvvvXHvvffGokWL4rzzzivr/YQdbdy4MY488si47rrrery+XI//bdu2xdSpU2Pjxo3xyCOPxO233x4///nP49JLLy3dnYcd7Go9REScccYZXZ4v7rnnni7XWw9Ug7lz58aFF14Y8+fPj9mzZ8fWrVtj8uTJsXHjxsI+nh/oK3ZnPUR4fqCEMkrqgx/8YHbBBRd02XbYYYdll112WYUmgnfn8ssvz4488sger+vs7MwaGxuzq6++urDtjTfeyOrq6rIf/ehHWZZl2euvv54NGDAgu/322wv7rFy5MqupqcnuvffeLMuy7Omnn84iIps/f35hn3nz5mURkT3zzDMluFew5yIiu/POOwuXy/n4v+eee7Kampps5cqVhX1uu+22LJ/PZ21tbSW5v7AzO66HLMuy888/P/vkJz/Z6/dYD1SrNWvWZBGRzZ07N8syzw/0bTuuhyzz/EBpOZOrhLZs2RILFiyIyZMnd9k+efLkePTRRys0Fbx7zz33XDQ1NcW4cePiM5/5TLz44osREbFs2bJobW3t8pjP5/PxkY98pPCYX7BgQbz55ptd9mlqaooJEyYU9pk3b17U1dXFcccdV9jn+OOPj7q6OmuHvVY5H//z5s2LCRMmRFNTU2Gfj33sY9HR0RELFiwo6f2EPTFnzpyor6+P9773vfGlL30p1qxZU7jOeqBatbW1RUTE8OHDI8LzA33bjuthO88PlIrIVUKvvfZabNu2LRoaGrpsb2hoiNbW1gpNBe/OcccdFz/96U/jvvvui5tuuilaW1vjhBNOiLVr1xYe1zt7zLe2tsbAgQNjv/322+k+9fX13Y5dX19v7bDXKufjv7W1tdtx9ttvvxg4cKA1wl5jypQp8e///u/xwAMPxPe///14/PHH49RTT42Ojo6IsB6oTlmWxcyZM+Okk06KCRMmRITnB/quntZDhOcHSqt/pQfoC3K5XJfLWZZ12wapmDJlSuHPRxxxREyaNCne8573xC233FJ4w8h38pjfcZ+e9rd2SEG5Hv/WCHu7c845p/DnCRMmxDHHHBNjxoyJu+++O84+++xev896IGXTp0+PP/zhD/HII490u87zA31Nb+vB8wOl5EyuEhoxYkT069evWyVes2ZNt6IMqRo6dGgcccQR8dxzzxU+ZXFnj/nGxsbYsmVLrFu3bqf7vPLKK92O9eqrr1o77LXK+fhvbGzsdpx169bFm2++aY2w1xo5cmSMGTMmnnvuuYiwHqg+F110Udx1113x4IMPxqhRowrbPT/QF/W2Hnri+YFiErlKaODAgTFx4sSYPXt2l+2zZ8+OE044oUJTQXF1dHTEH//4xxg5cmSMGzcuGhsbuzzmt2zZEnPnzi085idOnBgDBgzoss/q1avjqaeeKuwzadKkaGtri9/97neFfR577LFoa2uzdthrlfPxP2nSpHjqqadi9erVhX1+85vfRD6fj4kTJ5b0fsI7tXbt2lixYkWMHDkyIqwHqkeWZTF9+vT4xS9+EQ888ECMGzeuy/WeH+hLdrUeeuL5gaIq57vc90W33357NmDAgOzHP/5x9vTTT2czZszIhg4dmr300kuVHg3ekUsvvTSbM2dO9uKLL2bz58/Ppk2bltXW1hYe01dffXVWV1eX/eIXv8gWL16cffazn81GjhyZtbe3F27jggsuyEaNGpXdf//92ZNPPpmdeuqp2ZFHHplt3bq1sM8ZZ5yRvf/978/mzZuXzZs3LzviiCOyadOmlf3+wl9bv359tnDhwmzhwoVZRGTXXHNNtnDhwuzll1/Osqx8j/+tW7dmEyZMyE477bTsySefzO6///5s1KhR2fTp08v3w6DP29l6WL9+fXbppZdmjz76aLZs2bLswQcfzCZNmpQdeOCB1gNV5+/+7u+yurq6bM6cOdnq1asLX5s2bSrs4/mBvmJX68HzA6UmcpXBD3/4w2zMmDHZwIEDs6OPPrrLx6dCas4555xs5MiR2YABA7Kmpqbs7LPPzpYsWVK4vrOzM7v88suzxsbGLJ/PZx/+8IezxYsXd7mNzZs3Z9OnT8+GDx+eDR48OJs2bVq2fPnyLvusXbs2O/fcc7Pa2tqstrY2O/fcc7N169aV4y5Crx588MEsIrp9nX/++VmWlffx//LLL2dTp07NBg8enA0fPjybPn169sYbb5Ty7kMXO1sPmzZtyiZPnpwdcMAB2YABA7LRo0dn559/frfHuvVANehpHUREdvPNNxf28fxAX7Gr9eD5gVLLZVmWle+8MQAAAAAoPu/JBQAAAEDyRC4AAAAAkidyAQAAAJA8kQsAAACA5IlcAAAAACRP5AIAAAAgeSIXAAAAAMkTuQAAAABInsgFAAAAQPJELgAAAACSJ3IBAAAAkLz/D+CcAeUHmOIdAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1500x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "device=torch.device('cuda:0' if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "model=RNN(batch_size, 1).to(device)\n",
    "\n",
    "criterion=nn.MSELoss()\n",
    "optimizer=optim.Adam(model.parameters(), lr=0.01)\n",
    "lr_scheduler=torch.optim.lr_scheduler.ExponentialLR(optimizer, gamma=0.9)\n",
    "losses=[]\n",
    "Exp_Smoothed_Loss=0\n",
    "for i in range(50):\n",
    "    for x, y in dl:\n",
    "        # 将x计算预测值\n",
    "        y_pred=model(x.float().T.unsqueeze(-1).to(device))\n",
    "        # 计算损失\n",
    "        loss=criterion(y_pred, y.float().unsqueeze(1).to(device))\n",
    "        losses.append(loss.item())\n",
    "        if i==0:\n",
    "            Exp_Smoothed_Loss=loss.item()\n",
    "        else:\n",
    "            Exp_Smoothed_Loss=0.01*loss.item()+0.99*Exp_Smoothed_Loss\n",
    "        # 反向传播\n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "    lr_scheduler.step()\n",
    "    if i%20==0:\n",
    "        print(\"第%s次epoch，Smoothed Loss=%s，LR=%s\" % (i, Exp_Smoothed_Loss, lr_scheduler.get_last_lr()))\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (15.0, 5.0)\n",
    "\n",
    "i=np.arange(len(losses))+1\n",
    "plt.plot(i,np.array(losses))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ready-princess",
   "metadata": {},
   "source": [
    "最后可以检查预测效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "arabic-frontier",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzoAAAMtCAYAAABXYgSXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAACBXUlEQVR4nO3dfXyU9Z3v//dMSDIkJgMJhRkUJSK1DhE1WEpKcVskNsWNtHV3T29w226XLVTarm5PFbduzM/T0p7tObZdekCt265m1b1BK6k0W10V1I0bNSDEqEUc1OJECpFJBJOQmev3R5yQSebmmsncXDPzej4eeWwzueaabzodN2+/38/nYzMMwxAAAAAA5BF7thcAAAAAAKlG0AEAAACQdwg6AAAAAPIOQQcAAABA3iHoAAAAAMg7BB0AAAAAeYegAwAAACDvTMv2AswIBoN66623VFFRIZvNlu3lAAAAAMgSwzA0MDCguXPnym6Pvm+TE0Hnrbfe0rx587K9DAAAAAAW8eabb+qss86K+vOcCDoVFRWSRn+ZysrKLK8GAAAAQLb09/dr3rx5YxkhmpwIOqHjapWVlQQdAAAAAHFLWmhGAAAAACDvEHQAAAAA5B2CDgAAAIC8Q9ABAAAAkHcIOgAAAADyDkEHAAAAQN4h6AAAAADIOwQdAAAAAHmHoAMAAAAg7xB0AAAAAOQdgg4AAACAvEPQAQAAAJB3CDoAAAAA8g5BBwAAAEDeIegAAAAAyDsEHQAAAAB5h6ADAAAAIO8QdAAAAADkHYIOAAAAgLxD0AEAAACQdwg6AAAAAPIOQQcAAABA3iHoAAAAAMg7BB0AAAAAeYegAwAAACDvEHQAAAAA5B2CDgAAAIC8My3bCwAAAABgLYGgoU5vn44MDGp2hUNLa6pUZLdle1kJIegAAAAAkDQacLY89qp+8bRXx987NfZ4VXmJPn3xXDV4XDkTemyGYRjZXkQ8/f39cjqd8vv9qqyszPZyAAAAgLzT3u3TjQ/s1/GTp2Je53Y61NzkUWOtO0MrC2c2G1CjAwAAABS49m6f1rd2xQ05kuTzD2pDa5fau30ZWFnyCDoAAABAAQsEDbW09ST8vJa2HgWC1j0cRtABAAAAClint08+/2BCzzE0urPT6e1Lz6JSgGYEAAAAQB4aHgnqn/7rkJ49dExlJdN0dd1Z+uh5syY1EjgykFjISdVz042gAwAAAOSJQNDQM68d09//x8va+6Y/7Ge/2vuWykqK9H//7KKwRgKzKxxJv95UnptuBB0AAAAgD5jpmnZyOKD1rV3atrZuLOwsramS2+lQr39QZitubJJcztH5OlZFjQ4AAACQ4xLpmiZJt+x4cayRQJHdpuYmj6TRABNP6JrmJo+l5+kQdAAAAIAcFggaumVHYl3TevuHwhoJNNa6tXVtnVzO8KNoZSVFKi8tCnvM5XRo67gdIavi6BoAAACQA043F+hTWUmRrr7kLH1kQbXu6Tik3v7EmwJMbCTQWOtWg8elTm+fjgwManbF6aNpEx+z8k5OCEEHAAAAsLjvPfyi7nzyUNhjv9r71pTuGamRQJHdpvoF1ZMej/SY1RF0AAAAAAtbd/ezeqTnSErvObNsmqUbCaQCNToAAACARbW98FbKQ44kfe/TF+bE8bOpIOgAAAAAFhQIGrruX/am/L5fu6xGqxfPTfl9rYajawAAAIAFPXXgDxoJmp1sE195SZH+/k8u0urF1u6WlioEHQAAAMCC7nzytSk9f/mCan2golRnzpyuj547S8sWVOf9cbXxCDoAAACABb3RdzKp59k0Ouvm7q9+pKCCzUTU6AAAAAAWMzwSVK8/8dk4kmRIam7yFHTIkQg6AAAAgKW0d/u0bPOjGg4kV59z7qwyNdYWRh1OLBxdAwAAALIkEDTU6e3TkYFBza5w6J0TQ/r6vXumdM/jJ0+laHW5jaADAAAAZEF7t08tbT3yjTuiZkvBabPiaRzakgg6AAAAQMa1d/u0obVLEw+nGSnoJn3lYtfUb5IHiHsAAABABgWChlraeiaFnFS5/EMEHYmgAwAAAGRUp7cv7LhaKhUX2bTs3Oq03DvXEHQAAACADDoykJ6QI0lfWDqv4NtKhxB0AAAAgAw6eOTdtN27sXZu2u6da2hGAAAAAGTIV3/Zqf98+Q9pubfb6dDSmqq03DsXEXQAAACADGj66W7tf2sgLfe2SWpu8nBsbRyCDgAAAJBmX/1lZ9pCjtvpUHOTR4217rTcP1cRdAAAAIApCAQNdXr7dGRgULMrRo+Pjd9ZaXvhrZQeV3NVlur//OnFOnpiKOLrYRRBBwAAAEhSe7dPt+zoUW//6U5qrkqHbrlqdIclEDT0nX/fl9LXvOWqRVq+cFZK75mPCDoAAABAEtq7fVrf2jXp8d7+Qa1v7dK2tXWqcBTrvVOBlLzejLJi/eCzF3JEzSSCDgAAAJCgQNDQjQ/sj3nNpgf26wtLz57ya5WXFmnbF5boowtncUQtAQQdAAAAIEHPvHZMx0+einnNOydP6fDx96b0OjZJ/+dPL9KK8z8wpfsUIoIOAAAAkKCOg8fMXWgYSb8G3dSmhqADAAAAJMxcgDlzZplmlBXH3f2Z6E/qztIP/2QxR9WmwJ7tBQAAAAC5pv5cc13PPrpglv50yZkJ3//pg0cTfg7CEXQAAACABC1bUK0ZZcUxr5kxfZr8753SnU8eSvj+Pv+gOr19Sa4OEkEHAAAASFiR3aYffPbCmNeMBA19+99fSPo1jgwMxr8IURF0AAAAgCQ01rq1bW2dnNMjl72/OxTQyeHkZ+jMrnAk/VwQdAAAAICkNXhcml6c+v5ebqdDS2uqUn7fQkLXNQAAAOB9gaChTm+fjgwManbFaNiI1fms09un3v7UHjGzSWpu8tBxbYoIOgAAAICk9m6fWtp65POfDi7xZtmkuo5mZlmxNn/2QmbnpABBBwAAAAWvvdunDa1dk6bj9PoHtb61S9etWqj5s8on7fI82tObsjV8qtalLV+oYycnRQg6AAAAKGiBoKGWtp6II0BDj9326IGxx0K7PCs/NEe/3pe6oPPn9fMJOSlE0AEAAEBB6/T2hR1Xi8f3/i7PZQtnRQxHyZhmF80HUoyuawAAAChoydbZ7D5wNGVraPjQbHZzUoygAwAAgIJmhXk1v+k5ovZuX7aXkVcIOgAAAChoS2uq5HY6lM39FJuklrYeBYKpOgwHgg4AAAAKWpHdpuYmjyRlLewYGq396fT2ZWkF+YegAwAAgILXWOvW1rV1cjmze4wt1XN5ChlBBwAAANBo2PnAGSVZXYMV6oXyBUEHAAAAkPRg1++173B/2u4/vdgW9WicTaPzeWgxnTrM0QEAAIDlDI8EdU/HIb3ed1LnVJXpmvr5KpmW+n9HHwga6vT26T9e9OmX//V6yu8vSY5iu3509UWaNs2mDa1dsklh83dC4ae5yUOL6RSyGYZh+dYO/f39cjqd8vv9qqyszPZyAAAAkEabd/bozie9Gt+AzG6T1q2o0abVnpS9Tnu3Ty1tPQkNC03WjLJi/eCzF0rSpNd0Ox1qbvKosdad9nXkA7PZgKADAAAAy9i8s0e37/ZG/fnXLksu7IR2bo4MDGpWeamePXRMP/7PV6ey1KRsW1unBo9rbC2zK0aPq7GTY57ZbMDRNQAAAFjC8EhQdz4ZPeRI0p1PevU3V3wooWNsmdy5iaelrUcNHpfqF1Rneyl5j2YEAAAAsIR7Og4p3rzMoDF6XUggaKjj4DE9tPewOg4emzRws73bpw2tXZYIORKzcjKJHR0AAABYwut9JxO6LtJOzfh6l0DQUEtbj6xWp8GsnMwg6AAAAMASzqkqM31daKdmYojp9Q9qQ2uXtq6tk3N6iWV2csZjVk5mcHQNAAAAlnBN/XzFq8m326QvfOScqDs1xvtfNz2wXzdufyENq5waZuVkDkEHAAAAllAyza51K2piXrNuRY32vnk87k5N38lTer3vvVQuLyWYlZM5BB0AAABYxqbVHn3tsppJOzt22+nW0rla47L4zEpm5WQQNToAAACwlE2rPfqbKz6kezoO6fW+kzqnqkzX1M8fayltlRqX2rmV6n6r3/T1966rT+NqMBFBBwAAAJZTMs2ur644N+LPltZUqaq8WH0nTmV4VeE+fv5s00Fn8VmVOsPBn96ZxNE1AAAA5JQiu02fufjMbC9D9Quq5XY6FK/iZvGZldqxcUVG1oTTCDoAAADIOas8rqy+vtvp0LJzq9Xc5JGkqGHntj+7SDu+QcjJBoIOAAAAcs47J4az+vqh7mmNtW5tXVsnlzO8bsjtdGjb2jp9pu6sLK0QHBQEAABATgkEDd36cE/WXv+6VR8M657WWOtWg8elTm+fjgwManbF6Kwc2khnF0EHAAAAOaXT2xd3jk66uCpLtXHleZMeL7LbVL+gOgsrQjQcXQMAAEBO6e3PTsixSbrlqkXs1OQIdnQAAABgOYGgEfEoWCBo6MGu32d8Pa7KUt1y1SIGfuYQgg4AAAAspb3bp5a2nrDjaW6nQ1dd5Na/PPd7HT+Z2fk51636oDauPI+dnBxD0AEAAIBltHf7tKG1S8aEx33+Qd2+25vRtRQXSf/w+Tp2cXIUQQcAAACWEAgaamnrmRRyMs0u6a5rLtVlF8xmFyeHEXQAAACQUdHqb7LZTW28b61aqE8smpP080O/35t9J/Qvz76pd4dGdHZVmW77H5foDAd/fmcK/00DAAAgY6LV3zQ3eTQ0EsziykaVFtm0ceXCpJ8f6feTpFfefle1t/yHFp9VqR0bV0x1mTCB9tIAAADIiFD9zcQQ0Osf1IbWLh06eiJLKztt/ceTbzoQ7fcbb9/v+3XVlieTXR4SQNABAABA2sWqvwk9dl/nG3JVOjK5rDBlJUX65uXJ7eYkUl+07/f9endwJKnXgXkEHQAAAKRdvPobQ1Jv/5CWL6jK3KIm+L9/dlHSuzmJ1hdd9y97knodmEfQAQAAQNodGTAXAl7vO5nmlUzmqizVtrVTayNt9vcLeeOd95J+LZgzpaCzefNm2Ww2/fVf/3XM63bt2qUlS5bI4XDo3HPP1bZt26bysgAAAMgxsyvMHUl77vXj6V3IBJ++eK6evvHyKc/KMfv7hZw9c/qUXg/xJR10nn32Wd1xxx1avHhxzOu8Xq9Wr16tFStWaM+ePbrpppv0zW9+U9u3b0/2pQEAAJBjltZUye10yGpTaf50ybyUzMoJ/X5m3fY/LpnyayK2pILOu+++qy9+8Yu68847NXPmzJjXbtu2TWeffbZ+/OMf64ILLtBf/uVf6i/+4i/0ox/9KKkFAwAAIPcU2W1qbvJIkmXCzoyyYi1bUJ2Se4V+PzO/2+KzKpmnkwFJBZ1rr71WV155pVatWhX32o6ODl1xxRVhj33yk5/Uc889p1OnTkV8ztDQkPr7+8O+AAAAkNsaa93aurZOrgR2PtLpB5+9MCW7OSGh3y/Wzg5zdDIn4Sh5//33q6urS88++6yp63t7ezVnTvhk2Tlz5mhkZERHjx6V2z35POTmzZvV0tKS6NIAAABgcY21bjV4XOr09unIwKBue+R3OnQssw0IXJWluuWqRVOuy4lk/O/3Zt8J/cuzb+rdoRGdXVWm2/7HJezkZFBC/02/+eab+ta3vqXf/va3cjjMJ3GbLTwpG4YR8fGQTZs26frrrx/7vr+/X/PmzUtkqQAAALCoIrtN9QuqtaPr9xkLOdXlJVpz8Vw1eFxaWlOV0p2ciUK/X/2Cav3Zh89O2+sgtoSCzvPPP68jR45oyZIlY48FAgHt3r1bW7Zs0dDQkIqKisKe43K51NvbG/bYkSNHNG3aNFVXRz4TWVpaqtLS0kSWBgAAAIsJBI2xnZvZFY6wgLFzn0/f/NcX0r6G2rkV+tsrF6U93MB6Ego6l19+ufbv3x/22Fe+8hV96EMf0g033DAp5EhSfX292trawh777W9/q0svvVTFxcVJLBkAAABWt3PfW/ruQ93qO3G6JtvtdIw1JPj6vV0ZWUePb0BLzplJyClACQWdiooK1dbWhj1WXl6u6urqscc3bdqkw4cP6+6775YkrV+/Xlu2bNH111+vdevWqaOjQ3fddZfuu+++FP0KAAAAsJLNO3t0+27vpMd9/kGtb+3SjLLM/cvuoCHd03FIX11xbsZeE9YwpYGhkfh8Pr3xxhtj39fU1Gjnzp164okndPHFF+vWW2/VT3/6U1199dWpfmkAAABk2c59voghZ7zjJyN33k2X1/sy2+wA1mAzQp0BLKy/v19Op1N+v1+VlZXZXg4AAAAiCAQNffh7j6rvxHC2lxLm5isvYEcnj5jNBinf0QEAAEBh6vT2WS7k2G3SNfXzs70MZAFBBwAAAClxZGAw20uYZN2KGpVM40/eQsTEIgAAAKTE7ArzcxZTzSZpfD2G3TYacjat9mRrScgygg4AAABS4p0sHVuz26R9zZ/Uvzz7hl7vO6lzqsp0Tf18dnIKHEEHAAAAUxYIGrr14Z6svHbQkPYf9tNwAGEIOgAAAJA0Glb+69Wj2t71e50cHtGH51frSx81tzPS6e2Tz5+9Gh0r1gchuwg6AAAAUHu3T9f/6ws6ORwYe+y3PUf0/d+8pL+KUesSCBrq9PZp535fppYaUTbrg2BNBB0AAIAC197t0/rWrog/MwyNDQCdGHbau31qaevJ6k6OJJWVFGlpTVVW1wDroUILAACggAWChpofejHudXc+6dXwSHDs+/Zunza0dmU95EjSrDNKVGS3ZXsZsBiCDgAAQAHr9Pbp7YGhuNcFDemejkOSRsNRS1tPWDvnbDp7Zlm2lwALIugAAAAUsESK+F/vOykp+40HJlpHtzVEQI0OAABAAUukiN8wDD2097AOvP1uGleUmNJpdn3sgx/I9jJgQQQdAACAAra0pkpzKkpNHV+755k3dM8zb2RgVeb95HMXU5+DiDi6BgAAUMCK7Da1rFmU7WUkzFVZqm1r69RY6872UmBR7OgAAAAUuMZat7atrZs0R8cq3E6Hbr7So5nlJToyMKjZFQ4traliJwcxEXQAAACgxlq3Gjwu/derR7W96/c6OTyiD5xRqn/ufDPbS9OP/uQiLV84K9vLQI4h6AAAAEDS6DG2FR/8gFa8X9z/o/94JcsrGnX0RPz6IWAianQAAAAQ0YEjA9legqTEOsMBIQQdAAAATLJzn0//8eLb2V6G3M7RehwgURxdAwAAyBGBoKFOb1/aC/Lbu336+r1dKb9vomySmps8NB1AUgg6AAAAOaC926eWth75/INjj7mdDjU3eVLaYjkQNHTLjhdTdr9kpeN3Q2Eh6AAAAFhce7dPG1q7ZEx4vNc/qA2tXdo6hXkyE3eJOg4eVW9/5ov/Z0wv1s++UKejJ4ZoH42UIOgAAABYWCBoqKWtZ1LIkSRDo8e7Wtp61OBxJRwMIu0SZcvx907JbrdpzcVnZnspyBM0IwAAALCwTm9fzCBiSPL5B9Xp7UvovqFdIiuEnJAjA9ZZC3IfQQcAAMDCzP7xn0hIGB4J6qYH90fcJcom2kgjlQg6AAAAFmb2j3+z17V3+/SR7z+qvhOnprKslLKJNtJIPWp0AAAALGxpTZXcTod6/YMRd2BsklwRQkKkVtSP9PRqfWv220aPF6oqoo00Uo2gAwAAYGFFdpuamzza0NolmxQWdqKFhEhNBlyVpeofHMnImqP56sdqtHO/L3xdtJFGmtgMw7Da8cxJ+vv75XQ65ff7VVlZme3lAAAAZJzZOTrRWlFn25bPXaw/vvjMjA09Rf4ymw3Y0QEAAMgBjbVuNXhcMUPC6LDPyK2os+17v3lZ06bZ1VjrVv2C6mwvBwWAoAMAAJAjiuy2mCFhy2MH1NtvzRbNqRhuCiSCrmsAAAB5oL3bp9sePZDtZUQV2mVqaetRIGjFPSfkG4IOAABAjgsEDbW09WR7GXElO9wUSAZBBwAAIMd1evvCmhRYXSLDTYFkEXQAAAByXK4FB7PDTYGpIOgAAADkuFwKDu4Iw02BdCDoAAAA5LilNVWaUVac7WXEZdPk4aZAutBeGgAAIAfEGrT5SE+vjp88ldX1lZcW6cRQQDYp4hyfmWXF2vzZC2ktjYwh6AAAAFhce7dPLW09YQ0H3E6Hbr7yAjnLSnTD9n1ZW9v0YrvW/9ECbVy5UI/09E5a54zpxfrK8vnauHIhOznIKJthGJZvZN7f3y+n0ym/36/KyspsLwcAACBj2rt92tDaFXGXxAr+dvUFWnfZuWPfx9p5AlLBbDZgRwcAAMCiQvNxrBpyJGnWGSVh3xfZbapfUJ2l1QCnEXQAAABSLFW7GrkwH8flnJ7tJQAREXQAAABSKFo9TXOTJ+FCfKvPx6FVNKyM9tIAAAApEqqnmbgL0+sf1IbWLrV3+xK6n5Xn49AqGlZH0AEAAEiBWPU0ocda2noUCJqvuFlaUyW30yGrRQm306Gta+toFQ1L4+gaAABACsSrpzEk+fyD6vT2mS7WL7Lb1Nzk0YbWrqjzaTLpq8vna5XHRSc15ASCDgAAQAqYradJpO4mEDTknF6iryyfr3997k29OxRIdnlTUlVerO9/hmGfyC0EHQAAgBQwW09j9rpITQ2yobq8RB2bLlfJNCoekFsIOgAAACkQqqfp9Q9GPGJmk+Qy2aXMSkNCv/eZWkIOchL/qwUAAEiBUD2NpEnNA0Lfm+lSFggaumXHi1kPOXab9P++QMMB5C6CDgAAQIo01rq1dW2dXM7w42muBLqU/fQ/D6i3fyhdSzRty+cv0erFhBzkLo6uAQAApFBjrVsNHpc6vX06MjCo2RUO013KNu/s0e27vRlYZXQzy4q1+bM0HkDuI+gAAACkWJHdZrqFdMjOfb6shpxP1bq0dtk5WnZuNa2jkRcIOgAAAFkWCBr6zvZ9WXv9r3z0HDVfVZu11wfSgRodAACALNvy2AG9OzSStde/YhHH1JB/CDoAAABZFAgaun33a1l7fbfJltdAriHoAAAAZEkgaOiu3a/p5HAga2u46iI3NTnIS9ToAAAAZFggaGjLYwd0e5ZDjiTdsdurS86eSZc15B2CDgAAQBoEgkbEFtPt3T7d+MB+HT95KttLHNPS1qMGj4udHeQVgg4AAECKtXf71NLWI59/cOwxt9Ohqy5y647dXhlZXNtEhiSff1Cd3r6EW2IDVkbQAQAASKH2bp82tHZNCjM+/2DWh4HGcmRgMP5FQA6hGQEAAECKBIKGWtp6LLVjY9bsCke2lwCkFDs6AACgoEWrpUlGp7cv7LhaLrBJctFiGnmIoAMAAApWtFqa5iZPUl3IevtzL+RIUnOTh0YEyDscXQMAAAUpVEszcQem1z+oDa1dau/2JXzPpw/8IVXLS7lLz5khtzP8eJrL6dDWtXW0lkZeYkcHAAAUnFi1NIZGdzoSbbnc3u3Tv3cdTuUyU+qa+vn648VzU3ZMD7A6gg4AACg48WppEm25HApOVja7wqEiu40W0igYHF0DAAAFx2wrZbPXWb0JQVV5Mc0GUHAIOgAAoOCYbaVs9jqrz6D5X2tqOaKGgkPQAQAABWdpTZXcToei/elv02j3NbO7IFaeQfO1y2q0evHcbC8DyDiCDgAAKDhFdpuamzySNCnsJNNyORScsmlGWXHY99XlJfp/X6jTptWeLK0IyC6aEQAAgILUWOvW1rV1k+bouOLM0QkEDT3z2jF1HDwmyVD9ubO0bEG1mps8Wt/alaHVh7NJckyz65//8iM6+u4QHdUAEXQAAECeCwSNqC2VG2vdavC4TLdcbu/26cYH9uv4yVNjj215/KAcxXbNry7LyO8TiSGpt39IdptNay4+M2vrAKyEoAMAAPJWe7dv0o6Ne8KOjdmWy+3dvqg7NoOngnq5993ULHoKrN4UAcgkanQAAEBeau/2aUNr16S2z73+QW1o7VJ7t8/0vQJBQ7fssPacHMnaTRGATCPoAACAvBMa4GlE+FnosZa2HgWCka6YrNPbp95+6+6WJNolDigEBB0AAJB34g3wNCT5/IPq9PaZup+Vj4Ql0yUOKAQEHQAAkHfMBhOz11npSNjELONyOrR1bV3ULnFAoaIZAQAAyDtmg4nZ65bWVMlV6cj68bVPXzxX//tPLtLzr79jqkscUMgIOgAAIO+EBnj2+gcj1unYNLoTYrampchu05qL3bp9tzel60zUn146TyXT7Ka6xAGFjqNrAAAg7xTZbWpu8kg6XcMSkkxNS3u3L+shZ2ZZsZadS8ABzCLoAACAnBYIGuo4eEwP7T2sjoPHxjqpNda6tXVtnVzO8ONpida0DI8EtemB/Slfd6I2f/ZCjqgBCeDoGgAAyFnxBoI21rrV4HGp09uXcE1LIGhoy2MHdMeTr+nEUCCdv0ZMzunT9MOrF9NsAEgQQQcAAOSUQNBQp7dPj/T06h+fPjTp56GBoKFdmyK7LeGalvZun258YL+OnzyVolUnr6xkmho8rmwvA8g5BB0AAJAzIu3gTGRotA6npa1HDR5XQse9RndxXtVtj/5u6otNkdC8HxoQAIkh6AAAgJzQ3u3ThtauiF3UJho/ENRsQGjv9umWHS+qt39oSutMBysPLAWsiqADAAAsLxA01NLWYyrkjGc2ICQSorLBSgNLgVxB0AEAAJYWCBr65dPemMfVojETEAJBQ7fseNGSISfReT8ATiPoAAAAyzJTkxNJIgHhH/7zgCWPqyUz7wfAaQQdAABgSckeJ0skIGze2ZP1QaCSVF5apIrSaWGByzWuTTaAxBF0AACA5SRbkyOZDwi/3nvYEiFHkk4MBXTHNZfKbrMlPO8HQGQEHQAAYDmd3r6Ej6t9dfl8rfK4TAWEnft82nj/3imsMPWOvjukNRefme1lAHmDoAMAACwnkXbK7gSPeLV3+/T1e7uSXVra0FkNSC2CDgAAsJxE/ui/+UrzIScQNHTdv+xNclXpQWc1ID3s2V4AAADAREtrquR2xg87Nkm3PtyjQNBcNc9tj7yi904Fp7i61KGzGpA+BB0AAGA5RXabmps8ca8zJPn8g+r09sW99nsP92jL4wdTsLrUcTkd2rq2js5qQBpwdA0AAFhSY61bX10+X3c9fSjuteNregJBQ53evrDuZf+7/SXd+aQ1OqzdfOUFmlVRSmc1IM0IOgAAwLJWeVymgk6opifSgFFXpUO9/Yl1cEunWRWldFcDMoCgAwAALCtUq9PrH4w4U2d8IX+0AaNWCjkS3dWATKFGBwAAWNb4Wp2JB7zGF/JLSnrAaCa56a4GZAxBBwAAWFpjrVtb19bJNaEL2/hC/mQGjGYD3dWAzOHoGgAAsLzGWrcaPK5JTQZCoSGRAaPZct2qhXRXAzKIoAMAAHJCkd2m+gXVEX8264zSDK8mMW6nQxtXLsz2MoCCwtE1AACQ84IB61bn2MSRNSAb2NEBAAAZFWnOTbIhIHSve/779RSvMjXcToeamzwcWQOygKADAADSZmKoeefEkG59+KWwxgFmw8DEex0bGNLftXWr78SpdP8aCakuL9Gai+eqweNiICiQRQQdAACQFpGGd0bS6x/UhtausQ5qU7lXpl1dd6b+16cv1N43j6dkhwpA6hB0AABAykUb3hmJodE6lpa2HjV4XJNCQiL3yrQ/WTJP00uKojZJAJA9NCMAAAApFQgaCQ/vNCT5/IPq9PZN+V6ZlAttrYFCRdABAAApNZXhneODQyBo6JdPey13XG282RWO+BcByAqOrgEAgJSayi5HKDhYtSYnxCbJ5RytxwFgTQQdAACQUsnscowPDlauyRmP2TiAtXF0DQAApNTSmiq5nQ6ZjQCh65qbPJJk6ZocSbLbpJ994RJm4wAWR9ABAAApVWS3jYUWM2HH5XToZ1+4RM7pJbrtkVcse1wtJGhIM8tLs70MAHFwdA0AAKRcY61bW9fWTaqzcTsduvlKj2aWl4wbIjqsWx+2bj1OJHRbA6yPoAMAANKisdatBo9Lnd6+qMM027t9uvZe69fjTES3NcD6CDoAACBtiuy2qMM0h0eCuunB7pwKOXRbA3IHNToAACDj2rt9Wrb5UfWdGM72Ukwb3zSBbmuA9bGjAwAAMmrnvrf09Xv3ZHsZCXM5HWpu8tBtDcgRCe3obN26VYsXL1ZlZaUqKytVX1+v3/zmN1Gvf+KJJ2Sz2SZ9vfzyy1NeOAAAyD079/m08b7cCTlV5cX66vL5um/dMj11w0pCDpBDEtrROeuss/SDH/xA5513niTpn/7pn7RmzRrt2bNHixYtivq8V155RZWVlWPff+ADH0hyuQAAIFe1d/v09Xu7sr2MSS5bOEvzZ5XrnKoyfeEj52jvm8ejNk8AkDsSCjpNTU1h33/ve9/T1q1b9cwzz8QMOrNnz9aMGTOSWiAAAMh9gaChlraebC8joq/90QItP2/W2PfRmicAyC1JNyMIBAK6//77deLECdXX18e89pJLLpHb7dbll1+uxx9/PO69h4aG1N/fH/YFAAByV6e3z7Jzco6+O5TtJQBIg4SDzv79+3XGGWeotLRU69ev14MPPiiPxxPxWrfbrTvuuEPbt2/XAw88oPPPP1+XX365du/eHfM1Nm/eLKfTOfY1b968RJcJAAAsxMoDNpmJA+Qnm2EYCbWvHx4e1htvvKHjx49r+/bt+vnPf65du3ZFDTsTNTU1yWazaceOHVGvGRoa0tDQ6X+70t/fr3nz5snv94fV+gAAgNzQcfCYPn/nM9lexiRup0NP3bCSOhwgh/T398vpdMbNBgnv6JSUlOi8887TpZdeqs2bN+uiiy7ST37yE9PPX7ZsmQ4cOBDzmtLS0rHObqEvAACQu5bWVGlGWXG2lzEJM3GA/DXlgaGGYYTtvsSzZ88eud20ZgQAANl13aoP0i4ayGMJdV276aab9KlPfUrz5s3TwMCA7r//fj3xxBNqb2+XJG3atEmHDx/W3XffLUn68Y9/rPnz52vRokUaHh5Wa2urtm/fru3bt6f+NwEAAJbV6e3T8ZOnsr2MMa7KUm1ceV62lwEgjRIKOm+//bauueYa+Xw+OZ1OLV68WO3t7WpoaJAk+Xw+vfHGG2PXDw8P69vf/rYOHz6s6dOna9GiRXr44Ye1evXq1P4WAADAUgJBQ53ePh0ZGNSsM0r11IE/ZHtJkqTQIbVbrlrEkTUgzyXcjCAbzBYcAQCA7AoEDW157FX94mmvjr9nnR2cELfToeYmD0fWgBxmNhsktKMDAAAQTXu3Tzc+sN9SR9TGqyov1s1XXkDIAQrElJsRAAAAtHf7tL61y7IhR5LeOXFK1967R+3dvmwvBUAGEHQAAMCUBIKGWtp6sr2MuEJn9VvaehQIWv7kPoApIugAAIAp6fT2yecfzPYyTDEk+fyD6vT2ZXspANKMoAMAAJIWCBp6+lVrdFRLxJGB3AhmAJJHMwIAAJCU9m6fWtp6cmY3Z7zZFY5sLwFAmhF0AABAwtq7fdrQ2qVcq3SxSXI5HVpaU5XtpQBIM46uAQCAhISaD1g95EwcBxr6vrnJw7BQoAAQdAAAQEJypfmAs6w47HuX06Gta+uYowMUCI6uAQCAhDza05vtJcRlk+SYZtc//+VHdPTdIc2uGD2uxk4OUDgIOgAAYEwgaKjT26cjA4MRw0EgaOiBPYezuEJzDEm9/UOy22xac/GZ2V4OgCwg6AAAAEmRu6i5nQ41N3nGjns9c/CY3jl5KltLTBhtpIHCRdABAABRu6j1+ge1obVLW9fWSZJu+Pd9mV/cFNBGGihcBB0AAApcrC5qhkbrXW7cvk/H3xvJ8MqSRxtpAHRdAwCgwMXromZIORdyJNpIA4WOHR0AAApcrtWxlJcW6XOXztMqj0vvnBjSrQ+/FBbUXBPqigAUJoIOAAAFLlfqWGaUFesrH63RxpXnhe3UfLLWHbNTHIDCRNABAKDALa2pktvpUK9/MGKdjhV8/ePn6m+u+FDEAFNkt6l+QXUWVgXAyqjRAQCgwBXZbWpu8kg6Xd9iNSsWzmaXBkBCCDoAAECNtW5tXVsnZ1lxtpcyiZvuaQCSQNABACCHDY8EddeTr+nvHurWXU++puGRYNL3avC4VFpkrT8NbKJ7GoDkUKMDAECO2ryzR3c+6VVwXGHN93a+pHUrarRptSfh+2157IDeHhhK4Qqnxk33NABTQNABACAHbd7Zo9t3eyc9HjQ09ngiYae926fbHj2QsvUlo7ykSNvWLlHfyWG6pwGYMmvtTwMAgLiGR4K688nJIWe8O5/0mj7GNjwS1Lf/7YVULG1KTgwHNK3IrjUXn6n6BdWEHABTwo4OAAA55p6OQ2HH1SIJGqPXfXXFuZN+Fgga6vT26a3j7+mhvYf19KtHFbBIX+lcG14KwLoIOgAA5JjX+04mfV17t08tbT3y+a0ZKHJleCkA6yPoAACQY86pKkvquvZunza0dll2KChtpAGkEjU6AABYRCBoqOPgMT2097A6Dh5TIMr5tGvq5yte+YrdNnrd+Hu3tPVYNuRItJEGkFrs6AAAYAGRjpRFa69cMs2udStqInZdC1m3okYl007/+8xOb59lj6tJ0nWrFtJGGkBKsaMDAECWhY6UTQwivf5BbWjtUnu3b9JzNq326GuX1Uza2bHbpK9dNnmOTm+/dUOO2+nQxpULs70MAHnGZhiGlXexJUn9/f1yOp3y+/2qrKzM9nIAAEiZQNDQx374WNTdFpskl9Ohp25YGfFY1/BIUPd0HNLrfSd1TlWZvvCRc7T3zeM6MjCo2RUOLTlnprY+cVB37HpVJ06ZazedadvW1rGbA8A0s9mAo2sAAGRRvCNlhiSff1Cd3j7VL6ie9POSafaxFtLt3T6t/D9PhN3PZpOs/K80r1v1QUIOgLQg6AAAkEVm58bEuy5aRzUrhxxXZak2rjwv28sAkKcIOgAAZJHZuTGzziiN+rNc6Kg2XugA3i1XLaLLGoC0oRkBAABZtLSmSm6nQ/H+3P+bf90bsSmBZP2OauUl4X9uuJwObaUuB0CasaMDAEAWFdltam7yaENrl2xS1F2Zt/uHtKG1K2JAMHv8LVv+v6tqNXdm2ViDhKU1VezkAEg7dnQAAMiQaANBG2vd2rq2TnMqox9jM97/uunB/RoeCe+eZvb4W7bMnVmm+gXVWnPxmapfUE3IAZAR7OgAAJAB8QaCNta6VVFarC/e9d8x79N34pSWbf5Pff8ztWM7O6Hjb73+QcvV6bgqS7W0pirbywBQgNjRAQAgzcwOBD16YsjU/fpODIc9L3T8TVLcWp9M+/zSs9nBAZAVBB0AANIoVke00GMtbT0KBI2Ej6CFniedPv5WOd1ahzXmzyrP9hIAFCiCDgAAaZTIQFCzHdgmPi9kaCQo/3sjU15zKlm9fghA/iLoAACQRokMBB1/BC3R+6+7+1l96/69iS4vbWwarUGiPgdAthB0AABII7M7GqHrQkfQqsqLTT1vVnmprv3n5/RIz5Gk15guzU0e6nMAZA1BBwCANIp3HC3SzkdjrVvPbFqlqvKSmPcuLy3S1+99Xg/vfzt1C06Rv171QQaCAsgqgg4AAGkUqyNa6PtIOx8l0+z6/mdqZYvwvJATQwHL1eSEzJ9Vlu0lAChwBB0AANIsdBzN5Qw/xuZyOrR1bV3UnY9oz8sFNCEAkG02wzCsNltskv7+fjmdTvn9flVWVmZ7OQAAJCUQNNTp7dORgUHNrhg9rmamhiUQNPTMwWN6+uAfdHfH63p3KJCB1SbHptEA99QNK6nPAZAWZrOBtZrtAwCQx4rsNtUvqE74eY/09KqlrSdmm2oriHUUDwAyjaADAICFtXf7tKG1K+LAUatxOR1qbvLQhACAJRB0AACwqEDQUEtbj6VDznWrFmr+rPKEjuIBQCYQdAAAyJJ4NTud3j5LH1ebWVasjSsXEm4AWBJBBwCALGjv9k2qu3FPOPrV22/dkCNJ75w8pU5vX1J1RwCQbrSXBgAgw0J1NxN3a3r9g9rQ2qX2bp8k6ejAUDaWl5AjA9YOYwAKF0EHAIAMilV3E3qspa1HgaChh/e9lcmlJYV5OQCsiqNrAABkULy6G0OSzz+of/jPA9r7e3/mFpag0LycpTVV2V4KAETEjg4AABlk9qjXnU+9luaVJI95OQByATs6AABk0KGjJ01dd2IokOaVJI95OQByAUEHAIAMCQQN3df5RraXkRTm5QDINQQdAAAypNPbZ/mW0RNVlRfr+5+5kN0bADmHGh0AADIkF1sx3/zHiwg5AHISQQcAgAzJxVbMrsrcWzMASAQdAAAyZmlNlWaUFWd7GaZVlRfTPhpAzqJGBwCANAkEDXV6+3RkYFCzKxxacs5MRZwUalH/a00tTQcA5CyCDgAAadDe7VNLW0/YcNCZZcU6/t6pLK7KvK9dVqPVi+dmexkAkDSCDgAAKdbe7dOG1q5JmzfvnLReyKkqL1HfieGx76vLS3TrmlqtXkwDAgC5jaADAEAKBYKGWtp6LH9CzabRwZ+7/ucn9Pzr74wdr2NGDoB8QdABACCFOr19YcfVrCgUY5qbPCqZZlf9guqsrgcA0oGgAwBACuXCrByX06HmJg/zcQDkNYIOAAApZOVZOX9/9YU6q6qc42kACgJBBwCAFHrnxLDsNilowSKds6rKOaYGoGAQdAAASJH2bp+uvXdytzWryIVjdQCQKgQdAADimDj4M9LRr1zotjarvDTbSwCAjCHoAAAQQ6TBn+4Ixfy50G3tb/7tBd1yFU0IABQGe7YXAACAVYUGf04MML3+QW1o7VJ7t0/S6G7OU6/+IRtLTMjb/eHrBoB8xo4OAAARxDqKZmh0Fk1LW4+CQUO3PvyS5XdzpPB1N3hcdF4DkNfY0QEAIIJ4R9EMST7/oL5+756cCDkhoXV3evuyvRQASCuCDgAAEeR7h7J8//0AgKADAEAEVh78mQr5/vsBAEEHAIAIltZUye10KJeqWD58zkzd9mcXqaq8JOq6bRrtGre0piqTSwOAjCPoAAAQQZHdpuYmjyTlTNi5YpFLn6k7S9//TK2kyesOfd/c5KERAYC8R9ABACCKxlq3tq6tk7OsONtLMWXWGSWSTq/b5Qw/nuZyOrR1bR1zdAAUBNpLAwAQh//kqWwvwRSXc/rYf26sdavB41Knt09HBgY1u2L0uBo7OQAKBUEHAIAohkeCuunB7oizdKwmUt1Nkd2m+gXVWVoRAGQXR9cAAIigvdunZZsfVd+J4WwvJS6bqLsBgInY0QEAYIL2bp82tHZZcidnRlmxjo87Sud2OtTc5KHuBgAmIOgAADBOIGiopa3HciEnFGiouwEAcwg6AACM0+ntk88/mO1ljPmL5fPV4HGFBRrqbgAgPmp0AAAYp7ffOiFHkn7T3cuuDQAkgaADAMD72rt9uvXXL2Z7GWF8/kF1evuyvQwAyDkcXQMAFLxA0NCWxw7otkcPZHspER0ZsNYuEwDkAoIOAKCgtXf7dMuOHssdWRtvdoUj20sAgJxD0AEAFCwrt5EOCQ0CDQQNuq0BQAIIOgCAgmTVNtITfe7DZ+uRnl61tPWEdYNjfg4AxEYzAgBAQbJaG+lojp8c0obWrklr7fUPakNrl9q7fVlaGQBYG0EHAFCQHu3pzfYSTPn3rt9H3HUKPdbS1qNA0Or7UgCQeQQdAEDBae/26a6nD2V7GaYMDAai/swQ7acBIBqCDgCgoAyPBHXTg/uzvYyUov00AExG0AEAFIz2bp+Wbf5P9Z04le2lmFJdXmLqOtpPA8BkBB0AQEFo7/ZpfWuX+k4MZ3spcdne/7p1Ta3cToeiNZG26XT7aQBAONpLAwDyQqQ5M9Jod7Ve/3va9IB1j6sV2aVA8PT3rnGto+12aUNrl2xSWFOCUPhpbvIwTwcAIiDoAAByXnu3b9KcmRllxZKk4yetf0xtVnmJ/u//uERH3x2aNAy0sdatrWvrJv1+LuboAEBMBB0AQE5r7/ZpQ2vXpBbMuRBwQt4eGJbdZtOai8+M+PPGWrcaPK5JO1bs5ABAdAQdAEDOCgQNtbT1RJwzk2vidU4rsttUv6A6Q6sBgNxHMwIAQM7q9PaFHefKZXROA4DUYkcHAJCz8mF+jE2j9TbjO6dFaqzAMTUASAxBBwCQs3J9FyRS57RIjRXcNB4AgIRxdA0AkLOW1lTFnDNjdS6nQ1vX1o0FmFBjhYnH8Xr9g9rQ2qX2bl82lgkAOYkdHQCApcU6xlVkt6m5yRNxzoxV3XzlBZpVUTrpd4nVWMHQ6O5PS1uPGjwujrEBgAkEHQCAZZk5xhWaM/Ptf9und4dGsrXUuEK1OF9eXhMxqMRrrGBI8vkH1ento/saAJjA0TUAgCWZPcYVCBp6pXfA8iFHCq/FmejRnl5T98qHBgwAkAns6AAALMfsMa7g+9e9PTCU4RUmxhWnmUAgaOjBvYdN3SvXGzAAQKYQdAAAlmP2GNfX792TuUUlKFotTiSd3j71nTgV957V5SVhbagBANERdAAAlpPrx7PcMWpxIjH7+665eC6NCADAJGp0AACWk+vHs2LV4kRi9vdt8LiSXRIAFByCDgDAcnJ1Po7dJv2/L9QlPNjTzO/rdjo4tgYACSDoAAAsJzQfJ9d8Y+V5Wr04sZAjhf++E8OO7f2vRHeJAKDQEXQAAJbUWOvWX11Wk+1lJOTcD5yR9HND84BczvBjbC6nQ1vXJr5LBACFjmYEAABLCgQN7XjBl+1lJGSqtUWNtW41eFzq9PbpyMCgqY5tAIDICDoAAEuK12LaalJVQ1Nkt6l+QXUKVgQAhS2ho2tbt27V4sWLVVlZqcrKStXX1+s3v/lNzOfs2rVLS5YskcPh0Lnnnqtt27ZNacEAgMKQay2mqaEBAGtJKOicddZZ+sEPfqDnnntOzz33nFauXKk1a9boxRdfjHi91+vV6tWrtWLFCu3Zs0c33XSTvvnNb2r79u0pWTwAIH/NOqM020swbcvnLqaGBgAsxmYYhjGVG1RVVenv//7v9dWvfnXSz2644Qbt2LFDL7300thj69ev1wsvvKCOjo6o9xwaGtLQ0NDY9/39/Zo3b578fr8qKyunslwAQI54+sBRffGu/872MkxxOx1qbvIQdgAgA/r7++V0OuNmg6S7rgUCAd1///06ceKE6uvrI17T0dGhK664IuyxT37yk3ruued06tSpqPfevHmznE7n2Ne8efOSXSYAIIcEgoY6Dh7TQ3sP655nDmV7Oab1+ge1obVL7d251TwBAPJZws0I9u/fr/r6eg0ODuqMM87Qgw8+KI8n8qyD3t5ezZkzJ+yxOXPmaGRkREePHpXbHfnffG3atEnXX3/92PehHR0AQP5q7/appa0npxoQhBganXXT0tajBo+LWh0AsICEg87555+vvXv36vjx49q+fbu+9KUvadeuXVHDjs0W/g/70Em5iY+PV1paqtLS3DmbDQCYmvZunza0dmlKZ6mnYH6VQ4f6phawDEk+/6A6vX10TQMAC0g46JSUlOi8886TJF166aV69tln9ZOf/ES33377pGtdLpd6e3vDHjty5IimTZum6mr+nwAAYPS4WktbT9ZCjiQdOxn9OHWicq1bHADkq6RrdEIMwwhrHDBefX29HnnkkbDHfvvb3+rSSy9VcXHxVF8aAJAHrDAvZ2AwkLJ7TXVoKAAgNRIKOjfddJOefPJJHTp0SPv379ff/u3f6oknntAXv/hFSaO1NX/+538+dv369ev1+uuv6/rrr9dLL72kf/zHf9Rdd92lb3/726n9LQAAOavX/162l5ASNqVuaCgAYOoSOrr29ttv65prrpHP55PT6dTixYvV3t6uhoYGSZLP59Mbb7wxdn1NTY127typ6667Tj/72c80d+5c/fSnP9XVV1+d2t8CADBlgaChTm+fjgwManbF6B/s6SqqD73Wb1/06Z8730zLa2RS6L8lhoYCgHVMeY5OJpjtlQ0ASE6kjmfpmg3T3u3TLTteVG9/5GPPuYg5OgCQOWazQcLNCAAA+SVax7PQbJita+tS9gd8e7dP61u7UnIvq7j5ygv05eU17OQAgMVMuRkBACB3xep4Fnqspa1HgeDUN/8DQUM3PrB/yvfJpFjRJVSTQ8gBAGtiRwcACli8jmepnA3zzMFjOp7CNs7pYJNUVV6i7155gVzO6XrnxLCuvXd0B8qYcJ1ETQ4AWBlBBwAKmNmZL6mYDdPx2tEp3yOdQnHle5+pDTuqt9VeN6l+yUVNDgBYHkEHAAqY2ZkvqZgNY/XWN9HCS2OtWw0eV8Y60gEAUoOgAwAFbGlNldxOh3r9gxHrdGwaDQBTmQ0zPBLUPR2HdM8zh5K+R6q5nQ7dfOUFmlleaiq8FNltUz66BwDILIIOABSwIrtNzU0ebWjtkk2prUMZHgnqz//xv/XMa32pWGpKbPzEAi0/7wPsyABAAaDrGgAUuMZat7aurZPLGX48zeV0JN1a+nsP9+iD3/2NpUKOJC2cU6H6BdWEHAAoAOzoAABSWoey7u5n9UjPkTSscupSUWsEAMgNBB0AgKTU1KHs6Pq9JUNOKmqNAAC5haNrAICU+PXet/TNf30h28uYhJk3AFCY2NEBAEzZ5p09un23N9vLkCR9oLxYfzhxejApM28AoDARdAAACQsEjbF6Hu8fTlgm5Hztshp9p/ECZt4AAAg6AADzhkeCuumBfdq5v1cnTwWyvZwwS86eoU2rPZLEzBsAAEEHAGDO9x7u0c+f9EYcLGoFB4+eUCBosHsDAJBEMwIAgAnr7n5Wd1o45EjS8ZOn1Om11tweAED2sKMDAIgqEDT040dfsWTL6Eh6/e9lewkAAIsg6AAAJIU3GJhd4dA7J4b0//36JfX2D2Z7aabd+vBLml5SRIc1AABBBwBy3cSAkkyXsfZun1raeuTz506oieSdE8Pa0NqlrWvrCDsAUOAIOgCQwyIFFHeCc2Pau33a0Npl6fobswyNDghtaetRg8dFYwIAKGA0IwCAHBUKKBN3YXr9g9rQ2qX2bl/cewSChlraevIi5IQYknz+QRoTAECBI+gAQA6KFVBCj7W09SgQjB1hOr19OX9cLZojA/n5ewEAzCHoAEAOihdQ4u1qBIKGOg4e029M7PrkqtkVjmwvAQCQRdToAEAOMrtbEem6fGk8EIvbOdqUAQBQuAg6AJCDzO5WTLxu57639PV796RjSZbS3OShEQEAFDiOrgFADlpaUyW306Fof8rbNHlXY+c+nzbel/8h57pVH6S1NACAoAMAuajIblNzk0eSJoWd0PfjdzXau336+r1ditObIOe5Kku1ceV52V4GAMACCDoAkKMaa93aurZOLmf48TSX0xE2MDPUoc3KimySq3JqzQNskm65ahFH1gAAkqjRAYCc1ljrVoPHpU5vn44MDGp2xehxtfF/7OdCC+lPX3KmGjxztL61K6nnV5UX6/ufuZAjawCAMQQdAMhxRXab6hdUR/251efJ2CRt/uxilUyza9vaOt34wH4dP3kqoXvc/MeLCDkAgDAEHQDIc1afJ/NXl9WoZNroSerQDtUzrx1Tx8Fj+v07J/WrvW/FvcdUj70BAPIPQQcALOa94YC+v7NHh46d1PzqMt202qPpJUVJ3y/Uoc2Kx9dW187RptWesMeK7DYtP2+Wlp83S4Ggof/29qnXP6hIfRRsGq1JYmYOAGAimhEAgIWsu/tZXfB37brnmTf05IGjuueZN3TB37Vr3d3PJn3PIrtNN195QQpXmRql0+z6hy8siXlNot3lAAAIIegAgEX85T89q0d6jkT82SM9R6YUdpxlJUk/N13M7lKZ7S4HAMB4HF0DAAvY0XVYj74UOeSEPNJzRO8NB5I6xtZx8FiyS0ub4ydPqdPbF7ORQoiZ7nIAAIxH0AGALGvv9umb/7rX1LXf39mjWz99YRKvYs1JoYl0hIvXXQ4AgPE4ugYAWZToMM9Dx04m9Tr1585K6nnpZvWOcACA3MWODgBkUaLDPOdXl4395+GRoO7pOKTX+07qnKoyXVM/f6xN80TLFlRrRllxwvNp0oVuaQCAdCPoAEAWJTrM86b3WzFv3tmjO5/0KjjuRNr3dr6kdStqwto1B4LGWF3Ln3/kHP308VdTsu6pMkS3NABAehF0ACCLEjm61eCZreklRdq8s0e37/ZO+nnQ0Njjm1Z71N7tU0tbjyXn5/zF8vl0SwMApBVBBwCyKDTMM9pAzJBVF8zWnX/+YQ2PBHXnk5NDznh3PunVhXNn6Bv377FoCwKpweOK+fPxO1F0WAMAJIOgAwBZFBqIuaG1SzZF7o320z+7WFfVnSlJuqfjUNhxtUiChrTpV/stG3Kqyou15JyZUX8eaSfK7XSoucnDLhAAwDS6rgFAlkUbiOl2OrRtbd1YyJGk1/vMdV0bGBxJ6RpTqe/EKf3R3z+u9m7fpJ+1d/u0obVr0nG7Xv+gNrR2RXwOAACRsKMDABZgdiDmOVVlUe6QW0LBZevaurFdmlCr7Ug7UYZGO7W1tPWowePiGBsAIC52dADAIkIDMddcfKbqF1RH/GP+mvr5yoe/8UNhpqWtR4H3z+LFa7VtSPL5B9Xp7Uv/AgEAOY+gAwA5pGSaXetW1GR7GSkxMbiYbbWdaEtuAEBhIugAQI75TuMF+uPFbuXBxo6k08HFbKvtRFpyAwAKF0EHAHJIe7dPH/vhY/r1Pp8lu6qdUVqkpfOjd1SLJBRcQq22owU4m0YbNCytqZraIgEABYGgAwA5IlpHMiv48/pzdN+6ZXqh+ZP64rJzTD9vfHAJtdqWNCnshL5vbvLQiAAAYApBBwByQKyOZFbwqVr3WAOFRI6WTQwu0Vptu5yOsA5tAADEQ3tpALCoQNDQM68dU8fBY/r9OyctuZNj02gIGX+cLHQErdc/GDWY2W3Sls9HDi5mW20DABALQQcALKi926cbtu+T/z3rDv4MmbgrEzqCtqG1SzYpYtjZ8vlLtHpx9N2ZUKttAACSxdE1ALCY9m6f1rd2WT7kuGMcJ4t2BM3tdGjb2jqtXjw3U8sEABQodnQAwEICQUM3bt+f7WXEdfOVF+jLy2tiHifjCBoAIJsIOgBgIc8cPKbj753K9jLieuv4e6YCC0fQAADZwtE1ALCQjteOZnsJptz19CG1d/uyvQwAAKIi6ACApeTOsa6Wth4FglZteA0AKHQEHQDIokDQUMfBY3po72F1HDymj4xr02x1Pv+gOr192V4GAAARUaMDAFnS3u1TS1tP2HwcV2WpykuKdGI4kMWVmXdkwHqzfQAAkAg6AJAV7d0+bWjtmjRj5u3+oahDNq1odoUj/kUAAGQBR9cAIMMCQUMtbT0RA42h0SodxzTr/+PZ7RxtFw0AgBVZ//+TAkCe6fT2hR1Xm8iQNDgSzNyCktTc5GEmDgDAsji6BgAZZuW6Fptk6ujcdasWqrHWPenxQNBgQCgAwBIIOgCQYVata7lm2dm6ofECXfb3j6vvxHDU69xOhzauXDjp8UjNFdxOh5qbPBFDEQAA6cTRNQDIsKU1VXI7HZaamON2OlR/brUabtsVNeTY3v+KdGQt1Fxh4pG8Xv+gNrR2MVwUAJBxBB0AyLAiu03NTR5LdVf748UuXXvvnpi1Qy6nQ1vX1k3anYnXXEFiuCgAIPM4ugYAWRAMGiqdZteQRZoObO86HDN4VZUXa9f//IRKInSDM9NcITRctH5B9dQXCwCACQQdAMiwzTt7dPtub7aXEabvxKm4P3/+9XfGgsr4pgMH3h4w9RpWbsIAAMg/BB0ASJHhkaDu6Tik1/tO6pyqMl1TP3/SDsjOfT7LhRyzQkElUtMBM6zahAEAkJ8IOgCQApt39ujOJ70aX4byvZ0vad2KGm1a7ZE0ugvy3Ye6s7TCyGySqspLdCxGl7WQ2RWOsaYDiVTb2DRa38NwUQBAJtGMAACmKHQUbWKtfdCQbt/t1eadPZJGa1litW3OllvX1MbsAmfTaFe2JefMjNp0IJrQPRkuCgDINIIOAEzB8EhQdz4Z+yjanU96NTwS1KM9vRlalTlV5cXaurZOqxe71dw0uus0MYqMDyrPv/5OwsfVonVqAwAg3Ti6BgBTcE/HoUk7ORMFDemmB/br37t+n5lFmVBdXqKOTZeP1RA11rq1dW3dpNob17iBnw/tPWzq3hs/cZ4WzjlDsytGj6uxkwMAyAaCDgBMwet9J01dt+OFt9K8ksR87zO1kxolNNa61eBxjXVTmxhUzDYTWH7eLNpIAwCyjqADAFNwTlWZqeuGA9aYl2O3SVs+H/0oWZHdFjWkLK2pktvpUK9/MGKdDk0HAABWQo0OAEzBNfXzZaWTWbb31xJtSVs+f4lWLz4dcgJBQx0Hj+mhvYfVcfCYAjHO4RXZbaZqeTiqBgCwAnZ0AGAKSqbZtW5FjWVm4/zVihpdcvbMSbU27nG1NiGR5uFEum48M7U8AABYgc0wjEQ6hWZFf3+/nE6n/H6/Kisrs70cAAUmEDQm1a1ICnvssZd7dddT4Y0JbBrdBRmJ160gBew2TZrZE63WRlLUeTihK+J1Sot3fwAA0sVsNmBHBwBiiLTrMaOsWCMBQ+8OjYw95nY69JPPXaIj/YN6ve+kTg6N6N+7Dmck5EjSV5bPHws5Uuxam0DQiDoPx9Bo2Glp61GDxxU1vMS6PwAAVkCNDgBEEdr1mDg75vjJU2EhR5J8/kF98749OnPmdDU3LdKjLx/J5FL1i6cPaXjEXMODTm9fzHk4hkZ/n05vX4pWBwBA5hF0ACCCWLse0Rga3Qn5r1eP6vjJU+laWkRBY3SmjxlHBswN/TR7HQAAVsTRNQCIIN6uRzQ+/6C2Z2kw6KFjJ9Vx8Fjcuhmz83DMXgcAgBURdAAggqnsZuz+3R9SuBLz/u35N3XPM6+PfR+tgxrzcAAAhYCjawAQwVR2M/oyfGwtZPBUeI1Or39QG1q71N7tC3uceTgAgEJA0AGACEK7Hrn8p35ot6alrWfSINDQPByXMzzQuZyOuK2lAQDIBRxdA4AIQrseG1q7ZJMSakpgJeM7qE1sB91Y61aDx8U8HABAXiLoAEAUoV2PG7bvk/+9kfhPSLOr686UY5pdRwaGZMjQoy+ZrwWKVnPEPBwAQL4i6ABAHNkOOa7KUt1y1aKw42QP7T2cUNChgxoAoNAQdABYWiBoZO1oVWiWjhUlElzcdFADABQggg4Ay2rv9qmlrSdsnk20lsnpkOwsnVR7u39IG1q7wpoExGsRPV6mOqhlM5QCADCRzTAMy9fY9vf3y+l0yu/3q7KyMtvLAZAB7d0+bWjtmvRHfOjP5nR2BhseCeqejkN64nd/0JMHjqblNRIVmm3z1A0rx8JD6L8jKXKzhBllxfrBZy/MSCjMdigFABQOs9mAoAPAcgJBQx/74WNRd1Mi/dEfet5UdxQ27+zRnU96FbToPxnvW7csrHlApIAxo6xYX/lojTauPC8jOyrZDKUAgMJjNhtwdA2A5cQ7MhapZXIqdhQ27+zR7bu9U1p7uk3snpbtFtGhOqZIudDQaNhpaetRg8fFMTYAQEYRdABYTrRWyNGui7aj0OsfnFTbMt74HaCqshLdYfGQI0VuQpDNFtHJhFIAADKBoAPAcsx2FJtd4Uh6RyHSDpCVhY7rWa17WqKhFACATLFnewEAMFGoo1i0g042nW6ZnMiOQkhoByiXQo6Uue5piUgklAIAkEkEHQCWU2S3qbnJI0mTws7EP/oT3VGItQNkVS6nw7IF/YmEUgAAMomjawAsqbHWra1r6yYdL3NNaDCQ6I6CVWbjxOKqLNXnl56t+bPKLT+PJhRKN7R2yabwNtdW3okCAOQ/gg4AyzLTUSze4MyJtS1WrxXZ+InzdF3DB3MqGJgNpQAAZBJBB4ClxesoluiOgtVrRZafNyunQk5ItttcAwAwEUEHQM5LZEdhaU2VziidpneHRrKx1Kis2lUtEdlscw0AwEQEHQB5IdqOgiR1HDw29tiSc2ZqOBDM8mojo5YFAIDUIegAyBsTdxR27vPpuw91q+/E8NhjjmK7hkesFXRmTC/WD66+kFoWAABSiKADIC9t3tmj23d7Jz0+eMpaIUeSfvbFOi0/b1a2lwEAQF4h6ADIOzv3vRUx5FhNqC5n2bnUtQAAkGoEHQBpFwgaGevGFQga+u5D3Wm5dyoxYwYAgPQi6ABIq/Zu36RuaO4UzVeJFKA6vX3qO3FqqstOuRnTi3X8vdPrYsYMAADpRdABkDbt3T5taO2aNMjT5x/U+tYubVtbl/Qf+tEC1AXuiimsOH1+9sU62W02ZswAAJAhBB0AaREIGmpp65kUcsa78YH9avC4Ev6DP1aAGh98rML9fh0OwQYAgMyxZ3sBAPJTp7cvbug4fvKUtjz2akL3DQQN3fjA/pgBymqowwEAIPMIOgDS4siAuZ2VX/yXV4Fg5NgSCBrqOHhMD+09rI6DxxQIGtry2AEdP2m9Gpxo/uHzl1CHAwBAFnB0DUBazK5wmLru+MlT6vT2hQ36lCLX4LgqHeofzJ2Qs27FfDVdNDfbywAAoCARdACkxTsnhmSTTB0xm7j7E60Gp7ffOvU3xXZp48oP6t2hU7rrKa/Gb0rZbdK6FTXatNqTvQUCAFDgCDoAUq6926dr791juo5m/O6PmSYGVvCPX16qFR/8gCTpf37yQ7qn45Be7zupc6rKdE39fJVM42QwAADZRNABkLRIc2wkmQ4qNklzKks1EgjqR//xsiSbnNOLLdk5baKj7w6N/eeSaXZ9dcW5WVwNAACYiKADICnR5th87sPzTAcVQ1L/4Iiu+cfONK0yffpODGd7CQAAIAaCDoCERa2h8Q/qtkcPmLpHWUmRTg4HdHI4kPoFZkDVGaXZXgIAAIiBQ+QAEhKrhiaRuprSHK9hcVWa6yoHAACyI6G/NDZv3qwPf/jDqqio0OzZs/XpT39ar7zySsznPPHEE7LZbJO+Xn755SktHEB2mBkEGotNUlV5sd7JoVk4E7mdp+uRAACANSUUdHbt2qVrr71WzzzzjB555BGNjIzoiiuu0IkTJ+I+95VXXpHP5xv7WrhwYdKLBpA9ZgeBSqOhJtL3n7n4zJStJ5Ns7381N3lUZJ/42wEAACtJqEanvb097Ptf/OIXmj17tp5//nlddtllMZ87e/ZszZgxI+EFArAWs4NAr1v1Qd3/7BvhAz+dDjU3eeScXqK7nj6UphWmT2j9jbXubC8FAADEMaVmBH6/X5JUVRX/CMcll1yiwcFBeTweffe739UnPvGJqNcODQ1paOh069b+/v6pLBNACi2tqZLb6VCvfzBiTY5No4Fg48rztHHleZPaTxfZbQoEDbkqS9XbPxThDtbx1Y/N18oPzVHHwWOSDNWfO0vLFlRne1kAAMAEm2EYSc3lMwxDa9as0TvvvKMnn3wy6nWvvPKKdu/erSVLlmhoaEj33HOPtm3bpieeeCLqLtAtt9yilpaWSY/7/X5VVlYms1wAKRSt65o0GnS2rq2Lu+vR3u3T+tautKxvqs4onab/ffVi2e2K2EKbXR0AALKnv79fTqczbjZIOuhce+21evjhh/XUU0/prLPOSui5TU1Nstls2rFjR8SfR9rRmTdvHkEHsJDNO3t055NeBcf9E8Ruk9atqNGm1R5T92jv9unGB/br+ITGBDablNw/mabub1dfoL/4WI0e6emNGOZClTlmwhwAAEg9s0EnqaNr3/jGN7Rjxw7t3r074ZAjScuWLVNra2vUn5eWlqq0lBkVgFW1d/t0x27vpBBgGNIdu7265OyZpkJAY61bDR6Xnjl4TB2vHZVkU/2CavlPDuvae/ck1K46FcpLivQXH6uRpJgttG3v/7zB46IpAQAAFpVQ0DEMQ9/4xjf04IMP6oknnlBNTU1SL7pnzx653fybUMBKAkEjYj1NpOtSGQKK7DYtXzhLyxfOGnusvdsnZ1nxpJ2edPvLFTUqstvUcfBYzBbahiSff1Cd3j7VU7MDAIAlJRR0rr32Wt1777166KGHVFFRod7eXkmS0+nU9OnTJUmbNm3S4cOHdffdd0uSfvzjH2v+/PlatGiRhoeH1draqu3bt2v79u0p/lUAJKu922e6FiXeHJ2phIBA0NCWxw7otkcPJPS8VCgvKdI3L/+gJPMttBNptQ0AADIroaCzdetWSdLHP/7xsMd/8Ytf6Mtf/rIkyefz6Y033hj72fDwsL797W/r8OHDmj59uhYtWqSHH35Yq1evntrKAaREtMYCvf5BbWjtmlSLkq4Q0N7t0y07etTbn53w8H/+7KKxHSizLbTNXgcAADIv4aNr8fzyl78M+/473/mOvvOd7yS0KACZkcwxtHSEgFhd3NJtZtk0ffaSs+ScXqJA0FCR3Wa6hfbSmvit9QEAQHbYs70AANmTyDG0kFAIiFZ9Y9PosTezISBW2EqHmWXF+uevfkR/sXy+qspL9M7JEd319CF9/s5n9LEfPqb2bp+K7DY1N412jpv4e4a+b27y0IgAAAALI+gABSyZY2ipDAGBoKFfPu2NGbZS7Z2Tp/Tc6336xdOH1HdiOOxnoeN67d0+Nda6tXVtnVzO8J0pl9NBa2kAAHJAUu2lAeSHZI+hhULAxAYGLpPDNIdHgrrpgf3aud+nk6cCiS98in7x9CFTx/VC7a/NdKMDAADWQtABCthUalGSDQGbd/bojie9WRsIKknH34vetnpi17giu40W0gAA5CCOrgEFbKrH0EIhYM3FZ46Fglg27+zR7buzF3JskmaUFZu6ltbRAADkNoIOUOAyVYsyPBLUnU96U3KvZIQi2Fc+am7QMa2jAQDIbRxdA5CRWpR7Og4pmMXjaqH6oQaPS/c/+watowEAyHMEHQCSlPZalNf7Tqbt3vHcfOUF+vLymrHg1tzk0YbWLtmksLBD62gAAPIHR9cAxBQIGuo4eEwP7T2sjoPHFEhyW2bezLIUryy+0Eyf8SFHytxxPQAAkD3s6ACIqr3bN6mFtNvp0M1XejSzvMT0Mbf2bp9++JuXMrHkSaLtztA6GgCA/GYzjGw2eTWnv79fTqdTfr9flZWV2V4OUBDau33a0NoVsY5lIneM+Tnt3T6tb+1K/QLjmFk2TZs/u5jdGQAA8ozZbMDRNQCTBIKGWtp6TIUcSer1D2pDa5fau31jz+84eEwP7jmsb//bC+lbaBRnlE7Tf9/UQMgBAKCAcXQNwCSd3r6w42rxGBqth2lp61EwaOjWh19K6Pmp9qM/XaySafx7HAAAChl/CQCYJJlhmYYkn39QX793T9ZCzhkldl23aqGGRoJTapwAAAByHzs6ACbJ1WGZZziKddujB8a+j1U7BAAA8hs7OgAmWVpTpRnTi7O9jIT19g+Ffz+hdggAABQOgg5QQMzMxAkEDXV6+9I6PDRTQr9dS1sPx9gAACgwHF0DCkS0mTjjj3ZFuibXhWqH8iW8AQAAcwg6QAGINhMndLRr69o6STI9NycXJdNgAQAA5C6CDpDnYs3ECbWFvmXHi5JsORdyqstL9Of183Xbo7+Le22uNlgAAADJIegAeSpUa/P0q0djHkUzNLmIPxdUlRerY9PlKrLbdP+zb6jXPxgxqNkkuZwOLa2pyvQSAQBAFhF0gDyUrlqbspIinRwOpPSeybBJ+v5nLhwbCtrc5NGG1i7ZpLCwY3v//zY3eVRktwkAABQOuq4BeSZUj5OOhgJWCDkzphdr69q6sNk4jbVubV1bJ5cz/Hiay+mYdC0AACgM7OgAeSRWPU40NklzKksl2fR2f+TjX5Jkt0lW6ND8sy/Wafl5syY93ljrVoPHpU5vn44MDGp2xehxNXZyAAAoTAQdII90evsS2skJRYBbrlokSVGPfxnKfsgJ1dosOzd6i+giu40W0gAAQBJH14C8kmgL5fFHu2Id//rq8vkpXGXiqLUBAACJYkcHyCNmWyhv/MQCLT/vA5OOdkU7/tXp7dNdTx9K06pHxToa55ow2BQAACAegg6QR5bWVMntdMRttXxdw/lRd0YiHf9acs7MtNfoBA3p5isv0KyKUs06o1QypKMnhqi1AQAASSHoAHmkyG5LutVyaO7OxEL+QNDQPR2HMlKjM6uiVGsuPjP9LwQAAPIeQQfIM6Fam4lzdGId/4o0d8ftdOiqi9x6aK9Pvf2pb1UdidmjdwAAAPHYDMOwQMPY2Pr7++V0OuX3+1VZWZnt5QA5IRA09Mxrx9Rx8JgkQ/XnztKyBdWTdnNCc3ey+Q+C0JG6p25YyRE1AAAQk9lswI4OkKce6ekN26XZ8vhBuSfs6iQzdydd6KgGAABSifbSQJ4JBA395NHfaX1r16SZOr3+QW1o7VJ7t09S4nN30sE9rsU1AABAqrCjA+SR9m6fbtnRE7WmxtDoMbGWth41eFwJz91JtetWLdTGlQvZyQEAAClH0AHyhNlaG0OSzz+oTm9fxor/zyidpneHRsa+n3iEDgAAINUIOkAeSKbW5sjAoP548dyYc3dSwTl9mp792wY9//o7k1pXAwAApAtBB8gDydTazK5wjM3dWd/alaaVST+8erFKptknDSEFAABIJ5oRAHkgkVobm0aPji2tqZIkrfzQnLSsyVVZqm00GQAAAFnCjg6QBxKptTEkra51qdPbp6U1Vbqn41DK1rHxE+dp4ZwzOJ4GAACyjqAD5IGlNVWmam3sNiloSHc9fUh3PX1IbqdDC2efkbJ1LD9vFkfUAACAJXB0DcgDoVobafRoWjTBCSnI5x/U7gNHU7KG8cfhAAAAso2gA+SJxlq3tq6tk8sZfozNVVmqGWXFaX/9m6/0cFQNAABYBkfXgDzSWOtWg2e0/ibUyjloGPriz/877a/tnJ7+MAUAAGAWQQfIM0V2W1idzEN7D2fkdTteO6rlC2dl5LUAAADi4egakOcS6cg2NRxbAwAA1sGODpDjAkEj7KjaxLbOj738dkbWQbc1AABgJQQdIIe1d/vU0tYjn//0wFC306HmJo8aa916bzignz/lTfs6ZpYVa9m5BB0AAGAdBB0gR7V3+7ShtWvS3Jxe/6A2tHbpry6r0b2db8qINVgnRTZ/9kI6rgEAAEsh6AA5KBA01NLWE3E4aOix23enfydn/O4RAACAlRB0gCyJV1sTS6e3L+y4WqbNKCvWzz5fp2ULqtnJAQAAlkTQAbIgXm1NPEcGshNyQpHmB5+9kFbSAADA0mgvDWRYqLZm4o5MqLamvdsX9x6ZaxkdzuV0aOvaOo6qAQAAy2NHB8igeLU1NkktbT1q8LhiHglbWlMlt9OhXv9gxHulyhmlRVq3YoHmzypL+HgdAABANhF0gAyKV1tjSPL5B9Xp7QubSxOpnqe5yaP1rV1pW2uRTeq6+QqVTGPjFwAA5B6CDpBBZmtrnn71D2O7J9Hqea66KL3Hx7700fmEHAAAkLMIOkAGma2t2fL4QW3vOqyrLnLrjt3eiLNy0t0+usHjSuv9AQAA0ol/XQtkUKi2xkyVi+/9MBNrVk66uJ2jx+MAAAByFUEHyKAiu03NTR5JMhV2Ms32/ldzk4emAwAAIKcRdIAMa6x1a+vaOrmc2WkRHQvtowEAQL6gRgfIgsZatxo8Lt32yO+05fFXs70cSdLNV16gLy+vYScHAADkBXZ0gCwItYtOf7WNebMqSgk5AAAgb7CjA2RYpHbRVmC2IxwAAEAuIOgAGdTe7dOG1i4L7eOMossaAADINxxdAzIkEDTU0tZjuZAj0WUNAADkH4IOkCGd3j7LHVeTpH/4/CV0WQMAAHmHoANkyJEB64WcdSvmq+miudleBgAAQMpRowNkiNli/5uvvECzKko164xS3f1fr+k/ev6Q8rXYbdK6FTXatNqT8nsDAABYAUEHyJB3TgzHvcbtdIzNsgkEDa1vfT4lr23TaCPrP/rgB3TZwlm6pn6+SqaxoQsAAPIXQQfIgEDQ0K0P98S97uYrLxhrCtDp7dPA4EhKXt/ldKi5yUMtDgAAKBgEHSADzDYimFleOvafp1rTU1Zs1+eXnq1VHpeW1lTRVQ0AABQUgg6QAWZDy/jrpjrA87NLztLNTYumdA8AAIBcxSF9IAPMhpbx1y2tqZLbmXzYsWn0yBwAAEAhIugAGbC0pkozyopjXjOzrFhLa6rGvi+y29Tc5FGyB87ueeYNfeyHj6m925fkHQAAAHIXQQewiEh7L421bm1dWydXZXI7O73+QW1o7SLsAACAgkPQATKg09un4ydPxbzm+MlT6vT2TXq8sdatp29cqetWLUz4dUPhqaWth2NsAACgoBB0gAzo9b83peuK7DZ9a9UHtW1t3aS6nenFsT/GhiSffzBiiAIAAMhXdF0DMqDPxLBQM9c11rrV4HGp09unIwODml3hUG//oK77l71x7z3VdtUAAAC5hKADZEDVGaXxL9Jo0AkEjZgzb4rsNtUvqB77vuPgMVP3nmq7agAAgFzC0TUgA944dsLUdT974mDCndKWnDNT8WaB2m2j1wEAABQKgg6QZoGgofs63zB9faKd0p5//R3F6zMQNEavAwAAKBQEHSDNOr196u0fMn19op3SzNbeUKMDAAAKCUEHSLNHe3oTfk4indLM1t5QowMAAAoJQQdIo/Zun+56+lDSzzezC7O0pkpup0PRynRsktxOh5bWVCW9DgAAgFxD0AHSJBA01NLWM6V7mNmFKbLb1NzkkaRJYSf0fXOTJ2YnNwAAgHxD0AHSpNPbJ58/+bqYGdOLTe/CNNa6tXVtnVwThom6nA5tXVunxlp30usAAADIRczRAdKk1//elJ7/pY+ek9AuTKRhoktrqtjJAQAABYmgA6RJ34nhKT3/X579vS5wVya0GzNxmCgAAECh4ugakCZVZ5RO6flv9yc2TwcAAACnEXSANHFVTq2dc6LzdAAAAHAaR9eQdoGgkdN1I4msf3gkqHs6Dun1vpOaN3O6XJUO9fYn35Bg/DwdjqQBAACYR9BBWrV3+9TS1hPWfcztdKi5yZMTncDau31qfqhbbw+crreZU1GiljW1k9a/eWeP7nzSq/GbL6mKc2bm6QAAAOA0jq4hbdq7fdrQ2jWpxXKvPzdqT9q7fVrf2hUWciTp7YFhrW/t0s59b409tnlnj27fHR5ypNPHz0qnTe2jZmaeDgAAAE4j6CAtQsMyI1WW5ELtSSBo6Pp/fSHmNV+/d4/aXnhLT/7uD7pjtzfmtUMjQX3SM0eOCIHHFmPbx6bRHTCz83QAAAAwiqNrSIt4wzKtXnvyXweO6uRwIO5137hvj+l7/kfP2xEfN6JkvVD+aW7y5FRNEwAAgBWwo4O0MFtTYtXak+17fp/x15yYZVxOh7aurcuJWiYAAACrYUcHaWG2psSqtSdmdnNSLWhIN195gWZVlOZkdzoAAAArYUcHabG0pkpupyNq1zGr1558eH521jWrolRrLj5T9QuqCTkAAABTQNBBWhTZbWpu8kia3GI5F2pPvvTR+TGbBKTL0YEhyzZoAAAAyCUEHaRNY61bW9fWyeUMP56WC7UnJdPs+qsVNRl/3Vsffkkf++Fjlm+9DQAAYHU2w4jW88k6+vv75XQ65ff7VVlZme3lIEGBoKFOb5+ODAzmXO3J5p09umO3N2Kb7HQJ/Tdj9TAIAACQDWazAc0IkHZFdpslW0hPFCmQfafxAv1qz1t6e2AoY+swNBp2Wtp61OBx5UwoBAAAsBKCDgpKtN2l9m6fWtp6wmb/uJ0Ofe7D81IecmZMn6ZVF8zRv3cdjnqN1ecMAQAAWB1BByll5WNq0cLMVRe5Ix5P6/UP6rZHD6Ts9ctLivRXl52rjSsX6tf73ooZdEKsOmcIAADA6gg6SJloQaK5yZP1WpP2bp82tHZFDDO37/ZGfE6q6nI+VevS2mXnaNm5p1tG5/qcIQAAAKuj6xpSIhQkxoccaTRIbGjtymoXsUDQUEtbT8Tgks4mA26nQ9vW1mnr2iVaft6ssJ2tXJ8zBAAAYHXs6GDK4gWJbBfWd3r7JgWwdKkqL9ZnLj5TqzyumMf2QnOGNrR2yabwwJULc4YAAACsjqCDKYsXJLJdWJ/OOpd1K2q08kNzkqpJCs0Zmnjcz2WR434AAAC5jKCDKTMbJLJVWJ/OOpdf7/Ppxk9dkPTOS2OtWw0el2UbOAAAAOQqgg6mzOqF9UtrqlRVXqy+E6dSfu9U7FTlypwhAACAXEIzAkxZOgvrA0FDHQeP6aG9h9Vx8JgCwcTbBxTZbfrMxWcm/DyzaAENAABgPezoYMrSVVifynbVqzwu3fX0objX3XzlBZpVUaqjA0O69eGXTN2bFtAAAADWw44OUiJUWO9yhv/R73I6tHVtXcLBJNXtquPtOkmjIerLy2u05uIz9eXlNXJVlsa9Ly2gAQAArIkdHaRMqgrr09GuOtauU8h7pwJ6pKdXjbVuFdltuuWqRVrf2hXzvrSABgAAsCZ2dJBSocL6NRefqfoF1UmFgETaVYfEq+UJBA05p5foK8vnq7w0cr4/fvKU1rd26da2F9Vx8JgaPC5tW1unGWXFk66dWVasbUnsVAEAACAz2NGB5STarjpeLU+kn8dy19OHdNfTh+SqLNUtVy3S899t0DMHj6njtaOSRoPcsnOTC3EAAADIDIIOLCeRdtWhWp6JR9FCtTx/dVmN7tjtjXhULZ7e/iGtb+0a27lZvnBWEncBAABANnB0DZZjtl31knNmxqzlMSTd+WRyIWe8Gx/Yn1RbawAAAGQPQQeWE2ocIGlS2Bnfrvr519+JexwtFfnk+MlTeubgsanfCAAAABlD0IElmWlXnclBnaP1OQAAAMgVCQWdzZs368Mf/rAqKio0e/ZsffrTn9Yrr7wS93m7du3SkiVL5HA4dO6552rbtm1JLxiFo7HWraduWKn71i3TTz53se5bt0xP3bByrNNZZgd10ngAAAAglyQUdHbt2qVrr71WzzzzjB555BGNjIzoiiuu0IkTJ6I+x+v1avXq1VqxYoX27Nmjm266Sd/85je1ffv2KS8e+S9Wu2ozQ0DtttRElPoF1Sm4CwAAADLFZhhG0lUMf/jDHzR79mzt2rVLl112WcRrbrjhBu3YsUMvvfTS2GPr16/XCy+8oI6ODlOv09/fL6fTKb/fr8rKymSXizwU6romhQ8BDYWbv1wxX3c+eWhKrzGzrFjPfbeBdtIAAAAWYDYbTKlGx+/3S5KqqqqiXtPR0aErrrgi7LFPfvKTeu6553Tq1KmIzxkaGlJ/f3/YFxBJrFqev7qsRr/e1zvl19j82QsJOQAAADkm6Tk6hmHo+uuv18c+9jHV1tZGva63t1dz5swJe2zOnDkaGRnR0aNH5XZPniy/efNmtbS0JLs0FJjGWrcaPC51evvU639PfSeG9fvj7+n23d6E7uOcXiz/e6fD9/ihowAAAMgtSQedjRs3at++fXrqqafiXmuzhf/b8NBpuYmPh2zatEnXX3/92Pf9/f2aN29esktFASiy2+R/b1j/+z9eidtyOpr/98U62W02HRkY1OwKh5bWVLGTAwAAkKOSCjrf+MY3tGPHDu3evVtnnXVWzGtdLpd6e8OPDx05ckTTpk1TdXXkAu/S0lKVlpYmszQUqFCtTrIFZ2eUTtOyc6sJNgAAAHkioRodwzC0ceNGPfDAA3rsscdUU1MT9zn19fV65JFHwh777W9/q0svvVTFxcWJrRYYJxA01HHwmB7cc1g3Pbg/6ZAjSe8OjeiRnqnX8wAAAMAaEtrRufbaa3XvvffqoYceUkVFxdhOjdPp1PTp0yWNHjs7fPiw7r77bkmjHda2bNmi66+/XuvWrVNHR4fuuusu3XfffSn+VVBI2rt9amnrSfqYWiQ3PrBfDR5XxF2dQNBQp7ePY20AAAA5IqGgs3XrVknSxz/+8bDHf/GLX+jLX/6yJMnn8+mNN94Y+1lNTY127typ6667Tj/72c80d+5c/fSnP9XVV189tZWjYO3c59PX7+1K+X2PnzylZw4e0/KFs8IejxSqaFQAAABgbVOao5MpzNFByM59b2njfXsUTNP/ajd+YoG+/ckPjX0frfYntJezdW0dYQcAACCDMjJHB8ik9m6fvn5v+kLOqNPH0QJBQy1tPRFrf0KPtbT1KJDeBQEAACAJBB1YXiBo6OlXj+rG7fvT/lr1C053Auz09sWsATIk+fyD6vT2pX1dAAAASEzSc3SATEhH04FoZpYVa9m5p4POkQFzr2n2OgAAAGQOQQeWNdXZOIna/NkLwzqpza5wmHqe2esAAACQORxdgyXFqo9JNbfToW0RmgosramS2+lQtCbStvefu7SmKu1rBAAAQGLY0YElxauPicRmk8z2EKwuL9Gai+eqweOKOhOnyG5Tc5NHG1q7ZJPCQlfo6uYmD/N0AAAALIigA0vq7U+87uVnn79Edrst4sybm6/0aGZ5ScIDPxtr3dq6tm7SPV3M0QEAALA0gg6yKhA01OntmxRA+t4dSvheL/z+uDat9qjB44p4z2Q11rpTfk8AAACkF0EHWROpo5r7/Z2SqvKShO93+26vLjprplYvdoe1iU6FIrst5fcEAABA+hB0kBXROqr1+ge1obVLNbPKkrrvzQ9165O1LnZbAAAAChxd15BxsTqqGe9/vXb0ZFL3PnZimAGeAAAAIOgg85LpqJYIBngCAACAo2uYkmjNBGJJdxA5lORuEAAAAPIHQQdJi9VMIFbb5dkVjrSu6/5n39DGledRpwMAAFDAOLqGpISaCUw8gubzD2p9a5d+8ugBBYKRp3curamS2+lQumKIzz9InQ4AAECBI+ggYbGaCYTc9ujvtPwH/6n2bt+knxXZbWpu8qRvgaJOBwAAoNARdApcIGio4+AxPbT3sDoOHou6CzOe2WYCvf1D2tDaFTHsNNa69bMv1KVtVyfdx+MAAABgbdToFLBoNTY3X+nRzPKSqA0GEt0taWnrUYNn8mybnd2+mLtCybBJcjlH1wwAAIDCRdApUNEGdvr8g/r6vV1hj01sMJDIbomh0zUz9Quqxx4fHgnq4X2Td3qmIhSjmps8NCIAAAAocBxdK0BmamzG6/UPhh1BS6aZwMRdoHs6DqV8N8fldGjr2rqYHd8AAABQGNjRKUCJDuw0NLpbMv4IWnOTRxtau+I9dczEXaDX+1Iz6+bmKy/QrIpS0zN8AAAAUBgIOgUomY5kE4+gNda6tXVtnW7Z0aPe/tj3c79fMzN+uKhhTH0/x+106MvLawg3AAAAmISgU4Cm0pFsfEhqrHWrwePSlscO6LZHD0R9znunAvrf7S9pxwu+hHaSoqEWBwAAAPFQo1OApjKwc2JIKrLb9K1VH9S2tXWaUVYc8TnHT57S7bu9KQk5ErU4AAAAiI8dnQI0vsbGJpluClBVXhy1bXODx6VbdryYsjVO9LerL9DsSmpxAAAAYA47OgUqVGPjcpo/xvaZi8+MGjA6vX3q7R9K1fImmV1ZqjUXn6n6BdWEHAAAAMRF0ClgjbVuPXXDSt185QWmrl/lcUX9WTINDhIxlboiAAAAFB6CToErstv05eU1MWt2bDrdOS2aQ0dT0y46klhH5gAAAIBICDoYq9mRNCnsmOlw1t7t048f/V3a1ve/1tRyXA0AAAAJIehAUvSanXgdzgJBQy1tPaYbGiTqa5fVaPXiuWm6OwAAAPIVXdcwJjQXJzTU00yHs05v35TbRl+3aqHuf/bNsPtUl5fo1jW1Wr2YFtIAAABIHEEHYYrsNtUvqDZ9fSqaEMyfVa6nbliZUMACAAAAYiHoYEpS0Q1tdoUj4YAFAAAAxEKNDqZkaU2VyoqLkn5+dXkJHdUAAACQcgQdTEmR3abVF0afrxPPrXRUAwAAQBoQdDBl3//sYtmSyCqjHdVoNgAAAIDUI+hgykqm2fVXK2oSft4lZ89Mw2oAAAAAgg5SZNNqj752WY3MnkKzSWpp61EgmK4JPAAAAChkBB2kzKbVHr1866d085UX6ArPnJjXGpJ8/kF1evsyszgAAAAUFIJOgQsEDXUcPKaH9h5Wx8FjU95hKZlm15eX1+iDcypMXZ+KOTwAAADARMzRKWDt3T61tPXI5z8dNtxOh5qbPGqsjd4kIBA0og73jHTPWFIxhwcAAACYiKBToNq7fdrQ2qWJ+ze9/kFtaO3S1rV1EcNOpCBTVV6s/7WmVna7LeI9I7FJcjkdzNABAABAWhB08kSsXZZI17a09UQMJIZONwpo8LjC7hEtHPWdOKWv37tHZSVFpkJOSHOThxk6AAAASAuCTh5I9Ahap7cv5tGy8Y0C6hdUS4odjkJODgdMrdduk9atqIl5PA4AAACYCpoR5LjQLsvE4BI6gtbe7Zv0HLMNAMZfFy8cJcIwpDt2eyOuDQAAAEgFgk4Oi3cETYo8q8ZsA4Dx16WyO1qstQEAAACpQNDJYYkcQRtvaU2V3E6HolXH2DR69G18o4BUd0djjg4AAADSiaCTw5I5giZJRXabmps8kjQp7IS+n9goYGlNlc4oLUp2qabXBgAAAKQCQSeHmd1lmVVeOmkoaGOtW1vX1snlDL+Hy+mI2Fr6kZ5evTsUv9nAzLJiuSpLU/47AAAAAImg61oOCx1B6/UPRqzTsUlylhXrb/7tBfX2R+7I1uBxxW1LHaoFMmPzZy9Ug8elZw4e07X3dun4e6ciXsccHQAAAKQTOzo5LN4RNEPS8ZOnwkKOFN6RrchuU/2Caq25+MyxVtITd3/Mdly7btVCNda6VWS3afnCWfrB1RfKFmVtEnN0AAAAkD7s6OS40BG0iXN05lSWanAkqOMnJ++oRBsKGm0ez+pal6m1zJ9VbmptrhgzfgAAAIBUsBmGYfn+vv39/XI6nfL7/aqsrMz2ciwptPMSOoIWNAx98ef/Hfd5961bpvoF1WPzeCb+jyG0M2RG6F7x1hbpeBwAAABghtlswI5OnggdQQt5aO9hU887MjBoah6P3TY66DNaLVCsepuJawMAAADSjaCTpxIZCmqmBifaXE/qbQAAAGBFNCPIU4kMBTU7y6Zk2uT/ucwoK47YjhoAAADIJoJOnjIzFPRzHz5bv973lo4ODJm65/BIcNJj70RodgAAAABkG80I8kCsYv9IndRmlhWPtZ4Osb1fg5OoUH3OUzes5OgaAAAA0o5mBHkmWpiJ1hI61L554lDQQ0dP6LZHD0y6f7Jx15Dk8w+q09tHwwEAAABYBkEnB0QLM1dd5NYdu72TOqH1+ge1vrVL161aqPmzyseCkSR97IePpWWNZut8AAAAgEwg6KRRKubHRJtv4/MP6vbd3ojPCV07fufG7XTocx8+O253tWSZ7fIGAAAAZAJBJ03iHSmbaHgkqHs6Dun1vpM6p6pM19TPV5HdFnW+TaJ6/YO67dHfpeBO4eLN0AEAAACygaCTBtF2YXr9g9rQ2jWpHfP3Hu7Rz5/yhtXJfG/nS1p9oTtlOzDp6DjBDB0AAABYFe2lUywQNKLuwoQea2nrUeD9CZzr7n5Wdz7pndQMIGhIv97nS+taY6kqLwn7fmZZsWaUFYc95nI6mKEDAAAAS2JHJ8U6vX0xd2HGdyk7NjCoR3qOZG5xJrmdDu36n5/Q86+/E1ZfJGnKNUcAAABAJhB0Usxs97He/kH93UP707yaxNk0ehStZJo9YrtoWkgDAAAgFxB0Usxs97G+d4c0MBhI82oSE6tZAgAAAJBLCDoptrSmSm6nQ73+wYh1OqEuZRNrYGK5+pK5enDvWwqmo6OApD+vP0efqnVzFA0AAAB5g2YEKVZkt6m5ySPpdFeykPFdylzO6abuZ5O0+eqLtOXzdSlb40SfqnWrfkE1IQcAAAB5g6CTBo21bm1dWyeXM/wY2/guZaGdn3jKSor02Mtva/Vit65btTCl67Rp9LgaM3AAAACQbzi6liaNtW41eFxRu5SFdn4izdsZ7+RwQOtbu3TdqoU6dmI4ZetjBg4AAADymc0wJk5wsZ7+/n45nU75/X5VVlZmezkp1d7t0y07XlRv/1BGX5fGAwAAAMhFZrMBOzpZ1ljrVoWjWF/8+X+n/bWqyot18x8vkquSGTgAAADIbwQdCzj6bvp3c2ySvv+ZC9nBAQAAQEEg6FiA2dk7yeKYGgAAAAoNQccC4s3eScY1y85W3dkz5XJO55gaAAAACg5BxwLGd2CzSVMKO6GBpLdcVUu4AQAAQMFijo5FRJu9kwhaRgMAAACj2NFJQCBoRJ2LkwoTZ+88+bs/6N+7Dpt+votaHAAAAEASQce09m6fWtp65PMPjj2WjiL/IrtN9QuqJUl/vHiuHn35iI6fPBXxWpukqvISfffKC6jFAQAAAMbh6JoJ7d0+bWjtCgs5kuTzD2p9a5d+8ujvFAimfu5qkd2mH3z2wog/C8WZ732mVp+pO0v1C6oJOQAAAMD7CDpxBIKGWtp6YjYIuO3RA1r+g8fU3u1L+es31rq1bW2d3BNqd1xOh7aureOYGgAAABCBzTCM1G9FpFh/f7+cTqf8fr8qKysz+todB4/p83c+Y+pamzQpfKSqrifd9UEAAABALjCbDajRiePIwGD8i8ZpaetRg8elIrstpXU942t3AAAAAMTG0bU4ZleYb/dsaLRup9PbF7Wup9c/qA2tXWk55gYAAABgFEEnjqU1VXI7HUrkkFiv/72odT2hx1raetLSwAAAAAAAQSeuIrtNzU2ehJ7Td2J40k7OeON3fgAAAACkHkHHhMZat7aurZOrsjTmdTaN1uBUnRH7upCnXz3Krg4AAACQBgQdkxpr3Xr6xst13aoPRvx56Ghbc5NHrkpzdT1bHn9VH/thetpSAwAAAIWMoJOAIrtN31q1MO5cm0TqemhOAAAAAKQec3SSFG+uTajrmqSYw0al0d0gl9Ohp25YyWwcAAAAIAaz2YAdnSSF5tqsufhM1S+onhRQxup6nPGPsdGcAAAAAEgtgk4aNda69dQNK7XxEwtMXZ/ocFIAAAAAkRF00qzIbtPy8z5g6tpEhpMCAAAAiI6gkwHxmhOE2lIvranK5LIAAACAvEXQyYDxQ0cnhp3xbalpRAAAAACkBkEnQ6I1JxjflhoAAABAakzL9gIKSWOtWw0eV8y21AAAAACmjqCTYaG21AAAAADSh6NrAAAAAPIOQQcAAABA3iHoAAAAAMg7BB0AAAAAeYegAwAAACDvEHQAAAAA5B2CDgAAAIC8Q9ABAAAAkHcIOgAAAADyDkEHAAAAQN4h6AAAAADIOwQdAAAAAHmHoAMAAAAg7xB0AAAAAOQdgg4AAACAvEPQAQAAAJB3CDoAAAAA8g5BBwAAAEDeIegAAAAAyDsJB53du3erqalJc+fOlc1m069+9auY1z/xxBOy2WyTvl5++eVk1wwAAAAAMU1L9AknTpzQRRddpK985Su6+uqrTT/vlVdeUWVl5dj3H/jABxJ9aQAAAAAwJeGg86lPfUqf+tSnEn6h2bNna8aMGQk/DwAAAAASlbEanUsuuURut1uXX365Hn/88ZjXDg0Nqb+/P+wLAAAAAMxKe9Bxu9264447tH37dj3wwAM6//zzdfnll2v37t1Rn7N582Y5nc6xr3nz5qV7mQAAAADyiM0wDCPpJ9tsevDBB/XpT386oec1NTXJZrNpx44dEX8+NDSkoaGhse/7+/s1b948+f3+sDofAAAAAIWlv79fTqczbjbISnvpZcuW6cCBA1F/XlpaqsrKyrAvAAAAADArK0Fnz549crvd2XhpAAAAAAUg4a5r7777rl599dWx771er/bu3auqqiqdffbZ2rRpkw4fPqy7775bkvTjH/9Y8+fP16JFizQ8PKzW1lZt375d27dvN/2aodN1NCUAAAAAClsoE8SrwEk46Dz33HP6xCc+Mfb99ddfL0n60pe+pF/+8pfy+Xx64403xn4+PDysb3/72zp8+LCmT5+uRYsW6eGHH9bq1atNv+bAwIAk0ZQAAAAAgKTRjOB0OqP+fErNCDIlGAzqrbfeUkVFhWw2W7aXE1WoacKbb75JXVGe4b3NX7y3+Yn3NX/x3uYv3tv8ler31jAMDQwMaO7cubLbo1fiJLyjkw12u11nnXVWtpdhGg0U8hfvbf7ivc1PvK/5i/c2f/He5q9UvrexdnJCstKMAAAAAADSiaADAAAAIO8QdFKotLRUzc3NKi0tzfZSkGK8t/mL9zY/8b7mL97b/MV7m7+y9d7mRDMCAAAAAEgEOzoAAAAA8g5BBwAAAEDeIegAAAAAyDsEHQAAAAB5h6ADAAAAIO8QdBKwe/duNTU1ae7cubLZbPrVr34V8/onnnhCNptt0tfLL7+cmQXDlM2bN+vDH/6wKioqNHv2bH3605/WK6+8Evd5u3bt0pIlS+RwOHTuuedq27ZtGVgtEpHMe8vn1vq2bt2qxYsXj03Yrq+v129+85uYz+HzmhsSfW/5vOamzZs3y2az6a//+q9jXsfnNveYeW8z+bkl6CTgxIkTuuiii7Rly5aEnvfKK6/I5/ONfS1cuDBNK0Qydu3apWuvvVbPPPOMHnnkEY2MjOiKK67QiRMnoj7H6/Vq9erVWrFihfbs2aObbrpJ3/zmN7V9+/YMrhzxJPPehvC5ta6zzjpLP/jBD/Tcc8/pueee08qVK7VmzRq9+OKLEa/n85o7En1vQ/i85o5nn31Wd9xxhxYvXhzzOj63ucfsexuSkc+tgaRIMh588MGY1zz++OOGJOOdd97JyJqQGkeOHDEkGbt27Yp6zXe+8x3jQx/6UNhjX/va14xly5ale3mYAjPvLZ/b3DRz5kzj5z//ecSf8XnNbbHeWz6vuWVgYMBYuHCh8cgjjxh/9Ed/ZHzrW9+Kei2f29ySyHubyc8tOzoZcMkll8jtduvyyy/X448/nu3lIA6/3y9JqqqqinpNR0eHrrjiirDHPvnJT+q5557TqVOn0ro+JM/MexvC5zY3BAIB3X///Tpx4oTq6+sjXsPnNTeZeW9D+LzmhmuvvVZXXnmlVq1aFfdaPre5JZH3NiQTn9tpabkrJElut1t33HGHlixZoqGhId1zzz26/PLL9cQTT+iyyy7L9vIQgWEYuv766/Wxj31MtbW1Ua/r7e3VnDlzwh6bM2eORkZGdPToUbnd7nQvFQky+97yuc0N+/fvV319vQYHB3XGGWfowQcflMfjiXgtn9fcksh7y+c1d9x///3q6urSs88+a+p6Pre5I9H3NpOfW4JOGp1//vk6//zzx76vr6/Xm2++qR/96Ef8A9iiNm7cqH379umpp56Ke63NZgv73jCMiI/DGsy+t3xuc8P555+vvXv36vjx49q+fbu+9KUvadeuXVH/IObzmjsSeW/5vOaGN998U9/61rf029/+Vg6Hw/Tz+NxaXzLvbSY/txxdy7Bly5bpwIED2V4GIvjGN76hHTt26PHHH9dZZ50V81qXy6Xe3t6wx44cOaJp06apuro6nctEEhJ5byPhc2s9JSUlOu+883TppZdq8+bNuuiii/STn/wk4rV8XnNLIu9tJHxeref555/XkSNHtGTJEk2bNk3Tpk3Trl279NOf/lTTpk1TIBCY9Bw+t7khmfc2knR9btnRybA9e/aw3WoxhmHoG9/4hh588EE98cQTqqmpifuc+vp6tbW1hT3229/+VpdeeqmKi4vTtVQkKJn3NhI+t9ZnGIaGhoYi/ozPa26L9d5GwufVei6//HLt378/7LGvfOUr+tCHPqQbbrhBRUVFk57D5zY3JPPeRpKuzy1BJwHvvvuuXn311bHvvV6v9u7dq6qqKp199tnatGmTDh8+rLvvvluS9OMf/1jz58/XokWLNDw8rNbWVm3fvp3WiBZz7bXX6t5779VDDz2kioqKsX+D5HQ6NX36dEma9N6uX79eW7Zs0fXXX69169apo6NDd911l+67776s/R6YLJn3ls+t9d1000361Kc+pXnz5mlgYED333+/nnjiCbW3t0vi85rLEn1v+bzmhoqKikm1keXl5aqurh57nM9tbkrmvc3k55agk4DnnntOn/jEJ8a+v/766yVJX/rSl/TLX/5SPp9Pb7zxxtjPh4eH9e1vf1uHDx/W9OnTtWjRIj388MNavXp1xteO6LZu3SpJ+vjHPx72+C9+8Qt9+ctflqRJ721NTY127typ6667Tj/72c80d+5c/fSnP9XVV1+dqWXDhGTeWz631vf222/rmmuukc/nk9Pp1OLFi9Xe3q6GhgZJfF5zWaLvLZ/X/MHnNn9l83NrM0KVXQAAAACQJ2hGAAAAACDvEHQAAAAA5B2CDgAAAIC8Q9ABAAAAkHcIOgAAAADyDkEHAAAAQN4h6AAAAADIOwQdAAAAAHmHoAMAAAAg7xB0AAAAAOQdgg4AAACAvPP/A7bEl4jVAimtAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1000x1000 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model.eval()\n",
    "\n",
    "test_data = IR_data(X_test, Y_test)\n",
    "tdl = DataLoader(test_data,\n",
    "                 shuffle=False,\n",
    "                 batch_size=batch_size,\n",
    "                 drop_last=True)\n",
    "pred = torch.tensor([]).to(device)\n",
    "for x, y in tdl:\n",
    "    p = model(x.float().T.unsqueeze(-1).to(device))\n",
    "    pred = torch.cat([pred, p])\n",
    "\n",
    "result = pd.DataFrame({\n",
    "    'true_value': Y_test[:8000],\n",
    "    'Predicted': pred.squeeze(1).cpu().detach().numpy()\n",
    "})\n",
    "plt.rcParams['figure.figsize'] = (10.0, 10.0)\n",
    "plt.scatter(result['true_value'], result['Predicted'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "incorrect-terrain",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE = 0.03644\n"
     ]
    }
   ],
   "source": [
    "result['residual2'] = (result['true_value'] - result['Predicted'])**2\n",
    "print(\"RMSE = %1.5f\" % np.sqrt(result['residual2'].mean()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "proud-chart",
   "metadata": {},
   "source": [
    "# 长短期记忆模型（LSTM）\n",
    "\n",
    "RNN虽然建模了不同状态之间的转移过程，然而，对于序列中的长期依赖性却并不能够很好的建模。究其原因，如果我们观察RNN对于隐藏状态的更新方式，实际上是一个类似于马尔科夫链的过程，然而对于一个马尔科夫链而言，如果时间期比较长，那么比较长的历史之前对于现在的影响要么是指数级递减的（比如一个遍历的马尔科夫链），要么是爆炸的（比如随机游走）。\n",
    "\n",
    "而**长短期记忆模型**（**long short-term memory**，**LSTM**）使用“门”（gate）单元来建模不会随着时间消失或者爆炸的记忆，取得了非常好的效果。\n",
    "\n",
    "如果记$h_t$为第$t$期的隐藏状态，核心难点在于如何将过去较长一段时间的历史信息进行恰当的积累。长短期记忆模型使用了一个“**遗忘门**”（**forget gate**），负责决定如何使用过去的信息，以及在使用之后“遗忘”过去的信息。\n",
    "\n",
    "其结构如下：\n",
    "\n",
    "![](pic/LSTM.gv.png \"LSTM\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "written-softball",
   "metadata": {},
   "source": [
    "其中：\n",
    "\n",
    "* 每一个菱形代表一个“门”，即一个可以被学习的、依赖于$h_{t-1}$和$x_t$的、取值范围为$\\left(0,1\\right)$的数，可以使用Sigmoid激活函数产生\n",
    "* 通过遗忘门遗忘信息，外部输入门加入信息，从而产生一个更深层的“隐藏状态”，或称cell state——$c_t$，该“隐藏状态”决定了长期记忆和遗忘\n",
    "* 将经过遗忘门过滤掉的信息和新加入的信息相加，得到新的$c_t$\n",
    "* $c_t$也可以被一个门所关掉，即输出门\n",
    "\n",
    "其具体公式如下：\n",
    "\n",
    "![](pic/LSTMformula.png \"LSTM公式\")\n",
    "\n",
    "LSTM的用法与RNN基本一致，区别在于nn.LSTM所返回的函数多了一个输入c_0和一个输出c_n，即cell state。\n",
    "\n",
    "比如，我们稍加修改就可以得到LSTM版的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "ultimate-atlanta",
   "metadata": {},
   "outputs": [],
   "source": [
    "class LSTM(nn.Module):\n",
    "\n",
    "    def __init__(self, batch_size, feature_size, hidden_size=30):\n",
    "        super(LSTM, self).__init__()\n",
    "        self.batch_size = batch_size  #批大小\n",
    "        self.feature_size = feature_size  #特征数量\n",
    "        self.hidden_size = hidden_size  #隐藏状态大小\n",
    "\n",
    "        self.rnn = nn.LSTM(input_size=feature_size,\n",
    "                           hidden_size=hidden_size,\n",
    "                           num_layers=2,\n",
    "                           dropout=0.5)\n",
    "        self.layer2 = nn.Sequential(nn.Linear(hidden_size, 50),\n",
    "                                    nn.ReLU(inplace=True), nn.LayerNorm(50),\n",
    "                                    nn.Linear(50, 50), nn.ReLU(inplace=True),\n",
    "                                    nn.Dropout(p=0.5), nn.Linear(50, 1))\n",
    "        self.hidden = self.init_hidden()\n",
    "\n",
    "    def init_hidden(self):\n",
    "        h = torch.zeros(2, self.batch_size, self.hidden_size).to(device)\n",
    "        c = torch.zeros(2, self.batch_size, self.hidden_size).to(device)\n",
    "        return h, c\n",
    "\n",
    "    def forward(self, x):\n",
    "        out, h = self.rnn(x, self.hidden)\n",
    "        y = self.layer2(out[-1])\n",
    "        return y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "lonely-program",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "第0次epoch，Smoothed Loss=0.11098627746105194，LR=[0.009000000000000001]\n",
      "第20次epoch，Smoothed Loss=0.01474280677702954，LR=[0.0010941898913151245]\n",
      "第40次epoch，Smoothed Loss=0.014322486665278794，LR=[0.00013302794647291155]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABLkAAAGsCAYAAAAxGHVCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAk8UlEQVR4nO3dfbTVdZ3o8c8+PGwOeDiCCIcjR8TS0URN0QwfUZPRwDLXdK3MS82sRlNM0zslUzNqLT1mk5d7x9HSaTm0yoc1d7K8N7OwBHOEQsRELZ8fTsgJJTwHAQ8P53v/aNh65EHA/cB383qttZec3/6d/fts2F9++vZ39i6klFIAAAAAQMYaaj0AAAAAALxbIhcAAAAA2RO5AAAAAMieyAUAAABA9kQuAAAAALIncgEAAACQPZELAAAAgOz1r/UAb9fb2xsvv/xyNDU1RaFQqPU4AAAAANRQSilWrlwZra2t0dCw5eu1drrI9fLLL0dbW1utxwAAAABgJ9LR0RFjxozZ4v07XeRqamqKiD8PPnTo0BpPAwAAAEAtdXd3R1tbW6kZbclOF7k2/oji0KFDRS4AAAAAIiLe8W2tvPE8AAAAANkTuQAAAADInsgFAAAAQPZELgAAAACyJ3IBAAAAkD2RCwAAAIDsiVwAAAAAZE/kAgAAACB7IhcAAAAA2RO5AAAAAMieyAUAAABA9kQuAAAAALIncgEAAACQvf61HmBX8NiSrnht9brYf9RuMXLooFqPAwAAAFB3XMlVBdf89Pfx6e/+Oh58dnmtRwEAAACoSyIXAAAAANkTuQAAAADInsgFAAAAQPZELgAAAACyJ3IBAAAAkD2RCwAAAIDsiVwAAAAAZE/kAgAAACB7IhcAAAAA2RO5qqBQ+PM/U6TaDgIAAABQp0QuAAAAALIncgEAAACQPZELAAAAgOyJXAAAAABkT+QCAAAAIHsiFwAAAADZE7kAAAAAyJ7IBQAAAED2RC4AAAAAsidyVVFKtZ4AAAAAoD6JXAAAAABkT+QCAAAAIHsiFwAAAADZE7kAAAAAyN52R677778/Tj/99GhtbY1CoRA/+tGP+tyfUoorrrgiWltbo7GxMSZNmhSPP/54ueYFAAAAgE1sd+RatWpVHHrooXH99ddv9v5rr702rrvuurj++utjwYIF0dLSEqecckqsXLnyXQ8LAAAAAJvTf3u/4bTTTovTTjtts/ellGLmzJnxla98Jc4888yIiJg1a1aMGjUqbr311jj33HPf3bQAAAAAsBllfU+u559/Pjo7O2Py5MmlbcViMU444YR48MEHN/s9PT090d3d3ecGAAAAANujrJGrs7MzIiJGjRrVZ/uoUaNK971de3t7NDc3l25tbW3lHAkAAACAXUBFPl2xUCj0+TqltMm2jWbMmBFdXV2lW0dHRyVGqqmNzz2lGg8CAAAAUKe2+z25tqalpSUi/nxF1+jRo0vbly1btsnVXRsVi8UoFovlHAMAAACAXUxZr+QaN25ctLS0xOzZs0vb1q5dG3Pnzo2jjz66nIcCAAAAgJLtvpLr9ddfj2eeeab09fPPPx+PPPJIDB8+PPbee++4+OKL4+qrr4799tsv9ttvv7j66qtj8ODB8alPfaqsgwMAAADARtsduR566KE48cQTS19fcsklERExbdq0+Ld/+7f40pe+FGvWrInzzz8/VqxYEUcddVT8/Oc/j6ampvJNDQAAAABvsd2Ra9KkSZG28g7qhUIhrrjiirjiiivezVwAAAAAsM0q8umKAAAAAFBNIhcAAAAA2RO5AAAAAMieyAUAAABA9kSuKijUegAAAACAOidyVdGWP5MSAAAAgHdD5AIAAAAgeyIXAAAAANkTuQAAAADInsgFAAAAQPZELgAAAACyJ3IBAAAAkD2RCwAAAIDsiVwAAAAAZE/kqoJCodYTAAAAANQ3kauKUkq1HgEAAACgLolcAAAAAGRP5AIAAAAgeyIXAAAAANkTuQAAAADInsgFAAAAQPZELgAAAACyJ3IBAAAAkD2RCwAAAIDsiVxVUKj1AAAAAAB1TuSqolTrAQAAAADqlMgFAAAAQPZELgAAAACyJ3IBAAAAkD2RCwAAAIDsiVwAAAAAZE/kAgAAACB7IhcAAAAA2RO5AAAAAMieyFUFhUKh1iMAAAAA1DWRCwAAAIDsiVzVlGo9AAAAAEB9ErkAAAAAyJ7IBQAAAED2RC4AAAAAsidyAQAAAJA9kQsAAACA7IlcAAAAAGRP5AIAAAAgeyJXFRRqPQAAAABAnRO5AAAAAMieyFVFKVKtRwAAAACoSyIXAAAAANkTuQAAAADInsgFAAAAQPZELgAAAACyJ3IBAAAAkD2RCwAAAIDslT1yrV+/Pr761a/GuHHjorGxMfbdd9/42te+Fr29veU+FAAAAABERET/cj/gN77xjfj2t78ds2bNioMOOigeeuih+OxnPxvNzc1x0UUXlftwWSgUaj0BAAAAQH0re+SaN29efPSjH40pU6ZERMQ+++wTt912Wzz00EPlPhQAAAAAREQFflzx2GOPjV/84hfx1FNPRUTEb3/723jggQfiwx/+8Gb37+npie7u7j63epVSrScAAAAAqE9lv5Lry1/+cnR1dcUBBxwQ/fr1iw0bNsRVV10Vn/zkJze7f3t7e1x55ZXlHgMAAACAXUjZr+S644474vvf/37ceuut8fDDD8esWbPin/7pn2LWrFmb3X/GjBnR1dVVunV0dJR7JAAAAADqXNmv5Pq7v/u7uOyyy+ITn/hEREQcfPDB8eKLL0Z7e3tMmzZtk/2LxWIUi8VyjwEAAADALqTsV3KtXr06Ghr6Pmy/fv2it7e33IcCAAAAgIiowJVcp59+elx11VWx9957x0EHHRSLFi2K6667Lv76r/+63IcCAAAAgIioQOT653/+5/iHf/iHOP/882PZsmXR2toa5557bvzjP/5juQ8FAAAAABFRgcjV1NQUM2fOjJkzZ5b7oQEAAABgs8r+nlxsTqHWAwAAAADUNZELAAAAgOyJXAAAAABkT+SqolTrAQAAAADqlMgFAAAAQPZELgAAAACyJ3IBAAAAkD2RCwAAAIDsiVwAAAAAZE/kAgAAACB7IlcVFAq1ngAAAACgvolcAAAAAGRP5AIAAAAgeyJXFaVU6wkAAAAA6pPIBQAAAED2RC4AAAAAsidyAQAAAJA9kQsAAACA7IlcAAAAAGRP5AIAAAAgeyJXFRRqPQAAAABAnRO5AAAAAMieyAUAAABA9kSuKkqRaj0CAAAAQF0SuQAAAADInsgFAAAAQPZELgAAAACyJ3IBAAAAkD2RCwAAAIDsiVwAAAAAZE/kqoJCodYTAAAAANQ3kQsAAACA7IlcAAAAAGRP5AIAAAAgeyJXFaVU6wkAAAAA6pPIBQAAAED2RC4AAAAAsidyAQAAAJA9kQsAAACA7IlcAAAAAGRP5KqCQhRqPQIAAABAXRO5AAAAAMieyAUAAABA9kQuAAAAALInclVRqvUAAAAAAHVK5AIAAAAgeyIXAAAAANkTuQAAAADInsgFAAAAQPZELgAAAACyJ3JVQaFQ6wkAAAAA6pvIBQAAAED2RC4AAAAAsidyAQAAAJC9ikSuJUuWxKc//enYY489YvDgwfH+978/Fi5cWIlD5SWlWk8AAAAAUJf6l/sBV6xYEcccc0yceOKJ8dOf/jRGjhwZzz77bOy+++7lPhQAAAAAREQFItc3vvGNaGtri1tuuaW0bZ999in3YQAAAACgpOw/rnjXXXfFEUccER//+Mdj5MiRcdhhh8XNN9+8xf17enqiu7u7zw0AAAAAtkfZI9dzzz0XN954Y+y3337xs5/9LM4777z4whe+EN/73vc2u397e3s0NzeXbm1tbeUeCQAAAIA6V/bI1dvbG4cffnhcffXVcdhhh8W5554bn/vc5+LGG2/c7P4zZsyIrq6u0q2jo6PcIwEAAABQ58oeuUaPHh3ve9/7+mw78MAD46WXXtrs/sViMYYOHdrnVm8KhVpPAAAAAFDfyh65jjnmmHjyySf7bHvqqadi7Nix5T4UAAAAAEREBSLXF7/4xZg/f35cffXV8cwzz8Stt94aN910U1xwwQXlPhQAAAAAREQFIteRRx4Zd955Z9x2220xfvz4+PrXvx4zZ86Ms88+u9yHAgAAAICIiOhfiQedOnVqTJ06tRIPDQAAAACbKPuVXGxZqvUAAAAAAHVK5AIAAAAgeyIXAAAAANkTuQAAAADInsgFAAAAQPZEriooRKHWIwAAAADUNZELAAAAgOyJXAAAAABkT+QCAAAAIHsiFwAAAADZE7mqKKVaTwAAAABQn0QuAAAAALIncgEAAACQPZELAAAAgOyJXAAAAABkT+SqhkKtBwAAAACobyIXAAAAANkTuQAAAADInsgFAAAAQPZELgAAAACyJ3JVUUqp1iMAAAAA1CWRCwAAAIDsiVwAAAAAZE/kAgAAACB7IhcAAAAA2RO5AAAAAMieyFUFhVoPAAAAAFDnRC4AAAAAsidyAQAAAJA9kQsAAACA7IlcAAAAAGRP5KqiVOsBAAAAAOqUyAUAAABA9kQuAAAAALIncgEAAACQPZELAAAAgOyJXFVQKBRqPQIAAABAXRO5AAAAAMieyAUAAABA9kQuAAAAALIncgEAAACQPZGrilKq9QQAAAAA9UnkAgAAACB7IhcAAAAA2RO5AAAAAMieyAUAAABA9kSuKijUegAAAACAOidyAQAAAJA9kQsAAACA7IlcAAAAAGRP5AIAAAAgeyIXAAAAANkTuaoo1XoAAAAAgDpV8cjV3t4ehUIhLr744kofCgAAAIBdVEUj14IFC+Kmm26KQw45pJKHAQAAAGAXV7HI9frrr8fZZ58dN998cwwbNqxShwEAAACAykWuCy64IKZMmRIf+tCHtrpfT09PdHd397nVm0Kh1hMAAAAA1Lf+lXjQ22+/PR5++OFYsGDBO+7b3t4eV155ZSXGAAAAAGAXUfYruTo6OuKiiy6K73//+zFo0KB33H/GjBnR1dVVunV0dJR7JAAAAADqXNmv5Fq4cGEsW7YsJkyYUNq2YcOGuP/+++P666+Pnp6e6NevX+m+YrEYxWKx3GMAAAAAsAspe+Q6+eSTY/HixX22ffazn40DDjggvvzlL/cJXAAAAABQDmWPXE1NTTF+/Pg+24YMGRJ77LHHJtsBAAAAoBwq9umKbCqlVOsRAAAAAOpSRT5d8e3mzJlTjcMAAAAAsItyJRcAAAAA2RO5AAAAAMieyFUFhVoPAAAAAFDnRC4AAAAAsidyAQAAAJA9kQsAAACA7IlcAAAAAGRP5AIAAAAgeyIXAAAAANkTuQAAAADInsgFAAAAQPZELgAAAACyJ3JVQaFQqPUIAAAAAHVN5AIAAAAgeyIXAAAAANkTuQAAAADInsgFAAAAQPZELgAAAACyJ3IBAAAAkD2Rq4pSqvUEAAAAAPVJ5AIAAAAgeyIXAAAAANkTuaqgUOsBAAAAAOqcyAUAAABA9kQuAAAAALIncgEAAACQPZELAAAAgOyJXAAAAABkT+QCAAAAIHsiVxWlSLUeAQAAAKAuiVwAAAAAZE/kAgAAACB7Ilc1FGo9AAAAAEB9E7kAAAAAyJ7IBQAAAED2RC4AAAAAsidyAQAAAJA9kQsAAACA7IlcAAAAAGRP5KqilGo9AQAAAEB9ErkAAAAAyJ7IBQAAAED2RK4qKESh1iMAAAAA1DWRCwAAAIDsiVwAAAAAZE/kAgAAACB7IhcAAAAA2RO5AAAAAMieyAUAAABA9kQuAAAAALInclVRqvUAAAAAAHVK5AIAAAAgeyJXFRQKtZ4AAAAAoL6JXAAAAABkr+yRq729PY488shoamqKkSNHxhlnnBFPPvlkuQ8DAAAAACVlj1xz586NCy64IObPnx+zZ8+O9evXx+TJk2PVqlXlPhQAAAAARERE/3I/4D333NPn61tuuSVGjhwZCxcujOOPP77chwMAAACA8keut+vq6oqIiOHDh2/2/p6enujp6Sl93d3dXemRAAAAAKgzFX3j+ZRSXHLJJXHsscfG+PHjN7tPe3t7NDc3l25tbW2VHAkAAACAOlTRyDV9+vR49NFH47bbbtviPjNmzIiurq7SraOjo5IjAQAAAFCHKvbjihdeeGHcddddcf/998eYMWO2uF+xWIxisVipMXYqKdV6AgAAAID6VPbIlVKKCy+8MO68886YM2dOjBs3rtyHAAAAAIA+yh65Lrjggrj11lvjxz/+cTQ1NUVnZ2dERDQ3N0djY2O5D5eFQq0HAAAAAKhzZX9PrhtvvDG6urpi0qRJMXr06NLtjjvuKPehsrF67YaIiHhj3YYaTwIAAABQnyry44r09ZPFSyMi4n/94un44in713gaAAAAgPpT0U9XBAAAAIBqELkAAAAAyJ7IBQAAAED2RC4AAAAAsidyAQAAAJA9kQsAAACA7IlcAAAAAGRP5AIAAAAgeyIXAAAAANkTuQAAAADInsgFAAAAQPZELgAAAACyJ3IBAAAAkD2RCwAAAIDsiVwAAAAAZE/kAgAAACB7IhcAAAAA2RO5AAAAAMieyAUAAABA9kQuAAAAALIncgEAAACQPZELAAAAgOyJXAAAAABkT+QCAAAAIHsiFwAAAADZE7mq4OC9mku/Xr12fQ0nAQAAAKhPIlcVnPK+UaVfd61ZV8NJAAAAAOqTyFUFx++/Z+nXhSjUcBIAAACA+iRyVUHDW7rWC8tX1W4QAAAAgDolclXZN3/2ZK1HAAAAAKg7IleVPf5yV61HAAAAAKg7IlcVpPTmr99Y11u7QQAAAADqlMhVBfvuOaTP1/OfW16jSQAAAADqk8hVBU2DBvT5+oVXvfk8AAAAQDmJXAAAAABkT+SqgSf/uLLWIwAAAADUFZGrBp7344oAAAAAZSVy1cCrr/fUegQAAACAuiJyVUlr86DSrx9b0h0df1pdw2kAAAAA6ovIVSVf++j4Pl8fd+198ca6DTWaBgAAAKC+iFxVctIBIzfZ9u25z9ZgEgAAAID607/WA+wqGhoKm2ybee/T0a9QiGP3GxGH7T2sBlMBAAAA1AdXctXYt2Y/FR+74cHY0JtqPQoAAABAtkSuKvrN35+8xfsuvO3hKk4CAAAAUF9ErioaOXRQbOanFiMi4u7FnXHkVfdWdyAAAACAOiFyVdkzV304/vb4fTd73ysre2Kfy34S+1z2k1i7vrfKkwEAAADkS+SqsoaGQvz9hw+MffYYvNX9pvzvX1VpIgAAAID8FVJKO9U7nnd3d0dzc3N0dXXF0KFDaz1OxXX8aXUcd+19m72vZeig+NWXT4wB/bRIAAAAYNe0ra1IPamxtuGDY96MkzZ7X2f3G7HfV34aS15bU+WpAAAAAPIicu0ERjc3xgvXTNni/cdc88s4deb9MfepV6K3d6e68A4AAABgp+DHFXcyy7rfiA9c/Yut7vPVKQfG3xw7LgqFLXxUIwAAAECd2NZWJHLthFJKcc9jnfH5Hzy81f1+e/nkaG4cUKWpAAAAAKpP5KoDK1atjcO+Pvsd9/sfk/eP8ye9NxoaXNkFAAAA1JeaR64bbrghvvnNb8bSpUvjoIMOipkzZ8Zxxx33jt8ncm3q/z36cjy7bFX8z3uf2qb9Gwf0i3svPSH22r2xwpMBAAAAVFZNI9cdd9wR55xzTtxwww1xzDHHxHe+853413/913jiiSdi7733Lsvgu6Ilr62J/3zm1fjS/3n0XT3OCfvvGX//4QNj8MB+MWZYYxQKhUgpeY8vAAAAYKdT08h11FFHxeGHHx433nhjaduBBx4YZ5xxRrS3t2/1e0WubbNuQ2/c/9Qr8TezHqr1KDu1cSOGxPOvrqr1GBWxW7F/vN6zPhoH9Is16za8q8fq11CIDdvxyZ1/Maopnvzjynd1TMpvS6/31uZB8XLXG5tsP6ClKX7fuW1/jpP+Ys+Y8+Qr73rGcjli7LD40PtGxTU//f027f+BccPjN8//aav7bOn3aVsd2rZ7/LbjtR3+/rd6z55D4tlXqvd31167N8awIQPisSXdMbBfQ6zd0LtN33fyASPjd0u739Xv25YcMqY5xo0YEj9+5OWyP/bW7LV7Yyx5bU1ERPRvKMT6Cn2q8ZH7DIsFL6zYpn0/c/Q+8eLyVXHfDq7Bja//D+47PFKK+PV/rYVTD2qJ37zwp/jTqrU79Lg7YnTzoNizqRh/NWFM/Pq5P8VPFi8t22MP6FeIdRtSFPs3RM/6bXsNV8P723aPpV1r4o/dPX22n/gXe8Z9T76y2dfCxnP8wP4Nsfa/nsuI3QbGX01oi58+tjReXL46IiLGDGuMPXYrbvXvnmGDB8T63hRTDxkd855dHi/81/dur7f+u8LmzjcfnzAmdhvUP275zxe2+THf/nfzxj/D7XXImOZ49A9dpa8LhYh3+i+c/UbuFk8vez2GDuof3W+s3+5j9msoxMimYix9y99/Y/cYXPqziYhoKvaPlT1bf+zmxgHRtWZdn23bc37eUfuOGBLPbce/Iw8a0BAnHTAy7l7c2Wf7f584Nr4378V3/P4dfU47up7bhjfGH1as2eLrYOOf/0bDhwyMvz1+32gc0C8uv+vxzX7PiN2K8errf17H72/bPR7peK30vPZsKsYrK/9837DBA2LF6nWbfP+0iWNj1jb8Xm2Pjx22V9y5aElERBz73hHxwDOv7tDjHP2ePeLBZ5e/435twxuj409rNtn+1t+b7fHWf3eq1t/dDYWIbT21b+9/I22L940eGk8s7S7rY27JRw5tja+fMb5u3se7ZpFr7dq1MXjw4Pj3f//3+NjHPlbaftFFF8UjjzwSc+fO7bN/T09P9PS8uSC6u7ujra1N5NoBy1/viWdfWRX/7Tvzaj0KAAAAUEM//+Lxsf+oplqPURbbGrn6l/vAr776amzYsCFGjRrVZ/uoUaOis7Nzk/3b29vjyiuvLPcYu6Q9divGHrsV44Vrpmz2/t7eFEteWxP/99GX49p7nqzydOzMtvdqhVFDi5v8H2motuP2GxG/enrH/o8l5KRteGMs6+7Zqa5OejeO229ENA3qv8kVITvqrVc7sWXDhwys+FV77+ZKEii38XsNjceWbPsVM8ftNyLWbeiN+c9t/crvHbU9VxBBORT7N0TjgH61HqPqyh65Nnr7+ztt6T2fZsyYEZdccknp641XclF+DQ2FaBs+OM6f9N44f9J7az0OAAAAQNmUPXKNGDEi+vXrt8lVW8uWLdvk6q6IiGKxGMVisdxjAAAAALALaSj3Aw4cODAmTJgQs2fP7rN99uzZcfTRR5f7cAAAAABQmR9XvOSSS+Kcc86JI444IiZOnBg33XRTvPTSS3HeeedV4nAAAAAA7OIqErnOOuusWL58eXzta1+LpUuXxvjx4+Puu++OsWPHVuJwAAAAAOziCimlneozHrb1YyEBAAAAqH/b2orK/p5cAAAAAFBtIhcAAAAA2RO5AAAAAMieyAUAAABA9kQuAAAAALIncgEAAACQPZELAAAAgOyJXAAAAABkT+QCAAAAIHv9az3A26WUIiKiu7u7xpMAAAAAUGsbG9HGZrQlO13kWrlyZUREtLW11XgSAAAAAHYWK1eujObm5i3eX0jvlMGqrLe3N15++eVoamqKQqFQ63HKoru7O9ra2qKjoyOGDh1a63GgpqwHeJP1AH9mLcCbrAd4k/XARimlWLlyZbS2tkZDw5bfeWunu5KroaEhxowZU+sxKmLo0KEWJvwX6wHeZD3An1kL8CbrAd5kPRARW72CayNvPA8AAABA9kQuAAAAALInclVBsViMyy+/PIrFYq1HgZqzHuBN1gP8mbUAb7Ie4E3WA9trp3vjeQAAAADYXq7kAgAAACB7IhcAAAAA2RO5AAAAAMieyAUAAABA9kQuAAAAALInclXBDTfcEOPGjYtBgwbFhAkT4le/+lWtR4IddsUVV0ShUOhza2lpKd2fUoorrrgiWltbo7GxMSZNmhSPP/54n8fo6emJCy+8MEaMGBFDhgyJj3zkI/GHP/yhzz4rVqyIc845J5qbm6O5uTnOOeeceO2116rxFGGL7r///jj99NOjtbU1CoVC/OhHP+pzfzVf/y+99FKcfvrpMWTIkBgxYkR84QtfiLVr11biacNmvdN6+MxnPrPJ+eKDH/xgn32sB+pBe3t7HHnkkdHU1BQjR46MM844I5588sk++zg/sKvYlvXg/EAliVwVdscdd8TFF18cX/nKV2LRokVx3HHHxWmnnRYvvfRSrUeDHXbQQQfF0qVLS7fFixeX7rv22mvjuuuui+uvvz4WLFgQLS0tccopp8TKlStL+1x88cVx5513xu233x4PPPBAvP766zF16tTYsGFDaZ9PfepT8cgjj8Q999wT99xzTzzyyCNxzjnnVPV5wtutWrUqDj300Lj++us3e3+1Xv8bNmyIKVOmxKpVq+KBBx6I22+/Pf7jP/4jLr300so9eXibd1oPERGnnnpqn/PF3Xff3ed+64F6MHfu3Ljgggti/vz5MXv27Fi/fn1Mnjw5Vq1aVdrH+YFdxbashwjnByooUVEf+MAH0nnnnddn2wEHHJAuu+yyGk0E787ll1+eDj300M3e19vbm1paWtI111xT2vbGG2+k5ubm9O1vfzullNJrr72WBgwYkG6//fbSPkuWLEkNDQ3pnnvuSSml9MQTT6SISPPnzy/tM2/evBQR6fe//30FnhVsv4hId955Z+nrar7+77777tTQ0JCWLFlS2ue2225LxWIxdXV1VeT5wta8fT2klNK0adPSRz/60S1+j/VAvVq2bFmKiDR37tyUkvMDu7a3r4eUnB+oLFdyVdDatWtj4cKFMXny5D7bJ0+eHA8++GCNpoJ37+mnn47W1tYYN25cfOITn4jnnnsuIiKef/756Ozs7POaLxaLccIJJ5Re8wsXLox169b12ae1tTXGjx9f2mfevHnR3NwcRx11VGmfD37wg9Hc3GztsNOq5ut/3rx5MX78+GhtbS3t85d/+ZfR09MTCxcurOjzhO0xZ86cGDlyZOy///7xuc99LpYtW1a6z3qgXnV1dUVExPDhwyPC+YFd29vXw0bOD1SKyFVBr776amzYsCFGjRrVZ/uoUaOis7OzRlPBu3PUUUfF9773vfjZz34WN998c3R2dsbRRx8dy5cvL72ut/aa7+zsjIEDB8awYcO2us/IkSM3OfbIkSOtHXZa1Xz9d3Z2bnKcYcOGxcCBA60RdhqnnXZa/OAHP4hf/vKX8a1vfSsWLFgQJ510UvT09ESE9UB9SinFJZdcEscee2yMHz8+Ipwf2HVtbj1EOD9QWf1rPcCuoFAo9Pk6pbTJNsjFaaedVvr1wQcfHBMnToz3vOc9MWvWrNIbRu7Ia/7t+2xuf2uHHFTr9W+NsLM766yzSr8eP358HHHEETF27Nj4yU9+EmeeeeYWv896IGfTp0+PRx99NB544IFN7nN+YFezpfXg/EAluZKrgkaMGBH9+vXbpBIvW7Zsk6IMuRoyZEgcfPDB8fTTT5c+ZXFrr/mWlpZYu3ZtrFixYqv7/PGPf9zkWK+88oq1w06rmq//lpaWTY6zYsWKWLdunTXCTmv06NExduzYePrppyPCeqD+XHjhhXHXXXfFfffdF2PGjCltd35gV7Sl9bA5zg+Uk8hVQQMHDowJEybE7Nmz+2yfPXt2HH300TWaCsqrp6cnfve738Xo0aNj3Lhx0dLS0uc1v3bt2pg7d27pNT9hwoQYMGBAn32WLl0ajz32WGmfiRMnRldXV/zmN78p7fPrX/86urq6rB12WtV8/U+cODEee+yxWLp0aWmfn//851EsFmPChAkVfZ6wo5YvXx4dHR0xevToiLAeqB8ppZg+fXr88Ic/jF/+8pcxbty4Pvc7P7Areaf1sDnOD5RVNd/lfld0++23pwEDBqTvfve76YknnkgXX3xxGjJkSHrhhRdqPRrskEsvvTTNmTMnPffcc2n+/Plp6tSpqampqfSavuaaa1Jzc3P64Q9/mBYvXpw++clPptGjR6fu7u7SY5x33nlpzJgx6d57700PP/xwOumkk9Khhx6a1q9fX9rn1FNPTYccckiaN29emjdvXjr44IPT1KlTq/584a1WrlyZFi1alBYtWpQiIl133XVp0aJF6cUXX0wpVe/1v379+jR+/Ph08sknp4cffjjde++9acyYMWn69OnV+81gl7e19bBy5cp06aWXpgcffDA9//zz6b777ksTJ05Me+21l/VA3fn85z+fmpub05w5c9LSpUtLt9WrV5f2cX5gV/FO68H5gUoTuargX/7lX9LYsWPTwIED0+GHH97n41MhN2eddVYaPXp0GjBgQGptbU1nnnlmevzxx0v39/b2pssvvzy1tLSkYrGYjj/++LR48eI+j7FmzZo0ffr0NHz48NTY2JimTp2aXnrppT77LF++PJ199tmpqakpNTU1pbPPPjutWLGiGk8Rtui+++5LEbHJbdq0aSml6r7+X3zxxTRlypTU2NiYhg8fnqZPn57eeOONSj596GNr62H16tVp8uTJac8990wDBgxIe++9d5o2bdomr3XrgXqwuXUQEemWW24p7eP8wK7indaD8wOVVkgppepdNwYAAAAA5ec9uQAAAADInsgFAAAAQPZELgAAAACyJ3IBAAAAkD2RCwAAAIDsiVwAAAAAZE/kAgAAACB7IhcAAAAA2RO5AAAAAMieyAUAAABA9kQuAAAAALL3/wH5wWj+BznEZgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1500x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = LSTM(batch_size, 1).to(device)\n",
    "\n",
    "criterion = nn.MSELoss()\n",
    "optimizer = optim.Adam(model.parameters(), lr=0.01)\n",
    "lr_scheduler = torch.optim.lr_scheduler.ExponentialLR(optimizer, gamma=0.9)\n",
    "losses = []\n",
    "Exp_Smoothed_Loss = 0\n",
    "for i in range(50):\n",
    "    for x, y in dl:\n",
    "        # 将x计算预测值\n",
    "        y_pred = model(x.float().T.unsqueeze(-1).to(device))\n",
    "        # 计算损失\n",
    "        loss = criterion(y_pred, y.float().unsqueeze(1).to(device))\n",
    "        losses.append(loss.item())\n",
    "        if i == 0:\n",
    "            Exp_Smoothed_Loss = loss.item()\n",
    "        else:\n",
    "            Exp_Smoothed_Loss = 0.01 * loss.item() + 0.99 * Exp_Smoothed_Loss\n",
    "        # 反向传播\n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "    lr_scheduler.step()\n",
    "    if i % 20 == 0:\n",
    "        print(\"第%s次epoch，Smoothed Loss=%s，LR=%s\" %\n",
    "              (i, Exp_Smoothed_Loss, lr_scheduler.get_last_lr()))\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.rcParams['figure.figsize'] = (15.0, 5.0)\n",
    "\n",
    "i = np.arange(len(losses)) + 1\n",
    "plt.plot(i, np.array(losses))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "binding-means",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzoAAAMtCAYAAABXYgSXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB78klEQVR4nO3de3yb9X33/7fs2FZsbCVOMBLHOAcOigngQBYvQFtIaAg1dKX3Ntpwtx3L3QTSA6wbh5U7+Me60HvtDe0dFg5ltOAb2CHQxCXVTRgQDnNmwEmJaqBpcIAGGS9xIhsH24l0/f4wMj7ocEmWdF2SXs/HI39YvnRdX0/z5jffz/fzcRiGYQgAAAAA8kiR1QsAAAAAgHQj6AAAAADIOwQdAAAAAHmHoAMAAAAg7xB0AAAAAOQdgg4AAACAvEPQAQAAAJB3pli9ADPC4bA++OADVVZWyuFwWL0cAAAAABYxDEN9fX068cQTVVQUe98mJ4LOBx98oFNOOcXqZQAAAACwiffff18nn3xyzO/nRNCprKyUNPzDVFVVWbwaAAAAAFbp7e3VKaecMpIRYsmJoBMpV6uqqiLoAAAAAEh4pIVmBAAAAADyDkEHAAAAQN4h6AAAAADIOwQdAAAAAHmHoAMAAAAg7xB0AAAAAOQdgg4AAACAvEPQAQAAAJB3CDoAAAAA8g5BBwAAAEDeIegAAAAAyDsEHQAAAAB5h6ADAAAAIO8QdAAAAADkHYIOAAAAgLxD0AEAAACQdwg6AAAAAPIOQQcAAABA3iHoAAAAAMg7BB0AAAAAeYegAwAAACDvEHQAAAAA5B2CDgAAAIC8Q9ABAAAAkHcIOgAAAADyDkEHAAAAQN4h6AAAAADIO1OsXgAAAAAAewmFDbV19qi7b0A1lU4tqq1WcZHD6mUlhaADAAAAYITPH1BTS4cCwYGR1zwup9Y1erW8zmPhypJD6RoAAAAAScMhZ01z+5iQI0mB4IBWN7frJ8/uUShsWLS65BB0AAAAACgUNnTHlg7FizF3P/s7Lbnr3+XzB7K2rlQRdAAAAABow3N71NU7kPC6rt5BrWlut33YIegAAAAABc7nD+juZ/ck9Z6mlg5bl7ERdAAAAIACFgobamrpSOo9hobP7bR19mRmUWlA0AEAAAAKWFtnz4TmA2Z196X2vmwg6AAAAAAFbDJhpabSmcaVpBdzdAAAAIAClkpYcUhyu4YHidoVOzoAAABAAVtUWy2PyymHyesj161r9Kq4yOy7so+gAwAAABSw4iKH1jV6JclU2HG7nNq4sl7L6zyZXdgkUboGAAAAFLjldR5tXFmvppaOMY0JPC6nbr/iLE2vKFN334BqKofL1ey8kxNB0AEAAACg5XUeLfO61dbZk3OhJhqCDgAAAJCnQmEjqeBSXORQw5wZWVxh5hB0AAAAgDwTChu6+5m3df9L7+hoyBh53eNyal2j1/bna9KBZgQAAABAngiFDd2z7Xea97dbteGFvWNCjiQFggNa09wunz9g0Qqzhx0dAAAAIA/4/AHd9C+/0ZGhUNzrDElNLR1a5nXn7PkbM9jRAQAAAHKczx/Q6ub2hCEnIhAcUFtnT4ZXZS12dAAAAIAcFQob2rH3oG7etDvp93b3DSS+KIcRdAAAAIAc5PMHJsy9SUZNpTPNK7IXgg4AAABgc6Gwof/Yc0Cb2v+gPxw+IofDoVf3HUr5fuUlRVpUW53GFdoPQQcAAACwMZ8/oBv/eZc+PhpO2z3v+pMFed2IQCLoAAAAALYVaTKQTgtOrtKV9Sel9Z52RNc1AAAAwIZCYUM3/9sbab3n0rNqtGXtRWm9p12xowMAAADYTChs6J9efkfBgWOTvleRQ7pwzgzd/98v0NTS4jSsLjcQdAAAAAAb2frGB/r+Zr96+o+mfI/jyor1tYZZ+uM5M7V4zoy8P48TDUEHAAAAsIn1Wzt0/4udk77Pj/7bOVpe50nDinIXZ3QAAAAAG9j6RiAtIcdTVVbwIUci6AAAAACWC4UN3fQvu9Jyr97ByZ/ryQcEHQAAAMBi//H7Axo4lp45OceVcTpF4owOAAAAYJlQ2NCOvQd159O/Tds9/+4L89N2r1xG0AEAAACyZOhYWA+/0qltHR/qUP+Q9h/+OG07ORGX1LnTer9cRdABAAAAsuAHT/9WD760L6PPuG9lfUG2ko6GoAMAAABk2KpHXtW2ju6MPuO+lfV0WxuFoAMAAABkUMtvPshYyCmS9D8urtVfLz+LnZxxCDoAAADAJITChto6e9TdN6CaSqcW1VaPhI5Q2NDf/NsbaX3ejIpSff+Ks+R2TR3zLIxF0AEAAABS5PMHdMeWDnX1Doy85q5y6o4rvVpe59GOdw7q46OhtD3PIekHf1JHiZoJzNEBAAAAUuDzB7S6uX1MyJGkrt4BrW5ul88fUOveg2l7nsfl1EbO4ZjGjg4AAACQpFDY0C1P7o57za1P7tZXFp066WdNKy/RvdfUa/GcGZSpJYGgAwAAACRpxzsHdfjI0bjXHDpyVFVTS1J+RiTS3PWls7Vk3syU71OoKF0DAAAAkmS2JG3ne4dTfkZ1RSmlapNA0AEAAACSZpi66qU9/5XyE75/xVmEnEkg6AAAAABJaphtrpSsfyj1jmtu19SU3wuCDgAAAJC0xXNmaFp56udvEvG4hufxIHUEHQAAACBJxUUO3fWlszNyb4ekdY1eOqxNEkEHAAAASMHyOo/uW1kv19T0NTKeXl5CA4I0IegAAAAAKVrmdcuh9Oy8XF1/ol77/jJCTpoQdAAAAIAUbXhujw5/HH+ejlkXn15DuVoaMTAUAAAAiCEUNrRj70G1vnNAkkMNc2Zo8ewZKi5yKBQ2dP+L76TtWTWVzrTdCwQdAAAAICqfP6Bbntytw0c+3bHZ8PzvNa28RHd96WxVlpXoyCTaR49WWuygy1qaEXQAAACAcXz+gFY3t0f93uEjR7W6uV2X152Qtuf9j4tnU7aWZpzRAQAAAEYJhQ3dseW3Ca977q3utD3T66lK270wjKADAAAAjNLW2aOu3sGE1w0eM9L2zBse2ymfP5C2+4GgAwAAAIzR3TdgyXObWjoUCqcvPBU6gg4AAADwiVDY0IG+xLs56WZICgQH1NbZk/Vn5yuaEQAAAAAabkDQ1NKhQNCaHR3Jut2kfETQAQAAQMHz+QNa09wuqwvHmKWTPpSuAQAAoKCFwoaaWjoyHnIqSosVq4G0Q5LH5WSWThoRdAAAAFDQ2jp7slKu1j8UkiFNCDuRr9c1epmlk0YEHQAAABS0bJ6LmVZeohOqxpanuV1ObVxZr+V1nqytoxBwRgcAAAC2M3QsrEdb9+ndniM6rbpc1zbMUumUzPw3+myeizl85Kj+73X1KipyqLtvQDWVw+Vq7OSkH0EHAAAAtrJ+a4cefKlTo0fK/GDrm1p1Ua1uXeFN67NCYUP/+c7BtN4zkQP9g7rq3JOy+sxCRNABAACAbazf2qH7X+yc8HrY0MjrqYSdUNhQW2fPmF2UbR1dumPLb9XVm925OXRWyw6CDgAAAGxh6FhYD740MeSM9uBLnfqry85Mqowt2nycaeUlOnzkaMprTYVDw+dx6KyWHTQjAAAAgC082rpvTLlaNGFj+DqzIvNxxndVsyLkSHRWyyZ2dAAAAGAL7/YcSet12ZqPY4bb5dS6Ri+d1bKIoAMAAABbOK26PK3XZWs+Tjz/veE0XV7nobOaBShdAwAAgC1c2zBLibJAkWP4uohQ2FDr3oPavGu/WvceVGhU7Vs25+PEcnmdRw1zZhByLMCODgAAAGyhdEqRVl1UG7XrWsSqi2pHGhFEazLgcTl1+xVeTa8o1Z4P+zK+5lhoPGA9gg4AAABsI9I6evwcnSKHxszRiTQZGH/+JhAc0PWPtWdlrXUnVsn/Qa8c0ph10HjAHhyGYdjhfFZcvb29crlcCgaDqqqqsno5AAAAyLChY2E92rpP7/Yc0WnV5bq2YdbITk4obOjCHz5n+fmbn/z5uSqbUhR1V4nGA5ljNhuwowMAAADbKZ1SpOsumh31e3ZoMiAND/5smDNDy7zuCcNI2cmxHkEHAAAAOcUOTQY8o87fFBc51DBnhsUrwnh0XQMAAEBOqal0Wvp8hzh/kwsIOgAAAMgpi2qrVV1RYsmzPS6nNq6s5/xNDqB0DQAAADmluMihPzn3JD30yr6sPXPpWTW67sLZnL/JIezoAAAAIOcs9bqz+rxv/HEtgz9zDEEHAAAAOedQ/1DWnlVS7NBimg3kHIIOAAAAckoobOjOpzuy9rwfX72AnZwcRNABAABATsnmHJ0FJ1fpyvqTs/IspBfNCAAAAJBTunqzE3KWnlWjn33tgqTfFwobauvs0fs9/frnV9/XR4PHdGp1ue7+s/N0nJM/v7OF/0kDAADAdiJhobtvQDWVTi08bbpef/eQunoH9E8vvZPRZy87q0Y/vaZeU0uLk36vzx9QU0vHhB2ntz/8SHV3/D8tOLlKW9ZelK6lIg6CDgAAAGwlWlgockhhI/PPrq4o0X3Xnp/SmRyfP6A1ze2Kt8w3/tCrKze8RNjJAoIOAAAAbCNWWMhGyJGkaxfPSinkhMKGmlo64oaciDf+0KuPBo5RxpZhNCMAAACALSQTFjJl9vEVKb0v2QYJN/7zzpSeA/MIOgAAALCFbHZTi6Wm0pnS+7r7klv3e4c+Tuk5MI/9MgAAAGTV+EYDi2qrVVzkSDospJNDkts1vJZUJBuQTp0+NaXnwDyCDgAAALImWqMBj8updY3elHdTJityImddozflwaCLaqvlcTlN70jd/WfnpfQcmEfpGgAAALIi0mhgfBjoCg5oTXO7DvYNylmS/T9P3S6nNq6s1/I6T8r3KC5yaF2jV2Zi0oKTq2hEkAUOwzCsPO9lSm9vr1wul4LBoKqqqqxeDgAAAJIUChu68IfPxd3xyFYL6dHWfm6ublx2eso7OePFmqMTwRydyTObDSYVmdevXy+Hw6Hvfve7ca/bvn27Fi5cKKfTqdmzZ+u+++6bzGMBAACQY8w0Gsh2yJGkJXNnpi3kSNLyOo9evvkSPb5qsf7X1Wdr4anTdMYJx2nZWTXy3/F5Qk4Wpbxn9uqrr+qBBx7QggUL4l7X2dmpFStWaNWqVWpubtYrr7yi66+/Xscff7yuvvrqVB8PAACAHGJlo4FoJtt8IJ7iIoca5sxQw5wZ+tMLTk37/WFOSjs6H330kb761a/qwQcf1PTp0+Nee9999+nUU0/VPffco7POOkt/+Zd/qb/4i7/Qj370o5QWDAAAgNxjVaOBaNLRfAD2l1LQueGGG3TFFVdo6dKlCa9tbW3VZZddNua1z3/+83rttdd09OjRqO8ZHBxUb2/vmH8AAADIXZGuZHaIFeloPgD7S7p07YknnlB7e7teffVVU9d3dXXphBNOGPPaCSecoGPHjunAgQPyeCb+L9j69evV1NSU7NIAAABgU5GuZGua2+WQlO3jODcunadZMyvGzO1Bfksq6Lz//vv6zne+o2eeeUZOp/ntR4dj7P8iRRq9jX894tZbb9VNN9008nVvb69OOeWUZJYKAAAAi40fDLrM69bGlfVxu5JlwmdOP17fWXp61p4He0gq6Lz++uvq7u7WwoULR14LhUJ68cUXtWHDBg0ODqq4uHjMe9xut7q6usa81t3drSlTpmjGjBlRn1NWVqaysrJklgYAAAAb2frGB/r+Zr96+j89qhAZDLr9rz+nrz74H3r13WBW1nLxvJlZeQ7sJamgc+mll2r37t1jXvvGN76hM888UzfffPOEkCNJDQ0NamlpGfPaM888o/PPP18lJSUpLBkAAAB2tn5rh+5/sXPC64HggFY3t2tKkUPHstRLusghXdswKyvPgr0kFXQqKytVV1c35rWKigrNmDFj5PVbb71V+/fv1yOPPCJJWr16tTZs2KCbbrpJq1atUmtrqx566CE9/vjjafoRAAAAYBdb3whEDTmjZSvkSNKqi2pVOmVSoyORo9L+qQcCAb333nsjX9fW1mrr1q164YUXdO655+rOO+/UT3/6U2boAAAA5JlQ2ND3N/utXoak4Z2cb15cq1tXeK1eCiziMCKdAWyst7dXLpdLwWBQVVVVVi8HAAAAUbTuPahrHtxhybOv++PTdOL0cr3bc0SnVZfr2oZZ7OTkKbPZIOn20gAAAEA03X3Z66Q23tL5HjXMid7oCoWJmAsAAIC0qKk0P34k3awMWbAngg4AAADS4lD/kGXPtjJkwZ4IOgAAAJi0oWNhyxoRlJcWa1FttSXPhn1xRgcAAAApC4UNbXhujx548R31D4UsWcPM40pVXOSw5NmwL4IOAAAAUuLzB3TLk7t1+MhRS9dx6vRyS58PeyLoAAAAYMTQsbAebd2XsE2zzx/QmuZ22WFOyaqLZlu9BNgQQQcAAACSpB883aGfvdyp0VMWf7D1Ta26aOzgzVDYUFNLhy1CTtmUIl14+vFWLwM2RNABAACAVj3yqrZ1dE94PWxI97/YKUkjYaets0eBoD3aOf/kz8/lfA6iIugAAAAUuF/t+iBqyBntwZc6VV1Rpv2HP9Zbgb4srSw2d1WZ7rhyvpbXeaxeCmyKoAMAAFDAQmFDt/5yd8Lrwoa0/tdvZWFFE3lcTt1+hVfTK0rV3TegmkqnFtVWs5ODuAg6AAAABayts0d9A8esXkZcP/ryOVoyb6bVy0COYWAoAABAAevus8dZm3gO9A9avQTkIIIOAABAAaupdFq9hIRyYY2wH4IOAABAAVtUWy2Py75BwuMaPo8DJIugAwAAUMCKixxa1+iVXY/1//kFp9J0ACkh6AAAAOSIUNhQ696D2rxrv1r3HlQonJ6RncvrPNq4st6WOzuzZpZbvQTkKLquAQAA5ACfP6Cmlo4xgzo9LqfWNXrTMktmeZ1Hy7xu7XjnoK5vfl1Bm3Ri43wOUsWODgAAgM35/AGtaW4fE3IkqSs4oDXN7fL5A2l5TnGRQ6929tgi5DjE+RxMDjs6AAAANhYKG2pq6VC0IjVDw4GgqaVDy7zulM6yhMKG2jp71N03oG0dH+pXb6QnNE1G5KdY1+jlfA5SRtABAACwsbbOngk7OaMZkgLBAbV19qhhzoyk7h2tHM4O3GksyUPhIugAAADYmNmBnskO/oyUw6WnncHkOCRVV5Tq+1ecJbdrqhbVVrOTg0kj6AAAANiY2cP4yRzaHzoW1m1P7bZNyJGkH/xJHTs4SCuaEQAAANhYZKBnrP2NZA/t+/wB/dHfP6ue/qNpW+NkuF1ObVxZT8hB2rGjAwAAYGORgZ5rmtvlkMbswiR7aN/nD2h1c3smlmnaP37lPE2vKFN334BqKp2UqSFjCDoAAAA2FxnoOb5xQLxD+6O7qdVUOrXwtOm65cnd2Vz2GC5nsX745XPYuUHWEHQAAAByQGSg5+jwEms3JFo3tenlU3T4iHXzcaaWllj2bBQmh2EYdjiHFldvb69cLpeCwaCqqqqsXg4AAIBt2amb2miROMZ5HEyW2WxAMwIAAIA8EQobumNL9OGiVousqamlQ6GwHVeIfEPQAQAAyBMbntujrl57Df8cbfRwUyDTCDoAAAB5wOcP6O5n91i9DFOSHW4KpIKgAwAAkONCYUNNLR1WL8O0ZIabAqki6AAAAOS4ts6eMR3W7CyZ4abAZBB0AAAAclyulII5ZH64KTBZBB0AAIAcZ4dSsIqyYkmftpEeb3p5Ca2lkVUMDAUAAMgBobARc1joof4hy9ZVNsWh6z87V2svmadtHV0TBpVOm1qibyyZpbWXzGMnB1lF0AEAALA5nz8wIUB4XE6ta/RqmdetO5+2rhHBA9eer8+cUSNJWl7n0TKvO2YgA7KJoAMAAGBjPn9Aa5rbJwwBDQQHtLq5XYtmVVvaiGBK0diTEMVFDjXMmWHRaoBPEXQAAADSLF6ZWbL3aWrpmBByRmvbZ+3wzQP9g5Y+H4iFoAMAAJBG8crMkj2Inwtto+3QCAGIhq5rAAAAaRIpMxsfTrqCA1rT3C6fP5DU/ezcNtohZuLA3gg6AAAAaRCvzCzyWlNLh0LheIVoY9l1tyRShMdMHNgZQQcAACANEpWZGRpuINDWaf5MzaLaanlczpizaazidjmZiQPb44wOAABAGpgtM0umHK24yKF1jV6taW6XQ4rblCAbrlsyS0u9blpGIyewowMAAJAGZsvMki1HW17n0caV9XJNLUllWWkxbWqJ7ltZr9sb56thzgxCDnICQQcAACANEpWZTebw/s73Dunwx0cntb7JuPerlKkh9xB0AAAA0iBSZiZpQthJ9fB+KGzox8+8rftf7EzPIlPgcTm1eDYDQJF7CDoAAABpEikzc7vGlqcle3g/FDb0k2d/p7Pv8On/PPf7TCzVFIforIbcRTMCAACANFpe59Eyr1ttnT3q7htQTaUzqcP7Pn9Atzy5W4ePWFeqJknTy0u0/ktnU7KGnEXQAQAASLPiIoca5iRf7uXzB7S6uT0DKzKvbEqRrv/sHK29ZB47OchpBB0AAAAbCIUN3bJpt2XPd5YU6f6vLtSFpx9PwEFeIOgAAADYwHee2GlpZ7V7/uxcfebMGsueD6QbzQgAAAAstvWND/SrNwKWPf8bf3waZ3GQdwg6AAAAFgqFDX1/s9/SNUwrL7X0+UAmEHQAAAAs1NbZo55+azusPd72nkJhw9I1AOlG0AEAALBIKGzoxbe7rV6GunoH1dbZY/UygLSiGQEAAECWhcKGNjy3R/e/+I6ODIWsXo4kqbtvwOolAGlF0AEAAMiAUNiIOjTU5w/ob/7tN+odsEfAiaipdFq9BCCtCDoAAABp5vMH1NTSoUDw010Sj8upK8/x6P4XOy1c2UQOSW7XcBAD8glBBwAAII18/oDWNLdr/NH+QHDAliFHktY1ehkSirxD0AEAAAUtVolZqvdqaumYEHLsyu1yal2jlxk6yEsEHQAAULBilZil+sd/W2fPmHvZzfTyEm34Sr0OfDQ46VAH2B1BBwAAFKRYJWZdwQGtaW7XxpX1SYedrl77hhxJ+sEX67Rk7kyrlwFkBXN0AABAwYlXYhZ5ramlI+khmq/s+a9Jry1TvrDAoxULTrR6GUDWEHQAAEDBSVRiZmi4eUAyQzR9/oD+rX1/GlaXGcu8J1i9BCCrCDoAAKDgmB2Oafa6yA6RnTEnB4WGoAMAAAqO2T/6zV5n9yYE1RUlzMlBwSHoAACAgrOotloel1Ox+o05NNx9zWw4MLvzY5W/u6qO7mooOAQdAABQcIqLHFrX6JWkCWEnlSGadi4L++bFtTQhQEEi6AAAgLwWChtq3XtQm3ftV+vegyOd1JbXebRxZb3crrEhxe1yJt1a+rm3PkzrmtNhRkWp/vEr9bp1hdfqpQCWYI4OAADIW4kGgi6v82iZ1622zh519w2YGqIZChva8c5Bte49KMnQvgNH9KvdgSz8NImt/dwczTuhkmGggAg6AAAgT5kdCFpc5FDDnBmm73nLk7t1+MjR9C84DZbMPd70zwLkO0rXAABA3snEQFCfP6DVze22DDnJNk8ACgFBBwAA5J10DwQNhQ3dscWec3JSaZ4AFAJK1wAAQN5J90DQts4edfXas4W0e9SZIwCfIugAAIC8k+6BoHack3Pdklla6nXTdACIgaADAADyTmQgaFdwIOo5HYeGd0LMnmmx05wcDzs4gCmc0QEAAHkn3QNBF9VWq6K0OH0LTNENn5ujl2++hJADmEDQAQAAeSmdA0G3dXSpfyiU7iUm7cK5x1OmBphE6RoAAMhbqQwEjQiFjZEmBHf+6rdZWG18tI8GkkPQAQAAOS0SSGIFmWQGgkb4/AE1tXTEbVGdbbSPBpJD0AEAADkrWiCZ7GF9nz+gNc3tUZsYWME1dYp+ePUCzuUASeKMDgAAyEmRQDJ+16UrOKA1ze3y+QNJ3zMUNtTU0mGbkLPwtGlqv/0yQg6QAoIOAADIKaGwoVf2HNAtm3ZHDSSR15paOhQKJxdZdrxz0Bblag5Jqy6q1aY1SyhXA1JE6RoAAMgZZs/OGJICwQG1dfaYPp/j8wd0y6bdaVjl5FQ5p+g/b1uqqTZoZw3kMoIOAADICamcnenuM7c74/MHtLq5PbWFpdn/+vICQg6QBpSuAQAA20v17ExNpTPhNaGwoZv+eVdK60qnaeUlui/J+T4AYmNHBwAA2FoobOjnr3QmdXbGoeHBoGbmziz73y/oyNHwJFY4eV9Y4NFP/vw8zuMAaUTQAQAAtpXKPJtIVDAzd+a6n/+n3jlwZBIrnDyHpNffPWTpGoB8ROkaAACwpVjtoxNxu5zaaKIEbEv7H/Tvbx2YzBLTYnTjBADpw44OAACwnVTO5EybWqJ7v1qvxbNnJNzJ2fpGQN/+l99MbpFpZrZxAgBzCDoAAMB22jp7TO/kRCLNXVefrSVzZ0oaDkptnT3q7htQTeXwWZ1I+PH5A7r+MXt0WBvNTOMEAOYRdAAAgO0ks7vhKi/RXV86e6RULdq5Ho/LqXWNXi3zunWjDTqsjZZM4wQA5hF0AACA7SSzuzG1pFjLvG5JsWftdAUHtKa5XVecfYI+trjD2mjJNE4AkByaEQAAANtZVFstj8tc2Ikc5I93rify2q92f5i2NaaD2cYJAJLHjg4AALCd4iKH1jV6tbrZ3Fma7r6BhOd6kh02mim3X3GWZlaWTTg7BCC9CDoAAMCWltd5dOPSebr72T0Jr62pdNq+a1nkLM7Xl9QSboAsoHQNAADY1tpL5sldFbuEzaHhRgOLaqtt37XMEGdxgGwi6AAAANsqLnLojiu9cujTg/sR4w/yH+ofyvLqknPj0nmcxQGyiKADAABsbXmdRxtX1ss9rjnB6IP8obChO5/usGiF5syaWWH1EoCCwhkdAABge8vrPFrmdcccAprMgFGr2L20Dsg3BB0AAJATioscapgzI+r3unrtG3IYCApYg6ADAAByVihsqK2zR9vf6rZ6KVExEBSwDkEHAABkVSScRCtBS4bPH1BTS4etS9bcLqfWNXppQgBYgKADAAAyZnyoOdQ/qDuffnNMOPGYDAOj77XvwBHd/ezvMr38pPzFklm69KwTJEM60D/IQFDAYgQdAACQEWZ3XLqCA1rT3D7SQW0y97KC2aAGILsIOgAAIO18/oDWNLfLMHGtoeGzLE0tHVrmdU/YAUnmXtn25fqT9MMvn8OuDWBDzNEBAABpFQobamrpSCqYGJICwQG1dfZM+l7ZMr28hJAD2BhBBwAApNVkZtp09336vlDY0M9f6bRluZpD0vovnU3IAWyM0jUAAJBWo8NKsiJDNTmTA2CyCDoAACCtImElGaOHatr5TI4k3X4FIQfIBZSuAQCAtFpUWy2PyymzRV2jh2pKsu2ZHGl4rXc+3aFQ2K4rBBBB0AEAAGlVXOQYCS1mwo7b5RxpLT2Z8z3ZEKtpAgD7oXQNAACk3fI6jzaurJ9wzsbjcur2K7yaXlE6MkR0UW21JKl170H92h+waslJmcw5JADZQdABAAAZsbzOo2Vet9o6e8aEmmhzcuzaeCCWVM4hAcgugg4AAMiY4iKHGubMiPl9uzceGG900wQA9sYZHQAAYImhY2Hd9pQ/p0KONNw0gfk5gP2xowMAALIqFDa04bk9euDFd9Q/FLJ6Oaa5mZ8D5BSCDgAAyBqfP6Bbntytw0eOWr2UqIoc0oqzPVrmPUEzK8okh3Tgo8GY54sA2BdBBwAAZIVdz+NcPG+mZs2s0GnV5bq2YZZKp1DZD+SDpH6TN27cqAULFqiqqkpVVVVqaGjQr3/965jXv/DCC3I4HBP+vfXWW5NeOAAAyB2hsGHbQaDf/Mwc/X9X1em6i2YTcoA8ktSOzsknn6y77rpLc+fOlST94he/0FVXXaWdO3dq/vz5Md/39ttvq6qqauTr448/PsXlAgCAXGTnQaAHPhq0egkAMiCpoNPY2Djm6x/84AfauHGjduzYETfo1NTUaNq0aaafMzg4qMHBT/+PTm9vbzLLBAAANmPnAZvMxAHyU8r7s6FQSE888YT6+/vV0NAQ99rzzjtPHo9Hl156qZ5//vmE916/fr1cLtfIv1NOOSXVZQIAABuwa5jwMBMHyFtJB53du3fruOOOU1lZmVavXq2nnnpKXq836rUej0cPPPCANm3apCeffFJnnHGGLr30Ur344otxn3HrrbcqGAyO/Hv//feTXSYAALCRRbXV8rjsF3aYiQPkL4dhGEmdCxwaGtJ7772nw4cPa9OmTfrZz36m7du3xww74zU2NsrhcGjLli2mn9nb2yuXy6VgMDjmrA8AAMgd67d26P4XO61exoi/WDJL/7Mxduk9AHsymw2S3tEpLS3V3Llzdf7552v9+vU655xz9JOf/MT0+xcvXqw9e/Yk+1gAAJDDQmFDW34TsHoZYyzzuq1eAoAMmvQcHcMwxjQOSGTnzp3yeJgoDABAIbFT1zWHJDdnc4C8l1TQue2223T55ZfrlFNOUV9fn5544gm98MIL8vl8kobP1uzfv1+PPPKIJOmee+7RrFmzNH/+fA0NDam5uVmbNm3Spk2b0v+TAAAA2wiFDbV19qi7b0AzjyvTy3v+y+olSRoOORJnc4BCkFTQ+fDDD3XttdcqEAjI5XJpwYIF8vl8WrZsmSQpEAjovffeG7l+aGhI3/ve97R//35NnTpV8+fP19NPP60VK1ak96cAAAC24fMH1NTSYZsdnNHcLqfWNXq1vI7qEiDfJd2MwAo0IwAAIDdsfSOg6x9rt3oZUVVXlOjvrqrTigUnWr0UAJOQsWYEAAAA0Wx94wOtfdyeIUeSDvUf1Q2P7ZTPb6+mCAAyg6ADAAAmzecP6PrHdips4zqRyNKaWjoUsvNCAaQFQQcAAExKKGyoqaXD6mWYYkgKBAfU1tlj9VIAZBhBBwAATIqdWkeb1d2XW+sFkDyCDgAASFkobOiV39ujdXQyaiqdVi8BQIZNemAoAAAoTHZuIx0Lw0KBwkHQAQAASfP5A1rT3K5cOtLPsFCgsBB0AABAUiLNB3Ip5EgMCwUKDUEHAAAkJZeaD9x+xVmaWVmmmsrhcjV2coDCQdABAABJyYWOZZGzOF9fUku4AQoUQQcAAIwIhQ21dfaou28g5i7Ito4PLVqdOZzFASARdAAAwCeidVHzjDvXsvWNgH71RsCqJZrCWRwAEkEHAAAodhe1ruCA1jS3696vnCdXean+6l93WrK+RL576TzVHl/BWRwAIwg6AAAUuHhd1CKvrX18p8I2bLM2fscJACIIOgAAFDgzXdTsFHIckr7+x6fpsvkedm8AxETQAQCgwOVCF7XR7v1KvVYsYAcHQHwEHQAAClxNpdPqJZhCmRqAZBB0AAAocItqq+VxOdUVHIh6TscOrv/sbP3VZWdSpgbAtCKrFwAAAKxVXOTQukavpE9n0NjNRfNqCDkAkkLQAQAghw0dC+uhl97R/9zs10MvvaOhY+GU7rO8zqONK+vldtmvjM3jGm4ZDQDJoHQNAIActX5rhx58qXNMR7QfbH1Tqy6q1a0rvEnfb3mdR8eOGVr7hH1m5TgkrWv0spsDIGkEHQAActD6rR26/8XOCa+HDY28nmzY+dWu/frWE7vSsby0oPkAgMkg6AAAkGOGjoX14EsTQ85oD77Uqb+67EyVTjFXpR4rOGXb3644SzVVZaqpdDIjB8CkcEYHAIAc82jrvoQDPMPG8HVmbH0jYIuQI0k1VWW66tyT1DBnBiEHwKSwowMAQI55t+fIpK4LhQ21dfbog8Mfa+d7h/Qvr72fzuVNSq7M9AFgfwQdAAByzGnV5Slf5/MH1NTSoUBwIN3LmjS6qwFIJ0rXAADIMdc2zFKiqq4ix/B1o/n8Aa1pbrdlyJHorgYgvQg6AADkmNIpRVp1UW3ca1ZdVDumEUEobKippUMJjvZY5sal8+iuBiCtKF0DAMAmImdnuvsGEnYdi7SOHj9Hp8ihqHN02jp7bLuT43E5tfaSeVYvA0CeIegAAGAD0c7OJJojc+sKr/7qsjP1aOs+vdtzRKdVl+vahllRW0p399kz5EiUrAHIDIIOAAAWi5ydGV9W1hUc0Jrmdm1cWR8z7JROKdJ1F80e+ToUNtS69+CEXaHq8tIM/gSpu3Hp6ZSsAcgIgg4AABaKd3bGkOSQ1NTSoWVed8Jdj2i7Qu4qp86fNV3/3tGV1nWng7uqTGsvmWv1MgDkKZoRAABgoURnZwxJgeCAdrxzMO59YnVU6+od0K/eCOjjY/ZpQ+D45N8dV86nZA1AxhB0AACwkNmzMzf833b5/IGo37N7R7Xx3C5n3HI8AEgHStcAALBQTaXT1HWHPz4a87yOnTuqSZKzpEhrPjNXs2aWJ+wmBwDpQtABAMBCi2qr5XE51RUcMLUjE+28jp07qknS/V9ZqM+cVWP1MgAUGErXAADIkkhHtM279qt170GFwoaKixxa1+hN/GaNOq+zd+x5HbO7QlYom1KkC8843uplAChA7OgAAJAFiebkbFxZr1s27dbhj48mvNcNj7XrrqvPHilhS3ZXKJtW/tGplKkBsAQ7OgAAZFjMjmifzMnx+QNaXufRvV+pN3W/yHmdSHOCZHaFsm2p1231EgAUKIIOAAAZlGhOjjR87iYUNrR4zgx5XE6Z3f+IvC/CVV4y2eWmlcc13HgAAKxA0AEAIIPMzslp6+xJ+bzOT579nVY3t+vwkcRlb9m0rtFL2RoAyxB0AADIILMd0SLXRc7rTJtqbnfm+sfadfeze1JeXyZUlBbrPubkALAYQQcAgAwy2xFt9HXL6zy696vmzusETTQvyLZK5xQt42wOAIsRdAAAyKBIR7RYBVwORT/Lsnh2cud17KSrd1BtnT1WLwNAgSPoAACQQaPP3YwPLZGvo51life+XGD3IaYA8h9BBwCADIucu3G7xpaxuV1ObYxzliXW+8pLizO21nSx8xBTAIXBYRiG3WaLTdDb2yuXy6VgMKiqqiqrlwMAQEpCYUNtnT3q7htQTeVwuZqZrmSj37fvwBHd/ezvsrDa1Dg0HOBevvkSOq4ByAiz2WBKFtcEAEBBKy5yqGHOjJTet6i2Wjv2HtS6zb/NwMrSI14pHgBkG0EHAACb8/kDumNLh7p67X3uxe1yal2jl7bSAGyBoAMAgI35/AGtbm63ehkxVVeU6PYvzJe7ynwpHgBkA0EHAACLJDqzEwobuuXJ3RauMLG/u6pOKxacaPUyAGACgg4AABbw+QNqaulQIPhpOZpnXOnXjncO6vAR+w0EHW16RZnVSwCAqGgvDQBAlvn8Aa1pbh8TciSpKzigNc3t8vkDkqT/2HvAiuUlhXk5AOyKoAMAQBaFwoaaWjoUbbZD5LWmlg6Fwob2H/o4m0tLCfNyANgVpWsAAGRRW2fPhJ2c0QxJgeCA2jp79PGQfcvWIvNyFtVWW70UAIiKHR0AALLIbKnXB4eO6N/f+q8MryY1zMsBkAvY0QEAIIvMlnr97Wa/joUzvJgUMS8HQC4g6AAAkEWLaqs1rbwkYTe1gaP2SDnuqjL9zy94Nb2iLGYbbACwI4IOAABZtK2jy/Yto0f78Z+eqyVzZ1q9DABIGmd0AADIkkjHtVxy4KNBq5cAACkh6AAAkCWJOq7ZEe2jAeQqgg4AAFmSa8M1qytKaB8NIGcRdAAAyJJc2x35u6vqaDoAIGcRdAAAyJJIx7Vc8M2La7ViwYlWLwMAUkbXNQAAMiQUNtTW2TPSlnnhadN11GbDcY4rK9ZHg6GRr2dUlOrOq+q0YgEzcgDkNoIOAAAZ4PMH1NTSMab5wHFlxeofCsV5V/bdv/J8FRU5mJEDIO8QdAAASDOfP6A1ze0yxr0+eufEag5JbpdTi+fMINgAyEuc0QEAII1CYUN3bOmYEHLsJBJr1jV6CTkA8hY7OgAApNGG5/aoq9febaTdLqfWNXq1vI5zOADyFzs6AACkic8f0N3P7rF6GQndfgUhB0D+I+gAAJAGwyVrv7V6GQk5JN35dIdCYTsX1wHA5BF0AABIg7bOHnX1Dlq9jIQMSYHggNo6e6xeCgBkFGd0AABIYPw8nGgtmLv77H0uZ7xcWy8AJIugAwBAHNHm4XjGHeYPhQ1127wBwXg1lU6rlwAAGUXQAQAghljzcLqCA1rT3K6NK+sVDhv6/ma/evqPWrLGZEXm5yyqrbZ6KQCQUQQdAACiCIUNNbVEn4cTee27T+zSwLFwNpc1KczPAVBICDoAAETR1tkzplwtmlwKORLzcwAUFoIOAABR5Mth/e9eOle1xx8Xs4kCAOQrgg4AAFHk+mH98Q0TAKDQEHQAAIhiUW21PC6nuoIDUc/p2NGV55yoS8+qYfcGAMTAUAAAoioucmhdo1fSp4f47e7PLjhFV517khrmzCDkACh4BB0AAGJY5nXru0vnqWSK/f/f5fTyEi2ePcPqZQCAbVC6BgBAFNEGhdrZ+i+dzS4OAIxC0AEAYJxYg0LtiKYDABAdQQcAgFHiDQq1mkPSCVVl+vGfnqsDHw3SdAAA4iDoAAAwiplBoVaIRJk7rpyvJXNnWroWAMgFBB0AAEbp6rVfyJEkNyVqAJAUgg4AAJ/w+QO6/Ze7rV7GBLdfcZa+vqSWEjUASAJBBwAADYec1c3tVi8jqpmVZYQcAEiS/QcDAACQYUPHwrr1yTesXkZMNZVOq5cAADmHoAMAKGg+f0CL1/+7Dh05ZvVSojqurFiLaqutXgYA5BxK1wAABWvrGwFd/5g9y9Ui/vLC2SoucigUNtTW2aPuvgHaSgOACQQdAEBB2vrGB1r7+E6rlxFXRWmxvnXpPPn8ATW1dIxpe82gUACIj9I1AEDB8fkDuv6xnQrbcSroKD/+03O0raNLa5rbJ8z26QoOaE1zu3z+gEWrAwB7I+gAAApKKGyoqaXD6mXEdVzZFN23sl7LvG41tXQoWh6LvNbU0qGQ3RMbAFiA0jUAQEFp6+yZsDtiJxVlxWq/fZlKpxSpde/BuGs1JAWCA2rr7FHDnBnZWyQA5AB2dAAABSMUNvTynv+yehlx/fi/naPSKcP/77m7z1wgM3sdABQSdnQAAAXB5w/olid36/CRo1YvJarqihL9/Z+cPaa5gNn5OczZAYCJCDoAgLwVacm8raNL//TKPquXE9ftX5g/oYPaotpqeVxOdQUHop7TcUhyu5zM2QGAKAg6AIC8MH7OzKH+Qd359Ju2Po8zmrtq4q5McZFD6xq9WtPcLoc0JuxEJuisa/QyTwcAoiDoAAByXrQ5M7nEXVUWc1dmeZ1HG1fWT/j53MzRAYC4CDoAgJzm8we0prk9amlXrrjjyvlxd2WW13m0zOses2O1qLaanRwAiIOgAwDIWZGZOLkacqaVl+iuL51talemuMhBC2kASAJBBwCQs+w+EyeahadOU8OcGWqYPVOL58xgVwYAMoSgAwDIWbk4P+amy87Qkrkz414zvrECZWoAkDyCDgAgZ+Xa/Jjp5SVaPDt++Vm0xgoeGg8AQNKKrF4AAACpisyZyZW9jvVfOjvuzkykscL4cryu4IDWNLfL5w9keokAkDcIOgAAWwuFDbXuPajNu/arde9BhcKfth6IzJmxm/FZxuNy6r6V9XF3ZOI1Voi81tTSMebnBwDERukaAMC2zJRxLa/zaKm3Rts6uq1a5gRhQ7r9irM0s7LM9BmbRI0VDEmB4IDaOnvovgYAJrCjAwCwJbNlXC2/+cBWISdiZmWZrjr3JDWY7Kz2bEeXqfvmYgMGALACOzoAANsxW8Z17FhY33piVxZXZl4yjRJCYUNP7dqf9vsCQCFjRwcAYDtm5uMEggNaa9OQ43ENl6uZ1dbZo57+owmvm1FRmtR9AaCQEXQAALaT6+VZ6xq9Sc29MfvzXnXuiczTAQCTCDoAANvJ1fKsIof0j1+J310tGrM/7zKvO5VlAUBBIugAAGxnUW21qitKrF5G0r51yVytWJD8UE8z84CSLYcDgEJH0AEA2E5xkUN/cu5JVi8jabOPPy6l942eBzQ+7Dg++ZdsORwAFDqCDgDAlpbmYJnWZErultd5tHFlvdyusfdwu5zamGDYKABgItpLAwBsKVLOlaj7mp0c6h+a1PuX13m0zOtWW2ePuvsGTA8bBQBMxI4OAMCWioscuvKc3NrFuPPpDoXC0ab/mFdc5FDDnBlJDRsFAEyUVNDZuHGjFixYoKqqKlVVVamhoUG//vWv475n+/btWrhwoZxOp2bPnq377rtvUgsGABSGUNjQlt8ErF5GUgLBAbV19li9DACAkixdO/nkk3XXXXdp7ty5kqRf/OIXuuqqq7Rz507Nnz9/wvWdnZ1asWKFVq1apebmZr3yyiu6/vrrdfzxx+vqq69Oz08AAMgbobAxUrZ1oG8wp8rWInJ9BhAA5AuHYRiT2mOvrq7WP/zDP+i6666b8L2bb75ZW7Zs0Ztvvjny2urVq/Wb3/xGra2tpp/R29srl8ulYDCoqqqqySwXAGBTPn9ATS0dORluRnt81WI1zJlh9TIAIG+ZzQYpNyMIhUL613/9V/X396uhoSHqNa2trbrsssvGvPb5z39eDz30kI4ePaqSkugzEgYHBzU4ODjydW9vb6rLBADkAJ8/oDXN7Zrc6RZrOTTcIY1ZNwBgD0k3I9i9e7eOO+44lZWVafXq1Xrqqafk9XqjXtvV1aUTTjhhzGsnnHCCjh07pgMHDsR8xvr16+VyuUb+nXLKKckuEwCQI0JhQ00tHTkfciRm3QCAnSQddM444wzt2rVLO3bs0Jo1a/S1r31NHR0dMa93OMb+H/xIpdz410e79dZbFQwGR/69//77yS4TAJAj2jp7LC1Xq6ksnfQ9mHUDAPaTdOlaaWnpSDOC888/X6+++qp+8pOf6P77759wrdvtVldX15jXuru7NWXKFM2YEbt+uaysTGVlZckuDQCQg7qCH1v6/O6+yc2+uf2Ks/T1JbXs5ACAzUx6YKhhGGPO04zW0NCglpaWMa8988wzOv/882OezwEA5L9Id7VnfhvQP7/2B6uXk5LImRxCDgDYU1JB57bbbtPll1+uU045RX19fXriiSf0wgsvyOfzSRouOdu/f78eeeQRScMd1jZs2KCbbrpJq1atUmtrqx566CE9/vjj6f9JAAA5wecP6I4tv1VXb/T/SJYLOJMDAPaXVND58MMPde211yoQCMjlcmnBggXy+XxatmyZJCkQCOi9994bub62tlZbt27VjTfeqHvvvVcnnniifvrTnzJDBwBsaPQMm5rK4e5h6f4j3ucPaHVze1rvmUkel1NXnuPRlt8ExpwjcrucWtfo5UwOANjYpOfoZANzdAAgs6LNsPGk+Y/5UNjQwr/bpsNHjqblfpn03xtO0+V1npGwl40QCAAwJ+NzdAAA+SHWDJuu4IDWNLenrZvYjr0HcyLkSNLldZ4xQz+LixwMAQWAHJN0e2kAQP6IN8Mm8lpTS4dC4clv/re+E3t+ml04NLyTxdBPAMh9BB0AKGCJZtgYkgLBAbV19qThafYu9aLBAADkF0rXAKCAdfeZG9Rp9rpoIudbjKj7RtnjnFKk9V86W27XVB3qH9KdT3fQYAAA8hhBBwAKWE2lM63XjTfcSrpDXb2pB6V0GTgWlts1deSszefr3DQYAIA8RtABgAK2qLZaHpdTXcGBqPstkaGYqZxZsWMr6dE7UzQYAID8xhkdAChgxUUOrWv0Spp4gmYyZ1aGjoX1vX99Y/ILTLNUd6YAALmHoAMABW55nUcbV9bL7RobAtwuZ0qtpX3+gP7o75/VR4PH0rnMSaGbGgAUHkrXAABaXufRMu/kz6zEmsljJbqpAUBhIugAACRN/sxKKGzoji2/tTzkTHNO0eGBT3eT6KYGAIWJoAMASIsNz/1eXb2Dlq5henmJ/vO2pXr93UN0UwOAAkfQAQBMms8f0N3P/s7qZWj9l85W6ZQiuqkBAAg6AIDkRYaAdvcNaOZxZbrpX3ZZvSRdXuemPA0AMIKgAwBIytY3PtD3N/vV03/U6qWMccYJlVYvAQBgI7SXBgCYEgobur75NV3/2E7bhRxJ+ufX3lcobHUrBACAXbCjAwCIKVKi9mxHl37Ruk/HwlavKLZAcEA79h7UknkzrV4KAMAGCDoAgKh8/oCaWjoUCA5YvRTTbnisXXddfTZndQAABB0AwLDRDQb2HejX3c/usXpJSTv88VGtaW7XxpX1hB0AKHAEHQDIcaMDSqpzY3Jx9yaeppYOLfO6mZ8DAAWMoAMAOSxaQPG4nFrX6DW9o+HzB7SmuV35cozf0PB5nbbOHubpAEABo+saAOSoSEAZvwvTFRzQmuZ2+fyBhPcIhQ01tXTkTcgZrbsvP3anAACpIegAQA6KF1AirzW1dMRstxwKG2rde1B3b3s7b8rVxqupdFq9BACAhShdA4Ac1NbZEzegxCvfyrfzONF4XMNnlQAAhYugAwA5yGxZ1vjrtr7xga5/bGcmlmQr6xq9NCIAgAJH6RoA5CCzZVmjr9v6RkBrH8//kHPj0tNpLQ0AIOgAQC5aVFstj8upWHsWDo0t3/L5A7r+sXbFOLKTN9xVZVp7yVyrlwEAsAGCDgDkoOIih9Y1eiVpQtiJfB0p34o0LrCzkjT8fyOHpDuunE/JGgBAEkEHAHLW8jqPNq6sl9s1tozN7XJq48r6kfKtRI0L7OCuLy2Qx5V6l7TqipIxPzMAADQjAIActrzOo2Vet9o6e9TdN6CayuFytdG7GnafJ3PajKm6+vxTVOGckvLg0tu/MJ+QAwAYg6ADADYzdCysR1v36d2eIzqtulzXNsxS6ZTYG/DFRY4JLaRHs/M8mVkzpuqFv75E0qc7VKm0vnZX2fdnBABYg6ADADayfmuHHnypc0zTgB9sfVOrLqrVrSu8Kd0z0rjAjuVrNy47Y8zX43eoZh5Xpr/6l136sHcw6k6PQ8OleszMAQCMxxkdALCJHzzdoftf7JzQGS1sSPe/2Kn1W1NrKFBc5NDtV6QWkjJt5nFlE16L7FBdde5JWjJ3pu64cr6kxE0XAAAYjaADADbwq10f6MGXOuNe88BLnRo6Fk7p/tMrSlN6X8aZOJBjtukCAACjUboGABbz+QNa+0TiQZ6GIf3iP/Zp1cWzk35GV/DjVJaWcQf6B01dZ6bpAgAAoxF0AMBCyc64eXXfwZSCTk//UNLvyYZkGiUkaroAAMBoBB0AsFCyM27KSz/9P9tmurOFwobaOnu07+CRtK05HWgiAADINIIOAFgo2Rk3V9efLMlcdzafP5BSq+ZsMEQTAQBAZhF0AMBCyZRuVZQW64/nztT6rcPd2caLdGeTpPNOnZ7y8M1s+Isls2giAADIKIIOAFgoMuOmKziQMJT8+E/PUShsJOzO9uBLnTr+uA9sG3IkaZnXHff7kZI7Gg8AAFJF0AEACxUXObSu0as1ze1yKHq35WlTp+iuqxdoeZ1HD730zoQ5O+OFDenDPnPdzKxQXVGihadNj/n9aCV3HpdT6xq97AIBAExjjg4AWCzWnJhpU0t049J5ev32y0b+wH+3x15NBVLR039Un/mH5+XzByZ8z+cPaE1z+4RzRV3BAa1pbo/6HgAAomFHBwBswOycmNOqyy1aYXpFgsvogZ+RVtvRNqwMDXdqa2rp0DKvmzI2AEBC7OgAgE1E5sRcde5JapgzI+of89c2zFI+/I0fCTNNLR0KfVKLl6jVtiEpEBxQW2dP5hcIAMh5BB0AyCGlU4q06qJaq5eRFuODi9lW28m25AYAFCaCDgDkmL9Zfpa+sMCjPNjYkfRpcDHbajuZltwAgMJF0AGAHOLzB3ThD5/Tr94I2LJ99HFlxfr8/BNUXVFq+j2R4BJptR0rwDk03H1tUW315BcKAMh7NCMAgBwR6Uhmx4Dz3xtO0+V1Hi2qrVZbZ4/+328/NPW+0cElXqvtSPhZ1+ilEQEAwBR2dAAgB8TrSGYHl9d5RhooJHOGZnxwidVq2+1yjunQBgBAIuzoAIBNhcKGdrxzUK17D+oPh47E7UhmpfHlZGbP0Ny49PSowcVsq20AAOIh6ACADfn8Ad286Q0FPz5m9VLicmjirkzkrE1XcCDmDpS7qkxrL5kb876RVtsAAKSK0jUAsBmfP6DVze22DzkVZcVRy8kiZ20kTWgs4Pjk3x1XzmeHBgCQUQQdALCRUNjQLZt2W72MhBpmT9cb6z4f88wMZ20AAFajdA0AbGTH3oM6/PFRq5eR0J8vOi3hjgxnbQAAViLoAICNtL5zwOolmLLvQL+p6zhrAwCwCqVrAGArubHbcfeze+TzB6xeBgAAMRF0AMBCobCh1r0HtXnXfrXuPag/GtWm2c4ckppaOhQK23WyDwCg0FG6BgAW8fkDamrpGDMfx11VporSYvUPhSxcWWKGpEBwQG2dPZSmAQBsiaADABbw+QNa09w+Yc7Mh72DMWfP2FF3nz2HmAIAQOkaAGRZKGyoqaUjaqAxNFwWNm3qlJw4rVNT6Ux8EQAAFmBHBwCyrK2zZ0y52niGpMM2HxYqSR7XcLtoAADsiB0dAMiyfCn3WtfoZSYOAMC22NEBgCyzY7nXxfNm6jOnHy/X1BJ979/eSHj9dUtmaXmdZ8LrobDBgFAAgC0QdAAgyxbVVsvjcqorOGB54wHX1Cn64dULtLzOI58/oNue2m3qfUu97gmvResi53E5ta7RGzUUAQCQSZSuAUCaDB0L66GX3tH/3OzXQy+9o6Fj4ajXFRc5dPsVZ1keciSp95OzQJEucD39R+Ne71D0szmR948/e9QVHNCa5naGiwIAso4dHQBIg/VbO/TgS50aPT/zB1vf1KqLanXrCu+Ya33+gO58+s0srzC2ppYOGYaRMHhFCtDGn80x00WuqaVDy7xuytgAAFnDjg4ATNL6rR26/8WxIUeSwoZ0/4udWr+1Y+S1WDsfVokM/uzqHUx4bXVFqTaurJ9Qhmami1xkuCgAANlC0AGASRg6FtaDL3XGvebBlzo1dCwcd+cjF3z/irNGQk4obKh170Ft3rVfr/z+v0y9P1+6zQEAcgOlawAwCY+27puwkzNe2Bi+znuiyzY7Oalwu6ZKit50wAw7dpsDAOQvgg4ATMK7PUdMX/fB4Y8zvJrkOSS5XU4ZhqEPewej7jZFrllUWz1SepfMrtTo9wMAkC0EHQCYhNOqy01dd2QwpH9r/0OGV5OadY3DzRLWNLfLIY0JMY5x1yRbehergQEAAJnGGR0AmIRrG2Yp0d/vRQ7ppT3mzrFkk7uqbKS5wPI6jzaurJfbNba8zO1yjlyTqOlA1GeMej8AANnEjg4AJBAKG2rr7FF334BqKodLsCK7E6VTirTqolrd/2LshgQrzvboV2/Ya47MjUtP19pL5o7ZZVle59Eyrzvmz2q2mcDaz83VvBOOm/B+AACyiaADAHFEO3jvcTm1rtE7sksRmZMzfo5OkUNadVGtvCe6bBN0ihzShmvqtWJB9B2W4iKHGubMiPo9s80ElsydGfMeAABkC0EHAGKIdfC+KzigNc3tY0qybl3h1V9ddqYebd2nd3uO6LTqcl3bMEulU4r0k2f3ZH3t48/aRGy45rwxISfebtV4i2qr5XE51RUcSNi0AAAAqxF0ACCKeDNvDA3/Ud/U0qFlXveYMrbrLpo94T6Pt72X8fVGTC8v0Q++WKc7n34z7i6UZG63arTiIofWNXoTNi2gVA0AYAcEHQCIItHBe0NSIDigts6euGVaO/YeVFdv9mbnrP/S2Vpe59HnP2keEGunJpndqtEiTQvGByR3nIAEAIAVCDoAEIXZg/fxrvP5A7rpX36TriXFVVFWrB//t3NGgka8szap7FaNlqhpAQAAdkDQAYAozB68f/6tbs08rkyLZ8+YsGOyurk9U8ub4MhgSJeceYKpa9OxWxUvSAEAYAcEHQCIItHB+4hf7vpAv9z1gaaVl+iuT8rGQmFDtzy5O2trlYbDyS/+Y5/qTnIl3GVJx24VAAB2R9ABgCjiHbyP5vCRo1rd3K4bl54uwwjr8JGj2VjmGHdv+52OHA2NfB2rsYDZ3Sqz1wEAYEdFVi8AAOwqcvDe7TL/B//dz/5O9/z77zO4qthGhxzp08YCPv/YGT6R3apYJ2ocGg5JtIkGAOQygg4AxLG8zqOXb75Ef7viLKuXkrTILlRTS4dCoyaZRnarJE0IO7SJBgDkC4IOACRQXORQ8OMhq5eRktGNBUaLtVvldjljtpYGACCXcEYHAEzJ7d2NaI0FaBMNAMhnBB0AMGFKjv/xH6uxAG2iAQD5iqADAAmEwoZ+/h/7rF6Gvlx/ssrLinVadbmmlZfqr/7V3DBSGgsAAAoRQQcAEvg//75Hhz/OfrvoiCKHtOqiWt26wjvyWuveg6bfT2MBAEAhIugAsLVQ2LD0DInPH9A9/74na8+LJmxID7zYqfNOnT7SJMDMQNMih7ThmvOy1ljA6s8KAIDRCDoAbMvnD6ippUOB4KcH6WMNwcyEUNhQU0tHxp9jVlNLh5Z53SoucpgaaLrhmnqtWJCdkGP1ZwUAwHi0lwZgSz5/QGua28f84SzFHoKZTkPHwnropXe0pvn1Cc+3SrQ20bFaRHtcTt23Mrshx6rPCgCAWNjRAWA7kZ2UaLsUhoYbPY/e3Rj9vsmWTq3f2qEHX+pUOFY9mMXGt4m2ukV0qp8VAACZRtABYDttnT1xd1JG725EWiOno3Rq/dYO3f9i56TWnmnR2kRb2SI6lc8KAIBsIOgAsJ1owy3jXRcpnRq/qxApndq4sj5q2Bm9A1RdXqoHbBxyHJLcNmwTnexnBQBAthB0ANhOrOGW0a5LtXQq2g6QXUVWbcc20cl8VgAAZBPNCADYTqR1cqw/6R36dAhmMqVTEbEOz9uV2+WMuStltWQ+KwAAsokdHQC2E6918vjdjWRLp+LtANmFu6pM1yw6VbNmVth+Hk0ynxUAANlE0AFgS5HWyePLy9zjGgwkWzqVaAfIardefqb+8qLZORUMzH5WAABkE0EHgG2ZaZ0cKZ3qCg5E3aUZf4jf7ofi3S5nToWcCKvbXAMAMB5BB4CtJWqdnGzplN0Pxdt9ffFY2eYaAIDxaEYAIOdFSqfcrrEhIdoh/sgOkN1waB8AgPRiRwdAXjBbOlVc5NAXFnj04Ev2m5nDoX0AANKHoAMgbyQqnQqFDe3Ye1CPtL6bxVUlNm1qie66+mwO7QMAkEYEHQB5LRQ21NbZo2c7uvTUrv3q6T9q9ZImuPer9Voyd6bVywAAIK8QdADkLZ8/MKHlsZ1EOsItns0BfgAA0o2gAyAv+fwBrWlut+1gUIZpAgCQWXRdA5B3QmFDtzy521YhZ9rUkjFfR+sIBwAA0ocdHQAZFzknk+5BkrHuu+G5PTp8xF5nce79ar2KHA6GaQIAkCUEHQAZFe2cjMfl1LpG76R2M2Ld9/YrvPqnV+zVOtrzyTkcgg0AANlD6RqAjImckxnfDCAQHNDq5nb5/IG03rcrOKDrH2tX8ONjKa85EziHAwBA9hF0AGREKGyoqaUj7jmZW57crVA4uZM08c7f2OlMTsT/ueY8zuEAAGABgg6AjGjr7EnY1vnwkaPa8Nzvk7qvHc/fxLLqollqPOdEq5cBAEBBIugAyIjuPnOzax7+j86YuzqhsKHWvQe1edd+te49qKFjYT38yr40rjIzihzSNy+u1d9eMd/qpQAAULBoRgAgI2oqnaauO3zkqNo6e9QwZ+zQzGjNBqorSnX4Y3vt5hQ5pHv+2zn6r/4hvdtzRKdVl+vahlkqncJ/RwIAwEoEHQAZcah/UA6ZOzczfvcn1rDPnv6hdC0vbTZcU68VCziDAwCA3RB0AKSdzx/QDY/tNN0coLt3QD/6f29JcuiPaqt1x5bf2rKxwGgVJUX68Z+dS6MBAABsiqADIGXRBnZKSthtbTSHpB9sfWvk6w3Pp3+dmfDtpacTcgAAsDGCDoCUxBrY+ecXnJKw29podt+5iaWmytwZJAAAYA2CDoCkxTpD0xUc0N3P7rFkTdnmJugAAGBrtAUCkJR4g0BzdXcmWR7Xp2V6AADAnpIKOuvXr9cFF1ygyspK1dTU6Itf/KLefvvtuO954YUX5HA4Jvx766234r4PgD2ZGQQaj0PStKkl6VtQFjk++beu0aviIofVywEAAHEkFXS2b9+uG264QTt27NC2bdt07NgxXXbZZerv70/43rfffluBQGDk37x581JeNADrmB0EKg2Hgmhff2PJrHQtJ6vcLqc2rqynCQEAADkgqTM6Pp9vzNcPP/ywampq9Prrr+viiy+O+96amhpNmzYt6QUCsBezg0BvXHq6nnj1vTG7P26XU+savVrmdevxtvfU1TuYqWWmhbuqTNcsOlWzZlaMdJVjJwcAgNwwqWYEwWBQklRdnbhW/bzzztPAwIC8Xq++//3v63Of+1zMawcHBzU4+OkfQL29vZNZJoA0WlRbrWnlJTp85GjMa6aVl2jtJXO19pK5E9pPR4LCHVfO1+rm9mwt27TFtdN1zR+dNqZdduRnaOvsIewAAJAjUg46hmHopptu0oUXXqi6urqY13k8Hj3wwANauHChBgcH9eijj+rSSy/VCy+8EHMXaP369Wpqakp1aQAsFokBxUUONcyZEfWa5XUe3beyXrc8uXtCaHI4JMOCzgZlxQ7931UNI0EmVgvtdY1eytcAALA5h2Gk9ufEDTfcoKefflovv/yyTj755KTe29jYKIfDoS1btkT9frQdnVNOOUXBYFBVVVWpLBdAmrTuPahrHtyR8LrHVy2OGXJGC4UN7dh7UK3vHJA0HIyCR4Z0/WM707Da5Hzn0nm6cdnpkmK30I6EOM7qAABgjd7eXrlcroTZIKUdnW9961vasmWLXnzxxaRDjiQtXrxYzc3NMb9fVlamsrKyVJYGIMPMNiMwe11xkUNL5s3UknkzR17z+QMprW0yykuL9e1Lh5ukJGqh7ZDU1NKhZV43ZWwAANhUUl3XDMPQ2rVr9eSTT+q5555TbW1tSg/duXOnPB7+SyiQi8w2IzB73XiRkJFt//tPzxkJLYlaaBuSAsHhMzsAAMCektrRueGGG/TYY49p8+bNqqysVFdXlyTJ5XJp6tSpkqRbb71V+/fv1yOPPCJJuueeezRr1izNnz9fQ0NDam5u1qZNm7Rp06Y0/ygAsmFRbbU8Lqe6ggNRdzwcGu6ulspAzVDY0M9f6ZzUnJ5kuavKdMeV88eUoaV71woAAGRfUkFn48aNkqTPfvazY15/+OGH9fWvf12SFAgE9N577418b2hoSN/73ve0f/9+TZ06VfPnz9fTTz+tFStWTG7lANIqFDZidkgbrbjIoXWNXq1pbpdDGhN2IlenMlAz2sH/TJheXqIN19TrQP+gZlaUSQ7pwEeDat17cORnzvSuFQAAyLyUmxFkk9kDRwBSk0p3sXR2JIt18D8T7vukiUC89S/zunXhD59LuGv18s2XcEYHAIAsM5sNCDpAgZtMdzGzu0DxhMKGLvzhc1krV7tvZb0kJfyZI9dI0Xet6LoGAIA1CDoAEkoUMrKxc2G2XXW6eFxOGYahrt7BqN8f/TNv6+hijg4AADaT0fbSAPJDMt3FzMzEScW2jq6M3DeWRDtHo3/m5XUeLfO6J71rBQAAso+gAxQwq7uLbd65X//0yr6M3HuyIj9zcZEjYyEPAABkDkEHKGBWdhf7y1+06dk3/yvt900XOqoBAJDbkhoYCiC/RGbixCrEcmj4TEoqM3Hiue7n1oScyM/jrirL+s8MAACyi6ADFLDITBxJE/7wn8xMnHiaNvv1729ZE3Kk4Z/njivnj3kt2jWcwwEAILcRdIACt7zOo40r6+V2jS3VcrucaW2hPHQsrMvu3q6HW99Ny/2SNfrnydbPDAAArEN7aQCSkpuJk+z8nPVbO3T/i52ZWnpc06aW6N6v1mvx7BkT1piOOUAAACC7aC8NICmxuouNDwOH+gd159Nvmp4tY2XIcUi66+qztWTuzKjfp6MaAAD5i6ADICafPzBhYGY0XcEBrWlun1D2NXQsrAdfsibknFBZqqar6ihDAwCgQHFGB0BUPn9Aa5rbE4YcaXjIpiQ1tXQoFP60GvbR1n0KW1Qc+7//7DxCDgAABYwdHQAThMKGmlo6lExGMSQFggPasfegwoahJ9v/oNfePZSpJSbUuvdA1HM5AACgMBB0AEzQ1tljaicnmr/4xasaPBZO84qSt+H5vdrUvj/m2SEAAJDfKF0DMEF3X2ohR5ItQk5E5OyQzx+weikAACDLCDoAxgiFDR3oG7R6GWkR6+wQAADIf5SuAQUk0dwYs13Wcknk7FBbZw+tpAEAKCAEHaBARAsxo+ffRLqs5eu+x2TK8QAAQO4h6AB5avTuzb4D/br72T0TromcYbn3K+fpzqffzMmQ45BMrbum0pnppQAAABsh6AB5yGwJmqHhoPD9zX719B/NytrS5bOnz9RF847XV/7oNF3y4xfUFRyIGngcktyu4TI9AABQOAg6QJ5JtgTNkEyHHK+nSh2B3pTXlg7VFSX6+z85e0zL6HWNXq1pbp+wu+MY9X3m6QAAUFjougbkkVQGfSbD6pAjSbd/Yf6EuTjL6zzauLJebtfY8jS3y6mNK+uZowMAQAFiRwfII5MZ9FldUapD/UMxQ1KRQ7JDh2Z3VfSzNsvrPFrmdcftKgcAAAoHQQfII6l0FoucYbn9Cq9ueCx6+Zch60OOmbM2xUUOWkgDAABJlK4BeSXZzmKjz7CsWBC7/Ou6JbPSs8AUcdYGAAAkix0dII8sqq2Wx+WM2YFsPPeoOTpS7PKvts4ePfTKvoyuvbqiVGs+M1uv/P6AXn/vkPoGQjHXCQAAkAhBB8gjxUWOhB3Ivrv0dM2aWR7zDEu08q+Fp03P+Bmdnv4h1Z00TasunjNmBhBnbQAAQCoIOkCeiXQgGz9HJ9VdkVDY0KOt+7JyRidyxoizNgAAYLIIOkAeSqUDWbRdlG0dXaYGj6ZLsmeMAAAAYiHoAHkqmV0Rnz8wIdCUlxbryFAozrvSx0xHNQAAgGQQdIA8Zuasi88f0Jrm9gnNC7IVciLoqAYAANKJoAPkqWi7NJ5x53RCYUNNLR2mOrRlyvg1AQAApANzdIA8FNmlGX+2pis4oDXN7fL5A5Kkts6erJ2/iebGpfP08s2XEHIAAEDaEXSAPBNvlybyWlNLh0JhY6TLWbZ5XE7dt7Je31l6OuVqAAAgIyhdA/LMjr0H4+7SGJICwQG1dfZkrcvZdy6dp8WzZzAXBwAAZA1BB8gjPn9At2zabera7r4BfWHBifK4nBktX3NXlenbl84j2AAAgKyidA3IE5FzOYc/Pmrq+gN9g5KGu51l0lXnnkjIAQAAWUfQAfJAKt3T7nz6TV34w+d07Fhme65t+U1AobCVfd0AAEAhIugAeSDV7mldwQGtfWJnBlb0qch5IAAAgGwi6AB5INXuadnaZ7GquxsAAChcBB0gD2Sre1qq7L4+AACQfwg6QB5YVFstj8spux35d2h4Zs6i2mqrlwIAAAoMQQfIcaGwobbOHl1e55Yh2SbsRNaxrtFL1zUAAJB1zNEBcpjPH1BTS8eYRgQOh2SMOnzjripTz5GjGjoWzuhaqitK1dM/9OlzXU6ta/RqeZ0no88FAACIhqAD5KjI3JzxDQUinZyvWzJLS71uPffWh3rwpc6MrsXjcmr7X39Or797SN19A6qpHC5XYycHAABYhaAD5KBEc3Mckrb6u/S9z5+pr/5sR8bXc+U5HpVOKVLDnBkZfxYAAIAZnNEBLBIKG2rde1Cbd+1X696DSQ3VTDQ3x9Dw/JofPP1bZWNWJ0NBAQCA3bCjA1gg2tkaTxJnWszOpXn81T+kvMZkRIaCsqMDAADsgh0dIMsiZ2vG78h0BQe0prldPn8g4T3MzqXJ5i4LQ0EBAICdEHSALIp3tibyWlNLR8KAYse5OQwFBQAAdkLQAbLI7Nmats6euPcpLnJoXaNXkvVzcxgKCgAA7IigA2SR2fIuM9ctr/No48p6nVBVNtllpYyhoAAAwK5oRgBkkdnyrmTKwAaOZnYQaDwMBQUAAHZF0AGyKHK2pis4EHMGzrSpJQobhkJhY2SXJBQ21NbZM2YY57aOrqgDQ9OpyKEx7ak9Lqf+/IJTNWtmOUNBAQCArTkMw7D98Ive3l65XC4Fg0FVVVVZvRxgUiJd1yTFDSnuKqeuWXSqgh8P6Ze7PlBP/9CY7w0cC+nwkaMZWePUkiL97GsX6IJZ1Xr93UNjAhbBBgAAWMlsNiDoABaINkfHTu5bWU85GgAAsCWz2YBmBIAFltd59PLNl+j/XvdHmja1xOrljPC4nIQcAACQFzijA1ggcuam9Z0DOvxxZsrPkvXl+pP0wy+fQ2kaAADICwQdIMvsWrZ20enHE3IAAEDeIOgAWRRpRGDHg3HJtLQGAACwO87oAFkSChtqaumwZcjxuIY7qgEAAOQLgg6QJW2dPbYrV5Mkh6R1jV7K1gAAQF6hdA3Iku4++4Ucj8updY1euqwBAIC8Q9ABssQuZ2CWnVWjy+s88kybygBQAACQtwg6QJYsqq2Wx+VUV3DAknM6Dklul1P3XXs+4QYAAOQ9zugAWVJc5NC6Rq+lzQg4iwMAAAoFQQfIsmnlJVFf++bFtfK4MlPeNq28RBtX1nMWBwAAFAxK14AsiTdDJ3jkqM47dbr+ZvlZauvsUXffgF763QH9W/sf0vLse6+p15J5M9NyLwAAgFzAjg6QBYlm6BiSmlo6JEkNc2boqnNP0g+/vEDHlU3+v0VMLy/R4jkzJn0fAACAXELQAbLAzAydQHBAP3+lU6HwcBwqLnLoT88/edLPPnTkqLZ1dE36PgAAALmEoANkQVevuRk6dz79pi784XPy+QOSpGVe96Sf7dDwblEkQAEAABQCgg6QBT0fDZq+tis4oDXN7fL5AyMtqSfD0PBuUVtnz6TuAwAAkEsIOkAWTJs6sdNaLJF9l8iZnXWNXqWjIXR3n7ldJQAAgHxA0AGy4PDHR5O6fvQuzPI6jzaurI+5s1NdYS5E1VRmpnU1AACAHdFeGsiC6uPKUnpfZBdmeZ1Hy7xutXX2qKt3QD0fDaq6olRu11QtPG26Fv39szp8JHaYmlZeokW11SmtAQAAIBcRdIAscFeltpsyehemuMihhihtos00GUhH6RsAAEAuoXQNyIJD/eabEUjDwcTjcprahWnr7Im7myMNt5imGQEAACgkBB0gw0JhQ3c+/WZS7zE03ISguCjxXozZJgM0IwAAAIWEoANkmJlhoeNNLy8xPUPHbJMBmhEAAIBCQtABMiyVnZRkSs0is3Zi7f0kUwYHAACQLwg6QIbtO9Cf0vvMBqTiIofWNXolTWw6EPnabBkcAABAviDoABnk8wd097N7UnpvMqVmkVk77nGzdtwupzaurNfyOk9KawAAAMhVtJcG0igUNtTW2aPuvgHNrCjTHVs6UrpPdUWJFp42Pan3jJ610903oJrK4XI1dnIAAEAhIugAaeLzB9TU0pF044FoevqP6jP/8LzWNXqT2o2JNWsHAACg0FC6BqSBzx/Qmub2tISciK7ggNY0t8vnD6TtngAAAIWCoANMUihsqKmlQ0aa7xu5X1NLh0LhdN8dAAAgvxF0gElKZU6OWYakQHDAdKtpAAAADCPoAJOUypwcOz4DAAAgnxB0gEkIhQ0d6BvM+HOSaTUNAAAAuq4BCX08FNLfb+3QvoNHNGtGuW5b4dXU0uK0dlmLxaHhWTiLaqsz9gwAAIB8RNBBxo2eLZNrs11WPfKqtnV0j3z90h7p0R3vac7x5dr7X0cy+uzI/4TWNXpz5n9eAAAAdkHQQUZF2/XwuJxJz4exwviQM1qmQ440vJOTC/9zAgAAsCOCDjImMltmfGPkyHyYjSvrbftH/MdDoZghJ5O+eO6J+swZNXJX5dbOFwAAgN3QjAAZEW+2TC7Mh/n7rR2WPPeXuz7Q//K9peDHQ4QcAACASSDoICMSzZax+3yYfQczX5oWS2THy+cPWLYGAACAXEfpGjLC7NyXbM+HidcYYfT3nFOs+28AhoYbETS1dGiZ183ODgAAQAoIOsgIs3NfsjkfJl5jBEkZbxUd4ZCilvSNNnrHq2HOjIyvCQAAIN8QdJARi2qr5XE51RUciPpHfbbnw8RrjLC6uT0ra4j49qXzFAob2vD87xNem+0dLwAAgHzBGR1kRHGRY2SnZHzhVbbnw5hpjJAt08pL9O1L52nJ3Jmmrs/mjhcAAEA+IeggrUJhQ617D2rzrv1yTS3VvV85T27X2D/W3S5nVltLJ2qMkE13felsFRc5Rna8YsU8h4bL6rK14wUAAJBvKF1D2sQ6A3P7FV5NryiN2gAgG+xQ/jV+SGpkx2tNc/uEMzvZ3vECAADIRwQdpEW8MzA3PDY8HPSqc0+yZG1WlX9VOafoywtP1jKvO2q4W17n0caV9RPCoXtcKAIAAEDyCDqYtERnYKxulbyotlruqjJ19Q5m7ZnVFSXacetSlSZoU728zqNlXnfMltcAAABIDWd0MGl2Hw5aXOTQNYtOzcqzHJ/8+/s/OTthyIkoLnKoYc4MXXXuSWqYM4OQAwAAkAYEHUyaXYeDjjZrZkXS70klb2S70QIAAACio3QNk2bH4aCTefZ1S2bpxGlTdefTbya89m9XnCWvp0oH+gcpOwMAALARgg4mLZPDQUNhIy3nVxKtcbSt/i79zfIzTd23pqpMS+aZm4kDAACA7CHoYNIy1So5VrvqVDqSRda4urk94bWB4IB6PjLXuICBngAAAPbEGR2kRaRVcrqGg0baVY9vctAVHNCa5nb5/IGk7hcKG6p0luick6tMXV9dUcpATwAAgBzGjg7SJl2tktPdrtrnD+iWJ3fr8JGjptfQ0z+k26/w6obHGOgJAACQiwg6SKtIq+TJSKZddeRZsc7y+PwBU+Vq49359JtyV5Xpf1xcqy2/CTDQEwAAIMcQdGA7ybarjnWW5/YrztL/96vEndNi6eod1P0vduofv3KepleUMdATAAAghxB0YDvJtKuOnOUZX+bWFRzQ9Y/tTMt6bvulX69/fxnhBgAAIIfQjAC2E2kFnagRwMLTpsc9y5Muh48c1Y69B9N4RwAAAGQaQQe2E2kFLWlC2BndCOD1dw/FPcuTTq3vHMjKcwAAAJAeBB3Ykpl21WbP8qQHZWsAAAC5hDM6sK1E7aqzOaxzsp3kAAAAkF0EHdhavHbVkbM8mS5fm15eosWzCToAAAC5hNI15LQ/v+CUjD9j/ZfOpuMaAABAjmFHBzknFDa04bnf6+FXOnX446Npu2+RQwqPatfmYTAoAABAzkpqR2f9+vW64IILVFlZqZqaGn3xi1/U22+/nfB927dv18KFC+V0OjV79mzdd999KS8Yhc3nD2jh323T3c/+Lq0hRxoOObdfcZZ+8ufn6vFVi/XyzZcQcgAAAHJUUkFn+/btuuGGG7Rjxw5t27ZNx44d02WXXab+/v6Y7+ns7NSKFSt00UUXaefOnbrtttv07W9/W5s2bZr04lFYfP6AVje36/CR9Aac0WZWlumqc09Sw5wZlKsBAADksKRK13w+35ivH374YdXU1Oj111/XxRdfHPU99913n0499VTdc889kqSzzjpLr732mn70ox/p6quvjvqewcFBDQ4Ojnzd29ubzDKRh0JhQ00tHRl/zsyKsow/AwAAAJk3qWYEwWBQklRdXR3zmtbWVl122WVjXvv85z+v1157TUePRv8v8+vXr5fL5Rr5d8opmT9wDvsaOhbWHVv82RkOyiYOAABAXkg56BiGoZtuukkXXnih6urqYl7X1dWlE044YcxrJ5xwgo4dO6YDB6JPm7/11lsVDAZH/r3//vupLhM5bv3WDp15+6/16I73svK8Ax8NRn09FDbUuvegNu/ar9a9BxUa3bUAAAAAtpNy17W1a9fqjTfe0Msvv5zwWodj7H8mNwwj6usRZWVlKiujhKjQrd/aoftf7MzqM6MNIfX5A2pq6Rizo0RHNgAAAHtLaUfnW9/6lrZs2aLnn39eJ598ctxr3W63urq6xrzW3d2tKVOmaMYMhjAiuqFjYT34UvZCjkPD4WVR7dgyTJ8/oDXN7RPK5rqCA1rT3C6fP5C1NQIAAMC8pIKOYRhau3atnnzyST333HOqra1N+J6GhgZt27ZtzGvPPPOMzj//fJWUlCS3WuSdWCVhj7buU7arw9Y1esd0Wos0QIi2jMhrTS0dlLEBAADYUFKlazfccIMee+wxbd68WZWVlSM7NS6XS1OnTpU0fL5m//79euSRRyRJq1ev1oYNG3TTTTdp1apVam1t1UMPPaTHH388zT8Kck28krB9B4+kdM9p5SVyTilWV6/5xgWxytDaOnviNkAwJAWCA2rr7FHDHHYnAQAA7CSpoLNx40ZJ0mc/+9kxrz/88MP6+te/LkkKBAJ6771PD47X1tZq69atuvHGG3XvvffqxBNP1E9/+tOYraVRGCIlYeP3QrqCA1rd3J5yl4zDR47q/15Xr6Iih57t6NJTu/arp39id78ZFaW66twTtczr1qLa6qgzc7r7zIUls9cBAAAge5IKOpEmAvH8/Oc/n/DaZz7zGbW3tyfzKOQxMyVh4Unc/0D/4MjQz9uu8Kqts0ddvQPq+WhQ1RWlcrumxgw3o0VrTDCZ6wAAAJA9KXddA6Th0NLW2aPuvgHVVDpNBYhEJWGTNTp4FBc5Ui4rW1RbLY/Lqa7gQNRQ5pDkjtLAAAAAANYj6CBl0c7YVFeU6otpKglLxYyK0rQFj+Iih9Y1erWmuV0OaUzYifxU4xsYAAAAwB5SHhiKwhar7XJP/5D+6ZV9uubBHbrwh89Fbb+cyVKvO6+qS2vwWF7n0caV9XK7xq7Z7XJq48p65ugAAADYFDs6mCBROVq8MzajBT6ZNTM+EERKwtJdvrbqollasSD9wWN5nUfLvO6kS/QAAABgHYIOxojX8jkSVpI9Y9PU0qFlXvdIMIiUhK1uTm+DipbfBLTwtOqM7LJM5qwPAAAAso/SNYyIVY7W9cnOTKQMLZkzNqNnzYy2vM6jG5eePuk1j9bVOzhmnQAAAChcBJ0CFwobat17UE/t3K/bntodt+VzU0uHQmEjpTM20cLR2kvmyl1VlvS9EomsEwAAAIWL0rUCFq1MLZbROzOJ2i5HM7OiTK17D04443L2yS51dXRP6ueItU5KzQAAAAoXQadARcrUkt336O4biNt2eTyHJFd5if7qX3+jrt6x537qTqrStjSGnPHrBAAAQOGidK0Ame2aFk2kbC1W2+XRIiHo8JGjY0KONHzuJ1MhZ/Q6AQAAUJjY0SlAyXZNi3BXlY0Zxjm67fKzHV16atd+9fQfHfl+1dQpGjwW1sDR8IR7ZeoEjUPDM27SNTQUAAAAuYkdnQKUalnXNYtOnTA7JtJ2+fbG+Xr1b5fpxqXzNG1qiSQp+PGxqCEnUyIrW9foZcYNAABAgWNHpwClWtY1a2ZF3O9v6+jSPc/uydhuzXhFDml0czX3uHk/AAAAKFwEnQKUStc0KX5Amsy5n1Q4JG24pl7TK0ondHIDAAAACDoFKJmuaZK5cy+pnvtJhYedGwAAACRA0ClQka5piebomD33sq2jK80rHGt6eYm+dN5JWup1s3MDAACAhAg6BWx017TuvgHtO3BEj7e9N6YVtJlzL6GwoX957Q8ZXeuGr9RrydyZGX0GAAAA8gdBp8BFuqZFrL1k7kjwMXvuZcNze/TR4LGMrvPAR4MZvT8AAADyC0EHY4wPPomEwoYefmVfSs8ycz4oggGgAAAASAZzdDApbZ09Ovzx0cQXSqooLR7ztdvlHJm5E8/4QaUAAABAIuzoYFLMDh91OKTXvr9Mu94/PFIWFw4b+upD/5nwvdEGlQIAAADxEHQwKWZLygxD2vX+4TFlcZt37Tf13kSDSgEAAIDxKF1DSkJhQ617D6or+LEqyooTv0ETd3/MhiTO5wAAACBZ7OggaVvfCOj7m/3q6R9K6n17PuxT696DI53cFtVWy+Nyqis4ELUpgZlBpQAAAEA07OggKeu3duj6x9qTDjmStOH5vbrmwR268IfPyecPqLjIoXWNXkmfDiaNMDuoFAAAAIiGoAPTtr7xge5/sXPS9wkEB7SmuV0+f0DL6zzauLJebtfY8jS3y6mNK+vjDioFAAAAYqF0LU+EwkbSgz6Tvf/3N/vTdj9DUlNLh5Z53Vpe59Eyrzuj6wcAAEBhIejkAZ8/oKaWDgWCnx7297icWtfoTduOSFtnj3r6zc3LWfu5OZIc2vD87+NeFwgOqK2zRw1zZiQ9qBQAAACIh9K1HOfzB7SmuX1MyJGkrlHlYfFEuqdt3rVfrXsPKhSO1hbA/LwcSVoy93jNOd5cS+iu4Mem7wsAAACYxY5ODguFDTW1dETtWGZo+EB/pDwsWhlYMjtBZls8z6go1aLaav38FXNneVJpagAAAAAkwo5ODmvr7JmwkzOaoU/Lw8ZLdico0go6kTuvqpNkPsBUH1dm6joAAAAgGQSdHGa2nGz8dYl2gqThnaDRZWyRVtDx2gN88+JaFRVJF/7wOd37wl5Ta3NXMQwUAAAA6UfQyWFmy8nGX5fqTlCkFfT4nZ3qihL941fO03mnTo+6SxSLh2GgAAAAyBDO6OSwSDlZV3Ag6u6MQ8PzaMaHiVR3giTFbAUtDe/kRG9lMHFdEsNAAQAAkDkEnRwWKSdb09wuhzQmZMQLE6nuBI1+7vhW0K17D5reyXGnufU1AAAAMB6lazkuUk7mHldO5nY5tXFlfdQwEdkJirWX4lDyZWVmd4nWfm6OXr75EkIOAAAAMoodnTwQq5wsVllYqjtB8ZjdJVoy93jK1QAAAJBxBJ08Ea2cLJ7ITtD4OTrxyspCYUM79h5U6zsHJA0/b/HsGSoucqR8XggAAADIBIdhGGbOj1uqt7dXLpdLwWBQVVVVVi8nL4TChto6e9QV/Fg9/UOqPq5M7iqnFp42Xa+/e2jCzpDPH9AtT+7W4SNHx9xnWnmJ7vrS2Vpe5xmZzSNF3yWKVUoHAAAAmGU2GxB0MigSJsyUk2VzHYf6B3Xn02+O2cnxuJy68hyPtvwmEPX1+1/sjPuM+z4JMT5/YMIukYfmAwAAAEgTgo7F7PIHf7R1JGP8GZ5YPC6nXr75EhUXOWwT8AAAAJB/zGYDzuhkQKSEa3xA6AoOaE1ze9ZKuGKtIxlm3xsZMNowZ0bS54UAAACAdKO9dJqFwoaaWjqiBoTIa00tHQqFM7uRFm8dmWK2xTQAAACQaQSdNGvr7IlbJmbo090PK9eRCWZbTAMAAACZRulampnd1Rh9XSbOtGR7dyXZAaMAAABAJhF00szsrkbkukw1Lcj27kqyA0YBAACATKJ0Lc0igzNj/cnv0Ke7H5FmAeNLzCJNC3z+wMhrobCh1r0HtXnXfrXuPZjwjE+idZjl+OTfNy+u1bTykgnfn15eMtJaGgAAALAL2ktngJnBmcu8bl34w+dinqNxSHJ/0rJ5W0dXSrs+sdaRjNHPCYUN7dh7UK3vHJA03Flt8ewZ7OQAAAAga5ijY7FEJWmtew/qmgd3JLzPjUvn6Z5n90wIKqNDU6Kwk8ocnes/O0czKkpVfVyZ3FXMwgEAAIA9EHRsIF6Tgc279us7T+xKeI9pU0t0+OOjUb8X2fXZ/tef0+vvHorZzGD0OvZ82KcNz+9N+rlWDDsFAAAAxiPo2JzZHR0zqitK1dM/NPJ1vFCS6nPN7iABAAAAmWQ2G9CMwCKRZgHpMDrkSNGbGYx/brJFaNkcdgoAAABMFkHHIsVFDv3t5Wdm5N7xQklxkUPrGr2SlFLYycawUwAAAGCyCDoW+rBvMGP3joSS7/3LLr3y+wNjAs/yOo82rqyXe9yO0rSpE9tHR5PtYaQAAABAshgYaqF3e45k/BlP7fpAT+36QNPKS/SNP67VrJnlqql0apnXrWVe95hmCWHD0Fd/9p8J75ntYaQAAABAsgg6FjqtutzUdYtmTVfbvkOTetbhI0d197O/G/k6WsOCUNiQx+VUV3Ag6tydSJe3RbXVk1oLAAAAkGmUrlno2oZZSjSapsghfeuSeWl/drSGBfHO70S+XtfoZZ4OAAAAbI+gY6HSKUVadVFt3GtWXVSrP547M20d2iJiNSyIdX7H7XLSWhoAAAA5g9I1i926YngH5cGXOjW6QZpD0hcWuPU3y88a2WlZ3dye1meP7qLWMGfGyOvL6zwTzu+MH0IKAAAA2Bk7OjZw6wqv3rrzcn25/iSVlxZLGg4hLW906cIfPiefP6DldR7duDT9JWxS9C5qxUUONcyZoavOPUkNc2YQcgAAAJBTCDo28dxbH2pT+34dGQqNeX30WZq1l8xTdUVp2p9NFzUAAADkG4KODYTChppaOqJ2Oht9lkaSvnjuiWl7rkPD3dfoogYAAIB8Q9CxgbbOHgWCsYdwjj5Ls8zrTssz6aIGAACAfEbQsYFoZ2SieeX3B9TVO6DqihLT9/7iOR5999J5cleVjXmdLmoAAADIZ3RdswGzZ2Q2PP/7pO99cnW5vrvsdH3r0nl0UQMAAEDBIOjYwKLaanlcTnUFB6Ke05mc4TAT6aIGAAAAFAJK11IUChtq3XtQm3ftV+veg2OGbiYrMidH+vTsTLoQbgAAAFCI2NFJgc8fUFNLx5gGAh6XU+savSmfeVle59HGlfUT7jsZ08tLtHg2QQcAAACFx2EYRvqrpdKst7dXLpdLwWBQVVVVlq7F5w9oTXP7hBKzyE7MZA/4h8LGyFmaPR/2acPze1O+1300GwAAAECeMZsNKF1Lgtl5N5MtY2uYM0NXnXuSlsw93tR7xg8R9bichBwAAAAUNErXkpDMvJt0nI1ZVFutaeUlOnzkaNTvOzTcJnr7X39Or797iI5qAAAAwCcIOkkwO+/G7HWJbOvoihlypOFgta7Rq9IpRTQdAAAAAEahdC0JZufdmL0unkiZXDzTy0u0zOue9LMAAACAfEPQSUJk3k2sojCHhs/HLKqtnvSzEpXJSdKhI0fV1tkz6WcBAAAA+Yagk4R4824iX69r9KblfEy2y+QAAACAfELQSVJk3o3bNbY8ze1ymmotbXbQaDbL5AAAAIB8QzOCJERm3AweC+tHXz5HckgHPhqM2+ls9FycfQf69Xjbe+rqHRz5fqxBo5Eyua7gQNR21pGOa+kokwMAAADyDUHHJJ8/oKaWjjHnZiIhJVbHs2jvGa8rOKA1ze0TdoMiZXJrmtvlkMaEnXSXyQEAAAD5htI1E3z+gNY0t08ILJGQ4vMHTL9nvHiDRidbJgcAAAAUKnZ0Eoi0eY5WPmZoeHelqaVDy7zukd2VeO+JJt6g0eV1Hi3zukfK3xgICgAAACRG0EkgUZvnaCHFTGvoaGJ1UCsucjAQFAAAAEgCpWsJpNLmOdWWz3RQAwAAANKDHZ0EUmnznGxgoYMaAAAAkF7s6CQQafMc60SMQ8Pd10aHlETvGf9+iQ5qAAAAQDoRdBKItHmWNCG4xAop8d4zHh3UAAAAgPRzGIZhtjmYZXp7e+VyuRQMBlVVVWXJGuLN0YkVUmK9588vOFWzZpbTQQ0AAABIktlsQNBJQihsJN3mOZX3AAAAAIjObDagGUESUmnzTGtoAAAAIPs4owMAAAAg7xB0AAAAAOQdgg4AAACAvEPQAQAAAJB3CDoAAAAA8g5BBwAAAEDeIegAAAAAyDsEHQAAAAB5h6ADAAAAIO8QdAAAAADkHYIOAAAAgLxD0AEAAACQdwg6AAAAAPIOQQcAAABA3iHoAAAAAMg7BB0AAAAAeYegAwAAACDvEHQAAAAA5B2CDgAAAIC8Q9ABAAAAkHcIOgAAAADyDkEHAAAAQN4h6AAAAADIOwQdAAAAAHmHoAMAAAAg7xB0AAAAAOQdgg4AAACAvDPF6gWYYRiGJKm3t9filQAAAACwUiQTRDJCLDkRdPr6+iRJp5xyisUrAQAAAGAHfX19crlcMb/vMBJFIRsIh8P64IMPVFlZKYfDYfVyYurt7dUpp5yi999/X1VVVVYvB2nEZ5u/+GzzE59r/uKzzV98tvkr3Z+tYRjq6+vTiSeeqKKi2CdxcmJHp6ioSCeffLLVyzCtqqqKX9A8xWebv/hs8xOfa/7is81ffLb5K52fbbydnAiaEQAAAADIOwQdAAAAAHmHoJNGZWVlWrduncrKyqxeCtKMzzZ/8dnmJz7X/MVnm7/4bPOXVZ9tTjQjAAAAAIBksKMDAAAAIO8QdAAAAADkHYIOAAAAgLxD0AEAAACQdwg6AAAAAPIOQScJL774ohobG3XiiSfK4XDol7/8ZdzrX3jhBTkcjgn/3nrrrewsGKasX79eF1xwgSorK1VTU6MvfvGLevvttxO+b/v27Vq4cKGcTqdmz56t++67LwurRTJS+Wz5vbW/jRs3asGCBSMTthsaGvTrX/867nv4fc0NyX62/L7mpvXr18vhcOi73/1u3Ov4vc09Zj7bbP7eEnSS0N/fr3POOUcbNmxI6n1vv/22AoHAyL958+ZlaIVIxfbt23XDDTdox44d2rZtm44dO6bLLrtM/f39Md/T2dmpFStW6KKLLtLOnTt122236dvf/rY2bdqUxZUjkVQ+2wh+b+3r5JNP1l133aXXXntNr732mi655BJdddVV+u1vfxv1en5fc0eyn20Ev6+549VXX9UDDzygBQsWxL2O39vcY/azjcjK762BlEgynnrqqbjXPP/884Yk49ChQ1lZE9Kju7vbkGRs37495jV/8zd/Y5x55pljXvvmN79pLF68ONPLwySY+Wz5vc1N06dPN372s59F/R6/r7kt3mfL72tu6evrM+bNm2ds27bN+MxnPmN85zvfiXktv7e5JZnPNpu/t+zoZMF5550nj8ejSy+9VM8//7zVy0ECwWBQklRdXR3zmtbWVl122WVjXvv85z+v1157TUePHs3o+pA6M59tBL+3uSEUCumJJ55Qf3+/Ghoaol7D72tuMvPZRvD7mhtuuOEGXXHFFVq6dGnCa/m9zS3JfLYR2fi9nZKRu0KS5PF49MADD2jhwoUaHBzUo48+qksvvVQvvPCCLr74YquXhygMw9BNN92kCy+8UHV1dTGv6+rq0gknnDDmtRNOOEHHjh3TgQMH5PF4Mr1UJMnsZ8vvbW7YvXu3GhoaNDAwoOOOO05PPfWUvF5v1Gv5fc0tyXy2/L7mjieeeELt7e169dVXTV3P723uSPazzebvLUEng8444wydccYZI183NDTo/fff149+9CP+D7BNrV27Vm+88YZefvnlhNc6HI4xXxuGEfV12IPZz5bf29xwxhlnaNeuXTp8+LA2bdqkr33ta9q+fXvMP4j5fc0dyXy2/L7mhvfff1/f+c539Mwzz8jpdJp+H7+39pfKZ5vN31tK17Js8eLF2rNnj9XLQBTf+ta3tGXLFj3//PM6+eST417rdrvV1dU15rXu7m5NmTJFM2bMyOQykYJkPtto+L21n9LSUs2dO1fnn3++1q9fr3POOUc/+clPol7L72tuSeazjYbfV/t5/fXX1d3drYULF2rKlCmaMmWKtm/frp/+9KeaMmWKQqHQhPfwe5sbUvlso8nU7y07Olm2c+dOtlttxjAMfetb39JTTz2lF154QbW1tQnf09DQoJaWljGvPfPMMzr//PNVUlKSqaUiSal8ttHwe2t/hmFocHAw6vf4fc1t8T7baPh9tZ9LL71Uu3fvHvPaN77xDZ155pm6+eabVVxcPOE9/N7mhlQ+22gy9XtL0EnCRx99pN///vcjX3d2dmrXrl2qrq7WqaeeqltvvVX79+/XI488Ikm65557NGvWLM2fP19DQ0Nqbm7Wpk2baI1oMzfccIMee+wxbd68WZWVlSP/Bcnlcmnq1KmSNOGzXb16tTZs2KCbbrpJq1atUmtrqx566CE9/vjjlv0cmCiVz5bfW/u77bbbdPnll+uUU05RX1+fnnjiCb3wwgvy+XyS+H3NZcl+tvy+5obKysoJZyMrKio0Y8aMkdf5vc1NqXy22fy9Jegk4bXXXtPnPve5ka9vuukmSdLXvvY1/fznP1cgENB777038v2hoSF973vf0/79+zV16lTNnz9fTz/9tFasWJH1tSO2jRs3SpI++9nPjnn94Ycf1te//nVJmvDZ1tbWauvWrbrxxht177336sQTT9RPf/pTXX311dlaNkxI5bPl99b+PvzwQ1177bUKBAJyuVxasGCBfD6fli1bJonf11yW7GfL72v+4Pc2f1n5e+swIie7AAAAACBP0IwAAAAAQN4h6AAAAADIOwQdAAAAAHmHoAMAAAAg7xB0AAAAAOQdgg4AAACAvEPQAQAAAJB3CDoAAAAA8g5BBwAAAEDeIegAAAAAyDsEHQAAAAB55/8HCsn1lZgoMdQAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1000x1000 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model.eval()\n",
    "\n",
    "test_data = IR_data(X_test, Y_test)\n",
    "tdl = DataLoader(test_data,\n",
    "                 shuffle=False,\n",
    "                 batch_size=batch_size,\n",
    "                 drop_last=True)\n",
    "pred = torch.tensor([]).to(device)\n",
    "for x, y in tdl:\n",
    "    p = model(x.float().T.unsqueeze(-1).to(device))\n",
    "    pred = torch.cat([pred, p])\n",
    "\n",
    "result = pd.DataFrame({\n",
    "    'true_value': Y_test[:8000],\n",
    "    'Predicted': pred.squeeze(1).cpu().detach().numpy()\n",
    "})\n",
    "plt.rcParams['figure.figsize'] = (10.0, 10.0)\n",
    "plt.scatter(result['true_value'], result['Predicted'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "amended-market",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE = 0.03665\n"
     ]
    }
   ],
   "source": [
    "result['residual2'] = (result['true_value'] - result['Predicted'])**2\n",
    "print(\"RMSE = %1.5f\" % np.sqrt(result['residual2'].mean()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "selective-lemon",
   "metadata": {},
   "source": [
    "当然，同样，以上时间序列是一个比较简单的时间序列，也没有经过特别多的调参等优化，如果感兴趣可以尝试换其他的时间序列进行尝试和调优。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}