examples/demo-contextualsentence-multim.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "de081c32",
   "metadata": {},
   "source": [
    "Script to demo example usage of the Aspire Multi-Vector encoder which represents documents via contextual sentence embeddings and is trained to make multiple matches between documents using optimal transport, i.e the models:\n",
    "\n",
    "[`allenai/aspire-contextualsentence-multim-biomed`](https://huggingface.co/allenai/aspire-contextualsentence-multim-biomed)\n",
    "\n",
    "[`allenai/aspire-contextualsentence-multim-compsci`](https://huggingface.co/allenai/aspire-contextualsentence-multim-compsci)\n",
    "\n",
    "Requirements:\n",
    "- transformers version: 4.5.1\n",
    "- torch version: 1.8.1\n",
    "- geomloss version: 0.2.4"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07f5ede9",
   "metadata": {},
   "source": [
    "### Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "90b8a184",
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "99e62a39",
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import namedtuple\n",
    "import numpy as np\n",
    "import torch\n",
    "from torch import nn as nn\n",
    "from torch.autograd import Variable\n",
    "from transformers import AutoModel, AutoTokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "6a4bff9d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "plt.rcParams['figure.dpi'] = 150"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8af1a80e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import model, distance, and pre-proc code from examples script.\n",
    "from ex_aspire_consent_multimatch import AspireConSent, AllPairMaskedWasserstein\n",
    "from ex_aspire_consent_multimatch import prepare_abstracts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "2558da05",
   "metadata": {},
   "outputs": [],
   "source": [
    "rep_len_tup = namedtuple('RepLen', ['embed', 'abs_lens'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb506099",
   "metadata": {},
   "source": [
    "### Example documents and alignments"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d93f2291",
   "metadata": {},
   "source": [
    "For a handful examples, prepare the data, embed it, and visualize the optimal transport plans/alignments for the computed sentence vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b8520866",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize the tokenizer, model, and distance function.\n",
    "hf_model_name = 'allenai/aspire-contextualsentence-multim-compsci'\n",
    "aspire_tok = AutoTokenizer.from_pretrained(hf_model_name)\n",
    "aspire_mv_model = AspireConSent(hf_model_name)\n",
    "# Empty dict of hyper params will force class to use defaults.\n",
    "ot_distance = AllPairMaskedWasserstein({})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b9495eec",
   "metadata": {},
   "outputs": [],
   "source": [
    "ex_abstracts = [\n",
    "    {'TITLE': \"Team Self-Managing Behaviors and Team Effectiveness: The Moderating Effect of Task Routineness\",\n",
    "     'ABSTRACT': [\"This study investigates the role of team members’ self-managing behaviors in regard to three dimensions of team effectiveness.\",\n",
    "                  \"Furthermore, this study examines the moderating effect of task routineness on these relationships.\",\n",
    "                  \"The sample consists of 97 work teams (341 members and 97 immediate supervisors) drawn from a public safety organization.\",\n",
    "                  \"Results show that team self-managing behaviors are positively related to team performance, team viability, and team process improvement.\",\n",
    "                  \"Results also indicate that task routineness moderates the relationships that team self-managing behaviors have with team performance and team viability such that these relationships are stronger when the level of task routineness is low.\",\n",
    "                  \"However, this moderating effect is not significant in regard to the relationship between team self-managing behaviors and team process improvement.\",\n",
    "                  \"Taken together, these findings suggest that emphasis on team self-managing behaviors may enhance team effectiveness\"]},\n",
    "    {'TITLE': \"Overcoming Barriers to Self-Management in Software Teams\",\n",
    "     'ABSTRACT': [\"The basic work unit in innovative software organizations is the team rather than the individual.\",\n",
    "                  \"Such teams consist of a small number of people with complementary skills who are committed to a common purpose, set of performance goals, and approach for which they hold themselves mutually accountable.\",\n",
    "                  \"Work teams have many advantages, such as increased productivity, innovation, and employee satisfaction.\",\n",
    "                  \"However, their implementation doesn't always result in organizational success.\",\n",
    "                  \"It isn't enough to put individuals together and expect that they'll automatically know how to work effectively as a team.\",\n",
    "                  \"Lack of redundancy and conflict between team and individual autonomy are key issues when transforming from traditional command-and-control management to collaborative self-managing teams.\"]}\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "6812bc0a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pre-process the data.\n",
    "bert_batch, abs_lens, sent_token_idxs = prepare_abstracts(batch_abs=ex_abstracts,\n",
    "                                                          pt_lm_tokenizer=aspire_tok)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "2701c8d1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get sentence embeddings for the papers.\n",
    "clsreps, contextual_sent_reps = aspire_mv_model.forward(bert_batch=bert_batch,\n",
    "                                                        abs_lens=abs_lens,\n",
    "                                                        sent_tok_idxs=sent_token_idxs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "5d4470c8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([2, 7, 768])\n",
      "Number of sentences: 7, 6\n"
     ]
    }
   ],
   "source": [
    "# Check shapes\n",
    "print(contextual_sent_reps.shape)\n",
    "print(f'Number of sentences: {len(ex_abstracts[0][\"ABSTRACT\"])}, {len(ex_abstracts[1][\"ABSTRACT\"])}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "014b276e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Package the sentence embeds for use with the distance function\n",
    "query_embeds = contextual_sent_reps[0,:,:].unsqueeze(0)\n",
    "cand_embeds = contextual_sent_reps[1,:,:].unsqueeze(0)\n",
    "qt = rep_len_tup(embed=query_embeds.permute(0, 2, 1), abs_lens=[abs_lens[0]])\n",
    "ct = rep_len_tup(embed=cand_embeds.permute(0, 2, 1), abs_lens=[abs_lens[1]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "499d8f63",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compute the wasserstein distance and alignment between abstracts\n",
    "wd, intermediate_items = ot_distance.compute_distance(query=qt, cand=ct, return_pair_sims=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "d5836fb3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(7, 6)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get the transport plan\n",
    "transport_plan = intermediate_items[3].data.numpy()[0,:abs_lens[0],:abs_lens[1]]\n",
    "transport_plan.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "04b23696",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0: This study investigates the role of team members’ self-managing behaviors in regard to three dimensions of team effectiveness.\n",
      "1: Furthermore, this study examines the moderating effect of task routineness on these relationships.\n",
      "2: The sample consists of 97 work teams (341 members and 97 immediate supervisors) drawn from a public safety organization.\n",
      "3: Results show that team self-managing behaviors are positively related to team performance, team viability, and team process improvement.\n",
      "4: Results also indicate that task routineness moderates the relationships that team self-managing behaviors have with team performance and team viability such that these relationships are stronger when the level of task routineness is low.\n",
      "5: However, this moderating effect is not significant in regard to the relationship between team self-managing behaviors and team process improvement.\n",
      "6: Taken together, these findings suggest that emphasis on team self-managing behaviors may enhance team effectiveness\n",
      "\n",
      "0: The basic work unit in innovative software organizations is the team rather than the individual.\n",
      "1: Such teams consist of a small number of people with complementary skills who are committed to a common purpose, set of performance goals, and approach for which they hold themselves mutually accountable.\n",
      "2: Work teams have many advantages, such as increased productivity, innovation, and employee satisfaction.\n",
      "3: However, their implementation doesn't always result in organizational success.\n",
      "4: It isn't enough to put individuals together and expect that they'll automatically know how to work effectively as a team.\n",
      "5: Lack of redundancy and conflict between team and individual autonomy are key issues when transforming from traditional command-and-control management to collaborative self-managing teams.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwQAAAIXCAYAAAAizpljAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAABcSAAAXEgFnn9JSAAA+fElEQVR4nO3de5wtVXnn/8/3IHC4KOCFiOEWGcWIQQGNQFRCxsFE8a4xiSaKMb8Y5RJxNKMZFRMTTYwohDExiQSTiZrIGBEZr2gQFR1uAmoOIKJwFBEQkftFnt8fVa1t07t796ndvbu6Pm9f+1W9q1atevbuY7Of/ay1KlWFJEmSpGFaN+0AJEmSJE2PCYEkSZI0YCYEkiRJ0oCZEEiSJEkDZkIgSZIkDZgJgSRJkjRgJgSSJEnSgJkQSJIkSQNmQiBJkiQNmAmBJEmSNGAmBJIkSdKAmRBIkiRJA2ZCIEmSJA3YvaYdwCpU0w5AkiRpTJl2ADO22ufwZfkMdev5J6ya17hWmRDM47xv/XDaIQzGvrvdB4D3nHPllCMZjhc+ehcAbrnT3HelbL1589+y2+6aciADsr79r5vv+crxPV956/0Upwnxn5IkSZK6iyPR+8rfnCRJkjRgVggkSZLUXRzq31dWCCRJkqQBs0IgSZKk7pxD0Fv+5iRJkqQBs0IgSZKk7pxD0FsmBJIkSerOIUO95W9OkiRJGjArBJIkSerOIUO9ZYVAkiRJGjArBJIkSerOOQS95W9OkiRJGjArBJIkSerOOQS9ZYVAkiRJGjArBJIkSerOOQS9ZUIgSZKk7hwy1FumcpIkSdKAWSGQJElSdw4Z6i1/c5IkSdKAWSGQJElSd84h6C0rBJIkSdKAWSGQJElSd84h6C0TAkmSJHVnQtBb/uYkSZKkAbNCIEmSpO7WOam4r6wQSJIkSQNmhUCSJEndOYegt/zNSZIkSQPWq4QgybokL0xySpJz28cp7b6xX0uSr456LGf8kiRJa1ayPI+JhZf1Sd6Y5JIktyX5TpITk+y8xH4OSvKGJKcluSZJJdmwQPvd2zaLPU6cc96LFmn//k19L+bq25ChvwPuDfwv4AogwM7Ai4HHAb83vdAkSZIGbBUPGUqyHjgdOBC4CjgF2B04DDg0yQFVddmY3R0HPHIJl78JeM8Cx58HrAfOHHH8AuDL8+z/0hJiWFDfEoInVNVD5+z7T+CTSS4Zt5Oq2muhw5sUmSRJklar19IkA2cBh1TVTQBJjgbeBpwIHDRmX58A/g04G7gWOG+hxlV1LfCi+Y4leRjwQuBW4P+M6OJDVXXMmLFtkr4lBHcmeVhV/VRZpn0z75xSTJIkSZrg8J5JSrI5cET79OUzyQBAVR2b5IXAE5LsV1XnLtZfVb16Vt+7dwzvt9vtKVX1w459bbK+JQSHA6cluRK4kubb/N2AB+FwIUmSJN3T44Dtgcuq6vx5jp8M7A08FVg0IZiUJAF+q336zyt13fn0KiGoqs8keQjwGJq5A9AkBudU1d3Ti0ySJGngVu8cgpnx/qOG9pw3p91KeRzNPIZraIYhjbJfkrcC9wG+C3y6qs6YZCC9SggA2g/+X2KCEykkSZK0Oi20CuQi80Jn7NpuN444vnFOu5Xygnb7vqq6a4F2h7aPGa9PcgbwvKq6ehKBrNpUTpIkST2yepcd3bbd3jLi+M1z2i27JFsAz22fjhoudBVwDLAPsB3wQOBpwAaaCdCnJdlsEvH0rkIgSZKk4RizCrCQmaxi1EqS05gNfSiwA7Chqs6Zr0FVfRz4+KxdPwROTfIZmrkO+9EsWfrersFYIZAkSVJ3Wbc8j+5ubLfbjDi+dbu9acTx5TAzXGjJk4nbVZKOb58+aRLBWCGQJElSd6t02VGam9nCTxakmWvnOe2WVZLtgSfTVCz+ZRO7ubTd7jSJmKwQSJIkaS27oN3uO+L4zP4LVyAWgF8HtgTOrKpvbWIfO7TbiVQ1rBBIkiSpu9W77OjngRuAPZLsM8+9CJ7Tbj+yQvFs8nChWZ7dbidy34RV+5uTJEmSuqqqO4AT2qcnJPnxXIIkR9PclOxzVXX2rP2HJ9mQ5M2TjCXJbjT3H7gd+MAibY9Msu2cfZsneQPNCkW3AidNIi4rBJIkSepu9c4hAHgT8ETgQODSJGcCuwGPBa4DDpvT/v7AnswzRj/JS4CXtE+3bLe7JfnirGYvq6r5boT2fJpVjT5cVTcsEvNxwFuSfA34FrAeeBTwIOA24AVV9e1F+hiLCYEkSZLWtKq6LcnBwGuA3wKeAVwPvAd4XVVduYTudqZJJGZbP2fffUac+/x2+7/HuM6fAAcADwMeTpNIbATeBby9qi4eN+DFmBBIkiSpu9U7hwCAqroVeH37WKztMTQ3BVvSsTH6HfueClX1hk25xqYwIZAkSVJ3qzwh0Gj+5iRJkqQBs0IgSZKk7lb3pGItwAqBJEmSNGBWCCRJktSdcwh6y9+cJEmSNGBWCCRJktSdcwh6y4RAkiRJ3TlkqLf8zUmSJEkDZoVAkiRJ3TlkqLesEEiSJEkDZoVAkiRJncUKQW9ZIZAkSZIGzAqBJEmSOrNC0F9WCCRJkqQBs0IgSZKk7iwQ9JYJwTz23e0+0w5hcF746F2mHcLgbL25f7lX2nr/4q443/OV53s+XA4Z6i+HDEmSJEkDZh4/j+/deOe0QxiMHe+9OQBnf+OGKUcyHI958HYA3HJHTTmS4dh6i+Zbs1v907Jitmr+tHDbXdONY0hmKgP+bVk5M39bVgsrBP1lhUCSJEkaMCsEkiRJ6swKQX9ZIZAkSZIGzAqBJEmSOrNC0F8mBJIkSerOfKC3HDIkSZIkDZgVAkmSJHXmkKH+skIgSZIkDZgVAkmSJHVmhaC/rBBIkiRJA2aFQJIkSZ1ZIegvEwJJkiR1ZkLQXw4ZkiRJkgbMCoEkSZK6s0DQW1YIJEmSpAGzQiBJkqTOnEPQX1YIJEmSpAGzQiBJkqTOrBD0lxUCSZIkacCsEEiSJKkzKwT9ZUIgSZKk7swHesshQ5IkSdKAWSGQJElSZw4Z6i8rBJIkSVrzkqxP8sYklyS5Lcl3kpyYZOcl9nNQkjckOS3JNUkqyYZFzjmpbTfq8dIFzj00yRlJbkjyw/bnQ5cS82KsEEiSJKmz1VwhSLIeOB04ELgKOAXYHTgMODTJAVV12ZjdHQc8chND+Tjw3Xn2Xzxf4yRHtte7C/gUcDtwCHBqkqOq6vhNjOOnmBBIkiRprXstTTJwFnBIVd0EkORo4G3AicBBY/b1CeDfgLOBa4HzlhDHW6rqP8ZpmOShbWy3AwdX1Vmz9n8BeFuSj1bVpUu4/rwcMiRJkqTOkizLYwJxbQ4c0T59+UwyAFBVxwIXAk9Ist84/VXVq6vqz6vqk8D1nQMc7SiaL+//diYZaK9/CfBn7bEjJ3EhEwJJkiR1tloTAuBxwPbAZVV1/jzHT263T53ExSZoZp7AyfMc+0C7nUjMDhmSJEnSWjYz3n/U0J7z5rRbTs9K8mxgM+By4NSquseE5CTbA7u2T++RxFTVxiTXArsl2a6qbugS1CATgiRfHXWsqlYyFEmSpLVhmeYUL/K5ba8xupj5YL1xxPGNc9otpyPmPP+LJH8DHFVVd83aPxPL9VV184i+NgL3b9te1CWoNTVkqH1DJUmSpBnbtttbRhy/eU675XA+8FLgocDWwIOBlwM/AF4GvHVO+8VihgnGvdYqBE8Zp9Ei2aQlAkmSpCVarmVHx6wCLGQmsFGf8ZZ9vdSqOm7OrsuBdyb5LHAucESSY6vqyjkxLfS5dGJx965CkOSOEY87gZ+ddnySJElaVW5st9uMOL51u71pxPFlU1VfAT5MM6fgibMOLRYzTDDuPlYIrgb2rapr5h5IcuU87SVJkrTMVvGNya5ot6PuSLzznHYrbeY+AjvN2jcTyw5Jthkxj2BicfeuQgD8O/CQEcdOW8lAJEmS1FjFy45e0G73HXF8Zv+Fk7jYJtih3c6+P8IP+MkH/X3mnpBkZ5oJxVd0XWEIepgQVNWRVfWFEcdeutLxSJIkaVX7PHADsEeSe3y4Bp7Tbj+yciE1kmzJT+bAnjvn8MwX3c/hnp7bbicSc+8SAkmSJK1CWaZHR1V1B3BC+/SEJD8el5/kaGBv4HNVdfas/Ycn2ZDkzV2vn2TPJE9Pstmc/Q8A3g/sQlPFmPuF93HAj4CXJtl/1nkPAf64PXZ81/ign3MIJEmSpKV4E82k3QOBS5OcCewGPBa4DjhsTvv7A3vy0+P6AUjyEuAl7dMt2+1uSb44q9nLqmrmhmc7AR8CrkuyAfg2sCOwH3BvmvsJ/HrNuRlWVV2c5FXAscCZST4J3AEcAmwFHF1VFy/lTRjFhECSJEmdreJJxVTVbUkOBl4D/BbwDOB64D3A62Yt9zmOnWkSidnWz9l3n1k/XwK8A9gf2AP4ReD2dv+pwHFVdf2IuN+e5OvAq4DHt7vPBd5aVR9eQswLinfmvYf63o13TjuGwdjx3psDcPY3Os+H0Zge8+DtALjlDv+/v1K23qL5j+St/mlZMVs1f1q47a6F22ly1rdfMfq3ZeW0f1tWzafwXY/48LL88q/466etmte4VlkhkCRJUmeruUKghTmpWJIkSRowKwSSJEnqzApBf5kQSJIkqTMTgv5yyJAkSZI0YFYIJEmS1J0Fgt6yQiBJkiQNmBUCSZIkdeYcgv6yQiBJkiQNmBUCSZIkdWaFoL9MCCRJktSZ+UB/OWRIkiRJGjArBJIkSerMIUP9ZYVAkiRJGjArBJIkSerMAkF/WSGQJEmSBswKgSRJkjpzDkF/mRBIkiSpM/OB/nLIkCRJkjRgVggkSZLU2bp1lgj6ygqBJEmSNGBWCOax4703n3YIg/OYB2837RAGZ+st/CZnpW3ln5YVt97/yq04/7YMl3MI+ssKgSRJkjRgfnciSZKkzlx2tL9MCObxvvO/Pe0QBuM39/lZAP7pnCunHMlw/M6jdwHgtrumHMiAzAxb8T1fOTPv+c131HQDGZBt2qFC/jtfOQ6J06T4T0mSJEmdWSDoLxMCSZIkdeaQof5yUrEkSZI0YFYIJEmS1JkVgv6yQiBJkiQNmBUCSZIkdWaBoL+sEEiSJEkDZoVAkiRJnTmHoL9MCCRJktSZ+UB/OWRIkiRJGjArBJIkSerMIUP9ZYVAkiRJGjArBJIkSerMAkF/WSGQJEmSBswKgSRJkjpzDkF/mRBIkiSpM/OB/nLIkCRJkjRgVggkSZLUmUOG+ssKgSRJkta8JOuTvDHJJUluS/KdJCcm2XmJ/RyU5A1JTktyTZJKsmGB9psnOSTJCUnOTfL9JLcm+c8kf5XkASPOe1Hb96jH+5f6HoxihUCSJEmdreYCQZL1wOnAgcBVwCnA7sBhwKFJDqiqy8bs7jjgkUu4/EHAx9ufLwM+A2wOHAC8Enh+kl+uqotHnH8B8OV59n9pCTEsyIRAkiRJa91raZKBs4BDquomgCRHA28DTqT54D6OTwD/BpwNXAuct0j7u4H3AW+tqvNndibZDvhX4EnAP7bxzedDVXXMmLFtEhMCSZIkdbZa5xAk2Rw4on368plkAKCqjk3yQuAJSfarqnMX66+qXj2r793HaP9p4NPz7L8hyWHAd4ADkuxWVd9a9AUtA+cQSJIkaS17HLA9cNnsb+hnObndPnXFImpV1VXANe3TB6309WdYIZAkSVJnq7RAAD8Z7z9qaM95c9qtmCTbAzu0T787otl+Sd4K3Kdt8+mqOmOScZgQSJIkqbPlGjKU5KujjlXVXmN0sWu73Tji+MY57VbSy2k+j19UVZePaHNo+5jx+iRnAM+rqqsnEYRDhiRJkrSWbdtubxlx/OY57VZEkn2A/9k+/aN5mlwFHAPsA2wHPBB4GrCBZgL0aUk2m0QsVggkSZLU2XINGRqzCrCQmchqkeMrJskDgQ8C64F3VNVH57apqo/zk+VKAX4InJrkM8C5wH7A84D3do3HCoEkSZLWshvb7TYjjm/dbm8acXyi2uVGP0pzH4QP0NyLYGztKknHt0+fNImYepcQJDk0ycuT7Dln/+HTikmSJGnokizLYwKuaLej7ki885x2yybJVsCpwKNo7mfwgqq6exO6urTd7jSJuHo1ZCjJnwBPAC4E/ijJX1XVTIb0YuCEMftZaHJK5zglSZK0alzQbvcdcXxm/4XLGUSSe9FUBB4PfAF4VlXdsYndzaxMNJGqRq8SAuAZwL5VdVeSNwIfSPKgqvofTGH8lyRJkhqr9cZkwOeBG4A9kuwzz70IntNuP7JcAaR5c04CngJ8GXhKVd280DmLeHa7XfRGauPo25ChzarqLoCqug74VZpf7j+whNdSVXuNeixT3JIkSWtasjyPrtpv4WdGkZyQ5MdzCZIcDewNfK6qzp61//AkG5K8uXsEABwHPJ9mhaBDquoHi52Q5Mgk287Zt3mSNwDPBW6lSTI661uF4Kok+1bVedD8gpM8D/h74BHTDU2SJEmr1JuAJwIHApcmORPYDXgscB1w2Jz29wf2ZJ4x+kleArykfbplu90tyRdnNXvZzOfVJE8Hjmj3Xwm8dUQ15S1VtWHW8+OAtyT5GvAtmhWJHkVzR+PbaOYffHvhlz2eviUELwLumr2jnYjxu0nePZWIJEmStJqHDFFVtyU5GHgN8Fs0w9CvB94DvK6qrlxCdzvTJBKzrZ+z7z6zft5h1s//bYF+T6KpIMz4E+AA4GHAw2mGx28E3gW8vaouXkLMC+pVQlBVo+4wR1V9YSVjkSRJUn9U1a3A69vHYm2Pobkp2JKOjWh/EpswtKeq3rDUczZVrxICSZIkrU6ruECgRfRtUrEkSZKkCbJCIEmSpM5W8xwCLcyEQJIkSZ2ZD/SXQ4YkSZKkAbNCIEmSpM7WWSLoLSsEkiRJ0oBZIZAkSVJnFgj6ywqBJEmSNGBWCCRJktSZy472lxUCSZIkacCsEEiSJKmzdRYIesuEQJIkSZ05ZKi/HDIkSZIkDZgVAkmSJHVmgaC/rBBIkiRJA2aFQJIkSZ0FSwR9ZYVAkiRJGjArBJIkSerMZUf7y4RAkiRJnbnsaH85ZEiSJEkaMCsEkiRJ6swCQX9ZIZAkSZIGzAqBJEmSOltniaC3rBBIkiRJA2aFQJIkSZ1ZIOgvE4J5/OY+PzvtEAbndx69y7RDGJz1/r9/xfmer7xttvATykrz3/lwuexofzlkSJIkSRow8/h53HbXtCMYjplvknzPV87Me77VPodPN5ABufX8EwC45c6aciTDsfXmzTeV/m1ZOf49X3mrrRpjgaC/rBBIkiRJA7bKcktJkiT1kcuO9pcVAkmSJGnAJpoQJDlwkv1JkiSpH7JMDy2/SVcIPpfkK0mOSnLfCfctSZIkacImnRCcAfw8cCzw7ST/kuSXJ3wNSZIkrTJJluWh5TfRhKCqDgYeCrwVuB74TeD0JJckeXWSn5nk9SRJkrQ6rMvyPLT8Jj6puKouq6r/AewCPBv4OPBg4C3AFUlOTvKrMeWTJEmSpm7ZVhmqqh9V1b9X1ZOB3YE3AlcBzwROA76Z5HVJHrhcMUiSJGllOGSov1Zk2dGq2gh8HjiHn0wa34UmSbg8yVuTbL4SsUiSJEn6iWW9MVmSnYAXt4/daRKBC4B3AacDzwUOB45uY3nFcsYjSZKk5eGX+f018QpBknVJnprkFOBbwJ8CDwTeAxxQVftU1d9W1aVV9efAw4Bv0kxAliRJkrSCJlohSPKnwGHATjTVgK8Bfwe8p6pumO+cqvphkjOB355kLJIkSVo5jvfvr0lXCP4YuB/wPuCgqnpEVR0/KhmY5T+Af5pwLJIkSVohq33Z0STrk7yxXQ7/tiTfSXJikp2X2M9BSd6Q5LQk1ySpJBvGOG9dkj9MclGSW9tzP5Dk4Yucd2iSM5LckOSH7c+HLiXmxUx6DsGrgROr6vtLOamqTgJOmnAskiRJEknW08xfPZBm1ctTaOa3HgYcmuSAqrpszO6OAx65xOsH+FfgOcAPaFbcvD/NEv1PSXJwVX1pnvOObK93F/Ap4HbgEODUJEdV1fFLiWOUSScEv0Tz5h4+4X4lSZK0iq3yIUOvpUkGzgIOqaqbAJIcDbwNOBE4aMy+PgH8G3A2cC1w3hjnHEaTDFwKPL6qrm6v/2zgZOBfkjysqu6aOSHJQ9vYbgcOrqqzZu3/AvC2JB+tqkvHjHukSQ8Z+jWabEeSJEmaunZp+yPapy+fSQYAqupY4ELgCUn2G6e/qnp1Vf15VX0SuH7MMF7Zbl89kwy0ff0f4MPAHsDT55xzFM2X9387kwy051wC/Fl77Mgxr7+gSScElwPbTLhPSZIkrXJZpscEPA7YHrisqs6f5/jJ7fapk7ncT0vyc8DDgVtphgqNe/1D5xyf7QMjztkkk04I3gcc5N2HJUmStErMjPcfNbTnvDntluv6X6mqO8e5fpLtgV3bp/dIYtqb/l4L7JZku64BTnoOwZuBxwJnJPkfwEdGvHBJkiStIeuWaQ5Bkq+OOlZVe43RxcwH640jjm+c027SNuX6Mz9fX1U3L3De/du2F3UJcNIJwcU0VYddaMobleR7wG3ztK2q2mPC15ckSdIUrOI5xdu221tGHL95TrvVcP3Fzhl13iaZdEKw+5znoblLsSRJkrRkY1YBFjKTqtQix5fLYtff1HMmFvdEE4KqmvScBEmSJPXAKl529MZ2O2rhm63b7U0jji/39Wf2z77+YufABOP2A7wkSZLWsiva7ag7Eu88p91quP7MzzskGZUUTCxuEwJJkiR1lizPYwIuaLf7jjg+s//CiVxt9PUf0d4TYdHrV9UP+MkH/X3mnpBkZ5oJxVdU1Q1dA1yWhCDJryX5UJJvJ7k9ybvnHDs2yYOW49qSJEnSLJ8HbgD2SHKPD9c0dxAG+MhyXLyqLgf+E9gKeMoSrn/anOOzPXfEOZtk4glBknfSBPc0mlnPm/PTkx5+APwh8BuTvrYkSZKmY12yLI+uquoO4IT26Qmzh+AkORrYG/hcVZ09a//hSTYkeXPnABrHttu/TLLjrOs8i+Yz8+XAh+accxzwI+ClSfafdc5DgD9ujx0/ieAmOqk4yYuBlwJfAn6/qi5McvfsNlV1VpJv09xZ7dh5upEkSZIm6U3AE4EDgUuTnAnsRnP/rOuAw+a0vz+wJ7DT3I6SvAR4Sft0y3a7W5Ivzmr2sqqafSO0E4EnA88ENiQ5vb3GQTTL879g7r27quriJK+i+bx8ZpJPAncAh9BUG46uqovHfwtGm/Syo78PfB84tKquW6Dd14EHT/jakiRJmpLVu8gQVNVtSQ4GXgP8FvAM4HrgPcDrqurKJXS3M00iMdv6OfvuM+f6dyd5LnAU8GLgUJr7CPw78Pqqmvfma1X19iRfB14FPL7dfS7w1qr68BJiXtCkE4K9gM8skgwAfBfYf5E295BkW+AV7dO3AS8Hfofm7mxHVdU1S+1TkiRJ3a3iZUcBqKpbgde3j8XaHgMcs9Rji/T5I5pv+5c0QqaqTgVOXer1lmLSCcHdjDcv4UH85O5qS3EicCXNuqunAV8Ent0+/pox5yUscgvsTQhLkiRJ6qdJJwQbgEcn2bqq5r3VcpL7AY+iKXcs1Z5V9etJ1tFUGX6lmk/wb07y5U2MWZIkSR25ln1/TToh+Bfg7cD/SvJ7VXXX7INpaknH06w+9M+b0P/d8ONxWGfVT3+dP3adapFbYFsikCRJ0mBMOiF4J83wnRcCj0vy8Xb/3kn+imYCxUOBT9NM4liqa5Lcu6purKqnz+xM8kDg9m6hS5IkaVOt9jkEGm2i1Z12uaRfBf4W2BV4WXtoX+BoYA/g3cBTq+rueTtZuP9DqurGeQ7dDDxrk4KWJEmSBmzSFQLauQMvS/IGmrVVdwc2AzbSrED0nWW45o3AfImCJEmSVsA6CwS9NfGEYEa7BOjJy9W/JEmSVg8Tgv5yQrgkSZI0YBOtECRZ9EYPs1RV/ekkry9JkqTpcFJxf016yNAxNMt2jvoXMbOkZ9qfTQgkSZKkKZp0QnDYiP3rgF2AJwEHAP8LOGfC15YkSdKUOIegvyaaEFTVYvcW+JMkrwH+GPi7SV5bkiRJ0tKt+KTiqnozzRKkf77S15YkSdLySJbnoeW3bMuOLuIi4IlTurYkSZImbJ2f3ntrWsuO7sH0khFJkiRJrRX9UJ5ke+B1wKOAz6zktSVJkrR8vLlVf036PgTfWODwtsD9aJYcvRV4zSSvLUmSJGnpJl0h2H2BY3cCVwJnAH9RVV+b8LUlSZI0JU4h6K9JLztqtUiSJEnqkWWZQ5BkHXBfYDPg+qq6YzmuI0mSpNXBVYb6a2Lf6CfZOcmfJ/kycDtwNfAd4OYkG5K8Lckek7qeJEmSpO4mkhAkORy4GPgjYG+aykDax2bAQ4FXAF9J8vtzzl2f5PGTiEOSJEnT4Y3J+qtzQpDkj4DjgPXAB4BnALu0z7cCdgWeCZwMbAG8M8lr23PvC5wOHNw1DkmSJE3PuizPQ8uv0xyCJHsBbwKuB55ZVWfO02xj+zglyROADwGvT3IR8BfAnu0+SZIkSSus66TiI2mqDC8YkQz8lKr6bJLnA6fRJAEB3gsc2zEOSZIkTZGTivur65ChXwE2VNXHxj2hqj4K/Gf79G1V9YKq+lHHOCRJkiRtgq4JwYOAizbhvIsAqupVHa8vSZKkVcBJxf3VNSG4A9hyE87bErix47UlSZIkddR1DsHlwAFJ1lXV3eOc0N607ADgGx2vLUmSpFXCFYH6q2uF4DTgAcArl3DOf2/P+UjHa0uSJGmVyDL9T8uva0LwDuCHwJuTvCbJZqMaJtmsvf/Am4EbaO5dIEmSJGmKOg0ZqqrrkjyX5tv+NwF/kOQDwLnA92iWFd0R2A94DvCzwF3Ab1TVdV2uLUmSpNXDIUP91XUOAVX1qSSPB94DPAz4w3mazfwTuRh4UVV9qet1l9P6zu+Klsr3fOXdev4J0w5hcLbe3P9arjT/tqw833Opfybyf9uqOht4eJJfA54MPAq4H00icC1wAfBR4P9WVU3impIkSVo9rBD010Tz+PamYx+dZJ/TcNtd045gOGa+SfI9Xzkz7/ktd5qbr5SZysCtd045kAHZavNm++8Xfne6gQzIM/d+IAC7H+WaISvlm8cdOu0QtEZY2JMkSVJn8S5ivWVCIEmSpM4cMtRfXZcdlSRJktRjVggkSZLUmSOG+ssKgSRJkjRgVggkSZLU2TpLBL1lhUCSJEkaMBMCSZIkdbYuy/OYlCTrk7wxySVJbkvynSQnJtl5E/raPsk7knwrye3t9rgk28/TdvckNcbjxDnnvWiR9u/v8Hb8FIcMSZIkaU1Lsh44HTgQuAo4BdgdOAw4NMkBVXXZmH3dDzgLeAjwDeBDwF7AkcCTk+xfVdfNOuUm4D0LdPk8YD1w5ojjFwBfnmf/l8aJdxwmBJIkSepslU8heC1NMnAWcEhV3QSQ5GjgbcCJwEFj9vV2mmTgg8Dzququtq/jgSOAY4EXzjSuqmuBF83XUZKHtW1vBf7PiOt9qKqOGTO2TeKQIUmSJHW2jizLo6skm9N8UAd4+UwyAFBVxwIXAk9Ist8YfT0QeD5wJ/CymWSg9SrgGuD5SX5mzPB+u92eUlU/HPOciTMhkCRJ0lr2OGB74LKqOn+e4ye326eO0dev0Xx+/mxVXT37QFXdDpwKbNa2W1CSAL/VPv3nMa69bBwyJEmSpM5W8ZChR7bb80YcP29Ou659vXjMvh5HM4/hGuATC7TbL8lbgfsA3wU+XVVnjNH/2EwIJEmStGol+eqoY1W11xhd7NpuN444vnFOu5Xq6wXt9n1zhh7NdWj7mPH6JGfQzF+4esQ5S+KQIUmSJHW2ipcd3bbd3jLi+M1z2i17X0m2AJ7bPh01XOgq4BhgH2A74IHA04ANNBOgT0uy2RgxL8oKgSRJklatMasAC5lJK2qR4yvZ16HADsCGqjpnvgZV9XHg47N2/RA4NclngHOB/WiWLH3vmNccyQqBJEmSOluXLMtjAm5st9uMOL51u71pxPHl6GtmuNCSJxO3qyQd3z590lLPn48VAkmSJHW2iicVX9FuR92ReOc57Za1r/Zuxk+mqTL8yxjXnM+l7XanTTz/p1ghkCRJ0lp2Qbvdd8Txmf0XrlBfvw5sCZxZVd8a45rz2aHdjlPVWJQVAkmSJHU2oeE9y+HzwA3AHkn2medeBM9ptx8Zo6+PAXcDj0+yY1V9b+ZAki1p7mVwN/DRBfrY5OFCszy73Z7boY8fs0IgSZKkNauq7gBOaJ+ekOTH4/+THA3sDXyuqs6etf/wJBuSvHlOX1cB7wO2AN6ZZPaX638JPAB4b1V9d75YkuxGc/+B24EPLBR3kiOTbDtn3+ZJ3kCzQtGtwEkL9TEuKwSSJEnqbPUWCAB4E/BE4EDg0iRnArsBjwWuAw6b0/7+wJ7MP0b/D4H9ab6l35DkHGAv4BHAZcArFojj+TQrEX24qm5YJObjgLck+RrwLWA98CjgQcBtwAuq6tuL9DEWKwSSJEla06rqNuBg4E9p7iHwDJq7BL8H2Keqvr6Evq4FHgP8NU2l4Jk09wk4AfjF9vgoz2+3/3uMS/0J8DlgR+DXgF9pY38X8Kiq+uC4MS/GCoEkSZI6W+3fMlfVrcDr28dibY+huSnYqOPXA0e2j6XEMPY9FarqDUvpuwsTAkmSJHWWVT5mSKOt9mROkiRJ0jKyQiBJkqTOrA/0lxUCSZIkacB6XyFIst0YyzZJkiRpGa3iG5NpEb1PCIDzgQcv5YQkXx11rKo6ByRJkiT1Ra8SgiR3AsVPhqkVcK+Z/VW1xdSCkyRJGjDrA/3Vq4QA+Htga+CVVXUdQJLLq+rnltLJImvAWiKQJEnSYPRqUnFVvQz4B+BDSV44s3uKIUmSJAlIlueh5derhACgqj4H/FfgIUk+SXPLaEmSJE1RkmV5aPn1bcgQAFV1B/A/k/wC8EvTjkeSJEnqq14mBDOq6iLgomnHIUmSNHS9G3aiH/N3J0mSJA1YrysEkiRJWh0c799fVggkSZKkAbNCIEmSpM6sD/SXCYEkSZI6c8hQfzlkSJIkSRowKwSSJEnqzG+Z+8vfnSRJkjRgVggkSZLUmXMI+ssKgSRJkjRgVggkSZLUmfWB/jIhkCRJUmeOGOovhwxJkiRJA2aFQJIkSZ2tc9BQb1khkCRJkgbMCoEkSZI6cw5Bf1khkCRJkgbMCoEkSZI6i3MIessKgSRJkjRgVggkSZLUmXMI+suEQJIkSZ257Gh/OWRIkiRJGjArBJIkSerMIUP9ZYVAkiRJGjArBJIkSerMCkF/paqmHcNq4xsiSZL6YtV8DP/Ef16zLJ+hDvn5B6ya17hWWSGQJElSZ96YrL9MCCRJktTZOvOB3jIhmMdtd007guFY3/4L9D1fOTPv+S13ODpupWy9RfNfyef843lTjmQ4Tj5sXwBuvXPKgQzIVps32yu/f/t0AxmQXe675bRD0BphQiBJkqTOHDLUXy47KkmSpDUvyfokb0xySZLbknwnyYlJdt6EvrZP8o4k30pye7s9Lsn2I9qflKQWeLx0gWsdmuSMJDck+WH786FLjXkhVggkSZLU2WpedjTJeuB04EDgKuAUYHfgMODQJAdU1WVj9nU/4CzgIcA3gA8BewFHAk9Osn9VXTfi9I8D351n/8UjrnUkcBxwF/Ap4HbgEODUJEdV1fHjxLwYEwJJkiStda+lSQbOAg6pqpsAkhwNvA04EThozL7eTpMMfBB4XlXd1fZ1PHAEcCzwwhHnvqWq/mOciyR5aBvb7cDBVXXWrP1fAN6W5KNVdemYcY/kkCFJkiR1lmX6X+e4ks1pPqgDvHwmGQCoqmOBC4EnJNlvjL4eCDwfuBN42Uwy0HoVcA3w/CQ/0zlwOIrmy/u/nUkG2pgvAf6sPXbkBK5jQiBJkqTu1mV5HhPwOGB74LKqOn+e4ye326eO0dev0Xx+/mxVXT37QFXdDpwKbNa262pmnsDJ8xz7QLsdJ+ZFOWRIkiRJa9kj2+2otZ/Pm9Oua18vXqCvZyV5Nk3ScDlwalVtmNuonZy8a/v0HklMVW1Mci2wW5LtquqGMWIfyYRAkiRJna3iZUdnPlhvHHF845x2y9nXEXOe/0WSvwGOmjP8aOb866vq5gWudf+27UWjQ16cCYEkSZJWrSRfHXWsqvYao4tt2+0tI47fPKfdcvR1Ps2E5k/TfJB/IM2wojcBLwPuAF6xhOssdK0lMyGQJElSZ6t42dGZyGqR48vWV1UdN2fX5cA7k3wWOBc4IsmxVXXlmNcZea1NYUIgSZKkVWvMKsBCbmy324w4vnW7vWnE8eXqi6r6SpIPA88Bngj845jXWfK1FmJCIEmSpM5Wb4GAK9rtqDsS7zyn3Ur1NWPmPgI7zXOdHZJsM2IewaZca14uOypJkqS17IJ2u++I4zP7L1zhvmbs0G5n3x/hB/zkg/4+c09IsjPNhOIruq4wBCYEkiRJmoB1ybI8JuDzwA3AHknu8eGaZrgOwEfG6OtjwN3A45PsOPtAki1p7gtwN/DRcQJrz3lK+/TcOYdPmxPfbM9dQsyLMiGQJElSZ1mmR1dVdQdwQvv0hCQ/Hpef5Ghgb+BzVXX2rP2HJ9mQ5M1z+roKeB+wBc2k4NnD7/8SeADw3qr67qy+9kzy9CSbze4ryQOA9wO70FQevjAn9OOAHwEvTbL/rPMeAvxxe+z48d+J0ZxDIEmSpLXuTTSTdg8ELk1yJrAb8FjgOuCwOe3vD+zJT4/rn/GHwP7As4ENSc4B9gIeAVzGTy8fStvHh4DrkmwAvg3sCOwH3JtmGdJfr6qfWlGoqi5O8irgWODMJJ+kWZ70EGAr4OiqunhJ78IIVggkSZLU3WotEQBVdRtwMPCnNGv7PwPYHXgPsE9VfX0JfV0LPAb4a5pKwTOB7WiqEL/YHp/tEuAdNJOH92jbP7p9/kZg76q6ZMS13g48jeYeBo8H/ivN0KKnt8cmInOSEUHddtfijTQZ69sale/5ypl5z2+5w//vr5Stt2j+i/acfxx1p3tN2smHNfP6br1zyoEMyFabN9srv3/7dAMZkF3uuyWsosV9vnjZD5blPyz777H9qnmNa5VDhiRJktRZVk9uoiVyyJAkSZI0YFYIJEmS1NlkVgjVNJgQSJIkqTPzgf5yyJAkSZI0YFYIJEmS1J0lgt6yQiBJkiQNmBUCSZIkdeayo/1lhUCSJEkaMCsEkiRJ6sxlR/vLhECSJEmdmQ/0V68SgiQPq6oN7c/rgFcC+wMXAm+pqtvH7Oero45V1SRClSRJknqhb3MI3jvr5zcCvwScCOwGHDeViCRJktSUCJbjoWXXqwoBP/3P4lDgcVV1c5KPA+eP20lV7bXQ4U0NTpIkSeqbviUE90qyE01l4+6quhmgqu5K8qPphiZJkjRcLjvaX31LCO4NfJ62UpDkQVX1nST3xm/2JUmSpCXrVUJQVbuPOPQj4FkrGIokSZJmcdnR/upVQjBKVd0CXD7tOCRJkqS+WRMJgSRJkqbLAkF/mRBIkiSpOzOC3urbfQgkSZIkTZAVAkmSJHXmsqP9ZYVAkiRJGjArBJIkSerMZUf7ywqBJEmSNGBWCCRJktSZBYL+MiGQJElSd2YEveWQIUmSJGnArBBIkiSpM5cd7S8rBJIkSdKAWSGQJElSZy472l9WCCRJkqQBs0IgSZKkziwQ9JcJgSRJkrozI+gthwxJkiRJA2aFQJIkSZ257Gh/WSGQJEmSBswKgSRJkjpz2dH+skIgSZIkDZgVAkmSJHVmgaC/rBBIkiRJA2aFQJIkSd1ZIugtEwJJkiR15rKj/ZWqmnYMq41viCRJ6otV8yn80qtvXZbPUA/5ma0m8hqTrAdeA/wmsCvwfeBjwOurauMS+9oeOAZ4JvBA4LvAh4A3VNUP5rTdHDgYeBpwAPBzwFbAN4HTgL+oqmvmucaLgH9cIIx/rarfWErco1ghkCRJUmerednRNhk4HTgQuAo4BdgdOAw4NMkBVXXZmH3dDzgLeAjwDZpEYC/gSODJSfavqutmnXIQ8PH258uAzwCb0yQHrwSen+SXq+riEZe8APjyPPu/NE684zAhkCRJ0lr3Wppk4CzgkKq6CSDJ0cDbgBNpPriP4+00ycAHgedV1V1tX8cDRwDHAi+c1f5u4H3AW6vq/JmdSbYD/hV4Ek0l4MAR1/tQVR0zZmybxCFD91S33TXtEIZjfZuS+p6vHN/zled7vvJm3vOTzr5iuoEMyIsesysA/3TOlVOOZDh+59G7wCoaMnTZ95ZnyNAeO3YbMtQO2fkesD2w7+wP5e3xC4C9gUdX1bmL9PVA4NvAj4BdqurqWce2BK4E7gv87OxjC/S3E/Cd9unuVfWtWcdeRJMovHG5EwKXHZUkSdJa9jiaZOCyuclA6+R2+9Qx+vo1ms/Pn537gb+qbgdOBTZr2y2qqq4CZuYPPGicc5aDQ4YkSZLU3aqpVdzDI9vteSOOnzenXde+XjxmXzOTk3don353RLP9krwVuE/b5tNVdcY4/Y/LhECSJEmdLdeyo0m+OupYVe01Rhe7tttRKwltnNNupfoCeDnN5/GLquryEW0ObR8zXp/kDJr5C4sOSxqHQ4YkSZK0lm3bbm8ZcfzmOe1WpK8k+wD/s336R/M0uYpmadN9gO1oljd9GrCBZgL0aUk2GyPmRVkhkCRJUmfLtezomFWAhcxENmrS81Iin0hf7eTkDwLrgXdU1Ufntqmqj/OT5UoBfgicmuQzwLnAfsDzgPeOF/poVggkSZK0lt3YbrcZcXzrdnvTSvTVLjf6UZr7IHyA5l4EY2uXTD2+ffqkpZw7ihUCSZIkdbZ65xQzs/7wziOO7zyn3bL1lWQrmpWIHgV8AnhBVd09xnXnurTd7rQJ596DFQJJkiStZRe0231HHJ/Zf+Fy9pXkXjQVgccDXwCeVVV3jHHN+cysTDROVWNRJgSSJEnqLsv06O7zwA3AHu1E3rme024/MkZfH6O58/Djk+w4+0B7Y7Kntsc/OudYgJOApwBfBp5SVTez6Z7dbhe8kdq4TAgkSZLUWZbpf12138Kf0D49IcmPx/8nOZrmLsWfq6qzZ+0/PMmGJG+e09dVwPuALYB3tt/6z/hL4AHAe6tq7j0FjgOeT7NC0CFV9YPF4k5yZJJt5+zbPMkbgOcCt9IkGZ05h0CSJElr3ZuAJwIHApcmORPYDXgscB1w2Jz29wf2ZP4x+n8I7E/zLf2GJOcAewGPAC4DXjG7cZKnA0e0T68E3pr5l2R6S1VtmPX8OOAtSb4GfItmRaJH0dzR+Daa+QffXuR1j8WEQJIkSZ0t17Kjk1BVtyU5GHgN8FvAM4DrgfcAr6uqK5fQ17VJHgO8se3nmcDVNFWIN1TV9+ecssOsn//bAl2fRFNBmPEnwAHAw4CH0wyg2gi8C3h7VV08bsyLSdWoZVQHq267a9ohDMf6NiX1PV85vucrz/d85c285yedPc6iIZqEFz2muTHrP50z9ucqdfQ7j94FVtHiPld8//Zl+VC56323XDWvca2yQiBJkqTO/NTeX04qliRJkgbMCoEkSZI6W81zCLQwKwSSJEnSgFkhkCRJ0gRYIugrEwJJkiR15pCh/nLIkCRJkjRgVggkSZLUmQWC/rJCIEmSJA2YFQJJkiR15hyC/rJCIEmSJA2YFQJJkiR1FmcR9JYJgSRJkrozH+gthwxJkiRJA2aFQJIkSZ1ZIOivXlUIkvxekvu3P++W5PQk309yZpI9px2fJEmS1Dd9qxC8oqr+vv35HcB7gUOBQ4B3Ab88TidJvjrqWFV1i1CSJGmAXHa0v3pVIQA2m/XzrlX17qq6tapOAbafUkySJElSb/WtQvDFJK8E3g78vySHVNUnkuwH3DJuJ1W110KHuwYpSZI0NC472l99SwheBhwLXAF8D/j9JNcBXwF+d5qBSZIkDZr5QG/1KiGoqptpkoBtgQcDmwNXVtX3phuZJEmS1E+9SghmVNVNwIXTjkOSJEkNCwT91bdJxZIkSZImqJcVAkmSJK0uLjvaX1YIJEmSpAGzQiBJkqTOXHa0v6wQSJIkSQNmhUCSJEmdOYegv6wQSJIkSQNmQiBJkiQNmEOGJEmS1JlDhvrLCoEkSZI0YFYIJEmS1JnLjvaXFQJJkiRpwKwQSJIkqTPnEPSXCYEkSZI6Mx/oL4cMSZIkSQNmhUCSJEndWSLoLSsEkiRJ0oBZIZAkSVJnLjvaX1YIJEmSpAGzQiBJkqTOXHa0v6wQSJIkSQNmhUCSJEmdWSDoLxMCSZIkdWdG0FsOGZIkSZIGzAqBJEmSOnPZ0f6yQiBJkiQNWKpq2jGsNr4hkiSpL1bN1/K33bU8n6HW32v1vMa1yoRgDUjyVYCq2mvasQyF7/nK8z1feb7nK8/3fOX5nksOGZIkSZIGzYRAkiRJGjATAkmSJGnATAgkSZKkATMhkCRJkgbMhECSJEkaMJcdlSRJkgbMCoEkSZI0YCYEkiRJ0oCZEEiSJEkDZkIgSZIkDZgJgSRJkjRgJgSSJEnSgJkQSJIkSQNmQiBJkiQNmAnBGpDk4CRfS/L1JCcmude0Y1rLkvx1ko1J7pp2LEORZJckpyf5zyRfSfKmacc0BEk+meSCJBcmOTnJfaYd01Akead/Y1ZGkm8m+WqSL7ePh087JmmlmRD0XJLNgHcDz62q/wJsC/z2dKNa8/4V2G/aQQzMXcAfVdXPA/sCj0/ytCnHNATPqapHVtXewBXAK6Yd0BAkeTywzbTjGJgnVdWj2sfXph2MtNJMCPrvMcDGqvpq+/zdwLOmGM+aV1Wfq6qrpx3HkFTVVVV1TvvzHcCFwG7TjWrtq6obAJKsA7aecjiDkGRL4C3Af592LJKGw4Sg/3YGrpz1/Ip2n7QmJbkf8AzgU1MOZRCSfBi4Gvh54G1TDmcIXg+8u6qumXYgA3NqOzzuTQ671RCZEKwNNevnTC0KaZm1356eDBxbVf857XiGoKqeBjwQ+CLwsimHs6Yl2Rt4LPCP045lYB5XVfsAjwN+AXjllOORVpwJQf9dCew66/nOwMYpxSItm3a+zL8A51TV26cdz5BU1Y+AE4HfmXYsa9wvAQ8HLk/yTWCzdsLrDtMNa22rqo3t9kbg74H9pxuRtPJMCPrvHOBnZ62K8GLgg1OMR1oufwf8EHj1tAMZgiT3SbLTrF3PBr4yrXiGoKr+pqoeVFW7V9XuwI/an6+fdmxrVZJtZlbPaocKPZtmjpI0KCYEPdd+c/d7wMlJvg7cAvzzdKNa25K8K8lGmm/vNiZ517RjWuuS/BJNsvsY4Px2acAjpxzWWrcd8OEkFyW5EHgUcNR0Q5Im7meAz7b/xi8A7qSZ1C0NSqpq8VaSJEmS1iQrBJIkSdKAmRBIkiRJA2ZCIEmSJA2YCYEkSZI0YCYEkiRJ0oCZEEiSJEkDZkIgSZIkDZgJgSRJkjRgJgSSJEnSgJkQSFr1kmyT5BVJPpPk6iR3JLk+yVlJ/iTJrtOOcbYkv5ykkpw0Z/8x7f4Xde1LkqRJude0A5CkhSTZH/ggsBNwC/BF4GpgO+AxwP7Aq5McWlWfmlqgPZDkm8BuVZVpxyJJWj1MCCStWkn2Bj4NbAX8BfCnVXXzrOPrgGcAfwnsPI0Yl+gE4P3AVdMORJKkGSYEklalJAH+N00ycExVvXFum6q6G/hgktOBXVY4xCWrqmuBa6cdhyRJszmHQNJq9STgF4CNwJ8t1LCqbqiqrwAk2SnJq5OckeTb7XyD7yb5YJLHzHd+km8mqfbnlyS5MMmt7XnvSrL9iPN2T/K+JNcluSnJF5I8ZVScC80h2IS+xn6dM/MQgN3a5zXr8c05bbdIclSSs5PcmOTmJP8vye+2SZokaY2xQiBptZr5MPyBqrprCec9nWZ40deBi4AfAv8FeCZwaDvX4BPznZjkL4GjgLOBjwEHAv8f8PNJDqqqmtV2D+ALwI7AJcB5wM8BpwJ/u4R4N7WvpbzO7wLvAZ4DbNP+POPHFYsk2wAfBR7f7v8ccDdwAPAPNHM2XrqU1yZJWv1MCCStVvu02/OWeN7ngUdW1YWzdyZ5EvBh4J1JHjL7w/0sLwAeW1Vfbs+5P3AWzQfkg2nmM8x4J80H+HcCR7TDl0jyEuDvlxjzpvQ19uusqg3Ai5L8MrBNVb1oRJ9vpXmt/wy8rKpuavt8AE1y8vtJTq2q05b4+iRJq5hDhiStVvdrt9cs5aSqumjuh+R2/8eBDwB7AI8YcfrrZpKB9pxrgb9pnz5hZn/7jf4hwPXAq2c+wLfn/APNt/1j2dS+Or7O+eLYEXgJcDnwezPJQNvnNcDvt09/f57TJUk9ZoVA0mq1yePVk2wJ/Crwi8ADgC3aQ7/Qbh9CM8xmrvmGEl3Sbneate+X2u3/nb3q0SzvpxluNI5N7qvD65zPQcDmwMeq6va5B6vqgiQ30gwbkiStISYEklara4E9aT7oji3JL9AMmdl9gWb3HrF/4zz7Zr4p33LWvge12ytG9DNq/3w2qa+Or3M+M/38QZI/WKDdVkvoU5LUAyYEklarL9N8e74vzfKji2pXwfk3mg+3f9s+vgHcVFWV5M+B1zCi+jBiXsG8l5o5Zcz2E+2r6+scYbN2ez5wj6FIkqS1y4RA0mp1GvBy4LlJXj3mSkMPax/nVNV833I/eEKxfafd7jbi+K7L3NdyvM6Z6sh/VNXRm3C+JKmnnFQsabX6GPBVmjsQ//FCDZPcJ8lewA7trnsM/UmyA/DfJhTb59vtk9ulOuf6jWXua1Nf5x1tm/m+DPoM8COaJUs3m+e4JGmNMiGQtCq1w3deANwGHJPkzXM/MKfxNOAcmsmuX6dZN/9XkjxkVrv1NMNq7juh2L4OnE7zwfwtSX78tzTJYYw/oXhT+9rU1zlTjdhznji+DZxEMxH5n9slV39KkgOTPHmMlyVJ6hETAkmrVrsE6BOBq4H/AXwvyaeS/EuSjwBXAacAuwBXVtX3gHcD9wEuSPKRJB8Avgn8Cs0H3kn5A5olUQ8HvpbkvUnOaq+/pBuTLbWvDq/zw+329PauyP+Q5C2zjh9JUyn4TeAbST6b5P1J/iPJRppqxiFLfG2SpFXOhEDSqlZVn6e5A+9/p7mD8N7Ar9NMOP4m8EbgIVV1envKHwCvpFlP/7/S3GjrU8CjgW9NMK5Lgf1pJvfuSHPn4ADPAP51BfralNd5PPAmmpWTng38LrOGJFXVLTQf+F9Cc0O4R9Dc+XgP4DLg1cBfLeW1SZJWv4y/qIYkSZKktcYKgSRJkjRgJgSSJEnSgJkQSJIkSQNmQiBJkiQNmAmBJEmSNGAmBJIkSdKAmRBIkiRJA2ZCIEmSJA2YCYEkSZI0YCYEkiRJ0oCZEEiSJEkDZkIgSZIkDZgJgSRJkjRgJgSSJEnSgJkQSJIkSQNmQiBJkiQNmAmBJEmSNGD/P6/P0OH/5A2DAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 900x600 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Print the sentences and plot the optimal transport plan for the pair of abstracts.\n",
    "print('\\n'.join([f'{i}: {s}' for i, s in enumerate(ex_abstracts[0]['ABSTRACT'])]))\n",
    "print('')\n",
    "print('\\n'.join([f'{i}: {s}' for i, s in enumerate(ex_abstracts[1]['ABSTRACT'])]))\n",
    "h = sns.heatmap(transport_plan, linewidths=.7, cmap='Blues')\n",
    "h.set(xlabel='Candidate', ylabel='Query')\n",
    "h.tick_params(labelsize=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "963927df",
   "metadata": {},
   "outputs": [],
   "source": [
    "ex_abstracts = [\n",
    "    {'TITLE': \"Large scale metric learning from equivalence constraints\",\n",
    "     'ABSTRACT': [\"In this paper, we raise important issues on scalability and the required degree of supervision of existing Mahalanobis metric learning methods.\",\n",
    "                  \"Often rather tedious optimization procedures are applied that become computationally intractable on a large scale.\",\n",
    "                  \"Further, if one considers the constantly growing amount of data it is often infeasible to specify fully supervised labels for all data points.\",\n",
    "                  \"Instead, it is easier to specify labels in form of equivalence constraints.\",\n",
    "                  \"We introduce a simple though effective strategy to learn a distance metric from equivalence constraints, based on a statistical inference perspective.\",\n",
    "                  \"In contrast to existing methods we do not rely on complex optimization problems requiring computationally expensive iterations.\",\n",
    "                  \"Hence, our method is orders of magnitudes faster than comparable methods.\",\n",
    "                  \"Results on a variety of challenging benchmarks with rather diverse nature demonstrate the power of our method.\",\n",
    "                  \"These include faces in unconstrained environments, matching before unseen object instances and person re-identification across spatially disjoint cameras.\",\n",
    "                  \"In the latter two benchmarks we clearly outperform the state-of-the-art.\"]},\n",
    "    {'TITLE': \"PCCA: A new approach for distance learning from sparse pairwise constraints\",\n",
    "     'ABSTRACT': [\"This paper introduces Pairwise Constrained Component Analysis (PCCA), a new algorithm for learning distance metrics from sparse pairwise similarity/dissimilarity constraints in high dimensional input space, problem for which most existing distance metric learning approaches are not adapted.\",\n",
    "                  \"PCCA learns a projection into a low-dimensional space where the distance between pairs of data points respects the desired constraints, exhibiting good generalization properties in presence of high dimensional data.\",\n",
    "                  \"The paper also shows how to efficiently kernelize the approach.\",\n",
    "                  \"PCCA is experimentally validated on two challenging vision tasks, face verification and person re-identification, for which we obtain state-of-the-art results.\"]}\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "3faa2d74",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pre-process the data.\n",
    "bert_batch, abs_lens, sent_token_idxs = prepare_abstracts(batch_abs=ex_abstracts,\n",
    "                                                          pt_lm_tokenizer=aspire_tok)\n",
    "\n",
    "# Get sentence embeddings for the papers.\n",
    "clsreps, contextual_sent_reps = aspire_mv_model.forward(bert_batch=bert_batch,\n",
    "                                                        abs_lens=abs_lens,\n",
    "                                                        sent_tok_idxs=sent_token_idxs)\n",
    "\n",
    "# Package the sentence embeds for use with the distance function\n",
    "query_embeds = contextual_sent_reps[0,:,:].unsqueeze(0)\n",
    "cand_embeds = contextual_sent_reps[1,:,:].unsqueeze(0)\n",
    "qt = rep_len_tup(embed=query_embeds.permute(0, 2, 1), abs_lens=[abs_lens[0]])\n",
    "ct = rep_len_tup(embed=cand_embeds.permute(0, 2, 1), abs_lens=[abs_lens[1]])\n",
    "\n",
    "# Compute the wasserstein distance and alignment between abstracts\n",
    "wd, intermediate_items = ot_distance.compute_distance(query=qt, cand=ct, return_pair_sims=True)\n",
    "\n",
    "# Get the transport plan\n",
    "transport_plan = intermediate_items[3].data.numpy()[0,:abs_lens[0],:abs_lens[1]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "fb64fa4c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0: In this paper, we raise important issues on scalability and the required degree of supervision of existing Mahalanobis metric learning methods.\n",
      "1: Often rather tedious optimization procedures are applied that become computationally intractable on a large scale.\n",
      "2: Further, if one considers the constantly growing amount of data it is often infeasible to specify fully supervised labels for all data points.\n",
      "3: Instead, it is easier to specify labels in form of equivalence constraints.\n",
      "4: We introduce a simple though effective strategy to learn a distance metric from equivalence constraints, based on a statistical inference perspective.\n",
      "5: In contrast to existing methods we do not rely on complex optimization problems requiring computationally expensive iterations.\n",
      "6: Hence, our method is orders of magnitudes faster than comparable methods.\n",
      "7: Results on a variety of challenging benchmarks with rather diverse nature demonstrate the power of our method.\n",
      "8: These include faces in unconstrained environments, matching before unseen object instances and person re-identification across spatially disjoint cameras.\n",
      "9: In the latter two benchmarks we clearly outperform the state-of-the-art.\n",
      "\n",
      "0: This paper introduces Pairwise Constrained Component Analysis (PCCA), a new algorithm for learning distance metrics from sparse pairwise similarity/dissimilarity constraints in high dimensional input space, problem for which most existing distance metric learning approaches are not adapted.\n",
      "1: PCCA learns a projection into a low-dimensional space where the distance between pairs of data points respects the desired constraints, exhibiting good generalization properties in presence of high dimensional data.\n",
      "2: The paper also shows how to efficiently kernelize the approach.\n",
      "3: PCCA is experimentally validated on two challenging vision tasks, face verification and person re-identification, for which we obtain state-of-the-art results.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvcAAAIXCAYAAAAYO3E5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAABcSAAAXEgFnn9JSAAA3CUlEQVR4nO3de7htZVn///dnE7DloKDAFwwBJYEEUfAA4gEpv5aKJ8BDar/E9KchiKJRaJaUiXlASaKsJLlMsiQFkdSU8oRoKshJEUQMtoAiIXHawJb7+8ccC5bLNfeaa4259hxz7PfLa15jznF4nmfsNZne8573eEaqCkmSJEnTb8WkByBJkiRpPAzuJUmSpJ4wuJckSZJ6wuBekiRJ6gmDe0mSJKknDO4lSZKknjC4lyRJknrC4F6SJEnqCYN7SZIkqScM7iVJkqSeMLiXJEmSesLgXpIkSeoJg3tJkiSpJ35p0gPooJr0ACRJkkaUSQ9gxn32OnxZYqjbzz+xM+c4DQzu5/Gtq26e9BDUcY/cYfN7nq9eM8GBaCqsbD5pr7vprskORJ237f02vOe5ny1ayEqjOM3Dt4UkSZLai9XeXeBfQZIkSeoJM/eSJElqL5bGd4GZe0mSJKknzNxLkiSpPWvuO8G/giRJktQTZu4lSZLUnjX3nWBwL0mSpPYsy+kE/wqSJElST5i5lyRJUnuW5XSCmXtJkiSpJ8zcS5IkqT1r7jvBv4IkSZLUE2buJUmS1J41951g5l6SJEnqCTP3kiRJas+a+04wuJckSVJ7luV0gl+xJEmSpJ4wcy9JkqT2LMvphKkK7pOsAH4bOAjYvlm9CvgY8KGquntSY5MkSZImbaqCe+Bvgc2BvwKuAsIgyH8Z8ATgFaM0kuSSYduqqv0oJUmS1jfW3HfCtAX3T6qqXeas+w7w2SSXTWJAkiRJUldMW3B/V5LdqurS2SuT7AbcNWojVbX72jYvdXCSJEnrLWvuO2HagvvDgbOSXA1czSAQ3xF4ICOW5EiSJGkZGNx3wlQF91X1n0keCjyGey+ovRr4hhfTSpIkaX03VcE9QBPEf615SJIkqQtWeEFtF/j7iSRJktQTU5e5lyRJUgdZc98J/hUkSZKknjBzL0mSpPa8iVUnGNxLkiSpPctyOsG/giRJktQTZu4lSZLUnmU5nWDmXpIkSeoJM/eSJElqz5r7TvCvIEmSJPWEmXtJkiS1Z819J5i5lyRJknrCzL0kSZLas+a+EwzuJUmS1J5lOZ3gVyxJkiSpJ8zcS5IkqT3LcjrB4H4ej9xh80kPQVNkpf8VaUTb3m/DSQ9BU8TPFklL4UeHJEmS2rPmvhP8/USSJEnqCTP387j1zpr0ENRxm250b3bi+lvWTHAkmgZbbzb4qF3tW0ULmF2K4/tFC+lc6ZY1953QtbeFJEmSppHBfSf4V5AkSZJ6wsy9JEmS2vOC2k4wcy9JkiT1hJl7SZIktWfNfSf4V5AkSZJ6wsy9JEmS2rPmvhMM7iVJktSeZTmd4F9BkiRJvZdkZZJjk1yWZHWSa5KcnGT7RbSxRZIXJTk1ybeT3Jrk5iRfS3Jkkg2HHPfBJLWWx6vGdZ5m7iVJktReh8tykqwEzgb2A64FzgB2Ag4FDkzyuKq6YoSm3gC8CbgbOB84E9gaeDzwWOCQJL9RVbcNOf4zwHXzrP/u6Gezdgb3kiRJ6rs3MgjszwWeWlW3ACQ5Cng3cDKw/wjt3AK8DTipqn44szLJQ4HPAU8A/qjpbz5vr6rPL/EcRmJZjiRJklpLsiyPMYxrQ+CI5uWrZwJ7gKo6HrgQeFKSRy3UVlW9vareNDuwb9ZfDvxh8/K3Wg+6BYN7SZIk9dkTgC2AK6rq/Hm2n9Ysn9mynwua5QNbttOKZTmSJElqbRxZ9mXyiGZ53pDt583Zb6ke0iznq6mfcVCSg4ENgCuBM6vq0pb9/hyDe0mSJHVWkkuGbauq3UdoYodmuWrI9lVz9luqI5vlGWvZ54g5r/8iyV8DR1bVmpb9A5blSJIkaRyyTI/2NmuWw2awuXXOfovWTGX5FOCnwNvn2eV84FXALsAmDLL8r272Pwx451L7nsvMvSRJklpbrrKcEbPzazMzsFpg+9IaT/YHTmjaf1lVXTN3n6o6Yc6qK4GTknwR+CZwRJLjq+rqNmMBM/eSJEnqt5ub5aZDtm/SLG8Zsn2oJHsCpwMbMSit+fhijq+qi4FPMKjBf8pi+5+PmXtJkiS11uELaq9qlsPuRLv9nP1GkmRnBjel2gJ4S1W9b0mjg8ub5XZLPP7nrJfB/QIXZqzLoUiSJGl5zUxRufeQ7TPrLxy1wSQPBD4LbAucUFXHLn14bNksF/3LwXx6VZbTXG0sSZKkdayrN7ECzgFuAnZOstc82w9plp8c8Ty3ZJCxfzDwD8DrljqwJBsDz2hefnOp7czWq+Cee/9x1qqqdh/2WO4BSpIkad2pqjuBE5uXJya5p/Y+yVHAnsCXq+rrs9YfnuTSJMfNbivJJsC/AXsA/wK8ohYo+0iya5JnJ9lgzvqtgY8AD2Lw68JXlnqOs01dWU6SO4dton9fViRJkqZCh2vuAd7K4ILV/YDLk3wJ2BHYB7gBOHTO/lsBu/KLdfB/DuwL/AxYA3xgvvOuqpfOerkdg4tub0hyKfBDYBvgUcDmDObZf/5CXxJGNXXBPfAjYO+qun7uhiStpw+SJEnSEnQ4tq+q1UkOAI4BXgQ8B7gROAV48yKmoJypj9+gaWeYl856fhnwXgZfCnYGHgvc0aw/k0HN/o0j9r+gTNsFpEn+EvhIVf3CTxdJ/qaqXtWyi7r1zun6N9G6t+lG936CXX/LWG4opx7berNBHmW1bxUtYOWslJvvFy2keb90JqS+34s+tCwB1E2n/nZnznEaTF3mvqpes5ZtbQN7SZIkLUHHy3LWG9aoS5IkST0xdZl7SZIkdY+Z+24wcy9JkiT1hJl7SZIktWbmvhsM7iVJktSawX03WJYjSZIk9YSZe0mSJLVn4r4TzNxLkiRJPWHmXpIkSa1Zc98NZu4lSZKknjBzL0mSpNbM3HeDmXtJkiSpJ8zcS5IkqTUz991gcC9JkqT2jO07wbIcSZIkqSfM3M9j04386qnRbb2Z/xlpNCt9q2gRfL9o2liW0w1m7iVJkqSeMC8gSZKk1szcd4PB/TxWr5n0CNR1s38uP/+/b57cQDQV9tpxc8DPFi1s9meL7xctxNItzce3hSRJklozc98NBveSJElqzeC+G7ygVpIkSeoJM/eSJElqz8R9J5i5lyRJknrCzL0kSZJas+a+G8zcS5IkST1h5l6SJEmtmbnvBoN7SZIktWZw3w2W5UiSJEk9YeZekiRJ7Zm47wQz95IkSVJPmLmXJElSa9bcd4OZe0mSJKknzNxLkiSpNTP33WDmXpIkSeoJM/eSJElqzcx9NxjcS5IkqTWD+26wLEeSJEnqianL3Cc5ENgR+FxVfXfW+sOr6sQR27hk2Laqaj9ISZKk9Y2J+06Yqsx9kj8F3gDsCnw2yWtmbX7ZZEYlSZIkdcO0Ze6fA+xdVWuSHAt8NMkDq+oPWcT3xarafW2bW45RkiRpvWPNfTdMVeYe2KCq1gBU1Q3AbwI7J/l7pu9cJEmSpLGatoD42iR7z7yoqjuBFzDI2u8xsVFJkiSt55Isy0OLM21lOS8F1sxeUVV3A7+b5AMTGZEkSZIwDu+GqQruq2rVWrZ9ZV2ORZIkSeqaqQruJUmS1E2W0HTDtNXcS5IkSRrCzL0kSZJaM3HfDWbuJUmSpJ4wcy9JkqTWrLnvBoN7SZIktWZs3w2W5UiSJEk9YeZekiRJra1YYeq+C8zcS5IkST1h5l6SJEmtWXPfDWbuJUmSpJ4wcy9JkqTWnAqzG8zcS5IkST1h5l6SJEmtmbjvBoN7SZIktWZZTjdYliNJkqTeS7IyybFJLkuyOsk1SU5Osv0i2tgiyYuSnJrk20luTXJzkq8lOTLJhms5dkWS1ya5KMntSa5P8tEkDxvPGQ6YuZ/HSv9VtAh77bj5pIegKeFnixbD94umTZcz90lWAmcD+wHXAmcAOwGHAgcmeVxVXTFCU28A3gTcDZwPnAlsDTweeCxwSJLfqKrb5vQf4J+BQ4CfAmcBWwEHA89IckBVfa3laQJm7iVJktR/b2QQ2J8L7FJVL6iqfYDXMwjOTx6xnVuAtwE7VNWjq+qFVfXrwMOBq4AnAH80z3GHMgjsLwd2q6pDqurJwPOA+wAfTjKWr/SpqnG00yd1420/m/QY1HFbbrLBPc+v+p87JjgSTYMd7r8xAKvXTHgg6rzZ2XrfL1pI837pTLr8kW85e1mCym+95ddbnWNTKvNjYAtg76o6f872C4A9gUdX1Tdb9PNbwKnAD6rqwXO2XQI8DHhuVZ0+Z9sZwLOAQ6rqX5fa/wwz95IkSeqzJzAI7K+YG9g3TmuWz2zZzwXN8oGzVyZ5MIPA/nYG5TjL1T9gzb0kSZLGoMM1949olucN2X7enP2W6iHN8roh/V9cVXctY/+Awb0kSZLGYLli+6akZV5VtfsITezQLFcN2b5qzn5LdWSzPGNC/QOW5UiSJKnfNmuWtw3Zfuuc/RYtyauApzCYCeft67r/2czcS5IkqbXlKssZMTu/NjMDG3bBb9sLdvcHTmjaf1lVXbPI/sfK4F6SJEl9dnOz3HTI9k2a5S2LbTjJnsDpwEbAa6rq40vof2b9ovufj8G9JEmSWuvu9bRc1SyH3Yl2+zn7jSTJzsBnGMzE85aqet+67H8Ya+4lSZLUZzNTVO49ZPvM+gtHbTDJA4HPAtsCJ1TVsSP0v0cz537r/tfG4F6SJEmtJVmWxxicA9wE7Jxkr3m2H9IsPznieW7JIGP/YOAfgNetbf+quhL4DoM70T6jbf8LMbiXJElSa8nyPNqqqjuBE5uXJya5p/Y9yVEM7k775ar6+qz1hye5NMlxP3+O2QT4N2AP4F+AV1TVKBfKHt8s35Fkm1ntHcTg7rRXMqjdb82ae0mSJPXdWxlMVbkfcHmSLwE7AvsANwCHztl/K2BXYLs56/8c2Bf4GbAG+MB8vy5U1UvnrDoZeDrwXODSJGc3fewPrAZeMuQGV4tmcC9JkqTWOnyHWqpqdZIDgGOAFwHPAW4ETgHeXFVXj9jUls1yg6adYV46p/+7kzyPwY2uXgYcyGB++48Df1xVQ2/UtVgZ7ZeE9UrdeNvPJj0GddyWm2xwz/Or/ueOCY5E02CH+28MwOo1Ex6IOm/lrJSb7xctpHm/dCai3ue4LyxLUPm1Y/bvzDlOAzP3kiRJaq3Difv1ihfUSpIkST1h5l6SJEmtdbnmfn1i5l6SJEnqCTP3kiRJas3EfTcY3EuSJKk1y3K6YarKcpJsluTNzWOTJL+f5KIkpybZetLjkyRJkiZp2jL3JwNXA5sAZwFfBQ5uHu8DXjhKI0mG3ijAef8lSZIWz8R9N0xbcL9rVT0/yQrgOuDXahCNH5fkW5MdmiRJkjRZ0xbc3w333ML33Pr5NPvI3xerave1bV7q4CRJktZX1tx3w1TV3APXJ9kcoKqePbMyybbAHRMblSRJktQBU5W5r6qnDtl0K3DQuhyLJEmS7mXmvhumKrgfpqpuBm6e9DgkSZLWV8b23TBtZTmSJEmShuhF5l6SJEmTZVlON5i5lyRJknrCzL0kSZJaM3HfDWbuJUmSpJ4wcy9JkqTWrLnvBoN7SZIktWZs3w2W5UiSJEk9YeZekiRJra0wdd8JZu4lSZKknjBzL0mSpNZM3HeDmXtJkiSpJ8zcS5IkqTWnwuwGM/eSJElST5i5lyRJUmsrTNx3gsG9JEmSWrMspxsM7uex5SYbTHoImiI73H/jSQ9BU2Kln7haBN8vkpbCjw5JkiS1ZuK+G7ygVpIkSeoJM/fz+LdLfjzpIajjnr77Nvc8X71mggPRVJgpr7j1zprsQNR5m250b+rTzxYtpGulW8HUfReYuZckSZJ6omPf+SRJkjSNnAqzGwzuJUmS1JpTYXaDZTmSJElST5i5lyRJUmsm7rvBzL0kSZLUE2buJUmS1NoKU/edYOZekiRJ6gkz95IkSWrNxH03GNxLkiSpNafC7AbLciRJkqSeMHMvSZKk1kzcd4OZe0mSJKknzNxLkiSpNafC7AYz95IkSVJPjDW4T7LfONuTJEnSdMgyPbQ4487cfznJxUmOTHL/MbctSZIkaS3GHdx/AfhV4Hjgh0k+nOTJY+7j5yS533K2L0mSpIUlWZaHFmeswX1VHQDsArwTuBH4LeDsJJclOTrJ/xlnf43zl6FNSZIkLcKKLM9DizP2C2qr6oqq+kPgQcDBwGeAhwBvB65KclqS38wSvooluSvJnc3yriR3AjvNej5qO5cMeyx2TJIkSVJXLNtsOVX1s6r6eFU9HdgJOBa4FngucBbwgyRvTrLtIpr9O+BUYNuq2rCqNgL+e9ZzSZIkTYBlOd2wTqbCrKpVwDnAN7j34ucHMQj4r0zyziQbjtDOYcDfA6cn+Z2Z1UsYz+7DHottS5IkSeqKZQ3uk2yX5E1JrmBQnnMQcAFwGLAr8EcMavOPAt4xSptV9WXg14GHJvksYMZekiRpwpLleWhxxn6H2iQrgGcALwee1vRxO3AK8P6q+tqs3d+W5EQGF8X+FvC6UfqoqjuBP0rycODxYxy+JEmSNLXGGtwn+TPgUGA7BqU33wb+Fjilqm6a75iq+t8kXwJ+e7H9VdVFwEVLH7EkSZLGwfr4bhh35v5NwB3APzHI0n9pxOM+zxJq5yVJktQNTlvZDeMO7o8GTq6q/1nMQVX1QeCDYx6LJEmStF4Zd3D/eAbTXh4+5nYlSZLUYZbldMO4Z8t5GrDVmNuUJEmSWkmyMsmxSS5LsjrJNUlOTrL9ItvZP8mfJDkryfVJKsmlCxzzwWa/YY9XtTu7e407c38lsOmY25QkSVLHdTlvn2QlcDawH4Obqp7BoNrkUODAJI+rqitGbO4E4BFLHMpngOvmWf/dJbb3C8Yd3P8T8IYk21bVfAOXJEmS1rU3MgjszwWeWlW3ACQ5Cng3cDKw/4ht/TvwL8DXgZ8A5y1iHG+vqs8vYv9FG3dwfxywD/CFJH8IfLKq7hpzH5IkSeqYFR2tuU+yIXBE8/LVM4E9QFUdn+R3gCcleVRVfXOh9qrq6Flt7zTu8bY17uD+uwzq+B8EnAZUkh8Dq+fZt6pq5zH3L0mSpAnoaGwP8ARgC+CKqjp/nu2nAXsCzwQWDO67btzB/U5zXgfYdsx9SJIkSaOaqY8fVj5z3pz9ltNBSQ4GNmBwreqZVbXWi3EXa6zBfVWNe/YdSZIkTYHlmgozySXDtlXV7iM0sUOzXDVk+6o5+y2nI+a8/oskfw0cWVVrxtGBwbgkSZL6bLNmeduQ7bfO2W85nA+8CtgF2AR4CPBq4KfAYcA7x9XRuMtyJEmStB5arpr7EbPzazMzslpg+7KpqhPmrLoSOCnJFxnU+R+R5PiqurptX8uSuU/ytCSnJ/lhkjuSfGDOtuOTPHA5+pYkSZJmublZDrsX0ybN8pYh25dNVV0MfIJBDf5TxtHm2DP3SU4CXsngW9DNwIb8/DeinwKvZVDfdPy4+5ckSdK619WpMIGrmuWwO9FuP2e/de3yZrndOBoba+Y+ycsY1BP9F/DIqrrf3H2q6lzghwymG5IkSZKW0wXNcu8h22fWX7gOxjKfLZvlWH45GHfm/pXA/wAHVtUNa9nvewwuJJAkSVIPdDdxzznATcDOSfaaZ677Q5rlJ9ftsCDJxsAzmpdjmWN/3MH97sB/LhDYA1wH7Dvmvsfm6btvM+khaIqs9LJ0jWjTjbr7/3zqHj9bNG2WayrMtqrqziQnAm8CTkzy1Kq6FSDJUQxuYPXlqvr6zDFJDgcOBz5eVce06T/JrsBuwCer6mez1m8N/C2Dm79eAHylTT8zxv3RcTejlfo8kHunHZIkSZKW01sZXLC6H3B5ki8BOwL7ADcAh87ZfytgV+apg0/ycuDlzcuNm+WOSb46a7fDqmrm5ljbAacDNyS5lEF5+jbAo4DNGVyH+vyqGjabz6KMO7i/FHh0kk2qat65RJM8AHgkPbi9ryRJkga6fPOkqlqd5ADgGOBFwHOAG4FTgDcvcgrK7Rl8KZht5Zx19531/DLgvQyqVnYGHgvc0aw/Ezihqm5cRP9rlTF9SRg0lhwJvIfBP9QrqmpNkruBD1bVyzL4veYfgRcCL6+qfxhb5+NTq8dyfzD12eyfy32/aCEz75fvX796sgNR5z1k65X3PPezRQtpPls6UwtzxMe/M76gcpb3PfdXO3OO02DcmfuTgIOB3wGekOQzzfo9k7wLOJDBnbn+g8EXAEmSJPVAV2vu1zdj/QWlqu4CfhP4G2AHBrfThcEUQ0cx+CniA8Azq+rucfYtSZIkre/Gfi1+U2t/WJI/AfYHdmJw161VDGbSuWbcfUqSJGmyVpi474Rlm2irqq4HTluu9iVJktQdBvfd0OULmyVJkiQtwlgz90n+eBG7V1X92Tj7lyRJ0mR4QW03jLss5y1AMXxappkpktI8N7iXJEmSxmTcwf3cu3vNWMHg1rq/ATwO+CvgG2PuW5IkSRNizX03jDW4r6qF5q7/0yTHAG8C/nacfUuSJEnru3V+QW1VHcdgWsy3reu+JUmStDyS5XlocZZtKswFXAQ8ZUJ9S5IkacxWGIl3wqSmwtyZyX2xkCRJknppnQbYSbYA3gw8EvjPddm3JEmSlo83T+qGcc9z//21bN4MeACDaTBvB44ZZ9+SJEnS+m7cmfud1rLtLuBq4AvAX1TVt8fctyRJkibEkvtuGPdUmP4iI0mSJE3IstTcJ1kB3B/YALixqu5cjn4kSZLUDc6W0w1jy7Qn2T7J25J8C7gD+BFwDXBrkkuTvDvJzuPqT5IkSdLPG0twn+Rw4LvAHwB7MsjYp3lsAOwCvA64OMkr5xy7MskTxzEOSZIkTYY3seqG1mU5Sf6Ae+82+1HgVOCbwPUMgvutgUcBLwYOBk5K8oCqeluS+wNnAp8BvjRCX7tV1aXN8xXA64F9gQuBt1fVHSOO+ZJh26pqlCYkSZI0ywoD8U5olblPsjvwVuBG4MlV9cKq+kRV/bCq7qyqO6pqVVWdUVXPBw4AbgL+OMkzgS8zCM5vH7HLU2c9PxZ4PHAysCNwQptzkSRJkqZd28z9axh8QXhJVS2Yea+qLyZ5MXAWcDqDzP6pwPEj9jf7O+GBwBOq6tYknwHOH3XQVbX72jaP2o4kSZIGvKC2G9rW3P8acGlVfXrUA6rqU8B3mpfvrqqXVNXPRjz8l5Jsl+SXgbur6tamzTXAqG1IkiRJvdQ2c/9ABjXzi3URsFtV/f4ij9scOIcmg5/kgVV1TZLNMeMuSZI0MSbuu6FtcH8nsPESjtsYuHmxB1XVTkM2/Qw4aAnjkCRJknqjbXB/JfC4JCuq6u5RDmhmuXkc8P2Wfd+jqm5rxiJJkqQJcLacbmhbc38Wg6kuX7+IY97QHPPJln1LkiSpI7JM/9PitA3u3wv8L3BckmOSbDBsxyQbJHkjcByD6TCdulKSJEkao1ZlOVV1Q5LnMcjCvxX4vSQfZXATqx8zuPB1GwY3sToE+GVgDfDCqrqhTd+SJEnqDstyuqH1HWqr6nNJngicAuwGvHae3Wb+3N8FXlpVX2vbryRJkqSf1zq4B6iqrwMPS/I04OnAI4EHMAjqfwJcAHwK+LeqcspKSZKknjFz3w1jCe5nNDeo+tQ425QkSZI0mrEG95IkSVo/xbtYdYLBvSRJklqzLKcb2k6FKUmSJKkjzNxLkiSpNatyusHMvSRJktQTZu4lSZLU2gpT951g5l6SJEnqCTP3kiRJas3ZcrrBzL0kSZLUE2buJUmS1Jol991gcD+Plf6raBF8v2hUD9l65aSHoCniZ4umzQqM7rvAshxJkiSpJ8wLzGP1mkmPQF03O6Pm+0ULmXm/fPV7P53oONR9+/7KFvc897NFC+narzuW5XSDmXtJkiSpJzr2nU+SJEnTyKkwu8HMvSRJktQTZu4lSZLU2gqL7jvB4F6SJEmtGdt3g2U5kiRJUk+YuZckSVJrluV0g5l7SZIkqSfM3EuSJKk1E/fdYOZekiRJ6gkz95IkSWrNjHE3GNxLkiSptViX0wl+yZIkSZJ6wsy9JEmSWjNv3w1m7iVJkqSeMHMvSZKk1ryJVTeYuZckSVLvJVmZ5NgklyVZneSaJCcn2X6R7eyf5E+SnJXk+iSV5NIRjluR5LVJLkpye3PsR5M8bOln9YvM3EuSJKm1Luftk6wEzgb2A64FzgB2Ag4FDkzyuKq6YsTmTgAescj+A/wzcAjwU+AsYCvgYOAZSQ6oqq8tps1hzNxLkiSp797IILA/F9ilql5QVfsArwe2Bk5eRFv/DrwJeCqw94jHHMogsL8c2K2qDqmqJwPPA+4DfDjJWJLuUxXcJ3lFkq2a5zsmOTvJ/yT5UpJdJz0+SZKk9VWyPI/248qGwBHNy1dX1S0z26rqeOBC4ElJHjVKe1V1dFW9rao+C9w44jBe3yyPrqofzWrrX4FPADsDzx6xrbWaquAeeF1V/aR5/l7gVOCXgXcB7x+1kSSXDHuMf8iSJEn9l2RZHmPwBGAL4IqqOn+e7ac1y2eOo7O5kjwYeBhwO4NynGXtf9qC+w1mPd+hqj5QVbdX1RkM/miSJEnSbDP18ecN2X7enP2Wq/+Lq+qu5e5/2i6o/WqS1wPvAf4ryVOr6t+bn1FuG7WRqtp9bZvbDlKSJGl9s1wZ47VVViwQ083YoVmuGrJ91Zz9xm2d9j9tmfvDgF2Aq4B9gE8nuZ5BWc7vTnJgkiRJ6qTNmuWwRPCtc/ab6v6nKnNfVbcCr0yyGfAQYEPg6qr68WRHJkmStH4bU338LxgxO782MwMbVp2x3LN4LtT/WE1VcD+jucr5wkmPQ5IkSZ13c7PcdMj2TZrlLUO2L3f/M+vH0v9UBveSJEnqlg7fxOqqZjnsTrTbz9lvqvs3uJckSVJry1WWMwYXNMthN5yaWb9cVSEz/e+RZMN5ZswZa//TdkGtJEmStBjnADcBOyfZa57thzTLTy5H51V1JfAdBneifcZy929wL0mSpNZWLNOjraq6EzixeXlikntq35McBewJfLmqvj5r/eFJLk1y3BiGAHB8s3xHkm1m9XMQ8CzgSuD0cXRkWY4kSZL67q3AU4D9gMuTfAnYkcHU6jcAh87ZfytgV2C7uQ0leTnw8ublxs1yxyRfnbXbYVU1+6ZZJwNPB54LXJrk7KaP/YHVwEuG3OBq0QzuJUmS1FqHa+6pqtVJDgCOAV4EPAe4ETgFeHNVXb2I5rZn8KVgtpVz1t13Tv93J3kecCTwMuBABvPbfxz446oaeqOuxUqVN2Sdo1avmfQQ1HUrZ30t9v2ihcy8X776vZ9OdBzqvn1/ZYt7nvvZooU0ny2diag/fuF1yxJUPnfPbTtzjtPAzL0kSZJaMwLvBoN7SZIktdbhqpz1irPlSJIkST1h5l6SJEmtrbAwpxPM3EuSJEk9YeZekiRJrVlz3w1m7iVJkqSeMHMvSZKk1mLNfSeYuZckSZJ6wsz9PFb6r6JF8P2iUc2++6i0ED9bNG2sue8GPzokSZLUmlNhdoPB/TxWr5n0CNR1szNqvl+0kJn3i+8VLWT2Z8srT7tkcgPRVHj/IbtPegjqIIN7SZIktWZZTjd4Qa0kSZLUE2buJUmS1JqZ+24wcy9JkiT1hJl7SZIkteZNrLrB4F6SJEmtrTC27wTLciRJkqSeMHMvSZKk1izL6QYz95IkSVJPmLmXJElSa06F2Q1m7iVJkqSeMHMvSZKk1qy57waDe0mSJLXmVJjdYFmOJEmS1BNm7iVJktSaZTndYOZekiRJ6gkz95IkSWrNqTC7wcy9JEmS1BNm7iVJktSaiftumKrMfZJ/TfKCJCsnPRZJkiSpa6Ytc78fcF/gr5OcDvxjVf3HYhtJcsmwbVW19NFJkiStp1ZYdN8JU5W5B66rqv8L7AF8Gzg+ydVJ3p5kjwmPTZIkab2VZXpocaYtc18AVXUN8C7gXU1Q/9vAWcCOIzVStftCfUiSJEnTZtoy97/wBa6qLq6qPwB2WvfDkSRJEmDqviOmLbh/+rANZbG8JEmS1nNTVZZTVddOegySJEn6RTHN3gnTlrmXJEmSNMRUZe4lSZLUTc6E2Q0G95IkSWrN2L4bLMuRJEmSesLMvSRJktozdd8JZu4lSZKknjBzL0mSpNacCrMbzNxLkiRJPWHmXpIkSa05FWY3GNxLkiSpNWP7brAsR5IkSeoJM/eSJElqz9R9J5i5lyRJknrCzL0kSZJacyrMbjBzL0mSJPWEmXtJkiS15lSY3ZCqmvQYusZ/EEmSNC06E1J/66qblyWGeuQOm3fmHKeBmXtJkiS1ZgTeDQb3kiRJas/ovhMM7uexes2kR6CuWznrvxzfL1rIzPvF94oWMvuz5bVnXDq5gWgqvPfZu016COogg3tJkiS15lSY3eBUmJIkSVJPmLmXJElSa06F2Q1m7iVJktR7SVYmOTbJZUlWJ7kmyclJtl9CW1skeW+S/05yR7M8IckWQ/b/YJJay+NVrU+wYeZekiRJrXU5cZ9kJXA2sB9wLXAGsBNwKHBgksdV1RUjtvUA4FzgocD3gdOB3YHXAE9Psm9V3TDk8M8A182z/rsjn8wCDO4lSZLUXpeje3gjg8D+XOCpVXULQJKjgHcDJwP7j9jWexgE9h8DXlBVa5q2/hI4Ajge+J0hx769qj6/xHMYiWU5kiRJ6q0kGzIIugFePRPYA1TV8cCFwJOSPGqEtrYFXgzcBRw2E9g3fh+4Hnhxkv8zrvEvlsG9JEmSWssy/W8MngBsAVxRVefPs/20ZvnMEdp6GoP4+YtV9aPZG6rqDuBMYINmv4mwLEeSJEl99ohmed6Q7efN2a9tWy9bS1sHJTmYwReAK4Ezq2qsd6wzuJckSVJryzUVZpJLhm2rqt1HaGKHZrlqyPZVc/ZbzraOmPP6L5L8NXDknBKfJbMsR5IkSX22WbO8bcj2W+fstxxtnQ+8CtgF2AR4CPBq4KfAYcA7R+h7JGbuJUmS1NpyTZYzYnZ+bWaGVgtsX7a2quqEOauuBE5K8kXgm8ARSY6vqqsXMZZ5mbmXJElSe1mmR3s3N8tNh2zfpFneMmT7crVFVV0MfIJBDf5TRjlmIQb3kiRJ6rOrmuWwO9FuP2e/ddXWjMub5XaLOGYoy3IkSZLU2pimrVwOFzTLvYdsn1l/4Tpua8aWzXKkbP9CzNxLkiSpz84BbgJ2TrLXPNsPaZafHKGtTwN3A09Mss3sDUk2ZjBX/t3Ap0YZWHPMM5qX3xzlmIUY3EuSJKm1ZHkebVXVncCJzcsTk9xTL5/kKGBP4MtV9fVZ6w9PcmmS4+a0dS3wT8BGDC6InV0F8w5ga+DUqrpuVlu7Jnl2kg1+/t8rWwMfAR7E4BeBr7Q/W8tyJEmS1H9vZXDB6n7A5Um+BOwI7APcABw6Z/+tgF2Zvw7+tcC+wMHApUm+AewO7AFcAbxuzv7bAacDNyS5FPghsA3wKGBzBnPjP7+qhs3Asyhm7iVJktRadyfLgapaDRwA/BmDOeqfA+wEnALsVVXfW0RbPwEeA7yPQQb/ucD9GPw68Nhm+2yXAe9lcOHszs3+j25eHwvsWVWXLe3MflHG9CVhnUiyK4M/yjUM/jE+BDwZ+Dbwu1V10YjtrO1OZw9bPZb7g6nPVs76zcv3ixYy837xvaKFzP5see0ZY70jvXrovc/eDZZvevlFu+y625YlqNxl2006c47TYNoy9x8APsvg54tzgNMYXGH8VuCkCY5LkiRp/dbl1P16ZNpq7jetqr8DSHJEVX2wWf+JJMeO2sgCdzqbnp8yJEmSOqLDU2GuV6Ytc0+SXZI8BtgkycOaddsBG052ZJIkSdJkTVvm/hgGZTl3AwcB70qyOYOrmX9/kgOTJElan41j2kq1N1XBfVV9msG0RQAkOQfYC/jh7PlEJUmSpPXRVAX3c1XV3Yzpbl6SJElaOhP33TB1NfeSJEmS5jfVmXtJkiR1hKn7TjC4lyRJUmtOhdkNluVIkiRJPWHmXpIkSa05FWY3mLmXJEmSesLMvSRJklozcd8NZu4lSZKknjBzL0mSpPZM3XeCwb0kSZJacyrMbrAsR5IkSeoJM/eSJElqzakwu8HMvSRJktQTZu4lSZLUmon7bjBzL0mSJPWEmXtJkiS1Zs19N6SqJj2GrvEfRJIkTYvOhNSrbrxjWWKo7bfcuDPnOA3M3EuSJGkMjMG7wOB+HqvXTHoE6rqVs/7L8f2ihcy8X3yvaCGzP1vus9fhkxuIpsLt55846SH8HMtyusELaiVJkqSeMHMvSZKk1kzcd4OZe0mSJKknzNxLkiSpNWvuu8HMvSRJktQTZu4lSZLUWqy67wSDe0mSJLVnbN8JluVIkiRJPWHmXpIkSa2ZuO8GM/eSJElST5i5lyRJUmtOhdkNZu4lSZKknjBzL0mSpNacCrMbDO4lSZLUnrF9J1iWI0mSJPWEmXtJkiS1ZuK+G8zcS5IkST1h5l6SJEmtORVmN5i5lyRJknpiqjL3SQK8BHg+8CBgDfBd4H1V9dVFtHPJsG1V1XaYkiRJ6x2nwuyGacvc/xWwN3AScB5wJvBR4D1JfmeSA5MkSZImbaoy98ATq+rhAEn+HfhCVR2b5D+Ac4FTRmmkqnZf2+b2w5QkSVq/WHPfDdOWuSfJZs3TrYENAarqfyc3IkmSJKkbpi1z/1fAeUm+BjweOAYgyTbAjyY5MEmSJGnSpiq4r6q/SfJF4GHAn1bV5c36HwO/NtHBSZIkrccsy+mGqQruAarq28C3Jz0OSZIkqWumLriXJElS9zgVZjdM3QW1kiRJkuZn5l6SJEmtWXPfDQb3kiRJas3Yvhssy5EkSZJ6wsy9JEmS2jN13wlm7iVJkqSeMHMvSZKk1pwKsxvM3EuSJEk9YeZekiRJrTkVZjeYuZckSVLvJVmZ5NgklyVZneSaJCcn2X4JbW2R5L1J/jvJHc3yhCRbrOWYFUlem+SiJLcnuT7JR5M8rNWJzWFwL0mSpNayTI+xjC1ZCZwN/DGwGXAGcDVwKHBekp0X0dYDgP8CjgTWAKcDNwOvAb7ebJ97TIB/Bt4DbA+cBVwCHAx8I8k+Sz23uQzuJUmS1F6Xo3t4I7AfcC6wS1W9oKr2AV4PbA2cvIi23gM8FPgYsGvT1h7A+4BfAY6f55hDgUOAy4HdquqQqnoy8DzgPsCHk4ylXN7gXpIkSb2VZEPgiOblq6vqlpltVXU8cCHwpCSPGqGtbYEXA3cBh1XVmlmbfx+4Hnhxkv8z59DXN8ujq+pHs/r/V+ATwM7Asxd1YkMY3EuSJKm1LNP/xuAJwBbAFVV1/jzbT2uWzxyhracxiJ+/ODtIB6iqO4AzgQ2a/QBI8mDgYcDtDMpx2vS/IIN7SZIk9dkjmuV5Q7afN2e/cbc18/ziqrqrZf8LcirMeaz0X0WL4PtFo/K9osW4/fwTJz0EaVGWayrMJJcM21ZVu4/QxA7NctWQ7avm7DfutsbZ/4L8v5pf5Cytc8z8RzXif0Baj/le0WL4ftGofK9Mh5W/tGwx1NDgfkSbNcvbhmy/dc5+425rnP0vyOBekiRJnTWGL3UzXzpqge3L1dZCx4yVNfeSJEnqs5ub5aZDtm/SLG8Zsr1tWwsdM7N+lP4XZHAvSZKkPruqWQ67E+32c/Ybd1vj7H9BBveSJEnqswua5d5Dts+sv3CZ2po5Zo9mzv02/S/I4F6SJEl9dg5wE7Bzkr3m2X5Is/zkCG19GrgbeGKSbWZvSLIxg7nq7wY+NbO+qq4EvsPgTrTPaNn/ggzuJUmS1FtVdScwM7fsiUnuqX1PchSwJ/Dlqvr6rPWHJ7k0yXFz2roW+CdgI+CkJLMnp3kHsDVwalVdN2cYx8/sM/tLQZKDgGcBVwKnL/0s75WqdXLhriRJkjQRSVYCnwf2Aa4FvgTs2Ly+Adi3qr43a/+3AH8CnFJVL53T1lbAV4GdgSuAbwC7A3s0r/etqp/MOWYFgzvRPhe4ETgb2ArYH7gD+PWq+so4ztXMvSRJknqtqlYDBwB/xmC++ecAOwGnAHvNDuxHaOsnwGOA9zHI4D8XuB+DXwceOzewb465G3ge8HrgGuBA4OHAx4FHjyuwBzP3kiRJUm+YuZckSZJ6wuBekiRJ6gmDe0mSJKknDO4lSZKknjC4lyRJknrC4F6SJEnqCYN7rVWSA5J8O8n3kpw8505s0j2SvC/JqiRrJj0WdVuSByU5O8l3klyc5K2THpO6Lclnk1yQ5MIkpyW576THJHWVwb2GSrIB8AHgeVX1K8BmwG9PdlTqsH8GHjXpQWgqrAH+oKp+FdgbeGKSZ014TOq2Q6rqEVW1J3AV8LpJD0jqKoN7rc1jgFVVdUnz+gPAQRMcjzqsqr5cVT+a9DjUfVV1bVV9o3l+J3Ahg9vAS/OqqpsAkqwANpnwcKROM7jX2mwPXD3r9VXNOkkaiyQPYHAb+M9NeCjquCSfAH4E/Crw7gkPR+osg3stpGY9z8RGIal3kmwMnAYcX1XfmfR41G1V9SxgW+CrwGETHo7UWQb3WpurgR1mvd4eWDWhsUjqkeaang8D36iq90x6PJoOVfUz4GTg/5v0WKSuMrjX2nwD+OUkD2tevwz42ATHI6k//hb4X+DoSQ9E3Zbkvkm2m7XqYODiSY1H6jqDew3VZEheAZyW5HvAbcCHJjsqdVWS9ydZBWzQTIn5/kmPSd2U5PEMkgWPAc5P8q0kr5nwsNRd9wM+keSiJBcCjwSOnOyQpO5KVS28lyRJkqTOM3MvSZIk9YTBvSRJktQTBveSJElSTxjcS5IkST1hcC9JkiT1hMG9JEmS1BMG95IkSVJPGNxLkiRJPWFwL0mSJPWEwb2kzkuyaZLXJfnPJD9KcmeSG5Ocm+RPk+ww6THOluTJSSrJB+esf0uz/qVt25IkaT6/NOkBSNLaJNkX+BiwHXAb8FXgR8D9gMcA+wJHJzmwqj43sYFOgSQ/AHasqkx6LJKk5WFwL6mzkuwJ/AdwH+AvgD+rqltnbV8BPAd4B7D9JMa4SCcCHwGunfRAJEn9ZHAvqZOSBPhHBoH9W6rq2Ln7VNXdwMeSnA08aB0PcdGq6ifATyY9DklSf1lzL6mrfgN4OLAK+PO17VhVN1XVxQBJtktydJIvJPlhU59/XZKPJXnMfMcn+UGSap6/PMmFSW5vjnt/ki2GHLdTkn9KckOSW5J8Jckzho1zbTX3S2hr5POcqdsHdmxe16zHD+bsu1GSI5N8PcnNSW5N8l9Jfrf5wiVJ6jAz95K6aiaw/WhVrVnEcc9mUMLzPeAi4H+BXwGeCxzY1Ob/+3wHJnkHcCTwdeDTwH7A/w/8apL9q6pm7bsz8BVgG+Ay4DzgwcCZwN8sYrxLbWsx53kdcApwCLBp83zGPb8kJNkU+BTwxGb9l4G7gccBf8/gGodXLebcJEnrlsG9pK7aq1met8jjzgEeUVUXzl6Z5DeATwAnJXno7EB9lpcA+1TVt5pjtgLOZRDsHsCg/n/GSQyC8ZOAI5oSIZK8HPi7RY55KW2NfJ5VdSnw0iRPBjatqpcOafOdDM71Q8BhVXVL0+bWDL5ovDLJmVV11iLPT5K0jliWI6mrHtAsr1/MQVV10dyAt1n/GeCjwM7AHkMOf/NMYN8c8xPgr5uXT5pZ32TanwrcCBw9E4w3x/w9gyz8SJbaVsvznG8c2wAvB64EXjET2DdtXg+8snn5ynkOlyR1hJl7SV215PruJBsDvwk8Ftga2KjZ9PBm+VAGpSxzzVeuc1mz3G7Wusc3y3+bPXvPLB9hUNIziiW31eI857M/sCHw6aq6Y+7Gqrogyc0MSnMkSR1lcC+pq34C7MogaB1ZkoczKEvZaS27bT5k/ap51s1ksDeete6BzfKqIe0MWz+fJbXV8jznM9PO7yX5vbXsd59FtClJWscM7iV11bcYZLX3ZjAl5oKa2Vz+hUGg+jfN4/vALVVVSd4GHMOQXwWG1OHP29XMISPuP9a22p7nEBs0y/OBXyj3kSRNB4N7SV11FvBq4HlJjh5xxpzdmsc3qmq+7PNDxjS2a5rljkO277DMbS3Hec78avH5qjpqCcdLkjrAC2olddWngUsY3Hn2TWvbMcl9k+wObNms+oXymiRbAv93TGM7p1k+vZk+cq4XLnNbSz3PO5t95kvs/CfwMwbTaG4wz3ZJ0hQwuJfUSU2JzEuA1cBbkhw3N/jNwLOAbzC40PN7DOZl/7UkD52130oGpSv3H9PYvgeczSDIfnuSez5LkxzK6BfTLrWtpZ7nzK8Eu84zjh8CH2RwEe6HmmlAf06S/ZI8fYTTkiRNiMG9pM5qpqV8CvAj4A+BHyf5XJIPJ/kkcC1wBvAg4Oqq+jHwAeC+wAVJPpnko8APgF9jELyOy+8xmKbzcODbSU5Ncm7T/6JuYrXYtlqc5yea5dnN3XD/PsnbZ21/DYMM/m8B30/yxSQfSfL5JKsY/Mrw1EWemyRpHTK4l9RpVXUOgzuvvoHBnWP3BJ7P4GLbHwDHAg+tqrObQ34PeD2D+dp/ncFNmT4HPBr47zGO63JgXwYXtm7D4I6xAZ4D/PM6aGsp5/mXwFsZzAB0MPC7zCr7qarbGATvL2dw87A9GNzxdmfgCuBo4F2LOTdJ0rqV0SeHkCRJktRlZu4lSZKknjC4lyRJknrC4F6SJEnqCYN7SZIkqScM7iVJkqSeMLiXJEmSesLgXpIkSeoJg3tJkiSpJwzuJUmSpJ4wuJckSZJ6wuBekiRJ6gmDe0mSJKknDO4lSZKknjC4lyRJknrC4F6SJEnqCYN7SZIkqScM7iVJkqSe+H/37j2I/P3uiQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 900x600 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Print the sentences and plot the optimal transport plan for the pair of abstracts.\n",
    "print('\\n'.join([f'{i}: {s}' for i, s in enumerate(ex_abstracts[0]['ABSTRACT'])]))\n",
    "print('')\n",
    "print('\\n'.join([f'{i}: {s}' for i, s in enumerate(ex_abstracts[1]['ABSTRACT'])]))\n",
    "h = sns.heatmap(transport_plan, linewidths=.7, cmap='Blues')\n",
    "h.set(xlabel='Candidate', ylabel='Query')\n",
    "h.tick_params(labelsize=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "b3e1149e",
   "metadata": {},
   "outputs": [],
   "source": [
    "ex_abstracts = [\n",
    "    {'TITLE': \"End-to-end learning for music audio\",\n",
    "     'ABSTRACT': [\"Content-based music information retrieval tasks have traditionally been solved using engineered features and shallow processing architectures.\",\n",
    "                  \"In recent years, there has been increasing interest in using feature learning and deep architectures instead, thus reducing the required engineering effort and the need for prior knowledge.\",\n",
    "                  \"However, this new approach typically still relies on mid-level representations of music audio, e.g. spectrograms, instead of raw audio signals.\",\n",
    "                  \"In this paper, we investigate whether it is possible to apply feature learning directly to raw audio signals.\",\n",
    "                  \"We train convolutional neural networks using both approaches and compare their performance on an automatic tagging task.\",\n",
    "                  \"Although they do not outperform a spectrogram-based approach, the networks are able to autonomously discover frequency decompositions from raw audio, as well as phase-and translation-invariant feature representations.\"]},\n",
    "    {'TITLE': \"Speech acoustic modeling from raw multichannel waveforms\",\n",
    "     'ABSTRACT': [\"Standard deep neural network-based acoustic models for automatic speech recognition (ASR) rely on hand-engineered input features, typically log-mel filterbank magnitudes.\",\n",
    "                  \"In this paper, we describe a convolutional neural network - deep neural network (CNN-DNN) acoustic model which takes raw multichannel waveforms as input, i.e. without any preceding feature extraction, and learns a similar feature representation through supervised training.\",\n",
    "                  \"By operating directly in the time domain, the network is able to take advantage of the signal's fine time structure that is discarded when computing filterbank magnitude features.\",\n",
    "                  \"This structure is especially useful when analyzing multichannel inputs, where timing differences between input channels can be used to localize a signal in space.\",\n",
    "                  \"The first convolutional layer of the proposed model naturally learns a filterbank that is selective in both frequency and direction of arrival, i.e. a bank of bandpass beamformers with an auditory-like frequency scale.\",\n",
    "                  \"When trained on data corrupted with noise coming from different spatial locations, the network learns to filter them out by steering nulls in the directions corresponding to the noise sources.\",\n",
    "                  \"Experiments on a simulated multichannel dataset show that the proposed acoustic model outperforms a DNN that uses log-mel filterbank magnitude features under noisy and reverberant conditions.\"]}\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "ffcfdc67",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pre-process the data.\n",
    "bert_batch, abs_lens, sent_token_idxs = prepare_abstracts(batch_abs=ex_abstracts,\n",
    "                                                          pt_lm_tokenizer=aspire_tok)\n",
    "\n",
    "# Get sentence embeddings for the papers.\n",
    "clsreps, contextual_sent_reps = aspire_mv_model.forward(bert_batch=bert_batch,\n",
    "                                                        abs_lens=abs_lens,\n",
    "                                                        sent_tok_idxs=sent_token_idxs)\n",
    "\n",
    "# Package the sentence embeds for use with the distance function\n",
    "query_embeds = contextual_sent_reps[0,:,:].unsqueeze(0)\n",
    "cand_embeds = contextual_sent_reps[1,:,:].unsqueeze(0)\n",
    "qt = rep_len_tup(embed=query_embeds.permute(0, 2, 1), abs_lens=[abs_lens[0]])\n",
    "ct = rep_len_tup(embed=cand_embeds.permute(0, 2, 1), abs_lens=[abs_lens[1]])\n",
    "\n",
    "# Compute the wasserstein distance and alignment between abstracts\n",
    "wd, intermediate_items = ot_distance.compute_distance(query=qt, cand=ct, return_pair_sims=True)\n",
    "\n",
    "# Get the transport plan\n",
    "transport_plan = intermediate_items[3].data.numpy()[0,:abs_lens[0],:abs_lens[1]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "7bebda63",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0: Content-based music information retrieval tasks have traditionally been solved using engineered features and shallow processing architectures.\n",
      "1: In recent years, there has been increasing interest in using feature learning and deep architectures instead, thus reducing the required engineering effort and the need for prior knowledge.\n",
      "2: However, this new approach typically still relies on mid-level representations of music audio, e.g. spectrograms, instead of raw audio signals.\n",
      "3: In this paper, we investigate whether it is possible to apply feature learning directly to raw audio signals.\n",
      "4: We train convolutional neural networks using both approaches and compare their performance on an automatic tagging task.\n",
      "5: Although they do not outperform a spectrogram-based approach, the networks are able to autonomously discover frequency decompositions from raw audio, as well as phase-and translation-invariant feature representations.\n",
      "\n",
      "0: Standard deep neural network-based acoustic models for automatic speech recognition (ASR) rely on hand-engineered input features, typically log-mel filterbank magnitudes.\n",
      "1: In this paper, we describe a convolutional neural network - deep neural network (CNN-DNN) acoustic model which takes raw multichannel waveforms as input, i.e. without any preceding feature extraction, and learns a similar feature representation through supervised training.\n",
      "2: By operating directly in the time domain, the network is able to take advantage of the signal's fine time structure that is discarded when computing filterbank magnitude features.\n",
      "3: This structure is especially useful when analyzing multichannel inputs, where timing differences between input channels can be used to localize a signal in space.\n",
      "4: The first convolutional layer of the proposed model naturally learns a filterbank that is selective in both frequency and direction of arrival, i.e. a bank of bandpass beamformers with an auditory-like frequency scale.\n",
      "5: When trained on data corrupted with noise coming from different spatial locations, the network learns to filter them out by steering nulls in the directions corresponding to the noise sources.\n",
      "6: Experiments on a simulated multichannel dataset show that the proposed acoustic model outperforms a DNN that uses log-mel filterbank magnitude features under noisy and reverberant conditions.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvcAAAIXCAYAAAAYO3E5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAABcSAAAXEgFnn9JSAAA76ElEQVR4nO3de7wkdXnn8c93EBgEFRWM4AgoCiqIghoUUdQ1xhuKl6yb6K4hIXEFvGLwtiJEV4xGlEiMucjqZqNmJUaiBjVBExVxA4KiKAEJCiOIqIiADBd59o+qA+3x9Jw+p7qnq3s+73n1q7rrV/WrX/U5M/P000/9KlWFJEmSpNm3ZtoDkCRJkjQeBveSJEnSnDC4lyRJkuaEwb0kSZI0JwzuJUmSpDlhcC9JkiTNCYN7SZIkaU4Y3EuSJElzwuBekiRJmhMG95IkSdKcMLiXJEmS5oTBvSRJkjQnDO4lSZKkOXGHaQ+gh2raA5AkSRpRpj2ABdvse+REYqgbzj2pN+c4Cwzul7DH0Z+a9hBmzoVve/JtzzfcMsWBzKi1A38Tff9WbuH9871bnYX374abpzuOWbTNlrc/9/dv5fzd62bw909aYHAvSZKk7mK1dx/4U5AkSZLmhJl7SZIkdRdL4/vAzL0kSZI0J8zcS5IkqTtr7nvBn4IkSZI0J8zcS5IkqTtr7nvB4F6SJEndWZbTC/4UJEmSpDlh5l6SJEndWZbTC2buJUmSpDlh5l6SJEndWXPfC/4UJEmSpDlh5l6SJEndWXPfC2buJUmSpDlh5l6SJEndWXPfCwb3kiRJ6s6ynF7wI5YkSZI0J8zcS5IkqTvLcnrBn4IkSZI0J8zcS5IkqTtr7nvBzL0kSZI0J8zcS5IkqTtr7nvB4F6SJEndGdz3gj8FSZIkaU6YuZckSVJ3a7ygtg/M3EuSJElzwsy9JEmSurPmvhf8KUiSJElzwsy9JEmSuvMmVr1gcC9JkqTuLMvpBX8KkiRJ0pwwcy9JkqTuLMvpBTP3kiRJ0pyYqcx9kjXAfwWeDaxrV68HPgr8dVXdOq2xSZIkbdasue+FmQrugb8A7gT8KXApEJog/3eAA4HfG6WTJOcPa6uq7qOUJElSryRZC7wW+E1gF+DHwKeAY6pq/Qr6OQh4HPCr7WMH4N+r6gEj7Htn4FU0ier7ALcAlwH/Cry6qq5bwSktadaC+8dW1R6L1n0L+KckF05jQJIkSaLXNfdtYH86cABwBXAqsBtwKPD0JI+qqotH7O5E4CGrGMMewD8D9wYuAf4R2BrYEzgcOB7Y7IL7m5M8oKouGFyZ5AHAzaN2UlV7bax5tYOTJElSL72OJrA/E3jSQoY8ySuBdwAnAweN2NdngP8LnAX8EDhnuR2SbEvzLcG9gCOAP6uBcpEke9N8k9DZrAX3RwKfTHIZzVcYBewK7MyIJTmSJEmagJ7W3CfZEnhJ+/KIwdKXqjohyQuBxyZ5WFV9Zbn+qurogb53G3EYr6Ypw3lHVb1niT6/MWI/y5qp4L6qPpfk/sAjuP2C2suAs72YVpIkaYr6W5ZzILA9cHFVnbtE+ynAPsDBwLLB/Uq1E8IcRpOUfue4+19spoJ7gDaI/3/tQ5IkSdqYhfr4YeUz5yzabtweBOwEnF9V30vy68CvAdsCFwN/V1WXjOtgMxfcS5IkqYcmVJazzCyHG7uOcsEu7XLYjDjrF203bgtjvCTJx4BnLmp/S5I/qKoTx3GwfhZHSZIkSeOxXbv82ZD26xdtN253bZdPBp4GHE1zvei9gNfQTO3+riRPGcfBzNxLkiSpuwnV3I+Ynd+YhYENmxFx0hcLbNEu7wAcX1VvH2j7oyQ70Mx9/3rgtK4HM3MvSZKkeXZtu9x2SPsd22XnOeaXOT40U24utrDukUm27nowM/eSJEnqrqdTYQKXtst1Q9rXLdpu3L4z8Py7G2nfArg7cHmXgxncS5Ikqbv+Bvdfa5f7DWlfWH/ehI5/HvBzmuD9bsCVi9rvPvC887cHvf0pSJIkSWNwBnANsHuSfZdof267/MQkDl5VPwG+0L58/BKbPK5dXlxVP+16PIN7SZIkdZdM5tFRVd0EnNS+PCnJbbX3SV5JcwOrL1bVWQPrj0xyQZLjOw+g8dZ2+T+T3GfgOLsDb2pfvnccB7IsR5IkSfPuzcATgQOAi5J8AdgV2B/4EXDoou13APakufnUL0hyGM0dZwEWLoDdNcmXBzY7vKpuu2lWVX06yTuAo4DzkpxBM0vPo2ku9D2NMd291uBekiRJ3fW35p6q2pDk8cBrgd8CDgGuBj4AvKGqLltBd+toPhQMWrto3Z2XGMOrkpwNvIQmqN8CuAB4P/Ceqvr5CsYwlMG9JEmS5l5V3QAc0z6W2/ZY4NiVto3Q74eBD69m31EZ3EuSJKm7Cd3ESitjcC9JkqTuelyWsznxpyBJkiTNCTP3kiRJ6s6ynF4wcy9JkiTNCTP3kiRJ6ixm7nvBzL0kSZI0J8zcS5IkqTMz9/1g5l6SJEmaE2buJUmS1J2J+14wuJckSVJnluX0g2U5kiRJ0pwwcy9JkqTOzNz3g5l7SZIkaU6YuZckSVJnZu77wcy9JEmSNCfM3EuSJKkzM/f9YHAvSZKk7ozte8GyHEmSJGlOmLmXJElSZ5bl9IPB/RIufNuTpz2EmbbW36pOfP9Wz/eum222nPYIZpu/f6vn7540Pv5TJEmSpM7M3PeDNfeSJEnSnDBzv4Sf3PDzaQ9h5my/zRa3Pf/hdbdMcSSzaYftbv+reMPNUxzIjFr4Sn+Dv3qrslBOctW1voErteOdbv+7e/YlP53iSGbTw+9zZwD+46oNUx7JbLrvjmunPYRfYOa+HwzuJUmS1JnBfT9YliNJkiTNCTP3kiRJ6s7EfS+YuZckSZLmhJl7SZIkdWbNfT+YuZckSZLmhJl7SZIkdWbmvh/M3EuSJElzwsy9JEmSOjNz3w8G95IkSerO2L4XLMuRJEnS3EuyNslxSS5MsiHJ5UlOTrJuhf0clOSNST6Z5KokleSCFfaxVZJvtvtuWNmZbJyZe0mSJHXW57KcJGuB04EDgCuAU4HdgEOBpyd5VFVdPGJ3JwIP6Tik1wEP6NjHkszcS5Ikad69jiawPxPYo6qeV1X7A0cBOwInr6CvzwCvB54E7LfSgSR5IPBa4C9Xuu8ozNxLkiSps75m7pNsCbykfXlEVV230FZVJyR5IfDYJA+rqq8s119VHT3Q924rHEuAvwB+ArwG+P2V7D8KM/eSJEmaZwcC2wMXV9W5S7Sf0i4P3gRjeVE7nqOq6upJHMDMvSRJkjrra+ae2+vjzxnSfs6i7SYiyU7AW4HPVtX/mdRxDO4lSZLU2aSC+yTnD2urqr1G6GKXdrl+SPv6RdtNyknAWuDFkzyIZTmSJEmaZ9u1y58Nab9+0XZjl+SZwLOBt1bVhZM6Dpi5lyRJ0jhMqCpnxOz8xiyMrJZpn4gkd6LJ2l8EHD/JY4HBvSRJkubbte1y2yHtd2yX1w1p7+otwDrgiVV144SOcRuDe0mSJHXW4wtqL22Xw+5Eu27RduN2MLABeEOSNyzRvlWSf2mfH1ZV3+5yMIN7SZIkzbOvtcthN5xaWH/eBMewFjhoSFsG2jrX/RvcS5IkqbMeZ+7PAK4Bdk+y7xJz3T+3XX5iEgevqt2GtSUp4MaqWjuu4zlbjiRJkjpLMpFHV1V1E80FrQAnJbmt9j7JK4F9gC9W1VkD649MckGSiV8AO25m7iVJkjTv3gw8ETgAuCjJF4Bdgf2BHwGHLtp+B2BPYKfFHSU5DDisfbl1u9w1yZcHNju8qobdNGuiDO4lSZLUXW+rcqCqNiR5PPBa4LeAQ4CrgQ8Ab6iqy1bQ3TqaDwWD1i5ad+fVj7Ybg3tJkiTNvaq6ATimfSy37bHAsSttW8WYxv6RyOBekiRJnfX4gtrNihfUSpIkSXPCzL0kSZI6M3PfD2buJUmSpDlh5l6SJEmdmbnvB4N7SZIkdWZw3w+W5UiSJElzYrPM3Cc5f1hbVW3KoUiSJM0HE/e9MFeZ+yR/Nu0xSJIkSdMyb5n7p42yUVXttbHmMY1FkiRps2HNfT/MXHCf5KZhTczZNxGSJEnSSsxccA9cCexXVVctbkhy2RTGI0mStNkzc98Psxjc/z1wf+CXgnvgk5t4LJIkSQKM7fth5oL7qnrpRtr++6YciyRJktQnMxfcS5IkqX8sy+kHL0CVJEmS5oSZe0mSJHVm4r4fzNxLkiRJc8LMvSRJkjqz5r4fDO4lSZLUmbF9P1iWI0mSJM0JM/eSJEnqbM0aU/d9YOZekiRJmhNm7iVJktSZNff9YOZekiRJmhNm7iVJktSZU2H2g5l7SZIkaU6YuZckSVJnJu77weBekiRJnVmW0w+W5UiSJElzwsy9JEmSOjNz3w9m7iVJkqQ5YeZekiRJnZm47wcz95IkSdKcMLiXJElSZ0km8hjj+NYmOS7JhUk2JLk8yclJ1q2wn4OSvDHJJ5NclaSSXLCR7bdM8qQkJyX5SpIfJ7khybeS/HGSHbuf3e0sy5EkSVJnfS7LSbIWOB04ALgCOBXYDTgUeHqSR1XVxSN2dyLwkBUc/iDg0+3zi4HPAVsCjwKOAp6f5HFV9e8r6HMoM/eSJEmad6+jCezPBPaoqudV1f40wfWOwMkr6OszwOuBJwH7jbD9rcCHgP2q6n5V9ZyqegZwP5qg/57A/1rB8TfKzL0kSZI66+tUmEm2BF7Svjyiqq5baKuqE5K8EHhskodV1VeW66+qjh7oe7cRtv8s8Nkl1l+T5FDgcuBRSXatqu8ue0LLMHMvSZKkeXYgsD1wcVWdu0T7Ke3y4E02olZVXQFc1b7ceRx9mrmXJElSZz1N3MPt9fHnDGk/Z9F2m0yS7YG7ti+/P44+De4lSZLUW0nOH9ZWVXuN0MUu7XL9kPb1i7bblI6gice/XlWXjKNDg3tJkiR11teae2C7dvmzIe3XL9puk0iyL/A/2pevHle/BveSJEnqbFKx/YjZ+Y1ZGFkt077JJLkn8FFgLfCuqjptXH17Qa0kSZLm2bXtctsh7Xdsl9cNaR+rJHcBTqOZZ/8jNNNxjo2Ze0mSJHXW47KcS9vlsDvRrlu03cQk2Qb4OPBQmvnyX1BVt47zGAb3S9h+my2mPYSZtsN2/lp1sc2W0x7B7Frrr14nO97JN7CLh9/nztMewsy6745rpz0EzbevtcthN5xaWH/eJAeR5A40mfrHAF8Cnl1VN437OJblSJIkqbNkMo8xOAO4Bti9vYh1see2y0+M5WhLSPO1xvuBpwFfBZ5WVddvbJ/VMk2zhH2P+6WbiGkZ577xCbc933DLFAcyowYzzh87byzT3G5WDtnnngB86/KJ/Ds59x64c1OG+sPr/Mu7UoPfVPpv38ot/Nt31bW+eavht22jqaqbkpwEvB44KcmTFgLrJK8E9gG+WFVnLeyT5EjgSODvq+q1YxjGicDzgQuAJ1XVT8bQ55L8rZAkSVJnPa65B3gz8ETgAOCiJF8AdgX2B34EHLpo+x2APYGdFneU5DDgsPbl1u1y1yRfHtjs8Ko6p93+mcBL2vWXAW8f8l69taouWOF5/RKDe0mSJM21qtqQ5PHAa4HfAg4BrgY+ALyhqi5bQXfraD4UDFq7aN3gRTh3HXj+axvp9/00mf1ODO4lSZLUWb8T91BVNwDHtI/ltj0WOHalbUO2fz9N4L5JGNxLkiSps56X5Ww2nC1HkiRJmhNm7iVJktSZift+MHMvSZIkzQkz95IkSerMmvt+MHMvSZIkzQkz95IkSerMzH0/GNxLkiSpM2P7frAsR5IkSZoTZu4lSZLUmWU5/WDmXpIkSZoTZu4lSZLUmYn7fjBzL0mSJM0JM/eSJEnqzJr7fjC4lyRJUmfG9v1gWY4kSZI0J8zcS5IkqbM1pu57wcy9JEmSNCfM3EuSJKkzE/f9YOZekiRJmhNm7iVJktSZU2H2g5l7SZIkaU6YuZckSVJna0zc94LBvSRJkjqzLKcfLMuRJEmS5oSZe0mSJHVm4r4fzNxLkiRJc8LMvSRJkjoLpu77wMy9JEmSNCfM3EuSJKkzp8LsB4N7SZIkdeZUmP0wc2U5SZ6e5Igkey5af+S0xiRJkiT1wUxl7pP8IfBY4Dzg1Un+uKr+pG3+HeCkEfs5f1hbVXUepyRJ0ubGxH0/zFRwDxwC7FdVtyQ5DvhIkp2r6jXgJdqSJEnavM1aWc4WVXULQFX9CHgysHuSv2IF51JVew17TGjckiRJc21NMpHHuCRZm+S4JBcm2ZDk8iQnJ1m3wn4OSvLGJJ9MclWSSnLBCPutSfLyJF9PckO770eSPGj1Z/XLZi24vyLJfgsvquom4Hk0Wfu9pzYqSZIk9VaStcDpwDHAdsCpwGXAocA5SXZfQXcnAscCTwV2GPH4Af4WeCewDvgkcD7wHODsJPuv4PgbNWvB/W8Dlw+uqKpbq+p3gcdMZUSSJEkimcxjTF4HHACcCexRVc+rqv2Bo4AdgZNX0NdngNcDTwL2W2bbBYcCzwUuAh5QVc+tqscBvwFsA/xNkrGUy89UzX1Vrd9I25c25VgkSZJ0u75OhZlkS+Al7csjquq6hbaqOiHJC4HHJnlYVX1luf6q6uiBvncbcRhHtcujq+rKgb7+Lsk/AM8Angn83Yj9DTVrmXtJkiRpJQ4Etgcurqpzl2g/pV0ePImDJ7kP8CDgBppynIkef6Yy95IkSeqnnibuAR7SLs8Z0n7Oou0mdfxvVNXNkz6+wb0kSZJ6a5n7E40y0+Eu7XJYeff6RduN2yY9vsG9JEmSOhvntJVjtl27/NmQ9usXbTfTxze4lyRJUm+N4T5EC586apn2SVnu+GM11gtqkxwwzv4kSZI0GzKhxxhc2y63HdJ+x3Z53ZD2SR9/Yf1Yjj/u2XK+mOQbSV6W5G5j7luSJElaqUvb5bA70a5btN1MH3/cwf2/Ag8ETgC+l+RvkjxuzMeQJElSzySZyGMMvtYuh91wamH9eeM42EaOv3c75/5Ejz/W4L6qHg/sAbwduBr4TeD0JBcmOTrJr4zzeJIkSeqHNZnMYwzOAK4Bdk+y7xLtz22XnxjL0RapqkuAb9HcifZpkz7+2G9iVVUXV9VrgHsDzwE+DdwXeCtwaZJTkjw5fb2NmSRJkuZGVd0EnNS+PCnJbbXvSV4J7AN8sarOGlh/ZJILkhw/pmGc0C7fluQeA8d5Ns3daS8BPjaOA01stpyq+jnw98DfJ1kH/C5wKPCs9rE+yV8Bf1lV35/UOCRJkjR5Pc/bvhl4InAAcFGSLwC7AvsDP6KJUQftAOwJ7LS4oySHAYe1L7dul7sm+fLAZodX1eBNs04GnkoTA1+Q5PT2GAcBG4AXDLnB1YqNPXO/lKpaT/OVyNncfvHzvYHjgEuSvH1IDZIkSZLUSVVtAB4PvIlmvvlDgN2ADwD7VtW3V9DdOpoPBfsDD23XrR1Ytz9w50XHvxX4DeAo4HLg6cCDaRLhD6+qL638rJY20Xnuk+wE/E772I0mqP8a8OfA6TQneSTwynYsr5jkeCRJkjQZ/U7cQ1XdABzTPpbb9ljg2JW2LdPnz2nKc05Ybtsuxp65T7ImycFJTgW+S/MJ6Z40n4weVVX7VtV7q+qiqnoL8ADgOzQX30qSJElapbFm7pO8iaZmaSeaLP03gb8APlBV1yy1T1X9tK17+q/jHIskSZI2nZ7X3G82xl2W83rgRuBDwJ9X1RdG3O9f2ES35JUkSdL4jWnaSnU07uD+aODkqvrxSnaqqvcD7x/zWCRJkqTNyriD+0fTXDh75Jj7lSRJUo9ZltMP476g9ik0c3ZKkiRJ2sTGnbm/BNh22a0kSZI0V8zb98O4M/cfAg5Kcs8x9ytJkiRpGeMO7o8HvgD8a5JneddZSZKkzcOaZCIPrcy4y3L+neYDw72BU4BK8gNgwxLbVlXtPubjS5IkaQqMw/th3MH9boteh+butJIkSZImbKzBfVWNu8xHkiRJM8CpMPvBYFySJEmaE+Muy5EkSdJmyMR9P0wkc5/kKUk+luR7SW5M8r5FbSck2XkSx5YkSZI2V2PP3Cd5D/AimotprwW25Bfva/AT4OXAeuCEcR9fkiRJm57TVvbDWDP3SX4H+O/AvwEPraq7LN6mqs4EvgccPM5jS5IkSZu7cWfuXwT8GHh6Vf1oI9t9G7jvmI8tSZKkKTFx3w/jDu73Aj63TGAP8H3gkWM+9tic+8YnTHsIM22tl2l3csg+3hpitR6487bTHsJM22E7//J24b99q7fjnXzz5oFTYfbDuC+ovXXEPncGrh/zsSVJkqTN2rg/Kl8APDzJHavqZ0ttkOTuwEOBr4z52JIkSZoSb57UD+MO7v8GeCfwp0l+r6puGWxM833NnwDbAX895mOPzYZblt9Gv2jw6+ht9j1yegOZUTece9Jtz7/xveumOJLZtPe9tgPghpunPJAZtc2WzdL3b+UW3jvw/47VWPi/w/dudSwF01LG/WvxHuA5wAuBA5N8ul2/T5I/Bp4O7AF8FvjAmI8tSZKkKbHmvh/G+g1KVd0MPBl4L7ALcHjbtB/wSmB34H3AwVV16ziPLUmSJG3uxv6FTltrf3iSNwIHAbsBW9DctOpzVXX5uI8pSZKk6Vpj4r4XJlatVVVXAadMqn9JkiT1h8F9P3hhsyRJkjQnxpq5T3LMCjavqnrTOI8vSZKk6fCC2n4Yd1nOsUABw3661S7TPje4lyRJksZk3MH9oUPWrwHuDfw68CjgT4Gzx3xsSZIkTYk19/0w1uC+qpabu/4Pk7wWeD3wF+M8tiRJkrS52+QX1FbV8TTTYr5lUx9bkiRJk5FM5qGVmdaNi78OPHFKx5YkSdKYrTES74VpTYW5O9P7YCFJkiTNpU0a3CfZPsk7gIcC/7Ypjy1JkqTJWTOhx7gkWZvkuCQXJtmQ5PIkJydZt4q+tk/yriTfTXJjuzwxyfYb2eeBSf53ksuS3Jzkp0m+lOT3k4ztVMc9z/1/bKR5O+DuNNNg3gC8dpzHliRJkpaSZC1wOnAAcAVwKrAbzUyPT0/yqKq6eMS+7g6cCdwf+A/gY8BewEuBpyZ5ZFX9aNE+BwKfAbYBzm/3vxvwGJqZJJ8A/JdOJ9kad2nMbhtpuxm4DPhX4I+q6ptjPrYkSZKmpOcl96+jCezPBJ5UVdcBJHkl8A7gZOCgEft6J01g/1HgeVV1S9vXnwAvAU4AXrhonz+hCeyPrqq3L6xMcn/gS8Dzkvx5VX1udad3u7GW5VTVmo08tq6q3arqhQb2kiRJ2hSSbEkTdAMcsRDYA1TVCcB5wGOTPGyEvu4JPJ8maX34QmDf+gPgKuD5SX5lYJ/tgH2Bn9F8kLhNVV0E/E378hErPLUlTaTmPsmaJDsk+ZUkW03iGJIkSeqPNclEHmNwILA9cHFVnbtE+ynt8uAR+noKTfz8+aq6crChqm4EPg5s0W634GbgVqCW6fvHIxx/WWML7pOsS/KWJF8FbgSuBC4Hrk9yQZJ3JNl9XMeTJEmSRvCQdnnOkPZzFm031r7aoP8LwLbAUYMbt2U5zweuoand72wsNfdJjgT+CFhLc8HsoC2APdrH4UleXlV/PrDvWuARVfWFcYxFkiRJm96kau6TnD+srar2GqGLXdrl+iHt6xdtN4m+Xgz8E/C2JC+kuaj2bsBjgW8Dh1bVD0c4/rI6Z+6TvBo4kSaw/whwCHDv9vU2NCf3LJqvPLYC3pPkde2+d6O5cvnxXcchSZKk6VmTyTzGYLt2+bMh7dcv2m7sfVXVt2jKg86lmVnnP9Pc0DU0Qf8lIxx7JJ0y90n2At4MXA08a0j2fX37ODXJY2m+cjgmyddpsv17MqavISRJkjRfRszOb8zCR4RhNe8r+Qixqr6SPAH4O5qZI58AnE0zRfzv0szk85R2Os7Odfddy3JeSpP9f8EoZTVV9fkkzwc+SRPQB/ggzZRBkiRJmlFjuvh1Eq5tl9sOab9ju7xuSHunvpLclaa65Q7AU6rqewN9vSHJXWhm83kVTaDfSdeynCcAF1TVp0bdoapOA77VvnxHVb2gqn7ecRySJEnSUi5tl8PuRLtu0Xbj7uvpNPX1Xx4I7Af933b5uBGOv6yuwf3OwNdXsd/XAarqDzoeX5IkST2QTOYxBl9rl/sNaV9Yf96E+loI+H86ZJ+F9Xcb4fjL6hrc3wRsvYr9tub2rzUkSZKkSTmDZqrJ3ZPsu0T7c9vlJ0bo61M0c9Y/Jsk9BhuSbE0zV/6twGkDTd9vl/sm2WKJPhduXvWdEY6/rK7B/SXAo5KM3E+77aOA/+h4bEmSJPVEX2fLqaqbgJPalyclua1ePskrgX2AL1bVWQPrj2zv03T8or6uAD7E7TNADl6/+jZgR+CDVfX9gfWforkH1H2ANw3GzUn2BP6wfXkKY9A1uP8kzUkctdyGA17V7jPKpyNJkiTNgEzoz5i8Gfh/wAHARUn+NsmXgXcAPwIOXbT9DjQzOu60RF8vBy4GngNckOTD7SyQL23Xv2Jw4/YDwatoZth5bXv8U5J8jqbMZ2fgH4H3dz/N7sH9u2jqhI5P8tohXzUAkGSLdn7742m+Gjmx47ElSZKkZVXVBpr7Kr2JZo76Q4DdgA8A+1bVt1fQ1w9pSmneTZPBfxZwF5pvB351qZtRVdVJNBPRfIxmRp1n0tTnnwscATyjqm5Z1cktkqph03SO2EHyRJos/JbA92im+vkK8AOaqS7vATyMpp7pXsAtwMFV9ZlOB56c2jCWt3bzsnbgS6lt9j1yegOZUTece9Jtz7/xvVFm4tKgve/V3CvkhpunPJAZtc2WzdL3b+UW3jsA/+9YuYX/O3zvVqd9/3oz/+RbP3txt6ByiNc8YffenOMs6DrPPVX1z0keQ/PJ5wE0X1UstvBD+Xfgt6vq/3U9riRJkqRf1Dm4B2gvQHhQkqcATwUeSnPXrQA/pKknOg34x+r6VYEkSZJ6ZxwXv6q7sQT3C9obVJ227IaSJEmSxm6swb0kSZI2TxnTHafUjcG9JEmSOrMspx9mKrhPsh23zx36Dpqpg/4b8HXgZVV11Yj9nD+szUsCJEmSNKu6znO/qZ0MbE8z2f8ngbvR3EDg6zRzjUqSJGkKksk8tDIzlbkH9qyq/9zetvf7wBPa2XeOT/LVUTupqr021txxjJIkSdJUzFpwfytAVd2a5MxF02r62U6SJGlK1phm74VZK8u5KsmdAKrqmQsrk9wTuHFqo5IkSZJ6YKYy91X1pCFN1wPP3pRjkSRJ0u2cLacfZiq4H6aqrgWunfY4JEmSpGmai+BekiRJ02XJfT8Y3EuSJKmzNc5t0guzdkGtJEmSpCHM3EuSJKkzy3L6wcy9JEmSNCfM3EuSJKkzp8LsBzP3kiRJ0pwwcy9JkqTO1lh03wsG95IkSerM2L4fLMuRJEmS5oSZe0mSJHVmWU4/mLmXJEmS5oSZe0mSJHVm4r4fzNxLkiRJc8LMvSRJkjozY9wPBveSJEnqLNbl9IIfsiRJkqQ5YeZekiRJnZm37wcz95IkSdKcMHMvSZKkzryJVT+YuZckSZLmhJl7SZIkdWbevh/M3EuSJElzwuBekiRJnSWTeYxvfFmb5LgkFybZkOTyJCcnWbeKvrZP8q4k301yY7s8Mcn2y+x35yR/mOQbSa5Pck37/E+TbLfqkxtgcC9JkqTOkkzkMaaxrQVOB44BtgNOBS4DDgXOSbL7Cvq6O/BvwMuAW4CPAdcCLwXOatuX2m8P4BvAG4A7Av8I/CuwJXA4sP3Kz+yXGdxLkiRp3r0OOAA4E9ijqp5XVfsDRwE7AievoK93AvcHPgrs2fa1N/Bu4H7ACYt3SLIt8CngXsARwO5V9RtV9Yyq2hN4MPDjVZ/dAIN7SZIkdbZmQo+ukmwJvKR9eURVXbfQVlUnAOcBj03ysBH6uifwfOBm4PCqumWg+Q+Aq4DnJ/mVRbu+GrgP8M6qek9V1WBjVX2jqn62wlNbksG9JEmS5tmBNCUvF1fVuUu0n9IuDx6hr6fQxM+fr6orBxuq6kbg48AW7XYAJFkDHAYUTdZ/opwKU5IkSZ2Nqz5+Ah7SLs8Z0n7Oou269vU7i/p6ELATcH5VfS/JrwO/BmwLXAz8XVVdMsKxR2JwL0mSpN5Kcv6wtqraa4QudmmX64e0r1+03bj7WhjjJUk+Bjxz0T5vSfIHVXXiCMdflmU5kiRJ6iwTeozBwhSTw2rar1+03bj7umu7fDLwNOBoYGeai2tfQ3Oa70ryFMbAzP0S1vqudHLDuSdNewgzbe97jWWa283SNltOewSzzfevG//vWD3fu/kwqbKcEbPzG7MwsFqmfVJ9bdEu7wAcX1VvH2j7oyQ7AK8CXg+ctoKxLMnMvSRJkubZte1y2yHtd2yX1w1p79rXtQPPl5pyc2HdI5NsPcIYNsrPykvYcMvy2+gXDWZd/vH8H0xvIDPqqXvd47bn/v6t3MLvn+/d6iy8f+uvvmm6A5lB6+661W3P/f1buYXfvX/597FM773Zedyed5v2EH5BjzPGl7bLYXeiXbdou3H39Z2B599dYp+F9i2AuwOXjzCOoXr8c5AkSZI6+1q73G9I+8L68ybU13nAz9vnS30iG7yj7SjfHmyUwb0kSZI6SzKRxxicAVwD7J5k3yXan9suPzFCX58CbgUek+Qegw1tSc3BbftttfNV9RPgC+3Lxy/R5+Pa5cVV9dMRxrBRBveSJEmaW1V1E7Aw28dJSW6rl0/ySmAf4ItVddbA+iOTXJDk+EV9XQF8CNgKeE+SwRL3twE7Ah+squ8vGsZb2+X/THKfgePsDrypffne1Z7jIGvuJUmS1Flvb2HVeDPwROAA4KIkXwB2BfYHfgQcumj7HYA9aW4+tdjLgUcCzwEuSHI2zVz2e9PclOoVi3eoqk8neQdwFHBekjNo3rJH01ycexpjunutmXtJkiR1lkzmMQ5VtYGmJOZNNHPUHwLsBnwA2Leqvr2Cvn4IPAJ4N00G/1nAXWi+HfjVtn2p/V4F/CZNDf6jgccAFwIvA55RVT9far+VMnMvSZKkuVdVNwDHtI/ltj0WOHYj7VcDL20fKxnDh4EPr2SflTK4lyRJUmdr+l6Ys5mwLEeSJEmaE2buJUmS1Nm46uPVjZl7SZIkaU6YuZckSVJnsea+F8zcS5IkSXPCzL0kSZI6s+a+HwzuJUmS1JlTYfaDZTmSJEnSnDBzL0mSpM4sy+kHM/eSJEnSnDBzL0mSpM7M3PeDmXtJkiRpTpi5lyRJUmfexKofDO4lSZLU2Rpj+16wLEeSJEmaE2buJUmS1JllOf1g5l6SJEmaE2buJUmS1JlTYfaDmXtJkiRpTpi5lyRJUmfW3PeDwb0kSZI6cyrMfrAsR5IkSZoTZu4lSZLUmWU5/WDmXpIkSZoTZu4lSZLUmVNh9oOZe0mSJGlOzHzmPsldquqaaY9DkiRpc2bivh/mIXN/7rQHIEmSJPXBTGXuk9wMFLd/OCzgDgvrq2qrEfs5f1hbVXUepyRJ0uZmjUX3vTBrmfu/BD4I3LOqtmyD+e8OPJckSdIUZEIPrcxMZe6r6vAkBwIfS/JXVfUBmuz9SvvZa2PNqx6gJEmSNEWzlrmnqr4I/Cfg/kn+CTBjL0mSNG2m7nthpjL3C6rqJuB/JHkw8Ohpj0eSJEnqg5nL3A+qqq9X1XunPQ5JkqTNXSb0Z2zjS9YmOS7JhUk2JLk8yclJ1q2ir+2TvCvJd5Pc2C5PTLL9iPtvleSbSSrJhhWfzEbMdHAvSZIkLSfJWuB04BhgO+BU4DLgUOCcJLuvoK+7A/8GvAy4BfgYcC3wUuCstn05rwMesIJTGJnBvSRJkjpLJvMYk9cBBwBnAntU1fOqan/gKGBH4OQV9PVO4P7AR4E92772Bt4N3A84YWM7J3kg8FqaWSDHzuBekiRJnfX1etokWwIvaV8eUVXXLbRV1QnAecBjkzxshL7uCTwfuBk4vKpuGWj+A+Aq4PlJfmXI/gH+AvgJ8JqVn83yDO4lSZI0zw4Etgcurqpzl2g/pV0ePEJfT6GJnz9fVVcONlTVjcDHgS3a7ZbyonY8R1XV1SMcb8UM7iVJktRdX1P38JB2ec6Q9nMWbTeRvpLsBLwV+GxV/Z8RjrUqMzkVpiRJkjYPSc4f1rbMjUkX7NIu1w9pX79ou0n1dRKwFnjxCMdZNYN7SZIkdTbOaSvHbLt2+bMh7dcv2m7sfSV5JvBs4LiqunCE46yawb0kSZJ6a8Ts/MYsfOqoZdon0leSO9Fk7S8Cjl/BsVbF4F6SJEmdjXHaynG7tl1uO6T9ju3yuiHtXft6C7AOeGJ70e1EGdxLkiSps/7G9lzaLofdiXbdou3G3dfBwAbgDUnesMQ+WyX5l/b5YVX17RHGMZTBvSRJkubZ19rlfkPaF9afN8G+1gIHDdknA22j1P1vlMG9JEmSuutv6v4M4Bpg9yT7LjHX/XPb5SdG6OtTwK3AY5Lco6p+sNCQZGuaLP2twGkL66tqt2GdJSngxqpaO8qJjMJ57iVJkjS3quommgtaAU5Kclu9fJJXAvsAX6yqswbWH5nkgiTHL+rrCuBDwFbAe5IMJsrfBuwIfLCqvj+Zs1memXtJkiR11uOpMAHeDDwROAC4KMkXgF2B/YEfAYcu2n4HYE9gpyX6ejnwSOA5wAVJzgb2AvYGLgZeMYHxj8zMvSRJkuZaVW0AHg+8iWaO+kOA3YAPAPuu5CLWqvoh8Ajg3TQZ/GcBd6H5duBX2/apMXMvSZKkzno8FSYAVXUDcEz7WG7bY4FjN9J+NfDS9tFlTGN/18zcS5IkSXPCzL0kSZI663nifrNhcC9JkqTujO57wbIcSZIkaU6YuZckSVJnPZ8Kc7Nh5l6SJEmaE2buJUmS1Fnfp8LcXJi5lyRJkuaEmXtJkiR1ZuK+HwzuJUmS1J3RfS9YliNJkiTNCTP3kiRJ6sypMPvBzL0kSZI0J8zcS5IkqTOnwuwHM/eSJEnSnEhVTXsMfeMbIkmSZkVv8uXfuvz6icRQD9x5296c4yywLEeSJEndGYL3gmU5kiRJ0pwwc7+EDbdMewSzZ+3Ab5Lv38oNvn/P/MuzpzeQGXXq7z0cgHO+89Mpj2Q27bfbnQF4/1mXTnkks+e3H7HLbc/9t2/lFv7t871bnbU9i+KcCrMfzNxLkiRJc6Jnn/kkSZI0i5wKsx/M3EuSJElzwsy9JEmSOjNx3w9m7iVJkqQ5YeZekiRJ3Zm67wWDe0mSJHXmVJj9YFmOJEmSNCfM3EuSJKkzp8LsBzP3kiRJ0pwwcy9JkqTOTNz3g5l7SZIkaU6YuZckSVJ3pu57weBekiRJnTkVZj9YliNJkiTNCYN7SZIkdZZM5jG+8WVtkuOSXJhkQ5LLk5ycZN0q+to+ybuSfDfJje3yxCTbL7HtlkmelOSkJF9J8uMkNyT5VpI/TrLjWE6wZXAvSZKkuZZkLXA6cAywHXAqcBlwKHBOkt1X0NfdgX8DXgbcAnwMuBZ4KXBW2z7oIODTwBHAXYDPAf8E7AAcBZyXZM/VnttiBveSJEnqLBN6jMnrgAOAM4E9qup5VbU/TXC9I3DyCvp6J3B/4KPAnm1fewPvBu4HnLBo+1uBDwH7VdX9quo5VfWMdttPA/cE/tfqT+0XGdxLkiRpbiXZEnhJ+/KIqrpuoa2qTgDOAx6b5GEj9HVP4PnAzcDhVXXLQPMfAFcBz0/yKwPH+GxV/VZVnTvYV1VdQ/PNAcCjkuy68rP7ZQb3kiRJ6q6/qfsDge2BixcH2K1T2uXBI/T1FJr4+fNVdeVgQ1XdCHwc2KLdbllVdQXNBwKAnUfZZzkG95IkSeosE/ozBg9pl+cMaT9n0Xabqi/aC3Dv2r78/ij7LMd57iVJktRbSc4f1lZVe43QxS7tcv2Q9vWLtttUfUFzke0dgK9X1SUj7rNRBveSJEnqbJzTVo7Zdu3yZ0Par1+03SbpK8m+wP9oX756hGOPxOBekiRJvTVidn5jFj521DLtm6yv9sLcjwJrgXdV1WkrGMNGGdxLkiSps/4m7rm2XW47pP2O7fK6Ie1j7SvJXYDTgN2Aj9BMxzk2XlArSZKkeXZpuxx2J9p1i7abWF9JtqGZUeehwGeAF1TVrSMcd2Rm7iVJktRZj2vuv9Yu9xvSvrD+vEn2leQONJn6xwBfAp5dVTeNcMwVMXMvSZKkeXYGcA2we3sR62LPbZefGKGvT9HccfYxSe4x2JBka5q58m+lKbsZbAvwfuBpwFeBp1XV9UyAwb0kSZLGoJ93sWqz4ye1L09Kclu9fJJXAvsAX6yqswbWH5nkgiTHL+rrCuBDwFbAe9ps/IK3ATsCH6yqxXPWn0hzZ9sLgCdV1U86n9gQluVIkiSpsx6X5QC8GXgicABwUZIvALsC+wM/Ag5dtP0OwJ7ATkv09XLgkcBzgAuSnA3sBewNXAy8YnDjJM8EXtK+vAx4e5Z+s95aVRes9MQWM7iXJEnSXKuqDUkeD7wW+C3gEOBq4APAG6rqshX09cMkjwCOa/t5FnAlzbcDb6yqHy/a5a4Dz39tI12/nyaz34nBvSRJkjrrd+IequoG4Jj2sdy2xwLHbqT9auCl7WO5vt5PE7hvEtbcS5IkSXPCzL0kSZI663nN/WZjpoL7JA9YuNAgyRqaO3o9kmYu0bdW1Y0j9nP+sLaqYXcTliRJkvpt1spyPjjw/Djg0cDJNFc7nziVEUmSJIlM6I9WZqYy9/zitRpPBw6squuTfBo4d9ROqmqvjTWvdnCSJEmbLePwXpi14P4OSXai+cbh1oU7e1XVLUl+Pt2hSZIkSdM1a8H9nWhuIRyAJDtX1eVJ7oQZd0mSpKkxcd8PMxXcV9VuQ5p+Djx7Ew5FkiRJ6p2ZCu6HqaqfAZdMexySJEmbK6fC7IdZmy1HkiRJ0hBzkbmXJEnSdDltZT8Y3EuSJKk7Y/tesCxHkiRJmhNm7iVJktSZift+MHMvSZIkzQkz95IkSerMqTD7wcy9JEmSNCfM3EuSJKkzp8LsBzP3kiRJ0pwwcy9JkqTOrLnvBzP3kiRJ0pwwuJckSZLmhGU5kiRJ6syynH4wcy9JkiTNCTP3kiRJ6sypMPvBzL0kSZI0J8zcS5IkqTNr7vvB4F6SJEmdGdv3g2U5kiRJ0pwwcy9JkqTuTN33gpl7SZIkaU6YuZckSVJnToXZD2buJUmSpDlh5l6SJEmdORVmP5i5lyRJkuaEmXtJkiR1ZuK+HwzuJUmS1J3RfS9YliNJkiTNCTP3kiRJ6sypMPvBzL0kSZI0J1JV0x5D3/iGSJKkWdGbdPmGWyYTQ629Q3/OcRYY3M+QJOcDVNVe0x7LrPG968b3rxvfv258/1bP964b3z/NIstyJEmSpDlhcC9JkiTNCYN7SZIkaU4Y3EuSJElzwuBekiRJmhMG95IkSdKccCpMSZIkaU6YuZckSZLmhMG9JEmSNCcM7iVJkqQ5YXAvSZIkzQmDe0mSJGlOGNxLkiRJc8LgXpIkSZoTBveSJEnSnDC4nxFJHp/km0m+neTkJHeY9phmRZJ3J1mf5JZpj2UWJbl3ktOTfCvJN5K8edpjmiVJ/inJ15Kcl+SUJHee9phmUZL3+Hd45ZJ8J8n5Sb7aPh407THNiiTbJfnfSS5MckGSF017TNIoDO5nQJItgPcBv1FV9wO2A/7rdEc1U/4WeNi0BzHDbgFeXVUPBPYDHpPkGVMe0yx5blU9pKr2AS4FXjHtAc2aJI8Btp32OGbYr1fVQ9vHN6c9mBlyAvCNqtoDeCDw91MejzQSg/vZ8AhgfVWd375+H/DsKY5nplTVF6vqymmPY1ZV1RVVdXb7/CbgPGDX6Y5qdlTVNQBJ1gB3nPJwZk6SrYG3Aq+a9li0+UhyJ+DpNAE+1fjBdEcljcbgfjasAy4beH1pu07apJLcHTgE+OcpD2WmJPkH4Eqa7N87pjycWXMM8L6qumraA5lhH29Lw95sSefI7kvzd/akJOckOTXJblMekzQSg/vZUQPPM7VRaLPVZlBPAU6oqm9NezyzpKqeAdwT+DJw+JSHMzOS7APsD/yvaY9lhh1YVfsCBwIPBo6a8nhmxZbAQ4FTq2o/4OPAyVMdkTQig/vZcBmwy8DrdcD6KY1Fm6H2uo+/Ac6uqndOezyzqKp+ThMc/Ldpj2WGPBp4EHBJku8AW7QXiN51usOaHVW1vl1eC/wl8MjpjmhmXAb8uKpOa19/kOaaI6n3DO5nw9nAvQZmOfgd4KNTHI82P38B/BQ4etoDmSVJ7pxkp4FVzwG+Ma3xzJqq+rOq2rmqdquq3YCft8+vnvbYZkGSbRdmZ2rLcZ5Dc82MltFep/WNJA9vV/0acP5GdpF6I1W1/FaauiRPAE4CtgI+D/x+VTkt3AiS/DnwNOBewPeAT1aVU5qNKMmjgS/SBKU/b1efXFV/Mr1RzYYk96b5IL6WprTuAuAlXuC9OkluqSprxkeU5L40v39rgC2AM4CXVdUNUx3YjGgTau+jmanpJ8CLLEnULDC4lyRJkuaEZTmSJEnSnDC4lyRJkuaEwb0kSZI0JwzuJUmSpDlhcC9JkiTNCYN7SZIkaU4Y3EuSJElzwuBekiRJmhMG95IkSdKcMLiX1HtJtk3yiiSfS3JlkpuSXJ3kzCR/mGSXaY9xUJLHJakk71+0/th2/W937UuSpKXcYdoDkKSNSfJI4KPATsDPgC8DVwJ3AR4BPBI4OsnTq+qfpzbQGZDkO8CuVZVpj0WSNBkG95J6K8k+wGeBbYA/At5UVdcPtK8BDgHeBqybxhhX6CTgw8AV0x6IJGk+GdxL6qUkAf4PTWB/bFUdt3ibqroV+GiS04F7b+IhrlhV/RD44bTHIUmaX9bcS+qrXwceDKwH/ufGNqyqa6rqGwBJdkpydJJ/TfK9tj7/+0k+muQRS+2f5DtJqn1+WJLzktzQ7vfnSbYfst9uST6U5EdJrkvypSRPGzbOjdXcr6Kvkc9zoW4f2LV9XQOP7yzadqskL0tyVpJrk1yf5N+S/G77gUuS1GNm7iX11UJg+5GqumUF+z2TpoTn28DXgZ8C9wOeBTy9rc3/zFI7Jnkb8DLgLOBTwAHA7wMPTHJQVdXAtrsDXwLuAVwInAPcB/g48N4VjHe1fa3kPL8PfAB4LrBt+3zBbd8kJNkWOA14TLv+i8CtwKOAv6K5xuG/r+TcJEmblsG9pL7at12es8L9zgAeUlXnDa5M8uvAPwDvSXL/wUB9wAuA/avqq+0+OwBn0gS7j6ep/1/wHppg/D3AS9oSIZIcBvzlCse8mr5GPs+qugD47SSPA7atqt8e0ufbac71r4HDq+q6ts8daT5ovCjJx6vqkys8P0nSJmJZjqS+unu7vGolO1XV1xcHvO36TwMfAXYH9h6y+xsWAvt2nx8Cf9a+fOzC+jbT/iTgauDohWC83eevaLLwI1ltXx3Pc6lx3AM4DLgE+L2FwL7t8yrgRe3LFy2xuySpJ8zcS+qrVdd3J9kaeDLwq8COwFZt04Pb5f1pSlkWW6pc58J2udPAuke3y38cnL1nwIdpSnpGseq+OpznUg4CtgQ+VVU3Lm6sqq8luZamNEeS1FMG95L66ofAnjRB68iSPJimLGW3jWx2pyHr1y+xbiGDvfXAup3b5aVD+hm2fimr6qvjeS5loZ8XJ3nxRrbbZgV9SpI2MYN7SX31VZqs9n40U2Iuq53N5f/SBKrvbR//AVxXVZXkLcBrGfKtwJA6/CUPtbDLiNuPta+u5znEFu3yXOCXyn0kSbPB4F5SX30SOAL4jSRHjzhjzgPax9lVtVT2+b5jGtvl7XLXIe27TLivSZznwrcW/1JVr1zF/pKkHvCCWkl99SngfJo7z75+YxsmuXOSvYC7tqt+qbwmyV2BXxvT2M5ol09tp49c7L9MuK/VnudN7TZLJXY+B/ycZhrNLZZolyTNAIN7Sb3Ulsi8ANgAHJvk+MXBbxrPAM6mudDz2zTzsj8hyf0HtltLU7pytzGN7dvA6TRB9luT3PZvaZJDGf1i2tX2tdrzXPiWYM8lxvE94P00F+H+dTsN6C9IckCSp45wWpKkKTG4l9Rb7bSUTwSuBF4D/CDJPyf5mySfAK4ATgXuDVxWVT8A3gfcGfhakk8k+QjwHeAJNMHruLyYZprOI4FvJvlgkjPb46/oJlYr7avDef5Duzy9vRvuXyV560D7S2ky+L8J/EeSzyf5cJJ/SbKe5luGJ63w3CRJm5DBvaReq6ozaO68+iqaO8fuA/xnmottvwMcB9y/qk5vd3kxcBTNfO3/ieamTP8MPBz47hjHdRHwSJoLW+9Bc8fYAIcAf7sJ+lrNef4J8GaaGYCeA/wuA2U/VfUzmuD9MJqbh+1Nc8fb3YGLgaOBP17JuUmSNq2MPjmEJEmSpD4zcy9JkiTNCYN7SZIkaU4Y3EuSJElzwuBekiRJmhMG95IkSdKcMLiXJEmS5oTBvSRJkjQnDO4lSZKkOWFwL0mSJM0Jg3tJkiRpThjcS5IkSXPC4F6SJEmaEwb3kiRJ0pwwuJckSZLmhMG9JEmSNCcM7iVJkqQ5YXAvSZIkzYn/D1+Cv2BDtqYAAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 900x600 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Print the sentences and plot the optimal transport plan for the pair of abstracts.\n",
    "print('\\n'.join([f'{i}: {s}' for i, s in enumerate(ex_abstracts[0]['ABSTRACT'])]))\n",
    "print('')\n",
    "print('\\n'.join([f'{i}: {s}' for i, s in enumerate(ex_abstracts[1]['ABSTRACT'])]))\n",
    "h = sns.heatmap(transport_plan, linewidths=.7, cmap='Blues')\n",
    "h.set(xlabel='Candidate', ylabel='Query')\n",
    "h.tick_params(labelsize=5)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}