egg.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Egg data object\n",
    "\n",
    "This tutorial will go over the basics of the `Egg` data object, the essential quail data structure that contains all the data you need to run analyses and plot the results. An egg is made up of two primary pieces of data: \n",
    "\n",
    "1. `pres` data - words/stimuli that were presented to a subject\n",
    "\n",
    "2. `rec` data - words/stimuli that were recalled by the subject.\n",
    "\n",
    "You cannot create an `egg` without both of these components.  Additionally, there are a few optional fields:\n",
    "\n",
    "1. `features` data - features that describe each of the stimuli (for example, the category of the word, word length, etc.). This field is required for fingerprint analysis (see the fingerprint tutorial).\n",
    "\n",
    "2. `dist_funcs` dictionary - this field allows you to control the distance functions for each of the stimulus features.  For more on this, see the fingerprint tutorial.\n",
    "\n",
    "3. `meta` dictionary - this is an optional field that allows you to store custom meta data about the dataset, such as the date collected, experiment version etc.\n",
    "\n",
    "There are also a few other fields and functions to make organizing and modifying `eggs` easier (discussed at the bottom).  Now, lets dive in and create an `egg` from scratch."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load in the library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import quail"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The `pres` data structure\n",
    "\n",
    "The first piece of an `egg` is the `pres` data, or in other words the words/stimuli that were presented to the subject. For a single subject's data, the form of the input will be a list of lists, where each list is comprised of the words presented to the subject during a particular study block. Let's create a fake dataset of one subject who saw two encoding lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "presented_words = [['cat', 'bat', 'hat', 'goat'],['zoo', 'animal', 'zebra', 'horse']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The `rec` data structure\n",
    "\n",
    "The second fundamental component of an egg is the `rec` data, or the words/stimuli that were recalled by the subject. Now, let's create the recall lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "recalled_words = [['bat', 'cat', 'goat', 'hat'],['animal', 'horse', 'zoo']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now have the two components necessary to build an `egg`, so let's do that and then take a look at the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "egg = quail.Egg(pres=presented_words, rec=recalled_words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's it!  We've created our first `egg`. Let's take a closer look at how the `egg` is setup. We can use the `info` method to get a quick snapshot of the `egg`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of subjects: 1\n",
      "Number of lists per subject: 2\n",
      "Number of words per list: 4\n",
      "Date created: Wed Sep  6 10:54:34 2017\n",
      "Meta data: {}\n"
     ]
    }
   ],
   "source": [
    "egg.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's take a closer look at how the `egg` is structured.  First, we will check out the `pres` field:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Subject</th>\n",
       "      <th>List</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">0</th>\n",
       "      <th>0</th>\n",
       "      <td>cat</td>\n",
       "      <td>bat</td>\n",
       "      <td>hat</td>\n",
       "      <td>goat</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>zoo</td>\n",
       "      <td>animal</td>\n",
       "      <td>zebra</td>\n",
       "      <td>horse</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                0       1      2      3\n",
       "Subject List                           \n",
       "0       0     cat     bat    hat   goat\n",
       "        1     zoo  animal  zebra  horse"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "egg.pres"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see above, the `pres` field was turned into a multi-index Pandas DataFrame organized by subject and by list. This is how the `pres` data is stored within an egg, which will make more sense when we consider larger datasets with more subjects.  Next, let's take a look at the `rec` data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Subject</th>\n",
       "      <th>List</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">0</th>\n",
       "      <th>0</th>\n",
       "      <td>bat</td>\n",
       "      <td>cat</td>\n",
       "      <td>goat</td>\n",
       "      <td>hat</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>animal</td>\n",
       "      <td>horse</td>\n",
       "      <td>zoo</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   0      1     2     3\n",
       "Subject List                           \n",
       "0       0        bat    cat  goat   hat\n",
       "        1     animal  horse   zoo  None"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "egg.rec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `rec` data is also stored as a DataFrame.  Notice that if the number of recalled words is shorter than the number of presented words (as it typically will be), those columns are filled with a `None` value. Now, let's create an `egg` with two subject's data and take a look at the result."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multisubject `eggs`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# presented words\n",
    "sub1_presented=[['cat', 'bat', 'hat', 'goat'],['zoo', 'animal', 'zebra', 'horse']]\n",
    "sub2_presented=[['cat', 'bat', 'hat', 'goat'],['zoo', 'animal', 'zebra', 'horse']]\n",
    "\n",
    "# recalled words\n",
    "sub1_recalled=[['bat', 'cat', 'goat', 'hat'],['animal', 'horse', 'zoo']]\n",
    "sub2_recalled=[['cat', 'goat', 'bat', 'hat'],['horse', 'zebra', 'zoo', 'animal']]\n",
    "\n",
    "# combine subject data\n",
    "presented_words = [sub1_presented, sub2_presented]\n",
    "recalled_words = [sub1_recalled, sub2_recalled]\n",
    "\n",
    "# create Egg\n",
    "multisubject_egg = quail.Egg(pres=presented_words,rec=recalled_words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see above, in order to create an `egg` with more than one subject's data, all you do is create a list of subjects.  Let's see how the `pres` data is organized in the egg with more than one subject:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Subject</th>\n",
       "      <th>List</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">0</th>\n",
       "      <th>0</th>\n",
       "      <td>cat</td>\n",
       "      <td>bat</td>\n",
       "      <td>hat</td>\n",
       "      <td>goat</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>zoo</td>\n",
       "      <td>animal</td>\n",
       "      <td>zebra</td>\n",
       "      <td>horse</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">1</th>\n",
       "      <th>0</th>\n",
       "      <td>cat</td>\n",
       "      <td>bat</td>\n",
       "      <td>hat</td>\n",
       "      <td>goat</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>zoo</td>\n",
       "      <td>animal</td>\n",
       "      <td>zebra</td>\n",
       "      <td>horse</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                0       1      2      3\n",
       "Subject List                           \n",
       "0       0     cat     bat    hat   goat\n",
       "        1     zoo  animal  zebra  horse\n",
       "1       0     cat     bat    hat   goat\n",
       "        1     zoo  animal  zebra  horse"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "multisubject_egg.pres"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks identical to the single subject data, but now we have two unique subject identifiers in the `DataFrame`. The `rec` data is set up in the same way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Subject</th>\n",
       "      <th>List</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">0</th>\n",
       "      <th>0</th>\n",
       "      <td>bat</td>\n",
       "      <td>cat</td>\n",
       "      <td>goat</td>\n",
       "      <td>hat</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>animal</td>\n",
       "      <td>horse</td>\n",
       "      <td>zoo</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">1</th>\n",
       "      <th>0</th>\n",
       "      <td>cat</td>\n",
       "      <td>goat</td>\n",
       "      <td>bat</td>\n",
       "      <td>hat</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>horse</td>\n",
       "      <td>zebra</td>\n",
       "      <td>zoo</td>\n",
       "      <td>animal</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   0      1     2       3\n",
       "Subject List                             \n",
       "0       0        bat    cat  goat     hat\n",
       "        1     animal  horse   zoo    None\n",
       "1       0        cat   goat   bat     hat\n",
       "        1      horse  zebra   zoo  animal"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "multisubject_egg.rec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you add more subjects, they are simply appended to the bottom of the df with a unique subject identifier."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The `features` data structure\n",
    "\n",
    "The `features` data structure is an optional field that can be added to an egg.  It contains features of the presented words that are required for the fingerprint analysis.  The `features` data is set up like the `pres` and `rec` data, but instead of a single word, there is a dictionary of features.  For example take the word \"cat\".  A `features` dictionary for this word might look something like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cat_features = {\n",
    "    'category' : 'animal',\n",
    "    'word_length' : 3,\n",
    "    'starting_letter' : 'c',\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can include any stimulus feature you want in this dictionary, such as the position of the word on the screen, the color, or perhaps the font of the word. To create the `features` data structure in an `egg`, use the format of the `pres` structure, but replace the words with dictionaries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# presented words\n",
    "presented_words=[['cat', 'bat', 'hat', 'goat'],['zoo', 'donkey', 'zebra', 'horse']]\n",
    "\n",
    "# presentation features\n",
    "presented_words_features = [\n",
    "    [\n",
    "        {\n",
    "            'category' : 'animal',\n",
    "            'word_length' : 3,\n",
    "            'starting_letter' : 'c'\n",
    "        },\n",
    "        {\n",
    "            'category' : 'object',\n",
    "            'word_length' : 3,\n",
    "            'starting_letter' : 'b'\n",
    "        },\n",
    "        {\n",
    "            'category' : 'object',\n",
    "            'word_length' : 3,\n",
    "            'starting_letter' : 'h'\n",
    "        },\n",
    "        {\n",
    "            'category' : 'animal',\n",
    "            'word_length' : 4,\n",
    "            'starting_letter' : 'g'\n",
    "        },\n",
    "    ],\n",
    "    [\n",
    "        {\n",
    "            'category' : 'place',\n",
    "            'word_length' : 3,\n",
    "            'starting_letter' : 'z'\n",
    "        },\n",
    "        {\n",
    "            'category' : 'animal',\n",
    "            'word_length' : 6,\n",
    "            'starting_letter' : 'd'\n",
    "        },\n",
    "        {\n",
    "            'category' : 'animal',\n",
    "            'word_length' : 5,\n",
    "            'starting_letter' : 'z'\n",
    "        },\n",
    "        {\n",
    "            'category' : 'animal',\n",
    "            'word_length' : 5,\n",
    "            'starting_letter' : 'h'\n",
    "        },\n",
    "    ],\n",
    "]\n",
    "\n",
    "# recalled words\n",
    "recalled_words=[['bat', 'cat', 'goat', 'hat'],['donkey', 'horse', 'zoo']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, simply pass the features to the `Egg` class using the `features` key word argument:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Subject</th>\n",
       "      <th>List</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">0</th>\n",
       "      <th>0</th>\n",
       "      <td>{u'category': u'animal', u'starting_letter': u...</td>\n",
       "      <td>{u'category': u'object', u'starting_letter': u...</td>\n",
       "      <td>{u'category': u'object', u'starting_letter': u...</td>\n",
       "      <td>{u'category': u'animal', u'starting_letter': u...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>{u'category': u'place', u'starting_letter': u'...</td>\n",
       "      <td>{u'category': u'animal', u'starting_letter': u...</td>\n",
       "      <td>{u'category': u'animal', u'starting_letter': u...</td>\n",
       "      <td>{u'category': u'animal', u'starting_letter': u...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                              0  \\\n",
       "Subject List                                                      \n",
       "0       0     {u'category': u'animal', u'starting_letter': u...   \n",
       "        1     {u'category': u'place', u'starting_letter': u'...   \n",
       "\n",
       "                                                              1  \\\n",
       "Subject List                                                      \n",
       "0       0     {u'category': u'object', u'starting_letter': u...   \n",
       "        1     {u'category': u'animal', u'starting_letter': u...   \n",
       "\n",
       "                                                              2  \\\n",
       "Subject List                                                      \n",
       "0       0     {u'category': u'object', u'starting_letter': u...   \n",
       "        1     {u'category': u'animal', u'starting_letter': u...   \n",
       "\n",
       "                                                              3  \n",
       "Subject List                                                     \n",
       "0       0     {u'category': u'animal', u'starting_letter': u...  \n",
       "        1     {u'category': u'animal', u'starting_letter': u...  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create egg object\n",
    "egg = quail.Egg(pres=presented_words, rec=recalled_words, features=presented_words_features)\n",
    "egg.features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining custom distance functions for the stimulus feature dimensions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As described in the fingerprint tutorial, the `features` data structure is used to estimate how subjects cluster their recall responses with respect to the features of the encoded stimuli.  Briefly, these estimates are derived by computing the similarity of neighboring recall words along each feature dimension.  For example, if you recall \"dog\", and then the next word you recall is \"cat\", your clustering by category score would increase because the two recalled words are in the same category.  Similarly, if after you recall \"cat\" you recall the word \"can\", your clustering by starting letter score would increase, since both words share the first letter \"c\".  This logic can be extended to any number of feature dimensions.\n",
    "\n",
    "Similarity between the words can be computed in a number of ways.  By default, the distance function for all textual features (like category, starting letter) is binary.  In other words, if the words are in the same category (cat, dog), there similarity would be 1, whereas if they are in different categories (cat, can) their similarity would be 0.  For numerical features (such as word length), by default similarity between words is computed using Euclidean distance.  However, the point of this digression is that you can define your own distance functions by passing a `dist_func` dictionary to the `Egg` class.  This could be for all feature dimensions, or only a subset. Let's see an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dist_funcs = {\n",
    "    'word_length' : lambda x,y: (x-y)**2\n",
    "}\n",
    "\n",
    "egg = quail.Egg(pres=presented_words, rec=recalled_words, features=presented_words_features, dist_funcs=dist_funcs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the example code above, similarity between words for the word_length feature dimension will now be computed using this custom distance function, while all other feature dimensions will be set to the default."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding meta data to an `egg`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lastly, we can add meta data to the `egg`.  We added this field to help researchers keep their eggs organized by adding custom meta data to the `egg` object. The data is added to the `egg` by passing the `meta` key word argument when creating the `egg`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of subjects: 1\n",
      "Number of lists per subject: 2\n",
      "Number of words per list: 4\n",
      "Date created: Wed Sep  6 10:54:42 2017\n",
      "Meta data: {'Researcher': 'Andy Heusser', 'Study': 'Egg Tutorial'}\n"
     ]
    }
   ],
   "source": [
    "meta = {\n",
    "    'Researcher' : 'Andy Heusser',\n",
    "    'Study' : 'Egg Tutorial'\n",
    "}\n",
    "\n",
    "egg = quail.Egg(pres=presented_words, rec=recalled_words, meta=meta)\n",
    "egg.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding `listgroup` and `subjgroup` to an `egg`\n",
    "\n",
    "While the `listgroup` and `subjgroup` arguments can be used within the `analyze` function, they can also be attached directly to the `egg`, allowing you to save condition labels for easy organization and easy data sharing.\n",
    "\n",
    "To do this, simply pass one or both of the arguments when creating the `egg`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# presented words\n",
    "sub1_presented=[['cat', 'bat', 'hat', 'goat'],['zoo', 'animal', 'zebra', 'horse']]\n",
    "sub2_presented=[['cat', 'bat', 'hat', 'goat'],['zoo', 'animal', 'zebra', 'horse']]\n",
    "\n",
    "# recalled words\n",
    "sub1_recalled=[['bat', 'cat', 'goat', 'hat'],['animal', 'horse', 'zoo']]\n",
    "sub2_recalled=[['cat', 'goat', 'bat', 'hat'],['horse', 'zebra', 'zoo', 'animal']]\n",
    "\n",
    "# combine subject data\n",
    "presented_words = [sub1_presented, sub2_presented]\n",
    "recalled_words = [sub1_recalled, sub2_recalled]\n",
    "\n",
    "# create Egg\n",
    "multisubject_egg = quail.Egg(pres=presented_words,rec=recalled_words, subjgroup=['condition1', 'condition2'],\n",
    "                            listgroup=['early','late'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Saving an `egg`\n",
    "\n",
    "Once you have created your egg, you can save it for use later, or to share with colleagues. To do this, simply call the `save` method with a filepath:\n",
    "\n",
    "```\n",
    "multisubject_egg.save('myegg')\n",
    "```\n",
    "\n",
    "This will save you `egg` using the package `pickle`. The result? A pickled egg!  To load this egg later, simply call the `load_egg` function with the path of the egg:\n",
    "\n",
    "```\n",
    "egg = quail.load_egg('myegg.egg')\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stacking `eggs`\n",
    "\n",
    "We now have two separate eggs, each with a single subject's data. Let's combine them by passing a `list` of `eggs` to the `stack_eggs` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# subject 1 data\n",
    "sub1_presented=[['cat', 'bat', 'hat', 'goat'],['zoo', 'animal', 'zebra', 'horse']]\n",
    "sub1_recalled=[['bat', 'cat', 'goat', 'hat'],['animal', 'horse', 'zoo']]\n",
    "\n",
    "# create subject 2 egg\n",
    "subject1_egg = quail.Egg(pres=sub1_presented, rec=sub1_recalled)\n",
    "\n",
    "# subject 2 data\n",
    "sub2_presented=[['cat', 'bat', 'hat', 'goat'],['zoo', 'animal', 'zebra', 'horse']]\n",
    "sub2_recalled=[['cat', 'goat', 'bat', 'hat'],['horse', 'zebra', 'zoo', 'animal']]\n",
    "\n",
    "# create subject 2 egg\n",
    "subject2_egg = quail.Egg(pres=sub2_presented, rec=sub2_recalled)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Subject</th>\n",
       "      <th>List</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">0</th>\n",
       "      <th>0</th>\n",
       "      <td>cat</td>\n",
       "      <td>bat</td>\n",
       "      <td>hat</td>\n",
       "      <td>goat</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>zoo</td>\n",
       "      <td>animal</td>\n",
       "      <td>zebra</td>\n",
       "      <td>horse</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">1</th>\n",
       "      <th>0</th>\n",
       "      <td>cat</td>\n",
       "      <td>bat</td>\n",
       "      <td>hat</td>\n",
       "      <td>goat</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>zoo</td>\n",
       "      <td>animal</td>\n",
       "      <td>zebra</td>\n",
       "      <td>horse</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                0       1      2      3\n",
       "Subject List                           \n",
       "0       0     cat     bat    hat   goat\n",
       "        1     zoo  animal  zebra  horse\n",
       "1       0     cat     bat    hat   goat\n",
       "        1     zoo  animal  zebra  horse"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stacked_eggs = quail.stack_eggs([subject1_egg, subject2_egg])\n",
    "stacked_eggs.pres"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cracking `eggs`\n",
    "\n",
    "You can use the `crack_egg` function to slice out a subset of subjects or lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Subject</th>\n",
       "      <th>List</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <th>0</th>\n",
       "      <td>cat</td>\n",
       "      <td>bat</td>\n",
       "      <td>hat</td>\n",
       "      <td>goat</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                0    1    2     3\n",
       "Subject List                     \n",
       "0       0     cat  bat  hat  goat"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cracked_egg = quail.crack_egg(stacked_eggs, subjects=[1], lists=[0])\n",
    "cracked_egg.pres"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, you can use the `crack` method, which does the same thing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Subject</th>\n",
       "      <th>List</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <th>0</th>\n",
       "      <td>zoo</td>\n",
       "      <td>animal</td>\n",
       "      <td>zebra</td>\n",
       "      <td>horse</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <th>0</th>\n",
       "      <td>zoo</td>\n",
       "      <td>animal</td>\n",
       "      <td>zebra</td>\n",
       "      <td>horse</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                0       1      2      3\n",
       "Subject List                           \n",
       "0       0     zoo  animal  zebra  horse\n",
       "1       0     zoo  animal  zebra  horse"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stacked_eggs.crack(subjects=[0,1], lists=[1]).pres"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}