week-6/stormy-daniels-payments-simulation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Week 6-2: Simulation to detect campaign finance payments\n",
    "\n",
    "This notebook replicates the analysis in the post [Statistical Model Strongly Suggests the Stormy Daniels Payoff Came from the Trump Campaign](https://medium.com/@whstancil/statistical-model-strongly-suggests-the-stormy-daniels-payoff-came-from-the-trump-campaign-7c09c300cb18) and analyzes a robustness problem in the argument: the probability of finding five payments that add up to something close to $130,000 increases quickly as the number of payments to choose from increases, and the authors arbitrarily chose 10.\n",
    "\n",
    "A deeper problem is that this statistical test has all sorts of underlying assumptions about how the data relates to reality. [For example](https://twitter.com/pbump/status/972216289384157188), why would the campaign choose to report an illegal payment in their public data? However, even on the test's own terms, the sensitivity on the choice of 10 makes the results look a lot less robust.\n",
    "\n",
    "From the original post:\n",
    "\n",
    " >As is well-known by now, in late October 2016, Donald Trump’s personal attorney Michael Cohen paid adult star Stormy Daniels \\$130,000 in order to purchase her silence about an alleged affair a decade earlier. The exact set of facts around this payment remains shrouded in mystery, with Cohen maintaining that he made the payment alone, without Trump’s knowledge, and with personal funds ... However, sharp-eyed observers have noted that, in late October 2016, the Trump campaign made a series of five large payments to Trump-affiliated entities, totaling \\$129,999.72. \n",
    "\n",
    "The core of the argument is:\n",
    "\n",
    "> Ultimately, our model suggests that the probability of a set of payments coincidentally coming so close to \\$130,000 is approximately 0.1%, or one out of one thousand. In other words, about 99.9% of the time, random chance would not produce a set of payments this close to \\$130,000. Therefore, the probability that the Trump campaign payments were related to the Daniels payoff is very high.\n",
    "\n",
    "Their model calculates the probability of getting this close if the last few payments were truly random, by repeatedly simulating a set of 10 campaign payments and seeing how close you can get to \\$130,000 by adding up some set of them. You can't get very close very often. However, there's no reason to choose to look at only the last 10 payments. If you look at the last 20 payments instead, the probability of finding a subset that some to within a dollar of \\$130,000 rises to 40%!\n",
    "\n",
    "The data is \"a list of all Trump campaign payments to Trump-affiliated entities\" compiled from FEC data by DC attorney and blogger Susan Simpson, available [here](from https://twitter.com/TheViewFromLL2/status/971771806414725130).\n",
    "\n",
    "### Things to think about\n",
    "\n",
    "Of course the issue of whether there was an illegal payoff here has since been overtaken by Cohen's guilty plea, but working through this problem touches on a number of deep-ish statistical ideas.\n",
    "\n",
    "1) This type of \"the alternative is super unlikely\" statistical argument appears in many places. It's the core idea of significance testing and p-values. See [this chapter](https://towcenter.gitbooks.io/curious-journalist-s-guide-to-data/content/analysis/arguing_from_the_odds.html) of *The Curious Journalist's Guide to Data.* \n",
    "\n",
    "2) But again, how does this data relate to reality? What are the FEC rules and typical campaign practice for what is reported and when? When politicians have pulled shady stuff in the past, how did it look in the data? We desperately need domain knowledge here. For an example of what that looks like, see [Fivethirtyeight's critique of statistical tests for tennis fixing](https://fivethirtyeight.com/features/why-betting-data-alone-cant-identify-match-fixers-in-tennis/).\n",
    "\n",
    "3) What is the correct number of payments to use as a baseline for simulation? In other words, what is our universe of events that were using to calculate the probability of this particular thing happening? The authors [note](https://medium.com/@whstancil/hi-david-its-not-fixed-at-5-that-s-just-how-many-payments-made-up-the-original-discovery-9c3696b7f694) \"the set was fixed at ten because we’re trying to estimate the odds of the original discovery, which was found in a series of eight or so payments\" which has a pleasant \"let's let the data dictate the model\" kind of rationale. But the whole concept of frequentist inference is that we reason about the statistics of the process, independent of observed data, so it's not clear to me if this argument makes sense. Or that any argument about the \"right\" or \"objective\" number of payments to check with this method can really be solid. I'd prefer to see a fully Bayesian attempt, modeling the generation process for the entire observed payment stream with and without the Stormy payoff.\n",
    "\n",
    "4) At the highest level of generality, the problem of arbitrarily chosen parameters is a red flag in any analysis. Whenever you see a chooseable constant, ask how different the answer would be if a different constant was chosen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import math\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load and clean data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>disbursement_date</th>\n",
       "      <th>recipient_name</th>\n",
       "      <th>recipient_state</th>\n",
       "      <th>disbursement_amount</th>\n",
       "      <th>disbursement_description</th>\n",
       "      <th>recipient_city</th>\n",
       "      <th>disbursement_purpose_category</th>\n",
       "      <th>pdf_url</th>\n",
       "      <th>Unnamed: 8</th>\n",
       "      <th>Unnamed: 9</th>\n",
       "      <th>Unnamed: 10</th>\n",
       "      <th>Unnamed: 11</th>\n",
       "      <th>Unnamed: 12</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4/30/15 0:00</td>\n",
       "      <td>TRUMP INTERNATIONAL HOTEL AND TOWER</td>\n",
       "      <td>NY</td>\n",
       "      <td>$1,380.54</td>\n",
       "      <td>TTW - TRAVEL: LODGING</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>TRAVEL</td>\n",
       "      <td>http://docquery.fec.gov/cgi-bin/fecimg/?201507...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5/5/15 0:00</td>\n",
       "      <td>TRUMP TOWER COMMERCIAL LLC</td>\n",
       "      <td>NY</td>\n",
       "      <td>$9,583.33</td>\n",
       "      <td>TTW - RENT</td>\n",
       "      <td>HICKSVILLE</td>\n",
       "      <td>ADMINISTRATIVE</td>\n",
       "      <td>http://docquery.fec.gov/cgi-bin/fecimg/?201507...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6/16/15 0:00</td>\n",
       "      <td>THE TRUMP CORPORATION</td>\n",
       "      <td>NY</td>\n",
       "      <td>$37,993.04</td>\n",
       "      <td>TTW  - ONE-TIME REIMBURSEMENT FOR FACILITY, RE...</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>OTHER</td>\n",
       "      <td>http://docquery.fec.gov/cgi-bin/fecimg/?201507...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6/16/15 0:00</td>\n",
       "      <td>TRUMP SOHO</td>\n",
       "      <td>NY</td>\n",
       "      <td>$3,240.96</td>\n",
       "      <td>TTW - TRAVEL: LODGING</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>TRAVEL</td>\n",
       "      <td>http://docquery.fec.gov/cgi-bin/fecimg/?201507...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6/16/15 0:00</td>\n",
       "      <td>TRUMP TOWER COMMERCIAL LLC</td>\n",
       "      <td>NY</td>\n",
       "      <td>$9,583.33</td>\n",
       "      <td>TTW - RENT</td>\n",
       "      <td>HICKSVILLE</td>\n",
       "      <td>ADMINISTRATIVE</td>\n",
       "      <td>http://docquery.fec.gov/cgi-bin/fecimg/?201507...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  disbursement_date                       recipient_name recipient_state  \\\n",
       "0      4/30/15 0:00  TRUMP INTERNATIONAL HOTEL AND TOWER              NY   \n",
       "1       5/5/15 0:00           TRUMP TOWER COMMERCIAL LLC              NY   \n",
       "2      6/16/15 0:00                THE TRUMP CORPORATION              NY   \n",
       "3      6/16/15 0:00                           TRUMP SOHO              NY   \n",
       "4      6/16/15 0:00           TRUMP TOWER COMMERCIAL LLC              NY   \n",
       "\n",
       "  disbursement_amount                           disbursement_description  \\\n",
       "0          $1,380.54                               TTW - TRAVEL: LODGING   \n",
       "1          $9,583.33                                          TTW - RENT   \n",
       "2         $37,993.04   TTW  - ONE-TIME REIMBURSEMENT FOR FACILITY, RE...   \n",
       "3          $3,240.96                               TTW - TRAVEL: LODGING   \n",
       "4          $9,583.33                                          TTW - RENT   \n",
       "\n",
       "  recipient_city disbursement_purpose_category  \\\n",
       "0       NEW YORK                        TRAVEL   \n",
       "1     HICKSVILLE                ADMINISTRATIVE   \n",
       "2       NEW YORK                         OTHER   \n",
       "3       NEW YORK                        TRAVEL   \n",
       "4     HICKSVILLE                ADMINISTRATIVE   \n",
       "\n",
       "                                             pdf_url  Unnamed: 8  Unnamed: 9  \\\n",
       "0  http://docquery.fec.gov/cgi-bin/fecimg/?201507...         NaN         NaN   \n",
       "1  http://docquery.fec.gov/cgi-bin/fecimg/?201507...         NaN         NaN   \n",
       "2  http://docquery.fec.gov/cgi-bin/fecimg/?201507...         NaN         NaN   \n",
       "3  http://docquery.fec.gov/cgi-bin/fecimg/?201507...         NaN         NaN   \n",
       "4  http://docquery.fec.gov/cgi-bin/fecimg/?201507...         NaN         NaN   \n",
       "\n",
       "   Unnamed: 10  Unnamed: 11  Unnamed: 12  \n",
       "0          NaN          NaN          NaN  \n",
       "1          NaN          NaN          NaN  \n",
       "2          NaN          NaN          NaN  \n",
       "3          NaN          NaN          NaN  \n",
       "4          NaN          NaN          NaN  "
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('data/15-16-cycle-disbursements-to-trump-properties.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "127"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Following the original post above, drop all payments labelled as RENT (but not RENTAL)\n",
    "# Also, drop everything later than October 26, when Cohen was notified the funds arrived \n",
    "# (see https://twitter.com/TheViewFromLL2/status/971771806414725130)\n",
    "payments = df.disbursement_amount[\n",
    "    ~(df.disbursement_description.str.contains('RENT') & ~df.disbursement_description.str.contains('RENTAL')) & \n",
    "    (pd.to_datetime(df.disbursement_date) < '10/27/2016')]\n",
    "len(payments)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Convert the payments column to numbers (right now it's a string) by dropping dollar signs and commas \n",
    "payments = payments.str.replace('$','').str.replace(',','').astype(float)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Following the original, drop any payment below $1000\n",
    "payments = payments[payments >= 1000]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "54"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(payments)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "194    13431.88\n",
      "195    18731.90\n",
      "196    79043.94\n",
      "197     8544.00\n",
      "200    10248.00\n",
      "Name: disbursement_amount, dtype: float64\n",
      "129999.72\n"
     ]
    }
   ],
   "source": [
    "# these are the payments thst Simpson spotted\n",
    "paylist = [194, 195, 196, 197, 200]\n",
    "print(payments[paylist])\n",
    "print(payments[paylist].sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Function for finding closest matching subset of payments\n",
    "\n",
    "To repeat the analysis, we need to know how close we can get to \\$130,000 by summing any possible subset of a set of payments. So we'll write that function first. This version operates by exhaustive search of all possible subsets, so it's slow but accurate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Return subset of the numbers that match target most closely\n",
    "def closest_match(numbers, target):\n",
    "    num_numbers = len(numbers)\n",
    "    num_subsets = int(math.pow(2,num_numbers))                        \n",
    "\n",
    "    closest_miss = 1e10\n",
    "    \n",
    "    # enumerate all subsets, by counting up through the 2^n combinations\n",
    "    for which_subset in range(0, num_subsets):\n",
    "        subset = []\n",
    "        idx = 0\n",
    "        while idx<num_numbers:\n",
    "            if int(which_subset) & (1 << idx):  # if this bit is set in binary representation,\n",
    "                  subset.append(numbers[idx])   # add in the corresponding element\n",
    "            idx+=1\n",
    "\n",
    "        miss = abs(sum(subset)-target)\n",
    "        if miss < closest_miss:\n",
    "            closest_miss = miss\n",
    "            closest_subset = subset\n",
    "        \n",
    "    return closest_subset\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sure enough, this finds the payments that Simpson marked as suspicious, when we run it on the last 10 transactions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "best_of_10 = closest_match(payments.tail(10).values, 130000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[13431.879999999999, 18731.900000000001, 79043.940000000002, 8544.0, 10248.0]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "best_of_10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "129999.72"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(best_of_10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It finds an even closer match on last 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "best_of_20 = closest_match(payments.tail(20).values, 130000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[7173.6000000000004,\n",
       " 1127.03,\n",
       " 1021.4299999999999,\n",
       " 2172.4499999999998,\n",
       " 1343.0,\n",
       " 13431.879999999999,\n",
       " 79043.940000000002,\n",
       " 8544.0,\n",
       " 16142.610000000001]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "best_of_20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "129999.94"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(best_of_20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A faster search \n",
    "Searching 2^20 subsets takes several seconds, and we want to run this thousands of times to do our simulation over random sets of payments. So for performance reasons, we're going to modify this function so that it just finds the first subset within threshold of the target -- not the closest. This lets it bail very much quicker, in most cases, and it tells us all we need to know to evaluate any given set of payments: is there *any* subset that gets sufficiently close?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Determine if there is a subset of the numbers that sum to within threshold of the target\n",
    "# Returns sum within threshold if a matching subset is found, else False.\n",
    "def threshold_match(numbers, target, threshold):\n",
    "    num_numbers = len(numbers)\n",
    "    num_subsets = int(math.pow(2,num_numbers))                        \n",
    "    \n",
    "    # enumerate all subsets, by counting up through the 2^n combinations\n",
    "    for which_subset in range(0, num_subsets):\n",
    "        sum = 0\n",
    "        idx = 0\n",
    "        while idx<num_numbers:\n",
    "            if int(which_subset) & (1 << idx):  # if this bit is set in binary representation,\n",
    "                  sum += numbers[idx]           # add in the corresponding element\n",
    "            idx+=1\n",
    "\n",
    "        # if we find a close sum, we are done early\n",
    "        if abs(sum-target) <= threshold:\n",
    "            return sum\n",
    "        \n",
    "    return False\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This should find the identified payments again"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "129999.72"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "threshold_match(payments.tail(10).values, 130000, 0.28)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And just to prove it's working, there isn't any set of payments within 10 that comes closer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "threshold_match(payments.tail(10).values, 130000, 0.27)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The main probabilistic analysis\n",
    "\n",
    "Now we're going to replicate the analysis in the post and look at transaction sets of different sizes. They analyzed 10 and 15 within the post, but we'll also look at 20 -- and see that getting close to \\$130,000 is dramatically more likely than when we only look at 15 items."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# subset_size: how large a group of random campaign finance payments are we looking through?\n",
    "# dollar_threshold: how close to $130,000 do we need to get to count it as a hit?\n",
    "# num_samaples: increase this for more accurate estimation of the probability\n",
    "def probability_of_closeness(subset_size, dollar_threshold, num_samples):\n",
    "    # how many sets of payments that get within dollar_threshold have we found?\n",
    "    hits = 0\n",
    "\n",
    "    # generate num_samples different sets of payments by sampling from all payments\n",
    "    for i in range(0, num_samples):\n",
    "        # pick subset_size numbers randomly from payments\n",
    "        subset = payments.sample(subset_size)\n",
    "\n",
    "        # Is there a close match to $130,000?\n",
    "        close_sum = threshold_match(subset.values, 130000, dollar_threshold)\n",
    "\n",
    "        # Count \"hits\", printing out the close sums we find\n",
    "        if close_sum:\n",
    "            print(f'{i}: {close_sum}')\n",
    "            hits += 1\n",
    "\n",
    "    # print the fraction of hits\n",
    "    return hits/num_samples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Armed with this function, we'll start by repeating the calculation with 10 payments, as the post does. We'll calculate the probability of getting within \\$1 of \\$130,000, taking 1000 samples for accuracy (as only 2 or 3 will get this close.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "495: 129999.54000000001\n",
      "642: 130000.54999999999\n"
     ]
    }
   ],
   "source": [
    "p10 = probability_of_closeness(10, 1, 1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.002"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've replicated the results. Probability of getting within \\$1 is 0.003, which agrees with the chart in the post (showing 0.001 for \\$0.24 and 0.005 for \\$1.44)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "13: 130000.61000000002\n",
      "32: 130000.45999999999\n",
      "37: 130000.55\n",
      "41: 129999.93\n",
      "54: 130000.97000000002\n",
      "56: 129999.59\n",
      "61: 129999.16\n",
      "65: 129999.97\n",
      "66: 130000.62000000001\n",
      "67: 130000.14\n",
      "68: 129999.56\n",
      "70: 130000.48\n",
      "73: 129999.18000000001\n",
      "76: 130000.65000000001\n",
      "79: 129999.24000000002\n",
      "89: 130000.14\n",
      "100: 130000.83\n",
      "114: 129999.18000000002\n",
      "131: 129999.87\n",
      "150: 129999.48999999999\n",
      "162: 130000.58\n",
      "169: 130000.28\n",
      "183: 130000.6\n",
      "184: 129999.38\n",
      "194: 129999.24\n",
      "216: 130000.94\n",
      "217: 129999.91\n",
      "219: 130000.81000000001\n",
      "220: 129999.11000000002\n",
      "234: 129999.36\n",
      "242: 129999.22\n",
      "249: 130000.91\n",
      "262: 129999.55000000002\n",
      "264: 129999.72\n",
      "273: 130000.38\n",
      "277: 130000.62\n",
      "282: 129999.99\n",
      "286: 129999.37000000001\n",
      "291: 129999.91\n",
      "297: 130000.0\n",
      "301: 130000.1\n",
      "310: 129999.31999999999\n",
      "311: 130000.86\n",
      "314: 130000.24\n",
      "330: 129999.98\n",
      "335: 129999.01\n",
      "336: 129999.9\n",
      "344: 130000.81999999999\n",
      "349: 129999.37\n",
      "356: 129999.67000000001\n",
      "360: 130000.40000000001\n",
      "367: 130000.48000000001\n",
      "377: 129999.52000000002\n",
      "383: 129999.31000000001\n",
      "393: 130000.83000000002\n",
      "416: 129999.82\n",
      "421: 130000.37\n",
      "426: 130000.73000000001\n",
      "429: 129999.71999999999\n",
      "431: 130000.13999999998\n",
      "435: 130000.67000000001\n",
      "464: 129999.97\n",
      "465: 129999.54000000001\n",
      "476: 129999.13\n",
      "479: 130000.17\n",
      "483: 130000.44\n",
      "484: 130001.0\n",
      "504: 129999.54\n",
      "525: 130000.12\n",
      "528: 129999.37000000001\n",
      "532: 130000.8\n",
      "534: 130000.68000000001\n",
      "544: 129999.77\n",
      "558: 129999.50000000003\n",
      "561: 130000.1\n",
      "565: 129999.88\n",
      "570: 129999.44000000002\n",
      "584: 129999.90000000002\n",
      "596: 130000.55\n",
      "609: 130000.02\n",
      "623: 129999.51000000001\n",
      "639: 129999.24000000002\n",
      "645: 130000.25\n",
      "646: 129999.19\n",
      "650: 130000.41\n",
      "653: 130000.93\n",
      "665: 130000.70999999999\n",
      "670: 129999.68000000001\n",
      "705: 129999.70999999999\n",
      "708: 130000.99\n",
      "712: 130000.67000000001\n",
      "730: 130000.56999999999\n",
      "732: 129999.11\n",
      "737: 129999.29000000001\n",
      "748: 129999.71\n",
      "751: 129999.82\n",
      "759: 129999.16\n",
      "760: 130000.67000000001\n",
      "761: 130000.0\n",
      "766: 130000.06999999999\n",
      "787: 130000.19\n",
      "795: 129999.75000000001\n",
      "807: 130000.09000000001\n",
      "814: 130000.33000000002\n",
      "819: 129999.39\n",
      "823: 129999.25000000001\n",
      "826: 130000.98000000001\n",
      "827: 129999.57\n",
      "847: 129999.78999999998\n",
      "861: 130000.40000000001\n",
      "869: 129999.73999999999\n",
      "873: 129999.39\n",
      "889: 130000.47\n",
      "896: 130000.11\n",
      "897: 129999.0\n",
      "899: 129999.78\n",
      "901: 130000.98000000001\n",
      "903: 129999.48000000001\n",
      "904: 130000.14\n",
      "909: 130000.16\n",
      "924: 129999.25\n",
      "929: 129999.01\n",
      "939: 129999.71000000002\n",
      "950: 129999.49\n",
      "952: 129999.26999999999\n",
      "960: 130000.22\n",
      "969: 130000.41999999998\n",
      "977: 130000.45999999999\n",
      "984: 129999.35\n",
      "997: 129999.73\n",
      "999: 130000.22\n"
     ]
    }
   ],
   "source": [
    "p15 = probability_of_closeness(15, 1, 1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.131"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p15"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now for the main event: how likely is it that we can get within \\$1 of \\$130,000 if we take 20 random payments? It turns out, pretty likely. \n",
    "\n",
    "1000 samples of size 20 takes about an hour to run, so we'll use only 100 this time. It's no problem here as hits are very common."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0: 129999.59000000001\n",
      "3: 130000.86\n",
      "5: 130000.45\n",
      "6: 130000.74\n",
      "7: 129999.71999999999\n",
      "9: 129999.0\n",
      "10: 129999.62\n",
      "11: 130000.85\n",
      "12: 129999.86\n",
      "13: 130000.9\n",
      "14: 130000.98000000001\n",
      "16: 130000.67000000001\n",
      "18: 129999.99\n",
      "19: 130000.97999999998\n",
      "20: 129999.36\n",
      "21: 130000.17\n",
      "22: 130000.43\n",
      "24: 130000.33999999998\n",
      "25: 129999.58000000002\n",
      "26: 129999.06000000001\n",
      "27: 129999.66999999998\n",
      "28: 129999.96\n",
      "29: 130000.18\n",
      "30: 129999.95999999998\n",
      "32: 130000.39\n",
      "33: 130000.19\n",
      "34: 130000.04000000001\n",
      "35: 129999.82999999999\n",
      "36: 129999.28\n",
      "37: 130000.86000000002\n",
      "39: 129999.82\n",
      "40: 129999.76\n",
      "41: 129999.10999999999\n",
      "42: 130000.68\n",
      "43: 130000.94\n",
      "44: 130000.71\n",
      "45: 130000.51999999999\n",
      "46: 130000.70000000001\n",
      "47: 130000.78\n",
      "49: 129999.07\n",
      "50: 130000.1\n",
      "51: 130000.71\n",
      "53: 129999.05999999998\n",
      "54: 129999.04\n",
      "56: 129999.52\n",
      "57: 129999.61000000002\n",
      "58: 130000.72\n",
      "59: 130000.29\n",
      "60: 130000.99\n",
      "61: 129999.75\n",
      "62: 130000.99\n",
      "63: 130000.05\n",
      "64: 130000.16999999998\n",
      "65: 130001.0\n",
      "66: 129999.97\n",
      "67: 129999.75\n",
      "68: 129999.98\n",
      "70: 129999.99\n",
      "71: 129999.45\n",
      "72: 129999.06\n",
      "74: 130000.79000000001\n",
      "75: 129999.90000000001\n",
      "76: 129999.6\n",
      "77: 129999.67000000001\n",
      "78: 130000.83000000002\n",
      "79: 130000.01999999999\n",
      "80: 129999.20000000001\n",
      "81: 130000.14\n",
      "82: 130000.81999999999\n",
      "83: 130000.90000000001\n",
      "86: 129999.37000000002\n",
      "87: 130000.34\n",
      "88: 129999.68999999999\n",
      "89: 129999.52\n",
      "90: 129999.05\n",
      "91: 129999.7\n",
      "92: 129999.68000000001\n",
      "93: 129999.56000000001\n",
      "94: 129999.47\n",
      "95: 130000.62\n",
      "96: 130000.69\n",
      "98: 129999.09000000001\n"
     ]
    }
   ],
   "source": [
    "p20 = probability_of_closeness(20, 1, 100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.82"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p20"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a major challenge for the argument in the post. Having a subset of the payments sum to within a dollar of \\$130,000 is very unlikely if we only look at the last 10 payments (p=0.003, or 0.3%), but not that unlikely if we look at the last 15 (p=0.131, or 13.1%) and a safe bet if you look at the last 20 (p=0.82, or 82%). Why did the authors of the analysis choose 10 and not 15 or 20? The result of the analysis is very sensitive to this \"magic\" number."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}