├── .gitignore ├── LICENSE ├── README.md ├── basic_analysis_in_python └── Descriptive Statistics and Exploratory Data Analysis.ipynb ├── dimensionality └── Reducing Dimensionality with Principal Component Analysis.ipynb ├── instrumental_variables_estimation ├── Instrumental-Variables Estimation.ipynb └── Using IVE to Recover the Treatment Effect.ipynb ├── multilevel_models └── Multilevel Models.ipynb ├── regression_discontinuity └── Regression Discontinuity Analysis.ipynb └── survival_analysis ├── DATA_LICENSE.txt ├── Discrete Time Survival Analysis in Python.ipynb └── wikipedia-validation.zip /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | 5 | # ignores unique to this repository 6 | *ipython_notebook_config.py 7 | *.ipynb_checkpoints 8 | *.dta 9 | survival_analysis/*.tsv 10 | 11 | 12 | # C extensions 13 | *.so 14 | 15 | # Distribution / packaging 16 | .Python 17 | env/ 18 | build/ 19 | develop-eggs/ 20 | dist/ 21 | downloads/ 22 | eggs/ 23 | lib/ 24 | lib64/ 25 | parts/ 26 | sdist/ 27 | var/ 28 | *.egg-info/ 29 | .installed.cfg 30 | *.egg 31 | 32 | # PyInstaller 33 | # Usually these files are written by a python script from a template 34 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 35 | *.manifest 36 | *.spec 37 | 38 | # Installer logs 39 | pip-log.txt 40 | pip-delete-this-directory.txt 41 | 42 | # Unit test / coverage reports 43 | htmlcov/ 44 | .tox/ 45 | .coverage 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | 50 | # Translations 51 | *.mo 52 | *.pot 53 | 54 | # Django stuff: 55 | *.log 56 | 57 | # Sphinx documentation 58 | docs/_build/ 59 | 60 | # PyBuilder 61 | target/ 62 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 J. Nathan Matias 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Statistical Research In Python 2 | In the Spring of 2015, I committed to carrying out my general exams in a way that made a contribution to public knowledge. I'm summarizing every book and article I read on AcaWiki ([the full list is here](http://acawiki.org/J_Nathan_Matias_General_Exams_Reading_List)). 3 | 4 | This repository on Github includes all the data analyses I carry out in the quantitative section on "[Statistical Methods for Computational Social Science](http://acawiki.org/J_Nathan_Matias_General_Exams_Reading_List#Statistical_Methods_for_Computational_Social_Science)." 5 | 6 | --J. Nathan Matias, PhD Student, MIT Media Lab & Center for Civic Media 7 | 8 | ## Code Examples 9 | To execute these code examples, you will need to have jupyter notebook 3.0, along with standard Python data/stats libraries pandas, statsmodels, numpy, seaborn, and scipy. 10 | 11 | ### Basic Analysis in Python 12 | * [Downloading Sample R Datasets and Doing Descriptive Statistics](http://nbviewer.ipython.org/github/natematias/research_in_python/blob/master/basic_analysis_in_python/Descriptive%20Statistics%20and%20Exploratory%20Data%20Analysis.ipynb) 13 | 14 | ## Regression Models for Categorical and Limited Dependent Variables. 15 | _Long, J. Scott. 1997. Regression Models for Categorical and Limited Dependent Variables. 1 edition. Thousand Oaks: SAGE Publications, Inc._ 16 | 17 | ## Methods Matter: Improving causal Inference in Educational and Social Science Research 18 | _Murnane, Richard J., and John B. Willett. 2010. Methods Matter: Improving causal Inference in Educational and Social Science Research, Oxford University Press._ 19 | * [Regression Discontinuity Analysis](http://nbviewer.ipython.org/github/natematias/research_in_python/blob/master/regression_discontinuity/Regression%20Discontinuity%20Analysis.ipynb) 20 | * [Instrumental Variables Estimation in Python](http://nbviewer.ipython.org/github/natematias/research_in_python/blob/master/instrumental_variables_estimation/Instrumental-Variables%20Estimation.ipynb) 21 | * [Using Instrumental-Variables Estimation to Recover the Treatment Effect in Quasi-Experiments](http://nbviewer.ipython.org/github/natematias/research_in_python/blob/master/instrumental_variables_estimation/Using%20IVE%20to%20Recover%20the%20Treatment%20Effect.ipynb) 22 | 23 | ## Applied Longitudinal Data Analysis 24 | _Singer, Judith D., and John B. Willett. 2003. Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. Oxford university press._ 25 | * [Multi-Level Models (Fixed and Random Effects)](http://nbviewer.ipython.org/github/natematias/research_in_python/blob/master/multilevel_models/Multilevel%20Models.ipynb) 26 | * [Fitting Discrete Time Hazard/Survival Models](http://nbviewer.ipython.org/github/natematias/research_in_python/blob/master/survival_analysis/Discrete%20Time%20Survival%20Analysis%20in%20Python.ipynb) 27 | -------------------------------------------------------------------------------- /instrumental_variables_estimation/Instrumental-Variables Estimation.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 3, 6 | "metadata": { 7 | "collapsed": false 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "# RESEARCH IN PYTHON: INSTRUMENTAL VARIABLES ESTIMATION\n", 12 | "# by J. NATHAN MATIAS March 18, 2015\n", 13 | "\n", 14 | "# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n", 15 | "# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n", 16 | "# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n", 17 | "# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n", 18 | "# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n", 19 | "# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\n", 20 | "# THE SOFTWARE." 21 | ] 22 | }, 23 | { 24 | "cell_type": "markdown", 25 | "metadata": {}, 26 | "source": [ 27 | "# Instrumental Variables Estimation" 28 | ] 29 | }, 30 | { 31 | "cell_type": "markdown", 32 | "metadata": {}, 33 | "source": [ 34 | "This section is taken from [Chapter 10](http://www.ats.ucla.edu/stat/stata/examples/methods_matter/chapter10/default.htm) of [Methods Matter](http://www.ats.ucla.edu/stat/examples/methods_matter/) by Richard Murnane and John Willett. The descriptions are taken from Wikipedia, for copyright reasons.\n", 35 | "\n", 36 | "In statistics, econometrics, epidemiology and related disciplines, the method of instrumental variables (IV) is used to estimate causal relationships when controlled experiments are not feasible or when a treatment is not successfully delivered to every unit in a randomized experiment.\n", 37 | "\n", 38 | "In linear models, there are two main requirements for using an IV:\n", 39 | "\n", 40 | "* The instrument *must* be correlated with the *endogenous explanatory variables*, conditional on the other covariates.\n", 41 | "* The instrument *cannot* be correlated with the *error term* in the explanatory equation (conditional on the other covariates), that is, the instrument cannot suffer from the same problem as the original predicting variable.\n" 42 | ] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": {}, 47 | "source": [ 48 | "# Example: Predicting Civic Engagement from College Attainment" 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": {}, 54 | "source": [ 55 | "Can we use college attainment (COLLEGE) to predict the probability of civic engagement (REGISTER)? College attainment is not randomized, and the arrow of causality may move in the opposite direction, so all we can do with standard regression is to establish a correlation.\n", 56 | "\n", 57 | "In this example, we use an _Instrumental Variable_ of distance between the student's school and a community college (DISTANCE), to estimate a causal relationship. This is possible only if this variable is related to college attainment and NOT related to the residuals of regressing COLLEGE on REGISTER. \n", 58 | "\n", 59 | "The python code listed here is roughly parallel to [the code listed in the textbook example](http://www.ats.ucla.edu/stat/stata/examples/methods_matter/chapter10/default.htm) for Methods Matter Chapter 10. If you're curious about how to do a similar example in R, check out \"[A Simple Instrumental Variables Problem](http://www.r-bloggers.com/a-simple-instrumental-variables-problem/)\" by Adam Hyland in R-Bloggers or Ani Katchova's \"[Instrumental Variables in R](https://www.youtube.com/watch?v=OwM3BgWEgUg) video on YouTube." 60 | ] 61 | }, 62 | { 63 | "cell_type": "code", 64 | "execution_count": 2, 65 | "metadata": { 66 | "collapsed": true 67 | }, 68 | "outputs": [], 69 | "source": [ 70 | "# THINGS TO IMPORT\n", 71 | "# This is a baseline set of libraries I import by default if I'm rushed for time.\n", 72 | "\n", 73 | "import codecs # load UTF-8 Content\n", 74 | "import json # load JSON files\n", 75 | "import pandas as pd # Pandas handles dataframes\n", 76 | "import numpy as np # Numpy handles lots of basic maths operations\n", 77 | "import matplotlib.pyplot as plt # Matplotlib for plotting\n", 78 | "import seaborn as sns # Seaborn for beautiful plots\n", 79 | "from dateutil import * # I prefer dateutil for parsing dates\n", 80 | "import math # transformations\n", 81 | "import statsmodels.formula.api as smf # for doing statistical regression\n", 82 | "import statsmodels.api as sm # access to the wider statsmodels library, including R datasets\n", 83 | "from collections import Counter # Counter is useful for grouping and counting\n", 84 | "import scipy" 85 | ] 86 | }, 87 | { 88 | "cell_type": "markdown", 89 | "metadata": {}, 90 | "source": [ 91 | "# Acquire Dee Dataset from Methods Matter" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": 29, 97 | "metadata": { 98 | "collapsed": false 99 | }, 100 | "outputs": [], 101 | "source": [ 102 | "import urllib2\n", 103 | "import os.path\n", 104 | "if(os.path.isfile(\"dee.dta\")!=True):\n", 105 | " response = urllib2.urlopen(\"http://www.ats.ucla.edu/stat/stata/examples/methods_matter/chapter10/dee.dta\")\n", 106 | " if(response.getcode()==200):\n", 107 | " f = open(\"dee.dta\",\"w\")\n", 108 | " f.write(response.read())\n", 109 | " f.close()\n", 110 | "dee_df = pd.read_stata(\"dee.dta\")" 111 | ] 112 | }, 113 | { 114 | "cell_type": "markdown", 115 | "metadata": { 116 | "collapsed": false 117 | }, 118 | "source": [ 119 | "# Summary Statistics" 120 | ] 121 | }, 122 | { 123 | "cell_type": "code", 124 | "execution_count": 30, 125 | "metadata": { 126 | "collapsed": false 127 | }, 128 | "outputs": [ 129 | { 130 | "data": { 131 | "text/html": [ 132 | "
\n", 133 | "\n", 134 | " \n", 135 | " \n", 136 | " \n", 137 | " \n", 138 | " \n", 139 | " \n", 140 | " \n", 141 | " \n", 142 | " \n", 143 | " \n", 144 | " \n", 145 | " \n", 146 | " \n", 147 | " \n", 148 | " \n", 149 | " \n", 150 | " \n", 151 | " \n", 152 | " \n", 153 | " \n", 154 | " \n", 155 | " \n", 156 | " \n", 157 | " \n", 158 | " \n", 159 | " \n", 160 | " \n", 161 | " \n", 162 | " \n", 163 | " \n", 164 | " \n", 165 | " \n", 166 | " \n", 167 | " \n", 168 | " \n", 169 | " \n", 170 | " \n", 171 | " \n", 172 | " \n", 173 | " \n", 174 | " \n", 175 | " \n", 176 | " \n", 177 | " \n", 178 | " \n", 179 | " \n", 180 | " \n", 181 | " \n", 182 | " \n", 183 | " \n", 184 | " \n", 185 | " \n", 186 | " \n", 187 | " \n", 188 | " \n", 189 | " \n", 190 | " \n", 191 | " \n", 192 | "
registercollegedistance
count 9227.000000 9227.000000 9227.000000
mean 0.670857 0.547090 9.735992
std 0.469927 0.497805 8.702286
min 0.000000 0.000000 0.000000
25% 0.000000 0.000000 3.000000
50% 1.000000 1.000000 7.000000
75% 1.000000 1.000000 15.000001
max 1.000000 1.000000 35.000000
\n", 193 | "
" 194 | ], 195 | "text/plain": [ 196 | " register college distance\n", 197 | "count 9227.000000 9227.000000 9227.000000\n", 198 | "mean 0.670857 0.547090 9.735992\n", 199 | "std 0.469927 0.497805 8.702286\n", 200 | "min 0.000000 0.000000 0.000000\n", 201 | "25% 0.000000 0.000000 3.000000\n", 202 | "50% 1.000000 1.000000 7.000000\n", 203 | "75% 1.000000 1.000000 15.000001\n", 204 | "max 1.000000 1.000000 35.000000" 205 | ] 206 | }, 207 | "execution_count": 30, 208 | "metadata": {}, 209 | "output_type": "execute_result" 210 | } 211 | ], 212 | "source": [ 213 | "dee_df[['register','college', 'distance']].describe()" 214 | ] 215 | }, 216 | { 217 | "cell_type": "markdown", 218 | "metadata": { 219 | "collapsed": false 220 | }, 221 | "source": [ 222 | "# Cross-Tabulation" 223 | ] 224 | }, 225 | { 226 | "cell_type": "code", 227 | "execution_count": 31, 228 | "metadata": { 229 | "collapsed": false 230 | }, 231 | "outputs": [ 232 | { 233 | "name": "stdout", 234 | "output_type": "stream", 235 | "text": [ 236 | "college 0 1\n", 237 | "register \n", 238 | "0 1780 1257\n", 239 | "1 2399 3791\n", 240 | "chi2: 323\n", 241 | "p: 0.000\n", 242 | "df: 1.000\n", 243 | "expected:\n", 244 | "[[ 1375.48748239 1661.51251761]\n", 245 | " [ 2803.51251761 3386.48748239]]\n" 246 | ] 247 | } 248 | ], 249 | "source": [ 250 | "print pd.crosstab(dee_df.register, dee_df.college)\n", 251 | "chi2 = scipy.stats.chi2_contingency(pd.crosstab(dee_df.register, dee_df.college))\n", 252 | "print \"chi2: %(c)d\" % {\"c\":chi2[0]}\n", 253 | "print \"p: %(p)0.03f\" % {\"p\":chi2[1]}\n", 254 | "print \"df: %(df)0.03f\" % {\"df\":chi2[2]}\n", 255 | "print \"expected:\"\n", 256 | "print chi2[3]" 257 | ] 258 | }, 259 | { 260 | "cell_type": "markdown", 261 | "metadata": {}, 262 | "source": [ 263 | "# Correlation Matrix" 264 | ] 265 | }, 266 | { 267 | "cell_type": "code", 268 | "execution_count": 32, 269 | "metadata": { 270 | "collapsed": false 271 | }, 272 | "outputs": [ 273 | { 274 | "data": { 275 | "text/plain": [ 276 | "" 277 | ] 278 | }, 279 | "execution_count": 32, 280 | "metadata": {}, 281 | "output_type": "execute_result" 282 | }, 283 | { 284 | "data": { 285 | "image/png": [ 286 | "iVBORw0KGgoAAAANSUhEUgAAASYAAADtCAYAAAASnNgZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", 287 | "AAALEgAACxIB0t1+/AAAGdRJREFUeJzt3XmYXFWZx/FvVaeBsEQBQVlkQAZfGRhxQRQDEURFI2Aw\n", 288 | "ICMIxgUUFAEdBRlZBhBlccVH8WERlOgoAwrIqjCYCBGVKILgGyIgBlAQIayBdFfNH+cUqRTddatr\n", 289 | "u6du/z7Pc5+mlpM6VHe99Z7lvrdUrVYREUlJOe8OiIg0UmASkeQoMIlIchSYRCQ5CkwikhwFJhFJ\n", 290 | "zpSMx5cBq/ajIyKyklLeHchTKWMfU5VJ/gZNgN6r1uh9kkwayolIchSYRCQ5CkwikhwFJhFJjgKT\n", 291 | "iCRHgUlEkqPAJCLJUWASkeQoMIlIchSYRCQ5CkwikhwFJhFJjgKTiCRHgUlEkqPAJCLJUWASkeQo\n", 292 | "MIlIchSYRCQ5CkwikpxCBiYzK5mZ6kqLDKisq6QMHDMbcvfRvPshIu0rVMZkZlPcfdTMymZ2hZnt\n", 293 | "nHefRGTiChWY3H3EzMrAT4AngN/m3CURaUMhAlMMRjUzgbcDL3H3x+PjQ7l0TETaMvAXvIzDt5E4\n", 294 | "2f0CwrzZHsDpwKXuPqdPXUn+vUqE3ifJNNCBycxK7l6NGdE1wCrAEuDnhL6fDfzA3ffrQ3eSfq8S\n", 295 | "ovdJMg3sqlzD6ttZwN+Bi4EfAQ8CXwMOAf6STw9F8nX5sDXNOgDeudyT/JIYyMBkZlsTAtFDMVta\n", 296 | "ExgBTgS+BbwQeJm7nxmfX3L3zF+SSJEMTR3cKeSBDEzA9sBxZgawN/Br4FTgVuB24IuEYRwACkoy\n", 297 | "GZWGOwtMcVHpm8ArgWeAD7v7n+sefy9wGCEpuJUwQik1a9OqgQqpdatv84ENgZcAjxGGbScAawH7\n", 298 | "E96M+dr9LZPZ0NRy5pFhFrCKu78ROAr4Uu0BM5tKGKHs5O47EBaedottVh2rzUQMTGCKq2+VePM+\n", 299 | "4AxgAXA58HLgG8AOwEx3/6GCkkx25SmlzCPDdOAqAHe/Cdi27rFlwPbuvizenhLvmw5cOU6blg3E\n", 300 | "UM7MVnH3Z+N80ncJwegYYJ14+1bgaeCV7n5/rV1Rh3BZKXZ8zurAz4APurub2SqE4e2/AsuBT7j7\n", 301 | "Lf3tef+Y2e6Ev5ER4Fx3P7vh8RcB3wdWA+4HPuDuT5vZbOBIwurhXHf/evy7O4vwBVgFPuruf+zf\n", 302 | "/017ujDHNI0wIqkZNbOyu1fiZ+shADM7FFjD3X9mZu8Zr81EXnggMqYYlMrAQkIw3RR4D7A5MAc4\n", 303 | "Hti7/sNZ1KAUjZtiA5jZtsA8YDPCBwngQOCp2OZA4Nz+dbe/zGwY+DLwVuBNwEFmtn7D044FLnD3\n", 304 | "GcDvgI/Ev7EvALsQ5jEPMbN1gd2BShyyfA74fH/+TzpTGiplHhkeI0yP1KwUYOKpX6cT3q/ZrbRp\n", 305 | "VdKByczOj99gEDZNbgRcCHycMOn2deB+dz/B3a+YRFUFmqXYEPZzzQK87r5/q2uzCNjIzKb1vqu5\n", 306 | "2BJY7O5L3X058EtgRsNznnsPCUOPt8QP0JbxjIH1gCHgGXf/CfCR+NxNgUd63P+uGBouZx4ZbiCc\n", 307 | "SYGZvQH4Q8Pj3wZWBfasG9JltWlJskM5M9uMsAfpajPbhfA//DhhcvtU4G7C7u61gQeg8FlSvXFT\n", 308 | "bAB3vxEgrlrW/J4wOfmT+AezHrBGw79TFNOApXW3HydMzo73nCdqj8eTwN9NmLP8KfBU3f3nAXsC\n", 309 | "e/Ws5100tErHZ2L9GHirmd0Qb38grsStSTgP9YOEzPy6+Lf21bHatPPCSQYmMxt297vN7HjgM4Sh\n", 310 | "2hx338zMDgdeQ0inP+TuD+TX09y0ky6fC2xpZvMJQX4R8M8e9S8XZnYiYQHklcBNdQ+txfOznMcI\n", 311 | "wemh+PijtQfc/WIz+zFwHnBA/Im7zzGzI4GbzGxLd3+6N/8n3VEqdzZ4iF/0Bzfcvajuv8eLfI1t\n", 312 | "Jiy5wBR3dC+P4/0dCZPbrwJOikOPSwhD0PPc/bra0G0SZUsQAsvuwIUTSJe3A65z90/GOajt3P2Z\n", 313 | "Xnay39z9GAgruMDtZrY28CRhGHdaw9NrQ47zgXcA88xsLUKW9NY4r/kkIRvdH9jY3b9AWGSpxCNp\n", 314 | "Q6skPVPTVFKBqe6E3DJhj8RnCXNLh7Jir9KIu385Pn8yBiVokmK7+1njtHHgh2Z2NGFZ98A+9DMX\n", 315 | "8W/ok8DVhC+xc9z9ATNbBzjL3WcDJwHnm9mBhKxp37gqdwEhSC0HbgEuIKzcnWdmvwCGgcMGIaiX\n", 316 | "yoMbmJI7iTcGpV8BdwBvALYA3k3Y3X0G8E13v7affWqRTk5tjd6nPvn923bM/MJ+1TXzk/xdJJUx\n", 317 | "RfsBGwA7A+sSvrEuJsz8z4YQvNpZghSZTDqdY8pT7rlenA+oN0o41eTV7n4voZzJAuDHZjYdQEFJ\n", 318 | "JFt5ylDmkapcM6Y40V2bU9qNMPdxIfBR4Cozu5awynIQYWVu3bz6KjJoytkbKJOVW2Cq1VOKQek2\n", 319 | "4B7g1YS5pT2AIwjLuH8glDHZlHDqiTQws40J79kiwjL4noQh8BJCMD+GsJDwyCBM2vZDs/fM3Zc2\n", 320 | "azsoUs6IsuQylKu/mgnhQ3MPcDhhYnQdYLq7H0fYsbtNfM7O7n53Hv0dAFsSVtmOBnYC9iGsYK5H\n", 321 | "qH8+k7AK1bjJcDJr9p4VQqlcyjxS1feMKU5c14ZvZxGypJcSNsSdSZhj2svMfh43uv0FuM/d/9bv\n", 322 | "vg6QhYQTUtclnGbxCLCFuy82s00Ip/Bs5e4P5tjH1Iz7nuXaqy4qT8l9CrltuWwXiPuPLgNeQZg/\n", 323 | "+jRhLulywsbB97n7j7v9uj2mZfDW6H3qkzv3m5m5XWCLuVck+bvIa45pDcLw7e2EuaM5hHPgXkzY\n", 324 | "FnDNJN48KdIVLZykm6xcApO7PxF3ID9LGM4d4u6n1x5XUBLpXKc7v9up+xXvW8iKE6TvcvcPTfS1\n", 325 | "c1uVc/fHzOw4YCrwLTO7Gviru48qIIl0rgtzTM/V/TKz1xPqfs2qPRjPuTyTUOa6Gu9bDcDdd+7k\n", 326 | "hXPN9WLdm88QJmbvqbsck4h0qFQuZx4Z2qn7tQ2wupldbWbXxoA2YbkPQt39cXe/A1YM4USkc13Y\n", 327 | "+T1m3a/aDXe/0d2XNLR5EjjN3XclbJSeW9+m5b5PtEEvaQgn0kWlUvbRXDt1vxYBcwHc/U7gYcK5\n", 328 | "rxOSVGASke7pQsbUTpncDxBr0JvZhoSsa8LFHFOsLiAiXdCFU1Laqft1DvAdM5tXa9POSffJ1WMa\n", 329 | "YHqvWqP3qU/+fuT+mVMjLz7le0n+LpQxiRRUaYBP4lVgEimoUvbkdrIUmEQKapDLnigwiRTVAF+M\n", 330 | "QIFJpKDKQ8qYRCQ1CReCy6LAJFJQmmMSkfSUNMckIonRPiYRSY72MYlIcpQxiUh6Opxjaqe0bitt\n", 331 | "WjG4s2Mi0lRpaCjzyPBcaV3gKGI5k5pYWncesBmxtG5Wm1YpMIkUVbmUfTTXTmndrDatdb2dRiKS\n", 332 | "vi5kTO2U1m3aplWaYxIpqs5PSWmntG47bZ5HGZNIQXXhKintlNZtp83ztJIx6QIBrdN71Rq9T9k6\n", 333 | "34TUecbUTmnd57Vp54UzS+v+Zsb27fy7k8rr5i0ooZKxrdL71CfLfnBK5hfAau89MsnfheaYRIpK\n", 334 | "ZU9EJDk6JUVEkjM0uB/vwe25iDSnQnEikpyy5phEJDW6GIGIJEcZk4gkRxmTiCRHGZOIJEf7mEQk\n", 335 | "NVXt/BaR5OjyTSKSmmqHc0xZ9bvNbHfgGGAEONfdz473LwSWxqfd5e4fmuhrKzCJFFXnk9/P1e82\n", 336 | "s9cT6nfPAjCzYeDLhNK5TwE3mNklwOMA7r5zJy88uLmeiDRXLmcfzTWr370lsNjdl7r7cuCXwJuA\n", 337 | "bYDVzexqM7s2BrSJd72dRiKSvmp5KPPI0Kx+9zRWDNcgZEovAJ4ETnP3XYGPAnPbqfmtwCRSUNVS\n", 338 | "OfPI0Kx+99KGx9YCHgEWAXMB3P1O4GFgg4n2XYFJpKC6kDE1q9/9J2ALM1vbzFYBZgALCKV0vxTb\n", 339 | "bEjIrB6YaN81+S1SVJ1vsGxa89vMPglcTUhwznH3B8zsHOA7Zjav1qadq6So5ncXqOb3hOh96pPH\n", 340 | "br46s+b3tNfumuTvQhmTSFFpg6WIpKZSGtxTUgY3pErbzGyo4XaS6bx0qFTOPhKljGmSMbMp7j4S\n", 341 | "95ZsAyxz9zvy7pd0X2WAy56kGzKlJ2JQGgJuBY4H/mhmx5rZC/PtmXRdqZR9JEqBaZJo2H37KcI+\n", 342 | "lE8Bo8BOwMty6Jb0UKU0lHmkSkO5ScDMhtx9NM4llQm7dt8GvB44DRgG9jWz24Dl7p65zCzpa2Fn\n", 343 | "d7IUmAquYU7pTOAh4OfATcCbgWWEzGmWuz+bX0+l26oJZ0RZFJgKzMzKdXNKpwHvImRMTwNnE4Zz\n", 344 | "WwF7u/uVZlZStlQc1YTnkLIoMBWYu1fi8O0yQjB6G2HC++PA14BPu/vToC0DRZTyHFKWwR2Eyrga\n", 345 | "JrpXBdYH/uzutwAfJJSm+DDwYTMbjplVVdlSsVTKQ5lHqpQxFUxtTin+9wsIdXLmAbPM7O/A34Df\n", 346 | "AhsB73H3M3LrbB+NVwZ2jOcdDrzY3T9bd99XgD+5+7f70tkuqXZ4SmI7pXWz2rRKgalA4upbbU7p\n", 347 | "EmAqoVTFg4QSFofG+w4g/OHsZWZrufvjefW5H8YpA3upuz9Y95zVgHOA1wH/G+9bD/gusAUwcJtQ\n", 348 | "uzCUm2hp3UuBHYBVx2ozERrKFYSZrRO3BAwB/0Mo0HURcBhhOHccsD1wPrBPvH1I0YNSNFYZ2BkN\n", 349 | "z1kNOA/4PCuqH6xBeJ++xwBWROhCobiJltadEdtcOU6blikwFYCZnQKcbWYvdfdRYHXgUcIk97nA\n", 350 | "psAmwD+B+YRvuDe4+8359LjvxisD+xx3f9Tdf9Zw3z3u/us+9K8nurDBsp3Sus3atEyBqRiWEoZm\n", 351 | "x5nZOoQ5pEMJWdOvCdsERtz9GXe/DDjM3W/Lrbd9YmYnmtn/EYa10+oeqpWBLbQqpcwjw0RL6z6a\n", 352 | "0aZlmmMaYDFT2gTYl/CNdQRwKmEj5erAfxAC1MHu/qvaPqWYVRWeux8DYUEAuN3M1iasSM4g7Osq\n", 353 | "tErneccNwO7Ahc1K67Lye1pt0qZlCkwDKtZZHiaM4b9BGLaV488K8F/AicDq7v63ybxPKS4IjFUG\n", 354 | "dh3gLHef3dBkrG0TA7eVotp5YGqntO7z2rTzwiqt2wX9Lq1bd+7b6sCngfcT/kAOIQSmY4HLgY+4\n", 355 | "+zOxTSq7ulVat0/8z3/N/H3b5i9N8nehjGnA1J37NuTuT5nZycBywjdTLXOaAvy+FpQAEglK0ked\n", 356 | "7mPKkwLTAGnYp3ReHN8/DJxCyEQ+CQy7+0Hx+alkSZKDSnVw17YUmAZIXemSGwmrH9cBHwPWBfYk\n", 357 | "bJ68ru75CkqT2CBnTIMbUieRuKpUsxlh49+n3P0LhFW3rQkbBI939+sn80S3rFChnHmkShlT4hrq\n", 358 | "KR1CCExbAG82syXAeoStAlOBJ0CZUjNmtjGwB+FS1o8RMs0LgCWEzPMY4LPAI/VzdIOoWh3c7ycF\n", 359 | "psTVBaXfxbsuA75NOE9pf8CAD9af9yVNbQkcSNhgeRXh9JyXA0cSysLMJJQbPppwjuHAGk04I8qi\n", 360 | "wDQY5hA+LNPd/Wkz24/wAfoc8JC7/7Y2fFO2lGkh8H1CdnQVIUBt4e6LzWwTwqrmVkUI9MqYpNce\n", 361 | "JmRGOwNXAG8EfgZcUzchrqDUAnd/mJV3ff+h7rHawsGFfe1Uj2hVTnrtRuB64HwzuwbYG9izdmqJ\n", 362 | "ApKMZZAD0+D2fBJx94eAgwjf9I8Cu7n75Vp9k2YqLRypUsY0INz9PsIJuoBqdEu2Qc6YFJgGlIZv\n", 363 | "kkWT3yKSnNEeBCYzm0rY91XbP/d+d/9Hw3MOJEw9jAAn1U07LCHsHwNY4O5Hj/c6CkwiBdWjU1IO\n", 364 | "Bm5x9xPMbB/ClpXDaw+a2UsIZyO8lrDp95dxweZfgJvdfY9WXkSBSaSgKpWeBKbphJPGIewDO6bh\n", 365 | "8e2AG2Id8OVmthjYBtgc2MjMriNc4/AId1/EOBSYRAqq0vnlmz5EXTYU/Z0VNb2fVzudUFZ3rFrg\n", 366 | "9wMnu/tFZjadMBzcbrzXVmASKahOMyZ3P4dwSavnmNlFrKjpXavzXa+x5netvvodhDkn3P0GM9uw\n", 367 | "2WsP7nqiiDTVhYsRjOUGwulQAO8gXEy13q+BHc1s1XjB1S2BPxKqqh4OYGbbAPc2exFlTCIFNdqb\n", 368 | "OaZvEc5AmE+40u6+AGZ2BOE6c5eZ2dcJlwkrA0e7+zNm9kXgAjObScic5jR7EdX87oJ+1/wecHqf\n", 369 | "+uSKhcsz97rNfM1wkr8LZUwiBdWLfUz9osAkUlDa+S0iyRlN+SzdDApMIgU1yBcjUGASKShlTCKS\n", 370 | "nB5tF+gLBSaRgmq+EyhtCkwiBaWhnIgkR9sFRCQ5yphEJDmaYxKR5PQiY2qltG583nqESgRbu/uz\n", 371 | "rbarUdkTkYKqVrOPNtRK684AvksorbsSM9sVuAZYfyLt6mVmTK+bt2ACfZ60qg0/pTm9T9k6nrke\n", 372 | "He1GN54nq7QuhMvZ7wLcPMF2z8kMTL+6Y2nWUya9ebevyWdmD3HqRb35SyiSz8weApU96YtO55ja\n", 373 | "LK2Lu/88tq+/exorSu6O2a6e5phECmq00kpkGv87os3SuuN5jBCcWmqnOSaRgurRHFNWad2utFPG\n", 374 | "JFJQPZpjyiytW/fcala78SgwiRRUSyO5CXL3p4H3jHH/V8a472VZ7cajwCRSUJXRzuaY8qTAJFJQ\n", 375 | "vciY+kWBSaSgRlvKmNKkwCRSUBWdxCsiqakM8Fm8CkwiBdXa5HeaFJhECkpDORFJTmunpKRJgUmk\n", 376 | "oKoKTCKSGm0XEJHkVLUqJyKpUcYkIsnpRcbUQc3vErAEWBSfssDdjx7vdRSYRAqqRxlTrXb3CWa2\n", 377 | "D6F290pVLmPN7y+ycs3vzYGb3X2PVl5EheJECqpaqWYebZhOqNlN/PmWMZ5Tq/n9SN19rwU2MrPr\n", 378 | "zOxyM3t5sxdRxiRSUKMdXr+pyzW/7wdOdveLzGw6YTi43XivrcAk0gIzK7t7ZbzbKep0H1OXa37/\n", 379 | "FhiJ/+4NZrZhsydrKCeSwcymuHvFzNY0s40AUg9KEOaYso42tFvz+1hi9mVm2wD3NnuyMiaRJsys\n", 380 | "5O4jZvYy4FLgITNbk/Ahu9Hdk12T79E+pnZrfn8RuMDMZhIypznNXkSBSaQJd6+a2WqED+Rc4EHg\n", 381 | "LGAWcBNxeJKiTueYxtJBze+lwO6tvo6GciJjMLOhupsjhPmU7YETgeOAFwNvzaFrLauMVDKPVClj\n", 382 | "EmkQ55RGzGx94DWEjYEXA8cDfyBU8H87IUAlS4XiRAqibk5pE2AB4bLWpwOXAMPAbGAnYFd3vzu3\n", 383 | "jrag0oOhXL9oKCdSJ84pDQFfAOYD27v7ucC7CEveOwDvcvff5djNllQq1cwjVQpMIqw8p+TutWvY\n", 384 | "loGpcfL7w8Cq7r7M3R/Po48TVRmtZB6p0lBOJj0zG3L30Tin9BbgWuAywipchbDTeX3gj/n1cuKq\n", 385 | "A1xbV4FJJrW6oLQBYbPgBoRg9ArgAGBvYHVg59TnlBr1YrtAv2goJ5NaDEovIqywXQjsRlh5uwX4\n", 386 | "nbvPAvZy91ty7GZbenQSb18oMMmkZGbl+LNECEoHApsAvyDsSn4AuCbOLy3PqZsdGR0dzTxSpaGc\n", 387 | "TDp1+5TWJHwGjow/dwWOAL4KvBtY7u7L8utpZ1LOiLIoMMmkEueURsxsM8KmyVHgO8AnCKed/BdQ\n", 388 | "cfev5tjNrkh51S2LApNMKnWrbwuAi4DNCEO5p4FD4s+f5tfD7qn0YFWuldK68YTefeLNK2K1y5ZK\n", 389 | "8tZojkkmBTN7U/xwAKwNXEeoFlAFFgNnA0e5+6HuvjinbnZVZXQ082hDrbTuDOC7hNK6z4lVGPYl\n", 390 | "bEx9A/A2M/v3rHaNFJik8OIE987AXDO7EliDsEfpYMKpJnMJFQN+lFsneyCn0rr3Ek7Xqf3jw8Cy\n", 391 | "FtqtREM5Kbx4mskLCaVKbgXucveFZnYAcBJhn9K27v6nPPvZbZ2uurVTWtfdR4B/xi+D04CF7n6n\n", 392 | "mU0jnHc4ZrtGCkxSWLXVt3jzF4T5jVcDX4rzIFcQMqbfuPudOXWzZ/IqrRu3WJxLCESHxLsfA6Y1\n", 393 | "a1dPgUkKycyG3X15rBJwAPBLd9/PzI4ifFgWAg7s5+6t1q0eKJWRnuxTqpXW/Q1jlNaNmdIlwLXu\n", 394 | "fmqr7RopMEkhxaC0MSEAXQYsN7MqYXhxD+FDckpRgxJ0PpQbR9PSusAQMAMYNrN3xDZHjdduPApM\n", 395 | "UhhxN/eXgJOBfwCHESZjLwG+CXwMuN7dDzCzi9392dw62we9OIm3xdK6Uxsfj57XbjwKTFIIcQjx\n", 396 | "eWBdwvzFxoTg9BTw34QVuL2AZXHuqdBBCWh3O0AStF1AiuIrwEcJGydvi7d/QdjodxXwTmBP4Iy6\n", 397 | "CfFCG+STeJUxSSG4++FxI98BwB2EK3IsI5TFXQRsTdj0d2t+veyv+ZfsWMq7D+1SxiQDra5KwJqE\n", 398 | "iwa8HNiKcB7cbMLE67XAf06moDToFJhkoMUr5E4DbiZs/tsFeIiwX+l6wiWWlteVy5UBoMAkRTCV\n", 399 | "sHt7sbv/nlBbaSPgSWAbd78vz87JxGmOSYrgEeBXwOFmViFcjPIy4HPuviTXnklblDHJwItL/58m\n", 400 | "XCzgBOAg4GR3vyvXjknbFJikENz9HuB9hCvnbuvut+XbI+mEhnJSGO7+DPC3vPshnVPGJCLJUWAS\n", 401 | "keQoMIlIchSYRCQ5CkwikhwFJhFJjgKTiCRHgUlEkqPAJCLJUWASkeQoMIlIchSYRCQ5CkwikhwF\n", 402 | "JhFJjgKTiCRHgUlEkqPAJCLJUWASkeQoMIlIchSYRCQ5pWq1mncfRERWooxJRJKjwCQiyVFgEpHk\n", 403 | "KDCJSHIUmEQkOQpMIpKc/we122r+YxVm2AAAAABJRU5ErkJggg==\n" 404 | ], 405 | "text/plain": [ 406 | "" 407 | ] 408 | }, 409 | "metadata": {}, 410 | "output_type": "display_data" 411 | } 412 | ], 413 | "source": [ 414 | "sns.corrplot(dee_df[['register','college','distance']])" 415 | ] 416 | }, 417 | { 418 | "cell_type": "markdown", 419 | "metadata": {}, 420 | "source": [ 421 | "# Linear Regression of REGISTER on COLLEGE" 422 | ] 423 | }, 424 | { 425 | "cell_type": "code", 426 | "execution_count": 33, 427 | "metadata": { 428 | "collapsed": false 429 | }, 430 | "outputs": [ 431 | { 432 | "name": "stdout", 433 | "output_type": "stream", 434 | "text": [ 435 | " OLS Regression Results \n", 436 | "==============================================================================\n", 437 | "Dep. Variable: register R-squared: 0.035\n", 438 | "Model: OLS Adj. R-squared: 0.035\n", 439 | "Method: Least Squares F-statistic: 335.9\n", 440 | "Date: Wed, 18 Mar 2015 Prob (F-statistic): 1.03e-73\n", 441 | "Time: 22:15:12 Log-Likelihood: -5959.0\n", 442 | "No. Observations: 9227 AIC: 1.192e+04\n", 443 | "Df Residuals: 9225 BIC: 1.194e+04\n", 444 | "Df Model: 1 \n", 445 | "Covariance Type: nonrobust \n", 446 | "==============================================================================\n", 447 | " coef std err t P>|t| [95.0% Conf. Int.]\n", 448 | "------------------------------------------------------------------------------\n", 449 | "Intercept 0.5741 0.007 80.391 0.000 0.560 0.588\n", 450 | "college 0.1769 0.010 18.326 0.000 0.158 0.196\n", 451 | "==============================================================================\n", 452 | "Omnibus: 603.688 Durbin-Watson: 1.955\n", 453 | "Prob(Omnibus): 0.000 Jarque-Bera (JB): 1441.712\n", 454 | "Skew: -0.689 Prob(JB): 0.00\n", 455 | "Kurtosis: 1.640 Cond. No. 2.74\n", 456 | "==============================================================================\n", 457 | "\n", 458 | "Warnings:\n", 459 | "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n" 460 | ] 461 | } 462 | ], 463 | "source": [ 464 | "result = smf.ols(formula = \"register ~ college\", data = dee_df).fit()\n", 465 | "print result.summary()\n" 466 | ] 467 | }, 468 | { 469 | "cell_type": "markdown", 470 | "metadata": {}, 471 | "source": [ 472 | "# Two-Stage Least Squares Regression of REGISTER ~ COLLEGE where IV=DISTANCE\n", 473 | "###using statsmodels.formula.api.ols" 474 | ] 475 | }, 476 | { 477 | "cell_type": "markdown", 478 | "metadata": {}, 479 | "source": [ 480 | "In two-stage least squares regression, we regress COLLEGE on DISTANCE and use the predictions from that model as the predictors for REGISTER." 481 | ] 482 | }, 483 | { 484 | "cell_type": "code", 485 | "execution_count": 34, 486 | "metadata": { 487 | "collapsed": false 488 | }, 489 | "outputs": [ 490 | { 491 | "name": "stdout", 492 | "output_type": "stream", 493 | "text": [ 494 | "==============================================================================\n", 495 | " FIRST STAGE\n", 496 | "==============================================================================\n", 497 | " OLS Regression Results \n", 498 | "==============================================================================\n", 499 | "Dep. Variable: college R-squared: 0.012\n", 500 | "Model: OLS Adj. R-squared: 0.012\n", 501 | "Method: Least Squares F-statistic: 115.9\n", 502 | "Date: Wed, 18 Mar 2015 Prob (F-statistic): 7.35e-27\n", 503 | "Time: 22:15:12 Log-Likelihood: -6598.2\n", 504 | "No. Observations: 9227 AIC: 1.320e+04\n", 505 | "Df Residuals: 9225 BIC: 1.321e+04\n", 506 | "Df Model: 1 \n", 507 | "Covariance Type: nonrobust \n", 508 | "==============================================================================\n", 509 | " coef std err t P>|t| [95.0% Conf. Int.]\n", 510 | "------------------------------------------------------------------------------\n", 511 | "Intercept 0.6091 0.008 78.812 0.000 0.594 0.624\n", 512 | "distance -0.0064 0.001 -10.764 0.000 -0.008 -0.005\n", 513 | "==============================================================================\n", 514 | "Omnibus: 54.830 Durbin-Watson: 1.703\n", 515 | "Prob(Omnibus): 0.000 Jarque-Bera (JB): 1463.752\n", 516 | "Skew: -0.190 Prob(JB): 0.00\n", 517 | "Kurtosis: 1.086 Cond. No. 19.7\n", 518 | "==============================================================================\n", 519 | "\n", 520 | "Warnings:\n", 521 | "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n", 522 | "\n", 523 | "\n", 524 | "==============================================================================\n", 525 | " SECOND STAGE\n", 526 | "==============================================================================\n", 527 | " OLS Regression Results \n", 528 | "==============================================================================\n", 529 | "Dep. Variable: register R-squared: 0.001\n", 530 | "Model: OLS Adj. R-squared: 0.001\n", 531 | "Method: Least Squares F-statistic: 10.35\n", 532 | "Date: Wed, 18 Mar 2015 Prob (F-statistic): 0.00130\n", 533 | "Time: 22:15:12 Log-Likelihood: -6118.9\n", 534 | "No. Observations: 9227 AIC: 1.224e+04\n", 535 | "Df Residuals: 9225 BIC: 1.226e+04\n", 536 | "Df Model: 1 \n", 537 | "Covariance Type: nonrobust \n", 538 | "==================================================================================\n", 539 | " coef std err t P>|t| [95.0% Conf. Int.]\n", 540 | "----------------------------------------------------------------------------------\n", 541 | "Intercept 0.5157 0.049 10.632 0.000 0.421 0.611\n", 542 | "college_fitted 0.2837 0.088 3.216 0.001 0.111 0.457\n", 543 | "==============================================================================\n", 544 | "Omnibus: 658.930 Durbin-Watson: 1.938\n", 545 | "Prob(Omnibus): 0.000 Jarque-Bera (JB): 1638.898\n", 546 | "Skew: -0.726 Prob(JB): 0.00\n", 547 | "Kurtosis: 1.532 Cond. No. 23.4\n", 548 | "==============================================================================\n", 549 | "\n", 550 | "Warnings:\n", 551 | "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n" 552 | ] 553 | } 554 | ], 555 | "source": [ 556 | "print \"==============================================================================\"\n", 557 | "print \" FIRST STAGE\"\n", 558 | "print \"==============================================================================\"\n", 559 | "result = smf.ols(formula = \"college ~ distance\", data = dee_df).fit()\n", 560 | "print result.summary()\n", 561 | "dee_df['college_fitted'] = result.predict()\n", 562 | "\n", 563 | "print\n", 564 | "print\n", 565 | "print \"==============================================================================\"\n", 566 | "print \" SECOND STAGE\"\n", 567 | "print \"==============================================================================\"\n", 568 | "\n", 569 | "result = smf.ols(formula = \"register ~ college_fitted\", data=dee_df).fit()\n", 570 | "print result.summary()" 571 | ] 572 | }, 573 | { 574 | "cell_type": "markdown", 575 | "metadata": {}, 576 | "source": [ 577 | "^^^^^^ Not sure what's going on with the R2 statistic here (it's 0.001 here, versus 0.022 in the example), although everything else matches what we see from the Stata output in the published example " 578 | ] 579 | }, 580 | { 581 | "cell_type": "markdown", 582 | "metadata": {}, 583 | "source": [ 584 | "# Adding Covariates that do not satisfy the requirements of instrumental variables" 585 | ] 586 | }, 587 | { 588 | "cell_type": "markdown", 589 | "metadata": {}, 590 | "source": [ 591 | "In the case of the covariate of race/ethicity, we expect that there might be a relationship between race/ethnicity and distance to a community college, as well as a relationship between race/ethnicity and voter registration. \n", 592 | "\n", 593 | "While race/ethnicity fails the test for instrumental variables, it can still be included as a covariate in a multiple regression model. In such cases, it is essential to include covariates at both stages of a two-stage test." 594 | ] 595 | }, 596 | { 597 | "cell_type": "markdown", 598 | "metadata": {}, 599 | "source": [ 600 | "## Correlation Matrix" 601 | ] 602 | }, 603 | { 604 | "cell_type": "code", 605 | "execution_count": 35, 606 | "metadata": { 607 | "collapsed": false 608 | }, 609 | "outputs": [ 610 | { 611 | "data": { 612 | "text/plain": [ 613 | "" 614 | ] 615 | }, 616 | "execution_count": 35, 617 | "metadata": {}, 618 | "output_type": "execute_result" 619 | }, 620 | { 621 | "data": { 622 | "image/png": [ 623 | "iVBORw0KGgoAAAANSUhEUgAAASYAAADuCAYAAACUCKq3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", 624 | "AAALEgAACxIB0t1+/AAAIABJREFUeJztnXmYXFXR/z8zkxUI+yKCKGsRQUGIyL6ILKLsKAKCoOyC\n", 625 | "bC+rsigCKiKKGxJA4CXqjyUoO4iALGERAggC3xAEeRGQANkwBJKZ+f1Rp5NOZ2ZuT6+3Z+rzPP1k\n", 626 | "0n1P97e7b9etU6dOVVt3dzdBEAR5or3ZAoIgCEoJwxQEQe4IwxQEQe4IwxQEQe4IwxQEQe4IwxQE\n", 627 | "Qe4YkvH4bGB4I4QEQbAAbc0W0EzaMvKYusnfB5RHTZBPXaGpPPKoaVATU7kgCHJHbgyTmbWZWVy1\n", 628 | "giDIjDE1BDPrkNTZbB1BEOSDpntMZjZEUqeZtZvZrWa2TbM1BUHQXJpumCTNNbN24I/Au8BjTZYU\n", 629 | "BEGTaZphSsaowE7AjsCHJM1Mj3c0RVgQBE2nKekCafo2NwW7l8BjXbsAPwZulHRgozXVgDzqCk3l\n", 630 | "kUdNg5qGGyYza5PUnTyiO4FhwKvAXen1LgV+L2m/RmmqEXnUFZrKI4+aBjUNXZUrWX0bC/wHGA9c\n", 631 | "A7wJ/Aw4EvhXI3UFQZAvGmaYzGxd3BBNSd7SYsBc4Gzg18CSwGqSLk7Ht0mK8ppBMAhpZPB7E+AJ\n", 632 | "M3sV2Ah4FPgK8AHwLLAH8H7h4DBKQTB4qbthKlp9ux/4MPAhYAY+bfseMArYHzhY0v2R/R0EQV2n\n", 633 | "coXVt/TffwM/BzYAbgG+APwCuASYLemdWhqlZBB/BXwS98QOlvRiyTGLAH8Gvi5JZjYMD76vAcwB\n", 634 | "viXpqRpo2Rk4HZ+6Xi7p0pLHlwV+B4wAXgMOkvSeme0JnIwHZ8dJuihNg8cCa6X7D5f0jyq09fk5\n", 635 | "9aXdzJYHHge2lTSpnq/Z2xgzWwO4AugCngG+mRZXfgZsBsxMT7sLMAv4CbAhvuhyhqTby9Bb0fdX\n", 636 | "9PglwNuSTjWzA4GvpYdGAusBK0iakfnBDSLq5jGZ2bCUEtBhZuOAwpe7P/AK8DQe5B4p6Z3CuBpO\n", 637 | "4XYDhknaFDgFuKBE3xjgPmBV/AcOcAgwK405BLi8WhFmNhT/MWwHbAUcmn7QxZwBXC1pS+AJ4LD0\n", 638 | "QzwP2BafBh9pZsvgn2OXpM2B7wDnVCmx18+pL+3psd8A/23Qa+4GDO9hzE+A09Jn1wbsmu7fANhe\n", 639 | "0jbpNhM/94akz243YHSW0Eq/v6LxhwHrks4xSVcUNOHJxEeHUVqYuhkmSR+kH9dE3DP7GPBlYHXg\n", 640 | "QOAs4EvFV8oax5U2A25Pz/sIMKbk8WH4yami+z5eNGYSsJKZLV6ljtHAZEnTJc0BHgC27E0rcBvw\n", 641 | "OUldwOj0g1oO6ADel/RH5p/4HwOmVqmvr8+pL+3n44sWrzfoNTfDP5vSMRtIui/9fRvwueR5rwmM\n", 642 | "NbMHzOyg9Pj2wL/N7Gbc6/xTGVor+v4AzGxTPJ76G0rSEdKFcZ1S7ytwam6YzOzKNAUBd59XAq4F\n", 643 | "jgKOAS4CXpP0PUm31rGqwOJ4LKtAZ3G2uaQJkl4tGfMk8MX0PjbGDcKiNdAxvej/M/Gk0t6Oebfw\n", 644 | "eNpDuAd+Fb4Hn4oU7r8C/yx/VwN9vX1OPWpP05Epku5M9/f3++v3a/YypqPktQuf3aL4Z7MfvqPg\n", 645 | "SDP7BLAssLqkLwI/BH5bptZ+f39m9iHckzqKnj+f0/CLc9ADNTVMZrYqPj27w8y2BR7Ev8j9gR/h\n", 646 | "U49hwFKFMZK667QCNwMPrBdoT15IX1wOzDCz+3FvahLwTt9DesbMzjaze/CrcrHXNYqFvZwZRceM\n", 647 | "AqYVHpA0Hjfuw4EDiu4/EI8zjTWzkZVoLHrt3j6n6SWPFbQdBGyX3t/6wJVmtkKdX7OnMZ14bKn0\n", 648 | "2FnARZJmS3oXuBuP5byNxzdJXtZavQmswff3JdwQ3orHCfc1swPScy8JrCXpr729/mCnZobJzIZK\n", 649 | "egm/ChyFnwDbSloVv9pvgF/dT5dUifvfXx7E9+AVvJ+/lzFmI+BuSVsA1wGvS3o/Y0yPSDo9xRFW\n", 650 | "ANYws6VScH1L4KHetAKfB+4zs1Fm9tcUq+vGYzmdZra/mZ2ajn0P/2FmGdy+6Otzeh5Ys0T7BElb\n", 651 | "Sdo6vb8ngQMk/aeer9nHmCfMbKv09+fxuOFawAOpYsVQYHM8SP9A0XOsRx+JvNV+f5J+LmlMeo4f\n", 652 | "AL+TdFU6ZkvgL2V8ToOWmqzKpYzuOckd3wK4Cr+Sfj/FaP6EG8ErJN1dmLrVOVfpBvyq/mD6/0Fm\n", 653 | "tg+wmKSxvYwR8P/M7DS83vkh1YpICwDHA3fgn8Flkl43s6WBsZL2BL6Pex2HAFOAfdOq3NW4kZoD\n", 654 | "PAVcja/8XGFmfwWGAsdUajwTfX5OPWmv4rUqfk0zW2hM+vcE3GschufDXZdW5a7CDcgc/Lx7zsxe\n", 655 | "BH5tZgXDcniW0Eq/vx6eqvhcXwt4sYdjgkTVe+Vs/obcdjyL+1Q8tvQInqv0WXzV5PJ0fLVGKa/7\n", 656 | "mvKoKzSVRx41DWpqsok3GaWHgeeAjfEVkT3w7O6fA7+SVCvXNa8nUR51habyyKOmQU2tEiz3A1YE\n", 657 | "tgGWwacc44Hdk6uLmZUTfA6CIKgs+G1mpQatE99q8ilJr+DlTB4CbjCzzQDCKAVBUC799phSoLsQ\n", 658 | "U/oiHiS+Fg8k3m5mf8FXQQ7FV+iWqZ3cIAgGA/0yTMkoFZLhngFeBj6Fx5Z2AY7D8zj+jpcx+Ri+\n", 659 | "9aTpmNnKuMZJeN7J7viU81XceJ6OB+6nVrnKVSuNT0ma0gwdPWhZ4POSNL2vsYOBVjifWpmyDVPJ\n", 660 | "6tvpuFE6FrgXWBrYTNKZKVP5QDypcpuU25QHRuPL/1Px7QN748u2J+NbFXbCp6Sn4UXrmq3xf/Cl\n", 661 | "52bR1+c16A0TrXE+tSxlGaYUuC4YpbG4l/QRPCXgYvwL2MvM7pI03sz+Bfxb0hv1El4BE/EEz2Xw\n", 662 | "E2kqsKakyWa2Cp4Uuo6kZp5ExRqb7Wn2+nk1VVV+yP35dMtQy0zJ+cIc5XI1sux0gZR/dBOwNh4/\n", 663 | "OhGPJd2C73j/qqQb6qq2RFPOyKOu0FQeedRUNbcusnamYdpp1vO5fN/9iTEtik/fdsRjRwfi07UV\n", 664 | "8LSAOxuU0R0EQRl0jKyuA5pl183aB9+YPxf38I/EDXyfddDKoWzDJOndtFXjA3w6d6SkHxeJDKMU\n", 665 | "BDmifUjVztC8ullm9hm8BtZuAGnj+NnAupJmm9nv8FX6oaS6WaVj+qW9PwenglZn4lUnf21mH0ul\n", 666 | "J+pZJSAIggroGNmeecugr7pZs4FNJM1O/x+S7uutbla/6HeCZSpcdhIe2HtZ89sxBUGQI9qGtmXe\n", 667 | "Mui1blZyRKYAmNnRwKKS/tzXmP5Q0ZaUZJyeS6KizVIQ5JCO4VVXNeqzplkyOD/Ca+TvWc6Ycqla\n", 668 | "eRilIMgn7R1tmbcMsmqa/QYvYLh70ZSukjpoC5GZLvD41ptW8rx1Y8N7J0A+l3bzuOQcmsojj5qq\n", 669 | "ZsKYT2c6DZs+9rde33da0CqssIHXwNoQb1b7WLrdVzTkp8CNpWPUjw46BcrJYwqCoPFUbSgf/sxG\n", 670 | "mb/fjR95NJcGOTPGFB5T2eTxqhuayiOPmqqmY2gjG23Xlro2vAyCoHm0ZceQcksYpiAYoLQPqS7z\n", 671 | "u5mEYQqCAUoZq265pXUnoXWkkM1e9P/W/YaDQUv7kI7MW14Jj6mEkrpT6wGzJT3XbF1B0F/a2lv3\n", 672 | "ehqGqYRklDrwxLDJwM5mdhbe2XVan4ODIEe08qpc6yqvMSX7eU7AO8KegBfB2xpYrQmygqBi2trb\n", 673 | "M295JTwmFqhl3oYb6+l4edTPAOfjpRz2NbNngDmxDSdoBdqH5NfwZDHoDVNJTOlivM72XXjZ4M/i\n", 674 | "pRxOAHaT9EHzlAZB/8hzcDuLQW2YimqZd+Ce0a64x/QecCk+nVsH+JKk26KSQtBKtHK6wKA2TJK6\n", 675 | "imqZv4dP387CC8n/DDhR0nsQKQNB61Gtx5RVWjcdswjwZ+DrkpTum8j8Tjr/lPSN/r72oDRMJe3K\n", 676 | "hwPLA3dLesrMvo7vmj4YeNfMLgY6o5Nw0GrUIF2g19K6AGY2Bg9/fJi04d/MRgBI2qaaFx50hqkQ\n", 677 | "U0p/LwHMxEs37GZm/wHewA3TSsCXJf28Dhp2xnvzzQUul3RpL8cdC6wg6dSi+y4Enpf0m0ZoMbNl\n", 678 | "8TZFI4DX8DIWBS+yp6vlcnhNnnUriclVogdYAvh90WHr4/3dfgtchhcymwN8S9JT/dSTVZB/Ib0p\n", 679 | "NDAW7zPXDRwu6R9mtj5wEb7S+z5wQD3bO9UgxrRAad1kiIoZhhuq/y26bz1gETO7A7cvp6USu/2i\n", 680 | "dcP2FVDU3rzDzG4GxuNXgSn4j+lo4CfA5cCfgGFmNqrXJ6xMw9D0GtsBWwGHmtnyJceMMLNxeNeJ\n", 681 | "wpVoOTO7DW+VVZM4VzlagDOAqyVtCTwBHJbGjsEN+qpFGncA7sQ90IbpkfSGpG3SVfo04HHcMBwK\n", 682 | "zJK0Kd6c8vIKZM3zGoBT8PMlS+/OQJekzYHvAOekIT8Fjko6x+PGs260tbdl3jLos0yupAmSXi0Z\n", 683 | "81/gfEk7AIcD4yoprTtoDJOZLZ1SAjqAPwBvA9fj7WeG400WNgGuxLuqnol3gplZYymjgcmSpkua\n", 684 | "AzwAbFlyzAjgCvyELpw9iyZN/0vtSnSUo2XeVRMvMv+59HfhaqmiYzuBbfHmj43WU4gDXgQckRYp\n", 685 | "Ps78K/4kYCUzW7yfmvoqyN+jXkl/JBlwvNVZ4fP4iqRCRceheFyzbrR1dGTeMqikTO4kYByApBfw\n", 686 | "39mK/dU+KAyTmf0QuNTMPpKaJywCTMOD3JfjJ88qwDvA/cAsYGNJj9dBzuIs2GJ7Jj4VmYekaamw\n", 687 | "e/F9L0t6tNFaSo55t/B4T1dLSXdJeqcZehI7A8+kHwTAk3hLoUKZ1+VwA99fTb15Db3qTRfBK3BD\n", 688 | "+bt03xtJy6bAN4EL+6mlX9TAY6qkTO5BJK/SzD6Mf0av91f7oDBM+MnzSeBMM1sajyEdjVvzR/E0\n", 689 | "gbmS3pd0E3CMpGdqKcDMzjaze/ApYvFVexSVexiN0DKj6JhRuEHPq5798NZiBS4HZpjZ/bh3Nwm/\n", 690 | "+PSHvryG6SWPLaBX0oF4nGls6sOGme0N/BrYSdLb/dTSL2qwifcGYLaZPYgbm+PMbB8zO6SPMZcB\n", 691 | "i5vZffjM5KBKFo4GdPA7eUqrAPviV7Pj8K4OF+Ne01dwA3WEpIcLeUr1aEkl6fSkaQjwrJkthc/H\n", 692 | "t8RzqBpGP7UUrppXAp9nwRrPedMzRtJDRf/fCF9tPT7FxDaS9H4/5T2Ie2LX9uA1PA+sWarXzPYH\n", 693 | "VpZ0Hj5d6wK6zOyreNxra0l1vxhVu+UkTYePKLl7ofrdxStwaWFp/6pemAHsMZnZMHwePwb4Rbr9\n", 694 | "jBSkxA3UOsB2kv7QqDyl9MUdD9wBTAAuk/S6mS1tZtf3MKSnQHdNgt9lavk+8BUzewDfovOLMp66\n", 695 | "In3V6EmrgdNLnxI4xswm4N93X1f63ujVa0hxpYX0AtcB65vZX/H41DH4quDP8EL+483snrQ5vG60\n", 696 | "D2nPvOWVAdklpWjv2yLAicDX8JPnSDyudAZwC76i834aU21Wdx7rRoem8sijpqp589sHZp7Py59z\n", 697 | "RS7f94CbyhXtfeuQNMvMzsWvVgfhV9ej8Pf9ZLFbH1tNgoFGe/aqW24ZUIapOE8JuCLN/d8Gfohf\n", 698 | "FY8Hhko6NB0fe9+CAUsUissJRaVLJuCrKXfjy7LLALsDI9N9hePDKAUDlrYWri6Q3+hXP0irOQVW\n", 699 | "xXNVTkirIkcD6+JJi2dJujc25AaDgba29sxbXml5j6mkntKRuGFaE/ismb2KJ9XNxL2ldyE/npKZ\n", 700 | "rQzsgi/BzsC9uquBV3Ev73TgVGBqBcvcA4ZW+JxKND4laUozdBTTyh5TyxumIqP0RLrrJuA3+B6m\n", 701 | "/QHDN5nWbbNkFYzGl7Cn4svKe+MJeSfjJVh2wrd5nAbkUX+jaIXPqVjj/+D7L5tKGVtOckvLG6bE\n", 702 | "gfiJuZmk98xsP/xk/Q4wRdJjhelbXrylxER8u8Iy+A9uKrCmpMlmtgq+grhOTo1qI2mFz6lY49NN\n", 703 | "1DGPVg5+D4g8JjPbFT8pviTpVjP7JR4/O6ooIF5vo5THXJjQVB551FQ1M392Qub5PuqYC3L5vgeK\n", 704 | "xzQBuBe40szuBL4E7F7YWpIzLykIGkOOu6Bk0brKi0iBxkPxfVXTgC9KuiVW34LBTA3KnjSNgeIx\n", 705 | "Ienf+H4oIGp0BwFVxpiyqnemYxaoYlrOmLKkV6U8x6QqATGFCwYtNfCYeq3eCT1XMc0aUy4D1jAF\n", 706 | "wWCnBp14+6reCT1XMc0aUxZhmIJgoNLRkX3rm0pqfvc5plwGTIwpCIIS2qoOs1ZS87uSMQuRaZjm\n", 707 | "/Pq2/j5n3Tl/fFfuYkcn7hHOZ5Av2jqq9jv6qt5ZyzELkal849GlteCbT46NQO4MJqGpXPKmqfpV\n", 708 | "5eozv28AtkvVOwEOMrN9gMUkjS13TCUvnJn5/fBzpdVKm8vGo5fg/PH5a4qbjGXeUhTymNEcmhrE\n", 709 | "7P/3o0xjO2Lvk3L5vnPregS9UxpMrCS4GAwC2tqzbzklgt8tRlGZl8WAJST9u5LgYjAIyHFmdxZh\n", 710 | "mFqIVAp4rpmtBtwITEkG6lhgQiSUBguQY48oizBMLYSkbjMbgTdMHIfXHhqLJ7k9Asxtorwgb7Sw\n", 711 | "x9S6JnUQkZorFJiL54lsApwNnAmsAGzXBGlBnqk+wbJphMeUc4piSssDG+DlZMcDZ+E5Im3AjriB\n", 712 | "CoL5VJ9g2TTCMOWYopjSKsBDeKfZHwN/wrsM7wlsDewg6aWmCQ3ySfUJlk0jpnI5JsWUOoDzgPuB\n", 713 | "TSRdDuwKPAZsDuwq6Yk+niYYrLS1Zd9yShimHFIcUypU4cS/q5Ep+H0wMFzSbEkzm6ExaAHaO7Jv\n", 714 | "OaV1fb0BSuom3JliSp8D/oJ3fhkHdOGbJJcH/tE8lUFL0MKldcMw5Ygio7QiXoBrRdwYrQ0cgNcy\n", 715 | "XwTYJmJKQRbdOfaIsgjDlCOSUVoWX2G7FrgT+D7wFLCVpHFmNkzSB83UGbQIVSZYZpXJNbOd8Waj\n", 716 | "c4HLJV2a7p+IL9QA/FPSN/r72mGYcoCZtUvqSnXKz8QbJ/4e+CveM288cKeZrYGfILV+/R5PsKLH\n", 717 | "l8XbY40AXgMOSv379gFOBGYD10q6sBlagCXwz6vA+sDJki5JYz4D/EDSNo3UlT6jPfHGnN3AOEkX\n", 718 | "FY2pWldf1MBjmlcmN2m9IN2HmQ3Fm8qOAWYBD5rZn/Cu11T7nlp3EjpASHlKXYW9b/hJfAmwKXAc\n", 719 | "8E9gD7yZ5+xabzspOsG2A7YCDk3xrWLOAK6WtCXe8fgwM1sGOBf4LF5OdVcz+1QztEh6Q9I26cdw\n", 720 | "GvA4nhGPmZ2U/h7eaF3J4zgP2BZPiD3SzJaula5Mql+V66tM7mhgsqTpkuYAD+CfzXrAImZ2h5n9\n", 721 | "JRm0fhOGqYmkmNJcM1sVTwe4C/cAvgXcDXwb+JakyZL+VScZPZ1gW5YcM+8EBW7Dg/KrAU9JmpaM\n", 722 | "5cM9jGuUFmBeZ5yLgCOKDPhk3LBXszZeka60uXp0WjldDugACtPwWujqk+6OjsxbBn2VyV2c+dM1\n", 723 | "cE9pCeC/wPmSdgAOB8ZVUv0iDFMTKVp9ewhv2vkmPpXbHzgSnxrcXGcZvZ1gvR3zbnr8BWAdM1s+\n", 724 | "tfDZFg/MN0NLgZ2BZyS9ULhD0niq30NYsa70He+Be1H34NOeWunqm+rLnvRVJnd6yWOj8Nbtk/AV\n", 725 | "ZNL38Da+iNMvwjA1ATPbysxGpv8uhXtHN+JxiMnApcApko6WNLlOGs42s3vwLPLFix4qnGDFzCg6\n", 726 | "ZhQwTdI0fKp5PW5AJwJvNUNL0WP74dPgmlArXckIrYRP2w6olb4suts7Mm8ZPAjsBNBDmdzngTXN\n", 727 | "bCkzG4Z7kA/hHv8FacyH8c/k9f5qj+B3g0nTjW2AY5Jx+jZ+hTmC+VtNngauqacOSacnPUOAZ81s\n", 728 | "KdwN3xLvaFxM4QS9Evg8cF8aN0bSFmY2HA/U/7AZWooeGyPpoUo01EOXmY3CPd7tJH1gZv8FOmkQ\n", 729 | "3dWXPemztK6ZHQ/cgTs4l0l63cwuA35rZoXv5aC6NCMIakvaZrIkvrrxNL6cOtHMDsBTAxbBf2DP\n", 730 | "N0jP3F5OsKWBsZL2TLquNLNDgCnAvmlcp5k9jv/YLpb0z2ZoATCz5VhwulVKxYsGVXxG75nZ1biR\n", 731 | "moOnfVxdK12ZVLkql+J0R5TcPano8ZspCTVImouHIqoian7XiKya34UqAenv3YG9gE/h7u9xwO7A\n", 732 | "HOBvxTGSKsljLevQ1CBmPH5HptFbfMMdcvm+w2NqAGY2VNKcVCXgAOABSfuZ2Sl4kHsi3s10vxS7\n", 733 | "CYKq6W6LzO+gD5JRWhk3QDcBc8ysG49TvIzHJn4YRimoJd3trfvzbl3lOSflblyAJyG+BRwDvIIH\n", 734 | "uH8FfBO4V9IBZjY+tpkEtaY7x2VNsgjDVAfSyts5wDL4svHKuHGaBXwXDyjuBcxOsafcGKXk2e2C\n", 735 | "Bzln4LGvq/HKmcvg2zJOBaZKqvn2mFbR1ArEJt6glAuBr+HbI57BV99+jP+YvgV8Af9xbVIIiOeI\n", 736 | "0fheval4JvPewFr4Vpnt8WlnJ/7e3hzEmnJPDdIFmkbrKs8xko7F40kHAM/hGclH4dsSJuG5MJtI\n", 737 | "erppIntnIp4w+ShuBM4DXkyJns/j7+NVSY00AHnUlHu62joyb3kl0gVqxIl7tBf2vhU25P4S+CLw\n", 738 | "Dl4Gdzc8vnQK8H9FlSnrSR6XwUNTg5jy7KOZ6QLLfXyjXL7v8JhqSDJKi+O72/+D7x+bgucr3Yvv\n", 739 | "Tp/TIKMUDHJa2WMKw1R7RuLZ25MlPYnHRlbCp2/rSfp3M8UF9cPMljaz65uto0B3W3vmLa9E8Lv2\n", 740 | "TMVLgBxrZl14M8qbgO9IerWpyoK6IukdvKVWLsizR5RFGKYakzZrnognT34P32byhWr3kQVBf6k2\n", 741 | "j6mS0rpZY8olv75cCyPpZeCreOfcMZKeaa6iYDBSgxjTvNK6+KLNBYUH+qjquRveWmyhMf0hPKY6\n", 742 | "kRL93mi2jmDw0l39QuMCpXXNrMfSugBmVqjquQlewbOnMWUTHlMQDFBq4DFVUlq3rzFlk+kxjZ71\n", 743 | "SH+fs85szxc/2e+CeA1gJS69q++ksEZz8OdymaISNIgarLr1t7TutIwxZZNpmJbYcPv+PmfdGb3G\n", 744 | "Ss2W0CM5NQS5MpaJ0JRN1SdTV/WG6UF818K1fZXWZcGqnt19jCmbTMM0/fE7K3neurHEhtvz3OT8\n", 745 | "pQKNXmMlLr0rX+d2MpR5s5Z5zLLOo6aq6e6u+i1VUlp3oTGVvHAEv4NggNJFU0rr9jSm34RhCoIB\n", 746 | "SlcLr22FYQqCAUoN0gWaRhimIBigdHWHxxQEQc5oZY+pdU1q0DRS6eAg53R1t2fe8kp+lQW5JLWi\n", 747 | "6k5dZoMc00Vb5i2vhGEKysLM2sxss9SK6sN4d9lPmFnr1tYY4ITHFAwGvgPcb2Y74vuhlgReiWqc\n", 748 | "+aWbtsxbXongd1Au1+GdSW7CS1ksCtxjZtcA7+L1eGY1UV9QQp49oixaV3nQEFLdHSQ9h3tNlwMn\n", 749 | "4f3cHgJ2xYuFLdMsjUHPdHe3Zd7ySnhMQa+kri9zzOyjwLHAk8A4vCrnwcB1kr5pZqMkzWym1mBh\n", 750 | "OnNseLIIwxT0iqROM1sReAR4FnhN0pVmNgyvZX6dmX0Mn8rVnJ5Kt/Zy3LHACpJOTf/fBzgRmA1c\n", 751 | "K+nCeuows2XxvncjgNeAgyS9Z2a74004u9O4i9Pxp+I78IcCv5B0ZTX6eiPPMaQswjAFWWwHvAJs\n", 752 | "lwzVqXgI4Higq16eUlHp1jF4a/UHzezG4qaWZjYCuAz4NB4Dw8yWAc7FW2ZNx+Ng90p6ol46gDOA\n", 753 | "qyVdZWYnA4cBP03jPoWXBXnWzH6f/r+JpE3NbFF8WlwX6hFjMrOReEfp5fDicF+T9FbJMYcAh+KG\n", 754 | "/PuSbkm5b68yfxPwQ5JO6+11IsYULICZDSn8m06mucB6wNbpkM3wdlSv1rkV1bzSrZLmAIXSrcWM\n", 755 | "AK4AzmF+2ZLVgackTUs73R/uYVytdcwrQYuXlf1c+nsOvnq5SJG+HYCnzeyP+ELCjVVo65Ou7uxb\n", 756 | "BRyBf75bAlfhccd5mNmHgKOBTfH3el4y7qsDj0vaJt16NUoQHlNQhJkNkTTXzD6Ce0T/B/wC2Be4\n", 757 | "zcweA9YBNk8/+nrSW+nWeUiaBvzZzL5WdPcLwDqpMP67eNPR8fXUUXLMu0WPX4A3P/0vcL2k6Wna\n", 758 | "9xG8S/NquGFauwp9vVKnVbnNgB+mv2/Hp7jFbAQ8mIz4HDObjF/YVgdWMrO7gfeA4yRNohfCMAXz\n", 759 | "SEbpo8BE3O1eCVgX2APYG/8B3iVJ9dJgZmcDm+Ptf4rrOo/Ce/b1iaSpZnYccD3wNv5e3up7VNU6\n", 760 | "ZuCfzZT0+LRk3I8CPopPAa82s72SluckzQUmmdlsM1u2dDpUC6ot9Gxm38AXPYr5D/NrevdkpEfR\n", 761 | "syF/DThX0vVmthk+Hdyot9cOwzTISdO1Y4Bf4tO2I4E/pf//HE8HWAn4kqQZvT1PrZB0etI1BI/L\n", 762 | "lJZu7ZM0boykLcxsOPBX5l/h66XjQWAn4Erg88B9+DSzE3g/tY5/E5/WPYB/3j9JGfSL4ga05nRW\n", 763 | "6TFJugyP4c0jdRoubEcq1PkuprTmd8GQP4efX0h6ML33XokYU7ATHqS9AI+DXA9MxoO3Z+KlU5cD\n", 764 | "lm2kqORRFEq3TmB+6dbe2nB3F43rNLPHgfuBS6ppNlqmju8DX0ktjD6Dr7S9gBuqCWZ2P+41XCHp\n", 765 | "FuAJM3sUn8YdWa9pcVd3W+atAgpGGOYb4WIeBbYws+FmtgQeo/sHvkBwLICZrYcvqPRKW3ff/l53\n", 766 | "1Pwuj1as+Z3ylDrN7CDcQxqLBzM/iidSvoHHQXaoYaA7j/W186ipam5/8oPME3LH9Yf1632nVbkr\n", 767 | "gRXxTrv7SnozTZ8nS7rJzA7GV+XagXMk3ZCM1NXAYrjn9M2IMQU9kozS8ngw8n/wQPdM/MR7FvgQ\n", 768 | "8NU6r76SLsiEAAAOHklEQVQFdaKrq/a2VtJ7wJd7uP/Cor8vBS4teXw6nrtVFmGYBiFpanEN7iU9\n", 769 | "hl/F/oUHNk/G4wJHA3MkzW6WzqA68lzWJIuIMQ0yzGwRfEn9p/hy71X4Uu7f8KnbSfgVcdEwSq1N\n", 770 | "V1db5i2vhMc0yJA0y8yOAd4ETgBeKlpifwlf8l670JM+D5jZysAueNbwDGB3PF7xKr55+HTgVGCq\n", 771 | "pPcHq6ZSKkygzAVhmAYhkmaa2XlAB3BCShl4CQ8AT8iTUUqMBg7Bl51vx3Oq1sKnndvjq0Sd+L60\n", 772 | "N3t5jsGgaQEqXHXLBWGYBikpC/l7+I/nJDxN4NOSXm6qsJ6ZiG+SXQY3AlOBNSVNNrNV8ETGdUr2\n", 773 | "rw1GTQvQyh5TpAvUiFZMFwAwsyXxQPc19czoLiKPS/N51FQ11zyUbZq+vEl7Lt93BL8HOWm/2TkN\n", 774 | "MkpBA6lTgmVDiKlcgKSuZmsIak9XC3+rYZiCYIDSyjGmMExBMEDJc55SFmGYgmCAEh5TEAS5ox4x\n", 775 | "pnJK66bjlsMrEawr6YNyxxWIVbkgGKA0o7QugJntANwJLN+fccVkekxvL7laPzTXnyWAia/3WWOq\n", 776 | "KYxeA5ZcLG/LIB3scvjzuXLob7y4LlVkgx7orE+P5KzSuuBJu9viZYX7M24emYZptdXXyDqk4ey3\n", 777 | "RT6Dentt3NFsCQuRU0OQK2OZyJumqk/yaqdyFZbWRdJdaXzx3cV10XscV0ymYfrni5OzDmkoq62+\n", 778 | "BuPuz9s55Mbyuofrc4mqlL027mCXw59vtowFSIYyb1eWAZn5XW3wu8LSur1RqIte1riIMQUDmrRB\n", 779 | "eVDS1dWdeauArNK6NRkXq3LBgKXQjqrZOppFnTK/fw1cmYoNvo+39qK4tG7Rsd1Z43ojDFMwIDGz\n", 780 | "oZLmmFkH3ixgFvDztDdwUNBZB8NUTmndovtWK/q7x3G9EVO5YECSjFI7Xpnzs3jrpJmphfigoLs7\n", 781 | "+5ZXwjAFAwozOyU1mwRv1T0c2FbSCcB+lARzBzKdnd2Zt7wShikYMJjZlni527dSnal/4+23v5EO\n", 782 | "WRkYYWaLNUliQwmPKQiajJm1S7oP75R7HfB7vMvtz/Cut/fhTRcvkfRu85Q2jvCYgqCJpNW3LpjX\n", 783 | "OfcqvO72wXi7853xrrfbSxpvZm2DIY2glT2mWJULWprkKc1Nq28/Al7H40g7ArcCSwHHSro1Hd8G\n", 784 | "UK+23Hkizx5RFmGYgpalqMV5G97AcxM8u3gLPK60M26czsXjTYPCIBXIqOefa2IqF7QsySi14zGl\n", 785 | "WXig+0i8gec4fBPpcpKeGAxTt1IixhQEDcTMij39JYFPAPsAG0q6Cg94G7CKpLebIDEX1KnsSUMI\n", 786 | "wxS0FIVtJmbWbmY7AkOBrYFHgF+b2aaSxgKbS3q8HjElM9vZzB41swlmdnAfxx2bGosW33ehmR1W\n", 787 | "Ky190dXZnXnLKxFjCloGM2srCnRPxKdufwdOAfYC7gCuMbN1SDGlOmgYCvwEGINPHx80sxuLG1ua\n", 788 | "2Qg8AP9pfJpZqOh4FbAm8Fw9tJVS4SbdXBCGKWgZiryeK4B/AsfjHYTPAH6M717/SHGL8zoEu0fj\n", 789 | "m1WnA5jZA8zPnSowImm8Ezee4DlVZ+I76xsS76qHR1RFad024FVgUjrkIUmn9fY6MZULck/ykAp/\n", 790 | "jwSeBu4FDgN+AHwcb3M+RNJDdc5TKi54Bj0UPZM0TdKfS+57WdKjddLUI51dXZm3Cqi0tO7qwOOS\n", 791 | "tkm3Xo0ShMcU5JyilIB2YFPgbeAiYH88vvQy8BRwhqSXoT4pAWZ2NrA58Ek8nlVgFDC11q9XC+o0\n", 792 | "lau0tO6GwEpmdjfwHnCcpEk9jAXCMAU5pjjQDTyM73X7ED51uwn4DZ7hvZ+kCfVMnpR0ekET8KyZ\n", 793 | "LQX8F5/GnV/r16sF1U7lalxa9zXgXEnXm9lm+HRwo95eOwxTkFuKjNJpeGziK8AeeIb3X/BA8jBJ\n", 794 | "zzUqoztpOh4PtLcDl0l63cyWBsZK2rNkSE96GhKV7qoywbLGpXUfA+am533QzPrsKBKGKcgdaZtJ\n", 795 | "IQByAvA94M/AO8BP8fIl60m6OB3f0G0mkm4Gbi657x1gz5L7ruxh7Hfrq24+XXPrUsKyUCL3b/Sv\n", 796 | "tO4Z+Pd3vpmtB7zS18FhmIJcUTR9a8Ozts83s+WBrwG/ACYAawDzuiwMpm0m/aFO2QKVltb9AXC1\n", 797 | "me2Ee04H9vUiYZiC3FCSp3Qv8KSZXSXpRDObC5wMrAh8XdK9hcB4MzXnma461NatorTudHzvYlmE\n", 798 | "YQpygZktL+nN5CndgwdLJwJHmtlM4BhgMeCLwLpmdk9fLaaDSLAMgqows5vxDO7TcI9oGTw7+iw8\n", 799 | "hvFV4DpJR6fEvb3waV3TMLOV8WqZk/BVqt3xlaZXcf2nA6cCUyW93wyN9fCYGkUYpiAPnAVMMrNv\n", 800 | "44l54/H4xDjgbtxLmg4g6StmtqKkKU3SWmA0cAiew3Q7sDewFj7d3B4PEHfixvbNXp6jrnS3sMcU\n", 801 | "md9B0yiqEvA4/qM+G/+x/xaPMa0J/AE4StJTaZ8akl5vvNqFmAj8DngUN0znAS9KmowH5o8CXi3e\n", 802 | "Q9doOju7Mm95JTymoCmY2V6SrkuB7j/iGd1n41tL2vH8me8B35X0TIo95aZ5ZSqnUpxY+feix+5O\n", 803 | "f17bUFEltHKhuDBMQcMxs+3wKgA/ADqAkcD6+H6qK/A9cCOBb0iaXRgXaQH9o7M+eUwNIQxT0Azu\n", 804 | "AY7Dc1tewHOUngZuwWMz1wPjwihVRyt7TBFjChpO6mTyKzyr2/C8pLn4KtdLeCvvWwdjOdxa0jm3\n", 805 | "M/OWV8IwBU1B0hxgLB5TOsTMfp6W1beRdF9KtmzdS34O6O7uzrzllTBMQdNIxumXeM7PN81s7Ywh\n", 806 | "QT/omtuVecsrEWMKmkragnIhcG2hnlK6P7+X8xahszO/U7UswjAFTSfFl16GefvlwijVgHokWJZT\n", 807 | "Wjdt6N07/fdWSd8rtyRvgZjKBbkijFLt6OzszLxVQJ+ldc1sNbziwCaSNga2N7NPZI0rJdNjWm31\n", 808 | "NSoRX1f22yKfizV7bdyRfVCDufHiXIZt8mh8mqqpu7ubtrYFzuuqT/I6bUnJKq37CrBD0QVmKDC7\n", 809 | "jHELkGmY7n76vTL1NobPfmIk7zz512bLWIil19+KR54vt5hfY/jM2ktyxT35sgEHbtPGj67PV+zj\n", 810 | "pD07oEGdSwoU6k4V/l9ilGpCV5UxpkpK66b39E5K9TgfmCjpBTMrbuLQY0neYiLGFAQNJtWRmmtm\n", 811 | "q+DVOBcDLgFeSyuVNaFaj6nS0rqpr97luCE6Mt09A+8w0+u4YiLGFAQNJnV9WR7fvLw4/uO9Ad/I\n", 812 | "TKpzXjV1ijEVSutCD6V1k6f0J+BJSUcUTen6HFdKeExB0CBKVhw3A/4B3AYcDPwLb3X+j6J651XR\n", 813 | "XVnfuCz6LK2L733cEhhqZp9PY07pbVxvhGEKggZQVMt8RNoD+CKwCV7e5RjgY8BOqZTwzFq8ZrUx\n", 814 | "pp4os7TuyF6GLzSuN8IwBUGdKYopfRS41MxeBK4BLgC+hef8jAa2rZVRgiitGwRBH6SY0tJ4VYUH\n", 815 | "gUck3Z3iTNvisaUHJL1Uy9ftyvEm3SzCMAVBnSiJKX0Uj62cmxp07g98WdLOLNhyvGZ0d+d3L1wW\n", 816 | "YZiCoA4UxZRGAsOAt/AW598ws98AnwaWNLNFJM2qh4b7btg8n5nIZRCGKQhqTFFM6WPAxfjv7FDg\n", 817 | "cLzUyy54guGO9TJKrU4YpiCoMSmm9GG8UcF9eGD7YWADYG1gJbxRwf81T2W+iQTLIKgRqbECqZvL\n", 818 | "2sADwI+Af+L7xV4B1pP0UBilvgnDFAQ1IMWUCp7Sw3hM6Va8rvkv8Qadt+L5S0EGMZULgioxs/YU\n", 819 | "U/owXtLjSUl/N7PJwD7ApcBwYIykfzVTa6sQHlMQVEEKdHelPKXPA58FhqXp3BzgStxYbR1GqXzC\n", 820 | "MAVBhSSjVNiQ+zTwX7y5wn7ACZLmSLoK+LakfzRTa6sRU7kgqJAio3QP3jL8JtxLGgqca2ZzJF0Q\n", 821 | "VTn7T3hMQdBPCqtvieHAIsDGwCclfQBcCJyIVw4IKiA8piDoB0UZ3cvhzTrfwsuVXAf8yswOk/Qo\n", 822 | "vkE3qJDwmIKgHxRVCZgInAncjBfa3wtYAbjQzIY3UeKAIDymIOgHqULjiXhO0h+B8cBq+G9pY2Bo\n", 823 | "6igcVEF4TEGQgZkNS/8WqgW8B3wOTwM4CejC9729IikSKGtAGKYg6AMzW1bSB2a2EnComW2A5yat\n", 824 | "im8zWR/PX7q7iTIHHDGVC4JeMLOrgS4z+xHwB7yL7HLAdviG3G/iNa63kPRc04QOQMJjCoLemQZs\n", 825 | "CPwUX/rfBJ++3QmsJekQ4GBJf2+exIFJGKYgKMHMNkhB7hOAO4B1caP0Dt4A8gZgrJktQT67Crc8\n", 826 | "MZULgiLSHrdDgU/gQe7D8M6xX8XbWp8DHAKMlDS9t+cJqiM8piBIpL1vc/CWSp/BPaXXJZ2Jx5gO\n", 827 | "AE4Fpkt6rWlCBwFhmIKABeopLQV8Eu8muxRwiZktC/wWL19yca0aUga9E1O5YNDT1dVVyOheFbgd\n", 828 | "34j7Bl6Bcl/gC3gnky/Vsu9b0DvhMQWDnvb2dlI3k0vxfKStgb8Di+KF3sYDJ4dRahxhmILAaQeW\n", 829 | "ByZLegv4Lr737QVJ35D0VFPVDTJiKhcEzvt4sbdDzOwDPJFyCDClqaoGKeExBQFeNQD4Nm6cvgVs\n", 830 | "D+wa3UyaQ3hMQZCQ9JKZ7YcXfkPStCZLGrSEYQqCIlIFyg+arWOw09bdHRn1QRDki4gxBUGQO8Iw\n", 831 | "BUGQO8IwBUGQO8IwBUGQO8IwBUGQO8IwBUGQO8IwBUGQO/4/XMeBCWj26M0AAAAASUVORK5CYII=\n" 832 | ], 833 | "text/plain": [ 834 | "" 835 | ] 836 | }, 837 | "metadata": {}, 838 | "output_type": "display_data" 839 | } 840 | ], 841 | "source": [ 842 | "sns.corrplot(dee_df[['register','college','distance', 'black','hispanic','otherrace']])" 843 | ] 844 | }, 845 | { 846 | "cell_type": "markdown", 847 | "metadata": {}, 848 | "source": [ 849 | "## Two-Stage Least Squares Regression of REGISTER ~ COLLEGE + BLACK + HISPANIC + OTHERRACE where IV=DISTANCE" 850 | ] 851 | }, 852 | { 853 | "cell_type": "code", 854 | "execution_count": 36, 855 | "metadata": { 856 | "collapsed": false 857 | }, 858 | "outputs": [ 859 | { 860 | "name": "stdout", 861 | "output_type": "stream", 862 | "text": [ 863 | "==============================================================================\n", 864 | " FIRST STAGE\n", 865 | "==============================================================================\n", 866 | " OLS Regression Results \n", 867 | "==============================================================================\n", 868 | "Dep. Variable: college R-squared: 0.022\n", 869 | "Model: OLS Adj. R-squared: 0.021\n", 870 | "Method: Least Squares F-statistic: 51.24\n", 871 | "Date: Wed, 18 Mar 2015 Prob (F-statistic): 9.75e-43\n", 872 | "Time: 22:15:20 Log-Likelihood: -6554.4\n", 873 | "No. Observations: 9227 AIC: 1.312e+04\n", 874 | "Df Residuals: 9222 BIC: 1.315e+04\n", 875 | "Df Model: 4 \n", 876 | "Covariance Type: nonrobust \n", 877 | "==============================================================================\n", 878 | " coef std err t P>|t| [95.0% Conf. Int.]\n", 879 | "------------------------------------------------------------------------------\n", 880 | "Intercept 0.6431 0.009 71.039 0.000 0.625 0.661\n", 881 | "distance -0.0069 0.001 -11.636 0.000 -0.008 -0.006\n", 882 | "black -0.0577 0.016 -3.613 0.000 -0.089 -0.026\n", 883 | "hispanic -0.1162 0.013 -8.766 0.000 -0.142 -0.090\n", 884 | "otherrace 0.0337 0.024 1.404 0.160 -0.013 0.081\n", 885 | "==============================================================================\n", 886 | "Omnibus: 53.726 Durbin-Watson: 1.715\n", 887 | "Prob(Omnibus): 0.000 Jarque-Bera (JB): 1408.886\n", 888 | "Skew: -0.188 Prob(JB): 1.16e-306\n", 889 | "Kurtosis: 1.123 Cond. No. 62.3\n", 890 | "==============================================================================\n", 891 | "\n", 892 | "Warnings:\n", 893 | "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n", 894 | "\n", 895 | "\n", 896 | "==============================================================================\n", 897 | " SECOND STAGE\n", 898 | "==============================================================================\n", 899 | " OLS Regression Results \n", 900 | "==============================================================================\n", 901 | "Dep. Variable: register R-squared: 0.005\n", 902 | "Model: OLS Adj. R-squared: 0.004\n", 903 | "Method: Least Squares F-statistic: 10.53\n", 904 | "Date: Wed, 18 Mar 2015 Prob (F-statistic): 1.65e-08\n", 905 | "Time: 22:15:20 Log-Likelihood: -6103.0\n", 906 | "No. Observations: 9227 AIC: 1.222e+04\n", 907 | "Df Residuals: 9222 BIC: 1.225e+04\n", 908 | "Df Model: 4 \n", 909 | "Covariance Type: nonrobust \n", 910 | "==================================================================================\n", 911 | " coef std err t P>|t| [95.0% Conf. Int.]\n", 912 | "----------------------------------------------------------------------------------\n", 913 | "Intercept 0.5266 0.047 11.200 0.000 0.434 0.619\n", 914 | "college_fitted 0.2489 0.082 3.041 0.002 0.088 0.409\n", 915 | "black 0.0617 0.015 4.007 0.000 0.032 0.092\n", 916 | "hispanic 0.0283 0.015 1.879 0.060 -0.001 0.058\n", 917 | "otherrace -0.1067 0.023 -4.604 0.000 -0.152 -0.061\n", 918 | "==============================================================================\n", 919 | "Omnibus: 654.082 Durbin-Watson: 1.939\n", 920 | "Prob(Omnibus): 0.000 Jarque-Bera (JB): 1619.291\n", 921 | "Skew: -0.723 Prob(JB): 0.00\n", 922 | "Kurtosis: 1.543 Cond. No. 22.6\n", 923 | "==============================================================================\n", 924 | "\n", 925 | "Warnings:\n", 926 | "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n" 927 | ] 928 | } 929 | ], 930 | "source": [ 931 | "print \"==============================================================================\"\n", 932 | "print \" FIRST STAGE\"\n", 933 | "print \"==============================================================================\"\n", 934 | "result = smf.ols(formula = \"college ~ distance + black + hispanic + otherrace\", data = dee_df).fit()\n", 935 | "print result.summary()\n", 936 | "dee_df['college_fitted'] = result.predict()\n", 937 | "\n", 938 | "print\n", 939 | "print\n", 940 | "print \"==============================================================================\"\n", 941 | "print \" SECOND STAGE\"\n", 942 | "print \"==============================================================================\"\n", 943 | "\n", 944 | "result = smf.ols(formula = \"register ~ college_fitted + black + hispanic + otherrace\", data=dee_df).fit()\n", 945 | "print result.summary()" 946 | ] 947 | }, 948 | { 949 | "cell_type": "markdown", 950 | "metadata": {}, 951 | "source": [ 952 | "# Interactions Between the Endogenous Question Predictor and Exogenous Covariates in the Second Stage Model" 953 | ] 954 | }, 955 | { 956 | "cell_type": "markdown", 957 | "metadata": {}, 958 | "source": [ 959 | "In this case, we explore whether interactions between college and race/ethnicity are significant predictors of voter registration. Here, it's important to meet the \"rank condition\": that \"for every endogenous predictor included in the second stage, there must be at least one instrument included in the first stage.\"\n", 960 | "\n", 961 | "To do this, we need to create a series of stage-one instruments, one for the main effect, and one for each interaction. In " 962 | ] 963 | }, 964 | { 965 | "cell_type": "code", 966 | "execution_count": 37, 967 | "metadata": { 968 | "collapsed": false 969 | }, 970 | "outputs": [ 971 | { 972 | "name": "stdout", 973 | "output_type": "stream", 974 | "text": [ 975 | "==============================================================================\n", 976 | " FIRST STAGE\n", 977 | "==============================================================================\n", 978 | " OLS Regression Results \n", 979 | "==============================================================================\n", 980 | "Dep. Variable: college R-squared: 0.022\n", 981 | "Model: OLS Adj. R-squared: 0.021\n", 982 | "Method: Least Squares F-statistic: 29.57\n", 983 | "Date: Wed, 18 Mar 2015 Prob (F-statistic): 1.12e-40\n", 984 | "Time: 22:15:20 Log-Likelihood: -6553.3\n", 985 | "No. Observations: 9227 AIC: 1.312e+04\n", 986 | "Df Residuals: 9219 BIC: 1.318e+04\n", 987 | "Df Model: 7 \n", 988 | "Covariance Type: nonrobust \n", 989 | "======================================================================================\n", 990 | " coef std err t P>|t| [95.0% Conf. Int.]\n", 991 | "--------------------------------------------------------------------------------------\n", 992 | "Intercept 0.6452 0.010 64.147 0.000 0.625 0.665\n", 993 | "distance -0.0071 0.001 -9.830 0.000 -0.009 -0.006\n", 994 | "black -0.0651 0.023 -2.858 0.004 -0.110 -0.020\n", 995 | "hispanic -0.1276 0.019 -6.588 0.000 -0.166 -0.090\n", 996 | "otherrace 0.0590 0.035 1.681 0.093 -0.010 0.128\n", 997 | "distance:black 0.0009 0.002 0.442 0.658 -0.003 0.005\n", 998 | "distance:hispanic 0.0013 0.002 0.818 0.413 -0.002 0.004\n", 999 | "distance:otherrace -0.0030 0.003 -1.018 0.309 -0.009 0.003\n", 1000 | "==============================================================================\n", 1001 | "Omnibus: 53.396 Durbin-Watson: 1.715\n", 1002 | "Prob(Omnibus): 0.000 Jarque-Bera (JB): 1407.217\n", 1003 | "Skew: -0.188 Prob(JB): 2.67e-306\n", 1004 | "Kurtosis: 1.124 Cond. No. 93.0\n", 1005 | "==============================================================================\n", 1006 | "\n", 1007 | "Warnings:\n", 1008 | "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n", 1009 | " OLS Regression Results \n", 1010 | "==============================================================================\n", 1011 | "Dep. Variable: register R-squared: 0.005\n", 1012 | "Model: OLS Adj. R-squared: 0.004\n", 1013 | "Method: Least Squares F-statistic: 6.691\n", 1014 | "Date: Wed, 18 Mar 2015 Prob (F-statistic): 6.29e-08\n", 1015 | "Time: 22:15:20 Log-Likelihood: -6100.6\n", 1016 | "No. Observations: 9227 AIC: 1.222e+04\n", 1017 | "Df Residuals: 9219 BIC: 1.227e+04\n", 1018 | "Df Model: 7 \n", 1019 | "Covariance Type: nonrobust \n", 1020 | "=====================================================================================\n", 1021 | " coef std err t P>|t| [95.0% Conf. Int.]\n", 1022 | "-------------------------------------------------------------------------------------\n", 1023 | "Intercept 0.4640 0.056 8.360 0.000 0.355 0.573\n", 1024 | "college_fitted 0.3587 0.097 3.703 0.000 0.169 0.549\n", 1025 | "black 0.2780 0.163 1.702 0.089 -0.042 0.598\n", 1026 | "hispanic 0.1765 0.121 1.456 0.145 -0.061 0.414\n", 1027 | "otherrace 0.1742 0.176 0.988 0.323 -0.171 0.520\n", 1028 | "collegeXblack -0.3986 0.303 -1.315 0.189 -0.993 0.196\n", 1029 | "collegeXhispanic -0.2929 0.249 -1.177 0.239 -0.781 0.195\n", 1030 | "collegeXotherrace -0.4634 0.285 -1.623 0.105 -1.023 0.096\n", 1031 | "==============================================================================\n", 1032 | "Omnibus: 653.574 Durbin-Watson: 1.940\n", 1033 | "Prob(Omnibus): 0.000 Jarque-Bera (JB): 1616.409\n", 1034 | "Skew: -0.723 Prob(JB): 0.00\n", 1035 | "Kurtosis: 1.545 Cond. No. 88.9\n", 1036 | "==============================================================================\n", 1037 | "\n", 1038 | "Warnings:\n", 1039 | "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n" 1040 | ] 1041 | } 1042 | ], 1043 | "source": [ 1044 | "print \"==============================================================================\"\n", 1045 | "print \" FIRST STAGE\"\n", 1046 | "print \"==============================================================================\"\n", 1047 | "# generate the stage one main effect instrument\n", 1048 | "result = smf.ols(formula = \"college ~ distance + black + hispanic + otherrace +\" +\n", 1049 | " \"distance:black + distance:hispanic + distance:otherrace\", data = dee_df).fit()\n", 1050 | "dee_df['college_fitted'] = result.predict()\n", 1051 | "print result.summary()\n", 1052 | "\n", 1053 | "# generate the stage one interaction instrument for distance:black\n", 1054 | "# note that we have DROPPED the irrelevant terms. \n", 1055 | "# The full form for each interaction, which gives the exact same result, is:\n", 1056 | "# result = smf.ols(formula = \"college:black ~ distance + black + hispanic + otherrace +\" +\n", 1057 | "# \"distance:black + distance:hispanic + distance:otherrace\", data = dee_df).fit()\n", 1058 | "\n", 1059 | "result = smf.ols(formula = \"college:black ~ distance + black + distance:black\", data = dee_df).fit()\n", 1060 | "dee_df['collegeXblack'] = result.predict()\n", 1061 | "\n", 1062 | "\n", 1063 | "# generate the stage one interaction instrument for distance:hispanic\n", 1064 | "result = smf.ols(formula = \"college:hispanic ~ distance + hispanic + distance:hispanic\", data = dee_df).fit()\n", 1065 | "dee_df['collegeXhispanic'] = result.predict()\n", 1066 | "\n", 1067 | "# generate the stage one interaction instrument for distance:hispanic\n", 1068 | "result = smf.ols(formula = \"college:otherrace ~ distance + otherrace + distance:otherrace\", data = dee_df).fit()\n", 1069 | "dee_df['collegeXotherrace'] = result.predict()\n", 1070 | "\n", 1071 | "# generate the final model, that includes these interactions as predictors\n", 1072 | "result = smf.ols(formula = \"register ~ college_fitted + black + hispanic + otherrace +\" +\n", 1073 | " \"collegeXblack + collegeXhispanic + collegeXotherrace\", data = dee_df).fit()\n", 1074 | "print result.summary()" 1075 | ] 1076 | }, 1077 | { 1078 | "cell_type": "markdown", 1079 | "metadata": {}, 1080 | "source": [ 1081 | "^^^ in this particular case, we find no significant interactions and fall back on our previous model, which simply included race/ethnicity as a covariate" 1082 | ] 1083 | }, 1084 | { 1085 | "cell_type": "markdown", 1086 | "metadata": {}, 1087 | "source": [ 1088 | "# Binomial Regression: Logistic Model" 1089 | ] 1090 | }, 1091 | { 1092 | "cell_type": "markdown", 1093 | "metadata": {}, 1094 | "source": [ 1095 | "In this example, we use a logistic model with a two-stage least-squares regression. NOTE: This is not attempted in the textbook example, so I cannot be completely certain about this, unlike the above results." 1096 | ] 1097 | }, 1098 | { 1099 | "cell_type": "code", 1100 | "execution_count": 38, 1101 | "metadata": { 1102 | "collapsed": false 1103 | }, 1104 | "outputs": [ 1105 | { 1106 | "name": "stdout", 1107 | "output_type": "stream", 1108 | "text": [ 1109 | "==============================================================================\n", 1110 | " FIRST STAGE\n", 1111 | "==============================================================================\n", 1112 | " Generalized Linear Model Regression Results \n", 1113 | "==============================================================================\n", 1114 | "Dep. Variable: college No. Observations: 9227\n", 1115 | "Model: GLM Df Residuals: 9222\n", 1116 | "Model Family: Binomial Df Model: 4\n", 1117 | "Link Function: logit Scale: 1.0\n", 1118 | "Method: IRLS Log-Likelihood: -6253.9\n", 1119 | "Date: Wed, 18 Mar 2015 Deviance: 12508.\n", 1120 | "Time: 22:15:20 Pearson chi2: 9.23e+03\n", 1121 | "No. Iterations: 6 \n", 1122 | "==============================================================================\n", 1123 | " coef std err z P>|z| [95.0% Conf. Int.]\n", 1124 | "------------------------------------------------------------------------------\n", 1125 | "Intercept 0.5840 0.038 15.447 0.000 0.510 0.658\n", 1126 | "distance -0.0283 0.002 -11.456 0.000 -0.033 -0.023\n", 1127 | "black -0.2372 0.066 -3.621 0.000 -0.366 -0.109\n", 1128 | "hispanic -0.4744 0.055 -8.699 0.000 -0.581 -0.368\n", 1129 | "otherrace 0.1426 0.101 1.414 0.157 -0.055 0.340\n", 1130 | "==============================================================================\n", 1131 | "\n", 1132 | "\n", 1133 | "==============================================================================\n", 1134 | " SECOND STAGE\n", 1135 | "==============================================================================\n", 1136 | " Generalized Linear Model Regression Results \n", 1137 | "==============================================================================\n", 1138 | "Dep. Variable: register No. Observations: 9227\n", 1139 | "Model: GLM Df Residuals: 9222\n", 1140 | "Model Family: Binomial Df Model: 4\n", 1141 | "Link Function: logit Scale: 1.0\n", 1142 | "Method: IRLS Log-Likelihood: -5825.1\n", 1143 | "Date: Wed, 18 Mar 2015 Deviance: 11650.\n", 1144 | "Time: 22:15:20 Pearson chi2: 9.23e+03\n", 1145 | "No. Iterations: 6 \n", 1146 | "==================================================================================\n", 1147 | " coef std err z P>|z| [95.0% Conf. Int.]\n", 1148 | "----------------------------------------------------------------------------------\n", 1149 | "Intercept 0.0572 0.211 0.272 0.786 -0.356 0.470\n", 1150 | "college_fitted 1.1312 0.367 3.079 0.002 0.411 1.851\n", 1151 | "black 0.2899 0.073 3.995 0.000 0.148 0.432\n", 1152 | "hispanic 0.1284 0.068 1.886 0.059 -0.005 0.262\n", 1153 | "otherrace -0.4589 0.101 -4.566 0.000 -0.656 -0.262\n", 1154 | "==================================================================================\n" 1155 | ] 1156 | } 1157 | ], 1158 | "source": [ 1159 | "print \"==============================================================================\"\n", 1160 | "print \" FIRST STAGE\"\n", 1161 | "print \"==============================================================================\"\n", 1162 | "result = smf.glm(formula = \"college ~ distance + black + hispanic + otherrace\", \n", 1163 | " data=dee_df,\n", 1164 | " family=sm.families.Binomial()).fit()\n", 1165 | "print result.summary()\n", 1166 | "dee_df['college_fitted'] = result.predict()\n", 1167 | "\n", 1168 | "print\n", 1169 | "print\n", 1170 | "print \"==============================================================================\"\n", 1171 | "print \" SECOND STAGE\"\n", 1172 | "print \"==============================================================================\"#\n", 1173 | "result = smf.glm(formula = \"register ~ college_fitted + black + hispanic + otherrace\",\n", 1174 | " data=dee_df,\n", 1175 | " family=sm.families.Binomial()).fit()\n", 1176 | "print result.summary()" 1177 | ] 1178 | }, 1179 | { 1180 | "cell_type": "markdown", 1181 | "metadata": {}, 1182 | "source": [ 1183 | "# Binomial Regression: Probit Model" 1184 | ] 1185 | }, 1186 | { 1187 | "cell_type": "markdown", 1188 | "metadata": {}, 1189 | "source": [ 1190 | "In this example, we use a probit model with a two-stage least-squares regression. NOTE: This is not attempted in the textbook example, so I cannot be completely certain about this, unlike the above results." 1191 | ] 1192 | }, 1193 | { 1194 | "cell_type": "code", 1195 | "execution_count": 39, 1196 | "metadata": { 1197 | "collapsed": false 1198 | }, 1199 | "outputs": [ 1200 | { 1201 | "name": "stdout", 1202 | "output_type": "stream", 1203 | "text": [ 1204 | "==============================================================================\n", 1205 | " FIRST STAGE\n", 1206 | "==============================================================================\n", 1207 | "Optimization terminated successfully.\n", 1208 | " Current function value: 0.677791\n", 1209 | " Iterations 4\n", 1210 | " Probit Regression Results \n", 1211 | "==============================================================================\n", 1212 | "Dep. Variable: college No. Observations: 9227\n", 1213 | "Model: Probit Df Residuals: 9222\n", 1214 | "Method: MLE Df Model: 4\n", 1215 | "Date: Wed, 18 Mar 2015 Pseudo R-squ.: 0.01585\n", 1216 | "Time: 22:15:20 Log-Likelihood: -6254.0\n", 1217 | "converged: True LL-Null: -6354.7\n", 1218 | " LLR p-value: 1.862e-42\n", 1219 | "==============================================================================\n", 1220 | " coef std err z P>|z| [95.0% Conf. Int.]\n", 1221 | "------------------------------------------------------------------------------\n", 1222 | "Intercept 0.3640 0.023 15.566 0.000 0.318 0.410\n", 1223 | "distance -0.0176 0.002 -11.504 0.000 -0.021 -0.015\n", 1224 | "black -0.1474 0.041 -3.607 0.000 -0.227 -0.067\n", 1225 | "hispanic -0.2959 0.034 -8.709 0.000 -0.363 -0.229\n", 1226 | "otherrace 0.0888 0.062 1.423 0.155 -0.033 0.211\n", 1227 | "==============================================================================\n", 1228 | "\n", 1229 | "\n", 1230 | "==============================================================================\n", 1231 | " SECOND STAGE\n", 1232 | "==============================================================================\n", 1233 | "Optimization terminated successfully.\n", 1234 | " Current function value: 0.631312\n", 1235 | " Iterations 4\n", 1236 | " Probit Regression Results \n", 1237 | "==============================================================================\n", 1238 | "Dep. Variable: register No. Observations: 9227\n", 1239 | "Model: Probit Df Residuals: 9222\n", 1240 | "Method: MLE Df Model: 4\n", 1241 | "Date: Wed, 18 Mar 2015 Pseudo R-squ.: 0.003563\n", 1242 | "Time: 22:15:20 Log-Likelihood: -5825.1\n", 1243 | "converged: True LL-Null: -5845.9\n", 1244 | " LLR p-value: 1.962e-08\n", 1245 | "==================================================================================\n", 1246 | " coef std err z P>|z| [95.0% Conf. Int.]\n", 1247 | "----------------------------------------------------------------------------------\n", 1248 | "Intercept 0.0427 0.129 0.330 0.742 -0.211 0.296\n", 1249 | "college_fitted 0.6903 0.226 3.060 0.002 0.248 1.133\n", 1250 | "black 0.1752 0.044 4.021 0.000 0.090 0.261\n", 1251 | "hispanic 0.0782 0.042 1.879 0.060 -0.003 0.160\n", 1252 | "otherrace -0.2834 0.063 -4.533 0.000 -0.406 -0.161\n", 1253 | "==================================================================================\n" 1254 | ] 1255 | } 1256 | ], 1257 | "source": [ 1258 | "import patsy\n", 1259 | "print \"==============================================================================\"\n", 1260 | "print \" FIRST STAGE\"\n", 1261 | "print \"==============================================================================\"\n", 1262 | "a,b = patsy.dmatrices(\"college ~ distance + black + hispanic + otherrace\",\n", 1263 | " dee_df,return_type=\"dataframe\")\n", 1264 | "result = sm.Probit(a,b).fit()\n", 1265 | "print result.summary()\n", 1266 | "dee_df['college_fitted'] = result.predict()\n", 1267 | "\n", 1268 | "\n", 1269 | "print\n", 1270 | "print\n", 1271 | "print \"==============================================================================\"\n", 1272 | "print \" SECOND STAGE\"\n", 1273 | "print \"==============================================================================\"#\n", 1274 | "\n", 1275 | "a,b = patsy.dmatrices(\"register ~ college_fitted + black + hispanic + otherrace\",\n", 1276 | " dee_df,return_type=\"dataframe\")\n", 1277 | "result = sm.Probit(a,b).fit()\n", 1278 | "\n", 1279 | "print result.summary()" 1280 | ] 1281 | }, 1282 | { 1283 | "cell_type": "code", 1284 | "execution_count": null, 1285 | "metadata": { 1286 | "collapsed": true 1287 | }, 1288 | "outputs": [], 1289 | "source": [] 1290 | } 1291 | ], 1292 | "metadata": { 1293 | "kernelspec": { 1294 | "display_name": "Python 2", 1295 | "language": "python", 1296 | "name": "python2" 1297 | }, 1298 | "language_info": { 1299 | "codemirror_mode": { 1300 | "name": "ipython", 1301 | "version": 2 1302 | }, 1303 | "file_extension": ".py", 1304 | "mimetype": "text/x-python", 1305 | "name": "python", 1306 | "nbconvert_exporter": "python", 1307 | "pygments_lexer": "ipython2", 1308 | "version": "2.7.6" 1309 | } 1310 | }, 1311 | "nbformat": 4, 1312 | "nbformat_minor": 0 1313 | } 1314 | -------------------------------------------------------------------------------- /instrumental_variables_estimation/Using IVE to Recover the Treatment Effect.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": { 7 | "collapsed": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "# RESEARCH IN PYTHON: USING IVE TO RECOVER THE TREATMENT EFFECT\n", 12 | "# by J. NATHAN MATIAS March 18, 2015\n", 13 | "\n", 14 | "# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n", 15 | "# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n", 16 | "# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n", 17 | "# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n", 18 | "# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n", 19 | "# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\n", 20 | "# THE SOFTWARE." 21 | ] 22 | }, 23 | { 24 | "cell_type": "markdown", 25 | "metadata": {}, 26 | "source": [ 27 | "# Using Instrumental-Variables Estimation to Recover the Treatment Effect in Quasi-Experiments " 28 | ] 29 | }, 30 | { 31 | "cell_type": "markdown", 32 | "metadata": {}, 33 | "source": [ 34 | "This section is taken from [Chapter 11](http://www.ats.ucla.edu/stat/stata/examples/methods_matter/chapter11/default.htm) of [Methods Matter](http://www.ats.ucla.edu/stat/examples/methods_matter/) by Richard Murnane and John Willett. \n", 35 | "\n", 36 | "In Chapter 10, Murnane and Willett introduce instrumental variables estimation(IVE) as a method for carving out causal claims from observational data ([chapter summary](http://acawiki.org/Introducing_Instrumental-Variables_Estimation)) ([example code](http://nbviewer.ipython.org/github/natematias/research_in_python/blob/master/instrumental_variables_estimation/Instrumental-Variables%20Estimation.ipynb)). \n", 37 | "\n", 38 | "In Chapter 11, the authors explain how IVE can be used to \"recover\" the treatment effect in cases where random assignment is applied to an offer to participate, where not everyone takes the offer, and where other people participate through some other means. They use the example of research on the effectiveness of a financial aid offer on the likelihood of a student to finish 8th grade, using a subset of data from Bogotá from a study on \"[Vouchers for Private Schooling in Columbia](http://www.nber.org/papers/w8343)\" (2002) by Joshua Angrist, Eric Bettinger, Erik Bloom, Elizabeth King, and Michael Kremer ([full data here](http://economics.mit.edu/faculty/angrist/data1/data/angetal02), [subset data here](http://www.ats.ucla.edu/stat/stata/examples/methods_matter/chapter11/default.htm)).\n", 39 | "\n", 40 | "The dataset includes the following variables:\n", 41 | "* *finish8th*: did the student finish 8th grade or not (outcome variable)\n", 42 | "* *won_lottry*: won the lottery to receive offer of financial aid\n", 43 | "* *use_fin_aid*: did the student use financial aid of any kind (not exclusive to the lottery) or not\n", 44 | "* *base_age*: student age\n", 45 | "* *male*: is the student male or not" 46 | ] 47 | }, 48 | { 49 | "cell_type": "code", 50 | "execution_count": 1, 51 | "metadata": { 52 | "collapsed": true 53 | }, 54 | "outputs": [], 55 | "source": [ 56 | "# THINGS TO IMPORT\n", 57 | "# This is a baseline set of libraries I import by default if I'm rushed for time.\n", 58 | "\n", 59 | "import codecs # load UTF-8 Content\n", 60 | "import json # load JSON files\n", 61 | "import pandas as pd # Pandas handles dataframes\n", 62 | "import numpy as np # Numpy handles lots of basic maths operations\n", 63 | "import matplotlib.pyplot as plt # Matplotlib for plotting\n", 64 | "import seaborn as sns # Seaborn for beautiful plots\n", 65 | "from dateutil import * # I prefer dateutil for parsing dates\n", 66 | "import math # transformations\n", 67 | "import statsmodels.formula.api as smf # for doing statistical regression\n", 68 | "import statsmodels.api as sm # access to the wider statsmodels library, including R datasets\n", 69 | "from collections import Counter # Counter is useful for grouping and counting\n", 70 | "import scipy" 71 | ] 72 | }, 73 | { 74 | "cell_type": "markdown", 75 | "metadata": {}, 76 | "source": [ 77 | "# Acquire Dataset from Methods Matter" 78 | ] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "execution_count": 2, 83 | "metadata": { 84 | "collapsed": true 85 | }, 86 | "outputs": [], 87 | "source": [ 88 | "import urllib2\n", 89 | "import os.path\n", 90 | "if(os.path.isfile(\"colombia_voucher.dta\")!=True):\n", 91 | " response = urllib2.urlopen(\"http://www.ats.ucla.edu/stat/stata/examples/methods_matter/chapter11/colombia_voucher.dta\")\n", 92 | " if(response.getcode()==200):\n", 93 | " f = open(\"colombia_voucher.dta\",\"w\")\n", 94 | " f.write(response.read())\n", 95 | " f.close()\n", 96 | "voucher_df = pd.read_stata(\"colombia_voucher.dta\")" 97 | ] 98 | }, 99 | { 100 | "cell_type": "markdown", 101 | "metadata": {}, 102 | "source": [ 103 | "# Summary Statistics" 104 | ] 105 | }, 106 | { 107 | "cell_type": "code", 108 | "execution_count": 12, 109 | "metadata": { 110 | "collapsed": false 111 | }, 112 | "outputs": [ 113 | { 114 | "name": "stdout", 115 | "output_type": "stream", 116 | "text": [ 117 | "==============================================================================\n", 118 | " OVERALL SUMMARY\n", 119 | "==============================================================================\n", 120 | " id won_lottry male base_age finish8th \\\n", 121 | "count 1171.000000 1171.000000 1171.000000 1171.000000 1171.000000 \n", 122 | "mean 1357.010248 0.505551 0.504697 12.004270 0.681469 \n", 123 | "std 890.711584 0.500183 0.500192 1.347038 0.466106 \n", 124 | "min 3.000000 0.000000 0.000000 7.000000 0.000000 \n", 125 | "25% 616.000000 0.000000 0.000000 11.000000 0.000000 \n", 126 | "50% 1280.000000 1.000000 1.000000 12.000000 1.000000 \n", 127 | "75% 1982.500000 1.000000 1.000000 13.000000 1.000000 \n", 128 | "max 4030.000000 1.000000 1.000000 17.000000 1.000000 \n", 129 | "\n", 130 | " use_fin_aid \n", 131 | "count 1171.000000 \n", 132 | "mean 0.581554 \n", 133 | "std 0.493515 \n", 134 | "min 0.000000 \n", 135 | "25% 0.000000 \n", 136 | "50% 1.000000 \n", 137 | "75% 1.000000 \n", 138 | "max 1.000000 \n", 139 | "==============================================================================\n", 140 | " LOTTERY = 0\n", 141 | "==============================================================================\n", 142 | " id won_lottry male base_age finish8th \\\n", 143 | "count 579.000000 579 579.000000 579.000000 579.000000 \n", 144 | "mean 1460.998273 0 0.504318 12.036269 0.625216 \n", 145 | "std 960.839468 0 0.500414 1.351814 0.484486 \n", 146 | "min 4.000000 0 0.000000 7.000000 0.000000 \n", 147 | "25% 650.500000 0 0.000000 11.000000 0.000000 \n", 148 | "50% 1392.000000 0 1.000000 12.000000 1.000000 \n", 149 | "75% 2122.500000 0 1.000000 13.000000 1.000000 \n", 150 | "max 4030.000000 0 1.000000 16.000000 1.000000 \n", 151 | "\n", 152 | " use_fin_aid \n", 153 | "count 579.000000 \n", 154 | "mean 0.240069 \n", 155 | "std 0.427495 \n", 156 | "min 0.000000 \n", 157 | "25% 0.000000 \n", 158 | "50% 0.000000 \n", 159 | "75% 0.000000 \n", 160 | "max 1.000000 \n", 161 | "==============================================================================\n", 162 | " LOTTERY = 1\n", 163 | "==============================================================================\n", 164 | " id won_lottry male base_age finish8th \\\n", 165 | "count 592.000000 592 592.000000 592.000000 592.000000 \n", 166 | "mean 1255.305743 1 0.505068 11.972973 0.736486 \n", 167 | "std 804.217066 0 0.500397 1.342755 0.440911 \n", 168 | "min 3.000000 1 0.000000 9.000000 0.000000 \n", 169 | "25% 578.750000 1 0.000000 11.000000 0.000000 \n", 170 | "50% 1210.000000 1 1.000000 12.000000 1.000000 \n", 171 | "75% 1707.250000 1 1.000000 13.000000 1.000000 \n", 172 | "max 4006.000000 1 1.000000 17.000000 1.000000 \n", 173 | "\n", 174 | " use_fin_aid \n", 175 | "count 592.000000 \n", 176 | "mean 0.915541 \n", 177 | "std 0.278311 \n", 178 | "min 0.000000 \n", 179 | "25% 1.000000 \n", 180 | "50% 1.000000 \n", 181 | "75% 1.000000 \n", 182 | "max 1.000000 \n" 183 | ] 184 | } 185 | ], 186 | "source": [ 187 | "print \"==============================================================================\"\n", 188 | "print \" OVERALL SUMMARY\"\n", 189 | "print \"==============================================================================\"\n", 190 | "\n", 191 | "print voucher_df.describe()\n", 192 | "\n", 193 | "for i in range(2):\n", 194 | " print \"==============================================================================\"\n", 195 | " print \" LOTTERY = %(i)d\" % {\"i\":i}\n", 196 | " print \"==============================================================================\"\n", 197 | " print voucher_df[voucher_df['won_lottry']==i].describe()" 198 | ] 199 | }, 200 | { 201 | "cell_type": "markdown", 202 | "metadata": {}, 203 | "source": [ 204 | "# Two-stage Least Squares Logistic Regression" 205 | ] 206 | }, 207 | { 208 | "cell_type": "markdown", 209 | "metadata": {}, 210 | "source": [ 211 | " If you're interested to learn more on the rationale and process for doing this kind of analysis, Murnane and Willett introduce instrumental variables estimation(IVE) as a method for carving out causal claims from observational data ([chapter summary](http://acawiki.org/Introducing_Instrumental-Variables_Estimation)) ([example code](http://nbviewer.ipython.org/github/natematias/research_in_python/blob/master/instrumental_variables_estimation/Instrumental-Variables%20Estimation.ipynb)). \n" 212 | ] 213 | }, 214 | { 215 | "cell_type": "code", 216 | "execution_count": 15, 217 | "metadata": { 218 | "collapsed": false 219 | }, 220 | "outputs": [ 221 | { 222 | "name": "stdout", 223 | "output_type": "stream", 224 | "text": [ 225 | "==============================================================================\n", 226 | " FIRST STAGE\n", 227 | "==============================================================================\n", 228 | " Generalized Linear Model Regression Results \n", 229 | "==============================================================================\n", 230 | "Dep. Variable: use_fin_aid No. Observations: 1171\n", 231 | "Model: GLM Df Residuals: 1167\n", 232 | "Model Family: Binomial Df Model: 3\n", 233 | "Link Function: logit Scale: 1.0\n", 234 | "Method: IRLS Log-Likelihood: -488.00\n", 235 | "Date: Thu, 19 Mar 2015 Deviance: 975.99\n", 236 | "Time: 23:08:46 Pearson chi2: 1.16e+03\n", 237 | "No. Iterations: 7 \n", 238 | "==============================================================================\n", 239 | " coef std err z P>|z| [95.0% Conf. Int.]\n", 240 | "------------------------------------------------------------------------------\n", 241 | "Intercept 0.3455 0.731 0.472 0.637 -1.088 1.779\n", 242 | "won_lottry 3.5514 0.178 19.934 0.000 3.202 3.901\n", 243 | "male -0.1622 0.164 -0.992 0.321 -0.483 0.158\n", 244 | "base_age -0.1184 0.061 -1.946 0.052 -0.238 0.001\n", 245 | "==============================================================================\n", 246 | "\n", 247 | "\n", 248 | "==============================================================================\n", 249 | " SECOND STAGE\n", 250 | "==============================================================================\n", 251 | " Generalized Linear Model Regression Results \n", 252 | "==============================================================================\n", 253 | "Dep. Variable: finish8th No. Observations: 1171\n", 254 | "Model: GLM Df Residuals: 1167\n", 255 | "Model Family: Binomial Df Model: 3\n", 256 | "Link Function: logit Scale: 1.0\n", 257 | "Method: IRLS Log-Likelihood: -696.65\n", 258 | "Date: Thu, 19 Mar 2015 Deviance: 1393.3\n", 259 | "Time: 23:08:46 Pearson chi2: 1.17e+03\n", 260 | "No. Iterations: 6 \n", 261 | "======================================================================================\n", 262 | " coef std err z P>|z| [95.0% Conf. Int.]\n", 263 | "--------------------------------------------------------------------------------------\n", 264 | "Intercept 4.0756 0.604 6.753 0.000 2.893 5.258\n", 265 | "use_fin_aid_fitted 0.7743 0.192 4.036 0.000 0.398 1.150\n", 266 | "male -0.4175 0.130 -3.208 0.001 -0.673 -0.162\n", 267 | "base_age -0.2919 0.048 -6.077 0.000 -0.386 -0.198\n", 268 | "======================================================================================\n" 269 | ] 270 | } 271 | ], 272 | "source": [ 273 | "print \"==============================================================================\"\n", 274 | "print \" FIRST STAGE\"\n", 275 | "print \"==============================================================================\"\n", 276 | "result = smf.glm(formula = \"use_fin_aid ~ won_lottry + male + base_age\", \n", 277 | " data=voucher_df,\n", 278 | " family=sm.families.Binomial()).fit()\n", 279 | "voucher_df['use_fin_aid_fitted']= result.predict()\n", 280 | "print result.summary()\n", 281 | "\n", 282 | "print\n", 283 | "print\n", 284 | "print \"==============================================================================\"\n", 285 | "print \" SECOND STAGE\"\n", 286 | "print \"==============================================================================\"#\n", 287 | "result = smf.glm(formula = \" finish8th ~ use_fin_aid_fitted + male + base_age\", \n", 288 | " data=voucher_df,\n", 289 | " family=sm.families.Binomial()).fit()\n", 290 | "print result.summary()" 291 | ] 292 | }, 293 | { 294 | "cell_type": "markdown", 295 | "metadata": {}, 296 | "source": [ 297 | "# Interpreting the Local Average Treatment Effect " 298 | ] 299 | }, 300 | { 301 | "cell_type": "markdown", 302 | "metadata": {}, 303 | "source": [ 304 | "When we use IVE to \"recover\" the treatment effect, how can we actually describe the results? According to Murnane and Willett, \"an estimate of a treatment effect obtained by IV methods should be regarded as an estimated *local average treatment effect* (LATE). The chapter walks readers through the kinds of groups involved:\n", 305 | "\n", 306 | "\n", 307 | "\n", 308 | "\n", 309 | "\n", 310 | "\n", 311 | "\n", 312 | "\n", 313 | "
**won_lottery = 1****won_lottery = 0**
use_fin_aid=1
(*used financial aid form some source*)		use_fin_aid=0
(*did not use financial aid from any source*)		\"**Compliers**\"
use_fin_aid=1
(used financial aid from some source)		use_fin_aid=1
(used financial aid from some source)		\"**Always-Takers**\"
use_fin_aid=0
(did not use financial aid from any source)		use_fin_aid=0
(did not use financial aid from any source)		\"**Never-Takers**\"
\n", 314 | "\n", 315 | "Murnane and Willett offer a model that distinguishes among groups based on their compliance with \"the intent of the lottery\" (277), based on a paper by Angrist, Imbens and Rubin on \"[Identification of Causal Effects Using Instrumental Variables](http://business.baylor.edu/scott_cunningham/teaching/angrist-imbens-and-rubin.pdf)\" (1996):\n", 316 | "* *Compliers* \"are willing to have their behavior determined by the outcomes of the lottery, regardless of the particular experimental conditions to which they were assigned\" (278).\n", 317 | "* *Always-Takers* \"are families who will find and make use of financial aid to pay private-school fees\" regardless of the lottery. They may find aid outside the lottery\n", 318 | "* *Never-takers* are the mirror image of always-takers: \"they will not make use of financial aid to pay childrens' fees at a private secondary school under any circumstances\" (278)\n", 319 | "* (there are other possible groups, like \"defiers\" (Gennetian et all, 2005) who always do the opposite of what investigators ask them to do, but we make the assumption of \"no defiers\" in this dataset)\n", 320 | "\n", 321 | "In this context, IV estimates of the **local average treatment effect** (LATE) for this quasi-experiment only applies to \"compliers\"--and not to never-takers or always-takers. " 322 | ] 323 | }, 324 | { 325 | "cell_type": "code", 326 | "execution_count": null, 327 | "metadata": { 328 | "collapsed": true 329 | }, 330 | "outputs": [], 331 | "source": [] 332 | } 333 | ], 334 | "metadata": { 335 | "kernelspec": { 336 | "display_name": "Python 2", 337 | "language": "python", 338 | "name": "python2" 339 | }, 340 | "language_info": { 341 | "codemirror_mode": { 342 | "name": "ipython", 343 | "version": 2 344 | }, 345 | "file_extension": ".py", 346 | "mimetype": "text/x-python", 347 | "name": "python", 348 | "nbconvert_exporter": "python", 349 | "pygments_lexer": "ipython2", 350 | "version": "2.7.6" 351 | } 352 | }, 353 | "nbformat": 4, 354 | "nbformat_minor": 0 355 | } 356 | -------------------------------------------------------------------------------- /regression_discontinuity/Regression Discontinuity Analysis.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 27, 6 | "metadata": { 7 | "collapsed": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "%matplotlib inline\n", 12 | "\n", 13 | "# RESEARCH IN PYTHON: Regression Discontinuity Analysis\n", 14 | "# by J. NATHAN MATIAS March 23, 2015\n", 15 | "\n", 16 | "# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n", 17 | "# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n", 18 | "# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n", 19 | "# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n", 20 | "# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n", 21 | "# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\n", 22 | "# THE SOFTWARE." 23 | ] 24 | }, 25 | { 26 | "cell_type": "markdown", 27 | "metadata": {}, 28 | "source": [ 29 | "# Regression Discontinuity Analysis" 30 | ] 31 | }, 32 | { 33 | "cell_type": "markdown", 34 | "metadata": {}, 35 | "source": [ 36 | "This section is taken from [Chapter 9](http://www.ats.ucla.edu/stat/stata/examples/methods_matter/chapter9/default.htm) of [Methods Matter](http://www.ats.ucla.edu/stat/examples/methods_matter/) by Richard Murnane and John Willett. \n", 37 | "\n", 38 | "In chapter 9, Murnane and Willett introduce the practice of Regression Discontinuity Analysis, an method for estimating a causal effect in cases where a randomized trial is not possible. \n", 39 | "\n", 40 | "Regression Discontinuity analysis is possible in cases where some kind of cutoff determines who goes into one group versus another. Instead of looking at the effect of the predictor on the outcome for the entire population, we compare predicted outcomes on both sides of the cutoff.\n", 41 | "\n", 42 | "In *Methods Matter*, the authors use a paper by Joshua Angrist and Victor Lavy: \"[Using Maimonides Rule' to Estimate the Effect of Class Size on Scholastic Achievement](http://qje.oxfordjournals.org/content/114/2/533.short).\" In this paper, the authors were not able to randomize class size assigment, but they *were* able to take advantage of a rule in Israeli schools that split classes into smaller sizes if the enrollment cohort was 41 or higher. This offered Angrist and Lavy a natural experiment for their research question, since we might expect cohorts on both sides of the cutoff to be very similar, except for the cutoff (the equality of expectation assymption). \n", 43 | "\n", 44 | "**Regression Discontinuity** analysis uses observations further from the cutoff together with nearer ones \"to *project* the estimated treatment effect at the *cut-off*.\" Practically, this technique regresses the outcome on two predictors:\n", 45 | "* a continuous predictor (the **forcing variable**) \"that arrays observations (here, classrooms) along a continuum [here, class size] that includes an exogenously defined cut-point. The forcing predictor is usually recentered so it has a value of zero at, or just beyond the cut-point\" (180)\n", 46 | "* \"a dichotomous predictor that specifies on **which side of the exogenously defined cut-off a particular observation lies**, and consequently to which experimental conditions it has been assigned implicitly\" (180) (in this example, we construct the variable *small* from class sizes above and below the cut-point)\n", 47 | "\n", 48 | "The assumptions of this approach (beyond the assumptions of regression modeling) are:\n", 49 | "* assumption: the cutoff is \"as good as random.\" To test this assumption, we can:\n", 50 | " * verify that there is no selection bias in the application of the cutoff, perhaps with the [McCrary Density test (2008)](http://eml.berkeley.edu/~jmccrary/mccrary2006_DCdensity.pdf) on both sides of the cutoff. See also Urquiola and Verhoogen (2009) \"[Class-Size Caps, Sorting, and the Regression-Discontinuity Design](http://www.columbia.edu/~ev2124/research/Urquiola%26VerhoogenAER2009.pdf),\" for an in-depth exploration of how to observe these biases. \n", 51 | " * confirm that there is no unobserved influence (in the case of Angrist & Lavy, the authors consider a possible relationship between the socio-economic status of a region and cohort size)\n", 52 | "* assumption: the relationship between the outcome and forcing variable is well modeled, something that is sometimes addressed (like below) by limiting the \"bandwidth\" observations on both sides of the cutoff. These can be tested with:\n", 53 | " * sensitivity analyses at different bandwidths\n", 54 | " * sensitivity models that adopt a variety of possible functional forms\n", 55 | "\n", 56 | "## About The Angrist & Lavy Dataset\n", 57 | "In this analysis, we examine a dataset that includes school level data for:\n", 58 | "* *size*: fifth-grade cohort size\n", 59 | "* *intended_classsize*: average intended class size for each school\n", 60 | "* *observed_classize*: observed average class size for each school\n", 61 | "* *read*: average reading achievement in cohort\n" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": 28, 67 | "metadata": { 68 | "collapsed": true 69 | }, 70 | "outputs": [], 71 | "source": [ 72 | "# THINGS TO IMPORT\n", 73 | "# This is a baseline set of libraries I import by default if I'm rushed for time.\n", 74 | "\n", 75 | "import codecs # load UTF-8 Content\n", 76 | "import json # load JSON files\n", 77 | "import pandas as pd # Pandas handles dataframes\n", 78 | "import numpy as np # Numpy handles lots of basic maths operations\n", 79 | "import matplotlib.pyplot as plt # Matplotlib for plotting\n", 80 | "import seaborn as sns # Seaborn for beautiful plots\n", 81 | "from dateutil import * # I prefer dateutil for parsing dates\n", 82 | "import math # transformations\n", 83 | "import statsmodels.formula.api as smf # for doing statistical regression\n", 84 | "import statsmodels.api as sm # access to the wider statsmodels library, including R datasets\n", 85 | "from collections import Counter # Counter is useful for grouping and counting\n", 86 | "import scipy" 87 | ] 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "metadata": {}, 92 | "source": [ 93 | "# Acquire Dataset from Methods Matter" 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": 29, 99 | "metadata": { 100 | "collapsed": false 101 | }, 102 | "outputs": [], 103 | "source": [ 104 | "import urllib2\n", 105 | "import os.path\n", 106 | "if(os.path.isfile(\"class_size.dta\")!=True):\n", 107 | " response = urllib2.urlopen(\"http://www.ats.ucla.edu/stat/stata/examples/methods_matter/chapter9/angrist.dta\")\n", 108 | " if(response.getcode()==200):\n", 109 | " f = open(\"class_size.dta\",\"w\")\n", 110 | " f.write(response.read())\n", 111 | " f.close()\n", 112 | "class_df = pd.read_stata(\"class_size.dta\")" 113 | ] 114 | }, 115 | { 116 | "cell_type": "markdown", 117 | "metadata": {}, 118 | "source": [ 119 | "# Summary Statistics " 120 | ] 121 | }, 122 | { 123 | "cell_type": "code", 124 | "execution_count": 30, 125 | "metadata": { 126 | "collapsed": false 127 | }, 128 | "outputs": [ 129 | { 130 | "name": "stdout", 131 | "output_type": "stream", 132 | "text": [ 133 | "==============================================================================\n", 134 | " OVERALL SUMMARY\n", 135 | "==============================================================================\n", 136 | " read size intended_classize observed_classize\n", 137 | "count 2019.000000 2019.000000 2019.000000 2019.000000\n", 138 | "mean 74.379171 77.741951 30.955935 29.935116\n", 139 | "std 7.678460 38.810731 6.107924 6.545885\n", 140 | "min 34.799999 8.000000 8.000000 8.000000\n", 141 | "25% 69.855000 50.000000 27.000000 26.000000\n", 142 | "50% 75.379997 72.000000 31.666666 31.000000\n", 143 | "75% 79.839996 100.000000 35.666668 35.000000\n", 144 | "max 93.860001 226.000000 40.000000 44.000000\n" 145 | ] 146 | } 147 | ], 148 | "source": [ 149 | "print \"==============================================================================\"\n", 150 | "print \" OVERALL SUMMARY\"\n", 151 | "print \"==============================================================================\"\n", 152 | "print class_df.describe()" 153 | ] 154 | }, 155 | { 156 | "cell_type": "markdown", 157 | "metadata": {}, 158 | "source": [ 159 | "# Set up Forcing Variable and Cutoff Predictor" 160 | ] 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "metadata": {}, 165 | "source": [ 166 | "The Forcing Variable (*csize*) is centered around the cutoff, and the cutoff predictor (*small*) is a dichotomous variable that indicates which side of the cutoff an observation lands." 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": 34, 172 | "metadata": { 173 | "collapsed": false 174 | }, 175 | "outputs": [ 176 | { 177 | "data": { 178 | "image/png": [ 179 | "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEcCAYAAADA5t+tAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", 180 | "AAALEgAACxIB0t1+/AAAHwlJREFUeJzt3XuYXFWZ7/FvJSEkYIgZSBSMp9sZ5FUJIIgiyOSCEEWN\n", 181 | "wUucGefxCCIXiQ4PE4IYAzoHFGea8AjqIIZA8MioE4Yhw3DVkEkgc8AHBpQQffHWfcyoJOEkYAIR\n", 182 | "SNf5Y61Oik5fq/elqtbv8zz9pGvvqv2uvVP97rXXXmvtSrVaRURE0jGq7AKIiEixlPhFRBKjxC8i\n", 183 | "khglfhGRxCjxi4gkRolfRCQxY8ougLQGM5sJ/DPwBFAB9gU+5e6PZbDt5cB33f2eftZPAya5+/0j\n", 184 | "jZUHM7sQ2OzuN9Us+yLwuLv/Sw7xXgVc6u7zs962tAYlfslKFfihu38UwMxOAS4D5mS07YEGnHwY\n", 185 | "+B3QkImfvsue2wAad38KUNKXfinxS1Yq8afHnwBPAZjZ0cA1wC5gJ3AWcBDwv4G3AX8BvBu4CLgJ\n", 186 | "2AEcDPy7u1/as0EzGwMsB14HjAauAh4ATgd2mtkj7v5wzfsvAU4DNgP7AZcAs4ATgP2BM4H3xvgv\n", 187 | "AWvd/eJYG/+du19nZm8ArnX3WWb2n4ADr4/b/Gj83DeBQwlNp4vdfY2ZnRbjPU1I8v/UxzE73cw+\n", 188 | "BYwDLgBeCZzl7h+J5V8HfMjdfx9fV4CvAW8FxgJfAP4P8P147McB5wLPAN+N+3p3jL8PcFwsextw\n", 189 | "efz/+CVwjru/1Ef5pEWpjV+ydJKZrY4J8gbge3H5UmC+u88E/hG4yt0fBa4Hvg2cB3yCkLzagHmE\n", 190 | "5HZKPGkQ150LPOXu7wBOJiSvPwI3xm3WJv2jCCeTYwnJ/+C4qgo8EbexT4x1vLufALzezN5L/7Xx\n", 191 | "V8U4JxIS5rnAJwnNODNinG+YWc9J6WR3nw1s6Wd7P3H3k4GzgW+6+w+AI8zslWZ2eNzu72vefxpw\n", 192 | "oLsfR0jqx8bjtAU4lVDL37+n/O6+091nuvss4NfAue7eGf8/PhD/P/6bcOKUhCjxS5buc/dZMYke\n", 193 | "DXzfzMYBB7v7T+J77gcOj79fB5wE3OzuO+KyB939OXffBTwEHFaz/TfEz+Pu24ENwJ/FdbVXGz3v\n", 194 | "/ZG7V919J/Bwzbona97zYIzVu2w9are7yd0fj78/EMs2DXiPma0GbiFciRwCPOPuW+N719K3NXFf\n", 195 | "NgCvjsu+A/wVIRlf3+v9hxFq+Lj7tng1dBewDlgJ/C+gu1eZMbOvAz9z92VmNiXGWhHLPBv4H/2U\n", 196 | "T1qUEr/kZRN7as6/NbMj4u8zCM0lAB3x5wwze11cdpSZ7RNrzW8D1tds86fAnwOY2QTgCEJNtpu9\n", 197 | "v8tPAG81s4qZ7Us4EfXortnecWY2OjajTI9l28meK4Rjaj53kJm1x9/fATwO/Ixw43kWMJdwg/v3\n", 198 | "wMSYZAHe3vchCsvN7M1AZ1x2I/CRuJ939nr/Twk1fMxsopndCcwkNEu9C/gS8GVqrljM7DIAd/9S\n", 199 | "XLQF2Ai8P5b5K8AP+ymftCglfslKlT1NPT8E7gH+Nta2zwK+bmZrgc8AF5jZXOBQd78COB+4mXDP\n", 200 | "qQrcDjwI3OLuT9Rs/1vAgWZ2P7Aa+KK7bwYeAT5tZjN6CuPu6wmJ80HgVuDF+NOzrZ73/DOhxvwQ\n", 201 | "8Gt3X0loM++pxR/NnkT6EnCFmT1AaPb5FuGq5Q1m9h/AfwD/191fBD4F3BmPxST6bj6aZmargKsJ\n", 202 | "zT24+2+BZ4FV7t5d+2Z3/zdga9z/u+Pnfgx8Mpb1HwiJH6BqZscCFwOHxf+X1YQT7/mxbOti3A19\n", 203 | "lE1aWEWzc0qjiLXpr7n7iHsCmdlk4MPufm2s8a8HZrn7xhFs83F3P2Lwd46Mma0ELnD3X+UdS9Kk\n", 204 | "Xj3SSAbrtjkcWwhNPWfEbS4dSdKPcq0lmdl4wn2GVUr6kifV+EVEEqM2fpEMmdl2M1MvGWloSvwi\n", 205 | "2dIltDQ8tfFLsuL8QlcD2wkDny4FFhFGxT4HXOjuD8a5b64DevrAdwEfcffNZvbnhNG03YSxAqpM\n", 206 | "ScPTl1RSdzjwl4T5fi4HTnX3Y4BzgFvNbD/ClA7r3P0Ed/9TwknhY2Y2FlhB6IFzDHAvML6MnRAZ\n", 207 | "DiV+Sd1v3P03hBGsBwP3mdmjhBG0u4A/c/drgAfN7G/N7FrCaN39CQPIXnD31QDufguwta8gIo1E\n", 208 | "TT2Suu3x31GEbpR/2bMi3qTdaGZ/Txgxuwy4j/B3U6GP6REIg7xEGppq/CLBamC2mRmAmb0beIww\n", 209 | "4+Vs4KvufjNhVs5TCHPyPA5UzOzU+Jn3AJNLKLvIsCjxS+p6pm94gjB9wffM7DHiswTc/TnC5GdX\n", 210 | "mtmDwLWEydgOjVMZnwZcFpuHPkycilqkkWkAl4hIYgZt4zez44CvxAdRHEp4EEY3Ye6T+e5eNbOz\n", 211 | "CLWll4DL3f2OHMssIiIjMGBTj5ldRHhow75x0VXAInefTripNdfMXk2YcfEE4F2E2QvH5ldkEREZ\n", 212 | "icHa+H8BfJA9PReOcfeeh0rcRXgK0lsJfZxfdPdn42eOzKOwIiIycgMmfne/lZd3T6vtuvYHYCJw\n", 213 | "AOEZn72Xi4hIAxpuP/7aB0McAGwjPDRiQs3yCQw+iGU9ez/iTkREstN7jMluw038j5rZDHdfQ3i4\n", 214 | "8yrgR8CX4sMuxgFv5OWPy+vLtGHG7a3KADuVszJjlx1f+16eVPddxz0HQ038PX0+FwBL483bDYRH\n", 215 | "41XN7BrCAyRGEW7+vpB9UUVEJAvN2o+/Jc/CTRBf+16eVPddxz0HGrkrIpIYJX4RkcQo8YuIJEaJ\n", 216 | "X0QkMUr8IiKJUeIXEUmMEr+ISGKU+EVEEqPELyKSGCV+EZHEKPGLiCRGiV9EJDFK/CIiiVHiFxFJ\n", 217 | "jBK/iEhilPhFRBKjxC8ikhglfhGRxCjxi4gkRolfRCQxSvwiIolR4hcRSYwSv4hIYsYM9wNmNha4\n", 218 | "HjgUeBH4G2AHsBzoBtYD8929ml0xRUQkK/XU+M8CnnP3E+LvNwJLgEXuPh2oAHOzK6KISP4qlUpH\n", 219 | "pVLprP1pb2+n17KOssuZhXoS/5uAuwHc/UngNcBJ7r42rr8LODmb4omISNaG3dQDPAa8D7jNzN4O\n", 220 | "TAZqm3W2AxMzKJuISGGq1epCYGHvxUB78aXJVz2J/wbgjWZ2P7AOcOCgmvUTgG2DbGM9cHgdsWuV\n", 221 | "eQ+h7PsX2vf0YhcSf+HChaxYsWLA2PPmzaOjo9AWj5Y/7jnFrvS7olod3nbN7HjgQHf/dzM7FugA\n", 222 | "ngWucvc1ZvZNYJW79/ntyUiVAXYqZ2XGLjt+Evse23Hn1S5ra2tr6+rq6qpZtCLWEItSyr6Pe8WB\n", 223 | "U8eMGTN6+7anytr3JL5zRceup8bvwPfNbBGwE/gk4V7B0tjjZwNwS3ZFFCnXuFccOPXpZ3aWXYxC\n", 224 | "9G7umLNgZeeUSePbli2e3V5eqSRrw0787v7/gFP6WDVzxKURaQBKftLq6qnxi0gibl8yt53y29hL\n", 225 | "EU/4LFs8u+yiZE4jd0VEEqMav8ggUq71SmtSjV9EJDFK/CIiiVHiFxFJzLAHcDWIlhxU0QTxte/l\n", 226 | "KSV+TVdWHfcWiq0av4hIYpT4RQYxZ8HKzjMvv7fsYohkRolfRCQxDZ/4U3o4gohIERo+8YuISLYa\n", 227 | "fuRuSg9HEGk0KY9aLnKunn6mAqerq6uzZlFm02Grxi8i0mDyngq84Wv8KSu6FiB9S7nWW6SUv+9F\n", 228 | "TwWuGn8TSemBICKSn6ar8bfyHNm96YEgkhLdzytO0yV+EZEszFmwcq+mpV6mbnlmJ3MWrOwc4D0r\n", 229 | "bl8yN/Omp7ybF5X4m4jamqVoLX6FPQ+YCmzsZ/3GgyaOa9u09fn+Pj81bqPp7jko8Ys0mJRvcpZg\n", 230 | "Y6xQ9affpqZBrgQamm7uigyi7Ll6dFNfsqYav0iD0U39oMWbmUo17MRvZqOA64HDgG7gLGAXsDy+\n", 231 | "Xg/Md/dc2qLVzi0iMjL11PhnA/u7+4lmdjLw5bidRe6+1syuBeYCt2VYThHJQSP3bElZ3lc79ST+\n", 232 | "54GJZlYBJgIvAMe5+9q4/i7CyUGJP2O69E1Tzle5yfZsOWXzQ5Nev+M3+6+b++3O/t6z75TJ/HHT\n", 233 | "5j7Xnzdmv6k/3/+1O0I9t7nUk/jXAeOAnwEHAnOA6TXrtxNOCCJNQbXeNHu2pKyexH8RsM7dP29m\n", 234 | "U4HVwD416ycA27IonEhBkq31puwHk4/b+oPJx20d6UnvbzIvWf7qSfz7A8/G37fGbTxqZjPcfQ1w\n", 235 | "KrBqkG2sBw6vI3atMm/wlhJ7yqTxhcVfuHAhK1as6GvV7tjz5s2jo6PwZ+Bkvu89x3XZ4tltg7y1\n", 236 | "z/Wxq2cb+f+/5LbvQ9h2n+vz/E7evmR3E0oux7WR9z2jbff7oPZ6En8HcKOZ3U+o6X8OeARYamZj\n", 237 | "gQ3ALYNsY1odcYGXdW3L5enzQ1BlgAOap01bn++cMml8WxHxr7zyyr4GEbV1dXV11bxnRUdHR5G1\n", 238 | "3FyO/aatz3fGX9vriT3Ez49Uqvue299bmfs+lObFUaMqo+csWNk1wHvqbl4cduJ3923AB/pYNbOe\n", 239 | "Akhj0oRZIrkqtXlRA7gG0Xv4fJlD5zWGIU3qzdWySruprsQvkrCUuzSmTIl/EH00eai5Q0SaWsMl\n", 240 | "fvWpFilOI3dpVBNXfhpxds6emx792XjQxHEDfb7npkfmyp6lsUxF73ulUumoVCqdtT/t7e30WlZ4\n", 241 | "X1KRVtBwNf5IIwnlZTQ1sUh2GjXxSx9SuvRVd9I91JtLstaITT0iIpIj1fhFpDSDdOZQR46cKPH3\n", 242 | "kuoXUb2ppCQDjWDV5Hg5UeLfW6pfRM1QKWUZqDOHOnLkQIm/b6V8ERug1q3eVCIJ0M3dxtKwYxjK\n", 243 | "pjEUae675EM1/sajWreI5Eo1fhGRxCjxi4gkRk09AqQ9PW/K+y5pUuKXhtAAPZpKk8G+N+V+S3mU\n", 244 | "+AVoiOl5SxtH0OT7rvETMmxK/NJIUu7RVNe+N/t+D9bMpia2fCjxi6StDQY+gUyZNJ5NW5/vb/1A\n", 245 | "VyrSoJT4e1ENRORlpm4Z+DkIG4EV9W58CM1spTz9q9Up8Ysk7PYlcysDrZ+zYGXnQRPHtS1bPLu9\n", 246 | "oCJJAYad+M3s48Dp8eV44CjgROBqoBtYD8x396Z8cIRqICKSt7K7EA97AJe73+Tus9x9FvAw8Bng\n", 247 | "UmCRu08HKqitQ0SkYdXd1GNmxwJvcvdPm9kX3X1tXHUXMBu4LYsCpqTsWoCIFKPsLsQjaeNfBPxd\n", 248 | "/L22nXA7MHEE2xURKUKyPZrqSvxm9krgMHdfExd116yeAGwbZBPrgcP7WjFl0vieXwe7R9Dn+mF8\n", 249 | "vk9D/HwusX982Ex+DCxbPLttkLf2uT5O3XtAPfGb5LjnEr+Z932ksQdz+5LdV4+5bL/Mv7ea7fT5\n", 250 | "97TlmZ1seWZnv+sB3nHUa9rqiV/Qd67fG/f11vinA6tqXj9qZjPiieDUXuv6Mq2/FTVn1/YBPl+l\n", 251 | "n50a4uf7NYTPlxk7t/hNcNxzi9/M+z7S2EPU776P1Katz1cB5ixY2dXX+imTxrdt2vp8n+vYU+Nu\n", 252 | "ryd2zUmtT3MWrOycMml827LFszPf97K/c/Um/sOAX9a8XgAsNbOxwAbgljq3K+VJ9rKXkvd9JPd2\n", 253 | "Wvy+Tq5jCFJWV+J39yt7vf45MDOLAkljWH/ftyb9/hcP7d/zujJq1GioUO3etfsJYa8+9Lgd0046\n", 254 | "e2t82cp/hEpAORloHIHGEORHA7gE2PsPsFI5ba8ZI9va2tq6urp212w7H7tzxa8fvaPpJwcrexDT\n", 255 | "SHp4aOyI1EOJX/pUrVYXsveMj/12LxOR5qHELw1BYxgaU7zBybLFs8suSuHiFVhTzkAwGD16UUQk\n", 256 | "MarxS0MoeySjNJ5WrnGXTYlfZBBKQNJq1NQjIpIYJX4RkcSoqUdE+pVyM1cr92hSjV9EJDGq8TeW\n", 257 | "lOfLEXmZVq5xl02Jv7m08pwxDXvSUwKSVtNwiT/lEZxlzxnT4HTS6/ukp6s8GbaGS/ySJp30BjTQ\n", 258 | "Sa+ZT3hSkoZL/BrBKalp5JNeys1crdyjqeESfwMY8LJbN1dFpNkp8Q9PK7czizSUVq5xl02Jv5dG\n", 259 | "fiKQ/hDKoeMurUaJX5qCkq9IdpT4RRqcTnqSNSV+EelXyiednHs0lTpgUXP1iMhulUqlo1KpdNb+\n", 260 | "tLe302tZR9nlTECuHUlU4x+GlGs/IkUregxBPKHN63k97hUHTh0zZgw3XPJUZ83bVlSr1YUjjVX2\n", 261 | "2I26Er+ZfQ6YA+wDfB1YBywHuoH1wHx3V4LMWMqDacqU0nGPSa13Yut3wGSWiky8Q7Fz+9Mb29ra\n", 262 | "2rZvKyJasYbd1GNmM4Hj3f0EYCbwp8ASYJG7TwcqNONEOdLQ5ixY2Xnm5feWXQwp0M7tT288cOK4\n", 263 | "wuJVq9WF1Wq1vfans7OTXssKOenkrZ4a/2zgcTO7DTiAUDs4093XxvV3xffclk0RRdKWyhVHmVcb\n", 264 | "qakn8U8GXgu8j1Dbv51Qy++xHZg48qJJyhrtsl+KV6lQ6e6GyoCt4VKPehL/FuCn7v4S8KSZ7QRe\n", 265 | "U7N+AjBYq9h64PC+VkyZNL7n18HuEfS5fhifH4lS7l8UtG+DKST2hRdeyIoVe3daOHBiW1vP7/Pm\n", 266 | "zbsQuDDvspR93MuOX1bs7u7yYvdSePzbl+xuLR9J7H5PmfUk/geA84GrzOwQYD9glZnNcPc1wKnA\n", 267 | "qkG2Ma2/FTX9VtsH+HyVfnZqiJ+vS7zkblu2eHYpdZBNW5/vnDJpfBsD/IfmrN/jnrWOjg46Ovbq\n", 268 | "NVhY/FplH/ey41Pwca9UdseqvPQSu8aM2XMvslotPAmX8p3LO/awE7+732Fm083sR4Sbw+cBncBS\n", 269 | "MxsLbABuybSUAqg7aVl03KXV1NWd090/28fimSMrioiIFEEDuEQanK44JGuNmPgb9qHbIpK/nnb8\n", 270 | "SiX8lNCu3/Kaca4ePQxFJBGpduXMe8Biw9X4y57DYiC65BaRVtCMNf5kadqCcui4l6NapZpqjT9v\n", 271 | "SvwiIolpuKYe2UPTFgikM1ePFEc1/iZS9GyFItKamq7Gn9INVs1WKJKmvPOcavzDoJt8ItIKmq7G\n", 272 | "L5K33vdWANra2qhc0tVZs0j3VqRpqcYvIpKYSrXalM3lhU2VWlv7i71qRm/f9lRXzVuKrvm15DSx\n", 273 | "TRBf+55e7LLj5xZbNf5hUK8aEWkFTVfjL/thKKgGon1PL36qsUuLn3eeU41fRCQx6tUjTUEP3pZW\n", 274 | "VvQofdX4RUQaTN73E1Xjl4ZW++DtahVGjdrT3tqqD+jor/a3fZvmaGpVRY/SV41fpMGpN5lkrel6\n", 275 | "9UTqZZBY7NjG311zBVA0/b+nF7vs+OrHLyIi2VDiFxFJTF03d83sv4Bn4stfAVcAy4FuYD0w390z\n", 276 | "aUPqb8Ksri5NmJWSVr2RK1KGYSd+MxsH4O6zapb9G7DI3dea2bXAXOC2zEopIiKZqafGfxSwn5nd\n", 277 | "Ez//eeAYd18b198FzCajxK+HkYiIZKueNv4dQIe7vws4F7i51/rtwMSRFkxERPJRT43/SeAXAO7+\n", 278 | "czN7Gji6Zv0EYNsg21gPHF5H7FpltvmW3d6sfU8vdtnxU41ddvyRxO63K2g9if8M4EhgvpkdQkj0\n", 279 | "95rZDHdfA5wKrBpkG9PqiFurJfvWNkF87Xt5Ut13HfccDHsAl5mNAW4E2uKii4CngaXAWGADcFZW\n", 280 | "vXr60ZL/GU0QX/tenlT3Xcc9Bxq521yxy46vfS9Pqvuu454DDeASEUmMEr+ISGKU+GVIKpUwLbKI\n", 281 | "ND8lfhkyJX6R1qAHsciAej8IpXZaZM2fI9KcVOOXoVJTT0nUzCZZU+KXoarqQecirUFNPTKgnuac\n", 282 | "SiX8qHmnOCk+b1iKoRq/iEhiVOOXIVNTT7F0tSV5UY1fhqRaVRt/mXRzV7KkxC8ikhg19Yg0qNqb\n", 283 | "u71eq9lHRkQ1fhGRxKjGL9KgdHNX8qIav4hIYpT4RUQSo6YekQanJh7Jmmr8IiKJUeIXEUmMEr+I\n", 284 | "SGKU+EVEEqPELyKSmLp79ZjZFOAR4J1AN7A8/rsemO/u6okgItKA6qrxm9k+wHXADsI8IlcBi9x9\n", 285 | "enw9N7MSiohIpupt6ukArgV+F18f4+5r4+93ASePtGAiIpKPYSd+Mzsd2Ozu98ZFlfjTYzswceRF\n", 286 | "ExGRPNTTxn8GUDWzk4E3AzcBk2vWTwC2DbKN9cDhdcSuVeY9hLLvX2jf04tddvxUY5cdfySx+310\n", 287 | "0rATv7vP6PndzFYD5wIdZjbD3dcApwKrBtnMtOHG7aXKADuVszJjlx1f+16eVPddxz0HWczVUwUW\n", 288 | "AEvNbCywAbglg+2KiEgOKtXmfJhnS56FmyC+9r08qe67jnsONIBLRCQxSvwiIolR4hcRSYwSv4hI\n", 289 | "YpT4RUQSo8QvIpIYJX4RkcQo8YuIJEaJX0QkMUr8TaRSodKcA61FpJEo8YuIJCaLSdokZ5XK7vk6\n", 290 | "KtUqjBq1Z/6OarX0KWtFpMmoxi8ikhjV+JtAT62+Ugk/quWLyEioxi8ikhglfhGRxKipp4moiUdE\n", 291 | "sqAav4hIYpT4RUQSo8QvIpIYJX4RkcQo8YuIJEaJX0QkMcPuzmlmo4GlwGFAFTgX+COwHOgG1gPz\n", 292 | "3V1dD0VEGlA9Nf73Ad3ufiKwGPgysARY5O7TgQowN7siiohIload+N19JXBOfNkObAXe4u5r47K7\n", 293 | "gJMzKZ2IiGSurjZ+d99lZsuBq4GbYc80wcB2YOLIiyYiInmo++auu58OGHA9MK5m1QRg28iKNajK\n", 294 | "4G9pydhlx9e+pxk/1dhlx88t9rATv5l9zMw+F18+D+wCHjazGXHZqcDaPj8sIiKlq1SH+RBXMxtP\n", 295 | "6MHzamAf4ArgZ4SePmOBDcBZ6tUjItKYhp34RUSkuWkAl4hIYpT4RUQSo8QvIpKYpnsCl5ldAJwJ\n", 296 | "bI6LznH3Jwsuw7eAp939c4O+Odu4HwI+S5gq42Z3v6bA2H8FnA+8BDwOnFf0DXwz2w/4AfAJd/eC\n", 297 | "Yo4C/hE4kjA1ySfd/ZdFxK4pw3HAV9x9VsFx9wFuANqAfYHL3f32gmLvNTWMuz9RROyaMkwBHgHe\n", 298 | "WUKO+S/gmfjyV+5+Zpbbb8Ya/zHAx9x9Vvwp+j/kHGAaFPsYxPiHcAXwTuB44Dwz+5OCYo8HLgNm\n", 299 | "xqk6JhKm7iiMmR1L6Cb8Ooo99qcBY939BOBiwvQkhTGziwgJcN8i40Z/DWyOU7G8G/h6gbF7Tw3z\n", 300 | "pQJj95z0rgN2FBk3xh4HUJPjMk360JyJ/y3AIjO738wuLjKwmZ0AvI3whSh0YIe77wLe4O5/ACYD\n", 301 | "o4EXCgq/Ezje3XfG12MIYziKNJaQhAup6dd4B3A3gLs/BBxbcPxfAB+knIFEK4BL4++jCFd7hehn\n", 302 | "apgidQDXAr8rOC7AUcB+ZnaPma2KV3yZasbE/13CF+Ik4EQze28RQc3sYMIfwacpaTSfu3eb2QeB\n", 303 | "R4HVwHMFxa26+2YAM/sMsL+7/7CI2DVl+E9331hkzOgA4Nma17ti808h3P1WCky4vWLvcPftZjaB\n", 304 | "cBL4fMHxe6aGuQb4p6LimtnphCude+Oiov/edwAd7v4uwuzHN2f9nWuKNn4zuww4kfAf8H53fzYu\n", 305 | "vwM4GrijgNhHAp3AnYTBa/uZ2U/d/dt5xe4VH0Jb461m9q+EQXT/M/6bd+wqYeK9fwAOBT6UV8wB\n", 306 | "4r+zpEGBzxKmIekxyt27SyhHKczstcCtwDfc/XtFx3f3083ss8BDZvZGdy/iSvMMoGpmJwNvBm4y\n", 307 | "s7nu/lQBsQGeJFzp4e4/N7OngYOB/84qQFMkfne/BMDMDgAeN7M3EWq7JwHLiohdy8w+Tmh2yTXp\n", 308 | "18aPta7VZnaKu79gZjsI02XkHjvGX0po8vlAUQm4r2NfgnXAHGCFmb0d+EnJ5SmMmb0KuJdwI391\n", 309 | "wbE/Bkx19ysIzYrd8Sd37t4z/QxmtprQgaSopA/hxHMkMN/MDiFcdWba5NRUTT2xpn8xoZljLbDe\n", 310 | "3e8uqTiF1j5j2/53gLVmdj/hj+A7RcQ2s2OATxBuat9nZqvN7LQiYjeAfwV2mtk6wo3dC0oqRxlX\n", 311 | "O4sIN/Ivjf/nq3tuPBbgFuDNZraGcI/lfHf/Y0Gxy7YMOMDM1gLfA87I+ipTUzaIiCSmqWr8IiIy\n", 312 | "ckr8IiKJUeIXEUmMEr+ISGKU+EVEEqPELyKSGCV+kQGY2Rwz+7uyyyGSJfXjFxFJTFNM2SCSBzP7\n", 313 | "e8KMny8RZlwdQ5j/qBv4kbufGyfsmgF8Ebit9uOE6YK/ClwZ3zMaWO7uXy1oF0TqosQvSTKzecAJ\n", 314 | "hGkoxgIPEpL5eELi/0acJ6UK4O5dhAkBiTOkfpYwP/3ZQNXd32Jm+wL3mNnD7v5AwbskMmRq45dU\n", 315 | "TQe+7+4vxumHjyDMvPow8AXCbJS/pdeUvGZ2JGGu9g+5+wuEWUvfb2aPEk4ehxBOJiINSzV+SdWL\n", 316 | "1CR1M2sHPgocAbwHuNvM/pqaydHM7CDC5GFn1DwbYBSw0N1vi++ZDPyhiB0QqZdq/JKqtcAHzWxM\n", 317 | "fJbvKmALYcbXLxCmIz6y581mNoaQ9K9297U127kPODtu5xXA/YSntIk0LPXqkWSZ2eXA+wkVoK8R\n", 318 | "2vfPJjzroQv4OOGhMzMID3lfDjwG7EO4WriX8FSqJYRnQ4wBbnD3jiL3Q2S4lPhFRBKjph4RkcQo\n", 319 | "8YuIJEaJX0QkMUr8IiKJUeIXEUmMEr+ISGKU+EVEEqPELyKSmP8P1qMJ5OvF6GYAAAAASUVORK5C\n", 320 | "YII=\n" 321 | ], 322 | "text/plain": [ 323 | "" 324 | ] 325 | }, 326 | "metadata": {}, 327 | "output_type": "display_data" 328 | } 329 | ], 330 | "source": [ 331 | "def small(size):\n", 332 | " if(size>=41):\n", 333 | " return 1\n", 334 | " return 0\n", 335 | " \n", 336 | "# \"first\" distinguishes the groups that participate in the first diff.\n", 337 | "def first(group):\n", 338 | " groups = {1: 0, 2:0,\n", 339 | " 3: 1, 4:1}\n", 340 | " return groups[group]\n", 341 | "\n", 342 | "# SET UP Forcing Variable and Cutoff Predictor\n", 343 | "class_df['small'] = class_df['size'].map(small)\n", 344 | "class_df['csize'] = class_df['size'].map(lambda x: x-41)\n", 345 | "\n", 346 | "# summarize the read variable by each class size group\n", 347 | "class_df[(class_df['size']>=36) & (class_df['size']<=46)].boxplot(\"read\", \"csize\")\n", 348 | "plt.show()" 349 | ] 350 | }, 351 | { 352 | "cell_type": "markdown", 353 | "metadata": {}, 354 | "source": [ 355 | "# Regression Discontinuity" 356 | ] 357 | }, 358 | { 359 | "cell_type": "code", 360 | "execution_count": 36, 361 | "metadata": { 362 | "collapsed": false 363 | }, 364 | "outputs": [ 365 | { 366 | "name": "stdout", 367 | "output_type": "stream", 368 | "text": [ 369 | " OLS Regression Results \n", 370 | "==============================================================================\n", 371 | "Dep. Variable: read R-squared: 0.015\n", 372 | "Model: OLS Adj. R-squared: 0.010\n", 373 | "Method: Least Squares F-statistic: 3.103\n", 374 | "Date: Mon, 23 Mar 2015 Prob (F-statistic): 0.0459\n", 375 | "Time: 23:57:58 Log-Likelihood: -1536.6\n", 376 | "No. Observations: 423 AIC: 3079.\n", 377 | "Df Residuals: 420 BIC: 3091.\n", 378 | "Df Model: 2 \n", 379 | "Covariance Type: nonrobust \n", 380 | "==============================================================================\n", 381 | " coef std err t P>|t| [95.0% Conf. Int.]\n", 382 | "------------------------------------------------------------------------------\n", 383 | "Intercept 70.1188 1.151 60.908 0.000 67.856 72.382\n", 384 | "csize -0.1392 0.119 -1.168 0.243 -0.373 0.095\n", 385 | "small 3.9531 1.800 2.197 0.029 0.416 7.491\n", 386 | "==============================================================================\n", 387 | "Omnibus: 34.356 Durbin-Watson: 1.967\n", 388 | "Prob(Omnibus): 0.000 Jarque-Bera (JB): 40.549\n", 389 | "Skew: -0.711 Prob(JB): 1.57e-09\n", 390 | "Kurtosis: 3.529 Cond. No. 34.2\n", 391 | "==============================================================================\n", 392 | "\n", 393 | "Warnings:\n", 394 | "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n" 395 | ] 396 | }, 397 | { 398 | "data": { 399 | "text/plain": [ 400 | "" 401 | ] 402 | }, 403 | "execution_count": 36, 404 | "metadata": {}, 405 | "output_type": "execute_result" 406 | }, 407 | { 408 | "data": { 409 | "image/png": [ 410 | "iVBORw0KGgoAAAANSUhEUgAAAsYAAAGACAYAAACugaRZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", 411 | "AAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlYE8cbB/AviBwegHe1VkO1BhQUkcOjCoI/ihVbqReI\n", 412 | "Ih6tB2rV1kqt1LNqva8WlXoiaEVRqYpVPLBKPVqt961YtNYbBSGc+/sjzcKSBBKSMVl4P8+TR7NJ\n", 413 | "NsNmdnZ25p0ZE47jOBBCCCGEEFLJmRo6AYQQQgghhBgDqhgTQgghhBACqhgTQgghhBACgCrGhBBC\n", 414 | "CCGEAKCKMSGEEEIIIQCoYkwIIYQQQggAA1aMw8PDYW9vr/Ro3bo1vL298c033+DZs2eGSt4btWLF\n", 415 | "Ctjb2+Off/55Y9956tQppWPfqlUreHh4YODAgUhISFD6THx8POzt7XHmzJk3lk5dpKWlCZ7b29vj\n", 416 | "66+/ZvZ93t7eGDRokGDbs2fPkJ2dzeT7FPmm5MPR0RGdO3fG+PHj8ffffzP57rKUPBaDBg2Ct7e3\n", 417 | "QdKyb98+DBo0CO7u7mjTpg0++OADzJ49G0+fPjVIelh6k+fo2bNnMXHiRHh7e8PJyQldunRBeHg4\n", 418 | "7t27Z7A0aevx48eYOXMmfHx80Lp1a3h4eGDIkCFITExUeq+q85slVddIJycneHt7Y/bs2Xj16lW5\n", 419 | "971hwwa8//77aNOmDRYtWqTHVBsvbcqgtLQ0viwtrZxQdRwzMzPx/PlzvaS5NLm5uXj06BH//E3X\n", 420 | "I5KTkzFy5Eh4enrCyckJPj4+mDVrlk7lamFhIR48eFCuz169ehWffPIJX4cEyp/PzcqVAj2aMmUK\n", 421 | "atWqxT/PzMxESkoKduzYgUuXLmH79u2oWrWqAVPInq+vLyQSieA4vMnv/t///gcAyM/Px9OnT5GU\n", 422 | "lISvvvoKZ8+exfTp0/n3urm5YcGCBXj33XffeDq19e233yI1NRWbNm3ity1YsABNmjRh9p1TpkxB\n", 423 | "9erV+efJycmYNGkSdu3aBSsrK2bfO3LkSDRr1ox/LpPJcPbsWezatQtnz57FL7/8AhsbG2bfr46J\n", 424 | "iQn//1GjRjG7QSjNkiVLsHr1anh5eSEsLAxWVla4ceMGduzYgcTERGzduhXvvPPOG0+X2C1btgyr\n", 425 | "Vq1C8+bN0adPH9SrVw+3b99GXFwcDh48iE2bNqFVq1aGTmapHj58iL59+wIAevfujXfeeQfp6ek4\n", 426 | "cOAAJkyYgAsXLmDy5Mn8+0ue329K8WukTCbDrVu38PPPP+PixYvYsmULTE21a9+6fv065s2bB2dn\n", 427 | "Z/Tp0wcODg4skm2UipdJpfnll19gZWWF7OxsJCQkYOjQoUrvUXUcL126hFGjRmHx4sWoXbu2vpPP\n", 428 | "e/DgAYYOHYqRI0ciICCA2feokpeXh+nTp2PHjh1wdnZGcHAwbG1tcfnyZWzbtg0HDx7Eli1b8Pbb\n", 429 | "b2u138zMTISGhsLLywtjxozROl3ffPMN7t69i4kTJ6JevXq65XPOQCZPnsxJpVLuwYMHKl+fPn06\n", 430 | "J5VKuX379r3hlFUOJ0+e5KRSKbdixQql1woLC7lJkyaJ+vh37dqVGzRokEHTsHz58lLzuL72f/r0\n", 431 | "aZWvR0VFcVKplIuKimLy/aUxhuP/zz//cA4ODtzs2bOVXjt37hzXsmVLbvz48QZIGTs7duwoNU/o\n", 432 | "w/bt2zmpVMpNmzZN6bU7d+5w7u7u3Pvvv8/l5OS8sTSVx9SpU7m2bdty//77r9JrI0aM4Ozt7blb\n", 433 | "t24ZIGVypV0jY2NjOalUyh05ckTr/SYkJJT7s2I2cOBAztvbW6P3du/enRsxYgTn7+/P+fv7q3yP\n", 434 | "quP4pvK64vq9c+dOfhvr643CsmXLOKlUyq1atUrptTNnznCOjo5cQECA1vtNS0tTWyfRRKtWrbix\n", 435 | "Y8fyz3XJ50YbY6y4C7pw4YKBU1L5mJiYYNq0abCxsUFUVJShk1NunJEs6miodFT2c+j8+fMoLCzE\n", 436 | "+++/r/Sas7MzWrdujfPnzxsgZeKVl5eH+fPno2nTppg2bZrS63Z2dhg2bBiePn2KQ4cOGSCFmjt3\n", 437 | "7hzeffddNGjQQOm1kJAQAMBff/31ppOlEXd3dwDArVu3tP5sXl4eAKBatWp6TVNFceXKFdy5cwfu\n", 438 | "7u7w9PTEzZs3cenSJaX3lXYc31SZ/6avLU+fPsWaNWvQvn17jBgxQul1V1dXfPLJJ7h27dobv+7k\n", 439 | "5+cLenR0yedGWzG2tLQEoPzDHzlyBIGBgXB2doa7uzvGjRuH1NRUpc/v3r0bPXv2RJs2bdCjRw8k\n", 440 | "JiYiNDRUECPm7e2NiIgITJkyBa1bt4anpyfS09MByAvNIUOGwMXFBS4uLhg2bJjSD/3y5UuEh4fD\n", 441 | "y8sLTk5O+N///ofFixcjNzeXf09ubi6+++47+Pj4wMnJCV5eXpg5c6YgPkxVbNCLFy8wffp0dO7c\n", 442 | "GU5OTvDz88OaNWtQWFgo+Fzr1q1x7949jBgxAi4uLnB3d0d4eDj/d5RX9erV0bVrV1y5coWPl1IV\n", 443 | "K/jrr7+id+/ecHFxgaurK4YOHYqzZ88K9sVxHDZt2gR/f3+0adMGPj4+WLRoEWQyGf+egoIC/PTT\n", 444 | "T/jggw/g5OSEzp07Y/r06Xjx4gX/HkVcdEpKCmbMmIEOHTrA2dkZoaGhuHbtGv8+xbE8c+YM7O3t\n", 445 | "sWvXLn578Rhje3t7rFmzBuvXr0e3bt3g5OSEnj17Yv/+/YL0q4tNLrm9eAxieHg4fvjhBwCAj48P\n", 446 | "Bg0ahJ9//hn29vZITk5W2le/fv3Qp08fwd+5cuVKlb+NptSdQ5rkbQDYsmUL+vTpAxcXF7Ru3Rrd\n", 447 | "u3dXeaO0b98+fPzxx2jTpg169uyJ06dPK72nZHzfoEGDMHz4cBw7doyPC/Py8sLKlSuV0nv+/HmE\n", 448 | "hITAxcUFXbp0wcqVK7Fy5UrY29uX+vfXqFEDALBjxw6+kCxu06ZNOHz4sGBbZmYm5syZAy8vLzg7\n", 449 | "O6Nnz56Ii4sTvEebc/PgwYPo1KkTXFxcsH37dgDycmPWrFn85z/88ENByI/Cli1b0LNnTzg7O8PD\n", 450 | "wwNjxozRuCL06NEjhIWFoW3btujUqRNmz56NzMxMAEBGRgZat26N8ePHq/xOe3t73L59W+V+U1JS\n", 451 | "8PLlS/Tp00dtt3RwcDCOHj2K7t27q03fvXv3MHnyZHTp0gWOjo7w8PDAyJEjlf4+TcqX06dPIzg4\n", 452 | "GG5ubmjbti2CgoJw5MiRUo8PIM8f169fx7lz55Re69ChAy5fvozevXvz24qf36rGaBR/KMocQPPz\n", 453 | "TRv//vsvACiFht26dQthYWFwc3ODs7MzgoKCcPz4cf71QYMGYcqUKQDklf/i59D169cxevRouLm5\n", 454 | "oU2bNujfvz+SkpIE+1ect0uWLEHbtm3RsWNH3Lx5U6PvLk1iYiIGDhwIV1dXODo6wsfHBwsWLBBc\n", 455 | "S7UpM1JSUhAYGIi2bdvC19dXZcy4Or/88gsA+c2HIswwPj5e6TiUPI4rV64UbCte3v3777/46quv\n", 456 | "0L59e7Ru3RoBAQH89yiEh4eje/fuiImJgZubG9zd3VUev/j4eAwePBgA8PXXXyuVg/fu3cPIkSPR\n", 457 | "tm1beHh44Ouvv8bLly8F79G0DCrp119/RX5+Pvr166f2PZ9//jmOHz+O1q1bA1Af+1x8+6lTp9Ct\n", 458 | "WzcA4Mt2xfuzs7OxaNEieHt7w9HREd7e3oL6g6JeAgA7d+6Evb09vL291eZzTRg8xlid3377DQAE\n", 459 | "cSHx8fH45ptv0LFjR0yaNAkvX77Eli1b0K9fP2zbtg0SiQQAEBMTg1mzZsHd3R2BgYG4efMmvvzy\n", 460 | "S9SoUQNSqVTwPXv27EHz5s0xdepUPHnyBLa2tjhx4gRGjBiBli1bYvz48cjJyUF8fDwGDhyIdevW\n", 461 | "wdXVFQAwfvx4XL16FYMHD0a9evVw7tw5rFmzBunp6Zg5cyYAYObMmdi7dy9CQkLQpEkT3LhxAzEx\n", 462 | "Mbh37x7Wrl2r8m9/+fIlAgMD8c8//yAoKAh2dnY4fvw4Fi9ejKtXr2LJkiX8ewsLCxESEgJXV1eE\n", 463 | "h4fjwoUL2L59O2QyGZYuXarTb9C8eXMA8gKzQ4cOSq+fPn0aEyZMgJeXF/r374+srCxs3rwZQ4YM\n", 464 | "wZ49e/jYzRkzZmDr1q3o2rUrgoODcefOHaxbtw6pqalYsWIFAGDChAk4cOAAfH19ERoaitu3b2Pr\n", 465 | "1q04efIk4uLiULNmTf57p06divr162PMmDF48eIF1q5di88++wxHjhxBlSpVMH/+fMydOxe1a9fm\n", 466 | "Cwh1tm7dCo7jMGDAAFhaWmLjxo2YMGECmjVrhvfee0/rY6aoLAQGBuL169c4ePAgpkyZgvfeew8t\n", 467 | "W7bErFmzsH//fnh6evKfSUtLw4ULF/hKdvPmzbFgwQKlvKotVeeQpnlbEZsbEBCA/v374/Xr19i1\n", 468 | "axcWLVqE6tWrY8CAAQDk5+SUKVPQtm1bfPXVV7h79y5GjBiBwsJCNG7cWOWxUbhx4wYmTJiA/v37\n", 469 | "IygoCAkJCVi5ciVq167N7//SpUsICQnhf++srCxs2rQJJiYmZcYLenh4oHHjxjhw4AC6du0KX19f\n", 470 | "dOzYEe7u7rC2tlYau5Cbm4vg4GDcunUL/fv3h729PY4ePYqIiAjIZDIMGjRIq3MzPz8f06ZNw9Ch\n", 471 | "Q5GTkwNXV1dkZWVh4MCBePToEYKCgtCwYUP8/vvvmDNnDlJTU/Htt98CABISEjBjxgwEBAQgJCQE\n", 472 | "z58/x8aNGzFo0CAcPHiQr/SrExERgVatWuHLL7/E9evXERMTg5s3b2Ljxo2oWbMmvLy8kJycDJlM\n", 473 | "xt9AAcDevXthb28viFkvTtFy5uzsrPa7q1evXmos7tOnT9GvXz9YW1sjJCQEtra2uHr1KrZt24Yr\n", 474 | "V67g8OHDMDMz06h8uXPnDkaMGIFWrVph4sSJKCwsRFxcHEaPHo3NmzejXbt2atPRu3dv/PXXXxgw\n", 475 | "YAA8PDzg5eWF9u3bw97eXm3+UmxTnKPF5eXl4fvvvwfHcfz3anq+lebly5f8b5SXl4fbt29j9uzZ\n", 476 | "aNWqlaDydf36dQwYMAD169fHyJEjUaVKFezduxefffYZFi5ciA8//BCjRo2CnZ0dtm3bJhibcOHC\n", 477 | "BYSEhKBmzZoYOnQoqlWrht27d2PMmDGIiIhAcHAw/z1//vkn0tLSMHnyZNy/fx/NmzfX6LvViYuL\n", 478 | "Q0REBHx8fDBp0iTk5eXh119/5a+PkyZN4t+rSZmRkpKCTz/9FO+++y7Gjx+P58+fY8qUKTA1NYWt\n", 479 | "rW2px7qwsBB79+5Fw4YN4ejoCI7jUK9ePezduxfh4eEwNzcHAJXHsUWLFnj8+DG/zcnJCYD8JrVv\n", 480 | "374wMTHB4MGDYW1tjUOHDmHSpEl4/Pgxhg0bxn//w4cPsWrVKowbNw6PHz9WeZ65ublhxIgRWL16\n", 481 | "Nfr376+Uh0aPHg0fHx9MmTIFf/75J3bu3IlXr17xDTWalkGqXL58GUDp5395YqubN2+Or7/+GnPn\n", 482 | "zuXHPdWqVQu5ubkYMmQIzp8/j969e8PR0RHnz59HVFQU/vzzT2zatAlubm6YP38+vvrqK7i5uaFf\n", 483 | "v37Izs7GpUuXlPK5xsoVzKEHivipK1eucM+ePeMf9+7d4zZv3sw5OztzPXr04PLz8zmO47iMjAzO\n", 484 | "xcWFmzhxomA/T5484dzd3bmwsDCO4zguMzOTa9euHTdw4ECusLCQf9/GjRs5qVQqiHvs2rUr17Jl\n", 485 | "S+7x48f8toKCAs7Hx4cbMGCA4PNZWVmcr68v16tXL47jOO7p06ecVCrl1q1bJ0jP119/zQ0ZMoR/\n", 486 | "3rp1a27WrFmC9yxdupTr06cPl5WVxXGccmzQggULOKlUyiUlJQk+N2PGDE4qlXJHjx4VfG7evHmC\n", 487 | "9w0fPpxr1aoVJ5PJVB98rvQYY4Vt27ZxUqmU27t3L8dxyvFT06ZN49q1ayf4zPXr1zk/Pz/u119/\n", 488 | "5TiO427evMlJpVIuIiJC8L4lS5bwMXzJycmcVCrl5syZI3hPYmIiJ5VKufnz5wvS3LdvX8Fvs2bN\n", 489 | "Gk4qlXInTpzgt6mKcZVKpVx4eLjgedu2bbmnT5/y286fP89JpVJu8eLFaj+nbnvJ71QV8zVy5EjO\n", 490 | "zc2Ny83N5betWrWKc3BwEORDTSj2n5SUJDiH0tLSuF27dnHt27fnOnbsyL18+ZLjOM3zdm5uLteu\n", 491 | "XTulcy0jI4NzcnLiRo0axXEcx+Xn53MdOnTg+vbty5+nHMdx8fHxSudayfi+gQMHKsV/5eTkcO7u\n", 492 | "7lxgYCC/LSQkhHN3d+eeP3/Ob7ty5Qrn4ODA2dvbl3mM7t69ywUEBHBSqZR/ODg4cCEhIdzx48cF\n", 493 | "742JieGkUim3Z88ewfbg4GDu/fff5woLC7U+N0vGdy9fvpxzdHTkbty4Idi+ePFiTiqVcteuXeM4\n", 494 | "Tn4Ol4xtPHr0KOfv78+dPXtW7d+rOEeDgoK4goICfvuKFSs4qVTKHT58mOM4jjtw4IDSGIJ///2X\n", 495 | "c3BwKDUmXTH24+7du2rfoy5NinJj9erVnIODA3fnzh3B+xYtWsRfEzhOs/JFce6/ePGCf8+LFy+4\n", 496 | "Dz74gNu8eXOZaYuMjOQcHR0F+aNTp07c/PnzuYyMDMF7y4qbnz59Omdvb8/naU3PN3UU10hVjzZt\n", 497 | "2nDnz58XvH/gwIGcr68vl52dzW/Lz8/ngoODuU6dOvFljqo42L59+3IuLi6CeOucnBwuICCAa9Om\n", 498 | "DX98Fedteb9ble7duwvOecVnPT09uY8++kjwHZqUGQEBAVzXrl25zMxMfpvi2lFWjPHvv//OSaVS\n", 499 | "wTV75syZnFQq5RITEwXvVXUcVW2bPHky5+HhwT158kTw+YkTJ3JOTk7cs2fP+PdpOq6ntBjj7777\n", 500 | "TvDekJAQrlWrVvxvUFYZdPXqVbXf++mnn3L29val/p4lqYt9LrldVYyxIpZ+48aNgs/+9NNPnFQq\n", 501 | "5WJiYvhtJa/HusR7GzyUIiAgAB07duQfvr6+WLhwIXx8fBATE4MqVaoAkN95v379Gj4+Pnj+/Dn/\n", 502 | "MDU1hYeHB44fP46CggKcPHkSmZmZGDx4sOCOPygoSGUrS5MmTVCvXj3++ZUrV3D//n34+PjgxYsX\n", 503 | "/PdkZ2fDy8sLV69exePHj1GzZk1Uq1YNMTExOHDgALKysgAAc+bMwbp16/j9vfXWW9i7dy9/1wbI\n", 504 | "uxri4uLUzlRw+PBhNG/eHD4+PoLto0eP5l8vrmSXpb29PfLz83UOp1B0P6trmWvYsCEyMzMxe/Zs\n", 505 | "vuu1RYsWSExMhK+vLwDg6NGjAIpi9hSGDRuGhIQENGnShP97SsYs+fn5wc7OTilW0dfXV5AmRTdJ\n", 506 | "eaaJcXV1RZ06dZT2xWqqwJ49e+LVq1c4ceIEv23v3r1wd3cX5ENthIWFCc6hbt264ZtvvkGbNm0Q\n", 507 | "FxcHa2trAJrn7apVqyIlJYXv9VB48eIFqlevzuf1y5cv4/nz5/jkk0/48xQAPv74Y41mwbCysoKX\n", 508 | "lxf/3NzcHBKJhD/2L1++xJkzZ/DRRx8JZmxxcHBAx44dNTo2EokE8fHx2LRpE0JCQtCsWTNwHIdT\n", 509 | "p05h2LBhgtCQo0ePok6dOujRo4dgH/Pnz0dsbCxMTEy0Pjfd3NwEzw8cOIAWLVqgbt26gnJMsT/F\n", 510 | "+dKwYUPcuXMHK1eu5Kcv8vT0xC+//FJqD4hCaGioYLYCxfmn2L+npydq1qwp6GJOTEwEx3FKf39x\n", 511 | "it+5oKCgzDSo89lnn+H48eOws7Pjt8lkMv6cfv36NQDNypeGDRsCkPfMKVqzbG1tsX//fkErpzoj\n", 512 | "R45EcnIyvv32W3h5eaF69ep4+vQp1q5di169emlcpsTFxWHLli0YPXo0n6c1Pd/KsnDhQqxfvx7r\n", 513 | "16/HmjVrMG3aNDRu3BjBwcH4/fffAcjPzTNnzqBLly7Iysriv+vly5fo1q0bnj59iosXL6rc/9On\n", 514 | "T3HhwgV8/PHHgnhrc3NzDB8+HDKZDCkpKfx2Kysrvptc1+8G5KELa9asUUpTzZo1+bKm+HeXVmY8\n", 515 | "e/YMV65cQY8ePQS9Fh4eHhr1wO3ZswcA+PwFQG04hSYKCwuRlJQENzc3VKlSRXDO+/r6Ijc3V3Bs\n", 516 | "AeUyQ1v+/v6C546OjoL6gKZlkCqKMqV42BhLhw8fRs2aNZXO5ZCQENSoUYPZOAaDh1IsXLgQderU\n", 517 | "QX5+PpKTkxEbG4vu3btj+vTpfLcFAH4+1okTJ6rcj4mJCZ4/f87Podm0aVPB61WrVlU5LVPxSlHx\n", 518 | "75k/fz7mz5+v8nsePnyI+vXrY+bMmYiIiMC4ceNgbm4ONzc3fPDBB+jVqxef9unTp2P8+PH4+uuv\n", 519 | "YWZmBmdnZ3Tr1g19+vRR2x16//59QVe7Qt26dVGzZk2lWJ2SXReK79bl4gWAP5HUdY0EBwfj+PHj\n", 520 | "2Lx5MzZv3ozGjRvDy8sLffr04SuYiot6yd+jZs2afHjE/fv3YWNjo/J73n33XaU4K3V/b3lOVlbH\n", 521 | "Th1vb29Uq1YN+/fvh5eXF27fvo0bN25g9uzZ5d5neHg4pFIpCgsL8eeff2Lt2rVo3749vv/+e0EF\n", 522 | "VZu8bWZmhiNHjuDQoUO4e/cu/v77bz5OTXGcFb9tyThHU1NTpd9bFVXTE5qbm/PHPi0tDYWFhXyI\n", 523 | "VHGq8kVp3N3d+QFLDx8+xI4dO7Bq1SosW7YMH3/8MerXr48HDx6oLCMaNWrE/1/Xc/Pvv/9GTk6O\n", 524 | "ytAkExMT/vNhYWE4d+4cH0/dvHlzeHt7o2/fvhpNL1dySkVra2tYW1vz+zc3N4evry/27t3Lh1Ps\n", 525 | "3bsXLi4ufGVTlbp16wIAnj9/rn33ZDG5ublYsmQJLl++jLS0NNy/f5//3bn/4kU1KV/8/Pxw8OBB\n", 526 | "7Nu3D/v27UO9evXg6emJXr16aRSmAIDvhh8wYADy8/Px+++/Y/ny5bh48SJWrlwpmLJSlbNnz2LG\n", 527 | "jBno0qULxo4dy2/X5nwrjYuLiyAPAvLGEF9fX8yaNQv79u3j52yPjo5GdHS0yu9SxCWXpDiPi9+o\n", 528 | "KCjyUfG5ZUuGI+jy3YD8ZuvixYvYs2cP7ty5g7S0NL6iW3LKr7LKDHVlkuLvK62Cnpubi19//RU1\n", 529 | "atRAw4YNcf/+fQDy879atWpISUnBkydPtGrAePHiBTIzM3Hw4EEcPHhQ6fXi57xCyTqJtkqWOcXD\n", 530 | "cICyy6CHDx+q3Xe9evXAcRyePXumlCdZuH//Pt555x1Bwwsgr881bty41LTqwuAV4+InfefOnSGR\n", 531 | "SDB79mykp6fjxx9/5N+nuBjPmjVLKXZRwcbGBvn5+QAgqFQrqNpWcg5IxfeMHz8ebdq0Ufk9igLE\n", 532 | "398fnTt3RlJSEpKTk5GSkoITJ04gNjYW27Ztg7m5OTp06ICjR4/iyJEjOHLkCE6cOIF58+Zh48aN\n", 533 | "2LFjh9bxOIWFhUqxkdrOY6mpq1evwtTUVO2ddo0aNRAdHY3z588jKSkJx44dw+bNmxEbG4v58+fD\n", 534 | "39+fL7BKiwflShlZy/rv1XRey5LKW3G2tLREt27dcOjQIeTl5WHfvn0wNzfHBx98UK79AUCrVq34\n", 535 | "VoZOnTrB0dERYWFhGD58OGJiYpRuHMrK2xzHYfTo0Th69ChcXV3Rrl07BAUFwdXVlR/0ARQdu+KD\n", 536 | "KBU0OT5lHfvSzmULC4sy9x8dHY3c3FxBDB8gb2UcM2YMLCwssGjRIpw/fx7/+9//UFBQUO78AKjO\n", 537 | "qyUL9MLCQri6uiIsLEzlPhSVpAYNGiAhIQEnT57EoUOH8Ntvv/EDRdetW1dmq5Kqv4PjOEF6/P39\n", 538 | "sWPHDhw+fBhOTk64ePFiqfGFgLy8BuQDytSl4cmTJwgLCxMMKC3ujz/+wLBhw1CjRg0+5rtVq1a4\n", 539 | "d++eoJdCk/LFzMwMy5Ytw40bN3DgwAH89ttviI+Px/bt2zFx4kR89tlnKtN469Yt7NixAwEBAWjR\n", 540 | "ogW/3czMDJ07d4arqyu8vb2VBvqV9OjRI4wbNw5vvfUWFi5cKHhNm2uJtmxtbeHu7o6kpCRkZGTw\n", 541 | "59vAgQOVejMUFGNGSiqr/AUgyNcly19dvhuQX9NjYmLQsmVLtG3bFgEBAWjbti1mzpypVPEp6/ws\n", 542 | "rUwqq+EkOTkZGRkZAIpaiUvavXs3hg8fXup+ilMcGz8/P/Tv31/le0re6OpSBgFlXx8LCgo0KoNU\n", 543 | "cXFxQVxcHP766y+1FeOLFy9i/vz5CA0NVZsfFOkoi7Z1A30xeMW4pIEDB+L333/HoUOHsGHDBoSG\n", 544 | "hgIAXxmuVauW0p3OH3/8gYKCApibm/Pvu3v3rqDViuM4/P3332UOqFLcoVpZWSl9z6VLl/Dq1StY\n", 545 | "WlpCJpPh8uXLeO+999C7d2/07t0beXl5WLBgATZt2oQTJ07g/fffx9WrV9GgQQN8+OGH+PDDD8Fx\n", 546 | "HNavX4/58+cjMTFRZXff22+/jTt37ihtf/LkCV6/fl1qi46+ZGZm4vjx42jbtq3aAQtpaWlIT09H\n", 547 | "mzZt0KZNG3zxxRe4ffs2goODsW7dOvj7+/Mnz7179wQtTI8ePcLcuXMxaNAgNG7cGCdOnMCzZ8+U\n", 548 | "7pbv3r37Rv7e0piamgpGRwPlC9tQ8Pf3R0JCAs6cOYNDhw6hc+fOgsGFulKMnt+0aRMWLFiAb775\n", 549 | "BoDmefuPP/7A0aNHERYWJmgBy8/Px4sXL/iCXPFvyVlhOI7DgwcPBBWO8lDsX9W5cO/evTIvIElJ\n", 550 | "Sbh48SIGDBigMmxJURYoWlQaNWqEGzduKL0vOTkZiYmJmDRpks7n5ttvv43MzEyl45+RkYGTJ0/y\n", 551 | "Zdbt27dRWFiIDh068O89e/YsQkJCEB0dXWbF+P79+4Lz7fnz58jIyBBchNu3b4969erh8OHDePLk\n", 552 | "CczMzEqdSQKQXxjr1KmDnTt3Yvjw4Sovwnv27MGFCxfUhmQsX74cVlZW2LNnj6AFsGRrnibly6NH\n", 553 | "j3D//n20a9cOLVq0wJgxY/Do0SOEhITwg3JVSU9Px/r161G9enWV+dTKygqNGjWCmZn6S2ROTg5G\n", 554 | "jx6NzMxM/PTTT3zIkoKm51t5FRYW8oMEFd9lamqq9F23b9/G/fv31YbuKT6raiaSu3fvAkCp+VqX\n", 555 | "737w4AHm0K4DAAAgAElEQVRiYmLQq1cvzJs3T/DakydP1H5naWkxMTFROVOVogVYHcUsEeHh4UqV\n", 556 | "1cePH2PGjBl8vtdU7dq1YWVlhby8PKVj8+jRI1y+fJnp4k+qNG7cWKMySBVPT0+Ym5sjLi5O7YDK\n", 557 | "3bt348yZM3wjiqKMKM819O2338b58+eRn58vOBdzc3Nx//59vhdQ3wweY6zKzJkzYWNjg2XLlvGZ\n", 558 | "uWPHjrCwsMDatWv5liRAnmFHjhzJ36137twZVlZW/GwDComJiYKpv9RxcnJCvXr1EB0dLYhvev36\n", 559 | "NSZMmIDw8HCYmZnh5s2bCA4O5qdgAuR31YoZAKpUqYKXL1+if//+gvgpExMTODo6AlB/Z9e1a1fc\n", 560 | "vn1baaocxX6Kx1ixwHEc5syZg+zs7FILgdmzZ2PUqFGC42RnZ4eaNWvymVjR7bx161bBZ+Pj47F/\n", 561 | "/37UqFGDH1m9evVqwXuSkpKQmpparr/X1NRUb3M81q1bVzAdHCCfokyTNADKd8YdO3ZE7dq1ERcX\n", 562 | "h2vXrpUa01leX3zxBd555x3ExMTwc/VqmrcVITQlu8q3bdsGmUzG/z0tW7bE22+/jS1btghaaPbu\n", 563 | "3atzfDsg71Js27Yt9u7dK5jeMC0tDceOHSvz8x999BGysrIwb948pbygmL3AxsaGr2R6enryKz8W\n", 564 | "t3HjRiQnJ6NWrVo6n5ve3t64du2a0pR9q1atwtixY/mprz7//HNMmjRJ0Mrl4OCAqlWrKrVCq7Jt\n", 565 | "2zbBc8UIf8WUSIC8LOrRoweOHz+O5ORkdOjQoczVN83MzDB27FjcvXsXc+bMUXr92rVrWL58OerX\n", 566 | "r8+vKldSeno66tSpI/iujIwM7Ny5E0BRT4Em5cvq1asRGhoqWBq3QYMGfDiQOi4uLnj77bcRHR3N\n", 567 | "H/PiLly4gGvXrpXa4hUREYHLly9jxowZKqeD0vR8K4+nT5/i5MmTcHBwQI0aNVC/fn04Ojpi586d\n", 568 | "grjl/Px8fPPNNxg3bpzaFrp69erB0dERCQkJguOYm5uL9evXw8LCAp06dVKbFl2+WxGeVTL0Jzk5\n", 569 | "Gffu3dO6Z6527dpwdXVFQkKCYJzIuXPncOXKFbWfy8zMxNGjR9GkSRO+pbP4IygoCC1btsTt27dL\n", 570 | "nWqvZJlvZmaGLl26IDk5WekaMnfuXIwePVpQVmraWqwoA8oTPqhpGaRK7dq1MXjwYPz++++CsVQK\n", 571 | "x48fR2xsLOzt7fmyRtECXfz4Z2ZmIjk5WfD3Kv6m4mW1t7c3MjMzERMTI/ie2NhYZGVlMasLGV2L\n", 572 | "MSC/IH755ZeIiIjAtGnTsHbtWtSqVQsTJkzAvHnz0L9/f/Ts2RMFBQWIjY1Fbm4uv3RnzZo1MW7c\n", 573 | "OHz//fcIDQ2Fr68vUlNT8fPPP6Nq1aplZjwzMzNMnToVEyZMQEBAAPr06QNLS0ts374dDx48wMKF\n", 574 | "C2FqagonJyd4eHhgyZIl+OeffyCVSvHw4UNs3rwZzZo1Q8eOHWFmZoZevXrxP2Lbtm2Rnp6OzZs3\n", 575 | "o27dumpbZ0aMGMEvSxoUFISmTZvi5MmTOHjwIHx9fdG5c2e9Hevr169j9+7dAOQns6JicOHCBQwe\n", 576 | "PBhdu3ZV+9khQ4Zg6NChCA4O5uOqk5KSkJaWxrc0Ojg4oG/fvoiOjsbjx4/Rvn17fknTgIAASKVS\n", 577 | "SKVS+Pj4YNOmTfj333/h4eGB1NRUbNmyBU2aNFE5kXhZ6tSpg6tXr2LLli1wd3fXKR6yR48eWL9+\n", 578 | "PcaMGQNPT09cvnwZ+/fvLzMMRtH6vXbtWnTp0oW/AVC0zMXExKBatWpKF99nz57h+PHjsLe3L/eU\n", 579 | "bRYWFpg+fTqGDRuGqVOnYufOnRrnbRcXF9SoUQNz5szBgwcPYG1tjVOnTuHo0aNo1KgRPx+uiYkJ\n", 580 | "IiIiEBYWhv79++OTTz7Bo0ePEBsbCxsbG6XKaFnPVW2fPHkyBg0ahD59+iAwMBA5OTmIjo7W6Kbn\n", 581 | "k08+wW+//Yaff/4Z586dQ/fu3VG/fn08e/YMiYmJuHnzJhYtWsS32gUGBmLHjh2YMGECgoODIZFI\n", 582 | "cPToUaSkpGDu3LkwNTXV+dxUfH7MmDEIDAxE8+bN8ddff2Hnzp3w9PTkbySHDx+O8PBwDB48GH5+\n", 583 | "fuA4Drt370ZeXh4/LVVp/vzzT4wePRqenp44e/Ysdu/ejQ8//BAeHh6C9/n7+2PDhg1ISUnB999/\n", 584 | "X+Z+AaB///64cuUKNm/ejNOnT8Pf3x81a9bE1atXER8fD0tLSyxbtkztxPqenp6IiorC+PHj0alT\n", 585 | "Jzx58gTbt2/nB0spBt9pUr4EBwdj586dGDhwIPr37w9ra2ucPHkSZ86cweeff672bzA1NcWiRYsw\n", 586 | "dOhQ9O7dGz169ICTkxPMzMxw6dIl7N69G05OTnyPZUkxMTFISEiAm5sbrKyskJCQIMiTTZs2hbOz\n", 587 | "s0bnW1kOHjzI99pxHId///0X27ZtQ05ODiZMmMC/b+rUqRg8eDA++eQTBAUFoVatWti3bx/++usv\n", 588 | "fPHFF6UOiFV8tnfv3hgwYACqVauGhIQEXL16FVOnThWMh1F17pX3u5s3b45GjRph9erVyM3NRYMG\n", 589 | "DXDhwgX88ssvsLOzU2o11qTMCA8PR3BwMPr164fg4GBkZWVh48aNqFWrltrPHzhwALm5uYJ5q0sK\n", 590 | "CgpCREQE4uPjBYMPi1OU+Vu2bMHTp0/h7++PL7/8EidPnkRwcDCCg4PRqFEjHDt2DIcPH0ZgYKDg\n", 591 | "2qRpY47ipnL37t0oLCzUalloTcsgdRSV5/nz5yMpKQndunWDhYUFzp07h3379qFu3bqCqWJ9fHww\n", 592 | "e/ZszJo1C//88w+qVq2Kbdu2oXr16oLf19bWFqampkhKSsJbb72FDz74AH379sWuXbswb9483Lhx\n", 593 | "A61atcKlS5ewc+dOODs7q7351pXBKsZlzUOqOCApKSnYvXs3Pv74Y4SGhuKtt97C+vXrsXTpUlhY\n", 594 | "WMDR0RELFy4UjNQeMmQILCwssHHjRsybNw8SiQRLly7FzJkzVcYrlvTBBx9g7dq1WLVqFSIjI2Fq\n", 595 | "aooWLVogMjJSkGlWrFiBlStX4vDhw9i2bRtsbGzg5+eHzz//nG8JmDFjBho1aoS9e/di3759sLKy\n", 596 | "QseOHTFhwgS+sCt5LGxsbPDzzz9j6dKl2LdvH169eoUmTZpg8uTJgoK6tLk2NY3FSkpK4gcFmJqa\n", 597 | "wtraGq1atcLSpUvh5+en9nOAvCs2MjISa9aswY8//oicnBy0aNECixcvFnSzzJw5ExKJBNu2bcPh\n", 598 | "w4fRqFEjjBkzRtAavWzZMkRFRWHXrl04cuQI6tati8DAQIwbN05QKKv7u0puHzt2LL799lvMnTsX\n", 599 | "YWFhOlWMx48fj4KCAuzduxfHjx+Hs7MzNm7ciC+//LLUz/Xo0QMHDhxAfHw8zpw5I5hztGfPnoiJ\n", 600 | "iYGPj49SvOytW7cwefJkjBkzptSKcVm/c6dOndCzZ0/s2bMHa9aswejRozXK23Xq1MHq1auxcOFC\n", 601 | "REZGomrVqujYsSPi4+MRHx+PdevW4fnz56hduza8vLywevVqrFixAkuWLEGDBg0wZ84cbNy4USlt\n", 602 | "ZT1Xtd3Z2Rk//fQTlixZgqVLl6JWrVoYPHgwbt26hV9//VXt367Yz9KlS7Fr1y4kJCQgOjoar169\n", 603 | "gq2tLdq1a4dZs2bxvTeA/GYiOjoaS5cuxZ49e5CZmYlmzZph2bJlfAy4ruem4vPLli3D/v378fLl\n", 604 | "SzRq1AhhYWGCbv9evXrBxMQEmzZtwpIlS1BQUAAnJydERUVpFF+8ZMkS/PTTT5gzZw5sbW0xatQo\n", 605 | "jBkzRum9jo6OaNq0KR49eqQ2rlLV/mfOnIkuXbpgy5Yt2Lx5M54/f466desiICAAI0eOVBo0VfxY\n", 606 | "jB07Fvn5+UhMTMShQ4cgkUgwbNgw9OrVC+7u7jh58iS6deumUfnSrFkzrF+/HitXrsS6deuQmZkJ\n", 607 | "Ozs7pbl3VXF2dsaePXuwdu1anDhxgl/Yp2nTpggLC8OQIUPUxjBeunQJJiYm+OOPPwQLHikEBATA\n", 608 | "2dlZ42uJuuMMyFsWFapUqQIbGxu0bt0ac+fOFdzoODs7Y8uWLVi+fDk2bNiA/Px82NnZYd68eejV\n", 609 | "q5fKfav67Lp161BQUICWLVvihx9+EJRbqj6r7XcXZ25ujjVr1mDu3LnYuHEjCgsL0a5dO0RHR+PK\n", 610 | "lSuYPn06rly5gpYtW6r97pLbW7VqhejoaCxatAgrVqyAra0txo0bh3Pnzqld6XLPnj2oUqVKqWn1\n", 611 | "9/fH999/j8TERH7xiJLp6dChA7p3744jR47g5MmT8PX1xTvvvIO4uDgsW7YMcXFxyMrKQpMmTfD1\n", 612 | "118LZmvS5Lqt0KxZMwwcOBA7d+7EpUuX4OHhoXF9QNMySB1zc3P88MMPSEhIwI4dO7Bu3Tqkp6ej\n", 613 | "YcOGCAkJwYgRIwS9QbVr10ZUVBQWLVqE5cuXo3bt2ujXrx8kEgm++OIL/n1WVlYYP3481q5dizlz\n", 614 | "5kAikcDNzQ0bNmzAypUrsX//fiQkJKBhw4YYOXIkRo0aVWbvWXnjtU04ffU3G4nc3FzIZDKleC9A\n", 615 | "3n3m6+urFMtEyJv2119/ITAwEFFRUXrtAahonj59ys+EUNzIkSNx48YNpenRiPb8/PzQsmVLLF68\n", 616 | "2NBJIYQQgyuzH+f8+fP8Mpj37t1DUFAQgoODMX36dL7Zf9u2bejduzf69+9f6hx4b8Ljx4/h7u6u\n", 617 | "NC/i0aNHkZWVxa9GQ4ghbd26FQ0aNMD7779v6KQYtX79+inFuT99+hSnTp2ic1kPTp06hdTUVHzy\n", 618 | "ySeGTgohhBiFUkMpoqKikJCQwMd+zZ07FxMnToSbmxumTZuGQ4cOoU2bNoiOjkZ8fDxycnIQFBSE\n", 619 | "jh07ahSywELjxo3h5uaGH3/8Eenp6bCzs0NaWhpiY2NhZ2encvogQt6UqVOnIi0tDadOnUJ4eLjO\n", 620 | "U/NUdAEBAfjhhx/wxRdfwMPDA69eveIHlqkKDSCaUYQsnThxAg4ODnSDRggh/ym1Yty0aVOsXLkS\n", 621 | "X331FQD5qEJFfFuXLl1w4sQJfrBO1apVUbVqVTRt2hTXr183aGvOjz/+iMjISBw8eBCPHj1CnTp1\n", 622 | "0LNnT4wfP16j+U8JYeX58+e4ePEiAgMDBXMCE9XGjBmDOnXq4Oeff8bhw4dhYWGBdu3aYcWKFWVO\n", 623 | "vUjUMzMzw2+//QY7OzssWrTI0MkhhBCjUWrF2NfXVzD3X/Fw5OrVqyMjIwOZmZmCOVirV6/Oj1o3\n", 624 | "lJo1a+Krr77iK/SEGIvii9aQspmYmPCrkhH98ff3V1o6lhBCiJbzGBefWiYzMxPW1taoUaMGP70O\n", 625 | "IJ9qR9XAN0IIIYQQQoyZVhVjBwcHnD59GgBw7NgxuLq6onXr1vjjjz+Qm5uLjIwM3L59m7o4CSGE\n", 626 | "EEKI6Gg0j7FigFB4eDgiIiKQl5eHZs2awc/PDyYmJggJCcGAAQNQWFiIiRMnGmzgHSGEEEIIIeVV\n", 627 | "4eYxJoQQQgghpDy0CqUghBBCCCGkoqKKMSGEGIhEIoFEIjF0MgghhPyHKsaEEEIIIYSAKsaEEEII\n", 628 | "IYQAoIoxIYQQQgghAKhiTAghhBBCCACqGBNCCCGEEAKA5jEmhBBCCCEEALUYE0IIIYQQAoAqxoQQ\n", 629 | "QgghhACgijEhhBBCCCEAqGJMCCGEEEIIAKoYE0IIIYQQAoAqxoQQYjASiQQSicTQySCEEPIfqhgT\n", 630 | "QgghhBACqhgTQgghhBACgCrGhBBCCCGEAKCKMSGEEEIIIQCoYkwIIYQQQggAwITjOM7QiSCEEEII\n", 631 | "IcTQqMWYEEIIIYQQUMWYEEIIIYQQAFQxJoQQQgghBABVjAkhhBBCCAFAFWNCCCGEEEIAUMWYEEIM\n", 632 | "RiKRQCKRGDoZhBBC/kMVY0IIIYQQQkAVY0IIIYQQQgBQxZgQQgghhBAAVDEmhBBCCCEEAFWMCSGE\n", 633 | "EEIIAQCYcBzHGToRhBBCCCGEGBq1GBNCCCGEEAKqGBNCCCGEEAKAKsaEEEIIIYQAoIoxIYQQQggh\n", 634 | "AKhiTAghhBBCCACqGBNCiMFIJBJIJBJDJ4MQQsh/qGJMCCGEEEIIADNDJ4AQQggxNJkM2LBB/v/Q\n", 635 | "UMDS0pCpIYQYitYV49zcXEydOhV///03zMzMMHXqVFhZWSE8PBympqZ47733MG3aNJiYmLBILzEg\n", 636 | "unAQQioimQzw8wOSk+XPt24F9u+nMo6IE12rdaN1xTguLg6WlpbYunUr7t69i4kTJ+Ktt97CxIkT\n", 637 | "4ebmhmnTpuHQoUPo1q0bi/QSA6ELByGkotqwoahsA+T/37ABGDnSUCkipHzoWq07rWOMb926hS5d\n", 638 | "ugAA7Ozs8OjRI5w8eRJubm4AgC5duiAlJUW/qSQGp+7CQQghhBDjQNdq3WldMXZwcMCRI0cAAH/9\n", 639 | "9ReeP38OmUzGv16tWjVkZGToL4WEEFJBpaamIjU11dDJYEImA1atkj+KXSKMUmgo4OlZ9NzTU76N\n", 640 | "EFL5aF0x7t27N2rUqIEBAwYgKSkJdnZ2sLGx4V9//fo1rK2t9ZpIYnh04SDEcMRUyQSKunNHjZI/\n", 641 | "/PyMO92WlvLu5shI+YO6nolY0bVadyYcx3HafODcuXNIT09H165dcfHiRcyfPx81atTAkCFD4O7u\n", 642 | "jm+//RYdOnRA9+7dWaWZGAgF9BPy5pWMGfT0NP6K26pV8gpxcZGRFLNLyJtA12rdaD34zs7ODhMm\n", 643 | "TMDq1athbm6O7777DoWFhYiIiEBeXh6aNWsGPz8/FmklBmZpSRc2Qt40GhhGiOGIsZJJ12rdaN1i\n", 644 | "TAgh5M0RY+urGFu5CSmJ8nHlRBVjQggxYmK9OIuxpY2Q4sR4U0p0RyvfEUKIgUgkEgAodWYKxcAw\n", 645 | "qmQSQgh71GJMCCEGoknFWIzE2spNSHGUjysnajE2EOpmJIRUVDRgkKgjpmsf9dZUTlQxNgBaspEQ\n", 646 | "QkhlI8ZrH83wUPlovcAH0R0t2UgIqchokQGiCl37iBhQizEhhBC9oi5oQohY0eA7A6CAfkIIIZUN\n", 647 | "XfuIGFDF2EDENACBEEII0Qe69hFjRxVjQgghhJAKgm4+dEMVY1KhUQFBCCHaoXJTvChcRXdUMSYV\n", 648 | "FhUQhBgOVa7EicpNcaNlrHVH07WRCoumBiLEMBSVq1Gj5A8/P/k2Yvyo3CSVHVWMCSHEQCQSCb8s\n", 649 | "dEVClStCDIPmENcdVYxJhUUFBCGGkZen2TZjIpPJu6FXrarcrdtUboqbYg7xyEj5g8JgtEcLfBgI\n", 650 | "xd+xR4sMEEI0IcalilmxtAR27SqKU42MrJzHQcxoGWvdUMXYAKgQFmJ5k0AFBCFvXtWqmm0zFupC\n", 651 | "Pypj2SGTAb16FR2Phw/FcX2ixiaiLxRKYQAUf1eEBum8OdRVLF5i++3E1h0vxtAPVsR4faLrCNEn\n", 652 | "qhgTgxJjISxGdOEQLzH+dhTnSN4kuo4QfaKKsQGIrTWFiB9dOIxTamoqUlNTS30P/XbsiS30Q4FF\n", 653 | "T0JgIGBjU/Tcxka+jZDKgirGBiDW1hQWhTDdJBBSOjF284utlZt1OcSi7GR1jLduBV6+LHr+8qV8\n", 654 | "mzGj64iQ2EKvjA0NvjMQloPCWAxCYDVgkGaOeDNCQ+W/WfHVrCrzhYOwJbbBbCzLIVZlp9iOMUuW\n", 655 | "lvLj6u8vf751q/5+v/R04Qwdtrb62S8rMhng6wv89pv8eWwscOAAXVe1QRXjCkaMhbDYbhJYYpVe\n", 656 | "ugERL7F284sNq3JIbBVYMd5Ep6cD9vZFLd329kBqqu6V2PR0oGlT4NUr+fN9+4B794y7chwVVVQp\n", 657 | "BuT/j4oCxo41XJrEhkIpKhiKRywitu5c1ulVXPhHjhRHpZi6A+VCQ4HOnYued+5s/BUVll3brPKF\n", 658 | "2PJbYCBgbV303NpaP7HAYgz1GzVKOfxD0cqrixEjiirFgPz/I0bovl+WUlI020bUo4ox0YgYL3Ri\n", 659 | "u0kQW3pZEttNDRFiVblilS9Y5jdWZWd0tHKlLTpa9/0C4ruJZuX2bc22GZOOHTXbRtSjinEFw6oQ\n", 660 | "FtuFjigTU4tYZblJkEgkkEgkpb5nwwblrlExHAsWlStW+YJlfmNVdrJsGWRZVrDYd2Sk8kwakZG6\n", 661 | "77dvX822GZNPP1XuYfr0U8OlR4woxriCYbmcJ4sYPJbxd2KLlWOZXlptkRDDYVF2duyoPFuEPloG\n", 662 | "WZYVrPZtayuPKdb3IDkxxvdbWsoH29F4Eh1wpELJzuY4T0+OA+QPT0/5NmMVGVmUVsUjMlJ/+8/O\n", 663 | "lu8vMtK4j4MCq/SyPs76JrZ8XF5NmzblmjZtWup7srM5rnPnomPRuXPFPBaaYJUvXrzgOBubov3a\n", 664 | "2Mi36QuL8zo7m+M6dSpKc6dO+tk3y7JCbOVQYKByegMDDZ0qwhq1GFcwYhwBHRtb1FWs74FFrEaa\n", 665 | "s5w9wlh/qzeJZtEgqrDKF+rm7tXHuciyBdbUVPX/iX6wapUnxo1OJUK0JMa4aDFOgE8DgOTEGmPM\n", 666 | "itjyBcu4aBb5gmVZIbZyiHW8rpjGfVQm1GJcwYgtrlZd4W7MraZia5UHqAWWkNIEBgKTJxfN8qCv\n", 667 | "qc/EiGVZIbZyiGW8Lo37MF5UMa5gxFbwkDeHwjSMT2pqapnvYRluJLYFcFhRN/WZPhZFYNVYwbIR\n", 668 | "hGVZQeFtcmJsYKksqGJcAYmpAiS2Fm5AnGkmpCSxtlixqACpm/pMHxVjVjMFUSNIEbHmZWKcKMaY\n", 669 | "GJQYV1kSY5rFiOLv5FjFkopxrmhW8f0sF0WQyYBeveSVta1b5f+vzPmZBTHmZTEumlVZUIsxMTgx\n", 670 | "tXAriDHNYkItQOLHomWXVffzoEHA1KnCGONBg3TbpwKrNNM58mawDNFg0eJP+UJ31GJMCDE6YmwB\n", 671 | "YoVVyxLrFisxzdyydatyjHHJabqMDctzRGwtjqzyskwGdOtWlI+7ddPv8RDT6pCVCVWMCSHEiLEK\n", 672 | "3WEZEsTq4iy26b4A8aVZJgN8fYsqg76+xl85ZpWXf/wROHGi6PmJE/JtxiwvT7NtRD2qGBNCjI7Y\n", 673 | "KhPlJZFIIJFIynwfq7l7xTYnMKsKEMv8JrY0R0Upx7RHRem+XzGKjdVsG6lYKMaYEGJ0aMS9kNim\n", 674 | "VQsMBMLDi1aTs7Ex7nmBWc0cUXz/+h6TwCrNLGfoYIVVXO277wJ//qm8zZhVrarZNqKe1hXjwsJC\n", 675 | "fPPNN0hNTYWpqSlmzZqFKlWqIDw8HKampnjvvfcwbdo0mJiYsEgvIaSSoAGOcmIcTMNqiWVFN7+i\n", 676 | "RTM2Vr4Ag67HQiYDPvqoaL8PHuhnvywpZrtQ5IuHD/WTL8S4DDKrAY5r1siPaUaG/HnNmvJt+sLi\n", 677 | "hpemE9Wd1qEUx48fR3Z2NrZs2YKwsDAsWbIE8+bNw8SJExETEwOO43Do0CEWaSWEkEpHjINpWMU5\n", 678 | "surmF2P4AKt8wXoZZDGxtARaty563rq1/le+0/cAVUVPQmCg/LFrl3Hf4BkjrSvGlpaWyMjIAMdx\n", 679 | "yMjIQNWqVXH58mW4ubkBALp06YIUVX0xRCA9HQgKkj/S0w2dGqItsY3aJqQiUNfNb6z7ZSkrS7Nt\n", 680 | "2lIsg6yIiTb2lnOAXbz1hg3Kg+/0dVPK6sZG0fuhmDf7o4/oGqUtrSvGLi4uyM3NhZ+fH7799lsM\n", 681 | "GjQIHMfxr1erVg0Zin4HolJ6OiCRFGVciYQqx2IixlHbREhMNzaBgfIYXQVjj9cF2LUYs1qIg+UC\n", 682 | "HwCb/MayMi/GQZliayUVW69KZaJ1xfinn36Ci4sLfv31V+zevRuTJ09Gfn4+//rr169hbW2t10RW\n", 683 | "NKNGKcffKQZQEONHBY+4GdMcu6mpqUhNTS31PeridY3ZyZOabdMWq25+luEDrPKbqmxTRlYyCixu\n", 684 | "Eli1kgYGyhd7UbC2Nv6bUjH2fhgbrSvG2dnZqF69OgDA2toa+fn5aNmyJU6fPg0AOHbsGFxdXfWb\n", 685 | "SkLKSWwtNYQ9scXsinFe0ipVNNumLVbd/CzDB1jlt759NdtmTFj1trFqrIiOVl74JTpa9/0C7GaP\n", 686 | "YN37URloXTEeNmwYzp8/jwEDBiA0NBRffPEFIiIisGLFCgQGBqKgoAB+fn4s0lphREYqd41GRhou\n", 687 | "PRUVq5YaKnjETYwVTVZYhZQsWSKsCFepIt+mD2Kb05lVLHC1apptKw9WY2BYVWDFGHvOKkSKBk/q\n", 688 | "Tuvp2qytrfHDDz8obY/W121UJWBrK+/yKj7/pK2tQZNUIbGawufTT4G4uKICngoeIkYsp4HbtQso\n", 689 | "KCh6XlAg31YZp987dUqzbdpi1eKoGAOjCN9JTJRfr/RxjWI1RzKrKeZYTl3HakpDRe+HmOY9Nza0\n", 690 | "8p2B2NoCW7bIH1QpFhcxjtpmSUwD2QCaAF9BbCElYmWq4iqrapu2WM3EwHIMzH+TV5W5TVtijD1n\n", 691 | "SWyDJ40NVYxJhcV6mVcqeMQ5Q0dlWW7akMQ4kwYrrELnWM3EUFio2bbyYHVTKsbYc5bniNgaK4wN\n", 692 | "LQldAYlt+VhWxLissNh+O3Uxg8a8fKwx5QuJRAIAZc5MwUJoqHzluOIhQfq6QWDVTcwSq3OPVegc\n", 693 | "q5XvPDyAbduUt+kDy94aVitlstovy9UhxbZSprGhirGBpKeziTFmtWSqWIlpWWEx/nasYgZZE1O+\n", 694 | "oNAP9lhXJhShc/rEagwFy0F9LJcrZnVNFRtW+aIyoVAKA2C5wAfNsSteYvztaIYO9kJDleMc9bWq\n", 695 | "V8n8pq8YY7GFUogx3prV7CqsQ9D27y8KTdDXzYcYF81idV4T3VHF2ABYDm6gOXbFi/VvxyLuTKyD\n", 696 | "UwhbYluUhKbwK8J6FTkW4zNYXlNZxusWj93WVxw3jaHQHVWMKxhWo34JeyxbX1nN6UwzdLDHqmWX\n", 697 | "LqDixirERhG7rGh97dWr8g7gYjm4OCoKOHGi6PmJE/rpIWTVKl+ZUMXYAFgu8EHxiOLFsvWVZVcx\n", 698 | "zdBRREyjwVleQMUWSiHGcpNliA3LsBIWi4ewuqayDG9j2UNIZbJuqGJsAIpRyoquKn1Nng6Is4AX\n", 699 | "U2KOzI0AACAASURBVGWCJWp9FbfytMqnpqaWOSOFGKcdFFsoBbWevxmsYoFtbYFr14B27eSPa9fY\n", 700 | "LkiiDzQ+w4hxpELJzuY4T0+OA+QPT0/5NmMltvSKFR1n9iIji46v4hEZqZ99Z2fL9xUZqd/fjdV+\n", 701 | "ly9XPhbLl+tv/yywOhas9s0qv2Vnc1znzkX77NxZf2kODFROc2Cg7vtlVb6xzMfZ2RzXqVPRfjt1\n", 702 | "ojLZWNB0bRWMMc3RqgmaWubNEFu+IEIsppej+U7fDDEeZxaDwlhidR359FMgLk4417c+BxcXXwFR\n", 703 | "H6shKohtPnxjQ6EUFRDFFxVhFaYhxvAPyhdsia07XoxTlLHCanAqwO44s8pvrAaFAcCSJUCVKkXP\n", 704 | "q1SRbzNWLMPbWA2qFeNqpMaGKsbEoFhWJlgVECwvouTNYHFjQ6PBxUuMNwms8hvLuNpdu4CCgqLn\n", 705 | "BQXybboS22BPlsQ4H76xoYoxMSiWlQlWBYQYL6KkCMsbGzG1yrO8KWU5CFhsvTUsF3Jgkd9YDgrL\n", 706 | "ytJsm7ZYDfZkWVYEBgLW1kXPra31U5mntQx0RxXjCkhsFw5WqIAgqhjTjY1EIoFEIjHId7NcyIFV\n", 707 | "pZtVL5DYwmBYYjlt5KlTmm0zFizLiuho4NWrouevXsm36Ypmu9AdVYwNhGXsq5i6+VnGQ7EqIOgi\n", 708 | "KkQ3YuLEciEHVj1BrHqBWPZcsVx6mwWWcbWqBpjpY9AZqzKZ5YqIrBpuBg1SbokeNEj3/VYqhp4W\n", 709 | "ozJiOXUWyymjWGA9HQ6raYdYTu0kJizzMsspyoxl6rqmTZtyTZs2LfN9YpruiyVW032xJMbjzMqL\n", 710 | "FxxnY1N0HGxs5Nv0te/AQPlDX/tkeX1itW8xTpVobKjF2ACMqSvX0Fiv/sOq5UNMsaQsscrLrOOA\n", 711 | "WbUOsmg9F1svEEtiXPKe9QBjMfXWsFrcilXvB8tYeVYhKxRCqDuqGFcwYuvmZx0PRRVYcWJ988gi\n", 712 | "X7CqwIptui8ixCqWm+UNE8sKt6WlPK95euo3XEVs5wirhhuKMdYDQzdZV0asu3LF1M3PMtxBjMT0\n", 713 | "23Ecu7wsxu7n8qRZk1AKMa6ox4oYQynEdo6wDo8S07HgOPGdI3RN1R21GBsAzXdahGW4g9iIscuc\n", 714 | "VV6uLK2ZqampSE1NLfU9leVYaEKMrWFiC51jmV6WLbtimhKPJbqm6o6WhDYQFku8AuJcfpTVsRAb\n", 715 | "sS6PzeL3E+MS1qGh8vNN8RvqqwLL6liIsaxgvUSvmLDKb4RUdiYcx3GGTgTRn1Wr5K2NxUVGGnfl\n", 716 | "iuW67mJaM37FCmDcOOG25cuBsWMNk56KjFW+EFN+Y11W0DGWK3kD4umpvxsQFseCZXrT0wGJpGgx\n", 717 | "Dhsb/QzAE+N1jxWWv19lQS3GxKBYtlqJsUWMFGFZsWKVL6j3Q46OcRGWvR9i661Rt0Kdrn8Dy/mG\n", 718 | "WWJRxom159GYUMW4ghFb9xrLk1hsBQTLqYHEhmXFSmz5grdqFbBkiTxTWFrKHxYWwn/L2lbs/0Mb\n", 719 | "WOBGa0ucumCJHFjAydUSQzpbAmklPl+1KmBiolVSRXuMCQDx3XyIETXcGC+qGFcwYozNJHJiu6lh\n", 720 | "iWXFSqytSygoADIz5VdUxaOwsNy7MwewuPiGPwA4qnijiYnGlW3Fv553LbECFpDBEjLIK95tDlgC\n", 721 | "uWV8vrT9VqlS7r/VkFj3iomprA8NBWJjhTHi+ijjxNiowKqMo+uI7qhiXAGJ6W6f5UkstgJCrDc1\n", 722 | "Yrs4s5SeXhTrGBlZduykRCIBgDJnppDJgA1VwoCIMOExzs+Xv5iTI/xX3f/VbcvOlv+r6WfS04Xb\n", 723 | "SnD47yGw879HeZmZaVeR1rIFXaOKuhG1nrOscGubjw2NZVnPqnxjdYMu1uuIMaGKMTEo1vF3VECw\n", 724 | "xerizPJCx6p1qeTAosRE/QwskskAX9+iVrbY2GJTMJmZATVqyB/l3LfO5wfHAbm5SpXpnJcy7Nku\n", 725 | "Q5X8HHzoLYN5oQYVd20q99nZwgp6QUG5joHWtGzx9rxrieX/tZoLWs/zdKi0V6nCrMLNKh8D8vQp\n", 726 | "8jEg/78+0kwztxB9olkpCDESYhxNzHI0OKuWGlYj44OC5BfO4gIDgS1b1H9GkxZjVrOViDG/lSo/\n", 727 | "X7nyrGkLeFnv1aYl/k0wM0NuFUu8yimqbMtgifrvWKJeYx1ayC0sMH+5JQ78VrRPGSzR7UMLLIks\n", 728 | "8V5zc61bz8U2ewTL9LLad4U7rw2AWowJMRI0YEmIVUgQq5HxrKSkqN6ma8W4wuU3MzP5o3p1w6WB\n", 729 | "4+T94SUqyzmvcrAnLlveeu6TI289L2/oy3//N8uWIeO6DAVZObBCNuqavYT1ixzgoUx+k1BOX/33\n", 730 | "ENgHoKmKN2sZJz68qiUaNLLEnX/kle63JJYIeWYBLC9nKIyFhfw3h/hCugIDgfBw4Q16YKDu+61w\n", 731 | "57UBUMWYEFJuYovjZikyUt7tXPxCFxmp+347dlRuiTb21d4A8VVU9MLERN6Sam4u2GwBoHc7/X6V\n", 732 | "KYCGJY6xieIYFxSUO+48+4UMi+bkADny9mJr8xwMDpTBQpNQmJcvgceP1baemwEIKL4hFcBUHQ9E\n", 733 | "lSrgLC2Rk2OBnvnyFu5/v7BAkxaWMLXSPORF1f+H1rfAzWIztzi2s8SQThbA37q3novtBr0yoVAK\n", 734 | "QoxEyVjSzp3FsZyn2CpArBcw0PfgO5kM6NYNOHFC/rxTJyApSff0sl54QkyLWogVy3AjnQffFW89\n", 735 | "L1mRVjwv72DREtuepMnwKC3nv8APGSyQAxsLGcwLc97clDNatnZfvWuJwynCUJgPAyzR3lOHUBgz\n", 736 | "Mwql0AOqGFdAdOEQJ7FWjMVITOcIy3zB6jhQ/CR7Yj0WLPJcqflN0XquS9x5WTO3aPL/N1HVMjUF\n", 737 | "LC3BWVoiq8ACN92CYf/LAqPPE8aGQikqGBpF+2awKNxZjdhmTUyVTAVW8csssMwXYjoOAMVPFifG\n", 738 | "Y1HqDCs6KDWkq0oVoFo1+cNQFK3nJSrMrx7L8P0MGcwKcjBprAw1qpaz0l5im4lMhuoyGZxdTAER\n", 739 | "lMfGhirGFYwYC0uxYVW4ixHdiLHHckESVjc1rGLPs7I020aMU1SU8k1eVJTuA0mNfmrO4rHnNWsC\n", 740 | "kJ97Hw0vOkeS86nsNBZUMTak7dvl/T8mJprFD2nw/2ZnLNGrxIpTdf+2BK6o2bcZZQFtsSrcxTiQ\n", 741 | "jW7ExIvlTQ2risqpU5ptqwxYzWrAEqsZVgDq/SD6Q7UiQ6pbVz6hqmKZ18xM4OnTonimcsQk/e+/\n", 742 | "h8Dc/x6q/BeTpPOqUNou71p8m4WFPB0iwapwN/pWD2IQrBYkEeOFWVUxIaKiQ6/EOKuBm5vyDCtu\n", 743 | "boZJi6bEGComtpULjQ1VjA3Jyws4c0b1axynesJ6DeKM8jJz8OcJ+YpTbR1kMMvXcpBAZmbR/3Nz\n", 744 | "38yxMDfXTyVblwq7mZlGU+6wnD5LbK0eYlyKVWzE2JPAqjWa1ZR4AOW3N4HVTR4rLHtVWJ3XLFcu\n", 745 | "rCxoVgpSusLCouVeyzmVjkbvVbW9+OuFhez/VlNTjSrSBVUtkHzSEmlP5aEqtRtZolc/C5jV0LKC\n", 746 | "rm6byJrAWFQoxDriXluaTNcGsGkBYrUCIMB2xTAWx0KM+U2Ms9jQyndCLPJyeVbgJELUYkxKpwi1\n", 747 | "MHRpm59f7hG65a64y2TA8+dF//+v9bwKAO/iafsHwFI9/q1Vq7IPXynr9apVNZ6wnkUrtxi7+VmR\n", 748 | "yYBevYqOx8OH+qm0seyKZzlg0NZW/xd5sea34u0Fb6LtQFdijItmhdV5TXSndcV4586diI+PBwDk\n", 749 | "5OTg2rVriI2NxXfffQdTU1O89957mDZtGky0XAWGkFKZmQE1asgfhlK89byseS31WYHPyhJW0N/E\n", 750 | "FdDERL8Vcy0r52a5FjCBJTiIq/WcBVaVNpaVV8JeVFTRoi+A/P/6GATMEsubMRY9V6Gh8lmHirfK\n", 751 | "6yuMidV5zTLcqLLQumIcEBCAgAD5oo4zZ85E37598cMPP2DixIlwc3PDtGnTcOjQIXTr1k3viSXE\n", 752 | "oIy19by8E85ru+3FC+E2hob/98hFVeTAAgVmlrCZbQks1CFMpbyVei1az4mc2GJJxRjHzXKGB7HF\n", 753 | "W7OMBRZbq7ytrTwkigbflV+5Y4wvXryIBQsWYNOmTejSpQuOHTsGADh06BBOnDiBb7/9Vq8JJYQY\n", 754 | "EY7TvPW8HP8vyM7B+VMy5LyUTzxYyyoHTd7KgWmOTPidxtJ6Xvz/VlbA8OHy5qUyaBJjzCoWmGX8\n", 755 | "pFhjdsVUGVyxAhg3Trht+XLdK8YsfztW+2aVl1kdY4BtjD/RTbljjFevXo0xY8YAAIrXratVq4aM\n", 756 | "jAzdU0aIERPbRVTvFJVFCwsmu49aBYw6UGxDNhD5lYoLXVkzt6jYlpchQ+TSHNy/LYMFctCiiQxB\n", 757 | "veSrT2nc0p6eLtxWkq2tRhVjTbDqfmbZTWxpyW7qQVbnnthmhPn0UyAuTvj7ffqp7vtlGW/NMl+w\n", 758 | "wLJVXozT7VUW5aoYv3r1CqmpqXB3dwcAmBYbRf/69WtYW1vrJ3WEGCGW3XZES2Zm8kf16hp/ZO0q\n", 759 | "4PPbxTb8Dbx20OGCpGg9Lz7F4dtva/TRsmajEDMWFc2SMzHoc9VJsd3sWlrK/3YxpRlgky9YhcKw\n", 760 | "nJqTJbHlZWNTrpEtZ86cQfv27fnnDg4OOH36NADg2LFjcHV11U/qCDFC6lpUiP6EhsovbgpGHfOp\n", 761 | "aD23tgbq1wcaN9ZrTHJgoLybVUFfI/k3bFBewdHY87G6VSd1pbjZHTVK/vDzU90RUBmI6tz7j6Il\n", 762 | "OjJS/tBXQ8Wnnwo7fvTVKg+wO86Ul3VXrhbj1NRUNGnShH8eHh6OiIgI5OXloVmzZvDz89NbAgkh\n", 763 | "lQ/LLlexDbRi1eUqxlkpWHVti3G6NlY9V2ILd1Bg0RLNslXe0hLYtUs4SE4f+xZjXjY25aoYDxs2\n", 764 | "TPBcIpEgOjpaLwkixNiJrWIlVqxiPlldkFhhVYEVY8VYrF3bLLCOBaaKFFs0j7HxoklCCdESq247\n", 765 | "8mYoLkhbt8ofvXoZd1cjqwrsyZOabTMmrLq2WYcPyGTymRNWrTLuvEaEFDHtirAEX1/9/X6sQvLE\n", 766 | "GApjbGjlO0LKgVpUxEtsXY1nzmi2TVuqwqCNfbpmVl3brGfRYBHywHJWESKnLqbdmBdREWsojDGh\n", 767 | "FmNCCDEQiUTCz2WsjqpQAX2ED7z/vmbbjI3ipnTkSP1e8Fntlwbripe6mHZ9YNmyyyovVxZUMSak\n", 768 | "HFh1jf6/vfuPreqs4zj+aQvu0mGpJlOjI73akGVzcQujRoG2Rgs2JkYX19mGVbswWDGZCUVip5bS\n", 769 | "haV1yfiDoRfCPyjONVEJ8Q83p82kS2vWZSpmZfy3uzFFcEpNadqtwPGPw+3tL+Dc3vvcc773vl9J\n", 770 | "k9vDev16fjz3e5/n+zwPQ67uufxAcnH9XJUPbNsmbdiQ/n3DhtzNuId7LlcVoR3y1dQEO7YUlORF\n", 771 | "F6UUCIy1EX2uhkZdr4/M9fO5Gmp0uUqAq5nxs5agn/O6GLl6Ppqbpc7OuTuc5WK5PVdYpz3N9dbm\n", 772 | "lORFlAcEMDnpefX1nufvZuC/npwMO6pwJBLp85D6SSSi+76ex/XLh6Vcv6qqKq+qqiov8c3n8n7z\n", 773 | "PP/+SiT8n6jfay6fD1fneXLS82pr0+9ZW5ubmF3fF664uN8OHFh4Lg4cyM17I7qKvI8AQVEnZxvX\n", 774 | "D/lkbZMBq8/H1auLvy421u43RBuJMZAhVzWqLLNjm7Xr5zJeq4mmC67O85Ej0tBQ+vehodzsAtjc\n", 775 | "7G/imFJRkdvSDxf1y67uN9elFNRyR1TYXdawgaH4uVwNE7t8X66fexbvCxfva2043vXz4eI8Nzcv\n", 776 | "PMfNzdm/r8vyAVfn2WW5iqv7gjY5uko8z/PCTs5hA5O3bLN2/azF68r8yVD19bmbDOXqHLuM2RVr\n", 777 | "99v+/dKuXXOPPf201NGR3fu2tCzcXbC5WXruuezeV/J7RlM7TqYkEtlPQLP4jLg6F8geq1IgMGbQ\n", 778 | "2mbp+lmdGe/iQ9TVhiQuz7HFTQYsPR+Su2H+e+9dmBjfe2/27+uSy/vN2n0h2fuSFzXUGCMw6qGQ\n", 779 | "LxZrVK1NALJ4jpHmaqvw114Ldmwp2NQizdW5sNYORRGJMQLhYQNuzFWiaW1Sn0R7kQ+udmUrKwt2\n", 780 | "bCnY1CLN1bngC2/2SIxDYq331eXDZu1cwD2LyaArrj5A29oW7qjHqhR2JJPBjmUqkfA3IUlZtco/\n", 781 | "livWenZd4lxEEzXGIbBaP+kCu71hMRZrVNva/Pt39gSgmyWa8XhckpS8SUZjsc4RaS7aoaamhSUO\n", 782 | "TU3Zv29lpZ9gpyaGJRL+MdiwlHYIc9FjHAKLvSmuevBc90QznGuXtd4Ua8PER49KL7+c/v3ll3P3\n", 783 | "7NHjn+aqHSovD3ZsKWIx/5rV10f7HsZCsZh04oS/kkhzs/+aa5gZeowRiMUePFez+SV6orE4Sz27\n", 784 | "riZvSTbbC1dctUOuVqVgRNO2qSnp619PX79z57h+maLHOARWe1Nc9OBZPBdTU9LmzekeoM2bc9cT\n", 785 | "Tb21bVy/NGs9/tZYHMWDe1y/7NFjHILUUMfsGq5i/eBw2bPkqtbqyJGFQ9BHjkiPPZbd+9JTM5e1\n", 786 | "Xnlr18/1drfwuWqH6JXPD2vtELLHznchsLgrlFUuGjVXO0O53gnJUgNv8RlZyvULOvnOhdTIR+pL\n", 787 | "Xm2t9OKL0T7HVll69v71L+n226UrV/zfy8qkd96RPvax3Ly/y90Wc/2+rtuhsbHcT3K02HZGDT3G\n", 788 | "IXBZ+wr31q9fmBivXx9OLEFZ680slmckjIQY+Wep9nznznRSLPmvd+7MzZbQrtohV+/rsh0aG5Pi\n", 789 | "cel///N/f/55fzWQbJNjRhKyR40xCpar2eDbti1c/3Xbtuzf12W9NXVn7lmrl3e5KoXkrt6aOm67\n", 790 | "XLVDFtu3HTvSSbHkv54/4rRU1Pdnh8Q4BNY+QFOsfSC5aixjMX/IObUsV66Gn60t95Xi4r6w+IxY\n", 791 | "u34uV6Vw9aXU6hKMltpO1xt8WGKxHUIOeAjFxYue19zs/1y8GHY0Nzc56Xn19Z4n+T/19f6xKDtw\n", 792 | "IB1v6ufAgbCjCsfFi563alX6PKxalZv7zuV9MTnpeYmE/xP1e80il89HIrHwvROJ6L6vSxbbTlef\n", 793 | "T67OhcV26Nw5zysrS8dcVuYfywXazuxQYxwCi+sMFkvNZ6Hq7184bNffn/31c3lfWKrLBK7HYttZ\n", 794 | "WZmbmuL5XNW/uqyrddUOnTixsJb7xIns/7fmT6r95S+ZVJspSilCYLEeyiKWo3LP5XA87HI1BM3Q\n", 795 | "tn2u6l+pq/VdbzlRBEdijEAsfiBZjNkVzkU0xePxmSXb8s3lF0dX9dbW6rglnj0sztV9MTwc7Biu\n", 796 | "j3WMQ2B1nUFLa3GmWIzZFRfnwvXay4Uu7HWMLbZDFtEOYTEu7ov9+6Vdu+Yee/ppqaMj+/cuFiTG\n", 797 | "IaGhRCEgucpO0MTY0qYIAMLzzDPSd78799iBA9nvzFpMmHwXEiYWoRCwmLx7LjdnoR0CCgtza7JH\n", 798 | "jzFCR68VilWQHmPKVfKDdig/OM9uMYqXPXqMESprWxUDKDy0Q/nBeZ7LxZcERvGyx6oUCBVL16GY\n", 799 | "JZPJm9YXNzcv3ImsudltXMXGdTtkaec7lyy292NjUkuL/zM2lrv3tbqLYzGgxxgAIszV5iyuMWTu\n", 800 | "Y8MFu8bGpHg8/fw9/7yUTPoboGTL1cYv9Mpnjx5jhIo1PoHCY603zGWvPBsupFlr73fsWPildH69\n", 801 | "f9RY7JWPGhLjAmRp2M7igv1APllLJiR7H87X65XPBTZcSKO9T7P4XBcLSikKjMVhFJaMAq6PyTTu\n", 802 | "udzafP36hUn2+vW5eW+LLLX3iYRfPpH60rRqlX8sF1w9121t/v02e1UKEu7MsFxbgWFpJwBhc7lk\n", 803 | "lIvaZZebIsyvMa6tpcbYkrGx9GdqIpGb+mLXqO/PDj3GALJCI7x0Ye9854qr3jBXI2IuN0WIxfxE\n", 804 | "2NL1Q1osli55sHLdLPXKRxE9xgWGxb1ts5YAcb9lJ0hizDlOczUixjnGYrgvitOSeowPHz6sl156\n", 805 | "SdPT03rooYe0du1adXZ2qrS0VGvWrFF3d7dKSkpyHSsCoB7RLov14a6WHLLKxRcbzrF7tJtYDM9e\n", 806 | "ccp4VYpXXnlFf/3rX9Xf369jx47p7Nmz6uvrU0dHh5599ll5nqeBgQEXsSKg1DBKezuNuyXWZvJj\n", 807 | "LmtLlFnkciY/7SYAaQmJ8dDQkO644w595zvfUXt7u774xS9qdHRUNTU1kqS6ujoNF+taNECRYcmh\n", 808 | "NFdfbDjHaVaX+7K0hCbSePaKU8alFP/973917tw5HT58WGfPnlV7e7tmlymXl5drfHw8p0ECxcDi\n", 809 | "MjsMQbvHOZ7L2sQidr6zi2evOGWcGH/oQx9SdXW1li1bpk9+8pO65ZZbdOHChZl/n5iYUEVFRU6D\n", 810 | "BIqB1UbYWqLiylK+2NxsNYoUzrFd19v5LhdLwcE9nr3ik3EpxX333aeXrz3l58+f19TUlD73uc9p\n", 811 | "ZGREkjQ4OKh169blNkqgSFDnaJfVYX5rrJUlsPMd8s3aMxI1GfcYf+ELX9Crr76qBx54QFevXlV3\n", 812 | "d7c+8YlPqKurS9PT06qurlZjY6OLWAEg0uhdcsviyi3sfId8sviMRA3rGANLYG29YaAQWNzZ0+LO\n", 813 | "d7Rvdll8RqKGne+ADPGNHAjH9HSwY1Fibec72jcUu4xrjIFix3rDADJhae4A7ZttLDGXPXqMASAk\n", 814 | "QbaERtry5cGOAcUqFpNOnEiXUyQS0f8yFjXUGCMw6s5884ca6+ttDDVy/aInaGLs6tpZuyesPnuW\n", 815 | "cI5t4/plj8QYgfCwzUVCgVwIkhi7unZW7wlrz55FnGO7mHyXPWqMEQh1Z3NZqhmUuH6Wubp2Vu8J\n", 816 | "a8+eRZzjNNYELj4kxgAAAPOkRlV27PB/Ghujnxwz+S57lFIgEKvDrvBx/aKJUgpEEaUUPqtlCVy/\n", 817 | "7JAYI7CxsbkzXSsrw40HmaGxtIvJd8gXvjClWU2MkR0SYwRCYwkAhY9kMI3PveLEOsYI5HoTdYqx\n", 818 | "sUR+0JsJIEyxmJ8Iu2qHaOOiicQYQOSwLS0QjrY2/3mb3UtazJO3Uit05BptXHSxKgUCYaYr8snq\n", 819 | "UmKAdale0kTC/yFZc4M2LrroMUYgroeUAADR4KqXFLCAyXcAIqdYJr0E3RIaQGEpljbOIhJjAJFU\n", 820 | "DBNTSIyB4lUMbZxFJMYAEJKgiTEfoACQH9QYA0CEMXsdAPKHVSkAIMKYvQ4A+UOPMYCsMMwP3BjP\n", 821 | "CGAHNcYFiEYY+cLMavc4x7Zx/QBbSIwLDI0w8unQIWnHjrnHEgnWQM01vuzaxTMC2EIpRYG5Xj0i\n", 822 | "jXBuWUxULMYMHxsuYDE800DukRgDGbK4SoCrmNva/PeaPULBVuFAmqtnxGI7BFhAKUWBoZTCPYtD\n", 823 | "oy5jptcKuDEXz4jFdgiwgB7jAhOL+YkwiQryhWF+4MZ4RgA76DEGMmSxV95izMWALaGLg4seY55p\n", 824 | "wA0SY2AJLJYPWIy50JEYFz6XCSzPNJB7JMYAkCOZJiphJ8YkVu5RCwzYQo0xAOSAtVUCrMULAPlQ\n", 825 | "GnYAAFAIrreGeFRZi9eqtja/fCKFJQ2BaKPHGAAAR1gpCLCFGmMgQqj5tMvaKgHW4gWAfCAxBiKC\n", 826 | "RMU+a19srMULAK6RGAMRwex1AADCxeQ7AAAAQPQYA5HhupSCYXMAAG6MxBiIEFfJK/XLAADcHIkx\n", 827 | "UASoX46msHe+AwDMRY0xAAAAoCX2GN9///1auXKlJGn16tV69NFH1dnZqdLSUq1Zs0bd3d0qKSnJ\n", 828 | "ebAAloZSimiixxgAoiXjne/ee+89SdKxY8dmjrW3t6ujo0M1NTXq7u7WwMCAGhoachclgKyw+xYA\n", 829 | "ADeXcWJ85swZTU5OauvWrbp8+bJ27typ06dPq6amRpJUV1enoaEhEmMgYmIxaooBALiRjBPjFStW\n", 830 | "aOvWrWpqalIymdQjjzwy59/Ly8s1Pj6eswABAACAfMg4MY7H46qqqpp5XVlZqTfeeGPm3ycmJlRR\n", 831 | "UZG7CAGgQFFbDADRkvGqFMePH1dfX58k6fz585qYmNCGDRs0MjIiSRocHNS6detyGyUAAADgWMar\n", 832 | "Uly+fFmPP/64/vnPf0qSdu/ercrKSnV1dWl6elrV1dXat28fq1IAAADAFDb4AAAAAMQGHwAAAIAk\n", 833 | "EmMAAABAEokxAIQmHo/P7H4HAAgfiTEAAACgJaxjDAAAUAympqSjR/3XbW3+DqIobCTGAAAA80xN\n", 834 | "SY2N0smT/u/9/dILL5AcFzpKKQAAAOY5ejSdFEv+61TvMQoXPcYAig7DowCAxbDBB4CiMn94tL6e\n", 835 | "4VEAC1ltK/jinx0SYwBF5dAhaceOuccSCam9PZx4gKUiAXLP2jm2msxHCaUUAIrK9HSwY0CUMTEs\n", 836 | "P2IxW1+ar1cXben/Q9iYfAcARWpqyu9BP3TIfw07mBgGuEFiDKCoLF8e7FihS/U47tjh/zQ2khwD\n", 837 | "1rW1+eUTKfX1/jEER40xgKJCDZ6PWmvbuI9xPdbqoqOGGmMARSUW8xOIKHxwxONxSVIymQwnAJgV\n", 838 | "pfsY0WKtLjpq6DEGgJCEmRjT4wgAC9FjDABFiB5HAFiIHmMACAmlFAAQLaxKAQAAAIjEGAAAAJBE\n", 839 | "KQUAFC2WdQKAuUiMAaAIsSoFACxEKQUAFCG2FAaAhUiMAQAAAFFKAQBFiVIKAFiIxBgAihST7wBg\n", 840 | "LhJjAAgJG3wAQLRQYwwAAACIxBgAAACQRGIMAAAASCIxBgAAACSRGAMAAACSWJUCAAAAkESPMQAA\n", 841 | "ACCJxBgAAACQRGIMAAAASJKWhR0AAAAAcoOt3rPD5DsAAIACMDUlNTZKJ0/6v9fXSy+8QHKcCUop\n", 842 | "ACAk8Xhc8Xg87DAAFIijR9NJseS/TvUeIxgSYwAAAEBZJMb/+c9/VF9frzfffFNvvfWWWlpaY7O7\n", 843 | "aQAACMFJREFUtGXLFu3du1dUZwAAAORXW5tfPpFSX+8fQ3BLSoynp6e1Z88erVixQp7nqbe3Vx0d\n", 844 | "HXr22WfleZ4GBgZyHScAAABuIBbza4oTCf+H+uLMLSkxfuqpp9TS0qLbbrtNknT69GnV1NRIkurq\n", 845 | "6jQ8PJy7CAEAABBILCa1t/s/JMWZyzgxPn78uD784Q9r48aNkiTP8+aUTpSXl2t8fDx3EQIAAAB5\n", 846 | "kPE6xsePH1dJSYmGh4d15swZdXZ26uLFizP/PjExoYqKipwGCQCFKJlMhh0CAGCWjBPjX/ziFzOv\n", 847 | "W1tb1dPTo6eeekojIyP67Gc/q8HBQX3+85/PaZAAAACAa1nvfFdSUqLOzk51dXVpenpa1dXVamxs\n", 848 | "zEVsAAAAQN6w8x0AAAAgNvgAAAAAJJEYAwAAAJJIjAEgNPF4XPF4POwwAADXkBgDAAAAIjEGAAAA\n", 849 | "JJEYAwAAAJJIjAEAAABJJMYAAACAJDb4AAAAACTRYwwAAABIIjEGAAAAJJEYAwAAAJJIjAEAAABJ\n", 850 | "JMYAAACAJBJjAAhNPB5XPB4POwwAwDUkxgAAAIBIjAEAAABJJMYAAACAJGlZ2AEAwGKmpqSjR/3X\n", 851 | "bW1SLBZmNACAYkBiDCBypqakxkbp5En/9/5+6YUXSI4BAG5RSgEgco4eTSfFkv861XtcSJLJpJLJ\n", 852 | "ZNhhAACuITEGAAAAJJV4nueFHQQAzDa/lKK+nlIKAIB7JMYAIonJdwCAfCMxBgAAAESNMQAAACCJ\n", 853 | "xBgAQhOPxxWPx8MOAwBwDYkxAAAAIBJjAAAAQBKJMQAAACCJxBgAAACQRGIMAAAASGIdYwAAAEAS\n", 854 | "PcYAAACAJBJjAAAAQBKJMQAAACCJxBgAAACQRGIMAAAASCIxBoDQxONxxePxsMMAAFyzLNM/uHLl\n", 855 | "in70ox8pmUyqpKREPT09+sAHPqDOzk6VlpZqzZo16u7uVklJiYt4AQAAACcyToxfeukllZaW6rnn\n", 856 | "ntPIyIj2798vSero6FBNTY26u7s1MDCghoaGnAcLAAAAuJJxKUVDQ4OeeOIJSdI//vEPrVq1SqOj\n", 857 | "o6qpqZEk1dXVaXh4OLdRAgAAAI4tqca4rKxMnZ2devLJJ/XVr35VszfPKy8v1/j4eM4CBAAAAPIh\n", 858 | "41KKlL6+Pr377rtqamrS+++/P3N8YmJCFRUVOQkOAAAAyJeMe4xPnDihw4cPS5JisZhKS0t19913\n", 859 | "a2RkRJI0ODiodevW5TZKAChAyWRSyWQy7DAAANeUeLPrIAKYmppSZ2en3n33XV2+fFnbt2/Xpz71\n", 860 | "KXV1dWl6elrV1dXat28fq1IAAADAlIwTYwAAAKAQscEHAAAAIBJjAAAAQBKJMQAAACCJxBgAAACQ\n", 861 | "FEJi/Ic//EG7du2a8/umTZvU2tqq1tZWvfrqq/kOCRmYf/3+9re/6cEHH1RLS4sOHjwYYmQIwvM8\n", 862 | "1dbWzjxvqS3dEW1Xr17Vnj171NzcrNbWVr399tthh4QM3H///TPP3A9+8IOww0FAp06dUmtrqyTp\n", 863 | "rbfeUktLi7Zs2aK9e/eKdQuibfa1O336tOrq6maewd/97nc3/Nslb/CxFPv27dPQ0JDuuuuumWOj\n", 864 | "o6PavXu3Nm/enM9QsASLXb+9e/fqmWee0erVq7V9+3a98cYbuvPOO0OMEjfy9ttv69Of/rQOHToU\n", 865 | "dijIwB//+EdNT0+rv79fp06dUl9fn37605+GHRYCeO+99yRJx44dCzkSZOLIkSP67W9/q1tvvVWS\n", 866 | "1Nvbq46ODtXU1Ki7u1sDAwNqaGgIOUosZv61Gx0d1cMPP6yHH3440N/ntcd47dq1C75pjY6O6je/\n", 867 | "+Y22bNmiH//4x7py5Uo+Q0IG5l+/S5cu6f3339fq1aslSRs3btTw8HCYIeImRkdHdeHCBX3rW9/S\n", 868 | "9u3b9eabb4YdEgL4y1/+otraWknSPffco9dffz3kiBDUmTNnNDk5qa1bt+rb3/62Tp06FXZICKCq\n", 869 | "qkoHDx6c+bw7ffq0ampqJEl1dXV81kXY/Gv3+uuv609/+pMeeugh/fCHP9TExMQN/95Jj/GvfvUr\n", 870 | "/fznP59zrLe3V1/5ylf0yiuvzDm+YcMGNTQ06Pbbb9eePXvU39+vLVu2uAgLAQW9fpcuXdLKlStn\n", 871 | "fr/11lt19uzZvMWJG1vsOnZ3d+vRRx/Vl7/8Zb322mvavXu3fv3rX4cUIYKa/6yVlZXp6tWrKi1l\n", 872 | "mkjUrVixQlu3blVTU5OSyaS2bdum3//+91y7iNu8ebPeeeedmd9nd+iVl5drfHw8jLAQwPxrd889\n", 873 | "9+ib3/ym7rrrLh06dEgHDx7U97///ev+vZPEuKmpSU1NTYH+22984xv64Ac/KEn60pe+pBdffNFF\n", 874 | "SMhA0Ou3cuXKOd+8Ll26pIqKCpehIQOLXcepqSmVlZVJku677z5duHAhjNCQofnPGkmxHfF4XFVV\n", 875 | "VTOvKysr9e9//1sf/ehHQ44MmZj9vE1MTPBZZ8imTZtm8syGhgbt27fvhv99qC2r53n62te+pvPn\n", 876 | "z0uS/vznP+vuu+8OMyRkYOXKlVq+fLnOnj0rz/M0NDSkdevWhR0WbuAnP/mJfvazn0nyh3g//vGP\n", 877 | "hxwRgli7dq0GBwcl+RNe77jjjpAjQlDHjx9XX1+fJOn8+fO6dOmSbrvttpCjQqbuvPNOjYyMSJIG\n", 878 | "Bwf5rDPkkUce0d///ndJwfLMvE6+k6SSkhKVlJTMvH7yySf12GOP6ZZbbtGaNWv04IMP5jskZGD2\n", 879 | "9ZOknp4efe9739OVK1e0ceNGfeYznwkxOtzM9u3btXv3bp08eVLLli1Tb29v2CEhgE2bNmloaEjN\n", 880 | "zc2SxHUz5IEHHtDjjz8+UyLY29tLb78hqc+7zs5OdXV1aXp6WtXV1WpsbAw5MtxM6tr19PSop6dH\n", 881 | "y5Yt00c+8hE98cQTN/47jzVHAAAAADb4AAAAACQSYwAAAEASiTEAAAAgicQYAAAAkERiDAAAAEgi\n", 882 | "MQYAAAAkkRgDAAAAkkiMAQAAAEnS/wHvZVE3+a+WiQAAAABJRU5ErkJggg==\n" 883 | ], 884 | "text/plain": [ 885 | "" 886 | ] 887 | }, 888 | "metadata": {}, 889 | "output_type": "display_data" 890 | } 891 | ], 892 | "source": [ 893 | "window = class_df[(class_df['size']>=29) & (class_df['size']<=53)]\n", 894 | "\n", 895 | "result = smf.ols(formula = \"read ~ csize + small\", \n", 896 | " data = window).fit()\n", 897 | "print result.summary()\n", 898 | "\n", 899 | "plt.figure(num=None, figsize=(12, 6), dpi=80, facecolor='w', edgecolor='k')\n", 900 | "plt.scatter(window.csize,window.read, color=\"blue\")\n", 901 | "l=window[window.csize<0].csize.count()\n", 902 | "plt.plot(window.csize[0:l], result.predict()[0:l], '-', color=\"r\")\n", 903 | "plt.plot(window.csize[l:], result.predict()[l:], '-', color=\"r\")\n", 904 | "plt.axvline(x=-0.5,color=\"black\", linestyle=\"--\")\n", 905 | "plt.title(\"Regression Discontinuity: Reading Scores by Class Size Before and After the Cutoff\", fontsize=\"18\")" 906 | ] 907 | }, 908 | { 909 | "cell_type": "markdown", 910 | "metadata": {}, 911 | "source": [ 912 | "^^ In this model, we see a statistically significant effect of class size on reading scores (p<0.05), as expressed in the coefficient *small* and illustrated by plotting the predicted value from the model on the forcing variable." 913 | ] 914 | }, 915 | { 916 | "cell_type": "markdown", 917 | "metadata": {}, 918 | "source": [ 919 | "## Generalizing the Relationship Between the Outcome and the Forcing Variable\n", 920 | "\n", 921 | "Murnane and Willett explore a more complex example in Ludwig and Miller (2009) \"[Does Head Start Improve Children's Life Chances? Evidence from a Regression Discontinuity Design](http://qje.oxfordjournals.org/content/122/1/159.short).\" In this paper, the authors identify a possible exogenous cutoff in the deployment of Head Start when the US Office of Economic Opportunity (OEO) offered Head Start grant-writing support to communities with county-level povery rates below 59.2%. The authors fit two regression discontinuity models:\n", 922 | "\n", 923 | "**Model 1** \"examines differences in counties' funding for and participation in, the Head Start program as a consequence of receipt of grant-writing assistance or the lack of it. The results of these analyses showed that grant-writing assistance did indeed lead to differences in Head Start funding and participation immediately following receipt of the assistance.\" (187)\n", 924 | "**Model 2** focused on \"longer-term child health and schooling outcomes.\" In this model, \"poverty-rate was again the forcing variable and exogenous assignment to grant-writing assistance again defined the experimental conditions. However, now, the impact of *participating* in Head Start on future child outcomes is the substantive focus, and so grant-writing assistance--which remains the principal question predictor--has become an exogenously assigned expression of *intent to treat* by Head Start\" (187). This opens up two causal questions:\n", 925 | "* \"Did receipt of grant-writing assistance lead to increases in a county's participation in, and funding for, Head Start?\" (187) \n", 926 | "* \"Did grant-writing assistance (which constitutes an intent to treat with Head Start) lead to improvement in future child outcomes?\" (187) (particularly recuding child mortality)\n", 927 | "\n", 928 | "Murnane and Willett walk readers through the various models taken in this paper, as well as the sensitivity analyses conducted on the second set of models:\n", 929 | "* demonstrating that their approach \"*revealed no effects for outcomes that they hypothesized should not have been influenced by the OEO grant-writing assistance program*\" (193)\n", 930 | " * testing unlinked outcomes, like per-capita spending on non-Head-Start programs\n", 931 | " * testing outcomes from a period before the intervention\n", 932 | " \n", 933 | "## Planning Regression Discontinuity Designs Up Front\n", 934 | "Murnane and Willet note that Regression Discontinuity can be convenient as an initial study design in politically complicated situations: \"providing assistance to the 300 poorest counties, with the slightly less needy serving as controls, would be easier to defend ethically to members of Congress who were concerned about equity than would the assignment of poor counties randomly to treatment and control groups\" (195). When doing this, there are several costs:\n", 935 | "* Results apply only to observations near the cut-point (197)\n", 936 | "* Statistical power is also a problem: \"the sample size required in an impact evaluation with an RD design typically must be almost three times the size of the sample that would be required in an equivalent random-assignment experiment of the same statistical power\" (they cite Bloom, forthcoming, which I couldn't find) (197)\n", 937 | "\n", 938 | "## Further Reading\n", 939 | "Murnane and Willett conclude by suggesting that readers further interested in this technique read Thomas Cook's 2008 paper, \"[Waiting for Life to Arrive: A History of the Regression-Discontinuity Design in Psychology, Statistics, and Economics](http://www.sciencedirect.com/science/article/pii/S0304407607001108)\" in the journal of Econometrics.\n" 940 | ] 941 | } 942 | ], 943 | "metadata": { 944 | "kernelspec": { 945 | "display_name": "Python 2", 946 | "language": "python", 947 | "name": "python2" 948 | }, 949 | "language_info": { 950 | "codemirror_mode": { 951 | "name": "ipython", 952 | "version": 2 953 | }, 954 | "file_extension": ".py", 955 | "mimetype": "text/x-python", 956 | "name": "python", 957 | "nbconvert_exporter": "python", 958 | "pygments_lexer": "ipython2", 959 | "version": "2.7.6" 960 | } 961 | }, 962 | "nbformat": 4, 963 | "nbformat_minor": 0 964 | } 965 | -------------------------------------------------------------------------------- /survival_analysis/DATA_LICENSE.txt: -------------------------------------------------------------------------------- 1 | # Data was provided under the Creative Commons Attribution-Share-Alike 3.0 license 2 | # https://creativecommons.org/licenses/by-sa/3.0/us/ 3 | -------------------------------------------------------------------------------- /survival_analysis/wikipedia-validation.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/natematias/research_in_python/7da867fe3da8bd5450bb4fe1d05a06e66750634c/survival_analysis/wikipedia-validation.zip --------------------------------------------------------------------------------