├── .gitignore ├── Chapter_01.ipynb ├── Chapter_02.ipynb ├── Chapter_03.ipynb ├── Chapter_04.ipynb ├── Chapter_05.ipynb ├── Chapter_06.ipynb ├── Chapter_07.ipynb ├── Chapter_08.ipynb ├── Chapter_09.ipynb ├── Chapter_10.ipynb ├── Chapter_11.1.ipynb ├── Chapter_11.2.ipynb ├── Chapter_13.ipynb ├── Chapter_14.ipynb ├── EXP -- Chapter_09.ipynb ├── Extras_01__DGPs.ipynb ├── Extras_02__Additional_computations.ipynb ├── LICENSE ├── README.md ├── causal-pymc.yml ├── causal_book_py39_apple_m1_(experimental-by-ferrari-leo).txt ├── causal_book_py39_cuda117.yml ├── causal_model.png ├── data ├── ch_01_drug_data.csv ├── data_11_no_interaction_test.csv ├── data_11_no_interaction_train.csv ├── data_11_with_interaction_test.csv ├── data_11_with_interaction_train.csv ├── gt_social_media_data.csv ├── hillstrom_clean.csv ├── hillstrom_clean_label_mapping.json ├── hillstrom_original.csv ├── manga.csv ├── manga_processed.csv ├── ml_earnings.csv ├── ml_earnings_interaction_test.csv ├── ml_earnings_interaction_train.csv └── shpitser_thesis1.grapl ├── docs └── himsolt-gml-technical-report.pdf ├── errata ├── Errata - Early Print (ordered before June 13 2023).ipynb ├── Errata - Non-Early Print (ordered after June 13 2023).ipynb ├── img │ ├── ch_04__fig_4_1.png │ ├── ch_06__fig_6_3.png │ ├── ch_06__fig_6_4.png │ ├── ch_06__fig_6_5.png │ └── ch_07__fig_7_6.png └── minor-errors.txt ├── img ├── ch_03_graph_01 ├── ch_03_graph_01.png ├── ch_03_graph_02 ├── ch_03_graph_02.png ├── ch_04_graph_DAG ├── ch_04_graph_DAG.png ├── ch_04_graph_DCG ├── ch_04_graph_DCG.png ├── ch_04_graph_Fully connected ├── ch_04_graph_Fully connected.png ├── ch_04_graph_Partially connected ├── ch_04_graph_Partially connected.png ├── ch_04_graph_Undirected ├── ch_04_graph_Undirected.png ├── ch_04_graph_adj_00 ├── ch_04_graph_adj_00.png ├── ch_04_graph_adj_01 ├── ch_04_graph_adj_01.png ├── ch_04_graph_adj_02 ├── ch_04_graph_adj_02.png ├── ch_05_chain_00 ├── ch_05_chain_00.png ├── ch_05_collider_00 ├── ch_05_collider_00.png ├── ch_05_fork_00 ├── ch_05_fork_00.png ├── ch_05_markov_01 ├── ch_05_markov_01.png ├── ch_05_markov_02 ├── ch_05_markov_02.png ├── ch_06_confounding_00 ├── ch_06_confounding_00.png ├── ch_06_d_sep_00 ├── ch_06_d_sep_00.png ├── ch_06_d_sep_01 ├── ch_06_d_sep_01.png ├── ch_06_d_sep_02 ├── ch_06_d_sep_02.png ├── ch_06_d_sep_03 ├── ch_06_d_sep_03.png ├── ch_06_d_sep_04 ├── ch_06_d_sep_04.png ├── ch_06_equivalent_estimands_00 ├── ch_06_equivalent_estimands_00.png ├── ch_06_equivalent_estimands_01 ├── ch_06_equivalent_estimands_01.png ├── ch_06_gps_00 ├── ch_06_gps_00.png ├── ch_06_gps_01 ├── ch_06_gps_01.png ├── ch_06_gps_02 ├── ch_06_gps_02.png ├── ch_06_gps_03 ├── ch_06_gps_03.png ├── ch_06_icecream ├── ch_06_icecream.png ├── ch_06_instrumental_00 ├── ch_06_instrumental_00.png ├── ch_07_full_example ├── ch_07_full_example.png ├── ch_08_modularity ├── ch_08_modularity.png ├── ch_08_modularity_mod ├── ch_08_modularity_mod.png ├── ch_08_selection ├── ch_08_selection.png ├── ch_08_selection_02 ├── ch_08_selection_02.png ├── ch_08_selection_03 └── ch_08_selection_03.png └── models └── causal_bert_pytorch ├── CausalBert.py ├── README.md ├── __pycache__ ├── CausalBert.cpython-38.pyc └── CausalBert.cpython-39.pyc └── requirements.txt /.gitignore: -------------------------------------------------------------------------------- 1 | .ipynb_checkpoints -------------------------------------------------------------------------------- /Chapter_01.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "id": "d885824b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from itertools import combinations\n", 11 | "\n", 12 | "import numpy as np\n", 13 | "from scipy import stats\n", 14 | "\n", 15 | "import networkx as nx\n", 16 | "\n", 17 | "import matplotlib.pyplot as plt\n", 18 | "plt.style.use('fivethirtyeight')" 19 | ] 20 | }, 21 | { 22 | "cell_type": "code", 23 | "execution_count": 2, 24 | "id": "7a2494ed", 25 | "metadata": {}, 26 | "outputs": [], 27 | "source": [ 28 | "COLORS = [\n", 29 | " '#00B0F0',\n", 30 | " '#FF0000'\n", 31 | "]" 32 | ] 33 | }, 34 | { 35 | "cell_type": "markdown", 36 | "id": "a0fd06fd", 37 | "metadata": {}, 38 | "source": [ 39 | "# Chapter 01\n", 40 | "\n", 41 | "This chapter introduces the concept of causality and highlights similarities and differences between causal inference and statistical learning. A brief historical outline of the concept of causality is provided to help the reader understand a broader context. Finally, three motivating examples are provided (medicine, marketing, social policy) to demonstrate the importance of causal inference in terms of technical, practical and business perspectives. " 42 | ] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "id": "c6c49b0c", 47 | "metadata": {}, 48 | "source": [ 49 | "## Confounding " 50 | ] 51 | }, 52 | { 53 | "cell_type": "code", 54 | "execution_count": 33, 55 | "id": "9db62153", 56 | "metadata": {}, 57 | "outputs": [], 58 | "source": [ 59 | "# Let's set random seed for reproducibility\n", 60 | "np.random.seed(45)\n", 61 | "\n", 62 | "# `b` represents our confounder\n", 63 | "b = np.random.rand(100)\n", 64 | "\n", 65 | "# `a` and `c` are causally independent of each other, but they are both children of `b` \n", 66 | "a = b + .1 * np.random.rand(100)\n", 67 | "c = b + .3 * np.random.rand(100)" 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": 34, 73 | "id": "25652f3a", 74 | "metadata": {}, 75 | "outputs": [ 76 | { 77 | "name": "stdout", 78 | "output_type": "stream", 79 | "text": [ 80 | "0.9627497625297509\n" 81 | ] 82 | } 83 | ], 84 | "source": [ 85 | "# Let's check correlation between `a` and `c`\n", 86 | "coef, p_val = stats.pearsonr(a, c)\n", 87 | "\n", 88 | "print(coef)" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": 35, 94 | "id": "ac319993", 95 | "metadata": {}, 96 | "outputs": [ 97 | { 98 | "data": { 99 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAyoAAAH9CAYAAAD1bYh5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAACXeklEQVR4nOzde3hU1dn///cck3CMhBxIOHlI8wAVEBQQvhbEqk/bCyzSKKAtIlgpaG0rv4pWUdRHpKl+W60gSlFRqVILrfYgT0UsCILfUhGLFCICAjEJCQVCTpOZ2b8/diaZJJNkJplMZiaf13V5xezZe89aSdh77r3Wum/L6dOnDURERERERKKItbMbICIiIiIi0pgCFRERERERiToKVEREREREJOooUBERERERkaijQEVERERERKKOAhUREREREYk6ClRERERERCTqKFAREREREZGoo0BFRBp49dVXSU5O5ujRozH9Hh2lI9oeLT+PZcuWkZycTFFRUav7dlabQ2mjdKyamhrS0tJ45JFHOrspLYqWf18iEjoFKiIxzncT9v2XkpLC0KFDueOOOygsLOzs5omfDz74gGXLlnH69OnObor40e+lbfLz83G5XAwZMqSzmyIicUqBikicWLx4MatWreL//t//y6RJk1i3bh3f+MY3qKysDOk8M2bMoLCwkIEDB3ZQSyPzHtFo586dLF++nDNnzjTYHos/j1hsc3Oa+71Iy/bt2wegQEVEOoy9sxsgIuFx1VVXcdlllwHwve99j/POO49nnnmGv/zlL0yfPj3o89hsNmw2W6v7VVRU0K1btza1Ndj36Gjt6UM4RcvPIxSx2GYJr08//RS73U52dnZnN0VE4pRGVETi1Ne+9jUAjhw5AsAXX3zB3XffzWWXXUa/fv0YOHAgN954I/v3729wXKD53L51Af/+97+ZP38+559/PpmZmSQnJ7Nx48a6/Q4fPkxycjJf/epXG5zzxz/+MV/5yldafI9z585x//33M3z4cNLT08nOzmbKlCls27atwbkKCwu56667+K//+i/S0tIYNWoUv/rVrzAMo8WfR6A+jBs3rt3nDfbnumzZMpYuXQrAiBEj6qbqbdu2rdk59J9++ikzZsxg4MCB9OvXj6uvvpq//e1vAft16NAhfvzjH3P++eeTlZXF7NmzOXXqVIN9g/0Znzt3rtVztfZ3cttttzFw4EAGDRrEnXfeydmzZ0NuR0v+85//tPgeENzvtLnfy7PPPtvmv+9g3zuU/UL5PQdy6NAhFi1axJgxY8jMzOSiiy7i5ptvrrs+tMW+ffu46KKL2L9/P9dffz1ZWVlccsklrFmzptVjg/13A6H1/f/9v//HNddcQ3p6Ol/96lf5v//3/7b6b7ix/fv38/3vf5+cnBzS09O55JJLuO+++0I6h4iEh0ZUROLU4cOHAejTpw8AH330Edu3b2fKlCkMHDiQL7/8khdeeIFvfvOb7Ny5k/T09FbPOWfOHAYOHMjPfvYzqqurycvLY/v27UybNg2A7du3Y7VaOX78OEePHmXQoEEA7Nixg8svv7zFc//kJz/hD3/4A/PmzeO//uu/OHPmDP/4xz/45JNPuOKKKwA4efIkX//613G73cyePZuMjAw++OADHnzwQb788ksef/zxkPrgcrnafd5gf65TpkwhPz+fDRs28Nhjj5GSkgJATk4OX3zxRZPzfvbZZ/z3f/83TqeTBQsW0L17d9atW8eNN97ISy+9xJQpUxrsP3fuXNLT0/nZz37GoUOHeO6553A4HKxevTqkn3Gw52rJrbfeSmZmJg888ACffPIJa9eu5fjx43Uf+oNtR3veI9jfaXO/lyuvvJLHH3+8TX/fwb53W/7u2vq7+e1vf8u///1vpk2bRlZWFvn5+fzmN79h1qxZ7NixI6ifeWOffvopCQkJzJo1i5tuuolvfOMbvPLKK/zkJz8hKyuLa6+9ttlj23I9aq3v//73v/n2t79Nz549WbRoEU6nkxdffJHu3bsH3adNmzYxe/ZsMjMz+cEPfkDv3r3Zu3cvO3fuDP0HJCLtpkBFJE6cPXuW0tJSqqqq2LVrFz//+c9JSkqq+7Bw9dVXc9111zU45sYbb+Tyyy/n5ZdfZtGiRa2+x0UXXcTLL79c9/22bdsafMj54IMPmDx5Mrt27eKDDz5g0KBBlJaWcvDgQW699dYWz+37gPDYY481u8+jjz5KdXU127dvJy0tDTADj4yMDH7961/zgx/8oO7DY7B9aO95g/25fvWrX+Xiiy9mw4YNfOtb32q1nQ8//DAVFRW88847dU/rZ8+ezfjx47n33nv51re+hdVaPyj+la98heeee67ue8MweP7553niiSfo3bs3ENzPONhztSQzM5Pf/e53WCwWANLT08nLy+Pdd99l8uTJQbejPe8R7O+0pd/LuHHj2vT3Hex7t+Xvrq2/m5/85CdNpjkOGDCAn/70pxw/fpz+/fuH9PM/ffo0x48fp1evXrz77rtcdNFFAOTm5jJ8+HBefPHFFgOVtlyPWuv7//zP/+ByufjrX//K+eefD8BNN93EqFGjgurTkSNHmDt3LhMnTuSFF15o8POqrq4O6hwiEl6a+iUSJ6ZPn86FF17IsGHDuPXWW0lPT+f1118nMzMToMFNt6KiglOnTtG7d28uvPBC9uzZE9R7zJ07t8H348ePZ//+/fznP/8BzCfLV1xxBWPGjKn7gLd9+3YMw2h1RKVnz57s3r2bgoKCgK8bhsEf//hHrr32Wmw2G6WlpXX/XXXVVXi9XrZv3x5yH9p73nD8XBvzeDxs3ryZ//7v/24wpahXr17ceuutHD9+vG4hc3P9mjBhAh6Ph+PHj9dta+1nHMq5WnLbbbfVBRAA8+fPB+B///d/Q2pHW98jXH8rbfn7Dva929rGtv5u/P9Oz5w5Q2lpKT179gTMNMOh+vTTT+va4wtSAJKTkxkxYkSrqYDb8u+mpb77/5vxBSkAffv25YYbbgiqT8uWLcNisbBy5comQV1CQkJQ5xCR8NKIikicWL58OTk5OSQkJNC/f3/69+/f4INcVVUVjz32GOvXr2+Sttg33aU1gwcPbvD9+PHjMQyDHTt2MHr0aA4fPsz48eNxu9289tprgPnhrlevXk3m9Te2dOlSFi5cyFe/+lWGDx/O17/+dXJzc8nJyQGgpKSE06dP88orr/DKK68EPEdJSUnIfWjvecPxcw30fuXl5U3WPQB1P48vvviCiy++uG77gAEDGuyXnJwMUPchG1r/GYdyrpZceOGFDb5PSUkhOTmZY8eOhdSOtr5HuP5W2vL3Hex7t7WNbfndeDweXn/9dZ5//nkOHjxIeXl53WsOhyPk0RSoD1QCjZRardZWk1S05d9NS30vKSmhoqIi4MJ+/0CqOS6Xiz//+c9Mnz69brqsiHQ+BSoicWLUqFF1Wb8CWbx4MWvXruX73/8+48aNo1evXlitVu699168Xm9Q75GUlNTg+5EjR9K9e3d27NhBdXU13bp1Y+TIkdTU1PDII49w8uTJuvn7/tOUApk+fToTJkzgr3/9K++++y6rVq3il7/8Jc888ww33nhjXRu/853vcPPNNwc8xwUXXBByH9p73nD8XEPR3MLg5jJw+e/f2s84lHO1xD9Abk872voe4fpbacvfd7Dv3dY2tuV384Mf/IA33niDG264gXnz5tG3b18SEhJ49NFHqaysxOFwNP9DaMa+ffs477zzmgQPXq+Xffv28a1vfavF49vy76alvvv639rfXnMOHz7MuXPnuOSSS1rdV0QiR4GKSBexYcMGZsyY0WSB7unTp9v8BNFut3PppZeyY8cOXC4Xl112GQ6Hg9GjR5OQkMDbb7/Nvn37uP7664M6X0ZGBnPmzGHOnDmcPn2aq6++muXLl3PjjTfSt29fevXqhdvtZtKkSW1qbyDtPW8oP9dAH6Kaa1P37t05ePBgk9fy8/MB2ly/pKWfcbh89tlnDUY8SktLOXPmTIMPte1tR0vvEervtLnfS1v+voN9b4/H0yF/z40dOHCA9evXs2TJEn7yk5/UbS8rK+PTTz9tkpQhWJ9++mnAwOGPf/wjJSUlTJ06tcXjw309Sk1NpVu3bgH/zRw6dKjV4331poL9NyoikaE1KiJdhM1ma/Jk8Y033uDLL79s13nHjx/P3r17eeeddxg/fjxgzuceNWoUTz31FB6Pp257czweT5Nie8nJyQwaNKiuWrjNZmPq1Kn86U9/CjiH/cyZM22aa9/e84byc/VNh2mtArrNZuOqq65i06ZNfPbZZ3Xby8rKeOGFF+jfvz/Dhg1r8RyNBfMzDpfnn3++wc/k2WefBcwF1OFqR0vvEervtKXfS6h/38G+d0f9PTd24sQJoOFUOa/Xy49//GMqKipanZLZnP3791NSUsKBAwfqtp09e5Zly5Zx2WWXcdVVV7V4fLivRzabjcmTJ/P222/XZTwEc/rc7373u1aPv+CCC7BYLPz9739v8lo4fg8i0jYaURHpIr7xjW/w2muv0bNnT4YOHconn3zChg0bmqzZCNXll1+Ox+Opm7/vM2HCBH7xi1+QlJTEyJEjWzxHWVkZQ4cOZcqUKXz1q1+lV69e7Ny5k3feeYfbbrutbr+HHnqI7du389///d9897vfZejQoXVPht966y3++c9/BpVmubH2nDeUn6tvWskjjzzC9OnTcTqddfVuGnvggQd47733+MY3vsG8efPq0hMfP36cF198sdWpdI0F+zMOh4KCAnJzc7n22mv517/+xUsvvcTEiRO56qqrOH36dFja0dJ7QGi/0+Z+L6mpqW36+w72vTvq79nfxRdfTPfu3bn//vspKCjA7Xbz+9//HrfbDRAwUElOTmbChAn8+c9/DnjOo0ePcvbsWS655BJuvPFGFixYgNvt5je/+Q1nzpxh/fr1rbarI65H9913H++++27dvxmHw8GLL77IgAEDmgTHjfXq1Yubb76Zl19+Ga/Xy9e+9jXKy8v54IMPGDFiBPfcc0+b2yUibadARaSLePzxx3E4HGzcuJFXXnmFkSNH8vvf/54HHnigXee97LLLcDqdAFx66aV1231ZkC699NK615vTrVs35s2bx5YtW/jrX/+K2+1m0KBBPPLII/zgBz+o269v375s3ryZvLw8/vznP/Piiy/Su3dvLrroIhYvXsx5553Xpj6057yh/Fwvu+wy7r//fl588UUWLlyI1+vlrbfeCnje7Oxs3n77bZYuXcozzzyDy+Xi4osv5rXXXuOaa64JuY/B/ozD4Te/+Q1PPPEEjzzyCGCmiPWlIg5XO1p6Dwjtd9rc7yU1NbVNf9/BvndH/T37S01N5cUXX+T+++9n6dKlnH/++dx2221UVVVx3333NRmZO3fuHGBOzWuObyH9k08+ye9//3see+wxvF4vX//611myZElQwUZHXI+GDh3Kxo0buf/++8nLyyM1NZW5c+eSmprKHXfc0erxP//5z8nMzOT3v/89b7/9Nr169WLUqFGtrrcRkY5jOX36dGglW0VERAJYtmwZy5cv58CBA+0eCZDO8b//+7/ceOONvP/++yFPLxQRCTetURERERHALOI6ffp0BSkiEhU09UtEREQA6qbSiYhEA42oiIiIiIhI1NEaFRERERERiToaURERERERkaijQEVERERERKKOAhUREREREYk6ClRERERERCTqKFAREREREZGoo0BFRERERESijgIVERERERGJOgpUREREREQk6ihQERERERGRqKNARUREREREoo4CFRERERERiToKVEREREREJOooUBERERERkaijQEVERERERKKOAhUREREREYk6ClRERERERCTqKFAREREREZGoo0BFRERERESijgIVERERERGJOgpUREREREQk6ihQERERERGRqKNARUREREREoo4CFRERERERiToKVEREREREJOooUBERERERkaijQEVEROLC9u3bmTFjBkOGDCE5OZlXX321xf23bdvGzJkzycnJoV+/fowfP56XX365yT7JyclN/jt48GBHdkVERAB7ZzdAREQkHMrLyxk6dCgzZ85k/vz5re7/4YcfMmzYMO666y4yMjLYvHkzP/rRj0hMTCQ3N7fBvjt37uS8886r+75v375hb7+IiDRkOX36tNHZjRAREQmnrKwsfv7zn3PTTTeFdNwtt9yCx+OpG1nZtm0bU6ZM4dChQ6SkpHREU0VEpBma+iUiIlKrrKyM5OTkJtsnTZpETk4OU6dOZevWrZFvmIhIF6SpXyIiIsDbb7/N3//+dzZt2lS3LSMjgyeffJJRo0bhcrl4/fXXue666/jTn/7EhAkTOrG1IiLxT4GKiIh0eTt37uS2225j+fLljB49um57dnY22dnZdd+PGTOGL774gqefflqBiohIB4vrqV/5+fmd3YRO05X7Duq/+q/+S/A++OADcnNzuffee5k7d26r+48ePZrPP/+81f26+u9B/Vf/u7Ku3P9w9j2uAxUREZGWbN++ndzcXH7605+yYMGCoI755JNPSE9P7+CWiYiIpn6JiEhcOHfuXN1Ih9fr5fjx4+zdu5fzzjuPAQMGsHTpUnbv3s2bb74JmBm9brzxRubOncsNN9xAUVERADabrS798IoVKxg4cCBDhgzB5XKxfv16/vznP7N27drO6aSISBeiQEVEJEYVVMGzJ6DIBelOmJ8FmYmd3arO89FHHzFlypS675ctW8ayZcuYOXMmK1eupLCwkMOHD9e9vm7dOioqKnj66ad5+umn67YPGDCATz75BICamhoeeOABvvzySxITExkyZAjr16/nmmuuiVzHRESiVKD7UDgpUBERiUEFVXDbfihzg9UC+8/B7rPw/JDOblnnueKKKzh9+nSzr69cubLJ9423NXbXXXdx1113haN5IiJxpbn70H2JNrJbPzwoWqMiIhKDnj1h3hw8QEE1HKuGg+Xwi6Od3TIREekKfPchq8X83moxv3+tPDls76FARUQkBhW5zCDlSBWUeaDKC+e88FYJFLttnd08ERGJc0UucHvhRDUcrTK/ur1Q6gnfhC1N/RIRiUHpTni3GrxG/TbDAAzzaZYqfIiISLj5r0nZVwaHq8AALBbzHlThgf9K8ITt/RSoiIjEoPlZ8NtC88bgu0HYLJDqCO/TLBEREWi4JsXthUNVUG2ADXBQey+q/RouupuJiMSgzESYnAxvloDbMC/mmQlgt0KKxd3ZzRMRkSjT3kyRvjUpbi8crYYaAyyAF3ADvW2Q4YRKb/imHytQERGJQQVVcKjSXLxoMwCLOT94WHeY0e000KeTWygiItGipUyRwQYrRS7z2JNu8PgFKTbAYTG/2oAUW/gelmkxvYhIDHr2BLgMGJQAPe2QaIXuVri4B6TZwzc/WEREYl9zGbqePRH8OdKd5rpId+2UYztmsOLj8pr3oxndT4et3RpRERGJQb4nW04bZPmNspd70SMoERFpwHfP8Ge1mNuDNT/LHIWxV5vrIq1WSDSgm80cYbmgmzlCU34sfA/LdDsTEYlB3a1woqo+JaTLYz7pSnd2dstERCTa+EZD/IV6z8hMNAORKanmyEkPqxmcZCXAsB7wyrDQ1rwEQ4GKiEiMKaiCT86ZoydVXnP4/mg1OC3mEy8REYlOBVWw5BDcvt/8WlAVmfedn2UGF75gxWuY34d6z8hMhCdz4O+jYWY/uCCxNomLw5xGFu7+RDxQ2b59OzNmzGDIkCEkJyfz6quvtnrMvn37+OY3v0lGRgZDhgxh+fLlGIbR6nEiIvHo2RNQ7YXzk6CnDZJs0M0Kw3uG/2mWiIiEh29B+3un4EA5/K0Uvv5P+O4nHR+0+EZDJvWBnO7mV99C+rYET5mJZpBzqjYL2BdVZr9u2x/eosMRX6NSXl7O0KFDmTlzJvPnz291/7NnzzJt2jTGjx/Pu+++S35+PgsXLqRbt27ceeedEWixiEho2psCsjW+ucZuj5mzHgALnAxhrrGIiESW/4L2GgO+qAaPF3aeNafxhpqFK1SZifDwhQ23tScbWHML9MNZdDjigco111zDNddcA8CCBQta3f93v/sdlZWVrFy5kqSkJIYOHcrBgwdZsWIFd9xxB5ZwVpUREWmncKSAbOncz56APWVwsgqqMBcwejCH8bf8Bz46Az3C0REREQkr/wXtJ13mddtiMbNo+WfhahxMdKSWsoG11o7mFuiHs+hw1K9R+fDDD7n88stJSkqq23bVVVfx5ZdfcvTo0U5smYhIU+FIARmI/5QBmwFnvVDhNZ/KeWqHVWq8cPO+8A67i4hIePgvaK+p/WoYYPe7XxS5IruOpT3ZwJpboB/OOipRn564uLiYzMzMBttSU1PrXhs8eHDA4/Lz8xt87Yq6ct9B/Vf/O6f/+afSqXE3TaPyWYmLfG9Rk+3FbhuvlSdT4rHT1+ZmRvfTAeugPHUmhZKqbnU3lCQcnMOKAVgxzHz2FiirMXitPJm0IPufnZ0dSvdERKSNfOl9y9xmgcRKA2wWSK39NO41zIyOHTUqH0i603wP/2Al2Gxg/v2xWuoX6M9IPE24ig5HfaACNJne5VtI39K0r+zsbPLz87vsTbgr9x3Uf/W/8/qffQhOnGp60c/olcjL1l4N1q0APLYfygyw2uCEAZ9X9Ql4Q3LthyS/7xOB8hqz2Jaz9s0MAxLs5rB7V/79i4hEI9+C9mdPwJFK80N+T5tZD8v3IR+g1AWlbnNKmN1iFlLsqClhzQUbwWQDy0yEu/rDHQeh3APdbXD/YEg7Hb46KlEfqKSlpVFcXNxgW0lJCVA/siIiEi0CXfQTrLC3zKwk7/+ELKdb8HODGz/1SrXD6Rrw1r5u1D6ZS7GHd9hdRETCx39Be6DEK/fkw7Fqc0qvxWJe2ys8cLii49rjC55CTQDz0RmY928zkLJY4Kzb/P7pZCfhelQW9YHKmDFjeOihh6iqqiIx0fypbdmyhX79+jFo0KBObp2ISEOBLvrn3PCPs00Dku1nIKnRSsHm5gY3DoDsVhjSDYpqzFTFDlttkOIM77C7iIh0jEBZuE646oMUML96DHN7JNsRjLs/qw9SwPzq8sLys2lcF6a2RXwx/blz59i7dy979+7F6/Vy/Phx9u7dy7FjxwBYunQpU6dOrdv/O9/5DklJSSxYsIBPP/2UN998k1/+8pcsWLBAGb9EJCr5Lvqrhphfy72BFytC8JWCA+XAf204bBkN38uEyX3gmr7mPoHWuIiISPTrnwC2Rp/ObVZze7QprakPUnwsFjjtieE6Kh999BFTpkyp+37ZsmUsW7aMmTNnsnLlSgoLCzl8+HDd671792bjxo0sWrSIK6+8kuTkZBYuXMgdd9wR6aaLSBcR7joozS1WnNDbrDB/pBLcmBfkwUnNzw1u7qlX421dO42CiEjsGpwEAyvMIKDGMBfdpzjM7dEmxQFF1Q2DFcOA5DA+LIt4oHLFFVdw+vTpZl9fuXJlk23Dhg3jr3/9awe2SkTE1BF1UJpbrPjdDPj/GkUVhhH4HCIiEv989wuHpfXF7R1dXLg1T1wE1+2tn/5lGOC0wj29ioHwLM+I+joqIiKR1BF1UDIT4dELzHUl/3GbXx+9ADaWmAvssxJhUKL51WW0v+aKiIjEpkDTfAM9KPOvrXWg3Px62/6Wa66Euz7LJb3hj8NhaA9Ic5pf/zgchiaGb0FN1C+mFxGJpNaKX/k/wepe+6in3Nvy06yCKrj/c3B74Ty7+fX+z81h87YW2hIRkfgUzOL2UCvKd8RsATCDlXdHN9yWXxx437ZQoCIi4qel4lf+F3oPcLjSrGMyKAH2W5u/6Dd3Q6n0muduS6EtERHpeJ09vao5oVaUDzWwiRaa+iUi4md+ljkf2JeNy39+sP+F/qQLDMy0kSfdLU8Ra+6GkuVs/r1ERKRztWV6VaSkO4PPGgnN34cOV4R3Oli4KVAREfHT0vxg/wt9Te0NwmIxqwdD80+zmruhnN8tuLnIEpzt27czY8YMhgwZQnJyMq+++mqrx+zbt49vfvObZGRkMGTIEJYvX47RKKPB+++/z8SJE0lPT2fEiBGsWbOmo7ogIlGkI9YsBhLq2pGCKrM+15cuOFEFLk/rD7oC3YeqPLC7LDoDMR9N/RKRuOcbuj9cYRbN6p9QnwY4UFDQ3Pxg/2lhDgtUYWY5sfvVRAn0NKu5rF++94/mYfdYUl5eztChQ5k5cybz589vdf+zZ88ybdo0xo8fz7vvvkt+fj4LFy6kW7du3HnnnQAcOXKEG264gZtuuonnnnuOnTt3cvfdd5OSksJ114WrpJmIRKNQp1e1RahrR/z3T3NCcbU5qj+lLywa1PyDrkD3oTIP9LZH93QwBSoiEtd8F/VSFxyrNqdqHagw89SHupDQ/0Kf6oRzleaFPdXe8tOsQNXqo2Weczy55ppruOaaawBYsGBBq/v/7ne/o7KykpUrV5KUlMTQoUM5ePAgK1as4I477sBisfDCCy+QkZFBXl4eADk5OfzjH//g17/+tQIVkSgWjrUlLa1ZDJdQ147472/FzBbpNaCHveX+BboPHamELxqNnkRbQhcFKiIS13wX9VK3GaRYap8kldaYr9+8D7K7BXcja3yhH9vL3N5a1i/fsdHyhEpMH374IZdffjlJSfWV1K666ir+53/+h6NHjzJ48GA+/PBDJk+e3OC4q666it/+9rfU1NTgcDgi3WwRaUW4Mly1NBoeLqGO2rRnlKfxfWjJITNYieaELgpURCSu+S7qbqNh9dxqL3xRDQ7MC3OwNzIFHPGjuLiYzMzMBttSU1PrXhs8eDDFxcVMmjSpyT5ut5vS0lIyMjJafI/8/PwWX4936r/63xmeOpNCSVW3Bh/AS6ph2b8q+GHv0pDOdV+ijdfKkyn12EmxuZmReJryYx6C6Vkw/XeX9eVoZU88WLBbDPpYPdgwcFJBfn7TtjrLU6hs1DevQbP7t+Rat433azIo91rrArHuVi/XugrJz29fdflQfvfZ2dnNvqZARUTimm/o3l5bNdcXrNQY5kW5W+1VMBrn5krHs1gaPpr0LaT33x7MPs1p6QYc7/Lz89V/9b9D36O56V2u/ZAUYH9X90Sys/u0eGxj2cCEBlv6BNW2YPpfUAVHz4DLYo741wAFXjvDusO9wxLJTGz6Xvc2Gi3yjfLcOyTw/i3JBl4J+HO4IKTzNBbO370CFRGJa76he5cXKjzmzcBmBRtmLZTeVjhRbY642GtTNUrXkJaWRnFxw8pkJSUlQP3ISnP72O12+vQJ7UOBiIRPS9O7Wltb0lHFD0P17AlwGWYtrpPu2vsQcHGPlqcRh3PNY7TPElCgIiJxzf+i7p/163g1nHFBgQuq/VI2/r8y8yamhe7xb8yYMTz00ENUVVWRmGj+wrds2UK/fv0YNGhQ3T5//vOfGxy3ZcsWLrnkEq1PEelELS1Cb21tSbQUP/RNTXbaIMtWv73c2/JxwQQX0VqoMlSqoyIicc93UX/5Ynh3NKz9KrwyDM55odIAL+Z/BnCmBpZ+Ht0FsCSwc+fOsXfvXvbu3YvX6+X48ePs3buXY8eOAbB06VKmTp1at/93vvMdkpKSWLBgAZ9++ilvvvkmv/zlL1mwYEHdtK45c+ZQUFDA4sWLOXDgAGvXrmXdunXccccdndJHETG1tKi8pXpYrR0bSaEWbQxWNBeqDJVGVEQkanXkE6HMROhhM7N/GZhrV+wW8//fPGlmAuvMKQESuo8++ogpU6bUfb9s2TKWLVvGzJkzWblyJYWFhRw+fLju9d69e7Nx40YWLVrElVdeSXJyMgsXLmwQhAwePJj169dz3333sWbNGjIyMli+fLlSE4t0stamdzU36lBQBfkVZrYrp9VMNe+wdE62q47KKhYtI0bhoEBFRKJSJOYQ261mcOK/JtrlNbfFwwW+q7niiis4ffp0s6+vXLmyybZhw4bx17/+tcXz/p//83/YunVre5snImHUlg/5H50xU9KXuaHCC1aPOc1qYAL0cYQ37XAwOqrGVrSMGIWDAhURiUqReCI0oTccr6qvr1KbzInujSbFxuoFXkQkXoX6Ib+gCm7aZxb/9VjMZCoewO6BJFvnjZp3xGL2SBSqjBQFKiISlSLxRGjRINhbBkerwI15QbRgPlnzF6sXeBGReBbKh/xnT8A5t5kC2PdQygBcQJYzvqb2RqJQZaQoUBGRqBSJJ0KZiebCev8nctP6wv2fx8cFXkQknrRn3WKRy3wg5b923YKZSOVEnI2Yd9SUss6gQEVEolKknggFeiIXLxd4EZF4UFAFvzgKb5UABqQ6YL81tHWL6c7aNYnUBysGYLOYKevjTbTXRwmWAhURiUqd+UQoXi7wIiKxzpdY5WC5mVLeMMyF8IMSQlu3OD8L3iiCaq+5NsVbW+R3UAIMDlTGXqKCAhURiVoKGEREujZfYpVqr5mV0QAsBhS6YGBS8OsWMxPh1WFm1q9yDziskGKHFKem9kYzBSoi0inipWquiIgEL9Rrf5EL3LXFeT3Urys544EqT2jrFi/pDX8bpXtPLFGgIiIRF4kaKSIiEl3acu1Pd8K7NWDF/M/ADFYAyjyhj4ZopD62WFvfJfxWr17N8OHDSU9PZ+LEiezYsaPF/Tdv3szVV19N//79ueCCC5g5cyafffZZhForIqEqqIIlh+D2/ebXgqqGr928D/adgy9d4PI0rJEiIiLxqaX6WM2ZnwXUFuF1WsGGuQA+2QGje+nhVryLeKCyYcMGFi9ezN13383WrVsZM2YMubm5HDt2LOD+R44cYdasWVx++eVs3bqVP/zhD1RVVZGbmxvhlotIMHxPzN47BQfKza+37Te3+177vAKqDThTAwcrze+/dMHhis5uvYiIdJS21MfKTIQpfaGHFZKsZp2rCxLBasCBiqYPwyS+RDxQeeaZZ5g1axazZ88mJyeHvLw80tPTWbNmTcD9P/74Y2pqanjwwQe54IILGD58OD/+8Y85fPgwpaWlEW69iLSmpSdmvtecVvB6zcJbbqDCMLfvLtMNR0QkXqU7zWxb/oKpj7VoEHylOwxIgFQnHK82M3/ZjIYPwyT+RDRQcblc7Nmzh8mTJzfYPnnyZHbt2hXwmJEjR+JwOFi7di0ej4eysjJ++9vfMmrUKFJSUiLRbBEJQUtPzHyvpTrBY6mfa+wxzIClymvmyhcRkfgzP8ush+ULVoKtj+VLVz+pD7gN6G410wo7beGZOtzSdGXpXBFdTF9aWorH4yE1NbXB9tTUVIqLiwMeM2jQIDZu3Mgtt9zCokWL8Hq9DB8+nDfeeKPF98rPz2/wtSvqyn0H9b+z+u8sT6GyqluTivJOzHldvteScFCBtS6Li80wKHfDxkIP073HSLN72tUO/f6D6392dnYHt0REuhJfVq/8U+lkH2qYVas99bF8i+CLXOa0Yn+tTR9rrb1K7hK9OiXrl8XS8HGrYRhNtvkUFRVx5513MmPGDKZPn865c+d47LHHuOWWW3jrrbewWgMPCmVnZ5Ofn99lb8Jdue+g/ndm/+9tdNH3PTG7d4h5xfe9lghU1GZycVrM64JhgN1mZZPzgnZlZdHvv2v3X0Q6h/+H/hq3kxOnmn7ob2/WrXSnGUw0fhgWSppify1NVw5ndjCl5G+biE79SklJwWazNRk9KSkpaTLK4vP888/TrVs3Hn74YUaMGMGECRN47rnn2L59e7PTxUSk8/gP0ed0N7/6blKZifDoBWC3msEJmE9LLBaz2rDNAqmOtj8ZExGRztOWrF6hauv0sea0ZYF/qFpKMiMti2ig4nQ6GTlyJFu2bGmwfcuWLYwdOzbgMZWVldhstgbbfN97vd6OaaiItIvvidmqIebXzETzgvyTA3DDv+BYJfS2QW+rWbjLaTFvNIMSzCCmrU/GRESk80TiQ39LD8Paoq0L/EMRiQAuXkV86tfChQu5/fbbGT16NGPHjmXNmjUUFhYyZ84cAJYuXcru3bt58803AbjmmmtYsWIFjz/+OLm5uZSVlfHII4/Qv39/Ro4cGenmi3Q5BVXmAvftZ8zvJ/Q2M7CEclPwPU06WA7nvOboSYUX+jmgusYMVLIS2v9kTEREOk+4p2U1J5xFG+dnmdPTGk9XDud9KBIBXLyKeKBy/fXXc+rUKfLy8igqKmLIkCGsX7+egQMHAlBYWMjhw4fr9p84cSKrV6/mV7/6FU8//TSJiYlceumlvPHGG3Tv3j3SzRfpUgqq4Hv/gv0VZmYuiwWOVcEn5+ClYcEHK76nSe7a7y0W83xnvOYoisdiPhnTvF0Rkdjl/6EfYuPhU3sW+AcrUgFcPOqUxfTz5s1j3rx5AV9buXJlk23Tp09n+vTpHd0sEWnk2RNwtKo+SAEzpfCRytAWGvqeJjks4JuSa7GYaSbtVvh6n/AuWhQRkcjz/9D/WYmLi/okxsTDp3CO0AQSiVGbeNUpgYqIxIYilzkK4p+UzzDgrBf+eNL8PpibkO9pUqoTyqvMi7RhmBcgXaxFROKH70N/vreI7At7hXRsvGbGisSoTbxSoCIizUp3mhcJw6jPzOUywGJAhcfMXBJMvnnf06RTNZBoNZ8q2Sww+Tx48EJdrEVEurp4r2fS0aM28SqiWb9EJLbMz4JBiWZQYRjmVC0DSLBCqj34zCW+tMRn3eDxQu/aDF/HtZBQRESITGYsVaCPPQpURKRZmYmw9qtwYzoMSoJuNuhjh8EJ4KzNGh5s5pKNJZDigMFJZoavRJvSM4qIiKmjM2OplklsUqAiIi3KTIQnc2DXGLgl0wwynH6ljYLNXKL0jCIi0hxfPZMaAwqqzUQuJ6qge5g+qaqWSWxSoCIiQWtPReBIFNUSEZHOV1AFT51JCWmK1fwsc1rx4Uoo80Clx6y3tbcsPKMeelgWmxSoiEjQ2lMRuD1BjkgoVq9ezfDhw0lPT2fixIns2LGj2X2XLVtGcnJywP9OnjRT223bti3g6wcPHoxUl0Rihm+K1YdV3UKaYpWZCBf3MEdQEq3m/WFQgpnAJRyjHnpYFpuU9UtEQtLWzCVKzyiRsGHDBhYvXswTTzzBuHHjWL16Nbm5uezcuZMBAwY02f/OO+/k1ltvbbDt1ltvxWKxkJqa2mD7zp07Oe+88+q+79u3b8d0QiSAWEnd29IUq9buHeVeyArQp3CMeqiWSWxSoCIidUK5Ebblpqn0jNLRnnnmGWbNmsXs2bMByMvLY/PmzaxZs4YHH3ywyf49evSgR48edd8fP36cDz74gFWrVjXZNzU1lZSUlI5rvEgzYil1b3umWHVkBffmHpaBOT0t2gPArkqBikgc8QUP+afSyT4U2gW3oAq+9y9zAaMb8+Kw4z9m1q/MxIaBSXcrfHIOqr3Rf9OUrsPlcrFnzx7uvPPOBtsnT57Mrl27gjrHyy+/TO/evZk6dWqT1yZNmoTL5SInJ4dFixbxta99LSztFmlNa6MU4Rxtae+5fMGGv2CDjY4e9Wj8sCyWAsCuSoGKSJzwv+DWuJ2cCLIYo88vjsL+CvD4FXfcX2FuXzSo4cX8RJU5RH9+krnQLZShfZGOUlpaisfjaTJlKzU1leLi4laP93q9vPrqq8yYMYOEhIS67RkZGTz55JOMGjUKl8vF66+/znXXXcef/vQnJkyY0OI58/Pz29aZOKH+h6f/+afSqXE3/aT/WYmL7a4SHvhPBuVea92H+/eLvDxyXiFpdk9I71PstrX7XNe6bbxfk0G1YaW6ugqvAd2tXq51FZKf3/o57ku08Vp5MqUeOyk2NzMST1N+zENH/CU9dSaFkqpuDUZwSqph2b8q+GHv0nafvyv//YfS9+zs7GZfU6AiEifaMy8YYPuZ+iAFzK8ew9zeo9G53ZiFH0+6IDOh/v2UPUWigcXScN6JYRhNtgXyv//7vxw/fpzvfe97DbZnZ2c3uJGOGTOGL774gqeffrrVQKWlG3C8y8/PV/+D7H9roxjZh+DEqaZToi7qk8gmeuF2QJLfa24DNjkvCPnB0cuHqDuXywOn3FDkgQcqLuCVYcE99MoGnjgDC/9VSbktiRQHPHERpCdcENRITTbQ8F9Vn9A6EQLXfkgKtL17ItnZ7Xvfrvz3H86+K+uXSJwIR+rFxp/lfN83Pre9dsSlxi+DirKnSGdLSUnBZrM1GT0pKSlpMsoSyEsvvcTYsWMZMmRIq/uOHj2azz//vM1tFfEJphBh46yJ1V4oqYHDFbCpFBqPU7T1wZHvWu/ywNFq8wFVtQGfVwRfHLGgCu7/HDyGhfPs4PbCTz8zpxZHW7FFZQKLfgpUROJEey+4E3pD42fOltrtjc+dagebpX5IVtlTJBo4nU5GjhzJli1bGmzfsmULY8eObfHYL7/8kv/93/9tMprSnE8++YT09PQ2t1XExzca7sEsdHisGg6Wm9NuffxTww9MhLNu6GUz9z3lMmuPhOPBke9af9LdcITdaQ2+OGKg0f0jleb6x2grtqi0+dFPU79E4oT/IkQI/YK7aJC5QP5IZf1i+sFJ5nZouMDRboUh3WB4T3OtijKlSLRYuHAht99+O6NHj2bs2LGsWbOGwsJC5syZA8DSpUvZvXs3b775ZoPjXnnlFbp37860adOanHPFihUMHDiQIUOG4HK5WL9+PX/+859Zu3ZtRPok8a3IZQYpR6oaPhB6q8S8/vquq76F4EsOmddp34f+VAdUVENxtZnatz0ftn33kS+q6oMUqwVSncGP0gQa3XcH2C8apgsrbX70U6AiEif8L7iflbi4qE9iSBfczER4aVjzF2xdzCUWXH/99Zw6dYq8vDyKiooYMmQI69evZ+DAgQAUFhZy+PDhBscYhsHLL79Mbm4u3bp1a3LOmpoaHnjgAb788ksSExPrznnNNddEpE8S39Kd8G41uD316/8smMFLoDWGjQMBp80sjOixmIV423N99t1Hbt5nTvdyWs0gxWEJfpQmUNavQB82o2WKldLmRzcFKiJxxHfBzfcWkX1hrzYfH+prItFk3rx5zJs3L+BrK1eubLLNYrGwd+/eZs931113cdddd4WtfRJf2pvOd34WvFIALuqDFAOoMsw1KI0FqjVit8LX+4TnGp2ZCK8Ma5jpMZRRGt+oTEm1+b3XMEfnDcOsMq9iixIKBSoicajYbePlZgpYxUp1YxGRaNdSHY5gZSaaU7bOlJvfW2rX/3mBEwGmRkWiwnp7pkT5jl32rwpc3RMbFFbUvUdCpUBFJM4UVMED/8nA7Qh841RxKxGR8GgpLfx3QzjP+UnweaM1KhbgTA3cvr/hB/tIratozyh6ZiL8sHdpkxS/GpWXUClQEYkzz56Acq+1Lqe+1QKnasw5x+UeM0NMqsOc16xCjSIibddiWvgQPmENToKBFVBaY2bvslqgwm1m9zpQ3vShkqbiSlehQEUkzhyugJMeG0aVWe8k2Q4FLnBgTimo9ECF11x86QtWOjvziohILAq0XqRukbg3+PP4pnM5LOa5TtSm8k11mK/7Hir94ij0sGv6lHQdqqMiEkcKqmB3GZwzbFR5zRvbkSqo8YDDat4EfRXnT/qlMY6GzCsiIrEmXHU4GtdJcQM2zOu0q7aaowczZXG0FU0U6UidEqisXr2a4cOHk56ezsSJE9mxY0eL+xuGwYoVK7jssstIS0sjJyeHhx56KDKNFYkhz56A3nawW8y7pi8o8WIWafTlwrdYwG0o84qISHv4Bxg53c2vbV3zl5loXotLa8zpLjWYD5uOVpvBSnE1YERf0USRjhTxqV8bNmxg8eLFPPHEE4wbN47Vq1eTm5vLzp07GTBgQMBjfvazn7Fp0yYefvhhhg0bxpkzZygqKopwy0WiX5ELEqyQaXNz1mKjxjAXZNoxp3kBDE40b3h9nOZNVVMHRETaLpzrRXyL89MSoKLKfMjkMeBkDWAxHzj509RdiXcRD1SeeeYZZs2axezZswHIy8tj8+bNrFmzhgcffLDJ/vn5+Tz33HNs376dnJycSDdXJKb45ks7LAaZCea2aq+5INNb+yTOBnyluzJ9iYhEG9/ifCvmQ6WTLnNxfR8nTOgN/zjbcP/Wpu4qHb3EuohO/XK5XOzZs4fJkyc32D558mR27doV8Ji//OUvDB48mHfeeYcRI0Zw8cUXM3/+fE6ePBmJJovElEDzpfs4zOJd4ZiaICIiHSfdWX/9dlggMwEGJMC1KbBoUGjrYXw1XrSmRWJZUCMqHo+Hp556ip07d9KjRw+GDBnC8OHDGTFiBOnp6UG/WWlpKR6Ph9TU1AbbU1NTKS4uDnjMkSNHOHbsGBs2bGDFihVYLBYeeOABZsyYwd/+9jes1sCxVn5+foOvXVFX7jvEZv+L3TZeK0+mxGOnr83NjO6nSbN7QjrHfYk2XvMkU+pxkWJzMyPxND2KPWZO/9oqYuXHIPZ+OqGJxd9/OAXb/+zs7A5uSev+9re/UVZWxvXXX9/ZTRHpVC0Vcwy1fkpLNV6U2lhiRVCBykMPPcQLL7zA1772NTZs2IDFUp+HLzU1lREjRrB+/fqg39T/eDAXyzfe5uP1eqmurmbVqlVcdNFFAKxatYpLL72Uf/7zn1x66aUBj8vOziY/Pz8qbsKdoSv3HWKz/wVV8Nh+KDPAaoMTBnxe1Sfk0Y9sIK1B//u0tHtcisXffzjFWv+XLVvG2LFjAwYqn3/+OYMGDcJms3VCy0Q6TnPTsloKRkJZD9NijReRGBFUoLJx40Z+9rOf8f3vf5++ffvypz/9iZMnT/Loo49SU1ODxxPcE9+UlBRsNluT0ZOSkpImoyw+6enp2O32uiAF4MILL8Rut3P8+PFmAxWRcIrEPN/GT7/cXjhYDtP2msP+mlss8ergwYPcc889AV/Ly8sDYOXKlZFskkiH8k3L8l3zGxd0DMeIR4s1XkRiRFBrVEpKShgxYkTdqEdiYiLXXXcd7733Ht26dWPBggVBvZnT6WTkyJFs2bKlwfYtW7YwduzYgMeMGzcOt9vN4cOH67YdOXIEt9vdbJYwkXCK1Dxf/6dfLg8croRTbvi8En77JXzvX5pbLPHJarViGEbA166++mref//9CLdIpGO1NC0rXMJV40WkMwUVqPTp04eKigqsVit9+vShtLQUgO7du/PDH/6Q5cuXB/2GCxcuZN26daxdu5YDBw5wzz33UFhYyJw5cwBYunQpU6dOrdt/0qRJjBgxgoULF/Lxxx/z8ccfs3DhQi699FIuueSSUPoq0iaRuKFAw0WUhS6owizwZQBlHthfYVYlFok3F198MZs3bw74WlpampKnSNyJxLSscNZ4EeksQQUqF198cd3CzAsvvLBBhq6MjAz+9a9/Bf2G119/PcuWLSMvL48rrriCnTt3sn79egYOHAhAYWFhg9ETq9XK66+/TmpqKt/61reYPn06WVlZrFu3rtmF9CLhFKl5vv5Pv8q9ZoBiBex+1eS3nwnve4pEg9tuu40XXniBt956q8lr+/bta3ZqsEis8n8w5dMR07J808hWDTG/hhKkFFTBkkNw+37zq0b0pTMEtUblBz/4AZ988gkAs2fP5t5772Xo0KFceOGFPPnkk2RkZIT0pvPmzWPevHkBXws0DzkjI4OXXnoppPcQaavG61G6W+trkPg0vqGEYw2L/yLKw1Xg8YDdahZsBDNYEYlH3/72t/nHP/7B9773Pb75zW9y3XXXkZGRwb///W8ef/xxrrvuus5uokhYtZTdKxq0toZGJFKCClQmTZrEpEmTAOrSAs+dOxeLxYLD4eD555/vyDaKREygi7PTYlZ7r/YGvqGE84Lue/p1zg2vFZmjKj4WzIJfIvHo0Ucf5ZJLLuHxxx/n9ttvr9s+btw4HnjggU5smUj4hZpqONKU2liiRciV6W02Gy+++CKffPIJx48fZ/jw4WRlRckjAJEQBBoFCXRxdhlwaU/oYQ98Q+mIC/qiQfDJOThSCW7Mf6iDk8ztIvFq+vTpTJ8+nSNHjlBYWEhGRgaDBw/u7GaJdIhwZffqCEptLNEi5EDF5+KLL+biiy8OZ1tEIqbxKMjeMnijyBy1cBmQagdnbdkGq8VcM/JkMzeUjrigZybCS8Oi92mbSEcaPHiwAhTpEJFINR8PlNpYokWbAxWRWOY/ClJjwBfV4PGagYoBVHhgUIIZrPguzs3d4Drqgh7NT9tERGKN1l0EL9rX0EjXoUBFuiT/UZCTLvMibLGY61HcmEHLSTf0s5oX52l9629wbi+8WwO/LYQpfeG7GYEv6NP6mplS9ORORKTzad1F8KJ9DY10HQpUpEvyHwWpqV2xbhjmCEp/pxm8JNnMvPP+a1fcXjhabaYKtljgrZNwoAIevQA2ltRf0Kf1hfs/15M7EZFooXUXodGovkQDBSrSJfkPazssUGmAzWKuTXFYIMNpBim+i7TvBnfSXR+kgDn6UuY2gxT/C/qSQ3pyJyISTaJ93UVr62e0vka6IlVMlC7Jv2LvuN7Q2w4JwJc1cKLKnALmPxfXV5zLbTSsZ+KwBH4ipyd3IiLRxb+oLkTXugvf+pn3TsGBcvPrbfvriyy29rpIvFKgIl2Wb1j78YtgYCLY/P41WCxQVF1flfec2wxe7JhTxMAMPFKdgZ/IRarqsIhIPOqIquj+D6hyuptfo2U6bkvrZ4J5XSReKVCRuFTstgV9k3v2hFnMMSsRBiWaX8954OZ99U+v/nHWDF4mn2c+gethhcGJYCPwE7lofnInEu9Wr17N8OHDSU9PZ+LEiezYsaPZfY8ePUpycnKT/955550G+73//vtMnDiR9PR0RowYwZo1azq6G11WR44e+B5QrRpifo2GIAVaH4XXKL10VQpUJO4UVMED/8kI+iYX6AZQWgPlnoZPr855IL8KxvWCAUlwYVLzT+Si+cmdSDzbsGEDixcv5u6772br1q2MGTOG3Nxcjh071uJxv//97zlw4EDdf1/72tfqXjty5Ag33HADY8aMYevWrfzkJz/hpz/9KX/84x87ujtdUlccPWhtFF6j9NJVaTG9xLRAiwt/cRQO1zjAC/baBfItLWQPtMDS5QWHXxjvq7XiwLw5eA0zmHn8ouaDD2VMEYm8Z555hlmzZjF79mwA8vLy2Lx5M2vWrOHBBx9s9rg+ffqQnp4e8LUXXniBjIwM8vLyAMjJyeEf//gHv/71r7nuuuvC34kuri2jB7G+0Ly1uiWqayJdlUZUJGYFmh7wvX/BH09ChWGjymte1I9Wm2mFm7vJTesLJTVwpBIKqs1pYN1skOIXxp90mbVVfMFLV3jCJxJrXC4Xe/bsYfLkyQ22T548mV27drV47He/+10uuugirr322iYjJR9++GGTc1511VV89NFH1NTUhKfxUifU0YNoXWgeyhTk1kbhNUovXZVGVCRmBZoecLTKrCrvY7GY6YRP1gS+yRVUmfVOetnMUROXF8664ZfZ8Kvj9ed3eevTF/tofrBIdCktLcXj8ZCamtpge2pqKsXFxQGP6dGjB4888gjjxo3Dbrfzl7/8hTlz5rBy5UpuvPFGAIqLi5k0aVKTc7rdbkpLS8nIyGi2Tfn5+e3rVIwLpv/FbhuvlSdT4rHT1+bm6wllvF+TSrnXWjd60N3q5VpXIfn5nibHvV/ZndNeGyk2Dw6LGeGUVMOyf1Xww96lHda31vr0wH8yKPdW1fXh/SIvj5xXSJrd0+xx3wXzk5kXyo9B459ea69HG/39d93+h9L37OzsZl9ToCIxK9D0ADfgtEKN18D3QM5iASyBh8h9wU6iDbJs5javAe+fbViV12qBCrdZENJH84NFopPF0vDCYBhGk20+KSkp3HnnnXXfX3LJJZw6dYpf/epXdYFKc+cMtL2xlm7A8S4/P7/Z/vumah2ugN1ltSnibXDCgM/dfXhiWMMiuuZUrgsaHP/Yfigz4LQFqizwpWFncIKZNh7A1T2R7Ow+kehqEysPwOGaGrA5sAOpDnBbYZPzgi4zJbil339X0JX7H86+K1CRmBVobYkdwAKpNjdnLTZqDHPblL6Bh8h9QUiNYU7vqjHMm9yRyoZrTHxTCzQ/WCR6paSkYLPZmoyelJSUNBllacno0aN59dVX675PS0sLeE673U6fPp3zQTiW+V9Pv3SZX894zEyKDkvgIrqN+Y+o2y1m2ngv5nU8M6FzHyQVVMFbJeYUZKvXbFuFBwYlaBReJFRaoyIxK1AK4EGJMDgJbBhkJsCABPhKd1g0KPA50p3mmpQjVVDmgaraqV+7zzacT6z5wSLRz+l0MnLkSLZs2dJg+5YtWxg7dmzQ5/nkk08aLKwfM2YM7733XpNzXnLJJTgcjna1uSvyDzJ8RXS9tQ+LILhptf4j6ql2c2quYZgPmzr7QdKzJwC/NTatTUEWkeZpREVili94aJzpBcy5ya7uia1mf5mfBW8UmQvlLbU3OpsFetqaZglTFi+R6Ldw4UJuv/12Ro8ezdixY1mzZg2FhYXMmTMHgKVLl7J7927efPNNANatW4fD4WD48OFYrVbefvttVq9ezUMPPVR3zjlz5vD888+zePFi5syZw65du1i3bh2rV6/ujC7GPP8gwzcaYqkd2YbgRkP8R9SdNnO04mQN9HGaD5I6M+tXkQvSEqDcE9wUZBFpngIViWnNBQ8/7F0a1NzkzEQY3RN2njWf7PnSGTttGqIXiUXXX389p06dIi8vj6KiIoYMGcL69esZOHAgAIWFhRw+fLjBMb/4xS84duwYNpuNCy+8kF//+tcN1qcMHjyY9evXc99997FmzRoyMjJYvny5UhO3kX+QkWo3p0V5aqfdBjsa0jhdr91qjp5Hw0h3uhP2A5lBTkEWkeYpUJEu7/xuZrYw/7UuWigvErvmzZvHvHnzAr62cuXKBt/PmjWLWbNmtXrO//N//g9bt24NS/u6Ov8gw2kzp+iWeWB0L3PqbjCjIc2NqEdDIODrX3W1Ubdepqe9+SnIItI8BSrS5amQlohI5IQryIjW6bi+/gU7BVlEmqdARbq8aH4yJyISj6I1yAiXzMTgpyCLSPM6JevX6tWrGT58OOnp6UycOJEdO3YEddyhQ4fo378/WVl61C3h5btprhpiflWQIiJiKqgi6ArrIiLhFPFAZcOGDSxevJi7776brVu3MmbMGHJzczl27FiLx7lcLm699VbGjx8foZZKe+jGJiIS+3w1T947BQfKza+37W94Tdf1XkQ6SsQDlWeeeYZZs2Yxe/ZscnJyyMvLIz09nTVr1rR43IMPPsiwYcOUZSUGBHNjExGR6Odf8wTMr2Xu2loh6HovIh0rooGKy+Viz549TJ48ucH2yZMns2vXrmaP27RpE5s2bWL58uUd3URpB99TtWl74WA5eGq3N76xiYhIbPCveeLjX5CxtUBGRKQ9IrqYvrS0FI/HQ2pqaoPtqampFBcXBzymsLCQu+66i5dffpmePXsG/V75+fkNvnZFkex7sdvGA//JoNxr5UuPg2rDQrnHIM3q5qxhw21Y+FN1Dde6Ckize1o/YRh05d89qP/qf3D9z87O7uCWSCzzr3ni45++vcVAJgrS9RRUKVGKSCzrlMuIxdLwqmYYRpNtPt///ve59dZbueyyy0J6j+zsbPLz87vsTTjSfX/5ELgdkGSBhGpwucFrgRNem1l5GCjDxmNVF0SkIFdX/t2D+q/+d+3+S/i0lr69xUDG2ylNruObluZr+/5zZl+ioSikiAQnolO/UlJSsNlsTUZPSkpKmoyy+GzdupXly5eTkpJCSkoKd955J+Xl5aSkpPDiiy9GoNUSDP+naql2sFmgxmvesAzD/D7VoSkBIiKxxJe+fVIfyOlufvX/oD8/ywxcvIb5fWfXofJf2H/zPih1aVqaSCyL6IiK0+lk5MiRbNmyhW9/+9t127ds2cLUqVMDHtM4dfFf/vIXnnjiCTZv3kxmZmZHNldC4P9UzWmDQQlwqHYxZU+7Gbw4beb3vrnNIiIS/VqqedJSHapIT75sPIJypBJqDPN+5Lv/+K+vEZHoF/GpXwsXLuT2229n9OjRjB07ljVr1lBYWMicOXMAWLp0Kbt37+bNN98EYOjQoQ2O/+ijj7BarU22S+dqPD3AboUUB/SyQaKtfj//uc0iItI2way9iNT6jGgp3th4Yb/TClVuOOmGrNr7kO5BIrEl4oHK9ddfz6lTp8jLy6OoqIghQ4awfv16Bg4cCJiL5w8fPhzpZkk7BXqqNq0v3P95w7nNTgucc5vD8lrYKCISumDWXvjv4wHerYbfFsKUvrBoUHxed/2nILs84DHMEZWzbnNU327t3GlpIhK6TllMP2/ePObNmxfwtZUrV7Z47E033cRNN93UEc2Sdgr0VM0/eOluhU/OwT/OamGjiEhbNZcS+BdHoYfdvN7mV0Clp3YKVFX9esG3TsKBiui87rZ3BMg3BdnthaPVZqDisILFMEdV4jlIE4lXUZA8UGJRsDcU/+BlySGo9gZe2BgN0wZERGJBoJTAbi+8VQL9nH7rM4BEa/1Cd4sF3IR23Y3U9LG2ZOhq3LZpfc1jDpabQYrFYp5rcBLYMIM4BSkisUWBioSsrSkfWyscJiIirQuUEvhkjfnVt81Ruz7jnNec+mXUBivdLMFfdzsqvW+g4KelwpGBAqrm2vboBTD/ALhd4LBAqtP8CrrXiMSiiKYnlujjn8pxySHz+9Y0vqG4veYTrGl7Wz5HurP+yZ6PFjaKiIQmUEpgLJCWUL9Pqt28Rlcb5uiCB7OsSYUHqjzBXXc7ouq8L8B47xQcKDe/3rYfDlc0XF9yohqOVcOm0sD3lObatrEErk2BAQmQmVAfpOheIxKbFKh0Yc3dMFoLVnwjIy4PfFEJByrhlBsKq1s+R7Tl2xcRiUWBaptM6WtOb/Jx2iARSLCYN3ob4AQMC5R5grvudsQoeHMBxgmXeU9wecz1JWVuqPLCKVfge0pLbdO9RiR+KFDpwvxvGDUGFLpg3zmzSFZLwUq603wid7Qaznj8ntTVTjFo7olba4XDREQkOL71f6uGmF8XDWr64dxmgwsT4b+6m+niu9mhlx1G9wruutsRo+DNBRj9E8z2n6xpuL4kLSHwPaWltuleIxI/tEalC/PdMGqM+qwwAJ+Vw9X/NG9mg5OaLp6cnwW/LzJvJrUzDrBgVp8/6TKH25t74hYt+fZFROJJoBTx59z1WRYza6eFeQ3zuh4MX32sUzVQWgMuL3SzmYvWfUJdbB9ofY2vTY9fZE4hDmZ9SePaXY1HTXSvEYkPClS6MN8N46TL7ymcFyoAVw3sPGNmjmm8eDIz0Qxidp6BM25w+1JAYgY9mgssItJ2zX34by0oaPzhvPGC81CnQGUmmovTb95nBikOq1nE9/7PzXsChL7YvqUAIzPRXF/y3qmmgUzje0qgwEx1uUTijwKVLsx3w/iidpqXYYDXAtbaYfcao/nMK4OTzCCmr70+Xz0W8w9Kc4FFRNqmpWxW/gV0gwkKwvFhfmOJOW0s1S9Q8J+KFUqmrmDa1NpISeNzadREJL4pUOnCfDeMm/fB5xXgsEG1B2p86S39bj4tDbsPSqhNjWlRQS0RkfZobrH53Z+ZGRZbCgqaG3Fpz4f51hbUt2WxfUtt0kiJiPhToNLFZSbCK8Pqn+CdqIJyjzmi4sEcVbGhYXcRkUhoLjAorYHz7E23+4KC1mqetDZtrLnXm1tT4rsntPRaW2Um1tdWKXKZX3V/EemaFKhIXdDxi6NwvMrM4GUY5o2xtAb6NFo86X+cht1FRMKnucAgxdFwRMW33RcUtFTzZH5WfRDj9sK7NfDbwvoRcGg+yGk8FavKY6Y3PlIJfR3gtIDLaNsamOZ0VKFJEYk9Sk8sgHnx72Fv+sQO4JQH7joQXDFIERFpu+ZqgDxxUcu1QVqaouULYtze2holHjjngbdOmgHBL442H+T4p/odkABnPWaK4y+qzIxiFgtc2iu8aYB97fUABbWFHw+Wm+0Uka5FgUoX01Il+iIXFLjMlMON5Ve2rxqxiEikrF69muHDh5Oens7EiRPZsWNHs/tu27aNmTNnkpOTQ79+/Rg/fjwvv/xyk32Sk5Ob/Hfw4MGwt725GiCX9G65NkhLdUV8QcxJd32NEgA3ZkCw/UzLa018o+fndzNHURKs9ftUe82HXL56LuEY8ThSaRaAPFBhpkau9MI5L7xVogdmIl2Npn51Ia0Np6c7zVTDgbhpXzViEZFI2LBhA4sXL+aJJ55g3LhxrF69mtzcXHbu3MmAAQOa7P/hhx8ybNgw7rrrLjIyMti8eTM/+tGPSExMJDc3t8G+O3fu5Lzzzqv7vm/fAHNiw6C5abUtTbdtKVvWsyfM673bL0gBM2GK1QLUppVvba1JR1Sqb6ygyuzH6RpzGjKYafMdte1sKaOYiMQfjah0IS3NYQbzhpbQzF+EHdVGEZHo98wzzzBr1ixmz55NTk4OeXl5pKens2bNmoD733333dx///2MGzeOwYMHM3fuXKZMmcKbb77ZZN/U1FTS09Pr/rPZbB3dnaC1VI3dN53Mjrn+EMzrf2rtKMyE3i1PK/PpiEr1jT17AnrW/lh9b+Wt/S/VoQdmIl2NApUupLWnYZmJsGYIOBodZwWyk1QbRUSim8vlYs+ePUyePLnB9smTJ7Nr166gz1NWVkZycnKT7ZMmTSInJ4epU6eydevW9jY37HwjLo2nYfmCmCmpZgDSwwqDE82Mjj3t5oL6lqaV+TS3fiac94YiFyTaoLfNbJ8F82uSBexWPTAT6Wo09asL8WWT8WBWo68xzD+AUT3r97kmFf73EvjhQThaBTYLXHMePBimucciIh2ltLQUj8dDampqg+2pqakUFxcHdY63336bv//972zatKluW0ZGBk8++SSjRo3C5XLx+uuvc9111/GnP/2JCRMmtHi+/Pz80DvSQX5ghel9bLxWnkypx06Kzc2MxNOUH/MA8F0wbwpeKD8GgVp+X2Lg45vrZaj9d5anUFnVjT5AlcWBxzCfrtkML/YaF9e6CsnP94R0zs4UTb//zqD+d93+h9L37OzsZl9ToBLHGufFn9YXPjgN+8rNIXXDMAORvWXmvr5A5JLesO2yzmy5iEjbWSwNh44Nw2iyLZCdO3dy2223sXz5ckaPHl23PTs7u8GNdMyYMXzxxRc8/fTTrQYqLd2AI6FJfZTz4ekGD536hHS+bKBhj5s/fvv+z9nkvCCkWlv3+q2lHOz1LyZsZdEgO5mJF4TU3s6Un5/f6b//zqT+d93+h7PvClTiVLHbxmMBFs5fkGhmVHEDdguk2s0c+FqgKCKxLiUlBZvN1mT0pKSkpMkoS2MffPABN9xwA/feey9z585t9b1Gjx7Nhg0b2tXeULRWsLG5YzqrHklBFTzwnwzcjtDeW8WERcSfApUY1tKN67XyZMqMpgvn99RAVoALvhYoikisczqdjBw5ki1btvDtb3+7bvuWLVuYOnVqs8dt376dG2+8kXvuuYcFCxYE9V6ffPIJ6enp7W1yUNoacLSUQKWjH0w9ewLKvVaS2vDeKiYsIj4KVKJcc8FIazeuEo8da6OENKGkoRQRiUULFy7k9ttvZ/To0YwdO5Y1a9ZQWFjInDlzAFi6dCm7d++uy+q1bds2brzxRubOncsNN9xAUVERADabrS798IoVKxg4cCBDhgzB5XKxfv16/vznP7N27dqI9KmtAUck0glH43uLSPxQoBLFCqpg9j6/qVqYa0xeGtb6jauvzc2JAAHJhN5mEa1AufZFRGLd9ddfz6lTp8jLy6OoqIghQ4awfv16Bg4cCEBhYSGHDx+u23/dunVUVFTw9NNP8/TTT9dtHzBgAJ988gkANTU1PPDAA3z55ZckJibWnfOaa66JSJ8af+ivMcyEKH88aX7f3NQoXwKVzngwFYlUxiIS/zolUFm9ejVPPfUURUVF/Nd//RfLli1j/PjxAffdtm0bK1as4J///Cdnz57l/PPP5wc/+AHf/e53I9zqyPvF0fqF7z77ys3t5d6Wn1bN6H6az6v6NAlIFg0yX9f8XxGJV/PmzWPevHkBX1u5cmWT7xtva+yuu+7irrvuClv7QuUfcNQYcKQKPF7z3vDeqeangbVUBLKjzc+C94u8uA09FBORtot4oNKRVYPjzfYzDYMUML/ffgauTamtNOyFk26z4rB/quE0u6fFBYma/ysiEhv8A46TLjNIsdUmQ2lpGlhnLkzPTIRHzisMOeuXiIi/iAcq/lWDAfLy8ti8eTNr1qzhwQcfbLL/3Xff3eD7uXPnsm3bNt588824D1TATCHsn1XTV1V4fhbs+A8cqgZP7T4W4JNz5pQx0IJEEZF44B9w/PGk+cAq1Q7O2nWILa396Mz7QJrdo3uQiLRLRCvTd3TV4Hgzobf51MwXnPjqnkzobd58hveEblZIskFPG5yfBNVe82YmIiLxwxdwXJcK/Zz1QQpo7YeIxK+Ijqh0VNXgQHwVMWOhKmix26z0W+Kx09fmZkb306TZPUz32vjQmsExj5NKrxUDSDI8fPmfcmbusvLvGie9seCwmJGM1wVe4LMSF/SJjb53JPVf/e/Kgu1/Vy1IFmt8GSCPVEJpjflwKtGmtR8iEt86ZTF9uKsGB5KdnR0TVUELqjALMxpgtcEJAz6v6sPzQ2BCIjx9Bm7eBxYP2IAKj5W/u5I5PwnOYS6qPz8JHLU/Pq8BF/VJBG9R1Pe9I8XC774jqf/qf1fuf7xpnI6+lx3OuCG7G5zfTWs/RCR+RXTqV3urBufm5gZdNThafHQGJu+GETvNrx+dafh6S2mGATaWQIoDBieB1QqGxZyffNIFqQ5zXUpxtbmvnqyJiMSfxveJBCv0dZhBysMXKkgRkfgV0UDFv2qwvy1btjB27Nhmj9u+fTu5ubn89Kc/DbpqcDgVVMGSQ3D7fvOrb7F6az46A9fthU/PmQsdPyqDyR/BiA/gJwfM87RWFMv/dbffwvoaw5yjPCgB+jghpztM6tN6pWIREYktKp4oIl1VxKd+dUTV4I7UWgX4ltz9Gbi8gAWqvPXbv6iG14rMDF0X92i5Urx//ny7pT4LmG+ql90KX++j7F4iIvGqMws3ioh0pogHKh1RNbgjtVYB3p9vsaMvZ3xRtRlU1DQuhoI5fetIpRmo9LQ3X5DLP39+qh0qPLWpKZ2a6iUi0hV0ZuFGEZHO1CmL6cNdNTicGgcbRyqDG3IPNPJyxm0WZPQ0815uzMXwLRXkalywa1ztZL1yrwpoiYh0BZ1ZuFFEpDN1SqASrQIFG6U1ZoaVBL/VPIGG3AONvPR1wNHqptXl66ZtYZ6ntYJcKtwoItK16T4gIl2RAhU/gYKNnjZzZKSvo37IPcEK59zm4nrfk61Aix0rvNDdai6Cr/KLVryGud5kcJKG7kVEREREAlGgQv10rz+eNNeApNrrq/4m2sxc9alO2H7GDDqKqs2AJtFWv7g+p1vTRfEur1k1PivBXKfyZZU5ZSvBCt9Jg0WDNHQvIiIiIhJIlw9U/Kd7VXrMrxUeM+2vs7bqb6oTDlRAggX+UwNnPXDOW79Pmds8l9MCR6vMtSf22u/71P6EHRYYmGSeb5KydImIiIiItCiidVSikf90r1Qn2KzgMeCkuz6zSrkbDpbDsWpzGphh1O8D5rEnXfU1Tnz6JUCP2mAHlKlFRERERCRYXX5ExX9ticMCgxPNoCPJZo58TOsLN/wLyjxmIOI2wAs4DfP/wQxATrjMDF9ZtVO5XB4oqDYX4vd2QJbTrCKsTC0iIiIiIq3r8iMq6c76EQ8wg5UMJ1yXak7P2lgC+FWEd9T+xNyYC+J9oyT9E+oDHpfHzPZV5oFTtSmKT7kVpIiIiIiIBKvLByrzs8xAo7npWUUuSPMLQiyYi+GdFhjXyxx1eX6ImcHLd46TbnNqmK+CvH+RSBERERERaV2Xn/rVWiGtdCfsp35KWI1h/tCmpMKTOfXn8a8c7K4NUnzrXiBwkUgREREREQmsywcq0HIhLf8AJDMBqr3mgvqTLlhyqD6o8Q94NpXCqdqRGF9xx0BFIkVERKA+Tb4qz4uI1OvyU79a4wtAJvWBgYlw1g29bGYGsPdOmamNC6rq9334Qtg4HL7SHWpLsSjbl4iINMuXJv+9U3CgvOm9RUSkq1KgEgRfADI4CVIcZqFHaH7tiX9wk9O9fh2Lno6JiEhj/mnyQesaRUR8NPUrBP6pjH2aW3vS0nQyERERn1DuLSIiXYlGVELQOJUxaO2JiIi0j+4tIiKBKVAJQWupjEVEREKle4uISGAKVEKgtSciItFv9erVDB8+nPT0dCZOnMiOHTta3H/fvn1885vfJCMjgyFDhrB8+XIMo+EQx/vvv8/EiRNJT09nxIgRrFmzJmzt1b1FRCQwrVEJkdaeiIhErw0bNrB48WKeeOIJxo0bx+rVq8nNzWXnzp0MGDCgyf5nz55l2rRpjB8/nnfffZf8/HwWLlxIt27duPPOOwE4cuQIN9xwAzfddBPPPfccO3fu5O677yYlJYXrrrsuLO3WvUVEpCmNqIiISNx45plnmDVrFrNnzyYnJ4e8vDzS09ObHQH53e9+R2VlJStXrmTo0KFcd9113HXXXaxYsaJuVOWFF14gIyODvLw8cnJymD17NjNnzuTXv/51JLsmItLlKFAREZG44HK52LNnD5MnT26wffLkyezatSvgMR9++CGXX345SUlJdduuuuoqvvzyS44ePVq3T+NzXnXVVXz00UfU1NSEuRciIuIT14FKdnZ2Zzeh03TlvoP6r/6r/11RaWkpHo+H1NTUBttTU1MpLi4OeExxcXHA/X2vtbSP2+2mtLS02fZ01d+Dj/qv/ndlXbn/4ex7XAcqIiLS9VgsDYuSGIbRZFtr+zfeHsw+IiISXgpUREQkLqSkpGCz2ZqMnpSUlDQZEfFJS0sLuD/Uj6w0t4/dbqdPnz7har6IiDSiQEVEROKC0+lk5MiRbNmypcH2LVu2MHbs2IDHjBkzhg8++ICqqqoG+/fr149BgwbV7fPee+81Oecll1yCw+EIbydERKSOAhUREYkbCxcuZN26daxdu5YDBw5wzz33UFhYyJw5cwBYunQpU6dOrdv/O9/5DklJSSxYsIBPP/2UN998k1/+8pcsWLCgblrXnDlzKCgoYPHixRw4cIC1a9eybt067rjjjk7po4hIV6E6KiIiEjeuv/56Tp06RV5eHkVFRQwZMoT169czcOBAAAoLCzl8+HDd/r1792bjxo0sWrSIK6+8kuTkZBYuXNggCBk8eDDr16/nvvvuY82aNWRkZLB8+fKw1VAREZHALKdPnzZa301ERERERCRyYnbq1+rVqxk+fDjp6elMnDiRHTt2tLj/vn37+OY3v0lGRgZDhgxh+fLldVlbYlEo/d+2bRszZ84kJyeHfv36MX78eF5++eUItjb8Qv39+xw6dIj+/fuTlZXVwS3sWKH23zAMVqxYwWWXXUZaWho5OTk89NBDkWlsBwi1/5s3b+bqq6+mf//+XHDBBcycOZPPPvssQq0Nn+3btzNjxgyGDBlCcnIyr776aqvHxNu1L5roPqT7UFe9D+ke1DXvQRD5+1BMBiobNmxg8eLF3H333WzdupUxY8aQm5vLsWPHAu5/9uxZpk2bRlpaGu+++y6PP/44Tz/9dMxWFQ61/x9++CHDhg3jpZde4oMPPmDu3Ln86Ec/4ne/+12EWx4eofbfx+VyceuttzJ+/PgItbRjtKX/P/vZz/jNb37DQw89xIcffsj69etj9ucQav+PHDnCrFmzuPzyy9m6dSt/+MMfqKqqIjc3N8Itb7/y8nKGDh3K448/3qBAYXPi7doXTXQf0n2oq96HdA/quvcgiPx9KCanfl111VUMGzaMp556qm7bqFGjuO6663jwwQeb7O/7x3Hw4MG6H2peXh5r1qzh008/jbk8+KH2P5BbbrkFj8cTk0+02tr/e++9lzNnzjBhwgR++tOfcuLEiUg0N+xC7X9+fj6XX34527dvJycnJ5JN7RCh9v+Pf/wjc+bM4eTJk9hsNgC2bt3K1KlTOXToECkpKRFrezhlZWXx85//nJtuuqnZfeLt2hdNdB/Sfair3od0D9I9yCcS96GYG1FxuVzs2bOHyZMnN9g+efJkdu3aFfCYDz/8kMsvv7xB5HfVVVfx5ZdfcvTo0Q5tb7i1pf+BlJWVkZycHObWdby29n/Tpk1s2rSJ5cuXd3QTO1Rb+v+Xv/yFwYMH88477zBixAguvvhi5s+fz8mTJyPR5LBqS/9HjhyJw+Fg7dq1eDweysrK+O1vf8uoUaNi+gYRjHi69kUT3Yd0H+qq9yHdg3QPClV7r30xF6iUlpbi8XiaFO9KTU1tUpDLp7i4OOD+vtdiSVv639jbb7/N3//+d2655ZYOaGHHakv/CwsLueuuu1i1ahU9e/aMRDM7TFv6f+TIEY4dO8aGDRtYsWIFq1atIj8/nxkzZuD1eiPR7LBpS/8HDRrExo0bWbZsGWlpaQwcOJBPP/2U119/PRJN7lTxdO2LJroP6T7UVe9DugfpHhSq9l77Yi5Q8Wk8VGQYRovDR4H2D7Q9VoTaf5+dO3dy2223sXz5ckaPHt1RzetwofT/+9//PrfeeiuXXXZZJJoWEaH03+v1Ul1dzapVq5gwYQLjx49n1apV7N69m3/+85+RaG7YhdL/oqIi7rzzTmbMmMG7777Ln/70J3r06MEtt9wSczfJtoi3a1800X1I9yF/Xek+pHuQ7kGhaM+1L+YClZSUFGw2W5MorKSkpEnE5pOWlhZwf6DZY6JVW/rv88EHH5Cbm8u9997L3LlzO7KZHaYt/d+6dSvLly8nJSWFlJQU7rzzTsrLy0lJSeHFF1+MQKvDpy39T09Px263c9FFF9Vtu/DCC7Hb7Rw/frxD2xtuben/888/T7du3Xj44YcZMWIEEyZM4LnnnmP79u0hTVOJRfF07Ysmug/pPtRV70O6B+keFKr2XvtiLlBxOp2MHDmSLVu2NNi+ZcsWxo4dG/CYMWPG8MEHH1BVVdVg/379+jFo0KAObW+4taX/YKaTy83N5ac//SkLFizo6GZ2mLb0f8eOHWzbtq3uv/vuu4+kpCS2bdvGt7/97Qi0Onza0v9x48bhdrsbFLk7cuQIbrebAQMGdGh7w60t/a+srKxbwOjj+z7en2bF07Uvmug+pPtQV70P6R6ke1Co2nvti7lABWDhwoWsW7eOtWvXcuDAAe655x4KCwuZM2cOAEuXLmXq1Kl1+3/nO98hKSmJBQsW8Omnn/Lmm2/yy1/+kgULFsTkkHuo/d+2bRu5ubnMmTOHG264gaKiIoqKiuoi2lgTav+HDh3a4L9+/fphtVoZOnRoTC7kDLX/kyZNYsSIESxcuJCPP/6Yjz/+mIULF3LppZdyySWXdFY32izU/l9zzTV8/PHHPP744xw6dIg9e/awcOFC+vfvz8iRIzupF21z7tw59u7dy969e/F6vRw/fpy9e/fWpcWM92tfNNF9SPehrnof0j2o696DIPL3IXuH9aQDXX/99Zw6dYq8vDyKiooYMmQI69evZ+DAgYC5aM0/cu/duzcbN25k0aJFXHnllSQnJ7Nw4ULuuOOOzupCu4Ta/3Xr1lFRUcHTTz/N008/Xbd9wIABfPLJJxFvf3uF2v94E2r/rVYrr7/+Ovfccw/f+ta3SExM5Morr+R//ud/sFpj71lFqP2fOHEiq1ev5le/+hVPP/00iYmJXHrppbzxxht07969s7rRJh999BFTpkyp+37ZsmUsW7aMmTNnsnLlyri/9kUT3Yd0H+qq9yHdg7ruPQgifx+KyToqIiIiIiIS32IvlBURERERkbinQEVERERERKKOAhUREREREYk6ClRERERERCTqKFAREREREZGoo0BFRERERESijgIVERERERGJOgpURDrQZ599RnJyMn/72986uykiItIF6T4ksUyBikgH+vjjjwEYNWpUJ7dERES6It2HJJYpUBHpQHv27GHgwIGkpKR0dlNERKQL0n1IYpkCFZEOtGfPHkaNGsWLL77I2LFjyczM5Oqrr2bPnj2d3TQREekCdB+SWGY5ffq00dmNEIlXgwYNolu3bgwfPpy5c+dSUVHBgw8+iMvl4h//+Afdu3fv7CaKiEgc031IYpm9sxsgEq8OHz7MmTNnuPTSS3nttdewWCwA9OjRg+985zvs2rWLyZMnd3IrRUQkXuk+JLFOU79EOohvWP2BBx6ouzkADB06FICSkpLOaJaIiHQRug9JrFOgItJBPv74Y84//3xGjhzZYHtRUREAmZmZndAqERHpKnQfklinQEWkg+zZs4eMjIwm2zds2ECvXr0YM2ZMJ7RKRES6Ct2HJNZpjYpIB/n4449JTEzE7XZjt5v/1E6cOMELL7zAwoULcTqdndxCERGJZ7oPSaxT1i+RDnD06FFGjBjBoEGDGDt2LDfddBMnTpzg8ccfJzMzkzfffFM3CBER6TC6D0k80NQvkQ7gW8D4+uuvU1lZyYwZM1i6dClTpkzh97//vW4OIiLSoXQfknigERUREREREYk6GlEREREREZGoo0BFRERERESijgIVERERERGJOgpUREREREQk6ihQERERERGRqKNARUREREREoo4CFRERERERiToKVEREREREJOooUBERERERkaijQEVERERERKKOAhUREREREYk6cR2o5Ofnd3YTwk59ig3x1qd46w+oTyIiItEurgMVERERERGJTQpUREREREQk6tg7uwEiIl2JpaAA57PPYi0qwpuejmv+fIzMzM5uloiISNRRoCIiEiGWggK63XYblJWB1Yp1/37su3dT8fzzClZEREQa0dQvEZF2shQUkLBkCUm3307CkiVYCgoC7ud89tm6IAUwv5aVmdtFRESkAY2oiIi0QyijJNaiovogpW6j1dwuIiIiDWhERUSkHUIZJfGmp4PX22ij19wuIiIiDShQERFph1BGSVzz50PPnvXBitcLPXua20VERKQBTf0SEWkHb3o61v37GwYrzYySGJmZVDz/fFizfvlnEctyOrHce68W5ouISFxQoCIi0g6u+fOx795dP/2rlVESIzOT6ocfDst7N14f06uykoTbblMWMRERiQua+iUi0g6+URL3pEl4c3JwT5oUsUBBWcRERCSeaURFRKSdwjlK4q+14pDKIiYiIvFMgYqISBQKJu1xKOtjREREYo2mfomIRKFgpnUpi5iIiMSziAcq27dvZ8aMGQwZMoTk5GReffXVVo/Zt28f3/zmN8nIyGDIkCEsX74cwzAi0FoRkc4RcFqX24190yaSbr+dhCVLABqsjzk7ZowW0ouISNyI+NSv8vJyhg4dysyZM5kfxFO/s2fPMm3aNMaPH8+7775Lfn4+CxcupFu3btx5550RaLGIRLvW1nLEoibTulwurEePQrducOBAg6lgvvUxJ/LzyY7xfouIiPhEPFC55ppruOaaawBYsGBBq/v/7ne/o7KykpUrV5KUlMTQoUM5ePAgK1as4I477sBisXR0k0UkigWzliNS7QhnsOSf9tji8WA5fBiLy4WRmIilpgbD4aibCtYRC/lFREQ6W9SvUfnwww+5/PLLSUpKqtt21VVX8eWXX3L06NFObJmIRINoSNHrC5bs772H9cAB7O+9R7fbbsNSUNDmc/rSHnsuvRRLcbG5/sThwFJRgfXIESw1NcrwJSIicS3qs34VFxeT2eipZGpqat1rgwcPDnhcfn5+g6/xRH2KDfHWp2jtz+D8fJJqappsr/zsM4600uZw9SnrqaewlZQ0XFNSUkL5smWc+OEPAXAUF5P22ms4Skqo6duX4hkzqElLa/3cNTX06tMHh8eDrbzc3Oh24/nyS2pSUznrdHLCrx+h9ik7Ozuk/UVERCIl6gMVoMn0Lt9C+pamfWVnZ5Ofnx93N2H1KTbEW5+iuT8J2dnYT5xokqLXdtFFLba5pT6FOo0ryeXC6jfq65PictEtO9sccXnssfqRnxMnSP3886Cmp9WdOyMDy9GjWDwesFqxeL1Y+/al+7331q1Liebfk4iISKiifupXWloaxcXFDbaVlJQA9SMrItJ1hTtFb1umcXnT0+vfv25jfT2T9kxPqzu304l30CC8PXtiJCTgueACZfgSEZG4FvWBypgxY/jggw+oqqqq27Zlyxb69evHoEGDOrFlIhINfGs5fCl63ZMmtesDfFuCitaCpfZUkG9wbqcTo18/vMOGUfnKKwpSREQkrkV86te5c+f4/PPPAfB6vRw/fpy9e/dy3nnnMWDAAJYuXcru3bt58803AfjOd77D8uXLWbBgAYsWLeKzzz7jl7/8JT/96U+V8UtEADNYCVfmq7YEFb5gqbnpYu2pIN/auUVEROJVxAOVjz76iClTptR9v2zZMpYtW8bMmTNZuXIlhYWFHD58uO713r17s3HjRhYtWsSVV15JcnIyCxcu5I477oh000UkTvmvSbHk50NVFSQm1u8QRFDROFiyFBSQsGSJGVx07w4JCVBdbQYrIU5PC2cgJiIiEisiHqhcccUVnD59utnXV65c2WTbsGHD+Otf/9qBrRKRrqpxHRZLdTWWggK8mZlmsNKGNS9NarvUTtvyXHoplvJyjYqIiIgEISayfomIdJTGa1KMhATIysJISsLIzm5TUBFwnYvLhdGjB1VPPtnq8eEuHikiIhKLFKiISJdlKSjAsWmTWVDR4cBITcVwODASEjCys6lctapNQUN7Fs83GY3Zvx/77t3K8CUiIl1O1Gf9EhHpCI7iYjMgOHUKS2UllrKy+orvtWtS2lpxvrV0xS1pTypjERGReKJARUS6pLTXXoOyMnMUxWYDwwCv1xxdqV2T0tagoT21XdozGiMiIhJPFKiISJfkKCkxAwL/QoqJiXj79KmbZtXWoKE9tV3aMxojIiIST7RGRUTCJpYWgdf07QsnTtQFK0ZWFobXi3vSpLDVPwmUUri1n5Fr/nzsu3fXj+S0IeuYiIhIPFCgIiJhEWuLwItnzCD1889bDAjCGTRYCgpw/uIXON96CwAjLS3gz0gFHkVEREwKVEQkLFpazxGNxQpr0tJaDQjCFTT4gjjLwYNYysrAYsFSUYF38GCMAD8jFXgUERFRoCIiYRKLi8CDCQjCETT4gjiL2w0Wi7nR68Vy8mT9WhgRERFpQIGKiIRFe9ZzdAZHcTEJL78ckelVviDOsNuxGEZ9sOKXCllEREQaUtYvEQmL9qTkjTRLQQHnP/BAyPVR2sqXyatBKmQAuz1qf0YiIiKdTYGKiIRFe1LydiRLQQEJS5aQdPvtJCxZUpd1y1ZeHrGiinVBnN1enwq5Rw9qpkyJip+RiIhINNLULxEJm2hbBN5cJjJvSgqeMK6naS3lsDJ5iYiIhE6BiojEreYykVkqK5sUVbRUV2PJzyfp9ttDCiSCTcscbUGciIhItNPULxGJW81lIjOysvB0714XrFiqq7GcOIGloiLkNSstpWUWERGRtlOgIiJxy7eIveFGL97zz+fwI4/UrafxJiXhzcyExERznxCCjVhMyywiIhILFKiISNxqKRNZTVoa1Q8/TOWqVRjZ2fVBik+QwUazwZBSDouIiLSLAhURiVvBZiJrT7ARS2mZRUREYokW04tIXAtmEbtr/nzsu3fXrzUJIdhQRi8REZGOoUBFRLq89gYbyuglIiISfgpURERQsCEiIhJtFKiISFxqrQijiIiIRLdOWUy/evVqhg8fTnp6OhMnTmTHjh0t7r9582auvvpq+vfvzwUXXMDMmTP57LPPItRaEYk1viKM9vfeC7kuioiIiESHiAcqGzZsYPHixdx9991s3bqVMWPGkJuby7FjxwLuf+TIEWbNmsXll1/O1q1b+cMf/kBVVRW5ubkRbrmIdCZLQQEJS5aQdPvtJCxZ0mLQEagIo+XUKZJuvrnueEdxcYRaLiIiIm0R8alfzzzzDLNmzWL27NkA5OXlsXnzZtasWcODDz7YZP+PP/6YmpoaHnzwQWw2GwA//vGPmTp1KqWlpaSkpES0/SKxKNanQflGSHzBh3X/fuy7dwdMNQxNizBaamqwfvEFFocDr9eLdf9+zn//fXjllZj6OYiIiHQlER1Rcblc7Nmzh8mTJzfYPnnyZHbt2hXwmJEjR+JwOFi7di0ej4eysjJ++9vfMmrUKAUpIkGIh2lQgUZIWqoc37guiuXkSfB4MByOuuNt5eVBVZ4XERGRzhHREZXS0lI8Hg+pqakNtqemplLczDSMQYMGsXHjRm655RYWLVqE1+tl+PDhvPHGGy2+V35+foOv8UR9ig3R0qesp57CVlLSYISBkhLKly3jxA9/GPR52tIfR3Exaa+9hqOkhJq+fSmeMYOatLSQzzM4P5+kmpom2ys/+4wjAdrluPZazn//fWzl5WC14qysxGq1Ut27N0Z1tbmT1cq5Zo6PZaH+nrKzszuoJSIiIu3TKVm/LBZLg+8Nw2iyzaeoqIg777yTGTNmMH36dM6dO8djjz3GLbfcwltvvYXVGnhQKDs7m/z8/Li7CatPsSGa+pTkcmFNSmqyPcXloluQbWxLfywFBXR77LH6kZATJ0j9/PNmp2u1JCE7G/uJEw2DLa8X20UXBW5Xdja88gq22uluJCVhVFTgTEys26W6spIezR0fo6Lp705ERKS9Ijr1KyUlBZvN1mT0pKSkpMkoi8/zzz9Pt27dePjhhxkxYgQTJkzgueeeY/v27c1OFxOReo2nQZkbveb2DhTqdK2WuObPh5496/sRROV4X12UylWrqHzlFUhJaXC8p3v3oCrPi4iISOeIaKDidDoZOXIkW7ZsabB9y5YtjB07NuAxlZWVdYvofXzfext/+BKRJtryIT8cGi9oNzdaze0h8lWOd0+ahDcnB/ekSVQ++ijOZ58NKgtYoOMPP/KIFtKLiIhEsYhP/Vq4cCG33347o0ePZuzYsaxZs4bCwkLmzJkDwNKlS9m9ezdvvvkmANdccw0rVqzg8ccfJzc3l7KyMh555BH69+/PyJEjI918kZjj+5Ae6axf3vR0rPv3N5mu1daRHP/K8daPPqLbzTdjKS/HcDiwpqS0mAWs8fEANXG2NkVERCTeRDxQuf766zl16hR5eXkUFRUxZMgQ1q9fz8CBAwEoLCzk8OHDdftPnDiR1atX86tf/Yqnn36axMRELr30Ut544w26d+8e6eaLxKTGH9IjwTV/Pvbdu+unf7UwktM4fXLNtGk4Nm4MGFhZCgrodtNNWE+dAosFS1UVRkUFXszpZpHup4iIiHSMTllMP2/ePObNmxfwtZUrVzbZNn36dKZPn97RzRKRMAp2JKdxjRTb3r04X3wRb2YmJCY2qZnifPZZLBUV4EvAYbFg8XiwlJa2aVqZiIiIRKdOCVREpGsIZiSn8aJ7S2kpFpcLS2kpRlZWg0X41Q8/bAYjTif40gyDGazU1HR4ggARERGJHAUqIlKnoyrYt3TeJovua2rMwMPtxvBt81uE701Px0hJwVJeXp8gwDAwlMVLREQkrihQERGg6RQs/ylXHXVeIzOz6aJ7hwMqKzHsfpcnv0X4vrUvXszRF1wujG7dqHjlFWXxEhERiSMRTU8sItErnHVPQjlv4/TJRkoKhtOJkZJi7t9oEb5v7UvN1VfjvvJKXN/7HuXvvIP3kkva1U4RERGJLhpREREgvHVPQjlvoEX3LWX98h2j7F4iIiLxTYGKiADhr3sSynkDBR7VGiERERHp0jT1S0SAjqtg31HnFRERkfimQEVEgPopWO5Jk/Dm5OCeNKnFSu+dfV4RERGJb5r6JSJ1Omrth9aUiIiISKgUqIh0cR1VO6Uz2xWtfRIREZHgKVAR6cJaq3Hi2yfrqadIcrmC/tDf3kAhmHZ1xLEiIiISPbRGRaQLa63Gie9Df68PP8R64AD2996j2223YSkoaPacvmPs770X9DGhtqujjhUREZHooUBFpAtrrcZJWz70hyNQaE9Nl46qByMiIiKRpUBFpAvzpqfXpw2u21hf46QtH/rDESi01q6OOlZERESihwIVkS6stRonbfnQH45AoT21V1S3RUREJD4oUBHpwlqrcdKWD/3hCBTaU3tFdVtERETig7J+iXRxLdU48X3oL1+2jJQgs375jmlveuD21F5R3RYREZHYp0BFRFpkZGZy4oc/pFt2dkjHKFAQERGR9lCgItJFRUNRxGhog4iIiEQnBSoicSaYD//RUBQxGtogIiIi0UuL6UXiSLDFFqOhKGI0tEFERESilwIVkTji/MUvsBw8iPXYMSwnToDbHfDDfzQURYyGNoiIiEj0UqAiEicsBQU433oLa1kZlqoqrGVlWI8eBbe7yYf/aCiKGA1tEBERkejVKYHK6tWrGT58OOnp6UycOJEdO3a0uL9hGKxYsYLLLruMtLQ0cnJyeOihhyLTWJEYUTdqYrHUfbV4PFhOnmzy4T8aiiJGQxtEREQkekV8Mf2GDRtYvHgxTzzxBOPGjWP16tXk5uayc+dOBgwYEPCYn/3sZ2zatImHH36YYcOGcebMGYo0PUSkAWtREUZaGpaKivoP/xYLFmjy4T9ctU7aIxraICIiItEr4oHKM888w6xZs5g9ezYAeXl5bN68mTVr1vDggw822T8/P5/nnnuO7du3k5OTE+nmisQMb3o61v378Q4ejOXkSaipAbudmilTAn74j4ZaJ9HQBhEREYlOQQUqHo+Hp556ip07d9KjRw+GDBnC8OHDGTFiBOkhzCd3uVzs2bOHO++8s8H2yZMns2vXroDH/OUvf2Hw4MG888473HDDDXi9XiZMmMAjjzxCampq0O8tEu9c8+dj370bo6zMDExqp1JVL1rU2U0TERERCVlQgcpDDz3ECy+8wNe+9jU2bNiAxTcHHkhNTWXEiBGsX7++1fOUlpbi8XiaBBipqakUFxcHPObIkSMcO3aMDRs2sGLFCiwWCw888AAzZszgb3/7G9bGWYNq5efnN/gaT9Sn2NAZfXLcdx9pr72Go7SUmpQUimfMgMOHSVu2DEdJCTV9+1I8YwY1aWkhn1u/o9gQap+ys7M7qCUiIiLtE1SgsnHjRn72s5/x/e9/n759+/KnP/2JkydP8uijj1JTU4PH4wnpTf0DHTAXyzfe5uP1eqmurmbVqlVcdNFFAKxatYpLL72Uf/7zn1x66aUBj8vOziY/Pz/ubsLqU2zotD5lZ8OECQA4gPMbFVXkxAlSP/885KKK+h3Fhnjsk4iIdF1BZf0qKSlhxIgRdcFEYmIi1113He+99x7dunVjwYIFQb1ZSkoKNputyehJSUlJs9O40tPTsdvtdUEKwIUXXojdbuf48eNBva9IV2IpKCBhyRKSbr+dpJtvhtLS+nolbjeWgwfpPm0aCUuWNCkEKSIiIhItggpU+vTpQ0VFBVarlT59+lBaWgpA9+7d+eEPf8jy5cuDejOn08nIkSPZsmVLg+1btmxh7NixAY8ZN24cbrebw4cP1207cuQIbre72SxhIpHiHxREwwf/xpXpbZ9/jvXYMXC5wOXCevSoWWeluLjZqvUiIiIi0SCoQOXiiy+um/d84YUXNlj4npGRwb/+9a+g33DhwoWsW7eOtWvXcuDAAe655x4KCwuZM2cOAEuXLmXq1Kl1+0+aNIkRI0awcOFCPv74Yz7++GMWLlzIpZdeyiWXXBL0+4qEW+OgIBo++DuffbZ+mheA01lXS8Vy8iQWj8ess+JwmPsEqFovIiIiEg2CClR+8IMf4Ha7AZg9ezbPPfccGzZs4OOPP+bJJ58kIyMj6De8/vrrWbZsGXl5eVxxxRXs3LmT9evXM3DgQAAKCwsbjJ5YrVZef/11UlNT+da3vsX06dPJyspi3bp1zS6kF4mEJkFBFHzwtxYV1bcHMHr3Nqd7nT6N5exZMAywWjF8Uy2t1iZV60VERESiQVCL6SdNmsSkSZMA6rJtzZ07F4vFgsPh4Pnnnw/pTefNm8e8efMCvrZy5com2zIyMnjppZdCeg+RjtY4KDA3tu2Dv6WgICyFD321VLBaweUyR3dsNrDZsLjdYBh4MzMxHI7aA7xNqtaLiIiIRIOQCz7abDZefPFFPvnkE44fP87w4cPJysrqiLaJRLUGQUHdxtA/+FsaZeay7t+PfffukDNzQX0tFcrK6qd62e0YgwdjANbDh+H0aejeva7OSuOq9SIiIiLRoM1zpy6++GK+8Y1vKEiRLss1fz707Gl+4IcmH/yDXWgfzilkRmYmFc8/j3vSJOjWDaNXL7yDB2M4HBgOB97zz4c+ffDm5OCeNKlNwZCIiIhIJIQ8oiISb9o67coXFAQ61rF9O90eeyyoUZJwTiHztav64YcBsL/3XsM1KzYb7quuqntdREREJFopUJEurb3TrvyDAn9pr70WcJQk4Re/wOjRo0Fg45tC5svORU0N2O14Ro0KuS/+QVPNtGl108CwWjXVS0RERGKKAhXp0lqadtWeUQdHSUmTURKLx4Pjrbfw9uvXICiqfPRR7B98gHXfPjMrl2Fg2GzY9u7FUlAQVMDUXMBV+eijODZubPcifREREZFIU6AiXVq4p1351PTtCydONDi3pbgYo/b8vvehrAzHxo24L74Yy5EjWNxuDLvdTB/scgUdMDUXcDk2btQ0LxEREYlJKkQiXZo3Pb1+MXzdxvan7C2eMaPpQnuor1/iUxsUWcvLMbKy8A4ahJGVBU5nSAFTRwVcIiIiIp1FIyrSpfmn8w3nOo6atLQmC+05dw77P/7RcEe/oCjYVMeBFv+HK1WyiIiISLRQoCJdWkuZu8Jxbv9pV5aCAux+60gaB0WBAqaaadNIWLKkwQL5pPvvD7gWRQvnRUREJJ4oUJG4E2q64eYyd4Vba0FR49cCBSWO3/8eevXCSEgwT+q3FqWjAi4RERGRzqBAReJCXXBy5Ai23bsxevaExMRW0w23tYZKW7UUFDV+LWHJkgYL5C0eD5aSEiwlJRi9e2OkpmI4HHVrUSIVcImIiIhEggIViXn+qXmthYVYzp7FOH0a76BB5qL0ZtINt7eGSkfzXyBvqanBeuQIuN1gGFjKyrCUl5tV5202rUURERGRuKOsXxLzGqTmrakBi6W+eCI0m/2qpRoq0cA/I5nl5Enz/202DJvNrLfi9WIpLq5bi2IpKCBhyRKSbr+dhCVLsBQUdHIPRERERNpOIyoS8xqk5nU4oKrKDFbcbrNuSTPZr6I9pW+DjGQ1NWYhSIcDb79+WM6cweJ24+3Th4rnnweI6tEhERERkVBpREVinv/Ig5GaagYfhoFht7eY/aqjaqiEi2/xvXvSJIy0NLw9e5rT2Xr0MGuuDBiA+9prMTIzo350SERERCRUClQk5rnmz68rrmg4HHgHDsTbpw+eceNwT5rU7KiC/3FAVKb09S2QL9+4EeMrXwF77SCoX1stBQU4Nm3CeuwY1oICLDU15j5RNDokIiIiEipN/ZKY19ZaKB1ZQyXcmmsr1E75OnUKS2UlVFVpkb2IiIjEBQUqEhfampo3llL6BmqrL4WxkZqKUVGBxeOpW2RvfOUrUTU6JCIiIhIKTf0SiWF1CQGcTryDBuHt2RMjMbFukX00jg6JiIiIBEOBikgMa5AQwOlsssheREREJFYpUBGJYbGQEEBERESkLbRGRSSGxVJCABEREZFQdMqIyurVqxk+fDjp6elMnDiRHTt2BHXcoUOH6N+/P1lZWR3cQpHY4VtkX7lqFdUPP6wgRUREROJCxAOVDRs2sHjxYu6++262bt3KmDFjyM3N5dixYy0e53K5uPXWWxk/fnyEWiqhshQUkLBkCUm3307CkiVYCgo6u0kiIiIiEqMiHqg888wzzJo1i9mzZ5OTk0NeXh7p6emsWbOmxeMefPBBhg0bxnXXXRehlkooLAUFdLvtNuzvvYf1wAHs771Ht9tuU7AiIiIiIm0S0TUqLpeLPXv2cOeddzbYPnnyZHbt2tXscZs2bWLTpk38/e9/58033+zoZkobOJ99FsrKzFS5YH4tK8P57LO45s+vW0NhdO+OAVjLyxsULRQRERER8RfRQKW0tBSPx0NqamqD7ampqRQXFwc8prCwkLvuuouXX36Znj17Bv1e+fn5Db7Gk2js0+D8fJJqappsr/7Xv7DffDOe8nK8Hg8JJ06Y27OyMGw2eP99HI88QvT1qP2i8ffUHvHWH1CfALKzszuoJSIiIu3TKVm/LBZLg+8Nw2iyzef73/8+t956K5dddllI75GdnU1+fn7c3YSjtU8J2dnYT5yoH1EB8Hpxnj6Nxe2GpCSsBQVYAAyDhDNnMLKywO0m7bXX6Pb0053V9A4Rrb+ntoq3/oD6JCIiEu0iukYlJSUFm83WZPSkpKSkySiLz9atW1m+fDkpKSmkpKRw5513Ul5eTkpKCi+++GIEWi3BaK6eh5GVVR+8+EZcLBYzeAGwWnGUlka+wSIiIiIS1SI6ouJ0Ohk5ciRbtmzh29/+dt32LVu2MHXq1IDHNE5d/Je//IUnnniCzZs3k6k0rFGjuXoezmefxXr0qBmsOBxQVQWGgWGv/dPzeqlJScHRuc0XERERkSgT8alfCxcu5Pbbb2f06NGMHTuWNWvWUFhYyJw5cwBYunQpu3fvrls0P3To0AbHf/TRR1it1ibbpfP56nn4c82fj333bigrw0hNxXLuHIbVipGaWjfqUjxjBoM7p8kiIiIiEqUiHqhcf/31nDp1iry8PIqKihgyZAjr169n4MCBgLl4/vDhw5FulnSQxiMt7rFjm2T9qikv7+xmNstSUKCq7yIiIiKdwHL69GmjsxvRUeJxYan6FDm+2jB1aZdrR4Aqnn++1WAlWvvUVvHWH1CfREREol3ECz6KdCRLQQEJS5aQdPvtJCxZ0q6Cky3VhhERERGRjtUp6YlFOkLjERDr/v3Yd+zAM3w4Fr+pZsFO3bIWFTVMtwzmeYuKOqD1IiIiIuJPgYrEDecvfoHl4EEsbjeG3Y7RuzfWQ4ewHj2KNyvLDFx27w5q6haANz0d6/79TWrDeNPTO7AXIiIiIgKa+iXtEM5pVuFoi/Ott7CWlWGpqsJaVob16FEsNTXgV7MllKlbzdWGcc2f30G9EBEREREfjahISOqyYB05gm33boyePcFqxfbuuzh/+1tqpkyhetGiiGfGqgs+LJa6rxaPx/ze4VelJYSpW83VhlHWLxEREZGOp0BFgua/BsRaWIjl7FmM//wHDAOLYYDFguOtt7AdOBD09KpwsRYVYaSlYamoqB8BsVrN4pKpqfU7hjh1K1BtGBERERHpeJr6JUFrkAWrpsYctaiuxuJy1Y9kuN2dkhnLm56OYbPhHTwYo2dPjMREjF698KamYthstTtp6paIiIhIrNCIigStQRYshwOqqsz/N/xK8TgcnZIZyzV/PvbduzHKysyRnNqgpPLRR3Fs3KipWyIiIiIxRoGKBM0/C5aRmorFV1HeN5pSu70zMmO1tJ6k+pJLItoWEREREWk/BSpdQN0C+HaOKvhGLSgrw3A48A4cCP/5jzn1y2bDSEszp1l10vSqYNeThOvnISIiIiIdR4FKnGtQBNHtbpCdy/Xd74Y0Laq5UQsgZj74BywKGUJtFRERERGJDAUqca5uAbzbbdYVqU3Z6/jDH3C88QZGVhZGQkLQH9ibG7WIlcxYDRICQIPaKrHSBxEREZGuQFm/4pxvAbzl5Mn6uiIA5eVYXC4spaW1OwZfDDGaCj2GqkFCgLqNkV/8LyIiIiItU6AS57zp6eD1YnG764MUH4vFTDPsE8QHdt/UKft772E9cAD7e+/R7bbbYiZY8f08Gm6M/OJ/EREREWmZApU455o/H3r2xLDb69MIW63Qo4f5vX/V9iA+sLc0dSoW+H4edcGKaquIiIiIRCUFKnHOtwC+ZsoUsxBijx54Bw82CyE6nRgpKeaOQX5gj/WpU76fh3vSJLw5ObgnTdJCehEREZEopMX0XYCRmUn1k0/iWrSoQXaummnTQi6G6F9LpX5j50+dCiXlcLBpjEVERESk8yhQiXHt/YAeajFE/1oqWK1RMXVKKYdFRERE4o+mfsWwzljYHo1Tp2J93YyIiIiINKURlRjWUTVBWhulibapU7G+bkZEREREmlKgEsM64gN6KNOoQpl21pGidd2MiIiIiLSdpn7FsI6oCRLsNKpwTztrTxFJpRwWERERiT+dEqisXr2a4cOHk56ezsSJE9mxY0ez+27bto2ZM2eSk5NDv379GD9+PC+//HIEWxu9OuIDerCjNOFcF5L06ad0//rXca5di33LFhx/+1tIQU80rpsRERERkfaJ+NSvDRs2sHjxYp544gnGjRvH6tWryc3NZefOnQwYMKDJ/h9++CHDhg3jrrvuIiMjg82bN/OjH/2IxMREcnNzI938qOL7gB7O6VfBTqMK17QzS0EBF/1//x/Ws2fBYoHqaizl5XghpLU20bZuRkRERETaJ+KByjPPPMOsWbOYPXs2AHl5eWzevJk1a9bw4IMPNtn/7rvvbvD93Llz2bZtG2+++WaXD1Qg/B/Qg00/HK51Ic5nn8VaWWkGKX7nsZSWajG8iIiISBcW0alfLpeLPXv2MHny5AbbJ0+ezK5du4I+T1lZGcnJyWFunUD9KI3n0kuxVFZCdTXunJwm+4Vr2pm1qAjD4QDDaPQGLi2GFxEREenCIjqiUlpaisfjITU1tcH21NRUiouLgzrH22+/zd///nc2bdrU4n75+fkNvsaT9vbJUVxMxm9+Q8+PPgKgbNQoCm+9lZq0tLrXz9+zBzeYIybbt8OePRx55JG6fQAc991H2muv4SgtpSYlheIZM6gpL4cQ2pfldNI7ORlbVRV4PObIiteLOyGBA9deS00M//7i7W8v3voD6hNAdnZ2B7VERESkfTolPbHFf5oPYBhGk22B7Ny5k9tuu43ly5czevToFvfNzs4mPz8/7m7C7e2TpaCAbg8+iHX/fiy1gUHSX/9KyhdfUPHSSxiZmSS8/DJ2txuSkuoPdLvJ2bSp4TSz7GyYMAEABzC4Le25917cN9+M0+nEUlqKpaYGo3t3XK+8wuBLLmlzPztbvP3txVt/QH0SERGJdhENVFJSUrDZbE1GT0pKSpqMsjT2wQcfcMMNN3Dvvfcyd+7cjmxmzGhLHRPns89iOXq0LkgBwDCwHjlSt3i9pYXy4a6dYmRmcuSRR8jZtKnT67GIiIiISPSIaKDidDoZOXIkW7Zs4dvf/nbd9i1btjB16tRmj9u+fTs33ngj99xzDwsWLIhASzteez/wh1KY0Z+1qAiLywVut7kuxDDAYsFy9izWI0eAFhbKd+/epvdsTU1amjJ2iYiIiEgDEa+jsnDhQtatW8fatWs5cOAA99xzD4WFhcyZMweApUuXNghatm3bRm5uLnPmzOGGG26gqKiIoqIiSkpKIt30sAlHscS21jExunfHUl5uBioej7kI3uOBmhpsu3djKShodqG8BcJWO0VEREREpCURD1Suv/56li1bRl5eHldccQU7d+5k/fr1DBw4EIDCwkIOHz5ct/+6deuoqKjg6aefJicnp+6/K6+8MtJND5twFEtsax0TA8wsW423WywYPXvifPbZZgsoWsrLA7/n4cNtriovIiIiIhJIpyymnzdvHvPmzQv42sqVK5t833hbrAtHscS21jGxlpfjPf98rPn5WGpqzI02G3TvDomJLa5DCfieVVXYdu/G9tlnWEpLweXC8cYbVLz6Kt4YXgwvIiIiIp0r4iMqYgYZddOq6jaGViyxrXVMvOnpYLfDeeeZWb2SksDhwEhIAK8Xo3YdSqBpaYHe01JWBt26Yf3iCyxlZViqq7GeOkW3m2/WyIqIiIiItJkClU4QjmKJzU3Pam1Ru++9jZQUc2TEMDBsNvP7nj0xoNlpaYHe0zN6NJazZ5sEXpaSErpNm6apYCIiIiLSJp0y9SvahDvlbmt8H/jb+55GZmbI2bIavPeRI1iOH8fo0wfLqVPQrRuOjRuxeDwYCQkYqangdDaYltb4PROWLIGtW/3ewIDqaixWK9biYuzvvReWzGAiIiIi0rV0+UClrWl+26stQUZHvLev/5bKSqw7dkBVlTmlq6oKo6IC76BBYLc3Oy3NNX8+jjfewFJVZdZl8Vv3YtjtDUZklIJYRERERILV5ad+hSMDV2ewFBSEJdOWr/+W0lJz+pYvI5jHg8XjwXLyZIvT0ozMTHPhfEqKOQpjt5trXhwOc0QGQk4UICIiIiLS5UdUwpGBK9LCOQpU13/fSIjFAgkJGIZhLrTv06fV83ovuYTyv/0N57PP4ti0CePkSbBYsH75JYbdjpGSElKiABERERGRLj+iEo4MXJEWzlGguv43qq1i9OqFd8AAaq69NqjgxzedrOLZZ7G43VjLy7FUVWEtK8NaUEDNtGkht01EREREuq4uH6iEIwNXpIVzFKi1LGCh/hwcGzdiZGVh9OqFkZhofs3KwrFxY8htExEREZGuq8tP/QpXBq5Iamuxx0ACZgHLysJ7/vlt+jlYi4rMtSqNjovmqXQiIiIiEn26fKACnZuBKxj+6ZOznE5qbr0V++7d9dO/2jkKFM7+hzOIEhEREZGuS4FKlGu8cL5XZSUJn39O5aOP4ti4MepGgVzz54c1iBIRERGRrkmBSpRrbuG8Y+PGqBwFisWpdCIiIiISfRSoRLlYTJ8c7VPpRERERCT6dfmsX9EuFtMni4iIiIi0lwKVKBeL6ZNFRERERNpLU7+iXOM1H2edTrrfe6/WfIiIiIhIXFOgEgP813ycyM8nW0GKiIiIiMQ5Tf0SEREREZGoo0BFRERERESijuX06dNGZzdCRERERETEn0ZUREREREQk6ihQERERERGRqKNARUREREREoo4CFRERERERiToKVEREREREJOrEdKCyevVqhg8fTnp6OhMnTmTHjh0t7r9v3z6++c1vkpGRwZAhQ1i+fDmGEV1Jz0Lp07Zt25g5cyY5OTn069eP8ePH8/LLL0ewtcEJ9ffkc+jQIfr3709WVlYHtzA0ofbHMAxWrFjBZZddRlpaGjk5OTz00EORaWyQQu3T5s2bufrqq+nfvz8XXHABM2fO5LPPPotQa1u3fft2ZsyYwZAhQ0hOTubVV19t9Zhovz6E2qdYuT6IiIg0J2YDlQ0bNrB48WLuvvtutm7dypgxY8jNzeXYsWMB9z979izTpk0jLS2Nd999l8cff5ynn36aX//61xFuefNC7dOHH37IsGHDeOmll/jggw+YO3cuP/rRj/jd734X4ZY3L9Q++bhcLm699VbGjx8foZYGpy39+dnPfsZvfvMbHnroIT788EPWr18fVf0KtU9Hjhxh1qxZXH755WzdupU//OEPVFVVkZubG+GWN6+8vJyhQ4fy+OOPk5SU1Or+sXB9CLVPsXB9EBERaUnM1lG56qqrGDZsGE899VTdtlGjRnHdddfx4IMPNtnf90Hx4MGDdTf5vLw81qxZw6efforFYolY25sTap8CueWWW/B4PFHz5LStfbr33ns5c+YMEyZM4Kc//SknTpyIRHNbFWp/8vPzufzyy9m+fTs5OTmRbGrQQu3TH//4R+bMmcPJkyex2WwAbN26lalTp3Lo0CFSUlIi1vZgZGVl8fOf/5ybbrqp2X1i4frgL5g+BRJt1wcREZGWxOSIisvlYs+ePUyePLnB9smTJ7Nr166Ax3z44YdcfvnlDZ5EXnXVVXz55ZccPXq0Q9sbjLb0KZCysjKSk5PD3Lq2aWufNm3axKZNm1i+fHlHNzEkbenPX/7yFwYPHsw777zDiBEjuPjii5k/fz4nT56MRJNb1ZY+jRw5EofDwdq1a/F4PJSVlfHb3/6WUaNGRV2QEqxovz6ESzRdH0RERFoTk4FKaWkpHo+H1NTUBttTU1MpLi4OeExxcXHA/X2vdba29Kmxt99+m7///e/ccsstHdDC0LWlT4WFhdx1112sWrWKnj17RqKZQWtLf44cOcKxY8fYsGEDK1asYNWqVeTn5zNjxgy8Xm8kmt2itvRp0P/f3v2DNrXGYRx/GqO0KigEIkHTDGaxDq0Y/7WDZnFQ1CJGdDMWHXLarVgUFxWJxUUJKlUHdShWlIq46NCKQevin0SoCIodChorIvgfSnsHSbDXVnvOvcl5j3w/kCGHE/g9ywvP2/OeRiLq7e1VOp1WMBhUbW2tBgcH1dPTU4mRy8L09eH/YNr6AADAn3iyqBT9+3GM8fHx3z6iMdn9k113k91MRQ8ePNCePXvU2dmp5cuXl2s8R+xk2rt3r3bv3q0VK1ZUYjRH7OQZGxvT9+/f1dXVpaamJjU2Nqqrq0sPHz7Uo0ePKjHutNjJVCgU1NbWph07dqivr083b97U3LlztWvXLiPKl1NeWB+cMnl9AABgKp4sKoFAQDNmzPhlp/Pdu3e/7IoWBYPBSe+XNOVvKslJpqKBgQElEgnt379fLS0t5RzTFieZ7t69q87OTgUCAQUCAbW1tenz588KBAK6cOFCBaaempM8CxYskN/vVzQaLV1bvHix/H6/hoeHyzrvdDjJdO7cOc2ePVuHDx9WfX29mpqadPbsWd27d8/WY4omMX19+C9MXR8AAPgTTxaVWbNmqaGhQf39/ROu9/f3a9WqVZP+ZuXKlRoYGNC3b98m3B8KhRSJRMo673Q4yST9eGVpIpHQvn37lEqlyj2mLU4y3b9/X9lstvQ5cOCAampqlM1m1dzcXIGpp+Ykz+rVqzU6OqpXr16Vrg0NDWl0dFThcLis806Hk0xfv34tHaIvKn736l9UTF8fnDJ5fQAA4E88WVQkybIsdXd369KlS3r+/Lk6Ojr05s0bJZNJSdKhQ4e0efPm0v3btm1TTU2NUqmUBgcHdePGDZ04cUKpVMqYRzvsZspms0okEkomk9q+fbsKhYIKhUJpJ9gEdjPV1dVN+IRCIfl8PtXV1RlxCNhunnXr1qm+vl6WZSmXyymXy8myLMViMS1btsytGBPYzbR+/XrlcjkdO3ZML1++1JMnT2RZlhYtWqSGhgaXUkz06dMn5fN55fN5jY2NaXh4WPl8vvTKZS+uD3YzeWF9AADgd/xuD+DU1q1b9f79ex0/flyFQkFLlizRlStXVFtbK+nHoeyfd7HnzZun3t5etbe3Kx6Pa/78+bIsS62trW5F+IXdTN3d3fry5YsymYwymUzpejgc1tOnTys+/2TsZjKd3Tw+n089PT3q6OjQxo0bVV1drXg8rqNHj8rnM2OfwG6mtWvX6vz58zp58qQymYyqq6sVi8V09epVzZkzx60YEzx+/FibNm0qfU+n00qn09q5c6fOnDnjyfXBbiYvrA8AAPyOZ/+PCgAAAIC/lxlbugAAAADwE4oKAAAAAONQVAAAAAAYh6ICAAAAwDgUFQAAAADGoagAAAAAMA5FBQAAAIBxKCoAAAAAjENRAQAAAGAcigpQAdevX9eGDRsUDoe1cOFCxeNx3bp1y+2xAAAAjOV3ewDgb3fw4EGdPn1ayWRSra2tqqqq0p07d/Tx40e3RwMAADBW1YcPH8bdHgL4W127dk0tLS26ePGitmzZ4vY4AAAAnkFRAcooFospGo3q8uXLbo8CAADgKZxRAcpkaGhIL168UHNzs9ujAAAAeA5FBSiT169fS5JCoZDLkwAAAHgPRQUok2JBefbsmcuTAAAAeA9v/QLKJBKJaM2aNUqn05KkpUuXamRkRLdv31Z7e7ui0ajLEwIAAJiLw/RAGY2MjOjIkSPq6+vT27dvFQwG1djYqFOnTmnmzJlujwcAAGAsigoAAAAA43BGBQAAAIBxKCoAAAAAjENRAQAAAGAcigoAAAAA41BUAAAAABiHogIAAADAOBQVAAAAAMahqAAAAAAwDkUFAAAAgHH+AUeJAFBdumFfAAAAAElFTkSuQmCC\n", 100 | "text/plain": [ 101 | "
" 102 | ] 103 | }, 104 | "metadata": {}, 105 | "output_type": "display_data" 106 | } 107 | ], 108 | "source": [ 109 | "variables = {\n", 110 | " 'a': a,\n", 111 | " 'b': b,\n", 112 | " 'c': c\n", 113 | "}\n", 114 | "\n", 115 | "plt.figure(figsize=(12, 7))\n", 116 | "\n", 117 | "for i, (var_1, var_2) in enumerate([('b', 'a'), ('b', 'c'), ('c', 'a')]):\n", 118 | " \n", 119 | " color = COLORS[1]\n", 120 | " \n", 121 | " if 'b' in [var_1, var_2]:\n", 122 | " color = COLORS[0]\n", 123 | " \n", 124 | " plt.subplot(2, 2, i + 1)\n", 125 | " plt.scatter(variables[var_1], variables[var_2], alpha=.8, color=color)\n", 126 | " \n", 127 | " plt.xlabel(f'${var_1}$', fontsize=16)\n", 128 | " plt.ylabel(f'${var_2}$', fontsize=16)\n", 129 | "\n", 130 | "plt.suptitle('Pairwise relationships between $a$, $b$ and $c$')\n", 131 | "plt.subplots_adjust(hspace=.25, wspace=.25)\n", 132 | "plt.show()" 133 | ] 134 | }, 135 | { 136 | "cell_type": "code", 137 | "execution_count": null, 138 | "id": "a1f0699b", 139 | "metadata": {}, 140 | "outputs": [], 141 | "source": [] 142 | } 143 | ], 144 | "metadata": { 145 | "kernelspec": { 146 | "display_name": "Python [conda env:causal_book_py38]", 147 | "language": "python", 148 | "name": "conda-env-causal_book_py38-py" 149 | }, 150 | "language_info": { 151 | "codemirror_mode": { 152 | "name": "ipython", 153 | "version": 3 154 | }, 155 | "file_extension": ".py", 156 | "mimetype": "text/x-python", 157 | "name": "python", 158 | "nbconvert_exporter": "python", 159 | "pygments_lexer": "ipython3", 160 | "version": "3.8.13" 161 | } 162 | }, 163 | "nbformat": 4, 164 | "nbformat_minor": 5 165 | } 166 | -------------------------------------------------------------------------------- /Extras_01__DGPs.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 387, 6 | "id": "de02baff", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import numpy as np\n", 11 | "import pandas as pd\n", 12 | "from scipy import stats\n", 13 | "\n", 14 | "from sklearn.model_selection import train_test_split\n", 15 | "\n", 16 | "import matplotlib.pyplot as plt\n", 17 | "plt.style.use('fivethirtyeight')" 18 | ] 19 | }, 20 | { 21 | "cell_type": "code", 22 | "execution_count": 2, 23 | "id": "702ababf", 24 | "metadata": {}, 25 | "outputs": [], 26 | "source": [ 27 | "COLORS = [\n", 28 | " '#00B0F0',\n", 29 | " '#FF0000'\n", 30 | "]" 31 | ] 32 | }, 33 | { 34 | "cell_type": "markdown", 35 | "id": "4d417229", 36 | "metadata": {}, 37 | "source": [ 38 | "# Additional code for data generating processes" 39 | ] 40 | }, 41 | { 42 | "cell_type": "markdown", 43 | "id": "ac23d7d9", 44 | "metadata": {}, 45 | "source": [ 46 | "## Chapter 09" 47 | ] 48 | }, 49 | { 50 | "cell_type": "markdown", 51 | "id": "7058fff7", 52 | "metadata": {}, 53 | "source": [ 54 | "### Post-training earnings data (simple)" 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "execution_count": 39, 60 | "id": "865c3739", 61 | "metadata": {}, 62 | "outputs": [], 63 | "source": [ 64 | "SAMPLE_SIZE = 200\n", 65 | "MAX_AGE = 50\n", 66 | "\n", 67 | "age = stats.halfnorm.rvs(loc=19, scale=10, size=SAMPLE_SIZE).astype(int)\n", 68 | "age = np.where(age > MAX_AGE, np.random.choice(np.arange(20, MAX_AGE)), age)\n", 69 | "\n", 70 | "took_a_course = stats.bernoulli(p=10/age).rvs().astype(bool)\n", 71 | "\n", 72 | "earnings = 75000 + took_a_course * 10000 + age * 1000 + age**2 * 50 + np.random.randn(SAMPLE_SIZE) * 2000\n", 73 | "earnings = earnings.round()\n", 74 | "\n", 75 | "earnings = pd.DataFrame(dict(\n", 76 | " age=age,\n", 77 | " took_a_course=took_a_course,\n", 78 | " earnings=earnings\n", 79 | "))" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": 40, 85 | "id": "ca46d6ce", 86 | "metadata": {}, 87 | "outputs": [ 88 | { 89 | "data": { 90 | "text/plain": [ 91 | "" 92 | ] 93 | }, 94 | "execution_count": 40, 95 | "metadata": {}, 96 | "output_type": "execute_result" 97 | }, 98 | { 99 | "data": { 100 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAb4AAAEJCAYAAADrQkIkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAABB4ElEQVR4nO3de1iU95n4/zcOiCNyUBgEgmBIcArEfmmy6ylbI2BVjDmYkGJ/bex6adAQf6Z+F0XiIbXLRk03ifmpMa7Sba80BxdqIm6I7maBRC2K1iIWXSSXKQJBEBRkkOMwvz8sow8zDzKcmblf15XrivfcDM9nHvTm83k+B6e6ujoTQgghhIMYNdQXIIQQQgwmKXxCCCEcihQ+IYQQDkUKnxBCCIcihU8IIYRDkcInhBDCoUjhE0II4VCk8AkhhHAoUviGQElJyVBfwqCRttonR2mro7QTHKutUviEEEI4FCl8QgghHIoUPiGEEA7lvoXv7bffJioqikmTJvHQQw8RHx/PxYsXVfNfffVVvLy82LVrlyLe0tLCunXrCAkJISAggCVLllBRUaHIqaurIyEhgaCgIIKCgkhISKCurk6RU1ZWRnx8PAEBAYSEhLB+/XpaW1sVOUVFRSxcuBA/Pz/CwsLYsWMHJpPsxS2EEKIHhe/EiRMsX76cY8eOkZmZibOzM88++yw3b960yD18+DDnzp3D39/f4rWUlBSOHDlCWloaWVlZNDQ0EB8fj9FoNOesWLGCwsJC0tPTycjIoLCwkJUrV5pfNxqNxMfHYzAYyMrKIi0tjczMTDZu3GjOuXXrFosXL8bX15fs7Gy2b9/Orl272L17t80fjhBCCPvjfL+EQ4cOKf68b98+goKCOHXqFLGxseb41atX2bBhA5999hlxcXGKr6mvr+eDDz5gz549REVFmd9n6tSp5ObmEhMTQ3FxMV9++SVHjx5l+vTpALzzzjvExsZSUlJCaGgo2dnZXLp0iQsXLhAYGAjA1q1bWbNmDZs3b8bDw4P09HSamprYu3cvWq2W8PBwLl++zHvvvcfq1atxcnLq2ycmhBBiwJQ2tJF6roHK20b8x2rY9Kg7we4u/fo9bH7GZzAY6OjowMvLyxxrb29nxYoVJCUlodfrLb6moKCAtrY2oqOjzbHAwED0ej2nT58GID8/n3HjxpmLHsCMGTNwc3NT5Oj1enPRA4iJiaGlpYWCggJzzsyZM9FqtYqcyspKSktLbW2uEEKIQVLa0Mazx2pJv9LEiWutpF9p4tljtZQ2tPXr97lvj6+rDRs2MHXqVKZNm2aObdu2jfHjx7N8+XKrX1NdXY1Go8Hb21sR1+l0VFdXm3O8vb0VPTInJyd8fHwUOTqdTvEe3t7eaDQaRU5AQIDF9+l8bfLkyVavcbDXsDjSmhlpq31ylLY6Sjth6Nu6udiFbxuUvbtvG4wkf1XBP+ttK36hoaGqr9lU+F577TVOnTrF0aNH0Wg0wJ1ngB999BHHjx+36aIATCaTRaHrTU7XeNeczokt3Q1zdvch9bfOoVtHIG21T47SVkdpJwyPthq+uQ60WsQbNW6Ehuosv6CXejzUmZKSwh/+8AcyMzMVvabjx49z7do19Ho93t7eeHt7U1ZWxuuvv054eDgAvr6+GI1GamtrFe9ZU1Nj7o35+vpSU1OjmH1pMpmora1V5HT27DrV1tZiNBq7zampqQGw6C0KIYQYPvzHaqzG/VTivdWjwpecnExGRgaZmZlMmTJF8dqKFSs4efIkx48fN//n7+9PYmIihw8fBiAyMhIXFxdycnLMX1dRUUFxcbH5md60adMwGAzk5+ebc/Lz82lsbFTkFBcXK5ZB5OTk4OrqSmRkpDknLy+P5uZmRY6/vz/BwcG2fDZCCCEG0aZH3XnQXVnkHnS/M8GlP913qDMpKYmDBw/y+9//Hi8vL6qqqgBwc3Nj3Lhx6HQ6i56Us7MzEydONHebPT09efHFF9myZQs6nY7x48ezceNGIiIimDNnDgB6vZ65c+eydu1a3n33XUwmE2vXrmX+/Pnm94mOjiYsLIxVq1aRmprKzZs32bJlC0uXLsXDwwOAuLg4duzYQWJiIklJSXzzzTfs3LmT9evXy4xOIYQYxoLdXfhsvjep5xq4dtuI3wDN6rxv4Ttw4AAAzzzzjCKenJxMSkpKj7/RG2+8gUajYdmyZTQ3NzN79mzef/9987NCgP3795OcnMxzzz0HQGxsLG+++ab5dY1Gw8GDB0lKSmLBggWMGTOGuLg4UlNTzTmenp58+umnJCUlERUVhZeXF6+88gqrV6/u8bUKIYQYGsHuLux/YsKAfg+nuro62dJkkA2Hh8iDRdpqnxylrY7STnCstspenUIIIRyKzev4hBBC2I/OnVKu1LgS8t2NAXmmNtxI4RNCCAfVuVPKtw1GQMOfbjVx9norn833tuviJ0OdQgjhoFLPNfyt6N31bYOR1HMNQ3RFg0MKnxBCOKjK20ar8WsqcXshhU8IIRzUYO2UMtxI4RNCCAc1WDulDDdS+IQQwkF17pTyQoiWxzyNvBCitfuJLSCzOoUQwqF17pRSUlJLaGjQUF/OoJAenxBCCIcihU8IIYRDkcInhBDCoUjhE0II4VCk8AkhhHAoUviEEEI4FCl8QgghHIoUPiGEEA5FCp8QQgiHIoVPCCGEQ5HCJ4QQwqFI4RNCCOFQ7lv43n77baKiopg0aRIPPfQQ8fHxXLx40fx6W1sbr7/+OrNmzSIgIAC9Xs+KFSsoKytTvE9LSwvr1q0jJCSEgIAAlixZQkVFhSKnrq6OhIQEgoKCCAoKIiEhgbq6OkVOWVkZ8fHxBAQEEBISwvr162ltbVXkFBUVsXDhQvz8/AgLC2PHjh2YTCZbPxshhBB26L6F78SJEyxfvpxjx46RmZmJs7Mzzz77LDdv3gTg9u3bnD9/nqSkJL766is++ugjKioqiIuLo7293fw+KSkpHDlyhLS0NLKysmhoaCA+Ph6j8e5JvytWrKCwsJD09HQyMjIoLCxk5cqV5teNRiPx8fEYDAaysrJIS0sjMzOTjRs3mnNu3brF4sWL8fX1JTs7m+3bt7Nr1y52797dLx+YEEKIkc2prq7Opq6QwWAgKCiIDz/8kNjYWKs5//u//8uMGTM4efIkERER1NfX8/DDD7Nnzx5+/OMfA1BeXs7UqVPJyMggJiaG4uJipk+fztGjR5kxYwYAeXl5xMbGcubMGUJDQ/nv//5vfvzjH3PhwgUCAwMBOHjwIGvWrKGkpAQPDw/S0tL45S9/yeXLl9FqtQD8+te/5je/+Q0XL17Eycmp1x9WfykpKSE0NHSoL2NQSFvtk6O01VHaCY7VVpuf8RkMBjo6OvDy8lLNaWhoADDnFBQU0NbWRnR0tDknMDAQvV7P6dOnAcjPz2fcuHFMnz7dnDNjxgzc3NwUOXq93lz0AGJiYmhpaaGgoMCcM3PmTHPR68yprKyktLTU1uYKIYSwMzYfRLthwwamTp3KtGnTrL7e2trKpk2bWLBgAQ888AAA1dXVaDQavL29Fbk6nY7q6mpzjre3t6JH5uTkhI+PjyJHp9Mp3sPb2xuNRqPICQgIsPg+na9NnjzZ6nWXlJT0pPn9ZrC/31CSttonR2mro7QT7Kut3fVebSp8r732GqdOneLo0aNoNBqL19vb20lISKC+vp6PP/74vu9nMpksCl1vcrrGu+Z0TmzpbphzMLv4jjSkIG21T47SVkdpJzhWW3s81JmSksIf/vAHMjMzrfaa2tvbWb58OUVFRRw+fJgJEyaYX/P19cVoNFJbW6v4mpqaGnNvzNfXl5qaGsXsS5PJRG1trSKns2fXqba2FqPR2G1OTU0NgEVvUQghhOPpUeFLTk4mIyODzMxMpkyZYvF6W1sby5Yto6ioiCNHjjBx4kTF65GRkbi4uJCTk2OOVVRUmCe0AEybNg2DwUB+fr45Jz8/n8bGRkVOcXGxYhlETk4Orq6uREZGmnPy8vJobm5W5Pj7+xMcHNyT5gohhLBj9y18SUlJfPTRRxw4cAAvLy+qqqqoqqrCYDAAd3p6P//5zzl79iwHDhzAycnJnNPU1ASAp6cnL774Ilu2bCE3N5fz58+zcuVKIiIimDNnDgB6vZ65c+eydu1azpw5Q35+PmvXrmX+/Pnm7nd0dDRhYWGsWrWK8+fPk5uby5YtW1i6dCkeHh4AxMXFodVqSUxM5OLFi2RmZrJz504SExOHxYxOIYQQQ+u+z/gOHDgAwDPPPKOIJycnk5KSQkVFBVlZWQDmItZpz549/PSnPwXgjTfeQKPRsGzZMpqbm5k9ezbvv/++4lnh/v37SU5O5rnnngMgNjaWN9980/y6RqPh4MGDJCUlsWDBAsaMGUNcXBypqanmHE9PTz799FOSkpKIiorCy8uLV155hdWrV9vyuQghhLBTNq/jE33nSA+Rpa32yVHaOlDtLG1oI/VcA5W3jfiP1bDpUXeC3V36/fvYwlHuKfRiOYMQQojeK21o49ljtXzbcHfXqrPXW/lsvveQFz9HIZtUCyHEIEo916AoegDfNhhJPdcwRFfkeKTwCSHEIKq8bbQav6YSF/1PCp8QQgwi/7GWm38A+KnERf+TwieEEINo06PuPOiuLHIPut+Z4CIGh0xuEUKIQRTs7sJn871JPdfAtdtG/IbJrE5HIoVPCCEGWbC7C/ufmHD/xL8ZjssfRjIpfEIIMYzJ8of+J8/4hBBiGJPlD/1PCp8QQgxjsvyh/8lQpxBC9NFAPoOT5Q/9TwqfEEL0wUA/g9v0qDtnr7cq3l+WP/SNDHUKIUQfDPQzuM7lDy+EaPmh32heCNHKxJY+kh6fEEL0wWA8g7N1+YPonvT4hBCiD+QZ3MgjhU8IIfpAtiAbeWSoUwgh+qA3W5DJTixDSwqfEEL0kS3P4GQnlqEnQ51CCDGIZCeWoSeFTwghBpHsxDL0ZKhTCCH6yJZndjILdOjdt8f39ttvExUVxaRJk3jooYeIj4/n4sWLihyTycS2bdv43ve+h5+fH08++SSXLl1S5LS0tLBu3TpCQkIICAhgyZIlVFRUKHLq6upISEggKCiIoKAgEhISqKurU+SUlZURHx9PQEAAISEhrF+/ntbWVkVOUVERCxcuxM/Pj7CwMHbs2IHJZLLlcxFCiB7pfGaXfqWJE9daSb/SxLPHailtaLOa/49TtDg7KWPOTnfiYnDct/CdOHGC5cuXc+zYMTIzM3F2dubZZ5/l5s2b5px3332XPXv2sGPHDrKzs9HpdCxevJiGhrtj1ikpKRw5coS0tDSysrJoaGggPj4eo/Fu937FihUUFhaSnp5ORkYGhYWFrFy50vy60WgkPj4eg8FAVlYWaWlpZGZmsnHjRnPOrVu3WLx4Mb6+vmRnZ7N9+3Z27drF7t27+/xhCSFEV7Y+s/vt5Sbau/we3m66ExeD475DnYcOHVL8ed++fQQFBXHq1CliY2MxmUzs3buXX/ziFzzzzDMA7N27l9DQUDIyMli2bBn19fV88MEH7Nmzh6ioKPP7TJ06ldzcXGJiYiguLubLL7/k6NGjTJ8+HYB33nmH2NhYSkpKCA0NJTs7m0uXLnHhwgUCAwMB2Lp1K2vWrGHz5s14eHiQnp5OU1MTe/fuRavVEh4ezuXLl3nvvfdYvXo1Tk5dftUSQog+sPWZnTzjG3o2T24xGAx0dHTg5eUFQGlpKVVVVURHR5tztFots2bN4vTp0wAUFBTQ1tamyAkMDESv15tz8vPzGTdunLnoAcyYMQM3NzdFjl6vNxc9gJiYGFpaWigoKDDnzJw5E61Wq8iprKyktLTU1uYKIUS3bH1mJ8/4hp7NhW/Dhg1MnTqVadOmAVBVVQWATqdT5Ol0OqqrqwGorq5Go9Hg7e3dbY63t7eiR+bk5ISPj48ip+v38fb2RqPRdJvT+efOHCGE6C+27twiO70MPZtmdb722mucOnWKo0ePotEob1zXIUSTyXTfYcWuOdbye5LTNW7tWrr7WoCSkpJur7W/Dfb3G0rSVvvkKG3tSTvfmeLE+1edud46Ct3oDlYFNdF6rYGSa/2TP1js6Z6GhoaqvtbjwpeSksKhQ4c4cuQIkydPNscnTpwI3OlN3TsEWVNTY+5p+fr6YjQaqa2txcfHR5Eza9Ysc05NTY2i0JlMJmpraxXv0zns2am2thaj0ajI6dqzq6mpASx7pffq7kPqb53PLB2BtNU+OUpbe9rOUGDO93v+vrbmDwZHuafQw6HO5ORkMjIyyMzMZMqUKYrXgoODmThxIjk5OeZYc3MzeXl55ud1kZGRuLi4KHIqKiooLi4250ybNg2DwUB+fr45Jz8/n8bGRkVOcXGxYhlETk4Orq6uREZGmnPy8vJobm5W5Pj7+xMcHNyjD0UIIYT9um/hS0pK4qOPPuLAgQN4eXlRVVVFVVUVBoMBuDN8+PLLL7Nz504yMzO5ePEiiYmJuLm5ERcXB4CnpycvvvgiW7ZsITc3l/Pnz7Ny5UoiIiKYM2cOAHq9nrlz57J27VrOnDlDfn4+a9euZf78+ebfQqKjowkLC2PVqlWcP3+e3NxctmzZwtKlS/Hw8AAgLi4OrVZLYmIiFy9eJDMzk507d5KYmCgzOoUQQtx/qPPAgQMA5qUKnZKTk0lJSQHg1VdfpampiXXr1lFXV8djjz3GoUOHcHe/+7D2jTfeQKPRsGzZMpqbm5k9ezbvv/++4lnh/v37SU5O5rnnngMgNjaWN9980/y6RqPh4MGDJCUlsWDBAsaMGUNcXBypqanmHE9PTz799FOSkpKIiorCy8uLV155hdWrV/fm8xFCCGFnnOrq6mRLk0HmSGPp0lb75ChtdZR2gmO1VTapFkII4VCk8AkhhHAoUviEEEI4FCl8QgghHIoUPiGEEA5FDqIVQjgEWw6LFfZNCp8Qwu51HhZ777l5Z6+38tl8byl+DkiGOoUQds/Ww2KFfZPCJ4Swe3L4q7iXFD4hhN2Tw1/FvaTwCSHsnhz+Ku4lk1uEEMPCQM66DHZ34bP53qSea+DabSN+MqvToUnhE0IMucGYdRns7sL+Jyb0y3uJkU2GOoUQQ643sy5LG9p46asbLPriOi99dYPShraBvkxhJ6THJ4QYcrbOuixtaOPJrOuU3757qlretWY+X6hT7SGerGzi5RP11LV04OU6ir3/4Mnj/tq+X7wYcaTHJ4QYcrbOutxwql5R9ADKb5vYcKreav7JyiaeOXaDqwYjt9pMXDUYeebYDU5WNvXtwsWIJIVPCDHkbJ11eaq6xab4yyfqae9y5Ha76U5cOB4Z6hRCDDlbZ13ebrf+PmrxupYOq/F6lbiwb1L4hBDDgi2zLt2cnWhpNVnGXZys5nu5juJWm+XzQk9XGfRyRHLXhRAjzoyJo63Hfa3H9/6DJ85daqKz0524cDxS+IQQw4ItyxO2Tfck0E35z1eg2yi2TbdeyB7313J4/gSCxmnwdHEiaJyGw/MnyKxOByVDnUKIIWfrAvZgdxc+j/WxaSeWx/21FL4ghU70sMd38uRJlixZQlhYGF5eXnz44YeK1w0GA+vWrSM8PBw/Pz/+7u/+jj179ihyWlpaWLduHSEhIQQEBLBkyRIqKioUOXV1dSQkJBAUFERQUBAJCQnU1dUpcsrKyoiPjycgIICQkBDWr19Pa2urIqeoqIiFCxfi5+dHWFgYO3bswGSyfB4ghLBNZ69sVaFrvy4a780C9s5ngkdidex/YoJsPyZ6rEeFr7GxkfDwcLZv345Wa/kb08aNG/mv//ov3n//fU6fPs0//dM/sXXrVj755BNzTkpKCkeOHCEtLY2srCwaGhqIj4/HaLz7w75ixQoKCwtJT08nIyODwsJCVq5caX7daDQSHx+PwWAgKyuLtLQ0MjMz2bhxoznn1q1bLF68GF9fX7Kzs9m+fTu7du1i9+7dvfqAhBB3dC4aT7/SxJ9uaUi/0sSTWdf7pfjJsUFiMPVoqHPevHnMmzcPgMTERIvX8/PziY+PZ/bs2QAEBwfzwQcf8Kc//YklS5ZQX1/PBx98wJ49e4iKigJg3759TJ06ldzcXGJiYiguLubLL7/k6NGjTJ8+HYB33nmH2NhYSkpKCA0NJTs7m0uXLnHhwgUCAwMB2Lp1K2vWrGHz5s14eHiQnp5OU1MTe/fuRavVEh4ezuXLl3nvvfdYvXo1Tk7WZ30J4Yhs2Ri6u0XjH//Ip0/XIccGicHUL5NbZsyYwdGjRykvLwfg9OnT/OUvfyEmJgaAgoIC2traiI6ONn9NYGAger2e06dPA3eK57hx48xFr/N93dzcFDl6vd5c9ABiYmJoaWmhoKDAnDNz5kxFzzQmJobKykpKS0v7o7lC2IV7e3AnrrXetwd3tsa2uC3k2CAxmPplcsuOHTtYu3YtjzzyCM7Od97yzTffZMGCBQBUV1ej0Wjw9vZWfJ1Op6O6utqc4+3treiROTk54ePjo8jR6XSK9/D29kaj0ShyAgICLL5P52uTJ0+22oaSkpLeNL3XBvv7DSVp6/D0T0UulN9W9u7Kb5v4f/+ngrciLIuZsX0M1n5XNra390u735nixPtXnbneOgrd6A5WBTXReq2Bkmt9fus+GUn3tK/sqa2hoaGqr/VL4du3bx+nT5/m448/ZtKkSfzxj39k8+bNBAUFMXfuXNWvM5lMFoWuNzld411zOie2dDfM2d2H1N86h24dgbR1+Lp0thKw3LnkUpMroaGTLeIzSmvIKrPcEmyGv5bQ0El9vp5QYM73+/w2/Wqk3dO+cKS29rnwNTU18atf/Yrf/va3xMbGAvDII49w4cIFdu3axdy5c/H19cVoNFJbW4uPz91nATU1NcyaNQsAX19fampqFIXOZDJRW1tr7rH5+vqahz071dbWYjQaFTmdvb97vw9g0VsUwrGpzXS2Ht823ZPCGzWUN94tlt2tnRvIg2WF6Is+P+Nra2ujra0NjUY5Pq/RaOjouPMXJDIyEhcXF3JycsyvV1RUUFxcbH6mN23aNAwGA/n5+eac/Px8GhsbFTnFxcWKZRA5OTm4uroSGRlpzsnLy6O5uVmR4+/vT3BwcF+bK4Td+Hud9V1O1OKda+deCNHymKeRF0K0fB7rY7WYda7Lu/f54bPHauXMPDEs9KjwGQwGCgsLKSwspKOjg/LycgoLCykrK8PDw4PHH3+crVu3cvz4cf7617/y4Ycf8sknn7Bo0SIAPD09efHFF9myZQu5ubmcP3+elStXEhERwZw5cwDQ6/XMnTuXtWvXcubMGfLz81m7di3z5883d7+jo6MJCwtj1apVnD9/ntzcXLZs2cLSpUvx8PAAIC4uDq1WS2JiIhcvXiQzM5OdO3eSmJgoMzqFuIetu5/A3bVz709t6XbtXG/W5QkxWJzq6uruu7L7+PHjPPXUUxbxn/zkJ+zdu5eqqiq2bt1KTk4ON2/eZNKkSSxdulSxfKC5uZnNmzeTkZFBc3Mzs2fP5q233lLM0Lx58ybJycl88cUXAMTGxvLmm2/i5eVlzikrKyMpKYmvv/6aMWPGEBcXR2pqKq6uruacoqIikpKSOHfuHF5eXixbtozk5ORhU/gcaSxd2jq8dQ5H9nT3k878KzUGQnzGqeYv+uI6J661WsR/6DeaI7Ej55HDSLynveVIbe1R4RP9y5F+wKSt9sPatmIPumusbiv2/3xpfSLMwkmufDS3b2v+BpO939N7OVJbZZNqIUSP2DJ8qbZDoOwcKIYDKXxCiB6xZVux6mbruddV4kIMJil8Qogece96oN3fjLMSv95svWtXrRIXYjDJsURCDGPDaS2c2twwa3Ff7SiuGix7dxO18ru2GHpS+IQYpmw9o26g3Wqz3ltrsBJ/0N2Zs9ct1+xNdpd/csTQk1+/hBimerMWzpZTzG1lywkKsum0GM7k1y8hhilbz6grbWgj5kg1NfesIsipaOJ/nvLtlx7iP07R8um3TbTf08FzdroT7yrY3YXP5nvbtEZQiMEihU+IYcrWM+oSvrqhKHoANS134scWTezz9fz2srLoAbSb7sQf97de/PY/MaHP31eI/iZDnUIMU7YOF+Zfb7cpbqsrt6wPm36rEhdiuJIenxDDlK3DhbadtXCHLbNGZYmCsBdS+IQYAXpSWpxU8tR2qLV11qgsURD2Qn5ihRimbD3aZ5rO+u+xanFbZ40+qLIUQZYoiJFGCp8Qw5SthenfnpiAj6sy5uN6J26NrbNGZYmCsBfyq5oQw5SthSnY3YX/ecq3x88EnVUGUDUq8XufOV6pNRDirX4skRDDmRQ+IYYpW5czgG1LCIrrrRdQtfi9719SUktoaFCPvo8Qw40MdQoxTA300GJDW4dNcSHshfT4hBhEtiwf6M3uJycrm3j5RD11LR14uY5i7z94Wl1cDtChMg+0Q3UeqBD2QQqfEH3UWcyu1LgS8t0N1eLUm02nbRm6PFnZxDPHbph3V7nVZuSZYzc4PH+C1eIX4j6Kv9y0HNZ8yF0GgoR9k59wIfrg3iUHf7ql6XbJQW82nbbFyyfqrW4p9vKJeqv5YeNHW41/TyUuhL2QwidEH9hSzGydpWmrGyqnm6vF/3GKlq5nyKptOi2EPZHCJ0Qf2FLMejNL05ZjhtSezanFu9t0Wgh71qPCd/LkSZYsWUJYWBheXl58+OGHFjnffPMNP/vZzwgKCsLf35/Zs2dTXFxsfr2lpYV169YREhJCQEAAS5YsoaKiQvEedXV1JCQkEBQURFBQEAkJCdTV1SlyysrKiI+PJyAggJCQENavX09ra6sip6ioiIULF+Ln50dYWBg7duzAZJL9BEX/G8gz6mzduSVE5dmc2jO7ge6BCjFc9ajwNTY2Eh4ezvbt29FqLYdB/vrXvzJ//nyCg4PJzMwkLy+PTZs24ebmZs5JSUnhyJEjpKWlkZWVRUNDA/Hx8RiNd/+SrVixgsLCQtLT08nIyKCwsJCVK1eaXzcajcTHx2MwGMjKyiItLY3MzEw2btxozrl16xaLFy/G19eX7Oxstm/fzq5du9i9e3evPiAhumPLcGHnLM0XQrT80G80L4Rou53YYuszQVuf2fWmByqEPejRrM558+Yxb948ABITEy1eT01NJTo6mn/5l38xxyZPnmz+//r6ej744AP27NlDVFQUAPv27WPq1Knk5uYSExNDcXExX375JUePHmX69OkAvPPOO8TGxlJSUkJoaCjZ2dlcunSJCxcuEBgYCMDWrVtZs2YNmzdvxsPDg/T0dJqamti7dy9arZbw8HAuX77Me++9x+rVq3Fykqnaov8M5Bl1th4DtOlRd/KqWihvvLsOL9BtlGqPctOj7py93qoorrIFmXAEfX7G19HRwdGjR9Hr9Tz//PM89NBDREVFcejQIXNOQUEBbW1tREdHm2OBgYHo9XpOnz4NQH5+PuPGjTMXPYAZM2bg5uamyNHr9eaiBxATE0NLSwsFBQXmnJkzZyp6pjExMVRWVlJaWtrX5gqhMJBn1PXqGKCuQ/rdDPHb2gMVwl70eR3f9evXMRgMvP3227z22mu8/vrrfP3117z00kuMHTuWBQsWUF1djUajwdvbW/G1Op2O6upqAKqrq/H29lb0yJycnPDx8VHk6HQ6xXt4e3uj0WgUOQEBARbfp/O1e3ui9yopKen9h9ALg/39hpI9t/W7BlfAcmiwoqGlz+32wPp7e9Jq9b03F7tQfltZtMpvm0j+qoJ/1qsX4vX3/HVpvVZLybWeXZ8939d7OUo7wb7aGhoaqvpanwtfR8edYZWFCxeyevVqAL7//e9TUFDAgQMHWLBggerXmkwmi0LXm5yu8a45nRNbuhvm7O5D6m+dQ7eOwN7b+kBxNZXXLYtKoMeYPu9lGfbdDf5isJxh+T3fcVbf2/DNdaDVIt6ocSM0VGcR7wt7v6+dHKWd4Fht7fNQp7e3N87Ozuj1ekV8ypQplJeXA+Dr64vRaKS2tlaRU1NTY+6N+fr6UlNTo5h9aTKZqK2tVeR09uw61dbWYjQau82pqakBsOgtCtFXvmOs/xXSqcRtYessUJmsIkTP9Plv5+jRo3n00UctusjffPMNkyZNAiAyMhIXFxdycnLMr1dUVFBcXGx+pjdt2jQMBgP5+fnmnPz8fBobGxU5xcXFimUQOTk5uLq6EhkZac7Jy8ujublZkePv709wcHBfmyuEgtojtP5YPWPrMzg5L0+InulR4TMYDBQWFlJYWEhHRwfl5eUUFhZSVlYGwJo1a/j000/57W9/y5UrV/jd737HoUOHWLFiBQCenp68+OKLbNmyhdzcXM6fP8/KlSuJiIhgzpw5AOj1eubOncvatWs5c+YM+fn5rF27lvnz55u739HR0YSFhbFq1SrOnz9Pbm4uW7ZsYenSpXh4eAAQFxeHVqslMTGRixcvkpmZyc6dO0lMTJQZnaLfVavsinJdJW6rckM7p6tbOV/bxunqVsoN7aq5MllFiJ5xqquru+/vpsePH+epp56yiP/kJz9h7969AHz44Ye8/fbbVFRUEBISwv/9v/+XuLg4c25zczObN28mIyOD5uZmZs+ezVtvvaWYoXnz5k2Sk5P54osvAIiNjeXNN9/Ey8vLnFNWVkZSUhJff/01Y8aMIS4ujtTUVFxd7x49XVRURFJSEufOncPLy4tly5aRnJw8bAqfI42lj8S22nKCwvfTr3HVYFnkgsZpKHzBr0/X0XXTabizRlBt0+nBNBLva284SjvBsdrao8In+pcj/YANh7baUsisnaDwoLtGtec09z+rOWtlcsvf61z470W+fbrugSyqfTUc7utgcJR2gmO1VY4lEnattKGNJ7+oUSzqzqtq4fNYH6uFrLvdUqwtPH/Q3dlq4Zvsbv2vli1FWG1z6Zv9NIwqhKOSTaqFXUs5Xa8oegDljR2knLZ+VM+3Ddafof1VJW7LlmW27r2ptrm0UQ6KFaJPpPAJu3bmuuW6tu7i1U0dVuNVKnFbTjiwde/NB8dZL3AhKnEhRM9I4RN2Tq1IWI/rxliP+6rEbTnhwNbTEMInuFqNh6nEhRA9I4VP2LW/87H+/EwtHuJhPf6gStyWReO2LjCXdXlCDAwpfGLEseVw1u0zPAkcq+ytBY51YvsMT6v5thYbW/I3Pepu9VrU3lvW5QkxMGRWpxhRrC03OHu9VbUgBLu78PlCHannGrh224jffWZSdhab3uRfqTUQ4j2u23ycnABTlz+rs+UYIyFEz0jhEyOKrcsN7tXTBau9Ljb3+Qap5xqszjDtybULIfqPFD4xotg6QcTWHqKtShvamP95DdeaOgANf7rVxPHKFo49ablO0NZrF0IMDHnGJ0YUWyeI2LqEAGx7hvjqybq/Fb27rjV18OrJuj5fuxBiYEiPT4womx51J6+qRTFkGOg2SnWCyED3EPOqrK8HtBbf9Kg7Z6+3WmyHJrM0hRhc0uMTI05be0e3f77XQPcQjSrP9TqsxGWWphDDg/T4xLDQ0z0sN5yqp6pFGatquRP/+Ec+Fvm29rJs3bJMN8aJyibLKuejsuBdZmkKMfSk8IkhZ8vw4tka68/b1OK2Lk/4rtH6EGiFSvzAE+N56ugN7u1zjvpbXAgxPEnhE0POtiUKamsG1NcS2NLLalIZNlWLP+6v5ciCCbx8op4bTe1M0Dqz9x88h/y8PCGEOil8YshduWW9t/atlfjf60aTVdZiNd4fnLouMFfErXvcX0vhC1qHOs9MiJFMJreIIXe92XpvrdpKfNt0TwLdlD+2gW6j2Dbd+hZkYNvyBFeVvxFqcSHEyCM9PjEgTlY2/W34T8uEgmvdDv95jXbiqkq8q2B3Fz6P9enxM7vShjaezLpO+e27RTTvWjOfL9RZ/RrvMaOobLJ8nuczRiqfEPZCCp/odycrm3j62I2/TfV3wmAw8vSxG2TOn2C1+FU2Wp8xqRa35ZndhlP1iqIHUH7bpDoLtLbZ+rO8GpW4EGLkkV9jRb976es6i/VtRtOduDX1KiOPanFbnFY5cFYt3qJS39TiQoiRR3p8ot+pnWKuFre22Lu7eE/X/AE0tll/E7W4RmUOi1pcCDHy9KjHd/LkSZYsWUJYWBheXl58+OGHqrmvvvoqXl5e7Nq1SxFvaWlh3bp1hISEEBAQwJIlS6ioqFDk1NXVkZCQQFBQEEFBQSQkJFBXV6fIKSsrIz4+noCAAEJCQli/fj2trcrf3ouKili4cCF+fn6EhYWxY8cOTKae7s0v+spJ5aNWi/tqrf8YWot3rvlLv9LEiWutpF9p4tljtaoTVtQKlrNKXG12aH/NGhVCDL0eFb7GxkbCw8PZvn07Wq36+qTDhw9z7tw5/P39LV5LSUnhyJEjpKWlkZWVRUNDA/Hx8RiNdycSrFixgsLCQtLT08nIyKCwsJCVK1eaXzcajcTHx2MwGMjKyiItLY3MzEw2btxozrl16xaLFy/G19eX7Oxstm/fzq5du9i9e3ePPhDRd+NVdi1Ri++f7WVRoDROd+Jd2bqlmJV5KgCoHYjQm1mjQoiRpUdDnfPmzWPevHkAJCYmWs25evUqGzZs4LPPPiMuLk7xWn19PR988AF79uwhKioKgH379jF16lRyc3OJiYmhuLiYL7/8kqNHjzJ9+nQA3nnnHWJjY83ro7Kzs7l06RIXLlwgMDAQgK1bt7JmzRo2b96Mh4cH6enpNDU1sXfvXrRaLeHh4Vy+fJn33nuP1atXd7seS/SPIHdnqpste2DB7tZ/3B7315I5/84i8PqWDjxdR6nOArV102nrq/LuxK2xddaoEGLk6ZfJLe3t7axYsYKkpCT0er3F6wUFBbS1tREdHW2OBQYGotfrOX36NAD5+fmMGzfOXPQAZsyYgZubmyJHr9ebix5ATEwMLS0tFBQUmHNmzpyp6JnGxMRQWVlJaWlpfzTXIdmyFu5BlQI3WSUOnYvA/Sj9WQCFL/ipLn2wddPpMSpjmmpxuDtr9Eisjv1PTJCiJ4Sd6ZfCt23bNsaPH8/y5cutvl5dXY1Go8Hb21sR1+l0VFdXm3O8vb0VPTInJyd8fHwUOTqdTvEe3t7eaDSabnM6/9yZI2wrZLY+V9v0qDsPuisLUX8dv7Mg0HoRUotv+cFYm+JCCPvX51mdJ06c4KOPPuL48eM2f63JZLIodL3J6RrvmtM5saW7Yc6SkpKeXXQ/Gezvd6+KJidWF7lS3nz395687xrZHdHCA1rLgcHNxS5826AsLN82GEn+qoJ/1lsvfu9MceL9q85cbx2FbnQHq4KaaL3WQMm1vl37htOugGXvbkPeTb7fYfnmuX91ASyLYu5f64hxvd63i7FiKO/rYHOUtjpKO8G+2trd9oF9LnzHjx/n2rVriiFOo9HI66+/zt69e7l48SK+vr4YjUZqa2vx8bm7aLimpoZZs2YB4OvrS01NjaLQmUwmamtrzT02X19f87Bnp9raWoxGoyKna8+upqYGwKIneK/B3GNxqPd0fPOrG5Q3Nyli5c2j+PDmBPZ/33JheE1xNWBZ4GoZS2ior9XvEQrM+X7/t/XmyQqr8bp2jdXv05tr762hvq+DyVHa6ijtBMdqa5+HOlesWMHJkyc5fvy4+T9/f38SExM5fPgwAJGRkbi4uJCTk2P+uoqKCoqLi83P9KZNm4bBYCA/P9+ck5+fT2NjoyKnuLhYsQwiJycHV1dXIiMjzTl5eXk0Nzcrcvz9/QkODu5rc+2CrRNEKgzW4+Uq8YGktjxhlEpcbe1glUpcCGH/etTjMxgMXLlyBYCOjg7Ky8spLCxk/PjxTJo0yaIn5ezszMSJE82/PXh6evLiiy+yZcsWdDod48ePZ+PGjURERDBnzhwA9Ho9c+fOZe3atbz77ruYTCbWrl3L/Pnzze8THR1NWFgYq1atIjU1lZs3b7JlyxaWLl2Kh4cHAHFxcezYsYPExESSkpL45ptv2LlzJ+vXr5cZnX9j6wSRZqP1IqEWH0iRPs6crrbcyizSx/qPsm6ME1cNlnFflaUVQgj716Me35///Gdmz57N7NmzaWpqYtu2bcyePZs33nijx9/ojTfeYNGiRSxbtowFCxbg5ubGJ598gkZz9x/b/fv388gjj/Dcc8/x/PPP88gjj7Bv3z7z6xqNhoMHDzJ27FgWLFjAsmXLWLRoEampqeYcT09PPv30UyorK4mKimLdunW88sorrF69usfXau82Pepuda1af0w+GWgTRlsvzmrxEA/rk14eVIkLIexfj3p8P/zhDy12UOnOhQsXLGJjxozh17/+Nb/+9a9Vv278+PH827/9W7fvPWnSJA4ePNhtTkREBF988UXPLtZBNbV2dPvne2mdR3HTyuta5/7b6rXzNIe6lg68ulnH19BuffsXg0p806PunL3eqlj03l8zTIUQI5NsUu2A1hy/SW2X+R61bXfi1gS4We9NPaASt9XJyiaeOXaDqwYjt9pMXDUYeebYDU5WNlnk2jpMG+zuwmfzvXkhRMsP/UbzQoiWz+Z7y9o8IRyYbFLtgP5YbX0Jglr8QXdnzl63fK27BemdG0lfqXEl5Lsb3e5+8vKJerp22NpNd+KFLyh7fb3pwdlyjJEQwv5J4RvGbDmFwJZclYMJVOO2FpvOBe938jX86VYTZ6+3qva06lTO/Km3Eu/swcmWYkKI3pLCN0wpi8cdasWjtKGN+Z/XcO2eKfrHK1s49qRPvxQEW4tNdxtJW+t5jdHALSudTVeVkVTpwQkh+kKe8Q1TtpxC8OrJOkXRA7jW1MGrJ+v6/bp6crjTFWtVDPhWJa52ZJQcJSWEGAjS4xumbFlkfvKa9dPE1eKuo6yfKO6q8muQLb1PgOvN1gtWtUq8pcP6GQqtHbLWTgjR/6THN0zZMnvR1md2oZ7W33uKStzWM/DUDpadqBL3Uqm4nmqVWAgh+kD+ZRmmNj3qTuBYZY8ncKxTv6w/Cxtv/TTx76nEbd3izHeM9R8rnUp87z94WpyI7ux0Jy6EEP1NCt9w1nWLtX7acs3WY4NsXTt3u836LE21+OP+Wg7Pn0DQOA2eLk4EjdNweP4E1TP5hBCiL+QZ3zCVeq6B8kZloShv7LA6M7I3p4zbMktz06PuHK9sUUyg8dOqb3FWVGe5l2Z3ceg8iFYKnRBi4EnhG0S2LOq2ZXjR2cn687xuDhm3aUlAuaHd4jSDqqYOyg3tKtdvaykWQojBI0Odg+TeU8z/dEtz31PMPVysFwl3K/HH/aw/m1OL22pZbp1FGTP9LW7N3/lYL+ZqcSGEGExS+AaJrTMj1ZawWYu/+7gXfl1mTPppR/Hu4169uVQL1c3Wn82pxVc/4mbxgzXqb3EhhBhqMtQ5SL5tsP58668qcVtOIQh2d+HYkz42beNlyxZntvrt5Sa6lsSOv8VlwooQYqhJ4esDW4rHd40qp5urxHtzCkFPn9mVNrQx7z+rqbp7SD1ff9fEfy3y7ZfiZ+vyByGEGEwy1NlL9z6zO3Gt9b7P7Awq592pxW1dcmCLV0/cVBQ9gKrmO3Fr1NaRq8VtLdpCCDGYpPD1kq3P7FQeh6nGB/IcuRNV1ouzWnziGOvv46cSH8iiLYQQfSVDnb1k60bMHSqTVdTiMHCnEKg8PlSNV7dYj1epxO9dJ3il1kCI9zg5OkgIMWxI4eslWzdidlJZ2tZPm7EMqHaVXqlaHO4W7ZKSWkJDgwbmwoQQohdkqLOXvEZbr1hqcYtpjveLDyOjVJqkFhdCiOFMeny9dL3JesVSi7u7OnGjxbLL5+7aP9VjIJcneLrAdSvDmp4ycimEGIGk8PVSo8o4n1p8um40X5RbVo/pur7vrlLa0MaczGpu3nP83pflTeQ+bX15gsYJjFaGXTUqNTjI3ZnrLZbrDYPd5cdHCDHy9Gio8+TJkyxZsoSwsDC8vLz48MMPza+1tbXx+uuvM2vWLAICAtDr9axYsYKysjLFe7S0tLBu3TpCQkIICAhgyZIlVFRUKHLq6upISEggKCiIoKAgEhISqKurU+SUlZURHx9PQEAAISEhrF+/ntZW5YGrRUVFLFy4ED8/P8LCwtixY0e/n+bdorIkTS2+fYan1WOGts/o+9E7S7NrFEUP4Gbrnbg1D46z/j4hanEP6127B1XiQggxnPWo8DU2NhIeHs727dvRapU7b9y+fZvz58+TlJTEV199xUcffURFRQVxcXG0t9/tJaSkpHDkyBHS0tLIysqioaGB+Ph4jMa7lWLFihUUFhaSnp5ORkYGhYWFrFy50vy60WgkPj4eg8FAVlYWaWlpZGZmsnHjRnPOrVu3WLx4Mb6+vmRnZ7N9+3Z27drF7t27e/0hWeOmspemWjzY3YXPF+p4IUTLY55GXgjR8vlC3X13V3npqxss+uI6L311Q3WN4Pkb1nuZavEf6KzvnhKpEpflCUIIe9Kjsap58+Yxb948ABITExWveXp68tlnnyli77zzDjNmzKC4uJiIiAjq6+v54IMP2LNnD1FRUQDs27ePqVOnkpubS0xMDMXFxXz55ZccPXqU6dOnm98nNjaWkpISQkNDyc7O5tKlS1y4cIHAwEAAtm7dypo1a9i8eTMeHh6kp6fT1NTE3r170Wq1hIeHc/nyZd577z1Wr16NUz9No+zN0KUtMx1LG9p4Mus65bfv9lTzrjXft1j2xKZH3Tl7vVWxDrG7QmbrMUZCCDGcDciszoaGO4u4vby8ACgoKKCtrY3o6GhzTmBgIHq9ntOnTwOQn5/PuHHjzEUPYMaMGbi5uSly9Hq9uegBxMTE0NLSQkFBgTln5syZip5pTEwMlZWVlJaW9lsbezN02dmDW1Xo2m0PDmDDqXpF0QMov21iw6n6vl04vVsc31m0j8Tq2P/EBCl6QogRq99nJ7S2trJp0yYWLFjAAw88AEB1dTUajQZvb29Frk6no7q62pzj7e2t6JE5OTnh4+OjyNHpdIr38Pb2RqPRKHICAgIsvk/na5MnT7Z63SUlJTa3dXeYE+9fdeZ66yh0oztYFdRO67W/UnLNMreiyYmVF1ypah0FaPjTrSa+Lm9k39QWHtBaPn88fW0M1n4vOV3VZOVaredCR7ftWn/Px9R6rdbqdfeH3ny2I5W01f44SjvBvtoaGhqq+lq/Fr729nYSEhKor6/n448/vm++yWSyKHS9yeka75rTObGlu2HO7j4k1a8B5ny/Z7lb/ruGqlbl0GhV6yj2VXnx8Y98LPI1Z7+zupWKRqOxvNYTFRZ5d4zqVbv6U+cwtSOQttofR2knOFZb+22os729neXLl1NUVMThw4eZMOHuVlu+vr4YjUZqa2sVX1NTU2Pujfn6+lJTU6OYfWkymaitrVXkdPbsOtXW1mI0GrvNqam5M7uxa29xMJ2tsT6sqRaPGG99KNFaXO0myu4EQghhqV/+bWxra2PZsmUUFRVx5MgRJk6cqHg9MjISFxcXcnJyzLGKigqKi4vNz/SmTZuGwWAgPz/fnJOfn09jY6Mip7i4WLEMIicnB1dXVyIjI805eXl5NDc3K3L8/f0JDg7uj+b2ilFlU06jyjKLm1bWzanF1R63yWM4IYSw1KPCZzAYKCwspLCwkI6ODsrLyyksLKSsrIz29nZ+/vOfc/bsWQ4cOICTkxNVVVVUVVXR1NQE3Jn5+eKLL7JlyxZyc3M5f/48K1euJCIigjlz5gCg1+uZO3cua9eu5cyZM+Tn57N27Vrmz59v7n5HR0cTFhbGqlWrOH/+PLm5uWzZsoWlS5fi4eEBQFxcHFqtlsTERC5evEhmZiY7d+4kMTGx32Z09oaryupwV5V9v2xZovC4n6vVXLW4EEI4sh4Vvj//+c/Mnj2b2bNn09TUxLZt25g9ezZvvPEGFRUVZGVlUVlZyZw5c9Dr9eb/Dh06ZH6PN954g0WLFrFs2TIWLFiAm5sbn3zyCRrN3fVh+/fv55FHHuG5557j+eef55FHHmHfvn3m1zUaDQcPHmTs2LEsWLCAZcuWsWjRIlJTU805np6efPrpp1RWVhIVFcW6det45ZVXWL16dX98Xr0W4Gb9o35AJW6LbdM9CezyPoFuo9g2ve+L44UQwt441dXV9e+WJsKql766QfqVJov4CyFaq0cPef272oQVqFv2gEWsc6/O4bbOzpEemEtb7Y+jtBMcq62y2eIg+ccpWg5926TYI1PjdCfeHwbq7D4hhLA3MvGvD3q6pRjAnqJGi42hjaY7cWvU+mpD34cTQoiRTXp8vVTa0MaTX9RQ3nh3skleVQufx/pYHWI8VdVqEesuPmuiC19VWRbSWROl9AkhRF9Ij6+XUk7XK4oeQHljBymnrW8p1mhlMXp38f/vh+OZ2GVS5kTXO3EhhBC9Jz2+Xjp5zcrJrMAfVeJjnaHFSudurModCHZ34b+e8h2WE1aEEGIkk8LXS/Uqj/PqVOL/Z4ILudcsX/w/E+6/MbQQQoj+I0Odg2Ssi/WPWi0uhBBiYMi/ur2ktgeMWvxWm/VneQ0qcSGEEANDCl8veaiMUKrF/cdqrMb9VOJCCCEGhhS+XopUeTanFt/0qDsPuiuLXHenngshhBgYUvh6SW2/a7X4vaeeP+Zp7NGp50IIIfqfzOrspaI6o01xuDtLs6SkltDQoIG6NCGEEN2QHl+vqU1KkckqQggxnEnh66W/1422KS6EEGJ4kMLXS3IGnhBCjEzyjK+Xgt1d+DzWR7YUE0KIEUYKXx/IlmJCCDHyyFCnEEIIhyKFTwghhEORwieEEMKhSOETQgjhUJzq6upkxbUQQgiHIT0+IYQQDkUKnxBCCIcihU8IIYRDkcInhBDCoUjhE0II4VCk8A2At99+m6ioKCZNmsRDDz1EfHw8Fy9eVOSYTCa2bdvG9773Pfz8/HjyySe5dOnSEF1x7/WkrS+//DJeXl6K/+bOnTtEV9x7+/fvZ9asWUyaNIlJkybxox/9iGPHjplft5d7Cvdvq73c067eeustvLy8WLdunTlmT/f1Xtbaaq/3tSspfAPgxIkTLF++nGPHjpGZmYmzszPPPvssN2/eNOe8++677Nmzhx07dpCdnY1Op2Px4sU0NDQM4ZXbridtBZgzZw7FxcXm/9LT04foinsvICCArVu38tVXX5GTk8Ps2bP56U9/yl/+8hfAfu4p3L+tYB/39F5nzpzhd7/7HREREYq4Pd3XTmptBfu7r9ZI4RsAhw4d4mc/+xnh4eFERESwb98+ampqOHXqFHDnN8i9e/fyi1/8gmeeeYbw8HD27t2LwWAgIyNjiK/eNvdraydXV1cmTpxo/m/8+PFDdMW99+STT/KjH/2IkJAQHn74YTZv3sy4ceM4c+aMXd1T6L6tnezhnnaqr6/npZdeYteuXXh5eZnj9nZfQb2tnezpvqqRwjcIDAYDHR0d5h+y0tJSqqqqiI6ONudotVpmzZrF6dOnh+gq+0fXtnbKy8vj4Ycf5rHHHmPNmjVcv359aC6wnxiNRv7whz/Q2NjItGnT7Pqedm1rJ3u6p52F7YknnlDE7fG+qrW1kz3dVzVyLNEg2LBhA1OnTjX/o1FVVQWATqdT5Ol0OiorKwf9+vpT17YCzJ07l6eeeorg4GCuXr1KamoqTz/9NLm5ubi6ug7h1dquqKiIefPm0dzcjJubG7///e+JiIgw/yNoT/dUra1gX/f0d7/7HVeuXGHfvn0Wr9nb39Xu2gr2dV+7I4VvgL322mucOnWKo0ePotFoFK85OTkp/mwymSxiI4laW59//nnz/0dERBAZGcnUqVM5duwYTz/99FBcaq+FhoZy/Phx6uvryczM5OWXX+Y///M/za/b0z1Va2t4eLjd3NOSkhJ+9atf8cUXXzB69GjVPHu4rz1pq73c1/uRwjeAUlJSOHToEEeOHGHy5Mnm+MSJEwGorq4mMDDQHK+pqbH4zXKkUGurNf7+/gQEBHDlypXBubh+NHr0aEJCQgD4wQ9+wLlz53jvvfdISkoC7OueqrV19+7dFrkj9Z7m5+dTW1vLzJkzzTGj0cgf//hHfvOb35ifVdvDfb1fW7/77juLXt1Iva/3I8/4BkhycjIZGRlkZmYyZcoUxWvBwcFMnDiRnJwcc6y5uZm8vDymT58+2JfaZ9211Zra2loqKyvNvwCMZB0dHbS2ttrdPbWms63WjNR7+uSTT/LHP/6R48ePm//7wQ9+wPPPP8/x48d5+OGH7ea+3q+t1nqBI/W+3o/0+AZAUlISBw8e5Pe//z1eXl7m5wRubm6MGzcOJycnXn75Zd566y1CQ0N5+OGH+dd//Vfc3NyIi4sb4qu3zf3aajAY2L59O08//TQTJ07k6tWr/OpXv0Kn07Fo0aIhvnrb/PKXv2TevHk88MAD5ll9J06c4D/+4z/s6p5C9221p3vauVbtXmPHjmX8+PGEh4cD2M19vV9b7em+3o8UvgFw4MABAJ555hlFPDk5mZSUFABeffVVmpqaWLduHXV1dTz22GMcOnQId3f3Qb/evrhfWzUaDRcvXuSTTz6hvr6eiRMn8sMf/pB///d/H3FtraqqIiEhgerqajw8PIiIiCAjI4OYmBjAfu4pdN/WpqYmu7mnPWFP97U79vR39X7kPD4hhBAORZ7xCSGEcChS+IQQQjgUKXxCCCEcihQ+IYQQDkUKnxBCCIcihU8IIYRDkcInhBDCoUjhE0II4VCk8AkhhHAo/z/n7Sl3T5FbTwAAAABJRU5ErkJggg==\n", 101 | "text/plain": [ 102 | "
" 103 | ] 104 | }, 105 | "metadata": {}, 106 | "output_type": "display_data" 107 | } 108 | ], 109 | "source": [ 110 | "plt.scatter(earnings.age, earnings.earnings)" 111 | ] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "execution_count": 41, 116 | "id": "100a985c", 117 | "metadata": {}, 118 | "outputs": [], 119 | "source": [ 120 | "earnings.to_csv('data/ml_earnings.csv', index=False)" 121 | ] 122 | }, 123 | { 124 | "cell_type": "markdown", 125 | "id": "567bbbbb", 126 | "metadata": {}, 127 | "source": [ 128 | "### Post-training earnings data (enhanced)" 129 | ] 130 | }, 131 | { 132 | "cell_type": "code", 133 | "execution_count": 139, 134 | "id": "2d26efeb", 135 | "metadata": {}, 136 | "outputs": [], 137 | "source": [ 138 | "# Train set large\n", 139 | "SAMPLE_SIZE = 5000\n", 140 | "MAX_AGE = 50\n", 141 | "\n", 142 | "age = stats.halfnorm.rvs(loc=19, scale=10, size=SAMPLE_SIZE).astype(int)\n", 143 | "age = np.where(age > MAX_AGE, np.random.choice(np.arange(20, MAX_AGE)), age)\n", 144 | "\n", 145 | "took_a_course = stats.bernoulli(p=10/age).rvs().astype(bool)\n", 146 | "python_proficiency = np.random.uniform(0, 1, SAMPLE_SIZE)\n", 147 | "\n", 148 | "noise = np.random.randn(SAMPLE_SIZE)\n", 149 | "\n", 150 | "earnings = 75000 + took_a_course * 10000 + took_a_course * python_proficiency * 5000 + age * 1000 + age**2 * 50 + noise * 2000\n", 151 | "earnings = earnings.round()\n", 152 | "\n", 153 | "earnings = pd.DataFrame(dict(\n", 154 | " age=age,\n", 155 | " python_proficiency = python_proficiency,\n", 156 | " took_a_course=took_a_course,\n", 157 | " earnings=earnings\n", 158 | "))" 159 | ] 160 | }, 161 | { 162 | "cell_type": "code", 163 | "execution_count": 140, 164 | "id": "99788d91", 165 | "metadata": {}, 166 | "outputs": [], 167 | "source": [ 168 | "earnings.to_csv('data/ml_earnings_interaction_train.csv', index=False)" 169 | ] 170 | }, 171 | { 172 | "cell_type": "code", 173 | "execution_count": 143, 174 | "id": "555e6273", 175 | "metadata": {}, 176 | "outputs": [], 177 | "source": [ 178 | "# Test set\n", 179 | "SAMPLE_SIZE = 100\n", 180 | "MAX_AGE = 50\n", 181 | "\n", 182 | "age = stats.halfnorm.rvs(loc=19, scale=10, size=SAMPLE_SIZE).astype(int)\n", 183 | "age = np.where(age > MAX_AGE, np.random.choice(np.arange(20, MAX_AGE)), age)\n", 184 | "\n", 185 | "python_proficiency = np.random.uniform(0, 1, SAMPLE_SIZE)\n", 186 | "\n", 187 | "noise = np.random.randn(SAMPLE_SIZE)\n", 188 | "\n", 189 | "earnings_0 = (75000 + 0 * 10000 + 0 * python_proficiency * 5000 + age * 5000 + age**2 * 50 + noise * 2000).round()\n", 190 | "earnings_1 = (75000 + 1 * 10000 + 1 * python_proficiency * 5000 + age * 5000 + age**2 * 50 + noise * 2000).round()\n", 191 | "true_effect = earnings_1 - earnings_0\n", 192 | "\n", 193 | "earnings_test = pd.DataFrame(dict(\n", 194 | " age=age,\n", 195 | " python_proficiency=python_proficiency,\n", 196 | " took_a_course=True,\n", 197 | " true_effect=true_effect\n", 198 | "))" 199 | ] 200 | }, 201 | { 202 | "cell_type": "code", 203 | "execution_count": 144, 204 | "id": "2763703c", 205 | "metadata": {}, 206 | "outputs": [], 207 | "source": [ 208 | "earnings_test.to_csv('data/ml_earnings_interaction_test.csv', index=False)" 209 | ] 210 | }, 211 | { 212 | "cell_type": "markdown", 213 | "id": "74afa358", 214 | "metadata": {}, 215 | "source": [ 216 | "## Chapter 11" 217 | ] 218 | }, 219 | { 220 | "cell_type": "markdown", 221 | "id": "283685f2", 222 | "metadata": {}, 223 | "source": [ 224 | "### Simulated data" 225 | ] 226 | }, 227 | { 228 | "cell_type": "code", 229 | "execution_count": 467, 230 | "id": "9db54f09", 231 | "metadata": {}, 232 | "outputs": [], 233 | "source": [ 234 | "SAMPLE_SIZE = 2000\n", 235 | "\n", 236 | "def get_data_dgp_11(sample_size=1000, with_interaction=True):\n", 237 | " chi = np.random.chisquare(12, (sample_size, 5)) / 30\n", 238 | " norm = np.random.normal(0, 1, (sample_size, 5))\n", 239 | " binom = np.random.binomial(1, [.3, .5, .7, .12, .9], (sample_size, 5))\n", 240 | " gumbel = np.random.gumbel(-2, 1, (sample_size, 5)) / 6\n", 241 | "\n", 242 | " X = np.concatenate([chi, norm, binom, gumbel], axis=1)\n", 243 | "\n", 244 | " T0 = np.zeros(sample_size)\n", 245 | " T1 = np.ones(sample_size)\n", 246 | " \n", 247 | " noise = np.random.randn(sample_size)\n", 248 | " \n", 249 | " coefs = np.random.gumbel(0, 10, X.shape[1] + 1)\n", 250 | " \n", 251 | " if with_interaction:\n", 252 | " interaction_term = X[:, 0] + X[:, 7]\n", 253 | " y0 = (coefs * np.concatenate([(T0 * interaction_term).reshape(-1, 1), X], axis=1)).sum(axis=1) + noise\n", 254 | " y1 = (coefs * np.concatenate([(T1 * interaction_term).reshape(-1, 1), X], axis=1)).sum(axis=1) + noise\n", 255 | " else:\n", 256 | " interaction_term = 1\n", 257 | " y0 = (coefs * np.concatenate([\n", 258 | " T0.reshape(-1, 1), \n", 259 | " X], \n", 260 | " axis=1))\\\n", 261 | " .sum(axis=1) + noise\n", 262 | " \n", 263 | " y1 = (coefs * np.concatenate([\n", 264 | " 10 * np.exp(\n", 265 | " T1 + X[:, 7:12]\\\n", 266 | " .sum(axis=1) / 1000).reshape(-1, 1), \n", 267 | " X], \n", 268 | " axis=1)).sum(axis=1) + noise\n", 269 | " \n", 270 | " return X, y0, y1, coefs\n", 271 | "\n", 272 | "\n", 273 | "def filter_outcomes(y, t):\n", 274 | " result = np.zeros_like(t, dtype=float) \n", 275 | " result[t == 0] = y[t == 0, 0] \n", 276 | " result[t == 1] = y[t == 1, 1] \n", 277 | " return result" 278 | ] 279 | }, 280 | { 281 | "cell_type": "markdown", 282 | "id": "51807226", 283 | "metadata": {}, 284 | "source": [ 285 | "#### No interaction" 286 | ] 287 | }, 288 | { 289 | "cell_type": "code", 290 | "execution_count": 468, 291 | "id": "08f5832e", 292 | "metadata": {}, 293 | "outputs": [], 294 | "source": [ 295 | "# No-interaction data\n", 296 | "X, y0, y1, coefs = get_data_dgp_11(SAMPLE_SIZE, False)\n", 297 | "\n", 298 | "# Generate treatments\n", 299 | "T = np.random.binomial(1, 0.5, SAMPLE_SIZE)\n", 300 | "\n", 301 | "# Train-test split\n", 302 | "X_train, X_test, y_train, y_test, T_train, T_test = train_test_split(X, np.stack([y0, y1]).T, T, test_size=.1)\n", 303 | "\n", 304 | "# Filter actual outcomes\n", 305 | "y_train_actual = filter_outcomes(y_train, T_train)\n", 306 | "y_test_actual = filter_outcomes(y_test, T_test)\n", 307 | "\n", 308 | "# To DF & update cols\n", 309 | "data_no_interaction_train = pd.DataFrame(\n", 310 | " np.concatenate(\n", 311 | " [\n", 312 | " X_train, \n", 313 | " T_train.reshape(-1, 1), \n", 314 | " y_train_actual.reshape(-1, 1)],\n", 315 | " axis=1\n", 316 | " )\n", 317 | ")\n", 318 | "\n", 319 | "data_no_interaction_train.columns = [f'x{i}' for i in range(20)] + ['treatment', 'outcome']\n", 320 | "\n", 321 | "\n", 322 | "data_no_interaction_test = pd.DataFrame(\n", 323 | " np.concatenate(\n", 324 | " [\n", 325 | " X_test, \n", 326 | " T_test.reshape(-1, 1), \n", 327 | " (y_test[:, 1] - y_test[:, 0]).reshape(-1, 1)],\n", 328 | " axis=1\n", 329 | " )\n", 330 | ")\n", 331 | "\n", 332 | "\n", 333 | "data_no_interaction_test.columns = [f'x{i}' for i in range(20)] + ['treatment', 'true_effect']\n", 334 | "\n", 335 | "# Store\n", 336 | "data_no_interaction_train.to_csv('data/data_11_no_interaction_train.csv', index=False)\n", 337 | "data_no_interaction_test.to_csv('data/data_11_no_interaction_test.csv', index=False)" 338 | ] 339 | }, 340 | { 341 | "cell_type": "code", 342 | "execution_count": 469, 343 | "id": "e87ab637", 344 | "metadata": {}, 345 | "outputs": [ 346 | { 347 | "data": { 348 | "text/plain": [ 349 | "(array([ 34.89751834, 8.61226375, -12.63565299, ..., 2.99937547,\n", 350 | " 14.48281498, 32.13379007]),\n", 351 | " array([ -9.92622448, -36.0636879 , -57.4257863 , ..., -41.66052399,\n", 352 | " -30.30810687, -12.62264189]))" 353 | ] 354 | }, 355 | "execution_count": 469, 356 | "metadata": {}, 357 | "output_type": "execute_result" 358 | } 359 | ], 360 | "source": [ 361 | "y0, y1" 362 | ] 363 | }, 364 | { 365 | "cell_type": "code", 366 | "execution_count": 470, 367 | "id": "9ad1f4f9", 368 | "metadata": {}, 369 | "outputs": [ 370 | { 371 | "data": { 372 | "text/plain": [ 373 | "array([-44.82374282, -44.67595165, -44.79013331, ..., -44.65989946,\n", 374 | " -44.79092185, -44.75643196])" 375 | ] 376 | }, 377 | "execution_count": 470, 378 | "metadata": {}, 379 | "output_type": "execute_result" 380 | } 381 | ], 382 | "source": [ 383 | "y1 - y0" 384 | ] 385 | }, 386 | { 387 | "cell_type": "markdown", 388 | "id": "31ce7420", 389 | "metadata": {}, 390 | "source": [ 391 | "#### With interaction" 392 | ] 393 | }, 394 | { 395 | "cell_type": "code", 396 | "execution_count": 475, 397 | "id": "71909b39", 398 | "metadata": {}, 399 | "outputs": [], 400 | "source": [ 401 | "# No-interaction data\n", 402 | "X, y0, y1, coefs = get_data_dgp_11(SAMPLE_SIZE, True)\n", 403 | "\n", 404 | "# Generate treatments\n", 405 | "T = np.random.binomial(1, 0.5, SAMPLE_SIZE)\n", 406 | "\n", 407 | "# Train-test split\n", 408 | "X_train, X_test, y_train, y_test, T_train, T_test = train_test_split(X, np.stack([y0, y1]).T, T, test_size=.1)\n", 409 | "\n", 410 | "# Filter actual outcomes\n", 411 | "y_train_actual = filter_outcomes(y_train, T_train)\n", 412 | "y_test_actual = filter_outcomes(y_test, T_test)\n", 413 | "\n", 414 | "# To DF & update cols\n", 415 | "data_with_interaction_train = pd.DataFrame(\n", 416 | " np.concatenate(\n", 417 | " [\n", 418 | " X_train, \n", 419 | " T_train.reshape(-1, 1), \n", 420 | " y_train_actual.reshape(-1, 1)],\n", 421 | " axis=1\n", 422 | " )\n", 423 | ")\n", 424 | "\n", 425 | "data_with_interaction_train.columns = [f'x{i}' for i in range(20)] + ['treatment', 'outcome']\n", 426 | "\n", 427 | "\n", 428 | "data_with_interaction_test = pd.DataFrame(\n", 429 | " np.concatenate(\n", 430 | " [\n", 431 | " X_test, \n", 432 | " T_test.reshape(-1, 1), \n", 433 | " (y_test[:, 1] - y_test[:, 0]).reshape(-1, 1)],\n", 434 | " axis=1\n", 435 | " )\n", 436 | ")\n", 437 | "\n", 438 | "\n", 439 | "data_with_interaction_test.columns = [f'x{i}' for i in range(20)] + ['treatment', 'true_effect']\n", 440 | "\n", 441 | "# Store\n", 442 | "data_with_interaction_train.to_csv('data/data_11_with_interaction_train.csv', index=False)\n", 443 | "data_with_interaction_test.to_csv('data/data_11_with_interaction_test.csv', index=False)" 444 | ] 445 | }, 446 | { 447 | "cell_type": "code", 448 | "execution_count": 474, 449 | "id": "72d62da4", 450 | "metadata": {}, 451 | "outputs": [ 452 | { 453 | "data": { 454 | "text/plain": [ 455 | "array([[ 1.63371979e+01, 1.59045881e+01],\n", 456 | " [-3.06609145e+00, -1.84422244e+00],\n", 457 | " [-1.03359389e+01, -7.87196605e+00],\n", 458 | " [ 4.46875521e+00, 4.96359667e+00],\n", 459 | " [ 3.18576088e+01, 3.37932190e+01],\n", 460 | " [ 1.19932029e+01, 2.98884169e+00],\n", 461 | " [ 9.69525909e+00, 7.93796477e+00],\n", 462 | " [-7.38440149e+00, -8.60629373e+00],\n", 463 | " [ 2.41254919e+01, 1.35643255e+01],\n", 464 | " [ 1.97529380e+01, 1.63711534e+01],\n", 465 | " [ 2.43068304e+01, 2.23882172e+01],\n", 466 | " [-7.98322104e+00, -1.16848962e+01],\n", 467 | " [-2.54805435e+00, -9.10154055e+00],\n", 468 | " [-2.12343129e+01, -2.11751544e+01],\n", 469 | " [ 1.99864813e+00, -3.96052447e+00],\n", 470 | " [ 1.54031350e+01, 1.24315489e+01],\n", 471 | " [ 7.02887405e+00, 5.32079794e+00],\n", 472 | " [ 2.07225787e+00, 3.98963467e+00],\n", 473 | " [ 1.05768464e+01, 1.16281929e+01],\n", 474 | " [ 2.58577213e+01, 2.11922848e+01],\n", 475 | " [ 1.11960625e+01, 6.92346165e+00],\n", 476 | " [ 8.63504905e+00, 7.25811536e+00],\n", 477 | " [ 1.45579537e+01, 1.14446180e+01],\n", 478 | " [-1.05771942e+01, -1.20518737e+01],\n", 479 | " [ 1.50904679e+01, 1.10366070e+01],\n", 480 | " [ 1.18356618e+01, 4.89932210e+00],\n", 481 | " [ 1.18242024e+01, 9.95327571e+00],\n", 482 | " [-4.42150929e+00, -8.56758507e-01],\n", 483 | " [ 1.90045567e+01, 2.25398094e+01],\n", 484 | " [-9.05976959e+00, -1.25219855e+01],\n", 485 | " [ 1.83118169e+01, 1.50464956e+01],\n", 486 | " [-9.71122700e+00, -9.93741894e+00],\n", 487 | " [-4.60208952e+00, 2.71032818e+00],\n", 488 | " [-2.15014674e+01, -2.04324464e+01],\n", 489 | " [ 1.31560593e+01, 1.00683814e+01],\n", 490 | " [-1.79174245e+01, -1.94410151e+01],\n", 491 | " [ 1.93439684e+01, 2.07561603e+01],\n", 492 | " [-1.58168095e+01, -1.03663941e+01],\n", 493 | " [-7.75661603e+00, -5.33883483e+00],\n", 494 | " [ 3.22820694e+01, 2.83894343e+01],\n", 495 | " [-2.01817097e+01, -1.76334410e+01],\n", 496 | " [ 3.17113259e+01, 2.81813370e+01],\n", 497 | " [-1.09985805e+01, -7.91712686e+00],\n", 498 | " [ 2.13866154e+01, 1.66148733e+01],\n", 499 | " [-1.48789697e+01, -8.17279022e+00],\n", 500 | " [ 9.17563124e+00, 6.02526627e+00],\n", 501 | " [ 2.58140460e+00, -1.12501476e-01],\n", 502 | " [-1.21727684e+01, -1.07267230e+01],\n", 503 | " [-1.90409191e+01, -1.65927276e+01],\n", 504 | " [ 3.72328902e+01, 3.79919913e+01],\n", 505 | " [-9.47953164e-01, 1.21423457e+00],\n", 506 | " [-1.45674899e+00, -1.87900795e+00],\n", 507 | " [-1.53365385e+01, -1.20907620e+01],\n", 508 | " [ 1.32460485e+01, 8.05235665e+00],\n", 509 | " [ 3.63131086e+00, 4.44464150e+00],\n", 510 | " [-3.49429150e+00, -4.56076985e+00],\n", 511 | " [ 3.10834849e+00, -8.10915916e-01],\n", 512 | " [ 2.77119072e+01, 2.41466627e+01],\n", 513 | " [-1.70212069e+01, -1.68258705e+01],\n", 514 | " [ 5.04745524e+01, 4.66274134e+01],\n", 515 | " [ 1.48456545e+00, 2.31260440e+00],\n", 516 | " [-1.18330778e+01, -8.95333768e+00],\n", 517 | " [ 2.81931054e+01, 2.69260371e+01],\n", 518 | " [-4.21986388e+00, -1.41826508e+00],\n", 519 | " [ 1.46745912e+01, 1.20520962e+01],\n", 520 | " [-1.11813974e+01, -1.13292031e+01],\n", 521 | " [-6.02535429e+00, -5.27047080e+00],\n", 522 | " [ 3.07129641e+01, 2.91119306e+01],\n", 523 | " [ 5.46496851e+00, 4.28464672e-02],\n", 524 | " [ 2.59619746e+01, 2.00919997e+01],\n", 525 | " [ 3.49669991e+00, 1.86626270e-01],\n", 526 | " [ 1.47507462e+01, 1.03204859e+01],\n", 527 | " [ 3.67698077e+01, 3.48365664e+01],\n", 528 | " [ 5.50672610e+00, 4.63873391e+00],\n", 529 | " [ 2.72345918e+01, 1.94359379e+01],\n", 530 | " [ 1.89310805e+01, 1.55254734e+01],\n", 531 | " [ 1.24083762e+01, 6.71134780e+00],\n", 532 | " [-6.70397101e+00, -2.09137370e-01],\n", 533 | " [-2.06016817e+01, -1.86689633e+01],\n", 534 | " [-1.27942259e+01, -9.91056303e+00],\n", 535 | " [ 1.77919844e+01, 1.51883790e+01],\n", 536 | " [ 9.71692034e+00, 4.06146703e+00],\n", 537 | " [-2.39694407e+00, -1.79774750e+00],\n", 538 | " [ 1.05421436e+01, 9.23487406e+00],\n", 539 | " [ 1.22045593e+00, -1.07875268e+00],\n", 540 | " [-1.07546228e+01, -4.74439361e+00],\n", 541 | " [-4.81063247e+00, -8.78817245e+00],\n", 542 | " [ 2.31450768e+01, 1.91748792e+01],\n", 543 | " [-4.18048254e-02, 4.66992569e+00],\n", 544 | " [-4.84710941e+01, -4.25253861e+01],\n", 545 | " [ 1.08864775e+01, 9.17476425e+00],\n", 546 | " [-3.27788678e+01, -2.98826665e+01],\n", 547 | " [-3.54064472e+00, -4.09085475e+00],\n", 548 | " [ 1.12191320e+01, 1.10812952e+01],\n", 549 | " [ 1.43260242e+01, 1.37793678e+01],\n", 550 | " [ 6.55637768e+00, 5.68611171e+00],\n", 551 | " [-1.11058627e+01, -6.48561277e+00],\n", 552 | " [ 7.85651786e+00, 8.00218141e+00],\n", 553 | " [ 4.46217704e+00, 1.14376013e+00],\n", 554 | " [ 3.29265255e+01, 3.49590882e+01],\n", 555 | " [ 4.85949912e+01, 4.11315565e+01],\n", 556 | " [ 1.19354415e+01, 9.37292040e+00],\n", 557 | " [ 3.73828496e+01, 2.93836980e+01],\n", 558 | " [ 3.30241330e+01, 3.16625772e+01],\n", 559 | " [ 1.03462408e+01, -2.98447923e-02],\n", 560 | " [ 1.71716305e+01, 1.27985762e+01],\n", 561 | " [ 2.39301432e+01, 1.67916835e+01],\n", 562 | " [-6.80227926e+00, -4.66061176e+00],\n", 563 | " [ 1.11362317e+01, 9.57382673e+00],\n", 564 | " [-1.40885447e+01, -1.69857739e+01],\n", 565 | " [ 7.84001035e+00, -1.13391887e+00],\n", 566 | " [ 1.29158828e+01, 1.19344714e+01],\n", 567 | " [-1.06241773e+01, -1.44117050e+01],\n", 568 | " [ 3.34716804e+01, 2.83302128e+01],\n", 569 | " [-1.55024300e+01, -4.42891211e+00],\n", 570 | " [-3.48955183e+01, -3.03897814e+01],\n", 571 | " [ 4.33012619e+00, -1.58450917e+00],\n", 572 | " [-6.73299617e+00, -3.80308113e+00],\n", 573 | " [-2.35171223e+00, -3.40258619e+00],\n", 574 | " [ 2.16014984e+01, 2.05111530e+01],\n", 575 | " [-1.14591606e+01, -7.76235530e+00],\n", 576 | " [-7.04712496e+00, -6.03325693e+00],\n", 577 | " [-9.36561770e+00, -1.05784975e+01],\n", 578 | " [ 1.51976132e+01, 1.75157483e+01],\n", 579 | " [-5.27283770e+00, -7.32338430e+00],\n", 580 | " [ 8.84887775e+00, 2.55961507e+00],\n", 581 | " [ 2.90433598e+01, 3.13938059e+01],\n", 582 | " [-4.87162731e+00, -1.53659872e+00],\n", 583 | " [ 7.03312019e+00, 1.35123136e+01],\n", 584 | " [ 6.21913815e+00, 7.58283931e+00],\n", 585 | " [ 1.76485937e+01, 1.86315969e+01],\n", 586 | " [ 2.07231446e+01, 1.73708759e+01],\n", 587 | " [-1.72123582e+01, -1.38098575e+01],\n", 588 | " [ 1.30620030e+01, 9.51586713e+00],\n", 589 | " [ 2.05136987e+01, 1.57194550e+01],\n", 590 | " [-1.11707578e+01, -1.03694898e+01],\n", 591 | " [ 1.24416031e+01, 1.70158000e+01],\n", 592 | " [-1.14513667e+01, -1.00988180e+01],\n", 593 | " [ 3.96631849e-01, 1.83286619e-01],\n", 594 | " [-1.13900816e+01, -1.51945068e+01],\n", 595 | " [ 2.78301418e+01, 2.35278376e+01],\n", 596 | " [-2.68684473e-01, -2.30994163e+00],\n", 597 | " [-5.79886629e+00, -6.99356703e+00],\n", 598 | " [ 2.65129618e+01, 2.79829307e+01],\n", 599 | " [-3.11016922e+01, -2.83476668e+01],\n", 600 | " [-3.29337223e+01, -3.42038410e+01],\n", 601 | " [-7.92670029e+00, -8.73191611e+00],\n", 602 | " [-2.13186348e+01, -1.76976884e+01],\n", 603 | " [ 7.22327839e+00, 9.56145219e+00],\n", 604 | " [ 2.51860980e+01, 2.35128088e+01],\n", 605 | " [-9.75730761e+00, -8.19911689e+00],\n", 606 | " [ 1.87096854e+01, 2.08531999e+01],\n", 607 | " [ 2.07770950e+01, 1.73692478e+01],\n", 608 | " [ 9.11028756e+00, 1.49462636e+01],\n", 609 | " [ 1.68582071e+01, 1.34435252e+01],\n", 610 | " [ 2.84208726e+01, 2.80328979e+01],\n", 611 | " [-1.52401652e+01, -1.42386803e+01],\n", 612 | " [-9.38910802e+00, -9.95319600e+00],\n", 613 | " [ 5.15012713e+00, 6.99167391e-01],\n", 614 | " [-3.92883724e+01, -3.68248844e+01],\n", 615 | " [ 1.87737433e+01, 1.72472071e+01],\n", 616 | " [ 3.44181087e+01, 3.48890706e+01],\n", 617 | " [-3.26906582e+01, -3.06290054e+01],\n", 618 | " [-2.40423520e+01, -1.45808145e+01],\n", 619 | " [ 2.13756242e+00, 5.27183576e+00],\n", 620 | " [ 4.90009592e+00, 5.61565090e+00],\n", 621 | " [ 4.54859419e+01, 3.76886129e+01],\n", 622 | " [ 2.31324656e+01, 2.85480940e+01],\n", 623 | " [ 3.24232398e+01, 2.99045715e+01],\n", 624 | " [ 4.88183641e+00, 3.50497753e+00],\n", 625 | " [ 9.65033120e+00, 2.89810642e+00],\n", 626 | " [ 3.12306314e+01, 2.94777631e+01],\n", 627 | " [ 1.59728878e+01, 1.06858864e+01],\n", 628 | " [ 1.20997705e+00, 5.95213548e-01],\n", 629 | " [ 1.33539038e+01, 1.75907486e+01],\n", 630 | " [ 2.62493437e+00, 6.53876886e-01],\n", 631 | " [-7.84884410e-01, 3.13612900e+00],\n", 632 | " [-2.36139030e+00, -1.67070765e+00],\n", 633 | " [ 3.12325596e+00, 3.24636605e-02],\n", 634 | " [ 2.39675710e+01, 2.50891188e+01],\n", 635 | " [-3.65625038e+00, -4.68218168e+00],\n", 636 | " [ 6.93704714e+00, 8.36365197e+00],\n", 637 | " [-1.56590115e+01, -1.38351988e+01],\n", 638 | " [-3.86054744e+00, -7.50929234e+00],\n", 639 | " [ 3.56766317e+00, 4.16903836e+00],\n", 640 | " [ 6.94816559e+00, 4.98694702e-01],\n", 641 | " [ 1.98748626e+01, 1.86915632e+01],\n", 642 | " [ 9.63694712e+00, 5.44322195e+00],\n", 643 | " [ 6.35608252e+00, 7.74252966e+00],\n", 644 | " [ 2.29898145e+01, 1.74060334e+01],\n", 645 | " [ 2.21190507e+01, 1.57930569e+01],\n", 646 | " [-7.53326886e+00, -1.15775530e+01],\n", 647 | " [-1.41301319e+01, -1.06411886e+01],\n", 648 | " [-1.40032833e+01, -1.28802892e+01],\n", 649 | " [ 2.14106292e+01, 1.91283181e+01],\n", 650 | " [ 5.16093055e-01, 3.61359753e+00],\n", 651 | " [-1.72316863e+00, 3.20182049e+00],\n", 652 | " [ 3.89969272e+00, 6.56362203e+00],\n", 653 | " [ 2.11949427e+01, 1.98445588e+01],\n", 654 | " [ 4.79663284e+00, 7.20981922e+00]])" 655 | ] 656 | }, 657 | "execution_count": 474, 658 | "metadata": {}, 659 | "output_type": "execute_result" 660 | } 661 | ], 662 | "source": [ 663 | "y_test" 664 | ] 665 | }, 666 | { 667 | "cell_type": "markdown", 668 | "id": "628499f2", 669 | "metadata": {}, 670 | "source": [ 671 | "### Chapter 11 - NLP data" 672 | ] 673 | }, 674 | { 675 | "cell_type": "code", 676 | "execution_count": null, 677 | "id": "f2ac5b5a", 678 | "metadata": {}, 679 | "outputs": [], 680 | "source": [ 681 | "df = pd.read_csv('data/manga.csv') " 682 | ] 683 | }, 684 | { 685 | "cell_type": "code", 686 | "execution_count": null, 687 | "id": "e3f61a77", 688 | "metadata": {}, 689 | "outputs": [], 690 | "source": [ 691 | "# Check gender indicator\n", 692 | "df['gender_indicator'].unique()" 693 | ] 694 | }, 695 | { 696 | "cell_type": "code", 697 | "execution_count": null, 698 | "id": "2df35ef5", 699 | "metadata": {}, 700 | "outputs": [], 701 | "source": [ 702 | "# Produce female avatar indicator\n", 703 | "avatar = []\n", 704 | "\n", 705 | "for i in df['gender_indicator']:\n", 706 | " if i:\n", 707 | " avatar.append(np.random.choice([0, 1], p=[.15, .85]))\n", 708 | " else:\n", 709 | " avatar.append(np.random.choice([0, 1], p=[.97, .03]))\n", 710 | " \n", 711 | "df['female_avatar'] = avatar\n", 712 | "\n", 713 | "# Sanity\n", 714 | "df.groupby('gender_indicator')['female_avatar'].mean()" 715 | ] 716 | }, 717 | { 718 | "cell_type": "code", 719 | "execution_count": null, 720 | "id": "d33d6722", 721 | "metadata": {}, 722 | "outputs": [], 723 | "source": [ 724 | "# Produce stereotype indicator\n", 725 | "df['love_indicator'] = df['text'].str.contains('love|roman').astype('int')" 726 | ] 727 | }, 728 | { 729 | "cell_type": "code", 730 | "execution_count": null, 731 | "id": "1ba8a129", 732 | "metadata": {}, 733 | "outputs": [], 734 | "source": [ 735 | "# Add photo to the post at random\n", 736 | "df['has_photo'] = np.random.choice([0, 1], size=df.shape[0])" 737 | ] 738 | }, 739 | { 740 | "cell_type": "code", 741 | "execution_count": null, 742 | "id": "e2dcbd14", 743 | "metadata": {}, 744 | "outputs": [], 745 | "source": [ 746 | "# Produce the outcome\n", 747 | "noise = np.random.normal(0.01, .05, size=df.shape[0])\n", 748 | "probas = .8 + 0.1 * df['love_indicator'] * 1.2 * (0.1 + df['has_photo']) - 0.7 * df['female_avatar'] + noise\n", 749 | "probas = np.clip(probas, 0, 1)\n", 750 | "\n", 751 | "upvote = []\n", 752 | "\n", 753 | "for p in probas:\n", 754 | " upvote_ = np.random.choice([0, 1], p=[1 - p, p])\n", 755 | " upvote.append(upvote_)\n", 756 | " \n", 757 | "df['upvote'] = likes" 758 | ] 759 | }, 760 | { 761 | "cell_type": "code", 762 | "execution_count": null, 763 | "id": "02c2eee9", 764 | "metadata": {}, 765 | "outputs": [], 766 | "source": [ 767 | "df.drop(['gender_indicator', 'topic', 'love_indicator', 'author'], axis=1).to_csv('data/manga_processed.csv', index=False)" 768 | ] 769 | } 770 | ], 771 | "metadata": { 772 | "kernelspec": { 773 | "display_name": "Python [conda env:causal_book_py38]", 774 | "language": "python", 775 | "name": "conda-env-causal_book_py38-py" 776 | }, 777 | "language_info": { 778 | "codemirror_mode": { 779 | "name": "ipython", 780 | "version": 3 781 | }, 782 | "file_extension": ".py", 783 | "mimetype": "text/x-python", 784 | "name": "python", 785 | "nbconvert_exporter": "python", 786 | "pygments_lexer": "ipython3", 787 | "version": "3.8.13" 788 | } 789 | }, 790 | "nbformat": 4, 791 | "nbformat_minor": 5 792 | } 793 | -------------------------------------------------------------------------------- /Extras_02__Additional_computations.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "id": "1eaa963c", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import numpy as np\n", 11 | "import pandas as pd\n", 12 | "from scipy import stats\n", 13 | "\n", 14 | "import dowhy\n", 15 | "from dowhy import CausalModel\n", 16 | "\n", 17 | "import matplotlib.pyplot as plt\n", 18 | "plt.style.use('fivethirtyeight')" 19 | ] 20 | }, 21 | { 22 | "cell_type": "code", 23 | "execution_count": 2, 24 | "id": "fc74dab6", 25 | "metadata": {}, 26 | "outputs": [], 27 | "source": [ 28 | "COLORS = [\n", 29 | " '#00B0F0',\n", 30 | " '#FF0000'\n", 31 | "]" 32 | ] 33 | }, 34 | { 35 | "cell_type": "markdown", 36 | "id": "df565036", 37 | "metadata": {}, 38 | "source": [ 39 | "# Additional computations" 40 | ] 41 | }, 42 | { 43 | "cell_type": "markdown", 44 | "id": "e42ea37a", 45 | "metadata": {}, 46 | "source": [ 47 | "## Chapter 01 / Chapter 09" 48 | ] 49 | }, 50 | { 51 | "cell_type": "markdown", 52 | "id": "d28e80e8", 53 | "metadata": {}, 54 | "source": [ 55 | "### Solving Simpson's paradox with IPW" 56 | ] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "execution_count": 3, 61 | "id": "865c3739", 62 | "metadata": {}, 63 | "outputs": [ 64 | { 65 | "name": "stdout", 66 | "output_type": "stream", 67 | "text": [ 68 | "propensity_score_weighting\n", 69 | "0.04811580602068774\n" 70 | ] 71 | }, 72 | { 73 | "name": "stderr", 74 | "output_type": "stream", 75 | "text": [ 76 | "C:\\Users\\aleks\\anaconda3\\envs\\causal_book_py38\\lib\\site-packages\\sklearn\\utils\\validation.py:1111: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", 77 | " y = column_or_1d(y, warn=True)\n" 78 | ] 79 | } 80 | ], 81 | "source": [ 82 | "pd.read_csv('data/ch_01_drug_data.csv')\n", 83 | "\n", 84 | "gender = [1] * (24 + 56 + 17 + 25) + [0] * (3 + 39 + 6 +74)\n", 85 | "clot = [1] * 24 + [0] * 56 + [1] * 17 + [0] * 25 + [1] * 3 + [0] * 39 + [1] * 6 + [0] * 74\n", 86 | "drug = [0] * (24 + 56) + [1] * (17 + 25) + [0] * 42 + [1] * 80\n", 87 | "\n", 88 | "drug_data = pd.DataFrame(dict(\n", 89 | " gender=gender,\n", 90 | " clot=clot,\n", 91 | " drug=drug\n", 92 | "))\n", 93 | "\n", 94 | "# Construct the graph (the graph is constant for all iterations)\n", 95 | "nodes_drug = ['drug', 'clot', 'gender']\n", 96 | "edges_drug = [\n", 97 | " ('drug', 'clot'),\n", 98 | " ('gender', 'drug'),\n", 99 | " ('gender', 'clot')\n", 100 | "]\n", 101 | "\n", 102 | "# Generate the GML graph\n", 103 | "gml_string_drug = 'graph [directed 1\\n'\n", 104 | "\n", 105 | "for node in nodes_drug:\n", 106 | " gml_string_drug += f'\\tnode [id \"{node}\" label \"{node}\"]\\n'\n", 107 | "\n", 108 | "for edge in edges_drug:\n", 109 | " gml_string_drug += f'\\tedge [source \"{edge[0]}\" target \"{edge[1]}\"]\\n'\n", 110 | " \n", 111 | "gml_string_drug += ']'\n", 112 | "\n", 113 | "# Instantiate the CausalModel\n", 114 | "model_drug = CausalModel(\n", 115 | " data=drug_data,\n", 116 | " treatment='drug',\n", 117 | " outcome='clot',\n", 118 | " graph=gml_string_drug\n", 119 | ")\n", 120 | "\n", 121 | "# Identify effect\n", 122 | "estimand_drug = model_drug.identify_effect()\n", 123 | "\n", 124 | "# Get estimate (IPW weighting)\n", 125 | "estimate_drug = model_drug.estimate_effect(\n", 126 | " identified_estimand=estimand_drug,\n", 127 | " method_name='backdoor.propensity_score_weighting',\n", 128 | " target_units='ate'\n", 129 | ")\n", 130 | "\n", 131 | "print(estimate_drug.value)" 132 | ] 133 | }, 134 | { 135 | "cell_type": "code", 136 | "execution_count": null, 137 | "id": "f85982a5", 138 | "metadata": {}, 139 | "outputs": [], 140 | "source": [] 141 | } 142 | ], 143 | "metadata": { 144 | "kernelspec": { 145 | "display_name": "Python [conda env:causal_book_py38]", 146 | "language": "python", 147 | "name": "conda-env-causal_book_py38-py" 148 | }, 149 | "language_info": { 150 | "codemirror_mode": { 151 | "name": "ipython", 152 | "version": 3 153 | }, 154 | "file_extension": ".py", 155 | "mimetype": "text/x-python", 156 | "name": "python", 157 | "nbconvert_exporter": "python", 158 | "pygments_lexer": "ipython3", 159 | "version": "3.8.13" 160 | } 161 | }, 162 | "nbformat": 4, 163 | "nbformat_minor": 5 164 | } 165 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 Packt 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 |

Machine Learning Summit 2025

2 | 3 | ## Machine Learning Summit 2025 4 | **Bridging Theory and Practice: ML Solutions for Today’s Challenges** 5 | 6 | 3 days, 20+ experts, and 25+ tech sessions and talks covering critical aspects of: 7 | - **Agentic and Generative AI** 8 | - **Applied Machine Learning in the Real World** 9 | - **ML Engineering and Optimization** 10 | 11 | 👉 [Book your ticket now >>](https://packt.link/mlsumgh) 12 | 13 | --- 14 | 15 | ## Join Our Newsletters 📬 16 | 17 | ### DataPro 18 | *The future of AI is unfolding. Don’t fall behind.* 19 | 20 |

DataPro QR

21 | 22 | Stay ahead with [**DataPro**](https://landing.packtpub.com/subscribe-datapronewsletter/?link_from_packtlink=yes), the free weekly newsletter for data scientists, AI/ML researchers, and data engineers. 23 | From trending tools like **PyTorch**, **scikit-learn**, **XGBoost**, and **BentoML** to hands-on insights on **database optimization** and real-world **ML workflows**, you’ll get what matters, fast. 24 | 25 | > Stay sharp with [DataPro](https://landing.packtpub.com/subscribe-datapronewsletter/?link_from_packtlink=yes). Join **115K+ data professionals** who never miss a beat. 26 | 27 | --- 28 | 29 | ### BIPro 30 | *Business runs on data. Make sure yours tells the right story.* 31 | 32 |

BIPro QR

33 | 34 | [**BIPro**](https://landing.packtpub.com/subscribe-bipro-newsletter/?link_from_packtlink=yes) is your free weekly newsletter for BI professionals, analysts, and data leaders. 35 | Get practical tips on **dashboarding**, **data visualization**, and **analytics strategy** with tools like **Power BI**, **Tableau**, **Looker**, **SQL**, and **dbt**. 36 | 37 | > Get smarter with [BIPro](https://landing.packtpub.com/subscribe-bipro-newsletter/?link_from_packtlink=yes). Trusted by **35K+ BI professionals**, see what you’re missing. 38 | 39 | 40 | 41 | # Causal Inference and Discovery in Python 42 | 43 | Causal Inference and Discovery in Python 44 | 45 | This is the code repository for [Causal Inference and Discovery in Python](https://www.packtpub.com/product/causal-inference-and-discovery-in-python/9781804612989), published by Packt. 46 | 47 | **Unlock the secrets of modern causal machine learning with DoWhy, EconML, PyTorch and more** 48 | 49 | ## What is this book about? 50 | 51 | Causal methods present unique challenges compared to traditional machine learning and statistics. Learning causality can be challenging, but it offers distinct advantages that elude a purely statistical mindset. Causal Inference and Discovery in Python helps you unlock the potential of causality. 52 | 53 | You’ll start with basic motivations behind causal thinking and a comprehensive introduction to Pearlian causal concepts, such as structural causal models, interventions, counterfactuals, and more. Each concept is accompanied by a theoretical explanation and a set of practical exercises with Python code. 54 | 55 | Next, you’ll dive into the world of causal effect estimation, consistently progressing towards modern machine learning methods. Step-by-step, you’ll discover Python causal ecosystem and harness the power of cutting-edge algorithms. You’ll further explore the mechanics of how “causes leave traces” and compare the main families of causal discovery algorithms. 56 | 57 | The final chapter gives you a broad outlook into the future of causal AI where we examine challenges and opportunities and provide you with a comprehensive list of resources to learn more. 58 | 59 | This book covers the following exciting features: 60 | * Master the fundamental concepts of causal inference 61 | * Decipher the mysteries of structural causal models 62 | * Unleash the power of the 4-step causal inference process in Python 63 | * Explore advanced uplift modeling techniques 64 | * Unlock the secrets of modern causal discovery using Python 65 | * Use causal inference for social impact and community benefit 66 | 67 | If you feel this book is for you, get your [copy](https://www.amazon.com/Causal-Inference-Discovery-Python-learning/dp/1804612987/ref=sr_1_1?keywords=Causal+Inference+and+Discovery+in+Python&s=books&sr=1-1) today! 68 | 69 | 70 | ## Instructions and Navigations 71 | All of the code is organized into folders. 72 | 73 | The code will look like the following: 74 | ``` 75 | preds = causal_bert.inference( 76 | texts=df['text'], 77 | confounds=df['has_photo'], 78 | )[0] 79 | ``` 80 | 81 | **Following is what you need for this book:** 82 | 83 | This book is for machine learning engineers, data scientists, and machine learning researchers looking to extend their data science toolkit and explore causal machine learning. It will also help developers familiar with causality who have worked in another technology and want to switch to Python, and data scientists with a history of working with traditional causality who want to learn causal machine learning. It’s also a must-read for tech-savvy entrepreneurs looking to build a competitive edge for their products and go beyond the limitations of traditional machine learning. 84 | 85 | With the following software and hardware list you can run all code files present in the book (Chapter 1-15). 86 | 87 | ### Software and Hardware List 88 | 89 | | Chapter | Software required | OS required | 90 | | -------- | -------------------------------------------------------------------------------------| -----------------------------------| 91 | | 1-15 | Python 3.9 | Windows macOS, or Linux | 92 | | 1-15 | DoWhy 0.8 | Windows, macOS, or Linux | 93 | | 1-15 | EconML 0.12.0 | Windows, macOS, or Linux | 94 | | 1-15 | CATENets 0.2.3 | Windows, macOS, or Linux | 95 | | 1-15 | gCastle 1.0.3 | Windows, macOS, or Linux | 96 | | 1-15 | Causica 0.2.0 | Windows, macOS, or Linux | 97 | | 1-15 | Causal-learn 0.1.3.3 | Windows, macOS, or Linux | 98 | | 1-15 | Transformers 4.24.0 | Windows, macOS, or Linux | 99 | 100 | 101 | ## Join our Discord server Coding 102 | 103 | Join our Discord community to meet like-minded people and learn alongside more than 2000 members at [Discord](https://packt.link/infer) Coding 104 | 105 | 106 | ### Related products 107 | * Hands-On Graph Neural Networks Using Python [[Packt]](https://www.packtpub.com/product/hands-on-graph-neural-networks-using-python/9781804617526) [[Amazon]](https://www.amazon.com/Hands-Graph-Neural-Networks-Python/dp/1804617520/ref=sr_1_1?keywords=Hands-On+Graph+Neural+Networks+Using+Python&s=books&sr=1-1) 108 | 109 | * Applying Math with Python - Second Edition [[Packt]](https://www.packtpub.com/product/applying-math-with-python-second-edition/9781804618370) [[Amazon]](https://www.amazon.com/Applying-Math-Python-real-world-computational/dp/1804618373/ref=sr_1_1?keywords=Applying+Math+with+Python+-+Second+Edition&s=books&sr=1-1) 110 | 111 | ## Get to Know the Author 112 | [**Aleksander Molak**](https://www.linkedin.com/in/aleksandermolak/) is a Machine Learning Researcher and Consultant who gained experience working with Fortune 100, Fortune 500, and Inc. 5000 companies across Europe, the USA, and Israel, designing and building large-scale machine learning systems. On a mission to democratize causality for businesses and machine learning practitioners, Aleksander is a prolific writer, creator, and international speaker. As a co-founder of Lespire, an innovative provider of AI and machine learning training for corporate teams, Aleksander is committed to empowering businesses to harness the full potential of cutting-edge technologies that allow them to stay ahead of the curve. 113 | He's the host of the Causal AI-centered [Causal Bandits Podcast](https://causalbanditspodcast.com/). 114 | 115 | 116 | 117 | 118 | # Note from the Author: 119 | 120 | ## Environment installation 121 | 1. See the section **Using `graphviz` and GPU** below 122 | 123 | 2. To install the basic environment run: `conda env create -f causal_book_py39_cuda117.yml` 124 | 125 | 3. To install the environment for notebook `Chapter_11.2.ipynb` run: `conda env create -f causal-pymc.yml` 126 | 127 | **NOTE:** We added an experimental environment for Apple M1 as suggested by @ferrari-leo [here](https://github.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/issues/8). This environment hasn't been thoroughly tested so please use it at your own risk. 128 | 129 | ## Selecting the kernel 130 | After a successful installation of the environment, open your notebook and select the kernel `causal_book_py39_cuda117` 131 | 132 | For notebook `Chapter_11.2.ipynb` change kernel to `causal-pymc` 133 | 134 | ## Using `graphviz` and GPU 135 | 136 | **Note**: Depending on your system settings, you might need to install `graphviz` manually in order to recreate the graph plots in the code. 137 | Check https://pypi.org/project/graphviz/ for instructions 138 | specific to your operating system. 139 | 140 | **Note 2**: To use GPU you'll need to install CUDA 11.7 drivers. 141 | This can be done here: https://developer.nvidia.com/cuda-11-7-0-download-archive 142 | 143 | ## Citation 144 | 145 | ### BibTeX 146 | ```{bibtex} 147 | @book{Molak2023, 148 | title={Causal Inference and Discovery in Python: Unlock the secrets of modern causal machine learning with DoWhy, EconML, PyTorch and more}, 149 | author={Molak, Aleksander}, 150 | publisher={Packt Publishing}, 151 | address={Birmingham}, 152 | edition={1.}, 153 | year={2023}, 154 | isbn={1804612987}, 155 | note={\url{https://amzn.to/3RebWzn}} 156 | } 157 | ``` 158 | 159 | ### APA 160 | ``` 161 | Molak, A. (2023). Causal Inference and Discovery in Python: Unlock the secrets of modern causal machine learning with DoWhy, EconML, PyTorch and more. Packt Publishing. 162 | ``` 163 | 164 | ## ‼️ Known mistakes // errata 165 | For known errors and corrections check: 166 | 167 | * [Books purchased before ~12:00 PM on June 13, 2023](https://github.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/blob/main/errata/Errata%20-%20Early%20Print%20(ordered%20before%20June%2013%202023).ipynb) 168 | 169 | * [Books purchased after ~12:00 PM on June 13, 2023](https://github.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/blob/main/errata/Errata%20-%20Non-Early%20Print%20(ordered%20after%20June%2013%202023).ipynb) 170 | 171 | If you spotted a mistake, let us know at book(at)causalpython.io or just open an **issue** in this repo. Thank you 🙏🏼 172 | -------------------------------------------------------------------------------- /causal-pymc.yml: -------------------------------------------------------------------------------- 1 | name: causal-pymc 2 | channels: 3 | - conda-forge 4 | - defaults 5 | dependencies: 6 | - python=3.9 7 | - numba 8 | - pip 9 | - nb_conda 10 | - arviz 11 | - matplotlib 12 | - pandas 13 | - scipy 14 | - numpy 15 | - statsmodels 16 | - pydot 17 | - tqdm 18 | - ipywidgets 19 | - pip: 20 | - pymc>=4 21 | - CausalPy==0.0.8 -------------------------------------------------------------------------------- /causal_book_py39_apple_m1_(experimental-by-ferrari-leo).txt: -------------------------------------------------------------------------------- 1 | name: causal_book_py39 2 | channels: 3 | - defaults 4 | - conda-forge 5 | dependencies: 6 | - abseil-cpp=20230802.0=h313beb8_2 7 | - aiohttp=3.9.0=py39h80987f9_0 8 | - aiosignal=1.2.0=pyhd3eb1b0_0 9 | - anyio=3.5.0=py39hca03da5_0 10 | - appnope=0.1.2=py39hca03da5_1001 11 | - argon2-cffi=21.3.0=pyhd3eb1b0_0 12 | - argon2-cffi-bindings=21.2.0=py39h1a28f6b_0 13 | - arrow-cpp=11.0.0=hc7aafb3_2 14 | - asttokens=2.0.5=pyhd3eb1b0_0 15 | - async-timeout=4.0.3=py39hca03da5_0 16 | - attrs=23.1.0=py39hca03da5_0 17 | - aws-c-common=0.6.8=h80987f9_1 18 | - aws-c-event-stream=0.1.6=h313beb8_6 19 | - aws-checksums=0.1.11=h80987f9_2 20 | - aws-sdk-cpp=1.8.185=ha71a6ea_1 21 | - backcall=0.2.0=pyhd3eb1b0_0 22 | - beautifulsoup4=4.12.2=py39hca03da5_0 23 | - blas=1.0=openblas 24 | - bleach=4.1.0=pyhd3eb1b0_0 25 | - boost-cpp=1.82.0=h48ca7d4_2 26 | - bottleneck=1.3.5=py39heec5a64_0 27 | - brotli=1.0.9=h1a28f6b_7 28 | - brotli-bin=1.0.9=h1a28f6b_7 29 | - brotli-python=1.0.9=py39hc377ac9_7 30 | - bzip2=1.0.8=h620ffc9_4 31 | - c-ares=1.19.1=h80987f9_0 32 | - ca-certificates=2023.12.12=hca03da5_0 33 | - certifi=2023.11.17=py39hca03da5_0 34 | - cffi=1.16.0=py39h80987f9_0 35 | - charset-normalizer=2.0.4=pyhd3eb1b0_0 36 | - comm=0.1.2=py39hca03da5_0 37 | - contourpy=1.2.0=py39h48ca7d4_0 38 | - cryptography=41.0.7=py39hd4332d6_0 39 | - cycler=0.11.0=pyhd3eb1b0_0 40 | - datasets=2.12.0=py39hca03da5_0 41 | - debugpy=1.6.7=py39h313beb8_0 42 | - decorator=5.1.1=pyhd3eb1b0_0 43 | - defusedxml=0.7.1=pyhd3eb1b0_0 44 | - dill=0.3.6=py39hca03da5_0 45 | - entrypoints=0.4=py39hca03da5_0 46 | - exceptiongroup=1.2.0=py39hca03da5_0 47 | - executing=0.8.3=pyhd3eb1b0_0 48 | - filelock=3.13.1=py39hca03da5_0 49 | - fonttools=4.25.0=pyhd3eb1b0_0 50 | - freetype=2.12.1=h1192e45_0 51 | - frozenlist=1.4.0=py39h80987f9_0 52 | - fsspec=2023.10.0=py39hca03da5_0 53 | - future=0.18.3=py39hca03da5_0 54 | - gflags=2.2.2=hc377ac9_0 55 | - giflib=5.2.1=h80987f9_3 56 | - glog=0.5.0=hc377ac9_0 57 | - grpc-cpp=1.48.2=hc60591f_4 58 | - gtest=1.14.0=h48ca7d4_0 59 | - huggingface_hub=0.17.3=py39hca03da5_0 60 | - icu=73.1=h313beb8_0 61 | - idna=3.4=py39hca03da5_0 62 | - importlib-metadata=7.0.1=py39hca03da5_0 63 | - importlib_resources=6.1.1=py39hca03da5_1 64 | - ipykernel=6.25.0=py39h33ce5c2_0 65 | - ipython=8.15.0=py39hca03da5_0 66 | - ipython_genutils=0.2.0=pyhd3eb1b0_1 67 | - jedi=0.18.1=py39hca03da5_1 68 | - jinja2=3.1.2=py39hca03da5_0 69 | - jpeg=9e=h80987f9_1 70 | - jsonschema=4.19.2=py39hca03da5_0 71 | - jsonschema-specifications=2023.7.1=py39hca03da5_0 72 | - jupyter_client=7.4.9=py39hca03da5_0 73 | - jupyter_core=5.5.0=py39hca03da5_0 74 | - jupyter_server=1.23.4=py39hca03da5_0 75 | - jupyterlab_pygments=0.2.2=py39hca03da5_0 76 | - kiwisolver=1.4.4=py39h313beb8_0 77 | - krb5=1.20.1=hf3e1bf2_1 78 | - lcms2=2.12=hba8e193_0 79 | - lerc=3.0=hc377ac9_0 80 | - libboost=1.82.0=h0bc93f9_2 81 | - libbrotlicommon=1.0.9=h1a28f6b_7 82 | - libbrotlidec=1.0.9=h1a28f6b_7 83 | - libbrotlienc=1.0.9=h1a28f6b_7 84 | - libcurl=8.5.0=h3e2b118_0 85 | - libcxx=14.0.6=h848a8c0_0 86 | - libdeflate=1.17=h80987f9_1 87 | - libedit=3.1.20230828=h80987f9_0 88 | - libev=4.33=h1a28f6b_1 89 | - libevent=2.1.12=h02f6b3c_1 90 | - libffi=3.4.4=hca03da5_0 91 | - libgfortran=5.0.0=11_3_0_hca03da5_28 92 | - libgfortran5=11.3.0=h009349e_28 93 | - libiconv=1.16=h1a28f6b_2 94 | - libnghttp2=1.57.0=h62f6fdd_0 95 | - libopenblas=0.3.21=h269037a_0 96 | - libpng=1.6.39=h80987f9_0 97 | - libprotobuf=3.20.3=h514c7bf_0 98 | - libsodium=1.0.18=h1a28f6b_0 99 | - libssh2=1.10.0=h02f6b3c_2 100 | - libthrift=0.15.0=h73c2103_2 101 | - libtiff=4.5.1=h313beb8_0 102 | - libwebp=1.3.2=ha3663a8_0 103 | - libwebp-base=1.3.2=h80987f9_0 104 | - lightgbm=4.1.0=py39h313beb8_0 105 | - llvm-openmp=14.0.6=hc6e5704_0 106 | - lz4-c=1.9.4=h313beb8_0 107 | - markupsafe=2.1.3=py39h80987f9_0 108 | - matplotlib=3.8.0=py39hca03da5_0 109 | - matplotlib-base=3.8.0=py39h46d7db6_0 110 | - matplotlib-inline=0.1.6=py39hca03da5_0 111 | - mistune=2.0.4=py39hca03da5_0 112 | - multidict=6.0.4=py39h80987f9_0 113 | - multiprocess=0.70.14=py39hca03da5_0 114 | - munkres=1.1.4=py_0 115 | - nb_conda=2.2.1=py39hca03da5_1 116 | - nb_conda_kernels=2.3.1=py39hca03da5_0 117 | - nbclassic=1.0.0=py39hca03da5_0 118 | - nbclient=0.8.0=py39hca03da5_0 119 | - nbconvert=7.10.0=py39hca03da5_0 120 | - nbformat=5.9.2=py39hca03da5_0 121 | - ncurses=6.4=h313beb8_0 122 | - nest-asyncio=1.5.6=py39hca03da5_0 123 | - notebook=6.5.4=py39hca03da5_1 124 | - notebook-shim=0.2.3=py39hca03da5_0 125 | - numexpr=2.8.7=py39hecc3335_0 126 | - openjpeg=2.3.0=h7a6adac_2 127 | - openssl=3.0.12=h1a28f6b_0 128 | - orc=1.7.4=hdca1487_1 129 | - packaging=23.1=py39hca03da5_0 130 | - pandocfilters=1.5.0=pyhd3eb1b0_0 131 | - parso=0.8.3=pyhd3eb1b0_0 132 | - pexpect=4.8.0=pyhd3eb1b0_3 133 | - pickleshare=0.7.5=pyhd3eb1b0_1003 134 | - pillow=10.0.1=py39h3b245a6_0 135 | - pip=23.3.1=py39hca03da5_0 136 | - platformdirs=3.10.0=py39hca03da5_0 137 | - progressbar2=4.2.0=py39hca03da5_0 138 | - prometheus_client=0.14.1=py39hca03da5_0 139 | - prompt-toolkit=3.0.43=py39hca03da5_0 140 | - psutil=5.9.0=py39h1a28f6b_0 141 | - ptyprocess=0.7.0=pyhd3eb1b0_2 142 | - pure_eval=0.2.2=pyhd3eb1b0_0 143 | - pyarrow=11.0.0=py39hbfed03b_1 144 | - pycparser=2.21=pyhd3eb1b0_0 145 | - pygam=0.9.0=pyhd8ed1ab_2 146 | - pygments=2.15.1=py39hca03da5_1 147 | - pyopenssl=23.2.0=py39hca03da5_0 148 | - pyparsing=3.0.9=py39hca03da5_0 149 | - pysocks=1.7.1=py39hca03da5_0 150 | - python=3.9.18=hb885b13_0 151 | - python-dateutil=2.8.2=pyhd3eb1b0_0 152 | - python-fastjsonschema=2.16.2=py39hca03da5_0 153 | - python-tzdata=2023.3=pyhd3eb1b0_0 154 | - python-utils=3.3.3=py39hca03da5_0 155 | - python-xxhash=2.0.2=py39h1a28f6b_1 156 | - pytz=2023.3.post1=py39hca03da5_0 157 | - pyyaml=6.0.1=py39h80987f9_0 158 | - pyzmq=23.2.0=py39hc377ac9_0 159 | - re2=2022.04.01=hc377ac9_0 160 | - readline=8.2=h1a28f6b_0 161 | - referencing=0.30.2=py39hca03da5_0 162 | - regex=2023.10.3=py39h80987f9_0 163 | - requests=2.31.0=py39hca03da5_0 164 | - responses=0.13.3=pyhd3eb1b0_0 165 | - rpds-py=0.10.6=py39hf0e4da2_0 166 | - safetensors=0.4.0=py39h482802a_0 167 | - scipy=1.11.4=py39h20cbe94_0 168 | - seaborn=0.12.2=py39hca03da5_0 169 | - send2trash=1.8.2=py39hca03da5_0 170 | - setuptools=68.2.2=py39hca03da5_0 171 | - six=1.16.0=pyhd3eb1b0_1 172 | - snappy=1.1.10=h313beb8_1 173 | - sniffio=1.3.0=py39hca03da5_0 174 | - soupsieve=2.5=py39hca03da5_0 175 | - sqlite=3.41.2=h80987f9_0 176 | - stack_data=0.2.0=pyhd3eb1b0_0 177 | - terminado=0.17.1=py39hca03da5_0 178 | - tinycss2=1.2.1=py39hca03da5_0 179 | - tk=8.6.12=hb8d0fd4_0 180 | - tokenizers=0.13.2=py39h3dd52b7_1 181 | - tornado=6.3.3=py39h80987f9_0 182 | - tqdm=4.65.0=py39h86d0a89_0 183 | - traitlets=5.7.1=py39hca03da5_0 184 | - transformers=4.32.1=py39hca03da5_0 185 | - typing-extensions=4.9.0=py39hca03da5_1 186 | - typing_extensions=4.9.0=py39hca03da5_1 187 | - tzdata=2023d=h04d1e81_0 188 | - urllib3=1.26.18=py39hca03da5_0 189 | - utf8proc=2.6.1=h1a28f6b_0 190 | - wcwidth=0.2.5=pyhd3eb1b0_0 191 | - webencodings=0.5.1=py39hca03da5_1 192 | - websocket-client=0.58.0=py39hca03da5_4 193 | - wheel=0.41.2=py39hca03da5_0 194 | - xxhash=0.8.0=h1a28f6b_3 195 | - xz=5.4.5=h80987f9_0 196 | - yaml=0.2.5=h1a28f6b_0 197 | - yarl=1.9.3=py39h80987f9_0 198 | - zeromq=4.3.5=h313beb8_0 199 | - zipp=3.17.0=py39hca03da5_0 200 | - zlib=1.2.13=h5a0b063_0 201 | - zstd=1.5.5=hd90d995_0 202 | - pip: 203 | - absl-py==2.1.0 204 | - alembic==1.13.1 205 | - antlr4-python3-runtime==4.9.3 206 | - arviz==0.17.0 207 | - azure-common==1.1.28 208 | - azure-core==1.29.7 209 | - azure-identity==1.15.0 210 | - azure-mgmt-core==1.4.0 211 | - azure-storage-blob==12.19.0 212 | - azureml-mlflow==1.54.0.post1 213 | - blinker==1.7.0 214 | - cachetools==5.3.2 215 | - catenets==0.2.3 216 | - causal-learn==0.1.3.3 217 | - causalpy==0.2.0 218 | - causica==0.2.0 219 | - click==8.1.7 220 | - cloudpickle==3.0.0 221 | - cons==0.4.6 222 | - databricks-cli==0.18.0 223 | - dataclasses-json==0.5.14 224 | - docker==7.0.0 225 | - docstring-parser==0.15 226 | - dowhy==0.8 227 | - econml==0.14.1 228 | - etuples==0.3.9 229 | - fastprogress==1.0.3 230 | - flask==3.0.1 231 | - gcastle==1.0.3 232 | - gdown==5.0.0 233 | - gitdb==4.0.11 234 | - gitpython==3.1.41 235 | - google-auth==2.27.0 236 | - google-auth-oauthlib==1.2.0 237 | - grapl-causal==1.5.1 238 | - grpcio==1.60.0 239 | - gunicorn==21.2.0 240 | - h5netcdf==1.3.0 241 | - h5py==3.10.0 242 | - hydra-core==1.3.2 243 | - iniconfig==2.0.0 244 | - isodate==0.6.1 245 | - itsdangerous==2.1.2 246 | - jax==0.4.23 247 | - jaxlib==0.4.23 248 | - joblib==1.3.2 249 | - jsonargparse==4.27.3 250 | - jsonpickle==3.0.2 251 | - lightning-utilities==0.10.1 252 | - llvmlite==0.41.1 253 | - logical-unification==0.4.6 254 | - loguru==0.7.2 255 | - mako==1.3.0 256 | - markdown==3.5.2 257 | - markdown-it-py==3.0.0 258 | - marshmallow==3.20.2 259 | - mdurl==0.1.2 260 | - minikanren==1.0.3 261 | - ml-dtypes==0.3.2 262 | - mlflow==2.10.0 263 | - mlflow-skinny==2.10.0 264 | - mpmath==1.3.0 265 | - msal==1.26.0 266 | - msal-extensions==1.1.0 267 | - msrest==0.7.1 268 | - multipledispatch==1.0.0 269 | - mypy-extensions==1.0.0 270 | - networkx==2.8.7 271 | - numba==0.58.1 272 | - numpy==1.25.2 273 | - oauthlib==3.2.2 274 | - omegaconf==2.3.0 275 | - opt-einsum==3.3.0 276 | - pandas==1.5.3 277 | - patsy==0.5.6 278 | - pluggy==1.4.0 279 | - portalocker==2.8.2 280 | - protobuf==4.23.4 281 | - pyasn1==0.5.1 282 | - pyasn1-modules==0.3.0 283 | - pydot==2.0.0 284 | - pyjwt==2.8.0 285 | - pymc==5.10.3 286 | - pytensor==2.18.6 287 | - pytest==7.4.4 288 | - python-graphviz==0.20.1 289 | - pytorch-lightning==1.9.5 290 | - querystring-parser==1.2.4 291 | - requests-oauthlib==1.3.1 292 | - rich==13.7.0 293 | - rsa==4.9 294 | - scikit-learn==1.2.2 295 | - shap==0.41.0 296 | - slicer==0.0.7 297 | - smmap==5.0.1 298 | - sparse==0.15.1 299 | - sqlalchemy==2.0.25 300 | - sqlparse==0.4.4 301 | - statsmodels==0.14.1 302 | - sympy==1.12 303 | - tabulate==0.9.0 304 | - tensorboard==2.15.1 305 | - tensorboard-data-server==0.7.2 306 | - tensorboardx==2.6.2.2 307 | - tensordict==0.1.2 308 | - threadpoolctl==3.2.0 309 | - tomli==2.0.1 310 | - toolz==0.12.1 311 | - torch==2.1.2 312 | - torchmetrics==1.3.0.post0 313 | - types-pyyaml==6.0.12.12 314 | - typeshed-client==2.4.0 315 | - typing-inspect==0.9.0 316 | - werkzeug==3.0.1 317 | - xarray==2024.1.1 318 | - xarray-einstats==0.7.0 319 | -------------------------------------------------------------------------------- /causal_book_py39_cuda117.yml: -------------------------------------------------------------------------------- 1 | name: causal_book_py39_cuda117 2 | channels: 3 | - defaults 4 | - conda-forge 5 | - nvidia 6 | - pytorch 7 | dependencies: 8 | - pip 9 | - python=3.9 10 | - nb_conda 11 | - numpy 12 | - pandas 13 | - matplotlib 14 | - seaborn 15 | - pytorch 16 | - pytorch-cuda=11.7 17 | - pygam 18 | - lightgbm 19 | - transformers 20 | - pip: 21 | - econml 22 | - networkx==2.8.7 23 | - dowhy==0.8 24 | - gcastle==1.0.3 25 | - graphviz 26 | - causal-learn==0.1.3.3 27 | - grapl-causal==1.5.1 28 | - causica==0.2.0 29 | - catenets==0.2.3 30 | -------------------------------------------------------------------------------- /causal_model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/causal_model.png -------------------------------------------------------------------------------- /data/ch_01_drug_data.csv: -------------------------------------------------------------------------------- 1 | Drug,A,A,B,B 2 | Clot,1,0,1,0 3 | Female,24,56,17,25 4 | Male,3,39,6,74 5 | Total,27,95,23,99 6 | -------------------------------------------------------------------------------- /data/gt_social_media_data.csv: -------------------------------------------------------------------------------- 1 | date,twitter,linkedin,tiktok,instagram 2 | 2022-05-15,55,9,23,59 3 | 2022-05-16,54,18,20,59 4 | 2022-05-17,54,20,23,57 5 | 2022-05-18,54,20,21,55 6 | 2022-05-19,49,23,21,52 7 | 2022-05-20,46,18,22,56 8 | 2022-05-21,51,9,23,58 9 | 2022-05-22,47,9,27,59 10 | 2022-05-23,45,19,21,58 11 | 2022-05-24,49,21,23,53 12 | 2022-05-25,55,21,21,61 13 | 2022-05-26,53,19,22,68 14 | 2022-05-27,52,16,23,52 15 | 2022-05-28,46,8,24,59 16 | 2022-05-29,45,7,22,56 17 | 2022-05-30,45,9,24,61 18 | 2022-05-31,46,19,20,58 19 | 2022-06-01,51,21,22,56 20 | 2022-06-02,47,19,22,54 21 | 2022-06-03,46,17,21,56 22 | 2022-06-04,45,9,23,58 23 | 2022-06-05,47,9,23,60 24 | 2022-06-06,48,20,21,58 25 | 2022-06-07,46,23,21,57 26 | 2022-06-08,48,22,24,56 27 | 2022-06-09,48,20,23,55 28 | 2022-06-10,48,17,22,56 29 | 2022-06-11,47,9,25,54 30 | 2022-06-12,46,9,24,56 31 | 2022-06-13,46,20,22,54 32 | 2022-06-14,46,21,23,58 33 | 2022-06-15,47,21,23,58 34 | 2022-06-16,46,21,24,56 35 | 2022-06-17,51,17,23,57 36 | 2022-06-18,44,9,24,54 37 | 2022-06-19,43,8,26,59 38 | 2022-06-20,45,16,25,53 39 | 2022-06-21,53,23,24,56 40 | 2022-06-22,48,21,25,58 41 | 2022-06-23,48,22,26,55 42 | 2022-06-24,54,18,24,56 43 | 2022-06-25,54,9,25,57 44 | 2022-06-26,48,8,23,62 45 | 2022-06-27,51,19,24,58 46 | 2022-06-28,54,21,25,56 47 | 2022-06-29,49,20,25,61 48 | 2022-06-30,53,19,27,58 49 | 2022-07-01,56,17,26,55 50 | 2022-07-02,47,8,25,56 51 | 2022-07-03,49,8,28,58 52 | 2022-07-04,47,9,25,58 53 | 2022-07-05,52,20,22,61 54 | 2022-07-06,49,20,27,60 55 | 2022-07-07,49,21,26,56 56 | 2022-07-08,58,17,27,53 57 | 2022-07-09,61,10,30,55 58 | 2022-07-10,56,15,31,55 59 | 2022-07-11,59,27,27,55 60 | 2022-07-12,52,25,28,57 61 | 2022-07-13,54,25,26,55 62 | 2022-07-14,61,25,25,60 63 | 2022-07-15,51,22,25,58 64 | 2022-07-16,50,12,24,58 65 | 2022-07-17,53,13,28,57 66 | 2022-07-18,48,24,25,57 67 | 2022-07-19,50,26,25,57 68 | 2022-07-20,47,26,25,53 69 | 2022-07-21,49,24,26,55 70 | 2022-07-22,50,21,26,57 71 | 2022-07-23,47,9,26,55 72 | 2022-07-24,45,8,27,57 73 | 2022-07-25,47,19,26,59 74 | 2022-07-26,47,20,28,58 75 | 2022-07-27,47,20,27,56 76 | 2022-07-28,48,19,26,57 77 | 2022-07-29,47,18,27,58 78 | 2022-07-30,46,9,28,53 79 | 2022-07-31,48,9,27,58 80 | 2022-08-01,48,19,25,56 81 | 2022-08-02,51,21,25,56 82 | 2022-08-03,49,21,27,55 83 | 2022-08-04,47,20,24,54 84 | 2022-08-05,47,17,25,54 85 | 2022-08-06,47,10,26,57 86 | 2022-08-07,46,9,26,58 87 | 2022-08-08,46,19,24,59 88 | 2022-08-09,62,21,25,59 89 | 2022-08-10,52,20,25,56 90 | 2022-08-11,54,21,24,61 91 | 2022-08-12,60,18,24,59 92 | 2022-08-13,55,9,26,56 93 | 2022-08-14,52,9,25,55 94 | 2022-08-15,48,19,24,54 95 | 2022-08-16,49,21,27,55 96 | 2022-08-17,47,20,24,53 97 | 2022-08-18,45,19,24,53 98 | 2022-08-19,47,18,25,53 99 | 2022-08-20,47,9,25,54 100 | 2022-08-21,49,10,27,56 101 | 2022-08-22,44,20,23,55 102 | 2022-08-23,46,22,23,52 103 | 2022-08-24,49,22,23,52 104 | 2022-08-25,47,21,23,56 105 | 2022-08-26,56,17,23,54 106 | 2022-08-27,52,9,25,55 107 | 2022-08-28,51,9,26,57 108 | 2022-08-29,47,19,22,56 109 | 2022-08-30,47,21,22,52 110 | 2022-08-31,46,20,21,52 111 | 2022-09-01,47,18,22,55 112 | 2022-09-02,48,16,21,49 113 | 2022-09-03,50,8,24,52 114 | 2022-09-04,47,8,25,56 115 | 2022-09-05,48,10,27,56 116 | 2022-09-06,45,19,24,54 117 | 2022-09-07,49,20,21,54 118 | 2022-09-08,51,19,23,50 119 | 2022-09-09,49,16,21,52 120 | 2022-09-10,49,9,21,52 121 | 2022-09-11,52,9,22,55 122 | 2022-09-12,49,19,21,50 123 | 2022-09-13,54,21,20,52 124 | 2022-09-14,47,21,20,49 125 | 2022-09-15,45,20,30,51 126 | 2022-09-16,42,16,22,48 127 | 2022-09-17,47,9,23,52 128 | 2022-09-18,50,9,23,51 129 | 2022-09-19,48,20,21,48 130 | 2022-09-20,47,21,22,50 131 | 2022-09-21,47,22,23,51 132 | 2022-09-22,46,21,23,55 133 | 2022-09-23,46,18,21,49 134 | 2022-09-24,47,9,22,50 135 | 2022-09-25,48,9,23,53 136 | 2022-09-26,45,17,23,51 137 | 2022-09-27,46,20,23,47 138 | 2022-09-28,49,19,23,50 139 | 2022-09-29,45,18,30,56 140 | 2022-09-30,49,17,32,56 141 | 2022-10-01,47,9,36,64 142 | 2022-10-02,52,9,39,66 143 | 2022-10-03,53,19,31,60 144 | 2022-10-04,64,21,33,66 145 | 2022-10-05,58,20,34,68 146 | 2022-10-06,49,19,33,62 147 | 2022-10-07,51,17,22,49 148 | 2022-10-08,50,9,24,48 149 | 2022-10-09,52,9,22,52 150 | 2022-10-10,52,16,21,50 151 | 2022-10-11,51,20,22,50 152 | 2022-10-12,47,19,22,49 153 | 2022-10-13,46,20,23,48 154 | 2022-10-14,46,18,20,49 155 | 2022-10-15,49,9,22,48 156 | 2022-10-16,52,9,23,52 157 | 2022-10-17,47,19,22,52 158 | 2022-10-18,48,20,24,49 159 | 2022-10-19,46,19,24,50 160 | 2022-10-20,46,20,22,49 161 | 2022-10-21,49,18,22,47 162 | 2022-10-22,50,9,23,49 163 | 2022-10-23,58,8,22,53 164 | 2022-10-24,53,18,22,50 165 | 2022-10-25,53,21,22,50 166 | 2022-10-26,52,19,22,49 167 | 2022-10-27,56,19,28,49 168 | 2022-10-28,100,17,26,48 169 | 2022-10-29,75,8,25,49 170 | 2022-10-30,66,9,23,56 171 | 2022-10-31,69,17,21,83 172 | 2022-11-01,75,19,21,58 173 | 2022-11-02,64,21,23,51 174 | 2022-11-03,61,19,24,49 175 | 2022-11-04,76,17,28,52 176 | 2022-11-05,69,9,23,54 177 | 2022-11-06,62,8,25,51 178 | 2022-11-07,66,18,23,50 179 | 2022-11-08,60,18,24,47 180 | 2022-11-09,64,18,21,45 181 | 2022-11-10,61,19,22,49 182 | 2022-11-11,69,16,23,47 -------------------------------------------------------------------------------- /data/hillstrom_clean_label_mapping.json: -------------------------------------------------------------------------------- 1 | {"control": 0, "womans_email": 1, "mens_email": 2} -------------------------------------------------------------------------------- /data/ml_earnings.csv: -------------------------------------------------------------------------------- 1 | age,took_a_course,earnings 2 | 19,False,110579.0 3 | 28,False,142577.0 4 | 22,True,130520.0 5 | 25,True,142687.0 6 | 24,False,127832.0 7 | 23,False,125557.0 8 | 20,False,113922.0 9 | 23,False,124084.0 10 | 22,True,131905.0 11 | 28,False,138213.0 12 | 22,False,121170.0 13 | 32,False,154397.0 14 | 35,False,174856.0 15 | 33,False,163965.0 16 | 27,False,136437.0 17 | 22,True,132110.0 18 | 22,True,131140.0 19 | 34,False,166636.0 20 | 22,True,131868.0 21 | 30,True,157744.0 22 | 25,False,130011.0 23 | 41,True,207676.0 24 | 24,True,137985.0 25 | 30,True,160678.0 26 | 37,False,181514.0 27 | 33,False,159670.0 28 | 19,True,123372.0 29 | 26,False,134860.0 30 | 22,False,118836.0 31 | 32,True,167632.0 32 | 35,True,180750.0 33 | 24,False,122944.0 34 | 24,False,128146.0 35 | 25,False,133634.0 36 | 21,True,132726.0 37 | 24,True,137189.0 38 | 20,False,113015.0 39 | 22,False,120984.0 40 | 23,False,122567.0 41 | 22,True,130602.0 42 | 36,False,174964.0 43 | 32,False,161667.0 44 | 26,False,133173.0 45 | 22,False,119640.0 46 | 28,True,149414.0 47 | 36,False,177082.0 48 | 24,False,125944.0 49 | 19,False,111847.0 50 | 34,False,162654.0 51 | 20,True,123706.0 52 | 28,False,139394.0 53 | 23,False,126213.0 54 | 33,True,169772.0 55 | 23,True,135389.0 56 | 32,False,155655.0 57 | 21,False,115116.0 58 | 26,False,133087.0 59 | 23,True,134155.0 60 | 26,True,144015.0 61 | 25,True,143167.0 62 | 27,True,148107.0 63 | 33,False,162571.0 64 | 19,True,121933.0 65 | 25,False,127461.0 66 | 33,True,169929.0 67 | 21,False,117047.0 68 | 30,True,156543.0 69 | 36,False,173676.0 70 | 29,True,154168.0 71 | 23,True,133546.0 72 | 24,False,124253.0 73 | 24,False,128471.0 74 | 30,False,149020.0 75 | 19,False,112694.0 76 | 23,False,124771.0 77 | 30,True,163548.0 78 | 22,False,121280.0 79 | 19,False,108913.0 80 | 21,False,120143.0 81 | 19,True,125577.0 82 | 39,False,185360.0 83 | 33,True,169475.0 84 | 22,False,119802.0 85 | 26,True,143772.0 86 | 42,False,204343.0 87 | 21,False,115600.0 88 | 29,False,146446.0 89 | 22,False,119595.0 90 | 31,True,163592.0 91 | 23,True,134419.0 92 | 30,True,159419.0 93 | 24,True,138847.0 94 | 23,True,134703.0 95 | 23,False,125074.0 96 | 24,False,127307.0 97 | 34,False,165055.0 98 | 20,False,114155.0 99 | 20,False,117019.0 100 | 32,True,172076.0 101 | 42,False,201715.0 102 | 28,False,141481.0 103 | 27,False,139070.0 104 | 32,False,159917.0 105 | 19,False,108433.0 106 | 23,False,123586.0 107 | 33,True,170740.0 108 | 21,False,117066.0 109 | 24,False,124757.0 110 | 24,False,128756.0 111 | 23,False,123440.0 112 | 19,True,122551.0 113 | 34,True,179133.0 114 | 28,False,141329.0 115 | 23,False,125327.0 116 | 22,True,129636.0 117 | 34,False,167906.0 118 | 38,False,186144.0 119 | 41,False,201729.0 120 | 24,False,131654.0 121 | 20,True,126153.0 122 | 42,True,214445.0 123 | 22,False,121647.0 124 | 20,True,126501.0 125 | 28,False,141564.0 126 | 23,True,132636.0 127 | 20,False,118623.0 128 | 25,True,143428.0 129 | 27,False,139404.0 130 | 24,True,136236.0 131 | 23,True,134872.0 132 | 38,False,185658.0 133 | 34,False,166220.0 134 | 19,False,112008.0 135 | 30,False,153864.0 136 | 24,False,125082.0 137 | 34,True,178711.0 138 | 39,False,185525.0 139 | 33,False,163608.0 140 | 31,False,154288.0 141 | 35,False,173291.0 142 | 31,False,150776.0 143 | 30,False,149096.0 144 | 27,True,151007.0 145 | 26,True,146392.0 146 | 35,False,170960.0 147 | 45,True,233035.0 148 | 26,False,134897.0 149 | 21,False,116928.0 150 | 26,True,145022.0 151 | 20,True,124835.0 152 | 20,True,124358.0 153 | 43,False,210243.0 154 | 20,True,128241.0 155 | 21,False,115543.0 156 | 37,True,187627.0 157 | 22,False,118132.0 158 | 20,False,111554.0 159 | 20,False,118302.0 160 | 20,True,125639.0 161 | 29,True,155448.0 162 | 21,False,117786.0 163 | 21,False,119334.0 164 | 28,True,153722.0 165 | 22,True,132427.0 166 | 23,True,138425.0 167 | 33,True,170184.0 168 | 30,False,149355.0 169 | 48,False,239510.0 170 | 23,True,133744.0 171 | 28,False,144378.0 172 | 35,False,169430.0 173 | 21,True,128129.0 174 | 25,False,130482.0 175 | 29,False,145436.0 176 | 25,True,141578.0 177 | 27,True,147119.0 178 | 21,True,129610.0 179 | 28,True,154320.0 180 | 29,False,143885.0 181 | 24,False,127397.0 182 | 26,True,143563.0 183 | 22,False,121027.0 184 | 25,False,132476.0 185 | 21,False,116586.0 186 | 29,False,143288.0 187 | 39,False,190876.0 188 | 27,False,138602.0 189 | 23,False,123990.0 190 | 22,False,120377.0 191 | 29,False,147188.0 192 | 29,True,148466.0 193 | 19,True,124007.0 194 | 22,True,132727.0 195 | 28,False,143704.0 196 | 27,False,141665.0 197 | 35,True,180059.0 198 | 19,False,113031.0 199 | 38,False,184837.0 200 | 19,False,111382.0 201 | 32,False,155940.0 202 | -------------------------------------------------------------------------------- /data/ml_earnings_interaction_test.csv: -------------------------------------------------------------------------------- 1 | age,python_proficiency,took_a_course,true_effect 2 | 30,0.22387682273767961,True,11120.0 3 | 23,0.39415189924761007,True,11970.0 4 | 37,0.2146377127092679,True,11073.0 5 | 21,0.8690694065487253,True,14345.0 6 | 41,0.8339335515161994,True,14169.0 7 | 33,0.27165464274377815,True,11358.0 8 | 24,0.6749056709382142,True,13374.0 9 | 21,0.6548377191770705,True,13275.0 10 | 36,0.028713567977660337,True,10143.0 11 | 36,0.07706736417814364,True,10385.0 12 | 24,0.5969832080096646,True,12985.0 13 | 19,0.754652317177385,True,13773.0 14 | 22,0.9701476625375967,True,14850.0 15 | 23,0.516337378365627,True,12582.0 16 | 27,0.14763330181007217,True,10738.0 17 | 32,0.10665130841274617,True,10534.0 18 | 21,0.9034475785889416,True,14517.0 19 | 21,0.00754939322492898,True,10038.0 20 | 23,0.5856538108502074,True,12928.0 21 | 21,0.7731091931329046,True,13865.0 22 | 22,0.4155164444090621,True,12077.0 23 | 24,0.879776135206835,True,14399.0 24 | 30,0.9273541754244794,True,14637.0 25 | 22,0.8829521915372446,True,14415.0 26 | 25,0.41105913268579086,True,12055.0 27 | 25,0.6191696160038245,True,13096.0 28 | 24,0.04733337838583784,True,10236.0 29 | 41,0.47893314888488114,True,12395.0 30 | 44,0.09944093127332043,True,10498.0 31 | 25,0.2141457794330497,True,11071.0 32 | 34,0.3010400850697761,True,11505.0 33 | 19,0.8935352732069651,True,14468.0 34 | 44,0.5011724863627216,True,12506.0 35 | 19,0.18598147770314144,True,10930.0 36 | 19,0.23651636982734914,True,11182.0 37 | 19,0.9515226743334864,True,14757.0 38 | 20,0.5903458191540543,True,12952.0 39 | 22,0.614049897785869,True,13071.0 40 | 21,0.5685133860458594,True,12842.0 41 | 23,0.9657358123678771,True,14828.0 42 | 29,0.2939826080119571,True,11470.0 43 | 23,0.31206223355020046,True,11561.0 44 | 25,0.24394882853743838,True,11219.0 45 | 35,0.1888105446811692,True,10944.0 46 | 21,0.17212297381880481,True,10861.0 47 | 27,0.6787326814995325,True,13393.0 48 | 20,0.8485841294117298,True,14243.0 49 | 35,0.9484767287957341,True,14742.0 50 | 28,0.18462968527393953,True,10923.0 51 | 23,0.19854916503306286,True,10992.0 52 | 30,0.7868749315943776,True,13934.0 53 | 31,0.2560144316176236,True,11281.0 54 | 23,0.7931986715456523,True,13966.0 55 | 19,0.5984763023800789,True,12993.0 56 | 34,0.3218355012581319,True,11609.0 57 | 26,0.8773770629018882,True,14387.0 58 | 20,0.027998583565693846,True,10140.0 59 | 24,0.7016890963543043,True,13508.0 60 | 28,0.414826108610463,True,12074.0 61 | 24,0.4055053361791364,True,12027.0 62 | 27,0.672410497084009,True,13362.0 63 | 24,0.9188827199485727,True,14594.0 64 | 21,0.8581974673286622,True,14291.0 65 | 23,0.24638740966187578,True,11232.0 66 | 21,0.03482611933114499,True,10174.0 67 | 25,0.6901719167371854,True,13451.0 68 | 32,0.37286235605965223,True,11864.0 69 | 33,0.10630526540167351,True,10532.0 70 | 32,0.14191426691020936,True,10709.0 71 | 27,0.6947074917198913,True,13474.0 72 | 19,0.29825221382550793,True,11491.0 73 | 29,0.6289800314325729,True,13145.0 74 | 21,0.007712802008101538,True,10039.0 75 | 28,0.7339499732538902,True,13670.0 76 | 28,0.6641348201845104,True,13321.0 77 | 27,0.22998238811942795,True,11150.0 78 | 26,0.20427867542820322,True,11022.0 79 | 21,0.5058140845042293,True,12529.0 80 | 30,0.2150062667778866,True,11075.0 81 | 45,0.5996982119068462,True,12998.0 82 | 24,0.46955708258014006,True,12348.0 83 | 34,0.689212768736534,True,13446.0 84 | 28,0.688789641829235,True,13444.0 85 | 19,0.08368086340831893,True,10419.0 86 | 36,0.08353312239165811,True,10418.0 87 | 32,0.6602178774563141,True,13301.0 88 | 25,0.3034097182007709,True,11517.0 89 | 24,0.4722092843068547,True,12361.0 90 | 28,0.894533822251131,True,14473.0 91 | 24,0.012281682704884167,True,10061.0 92 | 27,0.4460295972542465,True,12230.0 93 | 22,0.17985420688312093,True,10899.0 94 | 22,0.8572867893318362,True,14287.0 95 | 29,0.9950724171687579,True,14976.0 96 | 34,0.45387654040874015,True,12270.0 97 | 21,0.27297914716043903,True,11365.0 98 | 30,0.7261709231998095,True,13630.0 99 | 29,0.4287144754158567,True,12143.0 100 | 25,0.17942665780924838,True,10897.0 101 | 19,0.45354055693928375,True,12268.0 102 | -------------------------------------------------------------------------------- /data/shpitser_thesis1.grapl: -------------------------------------------------------------------------------- 1 | "Shpitser thesis, Fig 4.1"; 2 | X; W1; W2; Y1; Y2; 3 | X -> Y1; 4 | W1 <-> Y1; 5 | W1 -> X; 6 | W1 <-> Y2; 7 | W1 <-> W2; 8 | W2 -> Y2; 9 | W2 <-> X; -------------------------------------------------------------------------------- /docs/himsolt-gml-technical-report.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/docs/himsolt-gml-technical-report.pdf -------------------------------------------------------------------------------- /errata/Errata - Early Print (ordered before June 13 2023).ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "d3201184", 6 | "metadata": {}, 7 | "source": [ 8 | "# Errata\n", 9 | "\n", 10 | "\n", 11 | "(Books ordered before ~12:00 PM GMT on June 13, 2023)" 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "id": "ea449a06", 17 | "metadata": {}, 18 | "source": [ 19 | "## Chapter 02\n", 20 | "\n", 21 | "\n", 22 | "**Page 22**\n", 23 | "\n", 24 | "\n", 25 | "In the book (p. 22) we say:\n", 26 | "_______________________________________________________________\n", 27 | "\n", 28 | "*(...) the probability of buying book A, given we bought book B, is 63.8%. This indicates a\n", 29 | "positive relationship between both variables (if there was no association between them, we would\n", 30 | "expect the result to be **50%**)*\n", 31 | "\n", 32 | "\n", 33 | "This is **incorrect**. \n", 34 | "\n", 35 | "_______________________________________________________________\n", 36 | "\n", 37 | "The **correct** statement should be:\n", 38 | "\n", 39 | "*(...) the probability of buying book A, given we bought book B, is 63.8%. This indicates a\n", 40 | "positive relationship between both variables (if there was no association between them, we would\n", 41 | "expect the result to be **39%**)*\n", 42 | "_______________________________________________________________\n", 43 | "\n", 44 | " \n", 45 | "*Thanks for David for spotting this mistake.*" 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "id": "952febd2", 51 | "metadata": {}, 52 | "source": [ 53 | "## Chapter 04\n", 54 | "\n", 55 | "\n", 56 | "**Figure 4.1** is **incorrect**.\n", 57 | "\n", 58 | "The **correct** version of **Figure 4.1** is the following:\n", 59 | "\n", 60 | "" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "id": "85952f08", 66 | "metadata": {}, 67 | "source": [ 68 | "## Chapter 06\n", 69 | "\n", 70 | "### You’re gonna keep ‘em d-separated\n", 71 | "\n", 72 | "\n", 73 | "**Figure 6.3** representing DAG **d)** in the *Keep 'em d-separated* game is **incorrect**.\n", 74 | "\n", 75 | "The **correct** version of **Figure 6.3** is the following:\n", 76 | "\n", 77 | "\n", 78 | "\n", 79 | "

\n", 80 | "\n", 81 | "\n", 82 | "**Figure 6.4** representing DAG **e)** in the *Keep 'em d-separated* game is **incorrect**.\n", 83 | "\n", 84 | "The **correct** version of **Figure 6.4** is the following:\n", 85 | "\n", 86 | "\n", 87 | "\n", 88 | "

\n", 89 | "\n", 90 | "\n", 91 | "**Figure 6.5** representing **all DAGs** in the *Keep 'em d-separated* game is **incorrect**.\n", 92 | "\n", 93 | "The **correct** version of **Figure 6.5** is the following:\n", 94 | "\n", 95 | "\n", 96 | "\n", 97 | "\n", 98 | "### Instrumental variables\n", 99 | "\n", 100 | "**Page 121**\n", 101 | "\n", 102 | "In the book (p. 121) one of the formulas for regression models is **incorrect**. \n", 103 | "\n", 104 | "The book says:\n", 105 | "\n", 106 | "_________\n", 107 | "\n", 108 | "\n", 109 | "*To calculate the causal effect of X on Y in a linear case, all we need to do is fit two linear regression\n", 110 | "models and compute the ratio of their coefficients!*\n", 111 | "\n", 112 | "*The two models are as follows:*\n", 113 | "\n", 114 | "*• Y ~ Z*\n", 115 | "\n", 116 | "*• Y ~ X*\n", 117 | "\n", 118 | "_____________\n", 119 | "\n", 120 | "\n", 121 | "The **correct** formulas should be:\n", 122 | "\n", 123 | "_____________\n", 124 | "\n", 125 | "*The two models are as follows:*\n", 126 | "\n", 127 | "*• Y ~ Z*\n", 128 | "\n", 129 | "*• X ~ Z*\n", 130 | "\n", 131 | "__________________\n", 132 | "\n", 133 | "\n", 134 | "The **code example** below uses the **correct formulas**." 135 | ] 136 | }, 137 | { 138 | "cell_type": "markdown", 139 | "id": "8c0953ef", 140 | "metadata": {}, 141 | "source": [ 142 | "## Chapter 07\n", 143 | "\n", 144 | "\n", 145 | "**Figure 7.6** is **incorrect**.\n", 146 | "\n", 147 | "The **correct** version of **Figure 7.6** is the following:\n", 148 | "\n", 149 | "" 150 | ] 151 | }, 152 | { 153 | "cell_type": "markdown", 154 | "id": "7575e3b4", 155 | "metadata": {}, 156 | "source": [ 157 | "## Chapter 13\n", 158 | "\n", 159 | "\n", 160 | "**Page 343** (point 3)\n", 161 | "\n", 162 | "\n", 163 | "In the book (p. 343) in point **3.** we say:\n", 164 | "\n", 165 | "_______________________________________________________________\n", 166 | "\n", 167 | "3. *We remove the edges between **P and Q**, and Q and S, as P ⫫ Q | R and Q ⫫ S | R (C).*\n", 168 | "\n", 169 | "\n", 170 | "This is **incorrect**. \n", 171 | "\n", 172 | "_______________________________________________________________\n", 173 | "\n", 174 | "The **correct** statement should be:\n", 175 | "\n", 176 | "3. *We remove the edges between **P and S**, and Q and S, as P ⫫ Q | R and Q ⫫ S | R (C).*\n", 177 | "\n", 178 | "_______________________________________________________________\n", 179 | "\n", 180 | " \n", 181 | "*Thanks for Miguel for spotting this mistake.*\n", 182 | " \n", 183 | "______________________________________________________\n", 184 | "

\n", 185 | "\n", 186 | "**Page 343** (point 4)\n", 187 | "\n", 188 | "In the book (p. 343) in point **4.** we say:\n", 189 | "_______________________________________________________________\n", 190 | "\n", 191 | "4. *We orient the edges **P -> R <- S** because R is a collider between **P and S** (D).*\n", 192 | "\n", 193 | "\n", 194 | "This is **incorrect**. \n", 195 | "\n", 196 | "_______________________________________________________________\n", 197 | "\n", 198 | "The **correct** statement should be:\n", 199 | "\n", 200 | "4. *We orient the edges **P -> R <- Q** because R is a collider between **P and Q** (D).*\n", 201 | "\n", 202 | "_______________________________________________________________\n", 203 | "\n", 204 | " \n", 205 | "*Thanks for Takashi for spotting this mistake.*" 206 | ] 207 | }, 208 | { 209 | "cell_type": "markdown", 210 | "id": "7d249ae5", 211 | "metadata": {}, 212 | "source": [ 213 | "## Chapter 14\n", 214 | "\n", 215 | "**Page 391** (bullet point 1)\n", 216 | "\n", 217 | "\n", 218 | "In the book (p. 391) in the first bullet point we say:\n", 219 | "\n", 220 | "_______________________________________________________________\n", 221 | "\n", 222 | "*- **X** is not an ancestor of **Y** - there might be...*\n", 223 | "\n", 224 | "\n", 225 | "This is **incorrect**. \n", 226 | "\n", 227 | "_______________________________________________________________\n", 228 | "\n", 229 | "The **correct** statement should be:\n", 230 | "\n", 231 | "*- **Y** is not an ancestor of **X** - there might be...*\n", 232 | "_______________________________________________________________\n", 233 | "\n" 234 | ] 235 | } 236 | ], 237 | "metadata": { 238 | "kernelspec": { 239 | "display_name": "Python 3 (ipykernel)", 240 | "language": "python", 241 | "name": "python3" 242 | }, 243 | "language_info": { 244 | "codemirror_mode": { 245 | "name": "ipython", 246 | "version": 3 247 | }, 248 | "file_extension": ".py", 249 | "mimetype": "text/x-python", 250 | "name": "python", 251 | "nbconvert_exporter": "python", 252 | "pygments_lexer": "ipython3", 253 | "version": "3.9.18" 254 | } 255 | }, 256 | "nbformat": 4, 257 | "nbformat_minor": 5 258 | } 259 | -------------------------------------------------------------------------------- /errata/Errata - Non-Early Print (ordered after June 13 2023).ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "d3201184", 6 | "metadata": {}, 7 | "source": [ 8 | "# Errata\n", 9 | "\n", 10 | "(Books ordered after ~12:00 PM GMT on June 13, 2023)" 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "id": "a9740489", 16 | "metadata": {}, 17 | "source": [ 18 | "## Chapter 02\n", 19 | "\n", 20 | "\n", 21 | "**Page 22**\n", 22 | "\n", 23 | "\n", 24 | "In the book (p. 22) we say:\n", 25 | "_______________________________________________________________\n", 26 | "\n", 27 | "*(...) the probability of buying book A, given we bought book B, is 63.8%. This indicates a\n", 28 | "positive relationship between both variables (if there was no association between them, we would\n", 29 | "expect the result to be **50%**)*\n", 30 | "\n", 31 | "\n", 32 | "This is **incorrect**. \n", 33 | "\n", 34 | "_______________________________________________________________\n", 35 | "\n", 36 | "The **correct** statement should be:\n", 37 | "\n", 38 | "*(...) the probability of buying book A, given we bought book B, is 63.8%. This indicates a\n", 39 | "positive relationship between both variables (if there was no association between them, we would\n", 40 | "expect the result to be **39%**)*\n", 41 | "_______________________________________________________________\n", 42 | "\n", 43 | " \n", 44 | "*Thanks for David for spotting this mistake.*" 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "id": "85952f08", 50 | "metadata": {}, 51 | "source": [ 52 | "## Chapter 06\n", 53 | "\n", 54 | "\n", 55 | "### Instrumental variables\n", 56 | "\n", 57 | "**Page 121**\n", 58 | "\n", 59 | "In the book (p. 121) one of the formulas for regression models is **incorrect**. \n", 60 | "\n", 61 | "The book says:\n", 62 | "\n", 63 | "_________\n", 64 | "\n", 65 | "\n", 66 | "*To calculate the causal effect of X on Y in a linear case, all we need to do is fit two linear regression\n", 67 | "models and compute the ratio of their coefficients!*\n", 68 | "\n", 69 | "*The two models are as follows:*\n", 70 | "\n", 71 | "*• Y ~ Z*\n", 72 | "\n", 73 | "*• Y ~ X*\n", 74 | "\n", 75 | "_____________\n", 76 | "\n", 77 | "\n", 78 | "The **correct** formulas should be:\n", 79 | "\n", 80 | "_____________\n", 81 | "\n", 82 | "*The two models are as follows:*\n", 83 | "\n", 84 | "*• Y ~ Z*\n", 85 | "\n", 86 | "*• X ~ Z*\n", 87 | "\n", 88 | "__________________\n", 89 | "\n", 90 | "\n", 91 | "The **code example** below uses the **correct formulas**." 92 | ] 93 | }, 94 | { 95 | "cell_type": "markdown", 96 | "id": "53434695", 97 | "metadata": {}, 98 | "source": [ 99 | "## Chapter 13\n", 100 | "\n", 101 | "\n", 102 | "**Page 343** (point 3)\n", 103 | "\n", 104 | "\n", 105 | "In the book (p. 343) in point **3.** we say:\n", 106 | "\n", 107 | "_______________________________________________________________\n", 108 | "\n", 109 | "3. *We remove the edges between **P and Q**, and Q and S, as P ⫫ Q | R and Q ⫫ S | R (C).*\n", 110 | "\n", 111 | "\n", 112 | "This is **incorrect**. \n", 113 | "\n", 114 | "_______________________________________________________________\n", 115 | "\n", 116 | "The **correct** statement should be:\n", 117 | "\n", 118 | "3. *We remove the edges between **P and S**, and Q and S, as P ⫫ Q | R and Q ⫫ S | R (C).*\n", 119 | "\n", 120 | "_______________________________________________________________\n", 121 | "\n", 122 | " \n", 123 | "*Thanks for Miguel for spotting this mistake.*\n", 124 | " \n", 125 | "______________________________________________________\n", 126 | "

\n", 127 | "\n", 128 | "**Page 343** (point 4)\n", 129 | "\n", 130 | "In the book (p. 343) in point **4.** we say:\n", 131 | "_______________________________________________________________\n", 132 | "\n", 133 | "4. *We orient the edges **P -> R <- S** because R is a collider between **P and S** (D).*\n", 134 | "\n", 135 | "\n", 136 | "This is **incorrect**. \n", 137 | "\n", 138 | "_______________________________________________________________\n", 139 | "\n", 140 | "The **correct** statement should be:\n", 141 | "\n", 142 | "4. *We orient the edges **P -> R <- Q** because R is a collider between **P and Q** (D).*\n", 143 | "\n", 144 | "_______________________________________________________________\n", 145 | "\n", 146 | " \n", 147 | "*Thanks for Takashi for spotting this mistake.*" 148 | ] 149 | }, 150 | { 151 | "cell_type": "markdown", 152 | "id": "9e86603e", 153 | "metadata": {}, 154 | "source": [ 155 | "## Chapter 14\n", 156 | "\n", 157 | "**Page 391** (bullet point 1)\n", 158 | "\n", 159 | "\n", 160 | "In the book (p. 391) in the first bullet point we say:\n", 161 | "\n", 162 | "_______________________________________________________________\n", 163 | "\n", 164 | "*- **X** is not an ancestor of **Y** - there might be...*\n", 165 | "\n", 166 | "\n", 167 | "This is **incorrect**. \n", 168 | "\n", 169 | "_______________________________________________________________\n", 170 | "\n", 171 | "The **correct** statement should be:\n", 172 | "\n", 173 | "*- **Y** is not an ancestor of **X** - there might be...*\n", 174 | "_______________________________________________________________\n", 175 | "\n" 176 | ] 177 | } 178 | ], 179 | "metadata": { 180 | "kernelspec": { 181 | "display_name": "Python 3 (ipykernel)", 182 | "language": "python", 183 | "name": "python3" 184 | }, 185 | "language_info": { 186 | "codemirror_mode": { 187 | "name": "ipython", 188 | "version": 3 189 | }, 190 | "file_extension": ".py", 191 | "mimetype": "text/x-python", 192 | "name": "python", 193 | "nbconvert_exporter": "python", 194 | "pygments_lexer": "ipython3", 195 | "version": "3.9.18" 196 | } 197 | }, 198 | "nbformat": 4, 199 | "nbformat_minor": 5 200 | } 201 | -------------------------------------------------------------------------------- /errata/img/ch_04__fig_4_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/errata/img/ch_04__fig_4_1.png -------------------------------------------------------------------------------- /errata/img/ch_06__fig_6_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/errata/img/ch_06__fig_6_3.png -------------------------------------------------------------------------------- /errata/img/ch_06__fig_6_4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/errata/img/ch_06__fig_6_4.png -------------------------------------------------------------------------------- /errata/img/ch_06__fig_6_5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/errata/img/ch_06__fig_6_5.png -------------------------------------------------------------------------------- /errata/img/ch_07__fig_7_6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/errata/img/ch_07__fig_7_6.png -------------------------------------------------------------------------------- /errata/minor-errors.txt: -------------------------------------------------------------------------------- 1 | ## Chapter 06 2 | 3 | - Code section "Front-door in practice" 4 | - `class GPSMemorySCM:`: 5 | - `u_y` is not used anywhere while it should be used in the `memory` computations [reported by Arash A.] 6 | 7 | -------------------------------------------------------------------------------- /img/ch_03_graph_01: -------------------------------------------------------------------------------- 1 | digraph { 2 | A 3 | X 4 | B 5 | Y 6 | A -> X 7 | X -> B 8 | A -> Y 9 | Y -> B 10 | } 11 | -------------------------------------------------------------------------------- /img/ch_03_graph_01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_03_graph_01.png -------------------------------------------------------------------------------- /img/ch_03_graph_02: -------------------------------------------------------------------------------- 1 | digraph { 2 | graph [nodesep=1.5 rankdir=LR ranksep=.6] 3 | A 4 | X 5 | Y 6 | A -> Y 7 | A -> X 8 | X -> Y 9 | } 10 | -------------------------------------------------------------------------------- /img/ch_03_graph_02.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_03_graph_02.png -------------------------------------------------------------------------------- /img/ch_04_graph_DAG: -------------------------------------------------------------------------------- 1 | digraph { 2 | A 3 | B 4 | C 5 | D 6 | A -> B 7 | B -> C 8 | A -> D 9 | D -> C 10 | } 11 | -------------------------------------------------------------------------------- /img/ch_04_graph_DAG.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_04_graph_DAG.png -------------------------------------------------------------------------------- /img/ch_04_graph_DCG: -------------------------------------------------------------------------------- 1 | digraph { 2 | A 3 | B 4 | C 5 | D 6 | A -> B 7 | A -> D 8 | B -> B 9 | B -> C 10 | D -> C 11 | C -> A 12 | } 13 | -------------------------------------------------------------------------------- /img/ch_04_graph_DCG.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_04_graph_DCG.png -------------------------------------------------------------------------------- /img/ch_04_graph_Fully connected: -------------------------------------------------------------------------------- 1 | graph { 2 | A 3 | B 4 | C 5 | D 6 | A -- B 7 | A -- C 8 | A -- D 9 | B -- C 10 | B -- D 11 | C -- D 12 | } 13 | -------------------------------------------------------------------------------- /img/ch_04_graph_Fully connected.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_04_graph_Fully connected.png -------------------------------------------------------------------------------- /img/ch_04_graph_Partially connected: -------------------------------------------------------------------------------- 1 | graph { 2 | A 3 | B 4 | C 5 | D 6 | A -- B 7 | A -- C 8 | B -- C 9 | } 10 | -------------------------------------------------------------------------------- /img/ch_04_graph_Partially connected.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_04_graph_Partially connected.png -------------------------------------------------------------------------------- /img/ch_04_graph_Undirected: -------------------------------------------------------------------------------- 1 | graph { 2 | A 3 | B 4 | C 5 | D 6 | A -- B 7 | B -- C 8 | A -- D 9 | D -- C 10 | } 11 | -------------------------------------------------------------------------------- /img/ch_04_graph_Undirected.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_04_graph_Undirected.png -------------------------------------------------------------------------------- /img/ch_04_graph_adj_00: -------------------------------------------------------------------------------- 1 | digraph { 2 | 0 3 | 1 4 | 0 -> 1 5 | } 6 | -------------------------------------------------------------------------------- /img/ch_04_graph_adj_00.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_04_graph_adj_00.png -------------------------------------------------------------------------------- /img/ch_04_graph_adj_01: -------------------------------------------------------------------------------- 1 | digraph { 2 | 0 3 | 1 4 | 2 5 | 0 -> 1 6 | 2 -> 1 7 | 2 -> 0 8 | } 9 | -------------------------------------------------------------------------------- /img/ch_04_graph_adj_01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_04_graph_adj_01.png -------------------------------------------------------------------------------- /img/ch_04_graph_adj_02: -------------------------------------------------------------------------------- 1 | digraph { 2 | 0 3 | 1 4 | 2 5 | 3 6 | 0 -> 2 7 | 1 -> 3 8 | 3 -> 2 9 | 3 -> 0 10 | } 11 | -------------------------------------------------------------------------------- /img/ch_04_graph_adj_02.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_04_graph_adj_02.png -------------------------------------------------------------------------------- /img/ch_05_chain_00: -------------------------------------------------------------------------------- 1 | digraph { 2 | A [pos="0,0!"] 3 | B [pos="1.5,0!"] 4 | C [pos="3,0!"] 5 | A -> B 6 | B -> C 7 | } 8 | -------------------------------------------------------------------------------- /img/ch_05_chain_00.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_05_chain_00.png -------------------------------------------------------------------------------- /img/ch_05_collider_00: -------------------------------------------------------------------------------- 1 | digraph { 2 | A [pos="0,0!"] 3 | B [pos="1.5,0!"] 4 | C [pos="3,0!"] 5 | A -> B 6 | C -> B 7 | } 8 | -------------------------------------------------------------------------------- /img/ch_05_collider_00.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_05_collider_00.png -------------------------------------------------------------------------------- /img/ch_05_fork_00: -------------------------------------------------------------------------------- 1 | digraph { 2 | A [pos="0,0!"] 3 | B [pos="1.5,0!"] 4 | C [pos="3,0!"] 5 | B -> A 6 | B -> C 7 | } 8 | -------------------------------------------------------------------------------- /img/ch_05_fork_00.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_05_fork_00.png -------------------------------------------------------------------------------- /img/ch_05_markov_01: -------------------------------------------------------------------------------- 1 | digraph { 2 | A [pos="0,2.75!"] 3 | B [pos="2,3!"] 4 | C [pos="0,1!"] 5 | A -> B 6 | C -> B 7 | C -> A 8 | } 9 | -------------------------------------------------------------------------------- /img/ch_05_markov_01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_05_markov_01.png -------------------------------------------------------------------------------- /img/ch_05_markov_02: -------------------------------------------------------------------------------- 1 | digraph { 2 | A [pos="0,2.75!"] 3 | B [pos="2,3!"] 4 | C [pos="0,1!"] 5 | A -> B 6 | C -> B 7 | A -> C 8 | } 9 | -------------------------------------------------------------------------------- /img/ch_05_markov_02.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_05_markov_02.png -------------------------------------------------------------------------------- /img/ch_06_confounding_00: -------------------------------------------------------------------------------- 1 | digraph { 2 | "Gene X Expression" [pos="4,0!"] 3 | Stress [pos="2,1.75!"] 4 | "Immune System Response" [pos="7.5,0!"] 5 | "Delay of Gratification" [pos="2.5,-1.75!"] 6 | "Gene X Expression" -> "Immune System Response" 7 | Stress -> "Gene X Expression" 8 | Stress -> "Immune System Response" 9 | Stress -> "Delay of Gratification" 10 | "Gene X Expression" -> "Delay of Gratification" 11 | } 12 | -------------------------------------------------------------------------------- /img/ch_06_confounding_00.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_confounding_00.png -------------------------------------------------------------------------------- /img/ch_06_d_sep_00: -------------------------------------------------------------------------------- 1 | digraph { 2 | X [pos="1,0!"] 3 | B [pos="2.5,0!"] 4 | Y [pos="4,0!"] 5 | X -> B 6 | B -> Y 7 | } 8 | -------------------------------------------------------------------------------- /img/ch_06_d_sep_00.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_d_sep_00.png -------------------------------------------------------------------------------- /img/ch_06_d_sep_01: -------------------------------------------------------------------------------- 1 | digraph { 2 | X [pos="1,0!"] 3 | B [pos="2.5,0!"] 4 | Y [pos="4,0!"] 5 | X -> B 6 | Y -> B 7 | } 8 | -------------------------------------------------------------------------------- /img/ch_06_d_sep_01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_d_sep_01.png -------------------------------------------------------------------------------- /img/ch_06_d_sep_02: -------------------------------------------------------------------------------- 1 | digraph { 2 | A [pos="1,1!"] 3 | X [pos="3,1!"] 4 | B [pos="3,0!"] 5 | Y [pos="1, 0!"] 6 | X -> B 7 | Y -> B 8 | X -> A 9 | A -> Y 10 | } 11 | -------------------------------------------------------------------------------- /img/ch_06_d_sep_02.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_d_sep_02.png -------------------------------------------------------------------------------- /img/ch_06_d_sep_03: -------------------------------------------------------------------------------- 1 | digraph { 2 | A [pos="1.5,0!"] 3 | X [pos="0,0!"] 4 | B [pos="3,0!"] 5 | Y [pos="4.5,0!"] 6 | X -> A 7 | B -> A 8 | B -> Y 9 | } 10 | -------------------------------------------------------------------------------- /img/ch_06_d_sep_03.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_d_sep_03.png -------------------------------------------------------------------------------- /img/ch_06_d_sep_04: -------------------------------------------------------------------------------- 1 | digraph { 2 | S -> Q 3 | S -> Y 4 | Q -> X 5 | Q -> Y 6 | X -> Z 7 | Z -> Y 8 | X -> P 9 | Y -> P 10 | } 11 | -------------------------------------------------------------------------------- /img/ch_06_d_sep_04.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_d_sep_04.png -------------------------------------------------------------------------------- /img/ch_06_equivalent_estimands_00: -------------------------------------------------------------------------------- 1 | digraph { 2 | X [pos="0,0!"] 3 | Y [pos="3,0!"] 4 | A [pos=".5,2!"] 5 | B [pos="1.75,1!"] 6 | X -> Y 7 | A -> X 8 | A -> B 9 | B -> Y 10 | } 11 | -------------------------------------------------------------------------------- /img/ch_06_equivalent_estimands_00.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_equivalent_estimands_00.png -------------------------------------------------------------------------------- /img/ch_06_equivalent_estimands_01: -------------------------------------------------------------------------------- 1 | digraph { 2 | X [pos="0,0!"] 3 | Y [pos="3,0!"] 4 | A [pos=".5,2!"] 5 | B [pos="1.75,1!"] 6 | A [style=dashed] 7 | X -> Y 8 | B -> Y 9 | A -> X [style=dashed] 10 | A -> B [style=dashed] 11 | } 12 | -------------------------------------------------------------------------------- /img/ch_06_equivalent_estimands_01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_equivalent_estimands_01.png -------------------------------------------------------------------------------- /img/ch_06_gps_00: -------------------------------------------------------------------------------- 1 | digraph { 2 | GPS [pos="0,0!"] 3 | Memory [pos="3,0!"] 4 | Motivation [pos="1.5,1.5!"] 5 | Motivation [style=dashed] 6 | Motivation -> GPS [style=dashed] 7 | Motivation -> Memory [style=dashed] 8 | } 9 | -------------------------------------------------------------------------------- /img/ch_06_gps_00.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_gps_00.png -------------------------------------------------------------------------------- /img/ch_06_gps_01: -------------------------------------------------------------------------------- 1 | digraph { 2 | GPS [pos="0,0!"] 3 | Memory [pos="3,0!"] 4 | Motivation [pos="1.5,1.5!"] 5 | Motivation [style=dashed] 6 | Motivation -> GPS [style=dashed] 7 | Motivation -> Memory [style=dashed] 8 | GPS -> Memory 9 | } 10 | -------------------------------------------------------------------------------- /img/ch_06_gps_01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_gps_01.png -------------------------------------------------------------------------------- /img/ch_06_gps_02: -------------------------------------------------------------------------------- 1 | digraph { 2 | GPS [pos="0,0!"] 3 | "Hippocampal vol." [pos="2.5,0!"] 4 | Memory [pos="5,0!"] 5 | Motivation [pos="2.5,1.5!"] 6 | Motivation [style=dashed] 7 | Motivation -> GPS [style=dashed] 8 | Motivation -> Memory [style=dashed] 9 | GPS -> "Hippocampal vol." 10 | "Hippocampal vol." -> Memory 11 | } 12 | -------------------------------------------------------------------------------- /img/ch_06_gps_02.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_gps_02.png -------------------------------------------------------------------------------- /img/ch_06_gps_03: -------------------------------------------------------------------------------- 1 | digraph { 2 | X [pos="0,0!"] 3 | Z [pos="2.5,0!"] 4 | Y [pos="5,0!"] 5 | U [pos="2.5,1.5!"] 6 | U [style=dashed] 7 | U -> X [style=dashed] 8 | U -> Y [style=dashed] 9 | X -> Z 10 | Z -> Y 11 | } 12 | -------------------------------------------------------------------------------- /img/ch_06_gps_03.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_gps_03.png -------------------------------------------------------------------------------- /img/ch_06_icecream: -------------------------------------------------------------------------------- 1 | digraph { 2 | ICE [pos="0,0!"] 3 | TMP [pos="1.5,.75!"] 4 | ACC [pos="3,0!"] 5 | TMP -> ICE 6 | TMP -> ACC 7 | } 8 | -------------------------------------------------------------------------------- /img/ch_06_icecream.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_icecream.png -------------------------------------------------------------------------------- /img/ch_06_instrumental_00: -------------------------------------------------------------------------------- 1 | digraph { 2 | Z [pos="0,0!"] 3 | X [pos="1.5,0!"] 4 | Y [pos="5,0!"] 5 | U [pos="3.25,1.5!"] 6 | U [style=dashed] 7 | U -> X [style=dashed] 8 | U -> Y [style=dashed] 9 | Z -> X 10 | X -> Y 11 | } 12 | -------------------------------------------------------------------------------- /img/ch_06_instrumental_00.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_06_instrumental_00.png -------------------------------------------------------------------------------- /img/ch_07_full_example: -------------------------------------------------------------------------------- 1 | digraph { 2 | S [pos="2,2.5!"] 3 | Q [pos="3,1!"] 4 | X [pos="3,0!"] 5 | Y [pos="1, 0!"] 6 | P [pos="1,2!"] 7 | S -> Q 8 | S -> Y 9 | Q -> X 10 | Q -> Y 11 | X -> P 12 | Y -> P 13 | X -> Y 14 | } 15 | -------------------------------------------------------------------------------- /img/ch_07_full_example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_07_full_example.png -------------------------------------------------------------------------------- /img/ch_08_modularity: -------------------------------------------------------------------------------- 1 | digraph { 2 | X [pos="1,2!"] 3 | Y [pos="4,.5!"] 4 | R [pos="2.5,.5!"] 5 | Z [pos="1, 0!"] 6 | X -> R 7 | Z -> R 8 | R -> Y 9 | X -> Y 10 | } 11 | -------------------------------------------------------------------------------- /img/ch_08_modularity.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_08_modularity.png -------------------------------------------------------------------------------- /img/ch_08_modularity_mod: -------------------------------------------------------------------------------- 1 | digraph { 2 | X [pos="1,2!"] 3 | Y [pos="4,.5!"] 4 | R [pos="2.5,.5!"] 5 | Z [pos="1, 0!"] 6 | R -> Y 7 | X -> Y 8 | } 9 | -------------------------------------------------------------------------------- /img/ch_08_modularity_mod.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_08_modularity_mod.png -------------------------------------------------------------------------------- /img/ch_08_selection: -------------------------------------------------------------------------------- 1 | digraph { 2 | T [pos="0,.5!"] 3 | Y [pos="1.5,0!"] 4 | C [pos="3,.5!"] 5 | T -> Y 6 | T -> C 7 | Y -> C 8 | } 9 | -------------------------------------------------------------------------------- /img/ch_08_selection.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_08_selection.png -------------------------------------------------------------------------------- /img/ch_08_selection_02: -------------------------------------------------------------------------------- 1 | digraph { 2 | T [pos="1.,1.5!"] 3 | Y [pos="0,0!"] 4 | C [pos="3,0!"] 5 | Z [pos="3,2!"] 6 | W [pos="0,2!"] 7 | Z -> C 8 | T -> C 9 | W -> Z 10 | W -> Y 11 | } 12 | -------------------------------------------------------------------------------- /img/ch_08_selection_02.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_08_selection_02.png -------------------------------------------------------------------------------- /img/ch_08_selection_03: -------------------------------------------------------------------------------- 1 | digraph { 2 | T [pos="1.,1.5!"] 3 | Y [pos="0,0!"] 4 | C [pos="3,0!"] 5 | Z [pos="3,2!"] 6 | W [pos="0,2!"] 7 | T -> Z 8 | Z -> C 9 | W -> Z 10 | W -> Y 11 | } 12 | -------------------------------------------------------------------------------- /img/ch_08_selection_03.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/img/ch_08_selection_03.png -------------------------------------------------------------------------------- /models/causal_bert_pytorch/CausalBert.py: -------------------------------------------------------------------------------- 1 | """ 2 | This code comes originally from https://github.com/rpryzant/causal-bert-pytorch 3 | At the time of writing the original code contained an error 4 | (https://github.com/rpryzant/causal-bert-pytorch/issues/6) 5 | that made using one of the methods (.ATE()) unadvisable. 6 | This version of code fixes this error. 7 | 8 | An extensible implementation of the Causal Bert model from 9 | "Adapting Text Embeddings for Causal Inference" 10 | (https://arxiv.org/abs/1905.12741) 11 | """ 12 | from collections import defaultdict 13 | import os 14 | import pickle 15 | 16 | import scipy 17 | from sklearn.model_selection import KFold 18 | 19 | from torch.utils.data import Dataset, TensorDataset, DataLoader, RandomSampler, SequentialSampler 20 | 21 | from transformers import BertTokenizer 22 | from transformers import BertModel, BertPreTrainedModel, AdamW, BertConfig 23 | from transformers import get_linear_schedule_with_warmup 24 | 25 | from transformers import DistilBertTokenizer 26 | from transformers import DistilBertModel, DistilBertPreTrainedModel 27 | 28 | from torch.nn import CrossEntropyLoss 29 | 30 | import torch 31 | import torch.nn as nn 32 | from scipy.special import softmax 33 | import numpy as np 34 | from scipy.special import logit 35 | from sklearn.linear_model import LogisticRegression 36 | 37 | from tqdm import tqdm 38 | import math 39 | 40 | CUDA = (torch.cuda.device_count() > 0) 41 | MASK_IDX = 103 42 | 43 | 44 | def platt_scale(outcome, probs): 45 | logits = logit(probs) 46 | logits = logits.reshape(-1, 1) 47 | log_reg = LogisticRegression(penalty='none', warm_start=True, solver='lbfgs') 48 | log_reg.fit(logits, outcome) 49 | return log_reg.predict_proba(logits) 50 | 51 | 52 | def gelu(x): 53 | return 0.5 * x * (1.0 + torch.erf(x / math.sqrt(2.0))) 54 | 55 | 56 | def make_bow_vector(ids, vocab_size, use_counts=False): 57 | """ Make a sparse BOW vector from a tensor of dense ids. 58 | Args: 59 | ids: torch.LongTensor [batch, features]. Dense tensor of ids. 60 | vocab_size: vocab size for this tensor. 61 | use_counts: if true, the outgoing BOW vector will contain 62 | feature counts. If false, will contain binary indicators. 63 | Returns: 64 | The sparse bag-of-words representation of ids. 65 | """ 66 | vec = torch.zeros(ids.shape[0], vocab_size) 67 | ones = torch.ones_like(ids, dtype=torch.float) 68 | if CUDA: 69 | vec = vec.cuda() 70 | ones = ones.cuda() 71 | ids = ids.cuda() 72 | 73 | vec.scatter_add_(1, ids, ones) 74 | vec[:, 1] = 0.0 # zero out pad 75 | if not use_counts: 76 | vec = (vec != 0).float() 77 | return vec 78 | 79 | 80 | 81 | class CausalBert(DistilBertPreTrainedModel): 82 | """The model itself.""" 83 | def __init__(self, config): 84 | super().__init__(config) 85 | 86 | self.num_labels = config.num_labels 87 | self.vocab_size = config.vocab_size 88 | 89 | self.distilbert = DistilBertModel(config) 90 | # self.dropout = nn.Dropout(config.hidden_dropout_prob) 91 | self.vocab_transform = nn.Linear(config.dim, config.dim) 92 | self.vocab_layer_norm = nn.LayerNorm(config.dim, eps=1e-12) 93 | self.vocab_projector = nn.Linear(config.dim, config.vocab_size) 94 | 95 | self.Q_cls = nn.ModuleDict() 96 | 97 | for T in range(2): 98 | # ModuleDict keys have to be strings.. 99 | self.Q_cls['%d' % T] = nn.Sequential( 100 | nn.Linear(config.hidden_size + self.num_labels, 200), 101 | nn.ReLU(), 102 | nn.Linear(200, self.num_labels)) 103 | 104 | self.g_cls = nn.Linear(config.hidden_size + self.num_labels, 105 | self.config.num_labels) 106 | 107 | self.init_weights() 108 | 109 | def forward(self, W_ids, W_len, W_mask, C, T, Y=None, use_mlm=True): 110 | if use_mlm: 111 | W_len = W_len.unsqueeze(1) - 2 # -2 because of the +1 below 112 | mask_class = torch.cuda.FloatTensor if CUDA else torch.FloatTensor 113 | mask = (mask_class(W_len.shape).uniform_() * W_len.float()).long() + 1 # + 1 to avoid CLS 114 | target_words = torch.gather(W_ids, 1, mask) 115 | mlm_labels = torch.ones(W_ids.shape).long() * -100 116 | if CUDA: 117 | mlm_labels = mlm_labels.cuda() 118 | mlm_labels.scatter_(1, mask, target_words) 119 | W_ids.scatter_(1, mask, MASK_IDX) 120 | 121 | outputs = self.distilbert(W_ids, attention_mask=W_mask) 122 | seq_output = outputs[0] 123 | pooled_output = seq_output[:, 0] 124 | # seq_output, pooled_output = outputs[:2] 125 | # pooled_output = self.dropout(pooled_output) 126 | 127 | if use_mlm: 128 | prediction_logits = self.vocab_transform(seq_output) # (bs, seq_length, dim) 129 | prediction_logits = gelu(prediction_logits) # (bs, seq_length, dim) 130 | prediction_logits = self.vocab_layer_norm(prediction_logits) # (bs, seq_length, dim) 131 | prediction_logits = self.vocab_projector(prediction_logits) # (bs, seq_length, vocab_size) 132 | mlm_loss = CrossEntropyLoss()( 133 | prediction_logits.view(-1, self.vocab_size), mlm_labels.view(-1)) 134 | else: 135 | mlm_loss = 0.0 136 | 137 | C_bow = make_bow_vector(C.unsqueeze(1), self.num_labels) 138 | inputs = torch.cat((pooled_output, C_bow), 1) 139 | 140 | # g logits 141 | g = self.g_cls(inputs) 142 | if Y is not None: # TODO train/test mode, this is a lil hacky 143 | g_loss = CrossEntropyLoss()(g.view(-1, self.num_labels), T.view(-1)) 144 | else: 145 | g_loss = 0.0 146 | 147 | # conditional expected outcome logits: 148 | # run each example through its corresponding T matrix 149 | # TODO this would be cleaner with sigmoid and BCELoss, but less general 150 | # (and I couldn't get it to work as well) 151 | Q_logits_T0 = self.Q_cls['0'](inputs) 152 | Q_logits_T1 = self.Q_cls['1'](inputs) 153 | 154 | if Y is not None: 155 | T0_indices = (T == 0).nonzero().squeeze() 156 | Y_T1_labels = Y.clone().scatter(0, T0_indices, -100) 157 | 158 | T1_indices = (T == 1).nonzero().squeeze() 159 | Y_T0_labels = Y.clone().scatter(0, T1_indices, -100) 160 | 161 | Q_loss_T1 = CrossEntropyLoss()( 162 | Q_logits_T1.view(-1, self.num_labels), Y_T1_labels) 163 | Q_loss_T0 = CrossEntropyLoss()( 164 | Q_logits_T0.view(-1, self.num_labels), Y_T0_labels) 165 | 166 | Q_loss = Q_loss_T0 + Q_loss_T1 167 | else: 168 | Q_loss = 0.0 169 | 170 | sm = nn.Softmax(dim=1) 171 | Q0 = sm(Q_logits_T0)[:, 1] 172 | Q1 = sm(Q_logits_T1)[:, 1] 173 | g = sm(g)[:, 1] 174 | 175 | return g, Q0, Q1, g_loss, Q_loss, mlm_loss 176 | 177 | 178 | 179 | class CausalBertWrapper: 180 | """Model wrapper in charge of training and inference.""" 181 | 182 | def __init__(self, g_weight=1.0, Q_weight=0.1, mlm_weight=1.0, 183 | batch_size=32): 184 | self.model = CausalBert.from_pretrained( 185 | "distilbert-base-uncased", 186 | num_labels=2, 187 | output_attentions=False, 188 | output_hidden_states=False) 189 | if CUDA: 190 | self.model = self.model.cuda() 191 | 192 | self.loss_weights = { 193 | 'g': g_weight, 194 | 'Q': Q_weight, 195 | 'mlm': mlm_weight 196 | } 197 | self.batch_size = batch_size 198 | 199 | 200 | def train(self, texts, confounds, treatments, outcomes, 201 | learning_rate=2e-5, epochs=3): 202 | dataloader = self.build_dataloader( 203 | texts, confounds, treatments, outcomes) 204 | 205 | self.model.train() 206 | optimizer = AdamW(self.model.parameters(), lr=learning_rate, eps=1e-8) 207 | total_steps = len(dataloader) * epochs 208 | warmup_steps = total_steps * 0.1 209 | scheduler = get_linear_schedule_with_warmup( 210 | optimizer, num_warmup_steps=warmup_steps, num_training_steps=total_steps) 211 | 212 | for epoch in range(epochs): 213 | losses = [] 214 | self.model.train() 215 | for step, batch in tqdm(enumerate(dataloader), total=len(dataloader)): 216 | if CUDA: 217 | batch = (x.cuda() for x in batch) 218 | W_ids, W_len, W_mask, C, T, Y = batch 219 | # while True: 220 | self.model.zero_grad() 221 | g, Q0, Q1, g_loss, Q_loss, mlm_loss = self.model(W_ids, W_len, W_mask, C, T, Y) 222 | loss = self.loss_weights['g'] * g_loss + \ 223 | self.loss_weights['Q'] * Q_loss + \ 224 | self.loss_weights['mlm'] * mlm_loss 225 | loss.backward() 226 | optimizer.step() 227 | scheduler.step() 228 | losses.append(loss.detach().cpu().item()) 229 | # print(np.mean(losses)) 230 | # if step > 5: continue 231 | return self.model 232 | 233 | 234 | def inference(self, texts, confounds, outcome=None): 235 | self.model.eval() 236 | dataloader = self.build_dataloader(texts, confounds, outcomes=outcome, 237 | sampler='sequential') 238 | Q0s = [] 239 | Q1s = [] 240 | Ys = [] 241 | for i, batch in tqdm(enumerate(dataloader), total=len(dataloader)): 242 | if CUDA: 243 | batch = (x.cuda() for x in batch) 244 | W_ids, W_len, W_mask, C, T, Y = batch 245 | g, Q0, Q1, _, _, _ = self.model(W_ids, W_len, W_mask, C, T, use_mlm=False) 246 | Q0s += Q0.detach().cpu().numpy().tolist() 247 | Q1s += Q1.detach().cpu().numpy().tolist() 248 | Ys += Y.detach().cpu().numpy().tolist() 249 | # if i > 5: break 250 | probs = np.array(list(zip(Q0s, Q1s))) 251 | preds = np.argmax(probs, axis=1) 252 | 253 | return probs, preds, Ys 254 | 255 | def ATE(self, C, W, Y=None, platt_scaling=False): 256 | Q_probs, _, Ys = self.inference(W, C, outcome=Y) 257 | if platt_scaling and Y is not None: 258 | Q0 = platt_scale(Ys, Q_probs[:, 0])[:, 0] 259 | Q1 = platt_scale(Ys, Q_probs[:, 1])[:, 1] 260 | else: 261 | Q0 = Q_probs[:, 0] 262 | Q1 = Q_probs[:, 1] 263 | 264 | return np.mean(Q1 - Q0) 265 | 266 | def build_dataloader(self, texts, confounds, treatments=None, outcomes=None, 267 | tokenizer=None, sampler='random'): 268 | def collate_CandT(data): 269 | # sort by (C, T), so you can get boundaries later 270 | # (do this here on cpu for speed) 271 | data.sort(key=lambda x: (x[1], x[2])) 272 | return data 273 | # fill with dummy values 274 | if treatments is None: 275 | treatments = [-1 for _ in range(len(confounds))] 276 | if outcomes is None: 277 | outcomes = [-1 for _ in range(len(treatments))] 278 | 279 | if tokenizer is None: 280 | tokenizer = DistilBertTokenizer.from_pretrained( 281 | 'distilbert-base-uncased', do_lower_case=True) 282 | 283 | out = defaultdict(list) 284 | for i, (W, C, T, Y) in enumerate(zip(texts, confounds, treatments, outcomes)): 285 | # out['W_raw'].append(W) 286 | encoded_sent = tokenizer.encode_plus(W, add_special_tokens=True, 287 | max_length=128, 288 | truncation=True, 289 | pad_to_max_length=True) 290 | 291 | out['W_ids'].append(encoded_sent['input_ids']) 292 | out['W_mask'].append(encoded_sent['attention_mask']) 293 | out['W_len'].append(sum(encoded_sent['attention_mask'])) 294 | out['Y'].append(Y) 295 | out['T'].append(T) 296 | out['C'].append(C) 297 | # if i > 100: break 298 | 299 | data = (torch.tensor(out[x]) for x in ['W_ids', 'W_len', 'W_mask', 'C', 'T', 'Y']) 300 | data = TensorDataset(*data) 301 | sampler = RandomSampler(data) if sampler == 'random' else SequentialSampler(data) 302 | dataloader = DataLoader(data, sampler=sampler, batch_size=self.batch_size) 303 | # collate_fn=collate_CandT) 304 | 305 | return dataloader 306 | 307 | 308 | if __name__ == '__main__': 309 | import pandas as pd 310 | 311 | df = pd.read_csv('testdata.csv') 312 | cb = CausalBertWrapper(batch_size=2, 313 | g_weight=0.1, Q_weight=0.1, mlm_weight=1) 314 | print(df.T) 315 | cb.train(df['text'], df['C'], df['T'], df['Y'], epochs=1) 316 | print(cb.ATE(df['C'], df.text, platt_scaling=True)) 317 | 318 | 319 | 320 | 321 | 322 | 323 | 324 | 325 | -------------------------------------------------------------------------------- /models/causal_bert_pytorch/README.md: -------------------------------------------------------------------------------- 1 | # This version 2 | 3 | This code comes originally from https://github.com/rpryzant/causal-bert-pytorch 4 | At the time of writing the original code contained an error 5 | (https://github.com/rpryzant/causal-bert-pytorch/issues/6) 6 | that made using one of the methods (.ATE()) unadvisable. 7 | This version of code fixes this error. 8 | 9 | I also fixed minor errors in the original README.md content 10 | 11 | Below starts the original README.md file: 12 | 13 | # Causal Bert -- in Pytorch! 14 | Pytorch implementation of ["Adapting Text Embeddings for Causal Inference" by Victor Veitch, Dhanya Sridhar, and David M. Blei](https://arxiv.org/pdf/1905.12741.pdf). 15 | 16 | # Quickstart 17 | 18 | ``` 19 | pip install -r requirements.txt 20 | python CausalBert.py 21 | ``` 22 | 23 | This will train a system on some test data and calculate an average treatment effect (ATE). 24 | 25 | # Description 26 | 27 | As input this system expects data where each row consists of: 28 | * Freeform **text** 29 | * A categorical variable (numerically coded) representing a **confound** 30 | * A binary **treatment variable** 31 | * A binary **outcome variable** 32 | 33 | Then the system will give the text to BERT, and use the BERT embeddings + confound to predict 34 | 1) _P(T | C, text)_ 35 | 2) _P(Y | T = 1, C, text)_ 36 | 3) _P(Y | T = 0, C, text)_ 37 | 4) The original masked language modeling objective of BERT. 38 | 39 | Once trained the resulting BERT embeddings will be sufficient for some causal inferences. 40 | 41 | # Example 42 | 43 | ``` 44 | df = pd.read_csv('testdata.csv') 45 | cb = CausalBertWrapper(batch_size=2, # init a model wrapper 46 | g_weight=0.1, Q_weight=0.1, mlm_weight=1) 47 | cb.train(df['text'], df['C'], df['T'], df['Y'], epochs=1) # train the model 48 | print(cb.ATE(df['C'], df['text'], platt_scaling=True)) # use the model to get an average treatment effect 49 | ``` 50 | 51 | 52 | # Usage 53 | 54 | **Initialize** the model wrapper (handles training and inference): 55 | 56 | ``` 57 | cb = CausalBertWrapper( 58 | batch_size=2, # batch size for training 59 | g_weight=1.0, # loss weight for P(T | C, text) prediction head 60 | Q_weight=0.1, # loss weight for P(Y | T, C, text) prediction heads 61 | mlm_weight=1) # loss weight for original MLM objective 62 | ``` 63 | 64 | Then **train** 65 | ``` 66 | cb.train( 67 | df['text'], # list of texts 68 | df['C'], # list of confounds 69 | df['T'], # list of treatments 70 | df['Y'], # list of outcomes 71 | epochs=1) # training epochs 72 | ``` 73 | 74 | Perform **inference** 75 | 76 | ``` 77 | ( ( P(Y=1|T=1), P(Y=0|T=1)), ( P(Y=1|T=0), P(Y=0|T=0) ), ... = cb.inference( 78 | df['text'], # list of texts 79 | df['C']) # list of confounds 80 | ``` 81 | 82 | Or estimate an **average treatment effect** 83 | 84 | ``` 85 | ATE = cb.ATE( 86 | df['text'], # list of texts 87 | df['C'], # list of confounds 88 | platt_scailing=False) # https://en.wikipedia.org/wiki/Platt_scaling 89 | ``` 90 | 91 | 92 | -------------------------------------------------------------------------------- /models/causal_bert_pytorch/__pycache__/CausalBert.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/models/causal_bert_pytorch/__pycache__/CausalBert.cpython-38.pyc -------------------------------------------------------------------------------- /models/causal_bert_pytorch/__pycache__/CausalBert.cpython-39.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PacktPublishing/Causal-Inference-and-Discovery-in-Python/de2a9df880e31f7217cb99b2c878feee03fc0cb1/models/causal_bert_pytorch/__pycache__/CausalBert.cpython-39.pyc -------------------------------------------------------------------------------- /models/causal_bert_pytorch/requirements.txt: -------------------------------------------------------------------------------- 1 | sklearn==0.0 2 | scipy==1.4.1 3 | torch==1.6.0 4 | keras==2.6.0 5 | transformers==4.12.2 6 | pandas==0.25.3 7 | --------------------------------------------------------------------------------