├── .gitignore ├── .vscode └── launch.json ├── 09_Gaussian_Process_Regression.ipynb ├── README.md ├── ex01 ├── Exercise_01.ipynb ├── Exercise_01.pdf └── Tutorial_01.pdf ├── ex02 ├── Exercise_02.ipynb ├── Exercise_02.pdf ├── Exercise_02_geist.ipynb └── Tutorial_02.pdf ├── ex03 ├── Exercise_03.ipynb ├── Exercise_03.pdf ├── MC_agent.py ├── board.py ├── game.py ├── human_agent.py ├── main.py └── random_agent.py ├── ex04 ├── Exercise_04.ipynb ├── Exercise_04.md ├── Exercise_04.pdf ├── data │ ├── total-confirmed-cases-of-covid-19-per-million-people.csv │ └── total-covid-deaths-per-million.csv └── figures │ ├── covid19_spread.png │ └── graphical_model.png ├── lindata.mat └── nlindata.mat /.gitignore: -------------------------------------------------------------------------------- 1 | *slide* 2 | *flipped* 3 | *.pyc 4 | -------------------------------------------------------------------------------- /.vscode/launch.json: -------------------------------------------------------------------------------- 1 | { 2 | // Use IntelliSense to learn about possible attributes. 3 | // Hover to view descriptions of existing attributes. 4 | // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 5 | "version": "0.2.0", 6 | "configurations": [ 7 | { 8 | "name": "Python: Current File", 9 | "type": "python", 10 | "request": "launch", 11 | "program": "${file}", 12 | "console": "integratedTerminal" 13 | } 14 | ] 15 | } -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ### Exercise 01 - Uncertainty 2 | 3 | Analyse importance of uncertainty in deep learning methods when extrapolating data 4 | 5 | ### Exercise 02 - Distributions 6 | 7 | Beta and Gamma distributions and conjugate priors 8 | 9 | ### Exercise 03 - Posterior Monte Carlo Sampling 10 | 11 | Autonomous agent that can play the game Battleships by using Monte Carlo sampling in order to approximate the posterior over hits given hit/miss observations -------------------------------------------------------------------------------- /ex01/Exercise_01.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Practical task \n", 8 | "You will model the famous [Keeling curve](https://www.esrl.noaa.gov/gmd/ccgg/trends/) using contemporary deep learning frameworks. This notebook will download the data and read it in as numpy arrays and provide example code in [pytorch](https://pytorch.org/). You are welcome to use other frameworks and change the dataset to better fit your model arhitecture. " 9 | ] 10 | }, 11 | { 12 | "cell_type": "code", 13 | "execution_count": null, 14 | "metadata": {}, 15 | "outputs": [], 16 | "source": [ 17 | "import numpy as np\n", 18 | "import matplotlib.pyplot as plt\n", 19 | "import os \n", 20 | "%matplotlib inline\n", 21 | "%matplotlib notebook\n" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "You can download the dataset from: ftp://aftp.cmdl.noaa.gov/products/trends/co2/co2_mm_mlo.txt\n", 29 | "\n", 30 | "or run the cell below which will check if the dataset is present in working directory and download it there if not." 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": null, 36 | "metadata": {}, 37 | "outputs": [], 38 | "source": [ 39 | "url=\"ftp://aftp.cmdl.noaa.gov/products/trends/co2/co2_mm_mlo.txt\"\n", 40 | "filename=os.path.split(url)[-1]\n", 41 | "\n", 42 | "if not os.path.exists(filename): \n", 43 | " from urllib import request\n", 44 | " print('Downloading data from:\\n'+url)\n", 45 | " request.urlretrieve(url,filename)\n" 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": {}, 51 | "source": [ 52 | "Read in the data as `numpy` array" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "metadata": {}, 59 | "outputs": [], 60 | "source": [ 61 | "data=np.genfromtxt(filename,comments='#',missing_values='-99.99',usecols=[0,1,3])\n", 62 | "\n", 63 | "Xfull=data[:,0]+(data[:,1]-1)/12.0\n", 64 | "Yfull=data[:,-1]\n", 65 | "\n", 66 | "missing_values=Yfull==-99.99\n", 67 | "\n", 68 | "Xfull=Xfull[~missing_values].reshape(-1,1)\n", 69 | "Yfull=Yfull[~missing_values].reshape(-1,1)\n", 70 | "\n", 71 | "data=[]\n", 72 | "\n", 73 | "\n", 74 | "# Use last 3 years as a test set \n", 75 | "test_length=3*12\n", 76 | "\n", 77 | "Xtrain_np = Xfull[:-test_length]\n", 78 | "Ytrain_np = Yfull[:-test_length]\n", 79 | "\n", 80 | "Xtest_np = Xfull[-test_length:]\n", 81 | "Ytest_np = Yfull[-test_length:]\n", 82 | "\n", 83 | "# Data for prediction\n", 84 | "Xpred_np = Xtrain_np[-1]+np.arange(1,40*12).reshape(-1,1)/12\n" 85 | ] 86 | }, 87 | { 88 | "cell_type": "markdown", 89 | "metadata": {}, 90 | "source": [ 91 | "Plot the train and test data" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": null, 97 | "metadata": {}, 98 | "outputs": [], 99 | "source": [ 100 | "plt.scatter(Xtrain_np,Ytrain_np,marker='.',linewidths=0.05)\n", 101 | "plt.scatter(Xtest_np,Ytest_np,marker='.',linewidths=0.05)\n", 102 | "plt.title(r'$CO_2$ concentration')\n", 103 | "plt.xlabel('year')\n", 104 | "plt.ylabel(r'$CO_2$ [ppm]')\n", 105 | "plt.show()" 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "metadata": {}, 111 | "source": [ 112 | "# Preprocess the data\n", 113 | "You are allowed to change the preprocessing and structure of the dataset to better suit your model architecture.\n", 114 | "\n", 115 | "Below is a simple example." 116 | ] 117 | }, 118 | { 119 | "cell_type": "code", 120 | "execution_count": null, 121 | "metadata": {}, 122 | "outputs": [], 123 | "source": [ 124 | "xshift=np.mean(Xtrain_np)\n", 125 | "yshift=np.mean(Ytrain_np)\n", 126 | "\n", 127 | "xscale=np.std(Xtrain_np)\n", 128 | "yscale=np.std(Ytrain_np)\n", 129 | "\n", 130 | "\n", 131 | "Xtrain_np-=xshift\n", 132 | "Xtrain_np/=xscale\n", 133 | "Ytrain_np-=yshift\n", 134 | "Ytrain_np/=yscale\n", 135 | "\n", 136 | "Xtest_np-=xshift\n", 137 | "Xtest_np/=xscale\n", 138 | "Ytest_np-=yshift\n", 139 | "Ytest_np/=yscale\n", 140 | "\n", 141 | "\n", 142 | "Xpred_np-=xshift\n", 143 | "Xpred_np/=xscale" 144 | ] 145 | }, 146 | { 147 | "cell_type": "markdown", 148 | "metadata": {}, 149 | "source": [ 150 | "# Task:\n", 151 | "Use a deep learning framwework of your choice to build and train a model to predict the CO$_2$ concentration up until year 2060.\n", 152 | "Below is a Pytorch example with 2 ways to construct a model that you can use as a starting point. \n", 153 | "\n", 154 | "See the [nn module](https://pytorch.org/docs/stable/nn.html#) of Pytorch for inspiration on models and modules." 155 | ] 156 | }, 157 | { 158 | "cell_type": "code", 159 | "execution_count": null, 160 | "metadata": {}, 161 | "outputs": [], 162 | "source": [ 163 | "import torch\n", 164 | "\n", 165 | "if torch.cuda.is_available(): \n", 166 | " device = torch.device(\"cuda:0\") \n", 167 | "else: \n", 168 | " device = torch.device(\"cpu\")\n", 169 | "print(device)\n", 170 | "# Load the data into pytorch tensors for training\n", 171 | "Xtrain=torch.FloatTensor(Xtrain_np).to(device)\n", 172 | "Ytrain=torch.FloatTensor(Ytrain_np).to(device)\n", 173 | "\n", 174 | "Xtest=torch.FloatTensor(Xtest_np).to(device)\n", 175 | "Ytest=torch.FloatTensor(Ytest_np).to(device)\n" 176 | ] 177 | }, 178 | { 179 | "cell_type": "code", 180 | "execution_count": null, 181 | "metadata": {}, 182 | "outputs": [], 183 | "source": [ 184 | "import torch.nn as nn\n", 185 | "\n", 186 | "\n", 187 | "\n", 188 | "#############\n", 189 | "# 1st way to define model\n", 190 | "#############\n", 191 | "class NN(nn.Module):\n", 192 | " def __init__(self):\n", 193 | " super(NN, self).__init__()\n", 194 | " self.linear1=nn.Linear(1,100)\n", 195 | " self.linear2=nn.Linear(100,1)\n", 196 | " \n", 197 | " def forward(self,x):\n", 198 | " x=nn.ReLU(self.linear1(x))\n", 199 | " x=self.linear2(x)\n", 200 | " return x\n", 201 | " \n", 202 | "model = NN\n", 203 | "\n", 204 | "\n", 205 | "#############\n", 206 | "# 2nd way to define model\n", 207 | "#############\n", 208 | "model = torch.nn.Sequential(\n", 209 | " nn.Linear(1,100),\n", 210 | " nn.ReLU(),\n", 211 | " nn.Linear(100,1)\n", 212 | ")\n", 213 | "\n", 214 | "\n", 215 | "model.to(device)\n", 216 | "\n", 217 | "criterion=nn.MSELoss()\n", 218 | "\n", 219 | "# Change optimizer and hyperparameters\n", 220 | "optimizer=torch.optim.SGD(model.parameters(),lr=1e-3,momentum=0.9,weight_decay=0)\n", 221 | "\n", 222 | "\n", 223 | "NUM_EPOCHS=500\n", 224 | "\n", 225 | "for i in range(NUM_EPOCHS):\n", 226 | " optimizer.zero_grad()\n", 227 | " pred=model(Xtrain)\n", 228 | " loss=criterion(pred,Ytrain)\n", 229 | " loss.backward()\n", 230 | " optimizer.step()\n", 231 | " if i%100==0:\n", 232 | " with torch.no_grad():\n", 233 | " test_loss=criterion(model(Xtest),Ytest)\n", 234 | " print(f'{i}: {loss.item()} {test_loss.item()}')\n", 235 | "\n", 236 | "\n" 237 | ] 238 | }, 239 | { 240 | "cell_type": "markdown", 241 | "metadata": {}, 242 | "source": [ 243 | "Plot the prediction and extrapolation for analysis" 244 | ] 245 | }, 246 | { 247 | "cell_type": "code", 248 | "execution_count": null, 249 | "metadata": {}, 250 | "outputs": [], 251 | "source": [ 252 | "Xpred=torch.FloatTensor(Xpred_np).to(device)\n", 253 | "\n", 254 | "with torch.no_grad():\n", 255 | " pred=model(Xtrain)\n", 256 | " predf=model(Xpred)\n", 257 | " \n", 258 | "plt.plot(Xtrain.cpu().numpy()*xscale+xshift,pred.cpu().numpy()*yscale+yshift,color='C0',label='Prediction')\n", 259 | "plt.plot(Xtrain_np*xscale+xshift,Ytrain_np*yscale+yshift,'x',color='C0',alpha=0.2,label='Train data')\n", 260 | "plt.plot(Xtest_np*xscale+xshift,Ytest_np*yscale+yshift,'o',color='C1',alpha=0.2,label='Test data')\n", 261 | "plt.plot(Xpred.cpu().numpy()*xscale+xshift,predf.cpu().numpy()*yscale+yshift,color='C3',label='Extrapolation')\n", 262 | "\n", 263 | "plt.legend()\n", 264 | "plt.xlabel(r'Year')\n", 265 | "plt.ylabel(r'CO$_2$ [ppm]')\n", 266 | "plt.show()" 267 | ] 268 | }, 269 | { 270 | "cell_type": "markdown", 271 | "metadata": {}, 272 | "source": [ 273 | "# Question:\n", 274 | "**How is your model performing, is the behaviour expected?** \n", 275 | "\n", 276 | "*answer:*\n", 277 | "\n", 278 | "\n", 279 | "**Do you trust the extrapolation why/why not?** \n", 280 | "\n", 281 | "*answer:*\n", 282 | "\n", 283 | "**How would you change the architecture or dataset to improve the extrapolation?**\n", 284 | "\n", 285 | "*answer:*\n" 286 | ] 287 | } 288 | ], 289 | "metadata": { 290 | "kernelspec": { 291 | "display_name": "Python 3", 292 | "language": "python", 293 | "name": "python3" 294 | }, 295 | "language_info": { 296 | "codemirror_mode": { 297 | "name": "ipython", 298 | "version": 3 299 | }, 300 | "file_extension": ".py", 301 | "mimetype": "text/x-python", 302 | "name": "python", 303 | "nbconvert_exporter": "python", 304 | "pygments_lexer": "ipython3", 305 | "version": "3.7.5" 306 | } 307 | }, 308 | "nbformat": 4, 309 | "nbformat_minor": 4 310 | } 311 | -------------------------------------------------------------------------------- /ex01/Exercise_01.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucasrm25/Probabilistic-Machine-Learning/ad017c952c027ef37e0d6ba0bb5e4d0226bdf75c/ex01/Exercise_01.pdf -------------------------------------------------------------------------------- /ex01/Tutorial_01.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucasrm25/Probabilistic-Machine-Learning/ad017c952c027ef37e0d6ba0bb5e4d0226bdf75c/ex01/Tutorial_01.pdf -------------------------------------------------------------------------------- /ex02/Exercise_02.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Exercise Sheet #2: Battleships (1/2)\n", 8 | "\n", 9 | "### Probabilistic Machine Learning\n", 10 | "\n", 11 | "- **Lecturer**: Prof. Philipp Hennig\n", 12 | "- **Term**: SoSe 2020\n", 13 | "- **Due Date**: Monday, 04 May 2020, 10am\n", 14 | "\n", 15 | "\n", 16 | "![battleship rules](https://upload.wikimedia.org/wikipedia/commons/e/e4/Battleships_Paper_Game.svg)\n", 17 | "\n", 18 | "Over the course of two weeks, we will implement an agent that can play the pen-and-paper game _Battleships_. The goal of this exercise sheet is to find exact prior probabilities of getting a hit by enumeration, and to update to a posterior given observations of hits and misses. This week we will understand why we can't construct the agent with this approach. \n", 19 | "\n", 20 | "Next week we will use Monte Carlo techniques to build an agent that we can play against :). Stay tuned!" 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": null, 26 | "metadata": {}, 27 | "outputs": [], 28 | "source": [ 29 | "import numpy as np\n", 30 | "import matplotlib\n", 31 | "import matplotlib.pyplot as plt" 32 | ] 33 | }, 34 | { 35 | "cell_type": "markdown", 36 | "metadata": {}, 37 | "source": [ 38 | "### **2.1** A priori probability for a hit with one boat\n", 39 | "\n", 40 | "**Tasks**\n", 41 | "1. Write a function that takes the length of a boat and that returns the prior probability to observe a hit.\n", 42 | "2. Plot the prior for a carrier (length 5).\n", 43 | "\n", 44 | "\n", 45 | "*Hint:* You can find this probability by enumerating all possible positions of the boat." 46 | ] 47 | }, 48 | { 49 | "cell_type": "code", 50 | "execution_count": null, 51 | "metadata": {}, 52 | "outputs": [], 53 | "source": [ 54 | "field_size = 10" 55 | ] 56 | }, 57 | { 58 | "cell_type": "markdown", 59 | "metadata": {}, 60 | "source": [ 61 | "_Note:_ Let `i` denote the _row_ index and `j` the _column_ index" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": null, 67 | "metadata": {}, 68 | "outputs": [], 69 | "source": [ 70 | "def boat_prior(boat_length, field_size):\n", 71 | " \"\"\"\n", 72 | " Computes the prior probability to get a hit given the boat length and the size of the board.\n", 73 | " For a single-boat setup only.\n", 74 | " :param boat_length: Length of the boat, type: int\n", 75 | " :param field_size: size of the board, type: int\n", 76 | " \n", 77 | " :returns: np.ndarray of size (field_size, field_size) containing the probability of a hit for every field.\n", 78 | " \"\"\"\n", 79 | " \n", 80 | " # number of positions that a boat fits on the fired position divided by all possible positions that the boat can assume\n", 81 | "\n", 82 | " #######################################\n", 83 | " #\n", 84 | " # YOUR CODE GOES HERE\n", 85 | " #\n", 86 | " #######################################\n", 87 | "\n", 88 | " return NotImplementedError\n", 89 | "\n", 90 | "L_carrier = 5\n", 91 | "carrier_prior = boat_prior(L_carrier, field_size)\n", 92 | "plt.imshow(carrier_prior)" 93 | ] 94 | }, 95 | { 96 | "cell_type": "code", 97 | "execution_count": null, 98 | "metadata": {}, 99 | "outputs": [], 100 | "source": [ 101 | "# Sanity check: carrier_prior should sum to the length of the boat\n", 102 | "carrier_prior.sum()" 103 | ] 104 | }, 105 | { 106 | "cell_type": "markdown", 107 | "metadata": {}, 108 | "source": [ 109 | "### **2.2** Updating to a posterior given hit/miss observations\n", 110 | "\n", 111 | "Let us define an `observation_board` that contains the observations. We use the following notation: \n", 112 | " `0` unseen field \n", 113 | " `-1` miss (water) \n", 114 | " `1` hit " 115 | ] 116 | }, 117 | { 118 | "cell_type": "markdown", 119 | "metadata": {}, 120 | "source": [ 121 | "**Task 1:** Write a function that takes as input the `observation_board` as well as the length of one boat, and that returns an array that contains the probability to get a hit at every coordinate of the field.\n", 122 | "\n", 123 | "*Hint:* Again, this can be achieved by enumeration. Take care of all the constraints imposed by the `observation_board`!" 124 | ] 125 | }, 126 | { 127 | "cell_type": "code", 128 | "execution_count": null, 129 | "metadata": {}, 130 | "outputs": [], 131 | "source": [ 132 | "def posterior_one_ship(observation_board, boat_length):\n", 133 | " \"\"\"\n", 134 | " Computes the posterior probability to get a hit given an array of observed hits and misses as well as the boat length.\n", 135 | " For a single-boat setup only.\n", 136 | " :param observation_board: the board containing unobserved locations (0), misses (-1), and hits (1), type: np.ndarray\n", 137 | " :param boat_length: Length of the boat, type: int\n", 138 | " \n", 139 | " :returns: np.ndarray of the same size as the observation board containing the probability of a hit \n", 140 | " for every field given the observations\n", 141 | " \"\"\"\n", 142 | " \n", 143 | " #######################################\n", 144 | " #\n", 145 | " # YOUR CODE GOES HERE\n", 146 | " #\n", 147 | " #######################################\n", 148 | "\n", 149 | " return NotImplementedError" 150 | ] 151 | }, 152 | { 153 | "cell_type": "markdown", 154 | "metadata": {}, 155 | "source": [ 156 | "**Task 2:** Test your function on the following `observation_board`s (just run the cells)" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "metadata": {}, 163 | "outputs": [], 164 | "source": [ 165 | "# board 1: Only misses\n", 166 | "obs_board_1 = np.zeros((field_size, field_size)); obs_board_1[[1, 2, 5, 8], [8, 3, 4, 6]] = -1\n", 167 | "\n", 168 | "# board 2: a few misses, one hit\n", 169 | "obs_board_2 = np.copy(obs_board_1); obs_board_2[7, 1] = 1\n", 170 | "\n", 171 | "# board 3: a few misses, two hits\n", 172 | "obs_board_3 = np.copy(obs_board_2); obs_board_3[6, 1] = 1\n", 173 | "\n", 174 | "# board 4: two for one boat impossible hits: This should cause an error\n", 175 | "obs_board_4 = np.copy(obs_board_2); obs_board_4[6, 2] = 1" 176 | ] 177 | }, 178 | { 179 | "cell_type": "code", 180 | "execution_count": null, 181 | "metadata": {}, 182 | "outputs": [], 183 | "source": [ 184 | "### VISUALIZATION\n", 185 | "from matplotlib import colors\n", 186 | "\n", 187 | "# making a custom discrete colormap for hits and misses\n", 188 | "cmap_discrete = colors.ListedColormap(['white', 'C0','darkred'])\n", 189 | "boundaries = [-1.5, -0.5, 0.5, 1.5]\n", 190 | "norm = colors.BoundaryNorm(boundaries, cmap_discrete.N, clip=True)\n", 191 | "\n", 192 | "# Plot the boards and the corresponding posterior\n", 193 | "f, axs = plt.subplots(2, 4, figsize=(12,6))\n", 194 | "\n", 195 | "cmap = matplotlib.cm.viridis # Can be any colormap that you want after the cm\n", 196 | "cmap.set_bad(color='w')\n", 197 | "\n", 198 | "# board 1\n", 199 | "post_1 = posterior_one_ship(obs_board_1, 5)\n", 200 | "axs[0, 0].imshow(obs_board_1, cmap=cmap_discrete, norm=norm)\n", 201 | "axs[1, 0].imshow(np.ma.masked_where(post_1 == 0, post_1), cmap=cmap)\n", 202 | "\n", 203 | "# board 2\n", 204 | "axs[0, 1].imshow(obs_board_2, cmap=cmap_discrete, norm=norm)\n", 205 | "axs[1, 1].imshow(posterior_one_ship(obs_board_2, 5))\n", 206 | "\n", 207 | "# board 3\n", 208 | "axs[0, 2].imshow(obs_board_3, cmap=cmap_discrete, norm=norm)\n", 209 | "axs[1, 2].imshow(posterior_one_ship(obs_board_3, 5))\n", 210 | "\n", 211 | "# board 4\n", 212 | "axs[0, 3].imshow(obs_board_4, cmap=cmap_discrete, norm=norm)\n", 213 | "try:\n", 214 | " axs[1, 3].imshow(posterior_one_ship(obs_board_4, 5))\n", 215 | "except:\n", 216 | " pass" 217 | ] 218 | }, 219 | { 220 | "cell_type": "code", 221 | "execution_count": null, 222 | "metadata": {}, 223 | "outputs": [], 224 | "source": [ 225 | "# board 4 should raise an error\n", 226 | "posterior_one_ship(obs_board_4, 5)" 227 | ] 228 | }, 229 | { 230 | "cell_type": "markdown", 231 | "metadata": {}, 232 | "source": [ 233 | "### **2.3** Towards battleship with more than one boat\n", 234 | "_Note: No coding required!_\n", 235 | "\n", 236 | "Next week's assignment will deal with the full game of Battleships, which, according to the above rules, contains seven ships.\n", 237 | "\n", 238 | "1. Think about how you would need to modify your above routines to compute \n", 239 | "        (a) the prior over ship locations \n", 240 | "        (b) the posterior over ship locations given hit/miss observations \n", 241 | " by enumerating all states. Describe the changes you would need to make to your code.\n", 242 | "2. Why will it be hard to compute the posterior with multiple boats?" 243 | ] 244 | }, 245 | { 246 | "cell_type": "markdown", 247 | "metadata": {}, 248 | "source": [] 249 | } 250 | ], 251 | "metadata": { 252 | "kernelspec": { 253 | "display_name": "Python 3", 254 | "language": "python", 255 | "name": "python3" 256 | }, 257 | "language_info": { 258 | "codemirror_mode": { 259 | "name": "ipython", 260 | "version": 3 261 | }, 262 | "file_extension": ".py", 263 | "mimetype": "text/x-python", 264 | "name": "python", 265 | "nbconvert_exporter": "python", 266 | "pygments_lexer": "ipython3", 267 | "version": "3.7.4" 268 | } 269 | }, 270 | "nbformat": 4, 271 | "nbformat_minor": 4 272 | } -------------------------------------------------------------------------------- /ex02/Exercise_02.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucasrm25/Probabilistic-Machine-Learning/ad017c952c027ef37e0d6ba0bb5e4d0226bdf75c/ex02/Exercise_02.pdf -------------------------------------------------------------------------------- /ex02/Exercise_02_geist.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Exercise Sheet #2" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "# Exercise Sheet #2: Battleships (1/2)\n", 15 | "\n", 16 | "### Probabilistic Machine Learning\n", 17 | "\n", 18 | "- **Lecturer**: Prof. Philipp Hennig\n", 19 | "- **Term**: SoSe 2020\n", 20 | "- **Due Date**: Monday, 04 May 2020, 10am\n", 21 | "\n", 22 | "\n", 23 | "![battleship rules](https://upload.wikimedia.org/wikipedia/commons/e/e4/Battleships_Paper_Game.svg)\n", 24 | "\n", 25 | "Over the course of two weeks, we will implement an agent that can play the pen-and-paper game _Battleships_. The goal of this exercise sheet is to find exact prior probabilities of getting a hit by enumeration, and to update to a posterior given observations of hits and misses. This week we will understand why we can't construct the agent with this approach. \n", 26 | "\n", 27 | "Next week we will use Monte Carlo techniques to build an agent that we can play against :). Stay tuned!" 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "execution_count": 2, 33 | "metadata": {}, 34 | "outputs": [], 35 | "source": [ 36 | "import numpy as np\n", 37 | "import matplotlib\n", 38 | "import matplotlib.pyplot as plt" 39 | ] 40 | }, 41 | { 42 | "cell_type": "markdown", 43 | "metadata": {}, 44 | "source": [ 45 | "### **2.1** A priori probability for a hit with one boat\n", 46 | "\n", 47 | "**Tasks**\n", 48 | "1. Write a function that takes the length of a boat and that returns the prior probability to observe a hit.\n", 49 | "2. Plot the prior for a carrier (length 5).\n", 50 | "\n", 51 | "\n", 52 | "*Hint:* You can find this probability by enumerating all possible positions of the boat." 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": 3, 58 | "metadata": {}, 59 | "outputs": [], 60 | "source": [ 61 | "field_size = 10" 62 | ] 63 | }, 64 | { 65 | "cell_type": "markdown", 66 | "metadata": {}, 67 | "source": [ 68 | "_Note:_ Let `i` denote the _row_ index and `j` the _column_ index" 69 | ] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "execution_count": 11, 74 | "metadata": {}, 75 | "outputs": [ 76 | { 77 | "output_type": "execute_result", 78 | "data": { 79 | "text/plain": "" 80 | }, 81 | "metadata": {}, 82 | "execution_count": 11 83 | }, 84 | { 85 | "output_type": "display_data", 86 | "data": { 87 | "text/plain": "
", 88 | "image/svg+xml": "\n\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n", 89 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPgAAAD8CAYAAABaQGkdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAC1BJREFUeJzt3f+rngUZx/H35dn0rKksmIVsoxVEImIpaxGDqGUxU7Qf+kFJoQj2S4pSENYv0T8Q9UMEQy2j6QhtEGLmqIZJaTqd88tmmSiOVVNCdGPTnXn1w7m14xqde577y9N13i84nOc5595zXeM8n+f+8jz3fUVmIqmm08ZuQFJ/DLhUmAGXCjPgUmEGXCrMgEuFGXCpMAMuFWbApcKW9PGgp8cZOc3yPh56fsuXjVMXmFk23uvl8enRSgMwdXS82kuOvDle8cNHRil7lMO8ka/HfMv1EvBplvOJ+GwfDz2/Cy8cpy7w8oUjvagBr5w37keOV+yb97nWm5V7Do9Wmwf3jFL2ofxtq+XcRJcKM+BSYQZcKsyAS4UZcKkwAy4VZsClwgy4VJgBlwoz4FJhrQIeEZsi4pmIeDYibuq7KUndmDfgETEF/Ai4FDgfuDoizu+7MUkL12YNvh54NjOfy8w3gG3Alf22JakLbQK+Cnhxzv39zc8kTbg2p4ue7DzA/zo3MSI2A5sBpnnPAtuS1IU2a/D9wJo591cDB05cKDO3ZOa6zFy3lDO66k/SArQJ+MPAhyPigxFxOnAV8Kt+25LUhXk30TNzJiKuA34DTAG3ZuZTvXcmacFaXbIpM+8B7um5F0kd85NsUmEGXCrMgEuFGXCpMAMuFWbApcIMuFSYAZcKM+BSYQZcKqyX6aIsXzbalM/FOuHznPNeHq02wEusHLH6eH/zlYw0zXbPH1st5hpcKsyAS4UZcKkwAy4VZsClwgy4VJgBlwoz4FJhBlwqzIBLhRlwqbA200VvjYiDEfHkEA1J6k6bNfhPgU099yGpB/MGPDPvB/41QC+SOuY+uFRYZwGPiM0R8UhEPHLs2OGuHlbSAnQW8HeMD1463gn4kv7DTXSpsDZvk90B/An4SETsj4iv9d+WpC60mQ9+9RCNSOqem+hSYQZcKsyAS4UZcKkwAy4VZsClwgy4VJgBlwoz4FJhBlwqrJfxwTPLThttjO9iHeH75Q88PFptgK18fLTai3F08cxf262bXYNLhRlwqTADLhVmwKXCDLhUmAGXCjPgUmEGXCrMgEuFGXCpMAMuFdbmuuhrIuL3EbE3Ip6KiBuGaEzSwrU52WQG+GZmPhoRZwG7ImJHZj7dc2+SFqjN+OC/Z+ajze3XgL3Aqr4bk7Rwp7QPHhFrgYuAh/poRlK3Wgc8Is4E7gJuzMxXT/L7t8cHzxx1fLA0CVoFPCKWMhvurZn5y5MtM3d88JJpxwdLk6DNUfQAbgH2Zub3+29JUlfarME3ANcCGyNid/P1hZ77ktSBNuODHwBigF4kdcxPskmFGXCpMAMuFWbApcIMuFSYAZcKM+BSYQZcKsyAS4UZcKmwXsYHH58eb4zvYh3he/17Xxit9tgW4+ji49PtlnMNLhVmwKXCDLhUmAGXCjPgUmEGXCrMgEuFGXCpMAMuFWbApcIMuFRYm8EH0xHx54h4vBkf/L0hGpO0cG1ONnkd2JiZh5oRRg9ExK8z88Gee5O0QG0GHyRwqLm7tPka51QxSaek7fDBqYjYDRwEdmSm44Ol/wOtAp6ZxzPzY8BqYH1EXHDiMnPHBx8/7PhgaRKc0lH0zHwF2AlsOsnv3h4fPLXc8cHSJGhzFP2ciFjR3F4GXALs67sxSQvX5ij6ucBtETHF7AvCLzLz7n7bktSFNkfR9wAXDdCLpI75STapMAMuFWbApcIMuFSYAZcKM+BSYQZcKsyAS4UZcKkwAy4VZsClwnqZDz51FFbsiz4eel5jzWuGcedUj23rCyPO6N433t98rOf5P4+2W841uFSYAZcKM+BSYQZcKsyAS4UZcKkwAy4VZsClwgy4VJgBlwprHfBmPtljEeE10aX/E6eyBr8B2NtXI5K613a66GrgMuDmftuR1KW2a/AfAN8C3uyxF0kdazN88HLgYGbumme5t8cHzxx1fLA0CdqswTcAV0TE88A2YGNE/PzEheaOD14y7fhgaRLMG/DM/HZmrs7MtcBVwO8y85reO5O0YL4PLhV2SpdsysydwM5eOpHUOdfgUmEGXCrMgEuFGXCpMAMuFWbApcIMuFSYAZcKM+BSYQZcKsyAS4X1Mj54yZE3WblnrHPCxztVdTGPLl6MI3yB0Z7nfzvS7torrsGlwgy4VJgBlwoz4FJhBlwqzIBLhRlwqTADLhVmwKXCDLhUmAGXCmv1WfRmbNFrwHFgJjPX9dmUpG6cyskmn8nMl3vrRFLn3ESXCmsb8ATui4hdEbH5ZAvMHR987Jjjg6VJ0HYTfUNmHoiI9wE7ImJfZt4/d4HM3AJsATj7zFXZcZ+S3oVWa/DMPNB8PwhsB9b32ZSkbswb8IhYHhFnvXUb+DzwZN+NSVq4Npvo7we2R8Rby9+emff22pWkTswb8Mx8DvjoAL1I6phvk0mFGXCpMAMuFWbApcIMuFSYAZcKM+BSYQZcKsyAS4UZcKmwXsYHc/gIPLinl4eez0ouHKXurMU5uhgW5whfYLTnOXmk1WKuwaXCDLhUmAGXCjPgUmEGXCrMgEuFGXCpMAMuFWbApcIMuFSYAZcKaxXwiFgREXdGxL6I2BsRn+y7MUkL1/Zkkx8C92bmlyLidOA9PfYkqSPzBjwizgY+BXwFIDPfAN7oty1JXWizif4h4CXgJxHxWETc3Mwoe4d3jA/m9c4blXTq2gR8CXAx8OPMvAg4DNx04kKZuSUz12XmuqWc0XGbkt6NNgHfD+zPzIea+3cyG3hJE27egGfmP4AXI+IjzY8+Czzda1eSOtH2KPr1wNbmCPpzwFf7a0lSV1oFPDN3A+t67kVSx/wkm1SYAZcKM+BSYQZcKsyAS4UZcKkwAy4VZsClwgy4VJgBlwqLzOz+QSNeAl54l/98JfByh+1Y29oVa38gM8+Zb6FeAr4QEfFIZo7yuXdrW7tabTfRpcIMuFTYJAZ8i7Wtbe1uTNw+uKTuTOIaXFJHJirgEbEpIp6JiGcj4r+u3Npj3Vsj4mBEPDlUzTm110TE75uJMU9FxA0D1p6OiD9HxONN7e8NVXtOD1PN5bjvHrju8xHxRETsjohHBq492KSgidlEj4gp4C/A55i9kuvDwNWZ2fsFHiPiU8Ah4GeZeUHf9U6ofS5wbmY+GhFnAbuALw70/w5geWYeioilwAPADZn5YN+15/TwDWYvB3Z2Zl4+YN3ngXWZOfj74BFxG/CHzLz5rUlBmflKH7UmaQ2+Hng2M59rpqdsA64conBm3g/8a4haJ6n998x8tLn9GrAXWDVQ7czMQ83dpc3XYK/4EbEauAy4eaiaY5szKegWmJ0U1Fe4YbICvgp4cc79/Qz0RJ8UEbEWuAh46H8v2WnNqYjYDRwEdsy5/v0QfgB8C3hzwJpvSeC+iNgVEZsHrNtqUlBXJingcZKfTcb+wwAi4kzgLuDGzHx1qLqZeTwzPwasBtZHxCC7KBFxOXAwM3cNUe8kNmTmxcClwNeb3bQhtJoU1JVJCvh+YM2c+6uBAyP1Mqhm//cuYGtm/nKMHprNxJ3ApoFKbgCuaPaFtwEbI+LnA9UmMw803w8C25ndRRzCoJOCJingDwMfjogPNgcergJ+NXJPvWsOdN0C7M3M7w9c+5yIWNHcXgZcAuwbonZmfjszV2fmWmb/1r/LzGuGqB0Ry5sDmjSbx58HBnkHZehJQW0nm/QuM2ci4jrgN8AUcGtmPjVE7Yi4A/g0sDIi9gPfzcxbhqjN7JrsWuCJZl8Y4DuZec8Atc8FbmvewTgN+EVmDvp21UjeD2yffW1lCXB7Zt47YP3BJgVNzNtkkro3SZvokjpmwKXCDLhUmAGXCjPgUmEGXCrMgEuFGXCpsH8DMLT57q+v2KoAAAAASUVORK5CYII=\n" 90 | }, 91 | "metadata": { 92 | "needs_background": "light" 93 | } 94 | } 95 | ], 96 | "source": [ 97 | "def boat_prior(boat_length, field_size):\n", 98 | " \"\"\"\n", 99 | " Computes the prior probability to get a hit given the boat length and the size of the board.\n", 100 | " For a single-boat setup only.\n", 101 | " :param boat_length: Length of the boat, type: int\n", 102 | " :param field_size: size of the board, type: int\n", 103 | " \n", 104 | " :returns: np.ndarray of size (field_size, field_size) containing the probability of a hit for every field.\n", 105 | " \"\"\"\n", 106 | " tmp_field = np.zeros((field_size, field_size))\n", 107 | " \n", 108 | " # Vertical boat position (all columns are the same)\n", 109 | " for i in range(field_size-boat_length+1):\n", 110 | " tmp_field[i:i+boat_length, :] += 1\n", 111 | " \n", 112 | " # Horizontal boat position are the same tabel just transposed\n", 113 | " tmp_field += tmp_field.T\n", 114 | " \n", 115 | " # Normalize table\n", 116 | " normalization_constant = boat_length/(2 * (field_size-boat_length+1) * field_size * boat_length)\n", 117 | " \n", 118 | " return normalization_constant * tmp_field\n", 119 | "\n", 120 | "L_carrier = 4\n", 121 | "carrier_prior = boat_prior(L_carrier, field_size = 7)\n", 122 | "plt.imshow(carrier_prior)" 123 | ] 124 | }, 125 | { 126 | "cell_type": "code", 127 | "execution_count": 12, 128 | "metadata": {}, 129 | "outputs": [ 130 | { 131 | "output_type": "execute_result", 132 | "data": { 133 | "text/plain": "3.9999999999999996" 134 | }, 135 | "metadata": {}, 136 | "execution_count": 12 137 | } 138 | ], 139 | "source": [ 140 | "# Sanity check: carrier_prior should sum to the length of the boat\n", 141 | "carrier_prior.sum()" 142 | ] 143 | }, 144 | { 145 | "cell_type": "markdown", 146 | "metadata": {}, 147 | "source": [ 148 | "### **2.2** Updating to a posterior given hit/miss observations\n", 149 | "\n", 150 | "Let us define an `observation_board` that contains the observations. We use the following notation: \n", 151 | " `0` unseen field \n", 152 | " `-1` miss (water) \n", 153 | " `1` hit " 154 | ] 155 | }, 156 | { 157 | "cell_type": "markdown", 158 | "metadata": {}, 159 | "source": [ 160 | "**Task 1:** Write a function that takes as input the `observation_board` as well as the length of one boat, and that returns an array that contains the probability to get a hit at every coordinate of the field.\n", 161 | "\n", 162 | "*Hint:* Again, this can be achieved by enumeration. Take care of all the constraints imposed by the `observation_board`!" 163 | ] 164 | }, 165 | { 166 | "cell_type": "code", 167 | "execution_count": 18, 168 | "metadata": {}, 169 | "outputs": [], 170 | "source": [ 171 | "def posterior_one_ship(observation_board, boat_length):\n", 172 | " \"\"\"\n", 173 | " Computes the posterior probability to get a hit given an array of observed hits and misses as well as the boat length.\n", 174 | " For a single-boat setup only.\n", 175 | " :param observation_board: the board containing unobserved locations (0), misses (-1), and hits (1), type: np.ndarray\n", 176 | " :param boat_length: Length of the boat, type: int\n", 177 | " \n", 178 | " :returns: np.ndarray of the same size as the observation board containing the probability of a hit \n", 179 | " for every field given the observations\n", 180 | " \"\"\"\n", 181 | " \n", 182 | " tmp_posterior = np.zeros((field_size, field_size))\n", 183 | " \n", 184 | " n_hits = np.count_nonzero(observation_board == 1)\n", 185 | " print('n_hits', n_hits)\n", 186 | " # Vertical boat position (all columns are the same)\n", 187 | " for i in range(field_size):\n", 188 | " for j in range(field_size):\n", 189 | " \n", 190 | " # Check if vertical ship fits\n", 191 | " if i < field_size-boat_length+1:\n", 192 | " tmp_ship1 = observation_board[i:i+boat_length, j]\n", 193 | " # we only consider possible ship locations that contain all hits and no misses\n", 194 | " if (n_hits == np.count_nonzero(tmp_ship1 == 1)) and not (-1 in tmp_ship1):\n", 195 | " tmp_posterior[i:i+boat_length, j] += 1\n", 196 | " # Check if horizontal ship fits\n", 197 | " if j < field_size-boat_length+1:\n", 198 | " tmp_ship2 = observation_board[i, j:j+boat_length]\n", 199 | " # we only consider possible ship locations that contain all hits and no misses\n", 200 | " if (n_hits == np.count_nonzero(tmp_ship2 == 1)) and not (-1 in tmp_ship2):\n", 201 | " tmp_posterior[i, j:j+boat_length] += 1\n", 202 | " \n", 203 | " # Remove already scored hits\n", 204 | " tmp_posterior[observation_board == 1] = 0\n", 205 | " \n", 206 | " # Compute normalization constant\n", 207 | " normalization_sum = np.sum(tmp_posterior)/boat_length\n", 208 | " \n", 209 | " if normalization_sum != 0:\n", 210 | " tmp_posterior = (1/normalization_sum)*tmp_posterior\n", 211 | " else:\n", 212 | " raise ValueError('All entries in the posterior table have zero probability. Check the observation_board for invalid entries.')\n", 213 | " return tmp_posterior" 214 | ] 215 | }, 216 | { 217 | "cell_type": "markdown", 218 | "metadata": {}, 219 | "source": [ 220 | "**Task 2:** Test your function on the following `observation_board`s (just run the cells)" 221 | ] 222 | }, 223 | { 224 | "cell_type": "code", 225 | "execution_count": 19, 226 | "metadata": {}, 227 | "outputs": [], 228 | "source": [ 229 | "# board 1: Only misses\n", 230 | "obs_board_1 = np.zeros((field_size, field_size)); obs_board_1[[1, 2, 5, 8], [8, 3, 4, 6]] = -1\n", 231 | "\n", 232 | "# board 2: a few misses, one hit\n", 233 | "obs_board_2 = np.copy(obs_board_1); obs_board_2[7, 1] = 1\n", 234 | "\n", 235 | "# board 3: a few misses, two hits\n", 236 | "obs_board_3 = np.copy(obs_board_2); obs_board_3[6, 1] = 1\n", 237 | "\n", 238 | "# board 4: two for one boat impossible hits: This should cause an error\n", 239 | "obs_board_4 = np.copy(obs_board_2); obs_board_4[6, 2] = 1" 240 | ] 241 | }, 242 | { 243 | "cell_type": "code", 244 | "execution_count": 20, 245 | "metadata": {}, 246 | "outputs": [ 247 | { 248 | "output_type": "stream", 249 | "name": "stdout", 250 | "text": "n_hits 0\nSum of posterior values: 4.999999999999999\nn_hits 1\nn_hits 2\nn_hits 2\n" 251 | }, 252 | { 253 | "output_type": "display_data", 254 | "data": { 255 | "text/plain": "
", 256 | "image/svg+xml": "\n\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n", 257 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsQAAAFnCAYAAAChNV9eAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X2wHXWd5/HPh3sTMnkgsJOokAQSZwArOgp4J6CsDsogARwZa7Q2OOpKzVSGWeOIZe0QZ2ulaq2p0vJh1BWlUsi4lgwplweXhQi6qw7OKBluYngIEStiIDEwJOqGBEbywHf/OCd4ONx7Tt8+3X267+/9qjrlPad/v9Pf2/nQfm+fPt2OCAEAAACpOmbYBQAAAADDREMMAACApNEQAwAAIGk0xAAAAEgaDTEAAACSRkMMAACApNEQAwAAIGk0xAAAAEgaDTEAAACSNlrGm47Mnh+j819SxlsjEYf3Pakjz+xzlesktyjCwSe2742IhVWuk+xiUMPY50pkF4MrKrulNMSj81+iE//jZ8t4ayTi8f9xZeXrJLcowqOfeOujVa+T7GJQw9jnSmQXgysqu5wyAQAAgKTREAMAACBpmRpi2yttP2x7u+21ZRcFFIXsoonILZqK7KKp+jbEtkckXSPpIknLJV1me3nZhQGDIrtoInKLpiK7aLIsR4hXSNoeEY9ExEFJ6yVdWm5ZQCHILpqI3KKpyC4aK0tDvEjSzo7nu9qvvYDt1bbHbY8feWZfUfUBg+ibXXKLGmKfi6Yiu2isLA3xRNd2ixe9ELEuIsYiYmxk9vzBKwMG1ze75BY1xD4XTUV20VhZGuJdkpZ0PF8saXc55QCFIrtoInKLpiK7aKwsDfG9kk61vcz2TEmrJN1WbllAIcgumojcoqnILhqr753qIuKw7TWS7pI0Iun6iNhaemXAgMgumojcoqnILpos062bI2KDpA0l1wIUjuyiicgtmorsoqm4Ux0AAACSRkMMAACApGU6ZQLY8fFLcs1buvaOgisBsiO3aCqyi6ZqanY5QgwAAICk0RADAAAgaTTEAAAASBoNMQAAAJJGQwwAAICk0RADAAAgaTTEAAAASBoNMQAAAJJGQwwAAICk0RADAAAgaTTEAAAASBoNMQAAAJJGQwwAAICkjQ67gDrY8fFLcs1buvaOgiupr5R+16Ygt/2l9Ls2CdntL6XftUnIbn9N/V05QgwAAICk0RADAAAgaTTEAAAASFrfhtj2Etvftb3N9lbbH6yiMGBQZBdNRG7RVGQXTZblS3WHJX04Ijbbnidpk+1vR8RDJdcGDIrsoonILZqK7KKx+h4hjojHI2Jz++f9krZJWlR2YcCgyC6aiNyiqcgummxK5xDbXirpTEkbJ1i22va47fEjz+wrpjqgIJNll9yiztjnoqnILpomc0Nse66kmyVdGRFPdS+PiHURMRYRYyOz5xdZIzCQXtklt6gr9rloKrKLJsrUENueoVa4b4iIW8otCSgO2UUTkVs0FdlFU2W5yoQlfVnStoj4TPklAcUgu2gicoumIrtosixHiM+V9B5Jb7a9pf24uOS6gCKQXTQRuUVTkV00Vt/LrkXEP0lyBbUAhSK7aCJyi6Yiu2gy7lQHAACApNEQAwAAIGlZ7lQ37S1de8ewSwCmjNyiqcgumorsTl8cIQYAAEDSaIgBAACQNBpiAAAAJI2GGAAAAEmjIQYAAEDSaIgBAACQNBpiAAAAJI2GGAAAAEmjIQYAAEDSaIgBAACQNBpiAAAAJI2GGAAAAEmjIQYAAEDSaIgBAACQNBpiAAAAJI2GGAAAAEmjIQYAAEDSMjfEtkds/8j27WUWBBSJ3KKpyC6aiuyiiaZyhPiDkraVVQhQEnKLpiK7aCqyi8bJ1BDbXizpEknXlVsOUBxyi6Yiu2gqsoumynqE+LOS/lrSc5MNsL3a9rjt8SPP7CukOGBA5BZNRXbRVGQXjdS3Ibb9VklPRsSmXuMiYl1EjEXE2Mjs+YUVCORBbtFUZBdNRXbRZFmOEJ8r6W22d0haL+nNtr9WalXA4MgtmorsoqnILhqrb0McER+JiMURsVTSKknfiYh3l14ZMAByi6Yiu2gqsosm4zrEAAAASNroVAZHxPckfa+USoCSkFs0FdlFU5FdNA1HiAEAAJA0GmIAAAAkbUqnTOCFdnz8ktxzl669o8BKgOzILZqK7KKpyG79cYQYAAAASaMhBgAAQNJoiAEAAJA0GmIAAAAkjYYYAAAASaMhBgAAQNJoiAEAAJA0GmIAAAAkjYYYAAAASaMhBgAAQNJoiAEAAJA0GmIAAAAkjYYYAAAASaMhBgAAQNJGh11Aky1de8ewSwCmjNyi05pPvDXXvC9cdXvBlfRHdlGEvJmX8uee7NYfR4gBAACQNBpiAAAAJC1TQ2z7eNs32f6x7W22X1d2YUARyC6aiNyiqcgumirrOcSfk3RnRLzD9kxJs0usCSgS2UUTkVs0FdlFI/VtiG0fJ+mNkt4nSRFxUNLBcssCBkd20UTkFk1FdtFkWU6ZeLmkPZL+3vaPbF9ne07JdQFFILtoInKLpiK7aKwsDfGopLMkfSkizpT0tKS13YNsr7Y9bnv8yDP7Ci4TyKVvdsktaoh9LpqK7KKxsjTEuyTtioiN7ec3qRX4F4iIdRExFhFjI7PnF1kjkFff7JJb1BD7XDQV2UVj9W2II+IJSTttn95+6XxJD5VaFVAAsosmIrdoKrKLJst6lYkPSLqh/Y3RRyRdXl5JQKHILpqI3KKpyC4aKVNDHBFbJI2VXAtQOLKLJiK3aCqyi6biTnUAAABIGg0xAAAAkpb1HGIAPaz5xFtzzfvCVbcXXAkApIH9LorEEWIAAAAkjYYYAAAASaMhBgAAQNJoiAEAAJA0GmIAAAAkjYYYAAAASaMhBgAAQNJoiAEAAJA0GmIAAAAkjYYYAAAASaMhBgAAQNJoiAEAAJA0GmIAAAAkjYYYAAAASRsddgGYuh0fvyTXvKVr7yi4Ehz1hatuH3YJtUdu64ns9kd264ns9kd2s+MIMQAAAJJGQwwAAICkZWqIbX/I9lbbD9q+0fassgsDikB20UTkFk1FdtFUfRti24sk/ZWksYh4laQRSavKLgwYFNlFE5FbNBXZRZNlPWViVNJv2R6VNFvS7vJKAgpFdtFE5BZNRXbRSH0b4oj4uaRPSXpM0uOS9kXEt8ouDBgU2UUTkVs0FdlFk2U5ZeIESZdKWibpJElzbL97gnGrbY/bHj/yzL7iKwWmKEt2yS3qhn0umorsosmynDLxh5J+FhF7IuKQpFskvb57UESsi4ixiBgbmT2/6DqBPPpml9yihtjnoqnILhorS0P8mKRzbM+2bUnnS9pWbllAIcgumojcoqnILhoryznEGyXdJGmzpAfac9aVXBcwMLKLJiK3aCqyiybLdOvmiLha0tUl1wIUjuyiicgtmorsoqm4Ux0AAACSRkMMAACApGU6ZQL1snTtHcMuAZgycoumIrtoKrKbHUeIAQAAkDQaYgAAACSNhhgAAABJoyEGAABA0miIAQAAkDQaYgAAACSNhhgAAABJoyEGAABA0miIAQAAkDQaYgAAACSNhhgAAABJoyEGAABA0miIAQAAkDQaYgAAACTNEVH8m9p7JD06yeIFkvYWvtL86laPVL+ahlHPKRGxsMoV9smtxL9LP3WrRyK7Ev8uWVDPEHIrNS67datHql9Njc1uKQ1xzxXa4xExVulKe6hbPVL9aqpbPcNSt+1APf3Vsaaq1XEb1K0m6qmnum2HutUj1a+mutUzFZwyAQAAgKTREAMAACBpw2iI1w1hnb3UrR6pfjXVrZ5hqdt2oJ7+6lhT1eq4DepWE/XUU922Q93qkepXU93qyazyc4gBAACAOuGUCQAAACStlIbY9krbD9vebnvtBMtt+/Pt5ffbPquMOjrWt8T2d21vs73V9gcnGHOe7X22t7QfHy25ph22H2iva3yC5VVvo9M7fvcttp+yfWXXmEq30TCQ3Uw1kd0aqlN265jb9jprk11y+xtkt29Ntclte33TM7sRUehD0oikn0p6uaSZku6TtLxrzMWSvinJks6RtLHoOrrWd6Kks9o/z5P0kwlqOk/S7WXW0bW+HZIW9Fhe6Taa4N/wCbWu7Te0bVT1g+xmrons1uxRt+zWMbftddYyu6nmtuN3J7u9a6plbjv+/aZFdss4QrxC0vaIeCQiDkpaL+nSrjGXSvpqtNwj6XjbJ5ZQiyQpIh6PiM3tn/dL2iZpUVnrK0il26jL+ZJ+GhG9blIxHZHdYpDd6tUquw3NrTS87KaaW4nsFoF9bgHKaIgXSdrZ8XyXXhymLGNKYXuppDMlbZxg8ets32f7m7ZfWXIpIelbtjfZXj3B8qFtI0mrJN04ybIqt1HVyG42ZLd+apvdGuVWqm92U82tRHazqGtupWmU3dES3tMTvNZ9KYssYwpne66kmyVdGRFPdS3erNYh/wO2L5b0DUmnlljOuRGx2/ZLJH3b9o8j4u7OcieYU8U2minpbZI+MsHiqrdR1chuNmS3fmqZ3ZrlVqphdhPPrUR2s6hdbqXpl90yjhDvkrSk4/liSbtzjCmU7RlqhfuGiLile3lEPBURB9o/b5A0w/aCsuqJiN3t/31S0q1qfWzUqfJt1HaRpM0R8a/dC6reRkNAdjMgu7VUu+zWLbft9dQxuynnViK7fdU0t9I0y24ZDfG9kk61vaz918MqSbd1jblN0nvb34w8R9K+iHi8hFoktb6BKenLkrZFxGcmGfOy9jjZXqHWtvlFSfXMsT3v6M+S3iLpwa5hlW6jDpdpko8/qtxGQ0J2+9dDduupVtmtW27b66hrdlPOrUR2+9VT19xK0yy7hZ8yERGHba+RdJda3z68PiK22r6ivfxaSRvU+lbkdknPSLq86Dq6nCvpPZIesL2l/drfSDq5o6Z3SPpL24cl/ZukVRFR1kcOL5V0azsro5L+ISLuHPI2ku3Zki6Q9Bcdr3XWVOU2qhzZzYTs1lANs1u33Eo1zG7quZXIbga1y600PbPLneoAAACQNO5UBwAAgKTREAMAACBpNMQAAABIGg0xAAAAkkZDDAAAgKTREAMAACBpNMQAAABIGg0xAAAAkkZDDAAAgKTREAMAACBpNMQAAABIGg0xAADoyfb1tp+0/eAky23787a3277f9llV1wgMgoYYAAD08xVJK3ssv0jSqe3HaklfqqAmoDA0xAAAoKeIuFvSL3sMuVTSV6PlHknH2z6xmuqAwdEQAwCAQS2StLPj+a72a0AjjA67AAAA0Hie4LWYcKC9Wq3TKjRnzpzXvuIVryizLkxzmzZt2hsRCwd9n1Ia4pk+NmZpTq65nnVsrnlHZlXf2x+aW/kqJUnHHMo3b+TZAdb5/57OPzmHX+tpHYxnJ9rBlmaQ3AJH7devCtk5TwXZxaAK2OfukrSk4/liSbsnGhgR6yStk6SxsbEYHx8fYLVIne1Hi3ifUrrIWZqjs31+rrkjv3Narnn7Tz8h17xB7H5Dpf3a8+buzHemy3E7juRe5+xbN+aem8fG+L+Vrk8aLLfAUf8nbipk5zwVZBeDKmCfe5ukNbbXSzpb0r6IeHzgwoCKcMoEAADoyfaNks6TtMD2LklXS5ohSRFxraQNki6WtF3SM5IuH06lQD40xAAAoKeIuKzP8pD0/orKAQqX6bN32yttP9y+4PbasosCikJ20UTkFgCq1bchtj0i6Rq1Lrq9XNJltpeXXRgwKLKLJiK3AFC9LEeIV0jaHhGPRMRBSevVugA3UHdkF01EbgGgYlka4kwX27a92va47fFDGuD6XkBx+maX3KKG2OcCQMWyNMSZLrYdEesiYiwixmYo37WEgYL1zS65RQ2xzwWAimVpiDNfbBuoGbKLJiK3AFCxLA3xvZJOtb3M9kxJq9S6ADdQd2QXTURuAaBifa9DHBGHba+RdJekEUnXR8TW0isDBkR20UTkFgCql+nGHBGxQa270ACNQnbRROQWAKqV6cYcAAAAwHRVyq2bPetYjfzOabnm7j/9hFzznlo6kmueJB2340iueXN3Dufvibz1znv4V7nXeedz/zPXvAuOeWfudQIAAFSBI8QAAABIGg0xAAAAkkZDDAAAgKTREAMAACBpNMQAAABIGg0xAAAAkkZDDAAAgKTREAMAACBpNMQAAABIGg0xAAAAkkZDDAAAerK90vbDtrfbXjvB8vm2/7ft+2xvtX35MOoE8qIhBgAAk7I9IukaSRdJWi7pMtvLu4a9X9JDEfEaSedJ+rTtmZUWCgyAhhgAAPSyQtL2iHgkIg5KWi/p0q4xIWmebUuaK+mXkg5XWyaQ3+iwC+g27+Ff5ZyXf513Pvi3ueZdcMw7c6/ziQ+9Pvfcp5aO5Jy3IPc6V77qv+Sa98zbz84177nv3JNrHgCgcIsk7ex4vktS9879C5Juk7Rb0jxJ/yEinpvozWyvlrRakk4++eTCiwXy4AgxAADoxRO8Fl3PL5S0RdJJks6Q9AXbx030ZhGxLiLGImJs4cKFxVYK5ERDDAAAetklaUnH88VqHQnudLmkW6Jlu6SfSXpFRfUBA6MhBgAAvdwr6VTby9pflFul1ukRnR6TdL4k2X6ppNMlPVJplcAAancOMQAAqI+IOGx7jaS7JI1Iuj4ittq+or38Wkkfk/QV2w+odYrFVRGxd2hFA1PUtyG2vUTSVyW9TNJzktZFxOfKLgwYFNlFE5Fb1FFEbJC0oeu1azt+3i3pLVXXBRQlyxHiw5I+HBGbbc+TtMn2tyPioZJrAwZFdtFE5BYAKtb3HOKIeDwiNrd/3i9pm1qXYAFqjeyiicgtAFRvSucQ214q6UxJGydY9vx1BWfNmPBKK8DQTJbdF+RWsyuvC+gl8z6X7ALAQDJfZcL2XEk3S7oyIp7qXt55XcGZI+ycUR+9stuZ2xk6djgFAhOYyj6X7ALAYDI1xLZnqLVjviEibim3JKA4ZBdNRG4BoFp9G+L2fcm/LGlbRHym/JKAYpBdNBG5BYDqZTlCfK6k90h6s+0t7cfFJdcFFIHsoonILQBUrO+X6iLinzTxfcyBWiO7aCJyCwDV49bNAAAASFopt24+OH9UP79wQa65B5Y8l2verFP255onSb/34b/LNe/AZ87Jvc5P/9FXc8+9Ze9ZueZt2r0k9zr37zgh17ynlo7kmnekYV+a37v6dbnmLVj3w4IrAaaG7AIAR4gBAACQOBpiAAAAJI2GGAAAAEmjIQYAAEDSaIgBAACQNBpiAAAAJI2GGAAAAEmjIQYAAEDSaIgBAACQNBpiAAAAJI2GGAAAAEmjIQYAAD3ZXmn7Ydvbba+dZMx5trfY3mr7H6uuERjE6LALAAAA9WV7RNI1ki6QtEvSvbZvi4iHOsYcL+mLklZGxGO2XzKcaoF8SmmIn5shHVjyXK65s07Zn2vea0/amWueJP3zknm55+b1x3MO5J57y9588wbZRvctfVWueXlz8NyMXNOGZsYf78k3cV2xdQBTRXaRwQpJ2yPiEUmyvV7SpZIe6hjzLkm3RMRjkhQRT1ZeJTAATpkAAAC9LJLUeURlV/u1TqdJOsH292xvsv3eyd7M9mrb47bH9+zJ+QcZUDAaYgAA0IsneC26no9Keq2kSyRdKOm/2j5tojeLiHURMRYRYwsXLiy2UiAnziEGAAC97JK0pOP5Ykm7JxizNyKelvS07bslvUbST6opERgMR4gBAEAv90o61fYy2zMlrZJ0W9eY/yXpDbZHbc+WdLakbRXXCeSWuSG2PWL7R7ZvL7MgoEjkFk1FdlEXEXFY0hpJd6nV5H49IrbavsL2Fe0x2yTdKel+Sf8i6bqIeHBYNQNTNZVTJj6o1n8Ix5VUC1AGcoumIruojYjYIGlD12vXdj3/pKRPVlkXUJRMR4htL1brRPnryi0HKA65RVORXQCoVtZTJj4r6a8lTXpR2c7LqBx5+ulCigMGNKXcHtKz1VUG9EZ2AaBCfRti22+V9GREbOo1rvMyKiNz5hRWIJBHntzO0LEVVQdMjuwCQPWyHCE+V9LbbO+QtF7Sm21/rdSqgMGRWzQV2QWAivVtiCPiIxGxOCKWqnWple9ExLtLrwwYALlFU5FdAKge1yEGAABA0qZ0p7qI+J6k75VSCVAScoumIrsAUA2OEAMAACBpUzpCnNUxh6S5O3P22jvn55p2n/LNk6TT79qba96dD/5t7nUu+2zuqTrp+5Fr3u43OPc65+aemYZ7zrgp17wLdUbBlQBTQ3YBgCPEAAAASBwNMQAAAJJGQwwAAICk0RADAAAgaTTEAAAASBoNMQAAAJJGQwwAAICk0RADAAAgaTTEAAAASBoNMQAAAJJGQwwAAICk0RADAICebK+0/bDt7bbX9hj3+7aP2H5HlfUBg6IhBgAAk7I9IukaSRdJWi7pMtvLJxn3CUl3VVshMDgaYgAA0MsKSdsj4pGIOChpvaRLJxj3AUk3S3qyyuKAIowOu4BuB5Y8l2verFP2517no/9+Rq55v/fhv8u9zk//p6/lnvthvTvXvEG20QHNyzVv7s58f3MdcyjXNABTdOFJZwy7BNTfIkk7O57vknR25wDbiyS9XdKbJf1+rzezvVrSakk6+eSTCy0UyIsjxAAAoBdP8Fp0Pf+spKsi4ki/N4uIdRExFhFjCxcuLKRAYFC1O0IMAABqZZekJR3PF0va3TVmTNJ625K0QNLFtg9HxDeqKREYDA0xAADo5V5Jp9peJunnklZJelfngIhYdvRn21+RdDvNMJok0ykTto+3fZPtH9veZvt1ZRcGFIHsoonILeokIg5LWqPW1SO2Sfp6RGy1fYXtK4ZbHVCMrEeIPyfpzoh4h+2ZkmaXWBNQJLKLJiK3qJWI2CBpQ9dr104y9n1V1AQUqW9DbPs4SW+U9D5Jal9y5WC5ZQGDI7toInILANXLcsrEyyXtkfT3tn9k+zrbc0quCygC2UUTkVsAqFiWhnhU0lmSvhQRZ0p6WtKLbttoe7XtcdvjR555uuAygVz6Zrczt4f07DBqBLpNeZ9LdgFgMFka4l2SdkXExvbzm9TaWb9A53UFR2ZzMAO10De7nbmdoWMrLxCYwJT3uWQXAAbTtyGOiCck7bR9evul8yU9VGpVQAHILpqI3AJA9bJeZeIDkm5of9v5EUmXl1cSUCiyiyYitwBQoUwNcURsUesuNECjkF00EbkFgGplujEHAAAAMF2VcuvmkWel43YcyTs716wDmpdzfdKsU/bnnpvXLXtf9B2ZzObuzPd3zCDbKO868+ZgZAhfmj+8YI72/km+G4JdeFK+de7b8Lv5Jg7JoW8szD13wbofFlgJAADF4QgxAAAAkkZDDAAAgKTREAMAACBpNMQAAABIGg0xAAAAkkZDDAAAgKTREAMAACBpNMQAAABIGg0xAAAAkkZDDAAAgKTREAMAACBpNMQAAABIGg0xAADoyfZK2w/b3m577QTL/9T2/e3HD2y/Zhh1AnnREAMAgEnZHpF0jaSLJC2XdJnt5V3DfibpDyLi1ZI+JmldtVUCgxkt401Hfn1Y8x7+Vc7ZJ+SaddyOnKuTpO/PzTVt9q0/yL3KvXedlnvuAw9+KNe8C455Z+51Vu2YeLrydY7ufVoL1v0w19x9G34317z5F2/PNW94mlYv+kknuxjACknbI+IRSbK9XtKlkh46OiAiOv8P8R5JiyutEBgQR4gBAEAviyTt7Hi+q/3aZP5M0jcnW2h7te1x2+N79uwpqERgMDTEAACgF0/wWkw40H6TWg3xVZO9WUSsi4ixiBhbuHBhQSUCgynllAkAADBt7JK0pOP5Ykm7uwfZfrWk6yRdFBG/qKg2oBCZjhDb/pDtrbYftH2j7VllFwYUgeyiicgtauZeSafaXmZ7pqRVkm7rHGD7ZEm3SHpPRPxkCDUCA+nbENteJOmvJI1FxKskjaj1HwNQa2QXTURuUTcRcVjSGkl3Sdom6esRsdX2FbavaA/7qKTflvRF21tsjw+pXCCXrKdMjEr6LduHJM3WBB+VADVFdtFE5Ba1EhEbJG3oeu3ajp//XNKfV10XUJS+R4gj4ueSPiXpMUmPS9oXEd8quzBgUGQXTURuAaB6WU6ZOEGt6w0uk3SSpDm23z3BuOcvo3LwyDPFVwpMUZbsdub2kJ4dRpnAC+TZ55JdABhMli/V/aGkn0XEnog4pNZJ86/vHtR5GZWZI7OLrhPIo292O3M7Q8cOpUigy5T3uWQXAAaTpSF+TNI5tmfbtqTz1TqpHqg7sosmIrcAULEs5xBvlHSTpM2SHmjP4R7lqD2yiyYitwBQvUxXmYiIqyVdXXItQOHILpqI3AJAtbh1MwAAAJJWyq2b49fP6shD+W5UM0+n5Zq3//QTcs2TpKeWjuSb96EXfc8lswNLnss999w/+VS+iW8/O/c65z38q1zz8uagaQ59Y2HOmdsLrQOYKrILABwhBgAAQOJoiAEAAJA0GmIAAAAkjYYYAAAASaMhBgAAQNJoiAEAAJA0GmIAAAAkjYYYAAAASaMhBgAAQNJoiAEAAJA0GmIAAAAkjYYYAAAASaMhBgAAQNJoiAEAQE+2V9p+2PZ222snWG7bn28vv9/2WcOoE8jLEVH8m9p7JD06yeIFkvYWvtL86laPVL+ahlHPKRGxsMoV9smtxL9LP3WrRyK7Ev8uWVBPj9zaHpH0E0kXSNol6V5Jl0XEQx1jLpb0AUkXSzpb0uci4ux+Kx0bG4vx8fECykeqbG+KiLFB32e0iGK69fo/A9vjRRRelLrVI9WvprrVU5Z+TUzdtgP19FfHmsrQpH2uVL+aqKevFZK2R8QjkmR7vaRLJT3UMeZSSV+N1lG2e2wfb/vEiHi8+nKBqSulIQYAANPGIkk7O57vUusocL8xiyS9qCG2vVrS6vbTZ20/WFypA+PTgv7qVtPpRbwJDTEAAOjFE7zWfb5lljGtFyPWSVon1e9oOPX0V7eabBdyzs0wvlS3bgjr7KVu9Uj1q6lu9QxL3bYD9fRXx5qqVsdtULeaqKe3XZKWdDxfLGl3jjFAbZXypToAADA92B5V60t150v6uVoZ9FhcAAAFgklEQVRfqntXRGztGHOJpDX6zZfqPh8RKzK8d+2ONlJPb3Wrqah6OGUCAABMKiIO214j6S5JI5Kuj4ittq9oL79W0ga1muHtkp6RdHnGt6/b0XDq6a9uNRVSD0eIAQAAkLRSziGu2wW8bS+x/V3b22xvtf3BCcacZ3uf7S3tx0dLrmmH7Qfa63rRCeFD2Eand/zuW2w/ZfvKrjGVbqNhILuZaiK7NVSn7NYxt+111ia75BaomYgo9KHWxyk/lfRySTMl3SdpedeYiyV9U61vpZ4jaWPRdXSt70RJZ7V/nqfWuVDdNZ0n6fYy6+ha3w5JC3osr3QbTfBv+IRaF2of2jaq+kF2M9dEdmv2qFt265jb9jprmd1Uc8uDR50eZRwhfv4C3hFxUNLRC3h3ev4C3hFxj6TjbZ9YQi2SpIh4PCI2t3/eL2mbWtdHrLNKt1GX8yX9NCJ63bVtOiK7xSC71atVdhuaW2l42Z12uR3kE4t+c0uq50/bddxv+we2X9OxrOcnCyXWNOknBEPaRv+5o5YHbR+x/e/aywrfRravt/2kJ7lOddEZKqMhnuzi3FMdUwrbSyWdKWnjBItfZ/s+29+0/cqSSwlJ37K9ya2LlHcb2jaStErSjZMsq3IbVY3sZkN266e22a1RbqX6Znda5datWz1fI+kiScslXWZ7edewiySd2n6slvSlKcwto56fSfqDiHi1pI/pxV/UelNEnBEFXV1hCr/n99vrPSMi/tsU5xZaT0R88mgtkj4i6R8j4pcdQwrdRpK+Imllj+WFZqiMq0wUegHvItmeK+lmSVdGxFNdizer9XHVAbfuyf4NtTZyWc6NiN22XyLp27Z/HBF3d5Y7wZwqttFMSW9TK+zdqt5GVSO72ZDd+qlldmuWW6mG2Z2muc19q2dJSzPMLbyeiPhBx/h71LqOcpmybKMy5hb1npdp8j/iChERd7f/oJ5MoRkq4whxLS/gbXuGWjvmGyLilu7lEfFURBxo/7xB0gzbC8qqJyJ2t//3SUm3qhXGTsO6yPlFkjZHxL92L6h6Gw0B2c2A7NZS7bJbt9y211PH7E7H3A7yiUUZR+mn+p5/pta55Ef1+2ShzJom+oRgqNvI9my1jtze3PFyGduon0IzVEZDfK+kU20va//lu0rSbV1jbpP03vb5H+dI2hcRL7rfeVFsW9KXJW2LiM9MMuZl7XGyvUKtbfOLkuqZY3ve0Z8lvUVS9zkylW6jDpP+1VflNhoSstu/HrJbT7XKbt1y215HXbM7HXM7yCcWZRylz/yett+kVkN8VcfL50bEWWr98fJ+228csJ6sNR39hOA1kv67Wp8QZJ1bRj1H/ZGkf+46XaKMbdRPoRkq/JSJKPcC3nmdK+k9kh6wvaX92t9IOrmjpndI+kvbhyX9m6RV7cPwZXippFvb+7lRSf8QEXcOeRsd/avvAkl/0fFaZ01VbqPKkd1MyG4N1TC7dcutVMPsTuPcDvKJxcwMc8uoR7ZfLek6SRdFxPN/eHR+smD76CcLd3fPL7qmztOMImKD7S+2PyEo45OMqbzni855L2kb9VNshqIGl7rgwYMHDx48eEyPh1p/cDwiaZl+cxnAV3aNuUQvvMTdv2SdW1I9J6v1h9Dru16fI2lex88/kLSyom30Mv3mBmorJD3W3l5D2UbtcfMl/VLSnLK3Ufv9lkp6cJJlhWaIWzcDAIDCxACfWEw2t4J6PirptyV9sf0pwuFoXS1hwk8WBqlnCjVN9gnBsLaRJL1d0rci4umO6aVsI9s3qnUt7gW2d0m6WtKMjnoKzRC3bgYAAEDSSrl1MwAAANAUNMQAAABIGg0xAAAAkkZDDAAAgKTREAMAACBpNMQAAABIGg0xAAAAkvb/AaDo7ai8qsKvAAAAAElFTkSuQmCC\n" 258 | }, 259 | "metadata": { 260 | "needs_background": "light" 261 | } 262 | } 263 | ], 264 | "source": [ 265 | "### VISUALIZATION\n", 266 | "from matplotlib import colors\n", 267 | "\n", 268 | "# making a custom discrete colormap for hits and misses\n", 269 | "cmap_discrete = colors.ListedColormap(['white', 'C0','darkred'])\n", 270 | "boundaries = [-1.5, -0.5, 0.5, 1.5]\n", 271 | "norm = colors.BoundaryNorm(boundaries, cmap_discrete.N, clip=True)\n", 272 | "\n", 273 | "# Plot the boards and the corresponding posterior\n", 274 | "f, axs = plt.subplots(2, 4, figsize=(12,6))\n", 275 | "\n", 276 | "cmap = matplotlib.cm.viridis # Can be any colormap that you want after the cm\n", 277 | "cmap.set_bad(color='w')\n", 278 | "\n", 279 | "# board 1\n", 280 | "post_1 = posterior_one_ship(obs_board_1, 5)\n", 281 | "print('Sum of posterior values: ', np.sum(post_1))\n", 282 | "axs[0, 0].imshow(obs_board_1, cmap=cmap_discrete, norm=norm)\n", 283 | "axs[1, 0].imshow(np.ma.masked_where(post_1 == 0, post_1), cmap=cmap)\n", 284 | "\n", 285 | "# board 2\n", 286 | "axs[0, 1].imshow(obs_board_2, cmap=cmap_discrete, norm=norm)\n", 287 | "axs[1, 1].imshow(posterior_one_ship(obs_board_2, 5))\n", 288 | "\n", 289 | "# board 3\n", 290 | "axs[0, 2].imshow(obs_board_3, cmap=cmap_discrete, norm=norm)\n", 291 | "axs[1, 2].imshow(posterior_one_ship(obs_board_3, 5))\n", 292 | "\n", 293 | "# board 4\n", 294 | "axs[0, 3].imshow(obs_board_4, cmap=cmap_discrete, norm=norm)\n", 295 | "try:\n", 296 | " axs[1, 3].imshow(posterior_one_ship(obs_board_4, 5))\n", 297 | "except:\n", 298 | " pass" 299 | ] 300 | }, 301 | { 302 | "cell_type": "code", 303 | "execution_count": 21, 304 | "metadata": {}, 305 | "outputs": [ 306 | { 307 | "output_type": "stream", 308 | "name": "stdout", 309 | "text": "n_hits 2\n" 310 | }, 311 | { 312 | "output_type": "error", 313 | "ename": "ValueError", 314 | "evalue": "All entries in the posterior table have zero probability. Check the observation_board for invalid entries.", 315 | "traceback": [ 316 | "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", 317 | "\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)", 318 | "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;31m# board 4 should raise an error\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mposterior_one_ship\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobs_board_4\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m5\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", 319 | "\u001b[0;32m\u001b[0m in \u001b[0;36mposterior_one_ship\u001b[0;34m(observation_board, boat_length)\u001b[0m\n\u001b[1;32m 40\u001b[0m \u001b[0mtmp_posterior\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m/\u001b[0m\u001b[0mnormalization_sum\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mtmp_posterior\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 41\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 42\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'All entries in the posterior table have zero probability. Check the observation_board for invalid entries.'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 43\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mtmp_posterior\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", 320 | "\u001b[0;31mValueError\u001b[0m: All entries in the posterior table have zero probability. Check the observation_board for invalid entries." 321 | ] 322 | } 323 | ], 324 | "source": [ 325 | "# board 4 should raise an error\n", 326 | "posterior_one_ship(obs_board_4, 5)" 327 | ] 328 | }, 329 | { 330 | "cell_type": "markdown", 331 | "metadata": {}, 332 | "source": [ 333 | "### **2.3** Towards battleship with more than one boat\n", 334 | "_Note: No coding required!_\n", 335 | "\n", 336 | "Next week's assignment will deal with the full game of Battleships, which, according to the above rules, contains seven ships.\n", 337 | "\n", 338 | "1. Think about how you would need to modify your above routines to compute \n", 339 | "        (a) the prior over ship locations \n", 340 | "        (b) the posterior over ship locations given hit/miss observations \n", 341 | " by enumerating all states. Describe the changes you would need to make to your code.\n", 342 | "2. Why will it be hard to compute the posterior with multiple boats?" 343 | ] 344 | }, 345 | { 346 | "cell_type": "markdown", 347 | "metadata": {}, 348 | "source": [ 349 | "#### Answers\n", 350 | "1.1 \n", 351 | "\n", 352 | "(a) In the above inplementation I basically use brute force to loop over all possible placements of the ships. With $N$ being the length of the quadratic game board and $n$ the size of the current boat, looping over all possible placement options of **one ship** requires worst case roundabout $(2N(N-n+1))$ iterations. However, after placing the first ship I will need to loop over possible placements of the second ship, where I will need to loop over possible palcements of the third ship, and so on. This procedure gets computationally quite costly rather quickly.\n", 353 | "\n", 354 | "(b) In my implementation for the posterior case, I would have the same problems looping over all possible positions of the boats with the addition of if statements in a lot of nested for loops.\n", 355 | "\n", 356 | "2. The number of possible boat placements is huge.\n" 357 | ] 358 | }, 359 | { 360 | "cell_type": "markdown", 361 | "metadata": {}, 362 | "source": [ 363 | "\n", 364 | "#### 1. Exam-Type Question\n", 365 | "The likelihood for the $N$ binary i.i.d. observations $X:=[x_1,...,x_N]$ is\n", 366 | "\\begin{equation}\n", 367 | "p(X|f) = \\prod_{i=1}^{N} p(x_i|f) = f^{C} (1-f)^{N-C},\n", 368 | "\\end{equation}\n", 369 | "with $C = \\sum_{i=1}^N x_i$.\n", 370 | "\n", 371 | "The posterior distribtuion of $p(f|x)$ is obtained via Bayes' Law as\n", 372 | "\\begin{align}\n", 373 | "p(f|X) &= \\frac{p(X|f)p(f|a,b)}{p(X)} \\\\\n", 374 | "&= C_0 f^{C} (1-f)^{N-C} f^{a-1} (1-f)^{b-1}, \\\\\n", 375 | "&= C_0 f^{C+a-1}(1-f)^{N-C+b-1},\n", 376 | "\\end{align}\n", 377 | "with $C_0 = B(a,b) p(X)= B(C+a, N-C+b)$ being the normalization constant. Note that here the posterior takes the same form as the prior being a Beta distirbution. In other words, the Beta distribution is a conjugate prior to the Bernoulli distribution.\n", 378 | "\n", 379 | "#### 2. Theory Question\n", 380 | "\n", 381 | "(a)\n", 382 | "\n", 383 | "With $Z = \\frac{X}{X+Y}$ and $\\bar{Z}=X+Y$. Solving for X and Y yields $Z = \\frac{X}{\\bar{Z}}$ and hence $X = Z\\bar{Z}$ as well as $Y=\\bar{Z}(1-Z)$. With $X \\sim \\mathcal{G}(X|a,1)$ and $Y \\sim \\mathcal{G}(Y|b,1)$ the joint distribution $f(X,Y) = f(X)f(Y)$ (these distributions are independent) reads in terms of $Z$ and $\\bar{Z}$\n", 384 | "\\begin{equation}\n", 385 | "f(\\bar{Z},Z) = \\frac{\\lambda^a}{\\Gamma (a)} \\cdot (\\bar{Z}Z)^{a-1} \\cdot e^{-\\lambda \\bar{Z}Z} \\cdot \\frac{\\lambda^b}{\\Gamma (b)} \\cdot \\big(\\bar{Z}(1-Z)\\big)^{b-1} \\cdot e^{-\\lambda \\bar{Z}(1-Z)} \\text{det}\\Bigg(\\begin{bmatrix} \\frac{\\partial X}{\\partial \\bar{Z}} & \\frac{\\partial X}{\\partial Z} \\\\ \\frac{\\partial Y}{\\partial \\bar{Z}} & \\frac{\\partial Y}{\\partial Z} \\end{bmatrix} \\Bigg)\n", 386 | "\\end{equation}\n", 387 | "with \n", 388 | "\\begin{equation}\n", 389 | "\\text{det}\\Bigg(\\begin{bmatrix} \\frac{\\partial X}{\\partial \\bar{Z}} & \\frac{\\partial X}{\\partial Z} \\\\ \\frac{\\partial Y}{\\partial \\bar{Z}} & \\frac{\\partial Y}{\\partial Z} \\end{bmatrix} \\Bigg) = \\bar{Z}.\n", 390 | "\\end{equation}\n", 391 | "\n", 392 | "Note that if we rearrange the above equation, one term only is a function of $Z$ and denotes the beta distribution $Z \\sim \\text{Beta}(a,b)$.\n", 393 | "\n", 394 | "(b)\n", 395 | "\\begin{align}\n", 396 | " E(X) &= \\int^{1}_{0} x f_B(x) dx \\\\\n", 397 | " &= \\frac{1}{B(a,b)}\\int^{1}_{0} x^{a} (1-x)^{b-1} f_B(x) dx \\\\\n", 398 | " &= \\frac{B(a+1,b)}{B(a,b)} \\\\\n", 399 | " &= \\frac{\\Gamma(a+1)\\Gamma(b)}{\\Gamma(a+b+1)} \\frac{\\Gamma(a+b)}{\\Gamma(a)\\Gamma(b)} \\\\\n", 400 | " &= \\frac{a}{a+b}\\frac{\\Gamma(a)\\Gamma(b)}{\\Gamma(a+b)} \\frac{\\Gamma(a+b)}{\\Gamma(a)\\Gamma(b)} \\\\\n", 401 | " &= \\frac{a}{a+b}\n", 402 | "\\end{align}\n" 403 | ] 404 | } 405 | ], 406 | "metadata": { 407 | "kernelspec": { 408 | "display_name": "Python 3", 409 | "language": "python", 410 | "name": "python3" 411 | }, 412 | "language_info": { 413 | "codemirror_mode": { 414 | "name": "ipython", 415 | "version": 3 416 | }, 417 | "file_extension": ".py", 418 | "mimetype": "text/x-python", 419 | "name": "python", 420 | "nbconvert_exporter": "python", 421 | "pygments_lexer": "ipython3", 422 | "version": "3.7.0-final" 423 | } 424 | }, 425 | "nbformat": 4, 426 | "nbformat_minor": 4 427 | } -------------------------------------------------------------------------------- /ex02/Tutorial_02.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucasrm25/Probabilistic-Machine-Learning/ad017c952c027ef37e0d6ba0bb5e4d0226bdf75c/ex02/Tutorial_02.pdf -------------------------------------------------------------------------------- /ex03/Exercise_03.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Exercise #3: Battleships (2/2)\n", 8 | "\n", 9 | "### Probabilistic Machine Learning\n", 10 | "\n", 11 | "- **Lecturer**: Prof. Philipp Hennig\n", 12 | "- **Term**: SoSe 2020\n", 13 | "- **Due Date**: Monday, 11 May 2020\n", 14 | "\n", 15 | "\n", 16 | "![battleship rules](https://upload.wikimedia.org/wikipedia/commons/e/e4/Battleships_Paper_Game.svg)\n", 17 | "\n", 18 | "In the previous exercise sheet on the game _battleships_, we studied prior and posterior probabilities of getting a hit by enumerating all possible locations of one boat. We established that it is computationally unfeasible to obtain the full posterior by enumeration for the number of ships in the rules.\n", 19 | "\n", 20 | "Hence, to render the problem tractable, we need to approximate the posterior over hits given hit/miss observations. We achieve this through _Monte Carlo_ sampling. Instead of enumerating _all_ board states, we randomly sample `nb_samples` valid board states (i.e. configurations of ship arrangements on the board) and use their sum to estimate the posterior.\n", 21 | "\n", 22 | "__The goal of this assignment is to write such a Monte Carlo sampler.__" 23 | ] 24 | }, 25 | { 26 | "cell_type": "markdown", 27 | "metadata": {}, 28 | "source": [ 29 | "### __3.1__ Getting started\n", 30 | "\n", 31 | "There are a few accompanying files that are required for the game. You will only need to alter `MC_agent.py`, all other vital routines are provided.\n", 32 | "- `main.py`: Run this script to play the game. You can select which players should play against each other: `\"human\"`, `\"random\"`, or `\"MC\"`. The first two are already implemented, the latter is our task here.\n", 33 | "- `board.py`: contains a class to track the current state of the board (observations), and to save the placement of the ships. \n", 34 | "- `game.py`: the game itself with relevant methods (initialize the game, calling the respective agents for their moves, checking if the game is over, etc.)\n", 35 | "- `human_agent.py`: The human agent (interactive input)\n", 36 | "- `random_agent.py`: The random agent (just selects a previously unobserved location at random)\n", 37 | "- `MC_Agent`: this is where you should implement the Monte Carlo Agent for the virtual opponent. Initially, it randomly selects a query location (i.e. you can play the game by running `main.py`, but the computer makes uninformed moves." 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "In the current state, you can already play the game against a random agent, or have two random agents play against each other." 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "metadata": {}, 51 | "outputs": [], 52 | "source": [ 53 | "from main import play_battleships\n", 54 | "%matplotlib inline\n", 55 | "play_battleships('random', 'random')" 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": {}, 61 | "source": [ 62 | "### __3.2__ Implement the Monte Carlo Agent\n", 63 | "\n", 64 | "Your task is to replace the method `select_observation` in the class `MCAgent` by a routine that samples board states to estimate the posterior probability of getting a hit, and then choosing the location that maximizes this probability.\n", 65 | "\n", 66 | "The state of observations is saved in the variable `self.observed_board`. A `0` is an unobserved location, `-1` a miss or a sunk ship, and a `1` a hit of a ship that is not yet sunk.\n", 67 | "\n", 68 | "In `select_observation`, we first define a variable `score_board` through which we will assign scores to each field. Therefore it has the shape of the `self.observed_board`.\n", 69 | "\n", 70 | "_The key idea of the Monte Carlo agent is to sum up many possible boat placements and choose the location with the largest score (most boats placed) as new query location._\n", 71 | "\n", 72 | "Here is how to proceed (see comments in the function):\n", 73 | "1. Check if there is an \"open\" hit, i.e. a hit that does not belong to a sunk ship. If so, reduce the number of samples (e.g. to a tenth) and force possible boats to pass through the hit(s)\n", 74 | "2. In case there is no open hit, simulate `nb_samples` boats, the location and orientation of which are chosen at random. Track them using the `score_board`.\n", 75 | "3. Find the location with the largest score and return its indices.\n", 76 | "\n", 77 | "Once you're done, you can play against your agent:" 78 | ] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "execution_count": null, 83 | "metadata": {}, 84 | "outputs": [], 85 | "source": [ 86 | "play_battleships('MC', 'human')" 87 | ] 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "metadata": {}, 92 | "source": [ 93 | "### __3.2__ Further analysis\n", 94 | "__Task 1:__ \n", 95 | "Surface the score matrix of your Monte Carlo agent and visualize it." 96 | ] 97 | }, 98 | { 99 | "cell_type": "markdown", 100 | "metadata": {}, 101 | "source": [] 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "metadata": {}, 106 | "source": [ 107 | "__Task 2:__ \n", 108 | "How could you evaluate your agent?" 109 | ] 110 | }, 111 | { 112 | "cell_type": "markdown", 113 | "metadata": {}, 114 | "source": [] 115 | }, 116 | { 117 | "cell_type": "markdown", 118 | "metadata": {}, 119 | "source": [ 120 | "__Task 3:__ \n", 121 | "How could you improve your agent?" 122 | ] 123 | }, 124 | { 125 | "cell_type": "markdown", 126 | "metadata": {}, 127 | "source": [] 128 | } 129 | ], 130 | "metadata": { 131 | "kernelspec": { 132 | "display_name": "Python 3", 133 | "language": "python", 134 | "name": "python3" 135 | }, 136 | "language_info": { 137 | "codemirror_mode": { 138 | "name": "ipython", 139 | "version": 3 140 | }, 141 | "file_extension": ".py", 142 | "mimetype": "text/x-python", 143 | "name": "python", 144 | "nbconvert_exporter": "python", 145 | "pygments_lexer": "ipython3", 146 | "version": "3.7.4" 147 | } 148 | }, 149 | "nbformat": 4, 150 | "nbformat_minor": 4 151 | } 152 | -------------------------------------------------------------------------------- /ex03/Exercise_03.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucasrm25/Probabilistic-Machine-Learning/ad017c952c027ef37e0d6ba0bb5e4d0226bdf75c/ex03/Exercise_03.pdf -------------------------------------------------------------------------------- /ex03/MC_agent.py: -------------------------------------------------------------------------------- 1 | import random 2 | import copy 3 | import numpy as np 4 | 5 | def fitShipRandom(board_sample, ship, hit, size): 6 | possiblePositions = [] 7 | # 1.1 horizontal 8 | for j in range( hit[1]-ship+1, hit[1]+1 ): 9 | i = hit[0] 10 | if j>=0 and j <= size-ship: 11 | tmp_ship = board_sample[i,j:j+ship] 12 | if np.all( [valid in [0,1] for valid in tmp_ship] ): 13 | possiblePositions.append( (i,j,'h') ) 14 | # 1.2 vertical 15 | for i in range( hit[0]-ship+1, hit[0]+1 ): 16 | j = hit[1] 17 | if i>=0 and i <= size-ship: 18 | tmp_ship = board_sample[i:i+ship,j] 19 | if np.all( [valid in [0,1] for valid in tmp_ship] ): 20 | possiblePositions.append( (i,j,'v') ) 21 | # 1.3 check if ship fits 22 | if np.size(possiblePositions)==0: 23 | return 0 24 | 25 | # 1.3 randomize valid ship position 26 | ship_placed = np.random.permutation(possiblePositions)[0] 27 | if ship_placed[2] == 'v': 28 | board_sample[ int(ship_placed[0]):int(ship_placed[0])+ship, int(ship_placed[1]) ] = 2 29 | else: 30 | board_sample[ int(ship_placed[0]), int(ship_placed[1]):int(ship_placed[1])+ship ] = 2 31 | 32 | return 1 33 | 34 | class MCAgent: 35 | 36 | def __init__(self, ships, size, nb_samples = 1000): 37 | self.nb_samples = nb_samples 38 | self.observed_board = np.zeros((size,size)) 39 | self.remaining_ships = ships 40 | self.size = size 41 | 42 | def select_observation(self): 43 | """ 44 | -------------------------------------------------- 45 | THIS IS THE MONTE CARLO SAMPLER YOU NEED TO ADAPT. 46 | -------------------------------------------------- 47 | 48 | Select the next location to be observed 49 | :returns: i_new: int, j_new: int 50 | """ 51 | 52 | # +----------+ 53 | # | Task 1 | 54 | # +----------+ 55 | # Check if there is already an "open" hit, i.e. a ship that has been hit but not sunk 56 | # These locations are handled by the observation_board as 1 57 | 58 | # +------------+ 59 | # | Task 1a) | 60 | # +------------+ 61 | # If there is already a hit, choose a random one to deal with next. 62 | # Create a score board including that hit, and reduce the number of samples to 1/10 63 | 64 | # +----------+ 65 | # | Task 2 | 66 | # +----------+ 67 | # Populate the score_board with possible boat placements 68 | 69 | # +----------+ 70 | # | Task 3 | 71 | # +----------+ 72 | # Having populated the score board, select a new position by choosing the location with the highest score. 73 | 74 | ''' 75 | Here is how to proceed: 76 | 77 | Hence, to render the problem tractable, we need to approximate the posterior over hits 78 | given hit/miss observations. We achieve this through Monte Carlo sampling. Instead of 79 | enumerating all board states, we randomly sample nb_samples valid board states (i.e. 80 | configurations of ship arrangements on the board) and use their sum to estimate the posterior. 81 | 82 | 1. Check if there is an "open" hit, i.e. a hit that does not belong to a sunk ship. 83 | If so, reduce the number of samples (e.g. to a tenth) and force possible boats to pass 84 | through the hit(s) 85 | 86 | 2. In case there is no open hit, simulate nb_samples boats, the location and orientation 87 | of which are chosen at random. Track them using the score_board. 88 | 89 | 3. Find the location with the largest score and return its indices. 90 | 91 | The goal of this assignment is to write such a Monte Carlo sampler. 92 | ''' 93 | 94 | # New board to collect the states sampled by the MC agent 95 | score_board = np.zeros_like(self.observed_board) 96 | 97 | nb = 0 98 | while True: 99 | 100 | board_sample = self.observed_board.copy() 101 | remaining_ships = np.random.permutation( self.remaining_ships ) 102 | 103 | 104 | isvalid = True 105 | for ship in remaining_ships: 106 | 107 | openhits = np.random.permutation(np.stack(np.where(board_sample==1)).T) 108 | 109 | # if there is an open hit, place a random ship on that hit 110 | if np.size(openhits) > 0: 111 | hit = openhits[0] 112 | if not fitShipRandom(board_sample, ship, hit, self.size): 113 | isvalid = False 114 | break 115 | 116 | # randomize ship position, since there are no open hits left 117 | else: 118 | unobserveds = np.random.permutation(np.stack(np.where(board_sample==0)).T) 119 | 120 | foundPosition = False 121 | for newhitcenter in unobserveds: 122 | foundPosition = fitShipRandom(board_sample, ship, newhitcenter, self.size) 123 | if foundPosition: 124 | break 125 | if not foundPosition: 126 | isvalid = False 127 | break 128 | 129 | if not isvalid: 130 | continue 131 | 132 | score_board[ np.where(board_sample==2) ] += 1 133 | nb += 1 134 | 135 | if nb==self.nb_samples: 136 | break 137 | 138 | score_board[ np.where( self.observed_board == 1 ) ] = 0 139 | 140 | # return the next location to query, i_new: int, j_new: int 141 | i_new, j_new = np.unravel_index(score_board.argmax(), score_board.shape) 142 | print(f"\nShooting position {i_new},{j_new}\n") 143 | return i_new, j_new, score_board 144 | 145 | 146 | def update_observations(self, i, j, observation, sunken_ship): 147 | """ 148 | i: 149 | j: 150 | observation: 151 | """ 152 | if observation: 153 | self.observed_board[i,j] = 1 154 | else: 155 | self.observed_board[i,j] = -1 156 | if not sunken_ship is None: 157 | i,j,l,h = sunken_ship 158 | self.remaining_ships.remove(l) 159 | if h: 160 | self.observed_board[i,j:j+l] = -1 161 | else: 162 | self.observed_board[i:i+l,j] = -1 163 | -------------------------------------------------------------------------------- /ex03/board.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import string 3 | import random 4 | import os 5 | 6 | # for usage with jupyter notebook 7 | from IPython.display import clear_output 8 | 9 | class Board: 10 | 11 | def __init__(self, size = 10): 12 | self.size = size 13 | self.state = np.zeros((size,size), dtype = bool) 14 | self.observed = np.zeros((size,size), dtype = bool) 15 | self.ships = [] 16 | 17 | def show_board(self, show_all = False): 18 | show = np.empty((self.size+1, self.size+1), dtype = str) 19 | #show[1:11,0]= list(string.ascii_uppercase)[:10] 20 | show[1:11,0]= list(range(0,10)) 21 | show[0,0] = " " 22 | show[0,1:11]= list(range(0,10)) 23 | for i in range(self.size): 24 | for j in range(self.size): 25 | if self.observed[i,j] or show_all: 26 | if self.state[i,j] == 1: 27 | show[i+1,j+1]= "X" 28 | else: 29 | show[i+1,j+1] = "." 30 | else: 31 | show[i+1,j+1] = "-" 32 | return show 33 | 34 | def place_ship(self,i,j,l,h): 35 | """ 36 | i: i-coordinate of ship 37 | j: j-coordinate of ship 38 | l: length of ship 39 | h: horizontal = True, vertical = False 40 | """ 41 | if not h: 42 | if i+l <= self.size and (self.state[i:i+l,j]==0).all() and j < self.size: 43 | self.state[i:i+l,j] = 1 44 | self.ships.append((i,j,l,h)) 45 | else: 46 | raise ValueError("invalid position!") 47 | else: 48 | if j+l <= self.size and (self.state[i,j:j+l]==0).all() and i < self.size: 49 | self.state[i,j:j+l] = 1 50 | self.ships.append((i,j,l,h)) 51 | else: 52 | raise ValueError("invalid position!") 53 | 54 | def observe(self,i,j): 55 | if i < self.size and j < self.size and not self.observed[i,j]: 56 | self.observed[i,j]= True 57 | sunken_ship = None 58 | for ship in self.ships: 59 | i1,j1,l,h = ship 60 | if h and (self.observed[i1,j1:j1+l]==1).all(): 61 | self.ships.remove(ship) 62 | sunken_ship = ship 63 | break 64 | elif (not h) and (self.observed[i1:i1+l,j1]==1).all(): 65 | self.ships.remove(ship) 66 | sunken_ship = ship 67 | break 68 | return self.state[i,j], sunken_ship 69 | else: 70 | raise ValueError("invalid observation!") 71 | 72 | def random_initialization(self, ships): 73 | """ 74 | randomly place ships with lengths as specified in list ships 75 | """ 76 | for l in ships: 77 | placed = False 78 | while not placed: 79 | # horizontal or vertical 80 | h = random.choice([True, False]) 81 | if h: 82 | # horizontal 83 | i = random.randint(0,self.size-1) 84 | j = random.randint(0,self.size-l-1) 85 | if (self.state[i,j:j+l]==0).all(): 86 | self.state[i,j:j+l] = 1 87 | self.ships.append((i,j,l,h)) 88 | placed = True 89 | else: 90 | # vertical 91 | i = random.randint(0,self.size-l-1) 92 | j = random.randint(0,self.size-1) 93 | if (self.state[i:i+l,j]==0).all(): 94 | self.state[i:i+l,j] = 1 95 | self.ships.append((i,j,l,h)) 96 | placed = True 97 | 98 | def manual_initialization(self,ships): 99 | """ 100 | ask user where to place the ships. 101 | """ 102 | for l in ships: 103 | 104 | error = False 105 | 106 | while True: 107 | try: 108 | # clear output 109 | cls() 110 | clear_output(wait=True) 111 | 112 | if error: 113 | print("Oops! Invalid Position. Try again...") 114 | error = False 115 | 116 | print("Please place your ship of length "+str(l)+". Your board state is:") 117 | show = self.show_board(show_all=True) 118 | for line in show: 119 | print(*line) 120 | i = input('Enter i-coordinate: ') 121 | j = input('Enter j-coordinate: ') 122 | orient = input('Enter orientation (h=horizontal, v=vertical): ') 123 | if orient == "h": 124 | h = True 125 | elif orient == "v": 126 | h = False 127 | else: 128 | raise ValueError("Orientation parameter has to be h or v.") 129 | 130 | self.place_ship(int(i),int(j),l,h) 131 | 132 | break 133 | 134 | except ValueError: 135 | error = True 136 | 137 | 138 | print("Placement of ships finished.") 139 | 140 | 141 | 142 | def cls(): 143 | os.system('cls' if os.name=='nt' else 'clear') 144 | -------------------------------------------------------------------------------- /ex03/game.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import random 3 | 4 | from board import Board, cls 5 | from MC_agent import MCAgent 6 | from human_agent import HumanAgent 7 | from random_agent import RandomAgent 8 | 9 | import matplotlib.pyplot as plt 10 | import time 11 | 12 | # for usage with jupyter notebook 13 | from IPython.display import clear_output 14 | 15 | class Game: 16 | 17 | def __init__(self, size, ships, nb_samples = 1000, 18 | player1 = "human", player2 = "random"): 19 | self.board_player1 = Board(size) 20 | self.board_player2 = Board(size) 21 | self.size = size 22 | self.ships = ships 23 | 24 | 25 | if player1 == "human": 26 | self.player1 = HumanAgent() 27 | elif player1 == "MC": 28 | self.player1 = MCAgent(ships = ships, 29 | size = size, 30 | nb_samples = nb_samples) 31 | elif player1 == "MC2": 32 | self.player1 = MCAgent(ships = ships, 33 | size = size, 34 | nb_samples = nb_samples) 35 | else: 36 | self.player1 = RandomAgent(size = size) 37 | 38 | if player2 == "human": 39 | self.player2 = HumanAgent() 40 | elif player2 == "MC": 41 | self.player2 = MCAgent(ships = ships.copy(), 42 | size = size, 43 | nb_samples = nb_samples) 44 | elif player2 == "MC2": 45 | self.player2 = MCAgent(ships = ships.copy(), 46 | size = size, 47 | nb_samples = nb_samples) 48 | else: 49 | self.player2 = RandomAgent(size = size) 50 | 51 | 52 | def print_gamestate(self): 53 | # clear output before the next move is printed. 54 | #cls() 55 | #clear_output(wait=True) 56 | 57 | show = np.empty((self.size+1, 2*(self.size+1)+3), dtype = str) 58 | show_player2 = self.board_player2.show_board() 59 | show_player1 = self.board_player1.show_board() 60 | show[0:self.size+1,0:self.size+1] = show_player2 61 | show[0:self.size+1,self.size+4:2*(self.size+1)+3] = show_player1 62 | print("") 63 | print("Player1's observations"+" "*3+"Player2's observations") 64 | print("") 65 | for line in show: 66 | print(*line) 67 | 68 | def initialize_game(self): 69 | # if isinstance(self.player1, HumanAgent): 70 | # self.board_player1.manual_initialization(self.ships) 71 | # else: 72 | self.board_player1.random_initialization(self.ships) 73 | # if isinstance(self.player2, HumanAgent): 74 | # self.board_player2.manual_initialization(self.ships) 75 | # else: 76 | self.board_player2.random_initialization(self.ships) 77 | 78 | def one_turn(self): 79 | print("Enter the coordinates of your next observation:") 80 | while True: 81 | try: 82 | i,j,scores1 = self.player1.select_observation() 83 | observation, sunken_ship = self.board_player2.observe(i,j) 84 | if not sunken_ship is None: 85 | i1,j1,l,h = sunken_ship 86 | print("Player 1 player has sunk ship at ("+str(i1)+","+str(j1)+") with length "+str(l)+"!") 87 | break 88 | except: 89 | print("Player 1 - Invalid observation. Try again.") 90 | self.player1.update_observations(i,j,observation,sunken_ship) 91 | while True: 92 | try: 93 | # handles the case i or j are empty 94 | i,j, scores2 = self.player2.select_observation() 95 | observation, sunken_ship = self.board_player1.observe(i,j) 96 | if not sunken_ship is None: 97 | i2,j2,l,h = sunken_ship 98 | print("Player 2 has sunk ship at ("+str(i2)+","+str(j2)+") with length "+str(l)+"!") 99 | break 100 | except: 101 | print("Player 2- Invalid observation. Try again.") 102 | self.player2.update_observations(i,j,observation,sunken_ship) 103 | clear_output(wait=True) 104 | self.print_gamestate() 105 | # cls() 106 | # input() 107 | 108 | 109 | def game_over(self): 110 | if self.board_player2.ships == []: 111 | if self.board_player1.ships == []: 112 | print("Game over! It's a draw!") 113 | self.winner = None 114 | return True 115 | else: 116 | print("Game over! Player 1 won!") 117 | self.winner = True 118 | return True 119 | elif self.board_player1.ships == []: 120 | print("Game over! Player 2 won!") 121 | self.winner = False 122 | return True 123 | return False 124 | -------------------------------------------------------------------------------- /ex03/human_agent.py: -------------------------------------------------------------------------------- 1 | import random 2 | import copy 3 | import numpy as np 4 | 5 | class HumanAgent: 6 | 7 | def __init__(self): 8 | pass 9 | 10 | def select_observation(self): 11 | i = input('Enter i-coordinate: ') 12 | j = input('Enter j-coordinate: ') 13 | return int(i),int(j), None 14 | 15 | def update_observations(self,i,j,observation, sunken_ship): 16 | pass 17 | -------------------------------------------------------------------------------- /ex03/main.py: -------------------------------------------------------------------------------- 1 | 2 | def play_battleships(player1, player2): 3 | """ 4 | Play battleships 5 | :param player1, player2: Select players from agents 'human', 'random', 'MC', 'MC2' 6 | """ 7 | import game 8 | 9 | game = game.Game(size = 10, ships = [5,4,3,2,2,1,1], player1=player1, player2=player2) 10 | # game = game.Game(size = 10, ships = [5]) 11 | 12 | game.initialize_game() 13 | game.print_gamestate() 14 | 15 | while not game.game_over(): 16 | game.one_turn() 17 | 18 | return game.winner 19 | 20 | 21 | if __name__ == '__main__': 22 | play_battleships('MC', 'MC2') -------------------------------------------------------------------------------- /ex03/random_agent.py: -------------------------------------------------------------------------------- 1 | import random 2 | import copy 3 | import numpy as np 4 | 5 | class RandomAgent: 6 | 7 | def __init__(self, size): 8 | self.observed_board = np.zeros((size,size), dtype = bool) 9 | self.size = size 10 | 11 | def select_observation(self): 12 | while True: 13 | i = random.randint(0,self.size-1) 14 | j = random.randint(0,self.size-1) 15 | if not self.observed_board[i,j]: 16 | return i,j, None 17 | 18 | def update_observations(self,i,j,observation, sunken_ship): 19 | self.observed_board[i,j] = True 20 | -------------------------------------------------------------------------------- /ex04/Exercise_04.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Exercise #4: Probabilistic Modelling of COVID-19 Mortality\n", 8 | "\n", 9 | "### Probabilistic Machine Learning\n", 10 | "\n", 11 | "- **Lecturer**: Prof. Philipp Hennig\n", 12 | "- **Term**: SoSe 2020\n", 13 | "- **Due Date**: Tuesday, 18 May 2020 at 10am\n", 14 | "\n", 15 | "![https://marlin-prod.literatumonline.com/pb-assets/journals/trends/molecular-medicine/TRMOME-D-20-00044.pdf](figures/covid19_spread.png)" 16 | ] 17 | }, 18 | { 19 | "cell_type": "markdown", 20 | "metadata": {}, 21 | "source": [ 22 | "## COVID-19 Time-Series Data\n", 23 | "\n", 24 | "**Task:** Download the number of confirmed infections and deaths by COVID-19 per million people from the following links:\n", 25 | "- https://ourworldindata.org/grapher/total-confirmed-cases-of-covid-19-per-million-people\n", 26 | "- https://ourworldindata.org/grapher/total-covid-deaths-per-million\n", 27 | "\n", 28 | "Load the data from file and retrieve the infection and mortality statistics for the following countries\n", 29 | "- Germany\n", 30 | "- Italy\n", 31 | "- France\n", 32 | "- Great Britain\n", 33 | "- Spain\n", 34 | "- China\n", 35 | "- USA\n", 36 | "\n", 37 | "for further analysis." 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": 7, 43 | "metadata": {}, 44 | "outputs": [ 45 | { 46 | "data": { 47 | "text/html": [ 48 | "
\n", 49 | "\n", 62 | "\n", 63 | " \n", 64 | " \n", 65 | " \n", 66 | " \n", 67 | " \n", 68 | " \n", 69 | " \n", 70 | " \n", 71 | " \n", 72 | " \n", 73 | " \n", 74 | " \n", 75 | " \n", 76 | " \n", 77 | " \n", 78 | " \n", 79 | " \n", 80 | " \n", 81 | " \n", 82 | " \n", 83 | " \n", 84 | " \n", 85 | " \n", 86 | " \n", 87 | " \n", 88 | " \n", 89 | " \n", 90 | " \n", 91 | " \n", 92 | " \n", 93 | " \n", 94 | " \n", 95 | " \n", 96 | " \n", 97 | " \n", 98 | " \n", 99 | " \n", 100 | " \n", 101 | " \n", 102 | " \n", 103 | " \n", 104 | " \n", 105 | " \n", 106 | " \n", 107 | " \n", 108 | " \n", 109 | " \n", 110 | " \n", 111 | " \n", 112 | " \n", 113 | " \n", 114 | " \n", 115 | " \n", 116 | " \n", 117 | " \n", 118 | " \n", 119 | " \n", 120 | " \n", 121 | " \n", 122 | " \n", 123 | " \n", 124 | " \n", 125 | " \n", 126 | " \n", 127 | " \n", 128 | " \n", 129 | " \n", 130 | " \n", 131 | " \n", 132 | " \n", 133 | " \n", 134 | " \n", 135 | " \n", 136 | " \n", 137 | " \n", 138 | " \n", 139 | " \n", 140 | " \n", 141 | " \n", 142 | " \n", 143 | " \n", 144 | " \n", 145 | " \n", 146 | " \n", 147 | " \n", 148 | " \n", 149 | " \n", 150 | " \n", 151 | " \n", 152 | " \n", 153 | " \n", 154 | " \n", 155 | " \n", 156 | " \n", 157 | " \n", 158 | " \n", 159 | " \n", 160 | " \n", 161 | " \n", 162 | " \n", 163 | " \n", 164 | " \n", 165 | " \n", 166 | " \n", 167 | " \n", 168 | " \n", 169 | " \n", 170 | " \n", 171 | " \n", 172 | " \n", 173 | " \n", 174 | " \n", 175 | " \n", 176 | " \n", 177 | " \n", 178 | " \n", 179 | " \n", 180 | " \n", 181 | " \n", 182 | " \n", 183 | " \n", 184 | " \n", 185 | " \n", 186 | " \n", 187 | " \n", 188 | " \n", 189 | " \n", 190 | " \n", 191 | " \n", 192 | " \n", 193 | " \n", 194 | " \n", 195 | " \n", 196 | " \n", 197 | " \n", 198 | " \n", 199 | " \n", 200 | " \n", 201 | " \n", 202 | " \n", 203 | " \n", 204 | " \n", 205 | " \n", 206 | " \n", 207 | " \n", 208 | " \n", 209 | " \n", 210 | " \n", 211 | " \n", 212 | " \n", 213 | " \n", 214 | " \n", 215 | " \n", 216 | " \n", 217 | " \n", 218 | " \n", 219 | " \n", 220 | " \n", 221 | " \n", 222 | " \n", 223 | " \n", 224 | " \n", 225 | " \n", 226 | " \n", 227 | " \n", 228 | " \n", 229 | " \n", 230 | " \n", 231 | " \n", 232 | " \n", 233 | " \n", 234 | " \n", 235 | " \n", 236 | " \n", 237 | " \n", 238 | " \n", 239 | " \n", 240 | " \n", 241 | " \n", 242 | " \n", 243 | " \n", 244 | " \n", 245 | " \n", 246 | " \n", 247 | " \n", 248 | " \n", 249 | " \n", 250 | " \n", 251 | " \n", 252 | " \n", 253 | " \n", 254 | " \n", 255 | " \n", 256 | " \n", 257 | " \n", 258 | " \n", 259 | " \n", 260 | " \n", 261 | " \n", 262 | " \n", 263 | " \n", 264 | " \n", 265 | " \n", 266 | " \n", 267 | " \n", 268 | " \n", 269 | " \n", 270 | " \n", 271 | " \n", 272 | " \n", 273 | " \n", 274 | " \n", 275 | " \n", 276 | " \n", 277 | " \n", 278 | " \n", 279 | " \n", 280 | " \n", 281 | " \n", 282 | " \n", 283 | "
date60616263646566676869...112113114115116117118119120121
country
China1.9710651.9953812.0245622.0467952.0725012.0940392.1148822.1343362.1530942.169074...3.2209573.2209573.2209573.2209573.2209573.2209573.2216523.2216523.2216523.221652
France0.0306400.0306400.0306400.0459600.0612810.0612810.1072410.1378810.1532020.291083...310.462841318.597841326.932003334.837200340.796738346.449874350.157350356.852256362.474751369.016455
Germany0.0000000.0000000.0000000.0000000.0000000.0000000.0000000.0000000.0000000.000000...54.87924958.23311460.79923863.50858765.64503567.31600068.62890070.57438072.98534375.050178
Italy0.3473270.4796410.5788780.8600471.3231491.7697122.4478253.2582543.8536716.053406...398.830151407.662170414.889871422.564134429.510666436.374501440.674734446.182342452.500378457.842591
Spain0.0000000.0000000.0000000.0000000.0000000.0213880.0641650.1069410.1069410.106941...455.183377464.487238473.898039481.747504489.832238495.992036495.992036509.509369519.1982180.000000
United Kingdom0.0000000.0000000.0000000.0000000.0000000.0000000.0147310.0147310.0294610.044192...243.187093255.384011266.623441276.021549287.334632299.310591305.394320310.697327319.329445384.423864
United States0.0000000.0030210.0060420.0181270.0271900.0332320.0362530.0422960.0513590.063444...128.515589136.140905141.340259150.944412154.128677160.690558165.787194169.923112176.297684184.185838
\n", 284 | "

7 rows × 62 columns

\n", 285 | "
" 286 | ], 287 | "text/plain": [ 288 | "date 60 61 62 63 64 65 \\\n", 289 | "country \n", 290 | "China 1.971065 1.995381 2.024562 2.046795 2.072501 2.094039 \n", 291 | "France 0.030640 0.030640 0.030640 0.045960 0.061281 0.061281 \n", 292 | "Germany 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 \n", 293 | "Italy 0.347327 0.479641 0.578878 0.860047 1.323149 1.769712 \n", 294 | "Spain 0.000000 0.000000 0.000000 0.000000 0.000000 0.021388 \n", 295 | "United Kingdom 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 \n", 296 | "United States 0.000000 0.003021 0.006042 0.018127 0.027190 0.033232 \n", 297 | "\n", 298 | "date 66 67 68 69 ... 112 \\\n", 299 | "country ... \n", 300 | "China 2.114882 2.134336 2.153094 2.169074 ... 3.220957 \n", 301 | "France 0.107241 0.137881 0.153202 0.291083 ... 310.462841 \n", 302 | "Germany 0.000000 0.000000 0.000000 0.000000 ... 54.879249 \n", 303 | "Italy 2.447825 3.258254 3.853671 6.053406 ... 398.830151 \n", 304 | "Spain 0.064165 0.106941 0.106941 0.106941 ... 455.183377 \n", 305 | "United Kingdom 0.014731 0.014731 0.029461 0.044192 ... 243.187093 \n", 306 | "United States 0.036253 0.042296 0.051359 0.063444 ... 128.515589 \n", 307 | "\n", 308 | "date 113 114 115 116 117 \\\n", 309 | "country \n", 310 | "China 3.220957 3.220957 3.220957 3.220957 3.220957 \n", 311 | "France 318.597841 326.932003 334.837200 340.796738 346.449874 \n", 312 | "Germany 58.233114 60.799238 63.508587 65.645035 67.316000 \n", 313 | "Italy 407.662170 414.889871 422.564134 429.510666 436.374501 \n", 314 | "Spain 464.487238 473.898039 481.747504 489.832238 495.992036 \n", 315 | "United Kingdom 255.384011 266.623441 276.021549 287.334632 299.310591 \n", 316 | "United States 136.140905 141.340259 150.944412 154.128677 160.690558 \n", 317 | "\n", 318 | "date 118 119 120 121 \n", 319 | "country \n", 320 | "China 3.221652 3.221652 3.221652 3.221652 \n", 321 | "France 350.157350 356.852256 362.474751 369.016455 \n", 322 | "Germany 68.628900 70.574380 72.985343 75.050178 \n", 323 | "Italy 440.674734 446.182342 452.500378 457.842591 \n", 324 | "Spain 495.992036 509.509369 519.198218 0.000000 \n", 325 | "United Kingdom 305.394320 310.697327 319.329445 384.423864 \n", 326 | "United States 165.787194 169.923112 176.297684 184.185838 \n", 327 | "\n", 328 | "[7 rows x 62 columns]" 329 | ] 330 | }, 331 | "metadata": {}, 332 | "output_type": "display_data" 333 | } 334 | ], 335 | "source": [ 336 | "import numpy as np\n", 337 | "import pandas as pd\n", 338 | "from datetime import datetime\n", 339 | "\n", 340 | "# Load data from file\n", 341 | "infections_df = pd.read_csv(\"data/total-confirmed-cases-of-covid-19-per-million-people.csv\")\n", 342 | "deaths_df = pd.read_csv(\"data/total-covid-deaths-per-million.csv\")\n", 343 | "\n", 344 | "# Filter data for given countries\n", 345 | "country_codes = [\"DEU\", \"ITA\", \"FRA\", \"GBR\", \"ESP\", \"CHN\", \"USA\"]\n", 346 | "infections_df = infections_df.loc[infections_df[\"Code\"].isin(country_codes)]\n", 347 | "infections_df.columns = [\"country\", \"code\", \"date\", \"infections per million\"]\n", 348 | "deaths_df = deaths_df.loc[deaths_df[\"Code\"].isin(country_codes)]\n", 349 | "deaths_df.columns = [\"country\", \"code\", \"date\", \"deaths per million\"]\n", 350 | "\n", 351 | "# Convert dates to integer starting Dec 31, 2019\n", 352 | "def daysfromdate(datestring, startdate=\"Dec 31, 2019\"):\n", 353 | " \"\"\"Return number of days from ``startdate`` given a date.\"\"\"\n", 354 | " dayzero = datetime.strptime(startdate, '%b %d, %Y')\n", 355 | " dt = datetime.strptime(datestring, '%b %d, %Y')\n", 356 | " return (dt - dayzero).days\n", 357 | "\n", 358 | "infections_df['date'] = infections_df['date'].apply(daysfromdate)\n", 359 | "deaths_df['date'] = deaths_df['date'].apply(daysfromdate)\n", 360 | "\n", 361 | "# Select data from February onward\n", 362 | "day0 = 31\n", 363 | "infections_df = infections_df.loc[infections_df[\"date\"] > day0]\n", 364 | "deaths_df = deaths_df.loc[deaths_df[\"date\"] > day0]\n", 365 | "\n", 366 | "# Pivot to long format\n", 367 | "infections_df = infections_df.pivot(index=\"country\", columns=\"date\", values=\"infections per million\")\n", 368 | "deaths_df = deaths_df.pivot(index=\"country\", columns=\"date\", values=\"deaths per million\")\n", 369 | "\n", 370 | "# Fill NaNs with zeros\n", 371 | "infections_df.fillna(value=0, inplace=True)\n", 372 | "deaths_df.fillna(value=0, inplace=True)\n", 373 | "\n", 374 | "# Display tables\n", 375 | "# display(infections_df)\n", 376 | "display(deaths_df.loc[:, 60:])\n", 377 | "\n", 378 | "# Store each timeseries in the row of a numpy array\n", 379 | "infections_permil = infections_df.to_numpy()\n", 380 | "deaths_permil = deaths_df.to_numpy()" 381 | ] 382 | }, 383 | { 384 | "cell_type": "markdown", 385 | "metadata": {}, 386 | "source": [ 387 | "## Probabilistic Model of Mortality\n", 388 | "\n", 389 | "We build a simple probabilistic model for the number of infections and the mortality rate for COVID-19. Let \n", 390 | "$X_{ct} \\in \\mathbb{N}$ denote the latent number of *infected patients* per million people per country $c$ and time point $t$, $Y_{ct} \\in \\mathbb{N}$ the number of *observed infections* per million people, $V_{ct} \\in \\mathbb{N}$ the number of observed *fatalities* caused by COVID-19 per million, $\\tau_c \\in [0,1]$ the *testing rate* of infected patients for disease per country and $\\rho \\in [0,1]$ the *mortality rate* assumed to be constant across time and countries. The following probabilistic graphical model illustrates our setup.\n", 391 | "![Probabilistic Graphical Model](figures/graphical_model.png)" 392 | ] 393 | }, 394 | { 395 | "cell_type": "markdown", 396 | "metadata": {}, 397 | "source": [ 398 | "**Task:** Making use of the (conditional) independence assumptions encoded in the model, derive the Markov blanket of each of the variables $X_{ct}$, $Y_{ct}$, $V_{ct}$, $\\rho$ and $\\tau_c$.\n", 399 | "\n", 400 | "_Hint:_ The Markov blanket of a node is the minimal set of nodes in a graph when conditioned upon imply (conditional) independence of all other nodes in the graph, i.e. $\\forall B \\in V: \\ P(A \\mid \\text{MB}(A), B) = P(A \\mid \\text{MB}(A))$" 401 | ] 402 | }, 403 | { 404 | "cell_type": "markdown", 405 | "metadata": {}, 406 | "source": [] 407 | }, 408 | { 409 | "cell_type": "markdown", 410 | "metadata": {}, 411 | "source": [ 412 | "We make the following assumptions in our probabilistic model. We observe the total number of tests performed and deaths occurred per day.\n", 413 | "- $Y_{ct} = y_{ct}$\n", 414 | "- $V_{ct} = v_{ct}$\n", 415 | "\n", 416 | "We assume that our observations are corrupted by Gaussian noise given the latent number of infections, the mortality and the testing rate, respectively.\n", 417 | "- $Y_{ct} \\mid X_{ct}, \\tau_c \\sim \\mathcal{N}(\\tau_c X_{ct}, \\sigma_y^2)$\n", 418 | "- $V_{ct} \\mid X_{ct}, \\rho \\sim \\mathcal{N}(\\rho X_{ct}, \\sigma_v^2)$\n", 419 | "\n", 420 | "where $\\sigma_y^2 = 10^{-2}$ and $\\sigma_v^2 = 10^{-4}$.\n", 421 | "\n", 422 | "**Task:** What is the underlying assumed relationship between the observed infections $Y_{ct}$ and the observed deaths $V_{ct}$, if the observation noise collapses to zero for $\\sigma_y^2 \\rightarrow 0$ and $\\sigma_v^2 \\rightarrow 0$?" 423 | ] 424 | }, 425 | { 426 | "cell_type": "markdown", 427 | "metadata": {}, 428 | "source": [] 429 | }, 430 | { 431 | "cell_type": "markdown", 432 | "metadata": {}, 433 | "source": [ 434 | "We model the latent mortality rate as a Beta distribution depending on the latent number of cases and the observed fatalities. Assume the following parameterization\n", 435 | "- $\\rho \\sim \\text{Beta}\\left(\\sum_c V_{ct} + M_\\rho \\mu_\\rho, \\sum_c X_{ct} - \\sum_c V_{ct} + M_\\rho(1-\\mu_\\rho) \\right)$,\n", 436 | "\n", 437 | "where $\\mu_\\rho=0.05$ is the prior belief about the mortality before observing any data. This choice is motivated by the mortality of the flu (0.001) and SARS (0.1). The hyperparameter $M_\\rho=1$ controls the strength of our prior belief about the mortality.\n", 438 | "\n", 439 | "We put a Gaussian prior on the number of latent infections with a constant prior mean function $(\\mu_x)_t = 10$ and variance $\\sigma_x^2 = 0.1(\\mu_x)_t$. modelling our belief over the progress of the disease before observing any actual cases.\n", 440 | "- $X_{ct} \\sim \\mathcal{N}((\\mu_x)_t, \\sigma_x^2)$\n", 441 | "\n", 442 | "Assuming we are given the observed infections and deaths, the testing and mortality rate, the current number of fatalities at time $t$, then the posterior over the latent number of infections according to our observational model is given by\n", 443 | "- $X_{ct} \\mid Y_{ct}, V_{ct}, \\rho, \\tau_c \\sim \\mathcal{N}(\\mu_{ct}, \\sigma_c^2)$,\n", 444 | "\n", 445 | "where $\\mu_{ct} = \\big(\\frac{\\tau_c y_{ct}}{\\sigma_y^2} +\\frac{\\rho v_{ct}}{\\sigma_v^2} + \\frac{(\\mu_x)_t}{\\sigma_x^2} \\big)\\sigma^{2}$ and $\\sigma^2_{ct} =\\big(\\frac{\\tau_c^2}{\\sigma_y^2} +\\frac{\\rho^2}{\\sigma_v^2} + \\frac{1}{\\sigma_x^2} \\big)^{-1}$." 446 | ] 447 | }, 448 | { 449 | "cell_type": "markdown", 450 | "metadata": {}, 451 | "source": [ 452 | "This can be derived as follows:\n", 453 | "\\begin{align}\n", 454 | "p(X_{ct} \\mid Y_{ct}, V_{ct}, \\rho, \\tau_c) &\\propto p(X_{ct}, Y_{ct}, V_{ct} \\mid \\rho, \\tau_c) = p(Y_{ct} \\mid X_{ct}, \\tau_c)p(V_{ct} \\mid X_{ct}, \\rho) p(X_{ct})\\\\\n", 455 | "&=\\mathcal{N}(Y_{ct}; \\tau_c X_{ct}, \\sigma_y^2)\\mathcal{N}(V_{ct}; \\rho X_{ct}, \\sigma_v^2)\\mathcal{N}(X_{ct}; (\\mu_x)_t, \\sigma_x^2)\\\\\n", 456 | "&\\propto \\exp \\bigg[ -\\frac{1}{2} \\Big(\\big(\\frac{\\tau_c^2}{\\sigma_y^2} +\\frac{\\rho^2}{\\sigma_v^2} + \\frac{1}{\\sigma_x^2} \\big)x_{ct}^2 - 2 \\big(\\frac{\\tau_c y_{ct}}{\\sigma_y^2} +\\frac{\\rho v_{ct}}{\\sigma_v^2} + \\frac{(\\mu_x)_t}{\\sigma_x^2} \\big)x_{ct} \\Big)\\bigg]\\\\\n", 457 | "&\\propto \\mathcal{N}(\\mu_{ct}, \\sigma_{ct}^2)\n", 458 | "\\end{align}\n", 459 | "\n" 460 | ] 461 | }, 462 | { 463 | "cell_type": "markdown", 464 | "metadata": {}, 465 | "source": [ 466 | "Finally, we estimate the per country testing rates also using a Beta distribution as above where the observations are given by the latent number of infections and the observed number of infections, like so\n", 467 | "- $\\tau_c \\sim \\text{Beta}\\left(Y_{ct} + M_\\tau \\mu_\\tau, X_{ct} - Y_{ct} + M_c(1-\\mu_\\tau) \\right)$\n", 468 | "\n", 469 | "where analoguously $\\mu_\\tau=0.1$ represents the prior belief about the testing rate, which we a priori assume to be the same across countries and $M_\\tau=1$ controls the prior influence. Clearly, this assumption is very restrictive as it assumes that the testing rate is constant over time and we do not actually incorporate any data about how many people are tested relative to the entire population in our model. However we can hope to extract some information from the difference in testing rate across countries.\n", 470 | "\n", 471 | "**Task:** Compute the expectation $\\mathbb{E}[\\rho]$ at a given time $t$. How does it behave for a large numbers of cases (and therefore fatalities)?" 472 | ] 473 | }, 474 | { 475 | "cell_type": "markdown", 476 | "metadata": {}, 477 | "source": [] 478 | }, 479 | { 480 | "cell_type": "markdown", 481 | "metadata": {}, 482 | "source": [ 483 | "### Inference via Markov-Chain Monte-Carlo\n", 484 | "\n", 485 | "We will now derive an inference procedure for our model based on the observed data. We will implement a Gibbs sampler, a form of Markov-Chain Monte-Carlo. To do so we need to be able to sample from the conditional distributions of every variable given all others.\n", 486 | "\n", 487 | "**Task:** Based on the derived Markov blankets and model assumptions state the conditional distributions (and observed variables) needed to perform one iteration of the Gibbs sampler.\n", 488 | "\n", 489 | "_Hint:_ If you had trouble deriving the Markov blankets, you can also read of conditional independence assumptions directly from the model description." 490 | ] 491 | }, 492 | { 493 | "cell_type": "markdown", 494 | "metadata": {}, 495 | "source": [] 496 | }, 497 | { 498 | "cell_type": "markdown", 499 | "metadata": {}, 500 | "source": [ 501 | "**Task:** Implement a Gibbs sampler to estimate the joint distribution of the latent variables given the observed number of infections and fatal cases over time." 502 | ] 503 | }, 504 | { 505 | "cell_type": "code", 506 | "execution_count": 2, 507 | "metadata": {}, 508 | "outputs": [], 509 | "source": [ 510 | "import numpy as np\n", 511 | "\n", 512 | "def gibbs(y, v, burnin=50, samples=10000):\n", 513 | " \"\"\"\n", 514 | " Gibbs sampler to infer parameters of the COVID-19 mortality model.\n", 515 | " \n", 516 | " Parameters\n", 517 | " ----------\n", 518 | " y : ndarray\n", 519 | " Observed infections.\n", 520 | " v : ndarray\n", 521 | " Observed fatalities.\n", 522 | " burnin : int\n", 523 | " Number of burn-in samples.\n", 524 | " samples : int\n", 525 | " Number of samples to draw.\n", 526 | " \"\"\"\n", 527 | " # Setup\n", 528 | " \n", 529 | " # Initialize Gibbs sampler (by sampling from priors if necessary)\n", 530 | " \n", 531 | " # Sample from conditional distributions\n", 532 | " \n", 533 | " # Testing rate\n", 534 | "\n", 535 | " \n", 536 | " # Observations\n", 537 | "\n", 538 | " \n", 539 | " # Latent infections\n", 540 | "\n", 541 | " \n", 542 | " # Mortality\n", 543 | "\n", 544 | " raise NotImplementedError" 545 | ] 546 | }, 547 | { 548 | "cell_type": "markdown", 549 | "metadata": {}, 550 | "source": [ 551 | "### Mortality Rate\n", 552 | "\n", 553 | "**Task:** Compute the posterior mean and 95% credible interval of the marginal distribution $p(\\rho \\mid Y_{ct}, V_{ct})$ for the mortality rate $\\rho$ using the Gibbs sampler. Draw $100\\,000$ samples to do so." 554 | ] 555 | }, 556 | { 557 | "cell_type": "code", 558 | "execution_count": 3, 559 | "metadata": { 560 | "scrolled": true 561 | }, 562 | "outputs": [], 563 | "source": [ 564 | "# Sampling\n", 565 | "np.random.seed(1)\n", 566 | "\n", 567 | "# Posterior mean of the mortality rate\n", 568 | "\n", 569 | "\n", 570 | "# Credible interval\n" 571 | ] 572 | }, 573 | { 574 | "cell_type": "markdown", 575 | "metadata": {}, 576 | "source": [ 577 | "**Task:** Create scatterplots between the testing rate and mortality early on in time and towards the present for China, Germany and Italy. What do you observe? Do your observations match your expectation?" 578 | ] 579 | }, 580 | { 581 | "cell_type": "code", 582 | "execution_count": 4, 583 | "metadata": { 584 | "scrolled": true 585 | }, 586 | "outputs": [], 587 | "source": [ 588 | "%matplotlib inline\n", 589 | "import matplotlib.pyplot as plt\n", 590 | "\n", 591 | "# Plot of testing rate versus mortality\n" 592 | ] 593 | }, 594 | { 595 | "cell_type": "markdown", 596 | "metadata": {}, 597 | "source": [] 598 | }, 599 | { 600 | "cell_type": "markdown", 601 | "metadata": {}, 602 | "source": [ 603 | "**Task:** Visualize the estimated mortality rate over time with its associated uncertainty based on the computed distribution. What do you observe? How does your estimate compare to the case fatality rate (CFR)? Are there differences and if yes why?" 604 | ] 605 | }, 606 | { 607 | "cell_type": "markdown", 608 | "metadata": {}, 609 | "source": [] 610 | }, 611 | { 612 | "cell_type": "code", 613 | "execution_count": 8, 614 | "metadata": {}, 615 | "outputs": [], 616 | "source": [ 617 | "# Setup\n", 618 | "days = np.arange(np.shape(infections_permil)[1]) + day0\n", 619 | "\n", 620 | "# Plot\n" 621 | ] 622 | }, 623 | { 624 | "cell_type": "markdown", 625 | "metadata": {}, 626 | "source": [ 627 | "## Model Evaluation and Improvement\n", 628 | "\n", 629 | "**Task:** What simplifying assumptions are you making in your model, which might be violated in reality?\n", 630 | "- ...\n", 631 | "\n", 632 | "**Task:** What can you say about the sensitivity of the model with regard to the testing rate $\\tau$ and other parameter / prior choices?\n", 633 | "- ...\n", 634 | "\n", 635 | "**Task:** How would you improve the model? Describe a more realistic probabilistic model of COVID-19 mortality based on your findings above.\n", 636 | "- ..." 637 | ] 638 | }, 639 | { 640 | "cell_type": "markdown", 641 | "metadata": {}, 642 | "source": [ 643 | "**Image Sources**:\n", 644 | "- Jiumeng Sun, Wanting He, Lifang Wang, et al. COVID-19: epidemiology, evolution, and cross-disciplinary perspectives. _Trends in Molecular Medicine_.\n", 645 | "\n", 646 | "**References**:\n", 647 | "-" 648 | ] 649 | } 650 | ], 651 | "metadata": { 652 | "kernelspec": { 653 | "display_name": "Python 3", 654 | "language": "python", 655 | "name": "python3" 656 | }, 657 | "language_info": { 658 | "codemirror_mode": { 659 | "name": "ipython", 660 | "version": 3 661 | }, 662 | "file_extension": ".py", 663 | "mimetype": "text/x-python", 664 | "name": "python", 665 | "nbconvert_exporter": "python", 666 | "pygments_lexer": "ipython3", 667 | "version": "3.6.9" 668 | } 669 | }, 670 | "nbformat": 4, 671 | "nbformat_minor": 4 672 | } 673 | -------------------------------------------------------------------------------- /ex04/Exercise_04.md: -------------------------------------------------------------------------------- 1 | # Exercise 04 2 | 3 | ## Question 1 4 | 5 | Consider the following model 6 | $$ y = \phi_X^\top \w + \epsilon$$ 7 | 8 | with prior $p(\w) = \mathcal N(\w;\b\mu,\Sigma)$ and i.i.d noise $\epsilon=\mathcal N(0,\sigma^2)$. 9 | 10 | The likelihood is then given by $p(\y|w,X) = \mathcal N(\y; \phi_X^\top \w,\Lambda)$, with $\Lambda=\sigma^2I $ 11 | 12 | #### (a) Marginal distribution (evidence) 13 | 14 | The marginal distribution (or evidence) is given by: 15 | 16 | $$ p(\y|X) = \int p(\y|\w) p(\w) d\w = \mathcal N(\y \,;\, \phi_X^\top \,\b\mu_\w \,,\, \phi_X^\top \,\Sigma_\w \, \phi_X + \Lambda) $$ 17 | 18 | #### (b) Posterior distribution 19 | 20 | The posterior is then given by 21 | 22 | $$ p(\w|\y,X) = \frac{p(\w) p(\y|\w,X)}{p(\y,X)} = \frac{p(\w) p(\y|\w,X)}{\int p(\y|\w,X) p(\w) d\w} $$ 23 | 24 | which is Gaussian, since the likelihood and the prior are Gaussians. The posterior can therefore be evaluated by making use of Gaussian algebraic properties, such that Bayesian inference becomes linear algebra 25 | 26 | $$ 27 | \boxed{ 28 | p(\w|\y,X) = \mathcal N ( 29 | \w 30 | \;;\; 31 | \b\mu+\Sigma \phi_X (\phi_X^\top \Sigma \phi_X+ \Lambda )^{-1} (\y-\phi_X^\top \b\mu ) 32 | \;,\; 33 | \Sigma - \Sigma \, \phi_X (\phi_X^\top \Sigma \phi_X + \Lambda )^{-1} \phi_X^\top \, \Sigma) 34 | ) 35 | } 36 | $$ 37 | 38 | --- 39 | 40 | ## Question 2 41 | 42 | #### (a) Maximum Likelihood Estimation (ML) 43 | 44 | Another way to find $\w$ is to use the **maximum likelihood estimation**. Note that we consider i.i.d. noise, such that the joint distribution $\y$ is independent when condtioned on $\w$, i.e. $\mathcal N(\y; \phi_X^\top \w,\Lambda) = \prod_i \, \mathcal N(y_i; \phi_{x_i}^\top \w, \sigma^2 )$ 45 | 46 | $$ 47 | \begin{align} 48 | \w_{ML} &= \min_w J(\w) \\ 49 | &= \min_w - \log \; p(\y|\w,X) \\ 50 | &= \min_w -\log \; \mathcal N(\y; \phi_X^\top \w,\Lambda) \\ 51 | &= \min_w \left( \frac{n}{2}\log(2\pi) + \frac{1}{2}\log |\Lambda| + (\y - \phi_X^\top \w )^\top \, \Lambda^{-1} (\y - \phi_X^\top \w ) \right) \\ 52 | \end{align} 53 | $$ 54 | 55 | According to the first order necessary optimality condition $\nabla_w J(\w) = 0 $ and remembering that $\Lambda$ is Hermitian we have 56 | 57 | $$ 58 | \begin{align} 59 | \nabla_w J(\w) &= -\phi_X \Lambda^{-1} (\y - \phi_X^\top \w ) - \phi_X \Lambda^{-1\;T} (\y - \phi_X^\top \w )\\ 60 | &= - 2 \; \phi_X \Lambda^{-1} (\y - \phi_X^\top \w ) \\ 61 | &= 0 62 | \end{align} 63 | $$ 64 | 65 | $$ 66 | \begin{align} 67 | \Rightarrow \quad \phi_X \Lambda^{-1} \phi_X^\top \w = \phi_X \Lambda^{-1} \y \quad \\ 68 | \boxed{ 69 | \w = \left( \phi_X \Lambda^{-1} \phi_X^\top \right)^{-1} \phi_X \Lambda^{-1} \y } 70 | \end{align} 71 | $$ 72 | 73 | #### (b) Maximum a-posteriori Estimation (MAP) 74 | 75 | By an analogous computation, we find $\w$ that maximizes the posterior distribution $p(\w|\y,X) = \mathcal N(\w; \b\mu',\Sigma')$, as defined in exercise 1b). We then get 76 | 77 | $$ 78 | \begin{align} 79 | \w_{MAP} &= \min_w J(\w) \\ 80 | &= \min_w - \log \; p(\w|\y,X) \\ 81 | &= \min_w -\log \; \mathcal N(\w; \b\mu',\Sigma') \\ 82 | &= \min_w \left( \frac{n}{2}\log(2\pi) + \frac{1}{2}\log |\Sigma'| + (\w - \b\mu')^\top \, \Sigma'^{-1} (\w - \b\mu' ) \right) \\ 83 | \end{align} 84 | $$ 85 | 86 | which is clearly minimized when $\w = \mathbb E_{p(\w|\y)}[\w] = \b\mu'$. This implies that 87 | 88 | $$ 89 | \begin{align} 90 | \boxed{ 91 | \w_{MAP} = \mu + \Sigma \, \phi_X (\phi_X^\top \,\Sigma \, \phi_X + \Lambda )^{-1} (\y-\phi_X^\top \mu ) \\ 92 | \quad \quad \;\; = (\Sigma^{-1} + \phi_X \Lambda^{-1} \phi_X^\top )^{-1} (\phi_X \Lambda^{-1} \y + \Sigma^{-1} \b\mu ) 93 | } 94 | \end{align} 95 | $$ 96 | 97 | Please note that both formulas above are the same. They are just the Schur complement of oneanother. 98 | 99 | For a particular choise $\b\mu=0$ we see that the MAP estimation is just a $\ell_2$-regularized least-square estimator 100 | 101 | $$ 102 | \begin{align} 103 | \w_{MAP} = (\Sigma^{-1} + \phi_X \Lambda^{-1} \phi_X^\top )^{-1} \phi_X \Lambda^{-1} \y 104 | \end{align} 105 | $$ 106 | 107 |
108 | 109 |
110 | 111 | Please note that another way of seeing the same thing is to view the posterior as $p(\w|\y,X)\propto p(\y|\w,X)p(\w)$, which implies that 112 | 113 | 114 | $$ 115 | \begin{align} 116 | \w_{MAP} &= \min_w J(\w) \\ 117 | &= \min_w - \log \; p(\y|\w,X) \;\; p(\w) \\ 118 | &= \min_w -\log \; \mathcal N(\y; \phi_X^\top \w,\Lambda) - \log \mathcal N(\w;\b\mu,\Sigma) \\ 119 | &= \min_w \;\; 120 | \underbrace{ \|\y - \phi_X^\top \w \|^2_{\Lambda^{-1}} }_{least-square} + 121 | \underbrace{ \|\w - \b\mu \|^2_{\Sigma^{-1}} }_{\ell_2-regularizer} \\ 122 | \end{align} 123 | $$ 124 | 125 | --- 126 | $\newcommand{\b}{\boldsymbol}$ 127 | $\newcommand{\y}{\boldsymbol{y}}$ 128 | $\newcommand{\w}{\boldsymbol{w}}$ 129 | $\newcommand{\bphi}{\boldsymbol{\phi}}$ -------------------------------------------------------------------------------- /ex04/Exercise_04.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucasrm25/Probabilistic-Machine-Learning/ad017c952c027ef37e0d6ba0bb5e4d0226bdf75c/ex04/Exercise_04.pdf -------------------------------------------------------------------------------- /ex04/figures/covid19_spread.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucasrm25/Probabilistic-Machine-Learning/ad017c952c027ef37e0d6ba0bb5e4d0226bdf75c/ex04/figures/covid19_spread.png -------------------------------------------------------------------------------- /ex04/figures/graphical_model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucasrm25/Probabilistic-Machine-Learning/ad017c952c027ef37e0d6ba0bb5e4d0226bdf75c/ex04/figures/graphical_model.png -------------------------------------------------------------------------------- /lindata.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucasrm25/Probabilistic-Machine-Learning/ad017c952c027ef37e0d6ba0bb5e4d0226bdf75c/lindata.mat -------------------------------------------------------------------------------- /nlindata.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucasrm25/Probabilistic-Machine-Learning/ad017c952c027ef37e0d6ba0bb5e4d0226bdf75c/nlindata.mat --------------------------------------------------------------------------------