├── .gitattributes ├── w5 practice.pdf ├── workshop03-slides.pdf ├── COMP90051_workshops_week02.pdf ├── COMP90051_workshops_week04.pdf ├── COMP90051_workshops_week05.pdf ├── COMP90051_workshops_week06.pdf ├── COMP90051_workshops_week07.pdf ├── COMP90051_workshops_week08.pdf ├── COMP90051_workshops_week09.pdf ├── COMP90051_workshops_week11.pdf ├── COMP90051_workshops_week12.pdf ├── README.md ├── Week10.ipynb └── Torch_Autograd.ipynb /.gitattributes: -------------------------------------------------------------------------------- 1 | # Auto detect text files and perform LF normalization 2 | * text=auto 3 | -------------------------------------------------------------------------------- /w5 practice.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HanXudong/COMP90051_Workshops/HEAD/w5 practice.pdf -------------------------------------------------------------------------------- /workshop03-slides.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HanXudong/COMP90051_Workshops/HEAD/workshop03-slides.pdf -------------------------------------------------------------------------------- /COMP90051_workshops_week02.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HanXudong/COMP90051_Workshops/HEAD/COMP90051_workshops_week02.pdf -------------------------------------------------------------------------------- /COMP90051_workshops_week04.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HanXudong/COMP90051_Workshops/HEAD/COMP90051_workshops_week04.pdf -------------------------------------------------------------------------------- /COMP90051_workshops_week05.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HanXudong/COMP90051_Workshops/HEAD/COMP90051_workshops_week05.pdf -------------------------------------------------------------------------------- /COMP90051_workshops_week06.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HanXudong/COMP90051_Workshops/HEAD/COMP90051_workshops_week06.pdf -------------------------------------------------------------------------------- /COMP90051_workshops_week07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HanXudong/COMP90051_Workshops/HEAD/COMP90051_workshops_week07.pdf -------------------------------------------------------------------------------- /COMP90051_workshops_week08.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HanXudong/COMP90051_Workshops/HEAD/COMP90051_workshops_week08.pdf -------------------------------------------------------------------------------- /COMP90051_workshops_week09.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HanXudong/COMP90051_Workshops/HEAD/COMP90051_workshops_week09.pdf -------------------------------------------------------------------------------- /COMP90051_workshops_week11.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HanXudong/COMP90051_Workshops/HEAD/COMP90051_workshops_week11.pdf -------------------------------------------------------------------------------- /COMP90051_workshops_week12.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HanXudong/COMP90051_Workshops/HEAD/COMP90051_workshops_week12.pdf -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # COMP90051_Workshops 2 | 3 | ## Quality of Tutor/Demonstrator Survey 4 | https://apps.eng.unimelb.edu.au/casmas/index.php?r=qoct/subjects 5 | 6 | ## Week 10 7 | 8 | 1. Week 10 notebook has been updated with more explanations 9 | 2. There are some mistakes in question 3, let's revirew it next week. Sorry!!! 10 | 11 | ## Week 9 timetable 12 | Due to the public holiday, I will take the following 3 tutes: 13 | 1. Monday 16:15 PAR-207-221 Bouverie St-B116 14 | 2. Wednesday 20:15 PAR-207-221 Bouverie St-B117 15 | 3. Thursday 20:15 PAR-207-221 Bouverie St-B117 16 | 17 | ## L2 Nrom / Shrinkage Methods 18 | 19 | https://web.stanford.edu/~hastie/Papers/ESLII.pdf 20 | The Elements ofStatistical Learning 21 | 22 | page 64 23 | 24 | In addition, notice that the intercept $\beta_0$ has been left out of the penalty term. Penalization of the intercept would make the procedure depend on the origin chosen for Y; that is, adding a constant $c$ to each of the targets $y_i$ would not simply result in a shift of the predictions by the same amount $c$. It can be shown (Exercise 3.5) that the solution to (3.41) can be separated into two parts, after reparametrization using centered inputs: each $x_{ij}$ gets replaced by $x_{ij} − \bar{x}_j$ . We estimate $\beta_0$ by $ \bar{y} = \frac{1}{N}\sum_1^{N}y_i $. The remaining coefficients get estimated by a ridge regression without intercept, using the centered $x_ij$ . Henceforth we assume that this centering has been done, so that the input matrix X has p (rather than p + 1) columns. 25 | -------------------------------------------------------------------------------- /Week10.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import numpy as np" 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": 2, 15 | "metadata": {}, 16 | "outputs": [ 17 | { 18 | "data": { 19 | "text/plain": [ 20 | "array([[0.5985 , 0.1827 ],\n", 21 | " [0.5922 , 0.00063]])" 22 | ] 23 | }, 24 | "execution_count": 2, 25 | "metadata": {}, 26 | "output_type": "execute_result" 27 | } 28 | ], 29 | "source": [ 30 | "np.array([[0.95, 0.29], [0.94, 0.001]])*0.7*0.9" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": 3, 36 | "metadata": {}, 37 | "outputs": [ 38 | { 39 | "data": { 40 | "text/plain": [ 41 | "array([[2.500e-05, 3.550e-04],\n", 42 | " [3.000e-05, 4.995e-04]])" 43 | ] 44 | }, 45 | "execution_count": 3, 46 | "metadata": {}, 47 | "output_type": "execute_result" 48 | } 49 | ], 50 | "source": [ 51 | "np.array([[1-0.95, 1-0.29], [1-0.94, 1-0.001]])*0.05*0.01" 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "execution_count": 4, 57 | "metadata": {}, 58 | "outputs": [ 59 | { 60 | "data": { 61 | "text/plain": [ 62 | "array([[0.598525 , 0.183055 ],\n", 63 | " [0.59223 , 0.0011295]])" 64 | ] 65 | }, 66 | "execution_count": 4, 67 | "metadata": {}, 68 | "output_type": "execute_result" 69 | } 70 | ], 71 | "source": [ 72 | "np.array([[0.95, 0.29], [0.94, 0.001]])*0.7*0.9+np.array([[1-0.95, 1-0.29], [1-0.94, 1-0.001]])*0.05*0.01" 73 | ] 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": {}, 78 | "source": [ 79 | "* P(J=T,M=T|B=T,E=T) = 0.598525\n", 80 | "* P(J=T,M=T|B=F,E=T) = 0.183055\n", 81 | "* P(J=T,M=T|B=T,E=F) = 0.59223\n", 82 | "* P(J=T,M=T|B=F,E=F) = 0.0011295" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "Given P(E=T)=0.002 and P(E=F)=0.998, this product is actuay the marginalization and its result can be regarded as:\n", 90 | "* P(E=T)P(J=T,M=T|B=T,E=T) + P(E=F)P(J=T,M=T|B=T,E=F) = P(J=T,M=T|B=T)\n", 91 | "* P(E=T)P(J=T,M=T|B=F,E=T) + P(E=F)P(J=T,M=T|B=F,E=F) = P(J=T,M=T|B=F)" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": 5, 97 | "metadata": {}, 98 | "outputs": [ 99 | { 100 | "data": { 101 | "text/plain": [ 102 | "array([0.59224259, 0.00149335])" 103 | ] 104 | }, 105 | "execution_count": 5, 106 | "metadata": {}, 107 | "output_type": "execute_result" 108 | } 109 | ], 110 | "source": [ 111 | "np.dot(np.array([0.002, 0.998]), np.array([[0.598525 , 0.183055 ],[0.59223 , 0.0011295]]))" 112 | ] 113 | }, 114 | { 115 | "cell_type": "markdown", 116 | "metadata": {}, 117 | "source": [ 118 | "* P(J=T,M=T|B=T) = 0.59224259\n", 119 | "* P(J=T,M=T|B=F) = 0.00149335" 120 | ] 121 | }, 122 | { 123 | "cell_type": "code", 124 | "execution_count": 6, 125 | "metadata": {}, 126 | "outputs": [ 127 | { 128 | "data": { 129 | "text/plain": [ 130 | "0.0020840992399999998" 131 | ] 132 | }, 133 | "execution_count": 6, 134 | "metadata": {}, 135 | "output_type": "execute_result" 136 | } 137 | ], 138 | "source": [ 139 | "np.dot(np.array([0.001, 0.999]), np.array([0.59224259, 0.00149335]))" 140 | ] 141 | }, 142 | { 143 | "cell_type": "markdown", 144 | "metadata": {}, 145 | "source": [ 146 | "Marginalization over B: P(J=T,M=T) = P(B=T)P(J=T,M=T|B=T)+P(B=F)P(J=T,M=T|B=F) = 0.0020840992399999998" 147 | ] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "execution_count": 7, 152 | "metadata": {}, 153 | "outputs": [ 154 | { 155 | "data": { 156 | "text/plain": [ 157 | "0.2841719715803937" 158 | ] 159 | }, 160 | "execution_count": 7, 161 | "metadata": {}, 162 | "output_type": "execute_result" 163 | } 164 | ], 165 | "source": [ 166 | "1/0.0020840992399999998 * 0.001 * 0.59224259" 167 | ] 168 | }, 169 | { 170 | "cell_type": "markdown", 171 | "metadata": {}, 172 | "source": [ 173 | "P(B=T|J=T,M=T) = 1/P(J=T,M=T) * P(J=T,M=T|B=T)P(B=T) = 0.2841719715803937" 174 | ] 175 | } 176 | ], 177 | "metadata": { 178 | "kernelspec": { 179 | "display_name": "Python 3", 180 | "language": "python", 181 | "name": "python3" 182 | }, 183 | "language_info": { 184 | "codemirror_mode": { 185 | "name": "ipython", 186 | "version": 3 187 | }, 188 | "file_extension": ".py", 189 | "mimetype": "text/x-python", 190 | "name": "python", 191 | "nbconvert_exporter": "python", 192 | "pygments_lexer": "ipython3", 193 | "version": "3.7.4" 194 | } 195 | }, 196 | "nbformat": 4, 197 | "nbformat_minor": 2 198 | } 199 | -------------------------------------------------------------------------------- /Torch_Autograd.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "Torch_Autograd", 7 | "version": "0.3.2", 8 | "provenance": [], 9 | "include_colab_link": true 10 | }, 11 | "kernelspec": { 12 | "name": "python3", 13 | "display_name": "Python 3" 14 | } 15 | }, 16 | "cells": [ 17 | { 18 | "cell_type": "markdown", 19 | "metadata": { 20 | "id": "view-in-github", 21 | "colab_type": "text" 22 | }, 23 | "source": [ 24 | "\"Open" 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "metadata": { 30 | "id": "OA1HdlaS7Kon", 31 | "colab_type": "code", 32 | "colab": {} 33 | }, 34 | "source": [ 35 | "import torch" 36 | ], 37 | "execution_count": 0, 38 | "outputs": [] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "metadata": { 43 | "id": "BcaePvX57Mhd", 44 | "colab_type": "code", 45 | "colab": {} 46 | }, 47 | "source": [ 48 | "a = torch.tensor(2.0)\n", 49 | "b = torch.tensor(3.0)\n", 50 | "w0 = torch.tensor(1.0, requires_grad = True)\n", 51 | "w1 = torch.tensor(1.0, requires_grad = True)\n", 52 | "v = torch.tensor(1.0, requires_grad = True)\n" 53 | ], 54 | "execution_count": 0, 55 | "outputs": [] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "metadata": { 60 | "id": "y0uiE1Vg7hpj", 61 | "colab_type": "code", 62 | "colab": {} 63 | }, 64 | "source": [ 65 | "u = a*w0 + b*w1 \n", 66 | "f = v*u" 67 | ], 68 | "execution_count": 0, 69 | "outputs": [] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "metadata": { 74 | "id": "w9Gl4fGp7kGW", 75 | "colab_type": "code", 76 | "colab": { 77 | "base_uri": "https://localhost:8080/", 78 | "height": 34 79 | }, 80 | "outputId": "541fc079-c2e1-4f15-b98b-eca1ba547e2a" 81 | }, 82 | "source": [ 83 | "print(u, f)" 84 | ], 85 | "execution_count": 4, 86 | "outputs": [ 87 | { 88 | "output_type": "stream", 89 | "text": [ 90 | "tensor(5., grad_fn=) tensor(5., grad_fn=)\n" 91 | ], 92 | "name": "stdout" 93 | } 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "metadata": { 99 | "id": "lUakB0oz7lQH", 100 | "colab_type": "code", 101 | "colab": {} 102 | }, 103 | "source": [ 104 | "criterion = torch.nn.MSELoss()\n", 105 | "loss = criterion(f, torch.tensor(10.0))" 106 | ], 107 | "execution_count": 0, 108 | "outputs": [] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "metadata": { 113 | "id": "2m1uU07n7qh9", 114 | "colab_type": "code", 115 | "colab": { 116 | "base_uri": "https://localhost:8080/", 117 | "height": 34 118 | }, 119 | "outputId": "91910569-69ac-4ab4-c54d-7e216d16efbb" 120 | }, 121 | "source": [ 122 | "print(loss)" 123 | ], 124 | "execution_count": 6, 125 | "outputs": [ 126 | { 127 | "output_type": "stream", 128 | "text": [ 129 | "tensor(25., grad_fn=)\n" 130 | ], 131 | "name": "stdout" 132 | } 133 | ] 134 | }, 135 | { 136 | "cell_type": "code", 137 | "metadata": { 138 | "id": "_PBn7gs57rrn", 139 | "colab_type": "code", 140 | "colab": {} 141 | }, 142 | "source": [ 143 | "loss.backward()" 144 | ], 145 | "execution_count": 0, 146 | "outputs": [] 147 | }, 148 | { 149 | "cell_type": "code", 150 | "metadata": { 151 | "id": "qYSaO2xn7s0Q", 152 | "colab_type": "code", 153 | "colab": { 154 | "base_uri": "https://localhost:8080/", 155 | "height": 34 156 | }, 157 | "outputId": "cd4ab832-e0a5-4906-87ba-3d1eff728ebc" 158 | }, 159 | "source": [ 160 | "print(w0.grad, w1.grad, v.grad)" 161 | ], 162 | "execution_count": 8, 163 | "outputs": [ 164 | { 165 | "output_type": "stream", 166 | "text": [ 167 | "tensor(-20.) tensor(-30.) tensor(-50.)\n" 168 | ], 169 | "name": "stdout" 170 | } 171 | ] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "metadata": { 176 | "id": "u9KrLSkb7t_n", 177 | "colab_type": "code", 178 | "colab": {} 179 | }, 180 | "source": [ 181 | "u = a*w0 + b*w1 \n", 182 | "f = v*u" 183 | ], 184 | "execution_count": 0, 185 | "outputs": [] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "metadata": { 190 | "id": "3KQElfzQ7xKM", 191 | "colab_type": "code", 192 | "colab": {} 193 | }, 194 | "source": [ 195 | "loss = criterion(f, torch.tensor(10.0))" 196 | ], 197 | "execution_count": 0, 198 | "outputs": [] 199 | }, 200 | { 201 | "cell_type": "code", 202 | "metadata": { 203 | "id": "Ow6Ztv697ysZ", 204 | "colab_type": "code", 205 | "colab": {} 206 | }, 207 | "source": [ 208 | "loss.backward()" 209 | ], 210 | "execution_count": 0, 211 | "outputs": [] 212 | }, 213 | { 214 | "cell_type": "code", 215 | "metadata": { 216 | "id": "JEYfQ5_c7zvp", 217 | "colab_type": "code", 218 | "colab": { 219 | "base_uri": "https://localhost:8080/", 220 | "height": 34 221 | }, 222 | "outputId": "c9b2be1c-4708-4aff-d335-a59617a356ac" 223 | }, 224 | "source": [ 225 | "print(w0.grad, w1.grad, v.grad)" 226 | ], 227 | "execution_count": 12, 228 | "outputs": [ 229 | { 230 | "output_type": "stream", 231 | "text": [ 232 | "tensor(-40.) tensor(-60.) tensor(-100.)\n" 233 | ], 234 | "name": "stdout" 235 | } 236 | ] 237 | } 238 | ] 239 | } --------------------------------------------------------------------------------