├── README.md ├── 2022-2 ├── 열유체공학실험_Week_4 │ └── module.py ├── README.md ├── 열유체공학실험_Week_10.ipynb ├── 열유체공학실험_Week_12.ipynb └── 열유체공학실험_Week_15.ipynb ├── 2023-2 ├── README.md ├── 열유체공학실험1_Week13_PyTorch_Autoencoder.ipynb ├── 열유체공학실험1_Week13_PyTorch_Denoising_Autoencoder.ipynb ├── 열유체공학실험1_Week8_유한차분법.ipynb ├── 열유체공학실험1_Week9_인공신경망.ipynb ├── 열유체공학실험1_Week11_PyTorch_Image_Classification.ipynb └── 열유체공학실험1_Week10_PyTorch_Binary_Classification.ipynb └── LICENSE /README.md: -------------------------------------------------------------------------------- 1 | # [기계공학부] 열유체공학실험 실습 코드 2 | ### 오류 지적은 rhworbs1124@naver.com로 부탁드립니다. 3 | -------------------------------------------------------------------------------- /2022-2/열유체공학실험_Week_4/module.py: -------------------------------------------------------------------------------- 1 | class calculator : 2 | def __init__(self, num1, num2) : 3 | self.num1 = num1 4 | self.num2 = num2 5 | 6 | def add(self) : 7 | return self.num1 + self.num2 8 | 9 | def subtract(self) : 10 | return self.num1 - self.num2 11 | 12 | def multiply(self) : 13 | return self.num1 * self.num2 14 | 15 | def divide(self) : 16 | return self.num1 / self.num2 -------------------------------------------------------------------------------- /2022-2/README.md: -------------------------------------------------------------------------------- 1 | ## 2022년도 2학기 강의 계획서 2 | 3 | |주차|주제|활동사항| 4 | |:---:|:---:|:---:| 5 | |1주차|오리엔테이션|수업에 관한 기본적인 설명| 6 | |2주차|Python 기초 (1)|Python 기초 문법 및 루프 소개| 7 | |3주차|Python 기초 (2)|Python 자료구조 및 함수 소개| 8 | |4주차|객체 지향 프로그래밍|객체 지향 프로그래밍 (OOP) 소개| 9 | |5주차|데이터 분석 대표 라이브러리|NumPy, Pandas, Matplotlib 사용법 소개| 10 | |6주차|API 문서와 GPU 연산 라이브러리|외부 라이브러리 사용을 위한 API 문서 및 CuPy 사용법 소개| 11 | |7주차|통계 이론 및 선형 회귀 모델|통계 이론 및 선형 회귀 모델 설명| 12 | |8주차|중간고사|휴강| 13 | |9주차|머신 러닝 기반 분류 및 회귀 모델|Scikit-Learn을 활용한 분류 및 회귀 모델 사용법 소개| 14 | |10주차|유한차분법|유한차분법 소개| 15 | |11주차|경사하강법|딥 러닝 훈련에 사용되는 경사하강법 소개| 16 | |12주차|인공신경망|인공신경망 및 오차 역전파법 설명| 17 | |13주차|딥 러닝 기반 분류 및 회귀 모델|Keras를 활용한 분류 및 회귀 모델 사용법 소개| 18 | |14주차|딥 러닝 기반 시계열 데이터 분석|Keras를 활용한 시계열 데이터 처리 방법 소개| 19 | |15주차|딥 러닝 기반 CFD 분석|Keras를 활용한 CFD 해석 방법 소개| 20 | |16주차|기말고사|휴강| 21 | -------------------------------------------------------------------------------- /2023-2/README.md: -------------------------------------------------------------------------------- 1 | ## 2023년도 2학기 강의 계획서 2 | 3 | |주차|주제|활동사항| 4 | |:---:|:---:|:---:| 5 | |1주차|오리엔테이션|수업에 관한 기본적인 설명| 6 | |2주차|Python 기초 (1)|Python 기초 문법 및 루프 소개| 7 | |3주차|Python 기초 (2)|Python 자료구조 및 함수 소개| 8 | |4주차|객체 지향 프로그래밍|객체 지향 프로그래밍 (OOP) 소개| 9 | |5주차|휴강|개천절| 10 | |6주차|데이터 분석 대표 라이브러리|NumPy, Pandas, Matplotlib 사용법 소개| 11 | |7주차|통계 이론 및 선형 회귀 모델|통계 이론 및 선형 회귀 모델 설명| 12 | |8주차|유한차분법과 경사하강법|유한차분법 및 딥 러닝 훈련에 사용되는 경사하강법 소개| 13 | |9주차|인공신경망|인공신경망 및 오차 역전파법 설명| 14 | |10주차|딥 러닝 기반 분류 및 회귀 모델|PyTorch를 활용한 분류 및 회귀 모델 사용법 소개| 15 | |11주차|딥 러닝 기반 합성곱 신경망 모델|PyTorch를 활용한 2차원 및 3차원 데이터 처리를 위한 합성곱 신경망 소개| 16 | |12주차|딥 러닝 기반 데이터 차원 축소 및 압축|PyTorch를 활용한 Autoencoder 알고리즘 소개| 17 | |13주차|OpenFOAM을 활용한 데이터 셋 생성 (1)|OpenFOAM을 활용한 CFD 해석 방법 소개| 18 | |14주차|OpenFOAM을 활용한 데이터 셋 생성 (2)|OpenFOAM을 활용한 CFD 해석 방법 소개| 19 | |15주차|딥 러닝 기반 CFD 분석|PyTorch를 활용한 CFD 해석 방법 소개| 20 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 3-Clause License 2 | 3 | Copyright (c) 2023, the respective contributors, as shown by the AUTHORS file. 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | * Redistributions of source code must retain the above copyright notice, this 10 | list of conditions and the following disclaimer. 11 | 12 | * Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | * Neither the name of the copyright holder nor the names of its 17 | contributors may be used to endorse or promote products derived from 18 | this software without specific prior written permission. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 21 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 22 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 23 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 24 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 26 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 27 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 | -------------------------------------------------------------------------------- /2023-2/열유체공학실험1_Week13_PyTorch_Autoencoder.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "view-in-github", 7 | "colab_type": "text" 8 | }, 9 | "source": [ 10 | "\"Open" 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "metadata": { 16 | "id": "QQ97dC_jLHvF" 17 | }, 18 | "source": [ 19 | "# Autoencoder Using PyTorch Framework" 20 | ] 21 | }, 22 | { 23 | "cell_type": "markdown", 24 | "metadata": { 25 | "id": "BX6s-j7fq3hA" 26 | }, 27 | "source": [ 28 | "## Check NVIDIA GPU Setting" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": null, 34 | "metadata": { 35 | "id": "D_I7cz1Sq6S1" 36 | }, 37 | "outputs": [], 38 | "source": [ 39 | "!nvidia-smi" 40 | ] 41 | }, 42 | { 43 | "cell_type": "markdown", 44 | "source": [ 45 | "## Load MNIST Dataset" 46 | ], 47 | "metadata": { 48 | "id": "1sg71j-zWuau" 49 | } 50 | }, 51 | { 52 | "cell_type": "code", 53 | "source": [ 54 | "from torchvision.datasets import MNIST\n", 55 | "from torchvision import transforms" 56 | ], 57 | "metadata": { 58 | "id": "UvcYeIfuWwE9" 59 | }, 60 | "execution_count": null, 61 | "outputs": [] 62 | }, 63 | { 64 | "cell_type": "code", 65 | "source": [ 66 | "trainDataset = MNIST(root=\"content\",\n", 67 | " train=True,\n", 68 | " transform=transforms.Compose([transforms.Resize((32,32)), transforms.ToTensor()]),\n", 69 | " download=True)\n", 70 | "testDataset = MNIST(root=\"content\",\n", 71 | " train=False,\n", 72 | " transform=transforms.Compose([transforms.Resize((32,32)), transforms.ToTensor()]),\n", 73 | " download=True)" 74 | ], 75 | "metadata": { 76 | "id": "TRURIUnrW_1e" 77 | }, 78 | "execution_count": null, 79 | "outputs": [] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": null, 84 | "metadata": { 85 | "id": "Lcjb8ky2PvGo" 86 | }, 87 | "outputs": [], 88 | "source": [ 89 | "from torch import nn\n", 90 | "import torch.nn.functional as F" 91 | ] 92 | }, 93 | { 94 | "cell_type": "code", 95 | "execution_count": null, 96 | "metadata": { 97 | "id": "VpMGcu1KP07P" 98 | }, 99 | "outputs": [], 100 | "source": [ 101 | "class myModel(nn.Module) :\n", 102 | " def __init__(self, opt) :\n", 103 | " super(myModel, self).__init__()\n", 104 | "\n", 105 | " inputDim, channels = opt[\"inputDim\"], opt[\"channels\"]\n", 106 | "\n", 107 | " self.encoder = nn.Sequential(nn.Conv2d(inputDim, channels, kernel_size=3, stride=1, padding=1),\n", 108 | " nn.ReLU(),\n", 109 | " nn.MaxPool2d(kernel_size=2, stride=2),\n", 110 | " nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1),\n", 111 | " nn.ReLU(),\n", 112 | " nn.MaxPool2d(kernel_size=2, stride=2),\n", 113 | " nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1),\n", 114 | " nn.ReLU(),\n", 115 | " nn.MaxPool2d(kernel_size=2, stride=2),\n", 116 | " nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1),\n", 117 | " nn.ReLU())\n", 118 | " self.decoder = nn.Sequential(nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1),\n", 119 | " nn.ReLU(),\n", 120 | " nn.Upsample(scale_factor=2),\n", 121 | " nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1),\n", 122 | " nn.ReLU(),\n", 123 | " nn.Upsample(scale_factor=2),\n", 124 | " nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1),\n", 125 | " nn.ReLU(),\n", 126 | " nn.Upsample(scale_factor=2),\n", 127 | " nn.Conv2d(channels, inputDim, kernel_size=3, stride=1, padding=1))\n", 128 | " self.bottleneck0 = nn.Linear(((opt[\"inputSize\"]//8)**2)*channels, 2)\n", 129 | " self.bottleneck1 = nn.Linear(2, ((opt[\"inputSize\"]//8)**2)*channels)\n", 130 | "\n", 131 | " def forward(self, input) :\n", 132 | " output = self.encoder(input)\n", 133 | " n, c, h, w = output.size()\n", 134 | "\n", 135 | " latentVector = self.bottleneck0(output.view(-1, c*h*w))\n", 136 | "\n", 137 | " output = self.decoder(self.bottleneck1(latentVector).view(n, c, h, w))\n", 138 | "\n", 139 | " return latentVector, output" 140 | ] 141 | }, 142 | { 143 | "cell_type": "markdown", 144 | "metadata": { 145 | "id": "3Wk4_cBzQU19" 146 | }, 147 | "source": [ 148 | "## Train DL Model" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": null, 154 | "metadata": { 155 | "id": "0QJcgBk-QSsg" 156 | }, 157 | "outputs": [], 158 | "source": [ 159 | "import torch\n", 160 | "from torch.utils.data import DataLoader\n", 161 | "from torch import optim\n", 162 | "\n", 163 | "from torchsummary import summary\n", 164 | "\n", 165 | "from tqdm import tqdm" 166 | ] 167 | }, 168 | { 169 | "cell_type": "markdown", 170 | "metadata": { 171 | "id": "phpTOt47RjcB" 172 | }, 173 | "source": [ 174 | "### Fix Seed" 175 | ] 176 | }, 177 | { 178 | "cell_type": "code", 179 | "execution_count": null, 180 | "metadata": { 181 | "id": "81NJ5JxbRnaU" 182 | }, 183 | "outputs": [], 184 | "source": [ 185 | "import random\n", 186 | "import numpy as np" 187 | ] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "execution_count": null, 192 | "metadata": { 193 | "id": "o8F_Z5d3RlH-" 194 | }, 195 | "outputs": [], 196 | "source": [ 197 | "def fixSeed(seed) :\n", 198 | " random.seed(seed)\n", 199 | " np.random.seed(seed)\n", 200 | " torch.manual_seed(seed)\n", 201 | " torch.cuda.manual_seed(seed)\n", 202 | " torch.cuda.manual_seed_all(seed)\n", 203 | " torch.backends.cudnn.deterministic = True\n", 204 | " torch.backends.cudnn.benchmark = False" 205 | ] 206 | }, 207 | { 208 | "cell_type": "markdown", 209 | "source": [ 210 | "## Create Average Meter Instance" 211 | ], 212 | "metadata": { 213 | "id": "Jh4VCYXSswFY" 214 | } 215 | }, 216 | { 217 | "cell_type": "code", 218 | "source": [ 219 | "class AverageMeter(object):\n", 220 | " def __init__(self):\n", 221 | " self.reset()\n", 222 | "\n", 223 | " def reset(self):\n", 224 | " self.val = 0\n", 225 | " self.avg = 0\n", 226 | " self.sum = 0\n", 227 | " self.count = 0\n", 228 | "\n", 229 | " def update(self, val, n=1):\n", 230 | " self.val = val\n", 231 | " self.sum += val*n\n", 232 | " self.count += n\n", 233 | " self.avg = self.sum / self.count" 234 | ], 235 | "metadata": { 236 | "id": "oGUnMpzusxqw" 237 | }, 238 | "execution_count": null, 239 | "outputs": [] 240 | }, 241 | { 242 | "cell_type": "markdown", 243 | "source": [ 244 | "## Training Code as a Function (Abstraction)" 245 | ], 246 | "metadata": { 247 | "id": "lhJ28-fU6Zk7" 248 | } 249 | }, 250 | { 251 | "cell_type": "code", 252 | "source": [ 253 | "def train(opt, trainDataset, testDataset, myModel, criterion) :\n", 254 | " fixSeed(opt[\"seed\"])\n", 255 | "\n", 256 | " trainDataLoader = DataLoader(trainDataset, batch_size=opt[\"batchSize\"], shuffle=True, drop_last=True)\n", 257 | " testDataLoader = DataLoader(testDataset, batch_size=opt[\"batchSize\"], shuffle=False, drop_last=False)\n", 258 | "\n", 259 | " fixSeed(opt[\"seed\"])\n", 260 | " model = myModel(opt)\n", 261 | " if opt[\"isCUDA\"] :\n", 262 | " model = model.cuda()\n", 263 | "\n", 264 | " summary(model, (opt[\"inputDim\"], opt[\"inputSize\"], opt[\"inputSize\"]))\n", 265 | "\n", 266 | " optimizer = optim.Adam(model.parameters(), lr=opt[\"lr\"])\n", 267 | "\n", 268 | " trainLoss, testLoss = AverageMeter(), AverageMeter()\n", 269 | " trainLossList, testLossList = [], []\n", 270 | " bestLoss = torch.inf\n", 271 | "\n", 272 | " for epoch in range(1, opt[\"epochs\"]+1) :\n", 273 | " trainBar = tqdm(trainDataLoader)\n", 274 | " trainLoss.reset()\n", 275 | "\n", 276 | " for data in trainBar :\n", 277 | " input, target = data\n", 278 | " if opt[\"isCUDA\"] :\n", 279 | " input = input.cuda()\n", 280 | " optimizer.zero_grad()\n", 281 | " pred = model(input)\n", 282 | " loss = criterion(pred[-1], input)\n", 283 | " loss.backward()\n", 284 | " optimizer.step()\n", 285 | "\n", 286 | " trainLoss.update(loss.item(), opt[\"batchSize\"])\n", 287 | " trainBar.set_description(desc=f\"[{epoch}/{opt['epochs']}] [Train] < Loss:{trainLoss.avg:.6f} >\")\n", 288 | "\n", 289 | " trainLossList.append(trainLoss.avg)\n", 290 | "\n", 291 | " testBar = tqdm(testDataLoader)\n", 292 | " testLoss.reset()\n", 293 | "\n", 294 | " for data in testBar :\n", 295 | " input, target = data\n", 296 | " if opt[\"isCUDA\"] :\n", 297 | " input = input.cuda()\n", 298 | "\n", 299 | " model.eval()\n", 300 | " with torch.no_grad() :\n", 301 | " pred = model(input)\n", 302 | " loss = criterion(pred[-1], input)\n", 303 | "\n", 304 | " testLoss.update(loss.item(), opt[\"batchSize\"])\n", 305 | "\n", 306 | " testBar.set_description(desc=f\"[{epoch}/{opt['epochs']}] [Test] < Loss:{testLoss.avg:.6f} >\")\n", 307 | "\n", 308 | " testLossList.append(testLoss.avg)\n", 309 | "\n", 310 | " if testLoss.avg < bestLoss :\n", 311 | " bestLoss = testLoss.avg\n", 312 | " torch.save(model.state_dict(), \"bestModel.pth\")\n", 313 | "\n", 314 | " torch.save(model.state_dict(), \"latestModel.pth\")\n", 315 | "\n", 316 | " return (trainLossList, testLossList)" 317 | ], 318 | "metadata": { 319 | "id": "Toe0P0WH6c9p" 320 | }, 321 | "execution_count": null, 322 | "outputs": [] 323 | }, 324 | { 325 | "cell_type": "markdown", 326 | "source": [ 327 | "## Create Training Option (Hyperparameter) Dictionary" 328 | ], 329 | "metadata": { 330 | "id": "ttFH7GYJq6c3" 331 | } 332 | }, 333 | { 334 | "cell_type": "code", 335 | "source": [ 336 | "opt = {\"inputSize\":32,\n", 337 | " \"seed\":42,\n", 338 | " \"inputDim\":1,\n", 339 | " \"channels\":64,\n", 340 | " \"batchSize\":16,\n", 341 | " \"lr\":1e-4,\n", 342 | " \"epochs\":10,\n", 343 | " \"isCUDA\":torch.cuda.is_available()}" 344 | ], 345 | "metadata": { 346 | "id": "tXzQHG9aq9xB" 347 | }, 348 | "execution_count": null, 349 | "outputs": [] 350 | }, 351 | { 352 | "cell_type": "markdown", 353 | "source": [ 354 | "## Train Model" 355 | ], 356 | "metadata": { 357 | "id": "VpE3M2q67qRu" 358 | } 359 | }, 360 | { 361 | "cell_type": "code", 362 | "source": [ 363 | "lossList = train(opt, trainDataset, testDataset, myModel, nn.L1Loss())" 364 | ], 365 | "metadata": { 366 | "id": "yV7Jgs_77c57" 367 | }, 368 | "execution_count": null, 369 | "outputs": [] 370 | }, 371 | { 372 | "cell_type": "markdown", 373 | "metadata": { 374 | "id": "XrgUhJC67uzu" 375 | }, 376 | "source": [ 377 | "## Plot Training vs. Test Loss Graph" 378 | ] 379 | }, 380 | { 381 | "cell_type": "code", 382 | "source": [ 383 | "import matplotlib.pyplot as plt" 384 | ], 385 | "metadata": { 386 | "id": "_lIUGO11pAtu" 387 | }, 388 | "execution_count": null, 389 | "outputs": [] 390 | }, 391 | { 392 | "cell_type": "code", 393 | "execution_count": null, 394 | "metadata": { 395 | "id": "mLsjFs3K7uzv" 396 | }, 397 | "outputs": [], 398 | "source": [ 399 | "plt.figure(figsize=(20,10))\n", 400 | "\n", 401 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), lossList[0], label=\"Training Loss\")\n", 402 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), lossList[1], label=\"Test Loss\")\n", 403 | "\n", 404 | "plt.xlabel(\"Epoch\")\n", 405 | "plt.ylabel(\"L1 Loss\")\n", 406 | "plt.legend(loc=\"best\")\n", 407 | "\n", 408 | "plt.show()" 409 | ] 410 | }, 411 | { 412 | "cell_type": "markdown", 413 | "source": [ 414 | "## Extract Latent Vector" 415 | ], 416 | "metadata": { 417 | "id": "RP4jlNz2e42r" 418 | } 419 | }, 420 | { 421 | "cell_type": "markdown", 422 | "source": [ 423 | "### Load Trained Model" 424 | ], 425 | "metadata": { 426 | "id": "5D-ZF3NhfGkO" 427 | } 428 | }, 429 | { 430 | "cell_type": "code", 431 | "source": [ 432 | "weights = torch.load(\"/content/bestModel.pth\")\n", 433 | "\n", 434 | "model = myModel(opt)\n", 435 | "model.load_state_dict(weights)\n", 436 | "if opt[\"isCUDA\"] :\n", 437 | " model = model.cuda()" 438 | ], 439 | "metadata": { 440 | "id": "lxfxB9W1fIPE" 441 | }, 442 | "execution_count": null, 443 | "outputs": [] 444 | }, 445 | { 446 | "cell_type": "markdown", 447 | "source": [ 448 | "### Get Model Structure" 449 | ], 450 | "metadata": { 451 | "id": "8jjN8rSngATu" 452 | } 453 | }, 454 | { 455 | "cell_type": "code", 456 | "source": [ 457 | "print(model)" 458 | ], 459 | "metadata": { 460 | "id": "Z_vtvoYCfOgg" 461 | }, 462 | "execution_count": null, 463 | "outputs": [] 464 | }, 465 | { 466 | "cell_type": "markdown", 467 | "source": [ 468 | "### Load Test Dataset" 469 | ], 470 | "metadata": { 471 | "id": "jKM5ipJXgDEZ" 472 | } 473 | }, 474 | { 475 | "cell_type": "code", 476 | "source": [ 477 | "testDataLoader = DataLoader(testDataset, batch_size=opt[\"batchSize\"], shuffle=False, drop_last=False)" 478 | ], 479 | "metadata": { 480 | "id": "qzOVvKahfPCx" 481 | }, 482 | "execution_count": null, 483 | "outputs": [] 484 | }, 485 | { 486 | "cell_type": "markdown", 487 | "source": [ 488 | "### Create Dictionary Instance for Saving Result" 489 | ], 490 | "metadata": { 491 | "id": "5hxQXQzpgEDq" 492 | } 493 | }, 494 | { 495 | "cell_type": "code", 496 | "source": [ 497 | "classDict = {0:[], 1:[], 2:[], 3:[], 4:[], 5:[], 6:[], 7:[], 8:[], 9:[]}" 498 | ], 499 | "metadata": { 500 | "id": "DnOuCtOEf5eN" 501 | }, 502 | "execution_count": null, 503 | "outputs": [] 504 | }, 505 | { 506 | "cell_type": "markdown", 507 | "source": [ 508 | "### Add Result" 509 | ], 510 | "metadata": { 511 | "id": "CfJybQL2gHUB" 512 | } 513 | }, 514 | { 515 | "cell_type": "code", 516 | "source": [ 517 | "for input, target in testDataLoader :\n", 518 | " if opt[\"isCUDA\"] :\n", 519 | " input = input.cuda()\n", 520 | " latentVector, output = model(input)\n", 521 | "\n", 522 | " for i, label in enumerate(target) :\n", 523 | " classDict[int(label)].append(latentVector[i].view(-1).detach().cpu().numpy())" 524 | ], 525 | "metadata": { 526 | "id": "Rg3_axMHfWcg" 527 | }, 528 | "execution_count": null, 529 | "outputs": [] 530 | }, 531 | { 532 | "cell_type": "code", 533 | "source": [ 534 | "classDict" 535 | ], 536 | "metadata": { 537 | "id": "EZzdrsPokeGN" 538 | }, 539 | "execution_count": null, 540 | "outputs": [] 541 | }, 542 | { 543 | "cell_type": "markdown", 544 | "source": [ 545 | "### Visualize Result" 546 | ], 547 | "metadata": { 548 | "id": "dtnr66aClFmj" 549 | } 550 | }, 551 | { 552 | "cell_type": "code", 553 | "source": [ 554 | "plt.figure(figsize=(10,10))\n", 555 | "\n", 556 | "for i in range(10) :\n", 557 | " xList, yList = [], []\n", 558 | " for subLatentVector in classDict[i] :\n", 559 | " xList.append(subLatentVector[0])\n", 560 | " yList.append(subLatentVector[1])\n", 561 | " plt.scatter(xList, yList, label=f\"class-{i}\", s=10)\n", 562 | "\n", 563 | "plt.legend(loc=\"best\")\n", 564 | "plt.show()" 565 | ], 566 | "metadata": { 567 | "id": "pduiqibYlIj2" 568 | }, 569 | "execution_count": null, 570 | "outputs": [] 571 | } 572 | ], 573 | "metadata": { 574 | "accelerator": "GPU", 575 | "colab": { 576 | "provenance": [], 577 | "authorship_tag": "ABX9TyMR7QdTd44+Mm66zhn6inoB", 578 | "include_colab_link": true 579 | }, 580 | "gpuClass": "standard", 581 | "kernelspec": { 582 | "display_name": "Python 3", 583 | "name": "python3" 584 | }, 585 | "language_info": { 586 | "name": "python" 587 | } 588 | }, 589 | "nbformat": 4, 590 | "nbformat_minor": 0 591 | } -------------------------------------------------------------------------------- /2023-2/열유체공학실험1_Week13_PyTorch_Denoising_Autoencoder.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "view-in-github", 7 | "colab_type": "text" 8 | }, 9 | "source": [ 10 | "\"Open" 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "metadata": { 16 | "id": "QQ97dC_jLHvF" 17 | }, 18 | "source": [ 19 | "# Denoising Autoencoder Using PyTorch Framework" 20 | ] 21 | }, 22 | { 23 | "cell_type": "markdown", 24 | "metadata": { 25 | "id": "BX6s-j7fq3hA" 26 | }, 27 | "source": [ 28 | "## Check NVIDIA GPU Setting" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": null, 34 | "metadata": { 35 | "id": "D_I7cz1Sq6S1" 36 | }, 37 | "outputs": [], 38 | "source": [ 39 | "!nvidia-smi" 40 | ] 41 | }, 42 | { 43 | "cell_type": "markdown", 44 | "source": [ 45 | "## Load MNIST Dataset" 46 | ], 47 | "metadata": { 48 | "id": "1sg71j-zWuau" 49 | } 50 | }, 51 | { 52 | "cell_type": "code", 53 | "source": [ 54 | "from torchvision.datasets import MNIST\n", 55 | "from torchvision import transforms" 56 | ], 57 | "metadata": { 58 | "id": "UvcYeIfuWwE9" 59 | }, 60 | "execution_count": null, 61 | "outputs": [] 62 | }, 63 | { 64 | "cell_type": "code", 65 | "source": [ 66 | "trainDataset = MNIST(root=\"content\",\n", 67 | " train=True,\n", 68 | " transform=transforms.Compose([transforms.Resize((32,32)), transforms.ToTensor()]),\n", 69 | " download=True)\n", 70 | "testDataset = MNIST(root=\"content\",\n", 71 | " train=False,\n", 72 | " transform=transforms.Compose([transforms.Resize((32,32)), transforms.ToTensor()]),\n", 73 | " download=True)" 74 | ], 75 | "metadata": { 76 | "id": "TRURIUnrW_1e" 77 | }, 78 | "execution_count": null, 79 | "outputs": [] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": null, 84 | "metadata": { 85 | "id": "Lcjb8ky2PvGo" 86 | }, 87 | "outputs": [], 88 | "source": [ 89 | "from torch import nn\n", 90 | "import torch.nn.functional as F" 91 | ] 92 | }, 93 | { 94 | "cell_type": "code", 95 | "execution_count": null, 96 | "metadata": { 97 | "id": "VpMGcu1KP07P" 98 | }, 99 | "outputs": [], 100 | "source": [ 101 | "class myModel(nn.Module) :\n", 102 | " def __init__(self, opt) :\n", 103 | " super(myModel, self).__init__()\n", 104 | "\n", 105 | " inputDim, channels = opt[\"inputDim\"], opt[\"channels\"]\n", 106 | "\n", 107 | " self.encoder = nn.Sequential(nn.Conv2d(inputDim, channels, kernel_size=3, stride=1, padding=1),\n", 108 | " nn.ReLU(),\n", 109 | " nn.MaxPool2d(kernel_size=2, stride=2),\n", 110 | " nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1),\n", 111 | " nn.ReLU(),\n", 112 | " nn.MaxPool2d(kernel_size=2, stride=2),\n", 113 | " nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1),\n", 114 | " nn.ReLU(),\n", 115 | " nn.MaxPool2d(kernel_size=2, stride=2),\n", 116 | " nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1),\n", 117 | " nn.ReLU())\n", 118 | " self.decoder = nn.Sequential(nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1),\n", 119 | " nn.ReLU(),\n", 120 | " nn.Upsample(scale_factor=2),\n", 121 | " nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1),\n", 122 | " nn.ReLU(),\n", 123 | " nn.Upsample(scale_factor=2),\n", 124 | " nn.Conv2d(channels, channels, kernel_size=3, stride=1, padding=1),\n", 125 | " nn.ReLU(),\n", 126 | " nn.Upsample(scale_factor=2),\n", 127 | " nn.Conv2d(channels, inputDim, kernel_size=3, stride=1, padding=1))\n", 128 | " self.bottleneck0 = nn.Linear(((opt[\"inputSize\"]//8)**2)*channels, 2)\n", 129 | " self.bottleneck1 = nn.Linear(2, ((opt[\"inputSize\"]//8)**2)*channels)\n", 130 | "\n", 131 | " def forward(self, input) :\n", 132 | " noisy = torch.clamp(input+torch.randn_like(input)*(50/255), 0, 1)\n", 133 | "\n", 134 | " output = self.encoder(noisy)\n", 135 | " n, c, h, w = output.size()\n", 136 | "\n", 137 | " latentVector = self.bottleneck0(output.view(-1, c*h*w))\n", 138 | "\n", 139 | " output = self.decoder(self.bottleneck1(latentVector).view(n, c, h, w))\n", 140 | "\n", 141 | " return latentVector, noisy, output" 142 | ] 143 | }, 144 | { 145 | "cell_type": "markdown", 146 | "metadata": { 147 | "id": "3Wk4_cBzQU19" 148 | }, 149 | "source": [ 150 | "## Train DL Model" 151 | ] 152 | }, 153 | { 154 | "cell_type": "code", 155 | "execution_count": null, 156 | "metadata": { 157 | "id": "0QJcgBk-QSsg" 158 | }, 159 | "outputs": [], 160 | "source": [ 161 | "import torch\n", 162 | "from torch.utils.data import DataLoader\n", 163 | "from torch import optim\n", 164 | "\n", 165 | "from torchsummary import summary\n", 166 | "\n", 167 | "from tqdm import tqdm" 168 | ] 169 | }, 170 | { 171 | "cell_type": "markdown", 172 | "metadata": { 173 | "id": "phpTOt47RjcB" 174 | }, 175 | "source": [ 176 | "### Fix Seed" 177 | ] 178 | }, 179 | { 180 | "cell_type": "code", 181 | "execution_count": null, 182 | "metadata": { 183 | "id": "81NJ5JxbRnaU" 184 | }, 185 | "outputs": [], 186 | "source": [ 187 | "import random\n", 188 | "import numpy as np" 189 | ] 190 | }, 191 | { 192 | "cell_type": "code", 193 | "execution_count": null, 194 | "metadata": { 195 | "id": "o8F_Z5d3RlH-" 196 | }, 197 | "outputs": [], 198 | "source": [ 199 | "def fixSeed(seed) :\n", 200 | " random.seed(seed)\n", 201 | " np.random.seed(seed)\n", 202 | " torch.manual_seed(seed)\n", 203 | " torch.cuda.manual_seed(seed)\n", 204 | " torch.cuda.manual_seed_all(seed)\n", 205 | " torch.backends.cudnn.deterministic = True\n", 206 | " torch.backends.cudnn.benchmark = False" 207 | ] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "source": [ 212 | "## Create Average Meter Instance" 213 | ], 214 | "metadata": { 215 | "id": "Jh4VCYXSswFY" 216 | } 217 | }, 218 | { 219 | "cell_type": "code", 220 | "source": [ 221 | "class AverageMeter(object):\n", 222 | " def __init__(self):\n", 223 | " self.reset()\n", 224 | "\n", 225 | " def reset(self):\n", 226 | " self.val = 0\n", 227 | " self.avg = 0\n", 228 | " self.sum = 0\n", 229 | " self.count = 0\n", 230 | "\n", 231 | " def update(self, val, n=1):\n", 232 | " self.val = val\n", 233 | " self.sum += val*n\n", 234 | " self.count += n\n", 235 | " self.avg = self.sum / self.count" 236 | ], 237 | "metadata": { 238 | "id": "oGUnMpzusxqw" 239 | }, 240 | "execution_count": null, 241 | "outputs": [] 242 | }, 243 | { 244 | "cell_type": "markdown", 245 | "source": [ 246 | "## Training Code as a Function (Abstraction)" 247 | ], 248 | "metadata": { 249 | "id": "lhJ28-fU6Zk7" 250 | } 251 | }, 252 | { 253 | "cell_type": "code", 254 | "source": [ 255 | "def train(opt, trainDataset, testDataset, myModel, criterion) :\n", 256 | " fixSeed(opt[\"seed\"])\n", 257 | "\n", 258 | " trainDataLoader = DataLoader(trainDataset, batch_size=opt[\"batchSize\"], shuffle=True, drop_last=True)\n", 259 | " testDataLoader = DataLoader(testDataset, batch_size=opt[\"batchSize\"], shuffle=False, drop_last=False)\n", 260 | "\n", 261 | " fixSeed(opt[\"seed\"])\n", 262 | " model = myModel(opt)\n", 263 | " if opt[\"isCUDA\"] :\n", 264 | " model = model.cuda()\n", 265 | "\n", 266 | " summary(model, (opt[\"inputDim\"], opt[\"inputSize\"], opt[\"inputSize\"]))\n", 267 | "\n", 268 | " optimizer = optim.Adam(model.parameters(), lr=opt[\"lr\"])\n", 269 | "\n", 270 | " trainLoss, testLoss = AverageMeter(), AverageMeter()\n", 271 | " trainLossList, testLossList = [], []\n", 272 | " bestLoss = torch.inf\n", 273 | "\n", 274 | " for epoch in range(1, opt[\"epochs\"]+1) :\n", 275 | " trainBar = tqdm(trainDataLoader)\n", 276 | " trainLoss.reset()\n", 277 | "\n", 278 | " for data in trainBar :\n", 279 | " input, target = data\n", 280 | " if opt[\"isCUDA\"] :\n", 281 | " input = input.cuda()\n", 282 | " optimizer.zero_grad()\n", 283 | " pred = model(input)\n", 284 | " loss = criterion(pred[-1], input)\n", 285 | " loss.backward()\n", 286 | " optimizer.step()\n", 287 | "\n", 288 | " trainLoss.update(loss.item(), opt[\"batchSize\"])\n", 289 | " trainBar.set_description(desc=f\"[{epoch}/{opt['epochs']}] [Train] < Loss:{trainLoss.avg:.6f} >\")\n", 290 | "\n", 291 | " trainLossList.append(trainLoss.avg)\n", 292 | "\n", 293 | " testBar = tqdm(testDataLoader)\n", 294 | " testLoss.reset()\n", 295 | "\n", 296 | " for data in testBar :\n", 297 | " input, target = data\n", 298 | " if opt[\"isCUDA\"] :\n", 299 | " input = input.cuda()\n", 300 | "\n", 301 | " model.eval()\n", 302 | " with torch.no_grad() :\n", 303 | " pred = model(input)\n", 304 | " loss = criterion(pred[-1], input)\n", 305 | "\n", 306 | " testLoss.update(loss.item(), opt[\"batchSize\"])\n", 307 | "\n", 308 | " testBar.set_description(desc=f\"[{epoch}/{opt['epochs']}] [Test] < Loss:{testLoss.avg:.6f} >\")\n", 309 | "\n", 310 | " testLossList.append(testLoss.avg)\n", 311 | "\n", 312 | " if testLoss.avg < bestLoss :\n", 313 | " bestLoss = testLoss.avg\n", 314 | " torch.save(model.state_dict(), \"bestModel.pth\")\n", 315 | "\n", 316 | " torch.save(model.state_dict(), \"latestModel.pth\")\n", 317 | "\n", 318 | " return (trainLossList, testLossList)" 319 | ], 320 | "metadata": { 321 | "id": "Toe0P0WH6c9p" 322 | }, 323 | "execution_count": null, 324 | "outputs": [] 325 | }, 326 | { 327 | "cell_type": "markdown", 328 | "source": [ 329 | "## Create Training Option (Hyperparameter) Dictionary" 330 | ], 331 | "metadata": { 332 | "id": "ttFH7GYJq6c3" 333 | } 334 | }, 335 | { 336 | "cell_type": "code", 337 | "source": [ 338 | "opt = {\"inputSize\":32,\n", 339 | " \"seed\":42,\n", 340 | " \"inputDim\":1,\n", 341 | " \"channels\":64,\n", 342 | " \"batchSize\":16,\n", 343 | " \"lr\":1e-4,\n", 344 | " \"epochs\":10,\n", 345 | " \"isCUDA\":torch.cuda.is_available()}" 346 | ], 347 | "metadata": { 348 | "id": "tXzQHG9aq9xB" 349 | }, 350 | "execution_count": null, 351 | "outputs": [] 352 | }, 353 | { 354 | "cell_type": "markdown", 355 | "source": [ 356 | "## Train Model" 357 | ], 358 | "metadata": { 359 | "id": "VpE3M2q67qRu" 360 | } 361 | }, 362 | { 363 | "cell_type": "code", 364 | "source": [ 365 | "lossList = train(opt, trainDataset, testDataset, myModel, nn.L1Loss())" 366 | ], 367 | "metadata": { 368 | "id": "yV7Jgs_77c57" 369 | }, 370 | "execution_count": null, 371 | "outputs": [] 372 | }, 373 | { 374 | "cell_type": "markdown", 375 | "metadata": { 376 | "id": "XrgUhJC67uzu" 377 | }, 378 | "source": [ 379 | "## Plot Training vs. Test Loss Graph" 380 | ] 381 | }, 382 | { 383 | "cell_type": "code", 384 | "source": [ 385 | "import matplotlib.pyplot as plt" 386 | ], 387 | "metadata": { 388 | "id": "_lIUGO11pAtu" 389 | }, 390 | "execution_count": null, 391 | "outputs": [] 392 | }, 393 | { 394 | "cell_type": "code", 395 | "execution_count": null, 396 | "metadata": { 397 | "id": "mLsjFs3K7uzv" 398 | }, 399 | "outputs": [], 400 | "source": [ 401 | "plt.figure(figsize=(20,10))\n", 402 | "\n", 403 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), lossList[0], label=\"Training Loss\")\n", 404 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), lossList[1], label=\"Test Loss\")\n", 405 | "\n", 406 | "plt.xlabel(\"Epoch\")\n", 407 | "plt.ylabel(\"L1 Loss\")\n", 408 | "plt.legend(loc=\"best\")\n", 409 | "\n", 410 | "plt.show()" 411 | ] 412 | }, 413 | { 414 | "cell_type": "markdown", 415 | "source": [ 416 | "## Extract Latent Vector" 417 | ], 418 | "metadata": { 419 | "id": "RP4jlNz2e42r" 420 | } 421 | }, 422 | { 423 | "cell_type": "markdown", 424 | "source": [ 425 | "### Load Trained Model" 426 | ], 427 | "metadata": { 428 | "id": "5D-ZF3NhfGkO" 429 | } 430 | }, 431 | { 432 | "cell_type": "code", 433 | "source": [ 434 | "weights = torch.load(\"/content/bestModel.pth\")\n", 435 | "\n", 436 | "model = myModel(opt)\n", 437 | "model.load_state_dict(weights)\n", 438 | "if opt[\"isCUDA\"] :\n", 439 | " model = model.cuda()" 440 | ], 441 | "metadata": { 442 | "id": "lxfxB9W1fIPE" 443 | }, 444 | "execution_count": null, 445 | "outputs": [] 446 | }, 447 | { 448 | "cell_type": "markdown", 449 | "source": [ 450 | "### Get Model Structure" 451 | ], 452 | "metadata": { 453 | "id": "8jjN8rSngATu" 454 | } 455 | }, 456 | { 457 | "cell_type": "code", 458 | "source": [ 459 | "print(model)" 460 | ], 461 | "metadata": { 462 | "id": "Z_vtvoYCfOgg" 463 | }, 464 | "execution_count": null, 465 | "outputs": [] 466 | }, 467 | { 468 | "cell_type": "markdown", 469 | "source": [ 470 | "### Load Test Dataset" 471 | ], 472 | "metadata": { 473 | "id": "jKM5ipJXgDEZ" 474 | } 475 | }, 476 | { 477 | "cell_type": "code", 478 | "source": [ 479 | "testDataLoader = DataLoader(testDataset, batch_size=opt[\"batchSize\"], shuffle=False, drop_last=False)" 480 | ], 481 | "metadata": { 482 | "id": "qzOVvKahfPCx" 483 | }, 484 | "execution_count": null, 485 | "outputs": [] 486 | }, 487 | { 488 | "cell_type": "markdown", 489 | "source": [ 490 | "### Create Dictionary Instance for Saving Result" 491 | ], 492 | "metadata": { 493 | "id": "5hxQXQzpgEDq" 494 | } 495 | }, 496 | { 497 | "cell_type": "code", 498 | "source": [ 499 | "classDict = {0:[], 1:[], 2:[], 3:[], 4:[], 5:[], 6:[], 7:[], 8:[], 9:[]}" 500 | ], 501 | "metadata": { 502 | "id": "DnOuCtOEf5eN" 503 | }, 504 | "execution_count": null, 505 | "outputs": [] 506 | }, 507 | { 508 | "cell_type": "markdown", 509 | "source": [ 510 | "### Add Result" 511 | ], 512 | "metadata": { 513 | "id": "CfJybQL2gHUB" 514 | } 515 | }, 516 | { 517 | "cell_type": "code", 518 | "source": [ 519 | "import cv2" 520 | ], 521 | "metadata": { 522 | "id": "fmP6OP91mjqk" 523 | }, 524 | "execution_count": null, 525 | "outputs": [] 526 | }, 527 | { 528 | "cell_type": "code", 529 | "source": [ 530 | "%mkdir \"output-samples\"" 531 | ], 532 | "metadata": { 533 | "id": "RX3wQ7Qnnucv" 534 | }, 535 | "execution_count": null, 536 | "outputs": [] 537 | }, 538 | { 539 | "cell_type": "code", 540 | "source": [ 541 | "numSample = 0\n", 542 | "\n", 543 | "for input, target in testDataLoader :\n", 544 | " if opt[\"isCUDA\"] :\n", 545 | " input = input.cuda()\n", 546 | " latentVector, noisy, output = model(input)\n", 547 | "\n", 548 | " for i, label in enumerate(target) :\n", 549 | " classDict[int(label)].append(latentVector[i].view(-1).detach().cpu().numpy())\n", 550 | "\n", 551 | " noisySample = noisy[i].squeeze(0).detach().cpu().numpy()\n", 552 | " outputSample = output[i].squeeze(0).detach().cpu().numpy()\n", 553 | " targetSample = input[i].squeeze(0).detach().cpu().numpy()\n", 554 | "\n", 555 | " noisySample = np.clip(noisySample*225, 0, 255)\n", 556 | " outputSample = np.clip(outputSample*225, 0, 255)\n", 557 | " targetSample = np.clip(targetSample*225, 0, 255)\n", 558 | "\n", 559 | " concat = np.hstack((noisySample, outputSample, targetSample))\n", 560 | "\n", 561 | " cv2.imwrite(f\"output-samples/sample-{numSample}.png\", concat)\n", 562 | " numSample += 1" 563 | ], 564 | "metadata": { 565 | "id": "Rg3_axMHfWcg" 566 | }, 567 | "execution_count": null, 568 | "outputs": [] 569 | }, 570 | { 571 | "cell_type": "code", 572 | "source": [ 573 | "classDict" 574 | ], 575 | "metadata": { 576 | "id": "IWeRK_fucS_r" 577 | }, 578 | "execution_count": null, 579 | "outputs": [] 580 | }, 581 | { 582 | "cell_type": "markdown", 583 | "source": [ 584 | "### Visualize Result" 585 | ], 586 | "metadata": { 587 | "id": "dtnr66aClFmj" 588 | } 589 | }, 590 | { 591 | "cell_type": "code", 592 | "source": [ 593 | "plt.figure(figsize=(10,10))\n", 594 | "\n", 595 | "for i in range(10) :\n", 596 | " xList, yList = [], []\n", 597 | " for subLatentVector in classDict[i] :\n", 598 | " xList.append(subLatentVector[0])\n", 599 | " yList.append(subLatentVector[1])\n", 600 | " plt.scatter(xList, yList, label=f\"class-{i}\", s=10)\n", 601 | "\n", 602 | "plt.legend(loc=\"best\")\n", 603 | "plt.show()" 604 | ], 605 | "metadata": { 606 | "id": "3YUP9bwZcbO8" 607 | }, 608 | "execution_count": null, 609 | "outputs": [] 610 | }, 611 | { 612 | "cell_type": "code", 613 | "source": [ 614 | "concat = []\n", 615 | "\n", 616 | "for i in range(10) :\n", 617 | " subConcat = []\n", 618 | " for j in range(10) :\n", 619 | " subConcat.append(cv2.imread(f\"/content/output-samples/sample-{i*10+j}.png\"))\n", 620 | " concat.append(np.vstack(subConcat))\n", 621 | "\n", 622 | "concat = np.hstack(concat)" 623 | ], 624 | "metadata": { 625 | "id": "-l4XfDj5rdC8" 626 | }, 627 | "execution_count": null, 628 | "outputs": [] 629 | }, 630 | { 631 | "cell_type": "code", 632 | "source": [ 633 | "plt.figure(figsize=(20,20))\n", 634 | "plt.imshow(concat)\n", 635 | "plt.show()" 636 | ], 637 | "metadata": { 638 | "id": "Kx2358DvmMUp" 639 | }, 640 | "execution_count": null, 641 | "outputs": [] 642 | } 643 | ], 644 | "metadata": { 645 | "accelerator": "GPU", 646 | "colab": { 647 | "provenance": [], 648 | "authorship_tag": "ABX9TyNKRuFD92UTinyck3yPDEEA", 649 | "include_colab_link": true 650 | }, 651 | "gpuClass": "standard", 652 | "kernelspec": { 653 | "display_name": "Python 3", 654 | "name": "python3" 655 | }, 656 | "language_info": { 657 | "name": "python" 658 | } 659 | }, 660 | "nbformat": 4, 661 | "nbformat_minor": 0 662 | } -------------------------------------------------------------------------------- /2022-2/열유체공학실험_Week_10.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [], 7 | "collapsed_sections": [], 8 | "include_colab_link": true 9 | }, 10 | "kernelspec": { 11 | "name": "python3", 12 | "display_name": "Python 3" 13 | }, 14 | "language_info": { 15 | "name": "python" 16 | } 17 | }, 18 | "cells": [ 19 | { 20 | "cell_type": "markdown", 21 | "metadata": { 22 | "id": "view-in-github", 23 | "colab_type": "text" 24 | }, 25 | "source": [ 26 | "\"Open" 27 | ] 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "metadata": { 32 | "id": "7gwxyxhlsk94" 33 | }, 34 | "source": [ 35 | "# Finite Difference Formulations" 36 | ] 37 | }, 38 | { 39 | "cell_type": "markdown", 40 | "metadata": { 41 | "id": "1xyiOPVGsoVG" 42 | }, 43 | "source": [ 44 | "## Import Library" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "metadata": { 50 | "id": "LtrLT9DxdfuS" 51 | }, 52 | "source": [ 53 | "import numpy as np\n", 54 | "import matplotlib.pyplot as plt" 55 | ], 56 | "execution_count": null, 57 | "outputs": [] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": { 62 | "id": "ZYT3Y9romF12" 63 | }, 64 | "source": [ 65 | "## First Order Derivative" 66 | ] 67 | }, 68 | { 69 | "cell_type": "markdown", 70 | "source": [ 71 | "### Create Function" 72 | ], 73 | "metadata": { 74 | "id": "udx47eo9ZMKR" 75 | } 76 | }, 77 | { 78 | "cell_type": "code", 79 | "metadata": { 80 | "id": "dYfFlrEsl7NL" 81 | }, 82 | "source": [ 83 | "def function(x) :\n", 84 | " return 3*np.power(x, 2) + 2*x + 1" 85 | ], 86 | "execution_count": null, 87 | "outputs": [] 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "source": [ 92 | "### Plot Graph" 93 | ], 94 | "metadata": { 95 | "id": "AV3J0Up9ZQQc" 96 | } 97 | }, 98 | { 99 | "cell_type": "code", 100 | "metadata": { 101 | "id": "Tj7x2WhnmfZb" 102 | }, 103 | "source": [ 104 | "x = np.arange(0, 10, 1e-3)\n", 105 | "y = function(x)" 106 | ], 107 | "execution_count": null, 108 | "outputs": [] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "metadata": { 113 | "id": "nt_LXUKhmk2K" 114 | }, 115 | "source": [ 116 | "plt.plot(x, y)\n", 117 | "plt.xlabel(\"x\")\n", 118 | "plt.ylabel(\"f(x)\")\n", 119 | "plt.title(\"Function Graph\")\n", 120 | "plt.show()" 121 | ], 122 | "execution_count": null, 123 | "outputs": [] 124 | }, 125 | { 126 | "cell_type": "markdown", 127 | "metadata": { 128 | "id": "PUl2Gf9omHpz" 129 | }, 130 | "source": [ 131 | "### First Forward Difference Approximation" 132 | ] 133 | }, 134 | { 135 | "cell_type": "markdown", 136 | "source": [ 137 | "#### Compute Gradient" 138 | ], 139 | "metadata": { 140 | "id": "m5mELZKwZS2R" 141 | } 142 | }, 143 | { 144 | "cell_type": "code", 145 | "metadata": { 146 | "id": "fKn59wZTnecN" 147 | }, 148 | "source": [ 149 | "def FFDA(fi, fiP1, delta) :\n", 150 | " gradient = (fiP1 - fi) / delta\n", 151 | " return gradient" 152 | ], 153 | "execution_count": null, 154 | "outputs": [] 155 | }, 156 | { 157 | "cell_type": "code", 158 | "metadata": { 159 | "id": "-g9ynhBFmnxk" 160 | }, 161 | "source": [ 162 | "xInput, delta = 5, 1e-3" 163 | ], 164 | "execution_count": null, 165 | "outputs": [] 166 | }, 167 | { 168 | "cell_type": "code", 169 | "metadata": { 170 | "id": "KaGzLIkRm51J" 171 | }, 172 | "source": [ 173 | "fi, fiP1 = function(xInput), function(xInput + delta)" 174 | ], 175 | "execution_count": null, 176 | "outputs": [] 177 | }, 178 | { 179 | "cell_type": "code", 180 | "metadata": { 181 | "id": "9zSrA6agnuzw" 182 | }, 183 | "source": [ 184 | "gradient = FFDA(fi, fiP1, delta)\n", 185 | "print(f\"Gradient : {gradient:.4f}\")" 186 | ], 187 | "execution_count": null, 188 | "outputs": [] 189 | }, 190 | { 191 | "cell_type": "markdown", 192 | "source": [ 193 | "#### Visualize Result" 194 | ], 195 | "metadata": { 196 | "id": "L5tiCFlIaAL0" 197 | } 198 | }, 199 | { 200 | "cell_type": "code", 201 | "metadata": { 202 | "id": "_Vz7Bs_yow-g" 203 | }, 204 | "source": [ 205 | "def linear(x, gradient, intercept) :\n", 206 | " return gradient*x + intercept" 207 | ], 208 | "execution_count": null, 209 | "outputs": [] 210 | }, 211 | { 212 | "cell_type": "code", 213 | "metadata": { 214 | "id": "IL5zmtpvon89" 215 | }, 216 | "source": [ 217 | "intercept = fi - gradient * xInput" 218 | ], 219 | "execution_count": null, 220 | "outputs": [] 221 | }, 222 | { 223 | "cell_type": "code", 224 | "metadata": { 225 | "id": "Pj-09Sy5o2Mj" 226 | }, 227 | "source": [ 228 | "difference = linear(x, gradient, intercept)" 229 | ], 230 | "execution_count": null, 231 | "outputs": [] 232 | }, 233 | { 234 | "cell_type": "code", 235 | "metadata": { 236 | "id": "mICoEKzTo7z4" 237 | }, 238 | "source": [ 239 | "plt.plot(x, y, label = \"original\")\n", 240 | "plt.plot(x, difference, label = \"gradient\")\n", 241 | "plt.scatter(xInput, function(xInput), label = \"point\")\n", 242 | "plt.xlabel(\"x\")\n", 243 | "plt.ylabel(\"f(x)\")\n", 244 | "plt.title(\"Function Graph\")\n", 245 | "plt.legend(loc = \"best\")\n", 246 | "plt.show()" 247 | ], 248 | "execution_count": null, 249 | "outputs": [] 250 | }, 251 | { 252 | "cell_type": "markdown", 253 | "metadata": { 254 | "id": "TMksCMLQpNl3" 255 | }, 256 | "source": [ 257 | "### First Backward Difference Approximation" 258 | ] 259 | }, 260 | { 261 | "cell_type": "markdown", 262 | "source": [ 263 | "#### Compute Gradient" 264 | ], 265 | "metadata": { 266 | "id": "tuR4-rYUcnIw" 267 | } 268 | }, 269 | { 270 | "cell_type": "code", 271 | "metadata": { 272 | "id": "g_WGBXNUpbKJ" 273 | }, 274 | "source": [ 275 | "def FBDA(fi, fiM1, delta) :\n", 276 | " gradient = (fi - fiM1) / delta\n", 277 | " return gradient" 278 | ], 279 | "execution_count": null, 280 | "outputs": [] 281 | }, 282 | { 283 | "cell_type": "code", 284 | "metadata": { 285 | "id": "9FfeWHC4p8SF" 286 | }, 287 | "source": [ 288 | "xInput, delta = 5, 1e-3" 289 | ], 290 | "execution_count": null, 291 | "outputs": [] 292 | }, 293 | { 294 | "cell_type": "code", 295 | "metadata": { 296 | "id": "vFu9tWUlpP46" 297 | }, 298 | "source": [ 299 | "fi, fiM1 = function(xInput), function(xInput - delta)" 300 | ], 301 | "execution_count": null, 302 | "outputs": [] 303 | }, 304 | { 305 | "cell_type": "code", 306 | "metadata": { 307 | "id": "o2Ct60Y1pkAF" 308 | }, 309 | "source": [ 310 | "gradient = FBDA(fi, fiM1, delta)\n", 311 | "print(f\"Gradient : {gradient:.4f}\")" 312 | ], 313 | "execution_count": null, 314 | "outputs": [] 315 | }, 316 | { 317 | "cell_type": "markdown", 318 | "source": [ 319 | "#### Visualize Result" 320 | ], 321 | "metadata": { 322 | "id": "zjKH5u8LcyMF" 323 | } 324 | }, 325 | { 326 | "cell_type": "code", 327 | "metadata": { 328 | "id": "wQH-h32FpuUo" 329 | }, 330 | "source": [ 331 | "intercept = fi - gradient * xInput" 332 | ], 333 | "execution_count": null, 334 | "outputs": [] 335 | }, 336 | { 337 | "cell_type": "code", 338 | "metadata": { 339 | "id": "9wNFDna2pwDj" 340 | }, 341 | "source": [ 342 | "difference = linear(x, gradient, intercept)" 343 | ], 344 | "execution_count": null, 345 | "outputs": [] 346 | }, 347 | { 348 | "cell_type": "code", 349 | "metadata": { 350 | "id": "kk2JkGOhpwiV" 351 | }, 352 | "source": [ 353 | "plt.plot(x, y, label = \"original\")\n", 354 | "plt.plot(x, difference, label = \"gradient\")\n", 355 | "plt.scatter(xInput, function(xInput), label = \"point\")\n", 356 | "plt.xlabel(\"x\")\n", 357 | "plt.ylabel(\"f(x)\")\n", 358 | "plt.title(\"Function Graph\")\n", 359 | "plt.legend(loc = \"best\")\n", 360 | "plt.show()" 361 | ], 362 | "execution_count": null, 363 | "outputs": [] 364 | }, 365 | { 366 | "cell_type": "markdown", 367 | "metadata": { 368 | "id": "YUQyGF3kqX5N" 369 | }, 370 | "source": [ 371 | "### Central Difference Approximation" 372 | ] 373 | }, 374 | { 375 | "cell_type": "markdown", 376 | "source": [ 377 | "#### Compute Gradient" 378 | ], 379 | "metadata": { 380 | "id": "1ewPaxTlc_vW" 381 | } 382 | }, 383 | { 384 | "cell_type": "code", 385 | "metadata": { 386 | "id": "ag8ofPr7qc2w" 387 | }, 388 | "source": [ 389 | "def CDA(fiP1, fiM1, delta) :\n", 390 | " gradient = (fiP1 - fiM1) / (2*delta)\n", 391 | " return gradient " 392 | ], 393 | "execution_count": null, 394 | "outputs": [] 395 | }, 396 | { 397 | "cell_type": "code", 398 | "metadata": { 399 | "id": "JSuQ1TilqU-H" 400 | }, 401 | "source": [ 402 | "xInput, delta = 5, 1e-3" 403 | ], 404 | "execution_count": null, 405 | "outputs": [] 406 | }, 407 | { 408 | "cell_type": "code", 409 | "metadata": { 410 | "id": "Z-VpWHwHqZyk" 411 | }, 412 | "source": [ 413 | "fiP1, fiM1 = function(xInput + delta), function(xInput - delta)" 414 | ], 415 | "execution_count": null, 416 | "outputs": [] 417 | }, 418 | { 419 | "cell_type": "code", 420 | "metadata": { 421 | "id": "GzT3yFBSqlA1" 422 | }, 423 | "source": [ 424 | "gradient = CDA(fiP1, fiM1, delta)\n", 425 | "print(f\"Gradient : {gradient:.4f}\")" 426 | ], 427 | "execution_count": null, 428 | "outputs": [] 429 | }, 430 | { 431 | "cell_type": "markdown", 432 | "source": [ 433 | "#### Visualize Result" 434 | ], 435 | "metadata": { 436 | "id": "GEOZKZU1decp" 437 | } 438 | }, 439 | { 440 | "cell_type": "code", 441 | "metadata": { 442 | "id": "-H_VL9aXqtg8" 443 | }, 444 | "source": [ 445 | "intercept = fi - gradient * xInput" 446 | ], 447 | "execution_count": null, 448 | "outputs": [] 449 | }, 450 | { 451 | "cell_type": "code", 452 | "metadata": { 453 | "id": "UV8gSM9Xqvbd" 454 | }, 455 | "source": [ 456 | "difference = linear(x, gradient, intercept)" 457 | ], 458 | "execution_count": null, 459 | "outputs": [] 460 | }, 461 | { 462 | "cell_type": "code", 463 | "metadata": { 464 | "id": "f9BdgIv1qwmX" 465 | }, 466 | "source": [ 467 | "plt.plot(x, y, label = \"original\")\n", 468 | "plt.plot(x, difference, label = \"gradient\")\n", 469 | "plt.scatter(xInput, function(xInput), label = \"point\")\n", 470 | "plt.xlabel(\"x\")\n", 471 | "plt.ylabel(\"f(x)\")\n", 472 | "plt.title(\"Function Graph\")\n", 473 | "plt.legend(loc = \"best\")\n", 474 | "plt.show()" 475 | ], 476 | "execution_count": null, 477 | "outputs": [] 478 | }, 479 | { 480 | "cell_type": "markdown", 481 | "metadata": { 482 | "id": "MR_4Ni7vrAbY" 483 | }, 484 | "source": [ 485 | "### Example" 486 | ] 487 | }, 488 | { 489 | "cell_type": "markdown", 490 | "source": [ 491 | "#### Harmonic Function" 492 | ], 493 | "metadata": { 494 | "id": "NQsGcroJdz1H" 495 | } 496 | }, 497 | { 498 | "cell_type": "markdown", 499 | "source": [ 500 | "##### Plot Graph" 501 | ], 502 | "metadata": { 503 | "id": "JBDD-5VXd3R0" 504 | } 505 | }, 506 | { 507 | "cell_type": "code", 508 | "metadata": { 509 | "id": "V0oPQooKq8qH" 510 | }, 511 | "source": [ 512 | "delta = 1e-2\n", 513 | "x = np.arange(0, 10, delta)\n", 514 | "y = np.sin(x)" 515 | ], 516 | "execution_count": null, 517 | "outputs": [] 518 | }, 519 | { 520 | "cell_type": "code", 521 | "metadata": { 522 | "id": "sYAQGiZXrNs9" 523 | }, 524 | "source": [ 525 | "plt.plot(x, y)\n", 526 | "plt.xlabel(\"x\")\n", 527 | "plt.ylabel(\"f(x)\")\n", 528 | "plt.title(\"Function Graph\")\n", 529 | "plt.show()" 530 | ], 531 | "execution_count": null, 532 | "outputs": [] 533 | }, 534 | { 535 | "cell_type": "markdown", 536 | "source": [ 537 | "##### Compute Gradient" 538 | ], 539 | "metadata": { 540 | "id": "NESuiBWKd9yV" 541 | } 542 | }, 543 | { 544 | "cell_type": "code", 545 | "metadata": { 546 | "id": "lj-qw8-krsV_" 547 | }, 548 | "source": [ 549 | "xP1, xM1 = x + delta, x - delta\n", 550 | "yP1, yM1 = np.sin(xP1), np.sin(xM1)" 551 | ], 552 | "execution_count": null, 553 | "outputs": [] 554 | }, 555 | { 556 | "cell_type": "code", 557 | "metadata": { 558 | "id": "NyFU2G1MrP2T" 559 | }, 560 | "source": [ 561 | "gradientFFDA = FFDA(y, yP1, delta)\n", 562 | "gradientFBDA = FBDA(y, yM1, delta)\n", 563 | "gradientCDA = CDA(yP1, yM1, delta)" 564 | ], 565 | "execution_count": null, 566 | "outputs": [] 567 | }, 568 | { 569 | "cell_type": "markdown", 570 | "source": [ 571 | "##### Visualize Result" 572 | ], 573 | "metadata": { 574 | "id": "WBHZWruiemX8" 575 | } 576 | }, 577 | { 578 | "cell_type": "code", 579 | "metadata": { 580 | "id": "_xHv9GB_sF22" 581 | }, 582 | "source": [ 583 | "plt.plot(x[:25], gradientFFDA[:25], label = \"FFDA\")\n", 584 | "plt.plot(x[:25], gradientFBDA[:25], label = \"FBDA\")\n", 585 | "plt.plot(x[:25], gradientCDA[:25], label = \"CDA\")\n", 586 | "plt.xlabel(\"x\")\n", 587 | "plt.ylabel(\"f'(x)\")\n", 588 | "plt.title(\"Derivative Graph\")\n", 589 | "plt.legend(loc = \"best\")\n", 590 | "plt.show()" 591 | ], 592 | "execution_count": null, 593 | "outputs": [] 594 | }, 595 | { 596 | "cell_type": "markdown", 597 | "source": [ 598 | "##### Compute RMSE Loss" 599 | ], 600 | "metadata": { 601 | "id": "WBDk8f-9eqyA" 602 | } 603 | }, 604 | { 605 | "cell_type": "code", 606 | "source": [ 607 | "def RMSE(yHat, y) :\n", 608 | " loss = np.sqrt(np.power((yHat- y), 2).mean())\n", 609 | "\n", 610 | " return loss" 611 | ], 612 | "metadata": { 613 | "id": "PlqCNmRges6j" 614 | }, 615 | "execution_count": null, 616 | "outputs": [] 617 | }, 618 | { 619 | "cell_type": "code", 620 | "metadata": { 621 | "id": "zp3ovcWzv6gI" 622 | }, 623 | "source": [ 624 | "rmseFFDA = RMSE(gradientFFDA, np.cos(x))\n", 625 | "rmseFBDA = RMSE(gradientFBDA, np.cos(x))\n", 626 | "rmseCDA = RMSE(gradientCDA, np.cos(x))" 627 | ], 628 | "execution_count": null, 629 | "outputs": [] 630 | }, 631 | { 632 | "cell_type": "code", 633 | "metadata": { 634 | "id": "fFzwhLslwbta" 635 | }, 636 | "source": [ 637 | "print(f\"FFDA RMSE : {rmseFFDA:.8f}\")\n", 638 | "print(f\"RBDA RMSE : {rmseFBDA:.8f}\")\n", 639 | "print(f\"CDA RMSE : {rmseCDA:.8f}\")" 640 | ], 641 | "execution_count": null, 642 | "outputs": [] 643 | }, 644 | { 645 | "cell_type": "markdown", 646 | "metadata": { 647 | "id": "Z_BUIZxguOkB" 648 | }, 649 | "source": [ 650 | "## Second Derivation with Forward Difference Approximation" 651 | ] 652 | }, 653 | { 654 | "cell_type": "markdown", 655 | "source": [ 656 | "### Create Function" 657 | ], 658 | "metadata": { 659 | "id": "MWWQWbo-gDFX" 660 | } 661 | }, 662 | { 663 | "cell_type": "code", 664 | "metadata": { 665 | "id": "RGCism-_gDFY" 666 | }, 667 | "source": [ 668 | "def function(x) :\n", 669 | " return 3*np.power(x, 2) + 2*x + 1" 670 | ], 671 | "execution_count": null, 672 | "outputs": [] 673 | }, 674 | { 675 | "cell_type": "markdown", 676 | "source": [ 677 | "### Plot Graph" 678 | ], 679 | "metadata": { 680 | "id": "aAYk-8grgDFY" 681 | } 682 | }, 683 | { 684 | "cell_type": "code", 685 | "metadata": { 686 | "id": "wl4ceUa-gDFY" 687 | }, 688 | "source": [ 689 | "x = np.arange(0, 10, 1e-3)\n", 690 | "y = function(x)" 691 | ], 692 | "execution_count": null, 693 | "outputs": [] 694 | }, 695 | { 696 | "cell_type": "code", 697 | "metadata": { 698 | "id": "jYdNgX4SgDFY" 699 | }, 700 | "source": [ 701 | "plt.plot(x, y)\n", 702 | "plt.xlabel(\"x\")\n", 703 | "plt.ylabel(\"f(x)\")\n", 704 | "plt.title(\"Function Graph\")\n", 705 | "plt.show()" 706 | ], 707 | "execution_count": null, 708 | "outputs": [] 709 | }, 710 | { 711 | "cell_type": "markdown", 712 | "source": [ 713 | "### Second Forward Difference Approximation" 714 | ], 715 | "metadata": { 716 | "id": "In0ziN7RgsFx" 717 | } 718 | }, 719 | { 720 | "cell_type": "markdown", 721 | "source": [ 722 | "#### Compute Gradient" 723 | ], 724 | "metadata": { 725 | "id": "DOble9bPgE7F" 726 | } 727 | }, 728 | { 729 | "cell_type": "code", 730 | "metadata": { 731 | "id": "QcHbYkedupMB" 732 | }, 733 | "source": [ 734 | "def SDFDA(fiP2, fiP1, fi, delta) :\n", 735 | " gradient = (fiP2 - 2*fiP1 + fi) / np.power(delta, 2)\n", 736 | " return gradient" 737 | ], 738 | "execution_count": null, 739 | "outputs": [] 740 | }, 741 | { 742 | "cell_type": "code", 743 | "source": [ 744 | "xInput, delta = 5, 1e-3" 745 | ], 746 | "metadata": { 747 | "id": "lkrme9OkgMxi" 748 | }, 749 | "execution_count": null, 750 | "outputs": [] 751 | }, 752 | { 753 | "cell_type": "code", 754 | "metadata": { 755 | "id": "MuFLRUhiuiIN" 756 | }, 757 | "source": [ 758 | "fiP2, fiP1, fi = function(xInput + 2 * delta), function(xInput + delta), function(xInput)" 759 | ], 760 | "execution_count": null, 761 | "outputs": [] 762 | }, 763 | { 764 | "cell_type": "code", 765 | "metadata": { 766 | "id": "p3SG-oHHu3r5" 767 | }, 768 | "source": [ 769 | "gradient = SDFDA(fiP2, fiP1, fi, delta)\n", 770 | "print(f\"Gradient : {gradient:.8f}\")" 771 | ], 772 | "execution_count": null, 773 | "outputs": [] 774 | }, 775 | { 776 | "cell_type": "markdown", 777 | "source": [ 778 | "### Example" 779 | ], 780 | "metadata": { 781 | "id": "u0ar8MCVhHEy" 782 | } 783 | }, 784 | { 785 | "cell_type": "markdown", 786 | "metadata": { 787 | "id": "DGoH15kKvRTt" 788 | }, 789 | "source": [ 790 | "#### Harmonic Function" 791 | ] 792 | }, 793 | { 794 | "cell_type": "markdown", 795 | "source": [ 796 | "##### Plot Graph" 797 | ], 798 | "metadata": { 799 | "id": "1F81U7CLhLdb" 800 | } 801 | }, 802 | { 803 | "cell_type": "code", 804 | "metadata": { 805 | "id": "LDpuXv_SvS2g" 806 | }, 807 | "source": [ 808 | "delta = 1e-2\n", 809 | "x = np.arange(0, 10, delta)\n", 810 | "y = np.sin(x)" 811 | ], 812 | "execution_count": null, 813 | "outputs": [] 814 | }, 815 | { 816 | "cell_type": "code", 817 | "metadata": { 818 | "id": "qmbJaHL2vVSb" 819 | }, 820 | "source": [ 821 | "plt.plot(x, y)\n", 822 | "plt.xlabel(\"x\")\n", 823 | "plt.ylabel(\"f(x)\")\n", 824 | "plt.title(\"Function Graph\")\n", 825 | "plt.show()" 826 | ], 827 | "execution_count": null, 828 | "outputs": [] 829 | }, 830 | { 831 | "cell_type": "markdown", 832 | "source": [ 833 | "##### Compute Gradient" 834 | ], 835 | "metadata": { 836 | "id": "sV11jr34hPq7" 837 | } 838 | }, 839 | { 840 | "cell_type": "code", 841 | "metadata": { 842 | "id": "uo-pe1mrvXYr" 843 | }, 844 | "source": [ 845 | "xP2, xP1 = x + 2*delta, x + delta\n", 846 | "yP2, yP1 = np.sin(xP2), np.sin(xP1)" 847 | ], 848 | "execution_count": null, 849 | "outputs": [] 850 | }, 851 | { 852 | "cell_type": "code", 853 | "metadata": { 854 | "id": "0pDFuwd6vkeh" 855 | }, 856 | "source": [ 857 | "gradient = SDFDA(yP2, yP1, y, delta)" 858 | ], 859 | "execution_count": null, 860 | "outputs": [] 861 | }, 862 | { 863 | "cell_type": "markdown", 864 | "source": [ 865 | "##### Visualize Result" 866 | ], 867 | "metadata": { 868 | "id": "0aYs9yeAhb41" 869 | } 870 | }, 871 | { 872 | "cell_type": "code", 873 | "metadata": { 874 | "id": "-Y6LMgpSvsD5" 875 | }, 876 | "source": [ 877 | "plt.plot(x, gradient, label = \"SDFDA\")\n", 878 | "plt.xlabel(\"x\")\n", 879 | "plt.ylabel(\"f'(x)\")\n", 880 | "plt.title(\"Derivative Graph\")\n", 881 | "plt.legend(loc = \"best\")\n", 882 | "plt.show()" 883 | ], 884 | "execution_count": null, 885 | "outputs": [] 886 | }, 887 | { 888 | "cell_type": "markdown", 889 | "source": [ 890 | "##### Compute RMSE Loss" 891 | ], 892 | "metadata": { 893 | "id": "rTEce-uehkq5" 894 | } 895 | }, 896 | { 897 | "cell_type": "code", 898 | "metadata": { 899 | "id": "IDmPErUgwlq7" 900 | }, 901 | "source": [ 902 | "rmse = RMSE(gradient, -np.sin(x))" 903 | ], 904 | "execution_count": null, 905 | "outputs": [] 906 | }, 907 | { 908 | "cell_type": "code", 909 | "source": [ 910 | "print(f\"RMSE : {rmse:.8f}\")" 911 | ], 912 | "metadata": { 913 | "id": "uqeGOER4hrHD" 914 | }, 915 | "execution_count": null, 916 | "outputs": [] 917 | } 918 | ] 919 | } -------------------------------------------------------------------------------- /2023-2/열유체공학실험1_Week8_유한차분법.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [], 7 | "include_colab_link": true 8 | }, 9 | "kernelspec": { 10 | "name": "python3", 11 | "display_name": "Python 3" 12 | }, 13 | "language_info": { 14 | "name": "python" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "markdown", 20 | "metadata": { 21 | "id": "view-in-github", 22 | "colab_type": "text" 23 | }, 24 | "source": [ 25 | "\"Open" 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "metadata": { 31 | "id": "7gwxyxhlsk94" 32 | }, 33 | "source": [ 34 | "# Finite Difference Formulations" 35 | ] 36 | }, 37 | { 38 | "cell_type": "markdown", 39 | "metadata": { 40 | "id": "1xyiOPVGsoVG" 41 | }, 42 | "source": [ 43 | "## Import Library" 44 | ] 45 | }, 46 | { 47 | "cell_type": "code", 48 | "metadata": { 49 | "id": "LtrLT9DxdfuS" 50 | }, 51 | "source": [ 52 | "import numpy as np\n", 53 | "import matplotlib.pyplot as plt" 54 | ], 55 | "execution_count": null, 56 | "outputs": [] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": { 61 | "id": "ZYT3Y9romF12" 62 | }, 63 | "source": [ 64 | "## First Order Derivative" 65 | ] 66 | }, 67 | { 68 | "cell_type": "markdown", 69 | "source": [ 70 | "### Create Function" 71 | ], 72 | "metadata": { 73 | "id": "udx47eo9ZMKR" 74 | } 75 | }, 76 | { 77 | "cell_type": "code", 78 | "metadata": { 79 | "id": "dYfFlrEsl7NL" 80 | }, 81 | "source": [ 82 | "def function(x) :\n", 83 | " return 3*np.power(x, 2) + 2*x + 1" 84 | ], 85 | "execution_count": null, 86 | "outputs": [] 87 | }, 88 | { 89 | "cell_type": "markdown", 90 | "source": [ 91 | "### Plot Graph" 92 | ], 93 | "metadata": { 94 | "id": "AV3J0Up9ZQQc" 95 | } 96 | }, 97 | { 98 | "cell_type": "code", 99 | "metadata": { 100 | "id": "Tj7x2WhnmfZb" 101 | }, 102 | "source": [ 103 | "x = np.arange(0, 10, 1e-3)\n", 104 | "y = function(x)" 105 | ], 106 | "execution_count": null, 107 | "outputs": [] 108 | }, 109 | { 110 | "cell_type": "code", 111 | "metadata": { 112 | "id": "nt_LXUKhmk2K" 113 | }, 114 | "source": [ 115 | "plt.plot(x, y)\n", 116 | "plt.xlabel(\"x\")\n", 117 | "plt.ylabel(\"f(x)\")\n", 118 | "plt.title(\"Function Graph\")\n", 119 | "plt.show()" 120 | ], 121 | "execution_count": null, 122 | "outputs": [] 123 | }, 124 | { 125 | "cell_type": "markdown", 126 | "metadata": { 127 | "id": "PUl2Gf9omHpz" 128 | }, 129 | "source": [ 130 | "### First Forward Difference Approximation" 131 | ] 132 | }, 133 | { 134 | "cell_type": "markdown", 135 | "source": [ 136 | "#### Compute Gradient" 137 | ], 138 | "metadata": { 139 | "id": "m5mELZKwZS2R" 140 | } 141 | }, 142 | { 143 | "cell_type": "code", 144 | "metadata": { 145 | "id": "fKn59wZTnecN" 146 | }, 147 | "source": [ 148 | "def FFDA(fi, fiP1, delta) :\n", 149 | " gradient = (fiP1 - fi) / delta\n", 150 | " return gradient" 151 | ], 152 | "execution_count": null, 153 | "outputs": [] 154 | }, 155 | { 156 | "cell_type": "code", 157 | "metadata": { 158 | "id": "-g9ynhBFmnxk" 159 | }, 160 | "source": [ 161 | "xInput, delta = 5, 1e-3" 162 | ], 163 | "execution_count": null, 164 | "outputs": [] 165 | }, 166 | { 167 | "cell_type": "code", 168 | "metadata": { 169 | "id": "KaGzLIkRm51J" 170 | }, 171 | "source": [ 172 | "fi, fiP1 = function(xInput), function(xInput + delta)" 173 | ], 174 | "execution_count": null, 175 | "outputs": [] 176 | }, 177 | { 178 | "cell_type": "code", 179 | "metadata": { 180 | "id": "9zSrA6agnuzw" 181 | }, 182 | "source": [ 183 | "gradient = FFDA(fi, fiP1, delta)\n", 184 | "print(f\"Gradient : {gradient:.4f}\")" 185 | ], 186 | "execution_count": null, 187 | "outputs": [] 188 | }, 189 | { 190 | "cell_type": "markdown", 191 | "source": [ 192 | "#### Visualize Result" 193 | ], 194 | "metadata": { 195 | "id": "L5tiCFlIaAL0" 196 | } 197 | }, 198 | { 199 | "cell_type": "code", 200 | "metadata": { 201 | "id": "_Vz7Bs_yow-g" 202 | }, 203 | "source": [ 204 | "def linear(x, gradient, intercept) :\n", 205 | " return gradient*x + intercept" 206 | ], 207 | "execution_count": null, 208 | "outputs": [] 209 | }, 210 | { 211 | "cell_type": "code", 212 | "metadata": { 213 | "id": "IL5zmtpvon89" 214 | }, 215 | "source": [ 216 | "intercept = fi - gradient * xInput" 217 | ], 218 | "execution_count": null, 219 | "outputs": [] 220 | }, 221 | { 222 | "cell_type": "code", 223 | "metadata": { 224 | "id": "Pj-09Sy5o2Mj" 225 | }, 226 | "source": [ 227 | "difference = linear(x, gradient, intercept)" 228 | ], 229 | "execution_count": null, 230 | "outputs": [] 231 | }, 232 | { 233 | "cell_type": "code", 234 | "metadata": { 235 | "id": "mICoEKzTo7z4" 236 | }, 237 | "source": [ 238 | "plt.plot(x, y, label = \"original\")\n", 239 | "plt.plot(x, difference, label = \"gradient\")\n", 240 | "plt.scatter(xInput, function(xInput), label = \"point\")\n", 241 | "plt.xlabel(\"x\")\n", 242 | "plt.ylabel(\"f(x)\")\n", 243 | "plt.title(\"Function Graph\")\n", 244 | "plt.legend(loc = \"best\")\n", 245 | "plt.show()" 246 | ], 247 | "execution_count": null, 248 | "outputs": [] 249 | }, 250 | { 251 | "cell_type": "markdown", 252 | "metadata": { 253 | "id": "TMksCMLQpNl3" 254 | }, 255 | "source": [ 256 | "### First Backward Difference Approximation" 257 | ] 258 | }, 259 | { 260 | "cell_type": "markdown", 261 | "source": [ 262 | "#### Compute Gradient" 263 | ], 264 | "metadata": { 265 | "id": "tuR4-rYUcnIw" 266 | } 267 | }, 268 | { 269 | "cell_type": "code", 270 | "metadata": { 271 | "id": "g_WGBXNUpbKJ" 272 | }, 273 | "source": [ 274 | "def FBDA(fi, fiM1, delta) :\n", 275 | " gradient = (fi - fiM1) / delta\n", 276 | " return gradient" 277 | ], 278 | "execution_count": null, 279 | "outputs": [] 280 | }, 281 | { 282 | "cell_type": "code", 283 | "metadata": { 284 | "id": "9FfeWHC4p8SF" 285 | }, 286 | "source": [ 287 | "xInput, delta = 5, 1e-3" 288 | ], 289 | "execution_count": null, 290 | "outputs": [] 291 | }, 292 | { 293 | "cell_type": "code", 294 | "metadata": { 295 | "id": "vFu9tWUlpP46" 296 | }, 297 | "source": [ 298 | "fi, fiM1 = function(xInput), function(xInput - delta)" 299 | ], 300 | "execution_count": null, 301 | "outputs": [] 302 | }, 303 | { 304 | "cell_type": "code", 305 | "metadata": { 306 | "id": "o2Ct60Y1pkAF" 307 | }, 308 | "source": [ 309 | "gradient = FBDA(fi, fiM1, delta)\n", 310 | "print(f\"Gradient : {gradient:.4f}\")" 311 | ], 312 | "execution_count": null, 313 | "outputs": [] 314 | }, 315 | { 316 | "cell_type": "markdown", 317 | "source": [ 318 | "#### Visualize Result" 319 | ], 320 | "metadata": { 321 | "id": "zjKH5u8LcyMF" 322 | } 323 | }, 324 | { 325 | "cell_type": "code", 326 | "metadata": { 327 | "id": "wQH-h32FpuUo" 328 | }, 329 | "source": [ 330 | "intercept = fi - gradient * xInput" 331 | ], 332 | "execution_count": null, 333 | "outputs": [] 334 | }, 335 | { 336 | "cell_type": "code", 337 | "metadata": { 338 | "id": "9wNFDna2pwDj" 339 | }, 340 | "source": [ 341 | "difference = linear(x, gradient, intercept)" 342 | ], 343 | "execution_count": null, 344 | "outputs": [] 345 | }, 346 | { 347 | "cell_type": "code", 348 | "metadata": { 349 | "id": "kk2JkGOhpwiV" 350 | }, 351 | "source": [ 352 | "plt.plot(x, y, label = \"original\")\n", 353 | "plt.plot(x, difference, label = \"gradient\")\n", 354 | "plt.scatter(xInput, function(xInput), label = \"point\")\n", 355 | "plt.xlabel(\"x\")\n", 356 | "plt.ylabel(\"f(x)\")\n", 357 | "plt.title(\"Function Graph\")\n", 358 | "plt.legend(loc = \"best\")\n", 359 | "plt.show()" 360 | ], 361 | "execution_count": null, 362 | "outputs": [] 363 | }, 364 | { 365 | "cell_type": "markdown", 366 | "metadata": { 367 | "id": "YUQyGF3kqX5N" 368 | }, 369 | "source": [ 370 | "### Central Difference Approximation" 371 | ] 372 | }, 373 | { 374 | "cell_type": "markdown", 375 | "source": [ 376 | "#### Compute Gradient" 377 | ], 378 | "metadata": { 379 | "id": "1ewPaxTlc_vW" 380 | } 381 | }, 382 | { 383 | "cell_type": "code", 384 | "metadata": { 385 | "id": "ag8ofPr7qc2w" 386 | }, 387 | "source": [ 388 | "def CDA(fiP1, fiM1, delta) :\n", 389 | " gradient = (fiP1 - fiM1) / (2*delta)\n", 390 | " return gradient" 391 | ], 392 | "execution_count": null, 393 | "outputs": [] 394 | }, 395 | { 396 | "cell_type": "code", 397 | "metadata": { 398 | "id": "JSuQ1TilqU-H" 399 | }, 400 | "source": [ 401 | "xInput, delta = 5, 1e-3" 402 | ], 403 | "execution_count": null, 404 | "outputs": [] 405 | }, 406 | { 407 | "cell_type": "code", 408 | "metadata": { 409 | "id": "Z-VpWHwHqZyk" 410 | }, 411 | "source": [ 412 | "fiP1, fiM1 = function(xInput + delta), function(xInput - delta)" 413 | ], 414 | "execution_count": null, 415 | "outputs": [] 416 | }, 417 | { 418 | "cell_type": "code", 419 | "metadata": { 420 | "id": "GzT3yFBSqlA1" 421 | }, 422 | "source": [ 423 | "gradient = CDA(fiP1, fiM1, delta)\n", 424 | "print(f\"Gradient : {gradient:.4f}\")" 425 | ], 426 | "execution_count": null, 427 | "outputs": [] 428 | }, 429 | { 430 | "cell_type": "markdown", 431 | "source": [ 432 | "#### Visualize Result" 433 | ], 434 | "metadata": { 435 | "id": "GEOZKZU1decp" 436 | } 437 | }, 438 | { 439 | "cell_type": "code", 440 | "metadata": { 441 | "id": "-H_VL9aXqtg8" 442 | }, 443 | "source": [ 444 | "intercept = fi - gradient * xInput" 445 | ], 446 | "execution_count": null, 447 | "outputs": [] 448 | }, 449 | { 450 | "cell_type": "code", 451 | "metadata": { 452 | "id": "UV8gSM9Xqvbd" 453 | }, 454 | "source": [ 455 | "difference = linear(x, gradient, intercept)" 456 | ], 457 | "execution_count": null, 458 | "outputs": [] 459 | }, 460 | { 461 | "cell_type": "code", 462 | "metadata": { 463 | "id": "f9BdgIv1qwmX" 464 | }, 465 | "source": [ 466 | "plt.plot(x, y, label = \"original\")\n", 467 | "plt.plot(x, difference, label = \"gradient\")\n", 468 | "plt.scatter(xInput, function(xInput), label = \"point\")\n", 469 | "plt.xlabel(\"x\")\n", 470 | "plt.ylabel(\"f(x)\")\n", 471 | "plt.title(\"Function Graph\")\n", 472 | "plt.legend(loc = \"best\")\n", 473 | "plt.show()" 474 | ], 475 | "execution_count": null, 476 | "outputs": [] 477 | }, 478 | { 479 | "cell_type": "markdown", 480 | "metadata": { 481 | "id": "MR_4Ni7vrAbY" 482 | }, 483 | "source": [ 484 | "### Example" 485 | ] 486 | }, 487 | { 488 | "cell_type": "markdown", 489 | "source": [ 490 | "#### Harmonic Function" 491 | ], 492 | "metadata": { 493 | "id": "NQsGcroJdz1H" 494 | } 495 | }, 496 | { 497 | "cell_type": "markdown", 498 | "source": [ 499 | "##### Plot Graph" 500 | ], 501 | "metadata": { 502 | "id": "JBDD-5VXd3R0" 503 | } 504 | }, 505 | { 506 | "cell_type": "code", 507 | "metadata": { 508 | "id": "V0oPQooKq8qH" 509 | }, 510 | "source": [ 511 | "delta = 1e-2\n", 512 | "x = np.arange(0, 10, delta)\n", 513 | "y = np.sin(x)" 514 | ], 515 | "execution_count": null, 516 | "outputs": [] 517 | }, 518 | { 519 | "cell_type": "code", 520 | "metadata": { 521 | "id": "sYAQGiZXrNs9" 522 | }, 523 | "source": [ 524 | "plt.plot(x, y)\n", 525 | "plt.xlabel(\"x\")\n", 526 | "plt.ylabel(\"f(x)\")\n", 527 | "plt.title(\"Function Graph\")\n", 528 | "plt.show()" 529 | ], 530 | "execution_count": null, 531 | "outputs": [] 532 | }, 533 | { 534 | "cell_type": "markdown", 535 | "source": [ 536 | "##### Compute Gradient" 537 | ], 538 | "metadata": { 539 | "id": "NESuiBWKd9yV" 540 | } 541 | }, 542 | { 543 | "cell_type": "code", 544 | "metadata": { 545 | "id": "lj-qw8-krsV_" 546 | }, 547 | "source": [ 548 | "xP1, xM1 = x + delta, x - delta\n", 549 | "yP1, yM1 = np.sin(xP1), np.sin(xM1)" 550 | ], 551 | "execution_count": null, 552 | "outputs": [] 553 | }, 554 | { 555 | "cell_type": "code", 556 | "metadata": { 557 | "id": "NyFU2G1MrP2T" 558 | }, 559 | "source": [ 560 | "gradientFFDA = FFDA(y, yP1, delta)\n", 561 | "gradientFBDA = FBDA(y, yM1, delta)\n", 562 | "gradientCDA = CDA(yP1, yM1, delta)" 563 | ], 564 | "execution_count": null, 565 | "outputs": [] 566 | }, 567 | { 568 | "cell_type": "markdown", 569 | "source": [ 570 | "##### Visualize Result" 571 | ], 572 | "metadata": { 573 | "id": "WBHZWruiemX8" 574 | } 575 | }, 576 | { 577 | "cell_type": "code", 578 | "metadata": { 579 | "id": "_xHv9GB_sF22" 580 | }, 581 | "source": [ 582 | "plt.plot(x[:25], gradientFFDA[:25], label = \"FFDA\")\n", 583 | "plt.plot(x[:25], gradientFBDA[:25], label = \"FBDA\")\n", 584 | "plt.plot(x[:25], gradientCDA[:25], label = \"CDA\")\n", 585 | "plt.xlabel(\"x\")\n", 586 | "plt.ylabel(\"f'(x)\")\n", 587 | "plt.title(\"Derivative Graph\")\n", 588 | "plt.legend(loc = \"best\")\n", 589 | "plt.show()" 590 | ], 591 | "execution_count": null, 592 | "outputs": [] 593 | }, 594 | { 595 | "cell_type": "markdown", 596 | "source": [ 597 | "##### Compute RMSE Loss" 598 | ], 599 | "metadata": { 600 | "id": "WBDk8f-9eqyA" 601 | } 602 | }, 603 | { 604 | "cell_type": "code", 605 | "source": [ 606 | "def RMSE(yHat, y) :\n", 607 | " loss = np.sqrt(np.power((yHat- y), 2).mean())\n", 608 | "\n", 609 | " return loss" 610 | ], 611 | "metadata": { 612 | "id": "PlqCNmRges6j" 613 | }, 614 | "execution_count": null, 615 | "outputs": [] 616 | }, 617 | { 618 | "cell_type": "code", 619 | "metadata": { 620 | "id": "zp3ovcWzv6gI" 621 | }, 622 | "source": [ 623 | "rmseFFDA = RMSE(gradientFFDA, np.cos(x))\n", 624 | "rmseFBDA = RMSE(gradientFBDA, np.cos(x))\n", 625 | "rmseCDA = RMSE(gradientCDA, np.cos(x))" 626 | ], 627 | "execution_count": null, 628 | "outputs": [] 629 | }, 630 | { 631 | "cell_type": "code", 632 | "metadata": { 633 | "id": "fFzwhLslwbta" 634 | }, 635 | "source": [ 636 | "print(f\"FFDA RMSE : {rmseFFDA:.8f}\")\n", 637 | "print(f\"RBDA RMSE : {rmseFBDA:.8f}\")\n", 638 | "print(f\"CDA RMSE : {rmseCDA:.8f}\")" 639 | ], 640 | "execution_count": null, 641 | "outputs": [] 642 | }, 643 | { 644 | "cell_type": "markdown", 645 | "metadata": { 646 | "id": "Z_BUIZxguOkB" 647 | }, 648 | "source": [ 649 | "## Second Derivation with Forward Difference Approximation" 650 | ] 651 | }, 652 | { 653 | "cell_type": "markdown", 654 | "source": [ 655 | "### Create Function" 656 | ], 657 | "metadata": { 658 | "id": "MWWQWbo-gDFX" 659 | } 660 | }, 661 | { 662 | "cell_type": "code", 663 | "metadata": { 664 | "id": "RGCism-_gDFY" 665 | }, 666 | "source": [ 667 | "def function(x) :\n", 668 | " return 3*np.power(x, 2) + 2*x + 1" 669 | ], 670 | "execution_count": null, 671 | "outputs": [] 672 | }, 673 | { 674 | "cell_type": "markdown", 675 | "source": [ 676 | "### Plot Graph" 677 | ], 678 | "metadata": { 679 | "id": "aAYk-8grgDFY" 680 | } 681 | }, 682 | { 683 | "cell_type": "code", 684 | "metadata": { 685 | "id": "wl4ceUa-gDFY" 686 | }, 687 | "source": [ 688 | "x = np.arange(0, 10, 1e-3)\n", 689 | "y = function(x)" 690 | ], 691 | "execution_count": null, 692 | "outputs": [] 693 | }, 694 | { 695 | "cell_type": "code", 696 | "metadata": { 697 | "id": "jYdNgX4SgDFY" 698 | }, 699 | "source": [ 700 | "plt.plot(x, y)\n", 701 | "plt.xlabel(\"x\")\n", 702 | "plt.ylabel(\"f(x)\")\n", 703 | "plt.title(\"Function Graph\")\n", 704 | "plt.show()" 705 | ], 706 | "execution_count": null, 707 | "outputs": [] 708 | }, 709 | { 710 | "cell_type": "markdown", 711 | "source": [ 712 | "### Second Forward Difference Approximation" 713 | ], 714 | "metadata": { 715 | "id": "In0ziN7RgsFx" 716 | } 717 | }, 718 | { 719 | "cell_type": "markdown", 720 | "source": [ 721 | "#### Compute Gradient" 722 | ], 723 | "metadata": { 724 | "id": "DOble9bPgE7F" 725 | } 726 | }, 727 | { 728 | "cell_type": "code", 729 | "metadata": { 730 | "id": "QcHbYkedupMB" 731 | }, 732 | "source": [ 733 | "def SDFDA(fiP2, fiP1, fi, delta) :\n", 734 | " gradient = (fiP2 - 2*fiP1 + fi) / np.power(delta, 2)\n", 735 | " return gradient" 736 | ], 737 | "execution_count": null, 738 | "outputs": [] 739 | }, 740 | { 741 | "cell_type": "code", 742 | "source": [ 743 | "xInput, delta = 5, 1e-3" 744 | ], 745 | "metadata": { 746 | "id": "lkrme9OkgMxi" 747 | }, 748 | "execution_count": null, 749 | "outputs": [] 750 | }, 751 | { 752 | "cell_type": "code", 753 | "metadata": { 754 | "id": "MuFLRUhiuiIN" 755 | }, 756 | "source": [ 757 | "fiP2, fiP1, fi = function(xInput + 2 * delta), function(xInput + delta), function(xInput)" 758 | ], 759 | "execution_count": null, 760 | "outputs": [] 761 | }, 762 | { 763 | "cell_type": "code", 764 | "metadata": { 765 | "id": "p3SG-oHHu3r5" 766 | }, 767 | "source": [ 768 | "gradient = SDFDA(fiP2, fiP1, fi, delta)\n", 769 | "print(f\"Gradient : {gradient:.8f}\")" 770 | ], 771 | "execution_count": null, 772 | "outputs": [] 773 | }, 774 | { 775 | "cell_type": "markdown", 776 | "source": [ 777 | "### Example" 778 | ], 779 | "metadata": { 780 | "id": "u0ar8MCVhHEy" 781 | } 782 | }, 783 | { 784 | "cell_type": "markdown", 785 | "metadata": { 786 | "id": "DGoH15kKvRTt" 787 | }, 788 | "source": [ 789 | "#### Harmonic Function" 790 | ] 791 | }, 792 | { 793 | "cell_type": "markdown", 794 | "source": [ 795 | "##### Plot Graph" 796 | ], 797 | "metadata": { 798 | "id": "1F81U7CLhLdb" 799 | } 800 | }, 801 | { 802 | "cell_type": "code", 803 | "metadata": { 804 | "id": "LDpuXv_SvS2g" 805 | }, 806 | "source": [ 807 | "delta = 1e-2\n", 808 | "x = np.arange(0, 10, delta)\n", 809 | "y = np.sin(x)" 810 | ], 811 | "execution_count": null, 812 | "outputs": [] 813 | }, 814 | { 815 | "cell_type": "code", 816 | "metadata": { 817 | "id": "qmbJaHL2vVSb" 818 | }, 819 | "source": [ 820 | "plt.plot(x, y)\n", 821 | "plt.xlabel(\"x\")\n", 822 | "plt.ylabel(\"f(x)\")\n", 823 | "plt.title(\"Function Graph\")\n", 824 | "plt.show()" 825 | ], 826 | "execution_count": null, 827 | "outputs": [] 828 | }, 829 | { 830 | "cell_type": "markdown", 831 | "source": [ 832 | "##### Compute Gradient" 833 | ], 834 | "metadata": { 835 | "id": "sV11jr34hPq7" 836 | } 837 | }, 838 | { 839 | "cell_type": "code", 840 | "metadata": { 841 | "id": "uo-pe1mrvXYr" 842 | }, 843 | "source": [ 844 | "xP2, xP1 = x + 2*delta, x + delta\n", 845 | "yP2, yP1 = np.sin(xP2), np.sin(xP1)" 846 | ], 847 | "execution_count": null, 848 | "outputs": [] 849 | }, 850 | { 851 | "cell_type": "code", 852 | "metadata": { 853 | "id": "0pDFuwd6vkeh" 854 | }, 855 | "source": [ 856 | "gradient = SDFDA(yP2, yP1, y, delta)" 857 | ], 858 | "execution_count": null, 859 | "outputs": [] 860 | }, 861 | { 862 | "cell_type": "markdown", 863 | "source": [ 864 | "##### Visualize Result" 865 | ], 866 | "metadata": { 867 | "id": "0aYs9yeAhb41" 868 | } 869 | }, 870 | { 871 | "cell_type": "code", 872 | "metadata": { 873 | "id": "-Y6LMgpSvsD5" 874 | }, 875 | "source": [ 876 | "plt.plot(x, gradient, label = \"SDFDA\")\n", 877 | "plt.xlabel(\"x\")\n", 878 | "plt.ylabel(\"f'(x)\")\n", 879 | "plt.title(\"Derivative Graph\")\n", 880 | "plt.legend(loc = \"best\")\n", 881 | "plt.show()" 882 | ], 883 | "execution_count": null, 884 | "outputs": [] 885 | }, 886 | { 887 | "cell_type": "markdown", 888 | "source": [ 889 | "##### Compute RMSE Loss" 890 | ], 891 | "metadata": { 892 | "id": "rTEce-uehkq5" 893 | } 894 | }, 895 | { 896 | "cell_type": "code", 897 | "metadata": { 898 | "id": "IDmPErUgwlq7" 899 | }, 900 | "source": [ 901 | "rmse = RMSE(gradient, -np.sin(x))" 902 | ], 903 | "execution_count": null, 904 | "outputs": [] 905 | }, 906 | { 907 | "cell_type": "code", 908 | "source": [ 909 | "print(f\"RMSE : {rmse:.8f}\")" 910 | ], 911 | "metadata": { 912 | "id": "uqeGOER4hrHD" 913 | }, 914 | "execution_count": null, 915 | "outputs": [] 916 | } 917 | ] 918 | } -------------------------------------------------------------------------------- /2022-2/열유체공학실험_Week_12.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [], 7 | "authorship_tag": "ABX9TyPm/u8RYdVdwo3pQB7DEqqg", 8 | "include_colab_link": true 9 | }, 10 | "kernelspec": { 11 | "name": "python3", 12 | "display_name": "Python 3" 13 | }, 14 | "language_info": { 15 | "name": "python" 16 | } 17 | }, 18 | "cells": [ 19 | { 20 | "cell_type": "markdown", 21 | "metadata": { 22 | "id": "view-in-github", 23 | "colab_type": "text" 24 | }, 25 | "source": [ 26 | "\"Open" 27 | ] 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "source": [ 32 | "# Neural Network" 33 | ], 34 | "metadata": { 35 | "id": "w4pwoaQBRZEq" 36 | } 37 | }, 38 | { 39 | "cell_type": "markdown", 40 | "source": [ 41 | "## Import Library" 42 | ], 43 | "metadata": { 44 | "id": "fCmajizLRbAv" 45 | } 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "metadata": { 51 | "id": "0Cpux903NhlD" 52 | }, 53 | "outputs": [], 54 | "source": [ 55 | "import numpy as np\n", 56 | "import matplotlib.pyplot as plt" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "source": [ 62 | "## Create Function" 63 | ], 64 | "metadata": { 65 | "id": "W95iyhCoRc4h" 66 | } 67 | }, 68 | { 69 | "cell_type": "code", 70 | "source": [ 71 | "def function(x1, x2, x3) :\n", 72 | " return np.power(x1,2) + 4*np.power(x2,2) - 10*x3" 73 | ], 74 | "metadata": { 75 | "id": "2tnz6kmNN_xM" 76 | }, 77 | "execution_count": null, 78 | "outputs": [] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "source": [ 83 | "deltaX = 1e-3\n", 84 | "x1, x2, x3 = np.arange(2, 6, deltaX), np.arange(-4, 0, deltaX), np.arange(0, 4, deltaX)" 85 | ], 86 | "metadata": { 87 | "id": "V0R2zcSSOQoL" 88 | }, 89 | "execution_count": null, 90 | "outputs": [] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "source": [ 95 | "y = function(x1, x2, x3)" 96 | ], 97 | "metadata": { 98 | "id": "xjzmw9TKOd1U" 99 | }, 100 | "execution_count": null, 101 | "outputs": [] 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "source": [ 106 | "### Visualize Histogram" 107 | ], 108 | "metadata": { 109 | "id": "SuF0CyfeRnef" 110 | } 111 | }, 112 | { 113 | "cell_type": "code", 114 | "source": [ 115 | "plt.figure(figsize=(10, 5))\n", 116 | "plt.hist(y, bins = 100)\n", 117 | "plt.xlabel(\"value\")\n", 118 | "plt.ylabel(\"frequency\")\n", 119 | "plt.title(\"Output Distribution\")\n", 120 | "plt.show()" 121 | ], 122 | "metadata": { 123 | "id": "njrULeh5Ofm3" 124 | }, 125 | "execution_count": null, 126 | "outputs": [] 127 | }, 128 | { 129 | "cell_type": "markdown", 130 | "source": [ 131 | "## Activation Function" 132 | ], 133 | "metadata": { 134 | "id": "PyI2MzoSRNG3" 135 | } 136 | }, 137 | { 138 | "cell_type": "code", 139 | "source": [ 140 | "np.random.seed(42)" 141 | ], 142 | "metadata": { 143 | "id": "rSIiOai7DH5T" 144 | }, 145 | "execution_count": null, 146 | "outputs": [] 147 | }, 148 | { 149 | "cell_type": "markdown", 150 | "source": [ 151 | "### Sigmoid" 152 | ], 153 | "metadata": { 154 | "id": "l84szhBbUau6" 155 | } 156 | }, 157 | { 158 | "cell_type": "code", 159 | "source": [ 160 | "def Sigmoid(input:np.array)->np.array :\n", 161 | " return np.power(1 + np.exp(-input), -1)" 162 | ], 163 | "metadata": { 164 | "id": "RnXY7nT_Uau6" 165 | }, 166 | "execution_count": null, 167 | "outputs": [] 168 | }, 169 | { 170 | "cell_type": "markdown", 171 | "source": [ 172 | "#### Random Sampling" 173 | ], 174 | "metadata": { 175 | "id": "mVlBXL-2Uau6" 176 | } 177 | }, 178 | { 179 | "cell_type": "code", 180 | "source": [ 181 | "numSample = 25" 182 | ], 183 | "metadata": { 184 | "id": "q0K3GK8wUau7" 185 | }, 186 | "execution_count": null, 187 | "outputs": [] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "source": [ 192 | "randomVariables = np.random.randn((numSample))" 193 | ], 194 | "metadata": { 195 | "id": "a-4iBSuLUau7" 196 | }, 197 | "execution_count": null, 198 | "outputs": [] 199 | }, 200 | { 201 | "cell_type": "markdown", 202 | "source": [ 203 | "#### Compare Result" 204 | ], 205 | "metadata": { 206 | "id": "4fWmulpmUau7" 207 | } 208 | }, 209 | { 210 | "cell_type": "code", 211 | "source": [ 212 | "plt.figure(figsize=(10, 5))\n", 213 | "plt.scatter(np.arange(numSample), randomVariables, label=\"Original\", marker=\"X\", s=250)\n", 214 | "plt.scatter(np.arange(numSample), Sigmoid(randomVariables), label=\"Sigmoid\", marker=\".\", s=250)\n", 215 | "\n", 216 | "for i in range(numSample) :\n", 217 | " deltaY = Sigmoid(randomVariables[i]) - randomVariables[i]\n", 218 | " if deltaY != 0 :\n", 219 | " plt.arrow(i, randomVariables[i], 0, deltaY, head_width = 0.25, head_length = 0.05, fc = \"k\", ec = \"k\")\n", 220 | "\n", 221 | "plt.xlabel(\"# sample\")\n", 222 | "plt.ylabel(\"value\")\n", 223 | "plt.title(\"Result Comparison (Sigmoid)\")\n", 224 | "plt.legend(loc=\"best\")\n", 225 | "plt.show()" 226 | ], 227 | "metadata": { 228 | "id": "AmqfiqDnUau7" 229 | }, 230 | "execution_count": null, 231 | "outputs": [] 232 | }, 233 | { 234 | "cell_type": "markdown", 235 | "source": [ 236 | "### Rectified Linear Unit (ReLU)" 237 | ], 238 | "metadata": { 239 | "id": "Oy9EMNZ_ROXR" 240 | } 241 | }, 242 | { 243 | "cell_type": "code", 244 | "source": [ 245 | "def ReLU(input:np.array)->np.array :\n", 246 | " return np.where(input > 0, input, 0)" 247 | ], 248 | "metadata": { 249 | "id": "Oci-Y5IgQExX" 250 | }, 251 | "execution_count": null, 252 | "outputs": [] 253 | }, 254 | { 255 | "cell_type": "markdown", 256 | "source": [ 257 | "#### Random Sampling" 258 | ], 259 | "metadata": { 260 | "id": "Irc3oiTgR8sd" 261 | } 262 | }, 263 | { 264 | "cell_type": "code", 265 | "source": [ 266 | "numSample = 25" 267 | ], 268 | "metadata": { 269 | "id": "DXWi0v8CS_DB" 270 | }, 271 | "execution_count": null, 272 | "outputs": [] 273 | }, 274 | { 275 | "cell_type": "code", 276 | "source": [ 277 | "randomVariables = np.random.randn((numSample))" 278 | ], 279 | "metadata": { 280 | "id": "99yDg2vTRRik" 281 | }, 282 | "execution_count": null, 283 | "outputs": [] 284 | }, 285 | { 286 | "cell_type": "markdown", 287 | "source": [ 288 | "#### Compare Result" 289 | ], 290 | "metadata": { 291 | "id": "BMfmn_kcR9_n" 292 | } 293 | }, 294 | { 295 | "cell_type": "code", 296 | "source": [ 297 | "plt.figure(figsize=(10, 5))\n", 298 | "plt.scatter(np.arange(numSample), randomVariables, label=\"Original\", marker=\"X\", s=250)\n", 299 | "plt.scatter(np.arange(numSample), ReLU(randomVariables), label=\"ReLU\", marker=\".\", s=250)\n", 300 | "\n", 301 | "for i in range(numSample) :\n", 302 | " deltaY = ReLU(randomVariables[i]) - randomVariables[i]\n", 303 | " if deltaY != 0 :\n", 304 | " plt.arrow(i, randomVariables[i], 0, deltaY, head_width = 0.25, head_length = 0.05, fc = \"k\", ec = \"k\")\n", 305 | "\n", 306 | "plt.xlabel(\"# sample\")\n", 307 | "plt.ylabel(\"value\")\n", 308 | "plt.title(\"Result Comparison (ReLU)\")\n", 309 | "plt.legend(loc=\"best\")\n", 310 | "plt.show()" 311 | ], 312 | "metadata": { 313 | "id": "3OvyNKxRRyi6" 314 | }, 315 | "execution_count": null, 316 | "outputs": [] 317 | }, 318 | { 319 | "cell_type": "markdown", 320 | "source": [ 321 | "## Build 2-Layer Neural Network" 322 | ], 323 | "metadata": { 324 | "id": "xgIvG2txRVat" 325 | } 326 | }, 327 | { 328 | "cell_type": "code", 329 | "source": [ 330 | "from tqdm import tqdm" 331 | ], 332 | "metadata": { 333 | "id": "g1XcEyC17t8X" 334 | }, 335 | "execution_count": null, 336 | "outputs": [] 337 | }, 338 | { 339 | "cell_type": "code", 340 | "source": [ 341 | "class TwoLayerNeuralNetwork :\n", 342 | " def __init__(self, numInputs:int, numHiddenLayerNodes:int, numOutputs:int, seed:int) :\n", 343 | " # Initialize Variables\n", 344 | " self.numInputs = numInputs\n", 345 | " self.numHiddenLayerNodes = numHiddenLayerNodes\n", 346 | " self.numOutputs = numOutputs\n", 347 | " self.seed = seed\n", 348 | "\n", 349 | " # Initialize Model Parameters\n", 350 | " self.initializeWeights()\n", 351 | " \n", 352 | " def initializeWeights(self) :\n", 353 | " # Fix Seed\n", 354 | " np.random.seed(self.seed)\n", 355 | "\n", 356 | " # Initialize Model Parameters\n", 357 | " self.layer1Weights = np.random.random((self.numInputs, self.numHiddenLayerNodes))\n", 358 | " self.layer2Weights = np.random.random((self.numHiddenLayerNodes, self.numOutputs))\n", 359 | "\n", 360 | " # Print Model Parameters\n", 361 | " print(\"Model Parameters Initialized!\")\n", 362 | " print(f\"# Parameters : {self.layer1Weights.size + self.layer2Weights.size}\")\n", 363 | " print(f\"Layer 1 Size : {self.layer1Weights.shape}\")\n", 364 | " print(f\"Layer 1 Weights: {self.layer1Weights.flatten()}\")\n", 365 | " print(f\"Layer 2 Size : {self.layer2Weights.shape}\")\n", 366 | " print(f\"Layer 2 Weights: {self.layer2Weights.flatten()}\")\n", 367 | " \n", 368 | " def LeakyReLU(self, input:np.array)->np.array :\n", 369 | " return np.where(input>0, input, -0.2*input)\n", 370 | "\n", 371 | " def predict(self, input:np.array)->np.array :\n", 372 | " output = self.LeakyReLU(np.matmul(input, self.layer1Weights))\n", 373 | " output = np.matmul(output, self.layer2Weights)\n", 374 | " return output\n", 375 | "\n", 376 | " def computeMSELoss(self, pred:np.array, target:np.array)->float :\n", 377 | " return np.power(target-pred, 2).mean()\n", 378 | "\n", 379 | " def backPropagation(self, input:np.array, target:np.array)->np.array :\n", 380 | " # Compute Each Layer Output\n", 381 | " stg1Output = np.matmul(input, self.layer1Weights)\n", 382 | " stg2Output = self.LeakyReLU(stg1Output)\n", 383 | " stg3Output = np.matmul(stg2Output, self.layer2Weights)\n", 384 | "\n", 385 | " # Compute Gradient of Each Parameter\n", 386 | " gradientLayer2 = -np.matmul(stg2Output.reshape(-1,1), (target-stg3Output).reshape(1,-1))\n", 387 | " gradientLayer1 = -np.matmul(input.reshape(-1,1), (target-stg3Output).mean() * (self.layer2Weights * np.where(stg2Output>0, 1, -0.2)).sum(axis=1).reshape(1,-1))\n", 388 | "\n", 389 | " return gradientLayer1, gradientLayer2\n", 390 | "\n", 391 | " def train(self, inputTrain:np.array, targetTrain:np.array, inputTest:np.array, targetTest:np.array,batchSize:int, learningRate:float)->list :\n", 392 | " # Create List Instance\n", 393 | " trainLossList, testLossList = [],[]\n", 394 | "\n", 395 | " # Initialize Varaibles\n", 396 | " loss = 0\n", 397 | " \n", 398 | " # Compute Iteration\n", 399 | " iteration = len(inputTrain) // batchSize\n", 400 | "\n", 401 | " print(\"Training Phase\")\n", 402 | " with tqdm(total = iteration) as pBar :\n", 403 | " for i in range(iteration) :\n", 404 | " # Initialize Varaibles\n", 405 | " gradientLayer1, gradientLayer2 = 0, 0\n", 406 | " subInputTrain, subTargetTrain = inputTrain[i*batchSize : (i+1)*batchSize], targetTrain[i*batchSize : (i+1)*batchSize]\n", 407 | "\n", 408 | " for j in range(batchSize) :\n", 409 | " # Feed Forward\n", 410 | " pred = self.predict(subInputTrain[j])\n", 411 | "\n", 412 | " # Compute MSE Loss\n", 413 | " loss += self.computeMSELoss(pred, subTargetTrain[j])\n", 414 | "\n", 415 | " # Compute Gradient of Each Data\n", 416 | " subGradientLayer1, subGradientLayer2 = self.backPropagation(subInputTrain[j], subTargetTrain[j])\n", 417 | " gradientLayer1 += subGradientLayer1\n", 418 | " gradientLayer2 += subGradientLayer2\n", 419 | "\n", 420 | " # Compute Average Gradient\n", 421 | " gradientLayer1 /= batchSize\n", 422 | " gradientLayer2 /= batchSize\n", 423 | "\n", 424 | " # Update Model Parameters\n", 425 | " self.layer1Weights -= learningRate * gradientLayer1\n", 426 | " self.layer2Weights -= learningRate * gradientLayer2\n", 427 | "\n", 428 | " # Update TQDM Bar\n", 429 | " pBar.update()\n", 430 | "\n", 431 | " # Compute Average Loss\n", 432 | " loss /= len(inputTrain)\n", 433 | " trainLossList.append(loss)\n", 434 | "\n", 435 | " # Initialize Varaibles\n", 436 | " loss = 0\n", 437 | "\n", 438 | " # Compute Iteration\n", 439 | " iteration = len(inputTest) // batchSize\n", 440 | "\n", 441 | " print(\"Test Phase\")\n", 442 | " with tqdm(total = iteration) as pBar :\n", 443 | " for i in range(iteration) :\n", 444 | " # Initialize Varaibles\n", 445 | " subInputTest, subTargetTest = inputTest[i*batchSize : (i+1)*batchSize], targetTest[i*batchSize : (i+1)*batchSize]\n", 446 | "\n", 447 | " for j in range(batchSize) :\n", 448 | " # Feed Forward\n", 449 | " pred = self.predict(subInputTest[j])\n", 450 | "\n", 451 | " # Compute MSE Loss\n", 452 | " loss += self.computeMSELoss(pred, subTargetTest[j])\n", 453 | "\n", 454 | " # Update TQDM Bar\n", 455 | " pBar.update()\n", 456 | "\n", 457 | " # Compute Average Loss\n", 458 | " loss /= len(inputTest)\n", 459 | " testLossList.append(loss)\n", 460 | "\n", 461 | " return trainLossList, testLossList" 462 | ], 463 | "metadata": { 464 | "id": "Da2IF_ftQlv0" 465 | }, 466 | "execution_count": null, 467 | "outputs": [] 468 | }, 469 | { 470 | "cell_type": "markdown", 471 | "source": [ 472 | "## Generate Dataset" 473 | ], 474 | "metadata": { 475 | "id": "iAJhSnm4kXdt" 476 | } 477 | }, 478 | { 479 | "cell_type": "code", 480 | "source": [ 481 | "inputDataset, targetDataset = [], []\n", 482 | "\n", 483 | "for i in range(len(y)) :\n", 484 | " inputDataset.append(np.array([x1[i], x2[i], x3[i]]))\n", 485 | " targetDataset.append(np.array(y[i]))\n", 486 | "\n", 487 | "inputDataset, targetDataset = np.array(inputDataset), np.array(targetDataset)" 488 | ], 489 | "metadata": { 490 | "id": "lmjqhjHDiOIq" 491 | }, 492 | "execution_count": null, 493 | "outputs": [] 494 | }, 495 | { 496 | "cell_type": "code", 497 | "source": [ 498 | "print(f\"Input Dataset Size : {inputDataset.shape}\")\n", 499 | "print(f\"Target Dataset Size : {targetDataset.shape}\")" 500 | ], 501 | "metadata": { 502 | "id": "t4mnF4nglXzo" 503 | }, 504 | "execution_count": null, 505 | "outputs": [] 506 | }, 507 | { 508 | "cell_type": "markdown", 509 | "source": [ 510 | "### Split Dataset" 511 | ], 512 | "metadata": { 513 | "id": "Xos9NQi170FH" 514 | } 515 | }, 516 | { 517 | "cell_type": "code", 518 | "source": [ 519 | "from sklearn.model_selection import train_test_split" 520 | ], 521 | "metadata": { 522 | "id": "EEnihcnE70_F" 523 | }, 524 | "execution_count": null, 525 | "outputs": [] 526 | }, 527 | { 528 | "cell_type": "code", 529 | "source": [ 530 | "xTrain, xTest, yTrain, yTest = train_test_split(inputDataset, targetDataset, test_size = 0.2, random_state = 42)" 531 | ], 532 | "metadata": { 533 | "id": "oi58nD_J730B" 534 | }, 535 | "execution_count": null, 536 | "outputs": [] 537 | }, 538 | { 539 | "cell_type": "code", 540 | "source": [ 541 | "print(f\"Training Dataset Size : {xTrain.shape[0]}\")\n", 542 | "print(f\"Test Dataset Size : {xTest.shape[0]}\")" 543 | ], 544 | "metadata": { 545 | "id": "BWb5KZ8n8AW9" 546 | }, 547 | "execution_count": null, 548 | "outputs": [] 549 | }, 550 | { 551 | "cell_type": "markdown", 552 | "source": [ 553 | "## Create Neural Network Instance" 554 | ], 555 | "metadata": { 556 | "id": "Pf2t03OAaMv9" 557 | } 558 | }, 559 | { 560 | "cell_type": "code", 561 | "source": [ 562 | "numInputs, numHiddenLayerNodes, numOutputs, seed = 3, 2, 1, 42" 563 | ], 564 | "metadata": { 565 | "id": "mADKNfkGmNQI" 566 | }, 567 | "execution_count": null, 568 | "outputs": [] 569 | }, 570 | { 571 | "cell_type": "code", 572 | "source": [ 573 | "model = TwoLayerNeuralNetwork(numInputs, numHiddenLayerNodes, numOutputs, seed)" 574 | ], 575 | "metadata": { 576 | "id": "YqEZn7XDUTzt" 577 | }, 578 | "execution_count": null, 579 | "outputs": [] 580 | }, 581 | { 582 | "cell_type": "markdown", 583 | "source": [ 584 | "## Determine Hyperparameters" 585 | ], 586 | "metadata": { 587 | "id": "AwbenO-LmWp8" 588 | } 589 | }, 590 | { 591 | "cell_type": "code", 592 | "source": [ 593 | "numEpoch = 200" 594 | ], 595 | "metadata": { 596 | "id": "xpHWA7vi1T7t" 597 | }, 598 | "execution_count": null, 599 | "outputs": [] 600 | }, 601 | { 602 | "cell_type": "code", 603 | "source": [ 604 | "batchSize, learningRate = 128, 1e-4" 605 | ], 606 | "metadata": { 607 | "id": "qHUEWNcBmaBP" 608 | }, 609 | "execution_count": null, 610 | "outputs": [] 611 | }, 612 | { 613 | "cell_type": "markdown", 614 | "source": [ 615 | "## Train Model" 616 | ], 617 | "metadata": { 618 | "id": "JeWyr2yWmZLg" 619 | } 620 | }, 621 | { 622 | "cell_type": "code", 623 | "source": [ 624 | "def trainModel(model, numEpoch, batchSize, learningRate) :\n", 625 | " trainLossList, testLossList = [],[]\n", 626 | " bestLoss = np.inf\n", 627 | "\n", 628 | " for epoch in range(numEpoch) :\n", 629 | " print(f\"[Current Epoch : {epoch + 1}]\")\n", 630 | " trainLoss, testLoss = model.train(xTrain, yTrain, xTest, yTest, batchSize, learningRate)\n", 631 | " trainLossList += trainLoss\n", 632 | " testLossList += testLoss\n", 633 | "\n", 634 | " if testLoss[0] < bestLoss :\n", 635 | " bestLoss = testLoss[0]\n", 636 | " bestWeight = [model.layer1Weights, model.layer2Weights]\n", 637 | "\n", 638 | " return trainLossList, testLossList, bestWeight" 639 | ], 640 | "metadata": { 641 | "id": "2D2R0Dxa2zuq" 642 | }, 643 | "execution_count": null, 644 | "outputs": [] 645 | }, 646 | { 647 | "cell_type": "code", 648 | "source": [ 649 | "trainLossList, testLossList, bestWeight = trainModel(model, numEpoch, batchSize, learningRate)" 650 | ], 651 | "metadata": { 652 | "id": "HP2tzYbmldOc" 653 | }, 654 | "execution_count": null, 655 | "outputs": [] 656 | }, 657 | { 658 | "cell_type": "markdown", 659 | "source": [ 660 | "## Plot Loss Curve" 661 | ], 662 | "metadata": { 663 | "id": "C8z7IiI4_vk4" 664 | } 665 | }, 666 | { 667 | "cell_type": "code", 668 | "source": [ 669 | "plt.figure(figsize=(10, 5))\n", 670 | "plt.plot(np.arange(len(trainLossList)), trainLossList, label = \"Train Loss\")\n", 671 | "plt.plot(np.arange(len(testLossList)), testLossList, label = \"Test Loss\")\n", 672 | "plt.xlabel(\"Epoch\")\n", 673 | "plt.ylabel(\"MSE Loss\")\n", 674 | "plt.title(\"Training Result\")\n", 675 | "plt.legend(loc = \"best\")\n", 676 | "plt.show()" 677 | ], 678 | "metadata": { 679 | "id": "sN66LcOC3OVW" 680 | }, 681 | "execution_count": null, 682 | "outputs": [] 683 | }, 684 | { 685 | "cell_type": "markdown", 686 | "source": [ 687 | "## Get Optimized Parameters" 688 | ], 689 | "metadata": { 690 | "id": "E7DyB6NW_yRw" 691 | } 692 | }, 693 | { 694 | "cell_type": "code", 695 | "source": [ 696 | "min(testLossList)" 697 | ], 698 | "metadata": { 699 | "id": "cbn50gJaAJoR" 700 | }, 701 | "execution_count": null, 702 | "outputs": [] 703 | }, 704 | { 705 | "cell_type": "code", 706 | "source": [ 707 | "print(bestWeight[0])\n", 708 | "print(bestWeight[1])" 709 | ], 710 | "metadata": { 711 | "id": "O9nlgjkJ_rlU" 712 | }, 713 | "execution_count": null, 714 | "outputs": [] 715 | }, 716 | { 717 | "cell_type": "markdown", 718 | "source": [ 719 | "## Inference Result with Trained Neural Network" 720 | ], 721 | "metadata": { 722 | "id": "J0wFo8Wi_1e3" 723 | } 724 | }, 725 | { 726 | "cell_type": "code", 727 | "source": [ 728 | "model.layer1Weights, model.layer2Weights = bestWeight[0], bestWeight[1]" 729 | ], 730 | "metadata": { 731 | "id": "eHsRVCeA_tB2" 732 | }, 733 | "execution_count": null, 734 | "outputs": [] 735 | }, 736 | { 737 | "cell_type": "code", 738 | "source": [ 739 | "yPredNN = []\n", 740 | "\n", 741 | "for subXTest in xTest :\n", 742 | " yPredNN.append(model.predict(subXTest)[0])" 743 | ], 744 | "metadata": { 745 | "id": "Wx_LgjLkASPv" 746 | }, 747 | "execution_count": null, 748 | "outputs": [] 749 | }, 750 | { 751 | "cell_type": "markdown", 752 | "source": [ 753 | "## Inference Result with Linear Regression Model" 754 | ], 755 | "metadata": { 756 | "id": "G_HA5HRkBFN0" 757 | } 758 | }, 759 | { 760 | "cell_type": "code", 761 | "source": [ 762 | "def getParameter(x: np.array, y: np.array) :\n", 763 | " xT = np.transpose(x)\n", 764 | " output = np.matmul(np.matmul(np.linalg.inv(np.matmul(xT, x)), xT), y)\n", 765 | "\n", 766 | " return output" 767 | ], 768 | "metadata": { 769 | "id": "lkTZZxg-BHAW" 770 | }, 771 | "execution_count": null, 772 | "outputs": [] 773 | }, 774 | { 775 | "cell_type": "code", 776 | "source": [ 777 | "betaHat = getParameter(xTrain, yTrain)" 778 | ], 779 | "metadata": { 780 | "id": "_6MgSyVSBSUa" 781 | }, 782 | "execution_count": null, 783 | "outputs": [] 784 | }, 785 | { 786 | "cell_type": "code", 787 | "source": [ 788 | "betaHat" 789 | ], 790 | "metadata": { 791 | "id": "qgWhYnMMBUo7" 792 | }, 793 | "execution_count": null, 794 | "outputs": [] 795 | }, 796 | { 797 | "cell_type": "code", 798 | "source": [ 799 | "yPredLR = np.matmul(xTest, betaHat).tolist()" 800 | ], 801 | "metadata": { 802 | "id": "QAwywSf3BWHV" 803 | }, 804 | "execution_count": null, 805 | "outputs": [] 806 | }, 807 | { 808 | "cell_type": "markdown", 809 | "source": [ 810 | "## Compare Result" 811 | ], 812 | "metadata": { 813 | "id": "vJ0XImuOBq-r" 814 | } 815 | }, 816 | { 817 | "cell_type": "markdown", 818 | "source": [ 819 | "### Compute MSE Loss" 820 | ], 821 | "metadata": { 822 | "id": "IHS6VmFwCaMz" 823 | } 824 | }, 825 | { 826 | "cell_type": "code", 827 | "source": [ 828 | "def MSELoss(yPred:np.array, yTrue:np.array)->float:\n", 829 | " return np.power(yPred-yTrue, 2).mean()" 830 | ], 831 | "metadata": { 832 | "id": "CN4r-nC-CbSY" 833 | }, 834 | "execution_count": null, 835 | "outputs": [] 836 | }, 837 | { 838 | "cell_type": "code", 839 | "source": [ 840 | "print(f\"Neural Network (NN) MSE Loss : {MSELoss(np.array(yPredNN), yTest)}\")\n", 841 | "print(f\"Linear Regression (LR) MSE Loss : {MSELoss(np.array(yPredLR), yTest)}\")" 842 | ], 843 | "metadata": { 844 | "id": "OdPVhb35CmnB" 845 | }, 846 | "execution_count": null, 847 | "outputs": [] 848 | }, 849 | { 850 | "cell_type": "markdown", 851 | "source": [ 852 | "### Plot Bar Chart" 853 | ], 854 | "metadata": { 855 | "id": "gyyBlTMXCY0_" 856 | } 857 | }, 858 | { 859 | "cell_type": "code", 860 | "source": [ 861 | "fig, ax = plt.subplots(figsize = (20, 10))\n", 862 | "idx = np.asarray([i for i in range(50)])\n", 863 | "width = 0.2\n", 864 | "\n", 865 | "ax.bar(idx, yTest[:50], width = width)\n", 866 | "ax.bar(idx+width, yPredNN[:50], width = width)\n", 867 | "ax.bar(idx+2*width, yPredLR[:50], width = width)\n", 868 | "ax.set_xticks(idx)\n", 869 | "ax.legend([\"Ground Truth\", \"NN\", \"LR\"])\n", 870 | "ax.set_xlabel(\"# samples\")\n", 871 | "ax.set_ylabel(\"Value\")\n", 872 | "ax.set_title(\"Result Comparison\")\n", 873 | "\n", 874 | "fig.tight_layout()\n", 875 | "plt.show()" 876 | ], 877 | "metadata": { 878 | "id": "wbfy-HIzBf7D" 879 | }, 880 | "execution_count": null, 881 | "outputs": [] 882 | } 883 | ] 884 | } -------------------------------------------------------------------------------- /2023-2/열유체공학실험1_Week9_인공신경망.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [], 7 | "authorship_tag": "ABX9TyP1S4rSDn02q1ruOP6xDJ3k", 8 | "include_colab_link": true 9 | }, 10 | "kernelspec": { 11 | "name": "python3", 12 | "display_name": "Python 3" 13 | }, 14 | "language_info": { 15 | "name": "python" 16 | } 17 | }, 18 | "cells": [ 19 | { 20 | "cell_type": "markdown", 21 | "metadata": { 22 | "id": "view-in-github", 23 | "colab_type": "text" 24 | }, 25 | "source": [ 26 | "\"Open" 27 | ] 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "source": [ 32 | "# Neural Network" 33 | ], 34 | "metadata": { 35 | "id": "w4pwoaQBRZEq" 36 | } 37 | }, 38 | { 39 | "cell_type": "markdown", 40 | "source": [ 41 | "## Import Library" 42 | ], 43 | "metadata": { 44 | "id": "fCmajizLRbAv" 45 | } 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "metadata": { 51 | "id": "0Cpux903NhlD" 52 | }, 53 | "outputs": [], 54 | "source": [ 55 | "import numpy as np\n", 56 | "import matplotlib.pyplot as plt" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "source": [ 62 | "## Create Function" 63 | ], 64 | "metadata": { 65 | "id": "W95iyhCoRc4h" 66 | } 67 | }, 68 | { 69 | "cell_type": "code", 70 | "source": [ 71 | "def function(x1, x2, x3) :\n", 72 | " return np.power(x1,2) + 4*np.power(x2,2) - 10*x3" 73 | ], 74 | "metadata": { 75 | "id": "2tnz6kmNN_xM" 76 | }, 77 | "execution_count": null, 78 | "outputs": [] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "source": [ 83 | "deltaX = 1e-3\n", 84 | "x1, x2, x3 = np.arange(2, 6, deltaX), np.arange(-4, 0, deltaX), np.arange(0, 4, deltaX)" 85 | ], 86 | "metadata": { 87 | "id": "V0R2zcSSOQoL" 88 | }, 89 | "execution_count": null, 90 | "outputs": [] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "source": [ 95 | "y = function(x1, x2, x3)" 96 | ], 97 | "metadata": { 98 | "id": "xjzmw9TKOd1U" 99 | }, 100 | "execution_count": null, 101 | "outputs": [] 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "source": [ 106 | "### Visualize Histogram" 107 | ], 108 | "metadata": { 109 | "id": "SuF0CyfeRnef" 110 | } 111 | }, 112 | { 113 | "cell_type": "code", 114 | "source": [ 115 | "plt.figure(figsize=(10, 5))\n", 116 | "plt.hist(y, bins = 100)\n", 117 | "plt.xlabel(\"value\")\n", 118 | "plt.ylabel(\"frequency\")\n", 119 | "plt.title(\"Output Distribution\")\n", 120 | "plt.show()" 121 | ], 122 | "metadata": { 123 | "id": "njrULeh5Ofm3" 124 | }, 125 | "execution_count": null, 126 | "outputs": [] 127 | }, 128 | { 129 | "cell_type": "markdown", 130 | "source": [ 131 | "## Activation Function" 132 | ], 133 | "metadata": { 134 | "id": "PyI2MzoSRNG3" 135 | } 136 | }, 137 | { 138 | "cell_type": "code", 139 | "source": [ 140 | "np.random.seed(42)" 141 | ], 142 | "metadata": { 143 | "id": "rSIiOai7DH5T" 144 | }, 145 | "execution_count": null, 146 | "outputs": [] 147 | }, 148 | { 149 | "cell_type": "markdown", 150 | "source": [ 151 | "### Sigmoid" 152 | ], 153 | "metadata": { 154 | "id": "l84szhBbUau6" 155 | } 156 | }, 157 | { 158 | "cell_type": "code", 159 | "source": [ 160 | "def Sigmoid(input:np.array)->np.array :\n", 161 | " return np.power(1 + np.exp(-input), -1)" 162 | ], 163 | "metadata": { 164 | "id": "RnXY7nT_Uau6" 165 | }, 166 | "execution_count": null, 167 | "outputs": [] 168 | }, 169 | { 170 | "cell_type": "markdown", 171 | "source": [ 172 | "#### Random Sampling" 173 | ], 174 | "metadata": { 175 | "id": "mVlBXL-2Uau6" 176 | } 177 | }, 178 | { 179 | "cell_type": "code", 180 | "source": [ 181 | "numSample = 25" 182 | ], 183 | "metadata": { 184 | "id": "q0K3GK8wUau7" 185 | }, 186 | "execution_count": null, 187 | "outputs": [] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "source": [ 192 | "randomVariables = np.random.randn((numSample))" 193 | ], 194 | "metadata": { 195 | "id": "a-4iBSuLUau7" 196 | }, 197 | "execution_count": null, 198 | "outputs": [] 199 | }, 200 | { 201 | "cell_type": "markdown", 202 | "source": [ 203 | "#### Compare Result" 204 | ], 205 | "metadata": { 206 | "id": "4fWmulpmUau7" 207 | } 208 | }, 209 | { 210 | "cell_type": "code", 211 | "source": [ 212 | "plt.figure(figsize=(10, 5))\n", 213 | "plt.scatter(np.arange(numSample), randomVariables, label=\"Original\", marker=\"X\", s=250)\n", 214 | "plt.scatter(np.arange(numSample), Sigmoid(randomVariables), label=\"Sigmoid\", marker=\".\", s=250)\n", 215 | "\n", 216 | "for i in range(numSample) :\n", 217 | " deltaY = Sigmoid(randomVariables[i]) - randomVariables[i]\n", 218 | " if deltaY != 0 :\n", 219 | " plt.arrow(i, randomVariables[i], 0, deltaY, head_width = 0.25, head_length = 0.05, fc = \"k\", ec = \"k\")\n", 220 | "\n", 221 | "plt.xlabel(\"# sample\")\n", 222 | "plt.ylabel(\"value\")\n", 223 | "plt.title(\"Result Comparison (Sigmoid)\")\n", 224 | "plt.legend(loc=\"best\")\n", 225 | "plt.show()" 226 | ], 227 | "metadata": { 228 | "id": "AmqfiqDnUau7" 229 | }, 230 | "execution_count": null, 231 | "outputs": [] 232 | }, 233 | { 234 | "cell_type": "markdown", 235 | "source": [ 236 | "### Rectified Linear Unit (ReLU)" 237 | ], 238 | "metadata": { 239 | "id": "Oy9EMNZ_ROXR" 240 | } 241 | }, 242 | { 243 | "cell_type": "code", 244 | "source": [ 245 | "def ReLU(input:np.array)->np.array :\n", 246 | " return np.where(input > 0, input, 0)" 247 | ], 248 | "metadata": { 249 | "id": "Oci-Y5IgQExX" 250 | }, 251 | "execution_count": null, 252 | "outputs": [] 253 | }, 254 | { 255 | "cell_type": "markdown", 256 | "source": [ 257 | "#### Random Sampling" 258 | ], 259 | "metadata": { 260 | "id": "Irc3oiTgR8sd" 261 | } 262 | }, 263 | { 264 | "cell_type": "code", 265 | "source": [ 266 | "numSample = 25" 267 | ], 268 | "metadata": { 269 | "id": "DXWi0v8CS_DB" 270 | }, 271 | "execution_count": null, 272 | "outputs": [] 273 | }, 274 | { 275 | "cell_type": "code", 276 | "source": [ 277 | "randomVariables = np.random.randn((numSample))" 278 | ], 279 | "metadata": { 280 | "id": "99yDg2vTRRik" 281 | }, 282 | "execution_count": null, 283 | "outputs": [] 284 | }, 285 | { 286 | "cell_type": "markdown", 287 | "source": [ 288 | "#### Compare Result" 289 | ], 290 | "metadata": { 291 | "id": "BMfmn_kcR9_n" 292 | } 293 | }, 294 | { 295 | "cell_type": "code", 296 | "source": [ 297 | "plt.figure(figsize=(10, 5))\n", 298 | "plt.scatter(np.arange(numSample), randomVariables, label=\"Original\", marker=\"X\", s=250)\n", 299 | "plt.scatter(np.arange(numSample), ReLU(randomVariables), label=\"ReLU\", marker=\".\", s=250)\n", 300 | "\n", 301 | "for i in range(numSample) :\n", 302 | " deltaY = ReLU(randomVariables[i]) - randomVariables[i]\n", 303 | " if deltaY != 0 :\n", 304 | " plt.arrow(i, randomVariables[i], 0, deltaY, head_width = 0.25, head_length = 0.05, fc = \"k\", ec = \"k\")\n", 305 | "\n", 306 | "plt.xlabel(\"# sample\")\n", 307 | "plt.ylabel(\"value\")\n", 308 | "plt.title(\"Result Comparison (ReLU)\")\n", 309 | "plt.legend(loc=\"best\")\n", 310 | "plt.show()" 311 | ], 312 | "metadata": { 313 | "id": "3OvyNKxRRyi6" 314 | }, 315 | "execution_count": null, 316 | "outputs": [] 317 | }, 318 | { 319 | "cell_type": "markdown", 320 | "source": [ 321 | "## Build 2-Layer Neural Network" 322 | ], 323 | "metadata": { 324 | "id": "xgIvG2txRVat" 325 | } 326 | }, 327 | { 328 | "cell_type": "code", 329 | "source": [ 330 | "from tqdm import tqdm" 331 | ], 332 | "metadata": { 333 | "id": "g1XcEyC17t8X" 334 | }, 335 | "execution_count": null, 336 | "outputs": [] 337 | }, 338 | { 339 | "cell_type": "code", 340 | "source": [ 341 | "class TwoLayerNeuralNetwork :\n", 342 | " def __init__(self, numInputs:int, numHiddenLayerNodes:int, numOutputs:int, seed:int) :\n", 343 | " # Initialize Variables\n", 344 | " self.numInputs = numInputs\n", 345 | " self.numHiddenLayerNodes = numHiddenLayerNodes\n", 346 | " self.numOutputs = numOutputs\n", 347 | " self.seed = seed\n", 348 | "\n", 349 | " # Initialize Model Parameters\n", 350 | " self.initializeWeights()\n", 351 | "\n", 352 | " def initializeWeights(self) :\n", 353 | " # Fix Seed\n", 354 | " np.random.seed(self.seed)\n", 355 | "\n", 356 | " # Initialize Model Parameters\n", 357 | " self.layer1Weights = np.random.random((self.numInputs, self.numHiddenLayerNodes))\n", 358 | " self.layer2Weights = np.random.random((self.numHiddenLayerNodes, self.numOutputs))\n", 359 | "\n", 360 | " # Print Model Parameters\n", 361 | " print(\"Model Parameters Initialized!\")\n", 362 | " print(f\"# Parameters : {self.layer1Weights.size + self.layer2Weights.size}\")\n", 363 | " print(f\"Layer 1 Size : {self.layer1Weights.shape}\")\n", 364 | " print(f\"Layer 1 Weights: {self.layer1Weights.flatten()}\")\n", 365 | " print(f\"Layer 2 Size : {self.layer2Weights.shape}\")\n", 366 | " print(f\"Layer 2 Weights: {self.layer2Weights.flatten()}\")\n", 367 | "\n", 368 | " def LeakyReLU(self, input:np.array)->np.array :\n", 369 | " return np.where(input>0, input, -0.2*input)\n", 370 | "\n", 371 | " def predict(self, input:np.array)->np.array :\n", 372 | " output = self.LeakyReLU(np.matmul(input, self.layer1Weights))\n", 373 | " output = np.matmul(output, self.layer2Weights)\n", 374 | " return output\n", 375 | "\n", 376 | " def computeMSELoss(self, pred:np.array, target:np.array)->float :\n", 377 | " return np.power(target-pred, 2).mean()\n", 378 | "\n", 379 | " def backPropagation(self, input:np.array, target:np.array)->np.array :\n", 380 | " # Compute Each Layer Output\n", 381 | " stg1Output = np.matmul(input, self.layer1Weights)\n", 382 | " stg2Output = self.LeakyReLU(stg1Output)\n", 383 | " stg3Output = np.matmul(stg2Output, self.layer2Weights)\n", 384 | "\n", 385 | " # Compute Gradient of Each Parameter\n", 386 | " gradientLayer2 = -np.matmul(stg2Output.reshape(-1,1), (target-stg3Output).reshape(1,-1))\n", 387 | " gradientLayer1 = -np.matmul(input.reshape(-1,1), (target-stg3Output).mean() * (self.layer2Weights * np.where(stg2Output>0, 1, -0.2)).sum(axis=1).reshape(1,-1))\n", 388 | "\n", 389 | " return gradientLayer1, gradientLayer2\n", 390 | "\n", 391 | " def train(self, inputTrain:np.array, targetTrain:np.array, inputTest:np.array, targetTest:np.array,batchSize:int, learningRate:float)->list :\n", 392 | " # Create List Instance\n", 393 | " trainLossList, testLossList = [],[]\n", 394 | "\n", 395 | " # Initialize Varaibles\n", 396 | " loss = 0\n", 397 | "\n", 398 | " # Compute Iteration\n", 399 | " iteration = len(inputTrain) // batchSize\n", 400 | "\n", 401 | " print(\"Training Phase\")\n", 402 | " with tqdm(total = iteration) as pBar :\n", 403 | " for i in range(iteration) :\n", 404 | " # Initialize Varaibles\n", 405 | " gradientLayer1, gradientLayer2 = 0, 0\n", 406 | " subInputTrain, subTargetTrain = inputTrain[i*batchSize : (i+1)*batchSize], targetTrain[i*batchSize : (i+1)*batchSize]\n", 407 | "\n", 408 | " for j in range(batchSize) :\n", 409 | " # Feed Forward\n", 410 | " pred = self.predict(subInputTrain[j])\n", 411 | "\n", 412 | " # Compute MSE Loss\n", 413 | " loss += self.computeMSELoss(pred, subTargetTrain[j])\n", 414 | "\n", 415 | " # Compute Gradient of Each Data\n", 416 | " subGradientLayer1, subGradientLayer2 = self.backPropagation(subInputTrain[j], subTargetTrain[j])\n", 417 | " gradientLayer1 += subGradientLayer1\n", 418 | " gradientLayer2 += subGradientLayer2\n", 419 | "\n", 420 | " # Compute Average Gradient\n", 421 | " gradientLayer1 /= batchSize\n", 422 | " gradientLayer2 /= batchSize\n", 423 | "\n", 424 | " # Update Model Parameters\n", 425 | " self.layer1Weights -= learningRate * gradientLayer1\n", 426 | " self.layer2Weights -= learningRate * gradientLayer2\n", 427 | "\n", 428 | " # Update TQDM Bar\n", 429 | " pBar.update()\n", 430 | "\n", 431 | " # Compute Average Loss\n", 432 | " loss /= len(inputTrain)\n", 433 | " trainLossList.append(loss)\n", 434 | "\n", 435 | " # Initialize Varaibles\n", 436 | " loss = 0\n", 437 | "\n", 438 | " # Compute Iteration\n", 439 | " iteration = len(inputTest) // batchSize\n", 440 | "\n", 441 | " print(\"Test Phase\")\n", 442 | " with tqdm(total = iteration) as pBar :\n", 443 | " for i in range(iteration) :\n", 444 | " # Initialize Varaibles\n", 445 | " subInputTest, subTargetTest = inputTest[i*batchSize : (i+1)*batchSize], targetTest[i*batchSize : (i+1)*batchSize]\n", 446 | "\n", 447 | " for j in range(batchSize) :\n", 448 | " # Feed Forward\n", 449 | " pred = self.predict(subInputTest[j])\n", 450 | "\n", 451 | " # Compute MSE Loss\n", 452 | " loss += self.computeMSELoss(pred, subTargetTest[j])\n", 453 | "\n", 454 | " # Update TQDM Bar\n", 455 | " pBar.update()\n", 456 | "\n", 457 | " # Compute Average Loss\n", 458 | " loss /= len(inputTest)\n", 459 | " testLossList.append(loss)\n", 460 | "\n", 461 | " return trainLossList, testLossList" 462 | ], 463 | "metadata": { 464 | "id": "Da2IF_ftQlv0" 465 | }, 466 | "execution_count": null, 467 | "outputs": [] 468 | }, 469 | { 470 | "cell_type": "markdown", 471 | "source": [ 472 | "## Generate Dataset" 473 | ], 474 | "metadata": { 475 | "id": "iAJhSnm4kXdt" 476 | } 477 | }, 478 | { 479 | "cell_type": "code", 480 | "source": [ 481 | "inputDataset, targetDataset = [], []\n", 482 | "\n", 483 | "for i in range(len(y)) :\n", 484 | " inputDataset.append(np.array([x1[i], x2[i], x3[i]]))\n", 485 | " targetDataset.append(np.array(y[i]))\n", 486 | "\n", 487 | "inputDataset, targetDataset = np.array(inputDataset), np.array(targetDataset)" 488 | ], 489 | "metadata": { 490 | "id": "lmjqhjHDiOIq" 491 | }, 492 | "execution_count": null, 493 | "outputs": [] 494 | }, 495 | { 496 | "cell_type": "code", 497 | "source": [ 498 | "print(f\"Input Dataset Size : {inputDataset.shape}\")\n", 499 | "print(f\"Target Dataset Size : {targetDataset.shape}\")" 500 | ], 501 | "metadata": { 502 | "id": "t4mnF4nglXzo" 503 | }, 504 | "execution_count": null, 505 | "outputs": [] 506 | }, 507 | { 508 | "cell_type": "markdown", 509 | "source": [ 510 | "### Split Dataset" 511 | ], 512 | "metadata": { 513 | "id": "Xos9NQi170FH" 514 | } 515 | }, 516 | { 517 | "cell_type": "code", 518 | "source": [ 519 | "from sklearn.model_selection import train_test_split" 520 | ], 521 | "metadata": { 522 | "id": "EEnihcnE70_F" 523 | }, 524 | "execution_count": null, 525 | "outputs": [] 526 | }, 527 | { 528 | "cell_type": "code", 529 | "source": [ 530 | "xTrain, xTest, yTrain, yTest = train_test_split(inputDataset, targetDataset, test_size = 0.2, random_state = 42)" 531 | ], 532 | "metadata": { 533 | "id": "oi58nD_J730B" 534 | }, 535 | "execution_count": null, 536 | "outputs": [] 537 | }, 538 | { 539 | "cell_type": "code", 540 | "source": [ 541 | "print(f\"Training Dataset Size : {xTrain.shape[0]}\")\n", 542 | "print(f\"Test Dataset Size : {xTest.shape[0]}\")" 543 | ], 544 | "metadata": { 545 | "id": "BWb5KZ8n8AW9" 546 | }, 547 | "execution_count": null, 548 | "outputs": [] 549 | }, 550 | { 551 | "cell_type": "markdown", 552 | "source": [ 553 | "## Create Neural Network Instance" 554 | ], 555 | "metadata": { 556 | "id": "Pf2t03OAaMv9" 557 | } 558 | }, 559 | { 560 | "cell_type": "code", 561 | "source": [ 562 | "numInputs, numHiddenLayerNodes, numOutputs, seed = 3, 2, 1, 42" 563 | ], 564 | "metadata": { 565 | "id": "mADKNfkGmNQI" 566 | }, 567 | "execution_count": null, 568 | "outputs": [] 569 | }, 570 | { 571 | "cell_type": "code", 572 | "source": [ 573 | "model = TwoLayerNeuralNetwork(numInputs, numHiddenLayerNodes, numOutputs, seed)" 574 | ], 575 | "metadata": { 576 | "id": "YqEZn7XDUTzt" 577 | }, 578 | "execution_count": null, 579 | "outputs": [] 580 | }, 581 | { 582 | "cell_type": "markdown", 583 | "source": [ 584 | "## Determine Hyperparameters" 585 | ], 586 | "metadata": { 587 | "id": "AwbenO-LmWp8" 588 | } 589 | }, 590 | { 591 | "cell_type": "code", 592 | "source": [ 593 | "numEpoch = 200" 594 | ], 595 | "metadata": { 596 | "id": "xpHWA7vi1T7t" 597 | }, 598 | "execution_count": null, 599 | "outputs": [] 600 | }, 601 | { 602 | "cell_type": "code", 603 | "source": [ 604 | "batchSize, learningRate = 128, 1e-4" 605 | ], 606 | "metadata": { 607 | "id": "qHUEWNcBmaBP" 608 | }, 609 | "execution_count": null, 610 | "outputs": [] 611 | }, 612 | { 613 | "cell_type": "markdown", 614 | "source": [ 615 | "## Train Model" 616 | ], 617 | "metadata": { 618 | "id": "JeWyr2yWmZLg" 619 | } 620 | }, 621 | { 622 | "cell_type": "code", 623 | "source": [ 624 | "def trainModel(model, numEpoch, batchSize, learningRate) :\n", 625 | " trainLossList, testLossList = [],[]\n", 626 | " bestLoss = np.inf\n", 627 | "\n", 628 | " for epoch in range(numEpoch) :\n", 629 | " print(f\"[Current Epoch : {epoch + 1}]\")\n", 630 | " trainLoss, testLoss = model.train(xTrain, yTrain, xTest, yTest, batchSize, learningRate)\n", 631 | " trainLossList += trainLoss\n", 632 | " testLossList += testLoss\n", 633 | "\n", 634 | " if testLoss[0] < bestLoss :\n", 635 | " bestLoss = testLoss[0]\n", 636 | " bestWeight = [model.layer1Weights, model.layer2Weights]\n", 637 | "\n", 638 | " return trainLossList, testLossList, bestWeight" 639 | ], 640 | "metadata": { 641 | "id": "2D2R0Dxa2zuq" 642 | }, 643 | "execution_count": null, 644 | "outputs": [] 645 | }, 646 | { 647 | "cell_type": "code", 648 | "source": [ 649 | "trainLossList, testLossList, bestWeight = trainModel(model, numEpoch, batchSize, learningRate)" 650 | ], 651 | "metadata": { 652 | "id": "HP2tzYbmldOc" 653 | }, 654 | "execution_count": null, 655 | "outputs": [] 656 | }, 657 | { 658 | "cell_type": "markdown", 659 | "source": [ 660 | "## Plot Loss Curve" 661 | ], 662 | "metadata": { 663 | "id": "C8z7IiI4_vk4" 664 | } 665 | }, 666 | { 667 | "cell_type": "code", 668 | "source": [ 669 | "plt.figure(figsize=(10, 5))\n", 670 | "plt.plot(np.arange(len(trainLossList)), trainLossList, label = \"Train Loss\")\n", 671 | "plt.plot(np.arange(len(testLossList)), testLossList, label = \"Test Loss\")\n", 672 | "plt.xlabel(\"Epoch\")\n", 673 | "plt.ylabel(\"MSE Loss\")\n", 674 | "plt.title(\"Training Result\")\n", 675 | "plt.legend(loc = \"best\")\n", 676 | "plt.show()" 677 | ], 678 | "metadata": { 679 | "id": "sN66LcOC3OVW" 680 | }, 681 | "execution_count": null, 682 | "outputs": [] 683 | }, 684 | { 685 | "cell_type": "markdown", 686 | "source": [ 687 | "## Get Optimized Parameters" 688 | ], 689 | "metadata": { 690 | "id": "E7DyB6NW_yRw" 691 | } 692 | }, 693 | { 694 | "cell_type": "code", 695 | "source": [ 696 | "min(testLossList)" 697 | ], 698 | "metadata": { 699 | "id": "cbn50gJaAJoR" 700 | }, 701 | "execution_count": null, 702 | "outputs": [] 703 | }, 704 | { 705 | "cell_type": "code", 706 | "source": [ 707 | "print(bestWeight[0])\n", 708 | "print(bestWeight[1])" 709 | ], 710 | "metadata": { 711 | "id": "O9nlgjkJ_rlU" 712 | }, 713 | "execution_count": null, 714 | "outputs": [] 715 | }, 716 | { 717 | "cell_type": "markdown", 718 | "source": [ 719 | "## Inference Result with Trained Neural Network" 720 | ], 721 | "metadata": { 722 | "id": "J0wFo8Wi_1e3" 723 | } 724 | }, 725 | { 726 | "cell_type": "code", 727 | "source": [ 728 | "model.layer1Weights, model.layer2Weights = bestWeight[0], bestWeight[1]" 729 | ], 730 | "metadata": { 731 | "id": "eHsRVCeA_tB2" 732 | }, 733 | "execution_count": null, 734 | "outputs": [] 735 | }, 736 | { 737 | "cell_type": "code", 738 | "source": [ 739 | "yPredNN = []\n", 740 | "\n", 741 | "for subXTest in xTest :\n", 742 | " yPredNN.append(model.predict(subXTest)[0])" 743 | ], 744 | "metadata": { 745 | "id": "Wx_LgjLkASPv" 746 | }, 747 | "execution_count": null, 748 | "outputs": [] 749 | }, 750 | { 751 | "cell_type": "markdown", 752 | "source": [ 753 | "## Inference Result with Linear Regression Model" 754 | ], 755 | "metadata": { 756 | "id": "G_HA5HRkBFN0" 757 | } 758 | }, 759 | { 760 | "cell_type": "code", 761 | "source": [ 762 | "def getParameter(x: np.array, y: np.array) :\n", 763 | " xT = np.transpose(x)\n", 764 | " output = np.matmul(np.matmul(np.linalg.inv(np.matmul(xT, x)), xT), y)\n", 765 | "\n", 766 | " return output" 767 | ], 768 | "metadata": { 769 | "id": "lkTZZxg-BHAW" 770 | }, 771 | "execution_count": null, 772 | "outputs": [] 773 | }, 774 | { 775 | "cell_type": "code", 776 | "source": [ 777 | "betaHat = getParameter(xTrain, yTrain)" 778 | ], 779 | "metadata": { 780 | "id": "_6MgSyVSBSUa" 781 | }, 782 | "execution_count": null, 783 | "outputs": [] 784 | }, 785 | { 786 | "cell_type": "code", 787 | "source": [ 788 | "betaHat" 789 | ], 790 | "metadata": { 791 | "id": "qgWhYnMMBUo7" 792 | }, 793 | "execution_count": null, 794 | "outputs": [] 795 | }, 796 | { 797 | "cell_type": "code", 798 | "source": [ 799 | "yPredLR = np.matmul(xTest, betaHat).tolist()" 800 | ], 801 | "metadata": { 802 | "id": "QAwywSf3BWHV" 803 | }, 804 | "execution_count": null, 805 | "outputs": [] 806 | }, 807 | { 808 | "cell_type": "markdown", 809 | "source": [ 810 | "## Compare Result" 811 | ], 812 | "metadata": { 813 | "id": "vJ0XImuOBq-r" 814 | } 815 | }, 816 | { 817 | "cell_type": "markdown", 818 | "source": [ 819 | "### Compute MSE Loss" 820 | ], 821 | "metadata": { 822 | "id": "IHS6VmFwCaMz" 823 | } 824 | }, 825 | { 826 | "cell_type": "code", 827 | "source": [ 828 | "def MSELoss(yPred:np.array, yTrue:np.array)->float:\n", 829 | " return np.power(yPred-yTrue, 2).mean()" 830 | ], 831 | "metadata": { 832 | "id": "CN4r-nC-CbSY" 833 | }, 834 | "execution_count": null, 835 | "outputs": [] 836 | }, 837 | { 838 | "cell_type": "code", 839 | "source": [ 840 | "print(f\"Neural Network (NN) MSE Loss : {MSELoss(np.array(yPredNN), yTest)}\")\n", 841 | "print(f\"Linear Regression (LR) MSE Loss : {MSELoss(np.array(yPredLR), yTest)}\")" 842 | ], 843 | "metadata": { 844 | "id": "OdPVhb35CmnB" 845 | }, 846 | "execution_count": null, 847 | "outputs": [] 848 | }, 849 | { 850 | "cell_type": "markdown", 851 | "source": [ 852 | "### Plot Bar Chart" 853 | ], 854 | "metadata": { 855 | "id": "gyyBlTMXCY0_" 856 | } 857 | }, 858 | { 859 | "cell_type": "code", 860 | "source": [ 861 | "fig, ax = plt.subplots(figsize = (20, 10))\n", 862 | "idx = np.asarray([i for i in range(50)])\n", 863 | "width = 0.2\n", 864 | "\n", 865 | "ax.bar(idx, yTest[:50], width = width)\n", 866 | "ax.bar(idx+width, yPredNN[:50], width = width)\n", 867 | "ax.bar(idx+2*width, yPredLR[:50], width = width)\n", 868 | "ax.set_xticks(idx)\n", 869 | "ax.legend([\"Ground Truth\", \"NN\", \"LR\"])\n", 870 | "ax.set_xlabel(\"# samples\")\n", 871 | "ax.set_ylabel(\"Value\")\n", 872 | "ax.set_title(\"Result Comparison\")\n", 873 | "\n", 874 | "fig.tight_layout()\n", 875 | "plt.show()" 876 | ], 877 | "metadata": { 878 | "id": "wbfy-HIzBf7D" 879 | }, 880 | "execution_count": null, 881 | "outputs": [] 882 | } 883 | ] 884 | } -------------------------------------------------------------------------------- /2023-2/열유체공학실험1_Week11_PyTorch_Image_Classification.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "view-in-github", 7 | "colab_type": "text" 8 | }, 9 | "source": [ 10 | "\"Open" 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "metadata": { 16 | "id": "QQ97dC_jLHvF" 17 | }, 18 | "source": [ 19 | "# Image Classification Using PyTorch Framework" 20 | ] 21 | }, 22 | { 23 | "cell_type": "markdown", 24 | "metadata": { 25 | "id": "BX6s-j7fq3hA" 26 | }, 27 | "source": [ 28 | "## Check NVIDIA GPU Setting" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": null, 34 | "metadata": { 35 | "id": "D_I7cz1Sq6S1" 36 | }, 37 | "outputs": [], 38 | "source": [ 39 | "!nvidia-smi" 40 | ] 41 | }, 42 | { 43 | "cell_type": "markdown", 44 | "source": [ 45 | "## Mount Google Drive" 46 | ], 47 | "metadata": { 48 | "id": "D2rYJLwSYhb4" 49 | } 50 | }, 51 | { 52 | "cell_type": "code", 53 | "source": [ 54 | "from google.colab import drive\n", 55 | "drive.mount('/content/drive')" 56 | ], 57 | "metadata": { 58 | "id": "pERmU-3xYjmh" 59 | }, 60 | "execution_count": null, 61 | "outputs": [] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "source": [ 66 | "## Unzip Dataset" 67 | ], 68 | "metadata": { 69 | "id": "rOpdIRxMYnQO" 70 | } 71 | }, 72 | { 73 | "cell_type": "code", 74 | "source": [ 75 | "!unzip \"/content/drive/MyDrive/dataset.zip\" # path to dataset" 76 | ], 77 | "metadata": { 78 | "id": "06rK1ga-YrPK" 79 | }, 80 | "execution_count": null, 81 | "outputs": [] 82 | }, 83 | { 84 | "cell_type": "markdown", 85 | "source": [ 86 | "## Get Number of Data" 87 | ], 88 | "metadata": { 89 | "id": "_HjBulAhqmrv" 90 | } 91 | }, 92 | { 93 | "cell_type": "code", 94 | "source": [ 95 | "from os import listdir\n", 96 | "from os.path import join" 97 | ], 98 | "metadata": { 99 | "id": "19RtVbwlqqwg" 100 | }, 101 | "execution_count": null, 102 | "outputs": [] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "source": [ 107 | "sourcePath = \"/content/\" # path to dataset" 108 | ], 109 | "metadata": { 110 | "id": "TZyDYt1crDIG" 111 | }, 112 | "execution_count": null, 113 | "outputs": [] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "source": [ 118 | "print(\"\")\n", 119 | "for className in listdir(join(sourcePath, \"train\")) :\n", 120 | " print(f\"{className}:{len(listdir(join(sourcePath, 'train', className)))}\")" 121 | ], 122 | "metadata": { 123 | "id": "nuLeStY3q_YY" 124 | }, 125 | "execution_count": null, 126 | "outputs": [] 127 | }, 128 | { 129 | "cell_type": "code", 130 | "source": [ 131 | "print(\"\")\n", 132 | "for className in listdir(join(sourcePath, \"test\")) :\n", 133 | " print(f\"{className}:{len(listdir(join(sourcePath, 'test', className)))}\")" 134 | ], 135 | "metadata": { 136 | "id": "_Pim74yHrsJ0" 137 | }, 138 | "execution_count": null, 139 | "outputs": [] 140 | }, 141 | { 142 | "cell_type": "markdown", 143 | "metadata": { 144 | "id": "JXUpV3puOOlX" 145 | }, 146 | "source": [ 147 | "## Create PyTorch DataLoader Class without Data Augmentation" 148 | ] 149 | }, 150 | { 151 | "cell_type": "code", 152 | "execution_count": null, 153 | "metadata": { 154 | "id": "yhWPVY7tOKZc" 155 | }, 156 | "outputs": [], 157 | "source": [ 158 | "from PIL import Image\n", 159 | "\n", 160 | "import torch\n", 161 | "from torch.utils.data import Dataset\n", 162 | "from torchvision import transforms" 163 | ] 164 | }, 165 | { 166 | "cell_type": "code", 167 | "execution_count": null, 168 | "metadata": { 169 | "id": "RTjqdBsVOVUu" 170 | }, 171 | "outputs": [], 172 | "source": [ 173 | "class myDataLoader(Dataset) :\n", 174 | " def __init__(self, opt, forMetric) :\n", 175 | " super(myDataLoader, self).__init__()\n", 176 | "\n", 177 | " self.opt = opt\n", 178 | " self.forMetric = forMetric\n", 179 | " self.imageDataset = self.getPathList()\n", 180 | " self.label = {\"cloudy\":[1,0,0,0],\n", 181 | " \"desert\":[0,1,0,0],\n", 182 | " \"green_area\":[0,0,1,0],\n", 183 | " \"water\":[0,0,0,1]} # One-Hot Encoding\n", 184 | "\n", 185 | " def __getitem__(self, index) :\n", 186 | " image = Image.open(self.imageDataset[0][index]).convert(\"RGB\") #4D to 3D\n", 187 | " image = self.transforms(image)\n", 188 | "\n", 189 | " label = self.label[self.imageDataset[1][index]]\n", 190 | "\n", 191 | " return {\"image\":image, \"label\":torch.as_tensor(label).float()}\n", 192 | "\n", 193 | " def __len__(self) :\n", 194 | " return len(self.imageDataset[1])\n", 195 | "\n", 196 | " def getPathList(self) :\n", 197 | " if self.forMetric :\n", 198 | " classPath = join(self.opt[\"dataRoot\"], \"test\")\n", 199 | " else :\n", 200 | " classPath = join(self.opt[\"dataRoot\"], \"train\")\n", 201 | "\n", 202 | " imagePathList, imageLabelList = [], []\n", 203 | " for className in listdir(classPath) :\n", 204 | " for imageName in listdir(join(classPath, className)) :\n", 205 | " imagePathList.append(join(classPath, className, imageName))\n", 206 | " imageLabelList.append(className)\n", 207 | "\n", 208 | " return (imagePathList, imageLabelList)\n", 209 | "\n", 210 | " def transforms(self, image) :\n", 211 | " if self.forMetric :\n", 212 | " myTransforms = transforms.Compose([transforms.Resize(self.opt[\"cropSize\"]),\n", 213 | " transforms.ToTensor()]) # Resize Process for Test\n", 214 | " else :\n", 215 | " myTransforms = transforms.Compose([transforms.RandomCrop(self.opt[\"cropSize\"]),\n", 216 | " transforms.ToTensor()]) # Random Crop for Training\n", 217 | " image = myTransforms(image)\n", 218 | "\n", 219 | " return image" 220 | ] 221 | }, 222 | { 223 | "cell_type": "markdown", 224 | "metadata": { 225 | "id": "Wz0-THJOPxvx" 226 | }, 227 | "source": [ 228 | "## Create PyTorch Image Classification Model" 229 | ] 230 | }, 231 | { 232 | "cell_type": "code", 233 | "execution_count": null, 234 | "metadata": { 235 | "id": "Lcjb8ky2PvGo" 236 | }, 237 | "outputs": [], 238 | "source": [ 239 | "from torch import nn\n", 240 | "import torch.nn.functional as F" 241 | ] 242 | }, 243 | { 244 | "cell_type": "code", 245 | "execution_count": null, 246 | "metadata": { 247 | "id": "VpMGcu1KP07P" 248 | }, 249 | "outputs": [], 250 | "source": [ 251 | "class myModel(nn.Module) :\n", 252 | " def __init__(self, opt) :\n", 253 | " super(myModel, self).__init__()\n", 254 | "\n", 255 | " inputDim, targetDim, channels = opt[\"inputDim\"], opt[\"targetDim\"], opt[\"channels\"]\n", 256 | "\n", 257 | " self.layer0 = nn.Sequential(nn.Conv2d(inputDim, channels, kernel_size=3, stride=1, padding=1),\n", 258 | " nn.ReLU(),\n", 259 | " nn.MaxPool2d(kernel_size=2, stride=2))\n", 260 | " self.layer1 = nn.Sequential(nn.Conv2d(channels, channels*2, kernel_size=3, stride=1, padding=1),\n", 261 | " nn.ReLU(),\n", 262 | " nn.MaxPool2d(kernel_size=2, stride=2))\n", 263 | " self.layer2 = nn.Sequential(nn.Conv2d(channels*2, channels*4, kernel_size=3, stride=1, padding=1),\n", 264 | " nn.ReLU(),\n", 265 | " nn.MaxPool2d(kernel_size=2, stride=2))\n", 266 | " self.layer3 = nn.Sequential(nn.Conv2d(channels*4, channels*4, kernel_size=3, stride=1, padding=1),\n", 267 | " nn.ReLU(),\n", 268 | " nn.MaxPool2d(kernel_size=2, stride=2))\n", 269 | " self.layer4 = nn.Sequential(nn.Conv2d(channels*4, channels*4, kernel_size=3, stride=1, padding=1),\n", 270 | " nn.ReLU(),\n", 271 | " nn.MaxPool2d(kernel_size=2, stride=2))\n", 272 | " self.layer5 = nn.Sequential(nn.Conv2d(channels*4, channels*4, kernel_size=3, stride=1, padding=1),\n", 273 | " nn.ReLU(),\n", 274 | " nn.MaxPool2d(kernel_size=2, stride=2))\n", 275 | " self.layer6 = nn.Linear(channels*4, targetDim)\n", 276 | "\n", 277 | " def forward(self, input) :\n", 278 | " output = self.layer0(input)\n", 279 | " output = self.layer1(output)\n", 280 | " output = self.layer2(output)\n", 281 | " output = self.layer3(output)\n", 282 | " output = self.layer4(output)\n", 283 | " output = self.layer5(output)\n", 284 | " output = F.adaptive_avg_pool2d(output, (1,1)).view(output.size(0), -1)\n", 285 | " output = self.layer6(output)\n", 286 | "\n", 287 | " return output" 288 | ] 289 | }, 290 | { 291 | "cell_type": "markdown", 292 | "metadata": { 293 | "id": "3Wk4_cBzQU19" 294 | }, 295 | "source": [ 296 | "## Train DL Model" 297 | ] 298 | }, 299 | { 300 | "cell_type": "code", 301 | "execution_count": null, 302 | "metadata": { 303 | "id": "0QJcgBk-QSsg" 304 | }, 305 | "outputs": [], 306 | "source": [ 307 | "from torch.utils.data import DataLoader\n", 308 | "from torch import optim\n", 309 | "\n", 310 | "from torchsummary import summary\n", 311 | "\n", 312 | "from tqdm import tqdm" 313 | ] 314 | }, 315 | { 316 | "cell_type": "markdown", 317 | "metadata": { 318 | "id": "phpTOt47RjcB" 319 | }, 320 | "source": [ 321 | "### Fix Seed" 322 | ] 323 | }, 324 | { 325 | "cell_type": "code", 326 | "execution_count": null, 327 | "metadata": { 328 | "id": "81NJ5JxbRnaU" 329 | }, 330 | "outputs": [], 331 | "source": [ 332 | "import random\n", 333 | "import numpy as np" 334 | ] 335 | }, 336 | { 337 | "cell_type": "code", 338 | "execution_count": null, 339 | "metadata": { 340 | "id": "o8F_Z5d3RlH-" 341 | }, 342 | "outputs": [], 343 | "source": [ 344 | "def fixSeed(seed) :\n", 345 | " random.seed(seed)\n", 346 | " np.random.seed(seed)\n", 347 | " torch.manual_seed(seed)\n", 348 | " torch.cuda.manual_seed(seed)\n", 349 | " torch.cuda.manual_seed_all(seed)\n", 350 | " torch.backends.cudnn.deterministic = True\n", 351 | " torch.backends.cudnn.benchmark = False" 352 | ] 353 | }, 354 | { 355 | "cell_type": "markdown", 356 | "source": [ 357 | "## Create Average Meter Instance" 358 | ], 359 | "metadata": { 360 | "id": "Jh4VCYXSswFY" 361 | } 362 | }, 363 | { 364 | "cell_type": "code", 365 | "source": [ 366 | "class AverageMeter(object):\n", 367 | " def __init__(self):\n", 368 | " self.reset()\n", 369 | "\n", 370 | " def reset(self):\n", 371 | " self.val = 0\n", 372 | " self.avg = 0\n", 373 | " self.sum = 0\n", 374 | " self.count = 0\n", 375 | "\n", 376 | " def update(self, val, n=1):\n", 377 | " self.val = val\n", 378 | " self.sum += val*n\n", 379 | " self.count += n\n", 380 | " self.avg = self.sum / self.count" 381 | ], 382 | "metadata": { 383 | "id": "oGUnMpzusxqw" 384 | }, 385 | "execution_count": null, 386 | "outputs": [] 387 | }, 388 | { 389 | "cell_type": "markdown", 390 | "source": [ 391 | "## Create Accuracy Computation Function" 392 | ], 393 | "metadata": { 394 | "id": "fBn8NVKk8Pqt" 395 | } 396 | }, 397 | { 398 | "cell_type": "code", 399 | "source": [ 400 | "def computeAcc(pred, target) :\n", 401 | " acc = (torch.argmax(pred, dim=1)==torch.argmax(target, dim=1)).sum()/pred.size(0)\n", 402 | "\n", 403 | " return acc" 404 | ], 405 | "metadata": { 406 | "id": "B8qpzXhw8Org" 407 | }, 408 | "execution_count": null, 409 | "outputs": [] 410 | }, 411 | { 412 | "cell_type": "markdown", 413 | "source": [ 414 | "## Training Code as a Function (Abstraction)" 415 | ], 416 | "metadata": { 417 | "id": "lhJ28-fU6Zk7" 418 | } 419 | }, 420 | { 421 | "cell_type": "code", 422 | "source": [ 423 | "def train(opt, myDataLoader, myModel, criterion) :\n", 424 | " fixSeed(opt[\"seed\"])\n", 425 | "\n", 426 | " trainDataLoader = DataLoader(myDataLoader(opt, forMetric=False), batch_size=opt[\"batchSize\"], shuffle=True, drop_last=True)\n", 427 | " testDataLoader = DataLoader(myDataLoader(opt, forMetric=True), batch_size=opt[\"batchSize\"], shuffle=False, drop_last=False)\n", 428 | "\n", 429 | " fixSeed(opt[\"seed\"])\n", 430 | " model = myModel(opt)\n", 431 | " if opt[\"isCUDA\"] :\n", 432 | " model = model.cuda()\n", 433 | "\n", 434 | " summary(model, (opt[\"inputDim\"], opt[\"cropSize\"], opt[\"cropSize\"]))\n", 435 | "\n", 436 | " optimizer = optim.Adam(model.parameters(), lr=opt[\"lr\"])\n", 437 | "\n", 438 | " trainLoss, testLoss = AverageMeter(), AverageMeter()\n", 439 | " trainAcc, testAcc = AverageMeter(), AverageMeter()\n", 440 | " trainLossList, testLossList = [], []\n", 441 | " trainAccList, testAccList = [], []\n", 442 | " bestAcc = 0\n", 443 | "\n", 444 | " for epoch in range(1, opt[\"epochs\"]+1) :\n", 445 | " trainBar = tqdm(trainDataLoader)\n", 446 | " trainLoss.reset(), trainAcc.reset()\n", 447 | "\n", 448 | " for data in trainBar :\n", 449 | " input, target = data[\"image\"], data[\"label\"]\n", 450 | " if opt[\"isCUDA\"] :\n", 451 | " input, target = input.cuda(), target.cuda()\n", 452 | "\n", 453 | " optimizer.zero_grad()\n", 454 | " pred = model(input)\n", 455 | " loss = criterion(pred, target)\n", 456 | " loss.backward()\n", 457 | " optimizer.step()\n", 458 | "\n", 459 | " trainLoss.update(loss.item(), opt[\"batchSize\"])\n", 460 | " trainAcc.update(computeAcc(pred, target).item(), opt[\"batchSize\"])\n", 461 | " trainBar.set_description(desc=f\"[{epoch}/{opt['epochs']}] [Train] < Accuracy:{trainAcc.avg:.6f} | Loss:{trainLoss.avg:.6f} >\")\n", 462 | "\n", 463 | " trainLossList.append(trainLoss.avg)\n", 464 | " trainAccList.append(trainAcc.avg)\n", 465 | "\n", 466 | " testBar = tqdm(testDataLoader)\n", 467 | " testLoss.reset(), testAcc.reset()\n", 468 | "\n", 469 | " for data in testBar :\n", 470 | " input, target = data[\"image\"], data[\"label\"]\n", 471 | " if opt[\"isCUDA\"] :\n", 472 | " input, target = input.cuda(), target.cuda()\n", 473 | "\n", 474 | " model.eval()\n", 475 | " with torch.no_grad() :\n", 476 | " pred = model(input)\n", 477 | " loss = criterion(pred, target)\n", 478 | "\n", 479 | " testLoss.update(loss.item(), opt[\"batchSize\"])\n", 480 | " testAcc.update(computeAcc(pred, target).item(), opt[\"batchSize\"])\n", 481 | " testBar.set_description(desc=f\"[{epoch}/{opt['epochs']}] [Test] < Accuracy:{testAcc.avg:.6f} | Loss:{testLoss.avg:.6f} >\")\n", 482 | "\n", 483 | " testLossList.append(testLoss.avg)\n", 484 | " testAccList.append(testAcc.avg)\n", 485 | "\n", 486 | " if testAcc.avg > bestAcc :\n", 487 | " bestAcc = testAcc.avg\n", 488 | " torch.save(model.state_dict(), \"bestModel.pth\")\n", 489 | "\n", 490 | " torch.save(model.state_dict(), \"latestModel.pth\")\n", 491 | "\n", 492 | " return (trainLossList, testLossList), (trainAccList, testAccList)" 493 | ], 494 | "metadata": { 495 | "id": "Toe0P0WH6c9p" 496 | }, 497 | "execution_count": null, 498 | "outputs": [] 499 | }, 500 | { 501 | "cell_type": "markdown", 502 | "source": [ 503 | "## Create Training Option (Hyperparameter) Dictionary" 504 | ], 505 | "metadata": { 506 | "id": "ttFH7GYJq6c3" 507 | } 508 | }, 509 | { 510 | "cell_type": "code", 511 | "source": [ 512 | "opt = {\"dataRoot\":\"/content/\",\n", 513 | " \"cropSize\":224,\n", 514 | " \"seed\":42,\n", 515 | " \"inputDim\":3,\n", 516 | " \"targetDim\":4,\n", 517 | " \"channels\":64,\n", 518 | " \"batchSize\":16,\n", 519 | " \"lr\":1e-4,\n", 520 | " \"epochs\":10,\n", 521 | " \"isCUDA\":torch.cuda.is_available()}" 522 | ], 523 | "metadata": { 524 | "id": "tXzQHG9aq9xB" 525 | }, 526 | "execution_count": null, 527 | "outputs": [] 528 | }, 529 | { 530 | "cell_type": "markdown", 531 | "source": [ 532 | "## Train Model" 533 | ], 534 | "metadata": { 535 | "id": "VpE3M2q67qRu" 536 | } 537 | }, 538 | { 539 | "cell_type": "code", 540 | "source": [ 541 | "lossList, accList = train(opt, myDataLoader, myModel, nn.CrossEntropyLoss())" 542 | ], 543 | "metadata": { 544 | "id": "yV7Jgs_77c57" 545 | }, 546 | "execution_count": null, 547 | "outputs": [] 548 | }, 549 | { 550 | "cell_type": "markdown", 551 | "metadata": { 552 | "id": "XrgUhJC67uzu" 553 | }, 554 | "source": [ 555 | "## Plot Training vs. Validation Loss Graph" 556 | ] 557 | }, 558 | { 559 | "cell_type": "code", 560 | "source": [ 561 | "import matplotlib.pyplot as plt" 562 | ], 563 | "metadata": { 564 | "id": "_lIUGO11pAtu" 565 | }, 566 | "execution_count": null, 567 | "outputs": [] 568 | }, 569 | { 570 | "cell_type": "code", 571 | "execution_count": null, 572 | "metadata": { 573 | "id": "mLsjFs3K7uzv" 574 | }, 575 | "outputs": [], 576 | "source": [ 577 | "plt.figure(figsize=(20,10))\n", 578 | "\n", 579 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), lossList[0], label=\"Training Loss\")\n", 580 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), lossList[1], label=\"Test Loss\")\n", 581 | "\n", 582 | "plt.xlabel(\"Epoch\")\n", 583 | "plt.ylabel(\"CE Loss\")\n", 584 | "plt.legend(loc=\"best\")\n", 585 | "\n", 586 | "plt.show()" 587 | ] 588 | }, 589 | { 590 | "cell_type": "markdown", 591 | "metadata": { 592 | "id": "CfMgjzvi9dYX" 593 | }, 594 | "source": [ 595 | "## Plot Training vs. Validation Accuracy Graph" 596 | ] 597 | }, 598 | { 599 | "cell_type": "code", 600 | "execution_count": null, 601 | "metadata": { 602 | "id": "a0N1aiik9dYd" 603 | }, 604 | "outputs": [], 605 | "source": [ 606 | "plt.figure(figsize=(20,10))\n", 607 | "\n", 608 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), accList[0], label=\"Training Accuracy\")\n", 609 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), accList[1], label=\"Validation Accuracy\")\n", 610 | "\n", 611 | "plt.xlabel(\"Epoch\")\n", 612 | "plt.ylabel(\"Accuracy\")\n", 613 | "plt.legend(loc=\"best\")\n", 614 | "\n", 615 | "plt.show()" 616 | ] 617 | }, 618 | { 619 | "cell_type": "markdown", 620 | "metadata": { 621 | "id": "4S2Ag-GAuL2P" 622 | }, 623 | "source": [ 624 | "## Image Classification Model with Dropout and BN" 625 | ] 626 | }, 627 | { 628 | "cell_type": "code", 629 | "execution_count": null, 630 | "metadata": { 631 | "id": "YHgSlZKZuL2P" 632 | }, 633 | "outputs": [], 634 | "source": [ 635 | "class myModel(nn.Module) :\n", 636 | " def __init__(self, opt) :\n", 637 | " super(myModel, self).__init__()\n", 638 | "\n", 639 | " inputDim, targetDim, channels = opt[\"inputDim\"], opt[\"targetDim\"], opt[\"channels\"]\n", 640 | "\n", 641 | " self.layer0 = nn.Sequential(nn.Conv2d(inputDim, channels, kernel_size=3, stride=1, padding=1, bias=False),\n", 642 | " nn.BatchNorm2d(channels),\n", 643 | " nn.ReLU(),\n", 644 | " nn.MaxPool2d(kernel_size=2, stride=2))\n", 645 | " self.layer1 = nn.Sequential(nn.Conv2d(channels, channels*2, kernel_size=3, stride=1, padding=1, bias=False),\n", 646 | " nn.BatchNorm2d(channels*2),\n", 647 | " nn.ReLU(),\n", 648 | " nn.MaxPool2d(kernel_size=2, stride=2))\n", 649 | " self.layer2 = nn.Sequential(nn.Conv2d(channels*2, channels*4, kernel_size=3, stride=1, padding=1, bias=False),\n", 650 | " nn.BatchNorm2d(channels*4),\n", 651 | " nn.ReLU(),\n", 652 | " nn.MaxPool2d(kernel_size=2, stride=2))\n", 653 | " self.layer3 = nn.Sequential(nn.Conv2d(channels*4, channels*4, kernel_size=3, stride=1, padding=1, bias=False),\n", 654 | " nn.BatchNorm2d(channels*4),\n", 655 | " nn.ReLU(),\n", 656 | " nn.MaxPool2d(kernel_size=2, stride=2))\n", 657 | " self.layer4 = nn.Sequential(nn.Conv2d(channels*4, channels*4, kernel_size=3, stride=1, padding=1, bias=False),\n", 658 | " nn.BatchNorm2d(channels*4),\n", 659 | " nn.ReLU(),\n", 660 | " nn.MaxPool2d(kernel_size=2, stride=2))\n", 661 | " self.layer5 = nn.Sequential(nn.Conv2d(channels*4, channels*4, kernel_size=3, stride=1, padding=1, bias=False),\n", 662 | " nn.BatchNorm2d(channels*4),\n", 663 | " nn.ReLU(),\n", 664 | " nn.MaxPool2d(kernel_size=2, stride=2))\n", 665 | " self.layer6 = nn.Sequential(nn.Dropout(),\n", 666 | " nn.Linear(channels*4, targetDim))\n", 667 | "\n", 668 | " def forward(self, input) :\n", 669 | " output = self.layer0(input)\n", 670 | " output = self.layer1(output)\n", 671 | " output = self.layer2(output)\n", 672 | " output = self.layer3(output)\n", 673 | " output = self.layer4(output)\n", 674 | " output = self.layer5(output)\n", 675 | " output = F.adaptive_avg_pool2d(output, (1,1)).view(output.size(0), -1)\n", 676 | " output = self.layer6(output)\n", 677 | "\n", 678 | " return output" 679 | ] 680 | }, 681 | { 682 | "cell_type": "code", 683 | "source": [ 684 | "regLossList, regAccList = train(opt, myDataLoader, myModel, nn.CrossEntropyLoss())" 685 | ], 686 | "metadata": { 687 | "id": "M9WmT2-1uwp9" 688 | }, 689 | "execution_count": null, 690 | "outputs": [] 691 | }, 692 | { 693 | "cell_type": "markdown", 694 | "source": [ 695 | "## Vanilla Network vs. Network with Dropout & BN" 696 | ], 697 | "metadata": { 698 | "id": "MeELzQAQvwS7" 699 | } 700 | }, 701 | { 702 | "cell_type": "markdown", 703 | "source": [ 704 | "### Loss Graph" 705 | ], 706 | "metadata": { 707 | "id": "rN6yYbf6BnB5" 708 | } 709 | }, 710 | { 711 | "cell_type": "code", 712 | "execution_count": null, 713 | "metadata": { 714 | "id": "G4e7bh6SvvUh" 715 | }, 716 | "outputs": [], 717 | "source": [ 718 | "plt.figure(figsize=(20,10))\n", 719 | "\n", 720 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), lossList[1], label=\"Vanilla Model Loss\")\n", 721 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), regLossList[1], label=\"Model with Dropout & BN Loss\")\n", 722 | "\n", 723 | "plt.xlabel(\"Epoch\")\n", 724 | "plt.ylabel(\"CE Loss\")\n", 725 | "plt.legend(loc=\"best\")\n", 726 | "\n", 727 | "plt.show()" 728 | ] 729 | }, 730 | { 731 | "cell_type": "markdown", 732 | "metadata": { 733 | "id": "ibN9ugUJvvUi" 734 | }, 735 | "source": [ 736 | "### Accuracy Graph" 737 | ] 738 | }, 739 | { 740 | "cell_type": "code", 741 | "execution_count": null, 742 | "metadata": { 743 | "id": "nbq-zemKvvUi" 744 | }, 745 | "outputs": [], 746 | "source": [ 747 | "plt.figure(figsize=(20,10))\n", 748 | "\n", 749 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), accList[1], label=\"Vanilla Model Loss\")\n", 750 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), regAccList[1], label=\"Model with Dropout & BN Loss\")\n", 751 | "\n", 752 | "plt.xlabel(\"Epoch\")\n", 753 | "plt.ylabel(\"Accuracy\")\n", 754 | "plt.legend(loc=\"best\")\n", 755 | "\n", 756 | "plt.show()" 757 | ] 758 | } 759 | ], 760 | "metadata": { 761 | "accelerator": "GPU", 762 | "colab": { 763 | "provenance": [], 764 | "authorship_tag": "ABX9TyOtpLMkdhS9+UhMOlRhneB/", 765 | "include_colab_link": true 766 | }, 767 | "gpuClass": "standard", 768 | "kernelspec": { 769 | "display_name": "Python 3", 770 | "name": "python3" 771 | }, 772 | "language_info": { 773 | "name": "python" 774 | } 775 | }, 776 | "nbformat": 4, 777 | "nbformat_minor": 0 778 | } -------------------------------------------------------------------------------- /2022-2/열유체공학실험_Week_15.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "view-in-github", 7 | "colab_type": "text" 8 | }, 9 | "source": [ 10 | "\"Open" 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "metadata": { 16 | "id": "FN4UOzUgry82" 17 | }, 18 | "source": [ 19 | "# CFD Analysis with Deep Neural Network (DNN)" 20 | ] 21 | }, 22 | { 23 | "cell_type": "markdown", 24 | "metadata": { 25 | "id": "c2TN2v3MtTI6" 26 | }, 27 | "source": [ 28 | "# Dataset" 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": { 34 | "id": "dTaOdNrgr3e-" 35 | }, 36 | "source": [ 37 | "## Download Dataset" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": null, 43 | "metadata": { 44 | "id": "C5I9LPqCrscm" 45 | }, 46 | "outputs": [], 47 | "source": [ 48 | "!wget https://zenodo.org/record/3666056/files/DeepCFD.zip?download=1" 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": { 54 | "id": "NokjVkvzuJJQ" 55 | }, 56 | "source": [ 57 | "## Unzip Dataset" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": null, 63 | "metadata": { 64 | "colab": { 65 | "background_save": true 66 | }, 67 | "id": "yQfiZHrmr5rI" 68 | }, 69 | "outputs": [], 70 | "source": [ 71 | "!mv \"/content/DeepCFD.zip?download=1\" \"/content/DeepCFD.zip\"" 72 | ] 73 | }, 74 | { 75 | "cell_type": "code", 76 | "execution_count": null, 77 | "metadata": { 78 | "id": "zuq5Hhlbtjpt" 79 | }, 80 | "outputs": [], 81 | "source": [ 82 | "!unzip \"/content/DeepCFD.zip\" -d \"/content\"" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": { 88 | "id": "JuAm7mBQtRfI" 89 | }, 90 | "source": [ 91 | "## Analyze Dataset" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": null, 97 | "metadata": { 98 | "id": "VNnepflZuLRE" 99 | }, 100 | "outputs": [], 101 | "source": [ 102 | "import pickle" 103 | ] 104 | }, 105 | { 106 | "cell_type": "code", 107 | "execution_count": null, 108 | "metadata": { 109 | "id": "AD0FB_AOuL3v" 110 | }, 111 | "outputs": [], 112 | "source": [ 113 | "x = pickle.load(open(\"/content/dataX.pkl\", \"rb\"))\n", 114 | "y = pickle.load(open(\"/content/dataY.pkl\", \"rb\"))" 115 | ] 116 | }, 117 | { 118 | "cell_type": "code", 119 | "execution_count": null, 120 | "metadata": { 121 | "id": "ioOYEsKTv3z2" 122 | }, 123 | "outputs": [], 124 | "source": [ 125 | "type(x)" 126 | ] 127 | }, 128 | { 129 | "cell_type": "code", 130 | "execution_count": null, 131 | "metadata": { 132 | "id": "J-cKuh1guY9i" 133 | }, 134 | "outputs": [], 135 | "source": [ 136 | "x.shape" 137 | ] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "execution_count": null, 142 | "metadata": { 143 | "id": "paDz4sqQuaS9" 144 | }, 145 | "outputs": [], 146 | "source": [ 147 | "y.shape" 148 | ] 149 | }, 150 | { 151 | "cell_type": "markdown", 152 | "metadata": { 153 | "id": "onqc7M7YudZ7" 154 | }, 155 | "source": [ 156 | "### Preprocess Dataset" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "metadata": { 163 | "id": "1CJdWPZCupkO" 164 | }, 165 | "outputs": [], 166 | "source": [ 167 | "import numpy as np" 168 | ] 169 | }, 170 | { 171 | "cell_type": "code", 172 | "execution_count": null, 173 | "metadata": { 174 | "id": "m4bEDDabub9e" 175 | }, 176 | "outputs": [], 177 | "source": [ 178 | "x = np.transpose(x, (0, 3, 2, 1))\n", 179 | "y = np.transpose(y, (0, 3, 2, 1))" 180 | ] 181 | }, 182 | { 183 | "cell_type": "code", 184 | "execution_count": null, 185 | "metadata": { 186 | "id": "ZuCzzK1HuoRs" 187 | }, 188 | "outputs": [], 189 | "source": [ 190 | "x.shape" 191 | ] 192 | }, 193 | { 194 | "cell_type": "code", 195 | "execution_count": null, 196 | "metadata": { 197 | "id": "CoYFU60furJN" 198 | }, 199 | "outputs": [], 200 | "source": [ 201 | "y.shape" 202 | ] 203 | }, 204 | { 205 | "cell_type": "markdown", 206 | "metadata": { 207 | "id": "BX81p99Bu6lR" 208 | }, 209 | "source": [ 210 | "## Visualize Dataset" 211 | ] 212 | }, 213 | { 214 | "cell_type": "code", 215 | "execution_count": null, 216 | "metadata": { 217 | "id": "6fDfhCBlu5Sf" 218 | }, 219 | "outputs": [], 220 | "source": [ 221 | "import matplotlib.pyplot as plt" 222 | ] 223 | }, 224 | { 225 | "cell_type": "markdown", 226 | "metadata": { 227 | "id": "Bz17gJ1mvzaF" 228 | }, 229 | "source": [ 230 | "### Input Data" 231 | ] 232 | }, 233 | { 234 | "cell_type": "code", 235 | "execution_count": null, 236 | "metadata": { 237 | "id": "N0k5EdgRu9Lw" 238 | }, 239 | "outputs": [], 240 | "source": [ 241 | "plt.figure(figsize=(30, 60))\n", 242 | "\n", 243 | "plt.subplot(131)\n", 244 | "plt.imshow(x[0,:,:,0])\n", 245 | "\n", 246 | "plt.subplot(132)\n", 247 | "plt.imshow(x[0,:,:,1])\n", 248 | "\n", 249 | "plt.subplot(133)\n", 250 | "plt.imshow(x[0,:,:,2])\n", 251 | "\n", 252 | "plt.show()" 253 | ] 254 | }, 255 | { 256 | "cell_type": "markdown", 257 | "metadata": { 258 | "id": "6i-j5L55v0hj" 259 | }, 260 | "source": [ 261 | "### Target Data" 262 | ] 263 | }, 264 | { 265 | "cell_type": "code", 266 | "execution_count": null, 267 | "metadata": { 268 | "id": "yDD41vayvDIK" 269 | }, 270 | "outputs": [], 271 | "source": [ 272 | "plt.figure(figsize=(30, 60))\n", 273 | "\n", 274 | "plt.subplot(131)\n", 275 | "plt.imshow(y[0,:,:,0])\n", 276 | "\n", 277 | "plt.subplot(132)\n", 278 | "plt.imshow(y[0,:,:,1])\n", 279 | "\n", 280 | "plt.subplot(133)\n", 281 | "plt.imshow(y[0,:,:,2])\n", 282 | "\n", 283 | "plt.show()" 284 | ] 285 | }, 286 | { 287 | "cell_type": "markdown", 288 | "metadata": { 289 | "id": "_CLNFhyZ2W4l" 290 | }, 291 | "source": [ 292 | "### Input Data" 293 | ] 294 | }, 295 | { 296 | "cell_type": "code", 297 | "execution_count": null, 298 | "metadata": { 299 | "id": "EdMMKvNJ2W4m" 300 | }, 301 | "outputs": [], 302 | "source": [ 303 | "plt.figure(figsize=(30, 60))\n", 304 | "\n", 305 | "plt.subplot(131)\n", 306 | "plt.imshow(x[-1,:,:,0])\n", 307 | "\n", 308 | "plt.subplot(132)\n", 309 | "plt.imshow(x[-1,:,:,1])\n", 310 | "\n", 311 | "plt.subplot(133)\n", 312 | "plt.imshow(x[-1,:,:,2])\n", 313 | "\n", 314 | "plt.show()" 315 | ] 316 | }, 317 | { 318 | "cell_type": "markdown", 319 | "metadata": { 320 | "id": "TM6v2Ku_2W4m" 321 | }, 322 | "source": [ 323 | "### Target Data" 324 | ] 325 | }, 326 | { 327 | "cell_type": "code", 328 | "execution_count": null, 329 | "metadata": { 330 | "id": "juWjjSKX2W4m" 331 | }, 332 | "outputs": [], 333 | "source": [ 334 | "plt.figure(figsize=(30, 60))\n", 335 | "\n", 336 | "plt.subplot(131)\n", 337 | "plt.imshow(y[-1,:,:,0])\n", 338 | "\n", 339 | "plt.subplot(132)\n", 340 | "plt.imshow(y[-1,:,:,1])\n", 341 | "\n", 342 | "plt.subplot(133)\n", 343 | "plt.imshow(y[-1,:,:,2])\n", 344 | "\n", 345 | "plt.show()" 346 | ] 347 | }, 348 | { 349 | "cell_type": "markdown", 350 | "metadata": { 351 | "id": "xQxpUU5ezI6h" 352 | }, 353 | "source": [ 354 | "### Split Dataset" 355 | ] 356 | }, 357 | { 358 | "cell_type": "code", 359 | "execution_count": null, 360 | "metadata": { 361 | "id": "6JdGU5bfzN9F" 362 | }, 363 | "outputs": [], 364 | "source": [ 365 | "from sklearn.model_selection import train_test_split" 366 | ] 367 | }, 368 | { 369 | "cell_type": "code", 370 | "execution_count": null, 371 | "metadata": { 372 | "id": "HWq43gk7zJ2k" 373 | }, 374 | "outputs": [], 375 | "source": [ 376 | "xTrain, xTest, yTrain, yTest = train_test_split(x, y, test_size=0.1, random_state=42)" 377 | ] 378 | }, 379 | { 380 | "cell_type": "code", 381 | "execution_count": null, 382 | "metadata": { 383 | "id": "0vjjzyGF6DVT" 384 | }, 385 | "outputs": [], 386 | "source": [ 387 | "xTrain, xValid, yTrain, yValid = train_test_split(xTrain, yTrain, test_size=0.1, random_state=42)" 388 | ] 389 | }, 390 | { 391 | "cell_type": "code", 392 | "execution_count": null, 393 | "metadata": { 394 | "id": "KfWMkmGMzXBM" 395 | }, 396 | "outputs": [], 397 | "source": [ 398 | "xTrain.shape" 399 | ] 400 | }, 401 | { 402 | "cell_type": "code", 403 | "execution_count": null, 404 | "metadata": { 405 | "id": "7ouabwvLzZNH" 406 | }, 407 | "outputs": [], 408 | "source": [ 409 | "xTest.shape" 410 | ] 411 | }, 412 | { 413 | "cell_type": "code", 414 | "execution_count": null, 415 | "metadata": { 416 | "id": "P80L6_2dzanC" 417 | }, 418 | "outputs": [], 419 | "source": [ 420 | "yTrain.shape" 421 | ] 422 | }, 423 | { 424 | "cell_type": "code", 425 | "execution_count": null, 426 | "metadata": { 427 | "id": "5_z9j4aWzcTF" 428 | }, 429 | "outputs": [], 430 | "source": [ 431 | "yTest.shape" 432 | ] 433 | }, 434 | { 435 | "cell_type": "markdown", 436 | "metadata": { 437 | "id": "x7vcCs95wMPF" 438 | }, 439 | "source": [ 440 | "## Build Convolutional Neural Network (CNN)" 441 | ] 442 | }, 443 | { 444 | "cell_type": "markdown", 445 | "metadata": { 446 | "id": "CXOvsqoUwt6r" 447 | }, 448 | "source": [ 449 | "### Import Keras" 450 | ] 451 | }, 452 | { 453 | "cell_type": "code", 454 | "execution_count": null, 455 | "metadata": { 456 | "id": "Jn4oIhJzv_dF" 457 | }, 458 | "outputs": [], 459 | "source": [ 460 | "import keras\n", 461 | "from keras.models import Model, load_model\n", 462 | "from keras.layers import Input, concatenate, Resizing\n", 463 | "from keras.layers.core import Dropout, Lambda\n", 464 | "from keras.layers.convolutional import Conv2D, Conv2DTranspose\n", 465 | "from keras.layers.pooling import MaxPooling2D" 466 | ] 467 | }, 468 | { 469 | "cell_type": "markdown", 470 | "metadata": { 471 | "id": "s74y2-gswfsW" 472 | }, 473 | "source": [ 474 | "### Fix Seed" 475 | ] 476 | }, 477 | { 478 | "cell_type": "code", 479 | "execution_count": null, 480 | "metadata": { 481 | "id": "k66f2GJZwg6Z" 482 | }, 483 | "outputs": [], 484 | "source": [ 485 | "import tensorflow as tf\n", 486 | "from keras import backend as K\n", 487 | "import random\n", 488 | "\n", 489 | "def fixSeed(numSeed = 42) :\n", 490 | " np.random.seed(numSeed)\n", 491 | " random.seed(numSeed)\n", 492 | " tf.random.set_seed(numSeed)\n", 493 | "\n", 494 | " sessionConf = tf.compat.v1.ConfigProto(intra_op_parallelism_threads=1, inter_op_parallelism_threads=1)\n", 495 | " sess = tf.compat.v1.Session(graph=tf.compat.v1.get_default_graph(), config=sessionConf)\n", 496 | " K.set_session(sess)" 497 | ] 498 | }, 499 | { 500 | "cell_type": "markdown", 501 | "metadata": { 502 | "id": "xqwhdbovw-3u" 503 | }, 504 | "source": [ 505 | "### Build U-Net Network" 506 | ] 507 | }, 508 | { 509 | "cell_type": "code", 510 | "execution_count": null, 511 | "metadata": { 512 | "id": "SKDKZlEJxBVz" 513 | }, 514 | "outputs": [], 515 | "source": [ 516 | "inputs = Input((x.shape[1], x.shape[2], x.shape[3]))\n", 517 | "\n", 518 | "c1 = Conv2D(16, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(inputs)\n", 519 | "c1 = Dropout(0.1)(c1)\n", 520 | "c1 = Conv2D(16, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(c1)\n", 521 | "p1 = MaxPooling2D((2, 2))(c1)\n", 522 | "\n", 523 | "c2 = Conv2D(32, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(p1)\n", 524 | "c2 = Dropout(0.1)(c2)\n", 525 | "c2 = Conv2D(32, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(c2)\n", 526 | "p2 = MaxPooling2D((2, 2))(c2)\n", 527 | "\n", 528 | "c3 = Conv2D(64, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(p2)\n", 529 | "c3 = Dropout(0.2)(c3)\n", 530 | "c3 = Conv2D(64, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(c3)\n", 531 | "p3 = MaxPooling2D((2, 2))(c3)\n", 532 | "\n", 533 | "c4 = Conv2D(128, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(p3)\n", 534 | "c4 = Dropout(0.2)(c4)\n", 535 | "c4 = Conv2D(128, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(c4)\n", 536 | "p4 = MaxPooling2D(pool_size=(2, 2))(c4)\n", 537 | "\n", 538 | "c5 = Conv2D(256, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(p4)\n", 539 | "c5 = Dropout(0.3)(c5)\n", 540 | "c5 = Conv2D(256, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(c5)\n", 541 | "\n", 542 | "u6 = Resizing(c4.shape[1], c4.shape[2], \"nearest\")(c5)\n", 543 | "u6 = Conv2D(128, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(u6)\n", 544 | "u6 = concatenate([u6, c4])\n", 545 | "c6 = Conv2D(128, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(u6)\n", 546 | "c6 = Dropout(0.2)(c6)\n", 547 | "c6 = Conv2D(128, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(c6)\n", 548 | "\n", 549 | "u7 = Resizing(c3.shape[1], c3.shape[2], \"nearest\")(c6)\n", 550 | "u7 = Conv2D(64, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(u7)\n", 551 | "u7 = concatenate([u7, c3])\n", 552 | "c7 = Conv2D(64, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(u7)\n", 553 | "c7 = Dropout(0.2)(c7)\n", 554 | "c7 = Conv2D(64, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(c7)\n", 555 | "\n", 556 | "u8 = Resizing(c2.shape[1], c2.shape[2], \"nearest\")(c7)\n", 557 | "u8 = Conv2D(32, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(u8)\n", 558 | "u8 = concatenate([u8, c2])\n", 559 | "c8 = Conv2D(32, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(u8)\n", 560 | "c8 = Dropout(0.1)(c8)\n", 561 | "c8 = Conv2D(32, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(c8)\n", 562 | "\n", 563 | "u9 = Resizing(c1.shape[1], c1.shape[2], \"nearest\")(c8)\n", 564 | "u8 = Conv2D(16, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(u9)\n", 565 | "u9 = concatenate([u9, c1], axis=3)\n", 566 | "c9 = Conv2D(16, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(u9)\n", 567 | "c9 = Dropout(0.1)(c9)\n", 568 | "c9 = Conv2D(16, (3, 3), activation=\"elu\", kernel_initializer=\"he_normal\", padding=\"same\")(c9)\n", 569 | "\n", 570 | "outputs = Conv2D(y.shape[-1], (1, 1))(c9)" 571 | ] 572 | }, 573 | { 574 | "cell_type": "markdown", 575 | "metadata": { 576 | "id": "iwX36XsMypXh" 577 | }, 578 | "source": [ 579 | "### Create Model Instance" 580 | ] 581 | }, 582 | { 583 | "cell_type": "code", 584 | "execution_count": null, 585 | "metadata": { 586 | "id": "nE7H9BxdxTW-" 587 | }, 588 | "outputs": [], 589 | "source": [ 590 | "K.clear_session()\n", 591 | "fixSeed()\n", 592 | "\n", 593 | "model = Model(inputs=[inputs], outputs=[outputs])" 594 | ] 595 | }, 596 | { 597 | "cell_type": "markdown", 598 | "metadata": { 599 | "id": "n_fb9wQMyv_D" 600 | }, 601 | "source": [ 602 | "### Summarize Model" 603 | ] 604 | }, 605 | { 606 | "cell_type": "code", 607 | "execution_count": null, 608 | "metadata": { 609 | "id": "hEhqapfFyvPo" 610 | }, 611 | "outputs": [], 612 | "source": [ 613 | "model.summary()" 614 | ] 615 | }, 616 | { 617 | "cell_type": "markdown", 618 | "metadata": { 619 | "id": "TgK-92TO2gaP" 620 | }, 621 | "source": [ 622 | "### Visualize Model" 623 | ] 624 | }, 625 | { 626 | "cell_type": "code", 627 | "execution_count": null, 628 | "metadata": { 629 | "id": "7kt43qqs2iAk" 630 | }, 631 | "outputs": [], 632 | "source": [ 633 | "from keras.utils import plot_model\n", 634 | "plot_model(model, to_file=\"model.png\")" 635 | ] 636 | }, 637 | { 638 | "cell_type": "markdown", 639 | "metadata": { 640 | "id": "EhFgsIaLy1L_" 641 | }, 642 | "source": [ 643 | "### Compile Model" 644 | ] 645 | }, 646 | { 647 | "cell_type": "code", 648 | "execution_count": null, 649 | "metadata": { 650 | "id": "Dk-yuBe6yxsx" 651 | }, 652 | "outputs": [], 653 | "source": [ 654 | "lr, batchSize, epoch = 1e-3, 8, 50" 655 | ] 656 | }, 657 | { 658 | "cell_type": "code", 659 | "execution_count": null, 660 | "metadata": { 661 | "id": "FXELb3pVy6wJ" 662 | }, 663 | "outputs": [], 664 | "source": [ 665 | "model.compile(loss=\"mean_absolute_error\", optimizer=keras.optimizers.Adam(learning_rate=lr))" 666 | ] 667 | }, 668 | { 669 | "cell_type": "markdown", 670 | "metadata": { 671 | "id": "eLDqfZg8zf1m" 672 | }, 673 | "source": [ 674 | "### Train Model" 675 | ] 676 | }, 677 | { 678 | "cell_type": "code", 679 | "execution_count": null, 680 | "metadata": { 681 | "id": "6uKqTI9Cy9M9" 682 | }, 683 | "outputs": [], 684 | "source": [ 685 | "history = model.fit(xTrain, yTrain, batch_size=batchSize, epochs=epoch, validation_data=(xValid, yValid))" 686 | ] 687 | }, 688 | { 689 | "cell_type": "markdown", 690 | "metadata": { 691 | "id": "4eMlV-Fd0ivc" 692 | }, 693 | "source": [ 694 | "### Visualize Training Procedure" 695 | ] 696 | }, 697 | { 698 | "cell_type": "code", 699 | "execution_count": null, 700 | "metadata": { 701 | "id": "7gYbTp3zzm4T" 702 | }, 703 | "outputs": [], 704 | "source": [ 705 | "plt.subplots(figsize = (20,5))\n", 706 | "plt.plot(np.arange(epoch), history.history[\"loss\"], label=\"Training MAE Loss\")\n", 707 | "plt.plot(np.arange(epoch), history.history[\"val_loss\"], label=\"Validation MAE Loss\")\n", 708 | "plt.xlabel(\"# Epoch\")\n", 709 | "plt.ylabel(\"MAE Loss\")\n", 710 | "plt.title(\"Loss\")\n", 711 | "plt.legend(loc=\"best\")\n", 712 | "plt.show()" 713 | ] 714 | }, 715 | { 716 | "cell_type": "markdown", 717 | "metadata": { 718 | "id": "GOtivnsm0pju" 719 | }, 720 | "source": [ 721 | "### Inference Result" 722 | ] 723 | }, 724 | { 725 | "cell_type": "code", 726 | "execution_count": null, 727 | "metadata": { 728 | "id": "8Lnf_G7J0mOB" 729 | }, 730 | "outputs": [], 731 | "source": [ 732 | "yPred = model.predict(xTest)" 733 | ] 734 | }, 735 | { 736 | "cell_type": "code", 737 | "execution_count": null, 738 | "metadata": { 739 | "id": "gJJURjcP0rlX" 740 | }, 741 | "outputs": [], 742 | "source": [ 743 | "yPred.shape" 744 | ] 745 | }, 746 | { 747 | "cell_type": "markdown", 748 | "metadata": { 749 | "id": "N16lM-m51l2w" 750 | }, 751 | "source": [ 752 | "### Reshape Array" 753 | ] 754 | }, 755 | { 756 | "cell_type": "code", 757 | "execution_count": null, 758 | "metadata": { 759 | "id": "N92FA6uD1pW5" 760 | }, 761 | "outputs": [], 762 | "source": [ 763 | "yTest = np.transpose(yTest, (0, 3, 2, 1))\n", 764 | "yPred = np.transpose(yPred, (0, 3, 2, 1))" 765 | ] 766 | }, 767 | { 768 | "cell_type": "code", 769 | "execution_count": null, 770 | "metadata": { 771 | "id": "4voAuMRg1v9I" 772 | }, 773 | "outputs": [], 774 | "source": [ 775 | "yTest.shape" 776 | ] 777 | }, 778 | { 779 | "cell_type": "code", 780 | "execution_count": null, 781 | "metadata": { 782 | "id": "B04XIs7y1xJ8" 783 | }, 784 | "outputs": [], 785 | "source": [ 786 | "yPred.shape" 787 | ] 788 | }, 789 | { 790 | "cell_type": "markdown", 791 | "metadata": { 792 | "id": "TM8N_Y4U0xCy" 793 | }, 794 | "source": [ 795 | "### Visualize Result" 796 | ] 797 | }, 798 | { 799 | "cell_type": "code", 800 | "execution_count": null, 801 | "metadata": { 802 | "id": "qUfl7qC30s9S" 803 | }, 804 | "outputs": [], 805 | "source": [ 806 | "def visualize(sample_y, out_y, error, s) :\n", 807 | " minu = np.min(sample_y[s, 0, :, :])\n", 808 | " maxu = np.max(sample_y[s, 0, :, :])\n", 809 | " \n", 810 | " minv = np.min(sample_y[s, 1, :, :])\n", 811 | " maxv = np.max(sample_y[s, 1, :, :])\n", 812 | " \n", 813 | " minp = np.min(sample_y[s, 2, :, :])\n", 814 | " maxp = np.max(sample_y[s, 2, :, :])\n", 815 | " \n", 816 | " mineu = np.min(error[s, 0, :, :])\n", 817 | " maxeu = np.max(error[s, 0, :, :])\n", 818 | " \n", 819 | " minev = np.min(error[s, 1, :, :])\n", 820 | " maxev = np.max(error[s, 1, :, :])\n", 821 | " \n", 822 | " minep = np.min(error[s, 2, :, :])\n", 823 | " maxep = np.max(error[s, 2, :, :])\n", 824 | " \n", 825 | " plt.figure()\n", 826 | " fig = plt.gcf()\n", 827 | " fig.set_size_inches(15, 10)\n", 828 | " plt.subplot(3, 3, 1)\n", 829 | " plt.title(\"CFD\", fontsize=18)\n", 830 | " plt.imshow(np.transpose(sample_y[s, 0, :, :]), cmap=\"jet\", vmin = minu, vmax = maxu, origin=\"lower\", extent=[0,260,0,120])\n", 831 | " plt.colorbar(orientation=\"horizontal\")\n", 832 | " plt.ylabel(\"Ux\", fontsize=18)\n", 833 | " plt.subplot(3, 3, 2)\n", 834 | " plt.title(\"CNN\", fontsize=18)\n", 835 | " plt.imshow(np.transpose(out_y[s, 0, :, :]), cmap=\"jet\", vmin = minu, vmax = maxu, origin=\"lower\", extent=[0,260,0,120])\n", 836 | " plt.colorbar(orientation=\"horizontal\")\n", 837 | " plt.subplot(3, 3, 3)\n", 838 | " plt.title(\"Error\", fontsize=18)\n", 839 | " plt.imshow(np.transpose(error[s, 0, :, :]), cmap=\"jet\", vmin = mineu, vmax = maxeu, origin=\"lower\", extent=[0,260,0,120])\n", 840 | " plt.colorbar(orientation=\"horizontal\")\n", 841 | "\n", 842 | " plt.subplot(3, 3, 4)\n", 843 | " plt.imshow(np.transpose(sample_y[s, 1, :, :]), cmap=\"jet\", vmin = minv, vmax = maxv, origin=\"lower\", extent=[0,260,0,120])\n", 844 | " plt.colorbar(orientation=\"horizontal\")\n", 845 | " plt.ylabel(\"Uy\", fontsize=18)\n", 846 | " plt.subplot(3, 3, 5)\n", 847 | " plt.imshow(np.transpose(out_y[s, 1, :, :]), cmap=\"jet\", vmin = minv, vmax = maxv, origin=\"lower\", extent=[0,260,0,120])\n", 848 | " plt.colorbar(orientation=\"horizontal\")\n", 849 | " plt.subplot(3, 3, 6)\n", 850 | " plt.imshow(np.transpose(error[s, 1, :, :]), cmap=\"jet\", vmin = minev, vmax = maxev, origin=\"lower\", extent=[0,260,0,120])\n", 851 | " plt.colorbar(orientation=\"horizontal\")\n", 852 | "\n", 853 | " plt.subplot(3, 3, 7)\n", 854 | " plt.imshow(np.transpose(sample_y[s, 2, :, :]), cmap=\"jet\", vmin = minp, vmax = maxp, origin=\"lower\", extent=[0,260,0,120])\n", 855 | " plt.colorbar(orientation=\"horizontal\")\n", 856 | " plt.ylabel(\"p\", fontsize=18)\n", 857 | " plt.subplot(3, 3, 8)\n", 858 | " plt.imshow(np.transpose(out_y[s, 2, :, :]), cmap=\"jet\", vmin = minp, vmax = maxp, origin=\"lower\", extent=[0,260,0,120])\n", 859 | " plt.colorbar(orientation=\"horizontal\")\n", 860 | " plt.subplot(3, 3, 9)\n", 861 | " plt.imshow(np.transpose(error[s, 2, :, :]), cmap=\"jet\", vmin = minep, vmax = maxep, origin=\"lower\", extent=[0,260,0,120])\n", 862 | " plt.colorbar(orientation=\"horizontal\")\n", 863 | " plt.tight_layout()\n", 864 | " plt.show()" 865 | ] 866 | }, 867 | { 868 | "cell_type": "code", 869 | "execution_count": null, 870 | "metadata": { 871 | "id": "ECLnz7yn0430" 872 | }, 873 | "outputs": [], 874 | "source": [ 875 | "error = np.abs(yPred-yTest)" 876 | ] 877 | }, 878 | { 879 | "cell_type": "code", 880 | "execution_count": null, 881 | "metadata": { 882 | "id": "CHmdEuHs0-ai" 883 | }, 884 | "outputs": [], 885 | "source": [ 886 | "visualize(yTest, yPred, error, 0)" 887 | ] 888 | }, 889 | { 890 | "cell_type": "code", 891 | "execution_count": null, 892 | "metadata": { 893 | "id": "Ibrg4zgn1KfJ" 894 | }, 895 | "outputs": [], 896 | "source": [ 897 | "visualize(yTest, yPred, error, 30)" 898 | ] 899 | }, 900 | { 901 | "cell_type": "code", 902 | "execution_count": null, 903 | "metadata": { 904 | "id": "dCvng6H91LBw" 905 | }, 906 | "outputs": [], 907 | "source": [ 908 | "visualize(yTest, yPred, error, -1)" 909 | ] 910 | } 911 | ], 912 | "metadata": { 913 | "accelerator": "GPU", 914 | "colab": { 915 | "provenance": [], 916 | "include_colab_link": true 917 | }, 918 | "gpuClass": "standard", 919 | "kernelspec": { 920 | "display_name": "Python 3", 921 | "name": "python3" 922 | }, 923 | "language_info": { 924 | "name": "python" 925 | } 926 | }, 927 | "nbformat": 4, 928 | "nbformat_minor": 0 929 | } -------------------------------------------------------------------------------- /2023-2/열유체공학실험1_Week10_PyTorch_Binary_Classification.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "view-in-github", 7 | "colab_type": "text" 8 | }, 9 | "source": [ 10 | "\"Open" 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "metadata": { 16 | "id": "QQ97dC_jLHvF" 17 | }, 18 | "source": [ 19 | "# Binary Classification Using PyTorch Framework" 20 | ] 21 | }, 22 | { 23 | "cell_type": "markdown", 24 | "metadata": { 25 | "id": "BX6s-j7fq3hA" 26 | }, 27 | "source": [ 28 | "## Check NVIDIA GPU Setting" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": null, 34 | "metadata": { 35 | "id": "D_I7cz1Sq6S1" 36 | }, 37 | "outputs": [], 38 | "source": [ 39 | "!nvidia-smi" 40 | ] 41 | }, 42 | { 43 | "cell_type": "markdown", 44 | "metadata": { 45 | "id": "e9G6xRWOLRDP" 46 | }, 47 | "source": [ 48 | "## Load Dataset" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": null, 54 | "metadata": { 55 | "id": "Ult3kyAaLFPo" 56 | }, 57 | "outputs": [], 58 | "source": [ 59 | "import numpy as np\n", 60 | "import pandas as pd\n", 61 | "import matplotlib.pyplot as plt" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": null, 67 | "metadata": { 68 | "id": "wRo265QfLvCz" 69 | }, 70 | "outputs": [], 71 | "source": [ 72 | "df = pd.read_csv(\"/content/smoking.csv\")" 73 | ] 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": { 78 | "id": "2-Ycu33nLzHi" 79 | }, 80 | "source": [ 81 | "### View DataFrame" 82 | ] 83 | }, 84 | { 85 | "cell_type": "code", 86 | "execution_count": null, 87 | "metadata": { 88 | "id": "aos3Fo5cLyWZ" 89 | }, 90 | "outputs": [], 91 | "source": [ 92 | "df.head(10)" 93 | ] 94 | }, 95 | { 96 | "cell_type": "markdown", 97 | "source": [ 98 | "## Get Number of Data" 99 | ], 100 | "metadata": { 101 | "id": "d9ynzMU56gaU" 102 | } 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": null, 107 | "metadata": { 108 | "id": "QlerJUNzMx8_" 109 | }, 110 | "outputs": [], 111 | "source": [ 112 | "df.shape" 113 | ] 114 | }, 115 | { 116 | "cell_type": "markdown", 117 | "metadata": { 118 | "id": "oOzaz_jFMmGF" 119 | }, 120 | "source": [ 121 | "### Get Input and Target Data" 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "execution_count": null, 127 | "metadata": { 128 | "id": "mu_5mt8-L1TX" 129 | }, 130 | "outputs": [], 131 | "source": [ 132 | "inputData, targetData = df.drop(columns=[\"smoking\"], axis=1), df[\"smoking\"]" 133 | ] 134 | }, 135 | { 136 | "cell_type": "markdown", 137 | "metadata": { 138 | "id": "CDrFJPifMpPW" 139 | }, 140 | "source": [ 141 | "### Get Input Feature Names" 142 | ] 143 | }, 144 | { 145 | "cell_type": "code", 146 | "execution_count": null, 147 | "metadata": { 148 | "id": "nagAy9abMi2k" 149 | }, 150 | "outputs": [], 151 | "source": [ 152 | "inputData.columns" 153 | ] 154 | }, 155 | { 156 | "cell_type": "markdown", 157 | "metadata": { 158 | "id": "XscKeIwAM9jT" 159 | }, 160 | "source": [ 161 | "### Get Data Type of Input Features" 162 | ] 163 | }, 164 | { 165 | "cell_type": "code", 166 | "execution_count": null, 167 | "metadata": { 168 | "id": "u8A5rsbTM_Rj" 169 | }, 170 | "outputs": [], 171 | "source": [ 172 | "inputData.info()" 173 | ] 174 | }, 175 | { 176 | "cell_type": "markdown", 177 | "source": [ 178 | "## Show Target Data" 179 | ], 180 | "metadata": { 181 | "id": "WpUJ3DRsxSbG" 182 | } 183 | }, 184 | { 185 | "cell_type": "code", 186 | "source": [ 187 | "targetData" 188 | ], 189 | "metadata": { 190 | "id": "1ZTPJIWHxUYd" 191 | }, 192 | "execution_count": null, 193 | "outputs": [] 194 | }, 195 | { 196 | "cell_type": "markdown", 197 | "metadata": { 198 | "id": "WkolzrgvNl_X" 199 | }, 200 | "source": [ 201 | "## Preprocessing" 202 | ] 203 | }, 204 | { 205 | "cell_type": "markdown", 206 | "source": [ 207 | "## One-Hot Encoding" 208 | ], 209 | "metadata": { 210 | "id": "plnqX-sB6rgP" 211 | } 212 | }, 213 | { 214 | "cell_type": "code", 215 | "source": [ 216 | "inputDataOHE = inputData.loc[:, [\"gender\",\"oral\",\"tartar\"]]" 217 | ], 218 | "metadata": { 219 | "id": "lws3xaV26s4L" 220 | }, 221 | "execution_count": null, 222 | "outputs": [] 223 | }, 224 | { 225 | "cell_type": "code", 226 | "source": [ 227 | "inputDataOHE" 228 | ], 229 | "metadata": { 230 | "id": "0aD_ULIX68BF" 231 | }, 232 | "execution_count": null, 233 | "outputs": [] 234 | }, 235 | { 236 | "cell_type": "code", 237 | "source": [ 238 | "inputDataOHE = pd.get_dummies(inputDataOHE)" 239 | ], 240 | "metadata": { 241 | "id": "y5fsS9ZO69zv" 242 | }, 243 | "execution_count": null, 244 | "outputs": [] 245 | }, 246 | { 247 | "cell_type": "code", 248 | "source": [ 249 | "inputDataOHE" 250 | ], 251 | "metadata": { 252 | "id": "qAURyVwI6_Y-" 253 | }, 254 | "execution_count": null, 255 | "outputs": [] 256 | }, 257 | { 258 | "cell_type": "markdown", 259 | "source": [ 260 | "### Min-Max Normalization" 261 | ], 262 | "metadata": { 263 | "id": "Ft0do09XpX9h" 264 | } 265 | }, 266 | { 267 | "cell_type": "code", 268 | "source": [ 269 | "def MinMaxNorm(dataFrame) :\n", 270 | " return (dataFrame-dataFrame.min())/(dataFrame.max()-dataFrame.min())" 271 | ], 272 | "metadata": { 273 | "id": "57Jco6WDzhqL" 274 | }, 275 | "execution_count": null, 276 | "outputs": [] 277 | }, 278 | { 279 | "cell_type": "code", 280 | "source": [ 281 | "inputDataMMN = MinMaxNorm(inputData.drop([\"gender\",\"oral\",\"tartar\"], axis=1))" 282 | ], 283 | "metadata": { 284 | "id": "-PY_od97xqA9" 285 | }, 286 | "execution_count": null, 287 | "outputs": [] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "source": [ 292 | "inputDataMMN" 293 | ], 294 | "metadata": { 295 | "id": "7W1rRKPuzvZJ" 296 | }, 297 | "execution_count": null, 298 | "outputs": [] 299 | }, 300 | { 301 | "cell_type": "markdown", 302 | "source": [ 303 | "## Merge Preprocessed Input Data" 304 | ], 305 | "metadata": { 306 | "id": "zFXqbbhj7O_U" 307 | } 308 | }, 309 | { 310 | "cell_type": "code", 311 | "source": [ 312 | "inputData = pd.concat([inputDataOHE, inputDataMMN], axis=1)" 313 | ], 314 | "metadata": { 315 | "id": "V_b_Wocw7Tn7" 316 | }, 317 | "execution_count": null, 318 | "outputs": [] 319 | }, 320 | { 321 | "cell_type": "code", 322 | "source": [ 323 | "inputData" 324 | ], 325 | "metadata": { 326 | "id": "tMYJCur27Zf_" 327 | }, 328 | "execution_count": null, 329 | "outputs": [] 330 | }, 331 | { 332 | "cell_type": "markdown", 333 | "metadata": { 334 | "id": "R3NJc3zONGtN" 335 | }, 336 | "source": [ 337 | "### Split Dataset" 338 | ] 339 | }, 340 | { 341 | "cell_type": "code", 342 | "execution_count": null, 343 | "metadata": { 344 | "id": "ZfH3USUYNATV" 345 | }, 346 | "outputs": [], 347 | "source": [ 348 | "from sklearn.model_selection import train_test_split" 349 | ] 350 | }, 351 | { 352 | "cell_type": "code", 353 | "source": [ 354 | "inputData, targetData = np.array(inputData), np.array(targetData)" 355 | ], 356 | "metadata": { 357 | "id": "RwBj3WU50AdY" 358 | }, 359 | "execution_count": null, 360 | "outputs": [] 361 | }, 362 | { 363 | "cell_type": "code", 364 | "execution_count": null, 365 | "metadata": { 366 | "id": "9RQa1TUVNZzl" 367 | }, 368 | "outputs": [], 369 | "source": [ 370 | "xTrain, xTest, yTrain, yTest = train_test_split(inputData, targetData, test_size=0.1, random_state=42)" 371 | ] 372 | }, 373 | { 374 | "cell_type": "code", 375 | "execution_count": null, 376 | "metadata": { 377 | "id": "dOOOvXOzz7qM" 378 | }, 379 | "outputs": [], 380 | "source": [ 381 | "xTrain, xValid, yTrain, yValid = train_test_split(xTrain, yTrain, test_size=0.1, random_state=42)" 382 | ] 383 | }, 384 | { 385 | "cell_type": "code", 386 | "execution_count": null, 387 | "metadata": { 388 | "id": "aQ-l4w6HOIHX" 389 | }, 390 | "outputs": [], 391 | "source": [ 392 | "print(xTrain.shape, yTrain.shape)\n", 393 | "print(xValid.shape, yValid.shape)\n", 394 | "print(xTest.shape, yTest.shape)" 395 | ] 396 | }, 397 | { 398 | "cell_type": "markdown", 399 | "metadata": { 400 | "id": "JXUpV3puOOlX" 401 | }, 402 | "source": [ 403 | "## Create PyTorch DataLoader Class" 404 | ] 405 | }, 406 | { 407 | "cell_type": "code", 408 | "execution_count": null, 409 | "metadata": { 410 | "id": "yhWPVY7tOKZc" 411 | }, 412 | "outputs": [], 413 | "source": [ 414 | "import torch\n", 415 | "from torch.utils.data import Dataset" 416 | ] 417 | }, 418 | { 419 | "cell_type": "code", 420 | "execution_count": null, 421 | "metadata": { 422 | "id": "RTjqdBsVOVUu" 423 | }, 424 | "outputs": [], 425 | "source": [ 426 | "class myDataLoader(Dataset) :\n", 427 | " def __init__(self, inputData:np.array, targetData:np.array) :\n", 428 | " # Inheritance\n", 429 | " super(myDataLoader, self).__init__()\n", 430 | "\n", 431 | " # Initialize Variable\n", 432 | " self.inputData = inputData\n", 433 | " self.targetData = targetData\n", 434 | "\n", 435 | " def __getitem__(self, index) :\n", 436 | " input = self.inputData[index, :]\n", 437 | " target = self.targetData[index]\n", 438 | "\n", 439 | " input = torch.as_tensor(input)\n", 440 | " target = torch.as_tensor(target).unsqueeze(0)\n", 441 | "\n", 442 | " return {\"input\":input.float(), \"target\":target.float()}\n", 443 | "\n", 444 | " def __len__(self) :\n", 445 | " return len(self.inputData)" 446 | ] 447 | }, 448 | { 449 | "cell_type": "markdown", 450 | "metadata": { 451 | "id": "Wz0-THJOPxvx" 452 | }, 453 | "source": [ 454 | "## Create PyTorch Regression Model" 455 | ] 456 | }, 457 | { 458 | "cell_type": "code", 459 | "execution_count": null, 460 | "metadata": { 461 | "id": "Lcjb8ky2PvGo" 462 | }, 463 | "outputs": [], 464 | "source": [ 465 | "from torch import nn\n", 466 | "import torch.nn.functional as F" 467 | ] 468 | }, 469 | { 470 | "cell_type": "code", 471 | "execution_count": null, 472 | "metadata": { 473 | "id": "VpMGcu1KP07P" 474 | }, 475 | "outputs": [], 476 | "source": [ 477 | "class myModel(nn.Module) :\n", 478 | " def __init__(self, inputDim:int, targetDim:int, channels:int) :\n", 479 | " # Inheritance\n", 480 | " super(myModel, self).__init__()\n", 481 | "\n", 482 | " # Create MLP Layer Instance\n", 483 | " self.layer0 = nn.Linear(inputDim, channels)\n", 484 | " self.layer1 = nn.Linear(channels, channels*2)\n", 485 | " self.layer2 = nn.Linear(channels*2, channels)\n", 486 | " self.layer3 = nn.Linear(channels, targetDim)\n", 487 | "\n", 488 | " def forward(self, input) :\n", 489 | " output = F.relu(self.layer0(input))\n", 490 | " output = F.relu(self.layer1(output))\n", 491 | " output = F.relu(self.layer2(output))\n", 492 | " output = self.layer3(output)\n", 493 | "\n", 494 | " return output" 495 | ] 496 | }, 497 | { 498 | "cell_type": "markdown", 499 | "source": [ 500 | "## Create Training Option (Hyperparameter) Dictionary" 501 | ], 502 | "metadata": { 503 | "id": "ttFH7GYJq6c3" 504 | } 505 | }, 506 | { 507 | "cell_type": "code", 508 | "source": [ 509 | "opt = {\"seed\":42,\n", 510 | " \"batchSize\":128,\n", 511 | " \"lr\":1e-3,\n", 512 | " \"epochs\":50,\n", 513 | " \"isCUDA\":torch.cuda.is_available()}" 514 | ], 515 | "metadata": { 516 | "id": "tXzQHG9aq9xB" 517 | }, 518 | "execution_count": null, 519 | "outputs": [] 520 | }, 521 | { 522 | "cell_type": "markdown", 523 | "metadata": { 524 | "id": "3Wk4_cBzQU19" 525 | }, 526 | "source": [ 527 | "## Train DL Model" 528 | ] 529 | }, 530 | { 531 | "cell_type": "code", 532 | "execution_count": null, 533 | "metadata": { 534 | "id": "0QJcgBk-QSsg" 535 | }, 536 | "outputs": [], 537 | "source": [ 538 | "from torch.utils.data import DataLoader\n", 539 | "from torch import optim\n", 540 | "\n", 541 | "from torchsummary import summary\n", 542 | "\n", 543 | "from tqdm import tqdm" 544 | ] 545 | }, 546 | { 547 | "cell_type": "markdown", 548 | "metadata": { 549 | "id": "phpTOt47RjcB" 550 | }, 551 | "source": [ 552 | "### Fix Seed" 553 | ] 554 | }, 555 | { 556 | "cell_type": "code", 557 | "execution_count": null, 558 | "metadata": { 559 | "id": "81NJ5JxbRnaU" 560 | }, 561 | "outputs": [], 562 | "source": [ 563 | "import random" 564 | ] 565 | }, 566 | { 567 | "cell_type": "code", 568 | "execution_count": null, 569 | "metadata": { 570 | "id": "o8F_Z5d3RlH-" 571 | }, 572 | "outputs": [], 573 | "source": [ 574 | "def fixSeed(seed) :\n", 575 | " random.seed(seed)\n", 576 | " np.random.seed(seed)\n", 577 | " torch.manual_seed(seed)\n", 578 | " torch.cuda.manual_seed(seed)\n", 579 | " torch.cuda.manual_seed_all(seed)\n", 580 | " torch.backends.cudnn.deterministic = True\n", 581 | " torch.backends.cudnn.benchmark = False" 582 | ] 583 | }, 584 | { 585 | "cell_type": "markdown", 586 | "source": [ 587 | "## Create Average Meter Instance" 588 | ], 589 | "metadata": { 590 | "id": "Jh4VCYXSswFY" 591 | } 592 | }, 593 | { 594 | "cell_type": "code", 595 | "source": [ 596 | "class AverageMeter(object):\n", 597 | " def __init__(self):\n", 598 | " self.reset()\n", 599 | "\n", 600 | " def reset(self):\n", 601 | " self.val = 0\n", 602 | " self.avg = 0\n", 603 | " self.sum = 0\n", 604 | " self.count = 0\n", 605 | "\n", 606 | " def update(self, val, n=1):\n", 607 | " self.val = val\n", 608 | " self.sum += val*n\n", 609 | " self.count += n\n", 610 | " self.avg = self.sum / self.count" 611 | ], 612 | "metadata": { 613 | "id": "oGUnMpzusxqw" 614 | }, 615 | "execution_count": null, 616 | "outputs": [] 617 | }, 618 | { 619 | "cell_type": "markdown", 620 | "source": [ 621 | "## Create Accuracy Computation Function" 622 | ], 623 | "metadata": { 624 | "id": "fBn8NVKk8Pqt" 625 | } 626 | }, 627 | { 628 | "cell_type": "code", 629 | "source": [ 630 | "def computeAcc(pred, target) :\n", 631 | " pred = torch.where(pred>0, 1, 0)\n", 632 | " acc = (pred==target).sum()/pred.size(0)\n", 633 | "\n", 634 | " return acc" 635 | ], 636 | "metadata": { 637 | "id": "B8qpzXhw8Org" 638 | }, 639 | "execution_count": null, 640 | "outputs": [] 641 | }, 642 | { 643 | "cell_type": "markdown", 644 | "source": [ 645 | "## Training Code as a Function (Abstraction)" 646 | ], 647 | "metadata": { 648 | "id": "lhJ28-fU6Zk7" 649 | } 650 | }, 651 | { 652 | "cell_type": "code", 653 | "source": [ 654 | "def train(opt, dataset, criterion) :\n", 655 | " fixSeed(opt[\"seed\"])\n", 656 | "\n", 657 | " trainDataLoader = DataLoader(myDataLoader(dataset[\"xTrain\"], dataset[\"yTrain\"]), batch_size=opt[\"batchSize\"], shuffle=True, drop_last=True)\n", 658 | " validDataLoader = DataLoader(myDataLoader(dataset[\"xValid\"], dataset[\"yValid\"]), batch_size=opt[\"batchSize\"], shuffle=False, drop_last=False)\n", 659 | "\n", 660 | " fixSeed(opt[\"seed\"])\n", 661 | " model = myModel(xTrain.shape[1], 1, 64)\n", 662 | " if opt[\"isCUDA\"] :\n", 663 | " model = model.cuda()\n", 664 | "\n", 665 | " summary(model, (1, dataset[\"xTrain\"].shape[1]))\n", 666 | "\n", 667 | " optimizer = optim.Adam(model.parameters(), lr=opt[\"lr\"])\n", 668 | "\n", 669 | " trainLoss, validLoss = AverageMeter(), AverageMeter()\n", 670 | " trainAcc, validAcc = AverageMeter(), AverageMeter()\n", 671 | " trainLossList, validLossList = [], []\n", 672 | " trainAccList, validAccList = [], []\n", 673 | " bestAcc = 0\n", 674 | "\n", 675 | " for epoch in range(1, opt[\"epochs\"]+1) :\n", 676 | " trainBar = tqdm(trainDataLoader)\n", 677 | " trainLoss.reset(), trainAcc.reset()\n", 678 | "\n", 679 | " for data in trainBar :\n", 680 | " input, target = data[\"input\"], data[\"target\"]\n", 681 | " if opt[\"isCUDA\"] :\n", 682 | " input, target = input.cuda(), target.cuda()\n", 683 | "\n", 684 | " optimizer.zero_grad()\n", 685 | " pred = model(input)\n", 686 | " loss = criterion(pred, target)\n", 687 | " loss.backward()\n", 688 | " optimizer.step()\n", 689 | "\n", 690 | " trainLoss.update(loss.item(), opt[\"batchSize\"])\n", 691 | " trainAcc.update(computeAcc(pred, target).item(), opt[\"batchSize\"])\n", 692 | " trainBar.set_description(desc=f\"[{epoch}/{opt['epochs']}] [Train] < Accuracy:{trainAcc.avg:.6f} | Loss:{trainLoss.avg:.6f} >\")\n", 693 | "\n", 694 | " trainLossList.append(trainLoss.avg)\n", 695 | " trainAccList.append(trainAcc.avg)\n", 696 | "\n", 697 | " validBar = tqdm(validDataLoader)\n", 698 | " validLoss.reset(), validAcc.reset()\n", 699 | "\n", 700 | " for data in validBar :\n", 701 | " input, target = data[\"input\"], data[\"target\"]\n", 702 | " if opt[\"isCUDA\"] :\n", 703 | " input, target = input.cuda(), target.cuda()\n", 704 | "\n", 705 | " model.eval()\n", 706 | " with torch.no_grad() :\n", 707 | " pred = model(input)\n", 708 | " loss = criterion(pred, target)\n", 709 | "\n", 710 | " validLoss.update(loss.item(), opt[\"batchSize\"])\n", 711 | " validAcc.update(computeAcc(pred, target).item(), opt[\"batchSize\"])\n", 712 | " validBar.set_description(desc=f\"[{epoch}/{opt['epochs']}] [Valid] < Accuracy:{validAcc.avg:.6f} | Loss:{trainLoss.avg:.6f} >\")\n", 713 | "\n", 714 | " validLossList.append(validLoss.avg)\n", 715 | " validAccList.append(validAcc.avg)\n", 716 | "\n", 717 | " if validAcc.avg > bestAcc :\n", 718 | " bestAcc = validAcc.avg\n", 719 | " torch.save(model.state_dict(), \"bestModel.pth\")\n", 720 | "\n", 721 | " torch.save(model.state_dict(), \"latestModel.pth\")\n", 722 | "\n", 723 | " return (trainLossList, validLossList), (trainAccList, validAccList)" 724 | ], 725 | "metadata": { 726 | "id": "Toe0P0WH6c9p" 727 | }, 728 | "execution_count": null, 729 | "outputs": [] 730 | }, 731 | { 732 | "cell_type": "markdown", 733 | "source": [ 734 | "## Train Model" 735 | ], 736 | "metadata": { 737 | "id": "VpE3M2q67qRu" 738 | } 739 | }, 740 | { 741 | "cell_type": "code", 742 | "source": [ 743 | "lossList, accList = train(opt,\n", 744 | " {\"xTrain\":xTrain, \"yTrain\":yTrain, \"xValid\":xValid, \"yValid\":yValid},\n", 745 | " nn.BCEWithLogitsLoss())" 746 | ], 747 | "metadata": { 748 | "id": "yV7Jgs_77c57" 749 | }, 750 | "execution_count": null, 751 | "outputs": [] 752 | }, 753 | { 754 | "cell_type": "markdown", 755 | "metadata": { 756 | "id": "XrgUhJC67uzu" 757 | }, 758 | "source": [ 759 | "## Plot Training vs. Validation Loss Graph" 760 | ] 761 | }, 762 | { 763 | "cell_type": "code", 764 | "execution_count": null, 765 | "metadata": { 766 | "id": "mLsjFs3K7uzv" 767 | }, 768 | "outputs": [], 769 | "source": [ 770 | "plt.figure(figsize=(20,10))\n", 771 | "\n", 772 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), lossList[0], label=\"Training Loss\")\n", 773 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), lossList[1], label=\"Validation Loss\")\n", 774 | "\n", 775 | "plt.xlabel(\"Epoch\")\n", 776 | "plt.ylabel(\"BCE Loss\")\n", 777 | "plt.legend(loc=\"best\")\n", 778 | "\n", 779 | "plt.show()" 780 | ] 781 | }, 782 | { 783 | "cell_type": "markdown", 784 | "metadata": { 785 | "id": "CfMgjzvi9dYX" 786 | }, 787 | "source": [ 788 | "## Plot Training vs. Validation Accuracy Graph" 789 | ] 790 | }, 791 | { 792 | "cell_type": "code", 793 | "execution_count": null, 794 | "metadata": { 795 | "id": "a0N1aiik9dYd" 796 | }, 797 | "outputs": [], 798 | "source": [ 799 | "plt.figure(figsize=(20,10))\n", 800 | "\n", 801 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), accList[0], label=\"Training Accuracy\")\n", 802 | "plt.plot(np.arange(0, opt[\"epochs\"], 1), accList[1], label=\"Validation Accuracy\")\n", 803 | "\n", 804 | "plt.xlabel(\"Epoch\")\n", 805 | "plt.ylabel(\"Accuracy\")\n", 806 | "plt.legend(loc=\"best\")\n", 807 | "\n", 808 | "plt.show()" 809 | ] 810 | }, 811 | { 812 | "cell_type": "markdown", 813 | "source": [ 814 | "## Inference Code as a Function (Abstraction)" 815 | ], 816 | "metadata": { 817 | "id": "TWl44qH29BS2" 818 | } 819 | }, 820 | { 821 | "cell_type": "code", 822 | "source": [ 823 | "def inference(opt, inputData, modelPath) :\n", 824 | " weights = torch.load(modelPath)\n", 825 | "\n", 826 | " model = myModel(xTrain.shape[1], 1, 64)\n", 827 | " model.load_state_dict(weights)\n", 828 | " if opt[\"isCUDA\"] :\n", 829 | " model = model.cuda()\n", 830 | "\n", 831 | " inputDataTensor = torch.as_tensor(inputData).float()\n", 832 | "\n", 833 | " predList = []\n", 834 | "\n", 835 | " model.eval()\n", 836 | "\n", 837 | " with torch.no_grad() :\n", 838 | " with tqdm(total=inputData.shape[0]) as pBar :\n", 839 | " for inputData in inputDataTensor :\n", 840 | " if opt[\"isCUDA\"] :\n", 841 | " inputData = inputData.cuda()\n", 842 | "\n", 843 | " pred = model(inputData)\n", 844 | " predList.append(pred.detach().cpu().item())\n", 845 | "\n", 846 | " pBar.update()\n", 847 | "\n", 848 | " return predList" 849 | ], 850 | "metadata": { 851 | "id": "jwfLPKdk9ClT" 852 | }, 853 | "execution_count": null, 854 | "outputs": [] 855 | }, 856 | { 857 | "cell_type": "markdown", 858 | "source": [ 859 | "## Inference Result" 860 | ], 861 | "metadata": { 862 | "id": "6LBdxtVa8Pjx" 863 | } 864 | }, 865 | { 866 | "cell_type": "code", 867 | "source": [ 868 | "predList = inference(opt, xTest, \"/content/bestModel.pth\")" 869 | ], 870 | "metadata": { 871 | "id": "kMi_gTRL8Pjy" 872 | }, 873 | "execution_count": null, 874 | "outputs": [] 875 | }, 876 | { 877 | "cell_type": "code", 878 | "source": [ 879 | "predList" 880 | ], 881 | "metadata": { 882 | "id": "FLS_XvM-BKgJ" 883 | }, 884 | "execution_count": null, 885 | "outputs": [] 886 | }, 887 | { 888 | "cell_type": "code", 889 | "source": [ 890 | "predList = np.where(np.array(predList)>0, 1, 0)" 891 | ], 892 | "metadata": { 893 | "id": "i9rGqtIYBLrF" 894 | }, 895 | "execution_count": null, 896 | "outputs": [] 897 | }, 898 | { 899 | "cell_type": "code", 900 | "source": [ 901 | "predList" 902 | ], 903 | "metadata": { 904 | "id": "5iVP3BNTBm1H" 905 | }, 906 | "execution_count": null, 907 | "outputs": [] 908 | }, 909 | { 910 | "cell_type": "markdown", 911 | "source": [ 912 | "## Quantitative Assessment" 913 | ], 914 | "metadata": { 915 | "id": "URfarsx7ARdH" 916 | } 917 | }, 918 | { 919 | "cell_type": "code", 920 | "source": [ 921 | "from sklearn.metrics import accuracy_score, confusion_matrix, classification_report" 922 | ], 923 | "metadata": { 924 | "id": "q7GlfZO6AXrU" 925 | }, 926 | "execution_count": null, 927 | "outputs": [] 928 | }, 929 | { 930 | "cell_type": "code", 931 | "source": [ 932 | "accScore = accuracy_score(yTest, predList)\n", 933 | "print(accScore)" 934 | ], 935 | "metadata": { 936 | "id": "3SH1Bt3p8Pjz" 937 | }, 938 | "execution_count": null, 939 | "outputs": [] 940 | }, 941 | { 942 | "cell_type": "code", 943 | "source": [ 944 | "cm = confusion_matrix(yTest, predList)\n", 945 | "print(cm)" 946 | ], 947 | "metadata": { 948 | "id": "J-hHT2HfB-Pj" 949 | }, 950 | "execution_count": null, 951 | "outputs": [] 952 | }, 953 | { 954 | "cell_type": "code", 955 | "source": [ 956 | "clsRp = classification_report(yTest, predList)\n", 957 | "print(clsRp)" 958 | ], 959 | "metadata": { 960 | "id": "zhT7r62ACAvK" 961 | }, 962 | "execution_count": null, 963 | "outputs": [] 964 | } 965 | ], 966 | "metadata": { 967 | "accelerator": "GPU", 968 | "colab": { 969 | "provenance": [], 970 | "authorship_tag": "ABX9TyNzNd3nrw0wfCxPH8kSpbm9", 971 | "include_colab_link": true 972 | }, 973 | "gpuClass": "standard", 974 | "kernelspec": { 975 | "display_name": "Python 3", 976 | "name": "python3" 977 | }, 978 | "language_info": { 979 | "name": "python" 980 | } 981 | }, 982 | "nbformat": 4, 983 | "nbformat_minor": 0 984 | } --------------------------------------------------------------------------------