├── datasets └── .gitkeep ├── models └── .gitkeep ├── 11장 모델 배포 ├── 예제 11.15 플라스크 실행 ├── 예제 11.19 패스트 API 실행 ├── 예제 11.35 애플리케이션 배포 ├── 예제 11.32 도커 컨테이너 삭제하기 ├── 예제 11.33 도커 이미지 삭제하기 ├── 예제 11.28 도커 허브 로그인 ├── 예제 11.24 도커 컨테이너 접속 ├── 예제 11.25 도커 컨테이너 중지하기 ├── 예제 11.30 도커 이미지 배포하기 ├── 예제 11.26 도커 컨테이너 재시작하기 ├── 예제 11.31 배포된 도커 이미지 다운로드 ├── 예제 11.22 도커 이미지 빌드하기 ├── 예제 11.27 도커 컨테이너를 이미지로 커밋하기 ├── 예제 11.23 도커 컨테이너 실행 ├── 예제 11.29 도커 이미지 푸시를 위한 태그 생성 ├── 예제 11.21 도커 파일 ├── demo.py ├── 예제 11.07 특잇값 분해.ipynb ├── 예제 11.08 특잇값 분해를 이용한 저계수 분해.ipynb ├── 예제 11.11 ONNX 형식 변환.ipynb ├── 예제 11.16 모델 추론 요청.ipynb ├── 예제 11.37 문장 생성.ipynb ├── 예제 11.20 모델 추론 요청.ipynb ├── 예제 11.34 애플리케이션 구성.ipynb ├── 예제 11.36 데이터 캐싱.ipynb ├── 예제 11.04 VGG-16 학습 후 동적 양자화.ipynb ├── 예제 11.01 BERT 모델 가지치기.ipynb ├── app_fastapi.py ├── app_flask.py ├── 예제 11.12 ONNX 런타임 실행.ipynb ├── 예제 11.03 양자화 결과 비교.ipynb ├── 예제 11.38 이미지 모델.ipynb ├── 예제 11.06 응답 기반 지식 증류를 활용한 모델 학습.ipynb ├── 예제 11.02 VGG-16 학습 후 정적 양자화.ipynb ├── 예제 11.17~11.18 패스트 API 모델 서빙.ipynb ├── 예제 11.13~11.14 플라스크 모델 서빙.ipynb ├── 예제 11.09 VGG-16 합성곱 계층 경량화.ipynb └── 예제 11.10 VGG-16 모델 경량화.ipynb ├── .gitignore ├── kaggle.png ├── pytorch.png ├── requirements.txt ├── 04장 파이토치 심화 ├── 예제 4.07 가중치 감쇠 적용 방식.ipynb ├── 예제 4.01 텐서 초깃값.ipynb ├── 예제 4.05 L1 정칙화 적용 방식.ipynb ├── 예제 4.06 L2 정칙화 적용 방식.ipynb ├── 예제 4.02 배치 정규화 수행.ipynb ├── 예제 4.09 그라디언트 클리핑 적용 방식.ipynb ├── 예제 4.08 드롭아웃 적용 방식.ipynb ├── 예제 4.03 가중치 초기화 함수 (1).ipynb └── 예제 4.04 가중치 초기화 함수 (2).ipynb ├── 02장 파이토치 설치 ├── 예제 2.01 파이토치 버전 확인.ipynb ├── 예제 2.02 파이토치 GPU 가속 확인.ipynb ├── 예제 2.04 파이토치 GPU 가속 확인.ipynb └── 예제 2.03 MPS 가속 확인.ipynb ├── 03장 파이토치 기초 ├── 예제 3.04 텐서 자료형 설정.ipynb ├── 예제 3.02 텐서 속성.ipynb ├── 예제 3.01 텐서 생성.ipynb ├── 예제 3.07 넘파이 배열의 텐서 변환.ipynb ├── 예제 3.03 텐서 차원 변환.ipynb ├── 예제 3.48 모델 구조 확인.ipynb ├── 예제 3.08 텐서의 넘파이 배열 변환.ipynb ├── 예제 3.06 텐서 장치 변환.ipynb ├── 예제 3.05 텐서 GPU 장치 설정.ipynb ├── 예제 3.09~3.10 비쌍체 t-검정.ipynb ├── 예제 3.47 모델 불러오기.ipynb ├── 예제 3.50 모델 상태 불러오기.ipynb ├── 예제 3.57 모델 구조와 초깃값.ipynb ├── 예제 3.11~3.17 단순 선형 회귀(넘파이).ipynb ├── 예제 3.25 zero_grad(), cost.backward(), optimizer.step().ipynb ├── 예제 3.26~3.28 신경망 패키지 적용.ipynb ├── 예제 3.18~3.24 단순 선형 회귀(파이토치).ipynb ├── 예제 3.34 편향 제거.ipynb ├── 예제 3.29~3.33 다중 선형 회귀.ipynb ├── 예제 3.52 체크포인트 불러오기.ipynb ├── 예제 3.58 단층 퍼셉트론 구조.ipynb ├── 예제 3.49 모델 상태 저장.ipynb ├── 예제 3.51 체크포인트 저장.ipynb └── 예제 3.59 다층 퍼셉트론 구조.ipynb ├── 부록 ├── 예제 B.01 PreTrainedConfig 클래스 구성.ipynb ├── 예제 C.01 모델 생성.ipynb ├── 예제 C.02 사용 가능한 사전 학습된 모델 확인.ipynb ├── 예제 E.01 체크포인트 기법.ipynb ├── 예제 B.05 ImageProcessor 클래스 이미지 데이터 변환.ipynb ├── 예제 B.04 PreTrainedFeatureExtractor 입력 특징 추출.ipynb ├── 예제 D.01 파이토치 컴파일러.ipynb ├── 예제 B.03 PreTrainedTokenizer의 사전 토큰화와 정규화.ipynb ├── 예제 B.06 Auto 클래스를 이용한 BERT와 GPT2 구축.ipynb ├── 예제 B.02 PreTrainedModel 클래스 구성.ipynb ├── 예제 C.03 미세 조정.ipynb ├── 예제 A.01 오토인코더.ipynb ├── 예제 B.07 Auto 클래스와 Trainer를 활용한 IMDB 분류 모델 학습 .ipynb ├── 예제 A.02 MNIST 데이터세트.ipynb ├── 예제 A.03 파이토치 라이트닝 모델 학습.ipynb └── 예제 A.04 콜백 정의.ipynb ├── 05장 토큰화 ├── 예제 5.10 청와대 청원 데이터 다운로드.ipynb ├── 예제 5.03 자소 단위 토큰화.ipynb ├── 예제 5.12 토크나이저 모델 학습.ipynb ├── 예제 5.09 spaCy 품사 태깅.ipynb ├── 예제 5.11 학습 데이터세트 생성.ipynb ├── 예제 5.14 어휘 사전 불러오기.ipynb ├── 예제 5.01~5.02 단어 및 글자 토큰화.ipynb ├── 예제 5.15 워드피스 토크나이저 학습.ipynb ├── 예제 5.16 워드피스 토큰화.ipynb ├── 예제 5.13 바이트 페어 인코딩 토큰화.ipynb ├── 예제 5.06~5.08 NLTK 실습.ipynb └── 예제 5.04~5.05 KoNLPy 토큰화.ipynb ├── 07장 트랜스포머 ├── 예제 7.10 GPT-2를 이용한 문장 생성.ipynb ├── 예제 7.09 문장 생성을 위한 GPT-2 모델의 구조.ipynb └── 예제 7.01 위치 인코딩.ipynb ├── 06장 임베딩 ├── 예제 6.02 TF-IDF 계산.ipynb ├── 예제 6.01 N-gram.ipynb ├── 예제 6.18 양방향 다층 신경망.ipynb ├── 예제 6.19 양방향 다층 장단기 메모리.ipynb ├── 예제 6.31 합성곱 모델.ipynb ├── 예제 6.15~6.17 fastText OOV 모델 실습.ipynb └── 예제 6.13~6.14 Word2Vec 모델 실습.ipynb ├── 10장 비전 트랜스포머 └── 예제 10.10~10.12 스윈 트랜스포머 모델 구조.ipynb ├── 08장 이미지 분류 ├── 예제 8.05~8.06 데이터 시각화.ipynb ├── 예제 8.14~8.17 클래스 활성화 맵 실습.ipynb ├── 예제 8.01~8.04 알렉스넷 모델 실습.ipynb └── 예제 8.18~8.20 Grad-CAM 실습.ipynb └── README.md /datasets/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /models/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.15 플라스크 실행: -------------------------------------------------------------------------------- 1 | python app_flask.py -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | */.ipynb_checkpoints/ 3 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.19 패스트 API 실행: -------------------------------------------------------------------------------- 1 | python app_fastapi.py -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.35 애플리케이션 배포: -------------------------------------------------------------------------------- 1 | streamlit run demo.py -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.32 도커 컨테이너 삭제하기: -------------------------------------------------------------------------------- 1 | docker rm -f mycontainer -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.33 도커 이미지 삭제하기: -------------------------------------------------------------------------------- 1 | docker rmi -f myapp:latest -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.28 도커 허브 로그인: -------------------------------------------------------------------------------- 1 | $ docker login 2 | username 3 | pwd -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.24 도커 컨테이너 접속: -------------------------------------------------------------------------------- 1 | docker exec -it mycontainer /bin/bash -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.25 도커 컨테이너 중지하기: -------------------------------------------------------------------------------- 1 | docker stop mycontainer 2 | docker ps -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.30 도커 이미지 배포하기: -------------------------------------------------------------------------------- 1 | docker push $username/myapp:newimage -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.26 도커 컨테이너 재시작하기: -------------------------------------------------------------------------------- 1 | docker start mycontainer 2 | docker ps -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.31 배포된 도커 이미지 다운로드: -------------------------------------------------------------------------------- 1 | docker pull $username/myapp:newimage -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.22 도커 이미지 빌드하기: -------------------------------------------------------------------------------- 1 | docker build -t myapp:latest . 2 | docker images -------------------------------------------------------------------------------- /kaggle.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wikibook/pytorchtrf/HEAD/kaggle.png -------------------------------------------------------------------------------- /pytorch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wikibook/pytorchtrf/HEAD/pytorch.png -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.27 도커 컨테이너를 이미지로 커밋하기: -------------------------------------------------------------------------------- 1 | docker commit mycontainer myapp:newimage 2 | docker images -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.23 도커 컨테이너 실행: -------------------------------------------------------------------------------- 1 | docker run -it -d -p 8000:8000 --name mycontainer myapp:latest 2 | docker ps -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.29 도커 이미지 푸시를 위한 태그 생성: -------------------------------------------------------------------------------- 1 | export DOCKER_ID_USER="$username" 2 | docker tag myapp:newimage $DOCKER_ID_USER/myapp:newimage 3 | docker images -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.21 도커 파일: -------------------------------------------------------------------------------- 1 | FROM pytorch/pytorch:2.0.1-cuda11.7-cudnn8-runtime 2 | 3 | COPY container /app 4 | 5 | WORKDIR /app 6 | 7 | RUN pip install --no-cache-dir -r requirements.txt 8 | 9 | RUN chmod +x /app/app_flask.py 10 | 11 | EXPOSE 8000 12 | 13 | CMD ["python", "app_flask.py"] -------------------------------------------------------------------------------- /11장 모델 배포/demo.py: -------------------------------------------------------------------------------- 1 | # demo.py 2 | import pandas as pd 3 | import streamlit as st 4 | 5 | 6 | st.set_page_config( 7 | page_title="데모 애플리케이션", 8 | page_icon=":shark:", 9 | layout="wide" 10 | ) 11 | 12 | df = pd.read_csv("../datasets/non_linear.csv") 13 | 14 | st.header(body="Demo Application") 15 | st.subheader(body="non_linear.csv") 16 | 17 | x = st.sidebar.selectbox(label="X 축", options=df.columns, index=0) 18 | y = st.sidebar.selectbox(label="Y 축", options=df.columns, index=1) 19 | 20 | col1, col2 = st.columns(2) 21 | with col1: 22 | st.dataframe(data=df, height=500, use_container_width=True) 23 | with col2: 24 | st.line_chart(data=df, x=x, y=y, height=500) 25 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | absl-py==1.4.0 2 | cython==3.0.2 3 | datasets==2.14.5 4 | evaluate==0.4.0 5 | fastapi==0.103.1 6 | flask==2.3.3 7 | gensim==4.1.2 8 | imgaug==0.4.0 9 | jamo==0.4.1 10 | konlpy==0.6.0 11 | korpora==0.2.0 12 | lightning==2.0.1 13 | nlpaug==1.1.11 14 | nltk==3.7 15 | opencv-python==4.8.0.76 16 | pandas==1.4.4 17 | pillow==9.2.0 18 | portalocker==2.7.0 19 | pycocotools==2.0.7 20 | requests==2.28.1 21 | sacremoses==0.0.53 22 | scikit-learn==1.0.2 23 | sentencepiece==0.1.97 24 | spacy==3.6.1 25 | streamlit==1.22.0 26 | tensorly==0.8.1 27 | timm==0.9.7 28 | tokenizers==0.13.3 29 | torch==2.0.1 30 | torchdata==0.6.1 31 | torchinfo==1.8.0 32 | torchtext==0.15.2 33 | torchvision==0.15.2 34 | transformers==4.33.2 35 | ultralytics==8.0.128 -------------------------------------------------------------------------------- /04장 파이토치 심화/예제 4.07 가중치 감쇠 적용 방식.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "57a49a26-1f29-4af5-aa6c-cb9f397a2201", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "optimizer = torch.optim.SGD(model.parameters(), lr=0.01, weight_decay=0.01)" 11 | ] 12 | } 13 | ], 14 | "metadata": { 15 | "kernelspec": { 16 | "display_name": "Python 3 (ipykernel)", 17 | "language": "python", 18 | "name": "python3" 19 | }, 20 | "language_info": { 21 | "codemirror_mode": { 22 | "name": "ipython", 23 | "version": 3 24 | }, 25 | "file_extension": ".py", 26 | "mimetype": "text/x-python", 27 | "name": "python", 28 | "nbconvert_exporter": "python", 29 | "pygments_lexer": "ipython3", 30 | "version": "3.9.13" 31 | } 32 | }, 33 | "nbformat": 4, 34 | "nbformat_minor": 5 35 | } 36 | -------------------------------------------------------------------------------- /02장 파이토치 설치/예제 2.01 파이토치 버전 확인.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "95ff9559-b7e7-401f-9b73-8c7a68eed848", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "print(torch.__version__)" 14 | ] 15 | } 16 | ], 17 | "metadata": { 18 | "kernelspec": { 19 | "display_name": "Python 3 (ipykernel)", 20 | "language": "python", 21 | "name": "python3" 22 | }, 23 | "language_info": { 24 | "codemirror_mode": { 25 | "name": "ipython", 26 | "version": 3 27 | }, 28 | "file_extension": ".py", 29 | "mimetype": "text/x-python", 30 | "name": "python", 31 | "nbconvert_exporter": "python", 32 | "pygments_lexer": "ipython3", 33 | "version": "3.9.13" 34 | } 35 | }, 36 | "nbformat": 4, 37 | "nbformat_minor": 5 38 | } 39 | -------------------------------------------------------------------------------- /02장 파이토치 설치/예제 2.02 파이토치 GPU 가속 확인.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "b64aadc9-efad-4431-b295-037bcd4eb913", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "print(torch.__version__)\n", 14 | "print(torch.cuda.is_available())" 15 | ] 16 | } 17 | ], 18 | "metadata": { 19 | "kernelspec": { 20 | "display_name": "Python 3 (ipykernel)", 21 | "language": "python", 22 | "name": "python3" 23 | }, 24 | "language_info": { 25 | "codemirror_mode": { 26 | "name": "ipython", 27 | "version": 3 28 | }, 29 | "file_extension": ".py", 30 | "mimetype": "text/x-python", 31 | "name": "python", 32 | "nbconvert_exporter": "python", 33 | "pygments_lexer": "ipython3", 34 | "version": "3.9.13" 35 | } 36 | }, 37 | "nbformat": 4, 38 | "nbformat_minor": 5 39 | } 40 | -------------------------------------------------------------------------------- /02장 파이토치 설치/예제 2.04 파이토치 GPU 가속 확인.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "9b111566-1683-426f-b56e-a791371fabfd", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "print(torch.__version__)\n", 14 | "print(torch.cuda.is_available())" 15 | ] 16 | } 17 | ], 18 | "metadata": { 19 | "kernelspec": { 20 | "display_name": "Python 3 (ipykernel)", 21 | "language": "python", 22 | "name": "python3" 23 | }, 24 | "language_info": { 25 | "codemirror_mode": { 26 | "name": "ipython", 27 | "version": 3 28 | }, 29 | "file_extension": ".py", 30 | "mimetype": "text/x-python", 31 | "name": "python", 32 | "nbconvert_exporter": "python", 33 | "pygments_lexer": "ipython3", 34 | "version": "3.9.13" 35 | } 36 | }, 37 | "nbformat": 4, 38 | "nbformat_minor": 5 39 | } 40 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.04 텐서 자료형 설정.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "48a684ea-c14c-4a19-8d0e-34eca770387c", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "tensor = torch.rand((3, 3), dtype=torch.float)\n", 14 | "print(tensor)" 15 | ] 16 | } 17 | ], 18 | "metadata": { 19 | "kernelspec": { 20 | "display_name": "Python 3 (ipykernel)", 21 | "language": "python", 22 | "name": "python3" 23 | }, 24 | "language_info": { 25 | "codemirror_mode": { 26 | "name": "ipython", 27 | "version": 3 28 | }, 29 | "file_extension": ".py", 30 | "mimetype": "text/x-python", 31 | "name": "python", 32 | "nbconvert_exporter": "python", 33 | "pygments_lexer": "ipython3", 34 | "version": "3.9.13" 35 | } 36 | }, 37 | "nbformat": 4, 38 | "nbformat_minor": 5 39 | } 40 | -------------------------------------------------------------------------------- /02장 파이토치 설치/예제 2.03 MPS 가속 확인.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "a67e7e2a-e030-4dca-99d6-c2417ef9c054", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "print(torch.__version__)\n", 14 | "print(torch.backends.mps.is_built())\n", 15 | "print(torch.backends.mps.is_available())" 16 | ] 17 | } 18 | ], 19 | "metadata": { 20 | "kernelspec": { 21 | "display_name": "Python 3 (ipykernel)", 22 | "language": "python", 23 | "name": "python3" 24 | }, 25 | "language_info": { 26 | "codemirror_mode": { 27 | "name": "ipython", 28 | "version": 3 29 | }, 30 | "file_extension": ".py", 31 | "mimetype": "text/x-python", 32 | "name": "python", 33 | "nbconvert_exporter": "python", 34 | "pygments_lexer": "ipython3", 35 | "version": "3.9.13" 36 | } 37 | }, 38 | "nbformat": 4, 39 | "nbformat_minor": 5 40 | } 41 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.02 텐서 속성.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "834bc616-5b5f-4833-b2bf-937ee587a267", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "tensor = torch.rand(1, 2)\n", 14 | "print(tensor)\n", 15 | "print(tensor.shape)\n", 16 | "print(tensor.dtype)\n", 17 | "print(tensor.device)" 18 | ] 19 | } 20 | ], 21 | "metadata": { 22 | "kernelspec": { 23 | "display_name": "Python 3 (ipykernel)", 24 | "language": "python", 25 | "name": "python3" 26 | }, 27 | "language_info": { 28 | "codemirror_mode": { 29 | "name": "ipython", 30 | "version": 3 31 | }, 32 | "file_extension": ".py", 33 | "mimetype": "text/x-python", 34 | "name": "python", 35 | "nbconvert_exporter": "python", 36 | "pygments_lexer": "ipython3", 37 | "version": "3.9.13" 38 | } 39 | }, 40 | "nbformat": 4, 41 | "nbformat_minor": 5 42 | } 43 | -------------------------------------------------------------------------------- /부록/예제 B.01 PreTrainedConfig 클래스 구성.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "5bc7e08e-5ebe-4f6b-8edf-ed575d26cc8f", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from transformers import BertConfig\n", 11 | "\n", 12 | "\n", 13 | "config = BertConfig.from_pretrained(\"bert-base-uncased\")\n", 14 | "print(config)\n", 15 | "print(BertConfig.__bases__)" 16 | ] 17 | } 18 | ], 19 | "metadata": { 20 | "kernelspec": { 21 | "display_name": "Python 3 (ipykernel)", 22 | "language": "python", 23 | "name": "python3" 24 | }, 25 | "language_info": { 26 | "codemirror_mode": { 27 | "name": "ipython", 28 | "version": 3 29 | }, 30 | "file_extension": ".py", 31 | "mimetype": "text/x-python", 32 | "name": "python", 33 | "nbconvert_exporter": "python", 34 | "pygments_lexer": "ipython3", 35 | "version": "3.9.13" 36 | } 37 | }, 38 | "nbformat": 4, 39 | "nbformat_minor": 5 40 | } 41 | -------------------------------------------------------------------------------- /05장 토큰화/예제 5.10 청와대 청원 데이터 다운로드.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "8e2d6d57-69dd-41e8-9deb-46adcb8498c9", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from Korpora import Korpora\n", 11 | "\n", 12 | "\n", 13 | "corpus = Korpora.load(\"korean_petitions\")\n", 14 | "dataset = corpus.train\n", 15 | "petition = dataset[0]\n", 16 | "\n", 17 | "print(\"청원 시작일 :\", petition.begin)\n", 18 | "print(\"청원 종료일 :\", petition.end)\n", 19 | "print(\"청원 동의 수 :\", petition.num_agree)\n", 20 | "print(\"청원 범주 :\", petition.category)\n", 21 | "print(\"청원 제목 :\", petition.title)\n", 22 | "print(\"청원 본문 :\", petition.text[:30])" 23 | ] 24 | } 25 | ], 26 | "metadata": { 27 | "kernelspec": { 28 | "display_name": "Python 3 (ipykernel)", 29 | "language": "python", 30 | "name": "python3" 31 | }, 32 | "language_info": { 33 | "name": "" 34 | } 35 | }, 36 | "nbformat": 4, 37 | "nbformat_minor": 5 38 | } 39 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.07 특잇값 분해.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "M = torch.rand((4, 3))\n", 14 | "U, s, V = torch.svd(M)\n", 15 | "composed_M = torch.mm(torch.mm(U, torch.diag(s)), V.t())\n", 16 | "print(M)\n", 17 | "print(composed_M)" 18 | ] 19 | } 20 | ], 21 | "metadata": { 22 | "kernelspec": { 23 | "display_name": "Python 3 (ipykernel)", 24 | "language": "python", 25 | "name": "python3" 26 | }, 27 | "language_info": { 28 | "codemirror_mode": { 29 | "name": "ipython", 30 | "version": 3 31 | }, 32 | "file_extension": ".py", 33 | "mimetype": "text/x-python", 34 | "name": "python", 35 | "nbconvert_exporter": "python", 36 | "pygments_lexer": "ipython3", 37 | "version": "3.9.13" 38 | } 39 | }, 40 | "nbformat": 4, 41 | "nbformat_minor": 5 42 | } 43 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.01 텐서 생성.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "cc368595-532b-4cc2-8895-2588b53c329e", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "print(torch.tensor([1, 2, 3]))\n", 14 | "print(torch.Tensor([[1, 2, 3], [4, 5, 6]]))\n", 15 | "print(torch.LongTensor([1, 2, 3]))\n", 16 | "print(torch.FloatTensor([1, 2, 3]))" 17 | ] 18 | } 19 | ], 20 | "metadata": { 21 | "kernelspec": { 22 | "display_name": "Python 3 (ipykernel)", 23 | "language": "python", 24 | "name": "python3" 25 | }, 26 | "language_info": { 27 | "codemirror_mode": { 28 | "name": "ipython", 29 | "version": 3 30 | }, 31 | "file_extension": ".py", 32 | "mimetype": "text/x-python", 33 | "name": "python", 34 | "nbconvert_exporter": "python", 35 | "pygments_lexer": "ipython3", 36 | "version": "3.9.13" 37 | } 38 | }, 39 | "nbformat": 4, 40 | "nbformat_minor": 5 41 | } 42 | -------------------------------------------------------------------------------- /05장 토큰화/예제 5.03 자소 단위 토큰화.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "c78c3102-d005-48dc-bb20-56756c989353", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from jamo import h2j, j2hcj\n", 11 | "\n", 12 | "\n", 13 | "review = \"현실과 구분 불가능한 cg. 시각적 즐거음은 최고! 더불어 ost는 더더욱 최고!!\"\n", 14 | "decomposed = j2hcj(h2j(review))\n", 15 | "tokenized = list(decomposed)\n", 16 | "print(tokenized)" 17 | ] 18 | } 19 | ], 20 | "metadata": { 21 | "kernelspec": { 22 | "display_name": "Python 3 (ipykernel)", 23 | "language": "python", 24 | "name": "python3" 25 | }, 26 | "language_info": { 27 | "codemirror_mode": { 28 | "name": "ipython", 29 | "version": 3 30 | }, 31 | "file_extension": ".py", 32 | "mimetype": "text/x-python", 33 | "name": "python", 34 | "nbconvert_exporter": "python", 35 | "pygments_lexer": "ipython3", 36 | "version": "3.9.13" 37 | } 38 | }, 39 | "nbformat": 4, 40 | "nbformat_minor": 5 41 | } 42 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.07 넘파이 배열의 텐서 변환.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "690717f5-b925-4a5d-a3bb-8a8c51367ae9", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "import numpy as np\n", 12 | "\n", 13 | "\n", 14 | "ndarray = np.array([1, 2, 3], dtype=np.uint8)\n", 15 | "print(torch.tensor(ndarray))\n", 16 | "print(torch.Tensor(ndarray))\n", 17 | "print(torch.from_numpy(ndarray))" 18 | ] 19 | } 20 | ], 21 | "metadata": { 22 | "kernelspec": { 23 | "display_name": "Python 3 (ipykernel)", 24 | "language": "python", 25 | "name": "python3" 26 | }, 27 | "language_info": { 28 | "codemirror_mode": { 29 | "name": "ipython", 30 | "version": 3 31 | }, 32 | "file_extension": ".py", 33 | "mimetype": "text/x-python", 34 | "name": "python", 35 | "nbconvert_exporter": "python", 36 | "pygments_lexer": "ipython3", 37 | "version": "3.9.13" 38 | } 39 | }, 40 | "nbformat": 4, 41 | "nbformat_minor": 5 42 | } 43 | -------------------------------------------------------------------------------- /부록/예제 C.01 모델 생성.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "ac336f12-8262-42f4-9139-7b8248175450", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import timm\n", 11 | "import torch\n", 12 | "\n", 13 | "\n", 14 | "model = timm.create_model(model_name=\"resnet34\", pretrained=True, num_classes=10)\n", 15 | "image = torch.randn(1, 3, 224, 224)\n", 16 | "logit = model(image)\n", 17 | "print(logit.shape)" 18 | ] 19 | } 20 | ], 21 | "metadata": { 22 | "kernelspec": { 23 | "display_name": "Python 3 (ipykernel)", 24 | "language": "python", 25 | "name": "python3" 26 | }, 27 | "language_info": { 28 | "codemirror_mode": { 29 | "name": "ipython", 30 | "version": 3 31 | }, 32 | "file_extension": ".py", 33 | "mimetype": "text/x-python", 34 | "name": "python", 35 | "nbconvert_exporter": "python", 36 | "pygments_lexer": "ipython3", 37 | "version": "3.9.13" 38 | } 39 | }, 40 | "nbformat": 4, 41 | "nbformat_minor": 5 42 | } 43 | -------------------------------------------------------------------------------- /04장 파이토치 심화/예제 4.01 텐서 초깃값.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "41270908-4d74-4038-855c-43dbf3380355", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "x = torch.FloatTensor(\n", 14 | " [\n", 15 | " [-0.6577, -0.5797, 0.6360],\n", 16 | " [0.7392, 0.2145, 1.523],\n", 17 | " [0.2432, 0.5662, 0.322]\n", 18 | " ]\n", 19 | ")" 20 | ] 21 | } 22 | ], 23 | "metadata": { 24 | "kernelspec": { 25 | "display_name": "Python 3 (ipykernel)", 26 | "language": "python", 27 | "name": "python3" 28 | }, 29 | "language_info": { 30 | "codemirror_mode": { 31 | "name": "ipython", 32 | "version": 3 33 | }, 34 | "file_extension": ".py", 35 | "mimetype": "text/x-python", 36 | "name": "python", 37 | "nbconvert_exporter": "python", 38 | "pygments_lexer": "ipython3", 39 | "version": "3.9.13" 40 | } 41 | }, 42 | "nbformat": 4, 43 | "nbformat_minor": 5 44 | } 45 | -------------------------------------------------------------------------------- /05장 토큰화/예제 5.12 토크나이저 모델 학습.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "b613f94b-4bad-4962-8f6f-8344972168d5", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from sentencepiece import SentencePieceTrainer\n", 11 | "\n", 12 | "\n", 13 | "SentencePieceTrainer.Train(\n", 14 | " \"--input=../datasets/corpus.txt\\\n", 15 | " --model_prefix=../models/petition_bpe\\\n", 16 | " --vocab_size=8000 model_type=bpe\"\n", 17 | ")" 18 | ] 19 | } 20 | ], 21 | "metadata": { 22 | "kernelspec": { 23 | "display_name": "Python 3 (ipykernel)", 24 | "language": "python", 25 | "name": "python3" 26 | }, 27 | "language_info": { 28 | "codemirror_mode": { 29 | "name": "ipython", 30 | "version": 3 31 | }, 32 | "file_extension": ".py", 33 | "mimetype": "text/x-python", 34 | "name": "python", 35 | "nbconvert_exporter": "python", 36 | "pygments_lexer": "ipython3", 37 | "version": "3.9.13" 38 | } 39 | }, 40 | "nbformat": 4, 41 | "nbformat_minor": 5 42 | } 43 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.08 특잇값 분해를 이용한 저계수 분해.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "M = torch.rand((4, 3))\n", 14 | "k = 2\n", 15 | "Uk, sk, Vk = torch.svd_lowrank(M, q=k)\n", 16 | "approximated_M = torch.mm(torch.mm(Uk, torch.diag(sk)), Vk.t())\n", 17 | "print(M)\n", 18 | "print(approximated_M)" 19 | ] 20 | } 21 | ], 22 | "metadata": { 23 | "kernelspec": { 24 | "display_name": "Python 3 (ipykernel)", 25 | "language": "python", 26 | "name": "python3" 27 | }, 28 | "language_info": { 29 | "codemirror_mode": { 30 | "name": "ipython", 31 | "version": 3 32 | }, 33 | "file_extension": ".py", 34 | "mimetype": "text/x-python", 35 | "name": "python", 36 | "nbconvert_exporter": "python", 37 | "pygments_lexer": "ipython3", 38 | "version": "3.9.13" 39 | } 40 | }, 41 | "nbformat": 4, 42 | "nbformat_minor": 5 43 | } 44 | -------------------------------------------------------------------------------- /05장 토큰화/예제 5.09 spaCy 품사 태깅.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "245492d6-02c8-4444-a878-1480ed0fa34f", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import spacy\n", 11 | "\n", 12 | "\n", 13 | "nlp = spacy.load(\"en_core_web_sm\")\n", 14 | "sentence = \"Those who can imagine anything, can create the impossible.\"\n", 15 | "doc = nlp(sentence)\n", 16 | "\n", 17 | "for token in doc:\n", 18 | " print(f\"[{token.pos_:5} - {token.tag_:3}] : {token.text}\")" 19 | ] 20 | } 21 | ], 22 | "metadata": { 23 | "kernelspec": { 24 | "display_name": "Python 3 (ipykernel)", 25 | "language": "python", 26 | "name": "python3" 27 | }, 28 | "language_info": { 29 | "codemirror_mode": { 30 | "name": "ipython", 31 | "version": 3 32 | }, 33 | "file_extension": ".py", 34 | "mimetype": "text/x-python", 35 | "name": "python", 36 | "nbconvert_exporter": "python", 37 | "pygments_lexer": "ipython3", 38 | "version": "3.9.13" 39 | } 40 | }, 41 | "nbformat": 4, 42 | "nbformat_minor": 5 43 | } 44 | -------------------------------------------------------------------------------- /05장 토큰화/예제 5.11 학습 데이터세트 생성.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "8e2d6d57-69dd-41e8-9deb-46adcb8498c9", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from Korpora import Korpora\n", 11 | "\n", 12 | "\n", 13 | "corpus = Korpora.load(\"korean_petitions\")\n", 14 | "petitions = corpus.get_all_texts()\n", 15 | "with open(\"../datasets/corpus.txt\", \"w\", encoding=\"utf-8\") as f:\n", 16 | " for petition in petitions:\n", 17 | " f.write(petition + \"\\n\")" 18 | ] 19 | } 20 | ], 21 | "metadata": { 22 | "kernelspec": { 23 | "display_name": "Python 3 (ipykernel)", 24 | "language": "python", 25 | "name": "python3" 26 | }, 27 | "language_info": { 28 | "codemirror_mode": { 29 | "name": "ipython", 30 | "version": 3 31 | }, 32 | "file_extension": ".py", 33 | "mimetype": "text/x-python", 34 | "name": "python", 35 | "nbconvert_exporter": "python", 36 | "pygments_lexer": "ipython3", 37 | "version": "3.9.13" 38 | } 39 | }, 40 | "nbformat": 4, 41 | "nbformat_minor": 5 42 | } 43 | -------------------------------------------------------------------------------- /04장 파이토치 심화/예제 4.05 L1 정칙화 적용 방식.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "83f61458-0ab6-4471-8714-0cbb5a35adcd", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "for x, y in train_dataloader:\n", 11 | " x = x.to(device)\n", 12 | " y = y.to(device)\n", 13 | "\n", 14 | " output = model(x)\n", 15 | "\n", 16 | " _lambda = 0.5\n", 17 | " l1_loss = sum(p.abs().sum() for p in model.parameters())\n", 18 | "\n", 19 | " loss = criterion(output, y) + _lambda * l1_loss" 20 | ] 21 | } 22 | ], 23 | "metadata": { 24 | "kernelspec": { 25 | "display_name": "Python 3 (ipykernel)", 26 | "language": "python", 27 | "name": "python3" 28 | }, 29 | "language_info": { 30 | "codemirror_mode": { 31 | "name": "ipython", 32 | "version": 3 33 | }, 34 | "file_extension": ".py", 35 | "mimetype": "text/x-python", 36 | "name": "python", 37 | "nbconvert_exporter": "python", 38 | "pygments_lexer": "ipython3", 39 | "version": "3.9.13" 40 | } 41 | }, 42 | "nbformat": 4, 43 | "nbformat_minor": 5 44 | } 45 | -------------------------------------------------------------------------------- /04장 파이토치 심화/예제 4.06 L2 정칙화 적용 방식.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "daae2f11-66a9-4f2b-90d6-4817b2aeb9e1", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "for x, y in train_dataloader:\n", 11 | " x = x.to(device)\n", 12 | " y = y.to(device)\n", 13 | "\n", 14 | " output = model(x)\n", 15 | "\n", 16 | " _lambda = 0.5\n", 17 | " l2_loss = sum(p.pow(2.0).sum() for p in model.parameters())\n", 18 | "\n", 19 | " loss = criterion(output, y) + _lambda * l2_loss" 20 | ] 21 | } 22 | ], 23 | "metadata": { 24 | "kernelspec": { 25 | "display_name": "Python 3 (ipykernel)", 26 | "language": "python", 27 | "name": "python3" 28 | }, 29 | "language_info": { 30 | "codemirror_mode": { 31 | "name": "ipython", 32 | "version": 3 33 | }, 34 | "file_extension": ".py", 35 | "mimetype": "text/x-python", 36 | "name": "python", 37 | "nbconvert_exporter": "python", 38 | "pygments_lexer": "ipython3", 39 | "version": "3.9.13" 40 | } 41 | }, 42 | "nbformat": 4, 43 | "nbformat_minor": 5 44 | } 45 | -------------------------------------------------------------------------------- /부록/예제 C.02 사용 가능한 사전 학습된 모델 확인.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "96ac87e7-9a74-40d2-ac1a-7a9330b6e5fa", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import timm\n", 11 | "\n", 12 | "\n", 13 | "avail_models = timm.list_models()\n", 14 | "avail_pretrained_models = timm.list_models(pretrained=True)\n", 15 | "print(\"사용 가능한 모델 수 :\", len(avail_models))\n", 16 | "print(\"사용 가능한 사전 학습 모델 수 :\", len(avail_pretrained_models))\n", 17 | "print(\"사용 가능한 모델 이름 :\", avail_pretrained_models[:5])" 18 | ] 19 | } 20 | ], 21 | "metadata": { 22 | "kernelspec": { 23 | "display_name": "Python 3 (ipykernel)", 24 | "language": "python", 25 | "name": "python3" 26 | }, 27 | "language_info": { 28 | "codemirror_mode": { 29 | "name": "ipython", 30 | "version": 3 31 | }, 32 | "file_extension": ".py", 33 | "mimetype": "text/x-python", 34 | "name": "python", 35 | "nbconvert_exporter": "python", 36 | "pygments_lexer": "ipython3", 37 | "version": "3.9.13" 38 | } 39 | }, 40 | "nbformat": 4, 41 | "nbformat_minor": 5 42 | } 43 | -------------------------------------------------------------------------------- /04장 파이토치 심화/예제 4.02 배치 정규화 수행.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "41270908-4d74-4038-855c-43dbf3380355", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import nn\n", 12 | "\n", 13 | "\n", 14 | "x = torch.FloatTensor(\n", 15 | " [\n", 16 | " [-0.6577, -0.5797, 0.6360],\n", 17 | " [0.7392, 0.2145, 1.523],\n", 18 | " [0.2432, 0.5662, 0.322]\n", 19 | " ]\n", 20 | ")\n", 21 | "\n", 22 | "print(nn.BatchNorm1d(3)(x))" 23 | ] 24 | } 25 | ], 26 | "metadata": { 27 | "kernelspec": { 28 | "display_name": "Python 3 (ipykernel)", 29 | "language": "python", 30 | "name": "python3" 31 | }, 32 | "language_info": { 33 | "codemirror_mode": { 34 | "name": "ipython", 35 | "version": 3 36 | }, 37 | "file_extension": ".py", 38 | "mimetype": "text/x-python", 39 | "name": "python", 40 | "nbconvert_exporter": "python", 41 | "pygments_lexer": "ipython3", 42 | "version": "3.9.13" 43 | } 44 | }, 45 | "nbformat": 4, 46 | "nbformat_minor": 5 47 | } 48 | -------------------------------------------------------------------------------- /부록/예제 E.01 체크포인트 기법.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch.utils import checkpoint as cp\n", 12 | "\n", 13 | "\n", 14 | "def function(x, y):\n", 15 | " z = x + y\n", 16 | " w = z * y\n", 17 | " return w\n", 18 | "\n", 19 | "x = torch.randn(1000, 1000, requires_grad=True)\n", 20 | "y = torch.randn(1000, 1000, requires_grad=True)\n", 21 | "result = cp.checkpoint(function, x, y)" 22 | ] 23 | } 24 | ], 25 | "metadata": { 26 | "kernelspec": { 27 | "display_name": "Python 3 (ipykernel)", 28 | "language": "python", 29 | "name": "python3" 30 | }, 31 | "language_info": { 32 | "codemirror_mode": { 33 | "name": "ipython", 34 | "version": 3 35 | }, 36 | "file_extension": ".py", 37 | "mimetype": "text/x-python", 38 | "name": "python", 39 | "nbconvert_exporter": "python", 40 | "pygments_lexer": "ipython3", 41 | "version": "3.9.13" 42 | } 43 | }, 44 | "nbformat": 4, 45 | "nbformat_minor": 5 46 | } 47 | -------------------------------------------------------------------------------- /05장 토큰화/예제 5.14 어휘 사전 불러오기.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "ab31ca0b-05de-471b-9c10-eb57bef25ca8", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from sentencepiece import SentencePieceProcessor\n", 11 | "\n", 12 | "\n", 13 | "tokenizer = SentencePieceProcessor()\n", 14 | "tokenizer.load(\"../models/petition_bpe.model\")\n", 15 | "\n", 16 | "vocab = {idx: tokenizer.id_to_piece(idx) for idx in range(tokenizer.get_piece_size())}\n", 17 | "print(list(vocab.items())[:5])\n", 18 | "print(\"vocab size :\", len(vocab))" 19 | ] 20 | } 21 | ], 22 | "metadata": { 23 | "kernelspec": { 24 | "display_name": "Python 3 (ipykernel)", 25 | "language": "python", 26 | "name": "python3" 27 | }, 28 | "language_info": { 29 | "codemirror_mode": { 30 | "name": "ipython", 31 | "version": 3 32 | }, 33 | "file_extension": ".py", 34 | "mimetype": "text/x-python", 35 | "name": "python", 36 | "nbconvert_exporter": "python", 37 | "pygments_lexer": "ipython3", 38 | "version": "3.9.13" 39 | } 40 | }, 41 | "nbformat": 4, 42 | "nbformat_minor": 5 43 | } 44 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.03 텐서 차원 변환.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "3461d6c4-8122-4ad0-ada6-13e276137e6b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "tensor = torch.rand(1, 2)\n", 13 | "print(tensor)\n", 14 | "print(tensor.shape)" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": null, 20 | "id": "48f0b467", 21 | "metadata": {}, 22 | "outputs": [], 23 | "source": [ 24 | "tensor = tensor.reshape(2, 1)\n", 25 | "print(tensor)\n", 26 | "print(tensor.shape)" 27 | ] 28 | } 29 | ], 30 | "metadata": { 31 | "kernelspec": { 32 | "display_name": "Python 3 (ipykernel)", 33 | "language": "python", 34 | "name": "python3" 35 | }, 36 | "language_info": { 37 | "codemirror_mode": { 38 | "name": "ipython", 39 | "version": 3 40 | }, 41 | "file_extension": ".py", 42 | "mimetype": "text/x-python", 43 | "name": "python", 44 | "nbconvert_exporter": "python", 45 | "pygments_lexer": "ipython3", 46 | "version": "3.9.13" 47 | } 48 | }, 49 | "nbformat": 4, 50 | "nbformat_minor": 5 51 | } 52 | -------------------------------------------------------------------------------- /07장 트랜스포머/예제 7.10 GPT-2를 이용한 문장 생성.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "c0b6b124-1b85-4d01-a0a0-be02caf42f77", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from transformers import pipeline\n", 11 | "\n", 12 | "\n", 13 | "generator = pipeline(task=\"text-generation\", model=\"gpt2\")\n", 14 | "outputs = generator(\n", 15 | " text_inputs=\"Machine learning is\",\n", 16 | " max_length=20,\n", 17 | " num_return_sequences=3,\n", 18 | " pad_token_id=generator.tokenizer.eos_token_id\n", 19 | ")\n", 20 | "print(outputs)" 21 | ] 22 | } 23 | ], 24 | "metadata": { 25 | "kernelspec": { 26 | "display_name": "Python 3 (ipykernel)", 27 | "language": "python", 28 | "name": "python3" 29 | }, 30 | "language_info": { 31 | "codemirror_mode": { 32 | "name": "ipython", 33 | "version": 3 34 | }, 35 | "file_extension": ".py", 36 | "mimetype": "text/x-python", 37 | "name": "python", 38 | "nbconvert_exporter": "python", 39 | "pygments_lexer": "ipython3", 40 | "version": "3.9.13" 41 | } 42 | }, 43 | "nbformat": 4, 44 | "nbformat_minor": 5 45 | } 46 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.11 ONNX 형식 변환.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import onnx\n", 12 | "from torchvision import models\n", 13 | "\n", 14 | "\n", 15 | "model = models.vgg16(num_classes=2)\n", 16 | "model.load_state_dict(torch.load(\"../models/VGG16.pt\"))\n", 17 | "model.eval()\n", 18 | "\n", 19 | "dummy_input = torch.randn(1, 3, 224, 224)\n", 20 | "onnx.export(model=model, args=dummy_input, f=\"../models/VGG16.onnx\")" 21 | ] 22 | } 23 | ], 24 | "metadata": { 25 | "kernelspec": { 26 | "display_name": "Python 3 (ipykernel)", 27 | "language": "python", 28 | "name": "python3" 29 | }, 30 | "language_info": { 31 | "codemirror_mode": { 32 | "name": "ipython", 33 | "version": 3 34 | }, 35 | "file_extension": ".py", 36 | "mimetype": "text/x-python", 37 | "name": "python", 38 | "nbconvert_exporter": "python", 39 | "pygments_lexer": "ipython3", 40 | "version": "3.9.13" 41 | } 42 | }, 43 | "nbformat": 4, 44 | "nbformat_minor": 5 45 | } 46 | -------------------------------------------------------------------------------- /04장 파이토치 심화/예제 4.09 그라디언트 클리핑 적용 방식.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "d8af4d15-75d3-4b30-988a-d244842b0953", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "for x, y in train_dataloader:\n", 11 | " x = x.to(device)\n", 12 | " y = y.to(device)\n", 13 | "\n", 14 | " output = model(x)\n", 15 | " loss = criterion(output, y)\n", 16 | " \n", 17 | " optimizer.zero_grad()\n", 18 | " loss.backward()\n", 19 | "\n", 20 | " torch.nn.utils.clip_grad_norm_(model.parameters(), 0.1)\n", 21 | "\n", 22 | " optimizer.step()" 23 | ] 24 | } 25 | ], 26 | "metadata": { 27 | "kernelspec": { 28 | "display_name": "Python 3 (ipykernel)", 29 | "language": "python", 30 | "name": "python3" 31 | }, 32 | "language_info": { 33 | "codemirror_mode": { 34 | "name": "ipython", 35 | "version": 3 36 | }, 37 | "file_extension": ".py", 38 | "mimetype": "text/x-python", 39 | "name": "python", 40 | "nbconvert_exporter": "python", 41 | "pygments_lexer": "ipython3", 42 | "version": "3.9.13" 43 | } 44 | }, 45 | "nbformat": 4, 46 | "nbformat_minor": 5 47 | } 48 | -------------------------------------------------------------------------------- /부록/예제 B.05 ImageProcessor 클래스 이미지 데이터 변환.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "9535f47e-2006-482e-b349-9569fae8928a", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from torchvision import datasets\n", 11 | "from transformers import CLIPImageProcessor\n", 12 | "\n", 13 | "\n", 14 | "dataset = datasets.FashionMNIST(root=\"../datasets\", download=True, train=False)\n", 15 | "image_processor = CLIPImageProcessor(\"openai/clip-vit-base-patch32\")\n", 16 | "\n", 17 | "image, label = dataset[0]\n", 18 | "output = image_processor(image)\n", 19 | "print(output)" 20 | ] 21 | } 22 | ], 23 | "metadata": { 24 | "kernelspec": { 25 | "display_name": "Python 3 (ipykernel)", 26 | "language": "python", 27 | "name": "python3" 28 | }, 29 | "language_info": { 30 | "codemirror_mode": { 31 | "name": "ipython", 32 | "version": 3 33 | }, 34 | "file_extension": ".py", 35 | "mimetype": "text/x-python", 36 | "name": "python", 37 | "nbconvert_exporter": "python", 38 | "pygments_lexer": "ipython3", 39 | "version": "3.9.13" 40 | } 41 | }, 42 | "nbformat": 4, 43 | "nbformat_minor": 5 44 | } 45 | -------------------------------------------------------------------------------- /부록/예제 B.04 PreTrainedFeatureExtractor 입력 특징 추출.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "d2c302e6-4335-4dfb-b90b-dd7809a6ee8c", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from datasets import load_dataset\n", 11 | "from transformers import Wav2Vec2FeatureExtractor\n", 12 | "\n", 13 | "\n", 14 | "mindsKR = load_dataset(\"PolyAI/minds14\", \"ko-KR\", split=\"train\")\n", 15 | "feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained(\"facebook/wav2vec2-base-960h\")\n", 16 | "transformed = feature_extractor(mindsKR[\"audio\"][0][\"array\"])\n", 17 | "print(transformed)" 18 | ] 19 | } 20 | ], 21 | "metadata": { 22 | "kernelspec": { 23 | "display_name": "Python 3 (ipykernel)", 24 | "language": "python", 25 | "name": "python3" 26 | }, 27 | "language_info": { 28 | "codemirror_mode": { 29 | "name": "ipython", 30 | "version": 3 31 | }, 32 | "file_extension": ".py", 33 | "mimetype": "text/x-python", 34 | "name": "python", 35 | "nbconvert_exporter": "python", 36 | "pygments_lexer": "ipython3", 37 | "version": "3.9.13" 38 | } 39 | }, 40 | "nbformat": 4, 41 | "nbformat_minor": 5 42 | } 43 | -------------------------------------------------------------------------------- /부록/예제 D.01 파이토치 컴파일러.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "1875ecb0-a43a-4e9c-b589-a5d46c0e1b33", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torchvision import models\n", 12 | "\n", 13 | "\n", 14 | "model = models.resnet18().cuda()\n", 15 | "optimizer = torch.optim.SGD(model.parameters(), lr=0.01)\n", 16 | "compiled_model = torch.compile(model)\n", 17 | "\n", 18 | "x = torch.randn(16, 3, 224, 224).cuda()\n", 19 | "optimizer.zero_grad()\n", 20 | "out = compiled_model(x)\n", 21 | "out.sum().backward()\n", 22 | "optimizer.step()" 23 | ] 24 | } 25 | ], 26 | "metadata": { 27 | "kernelspec": { 28 | "display_name": "Python 3 (ipykernel)", 29 | "language": "python", 30 | "name": "python3" 31 | }, 32 | "language_info": { 33 | "codemirror_mode": { 34 | "name": "ipython", 35 | "version": 3 36 | }, 37 | "file_extension": ".py", 38 | "mimetype": "text/x-python", 39 | "name": "python", 40 | "nbconvert_exporter": "python", 41 | "pygments_lexer": "ipython3", 42 | "version": "3.9.13" 43 | } 44 | }, 45 | "nbformat": 4, 46 | "nbformat_minor": 5 47 | } 48 | -------------------------------------------------------------------------------- /05장 토큰화/예제 5.01~5.02 단어 및 글자 토큰화.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "d86b53dc-ba59-48ab-9276-cf1d5b24c6a7", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "review = \" 현실과 구분 불가능한 cg. 시각적 즐거음은 최고! 더불어 ost는 더더욱 최고!!\"\n", 11 | "tokenized = review.split()\n", 12 | "print(tokenized)" 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": null, 18 | "id": "1582d2e5", 19 | "metadata": {}, 20 | "outputs": [], 21 | "source": [ 22 | "review = \"현실과 구분 불가능한 cg. 시각적 즐거음은 최고! 더불어 ost는 더더욱 최고!!\"\n", 23 | "tokenized = list(review)\n", 24 | "print(tokenized)" 25 | ] 26 | } 27 | ], 28 | "metadata": { 29 | "kernelspec": { 30 | "display_name": "Python 3 (ipykernel)", 31 | "language": "python", 32 | "name": "python3" 33 | }, 34 | "language_info": { 35 | "codemirror_mode": { 36 | "name": "ipython", 37 | "version": 3 38 | }, 39 | "file_extension": ".py", 40 | "mimetype": "text/x-python", 41 | "name": "python", 42 | "nbconvert_exporter": "python", 43 | "pygments_lexer": "ipython3", 44 | "version": "3.9.13" 45 | } 46 | }, 47 | "nbformat": 4, 48 | "nbformat_minor": 5 49 | } 50 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.16 모델 추론 요청.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import json\n", 11 | "import requests\n", 12 | "\n", 13 | "\n", 14 | "url = \"http://127.0.0.1:8000/predict\"\n", 15 | "headers = {\"content-type\": \"application/json\"}\n", 16 | "\n", 17 | "response = requests.post(\n", 18 | " url=url,\n", 19 | " headers=headers,\n", 20 | " data=json.dumps({\"text\": \"정말 재미 있어요!\"})\n", 21 | ")\n", 22 | "\n", 23 | "print(response.status_code)\n", 24 | "print(response.json())" 25 | ] 26 | } 27 | ], 28 | "metadata": { 29 | "kernelspec": { 30 | "display_name": "Python 3 (ipykernel)", 31 | "language": "python", 32 | "name": "python3" 33 | }, 34 | "language_info": { 35 | "codemirror_mode": { 36 | "name": "ipython", 37 | "version": 3 38 | }, 39 | "file_extension": ".py", 40 | "mimetype": "text/x-python", 41 | "name": "python", 42 | "nbconvert_exporter": "python", 43 | "pygments_lexer": "ipython3", 44 | "version": "3.9.13" 45 | } 46 | }, 47 | "nbformat": 4, 48 | "nbformat_minor": 5 49 | } 50 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.48 모델 구조 확인.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "1356b5c4-632b-400e-ae83-6d021b5a1366", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import nn\n", 12 | "\n", 13 | "\n", 14 | "class CustomModel(nn.Module):\n", 15 | " pass" 16 | ] 17 | }, 18 | { 19 | "cell_type": "code", 20 | "execution_count": null, 21 | "id": "98a8ee5a", 22 | "metadata": {}, 23 | "outputs": [], 24 | "source": [ 25 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 26 | "model = torch.load(\"../models/model.pt\", map_location=device)\n", 27 | "print(model)" 28 | ] 29 | } 30 | ], 31 | "metadata": { 32 | "kernelspec": { 33 | "display_name": "Python 3 (ipykernel)", 34 | "language": "python", 35 | "name": "python3" 36 | }, 37 | "language_info": { 38 | "codemirror_mode": { 39 | "name": "ipython", 40 | "version": 3 41 | }, 42 | "file_extension": ".py", 43 | "mimetype": "text/x-python", 44 | "name": "python", 45 | "nbconvert_exporter": "python", 46 | "pygments_lexer": "ipython3", 47 | "version": "3.9.13" 48 | } 49 | }, 50 | "nbformat": 4, 51 | "nbformat_minor": 5 52 | } 53 | -------------------------------------------------------------------------------- /04장 파이토치 심화/예제 4.08 드롭아웃 적용 방식.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "c8d730bd-5268-4006-a40d-2cb1d7ea215e", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from torch import nn\n", 11 | "\n", 12 | "\n", 13 | "class Net(nn.Module):\n", 14 | " def __init__(self):\n", 15 | " super().__init__()\n", 16 | " self.layer1 = nn.Linear(10, 10)\n", 17 | " self.dropout = nn.Dropout(p=0.5)\n", 18 | " self.layer2 = nn.Linear(10, 10)\n", 19 | "\n", 20 | " def forward(self, x):\n", 21 | " x = self.layer1(x)\n", 22 | " x = self.dropout(x)\n", 23 | " x = self.layer2(x)\n", 24 | " return x" 25 | ] 26 | } 27 | ], 28 | "metadata": { 29 | "kernelspec": { 30 | "display_name": "Python 3 (ipykernel)", 31 | "language": "python", 32 | "name": "python3" 33 | }, 34 | "language_info": { 35 | "codemirror_mode": { 36 | "name": "ipython", 37 | "version": 3 38 | }, 39 | "file_extension": ".py", 40 | "mimetype": "text/x-python", 41 | "name": "python", 42 | "nbconvert_exporter": "python", 43 | "pygments_lexer": "ipython3", 44 | "version": "3.9.13" 45 | } 46 | }, 47 | "nbformat": 4, 48 | "nbformat_minor": 5 49 | } 50 | -------------------------------------------------------------------------------- /05장 토큰화/예제 5.15 워드피스 토크나이저 학습.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "7520f0f2-d4d9-4660-9751-5cfc2726f6d7", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from tokenizers import Tokenizer\n", 11 | "from tokenizers.models import WordPiece\n", 12 | "from tokenizers.normalizers import Sequence, NFD, Lowercase\n", 13 | "from tokenizers.pre_tokenizers import Whitespace\n", 14 | "\n", 15 | "\n", 16 | "tokenizer = Tokenizer(WordPiece())\n", 17 | "tokenizer.normalizer = Sequence([NFD(), Lowercase()])\n", 18 | "tokenizer.pre_tokenizer = Whitespace()\n", 19 | "\n", 20 | "tokenizer.train([\"../datasets/corpus.txt\"])\n", 21 | "tokenizer.save(\"../models/petition_wordpiece.json\")" 22 | ] 23 | } 24 | ], 25 | "metadata": { 26 | "kernelspec": { 27 | "display_name": "Python 3 (ipykernel)", 28 | "language": "python", 29 | "name": "python3" 30 | }, 31 | "language_info": { 32 | "codemirror_mode": { 33 | "name": "ipython", 34 | "version": 3 35 | }, 36 | "file_extension": ".py", 37 | "mimetype": "text/x-python", 38 | "name": "python", 39 | "nbconvert_exporter": "python", 40 | "pygments_lexer": "ipython3", 41 | "version": "3.9.13" 42 | } 43 | }, 44 | "nbformat": 4, 45 | "nbformat_minor": 5 46 | } 47 | -------------------------------------------------------------------------------- /부록/예제 B.03 PreTrainedTokenizer의 사전 토큰화와 정규화.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "53e1fa8d-c53c-4c78-92c8-46669da174ca", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from transformers import BertTokenizerFast\n", 11 | "\n", 12 | "\n", 13 | "tokenizer = BertTokenizerFast.from_pretrained(\"bert-base-uncased\")\n", 14 | "\n", 15 | "sentence = \"THIS SENTENCÉ is written in UPPERCASE\"\n", 16 | "normalize = tokenizer.backend_tokenizer.normalizer.normalize_str(sentence)\n", 17 | "print(\"정규화 :\", normalize)\n", 18 | "\n", 19 | "sentence = \"이 문장은 삼중 공백을 포함하고 있습니다.\"\n", 20 | "pre_tokenize = tokenizer.backend_tokenizer.pre_tokenizer.pre_tokenize_str(sentence)\n", 21 | "print(\"사전 토큰화 :\", pre_tokenize)" 22 | ] 23 | } 24 | ], 25 | "metadata": { 26 | "kernelspec": { 27 | "display_name": "Python 3 (ipykernel)", 28 | "language": "python", 29 | "name": "python3" 30 | }, 31 | "language_info": { 32 | "codemirror_mode": { 33 | "name": "ipython", 34 | "version": 3 35 | }, 36 | "file_extension": ".py", 37 | "mimetype": "text/x-python", 38 | "name": "python", 39 | "nbconvert_exporter": "python", 40 | "pygments_lexer": "ipython3", 41 | "version": "3.9.13" 42 | } 43 | }, 44 | "nbformat": 4, 45 | "nbformat_minor": 5 46 | } 47 | -------------------------------------------------------------------------------- /06장 임베딩/예제 6.02 TF-IDF 계산.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "7b4a00f1-bdec-441d-bc13-121e0c296410", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from sklearn.feature_extraction.text import TfidfVectorizer\n", 11 | "\n", 12 | "\n", 13 | "corpus = [\n", 14 | " \"That movie is famous movie\",\n", 15 | " \"I like that actor\",\n", 16 | " \"I don’t like that actor\"\n", 17 | "]\n", 18 | "\n", 19 | "tfidf_vectorizer = TfidfVectorizer()\n", 20 | "tfidf_vectorizer.fit(corpus)\n", 21 | "tfidf_matrix = tfidf_vectorizer.transform(corpus)\n", 22 | "# tfidf_matrix = tfidf_vectorizer.fit_transform(corpus)\n", 23 | "\n", 24 | "print(tfidf_matrix.toarray())\n", 25 | "print(tfidf_vectorizer.vocabulary_)" 26 | ] 27 | } 28 | ], 29 | "metadata": { 30 | "kernelspec": { 31 | "display_name": "Python 3 (ipykernel)", 32 | "language": "python", 33 | "name": "python3" 34 | }, 35 | "language_info": { 36 | "codemirror_mode": { 37 | "name": "ipython", 38 | "version": 3 39 | }, 40 | "file_extension": ".py", 41 | "mimetype": "text/x-python", 42 | "name": "python", 43 | "nbconvert_exporter": "python", 44 | "pygments_lexer": "ipython3", 45 | "version": "3.9.13" 46 | } 47 | }, 48 | "nbformat": 4, 49 | "nbformat_minor": 5 50 | } 51 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.08 텐서의 넘파이 배열 변환.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "b1150d10-f744-4472-8e1f-7d6a8930c943", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "tensor = torch.cuda.FloatTensor([1, 2, 3])\n", 14 | "ndarray = tensor.detach().cpu().numpy()\n", 15 | "print(ndarray)\n", 16 | "print(type(ndarray))" 17 | ] 18 | }, 19 | { 20 | "cell_type": "code", 21 | "execution_count": null, 22 | "id": "9e8c5ce4", 23 | "metadata": {}, 24 | "outputs": [], 25 | "source": [ 26 | "import torch\n", 27 | "\n", 28 | "\n", 29 | "tensor = torch.FloatTensor([1, 2, 3]).to(\"mps\")\n", 30 | "ndarray = tensor.detach().cpu().numpy()\n", 31 | "print(ndarray)\n", 32 | "print(type(ndarray))" 33 | ] 34 | } 35 | ], 36 | "metadata": { 37 | "kernelspec": { 38 | "display_name": "Python 3 (ipykernel)", 39 | "language": "python", 40 | "name": "python3" 41 | }, 42 | "language_info": { 43 | "codemirror_mode": { 44 | "name": "ipython", 45 | "version": 3 46 | }, 47 | "file_extension": ".py", 48 | "mimetype": "text/x-python", 49 | "name": "python", 50 | "nbconvert_exporter": "python", 51 | "pygments_lexer": "ipython3", 52 | "version": "3.9.13" 53 | } 54 | }, 55 | "nbformat": 4, 56 | "nbformat_minor": 5 57 | } 58 | -------------------------------------------------------------------------------- /07장 트랜스포머/예제 7.09 문장 생성을 위한 GPT-2 모델의 구조.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "2481314a-b8de-4fed-aa97-b95b396c5f7a", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from transformers import GPT2LMHeadModel\n", 11 | "\n", 12 | "\n", 13 | "model = GPT2LMHeadModel.from_pretrained(pretrained_model_name_or_path=\"gpt2\")\n", 14 | "\n", 15 | "for main_name, main_module in model.named_children():\n", 16 | " print(main_name)\n", 17 | " for sub_name, sub_module in main_module.named_children():\n", 18 | " print(\"└\", sub_name)\n", 19 | " for ssub_name, ssub_module in sub_module.named_children():\n", 20 | " print(\"│ └\", ssub_name)\n", 21 | " for sssub_name, sssub_module in ssub_module.named_children():\n", 22 | " print(\"│ │ └\", sssub_name)" 23 | ] 24 | } 25 | ], 26 | "metadata": { 27 | "kernelspec": { 28 | "display_name": "Python 3 (ipykernel)", 29 | "language": "python", 30 | "name": "python3" 31 | }, 32 | "language_info": { 33 | "codemirror_mode": { 34 | "name": "ipython", 35 | "version": 3 36 | }, 37 | "file_extension": ".py", 38 | "mimetype": "text/x-python", 39 | "name": "python", 40 | "nbconvert_exporter": "python", 41 | "pygments_lexer": "ipython3", 42 | "version": "3.9.13" 43 | } 44 | }, 45 | "nbformat": 4, 46 | "nbformat_minor": 5 47 | } 48 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.37 문장 생성.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "# demo.py\n", 11 | "import streamlit as st\n", 12 | "from transformers import pipeline\n", 13 | "\n", 14 | "\n", 15 | "@st.cache_resource\n", 16 | "def load_model():\n", 17 | " return pipeline(task=\"text-generation\", model=\"gpt2\")\n", 18 | "\n", 19 | "\n", 20 | "model = load_model()\n", 21 | "\n", 22 | "text = st.text_input(\"텍스트 입력\", value=\"Machine learning is\")\n", 23 | "if text:\n", 24 | " result = model(\n", 25 | " text_inputs=text,\n", 26 | " max_length=30,\n", 27 | " num_return_sequences=3,\n", 28 | " pad_token_id=model.tokenizer.eos_token_id,\n", 29 | " )\n", 30 | " st.write(result)" 31 | ] 32 | } 33 | ], 34 | "metadata": { 35 | "kernelspec": { 36 | "display_name": "Python 3 (ipykernel)", 37 | "language": "python", 38 | "name": "python3" 39 | }, 40 | "language_info": { 41 | "codemirror_mode": { 42 | "name": "ipython", 43 | "version": 3 44 | }, 45 | "file_extension": ".py", 46 | "mimetype": "text/x-python", 47 | "name": "python", 48 | "nbconvert_exporter": "python", 49 | "pygments_lexer": "ipython3", 50 | "version": "3.9.13" 51 | } 52 | }, 53 | "nbformat": 4, 54 | "nbformat_minor": 5 55 | } 56 | -------------------------------------------------------------------------------- /부록/예제 B.06 Auto 클래스를 이용한 BERT와 GPT2 구축.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "08b4c895-fc76-47df-9367-38dcf636b375", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from transformers import AutoModel, AutoTokenizer\n", 12 | "\n", 13 | "\n", 14 | "bert = AutoModel.from_pretrained(\"bert-base-uncased\")\n", 15 | "bert_tokenizer = AutoTokenizer.from_pretrained(\"bert-base-uncased\")\n", 16 | "\n", 17 | "gpt2 = AutoModel.from_pretrained(\"gpt2\")\n", 18 | "gpt2_tokenizer = AutoTokenizer.from_pretrained(\"gpt2\")\n", 19 | "\n", 20 | "sentence = \"AutoClass is so convenient method\"\n", 21 | "\n", 22 | "with torch.no_grad():\n", 23 | " bert_output = bert(**bert_tokenizer(sentence, return_tensors=\"pt\"))\n", 24 | " gpt2_output = gpt2(**gpt2_tokenizer(sentence, return_tensors=\"pt\"))" 25 | ] 26 | } 27 | ], 28 | "metadata": { 29 | "kernelspec": { 30 | "display_name": "Python 3 (ipykernel)", 31 | "language": "python", 32 | "name": "python3" 33 | }, 34 | "language_info": { 35 | "codemirror_mode": { 36 | "name": "ipython", 37 | "version": 3 38 | }, 39 | "file_extension": ".py", 40 | "mimetype": "text/x-python", 41 | "name": "python", 42 | "nbconvert_exporter": "python", 43 | "pygments_lexer": "ipython3", 44 | "version": "3.9.13" 45 | } 46 | }, 47 | "nbformat": 4, 48 | "nbformat_minor": 5 49 | } 50 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.06 텐서 장치 변환.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "8c1a9d97-9c69-464f-84ee-37075b86075b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "cpu = torch.FloatTensor([1, 2, 3])\n", 14 | "gpu = cpu.cuda()\n", 15 | "gpu2cpu = gpu.cpu()\n", 16 | "cpu2gpu = cpu.to(\"cuda\")\n", 17 | "print(cpu)\n", 18 | "print(gpu)\n", 19 | "print(gpu2cpu)\n", 20 | "print(cpu2gpu)" 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": null, 26 | "id": "0ba10270", 27 | "metadata": {}, 28 | "outputs": [], 29 | "source": [ 30 | "# 애플 실리콘이 탑재된 맥 사용자의 경우\n", 31 | "import torch\n", 32 | "\n", 33 | "\n", 34 | "cpu = torch.FloatTensor([1, 2, 3])\n", 35 | "gpu = cpu.to(\"mps\")\n", 36 | "gpu2cpu = gpu.cpu()\n", 37 | "print(cpu)\n", 38 | "print(gpu)\n", 39 | "print(gpu2cpu)" 40 | ] 41 | } 42 | ], 43 | "metadata": { 44 | "kernelspec": { 45 | "display_name": "Python 3 (ipykernel)", 46 | "language": "python", 47 | "name": "python3" 48 | }, 49 | "language_info": { 50 | "codemirror_mode": { 51 | "name": "ipython", 52 | "version": 3 53 | }, 54 | "file_extension": ".py", 55 | "mimetype": "text/x-python", 56 | "name": "python", 57 | "nbconvert_exporter": "python", 58 | "pygments_lexer": "ipython3", 59 | "version": "3.9.13" 60 | } 61 | }, 62 | "nbformat": 4, 63 | "nbformat_minor": 5 64 | } 65 | -------------------------------------------------------------------------------- /04장 파이토치 심화/예제 4.03 가중치 초기화 함수 (1).ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "9a48813a-353b-478d-ae2d-f236ff2dab7f", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from torch import nn\n", 11 | "\n", 12 | "\n", 13 | "class Net(nn.Module):\n", 14 | " def __init__(self):\n", 15 | " super().__init__()\n", 16 | " self.layer = nn.Sequential(\n", 17 | " nn.Linear(1, 2),\n", 18 | " nn.Sigmoid()\n", 19 | " )\n", 20 | " self.fc = nn.Linear(2, 1)\n", 21 | " self._init_weights()\n", 22 | "\n", 23 | " def _init_weights(self):\n", 24 | " nn.init.xavier_uniform_(self.layer[0].weight)\n", 25 | " self.layer[0].bias.data.fill_(0.01)\n", 26 | "\n", 27 | " nn.init.xavier_uniform_(self.fc.weight)\n", 28 | " self.fc.bias.data.fill_(0.01)\n", 29 | "\n", 30 | "\n", 31 | "model = Net()" 32 | ] 33 | } 34 | ], 35 | "metadata": { 36 | "kernelspec": { 37 | "display_name": "Python 3 (ipykernel)", 38 | "language": "python", 39 | "name": "python3" 40 | }, 41 | "language_info": { 42 | "codemirror_mode": { 43 | "name": "ipython", 44 | "version": 3 45 | }, 46 | "file_extension": ".py", 47 | "mimetype": "text/x-python", 48 | "name": "python", 49 | "nbconvert_exporter": "python", 50 | "pygments_lexer": "ipython3", 51 | "version": "3.9.13" 52 | } 53 | }, 54 | "nbformat": 4, 55 | "nbformat_minor": 5 56 | } 57 | -------------------------------------------------------------------------------- /04장 파이토치 심화/예제 4.04 가중치 초기화 함수 (2).ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "3223e634-65ad-466d-b2eb-1192d8d8c165", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from torch import nn\n", 11 | "\n", 12 | "\n", 13 | "class Net(nn.Module):\n", 14 | " def __init__(self):\n", 15 | " super().__init__()\n", 16 | " self.layer = nn.Sequential(\n", 17 | " nn.Linear(1, 2),\n", 18 | " nn.Sigmoid()\n", 19 | " )\n", 20 | " self.fc = nn.Linear(2, 1)\n", 21 | " self.apply(self._init_weights)\n", 22 | "\n", 23 | " def _init_weights(self, module):\n", 24 | " if isinstance(module, nn.Linear):\n", 25 | " nn.init.xavier_uniform_(module.weight)\n", 26 | " nn.init.constant_(module.bias, 0.01)\n", 27 | " print(f\"Apply : {module}\")\n", 28 | "\n", 29 | "\n", 30 | "model = Net()" 31 | ] 32 | } 33 | ], 34 | "metadata": { 35 | "kernelspec": { 36 | "display_name": "Python 3 (ipykernel)", 37 | "language": "python", 38 | "name": "python3" 39 | }, 40 | "language_info": { 41 | "codemirror_mode": { 42 | "name": "ipython", 43 | "version": 3 44 | }, 45 | "file_extension": ".py", 46 | "mimetype": "text/x-python", 47 | "name": "python", 48 | "nbconvert_exporter": "python", 49 | "pygments_lexer": "ipython3", 50 | "version": "3.9.13" 51 | } 52 | }, 53 | "nbformat": 4, 54 | "nbformat_minor": 5 55 | } 56 | -------------------------------------------------------------------------------- /06장 임베딩/예제 6.01 N-gram.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "ee13d109-b972-429c-94c9-55dd7f37f749", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import nltk\n", 11 | "\n", 12 | "\n", 13 | "def ngrams(sentence, n):\n", 14 | " words = sentence.split()\n", 15 | " ngrams = zip(*[words[i:] for i in range(n)])\n", 16 | " return list(ngrams)\n", 17 | "\n", 18 | "sentence = \"안녕하세요 만나서 진심으로 반가워요\"\n", 19 | "\n", 20 | "unigram = ngrams(sentence, 1)\n", 21 | "bigram = ngrams(sentence, 2)\n", 22 | "trigram = ngrams(sentence, 3)\n", 23 | "\n", 24 | "print(unigram)\n", 25 | "print(bigram)\n", 26 | "print(trigram)\n", 27 | "\n", 28 | "unigram = nltk.ngrams(sentence.split(), 1)\n", 29 | "bigram = nltk.ngrams(sentence.split(), 2)\n", 30 | "trigram = nltk.ngrams(sentence.split(), 3)\n", 31 | "\n", 32 | "print(list(unigram))\n", 33 | "print(list(bigram))\n", 34 | "print(list(trigram))" 35 | ] 36 | } 37 | ], 38 | "metadata": { 39 | "kernelspec": { 40 | "display_name": "Python 3 (ipykernel)", 41 | "language": "python", 42 | "name": "python3" 43 | }, 44 | "language_info": { 45 | "codemirror_mode": { 46 | "name": "ipython", 47 | "version": 3 48 | }, 49 | "file_extension": ".py", 50 | "mimetype": "text/x-python", 51 | "name": "python", 52 | "nbconvert_exporter": "python", 53 | "pygments_lexer": "ipython3", 54 | "version": "3.9.13" 55 | } 56 | }, 57 | "nbformat": 4, 58 | "nbformat_minor": 5 59 | } 60 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.20 모델 추론 요청.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import io\n", 11 | "import json\n", 12 | "import base64\n", 13 | "import requests\n", 14 | "from PIL import Image\n", 15 | "\n", 16 | "\n", 17 | "url = \"http://127.0.0.1:8000/predict\"\n", 18 | "headers = {\"content-type\": \"application/json\"}\n", 19 | "\n", 20 | "image = Image.open(\"../datasets/images/dog.jpg\")\n", 21 | "with io.BytesIO() as buffer:\n", 22 | " image.save(buffer, format=\"JPEG\")\n", 23 | " buffer.seek(0)\n", 24 | " bytes = buffer.read()\n", 25 | "string = base64.b64encode(bytes).decode(\"utf-8\")\n", 26 | "\n", 27 | "response = requests.post(\n", 28 | " url=url,\n", 29 | " headers=headers,\n", 30 | " data=json.dumps({\"base64\": string})\n", 31 | ")\n", 32 | "\n", 33 | "print(response.status_code)\n", 34 | "print(response.json())" 35 | ] 36 | } 37 | ], 38 | "metadata": { 39 | "kernelspec": { 40 | "display_name": "Python 3 (ipykernel)", 41 | "language": "python", 42 | "name": "python3" 43 | }, 44 | "language_info": { 45 | "codemirror_mode": { 46 | "name": "ipython", 47 | "version": 3 48 | }, 49 | "file_extension": ".py", 50 | "mimetype": "text/x-python", 51 | "name": "python", 52 | "nbconvert_exporter": "python", 53 | "pygments_lexer": "ipython3", 54 | "version": "3.9.13" 55 | } 56 | }, 57 | "nbformat": 4, 58 | "nbformat_minor": 5 59 | } 60 | -------------------------------------------------------------------------------- /부록/예제 B.02 PreTrainedModel 클래스 구성.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "1b3fb489-b252-4383-b024-7e47925395f6", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from transformers import BertConfig, BertModel\n", 11 | "\n", 12 | "\n", 13 | "def print_architecture(model):\n", 14 | " for main_name, main_module in model.named_children():\n", 15 | " print(main_name)\n", 16 | " for sub_name, sub_module in main_module.named_children():\n", 17 | " print(\"└\", sub_name)\n", 18 | " for ssub_name, ssub_module in sub_module.named_children():\n", 19 | " print(\"│ └\", ssub_name)\n", 20 | " for sssub_name, sssub_module in ssub_module.named_children():\n", 21 | " print(\"│ └\", sssub_name)\n", 22 | "\n", 23 | "config = BertConfig.from_pretrained(\"bert-base-uncased\")\n", 24 | "config.update({\"num_hidden_layers\":2})\n", 25 | "model = BertModel(config)\n", 26 | "print_architecture(model)" 27 | ] 28 | } 29 | ], 30 | "metadata": { 31 | "kernelspec": { 32 | "display_name": "Python 3 (ipykernel)", 33 | "language": "python", 34 | "name": "python3" 35 | }, 36 | "language_info": { 37 | "codemirror_mode": { 38 | "name": "ipython", 39 | "version": 3 40 | }, 41 | "file_extension": ".py", 42 | "mimetype": "text/x-python", 43 | "name": "python", 44 | "nbconvert_exporter": "python", 45 | "pygments_lexer": "ipython3", 46 | "version": "3.9.13" 47 | } 48 | }, 49 | "nbformat": 4, 50 | "nbformat_minor": 5 51 | } 52 | -------------------------------------------------------------------------------- /부록/예제 C.03 미세 조정.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "c77e1e5b-e47d-4447-9e6e-b596ed549f7f", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from fastai.vision.all import (\n", 11 | " URLs,\n", 12 | " untar_data,\n", 13 | " ImageDataLoaders,\n", 14 | " Resize,\n", 15 | " get_image_files,\n", 16 | " error_rate,\n", 17 | " vision_learner,\n", 18 | ")\n", 19 | "\n", 20 | "\n", 21 | "def is_upper(x):\n", 22 | " return x[0].isupper()\n", 23 | "\n", 24 | "\n", 25 | "path = untar_data(URLs.PETS) / \"images\"\n", 26 | "print(\"Path:\", path)\n", 27 | "\n", 28 | "dls = ImageDataLoaders.from_name_func(\n", 29 | " path,\n", 30 | " get_image_files(path),\n", 31 | " bs=32,\n", 32 | " valid_pct=0.2,\n", 33 | " label_func=is_upper,\n", 34 | " item_tfms=Resize(224),\n", 35 | ")\n", 36 | "\n", 37 | "learn = vision_learner(dls, \"vit_tiny_patch16_224\", metrics=error_rate)\n", 38 | "learn.fine_tune(1)" 39 | ] 40 | } 41 | ], 42 | "metadata": { 43 | "kernelspec": { 44 | "display_name": "Python 3 (ipykernel)", 45 | "language": "python", 46 | "name": "python3" 47 | }, 48 | "language_info": { 49 | "codemirror_mode": { 50 | "name": "ipython", 51 | "version": 3 52 | }, 53 | "file_extension": ".py", 54 | "mimetype": "text/x-python", 55 | "name": "python", 56 | "nbconvert_exporter": "python", 57 | "pygments_lexer": "ipython3", 58 | "version": "3.9.13" 59 | } 60 | }, 61 | "nbformat": 4, 62 | "nbformat_minor": 5 63 | } 64 | -------------------------------------------------------------------------------- /06장 임베딩/예제 6.18 양방향 다층 신경망.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "41d5a81c-54be-4c00-9378-4be0122ba613", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import nn\n", 12 | "\n", 13 | "\n", 14 | "input_size = 128\n", 15 | "ouput_size = 256\n", 16 | "num_layers = 3\n", 17 | "bidirectional = True\n", 18 | "\n", 19 | "model = nn.RNN(\n", 20 | " input_size=input_size,\n", 21 | " hidden_size=ouput_size,\n", 22 | " num_layers=num_layers,\n", 23 | " nonlinearity=\"tanh\",\n", 24 | " batch_first=True,\n", 25 | " bidirectional=bidirectional,\n", 26 | ")\n", 27 | "\n", 28 | "batch_size = 4\n", 29 | "sequence_len = 6\n", 30 | "\n", 31 | "inputs = torch.randn(batch_size, sequence_len, input_size)\n", 32 | "h_0 = torch.rand(num_layers * (int(bidirectional) + 1), batch_size, ouput_size)\n", 33 | "\n", 34 | "outputs, hidden = model(inputs, h_0)\n", 35 | "print(outputs.shape)\n", 36 | "print(hidden.shape)" 37 | ] 38 | } 39 | ], 40 | "metadata": { 41 | "kernelspec": { 42 | "display_name": "Python 3 (ipykernel)", 43 | "language": "python", 44 | "name": "python3" 45 | }, 46 | "language_info": { 47 | "codemirror_mode": { 48 | "name": "ipython", 49 | "version": 3 50 | }, 51 | "file_extension": ".py", 52 | "mimetype": "text/x-python", 53 | "name": "python", 54 | "nbconvert_exporter": "python", 55 | "pygments_lexer": "ipython3", 56 | "version": "3.9.13" 57 | } 58 | }, 59 | "nbformat": 4, 60 | "nbformat_minor": 5 61 | } 62 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.34 애플리케이션 구성.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "# demo.py\n", 11 | "import pandas as pd\n", 12 | "import streamlit as st\n", 13 | "\n", 14 | "\n", 15 | "st.set_page_config(\n", 16 | " page_title=\"데모 애플리케이션\",\n", 17 | " page_icon=\":shark:\",\n", 18 | " layout=\"wide\"\n", 19 | ")\n", 20 | "\n", 21 | "df = pd.read_csv(\"../datasets/non_linear.csv\")\n", 22 | "\n", 23 | "st.header(body=\"Demo Application\")\n", 24 | "st.subheader(body=\"non_linear.csv\")\n", 25 | "\n", 26 | "x = st.sidebar.selectbox(label=\"X 축\", options=df.columns, index=0)\n", 27 | "y = st.sidebar.selectbox(label=\"Y 축\", options=df.columns, index=1)\n", 28 | "\n", 29 | "col1, col2 = st.columns(2)\n", 30 | "with col1:\n", 31 | " st.dataframe(data=df, height=500, use_container_width=True)\n", 32 | "with col2:\n", 33 | " st.line_chart(data=df, x=x, y=y, height=500)" 34 | ] 35 | } 36 | ], 37 | "metadata": { 38 | "kernelspec": { 39 | "display_name": "Python 3 (ipykernel)", 40 | "language": "python", 41 | "name": "python3" 42 | }, 43 | "language_info": { 44 | "codemirror_mode": { 45 | "name": "ipython", 46 | "version": 3 47 | }, 48 | "file_extension": ".py", 49 | "mimetype": "text/x-python", 50 | "name": "python", 51 | "nbconvert_exporter": "python", 52 | "pygments_lexer": "ipython3", 53 | "version": "3.9.13" 54 | } 55 | }, 56 | "nbformat": 4, 57 | "nbformat_minor": 5 58 | } 59 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.05 텐서 GPU 장치 설정.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "c8358bf9-2c72-44ab-8560-2c73cb5c4da3", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 14 | "cpu = torch.FloatTensor([1, 2, 3])\n", 15 | "gpu = torch.cuda.FloatTensor([1, 2, 3])\n", 16 | "tensor = torch.rand((1, 1), device=device)\n", 17 | "print(device)\n", 18 | "print(cpu)\n", 19 | "print(gpu)\n", 20 | "print(tensor)" 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": null, 26 | "id": "a9bce0cd", 27 | "metadata": {}, 28 | "outputs": [], 29 | "source": [ 30 | "# 애플 실리콘이 탑재된 맥 사용자의 경우\n", 31 | "import torch\n", 32 | "\n", 33 | "\n", 34 | "device = \"mps\" if torch.backends.mps.is_available() and torch.backends.mps.is_built() else \"cpu\"\n", 35 | "cpu = torch.FloatTensor([1, 2, 3])\n", 36 | "mps = torch.rand((1, 1), device=device)\n", 37 | "print(device)\n", 38 | "print(mps)" 39 | ] 40 | } 41 | ], 42 | "metadata": { 43 | "kernelspec": { 44 | "display_name": "Python 3 (ipykernel)", 45 | "language": "python", 46 | "name": "python3" 47 | }, 48 | "language_info": { 49 | "codemirror_mode": { 50 | "name": "ipython", 51 | "version": 3 52 | }, 53 | "file_extension": ".py", 54 | "mimetype": "text/x-python", 55 | "name": "python", 56 | "nbconvert_exporter": "python", 57 | "pygments_lexer": "ipython3", 58 | "version": "3.9.13" 59 | } 60 | }, 61 | "nbformat": 4, 62 | "nbformat_minor": 5 63 | } 64 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.36 데이터 캐싱.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "# demo.py\n", 11 | "import pandas as pd\n", 12 | "import streamlit as st\n", 13 | "\n", 14 | "\n", 15 | "st.set_page_config(\n", 16 | " page_title=\"데모 애플리케이션\",\n", 17 | " page_icon=\":shark:\",\n", 18 | " layout=\"wide\")\n", 19 | "\n", 20 | "@st.cache_data\n", 21 | "def load_data(path):\n", 22 | " return pd.read_csv(path)\n", 23 | "\n", 24 | "df = load_data(\"../datasets/non_linear.csv\")\n", 25 | "\n", 26 | "st.header(body=\"Demo Application\")\n", 27 | "st.subheader(body=\"non_linear.csv\")\n", 28 | "\n", 29 | "x = st.sidebar.selectbox(label=\"X 축\", options=df.columns, index=0)\n", 30 | "y = st.sidebar.selectbox(label=\"Y 축\", options=df.columns, index=1)\n", 31 | "\n", 32 | "col1, col2 = st.columns(2)\n", 33 | "with col1:\n", 34 | " st.dataframe(data=df, height=500, use_container_width=True)\n", 35 | "with col2:\n", 36 | " st.line_chart(data=df, x=x, y=y, height=500)" 37 | ] 38 | } 39 | ], 40 | "metadata": { 41 | "kernelspec": { 42 | "display_name": "Python 3 (ipykernel)", 43 | "language": "python", 44 | "name": "python3" 45 | }, 46 | "language_info": { 47 | "codemirror_mode": { 48 | "name": "ipython", 49 | "version": 3 50 | }, 51 | "file_extension": ".py", 52 | "mimetype": "text/x-python", 53 | "name": "python", 54 | "nbconvert_exporter": "python", 55 | "pygments_lexer": "ipython3", 56 | "version": "3.9.13" 57 | } 58 | }, 59 | "nbformat": 4, 60 | "nbformat_minor": 5 61 | } 62 | -------------------------------------------------------------------------------- /05장 토큰화/예제 5.16 워드피스 토큰화.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f7bee282-8abe-4914-9b4b-efe9d737e6e6", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from tokenizers import Tokenizer\n", 11 | "from tokenizers.decoders import WordPiece as WordPieceDecoder\n", 12 | "\n", 13 | "\n", 14 | "tokenizer = Tokenizer.from_file(\"../models/petition_wordpiece.json\")\n", 15 | "tokenizer.decoder = WordPieceDecoder()\n", 16 | "\n", 17 | "sentence = \"안녕하세요, 토크나이저가 잘 학습되었군요!\"\n", 18 | "sentences = [\"이렇게 입력값을 리스트로 받아서\", \"쉽게 토크나이저를 사용할 수 있답니다\"]\n", 19 | "\n", 20 | "encoded_sentence = tokenizer.encode(sentence)\n", 21 | "encoded_sentences = tokenizer.encode_batch(sentences)\n", 22 | "\n", 23 | "print(\"인코더 형식 :\", type(encoded_sentence))\n", 24 | "\n", 25 | "print(\"단일 문장 토큰화 :\", encoded_sentence.tokens)\n", 26 | "print(\"여러 문장 토큰화 :\", [enc.tokens for enc in encoded_sentences])\n", 27 | "\n", 28 | "print(\"단일 문장 정수 인코딩 :\", encoded_sentence.ids)\n", 29 | "print(\"여러 문장 정수 인코딩 :\", [enc.ids for enc in encoded_sentences])\n", 30 | "\n", 31 | "print(\"정수 인코딩에서 문장 변환 :\", tokenizer.decode(encoded_sentence.ids))" 32 | ] 33 | } 34 | ], 35 | "metadata": { 36 | "kernelspec": { 37 | "display_name": "Python 3 (ipykernel)", 38 | "language": "python", 39 | "name": "python3" 40 | }, 41 | "language_info": { 42 | "codemirror_mode": { 43 | "name": "ipython", 44 | "version": 3 45 | }, 46 | "file_extension": ".py", 47 | "mimetype": "text/x-python", 48 | "name": "python", 49 | "nbconvert_exporter": "python", 50 | "pygments_lexer": "ipython3", 51 | "version": "3.9.13" 52 | } 53 | }, 54 | "nbformat": 4, 55 | "nbformat_minor": 5 56 | } 57 | -------------------------------------------------------------------------------- /05장 토큰화/예제 5.13 바이트 페어 인코딩 토큰화.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "a6e188c9-da48-4968-8d56-d17f365ae03b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from sentencepiece import SentencePieceProcessor\n", 11 | "\n", 12 | "\n", 13 | "tokenizer = SentencePieceProcessor()\n", 14 | "tokenizer.load(\"../models/petition_bpe.model\")\n", 15 | "\n", 16 | "sentence = \"안녕하세요, 토크나이저가 잘 학습되었군요!\"\n", 17 | "sentences = [\"이렇게 입력값을 리스트로 받아서\", \"쉽게 토크나이저를 사용할 수 있답니다\"]\n", 18 | "\n", 19 | "tokenized_sentence = tokenizer.encode_as_pieces(sentence)\n", 20 | "tokenized_sentences = tokenizer.encode_as_pieces(sentences)\n", 21 | "print(\"단일 문장 토큰화 :\", tokenized_sentence)\n", 22 | "print(\"여러 문장 토큰화 :\", tokenized_sentences)\n", 23 | "\n", 24 | "encoded_sentence = tokenizer.encode_as_ids(sentence)\n", 25 | "encoded_sentences = tokenizer.encode_as_ids(sentences)\n", 26 | "print(\"단일 문장 정수 인코딩 :\", encoded_sentence)\n", 27 | "print(\"여러 문장 정수 인코딩 :\", encoded_sentences)\n", 28 | "\n", 29 | "decode_ids = tokenizer.decode_ids(encoded_sentences)\n", 30 | "decode_pieces = tokenizer.decode_pieces(encoded_sentences)\n", 31 | "print(\"정수 인코딩에서 문장 변환 :\", decode_ids)\n", 32 | "print(\"하위 단어 토큰에서 문장 변환 :\", decode_pieces)" 33 | ] 34 | } 35 | ], 36 | "metadata": { 37 | "kernelspec": { 38 | "display_name": "Python 3 (ipykernel)", 39 | "language": "python", 40 | "name": "python3" 41 | }, 42 | "language_info": { 43 | "codemirror_mode": { 44 | "name": "ipython", 45 | "version": 3 46 | }, 47 | "file_extension": ".py", 48 | "mimetype": "text/x-python", 49 | "name": "python", 50 | "nbconvert_exporter": "python", 51 | "pygments_lexer": "ipython3", 52 | "version": "3.9.13" 53 | } 54 | }, 55 | "nbformat": 4, 56 | "nbformat_minor": 5 57 | } 58 | -------------------------------------------------------------------------------- /06장 임베딩/예제 6.19 양방향 다층 장단기 메모리.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "bb9d7c08-8d28-45d0-887a-a63b6dea7798", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import nn\n", 12 | "\n", 13 | "\n", 14 | "input_size = 128\n", 15 | "output_size = 256\n", 16 | "num_layers = 3\n", 17 | "bidirectional = True\n", 18 | "proj_size = 64\n", 19 | "\n", 20 | "model = nn.LSTM(\n", 21 | " input_size=input_size,\n", 22 | " hidden_size=output_size,\n", 23 | " num_layers=num_layers,\n", 24 | " batch_first=True,\n", 25 | " bidirectional=bidirectional,\n", 26 | " proj_size=proj_size,\n", 27 | ")\n", 28 | "\n", 29 | "batch_size = 4\n", 30 | "sequence_len = 6\n", 31 | "\n", 32 | "inputs = torch.randn(batch_size, sequence_len, input_size)\n", 33 | "h_0 = torch.rand(\n", 34 | " num_layers * (int(bidirectional) + 1),\n", 35 | " batch_size,\n", 36 | " proj_size if proj_size > 0 else output_size,\n", 37 | ")\n", 38 | "c_0 = torch.rand(num_layers * (int(bidirectional) + 1), batch_size, output_size)\n", 39 | "\n", 40 | "outputs, (h_n, c_n) = model(inputs, (h_0, c_0))\n", 41 | "\n", 42 | "print(outputs.shape)\n", 43 | "print(h_n.shape)\n", 44 | "print(c_n.shape)" 45 | ] 46 | } 47 | ], 48 | "metadata": { 49 | "kernelspec": { 50 | "display_name": "Python 3 (ipykernel)", 51 | "language": "python", 52 | "name": "python3" 53 | }, 54 | "language_info": { 55 | "codemirror_mode": { 56 | "name": "ipython", 57 | "version": 3 58 | }, 59 | "file_extension": ".py", 60 | "mimetype": "text/x-python", 61 | "name": "python", 62 | "nbconvert_exporter": "python", 63 | "pygments_lexer": "ipython3", 64 | "version": "3.9.13" 65 | } 66 | }, 67 | "nbformat": 4, 68 | "nbformat_minor": 5 69 | } 70 | -------------------------------------------------------------------------------- /06장 임베딩/예제 6.31 합성곱 모델.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "e5776758-c4df-4bce-940f-f161561d0d56", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import nn\n", 12 | "\n", 13 | "\n", 14 | "class CNN(nn.Module):\n", 15 | " def __init__(self):\n", 16 | " super().__init__()\n", 17 | "\n", 18 | " self.conv1 = nn.Sequential(\n", 19 | " nn.Conv2d(\n", 20 | " in_channels=3, out_channels=16, kernel_size=3, stride=2, padding=1\n", 21 | " ),\n", 22 | " nn.ReLU(),\n", 23 | " nn.MaxPool2d(kernel_size=2, stride=2),\n", 24 | " )\n", 25 | "\n", 26 | " self.conv2 = nn.Sequential(\n", 27 | " nn.Conv2d(\n", 28 | " in_channels=16, out_channels=32, kernel_size=3, stride=1, padding=1\n", 29 | " ),\n", 30 | " nn.ReLU(),\n", 31 | " nn.MaxPool2d(kernel_size=2, stride=2),\n", 32 | " )\n", 33 | "\n", 34 | " self.fc = nn.Linear(32 * 32 * 32, 10)\n", 35 | "\n", 36 | " def forward(self, x):\n", 37 | " x = self.conv1(x)\n", 38 | " x = self.conv2(x)\n", 39 | " x = torch.flatten(x)\n", 40 | " x = self.fc(x)\n", 41 | " return x" 42 | ] 43 | } 44 | ], 45 | "metadata": { 46 | "kernelspec": { 47 | "display_name": "Python 3 (ipykernel)", 48 | "language": "python", 49 | "name": "python3" 50 | }, 51 | "language_info": { 52 | "codemirror_mode": { 53 | "name": "ipython", 54 | "version": 3 55 | }, 56 | "file_extension": ".py", 57 | "mimetype": "text/x-python", 58 | "name": "python", 59 | "nbconvert_exporter": "python", 60 | "pygments_lexer": "ipython3", 61 | "version": "3.9.13" 62 | } 63 | }, 64 | "nbformat": 4, 65 | "nbformat_minor": 5 66 | } 67 | -------------------------------------------------------------------------------- /05장 토큰화/예제 5.06~5.08 NLTK 실습.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "14de2b90-4221-4d6b-8011-a00b7361575b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import nltk\n", 11 | "\n", 12 | "\n", 13 | "nltk.download(\"punkt\")\n", 14 | "nltk.download(\"averaged_perceptron_tagger\")" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": null, 20 | "id": "49a31b66", 21 | "metadata": {}, 22 | "outputs": [], 23 | "source": [ 24 | "from nltk import tokenize\n", 25 | "\n", 26 | "\n", 27 | "sentence = \"Those who can imagine anything, can create the impossible.\"\n", 28 | "\n", 29 | "word_tokens = tokenize.word_tokenize(sentence)\n", 30 | "sent_tokens = tokenize.sent_tokenize(sentence)\n", 31 | "\n", 32 | "print(word_tokens)\n", 33 | "print(sent_tokens)" 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": null, 39 | "id": "8f146722", 40 | "metadata": {}, 41 | "outputs": [], 42 | "source": [ 43 | "from nltk import tag\n", 44 | "from nltk import tokenize\n", 45 | "\n", 46 | "\n", 47 | "sentence = \"Those who can imagine anything, can create the impossible.\"\n", 48 | "\n", 49 | "word_tokens = tokenize.word_tokenize(sentence)\n", 50 | "pos = tag.pos_tag(word_tokens)\n", 51 | "\n", 52 | "print(pos)" 53 | ] 54 | } 55 | ], 56 | "metadata": { 57 | "kernelspec": { 58 | "display_name": "Python 3 (ipykernel)", 59 | "language": "python", 60 | "name": "python3" 61 | }, 62 | "language_info": { 63 | "codemirror_mode": { 64 | "name": "ipython", 65 | "version": 3 66 | }, 67 | "file_extension": ".py", 68 | "mimetype": "text/x-python", 69 | "name": "python", 70 | "nbconvert_exporter": "python", 71 | "pygments_lexer": "ipython3", 72 | "version": "3.9.13" 73 | } 74 | }, 75 | "nbformat": 4, 76 | "nbformat_minor": 5 77 | } 78 | -------------------------------------------------------------------------------- /05장 토큰화/예제 5.04~5.05 KoNLPy 토큰화.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "00deb409-9bfc-44c5-9f02-745b75a786e5", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from konlpy.tag import Okt\n", 11 | "\n", 12 | "\n", 13 | "okt = Okt()\n", 14 | "\n", 15 | "sentence = \"무엇이든 상상할 수 있는 사람은 무엇이든 만들어 낼 수 있다.\"\n", 16 | "\n", 17 | "nouns = okt.nouns(sentence)\n", 18 | "phrases = okt.phrases(sentence)\n", 19 | "morphs = okt.morphs(sentence)\n", 20 | "pos = okt.pos(sentence)\n", 21 | "\n", 22 | "print(\"명사 추출 :\", nouns)\n", 23 | "print(\"구 추출 :\", phrases)\n", 24 | "print(\"형태소 추출 :\", morphs)\n", 25 | "print(\"품사 태깅 :\", pos)" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "execution_count": null, 31 | "id": "58de0a57", 32 | "metadata": {}, 33 | "outputs": [], 34 | "source": [ 35 | "from konlpy.tag import Kkma\n", 36 | "\n", 37 | "\n", 38 | "kkma = Kkma()\n", 39 | "\n", 40 | "sentence = \"무엇이든 상상할 수 있는 사람은 무엇이든 만들어 낼 수 있다.\"\n", 41 | "\n", 42 | "nouns = kkma.nouns(sentence)\n", 43 | "sentences = kkma.sentences(sentence)\n", 44 | "morphs = kkma.morphs(sentence)\n", 45 | "pos = kkma.pos(sentence)\n", 46 | "\n", 47 | "print(\"명사 추출 :\", nouns)\n", 48 | "print(\"문장 추출 :\", sentences)\n", 49 | "print(\"형태소 추출 :\", morphs)\n", 50 | "print(\"품사 태깅 :\", pos)" 51 | ] 52 | } 53 | ], 54 | "metadata": { 55 | "kernelspec": { 56 | "display_name": "Python 3 (ipykernel)", 57 | "language": "python", 58 | "name": "python3" 59 | }, 60 | "language_info": { 61 | "codemirror_mode": { 62 | "name": "ipython", 63 | "version": 3 64 | }, 65 | "file_extension": ".py", 66 | "mimetype": "text/x-python", 67 | "name": "python", 68 | "nbconvert_exporter": "python", 69 | "pygments_lexer": "ipython3", 70 | "version": "3.9.13" 71 | } 72 | }, 73 | "nbformat": 4, 74 | "nbformat_minor": 5 75 | } 76 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.09~3.10 비쌍체 t-검정.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "d87ad7f1-5af3-4a18-a0e7-f8871c6835bc", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import numpy as np\n", 11 | "import pandas as pd\n", 12 | "import seaborn as sns\n", 13 | "from scipy import stats\n", 14 | "from matplotlib import pyplot as plt\n", 15 | "\n", 16 | "\n", 17 | "man_height = stats.norm.rvs(loc=170, scale=10, size=500, random_state=1)\n", 18 | "woman_height = stats.norm.rvs(loc=150, scale=10, size=500, random_state=1)\n", 19 | "\n", 20 | "X = np.concatenate([man_height, woman_height])\n", 21 | "Y = [\"man\"] * len(man_height) + [\"woman\"] * len(woman_height)\n", 22 | "\n", 23 | "df = pd.DataFrame(list(zip(X, Y)), columns=[\"X\", \"Y\"])\n", 24 | "fig = sns.displot(data=df, x=\"X\", hue=\"Y\", kind=\"kde\")\n", 25 | "fig.set_axis_labels(\"cm\", \"count\")\n", 26 | "plt.show()" 27 | ] 28 | }, 29 | { 30 | "cell_type": "code", 31 | "execution_count": null, 32 | "id": "41a8d680-9da3-46c5-8514-a8c80c5afa23", 33 | "metadata": {}, 34 | "outputs": [], 35 | "source": [ 36 | "statistic, pvalue = stats.ttest_ind(man_height, woman_height, equal_var=True)\n", 37 | "\n", 38 | "print(\"statistic:\", statistic)\n", 39 | "print(\"pvalue :\", pvalue)\n", 40 | "print(\"*:\", pvalue < 0.05)\n", 41 | "print(\"**:\", pvalue < 0.001)" 42 | ] 43 | } 44 | ], 45 | "metadata": { 46 | "kernelspec": { 47 | "display_name": "Python 3 (ipykernel)", 48 | "language": "python", 49 | "name": "python3" 50 | }, 51 | "language_info": { 52 | "codemirror_mode": { 53 | "name": "ipython", 54 | "version": 3 55 | }, 56 | "file_extension": ".py", 57 | "mimetype": "text/x-python", 58 | "name": "python", 59 | "nbconvert_exporter": "python", 60 | "pygments_lexer": "ipython3", 61 | "version": "3.9.13" 62 | } 63 | }, 64 | "nbformat": 4, 65 | "nbformat_minor": 5 66 | } 67 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.04 VGG-16 학습 후 동적 양자화.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import os\n", 11 | "import torch\n", 12 | "from torch import nn\n", 13 | "from torch.ao import quantization\n", 14 | "from torchvision import models\n", 15 | "\n", 16 | "\n", 17 | "model = models.vgg16(num_classes=2)\n", 18 | "model.load_state_dict(torch.load(\"../models/VGG16.pt\"))\n", 19 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 20 | "model = model.to(device)\n", 21 | "model.eval()\n", 22 | "\n", 23 | "model_dynamic_quantized = quantization.quantize_dynamic(\n", 24 | " model=model,\n", 25 | " qconfig_spec={nn.Linear},\n", 26 | " dtype=torch.qint8\n", 27 | ")\n", 28 | "model_dynamic_quantized.eval()\n", 29 | "torch.save(model_dynamic_quantized.state_dict(), \"../models/PTDQ_VGG16.pt\")\n", 30 | "\n", 31 | "file_size = os.path.getsize(\"../models/VGG16.pt\") / 1e6\n", 32 | "print(\"양자화 적용 전:\")\n", 33 | "print(f\"파일 크기: {file_size:.2f} MB\")\n", 34 | "print(model.classifier)\n", 35 | "print(\"\\n\")\n", 36 | "\n", 37 | "file_size = os.path.getsize(\"../models/PTDQ_VGG16.pt\") / 1e6\n", 38 | "print(\"양자화 적용 후:\")\n", 39 | "print(f\"파일 크기: {file_size:.2f} MB\")\n", 40 | "print(model_dynamic_quantized.classifier)" 41 | ] 42 | } 43 | ], 44 | "metadata": { 45 | "kernelspec": { 46 | "display_name": "Python 3 (ipykernel)", 47 | "language": "python", 48 | "name": "python3" 49 | }, 50 | "language_info": { 51 | "codemirror_mode": { 52 | "name": "ipython", 53 | "version": 3 54 | }, 55 | "file_extension": ".py", 56 | "mimetype": "text/x-python", 57 | "name": "python", 58 | "nbconvert_exporter": "python", 59 | "pygments_lexer": "ipython3", 60 | "version": "3.9.13" 61 | } 62 | }, 63 | "nbformat": 4, 64 | "nbformat_minor": 5 65 | } 66 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.47 모델 불러오기.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "514213fa-55b7-433a-9be3-957046d88c83", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import nn\n", 12 | "\n", 13 | "\n", 14 | "class CustomModel(nn.Module):\n", 15 | " def __init__(self):\n", 16 | " super().__init__()\n", 17 | " self.layer = nn.Linear(2, 1)\n", 18 | "\n", 19 | " def forward(self, x):\n", 20 | " x = self.layer(x)\n", 21 | " return x" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "id": "466be7db", 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 32 | "model = torch.load(\"../models/model.pt\", map_location=device)\n", 33 | "print(model)" 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": null, 39 | "id": "aa460ad0", 40 | "metadata": {}, 41 | "outputs": [], 42 | "source": [ 43 | "with torch.no_grad():\n", 44 | " model.eval()\n", 45 | " inputs = torch.FloatTensor(\n", 46 | " [\n", 47 | " [1 ** 2, 1],\n", 48 | " [5 ** 2, 5],\n", 49 | " [11 ** 2, 11]\n", 50 | " ]\n", 51 | " ).to(device)\n", 52 | " outputs = model(inputs)\n", 53 | " print(outputs)" 54 | ] 55 | } 56 | ], 57 | "metadata": { 58 | "kernelspec": { 59 | "display_name": "Python 3 (ipykernel)", 60 | "language": "python", 61 | "name": "python3" 62 | }, 63 | "language_info": { 64 | "codemirror_mode": { 65 | "name": "ipython", 66 | "version": 3 67 | }, 68 | "file_extension": ".py", 69 | "mimetype": "text/x-python", 70 | "name": "python", 71 | "nbconvert_exporter": "python", 72 | "pygments_lexer": "ipython3", 73 | "version": "3.9.13" 74 | } 75 | }, 76 | "nbformat": 4, 77 | "nbformat_minor": 5 78 | } 79 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.50 모델 상태 불러오기.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "9b5fed6a-8384-450c-8434-d07433b83e4e", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import nn\n", 12 | "\n", 13 | "\n", 14 | "class CustomModel(nn.Module):\n", 15 | " def __init__(self):\n", 16 | " super().__init__()\n", 17 | " self.layer = nn.Linear(2, 1)\n", 18 | "\n", 19 | " def forward(self, x):\n", 20 | " x = self.layer(x)\n", 21 | " return x" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "id": "4e847da7", 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 32 | "model = CustomModel().to(device)\n", 33 | "\n", 34 | "model_state_dict = torch.load(\"../models/model_state_dict.pt\", map_location=device)\n", 35 | "model.load_state_dict(model_state_dict)" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "id": "d652e3c4", 42 | "metadata": {}, 43 | "outputs": [], 44 | "source": [ 45 | "with torch.no_grad():\n", 46 | " model.eval()\n", 47 | " inputs = torch.FloatTensor(\n", 48 | " [\n", 49 | " [1 ** 2, 1],\n", 50 | " [5 ** 2, 5],\n", 51 | " [11 ** 2, 11]\n", 52 | " ]\n", 53 | " ).to(device)\n", 54 | " outputs = model(inputs)" 55 | ] 56 | } 57 | ], 58 | "metadata": { 59 | "kernelspec": { 60 | "display_name": "Python 3 (ipykernel)", 61 | "language": "python", 62 | "name": "python3" 63 | }, 64 | "language_info": { 65 | "codemirror_mode": { 66 | "name": "ipython", 67 | "version": 3 68 | }, 69 | "file_extension": ".py", 70 | "mimetype": "text/x-python", 71 | "name": "python", 72 | "nbconvert_exporter": "python", 73 | "pygments_lexer": "ipython3", 74 | "version": "3.9.13" 75 | } 76 | }, 77 | "nbformat": 4, 78 | "nbformat_minor": 5 79 | } 80 | -------------------------------------------------------------------------------- /07장 트랜스포머/예제 7.01 위치 인코딩.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "6dd252f6-29e8-469f-9c4c-42fbc0d46fab", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import math\n", 11 | "import torch\n", 12 | "from torch import nn\n", 13 | "from matplotlib import pyplot as plt\n", 14 | "\n", 15 | "\n", 16 | "class PositionalEncoding(nn.Module):\n", 17 | " def __init__(self, d_model, max_len, dropout=0.1):\n", 18 | " super().__init__()\n", 19 | " self.dropout = nn.Dropout(p=dropout)\n", 20 | "\n", 21 | " position = torch.arange(max_len).unsqueeze(1)\n", 22 | " div_term = torch.exp(\n", 23 | " torch.arange(0, d_model, 2) * (-math.log(10000.0) / d_model)\n", 24 | " )\n", 25 | "\n", 26 | " pe = torch.zeros(max_len, 1, d_model)\n", 27 | " pe[:, 0, 0::2] = torch.sin(position * div_term)\n", 28 | " pe[:, 0, 1::2] = torch.cos(position * div_term)\n", 29 | " self.register_buffer(\"pe\", pe)\n", 30 | "\n", 31 | " def forward(self, x):\n", 32 | " x = x + self.pe[: x.size(0)]\n", 33 | " return self.dropout(x)\n", 34 | "\n", 35 | "\n", 36 | "encoding = PositionalEncoding(d_model=128, max_len=50)\n", 37 | "\n", 38 | "plt.pcolormesh(encoding.pe.numpy().squeeze(), cmap=\"RdBu\")\n", 39 | "plt.xlabel(\"Embedding Dimension\")\n", 40 | "plt.xlim((0, 128))\n", 41 | "plt.ylabel(\"Position\")\n", 42 | "plt.colorbar()\n", 43 | "plt.show()" 44 | ] 45 | } 46 | ], 47 | "metadata": { 48 | "kernelspec": { 49 | "display_name": "Python 3 (ipykernel)", 50 | "language": "python", 51 | "name": "python3" 52 | }, 53 | "language_info": { 54 | "codemirror_mode": { 55 | "name": "ipython", 56 | "version": 3 57 | }, 58 | "file_extension": ".py", 59 | "mimetype": "text/x-python", 60 | "name": "python", 61 | "nbconvert_exporter": "python", 62 | "pygments_lexer": "ipython3", 63 | "version": "3.9.13" 64 | } 65 | }, 66 | "nbformat": 4, 67 | "nbformat_minor": 5 68 | } 69 | -------------------------------------------------------------------------------- /부록/예제 A.01 오토인코더.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "0f695b0e-ea93-40dc-90cd-9fdf378d69a4", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "import lightning\n", 12 | "from torch import nn\n", 13 | "from torch.nn import functional as F\n", 14 | "\n", 15 | "\n", 16 | "class AutoEncoder(lightning.LightningModule):\n", 17 | " def __init__(self):\n", 18 | " super().__init__()\n", 19 | " self.encoder = nn.Sequential(\n", 20 | " nn.Linear(28 * 28, 128),\n", 21 | " nn.ReLU(),\n", 22 | " nn.Linear(128, 3)\n", 23 | " )\n", 24 | " self.decoder = nn.Sequential(\n", 25 | " nn.Linear(3, 128),\n", 26 | " nn.ReLU(),\n", 27 | " nn.Linear(128, 28 * 28)\n", 28 | " )\n", 29 | "\n", 30 | " def forward(self, x):\n", 31 | " embedding = self.encoder(x)\n", 32 | " return embedding\n", 33 | "\n", 34 | " def training_step(self, batch):\n", 35 | " x, _ = batch\n", 36 | " x = x.view(x.size(0), -1)\n", 37 | " z = self.encoder(x)\n", 38 | " x_hat = self.decoder(z)\n", 39 | " loss = F.mse_loss(x_hat, x)\n", 40 | " self.log(\"train_loss\", loss)\n", 41 | " return loss\n", 42 | "\n", 43 | " def configure_optimizers(self):\n", 44 | " optimizer = torch.optim.Adam(self.parameters(), lr=1e-3)\n", 45 | " return optimizer" 46 | ] 47 | } 48 | ], 49 | "metadata": { 50 | "kernelspec": { 51 | "display_name": "Python 3 (ipykernel)", 52 | "language": "python", 53 | "name": "python3" 54 | }, 55 | "language_info": { 56 | "codemirror_mode": { 57 | "name": "ipython", 58 | "version": 3 59 | }, 60 | "file_extension": ".py", 61 | "mimetype": "text/x-python", 62 | "name": "python", 63 | "nbconvert_exporter": "python", 64 | "pygments_lexer": "ipython3", 65 | "version": "3.9.13" 66 | } 67 | }, 68 | "nbformat": 4, 69 | "nbformat_minor": 5 70 | } 71 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.01 BERT 모델 가지치기.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch.nn.utils import prune\n", 12 | "from transformers import BertTokenizer, BertForSequenceClassification\n", 13 | "\n", 14 | "\n", 15 | "tokenizer = BertTokenizer.from_pretrained(\n", 16 | " pretrained_model_name_or_path=\"bert-base-multilingual-cased\",\n", 17 | " do_lower_case=False,\n", 18 | ")\n", 19 | "model = BertForSequenceClassification.from_pretrained(\n", 20 | " pretrained_model_name_or_path=\"bert-base-multilingual-cased\",\n", 21 | " num_labels=2\n", 22 | ")\n", 23 | "model.load_state_dict(torch.load(\"../models/BertForSequenceClassification.pt\"))\n", 24 | "\n", 25 | "print(\"가지치기 적용 전:\")\n", 26 | "print(model.bert.encoder.layer[0].attention.self.key.weight)\n", 27 | "\n", 28 | "parameters = [\n", 29 | " (model.bert.embeddings.word_embeddings, \"weight\"),\n", 30 | " (model.bert.encoder.layer[0].attention.self.key, \"weight\"),\n", 31 | " (model.bert.encoder.layer[1].attention.self.key, \"weight\"),\n", 32 | " (model.bert.encoder.layer[2].attention.self.key, \"weight\"),\n", 33 | "]\n", 34 | "prune.global_unstructured(\n", 35 | " parameters=parameters,\n", 36 | " pruning_method=prune.L1Unstructured,\n", 37 | " amount=0.2\n", 38 | ")\n", 39 | "\n", 40 | "print(\"가지치기 적용 후:\")\n", 41 | "print(model.bert.encoder.layer[0].attention.self.key.weight)" 42 | ] 43 | } 44 | ], 45 | "metadata": { 46 | "kernelspec": { 47 | "display_name": "Python 3 (ipykernel)", 48 | "language": "python", 49 | "name": "python3" 50 | }, 51 | "language_info": { 52 | "codemirror_mode": { 53 | "name": "ipython", 54 | "version": 3 55 | }, 56 | "file_extension": ".py", 57 | "mimetype": "text/x-python", 58 | "name": "python", 59 | "nbconvert_exporter": "python", 60 | "pygments_lexer": "ipython3", 61 | "version": "3.9.13" 62 | } 63 | }, 64 | "nbformat": 4, 65 | "nbformat_minor": 5 66 | } 67 | -------------------------------------------------------------------------------- /06장 임베딩/예제 6.15~6.17 fastText OOV 모델 실습.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f8e63add-f91e-421b-ad32-531ed802289f", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from Korpora import Korpora\n", 11 | "\n", 12 | "\n", 13 | "corpus = Korpora.load(\"kornli\")\n", 14 | "corpus_texts = corpus.get_all_texts() + corpus.get_all_pairs()\n", 15 | "tokens = [sentence.split() for sentence in corpus_texts]\n", 16 | "\n", 17 | "print(tokens[:3])" 18 | ] 19 | }, 20 | { 21 | "cell_type": "code", 22 | "execution_count": null, 23 | "id": "a7f20efe-1a33-4dc6-89e5-f66bf5c2e775", 24 | "metadata": {}, 25 | "outputs": [], 26 | "source": [ 27 | "from gensim.models import FastText\n", 28 | "\n", 29 | "\n", 30 | "fastText = FastText(\n", 31 | " sentences=tokens,\n", 32 | " vector_size=128,\n", 33 | " window=5,\n", 34 | " min_count=5,\n", 35 | " sg=1,\n", 36 | " max_final_vocab=20000,\n", 37 | " epochs=3,\n", 38 | " min_n=2,\n", 39 | " max_n=6\n", 40 | ")\n", 41 | "\n", 42 | "# fastText.save(\"../models/fastText.model\")\n", 43 | "# fastText = FastText.load(\"../models/fastText.model\")" 44 | ] 45 | }, 46 | { 47 | "cell_type": "code", 48 | "execution_count": null, 49 | "id": "da92edfb-febe-4ed3-a083-4d06122355ed", 50 | "metadata": {}, 51 | "outputs": [], 52 | "source": [ 53 | "oov_token = \"사랑해요\"\n", 54 | "oov_vector = fastText.wv[oov_token]\n", 55 | "\n", 56 | "print(oov_token in fastText.wv.index_to_key)\n", 57 | "print(fastText.wv.most_similar(oov_vector, topn=5))" 58 | ] 59 | } 60 | ], 61 | "metadata": { 62 | "kernelspec": { 63 | "display_name": "Python 3 (ipykernel)", 64 | "language": "python", 65 | "name": "python3" 66 | }, 67 | "language_info": { 68 | "codemirror_mode": { 69 | "name": "ipython", 70 | "version": 3 71 | }, 72 | "file_extension": ".py", 73 | "mimetype": "text/x-python", 74 | "name": "python", 75 | "nbconvert_exporter": "python", 76 | "pygments_lexer": "ipython3", 77 | "version": "3.9.13" 78 | } 79 | }, 80 | "nbformat": 4, 81 | "nbformat_minor": 5 82 | } 83 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.57 모델 구조와 초깃값.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "106f9290-44cb-4e25-92c7-678336595165", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import nn\n", 12 | "from torch import optim\n", 13 | "\n", 14 | "\n", 15 | "class CustomModel(nn.Module):\n", 16 | " def __init__(self):\n", 17 | " super().__init__()\n", 18 | "\n", 19 | " self.layer1 = nn.Sequential(\n", 20 | " nn.Linear(2, 2),\n", 21 | " nn.Sigmoid()\n", 22 | " )\n", 23 | " self.layer2 = nn.Sequential(\n", 24 | " nn.Linear(2, 1),\n", 25 | " nn.Sigmoid()\n", 26 | " )\n", 27 | " \n", 28 | " self.layer1[0].weight.data = torch.nn.Parameter(\n", 29 | " torch.Tensor([[0.4352, 0.3545],\n", 30 | " [0.1951, 0.4835]])\n", 31 | " )\n", 32 | "\n", 33 | " self.layer1[0].bias.data = torch.nn.Parameter(\n", 34 | " torch.Tensor([-0.1419, 0.0439])\n", 35 | " )\n", 36 | "\n", 37 | " self.layer2[0].weight.data = torch.nn.Parameter(\n", 38 | " torch.Tensor([[-0.1725, 0.1129]])\n", 39 | " )\n", 40 | "\n", 41 | " self.layer2[0].bias.data = torch.nn.Parameter(\n", 42 | " torch.Tensor([-0.3043])\n", 43 | " )\n", 44 | "\n", 45 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 46 | "model = CustomModel().to(device)\n", 47 | "criterion = nn.BCELoss().to(device)\n", 48 | "optimizer = optim.SGD(model.parameters(), lr=1)" 49 | ] 50 | } 51 | ], 52 | "metadata": { 53 | "kernelspec": { 54 | "display_name": "Python 3 (ipykernel)", 55 | "language": "python", 56 | "name": "python3" 57 | }, 58 | "language_info": { 59 | "codemirror_mode": { 60 | "name": "ipython", 61 | "version": 3 62 | }, 63 | "file_extension": ".py", 64 | "mimetype": "text/x-python", 65 | "name": "python", 66 | "nbconvert_exporter": "python", 67 | "pygments_lexer": "ipython3", 68 | "version": "3.9.13" 69 | } 70 | }, 71 | "nbformat": 4, 72 | "nbformat_minor": 5 73 | } 74 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.11~3.17 단순 선형 회귀(넘파이).ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "fbe8b0bf-3040-4f5d-958b-b1b1acecd49c", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import numpy as np\n", 11 | "\n", 12 | "x = np.array(\n", 13 | " [[1], [2], [3], [4], [5], [6], [7], [8], [9], [10],\n", 14 | " [11], [12], [13], [14], [15], [16], [17], [18], [19], [20],\n", 15 | " [21], [22], [23], [24], [25], [26], [27], [28], [29], [30]]\n", 16 | ")\n", 17 | "y = np.array(\n", 18 | " [[0.94], [1.98], [2.88], [3.92], [3.96], [4.55], [5.64], [6.3], [7.44], [9.1],\n", 19 | " [8.46], [9.5], [10.67], [11.16], [14], [11.83], [14.4], [14.25], [16.2], [16.32],\n", 20 | " [17.46], [19.8], [18], [21.34], [22], [22.5], [24.57], [26.04], [21.6], [28.8]]\n", 21 | ")" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "id": "ee421c53", 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "weight = 0.0\n", 32 | "bias = 0.0\n", 33 | "learning_rate = 0.005" 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": null, 39 | "id": "68591b3c", 40 | "metadata": {}, 41 | "outputs": [], 42 | "source": [ 43 | "for epoch in range(10000):\n", 44 | " y_hat = weight * x + bias\n", 45 | "\n", 46 | " cost = ((y - y_hat) ** 2).mean()\n", 47 | "\n", 48 | " weight = weight - learning_rate * ((y_hat - y) * x).mean()\n", 49 | " bias = bias - learning_rate * (y_hat - y).mean()\n", 50 | "\n", 51 | " if (epoch + 1) % 1000 == 0:\n", 52 | " print(f\"Epoch : {epoch+1:4d}, Weight : {weight:.3f}, Bias : {bias:.3f}, Cost : {cost:.3f}\")" 53 | ] 54 | } 55 | ], 56 | "metadata": { 57 | "kernelspec": { 58 | "display_name": "Python 3 (ipykernel)", 59 | "language": "python", 60 | "name": "python3" 61 | }, 62 | "language_info": { 63 | "codemirror_mode": { 64 | "name": "ipython", 65 | "version": 3 66 | }, 67 | "file_extension": ".py", 68 | "mimetype": "text/x-python", 69 | "name": "python", 70 | "nbconvert_exporter": "python", 71 | "pygments_lexer": "ipython3", 72 | "version": "3.9.13" 73 | } 74 | }, 75 | "nbformat": 4, 76 | "nbformat_minor": 5 77 | } 78 | -------------------------------------------------------------------------------- /11장 모델 배포/app_fastapi.py: -------------------------------------------------------------------------------- 1 | # app_fastapi.py 2 | import io 3 | import torch 4 | import base64 5 | from PIL import Image 6 | from torch.nn import functional as F 7 | from torchvision import models, transforms 8 | 9 | 10 | class VGG16Model: 11 | def __init__(self, weight_path): 12 | self.device = 'cuda' if torch.cuda.is_available() else 'cpu' 13 | self.transform = transforms.Compose( 14 | [ 15 | transforms.Resize(256), 16 | transforms.CenterCrop(224), 17 | transforms.ToTensor(), 18 | transforms.Normalize( 19 | mean=[0.48235, 0.45882, 0.40784], 20 | std=[0.229, 0.224, 0.225] 21 | ) 22 | ] 23 | ) 24 | self.model = models.vgg16(num_classes=2).to(self.device) 25 | self.model.load_state_dict(torch.load(weight_path, map_location=self.device)) 26 | self.model.eval() 27 | 28 | def preprocessing(self, data): 29 | decode = base64.b64decode(data) 30 | bytes = io.BytesIO(decode) 31 | image = Image.open(bytes) 32 | input_data = self.transform(image).to(self.device) 33 | return input_data 34 | 35 | @torch.no_grad() 36 | def predict(self, input): 37 | input_data = self.preprocessing(input) 38 | outputs = self.model(input_data.unsqueeze(0)) 39 | probs = F.softmax(outputs, dim=-1) 40 | 41 | index = int(probs[0].argmax(axis=-1)) 42 | label = "개" if index == 1 else "고양이" 43 | score = float(probs[0][index]) 44 | 45 | return { 46 | "label": label, 47 | "score": score 48 | } 49 | 50 | # app_fastapi.py 51 | import uvicorn 52 | from pydantic import BaseModel 53 | from fastapi import FastAPI, Depends, HTTPException 54 | 55 | 56 | app = FastAPI() 57 | vgg = VGG16Model(weight_path="./VGG16.pt") 58 | 59 | 60 | class Item(BaseModel): 61 | base64: str 62 | 63 | 64 | def get_model(): 65 | return vgg 66 | 67 | 68 | @app.post("/predict") 69 | async def inference(item: Item, model: VGG16Model = Depends(get_model)): 70 | try: 71 | return model.predict(item.base64) 72 | 73 | except Exception as e: 74 | raise HTTPException(status_code=500, detail=str(e)) 75 | 76 | 77 | if __name__ == "__main__": 78 | uvicorn.run(app="app_fastapi:app", host="0.0.0.0", port=8000, workers=2) -------------------------------------------------------------------------------- /06장 임베딩/예제 6.13~6.14 Word2Vec 모델 실습.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "a49f8cc3", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import pandas as pd\n", 11 | "from Korpora import Korpora\n", 12 | "from konlpy.tag import Okt\n", 13 | "\n", 14 | "\n", 15 | "corpus = Korpora.load(\"nsmc\")\n", 16 | "corpus = pd.DataFrame(corpus.test)" 17 | ] 18 | }, 19 | { 20 | "cell_type": "code", 21 | "execution_count": null, 22 | "id": "cd387120", 23 | "metadata": {}, 24 | "outputs": [], 25 | "source": [ 26 | "tokenizer = Okt()\n", 27 | "tokens = [tokenizer.morphs(review) for review in corpus.text]" 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "execution_count": null, 33 | "id": "c121d6cd", 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "from gensim.models import Word2Vec\n", 38 | "\n", 39 | "\n", 40 | "word2vec = Word2Vec(\n", 41 | " sentences=tokens,\n", 42 | " vector_size=128,\n", 43 | " window=5,\n", 44 | " min_count=1,\n", 45 | " sg=1,\n", 46 | " epochs=3,\n", 47 | " max_final_vocab=10000\n", 48 | ")" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": null, 54 | "id": "89b83369", 55 | "metadata": {}, 56 | "outputs": [], 57 | "source": [ 58 | "word2vec.save(\"../models/word2vec.model\")\n", 59 | "word2vec = Word2Vec.load(\"../models/word2vec.model\")" 60 | ] 61 | }, 62 | { 63 | "cell_type": "code", 64 | "execution_count": null, 65 | "id": "82db1c00", 66 | "metadata": {}, 67 | "outputs": [], 68 | "source": [ 69 | "word = \"연기\"\n", 70 | "print(word2vec.wv[word])\n", 71 | "print(word2vec.wv.most_similar(word, topn=5))\n", 72 | "print(word2vec.wv.similarity(w1=word, w2=\"연기력\"))" 73 | ] 74 | } 75 | ], 76 | "metadata": { 77 | "kernelspec": { 78 | "display_name": "Python 3 (ipykernel)", 79 | "language": "python", 80 | "name": "python3" 81 | }, 82 | "language_info": { 83 | "codemirror_mode": { 84 | "name": "ipython", 85 | "version": 3 86 | }, 87 | "file_extension": ".py", 88 | "mimetype": "text/x-python", 89 | "name": "python", 90 | "nbconvert_exporter": "python", 91 | "pygments_lexer": "ipython3", 92 | "version": "3.9.13" 93 | } 94 | }, 95 | "nbformat": 4, 96 | "nbformat_minor": 5 97 | } 98 | -------------------------------------------------------------------------------- /11장 모델 배포/app_flask.py: -------------------------------------------------------------------------------- 1 | # app_flask.py 2 | import torch 3 | from torch.nn import functional as F 4 | from transformers import BertTokenizer, BertForSequenceClassification 5 | 6 | 7 | class BertModel: 8 | device = "cuda" if torch.cuda.is_available() else "cpu" 9 | 10 | @classmethod 11 | def load_model(cls, weight_path): 12 | cls.tokenizer = BertTokenizer.from_pretrained( 13 | pretrained_model_name_or_path="bert-base-multilingual-cased", 14 | do_lower_case=False, 15 | ) 16 | cls.model = BertForSequenceClassification.from_pretrained( 17 | pretrained_model_name_or_path="bert-base-multilingual-cased", 18 | num_labels=2 19 | ).to(cls.device) 20 | cls.model.load_state_dict(torch.load(weight_path, map_location=cls.device)) 21 | cls.model.eval() 22 | 23 | @classmethod 24 | def preprocessing(cls, data): 25 | input_data = cls.tokenizer( 26 | text=data, 27 | padding="longest", 28 | truncation=True, 29 | return_tensors="pt" 30 | ).to(cls.device) 31 | return input_data 32 | 33 | @classmethod 34 | @torch.no_grad() 35 | def predict(cls, input): 36 | input_data = cls.preprocessing(input) 37 | outputs = cls.model(**input_data).logits 38 | probs = F.softmax(outputs, dim=-1) 39 | 40 | index = int(probs[0].argmax(axis=-1)) 41 | label = "긍정" if index == 1 else "부정" 42 | score = float(probs[0][index]) 43 | 44 | return { 45 | "label": label, 46 | "score": score 47 | } 48 | 49 | 50 | # app_flask.py 51 | import json 52 | from flask import Flask, request, Response 53 | 54 | 55 | app = Flask(__name__) 56 | 57 | 58 | @app.route("/predict", methods=["POST"]) 59 | def inference(): 60 | data = request.get_json() 61 | text = data["text"] 62 | 63 | try: 64 | return Response( 65 | response=json.dumps(BertModel.predict(text), ensure_ascii=False), 66 | status=200, 67 | mimetype="application/json", 68 | ) 69 | 70 | except Exception as e: 71 | return Response( 72 | response=json.dumps({"error": str(e)}, ensure_ascii=False), 73 | status=500, 74 | mimetype="application/json", 75 | ) 76 | 77 | 78 | if __name__ == "__main__": 79 | BertModel.load_model(weight_path="./BertForSequenceClassification.pt") 80 | app.run(host="0.0.0.0", port=8000) -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.12 ONNX 런타임 실행.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import time\n", 11 | "import torch\n", 12 | "import onnxruntime as ort\n", 13 | "from PIL import Image\n", 14 | "from torchvision import models\n", 15 | "from torchvision import transforms\n", 16 | "\n", 17 | "\n", 18 | "def to_numpy(tensor):\n", 19 | " return tensor.detach().cpu().numpy()\n", 20 | "\n", 21 | "\n", 22 | "image = Image.open(\"../datasets/images/cat.jpg\")\n", 23 | "transform = transforms.Compose(\n", 24 | " [\n", 25 | " transforms.Resize(256),\n", 26 | " transforms.CenterCrop(224),\n", 27 | " transforms.ToTensor(),\n", 28 | " transforms.Normalize(\n", 29 | " mean=[0.48235, 0.45882, 0.40784],\n", 30 | " std=[0.229, 0.224, 0.225],\n", 31 | " ),\n", 32 | " ]\n", 33 | ")\n", 34 | "input = transform(image).unsqueeze(0)\n", 35 | "\n", 36 | "model = models.vgg16(num_classes=2)\n", 37 | "model.load_state_dict(torch.load(\"../models/VGG16.pt\"))\n", 38 | "model.eval()\n", 39 | "\n", 40 | "with torch.no_grad():\n", 41 | " start_time = time.time()\n", 42 | " output = model(input)\n", 43 | " end_time = time.time()\n", 44 | " print(\"파이토치:\")\n", 45 | " print(output)\n", 46 | " print(end_time - start_time)\n", 47 | "\n", 48 | "\n", 49 | "ort_session = ort.InferenceSession(\"../models/VGG16.onnx\")\n", 50 | "\n", 51 | "start_time = time.time()\n", 52 | "ort_inputs = {ort_session.get_inputs()[0].name: to_numpy(input)}\n", 53 | "ort_outs = ort_session.run(output_names=None, input_feed=ort_inputs)\n", 54 | "end_time = time.time()\n", 55 | "print(\"ONNX:\")\n", 56 | "print(ort_outs)\n", 57 | "print(end_time - start_time)" 58 | ] 59 | } 60 | ], 61 | "metadata": { 62 | "kernelspec": { 63 | "display_name": "Python 3 (ipykernel)", 64 | "language": "python", 65 | "name": "python3" 66 | }, 67 | "language_info": { 68 | "codemirror_mode": { 69 | "name": "ipython", 70 | "version": 3 71 | }, 72 | "file_extension": ".py", 73 | "mimetype": "text/x-python", 74 | "name": "python", 75 | "nbconvert_exporter": "python", 76 | "pygments_lexer": "ipython3", 77 | "version": "3.9.13" 78 | } 79 | }, 80 | "nbformat": 4, 81 | "nbformat_minor": 5 82 | } 83 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.25 zero_grad(), cost.backward(), optimizer.step().ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "1ad48ab8-259f-44c4-8173-e82488c83d54", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import optim\n", 12 | "\n", 13 | "\n", 14 | "x = torch.FloatTensor([\n", 15 | " [1], [2], [3], [4], [5], [6], [7], [8], [9], [10],\n", 16 | " [11], [12], [13], [14], [15], [16], [17], [18], [19], [20],\n", 17 | " [21], [22], [23], [24], [25], [26], [27], [28], [29], [30]\n", 18 | "])\n", 19 | "y = torch.FloatTensor([\n", 20 | " [0.94], [1.98], [2.88], [3.92], [3.96], [4.55], [5.64], [6.3], [7.44], [9.1],\n", 21 | " [8.46], [9.5], [10.67], [11.16], [14], [11.83], [14.4], [14.25], [16.2], [16.32],\n", 22 | " [17.46], [19.8], [18], [21.34], [22], [22.5], [24.57], [26.04], [21.6], [28.8]\n", 23 | "])\n", 24 | "\n", 25 | "weight = torch.zeros(1, requires_grad=True)\n", 26 | "bias = torch.zeros(1, requires_grad=True)\n", 27 | "learning_rate = 0.001\n", 28 | "\n", 29 | "optimizer = optim.SGD([weight, bias], lr=learning_rate)\n", 30 | "\n", 31 | "for epoch in range(10000):\n", 32 | " hypothesis = weight * x + bias\n", 33 | " cost = torch.mean((hypothesis - y) ** 2)\n", 34 | " \n", 35 | " print(f\"Epoch : {epoch+1:4d}\")\n", 36 | " print(f\"Step [1] : Gradient : {weight.grad}, Weight : {weight.item():.5f}\")\n", 37 | "\n", 38 | " optimizer.zero_grad()\n", 39 | " print(f\"Step [2] : Gradient : {weight.grad}, Weight : {weight.item():.5f}\")\n", 40 | "\n", 41 | " cost.backward()\n", 42 | " print(f\"Step [3] : Gradient : {weight.grad}, Weight : {weight.item():.5f}\")\n", 43 | "\n", 44 | " optimizer.step()\n", 45 | " print(f\"Step [4] : Gradient : {weight.grad}, Weight : {weight.item():.5f}\")\n", 46 | " \n", 47 | " if epoch == 3:\n", 48 | " break\n" 49 | ] 50 | } 51 | ], 52 | "metadata": { 53 | "kernelspec": { 54 | "display_name": "Python 3 (ipykernel)", 55 | "language": "python", 56 | "name": "python3" 57 | }, 58 | "language_info": { 59 | "codemirror_mode": { 60 | "name": "ipython", 61 | "version": 3 62 | }, 63 | "file_extension": ".py", 64 | "mimetype": "text/x-python", 65 | "name": "python", 66 | "nbconvert_exporter": "python", 67 | "pygments_lexer": "ipython3", 68 | "version": "3.9.13" 69 | } 70 | }, 71 | "nbformat": 4, 72 | "nbformat_minor": 5 73 | } 74 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.26~3.28 신경망 패키지 적용.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "85c4e13c-31d5-44c4-b726-370b1f662f10", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import nn\n", 12 | "from torch import optim" 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": null, 18 | "id": "5c387cb7", 19 | "metadata": {}, 20 | "outputs": [], 21 | "source": [ 22 | "x = torch.FloatTensor([\n", 23 | " [1], [2], [3], [4], [5], [6], [7], [8], [9], [10],\n", 24 | " [11], [12], [13], [14], [15], [16], [17], [18], [19], [20],\n", 25 | " [21], [22], [23], [24], [25], [26], [27], [28], [29], [30]\n", 26 | "])\n", 27 | "y = torch.FloatTensor([\n", 28 | " [0.94], [1.98], [2.88], [3.92], [3.96], [4.55], [5.64], [6.3], [7.44], [9.1],\n", 29 | " [8.46], [9.5], [10.67], [11.16], [14], [11.83], [14.4], [14.25], [16.2], [16.32],\n", 30 | " [17.46], [19.8], [18], [21.34], [22], [22.5], [24.57], [26.04], [21.6], [28.8]\n", 31 | "])" 32 | ] 33 | }, 34 | { 35 | "cell_type": "code", 36 | "execution_count": null, 37 | "id": "b40fb176-5f47-46f0-97ae-deedf8cf4e4f", 38 | "metadata": {}, 39 | "outputs": [], 40 | "source": [ 41 | "model = nn.Linear(1, 1)\n", 42 | "criterion = nn.MSELoss()\n", 43 | "optimizer = optim.SGD(model.parameters(), lr=0.001)" 44 | ] 45 | }, 46 | { 47 | "cell_type": "code", 48 | "execution_count": null, 49 | "id": "0e553a7d", 50 | "metadata": {}, 51 | "outputs": [], 52 | "source": [ 53 | "for epoch in range(10000):\n", 54 | " output = model(x)\n", 55 | " cost = criterion(output, y)\n", 56 | "\n", 57 | " optimizer.zero_grad()\n", 58 | " cost.backward()\n", 59 | " optimizer.step()\n", 60 | "\n", 61 | " if (epoch + 1) % 1000 == 0:\n", 62 | " print(f\"Epoch : {epoch+1:4d}, Model : {list(model.parameters())}, Cost : {cost:.3f}\")" 63 | ] 64 | } 65 | ], 66 | "metadata": { 67 | "kernelspec": { 68 | "display_name": "Python 3 (ipykernel)", 69 | "language": "python", 70 | "name": "python3" 71 | }, 72 | "language_info": { 73 | "codemirror_mode": { 74 | "name": "ipython", 75 | "version": 3 76 | }, 77 | "file_extension": ".py", 78 | "mimetype": "text/x-python", 79 | "name": "python", 80 | "nbconvert_exporter": "python", 81 | "pygments_lexer": "ipython3", 82 | "version": "3.9.13" 83 | } 84 | }, 85 | "nbformat": 4, 86 | "nbformat_minor": 5 87 | } 88 | -------------------------------------------------------------------------------- /부록/예제 B.07 Auto 클래스와 Trainer를 활용한 IMDB 분류 모델 학습 .ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "82aa842d-9209-47df-b1c1-febf4f2c0dcf", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from datasets import load_dataset\n", 11 | "from transformers import AutoModelForSequenceClassification, AutoTokenizer\n", 12 | "from transformers import Trainer, TrainingArguments, DataCollatorWithPadding\n", 13 | "\n", 14 | "\n", 15 | "def preprocess_function(examples):\n", 16 | " return tokenizer(examples[\"text\"], truncation=True)\n", 17 | "\n", 18 | "imdb = load_dataset(\"imdb\")\n", 19 | "\n", 20 | "tokenizer = AutoTokenizer.from_pretrained(\"bert-base-uncased\")\n", 21 | "tokenized_imdb = imdb.map(preprocess_function, batched=True)\n", 22 | "\n", 23 | "data_collator = DataCollatorWithPadding(tokenizer=tokenizer)\n", 24 | "\n", 25 | "id2label = {0: \"부정\", 1: \"긍정\"}\n", 26 | "label2id = {\"부정\": 0, \"긍정\": 1}\n", 27 | "model = AutoModelForSequenceClassification.from_pretrained(\n", 28 | " \"bert-base-uncased\", num_labels=2, id2label=id2label, label2id=label2id\n", 29 | ")\n", 30 | "\n", 31 | "training_args = TrainingArguments(\n", 32 | " output_dir=\"imdb_classifier\",\n", 33 | " learning_rate=2e-5,\n", 34 | " per_device_train_batch_size=16,\n", 35 | " per_device_eval_batch_size=32,\n", 36 | " num_train_epochs=3,\n", 37 | " weight_decay=0.01,\n", 38 | " evaluation_strategy=\"epoch\",\n", 39 | " save_strategy=\"epoch\",\n", 40 | " load_best_model_at_end=True,\n", 41 | ")\n", 42 | "\n", 43 | "trainer = Trainer(\n", 44 | " model=model,\n", 45 | " args=training_args,\n", 46 | " train_dataset=tokenized_imdb[\"train\"],\n", 47 | " eval_dataset=tokenized_imdb[\"test\"],\n", 48 | " tokenizer=tokenizer,\n", 49 | " data_collator=data_collator\n", 50 | ")\n", 51 | "\n", 52 | "trainer.train()" 53 | ] 54 | } 55 | ], 56 | "metadata": { 57 | "kernelspec": { 58 | "display_name": "Python 3 (ipykernel)", 59 | "language": "python", 60 | "name": "python3" 61 | }, 62 | "language_info": { 63 | "codemirror_mode": { 64 | "name": "ipython", 65 | "version": 3 66 | }, 67 | "file_extension": ".py", 68 | "mimetype": "text/x-python", 69 | "name": "python", 70 | "nbconvert_exporter": "python", 71 | "pygments_lexer": "ipython3", 72 | "version": "3.9.13" 73 | } 74 | }, 75 | "nbformat": 4, 76 | "nbformat_minor": 5 77 | } 78 | -------------------------------------------------------------------------------- /부록/예제 A.02 MNIST 데이터세트.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "67ff4f1f-c6c6-464a-8507-a91fcbb6e9cd", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "import lightning\n", 12 | "from torch import nn\n", 13 | "from torch.nn import functional as F\n", 14 | "\n", 15 | "\n", 16 | "class AutoEncoder(lightning.LightningModule):\n", 17 | " def __init__(self):\n", 18 | " super().__init__()\n", 19 | " self.encoder = nn.Sequential(\n", 20 | " nn.Linear(28 * 28, 128),\n", 21 | " nn.ReLU(),\n", 22 | " nn.Linear(128, 3)\n", 23 | " )\n", 24 | " self.decoder = nn.Sequential(\n", 25 | " nn.Linear(3, 128),\n", 26 | " nn.ReLU(),\n", 27 | " nn.Linear(128, 28 * 28)\n", 28 | " )\n", 29 | "\n", 30 | " def forward(self, x):\n", 31 | " embedding = self.encoder(x)\n", 32 | " return embedding\n", 33 | "\n", 34 | " def training_step(self, batch):\n", 35 | " x, _ = batch\n", 36 | " x = x.view(x.size(0), -1)\n", 37 | " z = self.encoder(x)\n", 38 | " x_hat = self.decoder(z)\n", 39 | " loss = F.mse_loss(x_hat, x)\n", 40 | " self.log(\"train_loss\", loss)\n", 41 | " return loss\n", 42 | "\n", 43 | " def configure_optimizers(self):\n", 44 | " optimizer = torch.optim.Adam(self.parameters(), lr=1e-3)\n", 45 | " return optimizer" 46 | ] 47 | }, 48 | { 49 | "cell_type": "code", 50 | "execution_count": null, 51 | "id": "d05bcb0c-05fc-4045-ad7e-4661a49e5142", 52 | "metadata": {}, 53 | "outputs": [], 54 | "source": [ 55 | "import torchvision\n", 56 | "from torch.utils.data import random_split\n", 57 | "\n", 58 | "\n", 59 | "dataset = torchvision.datasets.MNIST(\n", 60 | " \"../datasets\", download=True, transform=torchvision.transforms.ToTensor()\n", 61 | ")\n", 62 | "train, valid = random_split(dataset, [55000, 5000])\n", 63 | "print(dataset)" 64 | ] 65 | } 66 | ], 67 | "metadata": { 68 | "kernelspec": { 69 | "display_name": "Python 3 (ipykernel)", 70 | "language": "python", 71 | "name": "python3" 72 | }, 73 | "language_info": { 74 | "codemirror_mode": { 75 | "name": "ipython", 76 | "version": 3 77 | }, 78 | "file_extension": ".py", 79 | "mimetype": "text/x-python", 80 | "name": "python", 81 | "nbconvert_exporter": "python", 82 | "pygments_lexer": "ipython3", 83 | "version": "3.9.13" 84 | } 85 | }, 86 | "nbformat": 4, 87 | "nbformat_minor": 5 88 | } 89 | -------------------------------------------------------------------------------- /10장 비전 트랜스포머/예제 10.10~10.12 스윈 트랜스포머 모델 구조.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "e102c369-4de8-4cdf-84b4-a8d64cc54364", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "window_size = 2\n", 14 | "coords_h = torch.arange(window_size)\n", 15 | "coords_w = torch.arange(window_size)\n", 16 | "coords = torch.stack(torch.meshgrid([coords_h, coords_w], indexing=\"ij\"))\n", 17 | "coords_flatten = torch.flatten(coords, 1)\n", 18 | "relative_coords = coords_flatten[:, :, None] - coords_flatten[:, None, :]\n", 19 | "\n", 20 | "print(relative_coords)\n", 21 | "print(relative_coords.shape)" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "id": "9d942af7-05c0-42e7-bfc8-a44957aced3d", 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "x_coords = relative_coords[0, :, :]\n", 32 | "y_coords = relative_coords[1, :, :]\n", 33 | "\n", 34 | "x_coords += window_size - 1 # X축에 대한 ③번 연산 과정\n", 35 | "y_coords += window_size - 1 # Y축에 대한 ③번 연산 과정\n", 36 | "x_coords *= 2 * window_size - 1 # ④번 연산 과정\n", 37 | "print(f\"X축에 대한 행렬:\\n{x_coords}\\n\")\n", 38 | "print(f\"Y축에 대한 행렬:\\n{y_coords}\\n\")\n", 39 | "\n", 40 | "relative_position_index = x_coords + y_coords # ⑤번 연산 과정\n", 41 | "print(f\"X, Y축에 대한 위치 행렬:\\n{relative_position_index}\")" 42 | ] 43 | }, 44 | { 45 | "cell_type": "code", 46 | "execution_count": null, 47 | "id": "46df5241-4ba4-4e74-b512-be1c903881ac", 48 | "metadata": {}, 49 | "outputs": [], 50 | "source": [ 51 | "num_heads = 1\n", 52 | "relative_position_bias_table = torch.Tensor(\n", 53 | " torch.zeros((2 * window_size - 1) * (2 * window_size - 1), num_heads)\n", 54 | ")\n", 55 | "\n", 56 | "relative_position_bias = relative_position_bias_table[relative_position_index.view(-1)]\n", 57 | "relative_position_bias = relative_position_bias.view(\n", 58 | " window_size * window_size, window_size * window_size, -1\n", 59 | ")\n", 60 | "print(relative_position_bias.shape)" 61 | ] 62 | } 63 | ], 64 | "metadata": { 65 | "kernelspec": { 66 | "display_name": "Python 3 (ipykernel)", 67 | "language": "python", 68 | "name": "python3" 69 | }, 70 | "language_info": { 71 | "codemirror_mode": { 72 | "name": "ipython", 73 | "version": 3 74 | }, 75 | "file_extension": ".py", 76 | "mimetype": "text/x-python", 77 | "name": "python", 78 | "nbconvert_exporter": "python", 79 | "pygments_lexer": "ipython3", 80 | "version": "3.9.13" 81 | } 82 | }, 83 | "nbformat": 4, 84 | "nbformat_minor": 5 85 | } 86 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.18~3.24 단순 선형 회귀(파이토치).ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "316898f4", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import optim" 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": null, 17 | "id": "1c883318", 18 | "metadata": {}, 19 | "outputs": [], 20 | "source": [ 21 | "x = torch.FloatTensor([\n", 22 | " [1], [2], [3], [4], [5], [6], [7], [8], [9], [10],\n", 23 | " [11], [12], [13], [14], [15], [16], [17], [18], [19], [20],\n", 24 | " [21], [22], [23], [24], [25], [26], [27], [28], [29], [30]\n", 25 | "])\n", 26 | "y = torch.FloatTensor([\n", 27 | " [0.94], [1.98], [2.88], [3.92], [3.96], [4.55], [5.64], [6.3], [7.44], [9.1],\n", 28 | " [8.46], [9.5], [10.67], [11.16], [14], [11.83], [14.4], [14.25], [16.2], [16.32],\n", 29 | " [17.46], [19.8], [18], [21.34], [22], [22.5], [24.57], [26.04], [21.6], [28.8]\n", 30 | "])" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": null, 36 | "id": "e3fc316c", 37 | "metadata": {}, 38 | "outputs": [], 39 | "source": [ 40 | "weight = torch.zeros(1, requires_grad=True)\n", 41 | "bias = torch.zeros(1, requires_grad=True)\n", 42 | "learning_rate = 0.001" 43 | ] 44 | }, 45 | { 46 | "cell_type": "code", 47 | "execution_count": null, 48 | "id": "01e776cd", 49 | "metadata": {}, 50 | "outputs": [], 51 | "source": [ 52 | "optimizer = optim.SGD([weight, bias], lr=learning_rate)" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "id": "7927188d", 59 | "metadata": {}, 60 | "outputs": [], 61 | "source": [ 62 | "for epoch in range(10000):\n", 63 | " hypothesis = weight * x + bias\n", 64 | " cost = torch.mean((hypothesis - y) ** 2)\n", 65 | "\n", 66 | " optimizer.zero_grad()\n", 67 | " cost.backward()\n", 68 | " optimizer.step()\n", 69 | "\n", 70 | " if (epoch + 1) % 1000 == 0:\n", 71 | " print(f\"Epoch : {epoch+1:4d}, Weight : {weight.item():.3f}, Bias : {bias.item():.3f}, Cost : {cost:.3f}\")" 72 | ] 73 | } 74 | ], 75 | "metadata": { 76 | "kernelspec": { 77 | "display_name": "Python 3 (ipykernel)", 78 | "language": "python", 79 | "name": "python3" 80 | }, 81 | "language_info": { 82 | "codemirror_mode": { 83 | "name": "ipython", 84 | "version": 3 85 | }, 86 | "file_extension": ".py", 87 | "mimetype": "text/x-python", 88 | "name": "python", 89 | "nbconvert_exporter": "python", 90 | "pygments_lexer": "ipython3", 91 | "version": "3.9.13" 92 | } 93 | }, 94 | "nbformat": 4, 95 | "nbformat_minor": 5 96 | } 97 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.03 양자화 결과 비교.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import os\n", 11 | "import time\n", 12 | "import torch\n", 13 | "from PIL import Image\n", 14 | "from torchvision import models\n", 15 | "from torchvision import transforms\n", 16 | "\n", 17 | "\n", 18 | "transform = transforms.Compose(\n", 19 | " [\n", 20 | " transforms.Resize(256),\n", 21 | " transforms.CenterCrop(224),\n", 22 | " transforms.ToTensor(),\n", 23 | " transforms.Normalize(\n", 24 | " mean=[0.48235, 0.45882, 0.40784],\n", 25 | " std=[0.229, 0.224, 0.225],\n", 26 | " ),\n", 27 | " ]\n", 28 | ")\n", 29 | "\n", 30 | "image = Image.open(\"../datasets/images/cat.jpg\")\n", 31 | "inputs = transform(image).unsqueeze(0)\n", 32 | "\n", 33 | "model = models.vgg16(num_classes=2)\n", 34 | "model.load_state_dict(torch.load(\"../models/VGG16.pt\"))\n", 35 | "\n", 36 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 37 | "model = model.to(device)\n", 38 | "model.eval()\n", 39 | "\n", 40 | "model_static_quantized = torch.jit.load(\"../models/PTSQ_VGG16.pt\")\n", 41 | "\n", 42 | "with torch.no_grad():\n", 43 | " start_time = time.time()\n", 44 | " outputs = model(inputs.to(device))\n", 45 | " file_size = os.path.getsize(\"../models/VGG16.pt\") / 1e6\n", 46 | " print(\"양자화 적용 전:\")\n", 47 | " print(f\"출력 결과: {outputs}\")\n", 48 | " print(f\"추론 시간: {time.time() - start_time:.4f}s\")\n", 49 | " print(f\"파일 크기: {file_size:.2f} MB\")\n", 50 | " print(\"\\n\")\n", 51 | " \n", 52 | "start_time = time.time()\n", 53 | "outputs = model_static_quantized(inputs)\n", 54 | "file_size = os.path.getsize(\"../models/PTSQ_VGG16.pt\") / 1e6\n", 55 | "end_time = time.time() - start_time\n", 56 | "print(\"양자화 적용 후:\")\n", 57 | "print(f\"출력 결과: {outputs}\")\n", 58 | "print(f\"추론 시간: {time.time() - start_time:.4f}s\")\n", 59 | "print(f\"파일 크기: {file_size:.2f} MB\")" 60 | ] 61 | } 62 | ], 63 | "metadata": { 64 | "kernelspec": { 65 | "display_name": "Python 3 (ipykernel)", 66 | "language": "python", 67 | "name": "python3" 68 | }, 69 | "language_info": { 70 | "codemirror_mode": { 71 | "name": "ipython", 72 | "version": 3 73 | }, 74 | "file_extension": ".py", 75 | "mimetype": "text/x-python", 76 | "name": "python", 77 | "nbconvert_exporter": "python", 78 | "pygments_lexer": "ipython3", 79 | "version": "3.9.13" 80 | } 81 | }, 82 | "nbformat": 4, 83 | "nbformat_minor": 5 84 | } 85 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.34 편향 제거.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "83500a5b-a66c-4ede-8455-af9195e07db4", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import nn\n", 12 | "from torch import optim\n", 13 | "from torch.utils.data import TensorDataset, DataLoader \n", 14 | "\n", 15 | "\n", 16 | "train_x = torch.FloatTensor([\n", 17 | " [1, 2], [2, 3], [3, 4], [4, 5], [5, 6], [6, 7]\n", 18 | "])\n", 19 | "train_y = torch.FloatTensor([\n", 20 | " [0.1, 1.5], [1, 2.8], [1.9, 4.1], [2.8, 5.4], [3.7, 6.7], [4.6, 8]\n", 21 | "])" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "id": "02b194de-ff26-49d3-8833-028bb2915c63", 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "train_dataset = TensorDataset(train_x, train_y)\n", 32 | "train_dataloader = DataLoader(train_dataset, batch_size=2, shuffle=True, drop_last=True)" 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": null, 38 | "id": "ac12a376-35c5-4ac9-a355-0d7919f7e6f4", 39 | "metadata": {}, 40 | "outputs": [], 41 | "source": [ 42 | "model = nn.Linear(2, 2, bias=False)\n", 43 | "criterion = nn.MSELoss()\n", 44 | "optimizer = optim.SGD(model.parameters(), lr=0.001)" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "id": "45ee590b-d644-4c65-b361-a65b848a7296", 51 | "metadata": {}, 52 | "outputs": [], 53 | "source": [ 54 | "for epoch in range(20000):\n", 55 | " cost = 0.0\n", 56 | " \n", 57 | " for batch in train_dataloader:\n", 58 | " x, y = batch\n", 59 | " output = model(x)\n", 60 | " \n", 61 | " loss = criterion(output, y)\n", 62 | " \n", 63 | " optimizer.zero_grad()\n", 64 | " loss.backward()\n", 65 | " optimizer.step()\n", 66 | " \n", 67 | " cost += loss\n", 68 | "\n", 69 | " cost = cost / len(train_dataloader)\n", 70 | " \n", 71 | " if (epoch + 1) % 1000 == 0:\n", 72 | " print(f\"Epoch : {epoch+1:4d}, Model : {list(model.parameters())}, Cost : {cost:.3f}\")" 73 | ] 74 | } 75 | ], 76 | "metadata": { 77 | "kernelspec": { 78 | "display_name": "Python 3 (ipykernel)", 79 | "language": "python", 80 | "name": "python3" 81 | }, 82 | "language_info": { 83 | "codemirror_mode": { 84 | "name": "ipython", 85 | "version": 3 86 | }, 87 | "file_extension": ".py", 88 | "mimetype": "text/x-python", 89 | "name": "python", 90 | "nbconvert_exporter": "python", 91 | "pygments_lexer": "ipython3", 92 | "version": "3.9.13" 93 | } 94 | }, 95 | "nbformat": 4, 96 | "nbformat_minor": 5 97 | } 98 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.29~3.33 다중 선형 회귀.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "83500a5b-a66c-4ede-8455-af9195e07db4", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import nn\n", 12 | "from torch import optim\n", 13 | "from torch.utils.data import TensorDataset, DataLoader \n", 14 | "\n", 15 | "\n", 16 | "train_x = torch.FloatTensor([\n", 17 | " [1, 2], [2, 3], [3, 4], [4, 5], [5, 6], [6, 7]\n", 18 | "])\n", 19 | "train_y = torch.FloatTensor([\n", 20 | " [0.1, 1.5], [1, 2.8], [1.9, 4.1], [2.8, 5.4], [3.7, 6.7], [4.6, 8]\n", 21 | "])" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "id": "02b194de-ff26-49d3-8833-028bb2915c63", 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "train_dataset = TensorDataset(train_x, train_y)\n", 32 | "train_dataloader = DataLoader(train_dataset, batch_size=2, shuffle=True, drop_last=True)" 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": null, 38 | "id": "ac12a376-35c5-4ac9-a355-0d7919f7e6f4", 39 | "metadata": {}, 40 | "outputs": [], 41 | "source": [ 42 | "model = nn.Linear(2, 2, bias=True)\n", 43 | "criterion = nn.MSELoss()\n", 44 | "optimizer = optim.SGD(model.parameters(), lr=0.001)" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "id": "45ee590b-d644-4c65-b361-a65b848a7296", 51 | "metadata": {}, 52 | "outputs": [], 53 | "source": [ 54 | "for epoch in range(20000):\n", 55 | " cost = 0.0\n", 56 | " \n", 57 | " for batch in train_dataloader:\n", 58 | " x, y = batch\n", 59 | " output = model(x)\n", 60 | " \n", 61 | " loss = criterion(output, y)\n", 62 | " \n", 63 | " optimizer.zero_grad()\n", 64 | " loss.backward()\n", 65 | " optimizer.step()\n", 66 | " \n", 67 | " cost += loss\n", 68 | "\n", 69 | " cost = cost / len(train_dataloader)\n", 70 | " \n", 71 | " if (epoch + 1) % 1000 == 0:\n", 72 | " print(f\"Epoch : {epoch+1:4d}, Model : {list(model.parameters())}, Cost : {cost:.3f}\")" 73 | ] 74 | } 75 | ], 76 | "metadata": { 77 | "kernelspec": { 78 | "display_name": "Python 3 (ipykernel)", 79 | "language": "python", 80 | "name": "python3" 81 | }, 82 | "language_info": { 83 | "codemirror_mode": { 84 | "name": "ipython", 85 | "version": 3 86 | }, 87 | "file_extension": ".py", 88 | "mimetype": "text/x-python", 89 | "name": "python", 90 | "nbconvert_exporter": "python", 91 | "pygments_lexer": "ipython3", 92 | "version": "3.9.13" 93 | } 94 | }, 95 | "nbformat": 4, 96 | "nbformat_minor": 5 97 | } 98 | -------------------------------------------------------------------------------- /08장 이미지 분류/예제 8.05~8.06 데이터 시각화.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "2e839263-a9a4-4aca-861a-2fddd6f98b98", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from torch.utils.data import DataLoader\n", 11 | "from torchvision.datasets import ImageFolder\n", 12 | "from torchvision import transforms\n", 13 | "\n", 14 | "\n", 15 | "hyperparams = {\n", 16 | " \"batch_size\": 4,\n", 17 | " \"learning_rate\": 0.0001,\n", 18 | " \"epochs\": 5,\n", 19 | " \"transform\": transforms.Compose(\n", 20 | " [\n", 21 | " transforms.Resize(256),\n", 22 | " transforms.CenterCrop(224),\n", 23 | " transforms.ToTensor(),\n", 24 | " transforms.Normalize(\n", 25 | " mean=[0.48235, 0.45882, 0.40784],\n", 26 | " std=[0.229, 0.224, 0.225]\n", 27 | " )\n", 28 | " ]\n", 29 | " )\n", 30 | "}\n", 31 | "\n", 32 | "train_dataset = ImageFolder(\"../datasets/pet/train\", transform=hyperparams[\"transform\"])\n", 33 | "test_dataset = ImageFolder(\"../datasets/pet/test\", transform=hyperparams[\"transform\"])\n", 34 | "\n", 35 | "train_dataloader = DataLoader(train_dataset, batch_size=hyperparams[\"batch_size\"], shuffle=True, drop_last=True)\n", 36 | "test_dataloader = DataLoader(test_dataset, batch_size=hyperparams[\"batch_size\"], shuffle=True, drop_last=True)" 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": null, 42 | "id": "594e9833-44e2-4ac4-a1d6-266e19c4a2c8", 43 | "metadata": {}, 44 | "outputs": [], 45 | "source": [ 46 | "import numpy as np\n", 47 | "from matplotlib import pyplot as plt\n", 48 | "\n", 49 | "\n", 50 | "mean=[0.48235, 0.45882, 0.40784]\n", 51 | "std=[0.229, 0.224, 0.225]\n", 52 | "\n", 53 | "images, labels = next(iter(train_dataloader))\n", 54 | "for image, label in zip(images, labels):\n", 55 | " image = image.numpy().transpose((1, 2, 0))\n", 56 | " image = ((std * image + mean) * 255).astype(np.uint8)\n", 57 | "\n", 58 | " plt.imshow(image)\n", 59 | " plt.title(train_dataset.classes[int(label)])\n", 60 | " plt.show()\n", 61 | " break" 62 | ] 63 | } 64 | ], 65 | "metadata": { 66 | "kernelspec": { 67 | "display_name": "Python 3 (ipykernel)", 68 | "language": "python", 69 | "name": "python3" 70 | }, 71 | "language_info": { 72 | "codemirror_mode": { 73 | "name": "ipython", 74 | "version": 3 75 | }, 76 | "file_extension": ".py", 77 | "mimetype": "text/x-python", 78 | "name": "python", 79 | "nbconvert_exporter": "python", 80 | "pygments_lexer": "ipython3", 81 | "version": "3.9.13" 82 | } 83 | }, 84 | "nbformat": 4, 85 | "nbformat_minor": 5 86 | } 87 | -------------------------------------------------------------------------------- /08장 이미지 분류/예제 8.14~8.17 클래스 활성화 맵 실습.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "14d53aaf-b04b-4df0-9079-85deb9faf606", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from torch import nn\n", 11 | "from torchvision import models\n", 12 | "\n", 13 | "\n", 14 | "model = models.resnet18(weights=\"ResNet18_Weights.IMAGENET1K_V1\").eval()\n", 15 | "features = nn.Sequential(*list(model.children())[:-2])" 16 | ] 17 | }, 18 | { 19 | "cell_type": "code", 20 | "execution_count": null, 21 | "id": "f7423527-4d43-4176-b2df-ba8468c594fc", 22 | "metadata": {}, 23 | "outputs": [], 24 | "source": [ 25 | "from PIL import Image\n", 26 | "from torchvision import transforms\n", 27 | "\n", 28 | "\n", 29 | "transform = transforms.Compose(\n", 30 | " [\n", 31 | " transforms.Resize((224, 224)),\n", 32 | " transforms.ToTensor(),\n", 33 | " transforms.Normalize(\n", 34 | " mean=[0.485, 0.456, 0.406],\n", 35 | " std=[0.229, 0.224, 0.225]\n", 36 | " ),\n", 37 | " ]\n", 38 | ")\n", 39 | "\n", 40 | "image = Image.open(\"../datasets/images/airplane.jpg\")\n", 41 | "target = transform(image).unsqueeze(0)\n", 42 | "\n", 43 | "output = model(target)\n", 44 | "class_idx = int(output.argmax())\n", 45 | "\n", 46 | "weights = model.fc.weight[class_idx].reshape(-1, 1, 1)\n", 47 | "features_output = features(target).squeeze()\n", 48 | "\n", 49 | "print(weights.shape)\n", 50 | "print(features_output.shape)" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": null, 56 | "id": "4611dd2d-64d4-4eda-b7d9-3583a2927129", 57 | "metadata": {}, 58 | "outputs": [], 59 | "source": [ 60 | "import torch\n", 61 | "from torch.nn import functional as F\n", 62 | "\n", 63 | "\n", 64 | "cam = features_output * weights\n", 65 | "cam = torch.sum(cam, dim=0)\n", 66 | "cam = F.interpolate(\n", 67 | " input=cam.unsqueeze(0).unsqueeze(0),\n", 68 | " size=(image.size[1], image.size[0]),\n", 69 | " mode=\"bilinear\",\n", 70 | ").squeeze().detach().numpy()" 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": null, 76 | "id": "92ce0bb0-7e57-401a-afb3-fe2e1e45297c", 77 | "metadata": {}, 78 | "outputs": [], 79 | "source": [ 80 | "import matplotlib.pyplot as plt\n", 81 | "\n", 82 | "\n", 83 | "plt.imshow(image)\n", 84 | "plt.imshow(cam, cmap=\"jet\", alpha=0.5)\n", 85 | "plt.axis(\"off\")\n", 86 | "plt.show()" 87 | ] 88 | } 89 | ], 90 | "metadata": { 91 | "kernelspec": { 92 | "display_name": "Python 3 (ipykernel)", 93 | "language": "python", 94 | "name": "python3" 95 | }, 96 | "language_info": { 97 | "codemirror_mode": { 98 | "name": "ipython", 99 | "version": 3 100 | }, 101 | "file_extension": ".py", 102 | "mimetype": "text/x-python", 103 | "name": "python", 104 | "nbconvert_exporter": "python", 105 | "pygments_lexer": "ipython3", 106 | "version": "3.9.13" 107 | } 108 | }, 109 | "nbformat": 4, 110 | "nbformat_minor": 5 111 | } 112 | -------------------------------------------------------------------------------- /부록/예제 A.03 파이토치 라이트닝 모델 학습.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "76e1a03b-6797-4e69-b9e7-7b1eb8512906", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "import lightning\n", 12 | "from torch import nn\n", 13 | "from torch.nn import functional as F\n", 14 | "\n", 15 | "\n", 16 | "class AutoEncoder(lightning.LightningModule):\n", 17 | " def __init__(self):\n", 18 | " super().__init__()\n", 19 | " self.encoder = nn.Sequential(\n", 20 | " nn.Linear(28 * 28, 128),\n", 21 | " nn.ReLU(),\n", 22 | " nn.Linear(128, 3)\n", 23 | " )\n", 24 | " self.decoder = nn.Sequential(\n", 25 | " nn.Linear(3, 128),\n", 26 | " nn.ReLU(),\n", 27 | " nn.Linear(128, 28 * 28)\n", 28 | " )\n", 29 | "\n", 30 | " def forward(self, x):\n", 31 | " embedding = self.encoder(x)\n", 32 | " return embedding\n", 33 | "\n", 34 | " def training_step(self, batch):\n", 35 | " x, _ = batch\n", 36 | " x = x.view(x.size(0), -1)\n", 37 | " z = self.encoder(x)\n", 38 | " x_hat = self.decoder(z)\n", 39 | " loss = F.mse_loss(x_hat, x)\n", 40 | " self.log(\"train_loss\", loss)\n", 41 | " return loss\n", 42 | "\n", 43 | " def configure_optimizers(self):\n", 44 | " optimizer = torch.optim.Adam(self.parameters(), lr=1e-3)\n", 45 | " return optimizer" 46 | ] 47 | }, 48 | { 49 | "cell_type": "code", 50 | "execution_count": null, 51 | "id": "d05bcb0c-05fc-4045-ad7e-4661a49e5142", 52 | "metadata": {}, 53 | "outputs": [], 54 | "source": [ 55 | "import torchvision\n", 56 | "from torch.utils.data import random_split\n", 57 | "\n", 58 | "\n", 59 | "dataset = torchvision.datasets.MNIST(\n", 60 | " \"../datasets\", download=True, transform=torchvision.transforms.ToTensor()\n", 61 | ")\n", 62 | "train, valid = random_split(dataset, [55000, 5000])\n", 63 | "print(dataset)" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": null, 69 | "id": "10eebcf7-b5cd-4714-9d98-12ebd3ed72c3", 70 | "metadata": {}, 71 | "outputs": [], 72 | "source": [ 73 | "from torch.utils.data import DataLoader\n", 74 | "\n", 75 | "\n", 76 | "autoencoder = AutoEncoder()\n", 77 | "trainer = lightning.Trainer(accelerator=\"gpu\", devices=1, max_epochs=1)\n", 78 | "trainer.fit(autoencoder, DataLoader(train), DataLoader(valid))" 79 | ] 80 | } 81 | ], 82 | "metadata": { 83 | "kernelspec": { 84 | "display_name": "Python 3 (ipykernel)", 85 | "language": "python", 86 | "name": "python3" 87 | }, 88 | "language_info": { 89 | "codemirror_mode": { 90 | "name": "ipython", 91 | "version": 3 92 | }, 93 | "file_extension": ".py", 94 | "mimetype": "text/x-python", 95 | "name": "python", 96 | "nbconvert_exporter": "python", 97 | "pygments_lexer": "ipython3", 98 | "version": "3.9.13" 99 | } 100 | }, 101 | "nbformat": 4, 102 | "nbformat_minor": 5 103 | } 104 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.38 이미지 모델.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "# demo.py\n", 11 | "import cv2\n", 12 | "import torch\n", 13 | "import numpy as np\n", 14 | "import streamlit as st\n", 15 | "from PIL import Image\n", 16 | "from ultralytics import YOLO\n", 17 | "from ultralytics.yolo.utils.plotting import Annotator\n", 18 | "\n", 19 | "\n", 20 | "@st.cache_resource\n", 21 | "def load_model():\n", 22 | " return YOLO(\"yolov8m-pose.pt\")\n", 23 | "\n", 24 | "\n", 25 | "def predict(frame, iou=0.7, conf=0.25):\n", 26 | " results = model(\n", 27 | " source=frame,\n", 28 | " device=\"0\" if torch.cuda.is_available() else \"cpu\",\n", 29 | " iou=0.7,\n", 30 | " conf=0.25,\n", 31 | " verbose=False,\n", 32 | " )\n", 33 | " result = results[0]\n", 34 | " return result\n", 35 | "\n", 36 | "\n", 37 | "def draw_boxes(result, frame):\n", 38 | " for boxes in result.boxes:\n", 39 | " x1, y1, x2, y2, score, classes = boxes.data.squeeze().cpu().numpy()\n", 40 | " cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 0, 255), 1)\n", 41 | " return frame\n", 42 | "\n", 43 | "\n", 44 | "def draw_keypoints(result, frame):\n", 45 | " annotator = Annotator(frame, line_width=1)\n", 46 | " for kps in result.keypoints:\n", 47 | " annotator.kpts(kps)\n", 48 | "\n", 49 | " for idx, kp in enumerate(kps):\n", 50 | " x, y, score = kp.data.squeeze().cpu().numpy()\n", 51 | " \n", 52 | " if score > 0.5:\n", 53 | " cv2.circle(frame, (int(x), int(y)), 3, (0, 0, 255), cv2.FILLED)\n", 54 | " cv2.putText(frame, str(idx), (int(x), int(y)), cv2.FONT_HERSHEY_COMPLEX, 1, (0, 0, 255), 1)\n", 55 | " \n", 56 | " return frame\n", 57 | "\n", 58 | "\n", 59 | "model = load_model()\n", 60 | "uploaded_file = st.file_uploader(\"파일 선택\", type=[\"PNG\", \"JPG\", \"JPEG\"])\n", 61 | "if uploaded_file is not None:\n", 62 | " if \"image\" in uploaded_file.type:\n", 63 | " with st.spinner(text=\"포즈 정보 추출 중..\"):\n", 64 | " pil_image = Image.open(uploaded_file).convert(\"RGB\")\n", 65 | " np_image = np.asarray(pil_image)\n", 66 | " cv_image = cv2.cvtColor(np_image, cv2.COLOR_RGB2BGR)\n", 67 | " \n", 68 | " result = predict(cv_image)\n", 69 | " image = draw_boxes(result, cv_image)\n", 70 | " image = draw_keypoints(result, image)\n", 71 | " st.image(image, channels=\"BGR\")" 72 | ] 73 | } 74 | ], 75 | "metadata": { 76 | "kernelspec": { 77 | "display_name": "Python 3 (ipykernel)", 78 | "language": "python", 79 | "name": "python3" 80 | }, 81 | "language_info": { 82 | "codemirror_mode": { 83 | "name": "ipython", 84 | "version": 3 85 | }, 86 | "file_extension": ".py", 87 | "mimetype": "text/x-python", 88 | "name": "python", 89 | "nbconvert_exporter": "python", 90 | "pygments_lexer": "ipython3", 91 | "version": "3.9.13" 92 | } 93 | }, 94 | "nbformat": 4, 95 | "nbformat_minor": 5 96 | } 97 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.06 응답 기반 지식 증류를 활용한 모델 학습.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import nn\n", 12 | "from torch import optim\n", 13 | "from torch.nn import functional as F\n", 14 | "\n", 15 | "\n", 16 | "class TeacherModel(nn.Module):\n", 17 | " def __init__(self, input_dim, hidden_dim, output_dim):\n", 18 | " super(TeacherModel, self).__init__()\n", 19 | " self.fc1 = nn.Linear(input_dim, hidden_dim)\n", 20 | " self.fc2 = nn.Linear(hidden_dim, output_dim)\n", 21 | "\n", 22 | " def forward(self, x):\n", 23 | " x = torch.relu(self.fc1(x))\n", 24 | " x = self.fc2(x)\n", 25 | " return x\n", 26 | "\n", 27 | "\n", 28 | "class StudentModel(nn.Module):\n", 29 | " def __init__(self, input_dim, hidden_dim, output_dim):\n", 30 | " super(StudentModel, self).__init__()\n", 31 | " self.fc1 = nn.Linear(input_dim, hidden_dim)\n", 32 | " self.fc2 = nn.Linear(hidden_dim, output_dim)\n", 33 | "\n", 34 | " def forward(self, x):\n", 35 | " x = torch.relu(self.fc1(x))\n", 36 | " x = self.fc2(x)\n", 37 | " return x\n", 38 | "\n", 39 | "\n", 40 | "def distillation_loss(y, labels, teacher_output, T, alpha):\n", 41 | " student_softmax = F.log_softmax(y / T, dim=1)\n", 42 | " teacher_softmax = F.softmax(teacher_output / T, dim=1)\n", 43 | " temperature_loss = T * T * 2.0 + alpha\n", 44 | "\n", 45 | " kld_loss = nn.KLDivLoss(reduction=\"batchmean\")(student_softmax, teacher_softmax)\n", 46 | " kld_loss = kld_loss * temperature_loss\n", 47 | " ce_loss = F.cross_entropy(y, labels) * (1.0 - alpha)\n", 48 | "\n", 49 | " total_loss = kld_loss + ce_loss\n", 50 | " return total_loss\n", 51 | "\n", 52 | "\n", 53 | "input_dim = 100\n", 54 | "output_dim = 10\n", 55 | "teacher = TeacherModel(input_dim, 256, output_dim)\n", 56 | "student = StudentModel(input_dim, 128, output_dim)\n", 57 | "optimizer = optim.Adam(student.parameters(), lr=0.001)\n", 58 | "\n", 59 | "input_data = torch.randn(1, input_dim)\n", 60 | "with torch.no_grad():\n", 61 | " teacher_output = teacher(input_data)\n", 62 | "\n", 63 | "optimizer.zero_grad()\n", 64 | "student_output = student(input_data)\n", 65 | "loss = distillation_loss(\n", 66 | " y=student_output,\n", 67 | " labels=torch.tensor([0]),\n", 68 | " teacher_output=teacher_output,\n", 69 | " T=0.1,\n", 70 | " alpha=0.5,\n", 71 | ")\n", 72 | "loss.backward()\n", 73 | "optimizer.step()\n", 74 | "\n", 75 | "print(\"Teacher Model Output:\", teacher_output)\n", 76 | "print(\"Student Model Output:\", student_output)" 77 | ] 78 | } 79 | ], 80 | "metadata": { 81 | "kernelspec": { 82 | "display_name": "Python 3 (ipykernel)", 83 | "language": "python", 84 | "name": "python3" 85 | }, 86 | "language_info": { 87 | "codemirror_mode": { 88 | "name": "ipython", 89 | "version": 3 90 | }, 91 | "file_extension": ".py", 92 | "mimetype": "text/x-python", 93 | "name": "python", 94 | "nbconvert_exporter": "python", 95 | "pygments_lexer": "ipython3", 96 | "version": "3.9.13" 97 | } 98 | }, 99 | "nbformat": 4, 100 | "nbformat_minor": 5 101 | } 102 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.02 VGG-16 학습 후 정적 양자화.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "from torch import nn\n", 12 | "from torch.ao import quantization\n", 13 | "from torchvision import models\n", 14 | "from torchvision import transforms\n", 15 | "from torch.utils.data import DataLoader\n", 16 | "from torchvision.datasets import ImageFolder\n", 17 | "\n", 18 | "\n", 19 | "class QuantizedVGG16(nn.Module):\n", 20 | " def __init__(self, model_fp32):\n", 21 | " super(QuantizedVGG16, self).__init__()\n", 22 | " self.quant = quantization.QuantStub()\n", 23 | " self.dequant = quantization.DeQuantStub()\n", 24 | " self.model_fp32 = model_fp32\n", 25 | " \n", 26 | " def forward(self, x):\n", 27 | " x = self.quant(x)\n", 28 | " x = self.model_fp32(x)\n", 29 | " x = self.dequant(x)\n", 30 | " return x\n", 31 | "\n", 32 | "hyperparams = {\n", 33 | " \"batch_size\": 4,\n", 34 | " \"learning_rate\": 0.0001,\n", 35 | " \"epochs\": 5,\n", 36 | " \"transform\": transforms.Compose(\n", 37 | " [\n", 38 | " transforms.Resize(256),\n", 39 | " transforms.CenterCrop(224),\n", 40 | " transforms.ToTensor(),\n", 41 | " transforms.Normalize(\n", 42 | " mean=[0.48235, 0.45882, 0.40784],\n", 43 | " std=[0.229, 0.224, 0.225],\n", 44 | " ),\n", 45 | " ]\n", 46 | " ),\n", 47 | "}\n", 48 | "\n", 49 | "model = models.vgg16(num_classes=2)\n", 50 | "model.load_state_dict(torch.load(\"../models/VGG16.pt\"))\n", 51 | "\n", 52 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 53 | "quantized_model = QuantizedVGG16(model).to(device)\n", 54 | "\n", 55 | "quantization_backend = \"fbgemm\"\n", 56 | "quantized_model.qconfig = quantization.get_default_qconfig(quantization_backend)\n", 57 | "\n", 58 | "model_static_quantized = quantization.prepare(quantized_model)\n", 59 | "\n", 60 | "calibration_dataset = ImageFolder(\n", 61 | " \"../datasets/pet/test\",\n", 62 | " transform=hyperparams[\"transform\"]\n", 63 | ")\n", 64 | "calibration_dataloader = DataLoader(\n", 65 | " calibration_dataset,\n", 66 | " batch_size=hyperparams[\"batch_size\"]\n", 67 | ")\n", 68 | "\n", 69 | "for i, (images, target) in enumerate(calibration_dataloader):\n", 70 | " if i >= 10:\n", 71 | " break\n", 72 | " model_static_quantized(images.to(device))\n", 73 | "\n", 74 | "model_static_quantized.to(\"cpu\")\n", 75 | "model_static_quantized = quantization.convert(model_static_quantized)\n", 76 | "\n", 77 | "torch.jit.save(torch.jit.script(model_static_quantized), \"../models/PTSQ_VGG16.pt\")" 78 | ] 79 | } 80 | ], 81 | "metadata": { 82 | "kernelspec": { 83 | "display_name": "Python 3 (ipykernel)", 84 | "language": "python", 85 | "name": "python3" 86 | }, 87 | "language_info": { 88 | "codemirror_mode": { 89 | "name": "ipython", 90 | "version": 3 91 | }, 92 | "file_extension": ".py", 93 | "mimetype": "text/x-python", 94 | "name": "python", 95 | "nbconvert_exporter": "python", 96 | "pygments_lexer": "ipython3", 97 | "version": "3.9.13" 98 | } 99 | }, 100 | "nbformat": 4, 101 | "nbformat_minor": 5 102 | } 103 | -------------------------------------------------------------------------------- /08장 이미지 분류/예제 8.01~8.04 알렉스넷 모델 실습.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f72bfb8a-fe55-4bb4-bab1-fd27cea72c56", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from torchvision import models\n", 11 | "from torchinfo import summary\n", 12 | "\n", 13 | "\n", 14 | "model = models.alexnet(weights=\"AlexNet_Weights.IMAGENET1K_V1\")\n", 15 | "summary(model, (1, 3, 224, 224), device=\"cpu\")" 16 | ] 17 | }, 18 | { 19 | "cell_type": "code", 20 | "execution_count": null, 21 | "id": "6c6d74f1-5526-4aa1-9f12-aff2d2731367", 22 | "metadata": {}, 23 | "outputs": [], 24 | "source": [ 25 | "with open(\"../datasets/imagenet_classes.txt\", \"r\") as file:\n", 26 | " classes = file.read().splitlines()\n", 27 | " \n", 28 | "print(f\"클래스 개수 : {len(classes)}\")\n", 29 | "print(f\"첫 번째 클래스 레이블 : {classes[0]}\")" 30 | ] 31 | }, 32 | { 33 | "cell_type": "code", 34 | "execution_count": null, 35 | "id": "a919bc32-b25a-40ec-9b09-ded68821ef27", 36 | "metadata": {}, 37 | "outputs": [], 38 | "source": [ 39 | "import torch\n", 40 | "from PIL import Image\n", 41 | "from torchvision import models, transforms\n", 42 | "\n", 43 | "\n", 44 | "transform = transforms.Compose(\n", 45 | " [\n", 46 | " transforms.Resize((224, 224)),\n", 47 | " transforms.ToTensor(),\n", 48 | " transforms.Normalize(\n", 49 | " mean=[0.485, 0.456, 0.406],\n", 50 | " std=[0.229, 0.224, 0.225]\n", 51 | " ),\n", 52 | " ]\n", 53 | ")\n", 54 | "\n", 55 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 56 | "model = models.alexnet(weights=\"AlexNet_Weights.IMAGENET1K_V1\").eval().to(device)\n", 57 | "\n", 58 | "tensors = []\n", 59 | "files = [\"../datasets/images/airplane.jpg\", \"../datasets/images/bus.jpg\"]\n", 60 | "for file in files:\n", 61 | " image = Image.open(file)\n", 62 | " tensors.append(transform(image))\n", 63 | "\n", 64 | "tensors = torch.stack(tensors)\n", 65 | "print(f\"입력 텐서의 크기 : {tensors.shape}\")" 66 | ] 67 | }, 68 | { 69 | "cell_type": "code", 70 | "execution_count": null, 71 | "id": "0cc3cddd-2615-4867-9697-e82d922aedf7", 72 | "metadata": {}, 73 | "outputs": [], 74 | "source": [ 75 | "import numpy as np\n", 76 | "from torch.nn import functional as F\n", 77 | "\n", 78 | "\n", 79 | "with torch.no_grad():\n", 80 | " outputs = model(tensors.to(device))\n", 81 | " probs = F.softmax(outputs, dim=-1)\n", 82 | " top_probs, top_idxs = probs.topk(5)\n", 83 | "\n", 84 | "top_probs = top_probs.detach().cpu().numpy()\n", 85 | "top_idxs = top_idxs.detach().cpu().numpy()\n", 86 | "top_classes = np.array(classes)[top_idxs]\n", 87 | "\n", 88 | "for idx, (cls, prob) in enumerate(zip(top_classes, top_probs)):\n", 89 | " print(f\"{files[idx]} 추론 결과\")\n", 90 | " for c, p in zip(cls, prob):\n", 91 | " print(f\" - {c:<30} : {p * 100:>5.2f}%\")" 92 | ] 93 | } 94 | ], 95 | "metadata": { 96 | "kernelspec": { 97 | "display_name": "Python 3 (ipykernel)", 98 | "language": "python", 99 | "name": "python3" 100 | }, 101 | "language_info": { 102 | "codemirror_mode": { 103 | "name": "ipython", 104 | "version": 3 105 | }, 106 | "file_extension": ".py", 107 | "mimetype": "text/x-python", 108 | "name": "python", 109 | "nbconvert_exporter": "python", 110 | "pygments_lexer": "ipython3", 111 | "version": "3.9.13" 112 | } 113 | }, 114 | "nbformat": 4, 115 | "nbformat_minor": 5 116 | } 117 | -------------------------------------------------------------------------------- /부록/예제 A.04 콜백 정의.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "503595df-8536-4e9f-a8aa-df7bad8fcd0b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "import lightning\n", 12 | "from torch import nn\n", 13 | "from torch.nn import functional as F\n", 14 | "\n", 15 | "\n", 16 | "class AutoEncoder(lightning.LightningModule):\n", 17 | " def __init__(self):\n", 18 | " super().__init__()\n", 19 | " self.encoder = nn.Sequential(\n", 20 | " nn.Linear(28 * 28, 128),\n", 21 | " nn.ReLU(),\n", 22 | " nn.Linear(128, 3)\n", 23 | " )\n", 24 | " self.decoder = nn.Sequential(\n", 25 | " nn.Linear(3, 128),\n", 26 | " nn.ReLU(),\n", 27 | " nn.Linear(128, 28 * 28)\n", 28 | " )\n", 29 | "\n", 30 | " def forward(self, x):\n", 31 | " embedding = self.encoder(x)\n", 32 | " return embedding\n", 33 | "\n", 34 | " def training_step(self, batch):\n", 35 | " x, _ = batch\n", 36 | " x = x.view(x.size(0), -1)\n", 37 | " z = self.encoder(x)\n", 38 | " x_hat = self.decoder(z)\n", 39 | " loss = F.mse_loss(x_hat, x)\n", 40 | " self.log(\"train_loss\", loss)\n", 41 | " return loss\n", 42 | "\n", 43 | " def configure_optimizers(self):\n", 44 | " optimizer = torch.optim.Adam(self.parameters(), lr=1e-3)\n", 45 | " return optimizer" 46 | ] 47 | }, 48 | { 49 | "cell_type": "code", 50 | "execution_count": null, 51 | "id": "d05bcb0c-05fc-4045-ad7e-4661a49e5142", 52 | "metadata": {}, 53 | "outputs": [], 54 | "source": [ 55 | "import torchvision\n", 56 | "from torch.utils.data import random_split\n", 57 | "\n", 58 | "\n", 59 | "dataset = torchvision.datasets.MNIST(\n", 60 | " \"../datasets\", download=True, transform=torchvision.transforms.ToTensor()\n", 61 | ")\n", 62 | "train, valid = random_split(dataset, [55000, 5000])\n", 63 | "print(dataset)" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": null, 69 | "id": "10eebcf7-b5cd-4714-9d98-12ebd3ed72c3", 70 | "metadata": {}, 71 | "outputs": [], 72 | "source": [ 73 | "from torch.utils.data import DataLoader\n", 74 | "\n", 75 | "\n", 76 | "autoencoder = AutoEncoder()\n", 77 | "trainer = lightning.Trainer(accelerator=\"gpu\", devices=1, max_epochs=1)\n", 78 | "trainer.fit(autoencoder, DataLoader(train), DataLoader(valid))" 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": null, 84 | "id": "3079aaa4-bef9-4e9e-9e28-bfdbf7b450f5", 85 | "metadata": {}, 86 | "outputs": [], 87 | "source": [ 88 | "import lightning\n", 89 | "from lightning.pytorch import callbacks\n", 90 | "\n", 91 | "\n", 92 | "early_stopping = callbacks.EarlyStopping(monitor=\"val_loss\")\n", 93 | "model_checkpoint = callbacks.ModelCheckpoint(monitor=\"val_loss\")\n", 94 | "\n", 95 | "trainer = lightning.Trainer(callbacks=[early_stopping, model_checkpoint])" 96 | ] 97 | } 98 | ], 99 | "metadata": { 100 | "kernelspec": { 101 | "display_name": "Python 3 (ipykernel)", 102 | "language": "python", 103 | "name": "python3" 104 | }, 105 | "language_info": { 106 | "codemirror_mode": { 107 | "name": "ipython", 108 | "version": 3 109 | }, 110 | "file_extension": ".py", 111 | "mimetype": "text/x-python", 112 | "name": "python", 113 | "nbconvert_exporter": "python", 114 | "pygments_lexer": "ipython3", 115 | "version": "3.9.13" 116 | } 117 | }, 118 | "nbformat": 4, 119 | "nbformat_minor": 5 120 | } 121 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.17~11.18 패스트 API 모델 서빙.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "# app_fastapi.py\n", 11 | "import io\n", 12 | "import torch\n", 13 | "import base64\n", 14 | "from PIL import Image\n", 15 | "from torch.nn import functional as F\n", 16 | "from torchvision import models, transforms\n", 17 | "\n", 18 | "\n", 19 | "class VGG16Model:\n", 20 | " def __init__(self, weight_path):\n", 21 | " self.device = 'cuda' if torch.cuda.is_available() else 'cpu'\n", 22 | " self.transform = transforms.Compose(\n", 23 | " [\n", 24 | " transforms.Resize(256),\n", 25 | " transforms.CenterCrop(224),\n", 26 | " transforms.ToTensor(),\n", 27 | " transforms.Normalize(\n", 28 | " mean=[0.48235, 0.45882, 0.40784],\n", 29 | " std=[0.229, 0.224, 0.225]\n", 30 | " )\n", 31 | " ]\n", 32 | " )\n", 33 | " self.model = models.vgg16(num_classes=2).to(self.device)\n", 34 | " self.model.load_state_dict(torch.load(weight_path, map_location=self.device))\n", 35 | " self.model.eval()\n", 36 | "\n", 37 | " def preprocessing(self, data):\n", 38 | " decode = base64.b64decode(data)\n", 39 | " bytes = io.BytesIO(decode)\n", 40 | " image = Image.open(bytes)\n", 41 | " input_data = self.transform(image).to(self.device)\n", 42 | " return input_data\n", 43 | " \n", 44 | " @torch.no_grad()\n", 45 | " def predict(self, input):\n", 46 | " input_data = self.preprocessing(input)\n", 47 | " outputs = self.model(input_data.unsqueeze(0))\n", 48 | " probs = F.softmax(outputs, dim=-1)\n", 49 | " \n", 50 | " index = int(probs[0].argmax(axis=-1))\n", 51 | " label = \"개\" if index == 1 else \"고양이\"\n", 52 | " score = float(probs[0][index])\n", 53 | "\n", 54 | " return {\n", 55 | " \"label\": label,\n", 56 | " \"score\": score\n", 57 | " }" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": null, 63 | "id": "0b322f29", 64 | "metadata": {}, 65 | "outputs": [], 66 | "source": [ 67 | "# app_fastapi.py\n", 68 | "import uvicorn\n", 69 | "from pydantic import BaseModel\n", 70 | "from fastapi import FastAPI, Depends, HTTPException\n", 71 | "\n", 72 | "\n", 73 | "app = FastAPI()\n", 74 | "vgg = VGG16Model(weight_path=\"./VGG16.pt\")\n", 75 | "\n", 76 | "\n", 77 | "class Item(BaseModel):\n", 78 | " base64: str\n", 79 | "\n", 80 | "\n", 81 | "def get_model():\n", 82 | " return vgg\n", 83 | "\n", 84 | "\n", 85 | "@app.post(\"/predict\")\n", 86 | "async def inference(item: Item, model: VGG16Model = Depends(get_model)):\n", 87 | " try:\n", 88 | " return model.predict(item.base64)\n", 89 | "\n", 90 | " except Exception as e:\n", 91 | " raise HTTPException(status_code=500, detail=str(e))\n", 92 | "\n", 93 | "\n", 94 | "if __name__ == \"__main__\":\n", 95 | " uvicorn.run(app=\"app_fastapi:app\", host=\"0.0.0.0\", port=8000, workers=2)" 96 | ] 97 | } 98 | ], 99 | "metadata": { 100 | "kernelspec": { 101 | "display_name": "Python 3 (ipykernel)", 102 | "language": "python", 103 | "name": "python3" 104 | }, 105 | "language_info": { 106 | "codemirror_mode": { 107 | "name": "ipython", 108 | "version": 3 109 | }, 110 | "file_extension": ".py", 111 | "mimetype": "text/x-python", 112 | "name": "python", 113 | "nbconvert_exporter": "python", 114 | "pygments_lexer": "ipython3", 115 | "version": "3.9.13" 116 | } 117 | }, 118 | "nbformat": 4, 119 | "nbformat_minor": 5 120 | } 121 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.52 체크포인트 불러오기.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "5fde76fd-d62f-48d3-960f-5b5e3e0b6139", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "import pandas as pd\n", 12 | "from torch import nn\n", 13 | "from torch import optim\n", 14 | "from torch.utils.data import Dataset, DataLoader\n", 15 | "\n", 16 | "\n", 17 | "class CustomDataset(Dataset):\n", 18 | " def __init__(self, file_path):\n", 19 | " df = pd.read_csv(file_path)\n", 20 | " self.x = df.iloc[:, 0].values\n", 21 | " self.y = df.iloc[:, 1].values\n", 22 | " self.length = len(df)\n", 23 | "\n", 24 | " def __getitem__(self, index):\n", 25 | " x = torch.FloatTensor([self.x[index] ** 2, self.x[index]])\n", 26 | " y = torch.FloatTensor([self.y[index]])\n", 27 | " return x, y\n", 28 | "\n", 29 | " def __len__(self):\n", 30 | " return self.length\n", 31 | "\n", 32 | "class CustomModel(nn.Module):\n", 33 | " def __init__(self):\n", 34 | " super().__init__()\n", 35 | " self.layer = nn.Linear(2, 1)\n", 36 | "\n", 37 | " def forward(self, x):\n", 38 | " x = self.layer(x)\n", 39 | " return x" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": null, 45 | "id": "0e41d65c", 46 | "metadata": {}, 47 | "outputs": [], 48 | "source": [ 49 | "train_dataset = CustomDataset(\"../datasets/non_linear.csv\")\n", 50 | "train_dataloader = DataLoader(train_dataset, batch_size=128, shuffle=True, drop_last=True)" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": null, 56 | "id": "044788d0", 57 | "metadata": {}, 58 | "outputs": [], 59 | "source": [ 60 | "\n", 61 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 62 | "model = CustomModel().to(device)\n", 63 | "criterion = nn.MSELoss().to(device)\n", 64 | "optimizer = optim.SGD(model.parameters(), lr=0.0001)" 65 | ] 66 | }, 67 | { 68 | "cell_type": "code", 69 | "execution_count": null, 70 | "id": "05c4d997", 71 | "metadata": {}, 72 | "outputs": [], 73 | "source": [ 74 | "\n", 75 | "checkpoint = torch.load(\"../models/checkpoint-6.pt\")\n", 76 | "model.load_state_dict(checkpoint[\"model_state_dict\"])\n", 77 | "optimizer.load_state_dict(checkpoint[\"optimizer_state_dict\"])\n", 78 | "checkpoint_epoch = checkpoint[\"epoch\"]\n", 79 | "checkpoint_description = checkpoint[\"description\"]\n", 80 | "print(checkpoint_description)" 81 | ] 82 | }, 83 | { 84 | "cell_type": "code", 85 | "execution_count": null, 86 | "id": "930c2c59", 87 | "metadata": {}, 88 | "outputs": [], 89 | "source": [ 90 | "\n", 91 | "for epoch in range(checkpoint_epoch + 1, 10000):\n", 92 | " cost = 0.0\n", 93 | "\n", 94 | " for x, y in train_dataloader:\n", 95 | " x = x.to(device)\n", 96 | " y = y.to(device)\n", 97 | "\n", 98 | " output = model(x)\n", 99 | " loss = criterion(output, y)\n", 100 | "\n", 101 | " optimizer.zero_grad()\n", 102 | " loss.backward()\n", 103 | " optimizer.step()\n", 104 | "\n", 105 | " cost += loss\n", 106 | " if (epoch + 1) % 1000 == 0:\n", 107 | " print(f\"Epoch : {epoch+1:4d}, Model : {list(model.parameters())}, Cost : {cost:.3f}\")" 108 | ] 109 | } 110 | ], 111 | "metadata": { 112 | "kernelspec": { 113 | "display_name": "Python 3 (ipykernel)", 114 | "language": "python", 115 | "name": "python3" 116 | }, 117 | "language_info": { 118 | "codemirror_mode": { 119 | "name": "ipython", 120 | "version": 3 121 | }, 122 | "file_extension": ".py", 123 | "mimetype": "text/x-python", 124 | "name": "python", 125 | "nbconvert_exporter": "python", 126 | "pygments_lexer": "ipython3", 127 | "version": "3.9.13" 128 | } 129 | }, 130 | "nbformat": 4, 131 | "nbformat_minor": 5 132 | } 133 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.13~11.14 플라스크 모델 서빙.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "# app_flask.py\n", 11 | "import torch\n", 12 | "from torch.nn import functional as F\n", 13 | "from transformers import BertTokenizer, BertForSequenceClassification\n", 14 | "\n", 15 | "\n", 16 | "class BertModel:\n", 17 | " device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 18 | "\n", 19 | " @classmethod\n", 20 | " def load_model(cls, weight_path):\n", 21 | " cls.tokenizer = BertTokenizer.from_pretrained(\n", 22 | " pretrained_model_name_or_path=\"bert-base-multilingual-cased\",\n", 23 | " do_lower_case=False,\n", 24 | " )\n", 25 | " cls.model = BertForSequenceClassification.from_pretrained(\n", 26 | " pretrained_model_name_or_path=\"bert-base-multilingual-cased\",\n", 27 | " num_labels=2\n", 28 | " ).to(cls.device)\n", 29 | " cls.model.load_state_dict(torch.load(weight_path, map_location=cls.device))\n", 30 | " cls.model.eval()\n", 31 | " \n", 32 | " @classmethod\n", 33 | " def preprocessing(cls, data):\n", 34 | " input_data = cls.tokenizer(\n", 35 | " text=data,\n", 36 | " padding=\"longest\",\n", 37 | " truncation=True,\n", 38 | " return_tensors=\"pt\"\n", 39 | " ).to(cls.device)\n", 40 | " return input_data\n", 41 | "\n", 42 | " @classmethod\n", 43 | " @torch.no_grad()\n", 44 | " def predict(cls, input):\n", 45 | " input_data = cls.preprocessing(input)\n", 46 | " outputs = cls.model(**input_data).logits\n", 47 | " probs = F.softmax(outputs, dim=-1)\n", 48 | " \n", 49 | " index = int(probs[0].argmax(axis=-1))\n", 50 | " label = \"긍정\" if index == 1 else \"부정\"\n", 51 | " score = float(probs[0][index])\n", 52 | "\n", 53 | " return {\n", 54 | " \"label\": label,\n", 55 | " \"score\": score\n", 56 | " }" 57 | ] 58 | }, 59 | { 60 | "cell_type": "code", 61 | "execution_count": null, 62 | "id": "0c2cf75a", 63 | "metadata": {}, 64 | "outputs": [], 65 | "source": [ 66 | "# app_flask.py\n", 67 | "import json\n", 68 | "from flask import Flask, request, Response\n", 69 | "\n", 70 | "\n", 71 | "app = Flask(__name__)\n", 72 | "\n", 73 | "\n", 74 | "@app.route(\"/predict\", methods=[\"POST\"])\n", 75 | "def inference():\n", 76 | " data = request.get_json()\n", 77 | " text = data[\"text\"]\n", 78 | "\n", 79 | " try:\n", 80 | " return Response(\n", 81 | " response=json.dumps(BertModel.predict(text), ensure_ascii=False),\n", 82 | " status=200,\n", 83 | " mimetype=\"application/json\",\n", 84 | " )\n", 85 | "\n", 86 | " except Exception as e:\n", 87 | " return Response(\n", 88 | " response=json.dumps({\"error\": str(e)}, ensure_ascii=False),\n", 89 | " status=500,\n", 90 | " mimetype=\"application/json\",\n", 91 | " )\n", 92 | "\n", 93 | "\n", 94 | "if __name__ == \"__main__\":\n", 95 | " BertModel.load_model(weight_path=\"../models/BertForSequenceClassification.pt\")\n", 96 | " app.run(host=\"0.0.0.0\", port=8000)" 97 | ] 98 | } 99 | ], 100 | "metadata": { 101 | "kernelspec": { 102 | "display_name": "Python 3 (ipykernel)", 103 | "language": "python", 104 | "name": "python3" 105 | }, 106 | "language_info": { 107 | "codemirror_mode": { 108 | "name": "ipython", 109 | "version": 3 110 | }, 111 | "file_extension": ".py", 112 | "mimetype": "text/x-python", 113 | "name": "python", 114 | "nbconvert_exporter": "python", 115 | "pygments_lexer": "ipython3", 116 | "version": "3.9.13" 117 | } 118 | }, 119 | "nbformat": 4, 120 | "nbformat_minor": 5 121 | } 122 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.58 단층 퍼셉트론 구조.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "01abc433-b0d8-458d-af31-1074012acdae", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "import pandas as pd\n", 12 | "from torch import nn\n", 13 | "from torch import optim\n", 14 | "from torch.utils.data import Dataset, DataLoader\n", 15 | "\n", 16 | "\n", 17 | "class CustomDataset(Dataset):\n", 18 | " def __init__(self, file_path):\n", 19 | " df = pd.read_csv(file_path)\n", 20 | " self.x1 = df.iloc[:, 0].values\n", 21 | " self.x2 = df.iloc[:, 1].values\n", 22 | " self.y = df.iloc[:, 2].values\n", 23 | " self.length = len(df)\n", 24 | "\n", 25 | " def __getitem__(self, index):\n", 26 | " x = torch.FloatTensor([self.x1[index], self.x2[index]])\n", 27 | " y = torch.FloatTensor([self.y[index]])\n", 28 | " return x, y\n", 29 | "\n", 30 | " def __len__(self):\n", 31 | " return self.length\n", 32 | "\n", 33 | "class CustomModel(nn.Module):\n", 34 | " def __init__(self):\n", 35 | " super().__init__()\n", 36 | "\n", 37 | " self.layer = nn.Sequential(\n", 38 | " nn.Linear(2, 1),\n", 39 | " nn.Sigmoid()\n", 40 | " )\n", 41 | "\n", 42 | " def forward(self, x):\n", 43 | " x = self.layer(x)\n", 44 | " return x" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "id": "bfeec018", 51 | "metadata": {}, 52 | "outputs": [], 53 | "source": [ 54 | "train_dataset = CustomDataset(\"../datasets/perceptron.csv\")\n", 55 | "train_dataloader = DataLoader(train_dataset, batch_size=64, shuffle=True, drop_last=True)" 56 | ] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "execution_count": null, 61 | "id": "8d3779d5", 62 | "metadata": {}, 63 | "outputs": [], 64 | "source": [ 65 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 66 | "model = CustomModel().to(device)\n", 67 | "criterion = nn.BCELoss().to(device)\n", 68 | "optimizer = optim.SGD(model.parameters(), lr=0.01)" 69 | ] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "execution_count": null, 74 | "id": "9ee09ab9", 75 | "metadata": {}, 76 | "outputs": [], 77 | "source": [ 78 | "for epoch in range(10000):\n", 79 | " cost = 0.0\n", 80 | "\n", 81 | " for x, y in train_dataloader:\n", 82 | " x = x.to(device)\n", 83 | " y = y.to(device)\n", 84 | "\n", 85 | " output = model(x)\n", 86 | " loss = criterion(output, y)\n", 87 | "\n", 88 | " optimizer.zero_grad()\n", 89 | " loss.backward()\n", 90 | " optimizer.step()\n", 91 | "\n", 92 | " cost += loss\n", 93 | "\n", 94 | " cost = cost / len(train_dataloader)\n", 95 | "\n", 96 | " if (epoch + 1) % 1000 == 0:\n", 97 | " print(f\"Epoch : {epoch+1:4d}, Cost : {cost:.3f}\")" 98 | ] 99 | }, 100 | { 101 | "cell_type": "code", 102 | "execution_count": null, 103 | "id": "f17bc138", 104 | "metadata": {}, 105 | "outputs": [], 106 | "source": [ 107 | "with torch.no_grad():\n", 108 | " model.eval()\n", 109 | " inputs = torch.FloatTensor([\n", 110 | " [0, 0],\n", 111 | " [0, 1],\n", 112 | " [1, 0],\n", 113 | " [1, 1]\n", 114 | " ]).to(device)\n", 115 | " outputs = model(inputs)\n", 116 | " \n", 117 | " print(\"---------\")\n", 118 | " print(outputs)\n", 119 | " print(outputs <= 0.5)" 120 | ] 121 | } 122 | ], 123 | "metadata": { 124 | "kernelspec": { 125 | "display_name": "Python 3 (ipykernel)", 126 | "language": "python", 127 | "name": "python3" 128 | }, 129 | "language_info": { 130 | "codemirror_mode": { 131 | "name": "ipython", 132 | "version": 3 133 | }, 134 | "file_extension": ".py", 135 | "mimetype": "text/x-python", 136 | "name": "python", 137 | "nbconvert_exporter": "python", 138 | "pygments_lexer": "ipython3", 139 | "version": "3.9.13" 140 | } 141 | }, 142 | "nbformat": 4, 143 | "nbformat_minor": 5 144 | } 145 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.49 모델 상태 저장.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "5fde76fd-d62f-48d3-960f-5b5e3e0b6139", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "import pandas as pd\n", 12 | "from torch import nn\n", 13 | "from torch import optim\n", 14 | "from torch.utils.data import Dataset, DataLoader\n", 15 | "\n", 16 | "\n", 17 | "class CustomDataset(Dataset):\n", 18 | " def __init__(self, file_path):\n", 19 | " df = pd.read_csv(file_path)\n", 20 | " self.x = df.iloc[:, 0].values\n", 21 | " self.y = df.iloc[:, 1].values\n", 22 | " self.length = len(df)\n", 23 | "\n", 24 | " def __getitem__(self, index):\n", 25 | " x = torch.FloatTensor([self.x[index] ** 2, self.x[index]])\n", 26 | " y = torch.FloatTensor([self.y[index]])\n", 27 | " return x, y\n", 28 | "\n", 29 | " def __len__(self):\n", 30 | " return self.length\n", 31 | "\n", 32 | "class CustomModel(nn.Module):\n", 33 | " def __init__(self):\n", 34 | " super().__init__()\n", 35 | " self.layer = nn.Linear(2, 1)\n", 36 | "\n", 37 | " def forward(self, x):\n", 38 | " x = self.layer(x)\n", 39 | " return x" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": null, 45 | "id": "ab088d05", 46 | "metadata": {}, 47 | "outputs": [], 48 | "source": [ 49 | "\n", 50 | "train_dataset = CustomDataset(\"../datasets/non_linear.csv\")\n", 51 | "train_dataloader = DataLoader(train_dataset, batch_size=128, shuffle=True, drop_last=True)\n", 52 | "\n", 53 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 54 | "model = CustomModel().to(device)\n", 55 | "criterion = nn.MSELoss().to(device)\n", 56 | "optimizer = optim.SGD(model.parameters(), lr=0.0001)" 57 | ] 58 | }, 59 | { 60 | "cell_type": "code", 61 | "execution_count": null, 62 | "id": "b509f77b", 63 | "metadata": {}, 64 | "outputs": [], 65 | "source": [ 66 | "\n", 67 | "for epoch in range(10000):\n", 68 | " cost = 0.0\n", 69 | "\n", 70 | " for x, y in train_dataloader:\n", 71 | " x = x.to(device)\n", 72 | " y = y.to(device)\n", 73 | "\n", 74 | " output = model(x)\n", 75 | " loss = criterion(output, y)\n", 76 | "\n", 77 | " optimizer.zero_grad()\n", 78 | " loss.backward()\n", 79 | " optimizer.step()\n", 80 | "\n", 81 | " cost += loss\n", 82 | "\n", 83 | " cost = cost / len(train_dataloader)\n", 84 | "\n", 85 | " if (epoch + 1) % 1000 == 0:\n", 86 | " print(f\"Epoch : {epoch+1:4d}, Model : {list(model.parameters())}, Cost : {cost:.3f}\")" 87 | ] 88 | }, 89 | { 90 | "cell_type": "code", 91 | "execution_count": null, 92 | "id": "8f921301", 93 | "metadata": {}, 94 | "outputs": [], 95 | "source": [ 96 | "with torch.no_grad():\n", 97 | " model.eval()\n", 98 | " inputs = torch.FloatTensor(\n", 99 | " [\n", 100 | " [1 ** 2, 1],\n", 101 | " [5 ** 2, 5],\n", 102 | " [11 ** 2, 11]\n", 103 | " ]\n", 104 | " ).to(device)\n", 105 | " outputs = model(inputs)\n", 106 | " print(outputs)" 107 | ] 108 | }, 109 | { 110 | "cell_type": "code", 111 | "execution_count": null, 112 | "id": "6c3d9ec7-995d-4179-a5d7-2b22278488a1", 113 | "metadata": {}, 114 | "outputs": [], 115 | "source": [ 116 | "torch.save(\n", 117 | " model.state_dict(),\n", 118 | " \"../models/model_state_dict.pt\"\n", 119 | ")\n" 120 | ] 121 | } 122 | ], 123 | "metadata": { 124 | "kernelspec": { 125 | "display_name": "Python 3 (ipykernel)", 126 | "language": "python", 127 | "name": "python3" 128 | }, 129 | "language_info": { 130 | "codemirror_mode": { 131 | "name": "ipython", 132 | "version": 3 133 | }, 134 | "file_extension": ".py", 135 | "mimetype": "text/x-python", 136 | "name": "python", 137 | "nbconvert_exporter": "python", 138 | "pygments_lexer": "ipython3", 139 | "version": "3.9.13" 140 | } 141 | }, 142 | "nbformat": 4, 143 | "nbformat_minor": 5 144 | } 145 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.09 VGG-16 합성곱 계층 경량화.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "import tensorly as tl\n", 12 | "from torch import nn\n", 13 | "from torchvision import models\n", 14 | "from tensorly import decomposition\n", 15 | "\n", 16 | "tl.set_backend(\"pytorch\")\n", 17 | "\n", 18 | "\n", 19 | "def cp_decomposition(layer, rank):\n", 20 | " weights, factors = decomposition.parafac(\n", 21 | " tensor=layer.weight.data,\n", 22 | " rank=rank,\n", 23 | " init=\"random\",\n", 24 | " normalize_factors=False\n", 25 | " )\n", 26 | " last, first, vertical, horizontal = factors\n", 27 | "\n", 28 | " pointwise_s_to_r_layer = nn.Conv2d(\n", 29 | " first.shape[0],\n", 30 | " first.shape[1],\n", 31 | " kernel_size=1,\n", 32 | " stride=1,\n", 33 | " padding=0,\n", 34 | " dilation=layer.dilation,\n", 35 | " bias=False,\n", 36 | " )\n", 37 | " depthwise_vertical_layer = nn.Conv2d(\n", 38 | " vertical.shape[1],\n", 39 | " vertical.shape[1],\n", 40 | " kernel_size=(vertical.shape[0], 1),\n", 41 | " stride=1,\n", 42 | " padding=(layer.padding[0], 0),\n", 43 | " dilation=layer.dilation,\n", 44 | " groups=vertical.shape[1],\n", 45 | " bias=False,\n", 46 | " )\n", 47 | " depthwise_horizontal_layer = nn.Conv2d(\n", 48 | " horizontal.shape[1],\n", 49 | " horizontal.shape[1],\n", 50 | " kernel_size=(1, horizontal.shape[0]),\n", 51 | " stride=layer.stride,\n", 52 | " padding=(0, layer.padding[0]),\n", 53 | " dilation=layer.dilation,\n", 54 | " groups=horizontal.shape[1],\n", 55 | " bias=False,\n", 56 | " )\n", 57 | " pointwise_r_to_t_layer = nn.Conv2d(\n", 58 | " last.shape[1],\n", 59 | " last.shape[0],\n", 60 | " kernel_size=1,\n", 61 | " stride=1,\n", 62 | " padding=0,\n", 63 | " dilation=layer.dilation,\n", 64 | " bias=True,\n", 65 | " )\n", 66 | " pointwise_r_to_t_layer.bias.data = layer.bias.data\n", 67 | "\n", 68 | " depthwise_horizontal_layer.weight.data = (\n", 69 | " torch.transpose(horizontal, 1, 0).unsqueeze(1).unsqueeze(1)\n", 70 | " )\n", 71 | " depthwise_vertical_layer.weight.data = (\n", 72 | " torch.transpose(vertical, 1, 0).unsqueeze(1).unsqueeze(-1)\n", 73 | " )\n", 74 | " pointwise_s_to_r_layer.weight.data = (\n", 75 | " torch.transpose(first, 1, 0).unsqueeze(-1).unsqueeze(-1)\n", 76 | " )\n", 77 | " pointwise_r_to_t_layer.weight.data = last.unsqueeze(-1).unsqueeze(-1)\n", 78 | "\n", 79 | " new_layers = [\n", 80 | " pointwise_s_to_r_layer,\n", 81 | " depthwise_vertical_layer,\n", 82 | " depthwise_horizontal_layer,\n", 83 | " pointwise_r_to_t_layer,\n", 84 | " ]\n", 85 | " return nn.Sequential(*new_layers)\n", 86 | "\n", 87 | "\n", 88 | "model = models.vgg16(num_classes=2)\n", 89 | "model.load_state_dict(torch.load(\"../models/VGG16.pt\"))\n", 90 | "model.eval()\n", 91 | "\n", 92 | "layer = model.features[0]\n", 93 | "layer_cp_decomposed = cp_decomposition(layer, rank=16)\n", 94 | "\n", 95 | "print(\"CP 분해 전 가중치 수:\", sum(param.numel() for param in layer.parameters()))\n", 96 | "print(\"CP 분해 후 가중치 수:\", sum(param.numel() for param in layer_cp_decomposed.parameters()))" 97 | ] 98 | } 99 | ], 100 | "metadata": { 101 | "kernelspec": { 102 | "display_name": "Python 3 (ipykernel)", 103 | "language": "python", 104 | "name": "python3" 105 | }, 106 | "language_info": { 107 | "codemirror_mode": { 108 | "name": "ipython", 109 | "version": 3 110 | }, 111 | "file_extension": ".py", 112 | "mimetype": "text/x-python", 113 | "name": "python", 114 | "nbconvert_exporter": "python", 115 | "pygments_lexer": "ipython3", 116 | "version": "3.9.13" 117 | } 118 | }, 119 | "nbformat": 4, 120 | "nbformat_minor": 5 121 | } 122 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.51 체크포인트 저장.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "5fde76fd-d62f-48d3-960f-5b5e3e0b6139", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "import pandas as pd\n", 12 | "from torch import nn\n", 13 | "from torch import optim\n", 14 | "from torch.utils.data import Dataset, DataLoader\n", 15 | "\n", 16 | "\n", 17 | "class CustomDataset(Dataset):\n", 18 | " def __init__(self, file_path):\n", 19 | " df = pd.read_csv(file_path)\n", 20 | " self.x = df.iloc[:, 0].values\n", 21 | " self.y = df.iloc[:, 1].values\n", 22 | " self.length = len(df)\n", 23 | "\n", 24 | " def __getitem__(self, index):\n", 25 | " x = torch.FloatTensor([self.x[index] ** 2, self.x[index]])\n", 26 | " y = torch.FloatTensor([self.y[index]])\n", 27 | " return x, y\n", 28 | "\n", 29 | " def __len__(self):\n", 30 | " return self.length\n", 31 | "\n", 32 | "class CustomModel(nn.Module):\n", 33 | " def __init__(self):\n", 34 | " super().__init__()\n", 35 | " self.layer = nn.Linear(2, 1)\n", 36 | "\n", 37 | " def forward(self, x):\n", 38 | " x = self.layer(x)\n", 39 | " return x" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": null, 45 | "id": "60b82f55", 46 | "metadata": {}, 47 | "outputs": [], 48 | "source": [ 49 | "\n", 50 | "train_dataset = CustomDataset(\"../datasets/non_linear.csv\")\n", 51 | "train_dataloader = DataLoader(train_dataset, batch_size=128, shuffle=True, drop_last=True)\n", 52 | "\n", 53 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 54 | "model = CustomModel().to(device)\n", 55 | "criterion = nn.MSELoss().to(device)\n", 56 | "optimizer = optim.SGD(model.parameters(), lr=0.0001)" 57 | ] 58 | }, 59 | { 60 | "cell_type": "code", 61 | "execution_count": null, 62 | "id": "42cb1d7e", 63 | "metadata": {}, 64 | "outputs": [], 65 | "source": [ 66 | "\n", 67 | "checkpoint = 1\n", 68 | "for epoch in range(10000):\n", 69 | " cost = 0.0\n", 70 | "\n", 71 | " for x, y in train_dataloader:\n", 72 | " x = x.to(device)\n", 73 | " y = y.to(device)\n", 74 | "\n", 75 | " output = model(x)\n", 76 | " loss = criterion(output, y)\n", 77 | "\n", 78 | " optimizer.zero_grad()\n", 79 | " loss.backward()\n", 80 | " optimizer.step()\n", 81 | "\n", 82 | " cost += loss\n", 83 | "\n", 84 | " cost = cost / len(train_dataloader)\n", 85 | "\n", 86 | " if (epoch + 1) % 1000 == 0:\n", 87 | " torch.save(\n", 88 | " {\n", 89 | " \"model\": \"CustomModel\",\n", 90 | " \"epoch\": epoch,\n", 91 | " \"model_state_dict\": model.state_dict(),\n", 92 | " \"optimizer_state_dict\": optimizer.state_dict(),\n", 93 | " \"cost\": cost,\n", 94 | " \"description\": f\"CustomModel 체크포인트-{checkpoint}\",\n", 95 | " },\n", 96 | " f\"../models/checkpoint-{checkpoint}.pt\",\n", 97 | " )\n", 98 | " checkpoint += 1" 99 | ] 100 | }, 101 | { 102 | "cell_type": "code", 103 | "execution_count": null, 104 | "id": "04c8c12a", 105 | "metadata": {}, 106 | "outputs": [], 107 | "source": [ 108 | "with torch.no_grad():\n", 109 | " model.eval()\n", 110 | " inputs = torch.FloatTensor(\n", 111 | " [\n", 112 | " [1 ** 2, 1],\n", 113 | " [5 ** 2, 5],\n", 114 | " [11 ** 2, 11]\n", 115 | " ]\n", 116 | " ).to(device)\n", 117 | " outputs = model(inputs)\n", 118 | " print(outputs)" 119 | ] 120 | } 121 | ], 122 | "metadata": { 123 | "kernelspec": { 124 | "display_name": "Python 3 (ipykernel)", 125 | "language": "python", 126 | "name": "python3" 127 | }, 128 | "language_info": { 129 | "codemirror_mode": { 130 | "name": "ipython", 131 | "version": 3 132 | }, 133 | "file_extension": ".py", 134 | "mimetype": "text/x-python", 135 | "name": "python", 136 | "nbconvert_exporter": "python", 137 | "pygments_lexer": "ipython3", 138 | "version": "3.9.13" 139 | } 140 | }, 141 | "nbformat": 4, 142 | "nbformat_minor": 5 143 | } 144 | -------------------------------------------------------------------------------- /08장 이미지 분류/예제 8.18~8.20 Grad-CAM 실습.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "b70bcb41-8c3d-4274-9c49-edffa0a56c75", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "\n", 12 | "\n", 13 | "class GradCAM:\n", 14 | " def __init__(self, model, main, sub):\n", 15 | " self.model = model.eval()\n", 16 | " self.register_hook(main, sub)\n", 17 | "\n", 18 | " def register_hook(self, main, sub):\n", 19 | " for name, module in self.model.named_children():\n", 20 | " if name == main:\n", 21 | " for sub_name, sub_module in module[-1].named_children():\n", 22 | " if sub_name == sub:\n", 23 | " sub_module.register_forward_hook(self.forward_hook)\n", 24 | " sub_module.register_full_backward_hook(self.backward_hook)\n", 25 | "\n", 26 | " def forward_hook(self, module, input, output):\n", 27 | " self.feature_map = output \n", 28 | "\n", 29 | " def backward_hook(self, module, grad_input, grad_output):\n", 30 | " self.gradient = grad_output[0]\n", 31 | " \n", 32 | " def __call__(self, x):\n", 33 | " output = self.model(x)\n", 34 | "\n", 35 | " index = output.argmax(axis=1)\n", 36 | " one_hot = torch.zeros_like(output) \n", 37 | " for i in range(output.size(0)):\n", 38 | " one_hot[i][index[i]] = 1\n", 39 | "\n", 40 | " self.model.zero_grad()\n", 41 | " output.backward(gradient=one_hot, retain_graph=True)\n", 42 | "\n", 43 | " a_k = torch.mean(self.gradient, dim=(2, 3), keepdim=True)\n", 44 | " grad_cam = torch.sum(a_k * self.feature_map, dim=1)\n", 45 | " grad_cam = torch.relu(grad_cam)\n", 46 | " return grad_cam" 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "execution_count": null, 52 | "id": "8f755211-80ad-4c19-a7f8-3bd50fd3afbc", 53 | "metadata": {}, 54 | "outputs": [], 55 | "source": [ 56 | "from PIL import Image\n", 57 | "from torch.nn import functional as F\n", 58 | "from torchvision import models\n", 59 | "from torchvision import transforms\n", 60 | "from matplotlib import pyplot as plt\n", 61 | "\n", 62 | "\n", 63 | "transform = transforms.Compose(\n", 64 | " [\n", 65 | " transforms.Resize((224, 224)),\n", 66 | " transforms.ToTensor(),\n", 67 | " transforms.Normalize(\n", 68 | " mean=[0.485, 0.456, 0.406],\n", 69 | " std=[0.229, 0.224, 0.225]\n", 70 | " ),\n", 71 | " ]\n", 72 | ")\n", 73 | "\n", 74 | "files = [\n", 75 | " \"../datasets/images/airplane.jpg\", \"../datasets/images/bus.jpg\",\n", 76 | " \"../datasets/images/dog.jpg\", \"../datasets/images/african_hunting_dog.jpg\"\n", 77 | "]\n", 78 | "images, tensors = [], []\n", 79 | "for file in files:\n", 80 | " image = Image.open(file)\n", 81 | " images.append(image)\n", 82 | " tensors.append(transform(image))\n", 83 | "tensors = torch.stack(tensors)\n", 84 | "\n", 85 | "model = GradCAM(\n", 86 | " model=models.resnet18(weights=\"ResNet18_Weights.IMAGENET1K_V1\"),\n", 87 | " main=\"layer4\",\n", 88 | " sub=\"conv2\"\n", 89 | ")\n", 90 | "grad_cams = model(tensors)\n", 91 | "\n", 92 | "for idx, image in enumerate(images):\n", 93 | " grad_cam = F.interpolate(\n", 94 | " input=grad_cams[idx].unsqueeze(0).unsqueeze(0),\n", 95 | " size=(image.size[1], image.size[0]),\n", 96 | " mode=\"bilinear\",\n", 97 | " ).squeeze().detach().numpy()\n", 98 | "\n", 99 | " plt.imshow(image)\n", 100 | " plt.imshow(grad_cam, cmap=\"jet\", alpha=0.5)\n", 101 | " plt.axis(\"off\")\n", 102 | " plt.show()" 103 | ] 104 | } 105 | ], 106 | "metadata": { 107 | "kernelspec": { 108 | "display_name": "Python 3 (ipykernel)", 109 | "language": "python", 110 | "name": "python3" 111 | }, 112 | "language_info": { 113 | "codemirror_mode": { 114 | "name": "ipython", 115 | "version": 3 116 | }, 117 | "file_extension": ".py", 118 | "mimetype": "text/x-python", 119 | "name": "python", 120 | "nbconvert_exporter": "python", 121 | "pygments_lexer": "ipython3", 122 | "version": "3.9.13" 123 | } 124 | }, 125 | "nbformat": 4, 126 | "nbformat_minor": 5 127 | } 128 | -------------------------------------------------------------------------------- /03장 파이토치 기초/예제 3.59 다층 퍼셉트론 구조.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "01abc433-b0d8-458d-af31-1074012acdae", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "import pandas as pd\n", 12 | "from torch import nn\n", 13 | "from torch import optim\n", 14 | "from torch.utils.data import Dataset, DataLoader" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": null, 20 | "id": "914e83e6", 21 | "metadata": {}, 22 | "outputs": [], 23 | "source": [ 24 | "class CustomDataset(Dataset):\n", 25 | " def __init__(self, file_path):\n", 26 | " df = pd.read_csv(file_path)\n", 27 | " self.x1 = df.iloc[:, 0].values\n", 28 | " self.x2 = df.iloc[:, 1].values\n", 29 | " self.y = df.iloc[:, 2].values\n", 30 | " self.length = len(df)\n", 31 | "\n", 32 | " def __getitem__(self, index):\n", 33 | " x = torch.FloatTensor([self.x1[index], self.x2[index]])\n", 34 | " y = torch.FloatTensor([self.y[index]])\n", 35 | " return x, y\n", 36 | "\n", 37 | " def __len__(self):\n", 38 | " return self.length\n", 39 | "\n", 40 | "class CustomModel(nn.Module):\n", 41 | " def __init__(self):\n", 42 | " super().__init__()\n", 43 | "\n", 44 | " self.layer1 = nn.Sequential(\n", 45 | " nn.Linear(2, 2),\n", 46 | " nn.Sigmoid()\n", 47 | " )\n", 48 | " self.layer2 = nn.Sequential(\n", 49 | " nn.Linear(2, 1),\n", 50 | " nn.Sigmoid()\n", 51 | " )\n", 52 | "\n", 53 | " def forward(self, x):\n", 54 | " x = self.layer1(x)\n", 55 | " x = self.layer2(x)\n", 56 | " return x" 57 | ] 58 | }, 59 | { 60 | "cell_type": "code", 61 | "execution_count": null, 62 | "id": "f153f6c6", 63 | "metadata": {}, 64 | "outputs": [], 65 | "source": [ 66 | "train_dataset = CustomDataset(\"../datasets/perceptron.csv\")\n", 67 | "train_dataloader = DataLoader(train_dataset, batch_size=64, shuffle=True, drop_last=True)" 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": null, 73 | "id": "8323e7c7", 74 | "metadata": {}, 75 | "outputs": [], 76 | "source": [ 77 | "\n", 78 | "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", 79 | "model = CustomModel().to(device)\n", 80 | "criterion = nn.BCELoss().to(device)\n", 81 | "optimizer = optim.SGD(model.parameters(), lr=0.01)" 82 | ] 83 | }, 84 | { 85 | "cell_type": "code", 86 | "execution_count": null, 87 | "id": "b353360e", 88 | "metadata": {}, 89 | "outputs": [], 90 | "source": [ 91 | "for epoch in range(10000):\n", 92 | " cost = 0.0\n", 93 | "\n", 94 | " for x, y in train_dataloader:\n", 95 | " x = x.to(device)\n", 96 | " y = y.to(device)\n", 97 | "\n", 98 | " output = model(x)\n", 99 | " loss = criterion(output, y)\n", 100 | "\n", 101 | " optimizer.zero_grad()\n", 102 | " loss.backward()\n", 103 | " optimizer.step()\n", 104 | "\n", 105 | " cost += loss\n", 106 | "\n", 107 | " cost = cost / len(train_dataloader)\n", 108 | "\n", 109 | " if (epoch + 1) % 1000 == 0:\n", 110 | " print(f\"Epoch : {epoch+1:4d}, Cost : {cost:.3f}\")" 111 | ] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "execution_count": null, 116 | "id": "43635367", 117 | "metadata": {}, 118 | "outputs": [], 119 | "source": [ 120 | "with torch.no_grad():\n", 121 | " model.eval()\n", 122 | " inputs = torch.FloatTensor([\n", 123 | " [0, 0],\n", 124 | " [0, 1],\n", 125 | " [1, 0],\n", 126 | " [1, 1]\n", 127 | " ]).to(device)\n", 128 | " outputs = model(inputs)\n", 129 | " \n", 130 | " print(\"---------\")\n", 131 | " print(outputs)\n", 132 | " print(outputs <= 0.5)" 133 | ] 134 | } 135 | ], 136 | "metadata": { 137 | "kernelspec": { 138 | "display_name": "Python 3 (ipykernel)", 139 | "language": "python", 140 | "name": "python3" 141 | }, 142 | "language_info": { 143 | "codemirror_mode": { 144 | "name": "ipython", 145 | "version": 3 146 | }, 147 | "file_extension": ".py", 148 | "mimetype": "text/x-python", 149 | "name": "python", 150 | "nbconvert_exporter": "python", 151 | "pygments_lexer": "ipython3", 152 | "version": "3.9.13" 153 | } 154 | }, 155 | "nbformat": 4, 156 | "nbformat_minor": 5 157 | } 158 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 파이토치 트랜스포머를 활용한 자연어 처리와 컴퓨터비전 심층학습 2 | 3 | ![pytorch.jpg](pytorch.png) 4 | 5 | - 책의 기준 경로는 `pytorchtrf` 디렉터리입니다. 6 | - 일부 예제는 이전 예제에서 생성된 데이터나 모델을 활용합니다. 7 | - 데이터세트는 `datasets` 디렉터리에 있다고 가정합니다. 8 | - 생성된 모델 파일은 `models` 디렉터리에 있다고 가정합니다. 9 | 10 | ## Datasets 11 | 12 | 이 책에서 사용된 데이터세트는 [캐글](https://www.kaggle.com/datasets/s076923/pytorch-transformer)에서 다운로드할 수 있습니다. 13 | 14 | > https://www.kaggle.com/datasets/s076923/pytorch-transformer 15 | 16 | ### Kaggle API download guide 17 | 18 | #### 토큰 생성 19 | 20 | 캐글에 로그인하여, [캐글 계정 설정](https://www.kaggle.com/settings/account) 페이지로 이동합니다. 21 | 22 | 캐글 계정 설정 페이지의 API 영역으로 이동하여, [Create New Token] 버튼으로 토큰을 생성합니다. 23 | 24 | ![kaggle.png](kaggle.png) 25 | 26 | #### 토큰 파일 경로 이동 27 | 28 | 생성된 토큰은 `kaggle.json` 파일로 제공됩니다. 29 | 30 | 다운로드 받은 `토큰 파일(kaggle.json)`을 캐글 설정 경로로 이동합니다. 31 | 32 | **Windows** 33 | > C:\Users\\\\\.kaggle\kaggle.json 34 | 35 | **MacOS / Linux** 36 | > ~/.kaggle/kaggle.json 37 | 38 | > chmod 600 ~/.kaggle/kaggle.json 39 | 40 | MacOS 및 Linux의 경우 chmod를 통해 권한을 변경합니다. 41 | 42 | #### 캐글 API 설치 43 | 44 | ``` 45 | pip install kaggle --upgrade 46 | ``` 47 | 48 | pip 또는 pip3를 통해 캐글 API를 설치합니다. 49 | 50 | #### 데이터세트 다운로드 51 | 52 | ``` 53 | kaggle datasets download -d s076923/pytorch-transformer 54 | ``` 55 | 56 | Kaggle API를 통해 데이터세트를 다운로드 받을 수 있습니다. 57 | 58 | 구글 코랩(Google Colaboratory)의 경우, [colab-datasets-download.ipynb](https://github.com/wikibook/pytorchtrf/blob/main/colab-datasets-download.ipynb) 파일에서 수행 방법을 확인해볼 수 있습니다. 59 | 60 | ## Notice 61 | 62 | 각각의 예제는 실행 환경 및 라이브러리/프레임워크 버전에 따라 결과가 다를 수 있습니다. 63 | 64 | 이 책에서 사용된 라이브러리/프레임워크 버전은 다음과 같습니다. 65 | 66 | ``` 67 | absl-py==1.4.0 68 | cython==3.0.2 69 | datasets==2.14.5 70 | evaluate==0.4.0 71 | fastapi==0.103.1 72 | flask==2.3.3 73 | gensim==4.1.2 74 | imgaug==0.4.0 75 | jamo==0.4.1 76 | konlpy==0.6.0 77 | korpora==0.2.0 78 | lightning==2.0.1 79 | nlpaug==1.1.11 80 | nltk==3.7 81 | opencv-python==4.8.0.76 82 | pandas==1.4.4 83 | pillow==9.2.0 84 | portalocker==2.7.0 85 | pycocotools==2.0.7 86 | requests==2.28.1 87 | sacremoses==0.0.53 88 | scikit-learn==1.0.2 89 | sentencepiece==0.1.97 90 | spacy==3.6.1 91 | streamlit==1.22.0 92 | tensorly==0.8.1 93 | timm==0.9.7 94 | tokenizers==0.13.3 95 | torch==2.0.1 96 | torchdata==0.6.1 97 | torchinfo==1.8.0 98 | torchtext==0.15.2 99 | torchvision==0.15.2 100 | transformers==4.33.2 101 | ultralytics==8.0.128 102 | ``` 103 | 104 | 일부 라이브러리/프레임워크는 안정적인(stable) 버전이 변경될 수 있으므로, 이 책에서 사용한 버전 설치가 불가능할 수 있습니다. 105 | 106 | 이런 경우 책에서 사용한 버전과 유사한 버전으로 설치합니다. 107 | 108 | ### MacOS M1/M2 : Device 109 | 110 | 이 책은 CUDA 가속을 기반으로 작성되어 애플 실리콘이 탑재된 맥에서는 일부 코드가 실행되지 않거나 GPU 가속이 되지 않습니다. 111 | 112 | 애플 실리콘이 탑재된 맥에서는 cuda 메서드가 지원되지 않으므로, MPS(Metal Performance Shaders) 가속을 사용합니다. 113 | 114 | 애플 실리콘 사용자는 다음과 같이 to 메서드로 MPS 가속을 적용할 수 있습니다. 115 | 116 | ```python 117 | import torch 118 | 119 | 120 | device = "mps" if torch.backends.mps.is_available() and torch.backends.mps.is_built() else "cpu" 121 | tensor = torch.FloatTensor([1, 2, 3]).to(device) 122 | ``` 123 | 124 | ### MacOS M1/M2 : Operator not implemented 125 | 126 | 파이토치의 일부 연산은 MPS 가속이 불가능할 수 있습니다. 127 | 128 | MPS 경고가 발생하는 경우 다음과 같이 환경 설정을 변경합니다. 129 | 130 | ```python 131 | import os 132 | 133 | 134 | os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1" 135 | ``` 136 | 137 | os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1" 설정은 MPS를 활성화하고, MPS를 지원하지 않는 환경에서도 파이토치가 작동할 수 있도록 설정합니다. 138 | 139 | 이 설정은 MPS 문제가 발생하는 환경에서 파이토치가 비활성화되지 않도록 합니다. 140 | 141 | MPS 가속 사용 여부는 사용중인 하드웨어와 파이토치 버전에 따라 다를 수 있으므로 상황에 맞게 설정을 조정해야 합니다. 142 | 143 | 144 | ### _torchtext.so : Symbol not found 145 | 146 | 일부 라이브러리는 현재 사용 중인 파이토치 버전과 호환되지 않을 수 있습니다. 147 | 148 | 예를 들어 토치 텍스트는 파이토치 버전과 호환되어야 합니다. 149 | 150 | 이러한 경우, 다음과 같이 삭제한 후 재설치하여 버전 호환성을 맞출 수 있습니다. 151 | 152 | ``` 153 | pip uninstall torch torchtext 154 | pip install torch torchtext 155 | ``` 156 | 157 | 파이토치를 재설치할 때, CUDA 버전과의 호환성을 반드시 확인합니다. 158 | 159 | ### Multi30k : Timeout 160 | 161 | Multi30k 데이터세트 다운로드 시 타임아웃 오류가 발생하는 경우, 데이터세트의 URL 주소를 변경해야 합니다. 162 | 163 | URL 주소를 변경하는 방법은 다음과 같습니다. 164 | 165 | ```python 166 | from torchtext.datasets import multi30k 167 | 168 | 169 | multi30k.URL["train"] = "https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/training.tar.gz" 170 | multi30k.URL["valid"] = "https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/validation.tar.gz" 171 | multi30k.URL["test"] = "https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/mmt16_task1_test.tar.gz" 172 | ``` 173 | 174 | ## Contacts 175 | 176 | - 윤대희 : [s076923@gmail.com](mailto:s076923@gmail.com) 177 | - 김동화 : [dhcycle25@gmail.com](mailto:dhcycle25@gmail.com) 178 | - 송종민 : [whdwhd93@naver.com](mailto:whdwhd93@naver.com) 179 | - 진현두 : [gusen0927@gmail.com](mailto:gusen0927@gmail.com) 180 | -------------------------------------------------------------------------------- /11장 모델 배포/예제 11.10 VGG-16 모델 경량화.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "f49f0bd8-df62-4097-9d3e-9378e48c8e9b", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "import torch\n", 11 | "import tensorly as tl\n", 12 | "from torch import nn\n", 13 | "from torchvision import models\n", 14 | "from tensorly import decomposition\n", 15 | "\n", 16 | "tl.set_backend(\"pytorch\")\n", 17 | "\n", 18 | "\n", 19 | "def cp_decomposition(layer, rank):\n", 20 | " weights, factors = decomposition.parafac(\n", 21 | " tensor=layer.weight.data,\n", 22 | " rank=rank,\n", 23 | " init=\"random\",\n", 24 | " normalize_factors=False\n", 25 | " )\n", 26 | " last, first, vertical, horizontal = factors\n", 27 | "\n", 28 | " pointwise_s_to_r_layer = nn.Conv2d(\n", 29 | " first.shape[0],\n", 30 | " first.shape[1],\n", 31 | " kernel_size=1,\n", 32 | " stride=1,\n", 33 | " padding=0,\n", 34 | " dilation=layer.dilation,\n", 35 | " bias=False,\n", 36 | " )\n", 37 | " depthwise_vertical_layer = nn.Conv2d(\n", 38 | " vertical.shape[1],\n", 39 | " vertical.shape[1],\n", 40 | " kernel_size=(vertical.shape[0], 1),\n", 41 | " stride=1,\n", 42 | " padding=(layer.padding[0], 0),\n", 43 | " dilation=layer.dilation,\n", 44 | " groups=vertical.shape[1],\n", 45 | " bias=False,\n", 46 | " )\n", 47 | " depthwise_horizontal_layer = nn.Conv2d(\n", 48 | " horizontal.shape[1],\n", 49 | " horizontal.shape[1],\n", 50 | " kernel_size=(1, horizontal.shape[0]),\n", 51 | " stride=layer.stride,\n", 52 | " padding=(0, layer.padding[0]),\n", 53 | " dilation=layer.dilation,\n", 54 | " groups=horizontal.shape[1],\n", 55 | " bias=False,\n", 56 | " )\n", 57 | " pointwise_r_to_t_layer = nn.Conv2d(\n", 58 | " last.shape[1],\n", 59 | " last.shape[0],\n", 60 | " kernel_size=1,\n", 61 | " stride=1,\n", 62 | " padding=0,\n", 63 | " dilation=layer.dilation,\n", 64 | " bias=True,\n", 65 | " )\n", 66 | " pointwise_r_to_t_layer.bias.data = layer.bias.data\n", 67 | "\n", 68 | " depthwise_horizontal_layer.weight.data = (\n", 69 | " torch.transpose(horizontal, 1, 0).unsqueeze(1).unsqueeze(1)\n", 70 | " )\n", 71 | " depthwise_vertical_layer.weight.data = (\n", 72 | " torch.transpose(vertical, 1, 0).unsqueeze(1).unsqueeze(-1)\n", 73 | " )\n", 74 | " pointwise_s_to_r_layer.weight.data = (\n", 75 | " torch.transpose(first, 1, 0).unsqueeze(-1).unsqueeze(-1)\n", 76 | " )\n", 77 | " pointwise_r_to_t_layer.weight.data = last.unsqueeze(-1).unsqueeze(-1)\n", 78 | "\n", 79 | " new_layers = [\n", 80 | " pointwise_s_to_r_layer,\n", 81 | " depthwise_vertical_layer,\n", 82 | " depthwise_horizontal_layer,\n", 83 | " pointwise_r_to_t_layer,\n", 84 | " ]\n", 85 | " return nn.Sequential(*new_layers)\n", 86 | "\n", 87 | "\n", 88 | "model = models.vgg16(num_classes=2)\n", 89 | "model.load_state_dict(torch.load(\"../models/VGG16.pt\"))\n", 90 | "model.eval()" 91 | ] 92 | }, 93 | { 94 | "cell_type": "code", 95 | "execution_count": null, 96 | "id": "1da11ad6", 97 | "metadata": {}, 98 | "outputs": [], 99 | "source": [ 100 | "import copy\n", 101 | "\n", 102 | "\n", 103 | "decomposed_model = copy.deepcopy(model)\n", 104 | "for idx, module in enumerate(decomposed_model.features):\n", 105 | " if isinstance(module, nn.Conv2d):\n", 106 | " rank = max(module.weight.data.numpy().shape) // 3\n", 107 | " decomposed_model.features[idx] = cp_decomposition(module, rank)\n", 108 | "\n", 109 | "print(\"CP 분해 전 가중치 수 :\", sum(param.numel() for param in model.parameters()))\n", 110 | "print(\"CP 분해 후 가중치 수 :\", sum(param.numel() for param in decomposed_model.parameters()))" 111 | ] 112 | } 113 | ], 114 | "metadata": { 115 | "kernelspec": { 116 | "display_name": "Python 3 (ipykernel)", 117 | "language": "python", 118 | "name": "python3" 119 | }, 120 | "language_info": { 121 | "codemirror_mode": { 122 | "name": "ipython", 123 | "version": 3 124 | }, 125 | "file_extension": ".py", 126 | "mimetype": "text/x-python", 127 | "name": "python", 128 | "nbconvert_exporter": "python", 129 | "pygments_lexer": "ipython3", 130 | "version": "3.9.13" 131 | } 132 | }, 133 | "nbformat": 4, 134 | "nbformat_minor": 5 135 | } 136 | --------------------------------------------------------------------------------