├── 01_Introduction └── TinyStories.ipynb ├── 02_NLP-Basics ├── 1_Tokenization.ipynb ├── 2_Token-Embeddings.ipynb └── homework │ └── 3_Homework.ipynb ├── 03_Data └── Data.ipynb ├── 04_Build_GPT-2 └── Build_GPT-2.ipynb ├── 05_Train_Eval_Gen_Functions ├── 1_Train_Eval_Gen_General.ipynb ├── 2_Train_Eval_Gen_LLM_specific_Walkthrough.ipynb ├── 3_Train_Eval_Gen_LLM_specific.ipynb └── homework │ └── 4_Homework.ipynb ├── 06_Train_GPT-2 ├── Train_GPT-2.ipynb ├── Train_GPT-2_Walkthrough.ipynb └── homework │ ├── Ghamarani │ ├── Train_GPT_2_(Checkpoint).ipynb │ └── checkpoint.pdf │ └── nima_kaveh │ ├── Train_GPT-2.ipynb │ ├── Train_GPT-2.pdf │ └── figure │ ├── 2.1.png │ ├── 2.2.png │ ├── 2.3.png │ ├── 3.png │ ├── 4.png │ ├── 5.1.png │ ├── 5.2.png │ ├── 6.png │ ├── 7.1.png │ └── 7.2.png ├── 07_Modal └── Train_GPT-2.ipynb ├── 08_Evaluation └── Evaluation.ipynb ├── README.md └── git /01_Introduction/TinyStories.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/01_Introduction/TinyStories.ipynb -------------------------------------------------------------------------------- /02_NLP-Basics/1_Tokenization.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/02_NLP-Basics/1_Tokenization.ipynb -------------------------------------------------------------------------------- /02_NLP-Basics/2_Token-Embeddings.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/02_NLP-Basics/2_Token-Embeddings.ipynb -------------------------------------------------------------------------------- /02_NLP-Basics/homework/3_Homework.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/02_NLP-Basics/homework/3_Homework.ipynb -------------------------------------------------------------------------------- /03_Data/Data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/03_Data/Data.ipynb -------------------------------------------------------------------------------- /04_Build_GPT-2/Build_GPT-2.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/04_Build_GPT-2/Build_GPT-2.ipynb -------------------------------------------------------------------------------- /05_Train_Eval_Gen_Functions/1_Train_Eval_Gen_General.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/05_Train_Eval_Gen_Functions/1_Train_Eval_Gen_General.ipynb -------------------------------------------------------------------------------- /05_Train_Eval_Gen_Functions/2_Train_Eval_Gen_LLM_specific_Walkthrough.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/05_Train_Eval_Gen_Functions/2_Train_Eval_Gen_LLM_specific_Walkthrough.ipynb -------------------------------------------------------------------------------- /05_Train_Eval_Gen_Functions/3_Train_Eval_Gen_LLM_specific.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/05_Train_Eval_Gen_Functions/3_Train_Eval_Gen_LLM_specific.ipynb -------------------------------------------------------------------------------- /05_Train_Eval_Gen_Functions/homework/4_Homework.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/05_Train_Eval_Gen_Functions/homework/4_Homework.ipynb -------------------------------------------------------------------------------- /06_Train_GPT-2/Train_GPT-2.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/Train_GPT-2.ipynb -------------------------------------------------------------------------------- /06_Train_GPT-2/Train_GPT-2_Walkthrough.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/Train_GPT-2_Walkthrough.ipynb -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/Ghamarani/Train_GPT_2_(Checkpoint).ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/Ghamarani/Train_GPT_2_(Checkpoint).ipynb -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/Ghamarani/checkpoint.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/Ghamarani/checkpoint.pdf -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/nima_kaveh/Train_GPT-2.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/nima_kaveh/Train_GPT-2.ipynb -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/nima_kaveh/Train_GPT-2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/nima_kaveh/Train_GPT-2.pdf -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/nima_kaveh/figure/2.1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/nima_kaveh/figure/2.1.png -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/nima_kaveh/figure/2.2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/nima_kaveh/figure/2.2.png -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/nima_kaveh/figure/2.3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/nima_kaveh/figure/2.3.png -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/nima_kaveh/figure/3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/nima_kaveh/figure/3.png -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/nima_kaveh/figure/4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/nima_kaveh/figure/4.png -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/nima_kaveh/figure/5.1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/nima_kaveh/figure/5.1.png -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/nima_kaveh/figure/5.2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/nima_kaveh/figure/5.2.png -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/nima_kaveh/figure/6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/nima_kaveh/figure/6.png -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/nima_kaveh/figure/7.1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/nima_kaveh/figure/7.1.png -------------------------------------------------------------------------------- /06_Train_GPT-2/homework/nima_kaveh/figure/7.2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/06_Train_GPT-2/homework/nima_kaveh/figure/7.2.png -------------------------------------------------------------------------------- /07_Modal/Train_GPT-2.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/07_Modal/Train_GPT-2.ipynb -------------------------------------------------------------------------------- /08_Evaluation/Evaluation.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/08_Evaluation/Evaluation.ipynb -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/howsam/Building-a-ChatGPT-like-Model-from-Scratch/HEAD/README.md -------------------------------------------------------------------------------- /git: -------------------------------------------------------------------------------- 1 | --------------------------------------------------------------------------------