├── LICENSE
└── README.md
/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2017 Amit Chaudhary
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # learning
2 |
3 | A running log of things I'm learning to build strong core software engineering skills while also expanding my knowledge of [adjacent technologies](http://www.effectiveengineer.com/blog/master-adjacent-disciplines) a little bit [everyday](https://jamesclear.com/continuous-improvement).
4 |
5 | **Updated**: Once a month | **Current** **Focus**: Generative AI
6 |
7 | ## Core Skills
8 |
9 | > Generic skills that are transferrable to any sort of software work I do
10 |
11 | ### Python Programming
12 |
13 | |Resource|Progress|
14 | |---|---|
15 | |[Datacamp: Writing Efficient Python Code](https://www.datacamp.com/courses/writing-efficient-python-code)|✅|
16 | |[Datacamp: Writing Functions in Python](https://www.datacamp.com/courses/writing-functions-in-python)|✅|
17 | |[Datacamp: Object-Oriented Programming in Python](https://www.datacamp.com/courses/object-oriented-programming-in-python)|✅|
18 | |[Datacamp: Intermediate Object-Oriented Programming in Python](https://www.datacamp.com/courses/intermediate-object-oriented-programming-in-python)|✅|
19 | |[Datacamp: Importing Data in Python (Part 1)](https://www.datacamp.com/courses/importing-data-in-python-part-1)|✅|
20 | |[Datacamp: Importing Data in Python (Part 2)](https://www.datacamp.com/courses/importing-data-in-python-part-2)|✅|
21 | |[Datacamp: Intermediate Python for Data Science](https://www.datacamp.com/courses/intermediate-python-for-data-science)|✅|
22 | |[Datacamp: Python Data Science Toolbox (Part 1)](https://www.datacamp.com/courses/python-data-science-toolbox-part-1)|✅|
23 | |[Datacamp: Python Data Science Toolbox (Part 2)](https://www.datacamp.com/courses/python-data-science-toolbox-part-2)|✅|
24 | |[Datacamp: Developing Python Packages](https://www.datacamp.com/courses/developing-python-packages)|✅|
25 | |[Datacamp: Conda Essentials](https://www.datacamp.com/courses/conda-essentials)|✅|
26 | |[Youtube: Tutorial: Sebastian Witowski - Modern Python Developer's Toolkit](https://www.youtube.com/watch?v=WkUBx3g2QfQ&feature=youtu.be)|✅|
27 | |[Datacamp: Working with Dates and Times in Python](https://www.datacamp.com/courses/working-with-dates-and-times-in-python)|✅|
28 | |[Datacamp: Command Line Automation in Python](https://www.datacamp.com/courses/command-line-automation-in-python)|⬜|
29 | |[Book: Python 201](https://leanpub.com/python201)|⬜|
30 | |[Book: Writing Idiomatic Python 3](https://www.amazon.com/Writing-Idiomatic-Python-Jeff-Knupp-ebook/dp/B00B5VXMRG)|⬜|
31 | |[Article: Python's many command-line utilities](https://www.pythonmorsels.com/cli-tools/)|⬜|
32 | |[Article: A Programmer’s Introduction to Unicode](https://www.reedbeta.com/blog/programmers-intro-to-unicode/)|⬜|
33 | |[Article: Exposing string types to maximize user happiness](https://stephantul.github.io/python/typing/2025/03/07/externalized-types/)|✅|
34 |
35 | ### Testing & Profiling
36 |
37 | |Resource|Progress|
38 | |---|---|
39 | |[Datacamp: Unit Testing for Data Science in Python](https://www.datacamp.com/courses/unit-testing-for-data-science-in-python)|✅|
40 | |[Book: Test Driven Development with Python](http://chimera.labs.oreilly.com/books/1234000000754/index.html)|⬜|
41 | |[Article: Introduction to Memory Profiling in Python](https://www.datacamp.com/tutorial/memory-profiling-python)|✅|
42 | |[Article: Profiling Python code with memory_profiler](https://www.wrighters.io/profiling-python-code-with-memory_profiler/)|✅|
43 | |[Article: How to Use "memory_profiler" to Profile Memory Usage by Python Code?](https://coderzcolumn.com/tutorials/python/how-to-profile-memory-usage-in-python-using-memory-profiler)|✅|
44 | |[Youtube: Debug Python inside Docker using debugpy and VSCode](https://www.youtube.com/watch?v=ywfsLKRLmf4)|✅|
45 |
46 |
47 | ### Data Structures and Algorithms
48 |
49 | |Resource|Progress|
50 | |---|---|
51 | |[Book: Grokking Algorithms](https://www.manning.com/books/grokking-algorithms)|✅|
52 | |[Book: The Tech Resume Inside Out](https://thetechresume.com)|✅|
53 | |[Neetcode: Algorithms and Data Structures for Beginners](https://neetcode.io/courses/dsa-for-beginners/0)|✅|
54 | |[Udacity: Intro to Data Structures and Algorithms](https://www.udacity.com/course/technical-interview--ud513)|✅|
55 |
56 |
57 | ### Linux & Command Line
58 |
59 | |Resource|Progress|
60 | |---|---|
61 | |[Datacamp: Introduction to Shell for Data Science](https://www.datacamp.com/courses/introduction-to-shell-for-data-science)|✅|
62 | |[Datacamp: Introduction to Bash Scripting](https://www.datacamp.com/courses/introduction-to-bash-scripting)|✅|
63 | |[Datacamp: Data Processing in Shell](https://www.datacamp.com/courses/data-processing-in-shell)|✅|
64 | |[MIT: The Missing Semester](https://www.youtube.com/playlist?list=PLyzOVJj3bHQuloKGG59rS43e29ro7I57J)|✅|
65 | |[Udacity: Linux Command Line Basics](https://www.udacity.com/course/linux-command-line-basics--ud595)|✅|
66 | |[Udacity: Shell Workshop](https://www.udacity.com/course/shell-workshop--ud206)|✅|
67 | |[Udacity: Configuring Linux Web Servers](https://www.udacity.com/course/configuring-linux-web-servers--ud299)|✅|
68 |
69 | ### Version Control
70 |
71 | |Resource|Progress|
72 | |---|---|
73 | |[Udacity: Version Control with Git](https://www.udacity.com/course/version-control-with-git--ud123)|✅|
74 | |[Datacamp: Introduction to Git for Data Science](https://www.datacamp.com/courses/introduction-to-git-for-data-science)|✅|
75 | |[Udacity: GitHub & Collaboration](https://www.udacity.com/course/github-collaboration--ud456)|✅|
76 | |[Udacity: How to Use Git and GitHub](https://www.udacity.com/course/how-to-use-git-and-github--ud775)|✅|
77 | |[Youtube: How to Use Git Worktree \| Checkout Multiple Git Branches at Once](https://youtu.be/s4BTvj1ZVLM)|✅|
78 |
79 | ### Databases
80 |
81 | |Resource|Progress|
82 | |---|---|
83 | |[Udacity: Intro to relational database](https://www.udacity.com/course/intro-to-relational-databases--ud197)|✅|
84 | |[Udacity: Database Systems Concepts & Design](https://www.udacity.com/course/database-systems-concepts-design--ud150)|⬜|
85 | |[Datacamp: Database Design](https://www.datacamp.com/courses/database-design)|⬜|
86 | |[Datacamp: Introduction to Databases in Python](https://www.datacamp.com/courses/introduction-to-relational-databases-in-python)|⬜|
87 | |[Datacamp: Intro to SQL for Data Science](https://www.datacamp.com/courses/intro-to-sql-for-data-science)|✅|
88 | |[Datacamp: Intermediate SQL](https://www.datacamp.com/courses/intermediate-sql)|⬜|
89 | |[Datacamp: Joining Data in PostgreSQL](https://www.datacamp.com/courses/joining-data-in-postgresql)|⬜|
90 | |[Udacity: SQL for Data Analysis](https://www.udacity.com/course/sql-for-data-analysis--ud198)|⬜|
91 | |[Datacamp: Exploratory Data Analysis in SQL](https://www.datacamp.com/courses/sql-for-exploratory-data-analysis)|⬜|
92 | |[Datacamp: Applying SQL to Real-World Problems](https://www.datacamp.com/courses/applying-sql-to-real-world-problems)|⬜|
93 | |[Datacamp: Analyzing Business Data in SQL](https://www.datacamp.com/courses/analyzing-business-data-in-sql)|⬜|
94 | |[Datacamp: Reporting in SQL](https://www.datacamp.com/courses/reporting-in-sql)|⬜|
95 | |[Datacamp: Data-Driven Decision Making in SQL](https://www.datacamp.com/courses/data-driven-decision-making-with-sql)|⬜|
96 | |[Datacamp: NoSQL Concepts](https://www.datacamp.com/courses/nosql-concepts)|⬜|
97 | |[Datacamp: Introduction to MongoDB in Python](https://www.datacamp.com/courses/introduction-to-using-mongodb-for-data-science-with-python)|⬜|
98 |
99 | ### Backend Engineering
100 |
101 | |Resource|Progress|
102 | |---|---|
103 | |[Udacity: Authentication & Authorization: OAuth](https://www.udacity.com/course/authentication-authorization-oauth--ud330)|✅|
104 | |[Udacity: HTTP & Web Servers](https://www.udacity.com/course/http-web-servers--ud303)|✅|
105 | |[Udacity: Client-Server Communication](https://www.udacity.com/course/client-server-communication--ud897)|⬜|
106 | |[Udacity: Designing RESTful APIs](https://www.udacity.com/course/designing-restful-apis--ud388)|✅|
107 | |[Udacity: Networking for Web Developers](https://www.udacity.com/course/networking-for-web-developers--ud256)|✅|
108 |
109 | ### Production System Design
110 |
111 | |Resource|Progress|
112 | |---|---|
113 | |[Book: Designing Machine Learning Systems](https://www.oreilly.com/library/view/designing-machine-learning/9781098107956/)|✅|
114 | |[Neetcode: System Design for Beginners](https://neetcode.io/courses/system-design-for-beginners/0)|✅|
115 | |[Neetcode: System Design Interview](https://neetcode.io/courses/system-design-interview)|✅|
116 | |[Datacamp: Customer Analytics & A/B Testing in Python](https://www.datacamp.com/courses/customer-analytics-ab-testing-in-python)|✅|
117 | |[Datacamp: A/B Testing in Python](https://www.datacamp.com/courses/ab-testing-in-python)|⬜|
118 | |[Udacity: A/B Testing](https://www.udacity.com/course/ab-testing--ud257)|⬜|
119 | |[Datacamp: MLOps Concepts](https://www.datacamp.com/courses/mlops-concepts)|✅|
120 | |[Datacamp: Machine Learning Monitoring Concepts](https://www.datacamp.com/courses/machine-learning-monitoring-concepts)|✅|
121 |
122 |
123 | ### Maths
124 |
125 | |Resource|Progress|
126 | |---|---|
127 | |[Datacamp: Foundations of Probability in Python](https://www.datacamp.com/courses/foundations-of-probability-in-python)|✅|
128 | |[Datacamp: Introduction to Statistics](https://www.datacamp.com/courses/introduction-to-statistics)|✅|
129 | |[Datacamp: Introduction to Statistics in Python](https://www.datacamp.com/courses/introduction-to-statistics-in-python)|✅|
130 | |[Datacamp: Hypothesis Testing in Python](https://www.datacamp.com/courses/hypothesis-testing-in-python)|✅|
131 | |[Datacamp: Statistical Thinking in Python (Part 1)](https://www.datacamp.com/courses/statistical-thinking-in-python-part-1)|✅|
132 | |[Datacamp: Statistical Thinking in Python (Part 2)](https://www.datacamp.com/courses/statistical-thinking-in-python-part-2)|✅|
133 | |[Datacamp: Experimental Design in Python](https://datacamp.com/courses/experimental-design-in-python)|✅|
134 | |[Datacamp: Practicing Statistics Interview Questions in Python](https://www.datacamp.com/courses/practicing-statistics-interview-questions-in-python)|⬜|
135 | |[edX: Essential Statistics for Data Analysis using Excel](https://www.edx.org/course/essential-statistics-data-analysis-using-microsoft-dat222x-1)|✅|
136 | |[Udacity: Intro to Inferential Statistics](https://www.udacity.com/course/intro-to-inferential-statistics--ud201)|✅|
137 | |[MIT 18.06 Linear Algebra, Spring 2005](https://www.youtube.com/playlist?list=PLE7DDD91010BC51F8)|✅|
138 | |[Udacity: Eigenvectors and Eigenvalues](https://www.udacity.com/course/eigenvectors-and-eigenvalues--ud104)|✅|
139 | |[Udacity: Linear Algebra Refresher](https://www.udacity.com/course/linear-algebra-refresher-course--ud953)|⬜|
140 | |[Youtube: Essence of linear algebra](https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab)|⬜|
141 |
142 | ## Specialization
143 |
144 |
145 | ### Traditional Machine Learning
146 |
147 | |Resource|Progress|
148 | |---|---|
149 | |[Book: Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition](https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/)|⬜|
150 | |[Book: A Machine Learning Primer](https://www.confetti.ai/assets/ml-primer/ml_primer.pdf)|✅|
151 | |[Book: Grokking Machine Learning](https://www.manning.com/books/grokking-machine-learning)|✅|
152 | |[Book: The StatQuest Illustrated Guide To Machine Learning](https://www.amazon.com/StatQuest-Illustrated-Guide-Machine-Learning/dp/B0BLM4TLPY)|✅|
153 | |[Datacamp: Ensemble Methods in Python](https://www.datacamp.com/courses/ensemble-methods-in-python)|✅|
154 | |[Datacamp: Extreme Gradient Boosting with XGBoost](https://www.datacamp.com/courses/extreme-gradient-boosting-with-xgboost)|⬜|
155 | |[Datacamp: Clustering Methods with SciPy](https://www.datacamp.com/courses/clustering-methods-with-scipy)|✅|
156 | |[Datacamp: Unsupervised Learning in Python](https://www.datacamp.com/courses/unsupervised-learning-in-python)|✅|
157 | |[Udacity: Segmentation and Clustering](https://www.udacity.com/course/segmentation-and-clustering--ud981)|✅|
158 | |[Datacamp: Intro to Python for Data Science](https://www.datacamp.com/courses/intro-to-python-for-data-science)|✅|
159 | |[edX: Implementing Predictive Analytics with Spark in Azure HDInsight](https://www.edx.org/course/implementing-predictive-analytics-spark-microsoft-dat202-3x-2)|✅|
160 | |[Datacamp: Supervised Learning with scikit-learn](https://www.datacamp.com/courses/supervised-learning-with-scikit-learn)|✅|
161 | |[Datacamp: Machine Learning with Tree-Based Models in Python](https://www.datacamp.com/courses/machine-learning-with-tree-based-models-in-python)|✅|
162 | |[Datacamp: Linear Classifiers in Python](https://www.datacamp.com/courses/linear-classifiers-in-python)|✅|
163 | |[Datacamp: Model Validation in Python](https://www.datacamp.com/courses/model-validation-in-python)|✅|
164 | |[Datacamp: Hyperparameter Tuning in Python](https://www.datacamp.com/courses/hyperparameter-tuning-in-python)|✅|
165 | |[Datacamp: HR Analytics in Python: Predicting Employee Churn](https://www.datacamp.com/courses/hr-analytics-in-python-predicting-employee-churn)|✅|
166 | |[Datacamp: Predicting Customer Churn in Python](https://www.datacamp.com/courses/predicting-customer-churn-in-python)|✅|
167 | |[Datacamp: Dimensionality Reduction in Python](https://www.datacamp.com/courses/dimensionality-reduction-in-python)|✅|
168 | |[Datacamp: Preprocessing for Machine Learning in Python](https://www.datacamp.com/courses/preprocessing-for-machine-learning-in-python)|✅|
169 | |[Datacamp: Data Types for Data Science](https://www.datacamp.com/courses/data-types-for-data-science)|✅|
170 | |[Datacamp: Cleaning Data in Python](https://www.datacamp.com/courses/cleaning-data-in-python)|✅|
171 | |[Datacamp: Feature Engineering for Machine Learning in Python](https://www.datacamp.com/courses/feature-engineering-for-machine-learning-in-python)|✅|
172 | |[Datacamp: Predicting CTR with Machine Learning in Python](https://www.datacamp.com/courses/predicting-ctr-with-machine-learning-in-python)|✅|
173 | |[Datacamp: Intro to Financial Concepts using Python](https://www.datacamp.com/courses/intro-to-financial-concepts-using-python)|✅|
174 | |[Datacamp: Fraud Detection in Python](https://www.datacamp.com/courses/fraud-detection-in-python)|✅|
175 |
176 |
177 | ### Deep Learning
178 |
179 | |Resource|Progress|
180 | |---|---|
181 | |[Article: An overview of gradient descent optimization algorithms](https://www.ruder.io/optimizing-gradient-descent)|✅|
182 | |[Book: Make Your Own Neural Network](https://www.amazon.com/Make-Your-Own-Neural-Network/dp/1530826608)|✅|
183 | |[Fast.ai: Practical Deep Learning for Coder (Part 1)](https://course.fast.ai/)|✅|
184 | |[Fast.ai: Practical Deep Learning for Coder (Part 2)](https://course.fast.ai/Lessons/part2.html) `9, 13,14,17,18(48:10),19`|⬜|
185 | |[Datacamp: Convolutional Neural Networks for Image Processing](https://www.datacamp.com/courses/convolutional-neural-networks-for-image-processing)|✅|
186 | |[Karpathy: Neural Networks: Zero to Hero](https://github.com/karpathy/nn-zero-to-hero/)|✅|
187 | |[Article: Weight Initialization in Neural Networks: A Journey From the Basics to Kaiming](https://towardsdatascience.com/weight-initialization-in-neural-networks-a-journey-from-the-basics-to-kaiming-954fb9b47c79)|⬜|
188 | |[Article: Things that confused me about cross-entropy](https://chris-said.io/2020/12/26/two-things-that-confused-me-about-cross-entropy/)|✅|
189 |
190 | ### Natural Language Processing
191 |
192 | |Resource|Progress|
193 | |---|---|
194 | |[Book: Natural Language Processing with Transformers](https://transformersbook.com/)|✅|
195 | |[Stanford CS224U: Natural Language Understanding \| Spring 2019](https://www.youtube.com/playlist?list=PLoROMvodv4rObpMCir6rNNUlFAn56Js20)|✅|
196 | |[Stanford CS224N: Stanford CS224N: NLP with Deep Learning \| Winter 2019](https://www.youtube.com/playlist?list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z)|✅|
197 | |[CMU: Low-resource NLP Bootcamp 2020](https://www.youtube.com/playlist?list=PL8PYTP1V4I8A1CpCzURXAUa6H4HO7PF2c)|✅|
198 | |[CMU Multilingual NLP 2020](http://demo.clab.cs.cmu.edu/11737fa20/)|✅|
199 | |[Datacamp: Feature Engineering for NLP in Python](https://www.datacamp.com/courses/feature-engineering-for-nlp-in-python)|✅|
200 | |[Datacamp: Natural Language Processing Fundamentals in Python](https://www.datacamp.com/courses/natural-language-processing-fundamentals-in-python)|✅|
201 | |[Datacamp: Regular Expressions in Python](https://www.datacamp.com/courses/regular-expressions-in-python)|✅|
202 | |[Datacamp: RNN for Language Modeling](https://www.datacamp.com/courses/recurrent-neural-networks-for-language-modeling-in-python)|✅|
203 | |[Datacamp: Natural Language Generation in Python](https://www.datacamp.com/courses/natural-language-generation-in-python)|✅|
204 | |[Datacamp: Building Chatbots in Python](https://www.datacamp.com/courses/building-chatbots-in-python)|✅|
205 | |[Datacamp: Sentiment Analysis in Python](https://www.datacamp.com/courses/sentiment-analysis-in-python)|✅|
206 | |[Datacamp: Machine Translation in Python](https://www.datacamp.com/courses/machine-translation-in-python)|✅|
207 | |[Article: The Unreasonable Effectiveness of Collocations](https://opensourceconnections.com/blog/2019/05/16/unreasonable-effectiveness-of-collocations/)|⬜|
208 | |[Article: FuzzyWuzzy: Fuzzy String Matching in Python](https://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/#)|✅|
209 | |[Article: Transformers: Origins](https://mark-riedl.medium.com/transformers-origins-1db4bdfcb3d1)|⬜|
210 |
211 | ### Generative AI
212 |
213 |
214 | #### LLM Theory
215 |
216 | |Resource|Progress|
217 | |---|---|
218 | |[Book: Hands-On Large Language Models: Language Understanding and Generation](https://www.amazon.com/Hands-Large-Language-Models-Understanding/dp/1098150961)|✅|
219 | |[Book: AI Engineering: Building Applications with Foundation Models](https://www.amazon.com/AI-Engineering-Building-Applications-Foundation/dp/1098166302)|✅|
220 | |[Book: Designing Large Language Model Applications](https://www.oreilly.com/library/view/designing-large-language/9781098150495/)|⬜|
221 | |[Book: Large Language Models: A Deep Dive: Bridging Theory and Practice](https://www.amazon.com/Large-Language-Models-Bridging-Practice/dp/3031656466)|⬜|
222 | |[Book: A Little Bit of Reinforcement Learning from Human Feedback](https://rlhfbook.com/)|✅|
223 | |[Stanford CS236: Deep Generative Models](https://www.youtube.com/playlist?list=PLoROMvodv4rPOWA-omMM6STXaWW4FvJT8) `18 lectures`|1/18|
224 | |[Article: You could have designed state of the art Positional Encoding](https://fleetwood.dev/posts/you-could-have-designed-SOTA-positional-encoding)|⬜|
225 | |[Article: From Digits to Decisions: How Tokenization Impacts Arithmetic in LLMs](https://huggingface.co/spaces/huggingface/number-tokenization-blog)|⬜|
226 | |[Article: SolidGoldMagikarp (plus, prompt generation)](https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldmagikarp-plus-prompt-generation)|⬜|
227 | |[Article: Sampling for Text Generation](https://huyenchip.com/2024/01/16/sampling.html)|⬜|
228 | |[Article: Scaling test-time compute - a Hugging Face Space by HuggingFaceH4](https://huggingface.co/spaces/HuggingFaceH4/blogpost-scaling-test-time-compute)|⬜|
229 | |[Article: DeepSeek R1's recipe to replicate o1 and the future of reasoning LMs](https://www.interconnects.ai/p/deepseek-r1-recipe-for-o1)|✅|
230 | |[Article: The Illustrated DeepSeek-R1](https://newsletter.languagemodels.co/p/the-illustrated-deepseek-r1)|✅|
231 | |[Article: A Visual Guide to Reasoning LLMs](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-reasoning-llms)|⬜|
232 | |[DeepLearning.AI: Pretraining LLMs](https://www.deeplearning.ai/short-courses/pretraining-llms)|✅|
233 | |[DeepLearning.AI: Reinforcement Learning from Human Feedback](https://www.deeplearning.ai/short-courses/reinforcement-learning-from-human-feedback)|✅|
234 | |[Karpathy: Intro to Large Language Models](https://www.youtube.com/watch?v=zjkBMFhNj_g) `1hr`|✅|
235 | |[Karpathy: Let's build the GPT Tokenizer](https://www.youtube.com/watch?v=zduSFxRajkE) `2hr13m`|✅|
236 | |[Karpathy: Let's reproduce GPT-2 (124M)](https://www.youtube.com/watch?v=l8pRSuU81PU) `4hr1m`|✅|
237 | |[Youtube: A Hackers' Guide to Language Models](https://www.youtube.com/watch?v=jkrNMKz9pWU) `1hr30m`|✅|
238 | |[Karpathy: Deep Dive into LLMs like ChatGPT](https://www.youtube.com/watch?v=7xTGNNLPyMI) `3h31m`|✅|
239 | |[Youtube: 5 Years of GPTs with Finbarr Timbers](https://www.youtube.com/watch?v=YA0pzBYAV2Q&list=PLKlhhkvvU8-YxMP9hjEYJTJDCaGszrJIh&index=8&t=43s) `55m`|✅|
240 | |[Youtube: Stanford CS229 I Machine Learning I Building Large Language Models (LLMs)](https://www.youtube.com/watch?v=9vM4p9NN0Ts) `1h44m`|✅|
241 | |[Youtube: LLaMA explained: KV-Cache, Rotary Positional Embedding, RMS Norm, Grouped Query Attention, SwiGLU](https://www.youtube.com/watch?v=Mn_9W1nCFLo) `1h10m`|✅|
242 | |[Youtube: CMU Advanced NLP Fall 2024 (7): Prompting and Complex Reasoning](https://www.youtube.com/watch?v=1Faf1cTe3T8&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=2)|⬜|
243 | |[Youtube: CMU Advanced NLP Fall 2024 (6): Instruction Tuning](https://www.youtube.com/watch?v=iWcGS0gCL1E&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=3)|⬜|
244 | |[Youtube: CMU Advanced NLP Fall 2024 (12): Domain Specific Modeling: Code and Math](https://www.youtube.com/watch?v=qHNUVpKO2dc&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=4)|⬜|
245 | |[Youtube: CMU Advanced NLP Fall 2024 (15): Tool Use and LLM Agent Basics](https://www.youtube.com/watch?v=a3SjRsqV9ZA&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=16)|⬜|
246 | |[Youtube: CMU Advanced NLP Fall 2024 (14): Ensembling and Mixture of Experts](https://www.youtube.com/watch?v=E4Rg4qTw4xw&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=15)|✅|
247 | |[Youtube: A little guide to building Large Language Models in 2024](https://www.youtube.com/watch?v=2-SPH9hIKT8) `1h15m`|✅|
248 | |[Youtube: How to approach post-training for AI applications](https://www.youtube.com/watch?v=grpc-Wyy-Zg) `22m`|✅|
249 | |[Youtube: Speculations on Test-Time Scaling (o1) `47m`](https://www.youtube.com/watch?v=6PEJ96k1kiw)|✅|
250 | |[Youtube: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://youtu.be/XMnxKGVnEUc) `1h19m`|✅|
251 | |[Youtube: How DeepSeek Changes the LLM Story](https://www.youtube.com/watch?v=0eMzc-WnBfQ)|✅|
252 | |[Youtube: MIT EI seminar, Hyung Won Chung from OpenAI. "Don't teach. Incentivize."](https://www.youtube.com/watch?v=kYWUEV_e2ss) `35m`|✅|
253 | |[Youtube: How I use LLMs](https://youtu.be/EWvNQjAaOHw) `2h7m`|✅|
254 | |[Youtube: Simple Diffusion Language Models](https://youtu.be/WjAUX23vgfg)|✅|
255 | |[Youtube: Introduction to Reasoning LLMs](https://www.youtube.com/watch?v=AZhUhGsgz4s) `1hr`|⬜|
256 | |[Article: Mamba Explained](https://thegradient.pub/mamba-explained/)|⬜|
257 | |[Article: A Visual Guide to Mamba and State Space Models](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-mamba-and-state)|⬜|
258 |
259 | #### Multi-modality
260 |
261 | |Resource|Progress|
262 | |---|---|
263 | |[Article: Understanding Multimodal LLMs](https://magazine.sebastianraschka.com/p/understanding-multimodal-llms)|⬜|
264 | |[Article: GPT-4 Vision Alternatives](https://blog.roboflow.com/gpt-4-vision-alternatives/)|⬜|
265 | |[Article: Computer-Using Agent](https://openai.com/index/computer-using-agent/)|✅|
266 | |[Youtube: AI Visions Live \| Merve Noyan \| Open-source Multimodality](https://www.youtube.com/watch?v=_TlhKHTgWjY) `54m`|✅|
267 | |[DeepLearning.AI: How Diffusion Models Work](https://www.deeplearning.ai/short-courses/how-diffusion-models-work/)|✅|
268 | |[DeepLearning.AI: Prompt Engineering for Vision Models](https://www.deeplearning.ai/short-courses/prompt-engineering-for-vision-models/)|⬜|
269 | |[DeepLearning.AI: Building Multimodal Search and RAG](https://www.deeplearning.ai/short-courses/building-multimodal-search-and-rag/)|✅|
270 | |[Pinecone: Embedding Methods for Image Search](https://www.pinecone.io/learn/series/image-search/)|0/8|
271 | |[Youtube: Lesson 9A 2022 - Stable Diffusion deep dive](https://youtu.be/0_BBRNYInx8)|✅|
272 | |[Article: Diffusion models are autoencoders](https://sander.ai/2022/01/31/diffusion.html)|✅|
273 | |[Article: Diffusion Language Models](https://sander.ai/2023/01/09/diffusion-language.html)|✅|
274 | |[Article: Guidance: a cheat code for diffusion models](https://sander.ai/2022/05/26/guidance.html)|✅|
275 | |[Article: Perspectives on diffusion](https://sander.ai/2023/07/20/perspectives.html)|✅|
276 | |[Article: The geometry of diffusion guidance](https://sander.ai/2023/08/28/geometry.html)|✅|
277 | |[Article: Diffusion is spectral autoregression](https://sander.ai/2024/09/02/spectral-autoregression.html)|✅|
278 | |[Article: Generative modelling in latent space](https://sander.ai/2025/04/15/latents.html)|✅|
279 | |[Youtube: Sander Dieleman - Generative modelling through iterative refinement](https://www.youtube.com/watch?v=9BHQvQlsVdE)|✅|
280 | |[Speech AI models: an introduction](https://thomwolf.io/blog/speech-ai.html)|⬜|
281 |
282 |
283 | #### Information Retrieval / RAG
284 |
285 | | Resource | Progress |
286 | | ---------------------------------------------------------------------------------------------------------------------------------------------- | -------- |
287 | | [Article: Pretrained Transformer Language Models for Search - part 1](https://blog.vespa.ai/pretrained-transformer-language-models-for-search-part-1/#) | ✅ |
288 | | [Article: Pretrained Transformer Language Models for Search - part 2](https://blog.vespa.ai/pretrained-transformer-language-models-for-search-part-2/) | ✅ |
289 | | [Article: Pretrained Transformer Language Models for Search - part 3](https://blog.vespa.ai/pretrained-transformer-language-models-for-search-part-3) | ✅ |
290 | | [Article: Pretrained Transformer Language Models for Search - part 4](https://blog.vespa.ai/pretrained-transformer-language-models-for-search-part-4) | ✅ |
291 | |[Article: How not to use BERT for Document Ranking](https://bergum.medium.com/how-not-to-use-bert-for-search-ranking-4586716428d9)|✅|
292 | | [Article: Understanding LanceDB's IVF-PQ index](https://lancedb.github.io/lancedb/concepts/index_ivfpq/) | ✅ |
293 | | [Article: A little pooling goes a long way for multi-vector representations](https://www.answer.ai/posts/colbert-pooling.html) | ✅ |
294 | |[Article: Levels of Complexity: RAG Applications](https://jxnl.github.io/blog/writing/2024/02/28/levels-of-complexity-rag-applications/)|✅|
295 | |[Article: Systematically Improving Your RAG](https://jxnl.github.io/blog/writing/2024/05/22/systematically-improving-your-rag/)|✅|
296 | |[Article: Stop using LGTM@Few as a metric (Better RAG)](https://jxnl.github.io/blog/writing/2024/02/05/when-to-lgtm-at-k/)|✅|
297 | |[Article: Low-Hanging Fruit for RAG Search](https://jxnl.github.io/blog/writing/2024/05/11/low-hanging-fruit-for-rag-search/)|✅|
298 | |[Article: What AI Engineers Should Know about Search](https://softwaredoug.com/blog/2024/06/25/what-ai-engineers-need-to-know-search)|✅|
299 | |[Article: Evaluating Chunking Strategies for Retrieval](https://research.trychroma.com/evaluating-chunking)|✅|
300 | |[Article: Sentence Embeddings. Introduction to Sentence Embeddings](https://osanseviero.github.io/hackerllama/blog/posts/sentence_embeddings/)|✅|
301 | |[Article: LambdaMART in Depth](https://softwaredoug.com/blog/2022/01/17/lambdamart-in-depth)|⬜|
302 | |[Article: Guided Generation with Outlines](https://medium.com/canoe-intelligence-technology/guided-generation-with-outlines-c09a0c2ce9eb)|✅|
303 | |[Article: RAG tricks from the trenches](https://duarteocarmo.com/blog/rag-tricks-from-the-trenches)|⬜|
304 | |[Article: Retrieval 101](https://isaacflath.com/blog/blog_post?fpath=posts%2F2025-03-17-Retrieval101.ipynb)|⬜|
305 | | [Course: Fullstack Retrieval](https://community.fullstackretrieval.com/) | ⬜ |
306 | |[DeepLearning.AI: Building and Evaluating Advanced RAG Applications](https://www.deeplearning.ai/short-courses/building-evaluating-advanced-rag/)|✅|
307 | |[DeepLearning.AI: Vector Databases: from Embeddings to Applications](https://www.deeplearning.ai/short-courses/vector-databases-embeddings-applications/)|✅|
308 | |[DeepLearning.AI: Advanced Retrieval for AI with Chroma](https://www.deeplearning.ai/short-courses/advanced-retrieval-for-ai/)|✅|
309 | |[DeepLearning.AI: Prompt Compression and Query Optimization](https://www.deeplearning.ai/short-courses/prompt-compression-and-query-optimization/)|✅|
310 | |[DeepLearning.AI: Large Language Models with Semantic Search](https://www.deeplearning.ai/short-courses/large-language-models-semantic-search) `1hr`|✅|
311 | |[DeepLearning.AI: Building Applications with Vector Databases](https://www.deeplearning.ai/short-courses/building-applications-vector-databases/)|✅|
312 | |[DeepLearning.AI: Knowledge Graphs for RAG](https://www.deeplearning.ai/short-courses/knowledge-graphs-rag/)|⬜|
313 | |[DeepLearning.AI: Preprocessing Unstructured Data for LLM Applications](https://www.deeplearning.ai/short-courses/preprocessing-unstructured-data-for-llm-applications/)|⬜|
314 | |[DeepLearning.AI: Embedding Models: From Architecture to Implementation](https://www.deeplearning.ai/short-courses/embedding-models-from-architecture-to-implementation)|✅|
315 | |[DeepLearning.AI: Retrieval Optimization - From Tokenization to Vector Quantization](https://www.deeplearning.ai/short-courses/retrieval-optimization-from-tokenization-to-vector-quantization/)|✅|
316 | |[Pinecone: Vector Databases in Production for Busy Engineers](https://www.pinecone.io/learn/series/vector-databases-in-production-for-busy-engineers/)|✅|
317 | |[Pinecone: Retrieval Augmented Generation](https://www.pinecone.io/learn/series/rag/)|✅|
318 | |[Pinecone: Faiss: The Missing Manual](https://www.pinecone.io/learn/series/faiss/)|✅|
319 | |[Pinecone: Natural Language Processing for Semantic Search](https://www.pinecone.io/learn/series/nlp/)|0/13|
320 | |[Youtube: Systematically improving RAG applications](https://youtu.be/RrDBV6odPKo?list=PLgIaq8VgndJvXkDSeReTl2u4rQMShkZ6V)|✅|
321 | |[Youtube: Back to Basics for RAG w/ Jo Bergum](https://www.youtube.com/watch?v=nc0BupOkrhI&list=PLgIaq8VgndJvXkDSeReTl2u4rQMShkZ6V&index=2)|✅|
322 | |[Youtube: Beyond the Basics of Retrieval for Augmenting Generation (w/ Ben Clavié)](https://www.youtube.com/watch?v=0nA5QG3087g&t=1287s)|✅|
323 | |[Youtube: RAG From Scratch](https://www.youtube.com/playlist?list=PLfaIDFEXuae2LXbO1_PKyVJiQ23ZztA0x) `14/14`|✅|
324 | |[Youtube: CMU Advanced NLP Fall 2024 (10): Retrieval and RAG](https://www.youtube.com/watch?v=KfQaYk4k9eM&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=6) `1h17m`|✅|
325 | |[Guidance: Token Healing](https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/token_healing.ipynb)|⬜|
326 | |[Youtube: What You See Is What You Search: Vision Language Models for PDF Retrieval [Jo Bergum]](https://youtu.be/qrbQUU4TrLM)|✅|
327 |
328 | #### Agentic Pattern
329 |
330 | |Resource|Progress|
331 | |---|---|
332 | |[Article: Tool Invocation - Demonstrating the Marvel of GPT's Flexibility](https://blog.jnbrymn.com/2024/01/30/the-marvel-of-GPT-generality.html)|⬜|
333 | |[Article: Introducing smolagents, a simple library to build agents](https://huggingface.co/blog/smolagents)|⬜|
334 | |[Article: What Problem Does The Model Context Protocol Solve?](https://www.aihero.dev/what-problem-does-model-context-protocol-solve)|✅|
335 | |[Anthropic: Building effective agents](https://www.anthropic.com/research/building-effective-agents)|✅|
336 | |[Anthropic: Building Effective Agents Cookbook](https://github.com/anthropics/anthropic-cookbook/tree/main/patterns/agents)|✅|
337 | |[OpenAI: Assistants & Agents Build Hour](https://vimeo.com/showcase/11333741/video/990334325)|✅|
338 | |[OpenAI: Function Calling Build Hour](https://vimeo.com/showcase/11333741/video/952127114)|✅|
339 | |[DeepLearning.AI: Functions, Tools and Agents with LangChain](https://www.deeplearning.ai/short-courses/functions-tools-agents-langchain/)|⬜|
340 | |[DeepLearning.AI: Building Agentic RAG with LlamaIndex](https://www.deeplearning.ai/short-courses/building-agentic-rag-with-llamaindex/)|✅|
341 | |[DeepLearning.AI: Multi AI Agent Systems with crewAI](https://www.deeplearning.ai/short-courses/multi-ai-agent-systems-with-crewai/)|✅|
342 | |[DeepLearning.AI: Building Towards Computer Use with Anthropic](https://www.deeplearning.ai/short-courses/building-towards-computer-use-with-anthropic/)|✅|
343 | |[DeepLearning.AI: Practical Multi AI Agents and Advanced Use Cases with crewAI](https://www.deeplearning.ai/short-courses/practical-multi-ai-agents-and-advanced-use-cases-with-crewai/)|⬜|
344 | |[DeepLearning.AI: LLMs as Operating Systems: Agent Memory](https://www.deeplearning.ai/short-courses/llms-as-operating-systems-agent-memory/)|✅|
345 | |[DeepLearning.AI: Serverless Agentic Workflows with Amazon Bedrock](https://www.deeplearning.ai/short-courses/serverless-agentic-workflows-with-amazon-bedrock/)|⬜|
346 | |[DeepLearning.AI: AI Agentic Design Patterns with AutoGen](https://www.deeplearning.ai/short-courses/ai-agentic-design-patterns-with-autogen/)|⬜|
347 | |[DeepLearning.AI: AI Agents in LangGraph](https://www.deeplearning.ai/short-courses/ai-agents-in-langgraph/)|⬜|
348 | |[DeepLearning.AI: Building Your Own Database Agent](https://www.deeplearning.ai/short-courses/building-your-own-database-agent/)|⬜|
349 | |[DeepLearning.AI: Function-Calling and Data Extraction with LLMs](https://www.deeplearning.ai/short-courses/function-calling-and-data-extraction-with-llms/) `59m`|✅|
350 | |[DeepLearning.AI: Evaluating AI Agents](https://www.deeplearning.ai/short-courses/evaluating-ai-agents/) `2h16m`|✅|
351 | |[DeepLearning.AI: Build Apps with Windsurf’s AI Coding Agents](https://www.deeplearning.ai/short-courses/build-apps-with-windsurfs-ai-coding-agents/) `1h10m`|✅|
352 | |[DeepLearning.AI: Building AI Browser Agents](https://www.deeplearning.ai/short-courses/building-ai-browser-agents)|⬜|
353 | |[Huggingface: Agents Course](https://huggingface.co/learn/agents-course/unit1/messages-and-special-tokens#base-models-vs-instruct-models)|Unit 1|
354 | |[Youtube: How to Evaluate Agents: Galileo’s Agentic Evaluations in Action](https://www.youtube.com/watch?v=QvStk5G8BZw)|✅|
355 | |[Youtube: Agent Response \| LangSmith Evaluation - Part 24](https://youtu.be/NbQKDfSw3gM?list=PLfaIDFEXuae0um8Fj0V4dHG37fGFU8Q5S)|✅|
356 | |[Youtube: Single Step \| LangSmith Evaluation - Part 25](https://youtu.be/AVPflFmRkd4?list=PLfaIDFEXuae0um8Fj0V4dHG37fGFU8Q5S)|✅|
357 | |[Youtube: Agent Trajectory \| LangSmith Evaluation - Part 26](https://youtu.be/pvlT056DAHs?list=PLfaIDFEXuae0um8Fj0V4dHG37fGFU8Q5S)|✅|
358 | |[Youtube: Evaluating Agents and Assistants: The AI Conference](https://www.youtube.com/watch?v=6uXWhmDRcMc)|✅|
359 | |[Youtube: How to Build, Evaluate, and Iterate on LLM Agents](https://youtu.be/0pnEUAwoDP0)|✅|
360 |
361 |
362 | #### Prompt Engineering
363 |
364 | |Resource|Progress|
365 | |---|---|
366 | |[Article: OpenAI Prompt Engineering](https://platform.openai.com/docs/guides/prompt-engineering)|⬜|
367 | |[Article: Prompting Fundamentals and How to Apply them Effectively](https://eugeneyan.com/writing/prompting/)|✅|
368 | |[Article: How I came in first on ARC-AGI-Pub using Sonnet 3.5 with Evolutionary Test-time Compute](https://params.com/@jeremy-berman/arc-agi)|✅|
369 | |[Anthropic Courses](https://github.com/anthropics/courses)|⬜|
370 | |[Anthropic: The Claude in Amazon Bedrock Course](https://www.anthropic.com/aws-reinvent-2024/course)|⬜|
371 | |[Article: Prompt Engineering(Liliang Weng)](https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/)|✅|
372 | |[Article: Prompt Engineering 201: Advanced methods and toolkits](https://amatria.in/blog/prompt201)|✅|
373 | |[Article: Optimizing LLMs for accuracy](https://platform.openai.com/docs/guides/optimizing-llm-accuracy)|✅|
374 | |[Article: Primers • Prompt Engineering](https://aman.ai/primers/ai/prompt-engineering/)|⬜|
375 | |[Article: Anyscale Endpoints: JSON Mode and Function calling Features](https://www.anyscale.com/blog/anyscale-endpoints-json-mode-and-function-calling-features)|⬜|
376 | |[Article: Guided text generation with Large Language Models](https://medium.com/productizing-language-models/guided-text-generation-with-large-language-models-d88fc3dcf4c)|⬜|
377 | |[Book: Prompt Engineering for LLMs](https://www.oreilly.com/library/view/prompt-engineering-for/9781098156145/)|⬜|
378 | |[DeepLearning.AI: Reasoning with o1](https://www.deeplearning.ai/short-courses/reasoning-with-o1/)|✅|
379 | |[OpenAI: Reasoning with o1 Build Hour](https://vimeo.com/showcase/11333741/video/1018737829)|✅|
380 | |[DeepLearning.AI: ChatGPT Prompt Engineering for Developers](https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/)|⬜|
381 | |[DeepLearning.AI: Prompt Engineering with Llama 2 & 3](https://www.deeplearning.ai/short-courses/prompt-engineering-with-llama-2/)|⬜|
382 | |[Wandb: LLM Engineering: Structured Outputs](https://www.wandb.courses/courses/steering-language-models)|⬜|
383 | |[Series: Prompt injection](https://simonwillison.net/series/prompt-injection/)|⬜|
384 | |[Youtube: Prompt Engineering Overview](https://www.youtube.com/watch?v=dOxUroR57xs) `1hr4m`|✅|
385 | |[Youtube: Prompt Engineering Workshop](https://youtu.be/htBTho6oEJA) `1h`|✅|
386 |
387 | #### Quantization
388 | |Resource|Progress|
389 | |---|---|
390 | |[Article: Quantization Fundamentals with Hugging Face](https://www.deeplearning.ai/short-courses/quantization-fundamentals-with-hugging-face/)|✅|
391 | |[DeepLearning.AI: Quantization in Depth](https://www.deeplearning.ai/short-courses/quantization-in-depth/)|⬜|
392 | |[DeepLearning.AI: Introduction to On-Device AI](https://www.deeplearning.ai/short-courses/introduction-to-on-device-ai/)|⬜|
393 | |[Article: A Visual Guide to Quantization](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-quantization)|⬜|
394 | |[Article: QLoRA and 4-bit Quantization](https://mccormickml.com/2024/09/14/qlora-and-4bit-quantization/)|⬜|
395 | |[Article: Understanding AI/LLM Quantisation Through Interactive Visualisations](https://smcleod.net/2024/07/understanding-ai/llm-quantisation-through-interactive-visualisations/)|⬜|
396 | |[Youtube: CMU Advanced NLP Fall 2024 (11): Distillation, Quantization, and Pruning](https://www.youtube.com/watch?v=DvVGkj4zhVU&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=5)|⬜|
397 | |[Article: LLM.int8() and Emergent Features](https://timdettmers.com/2022/08/17/llm-int8-and-emergent-features/)|⬜|
398 |
399 | #### Distributed Training
400 |
401 | |Resource|Progress|
402 | |---|---|
403 | |[Youtube: Slaying OOMs with PyTorch FSDP and torchao](https://youtu.be/UvRl4ansfCg) `49m`|✅|
404 | |[Youtube: Distributed Training with PyTorch: complete tutorial with cloud infrastructure and code](https://youtu.be/toUSzwR0EV8) `1h12m`|✅|
405 | |[Youtube: How DDP works \|\| Distributed Data Parallel ](https://youtu.be/bwNtfxEDjGA)|✅|
406 | |[Youtube: FSDP Explained](https://youtu.be/6pVn6khIgiI)|✅|
407 | |[Youtube: Lecture 48: The Ultra Scale Playbook](https://youtu.be/1E8GDR8QXKw) `3h3m`|`44:24`|
408 | |[Youtube: Invited Talk: PyTorch Distributed (DDP, RPC) - By Facebook Research Scientist Shen Li](https://youtu.be/3XUG7cjte2U)|✅|
409 | |[Youtube: Unit 9 \| Techniques for Speeding Up Model Training](https://www.youtube.com/playlist?list=PLaMu-SDt_RB403GN5DU7NYVoVmO5Vsgkh)|✅|
410 | |[Article: A Short Guide to PyTorch DDP](https://blog.hpc.qmul.ac.uk/pytorch-ddp/)|✅|
411 | |[Article: Scaling Deep Learning with PyTorch: Multi-Node and Multi-GPU Training Explained (with Code)](https://medium.com/@ashraf.kasem.94.0/scaling-deep-learning-with-pytorch-multi-node-and-multi-gpu-training-explained-with-code-ece8f03ea59b)|✅|
412 | |[Article: Accelerating PyTorch Model Training](https://magazine.sebastianraschka.com/p/accelerating-pytorch-model-training)|✅|
413 | |[Article: Meet Horovod: Uber’s Open Source Distributed Deep Learning Framework for TensorFlow](https://www.uber.com/blog/horovod/)|✅|
414 | |[Article: Distributed data parallel training in Pytorch](https://yangkky.github.io/2019/07/08/distributed-pytorch-tutorial.html)|✅|
415 | |[Article: Training on Multiple GPUs](https://d2l.ai/chapter_computational-performance/multiple-gpus.html)|✅|
416 |
417 |
418 | #### Parallel Computing
419 |
420 | |Resource|Progress|
421 | |---|---|
422 | |[Udacity: Intro to Parallel Programming](https://www.youtube.com/playlist?list=PLAwxTw4SYaPnFKojVQrmyOGFCqHTxfdv2) `458 videos`|299/458|
423 | |[Book: Programming Massively Parallel Processors: A Hands-on Approach](https://www.amazon.com/Programming-Massively-Parallel-Processors-Hands/dp/0124159923)|Ch. 2|
424 | |[Youtube: GPU Puzzles: Let's Play](https://youtu.be/K4T-YwsOxrM)|⬜|
425 |
426 | #### Inference Optimization
427 |
428 | |Resource|Progress|
429 | |---|---|
430 | |[Article: How to make LLMs go fast](https://vgel.me/posts/faster-inference/)|✅|
431 | |[Article: In the Fast Lane! Speculative Decoding - 10x Larger Model, No Extra Cost](https://docs.titanml.co/blog/speculative-decoding-unleashed/)|⬜|
432 | |[Article: Accelerating Generative AI with PyTorch II: GPT, Fast](https://pytorch.org/blog/accelerating-generative-ai-2/)|⬜|
433 | |[Article: Harmonizing Multi-GPUs: Efficient Scaling of LLM Inference](https://docs.titanml.co/blog/multi-gpu/)|⬜|
434 | |[Article: Multi-Query Attention is All You Need](https://fireworks.ai/blog/multi-query-attention-is-all-you-need)|⬜|
435 | |[Article: Transformers Inference Optimization Toolset](https://astralord.github.io/posts/transformer-inference-optimization-toolset/)|⬜|
436 | |[DeepLearning.AI: Efficiently Serving LLMs](https://www.deeplearning.ai/short-courses/efficiently-serving-llms/)|✅|
437 | |[Article: LLM Inference Series: 3. KV caching explained](https://medium.com/@plienhar/llm-inference-series-3-kv-caching-unveiled-048152e461c8)|⬜|
438 | |[Article: LLM Inference Series: 4. KV caching, a deeper look](https://medium.com/@plienhar/llm-inference-series-4-kv-caching-a-deeper-look-4ba9a77746c8)|⬜|
439 | |[Article: LLM Inference Series: 5. Dissecting model performance](https://medium.com/@plienhar/llm-inference-series-5-dissecting-model-performance-6144aa93168f)|⬜|
440 | |[Article: Transformer Inference Arithmetic](https://kipp.ly/transformer-inference-arithmetic/)|⬜|
441 | |[Article: Optimizing AI Inference at Character.AI](https://research.character.ai/optimizing-inference/)|⬜|
442 | |[Article: Optimizing AI Inference at Character.AI (Part Deux)](https://research.character.ai/optimizing-ai-inference-at-character-ai-part-deux/)|⬜|
443 | |[Article: llama.cpp guide - Running LLMs locally, on any hardware, from scratch](https://blog.steelph0enix.dev/posts/llama-cpp-guide/)|✅|
444 | |[Youtube: SBTB 2023: Charles Frye, Parallel Processors: Past & Future Connections Between LLMs and OS Kernels](https://www.youtube.com/watch?v=VxFtHqlMv8c)|✅|
445 | |[Youtube: Deploying Fine-Tuned Models](https://youtu.be/GzEcyBykkdo) `2h28m`|✅|
446 | |[Article: Compiling ML models to C for fun](https://bernsteinbear.com/blog/compiling-ml-models/)|⬜|
447 | |[Article: How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklog](https://siboehm.com/articles/22/CUDA-MMM)|⬜|
448 |
449 | #### Evals and Guardrails
450 |
451 | |Resource|Progress|
452 | |---|---|
453 | |[Article: Your AI Product Needs Evals](https://hamel.dev/blog/posts/evals)|✅|
454 | |[Article: Task-Specific LLM Evals that Do & Don't Work](https://eugeneyan.com/writing/evals/)|✅|
455 | |[Article: Evaluation & Hallucination Detection for Abstractive Summaries](https://eugeneyan.com/writing/abstractive/)|✅|
456 | |[Article: LLM-as-a-Judge vs Human Evaluation](https://www.galileo.ai/blog/llm-as-a-judge-vs-human-evaluation)|⬜|
457 | |[DeepLearning.AI: Automated Testing for LLMOps](https://www.deeplearning.ai/short-courses/automated-testing-llmops/)|✅|
458 | |[DeepLearning.AI: Red Teaming LLM Applications](https://www.deeplearning.ai/short-courses/red-teaming-llm-applications/)|✅|
459 | |[DeepLearning.AI: Evaluating and Debugging Generative AI Models Using Weights and Biases](https://www.deeplearning.ai/short-courses/evaluating-debugging-generative-ai/)|⬜|
460 | |[DeepLearning.AI: Quality and Safety for LLM Applications](https://www.deeplearning.ai/short-courses/quality-safety-llm-applications/)|⬜|
461 | |[OpenAI: Evals Build Hour](https://vimeo.com/showcase/11333741/video/1023317525)|✅|
462 | |[Youtube: Instrumenting & Evaluating LLMs](https://youtu.be/SnbGD677_u0) `2hr33m`|✅|
463 | |[Youtube: LLM Eval For Text2SQL](https://youtu.be/UGmenkjGXqM?list=PLgIaq8VgndJvt-HKMHPXehyJNNXQsAVHD) `51m`|✅|
464 | |[Youtube: A Deep Dive on LLM Evaluation](https://youtu.be/IsZVCnViwhk?list=PLgIaq8VgndJvt-HKMHPXehyJNNXQsAVHD) `49m`|✅|
465 |
466 | ### Finetuning and Distillation
467 |
468 | |Resource|Progress|
469 | |---|---|
470 | |[Article: Tokenization Gotchas](https://hamel.dev/notes/llm/finetuning/tokenizer_gotchas.html)|⬜|
471 | |[Article: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)](https://magazine.sebastianraschka.com/p/practical-tips-for-finetuning-llms)|⬜|
472 | |[OpenAI: GPT-4o mini Fine-Tuning Build Hour](https://vimeo.com/showcase/11333741/video/995989828)|✅|
473 | |[OpenAI: Distillation Build Hour](https://vimeo.com/showcase/11333741/video/1029408095)|✅|
474 | |[Article: How to Generate and Use Synthetic Data for Finetuning](https://eugeneyan.com/writing/synthetic/)|✅|
475 | |[DeepLearning.AI: Finetuning Large Language Models](https://www.deeplearning.ai/short-courses/finetuning-large-language-models/)|✅|
476 | |[Youtube: Fine-Tuning with Axolotl](https://youtu.be/mmsa4wDsiy0?list=PLgIaq8VgndJtZ_G6gxyuhHGLUy9zXV9JC) `2h10m`|✅|
477 | |[Youtube: Creating, Curating, and Cleaning Data for LLMs](https://youtu.be/HEGaei7k0zE?list=PLgIaq8VgndJtZ_G6gxyuhHGLUy9zXV9JC) `54m`|✅|
478 | |[Youtube: Best Practices For Fine Tuning Mistral](https://youtu.be/Z_oWzTuljss?list=PLgIaq8VgndJtZ_G6gxyuhHGLUy9zXV9JC) `23m`|✅|
479 | |[Youtube: Fine Tuning OpenAI Models - Best Practices](https://youtu.be/Q0GSZD0Na1s?list=PLgIaq8VgndJtZ_G6gxyuhHGLUy9zXV9JC)|✅|
480 | |[Youtube: When and Why to Fine Tune an LLM](https://youtu.be/cPn0nHFsvFg) `1h56m`|✅|
481 | |[Youtube: Napkin Math For Fine Tuning Pt. 1 w/Johno Whitaker](https://youtu.be/-2ebSQROew4)|✅|
482 | |[Youtube: Napkin Math For Fine Tuning Pt. 2 w/Johno Whitaker](https://youtu.be/u2fJ6K8FjS8)|✅|
483 | |[Youtube: Fine Tuning LLMs for Function Calling w/Pawel Garback](https://youtu.be/SEZ7j31u67A) `1h32m`|✅|
484 | |[Youtube: From Prompt to Model: Fine-tuning when you've already deployed LLMs in prod w/Kyle Corbitt](https://youtu.be/4EPZZkVrXC4) `32m`|✅|
485 | |[Youtube: Why Fine Tuning is Dead w/Emmanuel Ameisen](https://youtu.be/h1c_jmk97Ss) `50m`|✅|
486 | |[Benchmarking QLoRA+FSDP](https://github.com/AnswerDotAI/fsdp_qlora/blob/main/benchmarks_03_2024.md)|⬜|
487 |
488 | #### LLM System Design
489 |
490 | |Resource|Progress|
491 | |---|---|
492 | |[Article: What We’ve Learned From A Year of Building with LLMs](https://applied-llms.org/)|⬜|
493 | |[Article: Data Flywheels for LLM Applications](https://www.sh-reya.com/blog/ai-engineering-flywheel/)|⬜|
494 | |[Article: LLM From the Trenches: 10 Lessons Learned Operationalizing Models at GoDaddy](https://www.godaddy.com/resources/news/llm-from-the-trenches-10-lessons-learned-operationalizing-models-at-godaddy#h-3-prompts-aren-t-portable-across-models)|✅|
495 | |[Article: Emerging UX Patterns for Generative AI Apps & Copilots](https://www.tidepool.so/blog/emerging-ux-patterns-for-generative-ai-apps-copilots)|✅|
496 | |[Article: The Novice's LLM Training Guide](https://rentry.co/llm-training)|⬜|
497 | |[Article: Pushing ChatGPT's Structured Data Support To Its Limits](https://minimaxir.com/2023/12/chatgpt-structured-data/)|✅|
498 | |[Article: GPTed: using GPT-3 for semantic prose-checking](https://vgel.me/posts/gpted-launch/)|✅|
499 | |[Article: Don't worry about LLMs](https://vickiboykis.com/2024/05/20/dont-worry-about-llms/)|⬜|
500 | |[Article: Things we learned about LLMs in 2024](https://simonwillison.net/2024/Dec/31/llms-in-2024/)|⬜|
501 | |[Article: Data acquisition strategies for AI-first start-ups](https://press.airstreet.com/p/data-acquisition-strategies-for-ai?utm_source=substack&utm_medium=email)|⬜|
502 | |[DeepLearning.AI: Building Systems with the ChatGPT API](https://www.deeplearning.ai/short-courses/building-systems-with-chatgpt/)|⬜|
503 | |[DeepLearning.AI: Building Generative AI Applications with Gradio](https://www.deeplearning.ai/short-courses/building-generative-ai-applications-with-gradio/)|✅|
504 | |[DeepLearning.AI: Open Source Models with Hugging Face](https://www.deeplearning.ai/short-courses/open-source-models-hugging-face/)|⬜|
505 | |[DeepLearning.AI: Getting Started with Mistral](https://www.deeplearning.ai/short-courses/getting-started-with-mistral/)|⬜|
506 | |[LLMOps: Building with LLMs](https://www.comet.com/site/llm-course/)|⬜|
507 | |[LLM Bootcamp - Spring 2023](https://fullstackdeeplearning.com/llm-bootcamp/spring-2023/)|✅|
508 | |[Youtube: A Survey of Techniques for Maximizing LLM Performance](https://www.youtube.com/watch?v=ahnGLM-RC1Y)|✅|
509 | |[Youtube: Building Blocks for LLM Systems & Products: Eugene Yan](https://www.youtube.com/watch?v=LzeC1AQ-U5o)|✅|
510 | |[Youtube: Building LLM Applications](https://www.youtube.com/playlist?list=PLgIaq8VgndJtrxcelEdnXbvh9fXMHeAps)|0/8|
511 | |[Article: Emerging Architectures for LLM Applications](https://a16z.com/emerging-architectures-for-llm-applications/)|✅|
512 | |[Article: Patterns for Building LLM-based Systems & Products](https://eugeneyan.com/writing/llm-patterns/)|✅|
513 | |[DeepLearning.AI: LLMOps](https://www.deeplearning.ai/short-courses/llmops/)|⬜|
514 | |[DeepLearning.AI: Serverless LLM apps with Amazon Bedrock](https://www.deeplearning.ai/short-courses/serverless-llm-apps-amazon-bedrock/)|⬜|
515 | |[Youtube: Getting the Most Out of Your LLM Experiments](https://youtu.be/IfcDvtl6Z1Y) `48m`|✅|
516 |
517 |
518 | ## Technical Skills (Libraries/Frameworks/Tools)
519 |
520 | ### AWS
521 |
522 |
523 | |Resource|Progress|
524 | |---|---|
525 | |[Udemy: AWS Certified Developer - Associate 2018](https://www.udemy.com/aws-certified-developer-associate/)|✅|
526 |
527 | ### CSS
528 |
529 | |Resource|Progress|
530 | |---|---|
531 | |[Pluralsight: CSS Positioning](https://www.pluralsight.com/courses/css-positioning-1834)|✅|
532 | |[Pluralsight: Introduction to CSS](https://www.pluralsight.com/courses/css-intro)|✅|
533 | |[Pluralsight: CSS: Specificity, the Box Model, and Best Practices](https://app.pluralsight.com/interactive-courses/detail/c580b092-d94a-4ed8-8d2a-2f4d0b76f99f)|✅|
534 | |[Pluralsight: CSS: Using Flexbox for Layout](https://app.pluralsight.com/interactive-courses/detail/a089d0a5-4a4c-4c4e-b883-c1bc64009619)|✅|
535 | |[Code School: Blasting Off with Bootstrap](https://www.pluralsight.com/courses/code-school-blasting-off-with-bootstrap)|✅|
536 | |[Pluralsight: UX Fundamentals](https://www.pluralsight.com/courses/ux-fundamentals-2426)|✅|
537 | |[Codecademy: Learn SASS](https://www.codecademy.com/learn/learn-sass)|✅|
538 | |[CSS for Javascript Developers](https://css-for-js.dev/)|✅|
539 | |[Article: Create an illustration in Figma design](https://help.figma.com/hc/en-us/articles/13543867954711-Create-an-illustration-in-Figma-design)|✅|
540 | |[Book: Refactoring UI](https://refactoringui.com/book/)|⬜|
541 | |[Youtube: How to Make Your Website Not Ugly: Basic UX for Programmers](https://www.youtube.com/watch?v=Jf0cjocP8Wk) `48m`|⬜|
542 |
543 |
544 |
545 | ### Django
546 |
547 | |Resource|Progress|
548 | |---|---|
549 | |[Article: Django, HTMX and Alpine.js: Modern websites, JavaScript optional](https://www.saaspegasus.com/guides/modern-javascript-for-django-developers/htmx-alpine/)|✅|
550 |
551 | ### HTML
552 |
553 | |Resource|Progress|
554 | |---|---|
555 | |[Codecademy: Learn HTML](https://www.codecademy.com/learn/learn-html)|✅|
556 | |[Codecademy: Make a website](https://www.codecademy.com/en/courses/make-a-website)|✅|
557 | |[Article: Alternative Text](https://webaim.org/techniques/alttext/)|⬜|
558 |
559 | ### Langchain
560 |
561 | |Resource|Progress|
562 | |---|---|
563 | |[Pinecone: LangChain AI Handbook](https://www.pinecone.io/learn/series/langchain/)|0/11|
564 | |[DeepLearning.AI: LangChain for LLM Application Development](https://www.deeplearning.ai/short-courses/langchain-for-llm-application-development/)|⬜|
565 | |[DeepLearning.AI: LangChain: Chat with Your Data](https://www.deeplearning.ai/short-courses/langchain-chat-with-your-data/)|⬜|
566 |
567 |
568 | ### JavaScript
569 |
570 | |Resource|Progress|
571 | |---|---|
572 | |[Udacity: ES6 - JavaScript Improved](https://www.udacity.com/course/es6-javascript-improved--ud356)|✅|
573 | |[Udacity: Intro to Javascript](https://www.udacity.com/course/intro-to-javascript--ud803)|✅|
574 | |[Udacity: Object Oriented JS 1](https://www.udacity.com/course/object-oriented-javascript--ud015)|✅|
575 | |[Udacity: Object Oriented JS 2](https://www.udacity.com/course/object-oriented-javascript--ud711)|✅|
576 | |[Udemy: Understanding Typescript](https://www.udemy.com/understanding-typescript/)|✅|
577 | |[Codecademy: Learn JavaScript](https://www.codecademy.com/learn/learn-javascript)|✅|
578 | |[Codecademy: Jquery Track](https://www.codecademy.com/learn/learn-jquery)|✅|
579 | |[Pluralsight: Using The Chrome Developer Tools](https://www.pluralsight.com/courses/chrome-developer-tools)|✅|
580 |
581 |
582 | ### Matplotlib
583 |
584 | |Resource|Progress|
585 | |---|---|
586 | |[Datacamp: Introduction to Seaborn](https://www.datacamp.com/courses/introduction-to-seaborn)|✅|
587 | |[Datacamp: Introduction to Matplotlib](https://www.datacamp.com/courses/introduction-to-matplotlib)|✅|
588 |
589 |
590 | ### MLFlow
591 |
592 | |Resource|Progress|
593 | |---|---|
594 | |[Datacamp: Introduction to MLFlow](https://www.datacamp.com/courses/introduction-to-mlflow)|✅|
595 |
596 |
597 | ### Numpy
598 |
599 | |Resource|Progress|
600 | |---|---|
601 | |[Youtube: Numpy Array Broadcasting In Python Explained](https://youtu.be/oG1t3qlzq14)|✅|
602 |
603 |
604 | ### Nexxt.JS
605 |
606 | | Resource | Progress |
607 | | ----------------------------------------------------------------- | -------- |
608 | | [Docs: Start building with Next.js](https://nextjs.org/learn) | |
609 |
610 | ### Pandas
611 |
612 | |Resource|Progress|
613 | |---|---|
614 | |[Datacamp: Pandas Foundations](https://www.datacamp.com/courses/pandas-foundations)|✅|
615 | |[Datacamp: Pandas Joins for Spreadsheet Users](https://www.datacamp.com/courses/pandas-joins-for-spreadsheet-users)|✅|
616 | |[Datacamp: Manipulating DataFrames with pandas](https://www.datacamp.com/courses/manipulating-dataframes-with-pandas)|✅|
617 | |[Datacamp: Merging DataFrames with pandas](https://www.datacamp.com/courses/merging-dataframes-with-pandas)|✅|
618 | |[Datacamp: Data Manipulation with pandas](https://www.datacamp.com/courses/data-manipulation-with-pandas)|✅|
619 | |[Datacamp: Optimizing Python Code with pandas](https://www.datacamp.com/courses/optimizing-python-code-with-pandas)|✅|
620 | |[Datacamp: Streamlined Data Ingestion with pandas](https://www.datacamp.com/courses/streamlined-data-ingestion-with-pandas)|✅|
621 | |[Datacamp: Analyzing Marketing Campaigns with pandas](https://www.datacamp.com/courses/analyzing-marketing-campaigns-with-pandas)|✅|
622 | |[Datacamp: Analyzing Police Activity with pandas](https://www.datacamp.com/courses/analyzing-police-activity-with-pandas)|✅|
623 |
624 |
625 | ### PyTorch
626 |
627 | |Resource|Progress|
628 | |---|---|
629 | |[Article: PyTorch internals](https://blog.ezyang.com/2019/05/pytorch-internals/)|⬜|
630 | |[Article: Taking PyTorch For Granted](https://nrehiew.github.io/blog/pytorch/)|⬜|
631 | |[Datacamp: Introduction to Deep Learning with PyTorch](https://www.datacamp.com/courses/deep-learning-with-pytorch)|✅|
632 | |[Datacamp: Intermediate Deep Learning with PyTorch](https://app.datacamp.com/learn/courses/intermediate-deep-learning-with-pytorch)|⬜|
633 | |[Datacamp: Deep Learning for Text with PyTorch](https://www.datacamp.com/courses/deep-learning-for-text-with-pytorch)|⬜|
634 | |[Datacamp: Deep Learning for Images with PyTorch](https://www.datacamp.com/courses/deep-learning-for-images-with-pytorch)|⬜|
635 | |[Deeplizard: Neural Network Programming - Deep Learning with PyTorch](https://www.youtube.com/playlist?list=PLZbbT5o_s2xrfNyHZsM6ufI0iZENK9xgG)|✅|
636 |
637 |
638 | ### ReactJS
639 |
640 | |Resource|Progress|
641 | |---|---|
642 | |[Codecademy: Learn ReactJS: Part I](https://www.codecademy.com/learn/react-101)|✅|
643 | |[Codecademy: Learn ReactJS: Part II](https://www.codecademy.com/learn/react-102)|✅|
644 | |[NexxtJS: React Foundations](https://nextjs.org/learn/react-foundations)|⬜|
645 |
646 | ### Spacy
647 |
648 | |Resource|Progress|
649 | |---|---|
650 | |[Datacamp: Advanced NLP with spaCy](https://www.datacamp.com/courses/advanced-nlp-with-spacy)|✅|
651 |
652 | ### Tensorflow & Keras
653 |
654 | |Resource|Progress|
655 | |---|---|
656 | |[Datacamp: Introduction to TensorFlow in Python](https://www.datacamp.com/courses/introduction-to-tensorflow-in-python)|✅|
657 | |[Datacamp: Deep Learning in Python](https://www.datacamp.com/courses/deep-learning-in-python)|✅|
658 | |[Datacamp: Introduction to Deep Learning with Keras](https://www.datacamp.com/courses/deep-learning-with-keras-in-python)|✅|
659 | |[Datacamp: Advanced Deep Learning with Keras](https://www.datacamp.com/courses/advanced-deep-learning-with-keras-in-python)|✅|
660 | |[Deeplizard: Keras - Python Deep Learning Neural Network API](https://www.youtube.com/playlist?list=PLZbbT5o_s2xrwRnXk_yCPtnqqo4_u2YGL)|✅|
661 | |[Udacity: Intro to TensorFlow for Deep Learning](https://www.udacity.com/course/intro-to-tensorflow-for-deep-learning--ud187)|✅|
662 |
663 | ### VSCode
664 |
665 | |Resource|Progress|
666 | |---|---|
667 | |[VSCode Docs: Python Interactive window](https://code.visualstudio.com/docs/python/jupyter-support-py)|⬜|
668 |
669 | ## Miscellaneous
670 |
671 | ### Design
672 |
673 | |Resource|Progress|
674 | |---|---|
675 | |[Course: How to Visualize Value](https://visualizevalue.com/products/how-to-visualize-value)|✅|
676 |
677 | ### Finance
678 |
679 | |Resource|Progress|
680 | |---|---|
681 | |[Coursera: Financial Markets](https://www.coursera.org/learn/financial-markets-global)|⬜|
682 |
683 |
684 |
685 | ### Marketing
686 |
687 | |Resource|Progress|
688 | |---|---|
689 | |[Course: Build Once, Sell Twice](https://visualizevalue.com/products/build-once-sell-twice-the-productization-playbook)|✅|
690 |
691 | ### Search Engine Optimization (SEO)
692 |
693 | |Resource|Progress|
694 | |---|---|
695 | |[Course: Compound Content](https://visualizevalue.com/products/compound-content)|✅|
696 |
697 | ### Technical Writing
698 | |Resource|Progress|
699 | |---|---|
700 | |[Google: Technical Writing Course](https://developers.google.com/tech-writing/overview)|⬜|
--------------------------------------------------------------------------------