├── LICENSE └── README.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Amit Chaudhary 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # learning 2 | 3 | A running log of things I'm learning to build strong core software engineering skills while also expanding my knowledge of [adjacent technologies](http://www.effectiveengineer.com/blog/master-adjacent-disciplines) a little bit [everyday](https://jamesclear.com/continuous-improvement). 4 | 5 | **Updated**: Once a month | **Current** **Focus**: Generative AI 6 | 7 | ## Core Skills 8 | 9 | > Generic skills that are transferrable to any sort of software work I do 10 | 11 | ### Python Programming 12 | 13 | |Resource|Progress| 14 | |---|---| 15 | |[Datacamp: Writing Efficient Python Code](https://www.datacamp.com/courses/writing-efficient-python-code)|✅| 16 | |[Datacamp: Writing Functions in Python](https://www.datacamp.com/courses/writing-functions-in-python)|✅| 17 | |[Datacamp: Object-Oriented Programming in Python](https://www.datacamp.com/courses/object-oriented-programming-in-python)|✅| 18 | |[Datacamp: Intermediate Object-Oriented Programming in Python](https://www.datacamp.com/courses/intermediate-object-oriented-programming-in-python)|✅| 19 | |[Datacamp: Importing Data in Python (Part 1)](https://www.datacamp.com/courses/importing-data-in-python-part-1)|✅| 20 | |[Datacamp: Importing Data in Python (Part 2)](https://www.datacamp.com/courses/importing-data-in-python-part-2)|✅| 21 | |[Datacamp: Intermediate Python for Data Science](https://www.datacamp.com/courses/intermediate-python-for-data-science)|✅| 22 | |[Datacamp: Python Data Science Toolbox (Part 1)](https://www.datacamp.com/courses/python-data-science-toolbox-part-1)|✅| 23 | |[Datacamp: Python Data Science Toolbox (Part 2)](https://www.datacamp.com/courses/python-data-science-toolbox-part-2)|✅| 24 | |[Datacamp: Developing Python Packages](https://www.datacamp.com/courses/developing-python-packages)|✅| 25 | |[Datacamp: Conda Essentials](https://www.datacamp.com/courses/conda-essentials)|✅| 26 | |[Youtube: Tutorial: Sebastian Witowski - Modern Python Developer's Toolkit](https://www.youtube.com/watch?v=WkUBx3g2QfQ&feature=youtu.be)|✅| 27 | |[Datacamp: Working with Dates and Times in Python](https://www.datacamp.com/courses/working-with-dates-and-times-in-python)|✅| 28 | |[Datacamp: Command Line Automation in Python](https://www.datacamp.com/courses/command-line-automation-in-python)|⬜| 29 | |[Book: Python 201](https://leanpub.com/python201)|⬜| 30 | |[Book: Writing Idiomatic Python 3](https://www.amazon.com/Writing-Idiomatic-Python-Jeff-Knupp-ebook/dp/B00B5VXMRG)|⬜| 31 | |[Article: Python's many command-line utilities](https://www.pythonmorsels.com/cli-tools/)|⬜| 32 | |[Article: A Programmer’s Introduction to Unicode](https://www.reedbeta.com/blog/programmers-intro-to-unicode/)|⬜| 33 | |[Article: Exposing string types to maximize user happiness](https://stephantul.github.io/python/typing/2025/03/07/externalized-types/)|✅| 34 | 35 | ### Testing & Profiling 36 | 37 | |Resource|Progress| 38 | |---|---| 39 | |[Datacamp: Unit Testing for Data Science in Python](https://www.datacamp.com/courses/unit-testing-for-data-science-in-python)|✅| 40 | |[Book: Test Driven Development with Python](http://chimera.labs.oreilly.com/books/1234000000754/index.html)|⬜| 41 | |[Article: Introduction to Memory Profiling in Python](https://www.datacamp.com/tutorial/memory-profiling-python)|✅| 42 | |[Article: Profiling Python code with memory_profiler](https://www.wrighters.io/profiling-python-code-with-memory_profiler/)|✅| 43 | |[Article: How to Use "memory_profiler" to Profile Memory Usage by Python Code?](https://coderzcolumn.com/tutorials/python/how-to-profile-memory-usage-in-python-using-memory-profiler)|✅| 44 | |[Youtube: Debug Python inside Docker using debugpy and VSCode](https://www.youtube.com/watch?v=ywfsLKRLmf4)|✅| 45 | 46 | 47 | ### Data Structures and Algorithms 48 | 49 | |Resource|Progress| 50 | |---|---| 51 | |[Book: Grokking Algorithms](https://www.manning.com/books/grokking-algorithms)|✅| 52 | |[Book: The Tech Resume Inside Out](https://thetechresume.com)|✅| 53 | |[Neetcode: Algorithms and Data Structures for Beginners](https://neetcode.io/courses/dsa-for-beginners/0)|✅| 54 | |[Udacity: Intro to Data Structures and Algorithms](https://www.udacity.com/course/technical-interview--ud513)|✅| 55 | 56 | 57 | ### Linux & Command Line 58 | 59 | |Resource|Progress| 60 | |---|---| 61 | |[Datacamp: Introduction to Shell for Data Science](https://www.datacamp.com/courses/introduction-to-shell-for-data-science)|✅| 62 | |[Datacamp: Introduction to Bash Scripting](https://www.datacamp.com/courses/introduction-to-bash-scripting)|✅| 63 | |[Datacamp: Data Processing in Shell](https://www.datacamp.com/courses/data-processing-in-shell)|✅| 64 | |[MIT: The Missing Semester](https://www.youtube.com/playlist?list=PLyzOVJj3bHQuloKGG59rS43e29ro7I57J)|✅| 65 | |[Udacity: Linux Command Line Basics](https://www.udacity.com/course/linux-command-line-basics--ud595)|✅| 66 | |[Udacity: Shell Workshop](https://www.udacity.com/course/shell-workshop--ud206)|✅| 67 | |[Udacity: Configuring Linux Web Servers](https://www.udacity.com/course/configuring-linux-web-servers--ud299)|✅| 68 | 69 | ### Version Control 70 | 71 | |Resource|Progress| 72 | |---|---| 73 | |[Udacity: Version Control with Git](https://www.udacity.com/course/version-control-with-git--ud123)|✅| 74 | |[Datacamp: Introduction to Git for Data Science](https://www.datacamp.com/courses/introduction-to-git-for-data-science)|✅| 75 | |[Udacity: GitHub & Collaboration](https://www.udacity.com/course/github-collaboration--ud456)|✅| 76 | |[Udacity: How to Use Git and GitHub](https://www.udacity.com/course/how-to-use-git-and-github--ud775)|✅| 77 | |[Youtube: How to Use Git Worktree \| Checkout Multiple Git Branches at Once](https://youtu.be/s4BTvj1ZVLM)|✅| 78 | 79 | ### Databases 80 | 81 | |Resource|Progress| 82 | |---|---| 83 | |[Udacity: Intro to relational database](https://www.udacity.com/course/intro-to-relational-databases--ud197)|✅| 84 | |[Udacity: Database Systems Concepts & Design](https://www.udacity.com/course/database-systems-concepts-design--ud150)|⬜| 85 | |[Datacamp: Database Design](https://www.datacamp.com/courses/database-design)|⬜| 86 | |[Datacamp: Introduction to Databases in Python](https://www.datacamp.com/courses/introduction-to-relational-databases-in-python)|⬜| 87 | |[Datacamp: Intro to SQL for Data Science](https://www.datacamp.com/courses/intro-to-sql-for-data-science)|✅| 88 | |[Datacamp: Intermediate SQL](https://www.datacamp.com/courses/intermediate-sql)|⬜| 89 | |[Datacamp: Joining Data in PostgreSQL](https://www.datacamp.com/courses/joining-data-in-postgresql)|⬜| 90 | |[Udacity: SQL for Data Analysis](https://www.udacity.com/course/sql-for-data-analysis--ud198)|⬜| 91 | |[Datacamp: Exploratory Data Analysis in SQL](https://www.datacamp.com/courses/sql-for-exploratory-data-analysis)|⬜| 92 | |[Datacamp: Applying SQL to Real-World Problems](https://www.datacamp.com/courses/applying-sql-to-real-world-problems)|⬜| 93 | |[Datacamp: Analyzing Business Data in SQL](https://www.datacamp.com/courses/analyzing-business-data-in-sql)|⬜| 94 | |[Datacamp: Reporting in SQL](https://www.datacamp.com/courses/reporting-in-sql)|⬜| 95 | |[Datacamp: Data-Driven Decision Making in SQL](https://www.datacamp.com/courses/data-driven-decision-making-with-sql)|⬜| 96 | |[Datacamp: NoSQL Concepts](https://www.datacamp.com/courses/nosql-concepts)|⬜| 97 | |[Datacamp: Introduction to MongoDB in Python](https://www.datacamp.com/courses/introduction-to-using-mongodb-for-data-science-with-python)|⬜| 98 | 99 | ### Backend Engineering 100 | 101 | |Resource|Progress| 102 | |---|---| 103 | |[Udacity: Authentication & Authorization: OAuth](https://www.udacity.com/course/authentication-authorization-oauth--ud330)|✅| 104 | |[Udacity: HTTP & Web Servers](https://www.udacity.com/course/http-web-servers--ud303)|✅| 105 | |[Udacity: Client-Server Communication](https://www.udacity.com/course/client-server-communication--ud897)|⬜| 106 | |[Udacity: Designing RESTful APIs](https://www.udacity.com/course/designing-restful-apis--ud388)|✅| 107 | |[Udacity: Networking for Web Developers](https://www.udacity.com/course/networking-for-web-developers--ud256)|✅| 108 | 109 | ### Production System Design 110 | 111 | |Resource|Progress| 112 | |---|---| 113 | |[Book: Designing Machine Learning Systems](https://www.oreilly.com/library/view/designing-machine-learning/9781098107956/)|✅| 114 | |[Neetcode: System Design for Beginners](https://neetcode.io/courses/system-design-for-beginners/0)|✅| 115 | |[Neetcode: System Design Interview](https://neetcode.io/courses/system-design-interview)|✅| 116 | |[Datacamp: Customer Analytics & A/B Testing in Python](https://www.datacamp.com/courses/customer-analytics-ab-testing-in-python)|✅| 117 | |[Datacamp: A/B Testing in Python](https://www.datacamp.com/courses/ab-testing-in-python)|⬜| 118 | |[Udacity: A/B Testing](https://www.udacity.com/course/ab-testing--ud257)|⬜| 119 | |[Datacamp: MLOps Concepts](https://www.datacamp.com/courses/mlops-concepts)|✅| 120 | |[Datacamp: Machine Learning Monitoring Concepts](https://www.datacamp.com/courses/machine-learning-monitoring-concepts)|✅| 121 | 122 | 123 | ### Maths 124 | 125 | |Resource|Progress| 126 | |---|---| 127 | |[Datacamp: Foundations of Probability in Python](https://www.datacamp.com/courses/foundations-of-probability-in-python)|✅| 128 | |[Datacamp: Introduction to Statistics](https://www.datacamp.com/courses/introduction-to-statistics)|✅| 129 | |[Datacamp: Introduction to Statistics in Python](https://www.datacamp.com/courses/introduction-to-statistics-in-python)|✅| 130 | |[Datacamp: Hypothesis Testing in Python](https://www.datacamp.com/courses/hypothesis-testing-in-python)|✅| 131 | |[Datacamp: Statistical Thinking in Python (Part 1)](https://www.datacamp.com/courses/statistical-thinking-in-python-part-1)|✅| 132 | |[Datacamp: Statistical Thinking in Python (Part 2)](https://www.datacamp.com/courses/statistical-thinking-in-python-part-2)|✅| 133 | |[Datacamp: Experimental Design in Python](https://datacamp.com/courses/experimental-design-in-python)|✅| 134 | |[Datacamp: Practicing Statistics Interview Questions in Python](https://www.datacamp.com/courses/practicing-statistics-interview-questions-in-python)|⬜| 135 | |[edX: Essential Statistics for Data Analysis using Excel](https://www.edx.org/course/essential-statistics-data-analysis-using-microsoft-dat222x-1)|✅| 136 | |[Udacity: Intro to Inferential Statistics](https://www.udacity.com/course/intro-to-inferential-statistics--ud201)|✅| 137 | |[MIT 18.06 Linear Algebra, Spring 2005](https://www.youtube.com/playlist?list=PLE7DDD91010BC51F8)|✅| 138 | |[Udacity: Eigenvectors and Eigenvalues](https://www.udacity.com/course/eigenvectors-and-eigenvalues--ud104)|✅| 139 | |[Udacity: Linear Algebra Refresher](https://www.udacity.com/course/linear-algebra-refresher-course--ud953)|⬜| 140 | |[Youtube: Essence of linear algebra](https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab)|⬜| 141 | 142 | ## Specialization 143 |
144 | 145 | ### Traditional Machine Learning 146 | 147 | |Resource|Progress| 148 | |---|---| 149 | |[Book: Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition](https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/)|⬜| 150 | |[Book: A Machine Learning Primer](https://www.confetti.ai/assets/ml-primer/ml_primer.pdf)|✅| 151 | |[Book: Grokking Machine Learning](https://www.manning.com/books/grokking-machine-learning)|✅| 152 | |[Book: The StatQuest Illustrated Guide To Machine Learning](https://www.amazon.com/StatQuest-Illustrated-Guide-Machine-Learning/dp/B0BLM4TLPY)|✅| 153 | |[Datacamp: Ensemble Methods in Python](https://www.datacamp.com/courses/ensemble-methods-in-python)|✅| 154 | |[Datacamp: Extreme Gradient Boosting with XGBoost](https://www.datacamp.com/courses/extreme-gradient-boosting-with-xgboost)|⬜| 155 | |[Datacamp: Clustering Methods with SciPy](https://www.datacamp.com/courses/clustering-methods-with-scipy)|✅| 156 | |[Datacamp: Unsupervised Learning in Python](https://www.datacamp.com/courses/unsupervised-learning-in-python)|✅| 157 | |[Udacity: Segmentation and Clustering](https://www.udacity.com/course/segmentation-and-clustering--ud981)|✅| 158 | |[Datacamp: Intro to Python for Data Science](https://www.datacamp.com/courses/intro-to-python-for-data-science)|✅| 159 | |[edX: Implementing Predictive Analytics with Spark in Azure HDInsight](https://www.edx.org/course/implementing-predictive-analytics-spark-microsoft-dat202-3x-2)|✅| 160 | |[Datacamp: Supervised Learning with scikit-learn](https://www.datacamp.com/courses/supervised-learning-with-scikit-learn)|✅| 161 | |[Datacamp: Machine Learning with Tree-Based Models in Python](https://www.datacamp.com/courses/machine-learning-with-tree-based-models-in-python)|✅| 162 | |[Datacamp: Linear Classifiers in Python](https://www.datacamp.com/courses/linear-classifiers-in-python)|✅| 163 | |[Datacamp: Model Validation in Python](https://www.datacamp.com/courses/model-validation-in-python)|✅| 164 | |[Datacamp: Hyperparameter Tuning in Python](https://www.datacamp.com/courses/hyperparameter-tuning-in-python)|✅| 165 | |[Datacamp: HR Analytics in Python: Predicting Employee Churn](https://www.datacamp.com/courses/hr-analytics-in-python-predicting-employee-churn)|✅| 166 | |[Datacamp: Predicting Customer Churn in Python](https://www.datacamp.com/courses/predicting-customer-churn-in-python)|✅| 167 | |[Datacamp: Dimensionality Reduction in Python](https://www.datacamp.com/courses/dimensionality-reduction-in-python)|✅| 168 | |[Datacamp: Preprocessing for Machine Learning in Python](https://www.datacamp.com/courses/preprocessing-for-machine-learning-in-python)|✅| 169 | |[Datacamp: Data Types for Data Science](https://www.datacamp.com/courses/data-types-for-data-science)|✅| 170 | |[Datacamp: Cleaning Data in Python](https://www.datacamp.com/courses/cleaning-data-in-python)|✅| 171 | |[Datacamp: Feature Engineering for Machine Learning in Python](https://www.datacamp.com/courses/feature-engineering-for-machine-learning-in-python)|✅| 172 | |[Datacamp: Predicting CTR with Machine Learning in Python](https://www.datacamp.com/courses/predicting-ctr-with-machine-learning-in-python)|✅| 173 | |[Datacamp: Intro to Financial Concepts using Python](https://www.datacamp.com/courses/intro-to-financial-concepts-using-python)|✅| 174 | |[Datacamp: Fraud Detection in Python](https://www.datacamp.com/courses/fraud-detection-in-python)|✅| 175 | 176 | 177 | ### Deep Learning 178 | 179 | |Resource|Progress| 180 | |---|---| 181 | |[Article: An overview of gradient descent optimization algorithms](https://www.ruder.io/optimizing-gradient-descent)|✅| 182 | |[Book: Make Your Own Neural Network](https://www.amazon.com/Make-Your-Own-Neural-Network/dp/1530826608)|✅| 183 | |[Fast.ai: Practical Deep Learning for Coder (Part 1)](https://course.fast.ai/)|✅| 184 | |[Fast.ai: Practical Deep Learning for Coder (Part 2)](https://course.fast.ai/Lessons/part2.html) `9, 13,14,17,18(48:10),19`|⬜| 185 | |[Datacamp: Convolutional Neural Networks for Image Processing](https://www.datacamp.com/courses/convolutional-neural-networks-for-image-processing)|✅| 186 | |[Karpathy: Neural Networks: Zero to Hero](https://github.com/karpathy/nn-zero-to-hero/)|✅| 187 | |[Article: Weight Initialization in Neural Networks: A Journey From the Basics to Kaiming](https://towardsdatascience.com/weight-initialization-in-neural-networks-a-journey-from-the-basics-to-kaiming-954fb9b47c79)|⬜| 188 | |[Article: Things that confused me about cross-entropy](https://chris-said.io/2020/12/26/two-things-that-confused-me-about-cross-entropy/)|✅| 189 | 190 | ### Natural Language Processing 191 | 192 | |Resource|Progress| 193 | |---|---| 194 | |[Book: Natural Language Processing with Transformers](https://transformersbook.com/)|✅| 195 | |[Stanford CS224U: Natural Language Understanding \| Spring 2019](https://www.youtube.com/playlist?list=PLoROMvodv4rObpMCir6rNNUlFAn56Js20)|✅| 196 | |[Stanford CS224N: Stanford CS224N: NLP with Deep Learning \| Winter 2019](https://www.youtube.com/playlist?list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z)|✅| 197 | |[CMU: Low-resource NLP Bootcamp 2020](https://www.youtube.com/playlist?list=PL8PYTP1V4I8A1CpCzURXAUa6H4HO7PF2c)|✅| 198 | |[CMU Multilingual NLP 2020](http://demo.clab.cs.cmu.edu/11737fa20/)|✅| 199 | |[Datacamp: Feature Engineering for NLP in Python](https://www.datacamp.com/courses/feature-engineering-for-nlp-in-python)|✅| 200 | |[Datacamp: Natural Language Processing Fundamentals in Python](https://www.datacamp.com/courses/natural-language-processing-fundamentals-in-python)|✅| 201 | |[Datacamp: Regular Expressions in Python](https://www.datacamp.com/courses/regular-expressions-in-python)|✅| 202 | |[Datacamp: RNN for Language Modeling](https://www.datacamp.com/courses/recurrent-neural-networks-for-language-modeling-in-python)|✅| 203 | |[Datacamp: Natural Language Generation in Python](https://www.datacamp.com/courses/natural-language-generation-in-python)|✅| 204 | |[Datacamp: Building Chatbots in Python](https://www.datacamp.com/courses/building-chatbots-in-python)|✅| 205 | |[Datacamp: Sentiment Analysis in Python](https://www.datacamp.com/courses/sentiment-analysis-in-python)|✅| 206 | |[Datacamp: Machine Translation in Python](https://www.datacamp.com/courses/machine-translation-in-python)|✅| 207 | |[Article: The Unreasonable Effectiveness of Collocations](https://opensourceconnections.com/blog/2019/05/16/unreasonable-effectiveness-of-collocations/)|⬜| 208 | |[Article: FuzzyWuzzy: Fuzzy String Matching in Python](https://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/#)|✅| 209 | |[Article: Transformers: Origins](https://mark-riedl.medium.com/transformers-origins-1db4bdfcb3d1)|⬜| 210 | 211 | ### Generative AI 212 |
213 | 214 | #### LLM Theory 215 | 216 | |Resource|Progress| 217 | |---|---| 218 | |[Book: Hands-On Large Language Models: Language Understanding and Generation](https://www.amazon.com/Hands-Large-Language-Models-Understanding/dp/1098150961)|✅| 219 | |[Book: AI Engineering: Building Applications with Foundation Models](https://www.amazon.com/AI-Engineering-Building-Applications-Foundation/dp/1098166302)|✅| 220 | |[Book: Designing Large Language Model Applications](https://www.oreilly.com/library/view/designing-large-language/9781098150495/)|⬜| 221 | |[Book: Large Language Models: A Deep Dive: Bridging Theory and Practice](https://www.amazon.com/Large-Language-Models-Bridging-Practice/dp/3031656466)|⬜| 222 | |[Book: A Little Bit of Reinforcement Learning from Human Feedback](https://rlhfbook.com/)|✅| 223 | |[Stanford CS236: Deep Generative Models](https://www.youtube.com/playlist?list=PLoROMvodv4rPOWA-omMM6STXaWW4FvJT8) `18 lectures`|1/18| 224 | |[Article: You could have designed state of the art Positional Encoding](https://fleetwood.dev/posts/you-could-have-designed-SOTA-positional-encoding)|⬜| 225 | |[Article: From Digits to Decisions: How Tokenization Impacts Arithmetic in LLMs](https://huggingface.co/spaces/huggingface/number-tokenization-blog)|⬜| 226 | |[Article: SolidGoldMagikarp (plus, prompt generation)](https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldmagikarp-plus-prompt-generation)|⬜| 227 | |[Article: Sampling for Text Generation](https://huyenchip.com/2024/01/16/sampling.html)|⬜| 228 | |[Article: Scaling test-time compute - a Hugging Face Space by HuggingFaceH4](https://huggingface.co/spaces/HuggingFaceH4/blogpost-scaling-test-time-compute)|⬜| 229 | |[Article: DeepSeek R1's recipe to replicate o1 and the future of reasoning LMs](https://www.interconnects.ai/p/deepseek-r1-recipe-for-o1)|✅| 230 | |[Article: The Illustrated DeepSeek-R1](https://newsletter.languagemodels.co/p/the-illustrated-deepseek-r1)|✅| 231 | |[Article: A Visual Guide to Reasoning LLMs](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-reasoning-llms)|⬜| 232 | |[DeepLearning.AI: Pretraining LLMs](https://www.deeplearning.ai/short-courses/pretraining-llms)|✅| 233 | |[DeepLearning.AI: Reinforcement Learning from Human Feedback](https://www.deeplearning.ai/short-courses/reinforcement-learning-from-human-feedback)|✅| 234 | |[Karpathy: Intro to Large Language Models](https://www.youtube.com/watch?v=zjkBMFhNj_g) `1hr`|✅| 235 | |[Karpathy: Let's build the GPT Tokenizer](https://www.youtube.com/watch?v=zduSFxRajkE) `2hr13m`|✅| 236 | |[Karpathy: Let's reproduce GPT-2 (124M)](https://www.youtube.com/watch?v=l8pRSuU81PU) `4hr1m`|✅| 237 | |[Youtube: A Hackers' Guide to Language Models](https://www.youtube.com/watch?v=jkrNMKz9pWU) `1hr30m`|✅| 238 | |[Karpathy: Deep Dive into LLMs like ChatGPT](https://www.youtube.com/watch?v=7xTGNNLPyMI) `3h31m`|✅| 239 | |[Youtube: 5 Years of GPTs with Finbarr Timbers](https://www.youtube.com/watch?v=YA0pzBYAV2Q&list=PLKlhhkvvU8-YxMP9hjEYJTJDCaGszrJIh&index=8&t=43s) `55m`|✅| 240 | |[Youtube: Stanford CS229 I Machine Learning I Building Large Language Models (LLMs)](https://www.youtube.com/watch?v=9vM4p9NN0Ts) `1h44m`|✅| 241 | |[Youtube: LLaMA explained: KV-Cache, Rotary Positional Embedding, RMS Norm, Grouped Query Attention, SwiGLU](https://www.youtube.com/watch?v=Mn_9W1nCFLo) `1h10m`|✅| 242 | |[Youtube: CMU Advanced NLP Fall 2024 (7): Prompting and Complex Reasoning](https://www.youtube.com/watch?v=1Faf1cTe3T8&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=2)|⬜| 243 | |[Youtube: CMU Advanced NLP Fall 2024 (6): Instruction Tuning](https://www.youtube.com/watch?v=iWcGS0gCL1E&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=3)|⬜| 244 | |[Youtube: CMU Advanced NLP Fall 2024 (12): Domain Specific Modeling: Code and Math](https://www.youtube.com/watch?v=qHNUVpKO2dc&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=4)|⬜| 245 | |[Youtube: CMU Advanced NLP Fall 2024 (15): Tool Use and LLM Agent Basics](https://www.youtube.com/watch?v=a3SjRsqV9ZA&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=16)|⬜| 246 | |[Youtube: CMU Advanced NLP Fall 2024 (14): Ensembling and Mixture of Experts](https://www.youtube.com/watch?v=E4Rg4qTw4xw&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=15)|✅| 247 | |[Youtube: A little guide to building Large Language Models in 2024](https://www.youtube.com/watch?v=2-SPH9hIKT8) `1h15m`|✅| 248 | |[Youtube: How to approach post-training for AI applications](https://www.youtube.com/watch?v=grpc-Wyy-Zg) `22m`|✅| 249 | |[Youtube: Speculations on Test-Time Scaling (o1) `47m`](https://www.youtube.com/watch?v=6PEJ96k1kiw)|✅| 250 | |[Youtube: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://youtu.be/XMnxKGVnEUc) `1h19m`|✅| 251 | |[Youtube: How DeepSeek Changes the LLM Story](https://www.youtube.com/watch?v=0eMzc-WnBfQ)|✅| 252 | |[Youtube: MIT EI seminar, Hyung Won Chung from OpenAI. "Don't teach. Incentivize."](https://www.youtube.com/watch?v=kYWUEV_e2ss) `35m`|✅| 253 | |[Youtube: How I use LLMs](https://youtu.be/EWvNQjAaOHw) `2h7m`|✅| 254 | |[Youtube: Simple Diffusion Language Models](https://youtu.be/WjAUX23vgfg)|✅| 255 | |[Youtube: Introduction to Reasoning LLMs](https://www.youtube.com/watch?v=AZhUhGsgz4s) `1hr`|⬜| 256 | |[Article: Mamba Explained](https://thegradient.pub/mamba-explained/)|⬜| 257 | |[Article: A Visual Guide to Mamba and State Space Models](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-mamba-and-state)|⬜| 258 | 259 | #### Multi-modality 260 | 261 | |Resource|Progress| 262 | |---|---| 263 | |[Article: Understanding Multimodal LLMs](https://magazine.sebastianraschka.com/p/understanding-multimodal-llms)|⬜| 264 | |[Article: GPT-4 Vision Alternatives](https://blog.roboflow.com/gpt-4-vision-alternatives/)|⬜| 265 | |[Article: Computer-Using Agent](https://openai.com/index/computer-using-agent/)|✅| 266 | |[Youtube: AI Visions Live \| Merve Noyan \| Open-source Multimodality](https://www.youtube.com/watch?v=_TlhKHTgWjY) `54m`|✅| 267 | |[DeepLearning.AI: How Diffusion Models Work](https://www.deeplearning.ai/short-courses/how-diffusion-models-work/)|✅| 268 | |[DeepLearning.AI: Prompt Engineering for Vision Models](https://www.deeplearning.ai/short-courses/prompt-engineering-for-vision-models/)|⬜| 269 | |[DeepLearning.AI: Building Multimodal Search and RAG](https://www.deeplearning.ai/short-courses/building-multimodal-search-and-rag/)|✅| 270 | |[Pinecone: Embedding Methods for Image Search](https://www.pinecone.io/learn/series/image-search/)|0/8| 271 | |[Youtube: Lesson 9A 2022 - Stable Diffusion deep dive](https://youtu.be/0_BBRNYInx8)|✅| 272 | |[Article: Diffusion models are autoencoders](https://sander.ai/2022/01/31/diffusion.html)|✅| 273 | |[Article: Diffusion Language Models](https://sander.ai/2023/01/09/diffusion-language.html)|✅| 274 | |[Article: Guidance: a cheat code for diffusion models](https://sander.ai/2022/05/26/guidance.html)|✅| 275 | |[Article: Perspectives on diffusion](https://sander.ai/2023/07/20/perspectives.html)|✅| 276 | |[Article: The geometry of diffusion guidance](https://sander.ai/2023/08/28/geometry.html)|✅| 277 | |[Article: Diffusion is spectral autoregression](https://sander.ai/2024/09/02/spectral-autoregression.html)|✅| 278 | |[Article: Generative modelling in latent space](https://sander.ai/2025/04/15/latents.html)|✅| 279 | |[Youtube: Sander Dieleman - Generative modelling through iterative refinement](https://www.youtube.com/watch?v=9BHQvQlsVdE)|✅| 280 | |[Speech AI models: an introduction](https://thomwolf.io/blog/speech-ai.html)|⬜| 281 | 282 | 283 | #### Information Retrieval / RAG 284 | 285 | | Resource | Progress | 286 | | ---------------------------------------------------------------------------------------------------------------------------------------------- | -------- | 287 | | [Article: Pretrained Transformer Language Models for Search - part 1](https://blog.vespa.ai/pretrained-transformer-language-models-for-search-part-1/#) | ✅ | 288 | | [Article: Pretrained Transformer Language Models for Search - part 2](https://blog.vespa.ai/pretrained-transformer-language-models-for-search-part-2/) | ✅ | 289 | | [Article: Pretrained Transformer Language Models for Search - part 3](https://blog.vespa.ai/pretrained-transformer-language-models-for-search-part-3) | ✅ | 290 | | [Article: Pretrained Transformer Language Models for Search - part 4](https://blog.vespa.ai/pretrained-transformer-language-models-for-search-part-4) | ✅ | 291 | |[Article: How not to use BERT for Document Ranking](https://bergum.medium.com/how-not-to-use-bert-for-search-ranking-4586716428d9)|✅| 292 | | [Article: Understanding LanceDB's IVF-PQ index](https://lancedb.github.io/lancedb/concepts/index_ivfpq/) | ✅ | 293 | | [Article: A little pooling goes a long way for multi-vector representations](https://www.answer.ai/posts/colbert-pooling.html) | ✅ | 294 | |[Article: Levels of Complexity: RAG Applications](https://jxnl.github.io/blog/writing/2024/02/28/levels-of-complexity-rag-applications/)|✅| 295 | |[Article: Systematically Improving Your RAG](https://jxnl.github.io/blog/writing/2024/05/22/systematically-improving-your-rag/)|✅| 296 | |[Article: Stop using LGTM@Few as a metric (Better RAG)](https://jxnl.github.io/blog/writing/2024/02/05/when-to-lgtm-at-k/)|✅| 297 | |[Article: Low-Hanging Fruit for RAG Search](https://jxnl.github.io/blog/writing/2024/05/11/low-hanging-fruit-for-rag-search/)|✅| 298 | |[Article: What AI Engineers Should Know about Search](https://softwaredoug.com/blog/2024/06/25/what-ai-engineers-need-to-know-search)|✅| 299 | |[Article: Evaluating Chunking Strategies for Retrieval](https://research.trychroma.com/evaluating-chunking)|✅| 300 | |[Article: Sentence Embeddings. Introduction to Sentence Embeddings](https://osanseviero.github.io/hackerllama/blog/posts/sentence_embeddings/)|✅| 301 | |[Article: LambdaMART in Depth](https://softwaredoug.com/blog/2022/01/17/lambdamart-in-depth)|⬜| 302 | |[Article: Guided Generation with Outlines](https://medium.com/canoe-intelligence-technology/guided-generation-with-outlines-c09a0c2ce9eb)|✅| 303 | |[Article: RAG tricks from the trenches](https://duarteocarmo.com/blog/rag-tricks-from-the-trenches)|⬜| 304 | |[Article: Retrieval 101](https://isaacflath.com/blog/blog_post?fpath=posts%2F2025-03-17-Retrieval101.ipynb)|⬜| 305 | | [Course: Fullstack Retrieval](https://community.fullstackretrieval.com/) | ⬜ | 306 | |[DeepLearning.AI: Building and Evaluating Advanced RAG Applications](https://www.deeplearning.ai/short-courses/building-evaluating-advanced-rag/)|✅| 307 | |[DeepLearning.AI: Vector Databases: from Embeddings to Applications](https://www.deeplearning.ai/short-courses/vector-databases-embeddings-applications/)|✅| 308 | |[DeepLearning.AI: Advanced Retrieval for AI with Chroma](https://www.deeplearning.ai/short-courses/advanced-retrieval-for-ai/)|✅| 309 | |[DeepLearning.AI: Prompt Compression and Query Optimization](https://www.deeplearning.ai/short-courses/prompt-compression-and-query-optimization/)|✅| 310 | |[DeepLearning.AI: Large Language Models with Semantic Search](https://www.deeplearning.ai/short-courses/large-language-models-semantic-search) `1hr`|✅| 311 | |[DeepLearning.AI: Building Applications with Vector Databases](https://www.deeplearning.ai/short-courses/building-applications-vector-databases/)|✅| 312 | |[DeepLearning.AI: Knowledge Graphs for RAG](https://www.deeplearning.ai/short-courses/knowledge-graphs-rag/)|⬜| 313 | |[DeepLearning.AI: Preprocessing Unstructured Data for LLM Applications](https://www.deeplearning.ai/short-courses/preprocessing-unstructured-data-for-llm-applications/)|⬜| 314 | |[DeepLearning.AI: Embedding Models: From Architecture to Implementation](https://www.deeplearning.ai/short-courses/embedding-models-from-architecture-to-implementation)|✅| 315 | |[DeepLearning.AI: Retrieval Optimization - From Tokenization to Vector Quantization](https://www.deeplearning.ai/short-courses/retrieval-optimization-from-tokenization-to-vector-quantization/)|✅| 316 | |[Pinecone: Vector Databases in Production for Busy Engineers](https://www.pinecone.io/learn/series/vector-databases-in-production-for-busy-engineers/)|✅| 317 | |[Pinecone: Retrieval Augmented Generation](https://www.pinecone.io/learn/series/rag/)|✅| 318 | |[Pinecone: Faiss: The Missing Manual](https://www.pinecone.io/learn/series/faiss/)|✅| 319 | |[Pinecone: Natural Language Processing for Semantic Search](https://www.pinecone.io/learn/series/nlp/)|0/13| 320 | |[Youtube: Systematically improving RAG applications](https://youtu.be/RrDBV6odPKo?list=PLgIaq8VgndJvXkDSeReTl2u4rQMShkZ6V)|✅| 321 | |[Youtube: Back to Basics for RAG w/ Jo Bergum](https://www.youtube.com/watch?v=nc0BupOkrhI&list=PLgIaq8VgndJvXkDSeReTl2u4rQMShkZ6V&index=2)|✅| 322 | |[Youtube: Beyond the Basics of Retrieval for Augmenting Generation (w/ Ben Clavié)](https://www.youtube.com/watch?v=0nA5QG3087g&t=1287s)|✅| 323 | |[Youtube: RAG From Scratch](https://www.youtube.com/playlist?list=PLfaIDFEXuae2LXbO1_PKyVJiQ23ZztA0x) `14/14`|✅| 324 | |[Youtube: CMU Advanced NLP Fall 2024 (10): Retrieval and RAG](https://www.youtube.com/watch?v=KfQaYk4k9eM&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=6) `1h17m`|✅| 325 | |[Guidance: Token Healing](https://github.com/guidance-ai/guidance/blob/main/notebooks/tutorials/token_healing.ipynb)|⬜| 326 | |[Youtube: What You See Is What You Search: Vision Language Models for PDF Retrieval [Jo Bergum]](https://youtu.be/qrbQUU4TrLM)|✅| 327 | 328 | #### Agentic Pattern 329 | 330 | |Resource|Progress| 331 | |---|---| 332 | |[Article: Tool Invocation - Demonstrating the Marvel of GPT's Flexibility](https://blog.jnbrymn.com/2024/01/30/the-marvel-of-GPT-generality.html)|⬜| 333 | |[Article: Introducing smolagents, a simple library to build agents](https://huggingface.co/blog/smolagents)|⬜| 334 | |[Article: What Problem Does The Model Context Protocol Solve?](https://www.aihero.dev/what-problem-does-model-context-protocol-solve)|✅| 335 | |[Anthropic: Building effective agents](https://www.anthropic.com/research/building-effective-agents)|✅| 336 | |[Anthropic: Building Effective Agents Cookbook](https://github.com/anthropics/anthropic-cookbook/tree/main/patterns/agents)|✅| 337 | |[OpenAI: Assistants & Agents Build Hour](https://vimeo.com/showcase/11333741/video/990334325)|✅| 338 | |[OpenAI: Function Calling Build Hour](https://vimeo.com/showcase/11333741/video/952127114)|✅| 339 | |[DeepLearning.AI: Functions, Tools and Agents with LangChain](https://www.deeplearning.ai/short-courses/functions-tools-agents-langchain/)|⬜| 340 | |[DeepLearning.AI: Building Agentic RAG with LlamaIndex](https://www.deeplearning.ai/short-courses/building-agentic-rag-with-llamaindex/)|✅| 341 | |[DeepLearning.AI: Multi AI Agent Systems with crewAI](https://www.deeplearning.ai/short-courses/multi-ai-agent-systems-with-crewai/)|✅| 342 | |[DeepLearning.AI: Building Towards Computer Use with Anthropic](https://www.deeplearning.ai/short-courses/building-towards-computer-use-with-anthropic/)|✅| 343 | |[DeepLearning.AI: Practical Multi AI Agents and Advanced Use Cases with crewAI](https://www.deeplearning.ai/short-courses/practical-multi-ai-agents-and-advanced-use-cases-with-crewai/)|⬜| 344 | |[DeepLearning.AI: LLMs as Operating Systems: Agent Memory](https://www.deeplearning.ai/short-courses/llms-as-operating-systems-agent-memory/)|✅| 345 | |[DeepLearning.AI: Serverless Agentic Workflows with Amazon Bedrock](https://www.deeplearning.ai/short-courses/serverless-agentic-workflows-with-amazon-bedrock/)|⬜| 346 | |[DeepLearning.AI: AI Agentic Design Patterns with AutoGen](https://www.deeplearning.ai/short-courses/ai-agentic-design-patterns-with-autogen/)|⬜| 347 | |[DeepLearning.AI: AI Agents in LangGraph](https://www.deeplearning.ai/short-courses/ai-agents-in-langgraph/)|⬜| 348 | |[DeepLearning.AI: Building Your Own Database Agent](https://www.deeplearning.ai/short-courses/building-your-own-database-agent/)|⬜| 349 | |[DeepLearning.AI: Function-Calling and Data Extraction with LLMs](https://www.deeplearning.ai/short-courses/function-calling-and-data-extraction-with-llms/) `59m`|✅| 350 | |[DeepLearning.AI: Evaluating AI Agents](https://www.deeplearning.ai/short-courses/evaluating-ai-agents/) `2h16m`|✅| 351 | |[DeepLearning.AI: Build Apps with Windsurf’s AI Coding Agents](https://www.deeplearning.ai/short-courses/build-apps-with-windsurfs-ai-coding-agents/) `1h10m`|✅| 352 | |[DeepLearning.AI: Building AI Browser Agents](https://www.deeplearning.ai/short-courses/building-ai-browser-agents)|⬜| 353 | |[Huggingface: Agents Course](https://huggingface.co/learn/agents-course/unit1/messages-and-special-tokens#base-models-vs-instruct-models)|Unit 1| 354 | |[Youtube: How to Evaluate Agents: Galileo’s Agentic Evaluations in Action](https://www.youtube.com/watch?v=QvStk5G8BZw)|✅| 355 | |[Youtube: Agent Response \| LangSmith Evaluation - Part 24](https://youtu.be/NbQKDfSw3gM?list=PLfaIDFEXuae0um8Fj0V4dHG37fGFU8Q5S)|✅| 356 | |[Youtube: Single Step \| LangSmith Evaluation - Part 25](https://youtu.be/AVPflFmRkd4?list=PLfaIDFEXuae0um8Fj0V4dHG37fGFU8Q5S)|✅| 357 | |[Youtube: Agent Trajectory \| LangSmith Evaluation - Part 26](https://youtu.be/pvlT056DAHs?list=PLfaIDFEXuae0um8Fj0V4dHG37fGFU8Q5S)|✅| 358 | |[Youtube: Evaluating Agents and Assistants: The AI Conference](https://www.youtube.com/watch?v=6uXWhmDRcMc)|✅| 359 | |[Youtube: How to Build, Evaluate, and Iterate on LLM Agents](https://youtu.be/0pnEUAwoDP0)|✅| 360 | 361 | 362 | #### Prompt Engineering 363 | 364 | |Resource|Progress| 365 | |---|---| 366 | |[Article: OpenAI Prompt Engineering](https://platform.openai.com/docs/guides/prompt-engineering)|⬜| 367 | |[Article: Prompting Fundamentals and How to Apply them Effectively](https://eugeneyan.com/writing/prompting/)|✅| 368 | |[Article: How I came in first on ARC-AGI-Pub using Sonnet 3.5 with Evolutionary Test-time Compute](https://params.com/@jeremy-berman/arc-agi)|✅| 369 | |[Anthropic Courses](https://github.com/anthropics/courses)|⬜| 370 | |[Anthropic: The Claude in Amazon Bedrock Course](https://www.anthropic.com/aws-reinvent-2024/course)|⬜| 371 | |[Article: Prompt Engineering(Liliang Weng)](https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/)|✅| 372 | |[Article: Prompt Engineering 201: Advanced methods and toolkits](https://amatria.in/blog/prompt201)|✅| 373 | |[Article: Optimizing LLMs for accuracy](https://platform.openai.com/docs/guides/optimizing-llm-accuracy)|✅| 374 | |[Article: Primers • Prompt Engineering](https://aman.ai/primers/ai/prompt-engineering/)|⬜| 375 | |[Article: Anyscale Endpoints: JSON Mode and Function calling Features](https://www.anyscale.com/blog/anyscale-endpoints-json-mode-and-function-calling-features)|⬜| 376 | |[Article: Guided text generation with Large Language Models](https://medium.com/productizing-language-models/guided-text-generation-with-large-language-models-d88fc3dcf4c)|⬜| 377 | |[Book: Prompt Engineering for LLMs](https://www.oreilly.com/library/view/prompt-engineering-for/9781098156145/)|⬜| 378 | |[DeepLearning.AI: Reasoning with o1](https://www.deeplearning.ai/short-courses/reasoning-with-o1/)|✅| 379 | |[OpenAI: Reasoning with o1 Build Hour](https://vimeo.com/showcase/11333741/video/1018737829)|✅| 380 | |[DeepLearning.AI: ChatGPT Prompt Engineering for Developers](https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/)|⬜| 381 | |[DeepLearning.AI: Prompt Engineering with Llama 2 & 3](https://www.deeplearning.ai/short-courses/prompt-engineering-with-llama-2/)|⬜| 382 | |[Wandb: LLM Engineering: Structured Outputs](https://www.wandb.courses/courses/steering-language-models)|⬜| 383 | |[Series: Prompt injection](https://simonwillison.net/series/prompt-injection/)|⬜| 384 | |[Youtube: Prompt Engineering Overview](https://www.youtube.com/watch?v=dOxUroR57xs) `1hr4m`|✅| 385 | |[Youtube: Prompt Engineering Workshop](https://youtu.be/htBTho6oEJA) `1h`|✅| 386 | 387 | #### Quantization 388 | |Resource|Progress| 389 | |---|---| 390 | |[Article: Quantization Fundamentals with Hugging Face](https://www.deeplearning.ai/short-courses/quantization-fundamentals-with-hugging-face/)|✅| 391 | |[DeepLearning.AI: Quantization in Depth](https://www.deeplearning.ai/short-courses/quantization-in-depth/)|⬜| 392 | |[DeepLearning.AI: Introduction to On-Device AI](https://www.deeplearning.ai/short-courses/introduction-to-on-device-ai/)|⬜| 393 | |[Article: A Visual Guide to Quantization](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-quantization)|⬜| 394 | |[Article: QLoRA and 4-bit Quantization](https://mccormickml.com/2024/09/14/qlora-and-4bit-quantization/)|⬜| 395 | |[Article: Understanding AI/LLM Quantisation Through Interactive Visualisations](https://smcleod.net/2024/07/understanding-ai/llm-quantisation-through-interactive-visualisations/)|⬜| 396 | |[Youtube: CMU Advanced NLP Fall 2024 (11): Distillation, Quantization, and Pruning](https://www.youtube.com/watch?v=DvVGkj4zhVU&list=PL8PYTP1V4I8D4BeyjwWczukWq9d8PNyZp&index=5)|⬜| 397 | |[Article: LLM.int8() and Emergent Features](https://timdettmers.com/2022/08/17/llm-int8-and-emergent-features/)|⬜| 398 | 399 | #### Distributed Training 400 | 401 | |Resource|Progress| 402 | |---|---| 403 | |[Youtube: Slaying OOMs with PyTorch FSDP and torchao](https://youtu.be/UvRl4ansfCg) `49m`|✅| 404 | |[Youtube: Distributed Training with PyTorch: complete tutorial with cloud infrastructure and code](https://youtu.be/toUSzwR0EV8) `1h12m`|✅| 405 | |[Youtube: How DDP works \|\| Distributed Data Parallel ](https://youtu.be/bwNtfxEDjGA)|✅| 406 | |[Youtube: FSDP Explained](https://youtu.be/6pVn6khIgiI)|✅| 407 | |[Youtube: Lecture 48: The Ultra Scale Playbook](https://youtu.be/1E8GDR8QXKw) `3h3m`|`44:24`| 408 | |[Youtube: Invited Talk: PyTorch Distributed (DDP, RPC) - By Facebook Research Scientist Shen Li](https://youtu.be/3XUG7cjte2U)|✅| 409 | |[Youtube: Unit 9 \| Techniques for Speeding Up Model Training](https://www.youtube.com/playlist?list=PLaMu-SDt_RB403GN5DU7NYVoVmO5Vsgkh)|✅| 410 | |[Article: A Short Guide to PyTorch DDP](https://blog.hpc.qmul.ac.uk/pytorch-ddp/)|✅| 411 | |[Article: Scaling Deep Learning with PyTorch: Multi-Node and Multi-GPU Training Explained (with Code)](https://medium.com/@ashraf.kasem.94.0/scaling-deep-learning-with-pytorch-multi-node-and-multi-gpu-training-explained-with-code-ece8f03ea59b)|✅| 412 | |[Article: Accelerating PyTorch Model Training](https://magazine.sebastianraschka.com/p/accelerating-pytorch-model-training)|✅| 413 | |[Article: Meet Horovod: Uber’s Open Source Distributed Deep Learning Framework for TensorFlow](https://www.uber.com/blog/horovod/)|✅| 414 | |[Article: Distributed data parallel training in Pytorch](https://yangkky.github.io/2019/07/08/distributed-pytorch-tutorial.html)|✅| 415 | |[Article: Training on Multiple GPUs](https://d2l.ai/chapter_computational-performance/multiple-gpus.html)|✅| 416 | 417 | 418 | #### Parallel Computing 419 | 420 | |Resource|Progress| 421 | |---|---| 422 | |[Udacity: Intro to Parallel Programming](https://www.youtube.com/playlist?list=PLAwxTw4SYaPnFKojVQrmyOGFCqHTxfdv2) `458 videos`|299/458| 423 | |[Book: Programming Massively Parallel Processors: A Hands-on Approach](https://www.amazon.com/Programming-Massively-Parallel-Processors-Hands/dp/0124159923)|Ch. 2| 424 | |[Youtube: GPU Puzzles: Let's Play](https://youtu.be/K4T-YwsOxrM)|⬜| 425 | 426 | #### Inference Optimization 427 | 428 | |Resource|Progress| 429 | |---|---| 430 | |[Article: How to make LLMs go fast](https://vgel.me/posts/faster-inference/)|✅| 431 | |[Article: In the Fast Lane! Speculative Decoding - 10x Larger Model, No Extra Cost](https://docs.titanml.co/blog/speculative-decoding-unleashed/)|⬜| 432 | |[Article: Accelerating Generative AI with PyTorch II: GPT, Fast](https://pytorch.org/blog/accelerating-generative-ai-2/)|⬜| 433 | |[Article: Harmonizing Multi-GPUs: Efficient Scaling of LLM Inference](https://docs.titanml.co/blog/multi-gpu/)|⬜| 434 | |[Article: Multi-Query Attention is All You Need](https://fireworks.ai/blog/multi-query-attention-is-all-you-need)|⬜| 435 | |[Article: Transformers Inference Optimization Toolset](https://astralord.github.io/posts/transformer-inference-optimization-toolset/)|⬜| 436 | |[DeepLearning.AI: Efficiently Serving LLMs](https://www.deeplearning.ai/short-courses/efficiently-serving-llms/)|✅| 437 | |[Article: LLM Inference Series: 3. KV caching explained](https://medium.com/@plienhar/llm-inference-series-3-kv-caching-unveiled-048152e461c8)|⬜| 438 | |[Article: LLM Inference Series: 4. KV caching, a deeper look](https://medium.com/@plienhar/llm-inference-series-4-kv-caching-a-deeper-look-4ba9a77746c8)|⬜| 439 | |[Article: LLM Inference Series: 5. Dissecting model performance](https://medium.com/@plienhar/llm-inference-series-5-dissecting-model-performance-6144aa93168f)|⬜| 440 | |[Article: Transformer Inference Arithmetic](https://kipp.ly/transformer-inference-arithmetic/)|⬜| 441 | |[Article: Optimizing AI Inference at Character.AI](https://research.character.ai/optimizing-inference/)|⬜| 442 | |[Article: Optimizing AI Inference at Character.AI (Part Deux)](https://research.character.ai/optimizing-ai-inference-at-character-ai-part-deux/)|⬜| 443 | |[Article: llama.cpp guide - Running LLMs locally, on any hardware, from scratch](https://blog.steelph0enix.dev/posts/llama-cpp-guide/)|✅| 444 | |[Youtube: SBTB 2023: Charles Frye, Parallel Processors: Past & Future Connections Between LLMs and OS Kernels](https://www.youtube.com/watch?v=VxFtHqlMv8c)|✅| 445 | |[Youtube: Deploying Fine-Tuned Models](https://youtu.be/GzEcyBykkdo) `2h28m`|✅| 446 | |[Article: Compiling ML models to C for fun](https://bernsteinbear.com/blog/compiling-ml-models/)|⬜| 447 | |[Article: How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklog](https://siboehm.com/articles/22/CUDA-MMM)|⬜| 448 | 449 | #### Evals and Guardrails 450 | 451 | |Resource|Progress| 452 | |---|---| 453 | |[Article: Your AI Product Needs Evals](https://hamel.dev/blog/posts/evals)|✅| 454 | |[Article: Task-Specific LLM Evals that Do & Don't Work](https://eugeneyan.com/writing/evals/)|✅| 455 | |[Article: Evaluation & Hallucination Detection for Abstractive Summaries](https://eugeneyan.com/writing/abstractive/)|✅| 456 | |[Article: LLM-as-a-Judge vs Human Evaluation](https://www.galileo.ai/blog/llm-as-a-judge-vs-human-evaluation)|⬜| 457 | |[DeepLearning.AI: Automated Testing for LLMOps](https://www.deeplearning.ai/short-courses/automated-testing-llmops/)|✅| 458 | |[DeepLearning.AI: Red Teaming LLM Applications](https://www.deeplearning.ai/short-courses/red-teaming-llm-applications/)|✅| 459 | |[DeepLearning.AI: Evaluating and Debugging Generative AI Models Using Weights and Biases](https://www.deeplearning.ai/short-courses/evaluating-debugging-generative-ai/)|⬜| 460 | |[DeepLearning.AI: Quality and Safety for LLM Applications](https://www.deeplearning.ai/short-courses/quality-safety-llm-applications/)|⬜| 461 | |[OpenAI: Evals Build Hour](https://vimeo.com/showcase/11333741/video/1023317525)|✅| 462 | |[Youtube: Instrumenting & Evaluating LLMs](https://youtu.be/SnbGD677_u0) `2hr33m`|✅| 463 | |[Youtube: LLM Eval For Text2SQL](https://youtu.be/UGmenkjGXqM?list=PLgIaq8VgndJvt-HKMHPXehyJNNXQsAVHD) `51m`|✅| 464 | |[Youtube: A Deep Dive on LLM Evaluation](https://youtu.be/IsZVCnViwhk?list=PLgIaq8VgndJvt-HKMHPXehyJNNXQsAVHD) `49m`|✅| 465 | 466 | ### Finetuning and Distillation 467 | 468 | |Resource|Progress| 469 | |---|---| 470 | |[Article: Tokenization Gotchas](https://hamel.dev/notes/llm/finetuning/tokenizer_gotchas.html)|⬜| 471 | |[Article: Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)](https://magazine.sebastianraschka.com/p/practical-tips-for-finetuning-llms)|⬜| 472 | |[OpenAI: GPT-4o mini Fine-Tuning Build Hour](https://vimeo.com/showcase/11333741/video/995989828)|✅| 473 | |[OpenAI: Distillation Build Hour](https://vimeo.com/showcase/11333741/video/1029408095)|✅| 474 | |[Article: How to Generate and Use Synthetic Data for Finetuning](https://eugeneyan.com/writing/synthetic/)|✅| 475 | |[DeepLearning.AI: Finetuning Large Language Models](https://www.deeplearning.ai/short-courses/finetuning-large-language-models/)|✅| 476 | |[Youtube: Fine-Tuning with Axolotl](https://youtu.be/mmsa4wDsiy0?list=PLgIaq8VgndJtZ_G6gxyuhHGLUy9zXV9JC) `2h10m`|✅| 477 | |[Youtube: Creating, Curating, and Cleaning Data for LLMs](https://youtu.be/HEGaei7k0zE?list=PLgIaq8VgndJtZ_G6gxyuhHGLUy9zXV9JC) `54m`|✅| 478 | |[Youtube: Best Practices For Fine Tuning Mistral](https://youtu.be/Z_oWzTuljss?list=PLgIaq8VgndJtZ_G6gxyuhHGLUy9zXV9JC) `23m`|✅| 479 | |[Youtube: Fine Tuning OpenAI Models - Best Practices](https://youtu.be/Q0GSZD0Na1s?list=PLgIaq8VgndJtZ_G6gxyuhHGLUy9zXV9JC)|✅| 480 | |[Youtube: When and Why to Fine Tune an LLM](https://youtu.be/cPn0nHFsvFg) `1h56m`|✅| 481 | |[Youtube: Napkin Math For Fine Tuning Pt. 1 w/Johno Whitaker](https://youtu.be/-2ebSQROew4)|✅| 482 | |[Youtube: Napkin Math For Fine Tuning Pt. 2 w/Johno Whitaker](https://youtu.be/u2fJ6K8FjS8)|✅| 483 | |[Youtube: Fine Tuning LLMs for Function Calling w/Pawel Garback](https://youtu.be/SEZ7j31u67A) `1h32m`|✅| 484 | |[Youtube: From Prompt to Model: Fine-tuning when you've already deployed LLMs in prod w/Kyle Corbitt](https://youtu.be/4EPZZkVrXC4) `32m`|✅| 485 | |[Youtube: Why Fine Tuning is Dead w/Emmanuel Ameisen](https://youtu.be/h1c_jmk97Ss) `50m`|✅| 486 | |[Benchmarking QLoRA+FSDP](https://github.com/AnswerDotAI/fsdp_qlora/blob/main/benchmarks_03_2024.md)|⬜| 487 | 488 | #### LLM System Design 489 | 490 | |Resource|Progress| 491 | |---|---| 492 | |[Article: What We’ve Learned From A Year of Building with LLMs](https://applied-llms.org/)|⬜| 493 | |[Article: Data Flywheels for LLM Applications](https://www.sh-reya.com/blog/ai-engineering-flywheel/)|⬜| 494 | |[Article: LLM From the Trenches: 10 Lessons Learned Operationalizing Models at GoDaddy](https://www.godaddy.com/resources/news/llm-from-the-trenches-10-lessons-learned-operationalizing-models-at-godaddy#h-3-prompts-aren-t-portable-across-models)|✅| 495 | |[Article: Emerging UX Patterns for Generative AI Apps & Copilots](https://www.tidepool.so/blog/emerging-ux-patterns-for-generative-ai-apps-copilots)|✅| 496 | |[Article: The Novice's LLM Training Guide](https://rentry.co/llm-training)|⬜| 497 | |[Article: Pushing ChatGPT's Structured Data Support To Its Limits](https://minimaxir.com/2023/12/chatgpt-structured-data/)|✅| 498 | |[Article: GPTed: using GPT-3 for semantic prose-checking](https://vgel.me/posts/gpted-launch/)|✅| 499 | |[Article: Don't worry about LLMs](https://vickiboykis.com/2024/05/20/dont-worry-about-llms/)|⬜| 500 | |[Article: Things we learned about LLMs in 2024](https://simonwillison.net/2024/Dec/31/llms-in-2024/)|⬜| 501 | |[Article: Data acquisition strategies for AI-first start-ups](https://press.airstreet.com/p/data-acquisition-strategies-for-ai?utm_source=substack&utm_medium=email)|⬜| 502 | |[DeepLearning.AI: Building Systems with the ChatGPT API](https://www.deeplearning.ai/short-courses/building-systems-with-chatgpt/)|⬜| 503 | |[DeepLearning.AI: Building Generative AI Applications with Gradio](https://www.deeplearning.ai/short-courses/building-generative-ai-applications-with-gradio/)|✅| 504 | |[DeepLearning.AI: Open Source Models with Hugging Face](https://www.deeplearning.ai/short-courses/open-source-models-hugging-face/)|⬜| 505 | |[DeepLearning.AI: Getting Started with Mistral](https://www.deeplearning.ai/short-courses/getting-started-with-mistral/)|⬜| 506 | |[LLMOps: Building with LLMs](https://www.comet.com/site/llm-course/)|⬜| 507 | |[LLM Bootcamp - Spring 2023](https://fullstackdeeplearning.com/llm-bootcamp/spring-2023/)|✅| 508 | |[Youtube: A Survey of Techniques for Maximizing LLM Performance](https://www.youtube.com/watch?v=ahnGLM-RC1Y)|✅| 509 | |[Youtube: Building Blocks for LLM Systems & Products: Eugene Yan](https://www.youtube.com/watch?v=LzeC1AQ-U5o)|✅| 510 | |[Youtube: Building LLM Applications](https://www.youtube.com/playlist?list=PLgIaq8VgndJtrxcelEdnXbvh9fXMHeAps)|0/8| 511 | |[Article: Emerging Architectures for LLM Applications](https://a16z.com/emerging-architectures-for-llm-applications/)|✅| 512 | |[Article: Patterns for Building LLM-based Systems & Products](https://eugeneyan.com/writing/llm-patterns/)|✅| 513 | |[DeepLearning.AI: LLMOps](https://www.deeplearning.ai/short-courses/llmops/)|⬜| 514 | |[DeepLearning.AI: Serverless LLM apps with Amazon Bedrock](https://www.deeplearning.ai/short-courses/serverless-llm-apps-amazon-bedrock/)|⬜| 515 | |[Youtube: Getting the Most Out of Your LLM Experiments](https://youtu.be/IfcDvtl6Z1Y) `48m`|✅| 516 | 517 | 518 | ## Technical Skills (Libraries/Frameworks/Tools) 519 | 520 | ### AWS 521 | 522 | 523 | |Resource|Progress| 524 | |---|---| 525 | |[Udemy: AWS Certified Developer - Associate 2018](https://www.udemy.com/aws-certified-developer-associate/)|✅| 526 | 527 | ### CSS 528 | 529 | |Resource|Progress| 530 | |---|---| 531 | |[Pluralsight: CSS Positioning](https://www.pluralsight.com/courses/css-positioning-1834)|✅| 532 | |[Pluralsight: Introduction to CSS](https://www.pluralsight.com/courses/css-intro)|✅| 533 | |[Pluralsight: CSS: Specificity, the Box Model, and Best Practices](https://app.pluralsight.com/interactive-courses/detail/c580b092-d94a-4ed8-8d2a-2f4d0b76f99f)|✅| 534 | |[Pluralsight: CSS: Using Flexbox for Layout](https://app.pluralsight.com/interactive-courses/detail/a089d0a5-4a4c-4c4e-b883-c1bc64009619)|✅| 535 | |[Code School: Blasting Off with Bootstrap](https://www.pluralsight.com/courses/code-school-blasting-off-with-bootstrap)|✅| 536 | |[Pluralsight: UX Fundamentals](https://www.pluralsight.com/courses/ux-fundamentals-2426)|✅| 537 | |[Codecademy: Learn SASS](https://www.codecademy.com/learn/learn-sass)|✅| 538 | |[CSS for Javascript Developers](https://css-for-js.dev/)|✅| 539 | |[Article: Create an illustration in Figma design](https://help.figma.com/hc/en-us/articles/13543867954711-Create-an-illustration-in-Figma-design)|✅| 540 | |[Book: Refactoring UI](https://refactoringui.com/book/)|⬜| 541 | |[Youtube: How to Make Your Website Not Ugly: Basic UX for Programmers](https://www.youtube.com/watch?v=Jf0cjocP8Wk) `48m`|⬜| 542 | 543 | 544 | 545 | ### Django 546 | 547 | |Resource|Progress| 548 | |---|---| 549 | |[Article: Django, HTMX and Alpine.js: Modern websites, JavaScript optional](https://www.saaspegasus.com/guides/modern-javascript-for-django-developers/htmx-alpine/)|✅| 550 | 551 | ### HTML 552 | 553 | |Resource|Progress| 554 | |---|---| 555 | |[Codecademy: Learn HTML](https://www.codecademy.com/learn/learn-html)|✅| 556 | |[Codecademy: Make a website](https://www.codecademy.com/en/courses/make-a-website)|✅| 557 | |[Article: Alternative Text](https://webaim.org/techniques/alttext/)|⬜| 558 | 559 | ### Langchain 560 | 561 | |Resource|Progress| 562 | |---|---| 563 | |[Pinecone: LangChain AI Handbook](https://www.pinecone.io/learn/series/langchain/)|0/11| 564 | |[DeepLearning.AI: LangChain for LLM Application Development](https://www.deeplearning.ai/short-courses/langchain-for-llm-application-development/)|⬜| 565 | |[DeepLearning.AI: LangChain: Chat with Your Data](https://www.deeplearning.ai/short-courses/langchain-chat-with-your-data/)|⬜| 566 | 567 | 568 | ### JavaScript 569 | 570 | |Resource|Progress| 571 | |---|---| 572 | |[Udacity: ES6 - JavaScript Improved](https://www.udacity.com/course/es6-javascript-improved--ud356)|✅| 573 | |[Udacity: Intro to Javascript](https://www.udacity.com/course/intro-to-javascript--ud803)|✅| 574 | |[Udacity: Object Oriented JS 1](https://www.udacity.com/course/object-oriented-javascript--ud015)|✅| 575 | |[Udacity: Object Oriented JS 2](https://www.udacity.com/course/object-oriented-javascript--ud711)|✅| 576 | |[Udemy: Understanding Typescript](https://www.udemy.com/understanding-typescript/)|✅| 577 | |[Codecademy: Learn JavaScript](https://www.codecademy.com/learn/learn-javascript)|✅| 578 | |[Codecademy: Jquery Track](https://www.codecademy.com/learn/learn-jquery)|✅| 579 | |[Pluralsight: Using The Chrome Developer Tools](https://www.pluralsight.com/courses/chrome-developer-tools)|✅| 580 | 581 | 582 | ### Matplotlib 583 | 584 | |Resource|Progress| 585 | |---|---| 586 | |[Datacamp: Introduction to Seaborn](https://www.datacamp.com/courses/introduction-to-seaborn)|✅| 587 | |[Datacamp: Introduction to Matplotlib](https://www.datacamp.com/courses/introduction-to-matplotlib)|✅| 588 | 589 | 590 | ### MLFlow 591 | 592 | |Resource|Progress| 593 | |---|---| 594 | |[Datacamp: Introduction to MLFlow](https://www.datacamp.com/courses/introduction-to-mlflow)|✅| 595 | 596 | 597 | ### Numpy 598 | 599 | |Resource|Progress| 600 | |---|---| 601 | |[Youtube: Numpy Array Broadcasting In Python Explained](https://youtu.be/oG1t3qlzq14)|✅| 602 | 603 | 604 | ### Nexxt.JS 605 | 606 | | Resource | Progress | 607 | | ----------------------------------------------------------------- | -------- | 608 | | [Docs: Start building with Next.js](https://nextjs.org/learn) | | 609 | 610 | ### Pandas 611 | 612 | |Resource|Progress| 613 | |---|---| 614 | |[Datacamp: Pandas Foundations](https://www.datacamp.com/courses/pandas-foundations)|✅| 615 | |[Datacamp: Pandas Joins for Spreadsheet Users](https://www.datacamp.com/courses/pandas-joins-for-spreadsheet-users)|✅| 616 | |[Datacamp: Manipulating DataFrames with pandas](https://www.datacamp.com/courses/manipulating-dataframes-with-pandas)|✅| 617 | |[Datacamp: Merging DataFrames with pandas](https://www.datacamp.com/courses/merging-dataframes-with-pandas)|✅| 618 | |[Datacamp: Data Manipulation with pandas](https://www.datacamp.com/courses/data-manipulation-with-pandas)|✅| 619 | |[Datacamp: Optimizing Python Code with pandas](https://www.datacamp.com/courses/optimizing-python-code-with-pandas)|✅| 620 | |[Datacamp: Streamlined Data Ingestion with pandas](https://www.datacamp.com/courses/streamlined-data-ingestion-with-pandas)|✅| 621 | |[Datacamp: Analyzing Marketing Campaigns with pandas](https://www.datacamp.com/courses/analyzing-marketing-campaigns-with-pandas)|✅| 622 | |[Datacamp: Analyzing Police Activity with pandas](https://www.datacamp.com/courses/analyzing-police-activity-with-pandas)|✅| 623 | 624 | 625 | ### PyTorch 626 | 627 | |Resource|Progress| 628 | |---|---| 629 | |[Article: PyTorch internals](https://blog.ezyang.com/2019/05/pytorch-internals/)|⬜| 630 | |[Article: Taking PyTorch For Granted](https://nrehiew.github.io/blog/pytorch/)|⬜| 631 | |[Datacamp: Introduction to Deep Learning with PyTorch](https://www.datacamp.com/courses/deep-learning-with-pytorch)|✅| 632 | |[Datacamp: Intermediate Deep Learning with PyTorch](https://app.datacamp.com/learn/courses/intermediate-deep-learning-with-pytorch)|⬜| 633 | |[Datacamp: Deep Learning for Text with PyTorch](https://www.datacamp.com/courses/deep-learning-for-text-with-pytorch)|⬜| 634 | |[Datacamp: Deep Learning for Images with PyTorch](https://www.datacamp.com/courses/deep-learning-for-images-with-pytorch)|⬜| 635 | |[Deeplizard: Neural Network Programming - Deep Learning with PyTorch](https://www.youtube.com/playlist?list=PLZbbT5o_s2xrfNyHZsM6ufI0iZENK9xgG)|✅| 636 | 637 | 638 | ### ReactJS 639 | 640 | |Resource|Progress| 641 | |---|---| 642 | |[Codecademy: Learn ReactJS: Part I](https://www.codecademy.com/learn/react-101)|✅| 643 | |[Codecademy: Learn ReactJS: Part II](https://www.codecademy.com/learn/react-102)|✅| 644 | |[NexxtJS: React Foundations](https://nextjs.org/learn/react-foundations)|⬜| 645 | 646 | ### Spacy 647 | 648 | |Resource|Progress| 649 | |---|---| 650 | |[Datacamp: Advanced NLP with spaCy](https://www.datacamp.com/courses/advanced-nlp-with-spacy)|✅| 651 | 652 | ### Tensorflow & Keras 653 | 654 | |Resource|Progress| 655 | |---|---| 656 | |[Datacamp: Introduction to TensorFlow in Python](https://www.datacamp.com/courses/introduction-to-tensorflow-in-python)|✅| 657 | |[Datacamp: Deep Learning in Python](https://www.datacamp.com/courses/deep-learning-in-python)|✅| 658 | |[Datacamp: Introduction to Deep Learning with Keras](https://www.datacamp.com/courses/deep-learning-with-keras-in-python)|✅| 659 | |[Datacamp: Advanced Deep Learning with Keras](https://www.datacamp.com/courses/advanced-deep-learning-with-keras-in-python)|✅| 660 | |[Deeplizard: Keras - Python Deep Learning Neural Network API](https://www.youtube.com/playlist?list=PLZbbT5o_s2xrwRnXk_yCPtnqqo4_u2YGL)|✅| 661 | |[Udacity: Intro to TensorFlow for Deep Learning](https://www.udacity.com/course/intro-to-tensorflow-for-deep-learning--ud187)|✅| 662 | 663 | ### VSCode 664 | 665 | |Resource|Progress| 666 | |---|---| 667 | |[VSCode Docs: Python Interactive window](https://code.visualstudio.com/docs/python/jupyter-support-py)|⬜| 668 | 669 | ## Miscellaneous 670 | 671 | ### Design 672 | 673 | |Resource|Progress| 674 | |---|---| 675 | |[Course: How to Visualize Value](https://visualizevalue.com/products/how-to-visualize-value)|✅| 676 | 677 | ### Finance 678 | 679 | |Resource|Progress| 680 | |---|---| 681 | |[Coursera: Financial Markets](https://www.coursera.org/learn/financial-markets-global)|⬜| 682 | 683 | 684 | 685 | ### Marketing 686 | 687 | |Resource|Progress| 688 | |---|---| 689 | |[Course: Build Once, Sell Twice](https://visualizevalue.com/products/build-once-sell-twice-the-productization-playbook)|✅| 690 | 691 | ### Search Engine Optimization (SEO) 692 | 693 | |Resource|Progress| 694 | |---|---| 695 | |[Course: Compound Content](https://visualizevalue.com/products/compound-content)|✅| 696 | 697 | ### Technical Writing 698 | |Resource|Progress| 699 | |---|---| 700 | |[Google: Technical Writing Course](https://developers.google.com/tech-writing/overview)|⬜| --------------------------------------------------------------------------------