├── LICENSE └── README.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Snehil Sanyal 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 🏫🙋🏻Self Learn LLMs 2 | 3 | This repository constitutes some of the resources which I will use to learn about Large Language Models. I will also try to come up with a roadmap as I go forward in this self-learning journey, since a clear roadmap with milestones will be one of the best ways to learn about LLMs in a proper manner. 4 | 5 | For this, I will include a mix of theoretical and practical hands-on resources to learn. 6 | 7 | **PS:** Need to make this more visual 8 | 9 | **Edit: 6th Nov 2024** 10 | - [ ] Need to majorly refactor the repository. 11 | - [ ] Remove courses or resources that are not relevant. 12 | - [ ] In the model race, we can't go on listing models in the repository as it's tough to keep track of models and updates will make the previous models useless. Need to think of a better way to organize model zoo. 13 | - [ ] Maybe add 1-2 contributors or open the repository to contributions to help out. 14 | - [ ] How can we make it a gret learning experience, hosting notes and tutorials, open discussions, webpage? 15 | 16 | ## Courses 17 | 18 | ### 📝🗒️ NLP Fundamentals 19 | 20 | 1. [CS224N Natural Language Processing with Deep Learning, Stanford](https://web.stanford.edu/class/cs224n/) 21 | 2. [Natural Language Processing Specialization, Coursera](https://www.coursera.org/specializations/natural-language-processing) 22 | 23 | ### 🤖 Transformers 24 | 25 | 1. [HuggingFace LLM Course](https://huggingface.co/learn/llm-course/chapter1/1) 26 | 2. [CS25: Transformers United V2, Stanford](https://web.stanford.edu/class/cs25/) [CS25, Fall 2021 Version](https://web.stanford.edu/class/cs25/prev_years/2021_fall/) 27 | 28 | ### 🦜🗣️ Large Language Models 29 | 30 |
31 | Industrial and Open-Source courses 32 | 33 | 1. [Activeloop Learn](https://learn.activeloop.ai/), this initiative GenAI360 provides 3 free courses on RAGs, fine-tuning LLMs, LangChain and VectorDBs. 34 | 2. [LLM Course by Maxime Labonne](https://github.com/mlabonne/llm-course), Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks. 35 | 3. [Hands on LLM Course](https://github.com/iusztinpaul/hands-on-llms), Learn about LLMs, LLMOps, and Vector DBs for free by designing, training, and deploying a real-time financial advisor LLM system source code + video & reading materials. 36 | 4. [Full Stack Deep Learning](https://fullstackdeeplearning.com/llm-bootcamp/), started out as a deep learning bootcamp and evolved into LLM bootcamp around April 2023, now is free to take up. 37 | 5. [LLM University by Cohere](https://docs.cohere.com/docs/llmu), this course consists of 8 modules taught by the famous Luis Serrano, who is known for teaching concepts in a easy and visually appealing manner. The course contains topics like fundamentals, deployment, semantic search and RAG. 38 | 6. [Deeplearning.ai Short Courses](https://www.deeplearning.ai/courses/), Short courses by DL.AI on various domains of LLMs and Generative AI. These short courses are really useful as they have the perfect blend of theoretical and practical sessions. The courses are usually made in collaborations with companies like Hugging Face, Mistral, OpenAI, Microsoft, Meta, Google etc. 39 | 7. [LLM Zoomcamp by DataTalksClub](https://github.com/DataTalksClub/llm-zoomcamp), LLM Zoomcamp - a free online course about building a Q&A system. 40 | 8. [Applied LLMs Mastery 2024 Course by Aishwarya N Reganti](https://github.com/aishwaryanr/awesome-generative-ai-guide/tree/main/free_courses/Applied_LLMs_Mastery_2024), free 10 weeks course with a definite roadmap ranging from LLM Fundamentals, Tools and techniques, Deployment and evaluation to Challenges and future trends. 41 | 9. [Weights and Biases Courses](https://www.wandb.courses/collections), provides different courses on MLOps, LLM Powered Apps etc. 42 | 10. [LLM Models course, DataBricks x ed](https://www.edx.org/certificates/professional-certificate/databricks-large-language-models), professional certification by DataBricks. 43 | 11. [Deeplearning.ai](https://www.deeplearning.ai/short-courses/) offers various short courses on LLMs like LangChain for LLM App Development, Serverless LLMs with AWS Bedrock, Fine-tuning LLMs, LLMs with Semantic Search etc. 44 | 12. [Introduction to Generative AI Learning Path, Google Cloud](https://www.cloudskillsboost.google/paths/118). 45 | 13. [Arize University](https://courses.arize.com/courses/) hosts courses like llm-evaluation, llm agents tools and chains, llm-observability etc. 46 | 47 |
48 | 49 | 50 |
51 | University Courses 52 | 53 | 1. [CS 324, Stanford](https://stanford-cs324.github.io/winter2022/) 54 | 2. [COMP790-101: Large Language Models, UNC Chapel Hill](https://github.com/craffel/llm-seminar) 55 | 3. [Large Language Models S-23, ETH Zurich](https://rycolab.io/classes/llm-s23/) 56 | 4. [Foundations of Large Language Models, University of Waterloo](https://uwaterloo.ca/watspeed/programs-and-courses/foundations-large-language-models) 57 | 58 |
59 | 60 | 61 | ## 📕📗📙 Books 62 | 63 | 1. [Natural Language Processing with Transformers Book, by Lewis Tunstall, Leandro von Werra and Thomas Wolf](https://transformersbook.com/) | [Notebooks for the book](https://github.com/nlp-with-transformers/notebooks) 64 | 2. [Build a Large Language Model (From Scratch), by Sebastian Raschka](https://www.manning.com/books/build-a-large-language-model-from-scratch) | [Official Code Repository for the book](https://github.com/rasbt/LLMs-from-scratch) 65 | 3. [LLM Engineer's Handbook_ Master the art of engineering large language models from concept to production, by Paul Iusztin and Maxime Labonne ](https://www.packtpub.com/en-in/product/llm-engineers-handbook-9781836200062) | [Code for the book](https://github.com/PacktPublishing/LLM-Engineers-Handbook) 66 | 4. [Hands-On Large Language Models: Language Understanding and Generation, by Jay Alammar and Maarten Grootendorst](https://www.oreilly.com/library/view/hands-on-large-language/9781098150952/) | [Code repository for the book](https://github.com/HandsOnLLM/Hands-On-Large-Language-Models) 67 | 5. [AI Engineering: Building Applications with Foundation Models, by Chip Huyen](https://www.oreilly.com/library/view/ai-engineering/9781098166298/) | [Resources for the book](https://github.com/chiphuyen/aie-book) 68 | 69 | ## ✍🏻🛣️ Blogs and Guides 70 | 71 | **[Langchain Blogs](https://blog.langchain.dev/)** 72 | 73 | 1. [AIMultiple's blog on Large Language Models: Complete Guide in 2023](https://research.aimultiple.com/large-language-models/) 74 | 2. [Cohere Docs](https://docs.cohere.ai/docs/introduction-to-large-language-models) 75 | 3. [FutureSmart AI Blog on Building Chatbots using LangChain and ChatGPT](https://blog.futuresmart.ai/building-chatbot-using-langchain-and-chatgpt) 76 | 4. [Task-driven Autonomous Agent Utilizing GPT-4, Pinecone, and LangChain for Diverse Applications](https://yoheinakajima.com/task-driven-autonomous-agent-utilizing-gpt-4-pinecone-and-langchain-for-diverse-applications/) 77 | 78 | 79 | ## 📰 Newsletters 80 | 81 | ## 👨🏻‍💻👨🏻‍💻 Tutorials 82 | 83 | 1. [Let's build GPT: from scratch, in code, spelled out.](https://www.youtube.com/watch?v=kCc8FmEb1nY): 4 hour long YouTube video by Andrej Karpathy on building GPT from scratch. 84 | 2. [[1hr talk] Intro to Large Language Models](https://www.youtube.com/watch?v=zjkBMFhNj_g): 1 hour YouTube video by Andrej Karpathy on Introduction to LLMs. 85 | 3. [Let's build the GPT Tokenizer](https://www.youtube.com/watch?v=zduSFxRajkE): 2 hour long YouTube video by Andrej Karpathy on building the GPT Tokenizer. 86 | 4. [Let's reproduce GPT-2 (124M)](https://www.youtube.com/watch?v=l8pRSuU81PU): 4 hour long YouTube Video by Andrej Karpathy on reproducing GPT-2 (124M model). 87 | 88 | 89 | ## Libraries, Frameworks and Toolkits 90 | 91 | // Pipeline: Data Collection (Scraping) and Datasets, Pre-Training, Fine-Tuning, Deployment, Evaluations, Security, Guardrails. 92 | 93 | 1. [LLMStudio, H2O AI](https://github.com/h2oai/h2o-llmstudio) 94 | 2. [LLamaIndex](https://gpt-index.readthedocs.io/en/latest/index.html#ecosystem) 95 | 3. [NeMo Guardrails, NVIDIA, to prevent hallucinations and add programmable guardrails](https://github.com/NVIDIA/NeMo-Guardrails) 96 | 4. [MLC LLM, Develop optimize and deploy LLMs natively on everyone's devices](https://github.com/mlc-ai/mlc-llm)) 97 | 5. [LaMini LLM](https://github.com/lamini-ai/lamini) 98 | 99 | 100 | ## 🌍🌎🌏 Communities 101 | 102 | ## Usecases 103 | 1. [ShareGPT](https://sharegpt.com/) 104 | 105 | ## Projects and Ideas 106 | 107 | 108 | ## 🧑🏻‍🤝‍🧑🏻 People to Follow 109 | 110 | *People you should definitely follow to keep updated about LLMs. Researchers/Founders/Developers/AI Content Creators involved in LLM production/research/development* 111 | 112 | 1. [Sebastian Raschka](https://x.com/rasbt), he is a legend and will burst your hype-up LLM bubble with his amazing tweets, blogs and tutorials. Subscribe to his newsletter [Ahead of AI](https://magazine.sebastianraschka.com/) 113 | 2. [Andrej Karpathy](https://x.com/karpathy), so this legend worked in Tesla, took a break, started his YouTube channel to teach the fundamentals and blew us all with his amazing video on [implementing GPT from scratch](https://www.youtube.com/watch?v=kCc8FmEb1nY&t=2771s) and finally rejoined OpenAI. I guess you cannot lose a legend :D 114 | 3. [Jay Alammar](https://x.com/jayalammar), yup if you don't know about his ELI blog on Transformers go read that out first and be sure to follow him for updates. 115 | 4. [Tomaz Bratanic](https://x.com/tb_tomaz), he is the author of famous book Graph Algorithms for Data Science, and currently writes great blogs on Medium related to GPT, Langchain and stuff. 116 | 5. [Maxime Labonne](https://github.com/mlabonne) 117 | 6. [Paul Iusztin](https://github.com/iusztinpaul) 118 | --------------------------------------------------------------------------------