├── DataCamp - Python For Data Science Cheat Sheet.pdf ├── Deep Learning Notes.pdf ├── LICENSE ├── LightGBM CheatSheet.jpg ├── README.md └── big-o-cheatsheet.pdf /DataCamp - Python For Data Science Cheat Sheet.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/peggy1502/Amazing-Resources/5c100d6a7f5e926195e2c786eee8cbbf5e773a5d/DataCamp - Python For Data Science Cheat Sheet.pdf -------------------------------------------------------------------------------- /Deep Learning Notes.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/peggy1502/Amazing-Resources/5c100d6a7f5e926195e2c786eee8cbbf5e773a5d/Deep Learning Notes.pdf -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 Peggy Chang 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /LightGBM CheatSheet.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/peggy1502/Amazing-Resources/5c100d6a7f5e926195e2c786eee8cbbf5e773a5d/LightGBM CheatSheet.jpg -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Amazing Resources 2 | List of references and online resources related to data science, machine learning and deep learning. 3 | 4 | 75 | 76 | 77 | 78 | 79 | 80 | # 👍 Courses / Tutorials 81 | - fast.ai (https://www.fast.ai/) 82 | - Walk with fastai (https://walkwithfastai.com/) 83 | - Practical Deep Learning for Coders (https://course.fast.ai/) 84 | - Hugging Face Course (https://huggingface.co/course/chapter1) 85 | - MIT 6.S191: Introduction to Deep Learning (http://introtodeeplearning.com/) 86 | - Stanford University CS231n: Convolutional Neural Networks for Visual Recognition (http://cs231n.stanford.edu/) 87 | - Stanford University CS193p: Developing Applications for iOS using SwiftUI (https://cs193p.sites.stanford.edu/) 88 | - Stanford University CS224N: Natural Language Processing with Deep Learning | Winter 2021 (https://www.youtube.com/playlist?list=PLoROMvodv4rOSH4v6133s9LFPRHjEmbmJ) 89 | - Stanford University CS25: Transformers United (https://web.stanford.edu/class/cs25/index.html) 90 | - Stanford University CS230: Deep Learning (https://cs230.stanford.edu/) 91 | - Yann LeCun’s Deep Learning Course at CDS (https://cds.nyu.edu/deep-learning) 92 | - UC Berkeley - Full Stack Deep Learning (https://fullstackdeeplearning.com/) 93 | - New York University - PyTorch Deep Learning (https://atcold.github.io/pytorch-Deep-Learning/) 94 | - University of Amsterdam - UvA Deep Learning Tutorials! (https://uvadlc-notebooks.readthedocs.io/en/latest/) (https://www.youtube.com/playlist?list=PLdlPlO1QhMiAkedeu0aJixfkknLRxk1nA) 95 | - AI Hub (https://aihub.cloud.google.com/) 96 | - DEEP LEARNING WITH PYTORCH: A 60 MINUTE BLITZ (https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html) 97 | - Lightning Flash (https://lightning-flash.readthedocs.io/en/latest/index.html) 98 | - TensorFlow Tutorials (https://www.tensorflow.org/tutorials/) 99 | - Keras Guide (https://www.tensorflow.org/guide/keras/sequential_model) 100 | - Machine Learning Crash Course with TensorFlow APIs (https://developers.google.com/machine-learning/crash-course/ml-intro) 101 | - Crash Course - MXNet, Gluon (https://mxnet.apache.org/versions/1.8.0/api/python/docs/tutorials/getting-started/crash-course/index.html) 102 | - GluonCV: a Deep Learning Toolkit for Computer Vision (https://cv.gluon.ai/contents.html) 103 | - AutoGluon: AutoML for Text, Image, and Tabular Data (https://auto.gluon.ai/stable/index.html) 104 | - 10 minutes to pandas (https://pandas.pydata.org/pandas-docs/stable/user_guide/) 105 | - XGBoost (https://xgboost.readthedocs.io/en/latest/index.html) 106 | - XGBoost - Notes on Parameter Tuning (https://xgboost.readthedocs.io/en/latest/tutorials/param_tuning.html) 107 | - Kaggle (https://www.kaggle.com/learn) 108 | - The Super Duper NLP Repo (https://notebooks.quantumstat.com/) 109 | - Hugging Face Transformers Notebooks (https://huggingface.co/transformers/master/notebooks.html) 110 | - Hugging FaceCommunity Notebooks (https://huggingface.co/transformers/master/community.html) 111 | - Machine Learning for Beginners (https://github.com/microsoft/ML-For-Beginners) 112 | - BigQuery cookbook (https://support.google.com/analytics/answer/4419694#zippy=%2Cin-this-article) 113 | - PythonAlgos (https://pythonalgos.com/resources/) 114 | - Captum - an open source, extensible library for model interpretability built on PyTorch (https://captum.ai/docs/introduction) 115 | - Pinecone - A managed, cloud-native vector database with a simple API (https://www.pinecone.io/learn/) 116 | - ML YouTube Courses (https://github.com/dair-ai/ML-YouTube-Courses) 117 | - Towhee Codelabs (https://codelabs.towhee.io/) 118 | - Emma Ding - Data Science Resources (https://www.emmading.com/free-data-science-interview-resources) 119 | - NLPlanet - Practical NLP with Python (https://www.nlplanet.org/course-practical-nlp/index.html) 120 | - Haystack (https://haystack.deepset.ai/tutorials) 121 | - Zero to GPT (https://github.com/VikParuchuri/zero_to_gpt) 122 | - Interactive Coding Challenges (https://github.com/donnemartin/interactive-coding-challenges) 123 | - Machine Learning and AI Books (https://mltechniques.com/shop/) 124 | - Google Machine Learning Education (https://developers.google.com/machine-learning) 125 | - https://developers.google.com/machine-learning/guides/deep-learning-tuning-playbook/faq 126 | - Large Language Model Course - by Maxime Labonne (https://github.com/mlabonne/llm-course) 127 | - Parlance Labs - Educational resources on LLMs (https://parlance-labs.com/education/) 128 | - Start Machine Learning in 2024 - Become an expert for free! (https://github.com/louisfb01/start-machine-learning) 129 | - Start with Large Language Models (LLMs) - Become an expert for free! (https://github.com/louisfb01/start-llms) 130 | 131 | 132 | # 👍 Cheat Sheets 133 | - Jerry Hargrove - AWS Cloud Diagrams & Notes (https://www.awsgeek.com/) 134 | - Cheat Sheets for Machine Learning and Data Science - by Aqeel Anwar (https://sites.google.com/view/datascience-cheat-sheets) 135 | - Cheat Sheets for Machine Learning Interview Topics - by Aqeel Anwar (https://medium.com/swlh/cheat-sheets-for-machine-learning-interview-topics-51c2bc2bab4f) 136 | - ML Cheatsheet (https://ml-cheatsheet.readthedocs.io/en/latest/index.html) 137 | - Convolutional Neural Networks cheatsheet (https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-convolutional-neural-networks) 138 | - Deep Learning cheatsheet (https://stanford.edu/~shervine/teaching/cs-229/cheatsheet-deep-learning) 139 | - Probability Cheatsheet (http://www.wzchen.com/probability-cheatsheet/)(https://static1.squarespace.com/static/54bf3241e4b0f0d81bf7ff36/t/55e9494fe4b011aed10e48e5/1441352015658/probability_cheatsheet.pdf) 140 | - Bayesian (https://blogs.kent.ac.uk/jonw/files/2015/04/Puza2005.pdf) 141 | - https://github.com/kailashahirwar/cheatsheets-ai 142 | - pip command options (https://manpages.debian.org/stretch/python-pip/pip.1) 143 | - RAG cheatsheet (https://miro.com/app/board/uXjVNvklNmc=/) 144 | - 28 Jupyter Notebook Tips, Tricks, and Shortcuts (https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/) 145 | 146 | # 👍 AWS / SageMaker 147 | - How do I terminate active resources that I no longer need on my AWS account? (https://aws.amazon.com/premiumsupport/knowledge-center/terminate-resources-account-closure/) 148 | - AWS Resource Hub (https://resources.awscloud.com/) 149 | - AWS Machine Learning University (https://aws.amazon.com/machine-learning/mlu/) 150 | - Boto3 documentation (https://boto3.amazonaws.com/v1/documentation/api/latest/index.html#) 151 | - Amazon SageMaker Python SDK (https://sagemaker.readthedocs.io/en/stable/index.html) 152 | - Learn Python On AWS Workshop (https://learn-to-code.workshop.aws/) 153 | - Sagemaker Immersion Day - Self-Paced Lab (https://sagemaker-immersionday.workshop.aws/en/prerequisites/option2.html) 154 | - Data Engineering Immersion Day (https://catalog.us-east-1.prod.workshops.aws/workshops/976050cc-0606-4b23-b49f-ca7b8ac4b153/en-US) 155 | - QuickSight Workshops (https://catalog.workshops.aws/quicksight/en-US) 156 | - Starter kit for the AWS Deepracer Challenge (https://gitlab.aicrowd.com/deepracer/neurips-2021-aws-deepracer-starter-kit) 157 | - Amazon SageMaker Examples (https://github.com/aws/amazon-sagemaker-examples) 158 | - Amazon SageMaker Examples Notebooks (https://sagemaker-examples.readthedocs.io/en/latest/index.html) 159 | - Amazon SageMaker Course by Chandra Lingam (https://github.com/ChandraLingam/AmazonSageMakerCourse) 160 | - MLOps Workshop with Amazon SageMaker (https://github.com/aws-samples/amazon-sagemaker-mlops-workshop) 161 | - Managed Spot Training and Checkpointing for built-in XGBoost (https://github.com/aws-samples/amazon-sagemaker-managed-spot-training/blob/main/xgboost_built_in_managed_spot_training_checkpointing/xgboost_built_in_managed_spot_training_checkpointing.ipynb) 162 | - The Open Guide to Amazon Web Services (https://github.com/open-guides/og-aws) 163 | - 📺 Amazon SageMaker Technical Deep Dive Series (https://www.youtube.com/watch?v=uQc8Itd4UTs&list=PLhr1KZpdzukcOr_6j_zmSrvYnLUtgqsZz&index=2) 164 | - 📺 Improve Data Science Team Productivity Using Amazon SageMaker Studio - AWS Online Tech Talks (https://www.youtube.com/watch?v=-WzkbdioMJE) 165 | - Optimizing costs for machine learning with Amazon SageMaker (https://aws.amazon.com/blogs/machine-learning/optimizing-costs-for-machine-learning-with-amazon-sagemaker/) 166 | - Choosing the right GPU for deep learning on AWS (https://towardsdatascience.com/choosing-the-right-gpu-for-deep-learning-on-aws-d69c157d8c86) 167 | - Select right ML instances for training and inference jobs (https://pages.awscloud.com/rs/112-TZM-766/images/AL-ML%20for%20Startups%20-%20Select%20the%20Right%20ML%20Instance.pdf) 168 | - Deploy fast and scalable AI with NVIDIA Triton Inference Server in Amazon SageMaker (https://aws.amazon.com/blogs/machine-learning/deploy-fast-and-scalable-ai-with-nvidia-triton-inference-server-in-amazon-sagemaker/) 169 | - Accelerate BERT inference with Hugging Face Transformers and AWS Inferentia (https://huggingface.co/blog/bert-inferentia-sagemaker) 170 | - Static Quantization with Hugging Face `optimum` for ~3x latency improvements (https://www.philschmid.de/static-quantization-optimum)(https://github.com/philschmid/optimum-static-quantization) 171 | - HuggingFace SageMaker Forum (https://discuss.huggingface.co/c/sagemaker/17) 172 | - Using NGC with AWS Setup Guide (https://docs.nvidia.com/ngc/ngc-aws-setup-guide/) 173 | - kubernetes-sagemaker-demos (https://github.com/shashankprasanna/kubernetes-sagemaker-demos) 174 | - A quick guide to distributed training with TensorFlow and Horovod on Amazon SageMaker (https://towardsdatascience.com/a-quick-guide-to-distributed-training-with-tensorflow-and-horovod-on-amazon-sagemaker-dae18371ef6e) 175 | - AWS EC2 User Guide - Connect to your Linux instance (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstances.html) 176 | - How to set up a GPU instance for machine learning on AWS (https://kstathou.medium.com/how-to-set-up-a-gpu-instance-for-machine-learning-on-aws-b4fb8ba51a7c) 177 | - Running Jupyter notebooks with AWS (http://bebi103.caltech.edu.s3-website-us-east-1.amazonaws.com/2020b/content/lessons/lesson_04/aws_usage.html) 178 | - Access remote code in a breeze with JupyterLab via SSH (https://towardsdatascience.com/access-remote-code-in-a-breeze-with-jupyterlab-via-ssh-8c6a9ffaaa8c) 179 | - Setting up Jupyter on the Cloud (https://kiwidamien.github.io/setting-up-jupyter-on-the-cloud.html) 180 | - How to Connect AWS EC2 Instance using Session Manager (https://www.kodyaz.com/aws/connect-aws-ec2-instance-using-session-manager.aspx) 181 | - NVTabular Cloud Integration (https://nvidia-merlin.github.io/NVTabular/main/resources/cloud_integration.html) 182 | - Create real-time clickstream sessions and run analytics with Amazon Kinesis Data Analytics, AWS Glue, and Amazon Athena (https://aws.amazon.com/blogs/big-data/create-real-time-clickstream-sessions-and-run-analytics-with-amazon-kinesis-data-analytics-aws-glue-and-amazon-athena/) 183 | - Identifying and working with sensitive healthcare data with Amazon Comprehend Medical (https://lifesciences-resources.awscloud.com/healthcare-life-sciences-aws-for-industries/identifying-and-working-with-sensitive-healthcare-data-with-amazon-comprehend-medical) 184 | - Amazon Comprehend Medical – Natural Language Processing for Healthcare Customers (https://aws.amazon.com/blogs/aws/amazon-comprehend-medical-natural-language-processing-for-healthcare-customers/) 185 | - Extract and visualize clinical entities using Amazon Comprehend Medical (https://aws.amazon.com/blogs/machine-learning/extract-and-visualize-clinical-entities-using-amazon-comprehend-medical/) 186 | - AWS Simple Workflow vs AWS Step Functions vs Apache Airflow (https://digitalcloud.training/aws-simple-workflow-vs-aws-step-functions-vs-apache-airflow/) 187 | - Is it the end for Apache Airflow? (https://uncledata.medium.com/is-it-the-end-for-apache-airflow-81ef027becf4) 188 | - Scalable data preparation & ML using Apache Spark on AWS (https://github.com/debnsuma/sagemaker-studio-emr-spark) 189 | - Interactively fine-tune Falcon-40B and other LLMs on Amazon SageMaker Studio notebooks using QLoRA (https://aws.amazon.com/blogs/machine-learning/interactively-fine-tune-falcon-40b-and-other-llms-on-amazon-sagemaker-studio-notebooks-using-qlora/) 190 | - Boost your Resume with these Five AWS Projects: Easy, Intermediate, and Expert Levels with Repository Links (https://towardsaws.com/boost-your-resume-with-this-five-aws-projects-easy-intermediate-and-expert-levels-with-6224eef9e2ae) 191 | - Building End-to-End Machine Learning Pipelines with Amazon SageMaker: A Step-by-Step Guide (https://medium.com/anolytics/building-end-to-end-machine-learning-pipelines-with-amazon-sagemaker-a-step-by-step-guide-8531f73b38cd) 192 | - How to optimize AWS Lambda & Kinesis to process 5 million records per minute (https://towardsaws.com/how-to-optimize-aws-lambda-kinesis-to-process-5-million-messages-c3ed5a143c2d) 193 | - Deploying a Trained CTGAN Model on an EC2 Instance: A Step-by-Step Guide (https://tutorialsdojo.com/deploying-a-trained-ctgan-model-on-an-ec2-instance-a-step-by-step-guide/) 194 | - Serverless Model Deployment in AWS: Streamlining with Lambda, Docker, and S3 (https://tutorialsdojo.com/serverless-model-deployment-in-aws-streamlining-with-lambda-docker-and-s3/) 195 | - Demystifying AWS Storage: S3, EBS, and EFS (https://www.linkedin.com/comm/pulse/demystifying-aws-storage-s3-ebs-efs-neal-k-davis-zj84e) 196 | - Amazon AI Fairness and Explainability with Amazon SageMaker Clarify (https://www.linkedin.com/comm/pulse/amazon-ai-fairness-explainability-sagemaker-clarify-jon-bonso-vnfzc) 197 | - Introducing Amazon Kinesis Data Analytics Studio – Quickly Interact with Streaming Data Using SQL, Python, or Scala (https://aws.amazon.com/blogs/aws/introducing-amazon-kinesis-data-analytics-studio-quickly-interact-with-streaming-data-using-sql-python-or-scala/) 198 | - amazon-kinesis-data-generator (https://awslabs.github.io/amazon-kinesis-data-generator/) (https://awslabs.github.io/amazon-kinesis-data-generator/web/help.html) 199 | - Evaluate LLMs with Hugging Face Lighteval on Amazon SageMaker (https://www.philschmid.de/sagemaker-evaluate-llm-lighteval) 200 | - Chaim Rand 201 | - Part 1: Instance Selection for Deep Learning (https://medium.com/towards-data-science/instance-selection-for-deep-learning-7463d774cff0) 202 | - Part 2: Optimizing Instance Type Selection for AI Development in Cloud Spot Markets (https://medium.com/towards-data-science/optimizing-instance-type-selection-for-ai-development-in-cloud-spot-markets-a6e94804e8f3) 203 | - Managing Your Cloud-Based Data Storage with Rclone (https://medium.com/towards-data-science/managing-your-cloud-based-data-storage-with-rclone-32fff991e0b3) 204 | - Using Server-less Functions to Govern and Monitor Cloud-Based Training Experiments (https://medium.com/towards-data-science/using-server-less-functions-to-govern-and-monitor-cloud-based-training-experiments-755c43fba26b) 205 | - Part 1: A Simple Solution for Managing Cloud-Based ML-Training - How to Implement a Custom Training Solution Using Basic (Unmanaged) Cloud Service APIs (https://medium.com/towards-data-science/a-simple-solution-for-managing-cloud-based-ml-training-c80a69c6939a) 206 | - Part 2: How to Implement a Custom Training Solution Based on Amazon EC2 (https://medium.com/towards-data-science/how-to-implement-a-custom-training-solution-based-on-amazon-ec2-c91fcc2b145a) 207 | - Debugging and Tuning Amazon SageMaker Training Jobs with SageMaker SSH Helper (https://medium.com/towards-data-science/debugging-and-tuning-amazon-sagemaker-training-jobs-with-sagemaker-ssh-helper-51efeb4d03be) 208 | - Part 1: Optimizing the use of limited AI training accelerators - Maximizing the Utility of Scarce AI Resources: A Kubernetes Approach (https://medium.com/towards-data-science/maximizing-the-utility-of-scarce-ai-resources-a-kubernetes-approach-0230ba53965b) 209 | - Part 2: A Priority Based Scheduler for Amazon SageMaker Training Jobs (https://medium.com/towards-data-science/a-priority-based-scheduler-for-amazon-sagemaker-training-jobs-a225327e0a94) 210 | 211 | # 📺 Videos 212 | - StatQuest (https://statquest.org/video-index/) 213 | - Making Friends with Machine Learning (https://decision.substack.com/p/making-friends-with-machine-learning) 214 | - Artificial Intelligence - All in One (https://www.youtube.com/c/ArtificialIntelligenceAllinOne) 215 | - DeepLearningAI (https://www.youtube.com/c/Deeplearningai/playlists) 216 | - Deep Learning for Computer Vision (Andrej Karpathy, OpenAI) (https://www.youtube.com/watch?v=u6aEYuemt0M) 217 | - Deep Visualization Toolbox (https://www.youtube.com/watch?v=AgkfIQ4IGaM) 218 | - Nuts and Bolts of Applying Deep Learning (Andrew Ng) (https://www.youtube.com/watch?v=F1ka6a13S9I) 219 | - NIPS 2016 tutorial: "Nuts and bolts of building AI applications using Deep Learning" by Andrew Ng (https://www.youtube.com/watch?v=wjqaz6m42wU) 220 | - 3Blue1Brown (https://www.3blue1brown.com/) 221 | - Intellipaat - Data Science Online Course (https://www.youtube.com/watch?v=82pV44hr7kQ) 222 | - Jay Alammar (https://www.youtube.com/channel/UCmOwsoHty5PrmE-3QhUBfPQ/videos) 223 | - Abhishek Thakur (https://www.youtube.com/c/AbhishekThakurAbhi/playlists) 224 | - PyTorch Performance Tuning Guide - Szymon Migacz, NVIDIA (https://www.youtube.com/watch?v=9mS1fIYj1So) 225 | - NVIDIA Grandmaster Series (https://www.youtube.com/watch?v=bHuww-l_Sq0&list=PL5B692fm6--uXbxtmPJz5nu3Xmc1JUm3F) 226 | - Data Professor (https://www.youtube.com/c/DataProfessor/playlists) 227 | - Rasa Algorithm Whiteboard - Transformers & Attention 1: Self Attention (https://www.youtube.com/watch?v=yGTUuEx3GkA) 228 | - Smart Home (https://www.youtube.com/c/AlexTeo/featured) 229 | - Elliot Waite - Machine Learning, Coding, Math Animations (https://www.youtube.com/c/elliotwaite/videos) 230 | - James Briggs - NLP semantic search, vector similarity search (https://www.youtube.com/c/JamesBriggs/playlists) 231 | - Dataquest (https://www.youtube.com/channel/UC_lePY0Lm0E2-_IkYUWpI5A) 232 | - ByteByteGo (https://www.youtube.com/c/ByteByteGo/playlists) 233 | - Nicholas Renotte - Learn Machine Learning (https://www.youtube.com/@NicholasRenotte/playlists) 234 | - Andrej Karpathy - Neural Networks: Zero to Hero (https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ) (https://github.com/0ssamaak0/Karpathy-Neural-Networks-Zero-to-Hero) 235 | - NLP Summit (https://www.nlpsummit.org/) 236 | 237 | # 📚 Books 238 | - The Hundred-Page Machine Learning Book (http://themlbook.com/wiki/doku.php) 239 | - Machine Learning Engineering (http://www.mlebook.com/wiki/doku.php) 240 | - Approaching (Almost) Any Machine Learning Problem (https://github.com/abhishekkrthakur/approachingalmost/blob/master/AAAMLP.pdf) 241 | - The fastai book (https://github.com/fastai/fastbook) 242 | - https://books.google.com.sg/books?id=yATuDwAAQBAJ&pg=PA470&lpg=PA470&dq=AdaptiveConcatPool2d+vs+AdaptiveAvgPool2d&source=bl&ots=NKltks4CYL&sig=ACfU3U2xJo3iFtgSSLpQoUGEFYzrouhYzQ&hl=en&sa=X&ved=2ahUKEwiy29KGzNLwAhUUVH0KHTqSC3YQ6AEwCXoECAYQAw#v=onepage&q&f=false 243 | - Introduction to Probability for Data Science (https://probability4datascience.com/index.html) 244 | - Probabilistic Machine Learning: An Introduction (https://probml.github.io/pml-book/book1.html) 245 | - Dive into Deep Learning (https://d2l.ai/) 246 | - Personalized Machine Learning by Julian McAuley (https://cseweb.ucsd.edu/~jmcauley/pml/pml_book.pdf) 247 | - Machine Learning for Credit Card Fraud detection - Practical handbook (https://fraud-detection-handbook.github.io/fraud-detection-handbook/Foreword.html) 248 | - Deep Learning (https://www.deeplearningbook.org/) 249 | - Efficient Python Tricks and Tools for Data Scientists (https://khuyentran1401.github.io/Efficient_Python_tricks_and_tools_for_data_scientists/README.html) 250 | - Deep Learning Interviews: Hundreds of fully solved job interview questions from a wide range of key topics in AI (https://github.com/BoltzmannEntropy/interviews.ai) 251 | - Data Distribution Shifts and Monitoring (https://huyenchip.com/2022/02/07/data-distribution-shifts-and-monitoring.html) 252 | - Competitive Programmer’s Handbook (https://cses.fi/book/book.pdf) 253 | - Free and/or open source books on machine learning, statistics, data mining, etc (https://github.com/josephmisiti/awesome-machine-learning/blob/master/books.md) 254 | - Lucene in Action - Second Edition (https://livebook.manning.com/book/lucene-in-action-second-edition/appendix-b/) 255 | - Build a Large Language Model (From Scratch) - by Sebastian Raschka (https://github.com/rasbt/LLMs-from-scratch) 256 | - Machine Learning Q and AI book - by Sebastian Raschka (https://github.com/rasbt/MachineLearning-QandAI-book) 257 | - Hands-On Large Language Models - by Jay Alammar and Maarten Grootendorst (https://github.com/HandsOnLLM/Hands-On-Large-Language-Models) 258 | 259 | # 👍 Papers 260 | - 2015 Cyclical Learning Rates for Training Neural Networks (https://arxiv.org/abs/1506.01186) 261 | - 2017 Decoupled Weight Decay Regularization (https://arxiv.org/abs/1711.05101) 262 | - 2018 Mixed Precision Training (https://arxiv.org/pdf/1710.03740.pdf) 263 | - 2020 ReadNet: A Hierarchical Transformer Framework for Web Article Readability Analysis (https://link.springer.com/chapter/10.1007/978-3-030-45439-5_3) 264 | - 2021 A Survey of Transformers (https://arxiv.org/pdf/2106.04554.pdf) 265 | - 2022 Formal Algorithms for Transformers (https://arxiv.org/pdf/2207.09238.pdf) 266 | - 2023 Transformer models: an introduction and catalog (https://arxiv.org/pdf/2302.07730.pdf) (http://bit.ly/3YFqRn9) 267 | 268 | # 📑 Articles 269 | 270 | ## 🖼️ CNN 271 | - Applied Deep Learning - Part 4: Convolutional Neural Networks (https://towardsdatascience.com/applied-deep-learning-part-4-convolutional-neural-networks-584bc134c1e2) 272 | - CNNs from different viewpoints (https://medium.com/impactai/cnns-from-different-viewpoints-fab7f52d159c) 273 | - Image Kernels - Explained Visually (https://setosa.io/ev/image-kernels/) 274 | - Increase the Accuracy of Your CNN by Following These 5 Tips I Learned From the Kaggle Community (https://towardsdatascience.com/increase-the-accuracy-of-your-cnn-by-following-these-5-tips-i-learned-from-the-kaggle-community-27227ad39554) 275 | - A Beginner's Guide To Understanding Convolutional Neural Networks (https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/) 276 | - A Beginner's Guide To Understanding Convolutional Neural Networks Part 2 (https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/) 277 | - The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3) (https://adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html) 278 | - CNN Explainer (https://poloclub.github.io/cnn-explainer/) 279 | 280 | ## ↩️ RNN 281 | - Understanding LSTM Networks (http://colah.github.io/posts/2015-08-Understanding-LSTMs/) 282 | - A Visual Guide to Recurrent Layers in Keras (https://amitness.com/2020/04/recurrent-layers-keras/) 283 | 284 | ## ⁉️ NLP 285 | - A Visual Guide to FastText Word Embeddings (https://amitness.com/2020/06/fasttext-embeddings/) 286 | - The Illustrated Word2vec (https://jalammar.github.io/illustrated-word2vec/) 287 | - An Intuitive Understanding of Word Embeddings: From Count Vectors to Word2Vec (https://www.analyticsvidhya.com/blog/2017/06/word-embeddings-count-word2veec/) 288 | - Intuitive Understanding of Seq2seq model & Attention Mechanism in Deep Learning (https://medium.com/analytics-vidhya/intuitive-understanding-of-seq2seq-model-attention-mechanism-in-deep-learning-1c1c24aace1e) 289 | - How to Develop Word Embeddings in Python with Gensim (https://machinelearningmastery.com/develop-word-embeddings-python-gensim/) 290 | - Interactive Analysis of Sentence Embeddings (https://amitness.com/interactive-sentence-embeddings/) 291 | - Cosine Similarity for Vector Space Models (Part III) (https://blog.christianperone.com/2013/09/machine-learning-cosine-similarity-for-vector-space-models-part-iii/) 292 | 293 | ## 💏 NVIDIA Recommender Systems 294 | - Recommender Systems competitions solutions (https://github.com/NVIDIA-Merlin/competitions) 295 | - NVIDIA at RecSys 2022 (https://www.nvidia.com/en-us/events/recsys/) 296 | - Tutorials (https://recsys.acm.org/recsys22/tutorials/) 297 | - Tutorial Hands on Explainable Recommender Systems with Knowledge Graphs (https://explainablerecsys.github.io/recsys2022/) 298 | - Neural Re-ranking Tutorial (RecSys 22) (https://librerank-community.github.io/) 299 | - PIRS - Psychology-informed Recommender Systems (https://socialcomplab.github.io/pirs-psychology-informed-recsys/) 300 | - NVIDIA Merlin (https://medium.com/nvidia-merlin) 301 | - Merlin Jupyter Notebook Examples (https://catalog.ngc.nvidia.com/orgs/nvidia/resources/merlin_notebooks) 302 | - Merlin Models Example Notebooks (https://github.com/NVIDIA-Merlin/models/tree/main/examples) 303 | - NVIDIA Deep Learning Examples for Tensor Cores (https://github.com/NVIDIA/DeepLearningExamples) 304 | - Merlin Systems Example Notebook (https://github.com/NVIDIA-Merlin/systems/tree/main/examples) 305 | - Building a Four-Stage Recommender Pipeline (https://github.com/NVIDIA-Merlin/systems#building-a-four-stage-recommender-pipeline) 306 | - Exploring Production Ready Recommender Systems with Merlin (https://medium.com/nvidia-merlin/exploring-production-ready-recommender-systems-with-merlin-66bba65d18f2) 307 | - Recommender Models: Reducing Friction with Merlin Models (https://medium.com/nvidia-merlin/recommender-models-reducing-friction-with-merlin-models-4ea799fc3d89) 308 | - Scale faster with less code using Two Tower with Merlin (https://medium.com/nvidia-merlin/scale-faster-with-less-code-using-two-tower-with-merlin-c16f32aafa9f) 309 | - Transformers4Rec: Building Session-Based Recommendations with an NVIDIA Merlin Library (https://developer.nvidia.com/blog/transformers4rec-building-session-based-recommendations-with-an-nvidia-merlin-library/) 310 | - HugeCTR, a GPU-accelerated recommender framework (https://github.com/NVIDIA-Merlin/HugeCTR) 311 | - Recommender Systems at NVIDIA on Demand (https://www.nvidia.com/en-us/on-demand/search/?facet.mimetype[]=event%20session&layout=list&ncid=so-medi-419714&page=1&q=recommender%20systems&sort=date) 312 | - Recommender Systems Best Practices (https://resources.nvidia.com/en-us-recsys-white-paper/merlin-technical-ove) 313 | - Training a Recommender System on DGX A100 with 100B+ Parameters in TensorFlow 2 (https://developer.nvidia.com/blog/training-a-recommender-system-on-dgx-a100-with-100b-parameters-in-tensorflow-2/) 314 | - Recommender Systems, Not Just Recommender Models (2022-04-15)(https://medium.com/nvidia-merlin/recommender-systems-not-just-recommender-models-485c161c755e) 315 | - How NVIDIA Supports Recommender Systems feat. Even Oldridge | Stanford MLSys Seminar Episode 27 (https://www.youtube.com/watch?v=wPso35VkuCs) 316 | - 📺 Building and Deploying a Multi-Stage Recommender System with NVIDIA Merlin (https://www.youtube.com/watch?v=BQC-SGdIdD8) 317 | - Building and Deploying a Multi-Stage Recommender System with Merlin (https://resources.nvidia.com/en-us-merlin/bad-a-multi-stage-recommender?) 318 | - How to Build a Winning Deep Learning Powered Recommender System-Part 3 (https://developer.nvidia.com/blog/how-to-build-a-winning-deep-learning-powered-recommender-system-part-3/) (https://github.com/NVIDIA-Merlin/competitions/tree/main/WSDM_WebTour2021_Challenge) 319 | - 📺 Mastering Multilingual Recommender Systems | Grandmaster Series E9 | Winning Amazon’s 2023 KDD Cup (https://www.youtube.com/watch?v=IECznzY_Ko4) 320 | 321 | ## 💏 Collaborative Filtering / Recommender Systems 322 | - Microsoft - Recommenders - examples and best practices for building recommendation systems (https://github.com/microsoft/recommenders) 323 | - DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference (https://github.com/harvard-acc/DeepRecSys) 324 | - Google Courses - Recommendation Systems (https://developers.google.com/machine-learning/recommendation) 325 | - Dive Into Deep Learning - Chapter on Recommender Systems (http://d2l.ai/chapter_recommender-systems/index.html) 326 | - Reference End to End Architectures: 327 | - Alibaba Cloud - Recommender System: Ranking Algorithms and Training Architectures (https://www.alibabacloud.com/blog/recommender-system-ranking-algorithms-and-training-architectures_596643) 328 | - Basics of Recommender Systems (https://towardsdatascience.com/basics-of-recommender-systems-6f0fba58d8a) 329 | - Understanding Matrix Factorization for recommender systems (https://towardsdatascience.com/understanding-matrix-factorization-for-recommender-systems-4d3c5e67f2c9) 330 | - Building a Music Recommendation Engine with Probabilistic Matrix Factorization in PyTorch (https://towardsdatascience.com/building-a-music-recommendation-engine-with-probabilistic-matrix-factorization-in-pytorch-7d2934067d4a) 331 | - Customer Segmentation in Online Retail (https://towardsdatascience.com/customer-segmentation-in-online-retail-1fc707a6f9e6) 332 | - Sparse Matrices (https://www.youtube.com/watch?v=Lhef_jxzqCg) 333 | - scipy.sparse.csr_matrix example (https://stackoverflow.com/questions/53254104/cant-understand-scipy-sparse-csr-matrix-example) 334 | - Understanding min_df and max_df in scikit CountVectorizer (https://stackoverflow.com/questions/27697766/understanding-min-df-and-max-df-in-scikit-countvectorizer) 335 | - Build a Recommendation Engine With Collaborative Filtering (https://realpython.com/build-recommendation-engine-collaborative-filtering/) 336 | - Collaborative Filtering Recommendation with Co-Occurrence Algorithm (https://songxia-sophia.medium.com/collaborative-filtering-recommendation-with-co-occurrence-algorithm-dea583e12e2a) 337 | - Recommendation System Series (https://towardsdatascience.com/recommendation-system-series-part-1-an-executive-guide-to-building-recommendation-system-608f83e2630a) 338 | - Wayfair Tech Blog (https://www.aboutwayfair.com/careers/tech-blog?q=&s=0&f0=0000017b-63b5-d47e-adff-f7bda4220000) 339 | - TensorFlow Recommenders Tutorial (https://www.tensorflow.org/recommenders/examples/basic_retrieval) 340 | - Eugene Yan 341 | - System Design for Recommendations and Search (2021-06-27)(https://eugeneyan.com/writing/system-design-for-discovery/) 342 | - Patterns for Personalization in Recommendations and Search (2021-06-13)(https://eugeneyan.com/writing/patterns-for-personalization/) 343 | - Real-time Machine Learning For Recommendations (2021-01-10)(https://eugeneyan.com/writing/real-time-recommendations/) 344 | - Beating the Baseline Recommender with Graph & NLP in Pytorch (2020-01-13)(https://eugeneyan.com/writing/recommender-systems-graph-and-nlp-pytorch/#natural-language-processing-nlp-and-graphs) 345 | - Search, Rank, and Recommendations (https://medium.com/mlearning-ai/search-rank-and-recommendations-35cc717772cb)(https://www.kaggle.com/code/sbrvrm/search-ranking/notebook) 346 | - Vector representation of products Prod2Vec: How to get rid of a lot of embeddings (https://towardsdatascience.com/vector-representation-of-products-prod2vec-how-to-get-rid-of-a-lot-of-embeddings-26265361457c) 347 | - Deep Recommender Systems at Facebook feat. Carole-Jean Wu | Stanford MLSys Seminar Episode 24 (https://www.youtube.com/watch?v=5xcd0V9m6Xs) 348 | - Twitter's Recommendation Algorithm (https://blog.twitter.com/engineering/en_us/topics/open-source/2023/twitter-recommendation-algorithm) (https://github.com/twitter/the-algorithm) (https://github.com/twitter/the-algorithm-ml) 349 | - Personalized recommendations articles by Gaurav Chakravorty (https://www.linkedin.com/today/author/gauravchak?trk=article-ssr-frontend-pulse_more-articles) 350 | - Accelerating AI: Implementing Multi-GPU Distributed Training for Personalized Recommendations (https://multithreaded.stitchfix.com/blog/2023/06/08/distributed-model-training/) 351 | - How Instacart Uses Machine Learning-Driven Autocomplete to Help People Fill Their Carts (https://tech.instacart.com/how-instacart-uses-machine-learning-driven-autocomplete-to-help-people-fill-their-carts-9bc56d22bafb) 352 | - Is this the ChatGPT moment for recommendation systems? (https://www.shaped.ai/blog/is-this-the-chatgpt-moment-for-recommendation-systems) 353 | - Hands-on H&M Real-Time Personalized Recommender (https://github.com/decodingml/personalized-recommender-course) 354 | - Building a TikTok-like recommender (https://decodingml.substack.com/p/the-ultimate-recommender-system-framework) 355 | - Feature pipelines for TikTok-like recommenders (https://decodingml.substack.com/p/feature-pipeline-for-tiktok-like) 356 | - Training pipelines for TikTok-like recommenders (https://decodingml.substack.com/p/training-pipelines-for-tiktok-like) 357 | - Deploy scalable TikTok-like recommenders (https://decodingml.substack.com/p/deploy-scalable-tiktok-like-recommenders) 358 | - Using LLMs to build TikTok-like recommenders (https://decodingml.substack.com/p/using-llms-to-build-tiktok-like-recommenders) 359 | 360 | ## 👫 Similarity Search / ANNS / Vector Indexing 361 | - Getting started with Vector DBs in Python (https://code.dblock.org/2023/06/16/getting-started-with-vector-dbs-in-python.html) (https://github.com/dblock/vectordb-hello-world/) 362 | - Pinecone - A managed, cloud-native vector database with a simple API (https://www.pinecone.io/learn/) (https://docs.pinecone.io/docs/examples) 363 | - Weaviate (https://weaviate.io/blog.html) 364 | - Billion-scale Approximate Nearest Neighbor Search (https://matsui528.github.io/cvpr2020_tutorial_retrieval/) 365 | - PQk-means is a Python library for efficient clustering of large-scale data (https://github.com/DwangoMediaVillage/pqkmeans) 366 | - Nanopq (https://nanopq.readthedocs.io/en/latest/source/tutorial.html) 367 | - (https://speakerdeck.com/matsui_528/cvpr20-tutorial-billion-scale-approximate-nearest-neighbor-search?slide=84) 368 | - (https://github.com/matsui528/nanopq) 369 | - PQTable (http://yusukematsui.me/project/pqtable/pqtable.html) 370 | - Faiss - Facebook AI Similarity Search (https://github.com/facebookresearch/faiss/wiki) 371 | - Faiss tips (https://github.com/matsui528/faiss_tips) 372 | - https://speakerdeck.com/matsui_528/cvpr20-tutorial-billion-scale-approximate-nearest-neighbor-search?slide=108 373 | - Faiss on-disk example (https://davidefiocco.github.io/nearest-neighbor-search-with-faiss/) 374 | - NGT - Neighborhood Graph and Tree for Indexing High-dimensional Data (https://github.com/yahoojapan/NGT) 375 | - Milvus Bootcamp Solutions (https://github.com/milvus-io/bootcamp) 376 | - Add Similarity Search to DynamoDB with Faiss (https://medium.com/swlh/add-similarity-search-to-dynamodb-with-faiss-c68eb6a48b08)(https://github.com/ioannist/dynamodb-faiss-builder) 377 | - BERT models with Solr and Elasticsearch (https://github.com/DmitryKey/bert-solr-search) 378 | - Generative Feedback Loops with LLMs for Vector Databases (https://weaviate.io/blog/generative-feedback-loops-with-llms) 379 | - From zero to semantic search embedding model (https://blog.metarank.ai/from-zero-to-semantic-search-embedding-model-592e16d94b61) 380 | - Example of using factory pattern for your vectorstore implementation (https://github.com/trancethehuman/factory-pattern-vectorstore-interface) (https://www.youtube.com/watch?v=v1LyUJ5NFFU) 381 | - Accelerating Vector Search: Fine-Tuning GPU Index Algorithms (https://developer.nvidia.com/blog/accelerating-vector-search-fine-tuning-gpu-index-algorithms/) (https://github.com/rapidsai/raft/blob/HEAD/notebooks/VectorSearch_QuestionRetrieval.ipynb) 382 | - RAFT: Reusable Accelerated Functions and Tools for Vector Search and More (https://github.com/rapidsai/raft) 383 | - RAFT IVF-PQ tutorial (https://github.com/rapidsai/raft/blob/28b789404bedfa8dd82675fc4221f6db927c0422/notebooks/tutorial_ivf_pq.ipynb) 384 | - CAGRA: Highly Parallel Graph Construction and Approximate Nearest Neighbor Search for GPUs (Cuda Anns GRAph-based) (https://arxiv.org/pdf/2308.15136) (https://docs.rapids.ai/api/raft/nightly/pylibraft_api/neighbors/#cagra) 385 | - Geospatial Vector Search: Building an AI-Powered Geo-Aware News Search (https://levelup.gitconnected.com/geospatial-vector-search-building-an-ai-powered-geo-aware-news-search-6cbda8919465) 386 | - MyScale - A Deep Dive into SQL Vector Databases (https://myscale.com/blog/what-is-sql-vector-databases/) 387 | - Vector DB Comparison (https://superlinked.com/vector-db-comparison/) 388 | - How to build a real-time News Search Engine using Vector DBs - implementing a live news aggregating streaming pipeline with NewsAPI, NewsData, Apache Kafka, Bytewax, and Upstash Vector Database (https://medium.com/decodingml/how-to-build-a-real-time-news-search-engine-using-serverless-upstash-kafka-and-vector-db-6ba393e55024) (https://decodingml.substack.com/p/how-to-build-a-real-time-news-search) (https://github.com/decodingml/articles-code/tree/main/articles/ml_system_design/real_time_news_search_with_upstash_kafka_and_vector_db) 389 | - Hands-on Amazon Tabular Semantic Search (https://github.com/decodingml/tabular-semantic-search-tutorial) 390 | - Forget text-to-SQL: Use this natural query instead (https://decodingml.substack.com/p/forget-text-to-sql-use-this-natural) 391 | - Stop using text-to-SQL for search. Here's why. (https://decodingml.substack.com/p/stop-using-text-to-sql-for-search) 392 | 393 | ## 🆎 Code Search 394 | - A brief history of code search at GitHub (https://github.blog/2021-12-15-a-brief-history-of-code-search-at-github/) 395 | - The technology behind GitHub’s new code search (https://github.blog/2023-02-06-the-technology-behind-githubs-new-code-search/) 396 | - Regular Expression Matching with a Trigram Index or How Google Code Search Worked (https://swtch.com/~rsc/regexp/regexp4.html) 397 | 398 | ## 🌐 Search Engine / Information Retrieval 399 | - PROBABILISTIC DATA STRUCTURES FOR WEB ANALYTICS AND DATA MINING (https://highlyscalable.wordpress.com/2012/05/01/probabilistic-structures-web-analytics-data-mining/) 400 | - Big Data Counting: How To Count A Billion Distinct Objects Using Only 1.5KB Of Memory (http://highscalability.com/blog/2012/4/5/big-data-counting-how-to-count-a-billion-distinct-objects-us.html) 401 | - Roaring Bitmap (https://pypi.org/project/roaringbitmap/0.1/) (https://github.com/RoaringBitmap/RoaringBitmap) 402 | - A primer on Roaring bitmaps: what they are and how they work (https://vikramoberoi.com/a-primer-on-roaring-bitmaps-what-they-are-and-how-they-work/) (https://news.ycombinator.com/item?id=32692012) 403 | - Elias-Fano: quasi-succinct compression of sorted integers in C# (2016) (https://wolfgarbe.medium.com/elias-fano-quasi-succinct-compression-of-sorted-integers-in-c-89f92a8c9986) 404 | - Using Bitmaps to Perform Range Queries (https://www.featurebase.com/blog/range-encoded-bitmaps) 405 | - The anatomy of a Druid segment file (https://medium.com/engineers-optimizely/the-anatomy-of-a-druid-segment-file-bed89a93af1e#.46tincja7) 406 | - Information Retrieval resources (https://cwiki.apache.org/confluence/display/LUCENE/InformationRetrieval) 407 | - Information Retrieval - Lectures by Paolo Ferragina (http://didawiki.di.unipi.it/doku.php/magistraleinformatica/ir/ir22/start) 408 | - https://www.slideshare.net/VadimKirilchuk/numeric-rangequeries 409 | - NYU - Introduction to Data Compression - Web Search Engines (http://engineering.nyu.edu/~suel/cs6913/lec4-compress.pdf) 410 | - Decoding Billions of Integers Per Second Through Vectorization (https://people.csail.mit.edu/jshun/6886-s19/lectures/lecture19-1.pdf) 411 | - Smart way of storing data (https://towardsdatascience.com/smart-way-of-storing-data-d22dd5077340) 412 | - Google - Challenges in Building Large-Scale Information Retrieval Systems (http://static.googleusercontent.com/media/research.google.com/en//people/jeff/WSDM09-keynote.pdf) 413 | - A guide to Google Search ranking systems (https://developers.google.com/search/docs/appearance/ranking-systems-guide) 414 | - How to ARCHITECT a search engine like Google Search (https://newsletter.theaiedge.io/p/how-to-architect-a-search-engine) 415 | - Evaluation Metrics for Search and Recommendation Systems (https://weaviate.io/blog/retrieval-evaluation-metrics) 416 | - What AI Engineers Should Know about Search (https://softwaredoug.com/blog/2024/06/25/what-ai-engineers-need-to-know-search) 417 | - Building a full-text search engine in 150 lines of Python code (https://bart.degoe.de/building-a-full-text-search-engine-150-lines-of-code/) (https://github.com/bartdegoede/python-searchengine) 418 | - A search engine in 80 lines of Python (https://www.alexmolas.com/2024/02/05/a-search-engine-in-80-lines.html) (https://github.com/alexmolas/microsearch) 419 | 420 | ## ⬆️ Search Ranking 421 | - Using Cross-Encoders as reranker in multistage vector search (https://weaviate.io/blog/cross-encoders-as-reranker) 422 | - Bi-Encoder vs. Cross-Encoder (https://www.sbert.net/examples/applications/cross-encoder/#bi-encoder-vs-cross-encoder) 423 | - Improving information retrieval in the Elastic Stack: Introducing Elastic Learned Sparse Encoder, our new retrieval model (https://www.elastic.co/search-labs/may-2023-launch-information-retrieval-elasticsearch-ai-model) 424 | - Hybrid Search: SPLADE (Sparse Encoder) (https://medium.com/@sowmiyajaganathan/hybrid-search-splade-sparse-encoder-neural-retrieval-models-d092e5f46913) 425 | - What is a Sparse Vector? How to Achieve Vector-based Hybrid Search - and using SPLADE (https://qdrant.tech/articles/sparse-vectors/) 426 | - SPLADERunner (https://github.com/PrithivirajDamodaran/SPLADERunner) (https://huggingface.co/prithivida/Splade_PP_en_v1) 427 | - State-of-the-art MS MARCO Models (https://twitter.com/Nils_Reimers/status/1435544757388857345) 428 | - https://www.sbert.net/examples/training/ms_marco/README.html#marginmse 429 | - ranx ([raŋks]) is a library of fast ranking evaluation metrics (https://amenra.github.io/ranx/) 430 | - How Google Search ranking works (https://searchengineland.com/how-google-search-ranking-works-445141) 431 | - FAQ: All about the Google RankBrain algorithm (https://searchengineland.com/faq-all-about-the-new-google-rankbrain-algorithm-234440) 432 | - A guide to Google: Origins, history and key moments in search (https://searchengineland.com/guide/google) 433 | 434 | 435 | ## 👫 Lucene / Solr / Elasticsearch / BM25 436 | - Grokking Solr Trie Fields (http://mentaldetritus.blogspot.com/2013/01/grokking-solr-trie-fields.html) 437 | - Mastering ElasticSearch (https://hoclaptrinhdanang.com/downloads/pdf/elasticsearch/Mastering%20ElasticSearch-PDF.pdf) 438 | - Elasticsearch Kernel Analysis - Data Model (https://zhuanlan.zhihu.com/p/34680841) 439 | - Elasticsearch-Principle of Big Data (https://zhuanlan.zhihu.com/p/83961549) 440 | - Scaling Lucene and Solr (https://lucidworks.com/post/scaling-lucene-and-solr/) 441 | - Lucene Papers (https://cwiki.apache.org/confluence/display/LUCENE/LucenePapers) 442 | - Lucene, talk by Doug Cutting (https://lucene.sourceforge.net/talks/pisa/) 443 | - Lucene: The Good Parts (https://blog.parse.ly/lucene/) 444 | - Zhaofeng Zhou (Muluo) - Analysis of Lucene - Basic Concepts (https://alibaba-cloud.medium.com/analysis-of-lucene-basic-concepts-5ff5d8b90a53) 445 | - Zhaofeng Zhou (Muluo) - Lucene IndexWriter: An In-Depth Introduction (https://www.alibabacloud.com/blog/lucene-indexwriter-an-in-depth-introduction_594673) 446 | - What is term vector in Lucene (http://makble.com/what-is-term-vector-in-lucene) 447 | - What is Trie Data Structure in Lucene numeric range query (http://makble.com/what-is-trie-data-structure-in-lucene-numeric-range-query) (https://issues.apache.org/jira/browse/LUCENE-1673) 448 | - Lucene Performance (http://philosophyforprogrammers.blogspot.com/2010/09/lucene-performance.html) 449 | - Frame of Reference and Roaring Bitmaps (https://www.elastic.co/blog/frame-of-reference-and-roaring-bitmaps) (https://juejin.cn/post/7085352076595134494) 450 | - Class RoaringDocIdSet (https://lucene.apache.org/core/6_0_0/core/org/apache/lucene/util/RoaringDocIdSet.html) (https://issues.apache.org/jira/browse/LUCENE-5983) 451 | - Class Lucene50PostingsFormat (https://lucene.apache.org/core/7_2_1/core/org/apache/lucene/codecs/lucene50/Lucene50PostingsFormat.html) 452 | - Changing Bits - Lucene's PulsingCodec on "Primary Key" Fields (https://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html) 453 | - Changing Bits - Lucene performance with the PForDelta codec (https://blog.mikemccandless.com/2010/08/lucene-performance-with-pfordelta-codec.html) 454 | - Changing Bits - Lucene's new BlockPostingsFormat (https://blog.mikemccandless.com/2012/08/lucenes-new-blockpostingsformat-thanks.html) 455 | - Changing Bits - Using Finite State Transducers in Lucene (https://blog.mikemccandless.com/2010/12/using-finite-state-transducers-in.html) 456 | - Apache Lucene - Index File Formats (https://lucene.apache.org/core/3_0_3/fileformats.html) (https://stackoverflow.com/questions/71816188/what-is-elasticsearch-index-lucene-index-and-inverted-index) 457 | - Lucene-file format (https://zhuanlan.zhihu.com/p/354105864) 458 | - Apache Lucene - Scoring (https://lucene.apache.org/core/3_0_3/scoring.html) 459 | - Lucene JIRA: 460 | - More fine-grained control over the packed integer implementation that is chosen (https://issues.apache.org/jira/browse/LUCENE-4062) 461 | - Reduce reads for sparse DocValues (https://issues.apache.org/jira/browse/LUCENE-8374) 462 | - Simple9 (de)compression (https://issues.apache.org/jira/browse/LUCENE-2189) 463 | - Lucene index modeling - Why are skiplists used instead of btree? (https://stackoverflow.com/questions/66804510/lucene-index-modeling-why-are-skiplists-used-instead-of-btree) 464 | - How does lucene index documents? (https://stackoverflow.com/questions/2602253/how-does-lucene-index-documents#answer-43203339) 465 | - Skip List vs. Binary Search Tree (https://stackoverflow.com/questions/256511/skip-list-vs-binary-search-tree/28270537#28270537) 466 | - Uncle Cang - Lucene Source Code Series (https://juejin.cn/user/2559318800998141) 467 | - Chris - Articles on Lucene Index (https://www.amazingkoala.com.cn/Lucene/Index/) 468 | - Chris - Articles on Lucene Index File (https://www.amazingkoala.com.cn/Lucene/suoyinwenjian/) 469 | - Chris - Articles on Lucene Compressed Storage (https://www.amazingkoala.com.cn/Lucene/yasuocunchu/) 470 | - Chris - Articles on Lucene Tools (https://www.amazingkoala.com.cn/Lucene/gongjulei/) 471 | - Chris - Articles on Lucene Search (https://www.amazingkoala.com.cn/Lucene/Search/) 472 | - LuXugang/Lucene-7.x-9.x/tree/master/blog (https://github.com/LuXugang/Lucene-7.x-9.x/tree/master/blog) 473 | - Lucene Learning Summary (https://blog.csdn.net/jinhong_lu) 474 | - Lucene Underlying Principles and Optimization Experience Sharing (https://blog.csdn.net/njpjsoftdev) 475 | - Lucene PostingsFormat At-a-Glance (https://github.com/mocobeta/lucene-postings-format) (https://github.com/mocobeta/lucene-postings-format/blob/main/indexing_chain.md) 476 | - A Simple Tutorial of Lucene's Indexing and Search Systems (https://github.com/jiepujiang/LuceneTutorial) 477 | - 📺 What is in a Lucene index? Adrien Grand, Software Engineer, Elasticsearch (https://www.youtube.com/watch?v=T5RmMNDR5XI) 478 | - 📺 from Plain Schwarz (https://www.youtube.com/@PlainSchwarzUG) 479 | - Berlin Buzzwords 2015: Adrien Grand – Algorithms & data-structures that power Lucene & ElasticSearch (https://www.youtube.com/watch?v=eQ-rXP-D80U) 480 | - Berlin Buzzwords 2015: Ryan Ernst - Compression in Lucene (https://www.youtube.com/watch?v=kCQbFxqusN4&list=PLq-odUc2x7i-_qWWixXHZ6w-MxyLxEC7s&index=21) 481 | - Berlin Buzzwords 2015: Ivan Mamontov - Fast Decompression Lucene Codec (https://www.youtube.com/watch?v=2HQdbpgHfnQ&list=PLq-odUc2x7i-_qWWixXHZ6w-MxyLxEC7s&index=17) 482 | - Berlin Buzzwords 2017: Alan Woodward - How does a Lucene Query actually work? (https://www.youtube.com/watch?v=Z-yG-KvIuD8&list=PLq-odUc2x7i-9Nijx-WfoRMoAfHC9XzTt&index=3) 483 | - Berlin Buzzwords 2017: Adrien Grand - Running slow queries with Lucene (https://www.youtube.com/watch?v=p51vIDWHWqk&list=PLq-odUc2x7i-9Nijx-WfoRMoAfHC9XzTt&index=16) 484 | - (2020) Bruno Roustant – A Journey to Write a New Lucene PostingsFormat (https://www.youtube.com/watch?v=av0yQY3pklA&list=PLq-odUc2x7i_YTCOTQ6p3m-kqpvEXGvbT&index=44) 485 | - (2020) Uwe Schindler - Ask Me Anything: Lucene 9 (https://www.youtube.com/watch?v=RvoH_pVvXz0&list=PLq-odUc2x7i_YTCOTQ6p3m-kqpvEXGvbT&index=10) 486 | - BM25 for Python: Achieving high performance while simplifying dependencies with BM25S⚡ (https://huggingface.co/blog/xhluca/bm25s) (https://bm25s.github.io/) 487 | 488 | 489 | ## 📑 General ML/DL Articles 490 | - A Recipe for Training Neural Networks (http://karpathy.github.io/2019/04/25/recipe/) 491 | - The best machine learning and deep learning libraries (https://morioh.com/p/73998ba2a04e) 492 | - Neural Style Transfer with tf.keras (https://aihub.cloud.google.com/p/products%2F7f7495dd-6f66-4f8a-8c30-15f211ad6957) 493 | - Stock Prices Prediction Using Machine Learning and Deep Learning Techniques (with Python codes) (https://www.analyticsvidhya.com/blog/2018/10/predicting-stock-price-machine-learningnd-deep-learning-techniques-python) 494 | - I trained a model. What is next? (https://ternaus.blog/tutorial/2020/08/28/Trained-model-what-is-next.html) 495 | - How To Build and Deploy a Serverless Machine Learning App on AWS (https://towardsdatascience.com/how-to-build-and-deploy-a-serverless-machine-learning-app-on-aws-1468cf7ef5cb) $$ 496 | - Applied Deep Learning - Part 3: Autoencoders (https://towardsdatascience.com/applied-deep-learning-part-3-autoencoders-1c083af4d798) 497 | - Approaching (Almost) Any Machine Learning Problem (https://www.linkedin.com/pulse/approaching-almost-any-machine-learning-problem-abhishek-thakur) 498 | - KAGGLE ENSEMBLING GUIDE (https://mlwave.com/kaggle-ensembling-guide/) 499 | - Reading Larger than Memory CSVs with RAPIDS and Dask (https://medium.com/rapids-ai/reading-larger-than-memory-csvs-with-rapids-and-dask-e6e27dfa6c0f) 500 | - 28 Weekly Machine Learning Tricks And Resources That Are Pure Gems #1 (https://ibexorigin.medium.com/28-weekly-machine-learning-tricks-and-resources-that-are-pure-gems-1-8e5259a93c94) 501 | - 26 Weekly ML Tricks And Resources That Are Pure Gems, #2 (https://ibexorigin.medium.com/26-weekly-ml-tricks-and-resources-that-are-pure-gems-2-3be56841b1d9) 502 | - Hadoop vs. Spark vs. Kafka - How to Structure Modern Big Data Architecture? (https://nexocode.com/blog/posts/hadoop-spark-kafka-modern-big-data-architecture/) 503 | - Is it Better to Save Models Using Joblib or Pickle? (https://medium.com/nlplanet/is-it-better-to-save-models-using-joblib-or-pickle-776722b5a095) 504 | - How to Measure Drift in ML Embeddings (https://towardsdatascience.com/how-to-measure-drift-in-ml-embeddings-ee8adfe1e55e) (https://www.evidentlyai.com/blog/embedding-drift-detection) 505 | - Large Language Models in Molecular Biology (https://towardsdatascience.com/large-language-models-in-molecular-biology-9eb6b65d8a30) 506 | - Categorically: Don’t explode — encode! (https://github.com/PilCAki/machine-learning-tips/blob/main/Don't%20Explode%20-%20Encode!.ipynb) 507 | - Out-of-bag validation for random forests (https://medium.com/data-science-at-microsoft/out-of-bag-validation-for-random-forests-378f2b292560) (https://github.com/PilCAki/machine-learning-tips/blob/main/Out%20Of%20Bag%20Validation%20for%20Random%20Forests.ipynb) 508 | - Exploring Location Data Using a Hexagon Grid (https://towardsdatascience.com/exploring-location-data-using-a-hexagon-grid-3509b68b04a2) (https://github.com/sktahtin4/Helsinki-city-bikes) 509 | - H3: Uber’s Hexagonal Hierarchical Spatial Index (https://www.uber.com/en-FI/blog/h3/) (https://h3geo.org/docs/) 510 | - H3 hexagon data viewer (https://wolf-h3-viewer.glitch.me/) 511 | - How to use PostgreSQL for (military) geoanalytics tasks (https://klioba.com/how-to-use-postgresql-for-military-geoanalytics-tasks) (https://klioba.com/public/presentations/PostGIS_Warfare_Export.pdf) (http://download.geofabrik.de/osm-data-in-gis-formats-free.pdf) (https://download.geofabrik.de/) 512 | - Writing fast string ufuncs for NumPy 2.0 (https://labs.quansight.org/blog/numpy-string-ufuncs) 513 | 514 | #### Gradient / Momentum 515 | - Yes you should understand backprop (https://karpathy.medium.com/yes-you-should-understand-backprop-e2f06eab496b) (https://www.youtube.com/watch?v=i94OvYb6noo) 516 | - An overview of gradient descent optimization algorithms (https://ruder.io/optimizing-gradient-descent/) 517 | - Understanding Gradient Clipping (and How It Can Fix Exploding Gradients Problem) (https://neptune.ai/blog/understanding-gradient-clipping-and-how-it-can-fix-exploding-gradients-problem) 518 | - Vanishing and Exploding Gradients in Neural Network Models: Debugging, Monitoring, and Fixing (https://neptune.ai/blog/vanishing-and-exploding-gradients-debugging-monitoring-fixing) 519 | - Why Momentum Really Works (https://distill.pub/2017/momentum/) 520 | 521 | #### Optimization / Outliers / Overfitting / Regularization / Imbalance Dataset 522 | - Nuts and Bolts of Optimization (https://www.linkedin.com/pulse/nuts-bolts-optimization-chandra-mohan-lingam) 523 | - Finding Good Learning Rate and The One Cycle Policy (https://towardsdatascience.com/finding-good-learning-rate-and-the-one-cycle-policy-7159fe1db5d6) 524 | - Understanding Fastai's fit_one_cycle method (https://iconof.com/1cycle-learning-rate-policy/) 525 | - How to Choose an Activation Function for Deep Learning (https://machinelearningmastery.com/choose-an-activation-function-for-deep-learning/) 526 | - Dealing with Outliers Using Three Robust Linear Regression Models - Huber, RANSAC, Theil-Sen (https://developer.nvidia.com/blog/dealing-with-outliers-using-three-robust-linear-regression-models/) 527 | - Fighting Overfitting With L1 or L2 Regularization: Which One Is Better? (https://neptune.ai/blog/fighting-overfitting-with-l1-or-l2-regularization) 528 | - imbalanced-learn - tools dealing with classification with imbalanced classes (https://imbalanced-learn.org/stable/index.html) 529 | - Five mistakes to avoid when modeling with imbalanced datasets (https://medium.com/data-science-at-microsoft/five-mistakes-to-avoid-when-modeling-with-imbalanced-datasets-d58a8c09929c) (https://github.com/PilCAki/imbalanced-dataset-common-errors/blob/main/Imbalanced%20Dataset%20Examples.ipynb) 530 | 531 | #### Regression 532 | - A Comprehensive Overview of Regression Evaluation Metrics (https://developer.nvidia.com/blog/a-comprehensive-overview-of-regression-evaluation-metrics/) 533 | - A Comprehensive Guide to Interaction Terms in Linear Regression (https://developer.nvidia.com/blog/a-comprehensive-guide-to-interaction-terms-in-linear-regression/) 534 | 535 | ## ⏱️ Time Series 536 | - Predicting Credit Defaults Using Time-Series Models with Recursive Neural Networks and XGBoost (https://developer.nvidia.com/blog/predicting-credit-defaults-using-time-series-models-with-recursive-neural-networks-and-xgboost/) (https://github.com/daxiongshu/triton_amex) 537 | - Three Approaches to Encoding Time Information as Features for ML Models (https://developer.nvidia.com/blog/three-approaches-to-encoding-time-information-as-features-for-ml-models/) 538 | - A Comprehensive Guide on Interaction Terms in Time Series Forecasting (https://developer.nvidia.com/blog/a-comprehensive-guide-on-interaction-terms-in-time-series-forecasting/) 539 | - Skforecast - works with any regressor compatible with the scikit-learn API, including LightGBM, XGBoost, CatBoost, Keras, etc (https://skforecast.org/0.13.0/index.html) (https://github.com/skforecast/skforecast) 540 | 541 | ## 📊 EDA / Data Visualization 542 | - Plotly Fundamentals (https://plotly.com/python/plotly-fundamentals/) 543 | - CSS Color (https://developer.mozilla.org/en-US/docs/Web/CSS/color_value) 544 | - Flowing Data (https://flowingdata.com/) 545 | - Automatic EDA Libraries Comparisson (https://www.kaggle.com/andreshg/automatic-eda-libraries-comparisson/) 546 | - [TPS-Jun] This is Original EDA & VIZ 😉 (https://www.kaggle.com/subinium/tps-jun-this-is-original-eda-viz/) 547 | - Netflix Data Visualization (https://www.kaggle.com/joshuaswords/netflix-data-visualization?scriptVersionId=58425238)(https://www.kaggle.com/joshuaswords) 548 | - Custom Word Cloud (https://www.kaggle.com/tarzon/custom-word-cloud/notebook) 549 | - RAPIDS cuXfilter - Build a Fully Interactive Dashboard in a Few Lines of Python (https://medium.com/rapids-ai/build-a-fully-interactive-dashboard-in-a-few-lines-of-python-49959fb55fff) 550 | - Facets - visualizations for understanding and analyzing machine learning datasets (https://github.com/PAIR-code/facets) 551 | - Alternatives to box plots: Using beeswarm and raincloud plots to summarise ecological data (https://labs.ala.org.au/posts/2023-08-28_alternatives-to-box-plots/post.html) 552 | - How to Create a Beautiful Polar Histogram With Python and Matplotlib (https://dev.to/oscarleo/how-to-create-a-beautiful-polar-histogram-with-python-and-matplotlib-400l) 553 | - Data Visualisation Guide (https://data.europa.eu/apps/data-visualisation-guide/) 554 | - What is Gephi? Meet this useful network analysis tool (https://medium.com/@vespinozag/what-is-gephi-meet-this-useful-network-analysis-tool-628a1b42428c) (https://github.com/gephi/gephi) 555 | - The Perfect Way to Smooth Your Noisy Data - using Whittaker-Eilers Smoothing (https://towardsdatascience.com/the-perfect-way-to-smooth-your-noisy-data-4f3fe6b44440) 556 | - Scikit-learn Visualization Guide: Making Models Speak (https://www.dataleadsfuture.com/scikit-learn-visualization-guide-making-models-speak/) 557 | - 6 python libraries to make beautiful maps (https://medium.com/@alexroz/6-python-libraries-to-make-beautiful-maps-9fb9edb28b27) 558 | - Exploring ExplainerDashBoard, the easiest way to Develop Interactive DashBoards (https://towardsdatascience.com/build-dashboards-in-less-than-10-lines-of-code-835e9abeae4b) $$ 559 | - What I've Learned Building Interactive Embedding Visualizations (https://cprimozic.net/blog/building-embedding-visualizations-from-user-profiles/) (https://github.com/Ameobea/osu-beatmap-atlas/blob/main/notebooks/README.md) 560 | - Announcing Data Wrangler: Code-centric viewing and cleaning of tabular data in Visual Studio Code (https://devblogs.microsoft.com/python/announcing-data-wrangler-code-centric-viewing-and-cleaning-of-tabular-data-in-visual-studio-code/) 561 | 562 | ## 📘 Colab 563 | - Google Colab Tips for Power Users (https://amitness.com/2020/06/google-colaboratory-tips/) 564 | - Configuring Google Colab Like A Pro (https://medium.com/@robertbracco1/configuring-google-colab-like-a-pro-d61c253f7573) 565 | 566 | ## 🐍 Python 567 | - Advanced Python Topics Tutorial (https://www.geeksforgeeks.org/advanced-python-tutorials/) 568 | - Data manipulation with Python (https://www.mit.edu/~amidi/teaching/data-science-tools/study-guide/data-manipulation-with-python/) 569 | - Data Preprocessing Concepts with Python (https://pub.towardsai.net/data-preprocessing-concepts-with-python-b93c63f14bb6) 570 | - 7 Actionable Tips on How to Use Python to Become a Finance Guru (https://sdsclub.com/7-actionable-tips-on-how-to-use-python-to-become-a-finance-guru/) 571 | - 7 Cool Python Packages Kagglers Are Using Without Telling You (https://towardsdatascience.com/7-cool-python-packages-kagglers-are-using-without-telling-you-e83298781cf4) 572 | - Fastest Way to Read Excel in Python (https://hakibenita.com/fast-excel-python) 573 | - Efficiently iterating over rows in a Pandas DataFrame (https://towardsdatascience.com/efficiently-iterating-over-rows-in-a-pandas-dataframe-7dd5f9992c01) 574 | - Ten Python datetime pitfalls, and what libraries are (not) doing about it (https://dev.arie.bovenberg.net/blog/python-datetime-pitfalls/) 575 | - Whenever - Typed and DST-safe datetimes for Python (https://github.com/ariebovenberg/whenever) (https://whenever.readthedocs.io/en/latest/) 576 | 577 | ## 🗄️ Database / Storage 578 | - Blogs about streaming database (https://medium.com/@yingjunwu) 579 | - Rethinking Stream Processing and Streaming Databases (https://betterprogramming.pub/rethinking-stream-processing-and-streaming-databases-21076aaec375) 580 | - Why You Shouldn’t Invest In Vector Databases? (https://medium.com/data-engineer-things/why-you-shouldnt-invest-in-vector-databases-c0cd3f59d23c) 581 | - Building and operating a pretty big storage system called S3 (https://www.allthingsdistributed.com/2023/07/building-and-operating-a-pretty-big-storage-system.html) 582 | - MyScale - A Deep Dive into SQL Vector Databases (https://myscale.com/blog/what-is-sql-vector-databases/) 583 | - Building a weather data warehouse part I: Loading a trillion rows of weather data into TimescaleDB (https://aliramadhan.me/2024/03/31/trillion-rows.html) 584 | - How to use PostgreSQL for (military) geoanalytics tasks (https://klioba.com/how-to-use-postgresql-for-military-geoanalytics-tasks) (https://klioba.com/public/presentations/PostGIS_Warfare_Export.pdf) (http://download.geofabrik.de/osm-data-in-gis-formats-free.pdf) (https://download.geofabrik.de/) 585 | 586 | ## 🏃 Reinforcement Learning 587 | - Develop Your First AI Agent: Deep Q-Learning (https://towardsdatascience.com/develop-your-first-ai-agent-deep-q-learning-375876ee2472) 588 | 589 | ## 👍 Interesting and Fun 590 | - A Visual Guide to Regular Expression (https://amitness.com/regex/) 591 | - Machine Learning Glossary (https://developers.google.com/machine-learning/glossary/) 592 | - ConvNetJS - Deep Learning in your browser (https://cs.stanford.edu/people/karpathy/convnetjs/) 593 | - Classifier Playground (https://www.ccom.ucsd.edu/~cdeotte/programs/classify.html) 594 | - YOLO: Real-Time Object Detection (https://pjreddie.com/darknet/yolo/) 595 | - Coder One - A virtual playground to practice, compete, and experiment with machine learning (https://www.gocoder.one/) 596 | - Interactive demonstrations for ML courses (http://arogozhnikov.github.io/2016/04/28/demonstrations-for-ml-courses.html) 597 | - Tinker With a Neural Network (http://playground.tensorflow.org/) 598 | - Understanding ROC curves (http://www.navan.name/roc/) 599 | - Algorithm Visualizer (https://algorithm-visualizer.org/) 600 | - Sort Visualizer (https://adwait-algorithm-visualizer.netlify.app/) 601 | - VisuAlgo (https://visualgo.net/en) 602 | - Doodles-as-A-Service Repo (https://github.com/girliemac/a-picture-is-worth-a-1000-words) 603 | - AI art generator (https://app.wombo.art/) 604 | - Seeing Theory - Visual introduction to probability and statistics (https://seeing-theory.brown.edu/index.html#firstPage) 605 | - Answer Chat AI (https://www.answerchatai.com/) 606 | - AI Explorables (https://pair.withgoogle.com/explorables/) 607 | - Do Machine Learning Models Memorize or Generalize? (https://pair.withgoogle.com/explorables/grokking/) 608 | - Tensor Puzzles (https://github.com/srush/Tensor-Puzzles) 609 | - GPU Puzzles (https://github.com/srush/GPU-Puzzles) 610 | - Insanely Useful Websites (https://insanelyusefulwebsites.com) 611 | - The Markov-chain Monte Carlo Interactive Gallery (https://chi-feng.github.io/mcmc-demo/app.html) (https://chi-feng.github.io/mcmc-demo/) (https://github.com/chi-feng/mcmc-demo) 612 | - Interactive Gaussian process regression demo (https://chi-feng.github.io/gp-demo/) (https://github.com/chi-feng/gp-demo) 613 | - Visualizing {dplyr}’s mutate(), summarize(), group_by(), and ungroup() with animations (https://www.andrewheiss.com/blog/2024/04/04/group_by-summarize-ungroup-animations/) 614 | - Tidy Animated Verbs - inner_join(), left_join(), right_join(), full_join(), semi_join(), anti_join(), union(), union_all(), intersect(), setdiff(), pivot_wider(), pivot_longer(), spread(), gather() (https://www.garrickadenbuie.com/project/tidyexplain/) 615 | 616 | ## 👍 GitHub Repositories 617 | - PyTorch Image Models (https://github.com/rwightman/pytorch-image-models) 618 | - EfficientNet PyTorch (https://github.com/lukemelas/EfficientNet-PyTorch) 619 | - blurr - A library that integrates huggingface transformers with version 2 of the fastai framework (https://ohmeow.github.io/blurr/) 620 | - MIT 6.S191: Introduction to Deep Learning - Labs (https://github.com/aamini/introtodeeplearning/) 621 | - GitHub Profile README Generator (https://github.com/rahuldkjain/github-profile-readme-generator) 622 | - Applied ML (https://github.com/eugeneyan/applied-ml) 623 | - A detailed example of how to generate your data in parallel with PyTorch (https://github.com/shervinea/pytorch-data-generator) 624 | - Keras vs. PyTorch: Alien vs. Predator recognition with transfer learning (https://github.com/deepsense-ai/Keras-PyTorch-AvP-transfer-learning) 625 | - Deep Learning with Catalyst (https://github.com/catalyst-team/dl-course) 626 | - sgugger - Deep Learning Notebooks (https://github.com/sgugger/Deep-Learning) 627 | - Yang Zhang - fast.ai machine learning course notes (https://gist.github.com/yang-zhang/7ce6e6e7174c35ae26b7ce0dba1999d2) 628 | - ML Course Notes (https://github.com/dair-ai/ML-Course-Notes) 629 | - Tez: a simple pytorch trainer (https://github.com/abhi1thakur/tez) 630 | - Kaggle-Ensemble-Guide (https://github.com/MLWave/Kaggle-Ensemble-Guide) 631 | - Collection of useful data science topics along with code and articles (https://github.com/khuyentran1401/Data-science) 632 | - Data Science Articles (https://github.com/parulnith/Data-Science-Articles/blob/main/README.md) 633 | - James Le (https://github.com/khanhnamle1994) 634 | - Donne Martin (https://github.com/donnemartin) 635 | - Fraud-Detection-Handbook (https://github.com/Fraud-Detection-Handbook) 636 | - VADER-Sentiment-Analysis (https://github.com/cjhutto/vaderSentiment) 637 | - ECCO - Interfaces for Explaining Transformer (https://github.com/jalammar/ecco) 638 | - MLBoy (https://github.com/john-rocky) 639 | - Criteo (https://github.com/criteo) 640 | - NVIDIA Deep Learning Examples for Tensor Cores (https://github.com/NVIDIA/DeepLearningExamples) 641 | - Pinecone (https://github.com/pinecone-io/examples) 642 | - Surprise - a Python scikit for recommender systems that deal with explicit rating data (https://github.com/NicolasHug/Surprise) 643 | - PQk-means is a Python library for efficient clustering of large-scale data (https://github.com/DwangoMediaVillage/pqkmeans) 644 | - GraphEmbedding (https://github.com/shenweichen/GraphEmbedding) 645 | - Dataquest - Project Walkthroughs (https://github.com/dataquestio/project-walkthroughs) 646 | - Retinal Vessel Segmentation with data augmentation and Keras (https://github.com/onurboyar/Retinal-Vessel-Segmentation) 647 | - Visual Search with MXNet Gluon and HNSW (https://github.com/ThomasDelteil/VisualSearch_MXNet) 648 | - DAIR.AI - Democratizing Artificial Intelligence Research, Education, and Technologies (https://github.com/dair-ai) 649 | - ML Visuals (https://github.com/dair-ai/ml-visuals) 650 | - ML Notebooks - examples for all sorts of machine learning tasks and applications (https://github.com/dair-ai/ML-Notebooks) 651 | - ML YouTube Courses (https://github.com/dair-ai/ML-YouTube-Courses) 652 | - Transformer Recipe (https://github.com/dair-ai/Transformers-Recipe) 653 | - Graph Neural Networks (GNNs Recipe) (https://github.com/dair-ai/GNNs-Recipe) 654 | - NannyML estimates performance with Confidence-based Performance estimation (CBPE) - Predict Your Model’s Performance (Without Waiting for the Control Group)(https://towardsdatascience.com/predict-your-models-performance-without-waiting-for-the-control-group-3f5c9363a7da)(https://github.com/NannyML/nannyml) 655 | - Obsei (https://github.com/obsei/obsei) 656 | - nanoGPT - The simplest, fastest repository for training/finetuning medium-sized GPTs (https://github.com/karpathy/nanoGPT) 657 | - Trax — Deep Learning with Clear Code and Speed (https://github.com/google/trax/tree/af3a38917bd1bc69cf5d25ce007e16185f22f050) 658 | - Lightning-Hydra-Template (https://github.com/ashleve/lightning-hydra-template) 659 | - https://github.com/ChanCheeKean/DataScience/ 660 | - Tracking Progress in Natural Language Processing (https://github.com/sebastianruder/NLP-progress/tree/master) 661 | 662 | ## 👍 Kaggle 663 | - Kaggle Solutions (https://farid.one/kaggle-solutions/) 664 | - https://www.kaggle.com/bextuychiev/notebooks 665 | - Kaggle utils (https://github.com/daxiongshu/kaggle_utils) 666 | - Competition and Community Insights from NVIDIA’s Kaggle Grandmasters (https://developer.nvidia.com/blog/competition-and-community-insights-from-nvidias-kaggle-grandmasters/) 667 | 668 | ## 📘 DeepNote 669 | - Abid (https://deepnote.com/@abid) 670 | 671 | ## 👍 Blogs 672 | - Machine Learning Mastery (https://machinelearningmastery.com/start-here/) 673 | - Analytics Vidhya (https://www.analyticsvidhya.com/blog/) 674 | - Cassie Kozyrkov - Decision Intelligence (https://decision.substack.com/) 675 | - Kaggle Winner's Blog (https://medium.com/kaggle-blog) 676 | - Terrence Shin (https://terenceshin.com/) 677 | - Sylvain Gugger (https://sgugger.github.io/) 678 | - Jason Yosinski (https://yosinski.com/) 679 | - Lankinen (Fast.ai Lesson notes) (https://lankinen.medium.com/) 680 | - Zachary Mueller (https://muellerzr.github.io/fastblog/) 681 | - Jay Alammar (http://jalammar.github.io/) 682 | - colah's blog - Christopher Olah (https://colah.github.io/) 683 | - DataMuni (https://www.datamuni.com/) 684 | - Chris McCormick (https://mccormickml.com/) 685 | - Sebastian Ruder (https://ruder.io/) 686 | - Gilbert Tanner (https://ml-explained.com/) 687 | - Andrey Lukyanenko - Paper Review (https://andlukyane.com/blog/) 688 | - Jeremy Jordan (https://www.jeremyjordan.me/data-science/) 689 | - Mario - Kaggle Grandmaster (https://forecastegy.com/) 690 | - Chip Huyen (https://huyenchip.com/blog/) 691 | - Daniel Tunkelang - Search Fundamentals (https://dtunkelang.medium.com/) 692 | - Daniel Lemire - crazily fast code (https://lemire.me/en/) 693 | - NLPlanet (https://www.nlplanet.org/blog/index.html) 694 | - Uncle Cang decoding - Lucene source code series (https://juejin.cn/user/2559318800998141) 695 | - Philschmid's blog on Transformers & SageMaker (by Philipp Schmid) (https://www.philschmid.de/) 696 | - The AiEdge Newsletter (by Damien Benveniste) (https://newsletter.theaiedge.io/archive) 697 | - Ahead of AI (by Sebastian Raschka) (https://magazine.sebastianraschka.com/archive) (https://sebastianraschka.com/blog/) 698 | - The Kaitchup – AI on a Budget - by Benjamin Marie (https://kaitchup.substack.com/archive) 699 | - CheeKean (https://kean-chan.medium.com/) 700 | - Ryan O'Connor (https://www.assemblyai.com/blog/author/ryan/) 701 | - Norsbook’s KDP Journey (https://medium.com/norsbooks-kdp-journey) 702 | 703 | ## 👍 Company Tech Blogs 704 | - Data.gov.sg Blog (https://blog.data.gov.sg/) 705 | - GovTech’s Data Science and Artificial Intelligence Division (DSAID) (https://medium.com/dsaid-govtech) 706 | - Flipkart Tech Blog (https://tech.flipkart.com/) 707 | - RAPIDS AI (https://medium.com/rapids-ai) (https://github.com/rapidsai-community/notebooks-contrib/tree/main/getting_started_materials) 708 | - Pinterest Engineering (https://medium.com/pinterest-engineering) 709 | - Netflix Tech Blog (https://netflixtechblog.com/) 710 | - Uber Engineering (https://eng.uber.com/) 711 | - Meta AI (https://ai.facebook.com/blog) 712 | - Twitter Engineering (https://blog.twitter.com/engineering/en_us) 713 | - DoorDash Engineering Blog (https://doordash.engineering/blog/) 714 | - The Airbnb Tech Blog (https://medium.com/airbnb-engineering) 715 | - Zalando Engineering Blog (https://engineering.zalando.com/) 716 | - Linkedin Engineering Blog (https://engineering.linkedin.com/blog) 717 | - Microsoft Experimentation Platform (https://www.microsoft.com/en-us/research/group/experimentation-platform-exp/) 718 | - Etsy (https://codeascraft.com/) 719 | - The Unofficial Google Data Science Blog (https://www.unofficialgoogledatascience.com/) 720 | - Stitch Fix (https://multithreaded.stitchfix.com/blog/) 721 | - Lyft (https://eng.lyft.com/tagged/data-science) 722 | - Booking.com (https://booking.ai/) 723 | - Yelp Engineering Blog (https://engineeringblog.yelp.com/) 724 | - Spotify (https://engineering.atspotify.com/category/data-science/) 725 | - EXP (https://exp-platform.com/encyclopediamldm/) 726 | - Capital One (https://www.capitalone.com/tech/machine-learning/) 727 | - Google AI Blog (https://ai.googleblog.com/) 728 | - NVIDIA Merlin (https://medium.com/nvidia-merlin) 729 | - Criteo Tech Blog (https://medium.com/criteo-engineering)(https://labs.criteo.com/engineering-blog/) 730 | - Grab (https://engineering.grab.com/categories/data-science/) 731 | - Elucida (https://medium.com/elucidata/tagged/technology) 732 | - Zilliz (https://zilliz.com/learn) 733 | - Neptune (https://neptune.ai/blog) 734 | 735 | ## 🔟 Maths 736 | - Maths is Fun (https://www.mathsisfun.com/) 737 | - Matrix Multiplication (http://matrixmultiplication.xyz/) 738 | - The Matrix Calculus You Need For Deep Learning (https://explained.ai/matrix-calculus/) 739 | - What's behind matrix multiplication? (https://www.tivadardanka.com/blog/behind-matrix-multiplication) 740 | - Descriptive Matrix Operations with Einops - example with multi-query attention (https://www.kolaayonrinde.com/blog/2024/01/08/einops.html) 741 | - Geometric Mean (https://www.mathsisfun.com/numbers/geometric-mean.html) 742 | - Dot Product (https://www.mathsisfun.com/algebra/vectors-dot-product.html) 743 | - Things that confused me about cross-entropy (https://chris-said.io/2020/12/26/two-things-that-confused-me-about-cross-entropy/) 744 | - Mathematics for Machine Learning (https://github.com/dair-ai/Mathematics-for-ML) 745 | - Seeing Theory - Visual introduction to probability and statistics (https://seeing-theory.brown.edu/index.html#firstPage) 746 | - Calculus and Differentiation Primer (https://sebastianraschka.com/pdf/books/dlb/appendix_d_calculus.pdf) 747 | - Understanding Automatic Differentiation in 30 lines of Python (https://vmartin.fr/understanding-automatic-differentiation-in-30-lines-of-python.html) 748 | - What Are Floating-point Numbers? (https://www.baseclass.io/newsletter/floating-point-numbers) 749 | - Monte Carlo Simulation — a practical guide (https://towardsdatascience.com/monte-carlo-simulation-a-practical-guide-85da45597f0e) 750 | - Gradient Descent Algorithm — a deep dive (https://towardsdatascience.com/gradient-descent-algorithm-a-deep-dive-cf04e8115f21) 751 | - New Breakthrough Brings Matrix Multiplication Closer to Ideal (https://www.quantamagazine.org/new-breakthrough-brings-matrix-multiplication-closer-to-ideal-20240307/) 752 | 753 | 754 | ## 👍 Datasets 755 | - 24 Free Datasets for Building an Irresistible Portfolio (2022) (https://www.dataquest.io/blog/free-datasets-for-projects/) 756 | - Best Public Datasets for Machine Learning and Data Science (https://pub.towardsai.net/best-datasets-for-machine-learning-data-science-computer-vision-nlp-ai-c9541058cf4f) 757 | - Amazon Review Data (2018) (https://nijianmo.github.io/amazon/index.html) 758 | - Recommender Systems and Personalization Datasets (https://cseweb.ucsd.edu/~jmcauley/datasets.html) 759 | - Amazon product data (outdated?) (https://jmcauley.ucsd.edu/data/amazon/index.html) 760 | - Amazon Customer Reviews Dataset (https://s3.amazonaws.com/amazon-reviews-pds/readme.html) 761 | - Papers With Code (https://paperswithcode.com/datasets) 762 | - https://research.google/tools/ 763 | - Criteo - Terabyte Click Logs (https://labs.criteo.com/2013/12/download-terabyte-click-logs/) 764 | - Components (https://components.one/datasets) 765 | - Texmex (http://www.irisa.fr/texmex/ressources/index_en.php) 766 | - The MOVI Image Base (http://www.irisa.fr/texmex/ressources/bases/base_images_movi/index.html) 767 | - INRIA Holidays dataset for evaluation of image search & Copydays dataset for evaluation of copy detection (http://lear.inrialpes.fr/people/jegou/data.php#holidays) 768 | - Datasets for approximate nearest neighbor search (http://corpus-texmex.irisa.fr/) 769 | - Semantic Text Similarity Dataset Hub (https://github.com/brmson/dataset-sts) 770 | - GroupLens (https://grouplens.org/datasets/movielens/) 771 | - Phishing Detection (Designing a New Net for Phishing Detection with NVIDIA Morpheus)(https://developer.nvidia.com/blog/designing-a-new-net-for-phishing-detection-with-nvidia-morpheus/) 772 | - SPAM_ASSASSIN dataset (https://spamassassin.apache.org/old/publiccorpus/) 773 | - Enron Emails dataset (https://www.cs.cmu.edu/~enron/) 774 | - Clair dataset (https://www.kaggle.com/datasets/rtatman/fraudulent-email-corpus) 775 | - CSE-CIC-IDS2018 on AWS - Dataset for Network Intrusion Detection (https://www.unb.ca/cic/datasets/ids-2018.html) 776 | - MIcrosoft News Dataset (MIND) (https://docs.microsoft.com/en-us/azure/open-datasets/dataset-microsoft-news?tabs=azureml-opendatasets) (https://www.kaggle.com/datasets/arashnic/mind-news-dataset) 777 | - 190k+ Medium Articles (https://www.kaggle.com/datasets/fabiochiusano/medium-articles) 778 | - Unsplash (https://unsplash.com/data) 779 | - Randomizing Very Large Datasets (https://towardsdatascience.com/randomizing-very-large-datasets-e2b14e507725) 780 | - https://data.gov.sg/datasets 781 | - EarthData - NASA Earth Observation Data (https://www.earthdata.nasa.gov/) 782 | 783 | 784 | ## 👍 Synthetic Data 785 | - MOSTLY AI, the #1 Synthetic Data Platform (https://mostly.ai/) 786 | - No data? No problem! Generating synthetic training data at scale for NLP tasks using T0PP (https://medium.com/criteo-engineering/no-data-no-problem-generating-synthetic-training-data-at-scale-for-nlp-tasks-using-t0pp-198581643c5b) 787 | - Generating Synthetic Data with Transformers (https://developer.nvidia.com/blog/generating-synthetic-data-with-transformers-a-solution-for-enterprise-data-challenges/) 788 | - https://github.com/NVIDIA/NeMo/blob/r1.8.0/tutorials/nlp/Megatron_Synthetic_Tabular_Data_Generation.ipynb 789 | - Faiss SyntheticDataset (https://github.com/facebookresearch/faiss/blob/main/contrib/datasets.py#L72)(https://gist.github.com/mdouze/551ef6fa0722f2acf58fa2c6fce732d6#file-demo_pytorch_knn-ipynb) 790 | - YData Synthetic - generate synthetic tabular and time-series data (https://github.com/ydataai/ydata-synthetic) 791 | - Generate a synthetic domain-specific Q&A dataset in <30 minutes (https://decodingml.substack.com/p/problems-deploying-your-ml-models) 792 | - How to Generate and Use Synthetic Data for Finetuning (https://eugeneyan.com/writing/synthetic/) 793 | 794 | ## 🛠️ Utilities / Tools 795 | - Future Tools (https://www.futuretools.io/) 796 | - Scanner App for Math and Science (https://mathpix.com/) 797 | - Readability (https://pypi.org/project/readability/) 798 | - ipdb.set_trace() Commands (https://xxx-cook-book.gitbooks.io/python-cook-book/content/Debug/ipdb.html) 799 | - Writing Math Equations in Jupyter Notebook: A Naive Introduction (https://medium.com/analytics-vidhya/writing-math-equations-in-jupyter-notebook-a-naive-introduction-a5ce87b9a214) 800 | - Carbon - Create and share beautiful images of your source code (https://carbon.now.sh/) 801 | - Manim Community - How to Create Mathematical Animations (https://docs.manim.community/en/stable/tutorials.html) 802 | - diagrams.net (https://www.diagrams.net/) 803 | - PlotNeuralNet: Use LaTex for making neural networks diagrams (https://github.com/HarisIqbal88/PlotNeuralNet) 804 | - Keyword Tool (https://keywordtool.io/) 805 | - Octoparse web scraping (https://www.octoparse.com/blog/10-myths-about-web-scraping) 806 | - Capitalize My Title (https://capitalizemytitle.com/) 807 | - Free IEEE Citation Generator (https://www.citethisforme.com/citation-generator/ieee) 808 | - Shorten URLs (https://bitly.com/) 809 | - Profile Pic Maker (https://pfpmaker.com/) 810 | - Pictory - Online Video Creator (https://pictory.ai/) 811 | - CapCut - Free Video Editor (https://www.capcut.com/) 812 | - Video Creation (https://invideo.io/) 813 | - Free Video Recording & Live Streaming (https://obsproject.com/) 814 | - Boost your YouTube views (https://vidiq.com/) 815 | - Human X AI Generative Music (https://mubert.com/) 816 | - AI Speech Software (https://beta.elevenlabs.io/) 817 | - Digital People Text-to-Video (https://www.d-id.com/) 818 | - BlobCity AI Seed Projects (https://github.com/blobcity/ai-seed) 819 | - Meta AI Frameworks, Tools, Libraries, Models (https://ai.facebook.com/tools) 820 | - Resume Worded (https://resumeworded.com/resume-bullet-points) 821 | - Mockaroo - Random Data Generator (https://mockaroo.com/) 822 | - Flourish - Beautiful and easy data visualization and storytelling (https://flourish.studio/) 823 | - Flameshot - screenshot software (https://github.com/flameshot-org/flameshot) 824 | - Facets - visualizations for understanding and analyzing machine learning datasets (https://github.com/PAIR-code/facets) 825 | - Google - People + AI Research (PAIR) (https://research.google/teams/brain/pair/) 826 | - Google Research resources (https://research.google/tools/) 827 | - Frederik Brasz - Voronoi generator (https://cfbrasz.github.io/programs.html) 828 | - handcalcs - Python calculations in Jupyter (https://github.com/connorferster/handcalcs) 829 | - Towhee - open-source ML pipeline to encode unstructured data into embeddings (https://towhee.io/) 830 | - Gradio - Build & Share Delightful Machine Learning Apps (https://gradio.app/) 831 | - 🤗 Gradio example (https://huggingface.co/spaces/gradio/xgboost-income-prediction-with-explainability) 832 | - 🤗 Gradio example (https://huggingface.co/spaces/gradio/chatbot_dialogpt/blob/main/run.py) 833 | - 🤗 Gradio example (https://huggingface.co/spaces/gradio/chatbot/blob/main/app.py) 834 | - CurlWget (https://www.analyticsvidhya.com/blog/2021/08/load-dataset-directly-into-colab-from-anywhere-on-the-browser-using-curlwget-extension/) 835 | - GLTR - detect automatically generated text (http://gltr.io/) 836 | - Keenious - research explorer (https://keenious.com/) (https://medium.com/keenious/knowledge-graph-search-of-60-million-vectors-with-weaviate-7964657ec911) 837 | - AutoRegex - Effortless conversions between English and RegEx (https://www.autoregex.xyz/) 838 | - Stable Diffusion (https://huggingface.co/spaces/stabilityai/stable-diffusion) 839 | - Quarto - open-source scientific and technical publishing system (https://quarto.org/) 840 | - JSON Crack - Visualize JSON with graphs (https://jsoncrack.com/) 841 | - markmap (markdown + mindmap) (https://markmap.js.org/repl) 842 | - Newspaper3k: Article scraping & curation (https://github.com/codelucas/newspaper) 843 | - Trafilatura - Python package and command-line tool to gather text on the Web (https://trafilatura.readthedocs.io/en/latest/) 844 | - GoogleNews (https://github.com/Iceloof/GoogleNews) 845 | - Interactive network visualizations (https://pyvis.readthedocs.io/en/latest/index.html) 846 | - markdownify - Convert HTML to markdown (https://pypi.org/project/markdownify/) 847 | - ResumeWizard (https://resume-wizard.vercel.app/) 848 | - 12 AI Copywriting Tools to Improve Efficiency (https://ahrefs.com/blog/ai-copywriting/) 849 | - Icons, backgrounds, templates, graphics, etc for presentations: 850 | - https://www.flaticon.com/ 851 | - https://www.freepik.com/ 852 | - News API - Search worldwide news with code (https://newsapi.org/) 853 | - MkDocs: static site generator that's geared towards building project documentation (https://www.mkdocs.org/) (https://github.com/mkdocs/mkdocs) 854 | - Material for MkDocs (https://www.youtube.com/watch?v=Q-YA_dA8C20) (https://squidfunk.github.io/mkdocs-material/) (https://github.com/squidfunk/mkdocs-material) (https://www.stevemar.net/five-things-about-mkdocs/) (https://docs.markhh.com/pages/tools/mkdocs_demo/) (https://www.starfallprojects.co.uk/blog/mkdocs-material-blog-cover-image/) (https://www.codeinsideout.com/blog/site-setup/create-site-project/) (https://blog.ktz.me/making-mkdocs-tables-look-like-github-markdown-tables/) 855 | - data load tool - dlt (https://dlthub.com/docs/intro) 856 | - cleanlab automatically detects data and label issues in your ML datasets (https://docs.cleanlab.ai/stable/index.html) 857 | - Public APIs (https://github.com/public-apis/public-apis) 858 | - How to configure VS Code for AI, ML and MLOps development in Python 🛠️️ (https://mlops.community/how-to-configure-vs-code-for-ai-ml-and-mlops-development-in-python-%F0%9F%9B%A0%ef%b8%8f%ef%b8%8f/) 859 | - Great Tables - Absolutely Delightful Table-making in Python (https://posit-dev.github.io/great-tables/articles/intro.html) 860 | - The Design Philosophy of Great Tables (https://posit-dev.github.io/great-tables/blog/design-philosophy/) 861 | - Chunk visualizer (https://huggingface.co/spaces/m-ric/chunk_visualizer) 862 | - Bytewax - Stream processing purely in Python (https://bytewax.io/) 863 | - Unstructured - Preprocess and structure unstructured text documents (such as PDFs, XML and HTML) for use in downstream machine learning tasks (https://unstructured-io.github.io/unstructured/core/cleaning.html) 864 | - Upstash Serverless Kafka & Vector Database (https://upstash.com/) 865 | - Meet the NiceGUI: Your Soon-to-be Favorite Python UI Library (https://towardsdatascience.com/meet-the-nicegui-your-soon-to-be-favorite-python-ui-library-fb69f14bb0ac) (https://nicegui.io/documentation) 866 | - imodels - Interpretable ML package (https://github.com/csinva/imodels) 867 | - Video generation (https://klingai.com/) 868 | - Emergent Mind - the next generation of AI assistants for learning and research (https://www.emergentmind.com) 869 | - Semantic Scholar - free, AI-powered research tool for scientific literature (https://www.semanticscholar.org) 870 | - https://www.perplexity.ai/ 871 | - Overleaf online LaTeX editor (https://www.overleaf.com/) 872 | - Instaloader - download pictures (or videos) along with their captions and other metadata from Instagram (https://instaloader.github.io/index.html) 873 | - High Emotion Words (https://thepersuasionrevolution.com/380-high-emotion-persuasive-words/) 874 | - Heptabase - visually make sense of your learning, research, and projects (https://heptabase.com/) 875 | - Jupyter Agent (https://huggingface.co/spaces/data-agents/jupyter-agent) 876 | - Lovable - Build web apps without coding (https://lovable.dev/) 877 | 878 | ## 👍 Job / Interview / DS Portfolio 879 | - Interview Query - Questions and Blogs (https://www.interviewquery.com/p/data-science-interview-questions)(https://www.interviewquery.com/articles)(https://www.interviewquery.com/blog) 880 | - Machine Learning FAQ (https://sebastianraschka.com/faq/) 881 | - Mastering the Deep Learning Interview: Top 35 Questions and Expert Answers (https://medium.com/@riteshgupta.ai/mastering-the-deep-learning-interview-top-35-questions-and-expert-answers-aabb701f6e45) 882 | - Machine Learning Interviews Book (https://huyenchip.com/ml-interviews-book/) 883 | - Deep Learning Interviews: Hundreds of fully solved job interview questions from a wide range of key topics in AI (https://github.com/BoltzmannEntropy/interviews.ai) 884 | - Machine Learning Interviews (https://github.com/khangich/machine-learning-interview/) 885 | - NLP Interview Question and Answers in 2022 (https://www.mygreatlearning.com/blog/nlp-interview-questions/) 886 | - Machine Learning Interview Question and Answers in 2022 (https://www.mygreatlearning.com/blog/machine-learning-interview-questions/) 887 | - Machine learning algorithms interview - tips & resources (https://workera.ai/resources/machine-learning-algorithms-interview/) 888 | - Crack Data Science Interviews: Essential Machine Learning Concepts (https://towardsdatascience.com/crack-data-science-interviews-essential-machine-learning-concepts-afd6a0a6d1aa) 889 | - Top 20 AB Test Interview Questions And Answers (https://grabngoinfo.com/top-20-ab-test-interview-questions-and-answers/) 890 | - AB Testing 101 (https://medium.com/jonathans-musings/ab-testing-101-5576de6466b) 891 | - How to use Causal Inference when A/B testing is not available (https://towardsdatascience.com/how-to-use-causal-inference-when-a-b-testing-is-not-possible-c87c1252724a) 892 | - Glassdoor machine learning interview questions (https://www.glassdoor.sg/Interview/machine-learning-interview-questions-SRCH_KO0,16.htm?countryRedirect=true) 893 | - subreddit - ML question (https://www.reddit.com/r/MLQuestions/) 894 | - subreddit - Learning machine learning (https://www.reddit.com/r/learnmachinelearning/) 895 | - Ten Advanced SQL Concepts You Should Know for Data Science Interviews (https://towardsdatascience.com/ten-advanced-sql-concepts-you-should-know-for-data-science-interviews-4d7015ec74b0) 896 | - SQL Group By and Partition By Scenarios: When and How to Combine Data in Data Science (https://www.kdnuggets.com/sql-group-by-and-partition-by-scenarios-when-and-how-to-combine-data-in-data-science) 897 | - Creating a Data Science Portfolio (https://towardsdatascience.com/creating-a-data-science-portfolio-bd485382f49) 898 | - Build Your LLM Engineer Portfolio: A 3-Month Roadmap (https://pub.towardsai.net/build-your-llm-engineer-portfolio-a-3-month-roadmap-19826e39c185) $$ 899 | - Data Science Portfolio (https://github.com/MaartenGr/projects) 900 | - Github digital cv example (https://github.com/March-08/digital-cv) 901 | - How to Build a Data Science Portfolio Website using Python (https://towardsdatascience.com/how-to-build-a-data-science-portfolio-website-using-python-79531426fde5) 902 | - How to create a Medium-like personal blog for free in a day (https://medium.com/geekculture/how-to-create-a-medium-like-personal-blog-for-free-in-a-day-55ebd9551d9c) 903 | - How to Swiftly Launch a Free Website With GitHub Pages (https://www.stephenvinouze.com/how-to-swiftly-launch-a-free-website-with-github-pages/) 904 | - TYJ (https://tengyeejing.com/) 905 | - The Portfolio that Got Me a Data Scientist Job (https://towardsdatascience.com/the-portfolio-that-got-me-a-data-scientist-job-513cc821bfe4) 906 | - Set Up Your Portfolio Website in Less Than 10 Minutes with Github Pages (https://medium.com/@evanca/set-up-your-portfolio-website-in-less-than-10-minutes-with-github-pages-d0efa8ff56fd) (https://github.com/evanca/quick-portfolio) 907 | - Use Python and NLP to Boost Your Resume (https://medium.com/data-marketing-philosophy/use-python-and-nlp-to-boost-your-resume-e4691a58bcc9) 908 | - Facebook Field Guide to Machine Learning (https://research.facebook.com/blog/2018/05/the-facebook-field-guide-to-machine-learning-video-series/) 909 | - A Guide to Production Level Deep Learning (https://github.com/alirezadir/Production-Level-Deep-Learning) 910 | - Rules of ML (https://developers.google.com/machine-learning/guides/rules-of-ml) 911 | - Impactful and widely cited papers and literature on ML/DL/RL/AI (https://github.com/tirthajyoti/Papers-Literature-ML-DL-RL-AI) 912 | - Definitive Interview prep ROADMAP (https://www.codinginterview.com/interview-roadmap) 913 | - interviewing.io - Watch technical mock interviews (https://interviewing.io/recordings) 914 | - Interview Cake (https://www.interviewcake.com/) 915 | - Interview Questions (https://www.tryexponent.com/questions) 916 | - Free Interview Practice (https://www.pramp.com/) 917 | - Big-O Cheat Sheet (https://www.bigocheatsheet.com/) 918 | - Leetcode list by topics (more comprehensive): https://protegejj.gitbook.io/oj-practices/chapter1/dynamic-programming 919 | - Leetcode Company Tag (https://github.com/xizhengszhang/Leetcode_company_frequency) 920 | - Python_LeetCode_Coding (https://github.com/LeihuaYe/Python_LeetCode_Coding) 921 | - Software Engineering Interview Preparation (GitBook) (https://orrsella.gitbooks.io/soft-eng-interview-prep/content/) 922 | - 4 questions to ask in interviews to assess codebase health (https://www.educative.io/blog/questions-assess-codebase-health-interviews) 923 | - How to Tackle The 7 Most Common Types of Interview Questions (https://www.youtube.com/watch?v=vFdkhsN1PJo&t=137s) 924 | - Instamentor's Knowledge Center (https://instamentor.com/articles/) 925 | - Experimentation is a major focus of Data Science across Netflix (https://netflixtechblog.com/experimentation-is-a-major-focus-of-data-science-across-netflix-f67923f8e985) 926 | - Fighting Overfitting With L1 or L2 Regularization: Which One Is Better? (https://neptune.ai/blog/fighting-overfitting-with-l1-or-l2-regularization) 927 | - Machine Learning Projects You NEVER Knew Existed (https://www.youtube.com/watch?v=sw3o0rAazMg&list=RDCMUCHXa4OpASJEwrHrLeIzw7Yg&index=6) 928 | - https://resumaker.ai/app/build/templates/ 929 | - Scarlet Ink by Dave Anderson - Interview Advice (https://www.scarletink.com/tag/interviewing/) 930 | - How can I recover a job offer I rejected, or a job I quit? (https://www.scarletink.com/questions-answers-reddit-cscareerquestions-experiment/) 931 | - Writing and Speaking Clearly and Concisely (https://www.scarletink.com/writing-speaking-clearly-concisely/) 932 | - The Top 3 Resume Mistakes Costing You the Job (https://blog.bytebytego.com/p/the-top-3-resume-mistakes-costing) 933 | - The Amazon “secret” of controllable inputs for your career (https://levelupwithethanevans.substack.com/p/the-amazon-secret-of-controllable) 934 | - ML Systems Design Interview Guide (http://patrickhalina.com/posts/ml-systems-design-interview-guide/) 935 | - Conquering the 2024 Job Market: My Journey to Multiple DS/MLE Offers I — Job Search Summary and Strategy (https://bertmclee.medium.com/conquering-the-2024-job-market-my-journey-to-multiple-ds-mle-offers-i-job-search-summary-and-92bd41cdd7c8) 936 | 937 | ## 💰 Salary Negotiation 938 | - level.fyi (https://www.levels.fyi/Salaries/Data-Scientist/Singapore/) 939 | - Ten Rules for Negotiating a Job Offer (https://haseebq.com/my-ten-rules-for-negotiating-a-job-offer/) 940 | - HOW TO NEGOTIATE SALARY: 9 TIPS FROM A PRO SALARY NEGOTIATOR (https://fearlesssalarynegotiation.com/salary-negotiation-guide/) 941 | - HOW TO WRITE A SALARY NEGOTIATION EMAIL (WITH 11 PROVEN TEMPLATES AND A SAMPLE) (https://fearlesssalarynegotiation.com/salary-negotiation-email-sample/#ask-for-time-template) 942 | - Salary Negotiation: Make More Money, Be More Valued (https://www.kalzumeus.com/2012/01/23/salary-negotiation/) 943 | 944 | ## 👍 System Design 945 | - ByteByteGo - System Design 101 (https://blog.bytebytego.com/) (https://github.com/ByteByteGoHq/system-design-101) 946 | - System Design Interview (https://systeminterview.com/scale-from-zero-to-millions-of-users.php) 947 | - SYSTEM DESIGN INTERVIEW PREPARATION SERIES (https://www.codekarle.com/) 948 | - The complete guide to system design in 2022 (https://www.educative.io/blog/complete-guide-to-system-design#filestorage) 949 | - Systems Design Crash Course for ML Engineers (https://towardsdatascience.com/systems-design-crash-course-for-ml-engineers-aafae1cf1890) 950 | - The System Design Primer (https://github.com/donnemartin/system-design-primer) 951 | - awesome-scalability (https://github.com/binhnguyennus/awesome-scalability) 952 | - System Architecture (https://orrsella.gitbooks.io/soft-eng-interview-prep/content/topics/system-architecture.html) 953 | - Dynamo: Amazon’s Highly Available Key-value Store (https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf) 954 | - System Design Series by Sanil Khurana (https://medium.com/@sanilkhurana7) 955 | - System Design Series: The Ultimate Guide for Building High-Performance Data Streaming Systems from Scratch! (https://towardsdatascience.com/system-design-series-0-to-100-guide-to-data-streaming-systems-3dd584bd28fa) 956 | - ML Systems Design Interview Guide (http://patrickhalina.com/posts/ml-systems-design-interview-guide/) 957 | 958 | ## 🆎 Algorithms / Technical Coding 959 | - Dynamic Programming Patterns (https://leetcode.com/discuss/general-discussion/458695/dynamic-programming-patterns) 960 | - Binary Search Template (https://leetcode.com/discuss/general-discussion/786126/Python-Powerful-Ultimate-Binary-Search-Template.-Solved-many-problems) 961 | - Binary Search (https://www.youtube.com/watch?v=tgVSkMA8joQ) 962 | - Partition subset problem - all approaches explained (https://leetcode.com/problems/partition-equal-subset-sum/solutions/462699/Whiteboard-Editorial.-All-Approaches-explained) 963 | 964 | ## 📺 Videos for Algorithms / Technical Coding / Interview Prep 965 | - Neet Code (https://www.youtube.com/c/NeetCode/playlists) 966 | - MIT 6.006 Introduction to Algorithms, Fall 2011 (https://www.youtube.com/playlist?list=PLUl4u3cNGP61Oq3tWYp6V_F-5jb5L2iHb) 967 | - Vivekanand Khyade - Algorithm Every Day (https://www.youtube.com/user/vivekanandkhyade/playlists) 968 | - Gaurav Sen (https://www.youtube.com/channel/UCRPMAqdtSgd0Ipeef7iFsKw) 969 | - Coding Interview Solutions (https://www.youtube.com/playlist?list=PLot-Xpze53leF0FeHz2X0aG3zd0mr1AW_) 970 | - Kevin Naughton Jr. (https://www.youtube.com/c/KevinNaughtonJr/playlists) 971 | - Sai Anish Malla (https://www.youtube.com/channel/UCFBorf0jHu-1WNHGJZDNtew/playlists) 972 | - Back To Back SWE (https://www.youtube.com/c/BackToBackSWE/playlists) 973 | - Tech Dummies Narendra L (https://www.youtube.com/c/TechDummiesNarendraL/videos) 974 | - Data Interview Pro - Emma (https://www.youtube.com/c/DataInterviewPro/playlists) 975 | 976 | ## 🔢 Bit Hacks 977 | - Bit Hacks (https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-172-performance-engineering-of-software-systems-fall-2018/lecture-slides/MIT6_172F18_lec3.pdf) 978 | - Bit Twiddling Hacks (http://graphics.stanford.edu/~seander/bithacks.html) 979 | - Introduction to Low Level Bit Hacks (https://catonmat.net/low-level-bit-hacks) 980 | - Bitwise operations cheat sheet () 981 | - Bits, Bytes, Building With Binary (https://medium.com/basecs/bits-bytes-building-with-binary-13cb4289aafa) 982 | - The Binary Cheatsheet (https://timseverien.github.io/binary-cheatsheet/) 983 | - Binary cheatsheet for coding interviews (https://www.techinterviewhandbook.org/algorithms/binary/) 984 | - Bitwise Hacks for Competitive Programming (https://www.geeksforgeeks.org/bitwise-hacks-for-competitive-programming/) 985 | - Bitwise Operators in Python (https://realpython.com/python-bitwise-operators/) 986 | - Signed number representations (https://en.wikipedia.org/wiki/Signed_number_representations) 987 | - Pack and Unpack data (Masking) (https://www.youtube.com/watch?v=MWFwM9-2nlI) 988 | - Bit Packing or How to Love AND, OR or XOR (https://learnmongodbthehardway.com/article/bitflipping/) 989 | - BIt Packing (https://www.cs.waikato.ac.nz/~tcs/COMP317/bitpacking.html) 990 | 991 | ## 👍 Google foobar 992 | - https://codeforces.com/blog/entry/50841 993 | - Foobar Challenge: Google’s Secret Hiring Process (https://towardsdatascience.com/how-to-get-hired-by-google-b19806ad3c62) 994 | - Google Has a Secret Hiring Challenge Called Foobar (https://betterprogramming.pub/google-has-a-secret-hiring-challenge-called-foobar-14625bfcea7a)(https://github.com/FBosler/GoogleFoobar) 995 | - Dodge The Lasers — Fantastic Question From Google’s hiring challenge (https://towardsdatascience.com/dodge-the-lasers-fantastic-question-from-googles-hiring-challenge-72363d95fec) 996 | 997 | ## 🔦 PyTorch-Related 998 | - PyTorch 2 Internals (https://blog.christianperone.com/2023/12/pytorch-2-internals-talk/) 999 | - Torch Tensor Operations (https://jhui.github.io/2018/02/09/PyTorch-Basic-operations/) 1000 | - Optimize PyTorch Performance for Speed and Memory Efficiency (18 Tips) - (https://towardsdatascience.com/optimize-pytorch-performance-for-speed-and-memory-efficiency-2022-84f453916ea6) 1001 | - Faster Deep Learning Training with PyTorch – a 2021 Guide (https://efficientdl.com/faster-deep-learning-in-pytorch-a-guide/) 1002 | - How to fine tune VERY large model if it doesn’t fit on your GPU (https://bestasoff.medium.com/how-to-fine-tune-very-large-model-if-it-doesnt-fit-on-your-gpu-3561e50859af) 1003 | - Finetuning Torchvision Models (https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html) 1004 | - Transfer Learning with ResNet in PyTorch (https://www.pluralsight.com/guides/introduction-to-resnet) 1005 | - Notes in pytorch to deal with ConvNets (https://github.com/mortezamg63/Accessing-and-modifying-different-layers-of-a-pretrained-model-in-pytorch/blob/master/README.md) 1006 | - Some important Pytorch tasks - A concise summary from a vision researcher (https://spandan-madan.github.io/A-Collection-of-important-tasks-in-pytorch/) 1007 | - PyTorch layer dimensions: what size and why? (https://towardsdatascience.com/pytorch-layer-dimensions-what-sizes-should-they-be-and-why-4265a41e01fd) 1008 | - PyTorch :: Understanding Tensors (Part 1) (https://dev.to/tbhaxor/pytorch-understanding-tensors-part-1-od8) 1009 | - VGG16 Transfer Learning - Pytorch (https://www.kaggle.com/carloalbertobarbano/vgg16-transfer-learning-pytorch) 1010 | - Pytorch to fastai, Bridging the Gap (https://muellerzr.github.io/fastblog/2021/02/14/Pytorchtofastai.html) 1011 | - A detailed example of how to generate your data in parallel with PyTorch (https://stanford.edu/~shervine/blog/pytorch-how-to-generate-data-parallel) 1012 | - Cyclic Learning Rates and One Cycle Policy (https://github.com/nachiket273/One_Cycle_Policy) 1013 | - Cyclical LR and momentums (https://github.com/sgugger/Deep-Learning/blob/master/Cyclical%20LR%20and%20momentums.ipynb) 1014 | - Pytorch Loss Functions in Plain Python (https://zhang-yang.medium.com/pytorch-loss-funtions-in-plain-python-b79c05f8b53f) 1015 | - Automatic Mixed Precision Training for Deep Learning using PyTorch (https://debuggercafe.com/automatic-mixed-precision-training-for-deep-learning-using-pytorch/) 1016 | - A developer-friendly guide to mixed precision training with PyTorch (https://spell.ml/blog/mixed-precision-training-with-pytorch-Xuk7YBEAACAASJam) 1017 | - Making Pytorch Transformer Twice as Fast on Sequence Generation (https://scale.com/blog/pytorch-improvements) 1018 | - Transformer Details Not Described in The Paper (https://tunz.kr/post/4) 1019 | - A collection of tips for speeding up learning and reasoning with PyTorch (https://qiita.com/sugulu_Ogawa_ISID/items/62f5f7adee083d96a587) 1020 | - Implement Early Stopping in PyTorch (https://qiita.com/ku_a_i/items/ba33c9ce3449da23b503) 1021 | - BERT Fine-Tuning Tutorial with PyTorch (https://mccormickml.com/2019/07/22/BERT-fine-tuning/) 1022 | - Utilizing Transformer Representations Efficiently (https://www.kaggle.com/rhtsingh/utilizing-transformer-representations-efficiently) 1023 | - Visualize BERT sequence embeddings: An unseen way (https://towardsdatascience.com/visualize-bert-sequence-embeddings-an-unseen-way-1d6a351e4568) 1024 | - PyTorch Tutorial: Paddy Disease Identification (https://www.kaggle.com/code/manabendrarout/pytorch-tutorial-paddy-disease-identification) 1025 | - PYTORCH LIGHTNING TUTORIALS (https://pytorch-lightning.readthedocs.io/en/stable/tutorials.html) 1026 | - TUTORIAL 3: INITIALIZATION AND OPTIMIZATION (https://pytorch-lightning.readthedocs.io/en/stable/notebooks/course_UvA-DL/03-initialization-and-optimization.html) 1027 | - TUTORIAL 5: TRANSFORMERS AND MULTI-HEAD ATTENTION (https://lightning.ai/docs/pytorch/stable/notebooks/course_UvA-DL/05-transformers-and-MH-attention.html) 1028 | - University of Amsterdam - UvA Deep Learning Tutorials (https://uvadlc-notebooks.readthedocs.io/en/latest/) (https://www.youtube.com/playlist?list=PLdlPlO1QhMiAkedeu0aJixfkknLRxk1nA) 1029 | - Debugging in PyTorch (https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/guide3/Debugging_PyTorch.html) 1030 | - TorchMetrics - How do we use it, and what's the difference between .update() and .forward()? (https://sebastianraschka.com/blog/2022/torchmetrics.html) (https://github.com/rasbt/torchmetrics-blog/blob/main/torchmetrics-update-forward.ipynb) 1031 | - PyTorch training codes with AverageMeter & ProgressMeter (https://docs.openvino.ai/2023.0/notebooks/302-pytorch-quantization-aware-training-with-output.html) 1032 | - Hooks: the one PyTorch trick you must know (https://tivadardanka.com/blog/hooks-the-one-pytorch-trick-you-must-know) 1033 | - Chaim Rand 1034 | - Part 1: PyTorch Model Performance Analysis and Optimization - Use PyTorch Profiler and TensorBoard (https://medium.com/towards-data-science/pytorch-model-performance-analysis-and-optimization-10c3c5822869) 1035 | - Part 2: Identify and Reduce CPU Computation In Your Training Step (https://medium.com/towards-data-science/pytorch-model-performance-analysis-and-optimization-part-2-3bc241be91) 1036 | - Part 3: Reduce "Cuda Memcpy Async" Events and Why You Should Beware of Boolean Mask Operations (https://medium.com/towards-data-science/pytorch-model-performance-analysis-and-optimization-part-3-1c5876d78fe2) 1037 | - Part 4: Solving Bottlenecks on the Data Input Pipeline (https://medium.com/towards-data-science/solving-bottlenecks-on-the-data-input-pipeline-with-pytorch-profiler-and-tensorboard-5dced134dbe9) 1038 | - Part 5: How to Optimize Your DL Data-Input Pipeline with a Custom PyTorch Operator (https://medium.com/towards-data-science/how-to-optimize-your-dl-data-input-pipeline-with-a-custom-pytorch-operator-7f8ea2da5206) 1039 | - Part 6: Identify and Analyze Performance Issues in the Backward Pass with PyTorch Profiler, PyTorch Hooks, and TensorBoard (https://medium.com/towards-data-science/pytorch-model-performance-analysis-and-optimization-part-6-b87412a0371b) 1040 | - Part 7: Efficient Metric Collection in PyTorch: Avoiding the Performance Pitfalls of TorchMetrics (https://chaimrand.medium.com/efficient-metric-collection-in-pytorch-avoiding-the-performance-pitfalls-of-torchmetrics-0dea81413681) 1041 | - Chaim Rand 1042 | - Part 1: Accelerating PyTorch Training Workloads with FP8 (https://medium.com/towards-data-science/accelerating-pytorch-training-workloads-with-fp8-5a5123aec7d7) 1043 | - Part 2: PyTorch Native FP8 Data Types (https://medium.com/towards-data-science/pytorch-native-fp8-fedc06f1c9f7) 1044 | - Chaim Rand 1045 | - Part 1: Accelerating AI/ML Model Training with Custom Operators (https://medium.com/towards-data-science/accelerating-ai-ml-model-training-with-custom-operators-163ef2a04b12) 1046 | - Part 2: Unleashing the Power of Triton: Mastering GPU Kernel Optimization in Python (https://medium.com/towards-data-science/unleashing-the-power-of-triton-mastering-gpu-kernel-optimization-in-python-160a3f52701e) 1047 | - Part 3: The Rise of Pallas: Unlocking TPU Potential with Custom Kernels (https://medium.com/towards-data-science/the-rise-of-pallas-unlocking-tpu-potential-with-custom-kernels-67be10ab846a) 1048 | - Part 3A: Implementing Sequential Algorithms on TPU (https://medium.com/towards-data-science/implementing-sequential-algorithms-on-tpu-41d75c6aaa95) 1049 | - Part 4: On the Programmability of AWS Trainium and Inferentia (https://medium.com/towards-data-science/on-the-programmability-of-aws-trainium-and-inferentia-cd455826e26c) 1050 | - 1051 | 1052 | ## 🔦 PyTorch-Related Discussions 1053 | - How to modify a pretrained model (https://discuss.pytorch.org/t/how-to-modify-a-pretrained-model/60509) 1054 | - `Module.children()` vs `Module.modules()` (https://discuss.pytorch.org/t/module-children-vs-module-modules/4551) 1055 | - What is the distinct usage of the `AdaptiveConcatPool2d` layer? (https://forums.fast.ai/t/what-is-the-distinct-usage-of-the-adaptiveconcatpool2d-layer/7600) 1056 | - Splitting a pretrained model in groups of layers (https://forums.fast.ai/t/splitting-a-pretrained-model-in-groups-of-layers/33012/2) 1057 | - `x.data` (https://stackoverflow.com/questions/51743214/is-data-still-useful-in-pytorch) 1058 | - `model.eval()` vs with `torch.no_grad()` (https://discuss.pytorch.org/t/model-eval-vs-with-torch-no-grad/19615) 1059 | - How do we resume training by using the last LR? (https://www.kaggle.com/c/seti-breakthrough-listen/discussion/247574) 1060 | - Tensor view vs. permute (https://stackoverflow.com/questions/51143206/difference-between-tensor-permute-and-tensor-view-in-pytorch) 1061 | - Torch `stack()` vs. `cat()` (https://stackoverflow.com/questions/54307225/whats-the-difference-between-torch-stack-and-torch-cat-functions/54307331) 1062 | - Why do transformers use layer norm instead of batch norm? (https://stats.stackexchange.com/questions/474440/why-do-transformers-use-layer-norm-instead-of-batch-norm) 1063 | - Deep Learning normalization methods (https://tungmphung.com/deep-learning-normalization-methods/) 1064 | - PyTorch vs TensorFlow in 2022 (https://www.assemblyai.com/blog/pytorch-vs-tensorflow-in-2022/) 1065 | - What is the difference between `register_buffer` and `register_parameter` of `nn.Module` (https://discuss.pytorch.org/t/what-is-the-difference-between-register-buffer-and-register-parameter-of-nn-module/32723) (https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.register_buffer) 1066 | - `Model.named_parameters()` will lose some layer modules (https://discuss.pytorch.org/t/model-named-parameters-will-lose-some-layer-modules/14588) 1067 | - `model.parameters()`, `model.named_parameters()`, `model.state_dict()` (https://blog.csdn.net/qq_36429555/article/details/118609604) 1068 | 1069 | ## ⏩ fast.ai 1070 | - Jeremy Howard - Kaggle Notebooks (https://www.kaggle.com/jhoward/notebooks) 1071 | - fastkaggle (https://fastai.github.io/fastkaggle/) 1072 | 1073 | ## 📚 NVIDIA eBooks 1074 | - A Beginner’s Guide to Large Language Models (https://resources.nvidia.com/en-us-large-language-model-ebooks) 1075 | - End-To-End Speech AI Pipelines (https://resources.nvidia.com/en-us-speech-ai-ebooks-nurture/part-2) 1076 | 1077 | ## 🧞 Genomic Data Science 1078 | - https://github.com/fylls/genomic-data-science 1079 | - https://github.com/fylls/genome-sequencing 1080 | 1081 | ## 🤖 Transformer Architecture / Anatomy / Guide 1082 | - AI Canon - a curated list of resources (https://a16z.com/2023/05/25/ai-canon/) 1083 | - A Very Gentle Introduction to Large Language Models without the Hype - 38 min read (https://mark-riedl.medium.com/a-very-gentle-introduction-to-large-language-models-without-the-hype-5f67941fa59e) 1084 | - The Animated Transformer (https://prvnsmpth.github.io/animated-transformer/) 1085 | - Transformer Anatomy guide (https://www.kaggle.com/code/pastorsoto/transformer-anatomy-guide) 1086 | - Notebooks for the book: Natural Language Processing with Transformers (https://github.com/nlp-with-transformers/notebooks) 1087 | - The Annotated Transformer - with PyTorch codes 1088 | - Original: (http://nlp.seas.harvard.edu/2018/04/03/attention.html) 1089 | - v2022: (http://nlp.seas.harvard.edu/annotated-transformer/) 1090 | - Transformers: How Do They Transform Your Data? - explanation with codes (https://towardsdatascience.com/transformers-how-do-they-transform-your-data-72d69e383e0d) (https://github.com/maxime7770/Transformers-Insights) 1091 | - Attention? Attention! (https://lilianweng.github.io/posts/2018-06-24-attention/) 1092 | - The Transformer Family (https://lilianweng.github.io/posts/2020-04-07-the-transformer-family/) 1093 | - The Transformer Family Version 2.0 (https://lilianweng.github.io/posts/2023-01-27-the-transformer-family-v2/) 1094 | - Beautifully Illustrated: NLP Models from RNN to Transformer (https://towardsdatascience.com/beautifully-illustrated-nlp-models-from-rnn-to-transformer-80d69faf2109) 1095 | - Understanding Transformers: A Step-by-Step Math Example — Part 1 (https://medium.com/@fareedkhandev/understanding-transformers-a-step-by-step-math-example-part-1-a7809015150a) 1096 | - Solving Transformer by Hand: A Step-by-Step Math Example (https://levelup.gitconnected.com/understanding-transformers-from-start-to-end-a-step-by-step-math-example-16d4e64e6eb1) 1097 | - Building a Million-Parameter LLM from Scratch Using Python - A Step-by-Step Guide to Replicating LLaMA Architecture (https://levelup.gitconnected.com/building-a-million-parameter-llm-from-scratch-using-python-f612398f06c2) 1098 | - Understanding and Coding Self-Attention, Multi-Head Attention, Cross-Attention, and Causal-Attention in LLMs (https://magazine.sebastianraschka.com/p/understanding-and-coding-self-attention) 1099 | - https://github.com/rasbt/LLMs-from-scratch/blob/main/ch03/01_main-chapter-code/ch03.ipynb 1100 | - https://github.com/rasbt/LLMs-from-scratch/blob/main/ch03/01_main-chapter-code/multihead-attention.ipynb 1101 | - https://github.com/rasbt/LLMs-from-scratch/tree/main/ch03/02_bonus_efficient-multihead-attention 1102 | - Concept of self-attention (https://www.linkedin.com/posts/ugcPost-6882314741088321536-PNqG) (https://www.linkedin.com/feed/update/urn:li:activity:6879010048421445633) 1103 | - Transformers from Scratch - great explanation on dot products and matrix multiplication (https://e2eml.school/transformers.html) 1104 | - TRANSFORMERS FROM SCRATCH (https://peterbloem.nl/blog/transformers) 1105 | - Transformers From Scratch (https://blog.matdmiller.com/posts/2023-06-10_transformers/notebook.html) 1106 | - How Transformers work in deep learning and NLP: an intuitive introduction (https://theaisummer.com/transformer/) 1107 | - Getting Meaning from Text: Self-attention Step-by-step Video (https://pub.towardsai.net/getting-meaning-from-text-self-attention-step-by-step-video-7d8f49694f89) (https://www.youtube.com/watch?v=-9vVhYEXeyQ&t=570s) 1108 | - Transformer Architecture: The Positional Encoding (https://kazemnejad.com/blog/transformer_architecture_positional_encoding/) 1109 | - Explained: Multi-head Attention (Part 1) (https://storrs.io/attention/) 1110 | - Explained: Multi-head Attention (Part 2) (https://storrs.io/multihead-attention/) 1111 | - Deep learning explainer: a simple single cell classification model (https://storrs.io/sc-deep-learning-explainer/) 1112 | - Accelerating Large Language Models with Accelerated Transformers (https://pytorch.org/blog/accelerating-large-language-models/) 1113 | - Step-by-Step Illustrated Explanations of Transformer (https://medium.com/@yulemoon/detailed-explanations-of-transformer-step-by-step-dc32d90b3a98) 1114 | - An In-Depth Look at the Transformer Based Models (https://medium.com/@yulemoon/an-in-depth-look-at-the-transformer-based-models-22e5f5d17b6b) 1115 | - Anti-hype LLM reading list (https://gist.github.com/veekaybee/be375ab33085102f9027853128dc5f0e) 1116 | - GPT in 60 Lines of NumPy (https://jaykmody.com/blog/gpt-from-scratch/) 1117 | - x-transformers - A concise but fully-featured transformer, complete with a set of promising experimental features from various papers (https://github.com/lucidrains/x-transformers/tree/main) 1118 | - Transformer Taxonomy (https://kipp.ly/transformer-taxonomy/) 1119 | - Large Language Models: SBERT — Sentence-BERT (https://towardsdatascience.com/sbert-deb3d4aef8a4) 1120 | - How context sizes of 100k tokens and longer are achieved (https://www.linkedin.com/posts/andriyburkov_in-case-you-were-wondering-how-context-sizes-activity-7154922110354604032-Tx1a) 1121 | - KV caching (https://medium.com/@plienhar/llm-inference-series-3-kv-caching-unveiled-048152e461c8) (https://medium.com/@plienhar/llm-inference-series-4-kv-caching-a-deeper-look-4ba9a77746c8) 1122 | 1123 | ## 🤖 Transformer / Attention / LLM Visualization 1124 | - BertViz - tool for visualizing attention in Transformer model (https://github.com/jessevig/bertviz) 1125 | - LLM Visualization (https://bbycroft.net/llm) 1126 | - AttentionViz: A Global View of Transformer Attention (https://catherinesyeh.github.io/attn-docs/) 1127 | - Inside the Matrix: Visualizing Matrix Multiplication, Attention and Beyond (https://pytorch.org/blog/inside-the-matrix/) 1128 | - Explainable AI: Visualizing Attention in Transformers (https://generativeai.pub/explainable-ai-visualizing-attention-in-transformers-4eb931a2c0f8) (https://www.topbots.com/deconstructing-bert-part-1/) (https://www.topbots.com/deconstructing-bert-part-2/) (https://www.topbots.com/openai-gpt-2-visualization/) 1129 | - Deconstructing BERT, Part 2: Visualizing the Inner Workings of Attention (https://towardsdatascience.com/deconstructing-bert-part-2-visualizing-the-inner-workings-of-attention-60a16d86b5c1) 1130 | - Transformers Explained Visually (Part 3): Multi-head Attention, deep dive (https://towardsdatascience.com/transformers-explained-visually-part-3-multi-head-attention-deep-dive-1c1ff1024853) 1131 | - A Visual Guide to Vision Transformers (https://blog.mdturp.ch/posts/2024-04-05-visual_guide_to_vision_transformer.html) 1132 | - llama3 implemented from scratch (https://github.com/naklecha/llama3-from-scratch/blob/main/README.md) 1133 | - The Illustrated AlphaFold (https://elanapearl.github.io/blog/2024/the-illustrated-alphafold/) 1134 | - A Visual Guide to Quantization (https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-quantization) 1135 | - A Visual Guide to Mamba and State Space Models (https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-mamba-and-state) 1136 | - Transformer Explainer (https://poloclub.github.io/transformer-explainer/) 1137 | - Stable Diffusion Explained Step-by-Step with Visualization (https://medium.com/polo-club-of-data-science/stable-diffusion-explained-for-everyone-77b53f4f1c4) (https://poloclub.github.io/diffusion-explainer/) 1138 | - A Visual Guide to Mixture of Experts (MoE) (https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-mixture-of-experts) 1139 | 1140 | ## 🤖 Transformer Maths 1141 | - Numbers every LLM Developer should know (https://github.com/ray-project/llm-numbers#1-mb-gpu-memory-required-for-1-token-of-output-with-a-13b-parameter-model) 1142 | - Transformer Math 101 (https://blog.eleuther.ai/transformer-math/) 1143 | - LLM Parameter Counting (https://kipp.ly/transformer-param-count/) 1144 | - Transformer Inference Arithmetic (https://kipp.ly/transformer-inference-arithmetic/) 1145 | - Calculating GPU memory for serving LLMs (https://www.substratus.ai/blog/calculating-gpu-memory-for-llm) 1146 | - Can you run it? LLM version (https://huggingface.co/spaces/Vokturz/can-it-run-llm) 1147 | - 🤗 Model Memory Calculator (https://huggingface.co/spaces/hf-accelerate/model-memory-usage) 1148 | - VRAM Estimator - Estimate GPU VRAM usage of transformer-based models (https://vram.asmirnov.xyz/) 1149 | - Model training anatomy (https://huggingface.co/docs/transformers/model_memory_anatomy) 1150 | - Estimate the Memory Consumption of LLMs for Inference and Fine-tuning (https://kaitchup.substack.com/p/estimate-the-memory-consumption-of) (https://colab.research.google.com/drive/1J_r9gB849RL4R4PXC8o05KkNeUXhBXoO?usp=sharing) 1151 | - LLM Explorer (https://llm.extractum.io/) 1152 | - Memory Requirements for LLM Training and Inference (https://medium.com/@manuelescobar-dev/memory-requirements-for-llm-training-and-inference-97e4ab08091b) 1153 | - Memory-efficient Model Weight Loading (https://github.com/rasbt/LLMs-from-scratch/blob/main/ch05/08_memory_efficient_weight_loading/memory-efficient-state-dict.ipynb) 1154 | 1155 | ## 🤖 Transformer Libraries 1156 | - AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration [Paper] (https://github.com/mit-han-lab/llm-awq) 1157 | - AutoAWQ (https://github.com/casper-hansen/AutoAWQ) 1158 | - Microsoft DeepSpeed - Deep learning optimization software suite for both training and inference (https://github.com/microsoft/DeepSpeed) (https://www.deepspeed.ai/) 1159 | - DeepSpeed Chat - Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales (https://github.com/microsoft/DeepSpeedExamples/tree/master/applications/DeepSpeed-Chat) (https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-chat) 1160 | - DeepSpeed's Bag of Tricks for Speed & Scale (https://www.kolaayonrinde.com/blog/2023/07/14/deepspeed-train.html) 1161 | - 🤗 PEFT: Parameter-Efficient Fine-Tuning of Billion-Scale Models on Low-Resource Hardware (https://huggingface.co/blog/peft) (https://github.com/huggingface/peft) 1162 | - 🤗 PEFT Documentation (https://huggingface.co/docs/peft/index) 1163 | - 🤗 PEFT Examples (https://github.com/huggingface/peft/tree/main/examples) 1164 | - 🤗 PEFT Patch Release (https://github.com/huggingface/peft/releases) 1165 | - 🤗 TRL - Transformer Reinforcement Learning (https://github.com/huggingface/trl) (https://huggingface.co/docs/trl/index) 1166 | - bitsandbytes - 8-bit optimizers, matrix multiplication (LLM.int8()), and quantization functions (https://github.com/TimDettmers/bitsandbytes) 1167 | - GPTQ: Accurate Post-training Compression for Generative Pretrained Transformers (https://github.com/IST-DASLab/gptq) 1168 | - AutoGPTQ: LLMs quantization package with user-friendly apis, based on GPTQ algorithm (https://github.com/PanQiWei/AutoGPTQ) 1169 | - Quantize 🤗 Transformers models (https://huggingface.co/docs/transformers/main_classes/quantization) 1170 | - Which Quantization Method is Right for You? (GPTQ vs. GGUF vs. AWQ) (https://maartengrootendorst.substack.com/p/which-quantization-method-is-right) 1171 | - Optimum-Benchmark 🏋️ (https://github.com/huggingface/optimum-benchmark) 1172 | - NEFTune - add random noise to the embedding vectors of the training data during the forward pass of fine-tuning (https://github.com/neelsjain/NEFTune) 1173 | - LoRA+: Efficient Low Rank Adaptation of Large Models (https://github.com/nikhil-ghosh-berkeley/loraplus) 1174 | - DoRA: Weight-Decomposed Low-Rank Adaptation (https://github.com/catid/dora/tree/main) 1175 | - `tensor_parallel` - much faster than huggingface's `device_map` and lightweight than vLLM? (https://github.com/BlackSamorez/tensor_parallel) 1176 | - Nanotron - Minimalistic large language model 3D-parallelism training (https://github.com/huggingface/nanotron/) 1177 | - FastChat - platform for training, serving, and evaluating large language model based chatbots (https://github.com/lm-sys/FastChat) 1178 | - Half-Quadratic Quantization (HQQ) (https://mobiusml.github.io/hqq_blog/) (https://github.com/mobiusml/hqq) 1179 | - AQLM - Extreme Compression of Large Language Models via Additive Quantization (https://github.com/Vahe1994/AQLM) (https://towardsdatascience.com/the-aqlm-quantization-algorithm-explained-8cf33e4a783e) 1180 | - torchtune - A Native-PyTorch Library for LLM Fine-tuning (https://github.com/pytorch/torchtune) 1181 | - torchao: PyTorch Architecture Optimization (https://github.com/pytorch/ao/) 1182 | - PyTorch Native Architecture Optimization: torchao (https://pytorch.org/blog/pytorch-native-architecture-optimization/) 1183 | - Prefect - Modern workflow orchestration for data and ML engineers (https://www.prefect.io/) 1184 | - Modal - serverless platform to run generative AI models, large-scale batch jobs, job queues, etc (https://modal.com/) 1185 | - Beating Proprietary Models with a Quick Fine-Tune - Finetuning Quora Embeddings (https://modal.com/blog/fine-tuning-embeddings) (https://github.com/567-labs/fastllm/blob/main/applications/finetune-quora-embeddings/Readme.md) 1186 | - Fine-tune an LLM in minutes (ft. Llama 2, CodeLlama, Mistral, etc.) (https://modal.com/docs/examples/llm-finetuning) (https://github.com/modal-labs/llm-finetuning) 1187 | - Model Explorer - a powerful graph visualization tool that helps one understand, debug, and optimize ML models (https://ai.google.dev/edge/model-explorer) (https://research.google/blog/model-explorer/) 1188 | - Intel AutoRound - weight-only quantization algorithm designed specifically for low-bit LLM inference (https://github.com/intel/auto-round) (https://medium.com/intel-analytics-software/autoround-sota-weight-only-quantization-algorithm-for-llms-across-hardware-platforms-99fe6eac2861) 1189 | 1190 | 1191 | ## 🤖 Transformer Toolkit / Techniques / Methods 1192 | - 🤗 The Large Language Model Training Handbook (https://github.com/huggingface/llm_training_handbook) 1193 | - 🤗 The Large Language Model Training Playbook (https://github.com/huggingface/large_language_model_training_playbook) 1194 | - 🤗 Dataset map method - how to pass argument to the function (https://discuss.huggingface.co/t/dataset-map-method-how-to-pass-argument-to-the-function/16274) 1195 | - 🤗 Quantization (https://huggingface.co/docs/transformers/quantization) 1196 | - 🤗 Generation with LLMs - Common pitfalls, etc (https://huggingface.co/docs/transformers/main/llm_tutorial) 1197 | - 🤗 Getting the most out of LLMs - Optimizing LLMs for Speed and Memory (https://huggingface.co/docs/transformers/main/llm_tutorial_optimization) 1198 | - 🤗 LLM prompting guide (https://huggingface.co/docs/transformers/main/tasks/prompting) 1199 | - 🤗 Templates for Chat Models (https://huggingface.co/docs/transformers/main/chat_templating) 1200 | - 🤗 Text generation strategies & Decoding strategies (https://huggingface.co/docs/transformers/main/generation_strategies) 1201 | - Efficient Training Techniques (https://huggingface.co/docs/transformers/perf_train_gpu_one) 1202 | - Multimodal Toolkit - Transformers with Tabular Data (https://github.com/georgian-io/Multimodal-Toolkit) 1203 | - LiGO - Learning to grow machine-learning models - New LiGO technique accelerates training of large machine-learning models (https://news.mit.edu/2023/new-technique-machine-learning-models-0322) 1204 | - Finetuning Large Language Models (https://magazine.sebastianraschka.com/p/finetuning-large-language-models) 1205 | - Adapters (https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/v1.10.0/core/adapters/intro.html) (https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/02_NeMo_Adapters.ipynb) 1206 | - P-tuning and prompt tuning (https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/nlp/nemo_megatron/prompt_learning.html) (https://github.com/NVIDIA/NeMo/blob/stable/tutorials/nlp/Multitask_Prompt_and_PTuning.ipynb) 1207 | - Difference between P-tuning and Prefix Tuning (https://www.reddit.com/r/MachineLearning/comments/14pkibg/comment/jqkdam8/) 1208 | - https://github.com/huggingface/peft/issues/123 1209 | - RLHF: Reinforcement Learning from Human Feedback (https://huyenchip.com/2023/05/02/rlhf.html) 1210 | - Building LLM applications for production (https://huyenchip.com/2023/04/11/llm-engineering.html) 1211 | - Instruction tuning datasets to train (text and multi-modal) chat-based LLMs (GPT-4, ChatGPT,LLaMA,Alpaca) (https://github.com/yaodongC/awesome-instruction-dataset) 1212 | - A Gentle Introduction to 8-bit Matrix Multiplication for transformers at scale using Hugging Face Transformers, Accelerate and bitsandbytes (https://huggingface.co/blog/hf-bitsandbytes-integration) 1213 | - Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA (https://huggingface.co/blog/4bit-transformers-bitsandbytes) 1214 | - General notebook (https://colab.research.google.com/drive/1ge2F1QSK8Q7h0hn3YKuBCOAS0bK8E0wf?usp=sharing#scrollTo=OQdUx-aQScdR) 1215 | - Training notebook (https://colab.research.google.com/drive/1VoYNfYDKcKRQRor98Zbf2-9VQTtGJ24k?usp=sharing#scrollTo=jq0nX33BmfaC) 1216 | - Making LLMs lighter with AutoGPTQ and transformers (https://huggingface.co/blog/gptq-integration) (https://arxiv.org/pdf/2210.17323.pdf) 1217 | - Introduction to Quantization cooked in 🤗 with 💗🧑‍🍳 (https://huggingface.co/blog/merve/quantization) 1218 | - LLM.int8() and Emergent Features (https://timdettmers.com/2022/08/17/llm-int8-and-emergent-features/) 1219 | - 📺 State of GPT - Learn about the training pipeline of GPT assistants (https://www.youtube.com/watch?v=bZQun8Y4L2A&t=1s) 1220 | - Google "We Have No Moat, And Neither Does OpenAI" (https://www.semianalysis.com/p/google-we-have-no-moat-and-neither) 1221 | - Emerging Architectures for LLM Applications (https://a16z.com/2023/06/20/emerging-architectures-for-llm-applications/) 1222 | - The Secret Sauce behind 100K context window in LLMs: all tricks in one place (https://blog.gopenai.com/how-to-speed-up-llms-and-use-100k-context-window-all-tricks-in-one-place-ffd40577b4c) 1223 | - All You Need to Know to Build Your First LLM App (https://towardsdatascience.com/all-you-need-to-know-to-build-your-first-llm-app-eb982c78ffac) $$ 1224 | - Neural Networks: Zero to Hero - A course by Andrej Karpathy - Syllabus & links to videos (https://karpathy.ai/zero-to-hero.html) 1225 | - 📺 Let's build GPT: from scratch, in code, spelled out (https://www.youtube.com/watch?v=kCc8FmEb1nY) 1226 | - 📺 Let's reproduce GPT-2 (124M) (https://www.youtube.com/watch?v=l8pRSuU81PU) 1227 | - Decoding Strategies in Large Language Models (https://towardsdatascience.com/decoding-strategies-in-large-language-models-9733a8f70539) $$ 1228 | - Interactively fine-tune Falcon-40B and other LLMs on Amazon SageMaker Studio notebooks using QLoRA (https://aws.amazon.com/blogs/machine-learning/interactively-fine-tune-falcon-40b-and-other-llms-on-amazon-sagemaker-studio-notebooks-using-qlora/) (https://github.com/aws-samples/amazon-sagemaker-generativeai/blob/main/studio-notebook-fine-tuning/falcon-40b-qlora-finetune-summarize.ipynb) 1229 | - [Codes] CVPR 2023 - Scaling PyTorch Model Training With Minimal Code Changes (https://github.com/rasbt/cvpr2023) 1230 | - [Codes] LLM-finetuning-scripts (https://github.com/rasbt/LLM-finetuning-scripts/) 1231 | - [Codes] LoRA (https://github.com/rasbt/low-rank-adaptation-blog) (https://lightning.ai/lightning-ai/studios/code-lora-from-scratch?utm_source=substack&utm_medium=email) 1232 | - [Codes] Gradient Accumulation (https://github.com/rasbt/gradient-accumulation-blog/) 1233 | - [Codes] Optimizing PyTorch Memory Usage (https://github.com/rasbt/pytorch-memory-optim/) 1234 | - Patterns for Building LLM-based Systems & Products (https://eugeneyan.com/writing/llm-patterns/) 1235 | - Fine-tuning Alpaca and LLaMA: Training on a Custom Dataset - for sentiment analysis, using 🤗 PEFT LoRA (https://www.mlexpert.io/machine-learning/tutorials/alpaca-fine-tuning) (https://colab.research.google.com/drive/1X85FLniXx_NyDsh_F_aphoIAy63DKQ7d?usp=sharing) 1236 | - Extended Guide: Instruction-tune Llama 2 - focus on creating the instruction dataset (https://www.philschmid.de/instruction-tune-llama-2) 1237 | - Fine-tuning LLMs - in simple notes - ROUGE, BLEU metrics - (https://teetracker.medium.com/fine-tuning-llms-9fe553a514d0) 1238 | - Fine-tune Llama 2 with DPO - Direct Preference Optimization (https://huggingface.co/blog/dpo-trl) 1239 | - Fine-Tuning Llama 2.0 with Single GPU Magic (https://ai.plainenglish.io/fine-tuning-llama2-0-with-qloras-single-gpu-magic-1b6a6679d436) 1240 | - Understanding Llama2: KV Cache, Grouped Query Attention, Rotary Embedding and More (https://ai.plainenglish.io/understanding-llama2-kv-cache-grouped-query-attention-rotary-embedding-and-more-c17e5f49a6d7) 1241 | - Mastering BERT Model: Building it from Scratch with Pytorch (https://medium.com/data-and-beyond/complete-guide-to-building-bert-model-from-sratch-3e6562228891) 1242 | - Exploring Dolly 2.0: Fine Tuning Your Own ChatGPT-like Model (https://ai.plainenglish.io/exploring-dolly-2-0-a-guide-to-training-your-own-chatgpt-like-model-dd9b785ff1df) 1243 | - GPT Model Behind the Scene: Exploring it from scratch with Pytorch (https://ai.plainenglish.io/creating-and-exploring-gpt-from-scratch-ffe84ac415a9) 1244 | - Fine-Tune Your Own Llama 2 Model in a Colab Notebook (https://towardsdatascience.com/fine-tune-your-own-llama-2-model-in-a-colab-notebook-df9823a04a32) $$ (https://colab.research.google.com/drive/1PEQyJO1-f6j0S_XJ8DV50NkpzasXkrzd?usp=sharing) (https://mlabonne.github.io/blog/posts/Fine_Tune_Your_Own_Llama_2_Model_in_a_Colab_Notebook.html) 1245 | - Fine tune Llama v2 models on Guanaco Dataset (https://gist.github.com/younesbelkada/9f7f75c94bdc1981c8ca5cc937d4a4da) 1246 | - Fine-tune Llama 2 with dolly-15k on a free Google Colab instance (https://colab.research.google.com/drive/134o_cXcMe_lsvl15ZE_4Y75Kstepsntu?usp=sharing) 1247 | - Fine-tune Llama 2 with SFT and DPO (https://medium.com/@anchen.li/fine-tune-llama-2-with-sft-and-dpo-8b57cf3ec69) 1248 | - Fine-Tuning LLaMA 2 Models using a single GPU, QLoRA and AI Notebooks (https://blog.ovhcloud.com/fine-tuning-llama-2-models-using-a-single-gpu-qlora-and-ai-notebooks/) 1249 | - Fine-Tuning Embedding Model with PEFT and LoRA (https://medium.com/@kelvin.lu.au/fine-tuning-embedding-model-with-peft-and-lora-3b6f08987c24) 1250 | - [Video] Jeremy Howard: A Hackers' Guide to Language Models (https://www.youtube.com/watch?v=jkrNMKz9pWU) 1251 | - 🤗 The Alignment Handbook - Robust recipes to align language models with human and AI preferences (https://github.com/huggingface/alignment-handbook/tree/main) (https://www.linkedin.com/posts/philipp-schmid-a6a2bb196_what-if-we-could-distill-the-alignment-from-activity-7123567109128806400-jgEg) (https://www.linkedin.com/posts/thom-wolf_ai-knowledgesharing-opensource-activity-7123588019839713280-EqhV) (https://arxiv.org/pdf/2310.16944.pdf) 1252 | - Efficient Fine-Tuning with LoRA: A Guide to Optimal Parameter Selection for Large Language Models (https://www.databricks.com/blog/efficient-fine-tuning-lora-guide-llms) 1253 | - LoRA — Intuitively and Exhaustively Explained (https://towardsdatascience.com/lora-intuitively-and-exhaustively-explained-e944a6bff46b) $$ (https://github.com/DanielWarfield1/MLWritingAndResearch/blob/main/LoRA.ipynb) 1254 | - Fine-Tuning LLMs: LoRA or Full-Parameter? An in-depth Analysis with Llama 2 (https://www.anyscale.com/blog/fine-tuning-llms-lora-or-full-parameter-an-in-depth-analysis-with-llama-2) 1255 | - Out-of-Domain Finetuning to Bootstrap Hallucination Detection (https://eugeneyan.com/writing/finetuning/) (https://github.com/eugeneyan/visualizing-finetunes) 1256 | - Faster debug and development with tiny models, tokenizers and datasets (https://github.com/stas00/ml-engineering/blob/master/transformers/make-tiny-models.md) (https://huggingface.co/stas/tiny-random-llama-2) (https://huggingface.co/stas/tiny-random-llama-2/blob/main/make_tiny_model.py) 1257 | - Mastering LLM Techniques: Inference Optimization (https://developer.nvidia.com/blog/mastering-llm-techniques-inference-optimization/) 1258 | - The Novice's LLM Training Guide (https://rentry.org/llm-training) 1259 | - Fast Llama 2 on CPUs With Sparse Fine-Tuning and DeepSparse (https://neuralmagic.com/blog/fast-llama-2-on-cpus-with-sparse-fine-tuning-and-deepsparse/) 1260 | - LLM Distillation Playbook (by Predibase) - Practical best practices for distilling large language models (https://github.com/predibase/llm_distillation_playbook) 1261 | - Preference Tuning LLMs with Direct Preference Optimization Methods - evaluation of Direct Preference Optimization (DPO), Identity Preference Optimisation (IPO) and Kahneman-Tversky Optimisation (KTO) (https://huggingface.co/blog/pref-tuning) 1262 | - Merge Large Language Models with mergekit (https://towardsdatascience.com/merge-large-language-models-with-mergekit-2118fb392b54) 1263 | - 🤗 How to Fine-Tune LLMs in 2024 with Hugging Face (https://www.philschmid.de/fine-tune-llms-in-2024-with-trl) 1264 | - 🤗 makeMoE: Implement a Sparse Mixture of Experts Language Model from Scratch (https://huggingface.co/blog/AviSoori1x/makemoe-from-scratch) 1265 | - Function Calling Datasets, Training and Inference (https://www.youtube.com/watch?v=hHn_cV5WUDI) 1266 | - 📺 Fine tuning Optimizations - DoRA, NEFT, LoRA+, Unsloth (https://www.youtube.com/watch?v=ae2lbmtTY5A) 1267 | - DoRA Demystified: Visualising Weight-Decomposed Low-Rank Adaptation (https://shreyassk.substack.com/p/visualising-dora-weight-decomposed) (https://github.com/shreyassks/DoRA/) 1268 | - GaLore: Advancing Large Model Training on Consumer-grade Hardware (https://huggingface.co/blog/galore) (https://github.com/jiaweizzhao/galore) 1269 | - Memory-efficient LLM Training with GaLore (https://medium.com/@geronimo7/llm-training-on-consumer-gpus-with-galore-d25075143cfb) (https://github.com/geronimi73/3090_shorts/blob/main/nb_galore_llama2-7b.ipynb) 1270 | - An Overview of the LoRA Family (https://towardsdatascience.com/an-overview-of-the-lora-family-515d81134725) 1271 | - Efficiently fine-tune Llama 3 with PyTorch FSDP and Q-Lora (https://www.philschmid.de/fsdp-qlora-llama3) (https://github.com/philschmid/deep-learning-pytorch-huggingface/blob/main/training/scripts/run_fsdp_qlora.py) 1272 | - FlexAttention: The Flexibility of PyTorch with the Performance of FlashAttention (https://pytorch.org/blog/flexattention/) 1273 | - Full Attention, Standard Causal Masking, Sliding Window Attention, Prefix LM (Bidirectional + Causal), Document Masking, Stand-Alone Self-Attention Masking, NATTEN Masking, Alibi Bias, Tanh Soft-Capping, Nested Jagged Tensor, Flamingo Cross Attention 1274 | - Attention Gym - Helpful tools and examples for working with flex-attention (https://github.com/pytorch-labs/attention-gym) 1275 | - Quantization-Aware Training for Large Language Models with PyTorch (https://pytorch.org/blog/quantization-aware-training/) (https://pytorch.org/torchtune/main/tutorials/qat_finetune.html) 1276 | - 🤗 Efficient Deep Learning: A Comprehensive Overview of Optimization Techniques (https://huggingface.co/blog/Isayoften/optimization-rush) 1277 | - FLUTE: Flexible Lookup Table Engine for LUT-quantized LLMs (https://github.com/HanGuo97/flute) 1278 | - 🤗 Fine-tuning LLMs to 1.58bit: extreme quantization made easy - with BitNet archictecture (https://huggingface.co/blog/1_58_llm_extreme_quantization) 1279 | 1280 | ## 🤖 RAG 1281 | - A Guide on 12 Tuning Strategies for Production-Ready RAG Applications (https://towardsdatascience.com/a-guide-on-12-tuning-strategies-for-production-ready-rag-applications-7ca646833439#341d) 1282 | - Advanced RAG Techniques: an Illustrated Overview (https://pub.towardsai.net/advanced-rag-techniques-an-illustrated-overview-04d193d8fec6) 1283 | - Advanced RAG Techniques (https://www.pinecone.io/learn/advanced-rag-techniques/) 1284 | - A Cheat Sheet and Some Recipes For Building Advanced RAG (https://blog.llamaindex.ai/a-cheat-sheet-and-some-recipes-for-building-advanced-rag-803a9d94c41b) 1285 | - https://d3ddy8balm3goa.cloudfront.net/llamaindex/rag-cheat-sheet-final.svg 1286 | - RAG cheatsheet (https://miro.com/app/board/uXjVNvklNmc=/) 1287 | - From paper to prod! A guide to improving your semantic search with HyDE (https://aimodels.substack.com/p/from-paper-to-prod-a-guide-to-improving) 1288 | - Improving the Semantic Search Tool (https://puddles-of-water.medium.com/improving-the-semantic-search-tool-ef0442f7e972) 1289 | - Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval (https://huggingface.co/blog/embedding-quantization) 1290 | - 100x Faster — Scaling Your RAG App for Billions of Embeddings - Computing Cosine Similarity in parallel - using Chunkdot (https://medium.com/gitconnected/100x-faster-scaling-your-rag-app-for-billions-of-embeddings-ded34fccd16a) (https://github.com/rragundez/chunkdot/) 1291 | - Advanced Retrieval for AI with Chroma (https://www.deeplearning.ai/short-courses/advanced-retrieval-for-ai/) 1292 | - Chat with your code: RAG with Weaviate and LlamaIndex (https://lightning.ai/weaviate/studios/chat-with-your-code-rag-with-weaviate-and-llamaindex) 1293 | - The 4 Advanced RAG Algorithms You Must Know to Implement (https://medium.com/decodingml/the-4-advanced-rag-algorithms-you-must-know-to-implement-5d0c7f1199d2) 1294 | - FlashRAG: A Python Toolkit for Efficient RAG Research (https://github.com/ruc-nlpir/flashrag) 1295 | - ColPali: Efficient Document Retrieval with Vision Language Models (https://github.com/ManuelFay/colpali) (https://huggingface.co/vidore) 1296 | - Byaldi - wrapper around the ColPali (https://github.com/AnswerDotAI/byaldi/) 1297 | - Multimodal RAG using ColPali (with Byaldi) and Qwen2-VL (https://github.com/merveenoyan/smol-vision/blob/main/ColPali_%2B_Qwen2_VL.ipynb) 1298 | - Chat with your PDFs using byaldi + Claude (https://github.com/AnswerDotAI/byaldi/blob/main/examples/chat_with_your_pdf.ipynb) 1299 | - Multimodal RAG with ColPali and Gemini : Financial Report Analysis Application (https://learnopencv.com/multimodal-rag-with-colpali/) 1300 | - NVIDIA Introduces RankRAG: A Novel RAG Framework that Instruction-Tunes a Single LLM for the Dual Purposes of Top-k Context Ranking and Answer Generation in RAG (https://www.marktechpost.com/2024/07/09/nvidia-introduces-rankrag-a-novel-rag-framework-that-instruction-tunes-a-single-llm-for-the-dual-purposes-of-top-k-context-ranking-and-answer-generation-in-rag/) (https://arxiv.org/pdf/2407.02485) 1301 | - Late Chunking in Long-Context Embedding Models (https://jina.ai/news/late-chunking-in-long-context-embedding-models/) (https://github.com/jina-ai/late-chunking/) 1302 | - Reader-LM: Small Language Models for Cleaning and Converting HTML to Markdown (https://jina.ai/news/reader-lm-small-language-models-for-cleaning-and-converting-html-to-markdown/) (https://colab.research.google.com/drive/1wXWyj5hOxEHY6WeHbOwEzYAC0WB1I5uA#scrollTo=ad-fjFOQxoFG) 1303 | - Not RAG, but RAG Fusion? Understanding Next-Gen Info Retrieval. (https://pub.towardsai.net/not-rag-but-rag-fusion-understanding-next-gen-info-retrieval-477788da02e2) $$ 1304 | - Goodbye, Text2SQL: Why Table-Augmented Generation (TAG) is the Future of AI-Driven Data Queries! (https://ai.plainenglish.io/goodbye-text2sql-why-table-augmented-generation-tag-is-the-future-of-ai-driven-data-queries-892e24e06922) $$ 1305 | - LOTUS: A Query Engine For Processing Data with LLMs (https://github.com/TAG-Research/lotus) 1306 | - MinerU - converts PDFs into machine-readable formats (e.g., markdown, JSON)(https://github.com/opendatalab/mineru) 1307 | - xRx is a framework for multi-modal conversational AI system (https://github.com/8090-inc/xrx-core) 1308 | - GenAI with Python: RAG with LLM (Complete Tutorial) - with Pdf2image, PyTesseract, Ollama, ChromaDB, Streamlit (https://towardsdatascience.com/genai-with-python-rag-with-llm-complete-tutorial-c276dda6707b) $$ 1309 | 1310 | 1311 | ## 🤖 Lightning AI ⚡⚡⚡ 1312 | - 📖 Lightning Fabric Documentation (https://lightning.ai/docs/fabric/stable/) 1313 | - [Codes] Build Your Own Trainer (https://github.com/Lightning-AI/lightning/tree/master/examples/fabric/build_your_own_trainer) 1314 | - ✏️ Ahead of AI (by Sebastian Raschka) (https://magazine.sebastianraschka.com/archive) (https://sebastianraschka.com/blog/) 1315 | - [Course] Deep Learning Fundamentals (https://lightning.ai/pages/courses/deep-learning-fundamentals/) (https://github.com/Lightning-AI/dl-fundamentals) 1316 | - Accelerate PyTorch Code with Fabric (https://lightning.ai/pages/blog/accelerate-pytorch-code-with-fabric/) 1317 | - How to Speed Up PyTorch Model Training (https://lightning.ai/pages/community/tutorial/how-to-speed-up-pytorch-model-training/) 1318 | - Finetuning LLMs on a Single GPU Using Gradient Accumulation (https://lightning.ai/pages/blog/gradient-accumulation/) 1319 | - Accelerating LLaMA with Fabric: A Comprehensive Guide to Training and Fine-Tuning LLaMA (https://lightning.ai/pages/community/tutorial/accelerating-llama-with-fabric-a-comprehensive-guide-to-training-and-fine-tuning-llama/) 1320 | - Understanding Parameter-Efficient Finetuning of Large Language Models: From Prefix Tuning to LLaMA-Adapters (https://lightning.ai/pages/community/article/understanding-llama-adapters/) 1321 | - How To Finetune GPT Like Large Language Models on a Custom Dataset (https://lightning.ai/pages/blog/how-to-finetune-gpt-like-large-language-models-on-a-custom-dataset/) 1322 | - Code LoRA From Scratch (https://lightning.ai/lightning-ai/studios/code-lora-from-scratch) 1323 | - Parameter-Efficient LLM Finetuning With Low-Rank Adaptation (LoRA) (https://lightning.ai/pages/community/tutorial/lora-llm/) 1324 | - Efficient Initialization of Large Models (https://lightning.ai/pages/community/efficient-initialization-of-large-models/) 1325 | - Accelerating Large Language Models with Mixed-Precision Techniques (https://lightning.ai/pages/community/tutorial/accelerating-large-language-models-with-mixed-precision-techniques/) 1326 | - Faster PyTorch Training by Reducing Peak Memory (combining backward pass + optimizer step) (https://lightning.ai/pages/community/tutorial/faster-pytorch-training-by-reducing-peak-memory/) 1327 | - Falcon – A guide to finetune and inference (https://lightning.ai/pages/blog/falcon-a-guide-to-finetune-and-inference/) 1328 | - Finetuning Falcon LLMs More Efficiently With LoRA and Adapters (https://lightning.ai/pages/community/finetuning-falcon-efficiently/) 1329 | - The Falcon has landed in the Hugging Face ecosystem (https://huggingface.co/blog/falcon) (https://colab.research.google.com/drive/1BiQiw31DT7-cDp1-0ySXvvhzqomTdI-o?usp=sharing) 1330 | - Improve LLMs With Proxy-Tuning (https://lightning.ai/lightning-ai/studios/improve-llms-with-proxy-tuning) 1331 | 1332 | ## 🤖 LLM Leaderboard 1333 | - 🤗 Open LLM Leaderboard (https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) 1334 | - 🤗 Open LLM Leaderboard V2 (https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) 1335 | - 🤗 Massive Text Embedding Benchmark (MTEB) Leaderboard (https://huggingface.co/spaces/mteb/leaderboard) 1336 | - Hughes Hallucination Evaluation Model (HHEM) leaderboard (https://huggingface.co/spaces/vectara/leaderboard) (https://github.com/vectara/hallucination-leaderboard) 1337 | 1338 | ## 🤖 LLM Evaluation 1339 | - FastChat - platform for training, serving, and evaluating large language model based chatbots (https://github.com/lm-sys/FastChat) 1340 | - LLM Evaluation Metrics: Everything You Need for LLM Evaluation (https://www.confident-ai.com/blog/llm-evaluation-metrics-everything-you-need-for-llm-evaluation) 1341 | - DeepEval - open-source LLM evaluation framework specialized for unit testing LLM outputs (https://github.com/confident-ai/deepeval) 1342 | - Language Model Evaluation Harness - based on tasks (https://github.com/EleutherAI/lm-evaluation-harness) (https://github.com/EleutherAI/lm-evaluation-harness/blob/main/docs/interface.md) 1343 | - LLM Task-Specific Evals that Do & Don't Work (https://eugeneyan.com/writing/evals/) 1344 | - RULER: What’s the Real Context Size of Your Long-Context Language Models? Evaluate long-context language models with configurable sequence length and task complexity (https://github.com/NVIDIA/RULER) 1345 | 1346 | ## 🤖 Transformer Models / Timeline 1347 | - 7 Basic NLP Models to Empower Your ML Application (https://zilliz.com/learn/7-nlp-models) 1348 | - 7 models on HuggingFace you probably didn’t know existed (https://towardsdatascience.com/7-models-on-huggingface-you-probably-didnt-knew-existed-f3d079a4fd7c) 1349 | - ChatGPT, GenerativeAI and LLMs Timeline (https://github.com/hollobit/GenAI_LLM_timeline) 1350 | - AI / ML / LLM / Transformer Models Timeline and List (https://ai.v-gar.de/ml/transformer/timeline/index.html) 1351 | - Comprehensive LLM model zoo (https://crfm.stanford.edu/ecosystem-graphs/index.html?mode=table) 1352 | - Totally Open Chatgpt (https://github.com/nichtdax/awesome-totally-open-chatgpt) 1353 | - Open LLMs (https://github.com/eugeneyan/open-llms) 1354 | - TII open-sourced Falcon LLM (https://huggingface.co/tiiuae) 1355 | - Question generator with context (https://huggingface.co/voidful/bart-eqg-question-generator) 1356 | - T5 One Line Summary (https://huggingface.co/snrspeaks/t5-one-line-summary) 1357 | - T5-base fine-tuned on SQuAD for Question Generation by just prepending the answer to the context (https://huggingface.co/mrm8488/t5-base-finetuned-question-generation-ap) 1358 | - Parrot_paraphraser_on_T5 (https://huggingface.co/prithivida/parrot_paraphraser_on_T5) 1359 | - T0pp (https://huggingface.co/bigscience/T0pp) (https://huggingface.co/GroNLP/T0pp-sharded) 1360 | - Persimmon-8B: The best fully permissively-licensed model in the 8B class (https://www.adept.ai/blog/persimmon-8b) 1361 | - Mistral Transformer (https://github.com/mistralai/mistral-src) (https://mistral.ai/news/announcing-mistral-7b/) 1362 | - MiniGPT-v2: Large Language Model as a Unified Interface for Vision-Language Multi-task Learning (https://minigpt-v2.github.io/) 1363 | - 8 Top Open-Source LLMs for 2024 and Their Uses (https://www.datacamp.com/blog/top-open-source-llms) 1364 | - 01-ai/Yi-34B (https://huggingface.co/01-ai/Yi-34B) 1365 | - OLMo: Open Language Model (https://github.com/allenai/OLMo/tree/main) (https://github.com/allenai/OLMo/blob/main/scripts/train.py) 1366 | - Genstruct-7B, an instruction-generation model (https://huggingface.co/NousResearch/Genstruct-7B) (https://huggingface.co/NousResearch/Genstruct-7B/blob/main/notebook.ipynb) 1367 | - Grok-1, a 314 billion parameter Mixture-of-Experts model (https://github.com/xai-org/grok-1) 1368 | - Qwen1.5-MoE: Matching 7B Model Performance with 1/3 Activated Parameters (https://qwenlm.github.io/blog/qwen-moe/) 1369 | - Tiny but mighty: The Phi-3 small language models with big potential (https://news.microsoft.com/source/features/ai/the-phi-3-small-language-models-with-big-potential/) 1370 | - TinyStories: How Small Can Language Models Be and Still Speak Coherent English? (https://arxiv.org/pdf/2305.07759) 1371 | - OpenELM - An Efficient Language Model Family with Open-source Training and Inference Framework - using layer-wise scaling strategy (https://arxiv.org/abs/2404.14619) (https://huggingface.co/apple/OpenELM) (https://github.com/apple/corenet) 1372 | - Llama 3.1 - 405B, 70B & 8B with multilinguality and long context (https://huggingface.co/blog/llama31) 1373 | - HERMES 3 TECHNICAL REPORT - instruct and chat tuned models created by fine-tuning Llama 3.1 8B, 70B, and 405B (https://nousresearch.com/wp-content/uploads/2024/08/Hermes-3-Technical-Report.pdf) 1374 | - OLMoE - Open Mixture-of-Experts Language Models - fully open source (https://huggingface.co/allenai/OLMoE-1B-7B-0924) (https://arxiv.org/abs/2409.02060) 1375 | 1376 | 1377 | ## 🤖 Transformer / LLM Inference / Deployment 1378 | - 7 Ways To Speed Up Inference of Your Hosted LLMs (https://betterprogramming.pub/speed-up-llm-inference-83653aa24c47) 1379 | - vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention (https://github.com/vllm-project/vllm) (https://vllm.ai/) 1380 | - vLLM v0.6.0: 2.7x Throughput Improvement and 5x Latency Reduction (https://blog.vllm.ai/2024/09/05/perf-update.html) 1381 | - 🤗 TGI: Text Generation Inference - Fast optimized inference for LLMs (https://github.com/huggingface/text-generation-inference) 1382 | - LMDeploy: a toolkit for compressing, deploying, and serving LLM (https://github.com/InternLM/lmdeploy) 1383 | - OpenVINO: an open-source toolkit for optimizing and deploying AI inference (https://github.com/openvinotoolkit) (https://docs.openvino.ai/2023.0/home.html) 1384 | - How continuous batching enables 23x throughput in LLM inference while reducing p50 latency (https://www.anyscale.com/blog/continuous-batching-llm-inference) 1385 | - Squeeze more out of your GPU for LLM inference—a tutorial on Accelerate & DeepSpeed (https://preemo.medium.com/squeeze-more-out-of-your-gpu-for-llm-inference-a-tutorial-on-accelerate-deepspeed-610fce3025fd) 1386 | - Performance bottlenecks in deploying LLMs—a primer for ML researchers (https://preemo.medium.com/performance-bottlenecks-in-deploying-llms-a-primer-for-ml-researchers-c2b51c2084a8) 1387 | - Inference using the pre-trained Alpaca-LoRA (https://www.mlexpert.io/machine-learning/tutorials/alpaca-and-llama-inference) (https://colab.research.google.com/drive/15VstUxU48CT3mRudFrj3FIv6Z4cIXnon?usp=sharing) 1388 | - Optimizing your LLM in production (https://huggingface.co/blog/optimize-llm) 1389 | - StreamingLLM: Efficient Streaming Language Models with Attention Sinks (https://github.com/mit-han-lab/streaming-llm) (https://arxiv.org/pdf/2309.17453.pdf) 1390 | - S-LoRA: Serving Thousands of Concurrent LoRA Adapters (https://github.com/s-lora/s-lora) 1391 | - Recipe for Serving Thousands of Concurrent LoRA Adapters (https://lmsys.org/blog/2023-11-15-slora/) 1392 | - DeepSparse by Neural Magic - Sparsity-aware deep learning inference runtime for CPUs (https://github.com/neuralmagic/deepsparse/tree/main) 1393 | - SparseML by Neural Magic - an open-source model optimization toolkit that enables you to create inference-optimized sparse models using pruning, quantization, and distillation algorithms (https://github.com/neuralmagic/sparseml) 1394 | - Marlin - Mixed Auto-Regressive Linear kernel, an extremely optimized FP16xINT4 matmul kernel aimed at LLM inference (https://github.com/IST-DASLab/marlin) 1395 | - LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning (https://github.com/datamllab/LongLM) (https://www.reddit.com/r/LocalLLaMA/comments/18x8g6c/llm_maybe_longlm_selfextend_llm_context_window/) 1396 | - Flash-Decoding for long-context inference (https://pytorch.org/blog/flash-decoding/) 1397 | - 10 Ways To Run LLMs Locally And Which One Works Best For You (https://matilabs.ai/2024/02/07/run-llms-locally/) 1398 | - Towards 100x Speedup: Full Stack Transformer Inference Optimization (https://yaofu.notion.site/Towards-100x-Speedup-Full-Stack-Transformer-Inference-Optimization-43124c3688e14cffaf2f1d6cbdf26c6c) 1399 | - Deploy Deep Learning Models at Scale using NVIDIA Triton Inference Server (https://github.com/decodingml/articles-code/tree/main/articles/computer_vision/deploy_deep_learning_at_scale_nvidia_triton_server) 1400 | - LMDeploy - a toolkit for compressing, deploying, and serving LLM (https://github.com/InternLM/lmdeploy) 1401 | - LLM Inference Series: 5. Dissecting model performance (https://medium.com/@plienhar/llm-inference-series-5-dissecting-model-performance-6144aa93168f) 1402 | - How to compute LLM embeddings 3X faster with model quantization - with ONNX model quantization / ONNX transformer optimization (https://medium.com/nixiesearch/how-to-compute-llm-embeddings-3x-faster-with-model-quantization-25523d9b4ce5) 1403 | - A Hitchhiker’s Guide to Speculative Decoding (https://pytorch.org/blog/hitchhikers-guide-speculative-decoding/) 1404 | - Achieving Faster Open-Source Llama3 Serving with SGLang Runtime (vs. TensorRT-LLM, vLLM) (https://lmsys.org/blog/2024-07-25-sglang-llama3/) (https://github.com/sgl-project/sglang) 1405 | - Awesome Production Machine Learning - open source libraries that will help you deploy, monitor, version, scale, and secure your production machine learning (https://github.com/EthicalML/awesome-production-machine-learning) 1406 | - NanoFlow - a throughput-oriented high-performance serving framework for LLMs (https://github.com/efeslab/Nanoflow) (https://arxiv.org/pdf/2408.12757) 1407 | - GuideLLM - evaluating and optimizing the deployment of large language models (LLMs) (https://github.com/neuralmagic/guidellm) 1408 | - LLM Compressor - create compressed models for faster inference with vLLM (https://github.com/vllm-project/llm-compressor) (https://neuralmagic.com/blog/llm-compressor-is-here-faster-inference-with-vllm/) 1409 | - bitnet.cpp - Official inference framework for 1-bit LLMs (https://github.com/microsoft/BitNet) 1410 | - STRING - a training-free method to improve effective context length of popular RoPE-based LLMs (https://github.com/HKUNLP/STRING) 1411 | 1412 | ## 🤖 Transformer / LLM Platform / Software 1413 | - GPT4All - run open-source LLMs on your own computer (https://github.com/nomic-ai/gpt4all) 1414 | - LLaMA-Factory - Efficiently Fine-Tune 100+ LLMs in WebUI (https://github.com/hiyouga/LLaMA-Factory) 1415 | - Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs (https://github.com/facebookresearch/lingua) 1416 | - bolt.diy (Previously oTToDev) - Prompt, run, edit, and deploy full-stack web applications using any LLM you want! (https://github.com/stackblitz-labs/bolt.diy) 1417 | - WebUI - Run AI Agent in your browser (https://github.com/browser-use/web-ui) 1418 | - Free LLM API resources (https://github.com/cheahjs/free-llm-api-resources) 1419 | 1420 | 1421 | ## 🤖 Transformer / LLM Data Curator 1422 | - Curating Trillion-Token Datasets: Introducing NVIDIA NeMo Data Curator (https://developer.nvidia.com/blog/curating-trillion-token-datasets-introducing-nemo-data-curator/) 1423 | - ftfy: fixes text for you (https://github.com/rspeer/python-ftfy) 1424 | - Cosmopedia: how to create large-scale synthetic data for pre-training (https://huggingface.co/blog/cosmopedia) (https://github.com/huggingface/cosmopedia) 1425 | - DataTrove - a library to process, filter and deduplicate text data at a very large scale (https://github.com/huggingface/datatrove/) 1426 | - Guidance - control how LLM output is structured (https://github.com/guidance-ai/guidance) 1427 | - Large-scale Near-deduplication Behind BigCode (https://huggingface.co/blog/dedup) 1428 | - Dolma Toolkit - curation of large datasets for (pre)-training ML models (https://github.com/allenai/dolma) 1429 | - Distilabel - framework for synthetic data and AI feedback (https://github.com/argilla-io/distilabel) 1430 | - LLM Decontaminator (https://github.com/lm-sys/llm-decontaminator) 1431 | - Tutorial to demonstrate how to reproduce Zyda2 dataset, curated by Zyphra in collaboration with Nvidia using NeMo Curator (https://github.com/NVIDIA/NeMo-Curator/tree/main/tutorials/zyda2-tutorial) (https://www.zyphra.com/post/building-zyda-2) 1432 | 1433 | ## 🤖 Transformer / LLM Dataset 1434 | - DBPedia (https://www.dbpedia.org/resources/individual/) (http://downloads.dbpedia.org/wiki-archive/dbpedia-version-2016-04.html) (http://downloads.dbpedia.org/2016-04/core/) 1435 | - Common Crawl (https://commoncrawl.org/the-data/get-started/) 1436 | - c4 - A colossal, cleaned version of Common Crawl's web crawl corpus (https://tensorflow.org/datasets/catalog/c4) 1437 | - c4 processed version with five variants of the data: en, en.noclean, en.noblocklist, realnewslike, and multilingual (mC4). (https://huggingface.co/datasets/allenai/c4) 1438 | - RedPajama-V2: An open dataset with 30 trillion tokens for training large language models (https://www.together.ai/blog/redpajama-data-v2) (https://huggingface.co/datasets/togethercomputer/RedPajama-Data-V2) 1439 | - SlimPajama-627B - Extensively deduplicated, multi-corpora, open-source dataset for training LLM (https://www.cerebras.net/blog/slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama) (https://huggingface.co/datasets/cerebras/SlimPajama-627B) 1440 | - Falcon RefinedWeb - An English large-scale dataset (5 trillion tokens ) for the pretraining of LLM, built through stringent filtering and extensive deduplication of CommonCrawl (https://huggingface.co/datasets/tiiuae/falcon-refinedweb) (https://arxiv.org/abs/2306.01116) 1441 | - Instruction tuning datasets to train (text and multi-modal) chat-based LLMs (GPT-4, ChatGPT, LLaMA, Alpaca) (https://github.com/yaodongC/awesome-instruction-dataset) 1442 | - Python-Code-23k-ShareGPT (https://huggingface.co/datasets/ajibawa-2023/Python-Code-23k-ShareGPT) 1443 | - UltraFeedback Binarized - A pre-processed version of the UltraFeedback dataset and was used to train Zephyr-7Β-β (https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized) 1444 | - [Blog post] FineWeb: decanting the web for the finest text data at scale (https://huggingface.co/spaces/HuggingFaceFW/blogpost-fineweb-v1) 1445 | - FineWeb: 15T tokens (44TB disk space) of cleaned and deduplicated english web data from CommonCrawl, for LLM pretraining (https://huggingface.co/datasets/HuggingFaceFW/fineweb) 1446 | - FineWeb-Edu: 1.3T tokens of educational web pages filtered from 🍷 FineWeb dataset (https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) 1447 | - FineWeb-Edu-score-2: 5.4T tokens of educational web pages filtered from 🍷 FineWeb dataset (https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu-score-2) 1448 | - Dolma - Used to train OLMo on 3 trillion tokens from a diverse mix of web content, academic publications, code, books, and encyclopedic materials, with quality filtering, fuzzy deduplication (https://blog.allenai.org/dolma-3-trillion-tokens-open-llm-corpus-9a0ff4b8da64) (https://huggingface.co/datasets/allenai/dolma) 1449 | - TinyStories - synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary (https://huggingface.co/datasets/roneneldan/TinyStories) 1450 | - GenQA - over 10M cleaned and deduplicated instruction samples generated from a handful of carefully designed prompts (https://huggingface.co/datasets/tomg-group-umd/GenQA) 1451 | - Persona Hub - Scaling Synthetic Data Creation with 1,000,000,000 Personas (https://github.com/tencent-ailab/persona-hub) (https://huggingface.co/datasets/proj-persona/PersonaHub) 1452 | - FinePersonas - detailed personas for creating customized, realistic synthetic data (https://huggingface.co/datasets/argilla/FinePersonas-v0.1) 1453 | - APIGen Function-Calling Datasets (https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k) 1454 | - The Tome - Compiled from 9 publicly available datasets, curated and designed for training LLMs with a focus on instruction following (https://huggingface.co/datasets/arcee-ai/The-Tome) 1455 | - distilabel-intel-orca-dpo-pairs (for preference tuning) (https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs) 1456 | - MINT-1T: Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokens (https://blog.salesforceairesearch.com/mint-1t/) (https://huggingface.co/collections/mlfoundations/mint-1t-6690216ca4d0df7e518dde1c) 1457 | - Zyda-2 is a 5-trillion token dataset composed of filtered and cross-deduplicated DCLM, FineWeb-Edu, Zyda-1, and Dolma v1.7's Common Crawl portion (https://huggingface.co/datasets/Zyphra/Zyda-2) (https://www.zyphra.com/post/building-zyda-2) 1458 | 1459 | 1460 | ## 🤖 Transformer / LLM Sample Applications 1461 | - Text Summarization using T5: Fine-Tuning and Building Gradio App (https://learnopencv.com/text-summarization-using-t5/) 1462 | - Beyond Classification With Transformers and Hugging Face (https://towardsdatascience.com/beyond-classification-with-transformers-and-hugging-face-d38c75f574fb) 1463 | - Faster Text Classification with Naive Bayes and GPUs (https://developer.nvidia.com/blog/faster-text-classification-with-naive-bayes-and-gpus) 1464 | - Classifying Multimodal Data using Transformers (https://github.com/dsaidgovsg/multimodal-learning-hands-on-tutorial) 1465 | - Chris McCormick 1466 | - 📺 BERT Document Classification Tutorial with Code - including Semantic Similarity Search (https://www.youtube.com/watch?v=_eSGWNqKeeY) 1467 | - Combining Categorical and Numerical Features with Text in BERT (https://mccormickml.com/2021/06/29/combining-categorical-numerical-features-with-bert/) 1468 | - Smart Batching Tutorial - Speed Up BERT Training (https://mccormickml.com/2020/07/29/smart-batching-tutorial/) 1469 | - [Discussion] How to use Bert for long text classification? (https://stackoverflow.com/questions/58636587/how-to-use-bert-for-long-text-classification/63413589#63413589) 1470 | - Sentiment Analysis with BERT and Transformers by Hugging Face using PyTorch and Python (https://curiousily.com/posts/sentiment-analysis-with-bert-and-hugging-face-using-pytorch-and-python/) 1471 | - Custom Named Entity Recognition with BERT (https://towardsdatascience.com/custom-named-entity-recognition-with-bert-cf1fd4510804) 1472 | - The Guide to Multi-Tasking with the T5 Transformer (https://towardsdatascience.com/the-guide-to-multi-tasking-with-the-t5-transformer-90c70a08837b) 1473 | - A Full Guide to Finetuning T5 for Text2Text and Building a Demo with Streamlit (https://medium.com/nlplanet/a-full-guide-to-finetuning-t5-for-text2text-and-building-a-demo-with-streamlit-c72009631887) (https://colab.research.google.com/drive/1RFBIkTZEqbRt0jxpTHgRudYJBZTD3Szn?usp=sharing) 1474 | - Building a Knowledge Base from Texts: a Full Practical Example (https://medium.com/nlplanet/building-a-knowledge-base-from-texts-a-full-practical-example-8dbbffb912fa) 1475 | - Building a Personal Assistant from Scratch - intent classification, speech-to-text, and text-to-speech (https://medium.com/nlplanet/building-a-personal-assistant-from-scratch-db0814e62d34) 1476 | - How I Turned My Company’s Docs into a Searchable Database with OpenAI (https://towardsdatascience.com/how-i-turned-my-companys-docs-into-a-searchable-database-with-openai-4f2d34bd8736) 1477 | - How I Turned ChatGPT into an SQL-Like Translator for Image and Video Datasets (https://towardsdatascience.com/how-i-turned-chatgpt-into-an-sql-like-translator-for-image-and-video-datasets-7b22b318400a) 1478 | - 10 Exciting Project Ideas Using Large Language Models (LLMs) for Your Portfolio (https://towardsdatascience.com/10-exciting-project-ideas-using-large-language-models-llms-for-your-portfolio-970b7ab4cf9e) $$ 1479 | - Build a Telegram chatbot with any AI model under the hood (web scraping summarizer bot) (https://medium.com/@galperovich/build-a-telegram-chatbot-with-any-ai-model-under-the-hood-62f9a8675d81) (https://github.com/galinaalperovich/ai_summary_tg_bot) 1480 | - Put ChatGPT right into your messenger: build a Telegram bot with the new official OpenAI API (https://blog.gopenai.com/put-chatgpt-right-into-your-messenger-build-a-telegram-bot-with-the-new-official-openai-api-84f7c005de7f) (https://github.com/galinaalperovich/chatgpt-api-tg-bot) 1481 | - LangChain + Streamlit🔥+ Llama 🦙: Bringing Conversational AI to Your Local Machine 🤯 (https://ai.plainenglish.io/%EF%B8%8F-langchain-streamlit-llama-bringing-conversational-ai-to-your-local-machine-a1736252b172) 1482 | - Zero to One: A Guide to Building a First PDF Chatbot with LangChain & LlamaIndex — Part 1 (https://medium.com/how-ai-built-this/zero-to-one-a-guide-to-building-a-first-pdf-chatbot-with-langchain-llamaindex-part-1-7d0e9c0d62f) 1483 | - Choosing the Right Embedding Model: A Guide for LLM Applications (https://medium.com/@ryanntk/choosing-the-right-embedding-model-a-guide-for-llm-applications-7a60180d28e3) 1484 | - Financial Document Classification with LayoutLMv3 - using OCR on document images (https://www.mlexpert.io/machine-learning/tutorials/document-classification-with-layoutlmv3) (https://colab.research.google.com/drive/1I0Qyajp_DFzKvQfUwwRc9p6fs6NfI-Kx?usp=sharing) 1485 | - Making a web app generator with open ML models (https://huggingface.co/blog/text-to-webapp) 1486 | - Topic Modeling with Llama 2 (https://maartengrootendorst.substack.com/p/topic-modeling-with-llama-2) (https://towardsdatascience.com/topic-modeling-with-llama-2-85177d01e174) $$ (https://colab.research.google.com/drive/1QCERSMUjqGetGGujdrvv_6_EeoIcd_9M?usp=sharing) 1487 | - https://maartengr.github.io/BERTopic/algorithm/algorithm.html 1488 | - https://maartengr.github.io/BERTopic/getting_started/best_practices/best_practices.html 1489 | - Mastering PDFs: Extracting Sections, Headings, Paragraphs, and Tables with Cutting-Edge Parser (https://blog.llamaindex.ai/mastering-pdfs-extracting-sections-headings-paragraphs-and-tables-with-cutting-edge-parser-faea18870125) 1490 | - LayoutPDFReader (https://github.com/nlmatics/llmsherpa#layoutpdfreader) 1491 | - Multimodal Retrieval with Text Embedding and CLIP Image Embedding for Backyard Birds (https://github.com/wenqiglantz/multi_modal_retrieval_backyard_birds) 1492 | - LlamaIndex Chat (https://github.com/run-llama/chat-llamaindex) 1493 | - LoRA for semantic similarity tasks (https://huggingface.co/docs/peft/task_guides/semantic-similarity-lora) 1494 | - Forging a Personal Chatbot with OpenAI API, Chroma DB, HuggingFace Spaces, and Gradio 🔥 (https://mlops.community/forging-a-personal-chatbot-with-openai-api-chroma-db-huggingface-spaces-and-gradio-%f0%9f%94%a5/) 1495 | - How to Convert Any Text Into a Graph of Concepts (https://towardsdatascience.com/how-to-convert-any-text-into-a-graph-of-concepts-110844f22a1a) (https://github.com/rahulnyk/knowledge_graph) 1496 | - Text to Knowledge Graph Made Easy with Graph Maker (https://towardsdatascience.com/text-to-knowledge-graph-made-easy-with-graph-maker-f3f890c0dbe8) 1497 | - Embed English Wikipedia under 5 dollars (https://lightning.ai/lightning-ai/studios/embed-english-wikipedia-under-5-dollars~01hg0zg8fyybp7p1sma6g9dkzm) 1498 | - Fine-Tuning Mistral 7b in Google Colab with QLoRA (complete guide) (https://medium.com/@codersama/fine-tuning-mistral-7b-in-google-colab-with-qlora-complete-guide-60e12d437cca) 1499 | - Hands-on LLMs Course - Learn to Train and Deploy a Real-Time Financial Advisor (https://github.com/iusztinpaul/hands-on-llms) 1500 | - Building DoorDash’s Product Knowledge Graph with Large Language Models (https://doordash.engineering/2024/04/23/building-doordashs-product-knowledge-graph-with-large-language-models/) 1501 | - Musings on building a Generative AI product - at LinkedIn (https://www.linkedin.com/blog/engineering/generative-ai/musings-on-building-a-generative-ai-product) 1502 | - How We Finetuned a Large Language Model to Search Patents & Generate New Patents (https://www.activeloop.ai/resources/how-we-finetuned-a-large-language-model-to-search-patents-generate-new-patents/) 1503 | - Structured LLM Output and Function Calling with Guidance - and Tool Use (https://lightning.ai/lightning-ai/studios/structured-llm-output-and-function-calling-with-guidance) 1504 | - Function calling (https://platform.openai.com/docs/guides/function-calling) 1505 | - How to use functions with a knowledge base - to summarize arXiv articles (https://cookbook.openai.com/examples/how_to_call_functions_for_knowledge_retrieval) 1506 | - Fine-tuning examples (Style and tone, Structured output, Tool calling, Function calling) (https://platform.openai.com/docs/guides/fine-tuning/fine-tuning-examples) 1507 | - LLM Twin Course: Building Your Production-Ready AI Replica (https://github.com/decodingml/llm-twin-course) (https://medium.com/decodingml/an-end-to-end-framework-for-production-ready-llm-systems-by-building-your-llm-twin-2cc6bb01141f) 1508 | - Crawl data from Medium, Substack, Linkedin, GitHub 1509 | - Clean, normalize and load data into MongoDB 1510 | - Send database changes to RabbitMQ, and consume through Bytewax streaming pipeline 1511 | - Clean, chunk, embed and load into Qdrant vector DB (also refactor the cleaning, chunking, and embedding logic using Superlinked, and load and index the vectors to Redis vector search) 1512 | - Fine-tune LLM using QLoRA, use Comet ML experiment tracker 1513 | - Deploy as REST API on Qwak, query with advanced RAG 1514 | - From Posts to Reports: Leveraging LLMs for Social Media Data Mining - How to instruct LLMs to filter restaurant posts and extract giveaways, events, deals and discounts (https://medium.com/decodingml/from-posts-to-reports-leveraging-llms-for-social-media-data-mining-6ebe0e2cdeb1) (https://github.com/decodingml/articles-code/tree/main/articles/generative_ai/data_extraction_from_social_media_posts_using_llms) 1515 | - I Fine-Tuned the Tiny Llama 3.2 1B to Replace GPT-4o - for classification task (https://towardsdatascience.com/i-fine-tuned-the-tiny-llama-3-2-1b-to-replace-gpt-4o-7ce1e5619f3d) $$ (https://colab.research.google.com/drive/1T5-zKWM_5OD21QHwXHiV9ixTRR7k3iB9?usp=sharing) 1516 | 1517 | ## 🤖 MLOps 1518 | - The Full Stack 7-Steps MLOps Framework (https://github.com/iusztinpaul/energy-forecasting) 1519 | - MLOps-Basics (https://github.com/graviraja/MLOps-Basics) 1520 | 1521 | ## 🤖 Q&A 1522 | - Danswer: OpenSource Enterprise Question-Answering tool (https://github.com/danswer-ai/danswer) (https://docs.danswer.dev/introduction) 1523 | - DocsGPT: GPT-powered chat for documentation (https://docsgpt.arc53.com/) (https://github.com/arc53/DocsGPT) 1524 | - Host a Llama 2 API on GPU for Free (https://medium.com/@yuhongsun96/host-a-llama-2-api-on-gpu-for-free-a5311463c183) 1525 | - How to Augment LLMs with Private Data (https://medium.com/@yuhongsun96/how-to-augment-llms-with-private-data-29349bd8ae9f) 1526 | - How to build an AI that can answer questions about your website (https://platform.openai.com/docs/tutorials/web-qa-embeddings) 1527 | - openai-cookbook 1528 | - https://github.com/openai/openai-cookbook/tree/3826607431929af5d58ba442aa3c2893009f637b/examples/fine-tuned_qa 1529 | - https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb 1530 | - Fine-Tune Transformer Models For Question Answering On Custom Data (https://towardsdatascience.com/fine-tune-transformer-models-for-question-answering-on-custom-data-513eaac37a80) 1531 | - Revolutionise Your Q&A Bot with GPT-J: The Open-Source Game Changer as a Replacement for GPT-3 (https://medium.com/@maliahrajan/revolutionise-your-q-a-bot-with-gpt-j-the-open-source-game-changer-as-a-replacement-for-gpt-3-216bc4362b53) 1532 | - How to build a Q&A web application using Python and Anvil (https://www.section.io/engineering-education/building-a-qa-web-application/) 1533 | - Build a Deep Q&A Web App with Transformers and Anvil | Python Deep Learning App (https://www.youtube.com/watch?v=G1uGSkANZjQ) (https://github.com/nicknochnack/Q-A-Anvil-App/blob/main/Anvil-Tutorial.ipynb) 1534 | - Open Source Generative AI in Question-Answering (NLP) using Python (https://www.youtube.com/watch?v=L8U-pm-vZ4c) (https://docs.pinecone.io/docs/abstractive-question-answering) 1535 | - Build a Question Answering Engine (Towhee & Gradio Chatbot) (https://github.com/towhee-io/examples/blob/main/nlp/question_answering/1_build_question_answering_engine.ipynb) 1536 | - Question Generation using 🤗transformers (https://github.com/patil-suraj/question_generation) (https://colab.research.google.com/gist/nrjvarshney/39ed6c80e2fe293b9e7eca5bc3a45b7d/quiz.ipynb) (https://huggingface.co/mrm8488/t5-base-finetuned-question-generation-ap) 1537 | - Running Llama 2 on CPU Inference Locally for Document Q&A (https://towardsdatascience.com/running-llama-2-on-cpu-inference-for-document-q-a-3d636037a3d8) $$ (https://github.com/kennethleungty/Llama-2-Open-Source-LLM-CPU-Inference) 1538 | - Using LLaMA 2.0, FAISS and LangChain for Question-Answering on Your Own Data (https://medium.com/@murtuza753/using-llama-2-0-faiss-and-langchain-for-question-answering-on-your-own-data-682241488476) 1539 | - How to Fine-tune Llama 2 with LoRA for Question Answering: A Guide for Practitioners (https://deci.ai/blog/fine-tune-llama-2-with-lora-for-question-answering/) 1540 | - Generative AI Lifecycle Patterns (https://dr-arsanjani.medium.com/the-generative-ai-lifecycle-1b0c7d9463ec) 1541 | 1542 | ## 🤖 Discussion on LLM Padding / Formatting Function 1543 | - Why does the falcon QLoRA tutorial code use eos_token as pad_token? - use TemplateProcessing (https://discuss.huggingface.co/t/why-does-the-falcon-qlora-tutorial-code-use-eos-token-as-pad-token/45954/14?u=brando) 1544 | - Pad and eos distinction. (https://chat.openai.com/share/ebb9a9a2-71d3-4c97-a727-b6042494b9a9) 1545 | - LLaMA FastTokenizer does not add eos_token_id at the end. #22794 (https://github.com/huggingface/transformers/issues/22794) 1546 | - data_collator.py (https://github.com/huggingface/transformers/blob/main/src/transformers/data/data_collator.py#L747) 1547 | - https://huggingface.co/docs/transformers/main/llm_tutorial#wrong-padding-side 1548 | - https://huggingface.co/docs/transformers/main/model_doc/llama2#resources 1549 | - Challenges in Stop Generation within Llama 2 (https://towardsdatascience.com/challenges-in-stop-generation-within-llama-2-25f5fea8dea2) 1550 | - Padding Large Language Models — Examples with Llama 2 (https://towardsdatascience.com/padding-large-language-models-examples-with-llama-2-199fb10df8ff) $$ 1551 | - [SFTTrainer] Fix non packed dataset #444 - Example of formatting_func on alpaca dataset (https://github.com/huggingface/trl/pull/444) 1552 | 1553 | ## 🤖 Merging weights with quantized model 1554 | - Merging QLoRA weights with quantized model (https://gist.github.com/ChrisHayduk/1a53463331f52dca205e55982baf9930) 1555 | - LoRA Adapters: When a Naive Merge Leads to Poor Performance (https://kaitchup.substack.com/p/lora-adapters-when-a-naive-merge) 1556 | 1557 | ## 🤖 Merge / Fusion / MoE 1558 | - FuseChat: Knowledge Fusion of Chat Models (https://github.com/fanqiwan/FuseLLM/tree/main/FuseChat) 1559 | 1560 | ## 🤖 Prompt Engineering / Instructions 1561 | - Enprompt 360 - AI Prompts Generator (https://www.kickstarter.com/projects/enprompt360/enprompt-360) 1562 | - Awesome ChatGPT Prompts (https://prompts.chat/) 1563 | - Prompt Engineering Guide (https://www.promptingguide.ai/techniques) 1564 | - The Power of Prompt Engineering: Building Your Own Personal Assistant (https://ai.plainenglish.io/the-power-of-prompt-engineering-personalizing-your-ai-model-5a1b9671b8c5) 1565 | - What I Learned Pushing Prompt Engineering to the Limit (https://towardsdatascience.com/what-i-learned-pushing-prompt-engineering-to-the-limit-c40f0740641f) 1566 | - Fixing Hallucinations in LLMs (https://betterprogramming.pub/fixing-hallucinations-in-llms-9ff0fd438e33) 1567 | - New ChatGPT Prompt Engineering Technique: Program Simulation (https://towardsdatascience.com/new-chatgpt-prompt-engineering-technique-program-simulation-56f49746aa7b) 1568 | - AutoGPT Agent Custom Instruction (https://shard-tsunami-ffe.notion.site/AutoGPT-Agent-Custom-Instruction-9826d664c53e4f50a5f814378c19a89d) 1569 | - Practitioners guide to fine-tune LLMs for domain-specific use case (https://cismography.medium.com/practitioners-guide-to-fine-tune-llms-for-domain-specific-use-case-part-1-4561714d874f) 1570 | - Practical insights while fine-tuning LLMs for domain-specific use cases and best practices (https://cismography.medium.com/practical-insights-while-fine-tuning-llms-for-domain-specific-use-cases-and-best-practices-aa986c799777) 1571 | - A New Prompt Engineering Technique Has Been Introduced Called Step-Back Prompting (https://cobusgreyling.medium.com/a-new-prompt-engineering-technique-has-been-introduced-called-step-back-prompting-b00e8954cacb) 1572 | - The LangChain Implementation Of DeepMind’s Step-Back Prompting (https://cobusgreyling.medium.com/the-langchain-implementation-of-deepminds-step-back-prompting-9d698cf3e0c2) 1573 | - Open AI - Prompt engineering guide (https://platform.openai.com/docs/guides/prompt-engineering) 1574 | - Meta Llama - How-to guides - Prompting (https://www.llama.com/docs/how-to-guides/prompting) 1575 | - Inside the Leaked System Prompts of GPT-4, Gemini 1.5, Claude 3, and More (https://medium.com/gitconnected/inside-the-leaked-system-prompts-of-gpt-4-gemini-1-5-claude-3-and-more-4ecb3d22b447?sk=7e053318c47b260ee482a5c8b319dd83) 1576 | - https://gist.github.com/kennethleungty/74c8f1ad0c39ca006fddea5da449c390 / https://gist.github.com/kennethleungty/00b5a5d809fdda94eafe5d49ccff7729 / https://gist.github.com/kennethleungty/80ceeba091d7c777abe861ef46558363 / https://gist.github.com/kennethleungty/587693681583da71f90d2da28e733ec3 1577 | - DSPy - a framework for algorithmically optimizing LM prompts and weights (https://github.com/stanfordnlp/dspy) 1578 | - Your Language Model Deserves Better Prompting (https://weaviate.io/blog/dspy-optimizers) (https://github.com/weaviate/recipes/blob/main/integrations/weights_and_biases/wandb_logging_RAG_dspy_cohere.ipynb) 1579 | - Prompting Fundamentals and How to Apply them Effectively (https://eugeneyan.com/writing/prompting/) 1580 | - Discovering Preference Optimization Algorithms with and for Large Language Models - prompt an LLM to propose and implement new preference optimization loss functions based on previously-evaluated performance metrics (https://arxiv.org/pdf/2406.08414) 1581 | - Large Language Models Are Human-Level Prompt Engineers - Automatic Prompt Engineer (APE) (https://sites.google.com/view/automatic-prompt-engineer) 1582 | - Large Language Models as Optimizers - Optimization by PROmpting (OPRO) (https://github.com/google-deepmind/opro/) 1583 | - How to Make ChatGPT Write Like a Human: (7-Step Prompt) to Make Your Content Come Alive! (https://medium.com/@afghanbitani/how-to-make-chatgpt-write-like-a-human-7-step-prompt-to-make-your-content-come-alive-98e0cd51894f) $$ 1584 | 1585 | ## 🤖 Agent 1586 | - AutoGPT: the heart of the open-source agent ecosystem (https://github.com/Significant-Gravitas/AutoGPT) 1587 | - The Official AutoGPT Forge Tutorial Series (https://aiedge.medium.com/autogpt-forge-e3de53cc58ec) 1588 | - AutoGPT Tutorial: Creating an Agent Powered Research Assistant with Auto-GPT-Forge (https://lablab.ai/t/autogpt-tutorial-creating-a-research-assistant-with-auto-gpt-forge) 1589 | - Decoding Auto-GPT (https://maartengrootendorst.substack.com/p/decoding-auto-gpt) 1590 | - Empower Functions - a family of LLMs that offer GPT-4 level capabilities for real-world "tool using" use cases (https://github.com/empower-ai/empower-functions) 1591 | - Multi-Agent Software Development through Cross-Team Collaboration (https://arxiv.org/pdf/2406.08979v1) (https://github.com/OpenBMB/ChatDev) 1592 | - GenAI with Python: Build Agents from Scratch (Complete Tutorial) - with Ollama, LangChain, LangGraph (No GPU, No APIKEY) (https://towardsdatascience.com/genai-with-python-build-agents-from-scratch-complete-tutorial-4fc1e084e2ec) $$ 1593 | - Choosing Between LLM Agent Frameworks (https://towardsdatascience.com/choosing-between-llm-agent-frameworks-69019493b259) 1594 | - Atomic Agents (https://github.com/BrainBlend-AI/atomic-agents/) (https://generativeai.pub/forget-langchain-crewai-and-autogen-try-this-framework-and-never-look-back-e34e0b6c8068) 1595 | - V : AI Personal Trainer (https://github.com/pannaf/valkyrie) 1596 | 1597 | ## 🤖 LLM - Misc 1598 | - How Do Language Models put Attention Weights over Long Context? (https://yaofu.notion.site/How-Do-Language-Models-put-Attention-Weights-over-Long-Context-10250219d5ce42e8b465087c383a034e) (https://github.com/FranxYao/Long-Context-Data-Engineering) 1599 | - AI Watermarking 101: Tools and Techniques (https://huggingface.co/blog/watermarking) 1600 | - GGUF, the long way around (https://vickiboykis.com/2024/02/28/gguf-the-long-way-around/) 1601 | - A Comprehensive Guide to Modeling Techniques in Mixed-Integer Linear Programming - Convert ideas into mathematical expressions to solve operations research problems (https://towardsdatascience.com/a-comprehensive-guide-to-modeling-techniques-in-mixed-integer-linear-programming-3e96cc1bc03d) 1602 | - Mastering ML Configurations by leveraging OmegaConf and Hydra (https://decodingml.substack.com/p/mastering-ml-configurations-by-leveraging) 1603 | - UltraChat - example training script with Accelerator (https://github.com/thunlp/UltraChat/blob/main/train/train_legacy/train.py) 1604 | - Using LESS Data to Tune Models (https://www.cs.princeton.edu/~smalladi/blog/2024/04/04/dataselection/) 1605 | - Techniques for training large neural networks - Data parallelism, Pipeline parallelism, Tensor parallelism, Mixture-of-Experts (MoE), and other memory saving designs (https://openai.com/research/techniques-for-training-large-neural-networks) 1606 | - Large Scale Transformer model training with Tensor Parallel (TP) (https://pytorch.org/tutorials/intermediate/TP_tutorial.html) 1607 | - YaFSDP - a Sharded Data Parallelism framework (https://github.com/yandex/YaFSDP) (https://habr.com/ru/companies/yandex/articles/817509/) 1608 | - Best Embedding Model — OpenAI / Cohere / Google / E5 / BGE - An In-depth Comparison of Multilingual Embedding Models (https://medium.com/@lars.chr.wiik/best-embedding-model-openai-cohere-google-e5-bge-931bfa1962dc) topic dataset (https://github.com/LarsChrWiik/lars_datasets/tree/main/topics_dataset_50) 1609 | - Training and Finetuning Embedding Models with Sentence Transformers v3 (https://huggingface.co/blog/train-sentence-transformers) 1610 | - What can LLMs never do? (https://www.strangeloopcanon.com/p/what-can-llms-never-do) 1611 | - What We’ve Learned From A Year of Building with LLMs (https://applied-llms.org/) 1612 | - Uncensor any LLM with abliteration (https://huggingface.co/blog/mlabonne/abliteration) (https://colab.research.google.com/drive/1VYm3hOcvCpbGiqKZb141gJwjdmmCcVpR?usp=sharing) 1613 | - abliterator.py (https://github.com/FailSpy/abliterator) 1614 | - TransformerLens - A library for mechanistic interpretability of GPT-style language models (https://github.com/TransformerLensOrg/TransformerLens) (https://transformerlensorg.github.io/TransformerLens/) 1615 | - Training a 70B model from scratch: open-source tools, evaluation datasets, and learnings (https://imbue.com/research/70b-intro/) 1616 | - From bare metal to a 70B model: infrastructure set-up and scripts (https://imbue.com/research/70b-infrastructure/) 1617 | - Ensuring accurate model evaluations: open-sourced, cleaned datasets for models that reason and code (https://imbue.com/research/70b-evals/) 1618 | - Aleksa Gordić’s Post: Amazing list of techniques for improving the stability of training large ML models (LLMs, diffusion, etc) (https://www.linkedin.com/feed/update/urn:li:activity:7215624025639645184/) 1619 | - The AdEMAMix Optimizer: Better, Faster, Older. A simple modification of the Adam optimizer with a mixture of two Exponential Moving Average (EMA) (https://github.com/nanowell/AdEMAMix-Optimizer-Pytorch/) 1620 | - Generating Human-level Text with Contrastive Search in Transformers (https://huggingface.co/blog/introducing-csearch) 1621 | - A Contrastive Framework for Neural Text Generation (https://github.com/yxuansu/SimCTG) 1622 | - Open Source LLM Tools (https://huyenchip.com/llama-police) 1623 | - What I learned from looking at 900 most popular open source AI tools (https://huyenchip.com/2024/03/14/ai-oss.html) 1624 | - Imagen - Pytorch (https://github.com/lucidrains/imagen-pytorch) 1625 | - MinImagen - A Minimal implementation of the Imagen text-to-image model(https://github.com/AssemblyAI-Community/MinImagen) 1626 | - How Imagen Actually Works (https://www.assemblyai.com/blog/how-imagen-actually-works/) 1627 | - MinImagen - Build Your Own Imagen Text-to-Image Model (https://www.assemblyai.com/blog/minimagen-build-your-own-imagen-text-to-image-model/) 1628 | - How to Beat Proprietary LLMs With Smaller Open Source Models (https://www.aidancooper.co.uk/how-to-beat-proprietary-llms/) 1629 | - A Guide to Structured Outputs Using Constrained Decoding (https://www.aidancooper.co.uk/constrained-decoding/) 1630 | - The 6 Best LLM Tools To Run Models Locally (https://medium.com/@amosgyamfi/the-6-best-llm-tools-to-run-models-locally-eedd0f7c2bbd) 1631 | - You can now train a 70b language model at home - Training LLMs with QLoRA + FSDP (https://www.answer.ai/posts/2024-03-06-fsdp-qlora.html) (https://github.com/AnswerDotAI/fsdp_qlora/tree/main) 1632 | - Bugs in LLM Training - Gradient Accumulation Fix (https://unsloth.ai/blog/gradient) 1633 | - 🤗 Fixing Gradient Accumulation (https://huggingface.co/blog/gradient_accumulation) 1634 | - SynthID Text - Apply watermarks and identify AI-generated content (https://huggingface.co/blog/synthid-text) (https://deepmind.google/technologies/synthid/) 1635 | - How to Build ANYTHING You Imagine With DeepSeek-R1 (Zero Coding Required) (https://iamshobhitagarwal.medium.com/how-to-build-anything-you-imagine-with-deepseek-r1-zero-coding-required-free-0d582513e7ab) $$ 1636 | - BOLT.diy: Build UNLIMITED Apps in Minutes 🚀 (No Code Required) (https://www.youtube.com/watch?v=QQw47XzcKRQ) 1637 | - Form Setup Guide - NocoDB (https://www.youtube.com/watch?v=GEsFSadshpg) 1638 | - Email Automation - N8n (https://www.youtube.com/watch?v=gofNLKlrFCQ) 1639 | 1640 | 1641 | ## 🤖 Llama 1642 | - PEFT Finetuning Quick Start Notebook (https://github.com/meta-llama/llama-recipes/blob/main/recipes/quickstart/finetuning/quickstart_peft_finetuning.ipynb) 1643 | - https://github.com/meta-llama/llama-recipes/blob/main/src/llama_recipes/finetuning.py 1644 | - https://github.com/meta-llama/llama-recipes/blob/main/src/llama_recipes/utils/train_utils.py 1645 | - RAFT: Adapting Language Model to Domain Specific RAG (https://gorilla.cs.berkeley.edu/blogs/9_raft.html) 1646 | - https://github.com/meta-llama/llama-recipes/tree/main/recipes/use_cases/end2end-recipes/RAFT-Chatbot 1647 | - https://github.com/meta-llama/llama-recipes/blob/main/recipes/use_cases/end2end-recipes/RAFT-Chatbot/raft_utils.py 1648 | 1649 | 1650 | ## 🤖 Unsloth 1651 | - Make LLM Fine-tuning 2x faster with Unsloth and 🤗 TRL (https://huggingface.co/blog/unsloth-trl) 1652 | - https://unsloth.ai/blog 1653 | - https://github.com/unslothai/unsloth 1654 | - https://www.kaggle.com/code/danielhanchen/kaggle-mistral-7b-unsloth-notebook 1655 | 1656 | ## 🤖 Transformer Alternatives 1657 | - Retentive Networks (RetNet) Explained: The much-awaited Transformers-killer is here (https://medium.com/ai-fusion-labs/retentive-networks-retnet-explained-the-much-awaited-transformers-killer-is-here-6c17e3e8add8) 1658 | - Mamba Explained - The State Space Model taking on Transformers (https://www.kolaayonrinde.com/blog/2024/02/11/mamba.html) 1659 | - Introducing Jamba: AI21's Groundbreaking SSM-Transformer Model (https://www.ai21.com/blog/announcing-jamba) 1660 | - ModuleFormer: a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward experts (https://github.com/IBM/ModuleFormer) 1661 | - JetMoE: Reaching LLaMA2 Performance with 0.1M Dollars (https://research.myshell.ai/jetmoe) 1662 | - SUPRA: Scalable UPtraining for Recurrent Attention - uptrain a transformer to a linear RNN (https://github.com/TRI-ML/linear_open_lm) (https://huggingface.co/TRI-ML/mistral-supra) 1663 | - 📺 Understanding Mamba and State Space Models (https://www.youtube.com/watch?v=iskuX3Ak9Uk) 1664 | 1665 | ## 👍 Google AI/ML Use Cases 1666 | - 185 real-world gen AI use cases from the world's leading organizations (https://cloud.google.com/transform/101-real-world-generative-ai-use-cases-from-industry-leaders) 1667 | - Customers are putting Gemini to work (https://blog.google/products/google-cloud/gemini-at-work-ai-agents/) 1668 | - 1669 | ## Diffusion Models 1670 | - The ABCs of Diffusion Models (https://medium.com/decodingml/the-abcs-of-diffusion-models-51902a331068) (https://github.com/decodingml/articles-code/tree/main/articles/generative_ai/diffusion_models_fundamentals) 1671 | 1672 | 1673 | 1674 | 1675 | 1676 | 1677 | -------------------------------------------------------------------------------- /big-o-cheatsheet.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/peggy1502/Amazing-Resources/5c100d6a7f5e926195e2c786eee8cbbf5e773a5d/big-o-cheatsheet.pdf --------------------------------------------------------------------------------