└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # MLOps Roadmap 2024 2 | 3 | ## 1. Programming 4 | Programming skills are crucial for an MLOps engineer. Python is the most commonly used language in machine learning, making it important for collaboration with machine learning engineers and data scientists. 5 | 6 | ### 1.1. Python & IDEs 7 | Start learning Python through books and practice. 8 | - **Tutorial Suggestion:** https://realpython.com 9 | - **Book Suggestion:** [Python Crash Course, 3rd Edition](https://www.amazon.com/stores/Eric-Matthes/author/B01DPU378I?ref=ap_rdr&store_ref=ap_rdr&isDramIntegrated=true&shoppingPortalEnabled=true) by Eric Matthes 10 | - **Code Practice:** [LeetCode Python Problems](https://leetcode.com/problemset/) 11 | - **Courses:** [Learn Python 3](https://www.codecademy.com/learn/learn-python-3) 12 | - **Track Suggestions:** [Python fundamentals](https://app.datacamp.com/learn/skill-tracks/python-fundamentals), [Python programming](https://app.datacamp.com/learn/skill-tracks/python-programming) 13 | 14 | - **Important Topics:** [Installing Python, using virtual environments](https://marvelousmlops.substack.com/p/the-right-way-to-install-python-on), [IDE usage](https://mlops.community/how-to-configure-vs-code-for-ai-ml-and-mlops-development-in-python-🛠%EF%B8%8F%EF%B8%8F/), Python basics, [Pytest](https://app.datacamp.com/learn/skill-tracks/python-programming), [Packaging](https://www.freecodecamp.org/news/how-to-build-and-publish-python-packages-with-poetry/) 15 | 16 | ### 1.2. Bash Basics & Command Line Editors 17 | Understanding bash is essential for MLOps tasks. 18 | 19 | - **Book Suggestion:** _The Linux Command Line, 2nd Edition_ by William E. Shotts 20 | - **Course:** [Bash mastery](https://www.udemy.com/course/bash-mastery) 21 | - **Tutorials:** [VIM beginners guide](https://www.freecodecamp.org/news/vim-beginners-guide/), [VIM adventures](https://vim-adventures.com/), [VIM by Daniel Miessler](https://danielmiessler.com/p/vim/) 22 | 23 | ## 2. Containerization and Kubernetes 24 | These technologies are vital in modern software engineering. 25 | 26 | ### 2.1. Docker 27 | Docker is a popular open-source containerization platform used in MLOps. 28 | 29 | - **Docker Roadmap:** [Roadmap.sh Docker](https://roadmap.sh/docker) 30 | - **Tutorial:** [Full docker tutorial by Techworld by Nana](https://www.youtube.com/watch?v=3c-iBn73dDE) 31 | 32 | ### 2.2. Kubernetes 33 | Kubernetes is essential for machine learning model training and deployment. 34 | 35 | - **Kubernetes Roadmap:** [Roadmap.sh Kubernetes](https://roadmap.sh/kubernetes) 36 | - **Tutorial:** [Kubernetes course by freecodecamp.com](https://www.youtube.com/watch?v=d6WC5n9G_sM) 37 | - **Course:** [Kubernetes mastery](https://www.udemy.com/course/kubernetesmastery/) 38 | - **Tool:** [K9s CLI for Kubernetes](https://k9scli.io) 39 | 40 | ## 3. Machine Learning Fundamentals 41 | An MLOps engineer should have a basic understanding of machine learning models. 42 | 43 | - **Courses:** [MLCourse.ai](https://mlcourse.ai/), [Fast.ai](https://course.fast.ai) 44 | - **Book Suggestion:** _Applied Machine Learning and AI for Engineers_ by Jeff Prosise 45 | 46 | ## 4. MLOps Principles 47 | Awareness of MLOps principles and maturity factors is required. 48 | 49 | - **Books:** 50 | - _Designing Machine Learning Systems_ by Chip Huyen 51 | - _Introducing MLOps_ by Mark Treveil and Dataiku 52 | - **Assessment:** [MLOps maturity assessment](https://marvelousmlops.substack.com/p/mlops-maturity-assessment) 53 | - **Great resource on MLOps:** [ml-ops.org](https://ml-ops.org) 54 | 55 | ## 5. MLOps Components 56 | MLOps platforms consist of various components, from version control to feature stores. 57 | - [The minimum set of must-haves for MLOps article.](https://marvelousmlops.substack.com/p/the-minimum-set-of-must-haves-for) 58 | 59 | - [Made with ML MLOps Course](https://madewithml.com/courses/mlops/) 60 | 61 | - [The full stach 7-steps MLOps Framework](https://www.pauliusztin.me/courses/the-full-stack-7-steps-mlops-framework) 62 | - [End-to-end machine learning](https://www.udemy.com/course/sustainable-and-scalable-machine-learning-project-development/) 63 | 64 | 65 | ### 5.1. Version Control & CI/CD Pipelines 66 | Critical for traceable and reproducible ML model deployments. 67 | 68 | - **Books:** 69 | - _Learning GitHub Actions_ by Brent Laster 70 | - _Learning Git_ by Anna Skoulikari 71 | - **Tutorials & Courses:** [Git & GitHub for beginners](https://www.youtube.com/watch?v=RGOj5yH7evk), [Python to Production guide](https://www.udemy.com/course/setting-up-the-linux-terminal-for-software-development/), [Version Control Missing Semester](https://missing.csail.mit.edu/2020/version-control/), https://learngitbranching.js.org/ 72 | - **Tool:** [Pre-commit hooks](https://marvelousmlops.substack.com/p/welcome-to-pre-commit-heaven) 73 | 74 | ### 5.2. Orchestration 75 | Systems like Airflow and Mage are important in ML engineering. 76 | 77 | - **Course:** [Introduction to Airflow in Python](https://app.datacamp.com/learn/courses/introduction-to-airflow-in-python) 78 | - **Note:** Airflow is also featured in the _ML Engineering with Python_ book and [_The Full Stack 7-Steps MLOps Framework_](https://www.pauliusztin.me/courses/the-full-stack-7-steps-mlops-framework). 79 | 80 | ### 5.3. Experiment Tracking and Model Registries 81 | Logging metadata, parameters, and artifacts of training runs. 82 | 83 | - **Tool:** MLflow 84 | - **Courses:** [MLflow Udemy course](https://www.udemy.com/course/mlflow-course/), [End-to-end machine learning (MLflow piece)](https://www.udemy.com/course/sustainable-and-scalable-machine-learning-project-development/) 85 | 86 | ### 5.4. Data Lineage and Feature Stores 87 | Feature stores are a crucial component of MLOps infrastructure. 88 | 89 | - **Tutorial:** Creating a feature store with Feast [Part 1](https://kedion.medium.com/creating-a-feature-store-with-feast-part-1-37c380223e2f) [Part 2](https://kedion.medium.com/feature-storage-for-ml-with-feast-part-2-34df1971a8d3) [Part 3](https://kedion.medium.com/feature-storage-for-ml-with-feast-a061899fc4a2) 90 | - **Tool:** DVC for data tracking 91 | - **Course:** [End-to-end machine learning (DVC piece)](https://www.udemy.com/course/sustainable-and-scalable-machine-learning-project-development/) 92 | 93 | ### 5.5. Model Training & Serving 94 | Decisions depend on the organization's infrastructure. 95 | 96 | - **Repository Suggestion:** [ML Deployment k8s Fast API](https://github.com/sayakpaul/ml-deployment-k8s-fastapi/tree/main) 97 | - **Tutorial Suggestions:** [ML deployment with k8s FastAPI, Building an ML app with FastAPI](https://dev.to/bravinsimiyu/beginner-guide-on-how-to-build-a-machine-learning-app-with-fastapi-part-ii-deploying-the-fastapi-application-to-kubernetes-4j6g), [Basic Kubeflow pipeline](https://towardsdatascience.com/tutorial-basic-kubeflow-pipeline-from-scratch-5f0350dc1905), [Building and deploying ML pipelines](https://www.datacamp.com/tutorial/kubeflow-tutorial-building-and-deploying-machine-learning-pipelines?utm_source=google&utm_medium=paid_search&utm_campaignid=19589720818&utm_adgroupid=157156373991&utm_device=c&utm_keyword=&utm_matchtype=&utm_network=g&utm_adpostion=&utm_creative=683184494153&utm_targetid=dsa-2218886984380&utm_loc_interest_ms=&utm_loc_physical_ms=9064564&utm_content=&utm_campaign=230119_1-sea~dsa~tofu_2-b2c_3-eu_4-prc_5-na_6-na_7-le_8-pdsh-go_9-na_10-na_11-na-dec23&gad_source=1&gclid=Cj0KCQiA4Y-sBhC6ARIsAGXF1g7iSih9h2RGL27LwWY6dlPLhEss-e5Af8pnaBvdDynRh7IHIKi8sGgaApD-EALw_wcB), [KServe tutorial](https://towardsdatascience.com/kserve-highly-scalable-machine-learning-deployment-with-kubernetes-aa7af0b71202) 98 | 99 | ### 5.6. Monitoring & Observability 100 | Vital for the health and performance of ML systems. 101 | - [**ML Monitoring vs Observability article**](https://marvelousmlops.substack.com/p/ml-monitoring-vs-ml-observability) 102 | - **Course:** [Machine learning monitoring concepts](https://app.datacamp.com/learn/courses/machine-learning-monitoring-concepts), [Monitoring ML in Python](https://app.datacamp.com/learn/courses/monitoring-machine-learning-in-python) 103 | - **Tools:** [Prometheus, Grafana](https://www.udemy.com/course/mastering-prometheus-and-grafana/) 104 | 105 | ## 6. Infrastructure as Code: Terraform 106 | Essential for a reproducible MLOps framework. 107 | 108 | - **Course:** [Terraform course for beginners](https://www.youtube.com/watch?v=SLB_c_ayRMo) 109 | - **Video:** [8 Terraform best practices by Techworld by Nana](https://www.youtube.com/watch?v=gxPykhPxRW0) 110 | - **Book Suggestion:** _Terraform: Up and Running, 3rd Edition_ by Yevgeniy Brikman 111 | --------------------------------------------------------------------------------