├── Final-Logo.jpg └── README.md /Final-Logo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ahkarami/Deep-Learning-in-Production/HEAD/Final-Logo.jpg -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Deep-Learning-in-Production 2 | In this repository, I will share some useful notes and references about deploying deep learning-based models in production. 3 | 4 |

5 | Logo 6 |

7 | 8 | ## Convert PyTorch Models in Production: 9 | - [PyTorch Production Level Tutorials](https://pytorch.org/tutorials/#production-usage) [_Fantastic_] 10 | - [The road to 1.0: production ready PyTorch](https://pytorch.org/2018/05/02/road-to-1.0.html) 11 | - [PyTorch 1.0 tracing JIT and LibTorch C++ API to integrate PyTorch into NodeJS](http://blog.christianperone.com/2018/10/pytorch-1-0-tracing-jit-and-libtorch-c-api-to-integrate-pytorch-into-nodejs/) [_Good Article_] 12 | - [Model Serving in PyTorch](https://pytorch.org/blog/model-serving-in-pyorch/) 13 | - [PyTorch Summer Hackathon](https://pytorch.devpost.com/) [_Very Important_] 14 | - [Deploying PyTorch and Building a REST API using Flask](https://pytorch.org/tutorials/intermediate/flask_rest_api_tutorial.html) [_Important_] 15 | - [PyTorch model recognizing hotdogs and not-hotdogs deployed on flask](https://github.com/jaroslaw-weber/hotdog-not-hotdog) 16 | - [Serving PyTorch 1.0 Models as a Web Server in C++ ](https://github.com/Wizaron/pytorch-cpp-inference) [_Useful Example_] 17 | - [PyTorch Internals](http://blog.ezyang.com/2019/05/pytorch-internals/) [_Interesting & Useful Article_] 18 | - [Flask application to support pytorch model prediction](https://github.com/craigsidcarlson/PytorchFlaskApp) 19 | - [Serving PyTorch Model on Flask Thread-Safety](https://discuss.pytorch.org/t/serving-pytorch-model-on-flask-thread-safety/13921) 20 | - [Serving PyTorch Models on AWS Lambda with Caffe2 & ONNX](https://machinelearnings.co/serving-pytorch-models-on-aws-lambda-with-caffe2-onnx-7b096806cfac) 21 | - [Serving PyTorch Models on AWS Lambda with Caffe2 & ONNX (Another Version)](https://blog.waya.ai/deploy-deep-machine-learning-in-production-the-pythonic-way-a17105f1540e) 22 | - [Deep Dive into ONNX Runtime](https://medium.com/@mohsen.mahmoodzadeh/a-deep-dive-into-onnx-onnx-runtime-part-1-874517c66ffc) 23 | - [EuclidesDB - _multi-model machine learning feature database with PyTorch_](https://euclidesdb.readthedocs.io/en/latest/) 24 | - [EuclidesDB - GitHub](https://github.com/perone/euclidesdb/) 25 | - [WebDNN: Fastest DNN Execution Framework on Web Browser](https://github.com/mil-tokyo/webdnn) 26 | - [FastAI PyTorch Serverless API (with AWS Lambda)](https://github.com/alecrubin/pytorch-serverless/) 27 | - [FastAI PyTorch in Production (discussion)](http://forums.fast.ai/t/fastai-pytorch-in-production/16928) 28 | - [OpenMMLab Model Deployment Framework](https://github.com/open-mmlab/mmdeploy) 29 | - [PyTorch Native Architecture Optimization: torchao](https://pytorch.org/blog/pytorch-native-architecture-optimization/?utm_content=309679619&utm_medium=social&utm_source=twitter&hss_channel=tw-776585502606721024) 30 | - [TorchServe](https://github.com/pytorch/serve) [Great Tool] 31 | - [TorchServe Video Tutorial](https://www.youtube.com/watch?v=XlO7iQMV3Ik) 32 | - [LitServe: Easily serve AI models Lightning fast](https://github.com/Lightning-AI/litserve) 33 | 34 | ## Convert PyTorch Models to C++: 35 | - [**Loading a PyTorch Model in C++**](https://pytorch.org/tutorials/advanced/cpp_export.html) [_**Fantastic**_] 36 | - [**PyTorch C++ API**](https://pytorch.org/cppdocs/index.html) [_Bravo_] 37 | - [An Introduction To Torch (Pytorch) C++ Front-End](https://radicalrafi.github.io/posts/pytorch-cpp-intro/) [_Very Good_] 38 | - [Blogs on using PyTorch C++ API](https://discuss.pytorch.org/t/a-series-of-blogs-on-pytorch-c-api-transfer-learning-jupyter-notebook-with-libtorch-xeus-cling-and-more/54628) [_Good_] 39 | - [ATen: A TENsor library](https://github.com/pytorch/pytorch/tree/master/aten) 40 | - [Important Issue about PyTorch-like C++ interface](https://github.com/pytorch/pytorch/issues/3335) 41 | - [PyTorch C++ API Test](https://github.com/pytorch/pytorch/tree/master/test/cpp/api) 42 | - [PyTorch via C++](https://discuss.pytorch.org/t/pytorch-via-c/19234) [_Useful Notes_] 43 | - [AUTOGRADPP](https://github.com/pytorch/pytorch/tree/master/torch/csrc/api) 44 | - [PyTorch C++ Library](https://github.com/warmspringwinds/pytorch-cpp) 45 | - [Direct C++ Interface to PyTorch](https://github.com/ebetica/autogradpp) 46 | - [A Python module for compiling PyTorch graphs to C](https://github.com/lantiga/pytorch2c) 47 | 48 | ## Deploy TensorFlow Models in Production: 49 | - [How to deploy Machine Learning models with TensorFlow - _Part1_](https://towardsdatascience.com/how-to-deploy-machine-learning-models-with-tensorflow-part-1-make-your-model-ready-for-serving-776a14ec3198) 50 | - [How to deploy Machine Learning models with TensorFlow - _Part2_](https://towardsdatascience.com/how-to-deploy-machine-learning-models-with-tensorflow-part-2-containerize-it-db0ad7ca35a7) 51 | - [How to deploy Machine Learning models with TensorFlow - _Part3_](https://towardsdatascience.com/how-to-deploy-machine-learning-models-with-tensorflow-part-3-into-the-cloud-7115ff774bb6) 52 | - [Neural Structured Learning (NSL) in TensorFlow](https://medium.com/tensorflow/introducing-neural-structured-learning-in-tensorflow-5a802efd7afd) [_Great_] 53 | - [Building Robust Production-Ready Deep Learning Vision Models](https://medium.com/google-developer-experts/building-robust-production-ready-deep-learning-vision-models-in-minutes-acd716f6450a) 54 | - [Creating REST API for TensorFlow models](https://becominghuman.ai/creating-restful-api-to-tensorflow-models-c5c57b692c10) 55 | - ["How to Deploy a Tensorflow Model in Production" by _Siraj Raval_ on YouTube](https://www.youtube.com/watch?v=T_afaArR0E8) 56 | - [Code for the "How to Deploy a Tensorflow Model in Production" by _Siraj Raval_ on YouTube](https://github.com/llSourcell/How-to-Deploy-a-Tensorflow-Model-in-Production) 57 | - [How to deploy an Object Detection Model with TensorFlow serving](https://medium.freecodecamp.org/how-to-deploy-an-object-detection-model-with-tensorflow-serving-d6436e65d1d9) [_Very Good Tutorial_] 58 | - [Freeze Tensorflow models and serve on web](http://cv-tricks.com/how-to/freeze-tensorflow-models/) [_Very Good Tutorial_] 59 | - [How to deploy TensorFlow models to production using TF Serving](https://medium.freecodecamp.org/how-to-deploy-tensorflow-models-to-production-using-tf-serving-4b4b78d41700) [_Good_] 60 | - [How Zendesk Serves TensorFlow Models in Production](https://medium.com/zendesk-engineering/how-zendesk-serves-tensorflow-models-in-production-751ee22f0f4b) 61 | - [TensorFlow Serving Example Projects](https://github.com/Vetal1977/tf_serving_example) 62 | - [Serving Models in Production with TensorFlow Serving](https://www.youtube.com/watch?v=q_IkJcPyNl0) [_TensorFlow Dev Summit 2017 Video_] 63 | - [Building TensorFlow as a Standalone Project](https://tuatini.me/building-tensorflow-as-a-standalone-project/) 64 | - [TensorFlow C++ API Example](https://github.com/jhjin/tensorflow-cpp) 65 | - [TensorFlow.js](https://js.tensorflow.org/) 66 | - [Introducing TensorFlow.js: Machine Learning in Javascript](https://medium.com/tensorflow/introducing-tensorflow-js-machine-learning-in-javascript-bf3eab376db) 67 | 68 | ## Convert Keras Models in Production: 69 | - [Deep learning in production with Keras, Redis, Flask, and Apache](https://www.pyimagesearch.com/2018/02/05/deep-learning-production-keras-redis-flask-apache/) [_Rank: 1st & General Usefult Tutorial_] 70 | - [Deploying a Keras Deep Learning Model as a Web Application in Python](https://towardsdatascience.com/deploying-a-keras-deep-learning-model-as-a-web-application-in-p-fc0f2354a7ff) [_Very Good_] 71 | - [Deploying a Python Web App on AWS](https://towardsdatascience.com/deploying-a-python-web-app-on-aws-57ed772b2319) [_Very Good_] 72 | - [Deploying Deep Learning Models Part 1: Preparing the Model](https://blog.paperspace.com/deploying-deep-learning-models-flask-web-python/) 73 | - [Deploying your Keras model](https://medium.com/@burgalon/deploying-your-keras-model-35648f9dc5fb) 74 | - [Deploying your Keras model using Keras.JS](https://becominghuman.ai/deploying-your-keras-model-using-keras-js-2e5a29589ad8) 75 | - ["How to Deploy a Keras Model to Production" by _Siraj Raval_ on Youtube](https://github.com/llSourcell/how_to_deploy_a_keras_model_to_production) 76 | - [Deploy Keras Model with Flask as Web App in 10 Minutes](https://github.com/mtobeiyf/keras-flask-deploy-webapp) [Good Repository] 77 | - [Deploying Keras Deep Learning Models with Flask](https://towardsdatascience.com/deploying-keras-deep-learning-models-with-flask-5da4181436a2) 78 | - [keras2cpp](https://github.com/pplonski/keras2cpp) 79 | 80 | ## Deploy MXNet Models in Production: 81 | - [Model Server for Apache MXNet](https://github.com/awslabs/mxnet-model-server) 82 | - [Running the Model Server](https://github.com/awslabs/mxnet-model-server/blob/master/docs/server.md) 83 | - [Multi Model Server (MMS) Documentation](https://github.com/awslabs/multi-model-server/tree/master/docs) 84 | - [Introducing Model Server for Apache MXNet](https://aws.amazon.com/blogs/machine-learning/introducing-model-server-for-apache-mxnet/) 85 | - [Single Shot Multi Object Detection Inference Service](https://github.com/awslabs/mxnet-model-server/blob/master/examples/ssd/README.md) 86 | - [Amazon SageMaker](https://aws.amazon.com/sagemaker/) 87 | - [How can we serve MXNet models built with gluon api](https://discuss.mxnet.io/t/how-can-we-serve-mxnet-models-built-with-gluon-api/684) 88 | - [MXNet C++ Package](https://github.com/apache/incubator-mxnet/tree/master/cpp-package) 89 | - [MXNet C++ Package Examples](https://github.com/apache/incubator-mxnet/tree/master/cpp-package/example) 90 | - [MXNet Image Classification Example of C++](https://github.com/apache/incubator-mxnet/tree/master/example/image-classification/predict-cpp) 91 | - [MXNet C++ Tutorial](http://mxnet.incubator.apache.org/tutorials/c%2B%2B/basics.html) 92 | - [An introduction to the MXNet API](https://becominghuman.ai/an-introduction-to-the-mxnet-api-part-1-848febdcf8ab) [Very Good Tutorial for Learning MXNet] 93 | - [GluonCV](https://gluon-cv.mxnet.io/) 94 | - [GluonNLP](http://gluon-nlp.mxnet.io/) 95 | - [Model Quantization for Production-Level Neural Network Inference](https://medium.com/apache-mxnet/model-quantization-for-production-level-neural-network-inference-f54462ebba05) [_Excellent_] 96 | 97 | ## Deploy Machine Learning Models with Go: 98 | - [Cortex: Deploy machine learning models in production](https://github.com/cortexlabs/cortex) 99 | - [Cortex - Main Page](https://www.cortex.dev/) 100 | - [Why we deploy machine learning models with Go — not Python](https://towardsdatascience.com/why-we-deploy-machine-learning-models-with-go-not-python-a4e35ec16deb) 101 | - [Go-Torch](https://github.com/orktes/go-torch) 102 | - [Gotch - Go API for PyTorch](https://github.com/sugarme/gotch) 103 | - [TensorFlow Go Lang](https://www.tensorflow.org/install/lang_go) 104 | - [Go-onnx](https://github.com/dhdanie/goonnx) 105 | 106 | ## General Deep Learning Deployment Toolkits: 107 | - [OpenVINO Toolkit - Deep Learning Deployment Toolkit repository](https://github.com/openvinotoolkit/openvino) [_Great_] 108 | - [ClearML - ML/DL development and production suite](https://github.com/allegroai/clearml) 109 | - [Model Deployment Using Heroku: A Complete Guide on Heroku](https://www.analyticsvidhya.com/blog/2021/10/a-complete-guide-on-machine-learning-model-deployment-using-heroku/) [Good] 110 | - [NVIDIA Triton Inference Server](https://developer.nvidia.com/nvidia-triton-inference-server) [**Great**] 111 | - [NVIDIA Triton Inference Server - GitHub](https://github.com/triton-inference-server/server) [**Great**] 112 | - [Cohere Boosts Inference Speed With NVIDIA Triton Inference Server](https://txt.cohere.ai/nvidia-boosts-inference-speed-with-cohere/) 113 | - [NVIDIA Deep Learning Examples for Tensor Cores](https://github.com/NVIDIA/DeepLearningExamples) [Interesting] 114 | - [Deploying the Jasper Inference model using Triton Inference Server](https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/SpeechRecognition/Jasper/triton) [Useful] 115 | - [Nvidia MLOPs Course via Triton](https://analyticsindiamag.com/nvidia-is-offering-a-four-hour-self-paced-course-on-mlops/) 116 | - [Awesome Production Machine Learning](https://github.com/EthicalML/awesome-production-machine-learning) [Great] 117 | - [BentoML Model Serving](https://www.linkedin.com/posts/axelmdz_mlops-machinelearning-datascience-activity-7194637600861618176-iQaF?utm_source=share&utm_medium=member_desktop) 118 | 119 | ## Huawei Deep Learning Framework: 120 | - [MindSpore - Huawei Deep Learning Framework](https://github.com/mindspore-ai/mindspore) 121 | - [MindSpore - Tutorial](https://www.mindspore.cn/tutorial/en/0.1.0-alpha/quick_start/quick_start.html) 122 | 123 | ## General Deep Learning Compiler Stack: 124 | - [TVM Stack](https://tvm.ai/) 125 | 126 | ## Model Conversion between Deep Learning Frameworks: 127 | - [ONNX (Open Neural Network Exchange)](https://onnx.ai/) 128 | - [Tutorials for using ONNX](https://github.com/onnx/tutorials) 129 | - [MMdnn](https://github.com/Microsoft/MMdnn) [_Fantastic_] 130 | - [Convert Full ImageNet Pre-trained Model from MXNet to PyTorch](https://blog.paperspace.com/convert-full-imagenet-pre-trained-model-from-mxnet-to-pytorch/) [_Fantastic_, & Full ImageNet model means the model trained on ~ 14M images] 131 | 132 | ## Some Caffe2 Tutorials: 133 | - [Mnist using caffe2](http://vast.uccs.edu/~adhamija/blog/MNIST_singleGPU.html) 134 | - [Caffe2 C++ Tutorials and Examples](https://github.com/leonardvandriel/caffe2_cpp_tutorial) 135 | - [Make Transfer Learning of SqueezeNet on Caffe2](https://medium.com/@KazamiXHayato/make-transfer-learning-in-caffe2-21d96c47ba0e) 136 | - [Build Basic program by using Caffe2 framework in C++](https://medium.com/@KazamiXHayato/write-caffe2-program-in-c-5519e2646382) 137 | 138 | ## Some Useful Resources for Designing UI (Front-End Development): 139 | - [ReactJS vs Angular5 vs Vue.js](https://medium.com/@TechMagic/reactjs-vs-angular5-vs-vue-js-what-to-choose-in-2018-b91e028fa91d) 140 | - [A comparison between Angular and React and their core languages](https://medium.freecodecamp.org/a-comparison-between-angular-and-react-and-their-core-languages-9de52f485a76) 141 | - [A Guide to Becoming a Full-Stack Developer](https://medium.com/coderbyte/a-guide-to-becoming-a-full-stack-developer-in-2017-5c3c08a1600c) [_Very Good Tutorial_] 142 | - [Roadmap to becoming a web developer in 2018](https://github.com/kamranahmedse/developer-roadmap) [_Very Good Repository_] 143 | - [Modern Frontend Developer in 2018](https://medium.com/tech-tajawal/modern-frontend-developer-in-2018-4c2072fa2b9c) 144 | - [Roadmap to becoming a React developer in 2018](https://github.com/adam-golab/react-developer-roadmap) 145 | - [2019 UI and UX Design Trends](https://uxplanet.org/2019-ui-and-ux-design-trends-92dfa8323225) [_Good_] 146 | - [Streamlit](https://streamlit.io/) [_The fastest way to build custom ML tools_] 147 | - [Gradio](https://www.gradio.app/) [**Good**] 148 | - [Web Developer Monthly](https://medium.com/@andreineagoie/web-developer-monthly-july-2018-513e02f15fb6) 149 | - [23 Best React UI Component Frameworks](https://hackernoon.com/23-best-react-ui-component-libraries-and-frameworks-250a81b2ac42) 150 | - [9 React Styled-Components UI Libraries for 2018](https://blog.bitsrc.io/9-react-styled-components-ui-libraries-for-2018-4e1a0bd3e179) 151 | - [35 New Tools for UI Design](https://blog.prototypr.io/35-new-tools-for-ui-design-412cf1d701fd) 152 | - [5 Tools To Speed Up Your App Development](https://medium.com/swlh/5-tools-to-speed-up-your-app-development-6979d0e49e34) [_Very Good_] 153 | - [How to use ReactJS with Webpack 4, Babel 7, and Material Design](https://medium.freecodecamp.org/how-to-use-reactjs-with-webpack-4-babel-7-and-material-design-ff754586f618) 154 | - [Adobe Typekit](https://typekit.com/) [_Great fonts, where you need them_] 155 | - [Build A Real World Beautiful Web APP with Angular 6](https://medium.com/@hamedbaatour/build-a-real-world-beautiful-web-app-with-angular-6-a-to-z-ultimate-guide-2018-part-i-e121dd1d55e) 156 | - [You Don't Know JS](https://github.com/getify/You-Dont-Know-JS) 157 | - [JavaScript Top 10 Articles](https://medium.mybridge.co/javascript-top-10-articles-for-the-past-month-v-sep-2018-8f27a300d6c5) 158 | - [Web Design with Adobe XD](https://medium.freecodecamp.org/a-developers-guide-to-web-design-for-non-designers-1f64ce28c38d) 159 | - [INSPINIA Bootstrap Web Theme](https://wrapbootstrap.com/theme/inspinia-responsive-admin-theme-WB0R5L90S) 160 | - [A Learning Tracker for Front-End Developers](https://github.com/Syknapse/My-Learning-Tracker-first-ten-months) 161 | - [The best front-end hacking cheatsheets — all in one place](https://medium.freecodecamp.org/modern-frontend-hacking-cheatsheets-df9c2566c72a) [_Useful & Interesting_] 162 | - [GUI-fying the Machine Learning Workflow (Machine Flow)](https://towardsdatascience.com/gui-fying-the-machine-learning-workflow-towards-rapid-discovery-of-viable-pipelines-cab2552c909f) 163 | - [Electron - Build cross platform desktop apps with JavaScript](https://electronjs.org/) [_Very Good_] 164 | - [Opyrator - Turns Python functions into microservices with web API](https://github.com/ml-tooling/opyrator) [**Great**] 165 | - [A First Look at PyScript: Python in the Web Browser](https://realpython.com/pyscript-python-in-browser/) [**Interesting**] 166 | 167 | ## Mobile & Embedded Devices Development: 168 | - [PyTorch Mobile](https://pytorch.org/mobile/home/) [_Excellent_] 169 | - [Mobile UI Design Trends In 2018](https://uxplanet.org/mobile-ui-design-trends-in-2018-ccd26031dfd8) 170 | - [ncnn - high-performance neural network inference framework optimized for the mobile platform](https://github.com/Tencent/ncnn) [_Useful_] 171 | - [Alibaba - MNN](https://github.com/alibaba/MNN) 172 | - [Awesome Mobile Machine Learning](https://github.com/fritzlabs/Awesome-Mobile-Machine-Learning) 173 | - [EMDL - Embedded and Mobile Deep Learning](https://github.com/EMDL/awesome-emdl) 174 | - [Fritz - machine learning platform for iOS and Android](https://www.fritz.ai/) 175 | - [TensorFlow Lite](https://www.tensorflow.org/mobile/tflite/) 176 | - [Tiny Machine Learning: The Next AI Revolution](https://medium.com/@matthew_stewart/tiny-machine-learning-the-next-ai-revolution-495c26463868) 177 | - [TLT - NVIDIA Transfer Learning Toolkit](https://developer.nvidia.com/transfer-learning-toolkit) 178 | - [NVIDIA Jetson Inference](https://github.com/dusty-nv/jetson-inference) [_Great_] 179 | - [EfficientML.ai Lecture (MIT Course)](https://www.youtube.com/playlist?list=PL80kAHvQbh-pT4lCkDT53zT8DKmhE0idB) 180 | - [PyTorch Edge: Enabling On-Device Inference Across Mobile and Edge Devices with ExecuTorch](https://pytorch.org/blog/pytorch-edge/?utm_content=268296369&utm_medium=social&utm_source=linkedin&hss_channel=lcp-78618366) 181 | 182 | ## Back-End Development Part: 183 | - [Modern Backend Developer in 2018](https://medium.com/tech-tajawal/modern-backend-developer-in-2018-6b3f7b5f8b9) 184 | - [Deploying frontend applications — the fun way](https://hackernoon.com/deploying-frontend-applications-the-fun-way-bc3f69e15331) [_Very Good_] 185 | - [RabbitMQ](https://www.rabbitmq.com/) [_Message Broker Software_] 186 | - [Celery](http://www.celeryproject.org/) [_Distributed Task Queue_] 187 | - [Kafka](https://kafka.apache.org/) [_Distributed Streaming Platform_] 188 | - [Docker training with DockerMe](https://github.com/AhmadRafiee/Docker_training_with_DockerMe) 189 | - [Kubernetes - GitHub](https://github.com/kubernetes/kubernetes) 190 | - [Deploy Machine Learning Pipeline on Google Kubernetes Engine](https://towardsdatascience.com/deploy-machine-learning-model-on-google-kubernetes-engine-94daac85108b) 191 | - [An introduction to Kubernetes for Data Scientists](https://www.jeremyjordan.me/kubernetes/) 192 | - [Jenkins and Kubernetes with Docker Desktop](https://medium.com/@garunski/jenkins-and-kubernetes-with-docker-desktop-53a853486f7c) 193 | - [Helm: The package manager for Kubernetes](https://helm.sh/) 194 | - [Create Cluster using docker swarm](https://medium.com/tech-tajawal/create-cluster-using-docker-swarm-94d7b2a10c43) 195 | - [deepo - Docker Image for all DL Framewors](https://github.com/ufoym/deepo) 196 | - [Kubeflow](https://www.kubeflow.org/) [_deployments of ML workflows on Kubernetes_] 197 | - [kubespray - Deploy a Production Ready Kubernetes Cluster](https://github.com/kubernetes-sigs/kubespray) 198 | - [KFServing - Kubernetes for Serving ML Models](https://github.com/kubeflow/kfserving) 199 | - [Deploying a HuggingFace NLP Model with KFServing](http://www.pattersonconsultingtn.com/blog/deploying_huggingface_with_kfserving.html) [_Interesting_] 200 | - [Seldon Core - Deploying Machine Learning Models on Kubernetes](https://www.seldon.io/tech/products/core/) 201 | - [Seldon Core - GitHub](https://github.com/SeldonIO/seldon-core) 202 | - [Machine Learning: serving models with Kubeflow on Ubuntu, Part 1](https://ubuntu.com/blog/ml-serving-models-with-kubeflow-on-ubuntu-part-1) 203 | - [CoreWeave Kubernetes Cloud](https://github.com/coreweave/kubernetes-cloud/tree/master/online-inference/) 204 | - [MLOps References](https://github.com/visenger/mlops-references) [_DevOps for ML_] 205 | - [Data Version Control - DVC](https://dvc.org/) [_Great_] 206 | - [MLEM: package and deploy machine learning models](https://github.com/iterative/mlem) 207 | - [PySyft - A library for encrypted, privacy preserving deep learning](https://github.com/OpenMined/PySyft) 208 | - [LocalStack - A fully functional local AWS cloud stack](https://github.com/localstack/localstack) 209 | - [poetry: Python packaging and dependency management](https://python-poetry.org/) 210 | - [Netflix's Powerful Database Selections for Streaming at Scale](https://www.linkedin.com/posts/chandreshdesai_cloudcomputing-devops-softwareengineering-activity-7108414221482115072-HGil?utm_source=share&utm_medium=member_desktop) 211 | - [Ray: a unified framework for scaling AI and Python applications](https://github.com/ray-project/ray) 212 | 213 | ## GPU Management Libraries: 214 | - [GPUtil](https://github.com/anderskm/gputil) 215 | - [py3nvml](https://github.com/fbcotter/py3nvml) [_Python 3 binding to the NVIDIA Management Library_] 216 | - [PyCUDA - GitHub](https://github.com/inducer/pycuda) 217 | - [PyCUDA](https://mathema.tician.de/software/pycuda/) 218 | - [PyCUDA Tutorial](https://documen.tician.de/pycuda/) 219 | - [setGPU](https://github.com/bamos/setGPU) 220 | - [Monitor your GPUs](https://github.com/msalvaris/gpu_monitor) [**Excellent**] 221 | - [GPU-Burn - Multi-GPU CUDA stress test](https://github.com/wilicc/gpu-burn) [_Useful_] 222 | - [Grafana - Monitoring and Observability](https://github.com/grafana/grafana) [**Excellent**] 223 | - [Prometheus](https://prometheus.io/) [_Excellent for monitoring solution & extract required metrics_] 224 | - [OpenAI Triton: Open-Source GPU Programming for Neural Networks](https://openai.com/blog/triton/) 225 | 226 | ## Speed-up & Scalabale Python Codes: 227 | - [Numba - makes Python code fast](http://numba.pydata.org/) 228 | - [Dask - natively scales Python](https://dask.org/) 229 | - [What is Dask](https://medium.com/better-programming/what-is-dask-and-how-can-it-help-you-as-a-data-scientist-72adec7cec57) 230 | - [Ray - running distributed applications](https://github.com/ray-project/ray) 231 | - [Neural Network Distiller](https://github.com/NervanaSystems/distiller/) [_Distillation & Quantization of Deep Learning Models in PyTorch_] 232 | - [Autodistill](https://github.com/autodistill/autodistill) 233 | - [PyTorch Pruning Tutorial](https://pytorch.org/tutorials/intermediate/pruning_tutorial.html) 234 | - [Can you remove 99% of a neural network without losing accuracy? - An introduction to weight pruning](https://towardsdatascience.com/can-you-remove-99-of-a-neural-network-without-losing-accuracy-915b1fab873b) 235 | - [PocketFlow - An Automatic Model Compression (AutoMC) framework](https://github.com/Tencent/PocketFlow) [**Great**] 236 | - [Introducing the Model Optimization Toolkit for TensorFlow](https://medium.com/tensorflow/introducing-the-model-optimization-toolkit-for-tensorflow-254aca1ba0a3) 237 | - [TensorFlow Model Optimization Toolkit — Post-Training Integer Quantization](https://medium.com/tensorflow/tensorflow-model-optimization-toolkit-post-training-integer-quantization-b4964a1ea9ba) 238 | - [TensorFlow Post-training Quantization](https://www.tensorflow.org/lite/performance/post_training_quantization) 239 | - [Dynamic Quantization in PyTorch](https://pytorch.org/tutorials/advanced/dynamic_quantization_tutorial.html) 240 | - [Static Quantization in PyTorch](https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html) 241 | - [NVIDIA DALI - highly optimized data pre-processing in deep learning](https://github.com/NVIDIA/dali) 242 | - [Horovod - Distributed training framework](https://github.com/horovod/horovod) 243 | - [ONNX Float32 to Float16](https://github.com/onnx/onnx-docker/blob/master/onnx-ecosystem/converter_scripts/float32_float16_onnx.ipynb) 244 | - [Speeding Up Deep Learning Inference Using TensorRT](https://devblogs.nvidia.com/speeding-up-deep-learning-inference-using-tensorrt/) 245 | - [Speed up Training](https://ai.googleblog.com/2020/05/speeding-up-neural-network-training.html) 246 | - [Native PyTorch automatic mixed precision for faster training on NVIDIA GPUs](https://pytorch.org/blog/accelerating-training-on-nvidia-gpus-with-pytorch-automatic-mixed-precision/) 247 | - [JAX - Composable transformations of Python+NumPy programs](https://github.com/google/jax) 248 | - [TensorRTx - popular DL networks with tensorrt](https://github.com/wang-xinyu/tensorrtx) 249 | - [Speeding up Deep Learning Inference Using TensorFlow, ONNX, and TensorRT](https://devblogs.nvidia.com/speeding-up-deep-learning-inference-using-tensorflow-onnx-and-tensorrt/) 250 | - [TensorRT Developer Guide](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html) 251 | - [How to Convert a Model from PyTorch to TensorRT and Speed Up Inference](https://www.learnopencv.com/how-to-convert-a-model-from-pytorch-to-tensorrt-and-speed-up-inference/) [_Good_] 252 | - [cuDF: GPU DataFrame Library](https://github.com/rapidsai/cudf) 253 | - [cuDF example](https://colab.research.google.com/drive/12tCzP94zFG2BRduACucn5Q_OcX1TUKY3) 254 | 255 | ## Hardware Notes for Deep Learning: 256 | - [Hardware for Deep Learning](https://blog.inten.to/hardware-for-deep-learning-part-3-gpu-8906c1644664) 257 | 258 | ## MLOPs Courses & Resources: 259 | - [MLOps-Basics](https://github.com/graviraja/MLOps-Basics) [Great] 260 | - [MLOPs-Zoomcamp](https://github.com/DataTalksClub/mlops-zoomcamp) [Great] 261 | - [A collection of resources to learn about MLOPs](https://github.com/dair-ai/MLOPs-Primer) [Great] 262 | - [Awesome MLOPs](https://github.com/visenger/awesome-mlops) [Great] 263 | - [Data Science Topics & MLOPs](https://github.com/khuyentran1401/Data-science#mlops) [Great] 264 | - [MLEM: package and deploy machine learning models](https://github.com/iterative/mlem) 265 | - [DevOps Exercises](https://github.com/bregman-arie/devops-exercises) 266 | - [MlOPs Sample Project](https://github.com/AntonisCSt/Mlops_project_semicon) 267 | - [prefect: Orchestrate and observe all of your workflows](https://www.prefect.io/) 268 | - [DataTalks Club: The place to talk about data](https://datatalks.club/) 269 | - [OpenNMT CTranslate2: Fast inference engine for Transformer models](https://github.com/OpenNMT/CTranslate2/) 270 | 271 | ## Other: 272 | - [A Guide to Production Level Deep Learning](https://github.com/alirezadir/Production-Level-Deep-Learning) 273 | - [Facebook Says Developers Will Love PyTorch 1.0](https://medium.com/syncedreview/facebook-says-developers-will-love-pytorch-1-0-ba2f89ebc9cc) 274 | - [Some PyTorch Workflow Changes](https://github.com/pytorch/pytorch/issues/6032) 275 | - [wandb - A tool for visualizing and tracking your machine learning experiments](https://github.com/wandb/client) 276 | - [PyTorch and Caffe2 repos getting closer together](https://github.com/caffe2/caffe2/issues/2439#issuecomment-391155017) 277 | - [PyTorch or TensorFlow?](https://awni.github.io/pytorch-tensorflow/) 278 | - [Choosing a Deep Learning Framework in 2018: Tensorflow or Pytorch?](http://cv-tricks.com/deep-learning-2/tensorflow-or-pytorch/) 279 | - [Deep Learning War between PyTorch & TensorFlow](https://hub.packtpub.com/can-a-production-ready-pytorch-1-0-give-tensorflow-a-tough-time/) 280 | - [Embedding Machine Learning Models to Web Apps (Part-1)](https://towardsdatascience.com/embedding-machine-learning-models-to-web-apps-part-1-6ab7b55ee428) 281 | - [Deploying deep learning models: Part 1 an overview](https://towardsdatascience.com/deploying-deep-learning-models-part-1-an-overview-77b4d01dd6f7) 282 | - [Machine Learning in Production](https://medium.com/contentsquare-engineering-blog/machine-learning-in-production-c53b43283ab1) 283 | - [how you can get a 2–6x speed-up on your data pre-processing with Python](https://towardsdatascience.com/heres-how-you-can-get-a-2-6x-speed-up-on-your-data-pre-processing-with-python-847887e63be5) 284 | - [Making your C library callable from Python](https://medium.com/@shamir.stav_83310/making-your-c-library-callable-from-python-by-wrapping-it-with-cython-b09db35012a3) 285 | - [MIL WebDNN](https://mil-tokyo.github.io/webdnn/) 286 | - [Multi-GPU Framework Comparisons](https://medium.com/@iliakarmanov/multi-gpu-rosetta-stone-d4fa96162986) [_Great_] 287 | 288 | --------------------------------------------------------------------------------