├── .gitignore ├── .vscode └── settings.json ├── 1. Simple Flask APP ├── Docker-compose.yml ├── Dockerfile ├── app.py └── requirements.txt ├── 2. Passing Arguments APP ├── Docker-compose.yml ├── Dockerfile ├── app.py └── requirements.txt ├── 3. Simple ML APP ├── Docker-compose.yml ├── Dockerfile ├── app.py ├── dataset.csv ├── requirements.txt └── train.py ├── 4. Image Classification APP ├── Docker-compose.yml ├── Dockerfile ├── app.py ├── requirements.txt └── train.py └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | .vscode/ 2 | img.jpg 3 | -------------------------------------------------------------------------------- /.vscode/settings.json: -------------------------------------------------------------------------------- 1 | { 2 | "python.pythonPath": "C:\\Miniconda3\\envs\\root_env\\python.exe", 3 | "python.linting.pylintEnabled": true, 4 | "python.linting.flake8Enabled": false, 5 | "python.linting.enabled": true 6 | } -------------------------------------------------------------------------------- /1. Simple Flask APP/Docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3' 2 | 3 | services: 4 | simple_app: 5 | build: . 6 | ports: 7 | - "5000:5000" 8 | container_name: simple_app -------------------------------------------------------------------------------- /1. Simple Flask APP/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.6-slim-buster 2 | LABEL Gagandeep Singh 3 | COPY . /app 4 | WORKDIR /app 5 | RUN pip3 install -r requirements.txt 6 | CMD ["python3","app.py"] -------------------------------------------------------------------------------- /1. Simple Flask APP/app.py: -------------------------------------------------------------------------------- 1 | from flask import Flask 2 | 3 | app = Flask(__name__) 4 | 5 | 6 | @app.route('/') 7 | def hello_world(): 8 | return "Hello World" 9 | 10 | 11 | if __name__ == '__main__': 12 | app.run(host='0.0.0.0') 13 | -------------------------------------------------------------------------------- /1. Simple Flask APP/requirements.txt: -------------------------------------------------------------------------------- 1 | Flask -------------------------------------------------------------------------------- /2. Passing Arguments APP/Docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3' 2 | services: 3 | passing_args: 4 | build: . 5 | ports: 6 | - "5000:5000" 7 | container_name: passing_args -------------------------------------------------------------------------------- /2. Passing Arguments APP/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.6-slim-buster 2 | LABEL maintainer "Gagandeep Singh" 3 | COPY . /app 4 | WORKDIR /app 5 | RUN pip3 install -r requirements.txt 6 | CMD ["python3","app.py"] -------------------------------------------------------------------------------- /2. Passing Arguments APP/app.py: -------------------------------------------------------------------------------- 1 | from flask import Flask, request 2 | 3 | app = Flask(__name__) 4 | 5 | 6 | @app.route('/predict', methods=['POST']) 7 | def print_square(): 8 | recieved_value = int(request.get_json(force=True)) 9 | return str(recieved_value**2) 10 | 11 | 12 | if __name__ == "__main__": 13 | app.run(host='0.0.0.0') 14 | -------------------------------------------------------------------------------- /2. Passing Arguments APP/requirements.txt: -------------------------------------------------------------------------------- 1 | Flask -------------------------------------------------------------------------------- /3. Simple ML APP/Docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3' 2 | services: 3 | simple_ml_app: 4 | build: . 5 | ports: 6 | - "5000:5000" 7 | container_name: simple_ml_app -------------------------------------------------------------------------------- /3. Simple ML APP/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.6-slim-buster 2 | COPY . /app 3 | WORKDIR /app 4 | RUN pip3 install -r requirements.txt \ 5 | && python3 train.py 6 | CMD ["python3","app.py"] -------------------------------------------------------------------------------- /3. Simple ML APP/app.py: -------------------------------------------------------------------------------- 1 | import pickle 2 | from flask import Flask, request, jsonify 3 | 4 | app = Flask(__name__) 5 | 6 | 7 | model = None 8 | with open('model.pkl', 'rb') as f: 9 | model = pickle.load(f) 10 | 11 | 12 | @app.route('/predict', methods=['POST']) 13 | def predict(): 14 | parameters = request.get_json(force=True) 15 | f1 = int(parameters['f1']) 16 | f2 = int(parameters['f2']) 17 | f3 = int(parameters['f3']) 18 | spending_score = model.predict(([f1, f2, f3],))[0] 19 | return jsonify(spending_score) 20 | 21 | 22 | if __name__ == '__main__': 23 | app.run(host='0.0.0.0') 24 | -------------------------------------------------------------------------------- /3. Simple ML APP/dataset.csv: -------------------------------------------------------------------------------- 1 | CustomerID,Gender,Age,Annual Income (k$),Spending Score (1-100) 2 | 1,Male,19,15,39 3 | 2,Male,21,15,81 4 | 3,Female,20,16,6 5 | 4,Female,23,16,77 6 | 5,Female,31,17,40 7 | 6,Female,22,17,76 8 | 7,Female,35,18,6 9 | 8,Female,23,18,94 10 | 9,Male,64,19,3 11 | 10,Female,30,19,72 12 | 11,Male,67,19,14 13 | 12,Female,35,19,99 14 | 13,Female,58,20,15 15 | 14,Female,24,20,77 16 | 15,Male,37,20,13 17 | 16,Male,22,20,79 18 | 17,Female,35,21,35 19 | 18,Male,20,21,66 20 | 19,Male,52,23,29 21 | 20,Female,35,23,98 22 | 21,Male,35,24,35 23 | 22,Male,25,24,73 24 | 23,Female,46,25,5 25 | 24,Male,31,25,73 26 | 25,Female,54,28,14 27 | 26,Male,29,28,82 28 | 27,Female,45,28,32 29 | 28,Male,35,28,61 30 | 29,Female,40,29,31 31 | 30,Female,23,29,87 32 | 31,Male,60,30,4 33 | 32,Female,21,30,73 34 | 33,Male,53,33,4 35 | 34,Male,18,33,92 36 | 35,Female,49,33,14 37 | 36,Female,21,33,81 38 | 37,Female,42,34,17 39 | 38,Female,30,34,73 40 | 39,Female,36,37,26 41 | 40,Female,20,37,75 42 | 41,Female,65,38,35 43 | 42,Male,24,38,92 44 | 43,Male,48,39,36 45 | 44,Female,31,39,61 46 | 45,Female,49,39,28 47 | 46,Female,24,39,65 48 | 47,Female,50,40,55 49 | 48,Female,27,40,47 50 | 49,Female,29,40,42 51 | 50,Female,31,40,42 52 | 51,Female,49,42,52 53 | 52,Male,33,42,60 54 | 53,Female,31,43,54 55 | 54,Male,59,43,60 56 | 55,Female,50,43,45 57 | 56,Male,47,43,41 58 | 57,Female,51,44,50 59 | 58,Male,69,44,46 60 | 59,Female,27,46,51 61 | 60,Male,53,46,46 62 | 61,Male,70,46,56 63 | 62,Male,19,46,55 64 | 63,Female,67,47,52 65 | 64,Female,54,47,59 66 | 65,Male,63,48,51 67 | 66,Male,18,48,59 68 | 67,Female,43,48,50 69 | 68,Female,68,48,48 70 | 69,Male,19,48,59 71 | 70,Female,32,48,47 72 | 71,Male,70,49,55 73 | 72,Female,47,49,42 74 | 73,Female,60,50,49 75 | 74,Female,60,50,56 76 | 75,Male,59,54,47 77 | 76,Male,26,54,54 78 | 77,Female,45,54,53 79 | 78,Male,40,54,48 80 | 79,Female,23,54,52 81 | 80,Female,49,54,42 82 | 81,Male,57,54,51 83 | 82,Male,38,54,55 84 | 83,Male,67,54,41 85 | 84,Female,46,54,44 86 | 85,Female,21,54,57 87 | 86,Male,48,54,46 88 | 87,Female,55,57,58 89 | 88,Female,22,57,55 90 | 89,Female,34,58,60 91 | 90,Female,50,58,46 92 | 91,Female,68,59,55 93 | 92,Male,18,59,41 94 | 93,Male,48,60,49 95 | 94,Female,40,60,40 96 | 95,Female,32,60,42 97 | 96,Male,24,60,52 98 | 97,Female,47,60,47 99 | 98,Female,27,60,50 100 | 99,Male,48,61,42 101 | 100,Male,20,61,49 102 | 101,Female,23,62,41 103 | 102,Female,49,62,48 104 | 103,Male,67,62,59 105 | 104,Male,26,62,55 106 | 105,Male,49,62,56 107 | 106,Female,21,62,42 108 | 107,Female,66,63,50 109 | 108,Male,54,63,46 110 | 109,Male,68,63,43 111 | 110,Male,66,63,48 112 | 111,Male,65,63,52 113 | 112,Female,19,63,54 114 | 113,Female,38,64,42 115 | 114,Male,19,64,46 116 | 115,Female,18,65,48 117 | 116,Female,19,65,50 118 | 117,Female,63,65,43 119 | 118,Female,49,65,59 120 | 119,Female,51,67,43 121 | 120,Female,50,67,57 122 | 121,Male,27,67,56 123 | 122,Female,38,67,40 124 | 123,Female,40,69,58 125 | 124,Male,39,69,91 126 | 125,Female,23,70,29 127 | 126,Female,31,70,77 128 | 127,Male,43,71,35 129 | 128,Male,40,71,95 130 | 129,Male,59,71,11 131 | 130,Male,38,71,75 132 | 131,Male,47,71,9 133 | 132,Male,39,71,75 134 | 133,Female,25,72,34 135 | 134,Female,31,72,71 136 | 135,Male,20,73,5 137 | 136,Female,29,73,88 138 | 137,Female,44,73,7 139 | 138,Male,32,73,73 140 | 139,Male,19,74,10 141 | 140,Female,35,74,72 142 | 141,Female,57,75,5 143 | 142,Male,32,75,93 144 | 143,Female,28,76,40 145 | 144,Female,32,76,87 146 | 145,Male,25,77,12 147 | 146,Male,28,77,97 148 | 147,Male,48,77,36 149 | 148,Female,32,77,74 150 | 149,Female,34,78,22 151 | 150,Male,34,78,90 152 | 151,Male,43,78,17 153 | 152,Male,39,78,88 154 | 153,Female,44,78,20 155 | 154,Female,38,78,76 156 | 155,Female,47,78,16 157 | 156,Female,27,78,89 158 | 157,Male,37,78,1 159 | 158,Female,30,78,78 160 | 159,Male,34,78,1 161 | 160,Female,30,78,73 162 | 161,Female,56,79,35 163 | 162,Female,29,79,83 164 | 163,Male,19,81,5 165 | 164,Female,31,81,93 166 | 165,Male,50,85,26 167 | 166,Female,36,85,75 168 | 167,Male,42,86,20 169 | 168,Female,33,86,95 170 | 169,Female,36,87,27 171 | 170,Male,32,87,63 172 | 171,Male,40,87,13 173 | 172,Male,28,87,75 174 | 173,Male,36,87,10 175 | 174,Male,36,87,92 176 | 175,Female,52,88,13 177 | 176,Female,30,88,86 178 | 177,Male,58,88,15 179 | 178,Male,27,88,69 180 | 179,Male,59,93,14 181 | 180,Male,35,93,90 182 | 181,Female,37,97,32 183 | 182,Female,32,97,86 184 | 183,Male,46,98,15 185 | 184,Female,29,98,88 186 | 185,Female,41,99,39 187 | 186,Male,30,99,97 188 | 187,Female,54,101,24 189 | 188,Male,28,101,68 190 | 189,Female,41,103,17 191 | 190,Female,36,103,85 192 | 191,Female,34,103,23 193 | 192,Female,32,103,69 194 | 193,Male,33,113,8 195 | 194,Female,38,113,91 196 | 195,Female,47,120,16 197 | 196,Female,35,120,79 198 | 197,Female,45,126,28 199 | 198,Male,32,126,74 200 | 199,Male,32,137,18 201 | 200,Male,30,137,83 202 | -------------------------------------------------------------------------------- /3. Simple ML APP/requirements.txt: -------------------------------------------------------------------------------- 1 | pandas 2 | sklearn 3 | Flask -------------------------------------------------------------------------------- /3. Simple ML APP/train.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | import pickle 3 | from sklearn.ensemble import RandomForestRegressor 4 | from sklearn.model_selection import train_test_split 5 | from sklearn.preprocessing import LabelEncoder 6 | 7 | df = pd.read_csv('dataset.csv') 8 | 9 | df['Gender'] = pd.get_dummies(df['Gender'], prefix='Gender') 10 | 11 | features = df.drop(['Spending Score (1-100)','CustomerID'], axis=1) 12 | target = df['Spending Score (1-100)'] 13 | 14 | 15 | random = RandomForestRegressor(n_estimators=3) 16 | random.fit(features,target) 17 | 18 | with open('model.pkl','wb') as file: 19 | pickle.dump(random,file) 20 | print("Model saved successfully") -------------------------------------------------------------------------------- /4. Image Classification APP/Docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3' 2 | services: 3 | image_classification: 4 | build: . 5 | ports: 6 | - "5000:5000" 7 | container_name: image_classification -------------------------------------------------------------------------------- /4. Image Classification APP/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.6-slim-buster 2 | COPY . /app 3 | WORKDIR /app 4 | RUN pip3 install -r requirements.txt 5 | RUN python3 train.py 6 | CMD ["python3","app.py"] -------------------------------------------------------------------------------- /4. Image Classification APP/app.py: -------------------------------------------------------------------------------- 1 | from flask import Flask, request, jsonify 2 | from PIL import Image 3 | import numpy as np 4 | from tensorflow import keras 5 | 6 | model = keras.models.load_model('model.h5') 7 | 8 | app = Flask(__name__) 9 | 10 | 11 | @app.route('/predict', methods=['POST']) 12 | def classify_image(): 13 | label = None 14 | f = request.files['file'] 15 | print(f) 16 | img = Image.open(f).convert('L') 17 | img = np.array(img.resize((28, 28)))/255.0 18 | img = img.reshape(1, img.shape[0], img.shape[1], 1) 19 | label = int(np.argmax((model.predict(img)))) 20 | print(label) 21 | return jsonify(label) 22 | 23 | 24 | if __name__ == '__main__': 25 | # app.run(host='0.0.0.0') 26 | app.run() 27 | -------------------------------------------------------------------------------- /4. Image Classification APP/requirements.txt: -------------------------------------------------------------------------------- 1 | tensorflow 2 | keras 3 | Pillow 4 | Flask 5 | numpy -------------------------------------------------------------------------------- /4. Image Classification APP/train.py: -------------------------------------------------------------------------------- 1 | import pickle 2 | 3 | # from keras.datasets.mnist import load_data 4 | # from keras.layers import Dense, Flatten, Conv2D, MaxPooling2D, Dropout 5 | # from keras.models import Sequential 6 | # from keras.utils import to_categorical 7 | from tensorflow.keras.layers import Dense, Flatten, Conv2D, MaxPooling2D, Dropout 8 | from tensorflow.keras.datasets.mnist import load_data 9 | from tensorflow.keras.models import Sequential 10 | from tensorflow.keras.utils import to_categorical 11 | 12 | 13 | def create_model(n_filters, n_hidden, n_classes=10): 14 | model = Sequential() 15 | model.add(Conv2D(n_filters, (3, 3), padding='same',input_shape=(28, 28, 1), activation='relu')) 16 | model.add(MaxPooling2D((2, 2))) 17 | model.add(Conv2D(n_filters, (3, 3), padding='same', activation='relu')) 18 | model.add(Flatten()) 19 | model.add(Dense(n_hidden, activation='relu')) 20 | model.add(Dropout(0.3)) 21 | model.add(Dense(n_classes, activation='softmax')) 22 | model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy']) 23 | return model 24 | 25 | 26 | def train(model, x_train, y_train): 27 | x_train = x_train/255.0 28 | x_train = x_train.reshape(x_train.shape[0], 29 | x_train.shape[1], 30 | x_train.shape[2], 31 | 1) 32 | y_train = to_categorical(y_train) 33 | model.fit(x_train, y_train, batch_size=64, epochs=1, validation_split=0.2) 34 | return model 35 | 36 | 37 | if __name__ == '__main__': 38 | n_filters = 32 39 | n_hidden = 256 40 | (x_train, y_train), (x_test, y_test) = load_data() 41 | 42 | model = create_model(n_filters, n_hidden) 43 | trained_model = train(model, x_train, y_train) 44 | trained_model.save('model.h5') 45 | # tf.saved_model.save(pretrained_model, "/tmp/mobilenet/1/") 46 | # with open('model.pkl', 'wb') as f: 47 | # pickle.dump(trained_model, f) 48 | print("model saved") 49 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Docker Examples for just starters 2 | 3 | ### What is a docker and why to use it? 4 | Docker makes machine learning model deployment easy. For example- while building your model you setup production environment on your local system with all the dependencies. Now, you have to replace exact same steps on server and consider there are 4 servers. It will become a tedious task to replace and there also changes that might screw a step. With DOCKER you can create a Dockerfile which will be do all our tasks on as much machines as we want. 5 | 6 | ## Medium Article [link](https://towardsdatascience.com/docker-made-easy-for-data-scientists-b32efbc23165) 7 | 8 | ### How to write Dockerfile 9 | #### Elements 10 | 1. FROM 11 | ``` 12 | The image that you want to use. This is the base image. Example- ubuntu, redis, nginx. 13 | You can also specify the version that you want to pull. 14 | Example FROM ubuntu:18.04 15 | ``` 16 | 2. COPY 17 | ``` 18 | All the files that are required to run the project need to be copied inside docker container. 19 | Copy each file or individually or use '.' to copy all files to app direcotry inside container( COPY . /app). Use dockerignore file to remove redundant files. 20 | Example COPY . /app 21 | ``` 22 | 3. WORKDIR 23 | ``` 24 | This command is used to set the working directory. After you have copied the files inside container. Set that directory as working directory. 25 | Example WORKDIR /app 26 | ``` 27 | 4. RUN 28 | ``` 29 | Any step that is required before running the main file is done here. Example- running apt update, installing a particular package, etc. 30 | Example RUN apt update & apt install redis 31 | 32 | ``` 33 | ``` 34 | Each run statement creates a new layer. The goal is to minimise number of layers. Hence, try to merge all RUN statements into one. 35 | ``` 36 | 5. CMD 37 | ``` 38 | Enter the command here that you want to execute as soon as the docker build process completes. 39 | There can be only one CMD. 40 | Example CMD ["python", "app.py"] 41 | ``` 42 | ### Example Dockerfile 43 | ``` 44 | FROM ubuntu:latest 45 | RUN apt update &&\ 46 | apt install python3-pip 47 | COPY . /app 48 | WORKDIR /app 49 | RUN pip3 install -r requirements.txt 50 | CMD ["python", "app.py"] 51 | ``` 52 | 53 | ### NOTE 54 | The commands should be in order of least updating to most updating. Why? Everytime you make a change a new docker image will be build. 55 | Cache files will be used for the commands that remain unchanged. 56 | 57 | ### To build docker image, type 58 | ``` 59 | $ docker build -t simpleflask . 60 | ``` 61 | or 62 | ``` 63 | $ docker build . 64 | ``` 65 | ##### Make sure your Dockerfile is in the same directory. 66 | 67 | ### To view the build docker image 68 | ``` 69 | $ docker images 70 | ``` 71 | 72 | #### To run the docker image, type 73 | ``` 74 | $ docker run -d -p 5000:5000 simpleflask 75 | ``` 76 | ##### -d means running in deamon mode (background) or detached mode. 77 | ##### -p means the port 78 | 79 | #### To check if docker container is running 80 | ``` 81 | $ docker ps 82 | ``` 83 | You'll see a docker container there. 84 | 85 | #### To start or stop container 86 | ``` 87 | $ docker start/stop continaer_name or id 88 | ``` 89 | 90 | #### Calling flask API inside Docker container 91 | 1. To pass data inside it 92 | ``` 93 | import requests 94 | import json 95 | 96 | data = {'f1': 5, 97 | 'f2': 3, 98 | 'f3': 4} 99 | URL = 'http://127.0.0.1:5000/predict' 100 | 101 | output = requests.post(URL, json.dumps(data)) 102 | 103 | print(f"Status = {output.status}\nResult = {output.text}") 104 | ``` 105 | Just convert the data into JSON form and send a POST request 106 | 107 | ### Few best practices for creating Dockerfile 108 | 1. The part or lines that will be least updated should be placed at Top and the lines which will be modified most should be placed at the bottom. Why? Everytime you make an update to Dockerfile, all the lines after that will run again. 109 | 2. After any change you'll have to build the Dockerfile again and delete the old one. 110 | 3. In case when there are more frequent operations then do all compulsory step in one image and then use it as base image to reduce overall container size and build time. 111 | --------------------------------------------------------------------------------