├── .gitignore ├── .dockerignore ├── requirements.txt ├── models └── transformers │ └── put your model here.png ├── Dockerfile ├── make_req.py ├── app.py ├── save_hf_model.py └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | fapp_env 2 | _models 3 | yedek 4 | todos.txt 5 | readme.txt 6 | linkedin post -------------------------------------------------------------------------------- /.dockerignore: -------------------------------------------------------------------------------- 1 | .gitignore 2 | fapp_env 3 | README.md 4 | todos.txt 5 | make_req.py 6 | save_hf_models.py 7 | 8 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aslanismailgit/HuggingFace-Transformers-Model-Docker-Container/HEAD/requirements.txt -------------------------------------------------------------------------------- /models/transformers/put your model here.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aslanismailgit/HuggingFace-Transformers-Model-Docker-Container/HEAD/models/transformers/put your model here.png -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | # syntax=docker/dockerfile:1 2 | 3 | FROM python:3.8 4 | 5 | WORKDIR /app 6 | 7 | COPY requirements.txt requirements.txt 8 | RUN pip3 install -r requirements.txt 9 | 10 | COPY . . 11 | 12 | EXPOSE 5000 13 | CMD [ "python", "app.py"] -------------------------------------------------------------------------------- /make_req.py: -------------------------------------------------------------------------------- 1 | #%% 2 | import requests 3 | import json 4 | #%% 5 | # url = "http://127.0.0.1:5000/" # for flask 6 | url = "http://127.0.0.1:9000/" # for docker if exposed to 9000 7 | data = ["good", "bad"] 8 | j_data = json.dumps(data) 9 | headers = {'content-type': 'application/json', 'Accept-Charset': 'UTF-8'} 10 | r = requests.post(url, data=j_data, headers=headers) 11 | print(r, r.text) 12 | # %% 13 | # r 14 | # %% 15 | # sent_score = json.loads(r.text) 16 | # sent_score 17 | # %% 18 | # label = sent_score[0]["label"] 19 | # score = sent_score[0]["score"] 20 | # label, score 21 | # %% 22 | -------------------------------------------------------------------------------- /app.py: -------------------------------------------------------------------------------- 1 | from flask import Flask, request, jsonify 2 | import numpy as np 3 | from transformers import AutoTokenizer, TFAutoModelForSequenceClassification 4 | from transformers import pipeline 5 | 6 | app = Flask(__name__) 7 | 8 | 9 | @app.route('/', methods=['GET', 'POST']) 10 | def makecalc(): 11 | if request.method=="POST": 12 | data = request.get_json() 13 | print("data ---- > ", data) 14 | # prediction = np.array2string(model.predict(data)) 15 | results = classifier(data) 16 | 17 | return jsonify(results) 18 | return "Not a proper request method or data" 19 | 20 | 21 | if __name__ == '__main__': 22 | 23 | model_path = './models/transformers/' 24 | model = TFAutoModelForSequenceClassification.from_pretrained(model_path, local_files_only=True) 25 | print("----------- transformer model loaded ------------") 26 | tokenizer = AutoTokenizer.from_pretrained(model_path, local_files_only=True) 27 | print("----------- transformer tokenizer loaded ------------") 28 | classifier = pipeline('sentiment-analysis', model=model, tokenizer=tokenizer) 29 | print(classifier) 30 | 31 | app.run(debug=True, host='0.0.0.0') 32 | 33 | -------------------------------------------------------------------------------- /save_hf_model.py: -------------------------------------------------------------------------------- 1 | #%% import required libraries 2 | from transformers import pipeline 3 | from transformers import AutoTokenizer, TFAutoModelForSequenceClassification 4 | model_path = 'models/transformers/' # will be created automatically if not exists 5 | 6 | #%% download and save the model to local directory 7 | model_name = "nlptown/bert-base-multilingual-uncased-sentiment" 8 | 9 | model = TFAutoModelForSequenceClassification.from_pretrained(model_name, from_pt=True) 10 | tokenizer = AutoTokenizer.from_pretrained(model_name) 11 | classifier = pipeline('sentiment-analysis', model=model, tokenizer=tokenizer) 12 | classifier.save_pretrained(model_path) 13 | #%% test if it works 14 | classifier(["good"]) 15 | 16 | #%% load model from local directory if it works 17 | model = TFAutoModelForSequenceClassification.from_pretrained(model_path, local_files_only=True) 18 | print("----------- model loaded from local dir ------------") 19 | tokenizer = AutoTokenizer.from_pretrained(model_path, local_files_only=True) 20 | print("----------- tokenizer loaded from local dir ------------") 21 | classifier = pipeline('sentiment-analysis', model=model, tokenizer=tokenizer) 22 | 23 | classifier(["good"]) 24 | 25 | # %% 26 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # How to containerize a HuggingFace Transformers Model using Docker? 3 | 4 | *I assume you are already a little bit familiar with below libraries and docker.* 5 | **Required libraries:** Flask, transformers, tensorflow. (pip or conda as you wish, I used pip) 6 | - If you are using tensorflow, as I do, you will need PyTorch only if you are using a hf model trained on PyTorch, with the flag from_pt=true. But, to reload and re-use the model from local you don’t need PyTorch again, so will not be needed in your container. 7 | - Step 1: Load and save the transformer model in a local directory using save_hf_model.py 8 | - Step 2: Create a minimal flask app, in fact you can use the above one without changing anything. Just replace your model with the one in the models directory. Recommend to test your app at this level. 9 | - Step 3: Containerize the app using Dockerfile: 10 | `docker build --tag mlapp . ` 11 | `docker run -i -p 9000:5000 mlapp` *(add -d flag to run in detach mode in the background, you can change 9000 as you need)* 12 | - Check if your docker is up and running 13 | `docker ps` 14 | 15 | | CONTAINER ID | IMAGE | COMMAND | CREATED | STATUS | PORTS | NAMES | 16 | |--------------|--------|----------|----------|----------|-------|-------| 17 | | 1fbcac69069c | mlapp | "python app.py" | 50 seconds ago | Up 49 seconds | 0.0.0.0:9000->5000/tcp | crazy_pike | 18 | 19 | - Check if the container is responding `curl 127.0.0.1:9000 -v` 20 | - Step 4: Test your model with make_req.py. Please note that your data should be in the correct format, for example, as you tested your model in save_hf_model.py. 21 | - Step 5: To stop your docker container 22 | `docker stop 1fbcac69069c` 23 | 24 | Your model is now running in your container, ready to deploy anywhere. 25 | 26 | ## Happy machine learning! 27 | --------------------------------------------------------------------------------