├── .gitignore ├── Procfile ├── .slugignore ├── requirements.txt ├── app deployed.png ├── istockphoto-922783734-612x612.jpg ├── streamlit web app deployment screenshot.png ├── setup.sh ├── food_web_app.py ├── food_model.py └── readme.MD /.gitignore: -------------------------------------------------------------------------------- 1 | /train 2 | /valid -------------------------------------------------------------------------------- /Procfile: -------------------------------------------------------------------------------- 1 | web: sh setup.sh && streamlit run food_web_app.py 2 | -------------------------------------------------------------------------------- /.slugignore: -------------------------------------------------------------------------------- 1 | # these files are going to be ignore during deployment 2 | 3 | /train 4 | /valid -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | tensorflow-cpu 2 | pillow==6.2.0 3 | numpy==1.21.6 4 | streamlit==0.62.0 5 | pip 6 | -------------------------------------------------------------------------------- /app deployed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SohamBera16/foodetecto-An-AI-based-Food-category-detection-app-using-Streamlit-and-Heroku-Cloud/HEAD/app deployed.png -------------------------------------------------------------------------------- /istockphoto-922783734-612x612.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SohamBera16/foodetecto-An-AI-based-Food-category-detection-app-using-Streamlit-and-Heroku-Cloud/HEAD/istockphoto-922783734-612x612.jpg -------------------------------------------------------------------------------- /streamlit web app deployment screenshot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SohamBera16/foodetecto-An-AI-based-Food-category-detection-app-using-Streamlit-and-Heroku-Cloud/HEAD/streamlit web app deployment screenshot.png -------------------------------------------------------------------------------- /setup.sh: -------------------------------------------------------------------------------- 1 | mkdir -p ~/.streamlit/ 2 | echo "\ 3 | [server]\n\ 4 | port = $PORT\n\ 5 | enableCORS = false\n\ 6 | headless = true\n\ 7 | \n\ 8 | " > ~/.streamlit/config.toml 9 | -------------------------------------------------------------------------------- /food_web_app.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import streamlit as st 3 | import tensorflow as tf 4 | from PIL import Image, ImageOps 5 | 6 | def import_and_predict(image_data, model): 7 | 8 | size = (75,75) 9 | image = ImageOps.fit(image_data, size, Image.ANTIALIAS) 10 | image = image.convert('RGB') 11 | image = np.asarray(image) 12 | image = (image.astype(np.float32) / 255.0) 13 | 14 | img_reshape = image[np.newaxis,...] 15 | 16 | prediction = model.predict(img_reshape) 17 | 18 | return prediction 19 | 20 | model = tf.keras.models.load_model('best_model.hdf5') 21 | 22 | st.write(""" 23 | # Food item Prediction 24 | """ 25 | ) 26 | 27 | st.write("This is a simple image classification web app to predict food item name") 28 | 29 | file = st.file_uploader("Please upload an image file", type=["jpg", "png"]) 30 | # 31 | if file is None: 32 | st.text("You haven't uploaded an image file") 33 | else: 34 | image = Image.open(file) 35 | st.image(image, use_column_width=True) 36 | prediction = import_and_predict(image, model) 37 | 38 | if np.argmax(prediction) == 0: 39 | st.write("It is a apple pie!") 40 | elif np.argmax(prediction) == 1: 41 | st.write("It is a bibimbap!") 42 | elif np.argmax(prediction) == 2: 43 | st.write("It is a cannoli!") 44 | elif np.argmax(prediction) == 3: 45 | st.write("It is a edamame!") 46 | elif np.argmax(prediction) == 4: 47 | st.write("It is a falafel!") 48 | elif np.argmax(prediction) == 5: 49 | st.write("It is a french toast!") 50 | elif np.argmax(prediction) == 6: 51 | st.write("It is a ice cream!") 52 | elif np.argmax(prediction) == 7: 53 | st.write("It is a ramen!") 54 | elif np.argmax(prediction) == 8: 55 | st.write("It is a sushi!") 56 | else: 57 | st.write("It is a tiramisu!") 58 | 59 | st.text("Probability (0: apple pie, 1: bibimbap, 2: cannoli, 3: edamame, 4: falafel, 5: toast, 6: ice cream, 7: ramen, 8: sushi, 9: tiramisu") 60 | st.write(prediction) -------------------------------------------------------------------------------- /food_model.py: -------------------------------------------------------------------------------- 1 | import os 2 | from random import shuffle 3 | from tensorflow.keras.applications.inception_v3 import InceptionV3 4 | from tensorflow.keras.layers import Flatten, Dense, Dropout 5 | from tensorflow.keras.models import Model 6 | from tensorflow.keras.preprocessing.image import ImageDataGenerator 7 | import tensorflow as tf 8 | 9 | def image_gen_w_aug(train_parent_directory, test_parent_directory): 10 | 11 | train_datagen = ImageDataGenerator(rescale=1/255, 12 | rotation_range = 10, 13 | zoom_range = 0.2, 14 | width_shift_range=0.1, 15 | height_shift_range=0.1, 16 | validation_split = 0.2) 17 | 18 | 19 | 20 | test_datagen = ImageDataGenerator(rescale=1/255) 21 | 22 | train_generator = train_datagen.flow_from_directory(train_parent_directory, 23 | target_size = (75,75), 24 | batch_size = 128, 25 | class_mode = 'categorical', 26 | subset='training', 27 | shuffle=True) 28 | 29 | val_generator = train_datagen.flow_from_directory(train_parent_directory, 30 | target_size = (75,75), 31 | batch_size = 128, 32 | class_mode = 'categorical', 33 | subset = 'validation') 34 | 35 | test_generator = test_datagen.flow_from_directory(test_parent_directory, 36 | target_size=(75,75), 37 | batch_size = 32, 38 | class_mode = 'categorical') 39 | 40 | return train_generator, val_generator, test_generator 41 | 42 | 43 | def model_output_for_TL (pre_trained_model, last_output): 44 | 45 | x = Flatten()(last_output) 46 | 47 | # Dense hidden layer 48 | x = Dense(512, activation='relu')(x) 49 | x = Dropout(0.2)(x) 50 | 51 | # Output neuron. 52 | x = Dense(10, activation='softmax')(x) 53 | 54 | model = Model(pre_trained_model.input, x) 55 | 56 | return model 57 | 58 | 59 | train_dir = os.path.join('D:/archive/data/food-101-tiny/train') 60 | test_dir = os.path.join('D:/archive/data/food-101-tiny/valid') 61 | 62 | train_generator, validation_generator, test_generator = image_gen_w_aug(train_dir, test_dir) 63 | 64 | pre_trained_model = InceptionV3(input_shape = (75, 75, 3), 65 | include_top = False, 66 | weights = 'imagenet') 67 | 68 | for layer in pre_trained_model.layers: 69 | layer.trainable = False 70 | 71 | last_layer = pre_trained_model.get_layer('mixed5') 72 | last_output = last_layer.output 73 | 74 | model_TL = model_output_for_TL(pre_trained_model, last_output) 75 | model_TL.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy']) 76 | 77 | history_TL = model_TL.fit( 78 | train_generator, 79 | steps_per_epoch=10, 80 | epochs=10, 81 | verbose=1, 82 | validation_data = validation_generator) 83 | 84 | tf.keras.models.save_model(model_TL,'best_model.hdf5') 85 | 86 | test_TL = model_TL.predict(test_generator) 87 | 88 | print(test_generator[0]) 89 | print(test_TL[0]) 90 | 91 | -------------------------------------------------------------------------------- /readme.MD: -------------------------------------------------------------------------------- 1 | # Image based popular food items classification using AI and corresponding Web application deployment for end-users using Heroku 2 | 3 | ![A variety of foods](https://github.com/SohamBera16/foodetecto-An-AI-based-Food-category-detection-app-using-Streamlit-and-Heroku-Cloud/blob/main/istockphoto-922783734-612x612.jpg) 4 | 5 | * ### Problem Statement : 6 | In today's world, each and every one of us click thousands of images in our daily lives, and one of the most popular categories of these images is *food items*. No matter whether we go to a aesthetically appealing new restaurant or make a delicious meal in our home, we almost always make sure that the memory finds it place in our social media feeds. This massive amount of data can be leveraged by cutting-edge Artificial Intelligence technology to gain insights about the personal as well as regional/global trends of food consumption habits as well as various correlations with other factors. This knowledge can be extremely useful to figure out optimum diet plans, financial plans to tackle the overspending habits on junk foods, and how different food items contribute to the overall obesity factors to maintain our peak biological performance level. 7 | 8 | Hence, the first and most important step of this process is to make the machines smarter in terms of visual perception and understanding/categorization capabilities for different & popular food items. This project attempts to make an algorithm so that a computer can automatically detect a certain food type from an image and create the framework for further analysis and knowledge discovery. 9 | 10 | * ### Challenges: 11 | 12 | During Deployment: 13 | 1) Resolved python, pip, and tensorflow version incompatibility by performing effective modifications in the requirements.txt file 14 | 15 | 2) Problems with slugsize of the App 16 | Solutions tried: a) usage of .slugignore to delete redundant files during runtime of deployment, b) purging build caches to reduce the slug size , c) using Heroku buildpack for python 17 | 18 | - *Final working solution*: As Heroku cloud comes with a 500MB limitation and the new TensorFlow package is 489.6MB. To avoid dependencies and storage issues, I explored and figured out the change in the requirements.txt file which helped to reduce the slugsize in my use case - *Adding tensorflow-cpu instead of tensorflow reduced the slug size from 667.7MB to 377MB.* 19 | 20 | 21 | ![deployment](https://github.com/SohamBera16/foodetecto-An-AI-based-Food-category-detection-app-using-Streamlit-and-Heroku-Cloud/blob/main/app%20deployed.png) 22 | 23 | * ### Technology Implementation: 24 | Artificial Intelligence (AI)/Deep Learning techniques - in particular, the Inception V3 model has been used for implementing transfer learning along with the Convolutional Neural network (CNN) tech in order to automate the process of food item image detection. Afterwards, the Heroku platform was utilized to deploy the app into production for the demo of automated image classification results. 25 | 26 | * ### Dataset used: 27 | The dataset used for this project has been downloaded from - https://www.kaggle.com/datasets/msarmi9/food101tiny 28 | Some of the example food items to classify are - apple pie, ice cream, falafel, etc. 29 | 30 | * ### Demonstration of the app: 31 | 32 | ![web app](https://github.com/SohamBera16/foodetecto-An-AI-based-Food-category-detection-app-using-Streamlit-and-Heroku-Cloud/blob/main/streamlit%20web%20app%20deployment%20screenshot.png) 33 | 34 | * ### Results: 35 | Accuracy of detection achieved on the training dataset is 84.25%. 36 | The validation accuracy achieved on the test dataset was 86.13% which is quite good given that the analysis was performed on a comparatively very small dataset. 37 | 38 | * ### Future Steps: 39 | - Deploying the app using Docker and Kubernetes 40 | - Increasing the dataset 41 | - Testing different deep learning algorithms for better performance 42 | - Trying out the effects of various data preprocessing techniques on the final results 43 | --------------------------------------------------------------------------------