├── README.md ├── SpotGarbage.apk ├── SpotGarbage_GarbNet ├── deploy_garbnet.prototxt ├── garbnet_fcn.caffemodel └── garbnet_mean.binaryproto ├── garbnet_demo.html ├── garbnet_demo.ipynb ├── garbnet_demo.py ├── input ├── sample_1_garbage.jpg ├── sample_2_garbage.jpg ├── sample_3_garbage.jpg ├── sample_4_nongarbage.jpg └── sample_5_nongarbage.jpg └── output ├── output_sample_1_garbage.jpg ├── output_sample_2_garbage.jpg ├── output_sample_3_garbage.jpg ├── output_sample_4_nongarbage.jpg └── output_sample_5_nongarbage.jpg /README.md: -------------------------------------------------------------------------------- 1 | SpotGarbage 2 | =========== 3 | Smartphone App to Detect Garbage Using Deep Learning 4 | ---------------------------------------------------- 5 | 6 | ###Description 7 | * The **SpotGarbage_GarbNet** folder contains the caffe model related files for **GarbNet**. 8 | * The code to test GarbNet is given in files named **garbnet_demo** in HTML, standard Python and Jupyter Ipython Notebook formats. 9 | * The code requires the following dependencies installed: 10 | * Caffe (CPU or GPU) 11 | * OpenCV 12 | * PIL 13 | * Numpy 14 | 15 | ###Instructions 16 | * Put the images on which garbage needs to be detected in the **input** folder. 17 | * Run the **garbnet_demo.py** or run the code snippets through the Ipython notebook. 18 | * The predictions will be made and the images with garbage regions segmented will be saved in the **output** folder. 19 | 20 | ###SpotGarbage Apk 21 | The SpotGarbage apk can be installed on any smartphone having Android OS 4.0.3 or above. It has been successfully tested on Nexus 5, Nexus 6, Moto G3, Moto X and Lenovo K3 Note with Android OS 6.0 (Marshmallow). 22 | 23 | -------------------------------------------------------------------------------- /SpotGarbage.apk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KudaP/SpotGarbage/b5697fffb041fe73b035a704d52e0f5cc8ff779f/SpotGarbage.apk -------------------------------------------------------------------------------- /SpotGarbage_GarbNet/deploy_garbnet.prototxt: -------------------------------------------------------------------------------- 1 | name: "GarbageNetFCN" 2 | input: "data" 3 | input_shape { 4 | dim: 1 5 | dim: 3 6 | dim: 227 7 | dim: 227 8 | } 9 | layer { 10 | name: "conv1" 11 | type: "Convolution" 12 | bottom: "data" 13 | top: "conv1" 14 | param { 15 | lr_mult: 1 16 | decay_mult: 1 17 | } 18 | param { 19 | lr_mult: 2 20 | decay_mult: 0 21 | } 22 | convolution_param { 23 | num_output: 96 24 | kernel_size: 11 25 | stride: 4 26 | weight_filler { 27 | type: "gaussian" 28 | std: 0.01 29 | } 30 | bias_filler { 31 | type: "constant" 32 | value: 0 33 | } 34 | } 35 | } 36 | layer { 37 | name: "relu1" 38 | type: "ReLU" 39 | bottom: "conv1" 40 | top: "conv1" 41 | } 42 | layer { 43 | name: "pool1" 44 | type: "Pooling" 45 | bottom: "conv1" 46 | top: "pool1" 47 | pooling_param { 48 | pool: MAX 49 | kernel_size: 3 50 | stride: 2 51 | } 52 | } 53 | layer { 54 | name: "conv2" 55 | type: "Convolution" 56 | bottom: "pool1" 57 | top: "conv2" 58 | param { 59 | lr_mult: 1 60 | decay_mult: 1 61 | } 62 | param { 63 | lr_mult: 2 64 | decay_mult: 0 65 | } 66 | convolution_param { 67 | num_output: 256 68 | pad: 2 69 | kernel_size: 5 70 | group: 2 71 | weight_filler { 72 | type: "gaussian" 73 | std: 0.01 74 | } 75 | bias_filler { 76 | type: "constant" 77 | value: 0.1 78 | } 79 | } 80 | } 81 | layer { 82 | name: "relu2" 83 | type: "ReLU" 84 | bottom: "conv2" 85 | top: "conv2" 86 | } 87 | layer { 88 | name: "pool2" 89 | type: "Pooling" 90 | bottom: "conv2" 91 | top: "pool2" 92 | pooling_param { 93 | pool: MAX 94 | kernel_size: 3 95 | stride: 2 96 | } 97 | } 98 | layer { 99 | name: "conv3" 100 | type: "Convolution" 101 | bottom: "pool2" 102 | top: "conv3" 103 | param { 104 | lr_mult: 1 105 | decay_mult: 1 106 | } 107 | param { 108 | lr_mult: 2 109 | decay_mult: 0 110 | } 111 | convolution_param { 112 | num_output: 384 113 | pad: 1 114 | kernel_size: 3 115 | weight_filler { 116 | type: "gaussian" 117 | std: 0.01 118 | } 119 | bias_filler { 120 | type: "constant" 121 | value: 0 122 | } 123 | } 124 | } 125 | layer { 126 | name: "relu3" 127 | type: "ReLU" 128 | bottom: "conv3" 129 | top: "conv3" 130 | } 131 | layer { 132 | name: "conv4" 133 | type: "Convolution" 134 | bottom: "conv3" 135 | top: "conv4" 136 | param { 137 | lr_mult: 1 138 | decay_mult: 1 139 | } 140 | param { 141 | lr_mult: 2 142 | decay_mult: 0 143 | } 144 | convolution_param { 145 | num_output: 384 146 | pad: 1 147 | kernel_size: 3 148 | group: 2 149 | weight_filler { 150 | type: "gaussian" 151 | std: 0.01 152 | } 153 | bias_filler { 154 | type: "constant" 155 | value: 0.1 156 | } 157 | } 158 | } 159 | layer { 160 | name: "relu4" 161 | type: "ReLU" 162 | bottom: "conv4" 163 | top: "conv4" 164 | } 165 | layer { 166 | name: "conv5" 167 | type: "Convolution" 168 | bottom: "conv4" 169 | top: "conv5" 170 | param { 171 | lr_mult: 1 172 | decay_mult: 1 173 | } 174 | param { 175 | lr_mult: 2 176 | decay_mult: 0 177 | } 178 | convolution_param { 179 | num_output: 256 180 | pad: 1 181 | kernel_size: 3 182 | group: 2 183 | weight_filler { 184 | type: "gaussian" 185 | std: 0.01 186 | } 187 | bias_filler { 188 | type: "constant" 189 | value: 0.1 190 | } 191 | } 192 | } 193 | layer { 194 | name: "relu5" 195 | type: "ReLU" 196 | bottom: "conv5" 197 | top: "conv5" 198 | } 199 | layer { 200 | name: "pool5" 201 | type: "Pooling" 202 | bottom: "conv5" 203 | top: "pool5" 204 | pooling_param { 205 | pool: MAX 206 | kernel_size: 3 207 | stride: 2 208 | } 209 | } 210 | layer { 211 | name: "fc6_gb_conv" 212 | type: "Convolution" 213 | bottom: "pool5" 214 | top: "fc6_gb_conv" 215 | param { 216 | lr_mult: 1 217 | decay_mult: 1 218 | } 219 | param { 220 | lr_mult: 2 221 | decay_mult: 0 222 | } 223 | convolution_param { 224 | num_output: 512 225 | kernel_size: 6 226 | stride: 2 227 | weight_filler { 228 | type: "gaussian" 229 | std: 0.005 230 | } 231 | bias_filler { 232 | type: "constant" 233 | value: 0.1 234 | } 235 | } 236 | } 237 | layer { 238 | name: "relu6" 239 | type: "ReLU" 240 | bottom: "fc6_gb_conv" 241 | top: "fc6_gb_conv" 242 | } 243 | layer { 244 | name: "drop6" 245 | type: "Dropout" 246 | bottom: "fc6_gb_conv" 247 | top: "fc6_gb_conv" 248 | dropout_param { 249 | dropout_ratio: 0.5 250 | } 251 | } 252 | layer { 253 | name: "fc7_gb_conv" 254 | type: "Convolution" 255 | bottom: "fc6_gb_conv" 256 | top: "fc7_gb_conv" 257 | param { 258 | lr_mult: 10 259 | decay_mult: 10 260 | } 261 | param { 262 | lr_mult: 20 263 | decay_mult: 0 264 | } 265 | convolution_param { 266 | num_output: 256 267 | kernel_size: 1 268 | weight_filler { 269 | type: "gaussian" 270 | std: 0.005 271 | } 272 | bias_filler { 273 | type: "constant" 274 | value: 0.1 275 | } 276 | } 277 | } 278 | layer { 279 | name: "relu7" 280 | type: "ReLU" 281 | bottom: "fc7_gb_conv" 282 | top: "fc7_gb_conv" 283 | } 284 | layer { 285 | name: "drop7" 286 | type: "Dropout" 287 | bottom: "fc7_gb_conv" 288 | top: "fc7_gb_conv" 289 | dropout_param { 290 | dropout_ratio: 0.5 291 | } 292 | } 293 | layer { 294 | name: "fc8_gb_conv" 295 | type: "Convolution" 296 | bottom: "fc7_gb_conv" 297 | top: "fc8_gb_conv" 298 | param { 299 | lr_mult: 10 300 | decay_mult: 10 301 | } 302 | param { 303 | lr_mult: 20 304 | decay_mult: 0 305 | } 306 | convolution_param { 307 | num_output: 2 308 | kernel_size: 1 309 | weight_filler { 310 | type: "gaussian" 311 | std: 0.01 312 | } 313 | bias_filler { 314 | type: "constant" 315 | value: 0 316 | } 317 | } 318 | } 319 | layer { 320 | name: "prob" 321 | type: "Softmax" 322 | bottom: "fc8_gb_conv" 323 | top: "prob" 324 | } 325 | -------------------------------------------------------------------------------- /SpotGarbage_GarbNet/garbnet_fcn.caffemodel: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KudaP/SpotGarbage/b5697fffb041fe73b035a704d52e0f5cc8ff779f/SpotGarbage_GarbNet/garbnet_fcn.caffemodel -------------------------------------------------------------------------------- /SpotGarbage_GarbNet/garbnet_mean.binaryproto: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KudaP/SpotGarbage/b5697fffb041fe73b035a704d52e0f5cc8ff779f/SpotGarbage_GarbNet/garbnet_mean.binaryproto -------------------------------------------------------------------------------- /garbnet_demo.py: -------------------------------------------------------------------------------- 1 | import caffe 2 | import os 3 | from pylab import * 4 | import sys 5 | from PIL import Image 6 | import numpy as np 7 | import cv2 8 | 9 | def gatherImages(folder,imageNames=None): 10 | images = [] 11 | names = [] 12 | files = os.listdir(folder) 13 | total = len(files) 14 | print 'Total %d images in folder %s' % (total,folder) 15 | for i in os.listdir(folder): 16 | try: 17 | if imageNames is None or i in imageNames: 18 | example_image = folder+'/'+i 19 | input_image = Image.open(example_image) 20 | images.append(input_image) 21 | names.append(i) 22 | except: 23 | pass 24 | 25 | return images,names 26 | 27 | def resizeForFCN(image,size): 28 | w,h = image.size 29 | if w thresh: 42 | x1 = h*62 #stride 2 at fc6_gb_conv equivalent to 62 pixels stride in input 43 | y1 = k*62 44 | for hoff in range(x1,227+x1): 45 | if hoff < out_bn.shape[0]: 46 | for koff in range(y1,227+y1): 47 | if koff < out_bn.shape[1]: 48 | out_bn[hoff,koff] = 255 49 | edge = cv2.Canny(out_bn,200,250) 50 | box = cv2.dilate(edge,kernel,iterations = 3) 51 | 52 | or_im_ar = np.array(test_image) 53 | or_im_ar[:,:,1] = (or_im_ar[:,:,1] | box) 54 | or_im_ar[:,:,2] = or_im_ar[:,:,2] * box + or_im_ar[:,:,2] 55 | or_im_ar[:,:,0] = or_im_ar[:,:,0] * box + or_im_ar[:,:,0] 56 | 57 | return Image.fromarray(or_im_ar) 58 | 59 | 60 | def getPredictionsFor(images,names,size,thresh,output_folder): 61 | for i in range(len(images)): 62 | try: 63 | test_image = resizeForFCN(images[i],size) 64 | 65 | in_ = np.array(test_image,dtype = np.float32) 66 | in_ = in_[:,:,::-1] 67 | in_ -= np.array(mean.mean(1).mean(1)) 68 | in_ = in_.transpose((2,0,1)) 69 | 70 | net.blobs['data'].reshape(1,*in_.shape) 71 | net.blobs['data'].data[...] = in_ 72 | net.forward() 73 | 74 | probMap =net.blobs['prob'].data[0,1] 75 | print names[i]+'...', 76 | if len(np.where(probMap>thresh)[0]) > 0: 77 | print 'Garbage!' 78 | else: 79 | print 'Not Garbage!' 80 | 81 | out_ = getSegmentedImage(test_image, probMap,thresh) 82 | out_.save(output_folder + '/output_' + names[i]) 83 | except: 84 | pass 85 | 86 | 87 | mean_filename='SpotGarbage_GarbNet/garbnet_mean.binaryproto' 88 | deploy_filename = 'SpotGarbage_GarbNet/deploy_garbnet.prototxt' 89 | caffemodel_file = 'SpotGarbage_GarbNet/garbnet_fcn.caffemodel' 90 | 91 | proto_data = open(mean_filename, "rb").read() 92 | a = caffe.io.caffe_pb2.BlobProto.FromString(proto_data) 93 | mean = caffe.io.blobproto_to_array(a)[0] 94 | 95 | net = caffe.Net(deploy_filename,caffemodel_file,caffe.TEST) 96 | 97 | #specify 'input' folder containing images for prediction 98 | images,names = gatherImages('input') 99 | #specify 'output' folder to store segmented predictions 100 | getPredictionsFor(images,names,4,0.999,'output') -------------------------------------------------------------------------------- /input/sample_1_garbage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KudaP/SpotGarbage/b5697fffb041fe73b035a704d52e0f5cc8ff779f/input/sample_1_garbage.jpg -------------------------------------------------------------------------------- /input/sample_2_garbage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KudaP/SpotGarbage/b5697fffb041fe73b035a704d52e0f5cc8ff779f/input/sample_2_garbage.jpg -------------------------------------------------------------------------------- /input/sample_3_garbage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KudaP/SpotGarbage/b5697fffb041fe73b035a704d52e0f5cc8ff779f/input/sample_3_garbage.jpg -------------------------------------------------------------------------------- /input/sample_4_nongarbage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KudaP/SpotGarbage/b5697fffb041fe73b035a704d52e0f5cc8ff779f/input/sample_4_nongarbage.jpg -------------------------------------------------------------------------------- /input/sample_5_nongarbage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KudaP/SpotGarbage/b5697fffb041fe73b035a704d52e0f5cc8ff779f/input/sample_5_nongarbage.jpg -------------------------------------------------------------------------------- /output/output_sample_1_garbage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KudaP/SpotGarbage/b5697fffb041fe73b035a704d52e0f5cc8ff779f/output/output_sample_1_garbage.jpg -------------------------------------------------------------------------------- /output/output_sample_2_garbage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KudaP/SpotGarbage/b5697fffb041fe73b035a704d52e0f5cc8ff779f/output/output_sample_2_garbage.jpg -------------------------------------------------------------------------------- /output/output_sample_3_garbage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KudaP/SpotGarbage/b5697fffb041fe73b035a704d52e0f5cc8ff779f/output/output_sample_3_garbage.jpg -------------------------------------------------------------------------------- /output/output_sample_4_nongarbage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KudaP/SpotGarbage/b5697fffb041fe73b035a704d52e0f5cc8ff779f/output/output_sample_4_nongarbage.jpg -------------------------------------------------------------------------------- /output/output_sample_5_nongarbage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KudaP/SpotGarbage/b5697fffb041fe73b035a704d52e0f5cc8ff779f/output/output_sample_5_nongarbage.jpg --------------------------------------------------------------------------------