├── LICENSE ├── README.md ├── deepLabV3plus.ipynb └── images ├── deeplabv3+.png ├── model_eval.png ├── result1.png ├── result2.png ├── result3.png ├── result4.png ├── result5.png ├── result6.png └── scores.png /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 Berkay Mayalı 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 |

2 |

Clothes Segmentation using DeepLabV3+

3 |

4 | Train the DeepLabV3+ model on Colab with Tensorflow API. 5 |
6 |

7 |

8 | 9 |

10 | tf 11 |
12 |

13 |

14 | 15 |

16 | Table of Contents 17 |

18 | 19 | > * [Introduction](#introduction) 20 | > * [Training Details](#training-details) 21 | > * [Jupyter Notebook](#jupyter-notebook) 22 | > * [Data](#data) 23 | > * [Test Results](#test-results) 24 | > * [References](#references) 25 | 26 | 27 | # Introduction 28 | 29 | Recently, the fashion industry has undergone a digital transformation by expanding on online platforms. Especially after the pandemic, online platforms have grown considerably commercially and as an area where data scientists can develop applications and provide benefits. This growth allows the development of compelling use cases of artificial intelligence in the fashion industry. For example, extracting clothing products and their related features from pictures can improve consumers' shopping experience and improve work efficiency for fashion workers. Therefore, the automatic labelling and visual analysis of clothing images in this area have attracted increasing attention. 30 | 31 |
32 | example-result 33 |
34 | 35 | 36 | # Training Details 37 | 38 |
39 | example-result 40 |
41 | 42 | 43 | The DeepLab architecture has proven to be quite successful. This study used a pre-trained inceptionresnetv2 backbone network to parse clothing images into five main categories. 13520 images from iMaterialists (Fashion) 2020 FGVC7 for training have been resized to 256x256. In the study, to reduce the complex structure of the data set, 46 different categories were converted into 5 main categories ( 'upper-body', 'lower-body', 'whole-body', 'feet' and 'accessories'). The Adam optimization method and categorical cross-entropy loss were used for model optimization. 44 | 45 | This project uses DeepLabV3 as a deep learning model. However, instead of having 1 channel output from DeepLabV3 for the typical noticeable semantic segmentation task, it outputs 5 channels of output, each representing 5 main categories. 46 | 47 | 48 | 49 | 50 | # Jupyter Notebook 51 | You can train the DeepLabV3+ model. Click the button below to open it on Colab. 52 | 53 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/mberkay0/clothing-segmentation/blob/main/deepLabV3plus.ipynb) 54 | 55 |
56 | example-result 57 |
58 | 59 | 60 | # Data 61 | 62 | **Dataset Taxonomy** 63 | ``` 64 | +---------+-----------------------------------------+----------------+ 65 | | ClassId | Name | SuperCategory | 66 | +---------+-----------------------------------------+----------------+ 67 | | 0 | shirt, blouse | upperbody | 68 | | 1 | top, t-shirt, sweatshirt | upperbody | 69 | | 2 | sweater | upperbody | 70 | | 3 | cardigan | upperbody | 71 | | 4 | jacket | upperbody | 72 | | 5 | vest | upperbody | 73 | | 6 | pants | lowerbody | 74 | | 7 | shorts | lowerbody | 75 | | 8 | skirt | lowerbody | 76 | | 9 | coat | wholebody | 77 | | 10 | dress | wholebody | 78 | | 11 | jumpsuit | wholebody | 79 | | 12 | cape | wholebody | 80 | | 13 | glasses | head | 81 | | 14 | hat | head | 82 | | 15 | headband, head covering, hair accessory | head | 83 | | 16 | tie | neck | 84 | | 17 | glove | arms and hands | 85 | | 18 | watch | arms and hands | 86 | | 19 | belt | waist | 87 | | 20 | leg warmer | legs and feet | 88 | | 21 | tights, stockings | legs and feet | 89 | | 22 | sock | legs and feet | 90 | | 23 | shoe | legs and feet | 91 | | 24 | bag, wallet | others | 92 | | 25 | scarf | others | 93 | | 26 | umbrella | others | 94 | | 27 | hood | garment parts | 95 | | 28 | collar | garment parts | 96 | | 29 | lapel | garment parts | 97 | | 30 | epaulette | garment parts | 98 | | 31 | sleeve | garment parts | 99 | | 32 | pocket | garment parts | 100 | | 33 | neckline | garment parts | 101 | | 34 | buckle | closures | 102 | | 35 | zipper | closures | 103 | | 36 | applique | decorations | 104 | | 37 | bead | decorations | 105 | | 38 | bow | decorations | 106 | | 39 | flower | decorations | 107 | | 40 | fringe | decorations | 108 | | 41 | ribbon | decorations | 109 | | 42 | rivet | decorations | 110 | | 43 | ruffle | decorations | 111 | | 44 | sequin | decorations | 112 | | 45 | tassel | decorations | 113 | +---------+-----------------------------------------+----------------+ 114 | ``` 115 | More detailed explanation [here](https://www.kaggle.com/c/imaterialist-fashion-2020-fgvc7/overview/evaluation) 116 | 117 | 118 | 119 | ## Test Results 120 | 121 | Here are some visual and numerical results from the iMaterialist test dataset. Reasonably good results have been achieved in this dataset, which is complex, and labels are inconsistent. 122 | 123 | 124 | | loss | Avg. IoU (%) | F1-Score (%) | 125 | |:---:|:---:|:---:| 126 | | 0.3414 | 54.69 | 61.92 | 127 | 128 | 129 | ### Visual Results 130 |
131 | example-result 132 | example-result 133 | example-result 134 | example-result 135 | example-result 136 |
137 | 138 | 139 | # References 140 | 141 | * [iMaterialist (Fashion) 2020 at FGVC7](https://kaggle.com/competitions/imaterialist-fashion-2020-fgvc7) 142 | * [Based tensorflow repo model](https://github.com/tensorflow/models/tree/master/research/deeplab) 143 | * [DeepLabV3+](https://arxiv.org/pdf/1802.02611.pdf) 144 | * [Tensorflow](https://www.tensorflow.org/) 145 | 146 | 147 | -------------------------------------------------------------------------------- /images/deeplabv3+.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mberkay0/clothing-segmentation/1fb0c6ec3cd9102d3be0f5e9a1db743b389f193f/images/deeplabv3+.png -------------------------------------------------------------------------------- /images/model_eval.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mberkay0/clothing-segmentation/1fb0c6ec3cd9102d3be0f5e9a1db743b389f193f/images/model_eval.png -------------------------------------------------------------------------------- /images/result1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mberkay0/clothing-segmentation/1fb0c6ec3cd9102d3be0f5e9a1db743b389f193f/images/result1.png -------------------------------------------------------------------------------- /images/result2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mberkay0/clothing-segmentation/1fb0c6ec3cd9102d3be0f5e9a1db743b389f193f/images/result2.png -------------------------------------------------------------------------------- /images/result3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mberkay0/clothing-segmentation/1fb0c6ec3cd9102d3be0f5e9a1db743b389f193f/images/result3.png -------------------------------------------------------------------------------- /images/result4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mberkay0/clothing-segmentation/1fb0c6ec3cd9102d3be0f5e9a1db743b389f193f/images/result4.png -------------------------------------------------------------------------------- /images/result5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mberkay0/clothing-segmentation/1fb0c6ec3cd9102d3be0f5e9a1db743b389f193f/images/result5.png -------------------------------------------------------------------------------- /images/result6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mberkay0/clothing-segmentation/1fb0c6ec3cd9102d3be0f5e9a1db743b389f193f/images/result6.png -------------------------------------------------------------------------------- /images/scores.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mberkay0/clothing-segmentation/1fb0c6ec3cd9102d3be0f5e9a1db743b389f193f/images/scores.png --------------------------------------------------------------------------------