├── Code └── .gitkeep ├── Images ├── .gitkeep ├── curetsd_real.JPG ├── sign_types.png ├── coordinate_system.png └── curetsd_challenges.png ├── LICENSE └── README.md /Code/.gitkeep: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /Images/.gitkeep: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /Images/curetsd_real.JPG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/olivesgatech/CURE-TSD/HEAD/Images/curetsd_real.JPG -------------------------------------------------------------------------------- /Images/sign_types.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/olivesgatech/CURE-TSD/HEAD/Images/sign_types.png -------------------------------------------------------------------------------- /Images/coordinate_system.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/olivesgatech/CURE-TSD/HEAD/Images/coordinate_system.png -------------------------------------------------------------------------------- /Images/curetsd_challenges.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/olivesgatech/CURE-TSD/HEAD/Images/curetsd_challenges.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 OLIVES @GATech 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # CURE-TSD 2 | CURE-TSD: Challenging Unreal and Real Environments for Traffic Sign Detection 3 | 4 | [OLIVES Lab, Georgia Institute of Technology](https://ghassanalregib.info/) 5 | 6 | 7 | 8 | | CURE-TSD Real 11 | | 12 | CURE-TSD Unreal 15 | | 16 | 17 | 18 | 19 | 20 | 21 | ### Publications 22 | If you use CURE-TSD dataset or codes, please cite the papers listed below: 23 | 24 | [Traffic Sign Detection Under Challenging Conditions: A Deeper Look into Performance Variations and Spectral Characteristics 25 | ](https://arxiv.org/abs/1908.11262) 26 | 27 | ``` 28 | @ARTICLE{temel2019traffic, 29 | author={D. Temel and M. Chen and G. AlRegib}, 30 | journal={IEEE Transactions on Intelligent Transportation Systems}, 31 | title={Traffic Sign Detection Under Challenging Conditions: A Deeper Look into Performance Variations and Spectral Characteristics}, 32 | year={2019}, 33 | volume={}, 34 | number={}, 35 | pages={1-11}, 36 | doi={10.1109/TITS.2019.2931429}, 37 | ISSN={1524-9050}, 38 | url={https://arxiv.org/abs/1908.11262}} 39 | ``` 40 | 41 | [Traffic Signs in the Wild: Highlights from the IEEE Video and Image Processing Cup 2017 Student Competition [SP Competitions] 42 | ](https://arxiv.org/abs/1810.06169) 43 | 44 | ``` 45 | @ARTICLE{Temel2018_SPM, 46 | author={D. Temel and G. AlRegib}, 47 | journal={IEEE Sig. Proc. Mag.}, 48 | title={Traffic Signs in the Wild: Highlights from the IEEE Video and Image Processing Cup 2017 Student 49 | Competition [SP Competitions]}, 50 | year={2018}, 51 | volume={35}, 52 | number={2}, 53 | pages={154-161}, 54 | doi={10.1109/MSP.2017.2783449}, 55 | ISSN={1053-5888}, 56 | url={https://arxiv.org/abs/1810.06169}} 57 | ``` 58 | 59 | 60 | [Challenging Environments for Traffic Sign Detection: Reliability Assessment under Inclement Conditions](https://arxiv.org/abs/1902.06857) 61 | 62 | ``` 63 | @article{temel2019challenging, 64 | title={Challenging environments for traffic sign detection: Reliability assessment under inclement conditions}, 65 | author={Temel, Dogancan and Alshawi, Tariq and Chen, Min-Hung and AlRegib, Ghassan}, 66 | journal={arXiv preprint arXiv:1902.06857}, 67 | year={2019}, 68 | url={https://arxiv.org/abs/1902.06857} 69 | } 70 | ``` 71 | 72 | 73 | [CURE-TSR: Challenging unreal and real environments for traffic sign recognition](https://arxiv.org/abs/1712.02463) 74 | 75 | ``` 76 | @INPROCEEDINGS{Temel2017_NIPSW, 77 | Author = {D. Temel and G. Kwon and M. Prabhushankar and G. AlRegib}, 78 | Title = {{CURE-TSR: Challenging unreal and real environments for traffic sign recognition}}, 79 | Year = {2017}, 80 | booktitle = {Neural Information Processing Systems (NeurIPS) Workshop on Machine Learning for Intelligent Transportation Systems}, 81 | 82 | ``` 83 | 84 | 85 | 86 | ### Download Dataset 87 | The video sequences in the CURE-TSD dataset are grouped into two classes: real data and unreal data. Real data correspond to processed versions of sequences acquired from real world. Unreal data corresponds to synthesized sequences generated in a virtual environment. There are 49 real sequences and 49 unreal sequences that do not include any specific challenge. We separated the sequences into 70% and %30 splits. Therefore, we have 34 training videos and 15 test videos in both real and unreal sequences that are challenge-free. There are 300 frames in each video sequence. There are 49 challenge-free real video sequences processed with 12 different types of effects and 5 different challenge levels, which result in 2,989 (49*12*5+49) video sequences. Moreover, there are 49 synthesized video sequences processed with 11 different types of effects and 5 different challenge levels, which leads to 2,744 (49*11*5+49) video sequences. In total, there are 5,733 video sequences, which include around 1.72 million frames. To receive  the download link, please fill out this FORM to submit your information and agree the conditions of use. These information will be kept confidential and will not be released to anybody outside the MSL administration team. 88 | 89 | ### Challenging Conditions 90 |

91 | 92 |

93 | 94 | ### Traffic Signs 95 |

96 | 97 |

98 | 99 | ### File Name Format 100 | “sequenceType_sequenceNumber_challengeSourceType_challengeType_challengeLevel.mp4” 101 | 102 | * sequenceType: 103 | 01 – Real data 104 | 02 – Unreal data 105 | 106 | * sequenceNumber: 107 | A number in between [01 – 49] 108 | 109 | * challengeSourceType: 110 | 00 – No challenge source (which means no challenge) 111 | 01 – After affect 112 | 113 | * challengeType: 114 | 00 – No challenge 115 | 01 – Decolorization 116 | 02 – Lens blur 117 | 03 – Codec error 118 | 04 – Darkening 119 | 05 – Dirty lens 120 | 06 – Exposure 121 | 07 – Gaussian blur 122 | 08 – Noise 123 | 09 – Rain 124 | 10 – Shadow 125 | 11 – Snow 126 | 12 – Haze 127 | 128 | * challengeLevel: 129 | A number in between [01-05] where 01 is the least severe and 05 is the most severe challenge. 130 | 131 | ### Test Sequences 132 | We split the video sequences into 70% training set and 30% test set. The sequence numbers corresponding to test set are given below: 133 | 134 | [01_04_x_x_x, 01_05_x_x_x, 01_06_x_x_x, 01_07_x_x_x, 01_08_x_x_x, 01_18_x_x_x, 01_19_x_x_x, 01_21_x_x_x, 01_24_x_x_x, 01_26_x_x_x, 01_31_x_x_x, 01_38_x_x_x, 01_39_x_x_x, 01_41_x_x_x, 01_47_x_x_x, 02_02_x_x_x, 02_04_x_x_x, 02_06_x_x_x, 02_09_x_x_x, 02_12_x_x_x, 02_13_x_x_x, 02_16_x_x_x, 02_17_x_x_x, 02_18_x_x_x, 02_20_x_x_x, 02_22_x_x_x, 02_28_x_x_x, 02_31_x_x_x, 02_32_x_x_x, 02_36_x_x_x] 135 | 136 | The videos with all other sequence numbers are in the training set. 137 | Note that “x” above refers to the variations listed earlier. 138 | 139 | ### Coordinate System 140 |

141 | 142 |

143 | 144 | ### Annotation Format 145 | “sequenceType_sequenceNumber.txt“. 146 | 147 | Challenge source type, challenge type, and challenge level do not affect the annotations. Therefore, the video sequences that start with the same sequence type and the sequence number have the same annotations. 148 | 149 | * sequenceType: 150 | 01 – Real data 151 | 02 – Unreal data 152 | 153 | * sequenceNumber: 154 | A number in between [01 – 49] 155 | 156 | The format of each line in the annotation file (txt) should be: “frameNumber_signType_llx_lly_lrx_lry_ulx_uly_urx_ury”. 157 | 158 | * frameNumber: 159 | A number in between [001-300] 160 | 161 | * signType: 162 | 01 – speed_limit 163 | 02 – goods_vehicles 164 | 03 – no_overtaking 165 | 04 – no_stopping 166 | 05 – no_parking 167 | 06 – stop 168 | 07 – bicycle 169 | 08 – hump 170 | 09 – no_left 171 | 10 – no_right 172 | 11 – priority_to 173 | 12 – no_entry 174 | 13 – yield 175 | 14 – parking 176 | 177 | ### Challenging Condition Generation 178 | Adobe (c) After Effects version 14.1.0.57 was utilized to emulate challenging conditions with the following configurations: 179 | 180 | * Decolorization: Black & White Color Correction filter version 1:0. The filter settings were: Reds= 40, Yellows= 60, Greens= 40, Cyans= 60, Blues= 20, and Magentas= 80. We utilized multiple adjustment layers to compound the effect of the color correction filter and created multiple distinct levels of this challenge. 181 | * Lens Blur: Camera Lens Blur filter version 1:0. The filter settings were: Blur radius was set to 2; 4; 6; 8; and 10 for levels 1 – 5 and Iris Shape was Hexagan, everything else was set as default. 182 | * Codec Error: Time Displacement filter version 1:6. The filter settings were: Max Displacement Time was set to 0:1; 0:2; 0:3; 0:4; and 0:5 for levels 1 – 5, everything else was set as default. 183 | * Darkening: Exposure filter version 1:0. The filter was set to modify the master channel Exposure parameter to be -1;-3;-5;-7; and -9 for levels 1 – 5, everything else was set as default. 184 | * Dirty Lens: a set of dusty and smudged lens images superimposed on the video. 185 | * Exposure: Exposure filter version 1:0. The filter was set to modify the master channel Exposure parameter to be 1, 3, 5, 7, and 9 for levels 1 – 5, everything else was set as default. 186 | * Gaussian Blur: Gaussian Blur filter version 3:0. The filter settings were: Bluriness was 5; 10; 15; 20; and 25 for levels 1 􀀀 5, everything else was set as default. Unlike Lens Blur, Gaussian Blur is equally distributed in all directions, which leads to less structured blurred objects. 187 | * Sensor Noise: Noise filter version 2:6. The filter settings were: Amount of Noise parameter was set to 20; 40; 60; 70; and 71 using 5 adjustment layers that were compounded to generate levels 1 – 5, everything else was set as default. 188 | * Rain: Rain was implemented using Gradient Ramp generator version 3:2 with colors # 0F1E2D and # 5A7492 to create a blueish hue over the video, and CC Rainfall generator from Cycore Effects HD 1.8.2 version 1:1. The setting of CC Rainfall generator were: Drops was set to 10000; 20000; 50000; and 100000 and Opacity was 25% using 5 adjustment layers that were compounded to generates levels 1 – 5, everything else was set as default. 189 | * Shadow: Venetian Blinds filter version 2:3. The filter settings were: Transition Completeness was 47%, Direction was 0x + 0:0°, Width was 142, and Opacity was 15%; 30%; 45%; 60%, and 75% for levels 1-5, everything 190 | else was set as default. 191 | * Snow: Glow filter version 2:6 with color # FFFFF to create a white hue over the video, and CC Snowfall generator from Cycore Effects HD 1.8.2 version 1:1. The setting of Glow filter were: Glow Threshold was 55%, Glow Intensity was 1:4, Glow Operation was Screen, and Glow Dimension was Horizontal. The setting of CC Snowfall generator were: Drops 10000; 50000; 100000, and 140000 using 9 adjustment layers that were compounded to generates levels 1-5, everything else was set as default. 192 | * Haze: Ellipse Shape Layer filter version 1:0 with radial gradient fill using color # D6D6D6 in the center with 100% opacity and color # 000000 with 0% opacity on the edges, Smart Blur version 1:0, Exposure version 1:0, and Brightness & contrast version 1:0. The shape and focal point location of the Ellipse was manually controlled to closely follow the furthest point in the video, which created a sense of depth to the scene and emulated the behaviour of haze in realistic settings. The setting of Smart Blur filter were: Radius was 3, and Threshold was 25. The setting of Exposure filter for the master channel were: Radius was -1, and Gamma Correction was 1. The setting of Brightness & Contrast filter were: Brightness was -34, and contrast was -13. Additionally, we utilized Solid Layer, which was created from color # CECECE and opacity 10%; 20%; 30%; 40%, and 50% to add difficulty to levels 1 – 5. 193 | 194 | # Related Research Studies 195 | The following papers used the CURE-TSD dataset in their research studies. If you utilize or refer to CURE-TSD dataset, please email cantemel@gatech.edu for your publication to be listed here. 196 | 197 | 200 | --------------------------------------------------------------------------------