├── Fashion-OSCD ├── fashionOSCDpipeline.png ├── README.md └── generate_image_pairs.py ├── Pascal-OSCD ├── PascalOSCDpipeline.png ├── README.md └── generate_image_pairs.py ├── LICENSE └── README.md /Fashion-OSCD/fashionOSCDpipeline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AICyberTeam/OSCD-datasets/HEAD/Fashion-OSCD/fashionOSCDpipeline.png -------------------------------------------------------------------------------- /Pascal-OSCD/PascalOSCDpipeline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AICyberTeam/OSCD-datasets/HEAD/Pascal-OSCD/PascalOSCDpipeline.png -------------------------------------------------------------------------------- /Pascal-OSCD/README.md: -------------------------------------------------------------------------------- 1 | ## Introduction 2 | Pascal-OSCD is a dataset for one-shot conditional object detection, which is based on the Pascal VOC 2007 and 2012 datasets. We divide all 20 classes with a ratio of 4:1 so that there are 16 seen classes for training/validation and 4 unseen classes for test. Support images are cropped from object bounding 3 | boxes and paired with query images that contain the same class objects randomly. Training image pairs are sampled from the train&val subsets of Pascal VOC 2007 and 2012 datasets, while the validation/test pairs are generated from the test subset of Pascal VOC 2007 dataset. 4 | 5 | The building process is shown below. 6 | 7 | ![PascalOSCDpipeline](./PascalOSCDpipeline.png) 8 | 9 | ## Building dataset 10 | 11 | Step 1. Download Pascal VOC 2007 and 2012 datasets from: http://host.robots.ox.ac.uk/pascal/VOC/ 12 | 13 | Step 2. Build this dataset by following the example given in generate_image_pairs.py. 14 | 15 | ## References 16 | [Pascal VOC](http://host.robots.ox.ac.uk/pascal/VOC/) 17 | 18 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 AICyberTeam 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Fashion-OSCD/README.md: -------------------------------------------------------------------------------- 1 | ## Introduction 2 | Fashion-OSCD is a dataset for one-shot conditional object detection, which is constructed by randomly scaling and embedding samples of Fashion-MNIST dataset into a larger image. 3 | 4 | We make each image contain as much as 3 objects with multiple scales and aspect ratios. 7 classes are randomly selected as seen classes for training/validation, the other 3 unseen classes for test. Overall, we generate 8000 training, 8000 validation and 6999 test images. 5 | 6 | The building process is shown below. 7 | 8 | ![fashionOSCDpipeline](./fashionOSCDpipeline.png) 9 | 10 | ## Download 11 | 12 | This dataset can be downloaded from: 13 | 14 | OneDrive: https://1drv.ms/u/s!AgYiu2IzGlVAiVYmyr0fN4PSUeYX?e=jKZgdm 15 | 16 | BaidiWangpan: https://pan.baidu.com/s/1Q1l9o4UFRYlAHvN-c-h-7Q password: 0k3t 17 | 18 | The Fashion-MNIST dataset can be downloaded from https://github.com/zalandoresearch/fashion-mnist#get-the-data 19 | 20 | ## Building dataset 21 | 22 | Generate image pairs by following the example given in generate_image_pairs.py. 23 | 24 | ## References 25 | [fashion-mnist](https://github.com/zalandoresearch/fashion-mnist) 26 | 27 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # OSCD-datasets 2 | We build two benchmarks based on fashion-mnist and Pascal VOC to evaluate one-shot conditional object detection and provide convenience for the community. 3 | 4 | # 1. Fashion-OSCD 5 | 6 | ## 1.1 Introduction 7 | 8 | Fashion-OSCD is a dataset for one-shot conditional object detection, which is constructed by randomly scaling and embedding samples of Fashion-MNIST dataset into a larger image. 9 | 10 | We make each image contain as much as 3 objects with multiple scales and aspect ratios. 7 classes are randomly selected as seen classes for training/validation, the other 3 unseen classes for test. Overall, we generate 8000 training, 8000 validation and 6999 test images. 11 | 12 | The building process is shown below. 13 | 14 | ![fashionOSCDpipeline](./Fashion-OSCD/fashionOSCDpipeline.png) 15 | 16 | ## 1.2 Download 17 | 18 | This dataset can be downloaded from: 19 | 20 | OneDrive: https://1drv.ms/u/s!AgYiu2IzGlVAiVYmyr0fN4PSUeYX?e=jKZgdm 21 | 22 | BaiduNetdisk: https://pan.baidu.com/s/1Q1l9o4UFRYlAHvN-c-h-7Q password: 0k3t 23 | 24 | The Fashion-MNIST dataset can be downloaded from https://github.com/zalandoresearch/fashion-mnist#get-the-data 25 | 26 | ## 1.3 Building dataset 27 | 28 | Generate image pairs by following the example given in ./Fashion-OSCD/generate_image_pairs.py. 29 | 30 | ## 1.4 References 31 | [fashion-mnist](https://github.com/zalandoresearch/fashion-mnist) 32 | 33 | # 2. Pascal-OSCD 34 | 35 | ## 2.1 Introduction 36 | 37 | Pascal-OSCD is a dataset for one-shot conditional object detection, which is based on the Pascal VOC 2007 and 2012 datasets. We divide all 20 classes with a ratio of 4:1 so that there are 16 seen classes for training/validation and 4 unseen classes for test. Support images are cropped from object bounding 38 | boxes and paired with query images that contain the same class objects randomly. Training image pairs are sampled from the train&val subsets of Pascal VOC 2007 and 2012 datasets, while the validation/test pairs are generated from the test subset of Pascal VOC 2007 dataset. 39 | 40 | The building process is shown below. 41 | 42 | ![PascalOSCDpipeline](./Pascal-OSCD/PascalOSCDpipeline.png) 43 | 44 | ## 2.2 Building dataset 45 | 46 | Step 1. Download Pascal VOC 2007 and 2012 datasets from: http://host.robots.ox.ac.uk/pascal/VOC/ 47 | 48 | Step 2. Build this dataset by following the example given in ./Pascal-OSCD/generate_image_pairs.py. 49 | 50 | ## 2.3 References 51 | 52 | [Pascal VOC](http://host.robots.ox.ac.uk/pascal/VOC/) 53 | 54 | ## Citation 55 | If you find **Fashion-OSCD** and **Pascal-OSCD** useful in your research, please consider citing: 56 | 57 | ``` 58 | @article{FU2020, 59 | title = "OSCD: A one-shot conditional object detection framework", 60 | journal = "Neurocomputing", 61 | year = "2020", 62 | issn = "0925-2312", 63 | doi = "https://doi.org/10.1016/j.neucom.2020.04.092", 64 | url = "http://www.sciencedirect.com/science/article/pii/S0925231220306779", 65 | author = "Kun Fu and Tengfei Zhang and Yue Zhang and Xian Sun", 66 | keywords = "One-shot, Object detection", 67 | } 68 | 69 | @ARTICLE{8735792, 70 | author={K. {Fu} and T. {Zhang} and Y. {Zhang} and M. {Yan} and Z. {Chang} and Z. {Zhang} and X. {Sun}}, 71 | journal={IEEE Access}, 72 | title={Meta-SSD: Towards Fast Adaptation for Few-Shot Object Detection With Meta-Learning}, 73 | year={2019}, 74 | volume={7}, 75 | number={}, 76 | pages={77597-77606}, 77 | } 78 | ``` 79 | -------------------------------------------------------------------------------- /Fashion-OSCD/generate_image_pairs.py: -------------------------------------------------------------------------------- 1 | # encoding:utf-8 2 | # Generate training/validation/test image pairs by randomly pairing. 3 | import os, sys 4 | import cv2 5 | import copy 6 | import argparse 7 | import xml.etree.ElementTree as ET 8 | import pickle 9 | import random 10 | 11 | 12 | def get_file(path): 13 | allfile = [] 14 | for dirpath, dirnames, filenames in os.walk(path): 15 | for dir in dirnames: 16 | allfile.append(os.path.join(dirpath, dir)) 17 | for name in filenames: 18 | if name.find('.xml') == -1: 19 | continue 20 | allfile.append(os.path.join(dirpath, name)) 21 | return allfile 22 | 23 | 24 | def crop_img(img, boxes): 25 | img_crops = [] 26 | for box in boxes: 27 | x1, y1, x2, y2 = box[0] 28 | class_name = box[1] 29 | img_crop = img[y1:y2, x1:x2, :] 30 | img_crops.append([img_crop, class_name]) 31 | return img_crops 32 | 33 | 34 | def get_img_path(xml_path): 35 | xml_path_cp = xml_path.replace('Annotations', 'JPEGImages') 36 | return xml_path_cp.replace('.xml', '.jpg') 37 | 38 | 39 | def save_all_crops(img_crops, file_name, save_dir): 40 | base_name = os.path.basename(file_name) 41 | for idx, img_crop in enumerate(img_crops): 42 | rst_name = os.path.join(save_dir, base_name[:base_name.find('.jpg')] + '_' + str(idx) + base_name[ 43 | base_name.find( 44 | '.jpg'):]) 45 | cv2.imwrite(rst_name, img_crop[0]) 46 | 47 | 48 | def get_all_crop(xml_path): 49 | rtn_boxes = [] 50 | et = ET.parse(xml_path) 51 | element = et.getroot() 52 | 53 | element_objs = element.findall('object') 54 | for obj in element_objs: 55 | try: 56 | class_name = obj.find('name').text 57 | x1 = int(obj.find('bndbox').find('xmin').text) 58 | y1 = int(obj.find('bndbox').find('ymin').text) 59 | x2 = int(obj.find('bndbox').find('xmax').text) 60 | y2 = int(obj.find('bndbox').find('ymax').text) 61 | rtn_boxes.append([[x1, y1, x2, y2], class_name]) 62 | except Exception as e: 63 | continue 64 | return rtn_boxes 65 | 66 | 67 | def get_target_name(xml_path, all_target_name): 68 | et = ET.parse(xml_path) 69 | element = et.getroot() 70 | element_objs = element.findall('object') 71 | 72 | for obj in element_objs: 73 | class_name = obj.find('name').text 74 | if class_name not in all_target_name: 75 | all_target_name.append(class_name) 76 | if len(all_target_name) == 10: 77 | break 78 | return all_target_name 79 | 80 | 81 | # This function is to generate training/validation/test image pairs by randomly pairing. 82 | # Here, we just give an example of generating the training image pairs. 83 | def gen_siamese_data(support_img_class_path='./support_img_class.pickle', query_img_class_path='./query_img_class.pickle'): 84 | with open(support_img_class_path, 'rb') as f: 85 | support_img_class = pickle.load(f) 86 | with open(query_img_class_path, 'rb') as f: 87 | query_img_class = pickle.load(f) 88 | # 7 classes are randomly selected as seen classes for training/validation, the other 3 unseen classes for test. 89 | train_class_names = ['tshirt', 'sneaker', 'sandal', 'bag', 'coat', 'shirt', 'trouser'] 90 | test_class_names = ['ankle_boot', 'dress', 'pullover'] 91 | 92 | # the number of the selected image pairs for each class 93 | # for example, the number of training image pairs: 6000* 7 = 42,000 94 | num_per_class = 6000 95 | train_pairs = [] 96 | for index, class_name in enumerate(train_class_names): 97 | print('Process class {} : {}'.format(index, class_name)) 98 | n = num_per_class 99 | t = query_img_class[class_name] 100 | q = support_img_class[class_name] 101 | if len(t) > 6000: # to reduce the computation burden 102 | random.shuffle(t) 103 | t = t[:6000] 104 | if len(q) > 6000: 105 | random.shuffle(q) 106 | q = q[:6000] 107 | combs = list(product(t, q)) 108 | random.shuffle(combs) 109 | 110 | for pair in combs[:n]: 111 | train_pairs.append(pair) 112 | 113 | print('Generate genuine Done') 114 | print('Save genuine Data') 115 | 116 | with open('./train_pairs.pickle', 'wb') as f: 117 | pickle.dump(train_pairs, f, protocol=2) 118 | 119 | 120 | if __name__ == '__main__': 121 | print('Start job...') 122 | xml_path = 'path to Fashion OSCD dataset/train/xmls' 123 | save_dir = './SupportImages' # save the cropped support images 124 | 125 | xmls_path = get_file(xml_path) 126 | 127 | if not os.path.exists(save_dir): 128 | os.makedirs(save_dir) 129 | 130 | # get image filenames { class name：[image filenames...] } 131 | query_img_class = {} # query images 132 | support_img_class = {} # support images 133 | all_target_name = [] 134 | 135 | # get all class names in this dataset 136 | for xml_path in xmls_path: 137 | all_target_name = get_target_name(xml_path, all_target_name) 138 | 139 | for itarget_name in all_target_name: 140 | target_dir = os.path.join(save_dir, itarget_name) 141 | # make dir for each class to save the support images 142 | if not os.path.exists(target_dir): 143 | os.mkdir(target_dir) 144 | query_img_class[itarget_name] = [] 145 | support_img_class[itarget_name] = [] 146 | 147 | len_xmls = len(xmls_path) 148 | train_data = [] 149 | # crop the support objects from the query images according to the annotated bounding boxes, 150 | # then save them into the corresponding dir 151 | for index, xml_path in enumerate(xmls_path): 152 | boxes = get_all_crop(xml_path) 153 | img_path = get_img_path(xml_path) 154 | img = cv2.imread(img_path) 155 | img_crops = crop_img(img, boxes) 156 | 157 | train_data.append([img_path, boxes]) 158 | 159 | save_all_crops(img_crops, img_path, save_dir) 160 | # support image name: 'class name' + '_' + 'num.png' 161 | base_name = os.path.basename(img_path) 162 | for idx, img_crop in enumerate(img_crops): 163 | class_name = img_crop[1] 164 | support_img_path = os.path.join(save_dir, class_name, base_name[:base_name.find('.jpg')] + 165 | '_' + str(idx) + base_name[base_name.find('.jpg'):]) 166 | cv2.imwrite(support_img_path, img_crop[0]) 167 | 168 | # add the information of the support image to support_image_class 169 | if [support_img_path, class_name] not in support_img_class[class_name]: 170 | support_img_class[class_name].append([support_img_path, class_name]) 171 | # # add the information of the query image to query_image_class 172 | if [img_path, boxes] not in query_img_class[class_name]: 173 | query_img_class[class_name].append([img_path, boxes]) 174 | 175 | print('Process {}, percent {}.2f'.format(index, index / float(len_xmls) * 100)) 176 | 177 | print('Finished all images, save result...') 178 | # save support_img_class、query_img_class into disk 179 | support_img_class_path = './support_img_class.pickle' 180 | query_img_class_path = './query_img_class.pickle' 181 | with open(support_img_class_path, 'wb') as f: 182 | pickle.dump(support_img_class, f, protocol=2) 183 | with open(query_img_class_path, 'wb') as f: 184 | pickle.dump(query_img_class, f, protocol=2) 185 | # generate training/validation/test image pairs by randomly pairing. 186 | gen_siamese_data(support_img_class_path, query_img_class_path) -------------------------------------------------------------------------------- /Pascal-OSCD/generate_image_pairs.py: -------------------------------------------------------------------------------- 1 | # encoding:utf-8 2 | # 1. Get the information of all query images in Pascal VOC dataset and save it to the dict of query_img_class. 3 | # 2. Get the information of all support images in Pascal VOC dataset and save it to the dict of support_img_class, 4 | # crop all support objects from the query images according to the annotated bounding boxes then save them into disk. 5 | # 3. Generate training/validation/test image pairs by randomly pairing. 6 | import os, sys 7 | import cv2 8 | import copy 9 | import argparse 10 | import xml.etree.ElementTree as ET 11 | import pickle 12 | import random 13 | 14 | 15 | def get_file(path): 16 | allfile = [] 17 | for dirpath, dirnames, filenames in os.walk(path): 18 | for dir in dirnames: 19 | allfile.append(os.path.join(dirpath, dir)) 20 | for name in filenames: 21 | if name.find('.xml') == -1: 22 | continue 23 | allfile.append(os.path.join(dirpath, name)) 24 | return allfile 25 | 26 | 27 | def crop_img(img, boxes): 28 | img_crops = [] 29 | for box in boxes: 30 | x1, y1, x2, y2 = box[0] 31 | class_name = box[1] 32 | img_crop = img[y1:y2, x1:x2, :] 33 | img_crops.append([img_crop, class_name]) 34 | return img_crops 35 | 36 | 37 | def get_img_path(xml_path): 38 | xml_path_cp = xml_path.replace('Annotations', 'JPEGImages') 39 | return xml_path_cp.replace('.xml', '.jpg') 40 | 41 | 42 | def save_all_crops(img_crops, file_name, save_dir): 43 | base_name = os.path.basename(file_name) 44 | for idx, img_crop in enumerate(img_crops): 45 | rst_name = os.path.join(save_dir, base_name[:base_name.find('.jpg')] + '_' + str(idx) + base_name[ 46 | base_name.find( 47 | '.jpg'):]) 48 | cv2.imwrite(rst_name, img_crop[0]) 49 | 50 | 51 | def get_all_crop(xml_path): 52 | rtn_boxes = [] 53 | et = ET.parse(xml_path) 54 | element = et.getroot() 55 | 56 | element_objs = element.findall('object') 57 | for obj in element_objs: 58 | try: 59 | class_name = obj.find('name').text 60 | x1 = int(obj.find('bndbox').find('xmin').text) 61 | y1 = int(obj.find('bndbox').find('ymin').text) 62 | x2 = int(obj.find('bndbox').find('xmax').text) 63 | y2 = int(obj.find('bndbox').find('ymax').text) 64 | rtn_boxes.append([[x1, y1, x2, y2], class_name]) 65 | except Exception as e: 66 | continue 67 | return rtn_boxes 68 | 69 | 70 | def get_target_name(xml_path, all_target_name): 71 | et = ET.parse(xml_path) 72 | element = et.getroot() 73 | element_objs = element.findall('object') 74 | 75 | for obj in element_objs: 76 | class_name = obj.find('name').text 77 | if class_name not in all_target_name: 78 | all_target_name.append(class_name) 79 | if len(all_target_name) == 20: 80 | break 81 | return all_target_name 82 | 83 | 84 | # This function is to generate training/validation/test image pairs by randomly pairing. 85 | # Here, we just give an example of generating the training image pairs. 86 | def gen_siamese_data(support_img_class_path='./support_img_class.pickle', query_img_class_path='./query_img_class.pickle'): 87 | with open(support_img_class_path, 'rb') as f: 88 | support_img_class = pickle.load(f) 89 | with open(query_img_class_path, 'rb') as f: 90 | query_img_class = pickle.load(f) 91 | # We divide all 20 classes with a ratio of 4:1 so that there are 16 seen classes for training/validation, 92 | # and 4 unseen classes for test. 93 | train_class_names = ['pottedplant', 'sofa', 'tvmonitor', 'car', 'bottle', 'boat', 'chair', 'person', 'bus', 'train', 'horse', 94 | 'bicycle', 'dog', 'bird', 'motorbike', 'diningtable'] 95 | test_class_names = ['cow', 'sheep', 'cat', 'aeroplane'] 96 | 97 | # the number of the selected image pairs for each class 98 | # for example, the number of training image pairs: 6250 * 16 = 100,000 99 | num_per_class = 6250 100 | train_pairs = [] 101 | for index, class_name in enumerate(train_class_names): 102 | print('Process class {} : {}'.format(index, class_name)) 103 | n = num_per_class 104 | t = query_img_class[class_name] 105 | q = support_img_class[class_name] 106 | if len(t) > 6000: # to reduce the computation burden 107 | random.shuffle(t) 108 | t = t[:6000] 109 | if len(q) > 6000: 110 | random.shuffle(q) 111 | q = q[:6000] 112 | combs = list(product(t, q)) 113 | random.shuffle(combs) 114 | 115 | for pair in combs[:n]: 116 | train_pairs.append(pair) 117 | 118 | print('Generate genuine Done') 119 | print('Save genuine Data') 120 | 121 | with open('./train_pairs.pickle', 'wb') as f: 122 | pickle.dump(train_pairs, f, protocol=2) 123 | 124 | 125 | if __name__ == '__main__': 126 | print('Start job...') 127 | xml_path = 'path to VOC dataset/VOCdevkit/VOC2012/Annotations' 128 | save_dir = './SupportImages' # save the cropped support images 129 | 130 | xmls_path = get_file(xml_path) 131 | 132 | if not os.path.exists(save_dir): 133 | os.makedirs(save_dir) 134 | 135 | # get image filenames { class name：[image filenames...] } 136 | query_img_class = {} # query images 137 | support_img_class = {} # support images 138 | all_target_name = [] 139 | 140 | # get all class names in this dataset 141 | for xml_path in xmls_path: 142 | all_target_name = get_target_name(xml_path, all_target_name) 143 | 144 | for itarget_name in all_target_name: 145 | target_dir = os.path.join(save_dir, itarget_name) 146 | # make dir for each class to save the support images 147 | if not os.path.exists(target_dir): 148 | os.mkdir(target_dir) 149 | query_img_class[itarget_name] = [] 150 | support_img_class[itarget_name] = [] 151 | 152 | len_xmls = len(xmls_path) 153 | train_data = [] 154 | # crop the support objects from the query images according to the annotated bounding boxes, 155 | # then save them into the corresponding dir 156 | for index, xml_path in enumerate(xmls_path): 157 | boxes = get_all_crop(xml_path) 158 | img_path = get_img_path(xml_path) 159 | img = cv2.imread(img_path) 160 | img_crops = crop_img(img, boxes) 161 | 162 | train_data.append([img_path, boxes]) 163 | 164 | save_all_crops(img_crops, img_path, save_dir) 165 | # support image name: 'class name' + '_' + 'num.png' 166 | base_name = os.path.basename(img_path) 167 | for idx, img_crop in enumerate(img_crops): 168 | class_name = img_crop[1] 169 | support_img_path = os.path.join(save_dir, class_name, base_name[:base_name.find('.jpg')] + 170 | '_' + str(idx) + base_name[base_name.find('.jpg'):]) 171 | cv2.imwrite(support_img_path, img_crop[0]) 172 | 173 | # add the information of the support image to support_image_class 174 | if [support_img_path, class_name] not in support_img_class[class_name]: 175 | support_img_class[class_name].append([support_img_path, class_name]) 176 | # # add the information of the query image to query_image_class 177 | if [img_path, boxes] not in query_img_class[class_name]: 178 | query_img_class[class_name].append([img_path, boxes]) 179 | 180 | print('Process {}, percent {}.2f'.format(index, index / float(len_xmls) * 100)) 181 | 182 | print('Finished all images, save result...') 183 | # save support_img_class、query_img_class into disk 184 | support_img_class_path = './support_img_class.pickle' 185 | query_img_class_path = './query_img_class.pickle' 186 | with open(support_img_class_path, 'wb') as f: 187 | pickle.dump(support_img_class, f, protocol=2) 188 | with open(query_img_class_path, 'wb') as f: 189 | pickle.dump(query_img_class, f, protocol=2) 190 | # generate training/validation/test image pairs by randomly pairing. 191 | gen_siamese_data(support_img_class_path, query_img_class_path) --------------------------------------------------------------------------------