├── .gitignore
├── README.md
├── challenge_iccv_2025
├── Dockerfile.jetson
├── Dockerfile.train
├── README.md
├── data
│ ├── Fisheye1k.yaml
│ └── Fisheye8k.yaml
├── models
│ └── yolo11n_fisheye8k.pt
├── pycocotools
│ ├── __init__.py
│ ├── _mask.cpython-310-aarch64-linux-gnu.so
│ ├── _mask.cpython-311-x86_64-linux-gnu.so
│ ├── coco.py
│ ├── cocoeval.py
│ ├── cocoeval_modified.py
│ └── mask.py
├── run_evaluation_jetson.py
├── run_train_yolo.py
├── utils.py
└── yolo11n.pt
├── evaluation_Jetson
├── eval.py
├── eval_f1.py
└── pycocotools
│ ├── __init__.py
│ ├── _mask.c
│ ├── _mask.cpython-310-aarch64-linux-gnu.so
│ ├── _mask.pyx
│ ├── coco.py
│ ├── cocoeval.py
│ ├── cocoeval_modified.py
│ └── mask.py
├── evaluation_Linux
├── EVALUATION.md
├── README.md
├── eval.py
├── eval_f1.py
└── pycocotools
│ ├── __init__.py
│ ├── __pycache__
│ ├── __init__.cpython-38.pyc
│ ├── coco.cpython-38.pyc
│ ├── cocoeval.cpython-38.pyc
│ └── mask.cpython-38.pyc
│ ├── _mask.cpython-38-x86_64-linux-gnu.so
│ ├── coco.py
│ ├── cocoeval.py
│ ├── cocoeval_modified.py
│ └── mask.py
└── evaluation_Windows
├── EVALUATION.md
├── README.md
├── eval.py
├── eval_f1.py
└── pycocotools
├── __init__.py
├── __pycache__
├── __init__.cpython-310.pyc
├── __init__.cpython-39.pyc
├── coco.cpython-310.pyc
├── coco.cpython-39.pyc
├── cocoeval.cpython-39.pyc
├── cocoeval_modified.cpython-39.pyc
└── mask.cpython-39.pyc
├── _mask.cp39-win_amd64.pyd
├── coco.py
├── cocoeval.py
├── cocoeval_modified.py
└── mask.py
/.gitignore:
--------------------------------------------------------------------------------
1 | #challenge_iccv_2025/models/
2 | data/
3 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # FishEye8K: A Benchmark and Dataset for Fisheye Camera Object Detection
2 |
3 | **Authors:**
4 | [Munkhjargal Gochoo](https://github.com/moyog/), [Munkh-Erdene Otgonbold](https://github.com/omunkhuush/), [Erkhembayar Ganbold](https://github.com/Erkheme98), Ming-Ching Chang, [Ping-Yang Chen](https://github.com/pingyang1117), Byambaa Dorj, Hamad Al Jassmi, [Ganzorig Batnasan](https://github.com/ganzobtn/), Fady Alnajjar, Mohammed Abduljabbar, Fang-Pang Lin, Jun-Wei Hsieh
5 |
6 |
7 | 
8 |
9 |
10 | We provide detailed information on the new FishEye8K road object detection dataset. The dataset consists of 8,000 annotated images with 157K bounding boxes of five object classes. The figure displays sample images from each of the 18 cameras with wide-angle fisheye views. These cameras offer new possibilities for extensive coverage.
11 |
12 | **Results of YOLOv7-E6E model on the input size 1280x1280.**
13 |
14 | https://github.com/MoyoG/FishEye8K/assets/73123564/f7dd27db-7613-4e2c-b5f0-ec33c44deba4
15 |
16 |
17 |
18 | **Classes (Color)**
19 | -  - Bus
20 | -  - Bike
21 | -  - Car
22 | -  - Pedestrian
23 | -  - Truck
24 |
25 |
26 | # Dataset
27 | - [Click to download the Fisheye8K dataset](https://scidm.nchc.org.tw/en/dataset/fisheye8k)
28 | - To download the whole dataset, choose **Fisheye8K_all_including_train&test_update_2024Jan Update.zip**
29 | - Click on "**Explore**" and "**Go to resource**"
30 |
31 | 
32 |
33 | - Note: If you downloaded the older version **Fisheye8K_all_including_train&test_1.zip** before Jan 28, **MS COCO (train.json and test.json) labels should be replaced** with these updated versions [Click here to download correct MS COCO labels](https://drive.google.com/drive/folders/1J4nYmTwXoDc47_2tq4aSCXBkkrCxTCP3?usp=sharing)
34 |
35 | # Paper
36 | - [Click to download the paper](https://openaccess.thecvf.com/content/CVPR2023W/AICity/html/Gochoo_FishEye8K_A_Benchmark_and_Dataset_for_Fisheye_Camera_Object_Detection_CVPRW_2023_paper.html?fbclid=IwAR2UCUtrPydcCFxRjN67d7RyAyX-dNJAEi7mWPJjWhX3kTUA_SQ4AJyaFWc)
37 |
38 | # Train and Test
39 |
40 |
41 | 
42 |
43 |
44 |
45 | # Experimental results
46 |
47 | We evaluated using default yolo test.py
48 |
49 | - conf_thres=0.5
50 | - iou_thres=0.5
51 |
52 | | Model | Version | Input Size | Precision | Recall | mAP0.5 | mAP.5-.95 | f1\-score | APS | APM | APL | Inference[ms] |
53 | | ---------- | --------- | ---------- | --------- | ------ | ------ | --------- | --------- | ------ | ------ | ------ | ------------- |
54 | | YOLOv5 | YOLOv5l6 | 1280 | 0.7929 | 0.4076 | 0.6139 | 0.4098 | 0.535 | 0.1299 | 0.434 | 0.6665 | 22.7 |
55 | | | YOLOv5x6 | 1280 | 0.8224 | 0.4313 | 0.6387 | 0.4268 | 0.5588 | 0.133 | 0.452 | 0.6925 | 43.9 |
56 | | YOLOR | YOLOR-W6 | 1280 | 0.7871 | 0.4718 | 0.6466 | 0.4442 | 0.5899 | 0.1325 | 0.4707 | 0.6901 | 16.4 |
57 | | | YOLOR-P6 | 1280 | 0.8019 | 0.4937 | 0.6632 | 0.4406 | 0.6111 | 0.1419 | 0.4805 | 0.7216 | 13.4 |
58 | | YOLOv7 | YOLOv7-D6 | 1280 | 0.7803 | 0.4111 | 0.3977 | 0.2633 | 0.5197 | 0.1261 | 0.4462 | 0.6777 | 26.4 |
59 | | | YOLOv7-E6E| 1280 | 0.8005 | 0.5252 | 0.5081 | 0.3265 | 0.6294 | 0.1684 | 0.5019 | 0.6927 | 29.8 |
60 | | YOLOv7 | YOLOv7 | 640 | 0.7917 | 0.4373 | 0.4235 | 0.2473 | 0.5453 | 0.1108 | 0.4438 | 0.6804 | 4.3 |
61 | | | YOLOv7-X | 640 | 0.7402 | 0.4888 | 0.4674 | 0.2919 | 0.5794 | 0.1332 | 0.4605 | 0.7212 | 6.7 |
62 | | YOLOv8 | YOLOv8l | 640 | 0.7835 | 0.3877 | 0.612 | 0.4012 | 0.5187 | 0.1038 | 0.4043 | 0.6577 | 8.5 |
63 | | | YOLOv8x | 640 | 0.8418 | 0.3665 | 0.6146 | 0.4029 | 0.5106 | 0.0997 | 0.4147 | 0.7083 | 13.4 |
64 |
65 | # Citation
66 | ```
67 | @InProceedings{Gochoo_2023_CVPR,
68 | author = {Gochoo, Munkhjargal and Otgonbold, Munkh-Erdene and Ganbold, Erkhembayar and Hsieh, Jun-Wei and Chang, Ming-Ching and Chen, Ping-Yang and Dorj, Byambaa and Al Jassmi, Hamad and Batnasan, Ganzorig and Alnajjar, Fady and Abduljabbar, Mohammed and Lin, Fang-Pang},
69 | title = {FishEye8K: A Benchmark and Dataset for Fisheye Camera Object Detection},
70 | booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
71 | month = {June},
72 | year = {2023},
73 | pages = {5304-5312}
74 | }
75 |
76 |
--------------------------------------------------------------------------------
/challenge_iccv_2025/Dockerfile.jetson:
--------------------------------------------------------------------------------
1 | FROM ultralytics/ultralytics:latest-jetson-jetpack6
2 |
3 | WORKDIR /app
4 | COPY . .
5 |
6 | ENTRYPOINT ["python", "run_evaluation_jetson.py"]
7 |
--------------------------------------------------------------------------------
/challenge_iccv_2025/Dockerfile.train:
--------------------------------------------------------------------------------
1 | FROM ultralytics/ultralytics:latest
2 |
3 | WORKDIR /app
4 | COPY . .
5 | # COPY /data /app/data
6 | ENTRYPOINT ["python", "run_train_yolo.py"]
7 |
--------------------------------------------------------------------------------
/challenge_iccv_2025/README.md:
--------------------------------------------------------------------------------
1 | # 🐳 Docker Setup for AI City ICCV 2025 Track 4
2 |
3 | This guide provides a quick start tutorial for container submissions using a fine-tuned YOLOv11n model as a reference.
4 |
5 | **You should develop your own implementations of the get_model, preprocess_image, and postprocess_results functions in utils.py for your submission. Prize winners need to place their pretraining data (if applicable) and models in the shared Google Drive and upload training and evaluation containers on Docker Hub. Your training and evaluation scripts inside the container should load models from the mounted /models directory and the data from the /data directory.**
6 |
7 | # Evaluation Container Instruction
8 |
9 | ## 🔹 Pull the Prebuilt Docker Image
10 |
11 | Start by pulling the prebuilt Docker image designed for Jetson devices:
12 |
13 | ```bash
14 | docker pull ganzobtn/aicity_iccv_2025_track4_jetson:latest
15 | ```
16 | ## 🔹 Build Image Locally (Optional)
17 | If you'd prefer to build the Docker image locally:
18 |
19 | ```bash
20 | docker build -f Dockerfile.jetson -t ganzobtn/aicity_iccv_2025_track4_jetson:latest .
21 | ```
22 |
23 | ## 🔹 Run the Docker container
24 |
25 | ```bash
26 | IMAGE="ganzobtn/aicity_iccv_2025_track4_jetson:latest"
27 | DATA_DIR="/path/to/your/data"
28 | ```
29 |
30 | ```bash
31 | docker run -it --ipc=host --runtime=nvidia -v ${DATA_DIR}:/data ${IMAGE}
32 | ```
33 |
34 | 📁 Expected Directory Structure
35 |
36 | The `run_evaluation_jetson.py` script inside the container expects the following structure:
37 |
38 | - `/data/FishEye1K_eval/` Contains the groundtruth.json file, evaluation images and corresponding annotation files.
39 |
40 | - `models/yolo11n_fisheye8k.pt`
41 | The fine-tuned YOLOv11n model file used for inference.
42 |
43 |
44 |
45 | # Training Container Instruction
46 |
47 | This section provides a getting started guide for setting up and running the training Docker container for the challenge, which uses a YOLOv11n model finetuning pipeline.
48 |
49 | ## 🔹 Pull Prebuilt Docker Image
50 |
51 | You can use the prebuilt Docker image available on Docker Hub:
52 |
53 | ```bash
54 | docker pull ganzobtn/aicity_iccv_2025_track4_train:latest
55 | ```
56 | ---
57 |
58 | ## 🔹 Build the Docker Image Locally (Optional)
59 | If you'd prefer to build the image from the provided Dockerfile:
60 |
61 | ```bash
62 | docker build -f Dockerfile.train -t ganzobtn/aicity_iccv_2025_track4_train:latest .
63 | ```
64 |
65 | ## 🔹 Run the Docker Container
66 | Set your local paths and run the container:
67 |
68 | ```bash
69 | IMAGE="ganzobtn/aicity_iccv_2025_track4_train:latest"
70 |
71 | DATA_DIR="/path/to/your/data"
72 | MODEL_DIR="/path/to/your/models"
73 |
74 | docker run -it --ipc=host --runtime=nvidia \
75 | -v ${DATA_DIR}:/data \
76 | -v ${MODEL_DIR}:/models \
77 | ${IMAGE}
78 | ```
79 |
80 | 📁 Expected Directory Structure
81 | run_train_yolo.py script inside the container expects the following structure:
82 |
83 | - A dataset folder named Fisheye8K located inside /data.
84 |
85 | - Trained models and output logs will be saved to /models.
86 |
87 |
--------------------------------------------------------------------------------
/challenge_iccv_2025/data/Fisheye1k.yaml:
--------------------------------------------------------------------------------
1 | path: /data
2 |
3 | val: Fisheye1K_eval/images
4 | nc: 5
5 | names: ['Bus','Bike','Car','Pedestrian','Truck']
--------------------------------------------------------------------------------
/challenge_iccv_2025/data/Fisheye8k.yaml:
--------------------------------------------------------------------------------
1 | path: /data/Fisheye8K
2 | train: train/images
3 | val: test/images
4 |
5 | nc: 5
6 |
7 | # class names
8 | names: ['Bus','Bike','Car','Pedestrian','Truck']
--------------------------------------------------------------------------------
/challenge_iccv_2025/models/yolo11n_fisheye8k.pt:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/challenge_iccv_2025/models/yolo11n_fisheye8k.pt
--------------------------------------------------------------------------------
/challenge_iccv_2025/pycocotools/__init__.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tylin'
2 |
--------------------------------------------------------------------------------
/challenge_iccv_2025/pycocotools/_mask.cpython-310-aarch64-linux-gnu.so:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/challenge_iccv_2025/pycocotools/_mask.cpython-310-aarch64-linux-gnu.so
--------------------------------------------------------------------------------
/challenge_iccv_2025/pycocotools/_mask.cpython-311-x86_64-linux-gnu.so:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/challenge_iccv_2025/pycocotools/_mask.cpython-311-x86_64-linux-gnu.so
--------------------------------------------------------------------------------
/challenge_iccv_2025/pycocotools/coco.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tylin'
2 | __version__ = '2.0'
3 | # Interface for accessing the Microsoft COCO dataset.
4 |
5 | # Microsoft COCO is a large image dataset designed for object detection,
6 | # segmentation, and caption generation. pycocotools is a Python API that
7 | # assists in loading, parsing and visualizing the annotations in COCO.
8 | # Please visit http://mscoco.org/ for more information on COCO, including
9 | # for the data, paper, and tutorials. The exact format of the annotations
10 | # is also described on the COCO website. For example usage of the pycocotools
11 | # please see pycocotools_demo.ipynb. In addition to this API, please download both
12 | # the COCO images and annotations in order to run the demo.
13 |
14 | # An alternative to using the API is to load the annotations directly
15 | # into Python dictionary
16 | # Using the API provides additional utility functions. Note that this API
17 | # supports both *instance* and *caption* annotations. In the case of
18 | # captions not all functions are defined (e.g. categories are undefined).
19 |
20 | # The following API functions are defined:
21 | # COCO - COCO api class that loads COCO annotation file and prepare data structures.
22 | # decodeMask - Decode binary mask M encoded via run-length encoding.
23 | # encodeMask - Encode binary mask M using run-length encoding.
24 | # getAnnIds - Get ann ids that satisfy given filter conditions.
25 | # getCatIds - Get cat ids that satisfy given filter conditions.
26 | # getImgIds - Get img ids that satisfy given filter conditions.
27 | # loadAnns - Load anns with the specified ids.
28 | # loadCats - Load cats with the specified ids.
29 | # loadImgs - Load imgs with the specified ids.
30 | # annToMask - Convert segmentation in an annotation to binary mask.
31 | # showAnns - Display the specified annotations.
32 | # loadRes - Load algorithm results and create API for accessing them.
33 | # download - Download COCO images from mscoco.org server.
34 | # Throughout the API "ann"=annotation, "cat"=category, and "img"=image.
35 | # Help on each functions can be accessed by: "help COCO>function".
36 |
37 | # See also COCO>decodeMask,
38 | # COCO>encodeMask, COCO>getAnnIds, COCO>getCatIds,
39 | # COCO>getImgIds, COCO>loadAnns, COCO>loadCats,
40 | # COCO>loadImgs, COCO>annToMask, COCO>showAnns
41 |
42 | # Microsoft COCO Toolbox. version 2.0
43 | # Data, paper, and tutorials available at: http://mscoco.org/
44 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2014.
45 | # Licensed under the Simplified BSD License [see bsd.txt]
46 |
47 | import json
48 | import time
49 | import numpy as np
50 | import copy
51 | import itertools
52 | from . import mask as maskUtils
53 | import os
54 | from collections import defaultdict
55 | import sys
56 | PYTHON_VERSION = sys.version_info[0]
57 | if PYTHON_VERSION == 2:
58 | from urllib import urlretrieve
59 | elif PYTHON_VERSION == 3:
60 | from urllib.request import urlretrieve
61 |
62 |
63 | def _isArrayLike(obj):
64 | return hasattr(obj, '__iter__') and hasattr(obj, '__len__')
65 |
66 |
67 | class COCO:
68 | def __init__(self, annotation_file=None):
69 | """
70 | Constructor of Microsoft COCO helper class for reading and visualizing annotations.
71 | :param annotation_file (str): location of annotation file
72 | :param image_folder (str): location to the folder that hosts images.
73 | :return:
74 | """
75 | # load dataset
76 | self.dataset,self.anns,self.cats,self.imgs = dict(),dict(),dict(),dict()
77 | self.imgToAnns, self.catToImgs = defaultdict(list), defaultdict(list)
78 | if not annotation_file == None:
79 | print('loading annotations into memory...')
80 | tic = time.time()
81 | with open(annotation_file, 'r') as f:
82 | dataset = json.load(f)
83 | assert type(dataset)==dict, 'annotation file format {} not supported'.format(type(dataset))
84 | print('Done (t={:0.2f}s)'.format(time.time()- tic))
85 | self.dataset = dataset
86 | self.createIndex()
87 |
88 | def createIndex(self):
89 | # create index
90 | print('creating index...')
91 | anns, cats, imgs = {}, {}, {}
92 | imgToAnns,catToImgs = defaultdict(list),defaultdict(list)
93 | if 'annotations' in self.dataset:
94 | for ann in self.dataset['annotations']:
95 | imgToAnns[ann['image_id']].append(ann)
96 | anns[ann['id']] = ann
97 |
98 | if 'images' in self.dataset:
99 | for img in self.dataset['images']:
100 | imgs[img['id']] = img
101 |
102 | if 'categories' in self.dataset:
103 | for cat in self.dataset['categories']:
104 | cats[cat['id']] = cat
105 |
106 | if 'annotations' in self.dataset and 'categories' in self.dataset:
107 | for ann in self.dataset['annotations']:
108 | catToImgs[ann['category_id']].append(ann['image_id'])
109 |
110 | print('index created!')
111 |
112 | # create class members
113 | self.anns = anns
114 | self.imgToAnns = imgToAnns
115 | self.catToImgs = catToImgs
116 | self.imgs = imgs
117 | self.cats = cats
118 |
119 | def info(self):
120 | """
121 | Print information about the annotation file.
122 | :return:
123 | """
124 | for key, value in self.dataset['info'].items():
125 | print('{}: {}'.format(key, value))
126 |
127 | def getAnnIds(self, imgIds=[], catIds=[], areaRng=[], iscrowd=None):
128 | """
129 | Get ann ids that satisfy given filter conditions. default skips that filter
130 | :param imgIds (int array) : get anns for given imgs
131 | catIds (int array) : get anns for given cats
132 | areaRng (float array) : get anns for given area range (e.g. [0 inf])
133 | iscrowd (boolean) : get anns for given crowd label (False or True)
134 | :return: ids (int array) : integer array of ann ids
135 | """
136 | imgIds = imgIds if _isArrayLike(imgIds) else [imgIds]
137 | catIds = catIds if _isArrayLike(catIds) else [catIds]
138 |
139 | if len(imgIds) == len(catIds) == len(areaRng) == 0:
140 | anns = self.dataset['annotations']
141 | else:
142 | if not len(imgIds) == 0:
143 | lists = [self.imgToAnns[imgId] for imgId in imgIds if imgId in self.imgToAnns]
144 | anns = list(itertools.chain.from_iterable(lists))
145 | else:
146 | anns = self.dataset['annotations']
147 | anns = anns if len(catIds) == 0 else [ann for ann in anns if ann['category_id'] in catIds]
148 | anns = anns if len(areaRng) == 0 else [ann for ann in anns if ann['area'] > areaRng[0] and ann['area'] < areaRng[1]]
149 | if not iscrowd == None:
150 | ids = [ann['id'] for ann in anns if ann['iscrowd'] == iscrowd]
151 | else:
152 | ids = [ann['id'] for ann in anns]
153 | return ids
154 |
155 | def getCatIds(self, catNms=[], supNms=[], catIds=[]):
156 | """
157 | filtering parameters. default skips that filter.
158 | :param catNms (str array) : get cats for given cat names
159 | :param supNms (str array) : get cats for given supercategory names
160 | :param catIds (int array) : get cats for given cat ids
161 | :return: ids (int array) : integer array of cat ids
162 | """
163 | catNms = catNms if _isArrayLike(catNms) else [catNms]
164 | supNms = supNms if _isArrayLike(supNms) else [supNms]
165 | catIds = catIds if _isArrayLike(catIds) else [catIds]
166 |
167 | if len(catNms) == len(supNms) == len(catIds) == 0:
168 | cats = self.dataset['categories']
169 | else:
170 | cats = self.dataset['categories']
171 | cats = cats if len(catNms) == 0 else [cat for cat in cats if cat['name'] in catNms]
172 | cats = cats if len(supNms) == 0 else [cat for cat in cats if cat['supercategory'] in supNms]
173 | cats = cats if len(catIds) == 0 else [cat for cat in cats if cat['id'] in catIds]
174 | ids = [cat['id'] for cat in cats]
175 | return ids
176 |
177 | def getImgIds(self, imgIds=[], catIds=[]):
178 | '''
179 | Get img ids that satisfy given filter conditions.
180 | :param imgIds (int array) : get imgs for given ids
181 | :param catIds (int array) : get imgs with all given cats
182 | :return: ids (int array) : integer array of img ids
183 | '''
184 | imgIds = imgIds if _isArrayLike(imgIds) else [imgIds]
185 | catIds = catIds if _isArrayLike(catIds) else [catIds]
186 |
187 | if len(imgIds) == len(catIds) == 0:
188 | ids = self.imgs.keys()
189 | else:
190 | ids = set(imgIds)
191 | for i, catId in enumerate(catIds):
192 | if i == 0 and len(ids) == 0:
193 | ids = set(self.catToImgs[catId])
194 | else:
195 | ids &= set(self.catToImgs[catId])
196 | return list(ids)
197 |
198 | def loadAnns(self, ids=[]):
199 | """
200 | Load anns with the specified ids.
201 | :param ids (int array) : integer ids specifying anns
202 | :return: anns (object array) : loaded ann objects
203 | """
204 | if _isArrayLike(ids):
205 | return [self.anns[id] for id in ids]
206 | elif type(ids) == int:
207 | return [self.anns[ids]]
208 |
209 | def loadCats(self, ids=[]):
210 | """
211 | Load cats with the specified ids.
212 | :param ids (int array) : integer ids specifying cats
213 | :return: cats (object array) : loaded cat objects
214 | """
215 | if _isArrayLike(ids):
216 | return [self.cats[id] for id in ids]
217 | elif type(ids) == int:
218 | return [self.cats[ids]]
219 |
220 | def loadImgs(self, ids=[]):
221 | """
222 | Load anns with the specified ids.
223 | :param ids (int array) : integer ids specifying img
224 | :return: imgs (object array) : loaded img objects
225 | """
226 | if _isArrayLike(ids):
227 | return [self.imgs[id] for id in ids]
228 | elif type(ids) == int:
229 | return [self.imgs[ids]]
230 |
231 | def showAnns(self, anns, draw_bbox=False):
232 | """
233 | Display the specified annotations.
234 | :param anns (array of object): annotations to display
235 | :return: None
236 | """
237 | if len(anns) == 0:
238 | return 0
239 | if 'segmentation' in anns[0] or 'keypoints' in anns[0]:
240 | datasetType = 'instances'
241 | elif 'caption' in anns[0]:
242 | datasetType = 'captions'
243 | else:
244 | raise Exception('datasetType not supported')
245 | if datasetType == 'instances':
246 | import matplotlib.pyplot as plt
247 | from matplotlib.collections import PatchCollection
248 | from matplotlib.patches import Polygon
249 |
250 | ax = plt.gca()
251 | ax.set_autoscale_on(False)
252 | polygons = []
253 | color = []
254 | for ann in anns:
255 | c = (np.random.random((1, 3))*0.6+0.4).tolist()[0]
256 | if 'segmentation' in ann:
257 | if type(ann['segmentation']) == list:
258 | # polygon
259 | for seg in ann['segmentation']:
260 | poly = np.array(seg).reshape((int(len(seg)/2), 2))
261 | polygons.append(Polygon(poly))
262 | color.append(c)
263 | else:
264 | # mask
265 | t = self.imgs[ann['image_id']]
266 | if type(ann['segmentation']['counts']) == list:
267 | rle = maskUtils.frPyObjects([ann['segmentation']], t['height'], t['width'])
268 | else:
269 | rle = [ann['segmentation']]
270 | m = maskUtils.decode(rle)
271 | img = np.ones( (m.shape[0], m.shape[1], 3) )
272 | if ann['iscrowd'] == 1:
273 | color_mask = np.array([2.0,166.0,101.0])/255
274 | if ann['iscrowd'] == 0:
275 | color_mask = np.random.random((1, 3)).tolist()[0]
276 | for i in range(3):
277 | img[:,:,i] = color_mask[i]
278 | ax.imshow(np.dstack( (img, m*0.5) ))
279 | if 'keypoints' in ann and type(ann['keypoints']) == list:
280 | # turn skeleton into zero-based index
281 | sks = np.array(self.loadCats(ann['category_id'])[0]['skeleton'])-1
282 | kp = np.array(ann['keypoints'])
283 | x = kp[0::3]
284 | y = kp[1::3]
285 | v = kp[2::3]
286 | for sk in sks:
287 | if np.all(v[sk]>0):
288 | plt.plot(x[sk],y[sk], linewidth=3, color=c)
289 | plt.plot(x[v>0], y[v>0],'o',markersize=8, markerfacecolor=c, markeredgecolor='k',markeredgewidth=2)
290 | plt.plot(x[v>1], y[v>1],'o',markersize=8, markerfacecolor=c, markeredgecolor=c, markeredgewidth=2)
291 |
292 | if draw_bbox:
293 | [bbox_x, bbox_y, bbox_w, bbox_h] = ann['bbox']
294 | poly = [[bbox_x, bbox_y], [bbox_x, bbox_y+bbox_h], [bbox_x+bbox_w, bbox_y+bbox_h], [bbox_x+bbox_w, bbox_y]]
295 | np_poly = np.array(poly).reshape((4,2))
296 | polygons.append(Polygon(np_poly))
297 | color.append(c)
298 |
299 | p = PatchCollection(polygons, facecolor=color, linewidths=0, alpha=0.4)
300 | ax.add_collection(p)
301 | p = PatchCollection(polygons, facecolor='none', edgecolors=color, linewidths=2)
302 | ax.add_collection(p)
303 | elif datasetType == 'captions':
304 | for ann in anns:
305 | print(ann['caption'])
306 |
307 | def loadRes(self, resFile):
308 | """
309 | Load result file and return a result api object.
310 | :param resFile (str) : file name of result file
311 | :return: res (obj) : result api object
312 | """
313 | res = COCO()
314 | res.dataset['images'] = [img for img in self.dataset['images']]
315 |
316 | print('Loading and preparing results...')
317 | tic = time.time()
318 | if type(resFile) == str or (PYTHON_VERSION == 2 and type(resFile) == unicode):
319 | with open(resFile) as f:
320 | anns = json.load(f)
321 | elif type(resFile) == np.ndarray:
322 | anns = self.loadNumpyAnnotations(resFile)
323 | else:
324 | anns = resFile
325 | assert type(anns) == list, 'results in not an array of objects'
326 | annsImgIds = [ann['image_id'] for ann in anns]
327 | assert set(annsImgIds) == (set(annsImgIds) & set(self.getImgIds())), \
328 | 'Results do not correspond to current coco set'
329 | if 'caption' in anns[0]:
330 | imgIds = set([img['id'] for img in res.dataset['images']]) & set([ann['image_id'] for ann in anns])
331 | res.dataset['images'] = [img for img in res.dataset['images'] if img['id'] in imgIds]
332 | for id, ann in enumerate(anns):
333 | ann['id'] = id+1
334 | elif 'bbox' in anns[0] and not anns[0]['bbox'] == []:
335 | res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])
336 | for id, ann in enumerate(anns):
337 | bb = ann['bbox']
338 | x1, x2, y1, y2 = [bb[0], bb[0]+bb[2], bb[1], bb[1]+bb[3]]
339 | if not 'segmentation' in ann:
340 | ann['segmentation'] = [[x1, y1, x1, y2, x2, y2, x2, y1]]
341 | ann['area'] = bb[2]*bb[3]
342 | ann['id'] = id+1
343 | ann['iscrowd'] = 0
344 | elif 'segmentation' in anns[0]:
345 | res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])
346 | for id, ann in enumerate(anns):
347 | # now only support compressed RLE format as segmentation results
348 | ann['area'] = maskUtils.area(ann['segmentation'])
349 | if not 'bbox' in ann:
350 | ann['bbox'] = maskUtils.toBbox(ann['segmentation'])
351 | ann['id'] = id+1
352 | ann['iscrowd'] = 0
353 | elif 'keypoints' in anns[0]:
354 | res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])
355 | for id, ann in enumerate(anns):
356 | s = ann['keypoints']
357 | x = s[0::3]
358 | y = s[1::3]
359 | x0,x1,y0,y1 = np.min(x), np.max(x), np.min(y), np.max(y)
360 | ann['area'] = (x1-x0)*(y1-y0)
361 | ann['id'] = id + 1
362 | ann['bbox'] = [x0,y0,x1-x0,y1-y0]
363 | print('DONE (t={:0.2f}s)'.format(time.time()- tic))
364 |
365 | res.dataset['annotations'] = anns
366 | res.createIndex()
367 | return res
368 |
369 | def download(self, tarDir = None, imgIds = [] ):
370 | '''
371 | Download COCO images from mscoco.org server.
372 | :param tarDir (str): COCO results directory name
373 | imgIds (list): images to be downloaded
374 | :return:
375 | '''
376 | if tarDir is None:
377 | print('Please specify target directory')
378 | return -1
379 | if len(imgIds) == 0:
380 | imgs = self.imgs.values()
381 | else:
382 | imgs = self.loadImgs(imgIds)
383 | N = len(imgs)
384 | if not os.path.exists(tarDir):
385 | os.makedirs(tarDir)
386 | for i, img in enumerate(imgs):
387 | tic = time.time()
388 | fname = os.path.join(tarDir, img['file_name'])
389 | if not os.path.exists(fname):
390 | urlretrieve(img['coco_url'], fname)
391 | print('downloaded {}/{} images (t={:0.1f}s)'.format(i, N, time.time()- tic))
392 |
393 | def loadNumpyAnnotations(self, data):
394 | """
395 | Convert result data from a numpy array [Nx7] where each row contains {imageID,x1,y1,w,h,score,class}
396 | :param data (numpy.ndarray)
397 | :return: annotations (python nested list)
398 | """
399 | print('Converting ndarray to lists...')
400 | assert(type(data) == np.ndarray)
401 | print(data.shape)
402 | assert(data.shape[1] == 7)
403 | N = data.shape[0]
404 | ann = []
405 | for i in range(N):
406 | if i % 1000000 == 0:
407 | print('{}/{}'.format(i,N))
408 | ann += [{
409 | 'image_id' : int(data[i, 0]),
410 | 'bbox' : [ data[i, 1], data[i, 2], data[i, 3], data[i, 4] ],
411 | 'score' : data[i, 5],
412 | 'category_id': int(data[i, 6]),
413 | }]
414 | return ann
415 |
416 | def annToRLE(self, ann):
417 | """
418 | Convert annotation which can be polygons, uncompressed RLE to RLE.
419 | :return: binary mask (numpy 2D array)
420 | """
421 | t = self.imgs[ann['image_id']]
422 | h, w = t['height'], t['width']
423 | segm = ann['segmentation']
424 | if type(segm) == list:
425 | # polygon -- a single object might consist of multiple parts
426 | # we merge all parts into one mask rle code
427 | rles = maskUtils.frPyObjects(segm, h, w)
428 | rle = maskUtils.merge(rles)
429 | elif type(segm['counts']) == list:
430 | # uncompressed RLE
431 | rle = maskUtils.frPyObjects(segm, h, w)
432 | else:
433 | # rle
434 | rle = ann['segmentation']
435 | return rle
436 |
437 | def annToMask(self, ann):
438 | """
439 | Convert annotation which can be polygons, uncompressed RLE, or RLE to binary mask.
440 | :return: binary mask (numpy 2D array)
441 | """
442 | rle = self.annToRLE(ann)
443 | m = maskUtils.decode(rle)
444 | return m
445 |
--------------------------------------------------------------------------------
/challenge_iccv_2025/pycocotools/cocoeval.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tsungyi'
2 |
3 | import numpy as np
4 | import datetime
5 | import time
6 | from collections import defaultdict
7 | from . import mask as maskUtils
8 | import copy
9 |
10 | class COCOeval:
11 | # Interface for evaluating detection on the Microsoft COCO dataset.
12 | #
13 | # The usage for CocoEval is as follows:
14 | # cocoGt=..., cocoDt=... # load dataset and results
15 | # E = CocoEval(cocoGt,cocoDt); # initialize CocoEval object
16 | # E.params.recThrs = ...; # set parameters as desired
17 | # E.evaluate(); # run per image evaluation
18 | # E.accumulate(); # accumulate per image results
19 | # E.summarize(); # display summary metrics of results
20 | # For example usage see evalDemo.m and http://mscoco.org/.
21 | #
22 | # The evaluation parameters are as follows (defaults in brackets):
23 | # imgIds - [all] N img ids to use for evaluation
24 | # catIds - [all] K cat ids to use for evaluation
25 | # iouThrs - [.5:.05:.95] T=10 IoU thresholds for evaluation
26 | # recThrs - [0:.01:1] R=101 recall thresholds for evaluation
27 | # areaRng - [...] A=4 object area ranges for evaluation
28 | # maxDets - [1 10 100] M=3 thresholds on max detections per image
29 | # iouType - ['segm'] set iouType to 'segm', 'bbox' or 'keypoints'
30 | # iouType replaced the now DEPRECATED useSegm parameter.
31 | # useCats - [1] if true use category labels for evaluation
32 | # Note: if useCats=0 category labels are ignored as in proposal scoring.
33 | # Note: multiple areaRngs [Ax2] and maxDets [Mx1] can be specified.
34 | #
35 | # evaluate(): evaluates detections on every image and every category and
36 | # concats the results into the "evalImgs" with fields:
37 | # dtIds - [1xD] id for each of the D detections (dt)
38 | # gtIds - [1xG] id for each of the G ground truths (gt)
39 | # dtMatches - [TxD] matching gt id at each IoU or 0
40 | # gtMatches - [TxG] matching dt id at each IoU or 0
41 | # dtScores - [1xD] confidence of each dt
42 | # gtIgnore - [1xG] ignore flag for each gt
43 | # dtIgnore - [TxD] ignore flag for each dt at each IoU
44 | #
45 | # accumulate(): accumulates the per-image, per-category evaluation
46 | # results in "evalImgs" into the dictionary "eval" with fields:
47 | # params - parameters used for evaluation
48 | # date - date evaluation was performed
49 | # counts - [T,R,K,A,M] parameter dimensions (see above)
50 | # precision - [TxRxKxAxM] precision for every evaluation setting
51 | # recall - [TxKxAxM] max recall for every evaluation setting
52 | # Note: precision and recall==-1 for settings with no gt objects.
53 | #
54 | # See also coco, mask, pycocoDemo, pycocoEvalDemo
55 | #
56 | # Microsoft COCO Toolbox. version 2.0
57 | # Data, paper, and tutorials available at: http://mscoco.org/
58 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2015.
59 | # Licensed under the Simplified BSD License [see coco/license.txt]
60 | def __init__(self, cocoGt=None, cocoDt=None, iouType='segm'):
61 | '''
62 | Initialize CocoEval using coco APIs for gt and dt
63 | :param cocoGt: coco object with ground truth annotations
64 | :param cocoDt: coco object with detection results
65 | :return: None
66 | '''
67 | if not iouType:
68 | print('iouType not specified. use default iouType segm')
69 | self.cocoGt = cocoGt # ground truth COCO API
70 | self.cocoDt = cocoDt # detections COCO API
71 | self.evalImgs = defaultdict(list) # per-image per-category evaluation results [KxAxI] elements
72 | self.eval = {} # accumulated evaluation results
73 | self._gts = defaultdict(list) # gt for evaluation
74 | self._dts = defaultdict(list) # dt for evaluation
75 | self.params = Params(iouType=iouType) # parameters
76 | self._paramsEval = {} # parameters for evaluation
77 | self.stats = [] # result summarization
78 | self.ious = {} # ious between all gts and dts
79 | if not cocoGt is None:
80 | self.params.imgIds = sorted(cocoGt.getImgIds())
81 | self.params.catIds = sorted(cocoGt.getCatIds())
82 |
83 |
84 | def _prepare(self):
85 | '''
86 | Prepare ._gts and ._dts for evaluation based on params
87 | :return: None
88 | '''
89 | def _toMask(anns, coco):
90 | # modify ann['segmentation'] by reference
91 | for ann in anns:
92 | rle = coco.annToRLE(ann)
93 | ann['segmentation'] = rle
94 | p = self.params
95 | if p.useCats:
96 | gts=self.cocoGt.loadAnns(self.cocoGt.getAnnIds(imgIds=p.imgIds, catIds=p.catIds))
97 | dts=self.cocoDt.loadAnns(self.cocoDt.getAnnIds(imgIds=p.imgIds, catIds=p.catIds))
98 | else:
99 | gts=self.cocoGt.loadAnns(self.cocoGt.getAnnIds(imgIds=p.imgIds))
100 | dts=self.cocoDt.loadAnns(self.cocoDt.getAnnIds(imgIds=p.imgIds))
101 |
102 | # convert ground truth to mask if iouType == 'segm'
103 | if p.iouType == 'segm':
104 | _toMask(gts, self.cocoGt)
105 | _toMask(dts, self.cocoDt)
106 | # set ignore flag
107 | for gt in gts:
108 | gt['ignore'] = gt['ignore'] if 'ignore' in gt else 0
109 | gt['ignore'] = 'iscrowd' in gt and gt['iscrowd']
110 | if p.iouType == 'keypoints':
111 | gt['ignore'] = (gt['num_keypoints'] == 0) or gt['ignore']
112 | self._gts = defaultdict(list) # gt for evaluation
113 | self._dts = defaultdict(list) # dt for evaluation
114 | for gt in gts:
115 | self._gts[gt['image_id'], gt['category_id']].append(gt)
116 | for dt in dts:
117 | self._dts[dt['image_id'], dt['category_id']].append(dt)
118 | self.evalImgs = defaultdict(list) # per-image per-category evaluation results
119 | self.eval = {} # accumulated evaluation results
120 |
121 | def evaluate(self):
122 | '''
123 | Run per image evaluation on given images and store results (a list of dict) in self.evalImgs
124 | :return: None
125 | '''
126 | tic = time.time()
127 | print('Running per image evaluation...')
128 | p = self.params
129 | # add backward compatibility if useSegm is specified in params
130 | if not p.useSegm is None:
131 | p.iouType = 'segm' if p.useSegm == 1 else 'bbox'
132 | print('useSegm (deprecated) is not None. Running {} evaluation'.format(p.iouType))
133 | print('Evaluate annotation type *{}*'.format(p.iouType))
134 | p.imgIds = list(np.unique(p.imgIds))
135 | if p.useCats:
136 | p.catIds = list(np.unique(p.catIds))
137 | p.maxDets = sorted(p.maxDets)
138 | self.params=p
139 |
140 | self._prepare()
141 | # loop through images, area range, max detection number
142 | catIds = p.catIds if p.useCats else [-1]
143 |
144 | if p.iouType == 'segm' or p.iouType == 'bbox':
145 | computeIoU = self.computeIoU
146 | elif p.iouType == 'keypoints':
147 | computeIoU = self.computeOks
148 | self.ious = {(imgId, catId): computeIoU(imgId, catId) \
149 | for imgId in p.imgIds
150 | for catId in catIds}
151 |
152 | evaluateImg = self.evaluateImg
153 | maxDet = p.maxDets[-1]
154 | self.evalImgs = [evaluateImg(imgId, catId, areaRng, maxDet)
155 | for catId in catIds
156 | for areaRng in p.areaRng
157 | for imgId in p.imgIds
158 | ]
159 | self._paramsEval = copy.deepcopy(self.params)
160 | toc = time.time()
161 | print('DONE (t={:0.2f}s).'.format(toc-tic))
162 |
163 | def computeIoU(self, imgId, catId):
164 | p = self.params
165 | if p.useCats:
166 | gt = self._gts[imgId,catId]
167 | dt = self._dts[imgId,catId]
168 | else:
169 | gt = [_ for cId in p.catIds for _ in self._gts[imgId,cId]]
170 | dt = [_ for cId in p.catIds for _ in self._dts[imgId,cId]]
171 | if len(gt) == 0 and len(dt) ==0:
172 | return []
173 | inds = np.argsort([-d['score'] for d in dt], kind='mergesort')
174 | dt = [dt[i] for i in inds]
175 | if len(dt) > p.maxDets[-1]:
176 | dt=dt[0:p.maxDets[-1]]
177 |
178 | if p.iouType == 'segm':
179 | g = [g['segmentation'] for g in gt]
180 | d = [d['segmentation'] for d in dt]
181 | elif p.iouType == 'bbox':
182 | g = [g['bbox'] for g in gt]
183 | d = [d['bbox'] for d in dt]
184 | else:
185 | raise Exception('unknown iouType for iou computation')
186 |
187 | # compute iou between each dt and gt region
188 | iscrowd = [int(o['iscrowd']) for o in gt]
189 | ious = maskUtils.iou(d,g,iscrowd)
190 | return ious
191 |
192 | def computeOks(self, imgId, catId):
193 | p = self.params
194 | # dimention here should be Nxm
195 | gts = self._gts[imgId, catId]
196 | dts = self._dts[imgId, catId]
197 | inds = np.argsort([-d['score'] for d in dts], kind='mergesort')
198 | dts = [dts[i] for i in inds]
199 | if len(dts) > p.maxDets[-1]:
200 | dts = dts[0:p.maxDets[-1]]
201 | # if len(gts) == 0 and len(dts) == 0:
202 | if len(gts) == 0 or len(dts) == 0:
203 | return []
204 | ious = np.zeros((len(dts), len(gts)))
205 | sigmas = p.kpt_oks_sigmas
206 | vars = (sigmas * 2)**2
207 | k = len(sigmas)
208 | # compute oks between each detection and ground truth object
209 | for j, gt in enumerate(gts):
210 | # create bounds for ignore regions(double the gt bbox)
211 | g = np.array(gt['keypoints'])
212 | xg = g[0::3]; yg = g[1::3]; vg = g[2::3]
213 | k1 = np.count_nonzero(vg > 0)
214 | bb = gt['bbox']
215 | x0 = bb[0] - bb[2]; x1 = bb[0] + bb[2] * 2
216 | y0 = bb[1] - bb[3]; y1 = bb[1] + bb[3] * 2
217 | for i, dt in enumerate(dts):
218 | d = np.array(dt['keypoints'])
219 | xd = d[0::3]; yd = d[1::3]
220 | if k1>0:
221 | # measure the per-keypoint distance if keypoints visible
222 | dx = xd - xg
223 | dy = yd - yg
224 | else:
225 | # measure minimum distance to keypoints in (x0,y0) & (x1,y1)
226 | z = np.zeros((k))
227 | dx = np.max((z, x0-xd),axis=0)+np.max((z, xd-x1),axis=0)
228 | dy = np.max((z, y0-yd),axis=0)+np.max((z, yd-y1),axis=0)
229 | e = (dx**2 + dy**2) / vars / (gt['area']+np.spacing(1)) / 2
230 | if k1 > 0:
231 | e=e[vg > 0]
232 | ious[i, j] = np.sum(np.exp(-e)) / e.shape[0]
233 | return ious
234 |
235 | def evaluateImg(self, imgId, catId, aRng, maxDet):
236 | '''
237 | perform evaluation for single category and image
238 | :return: dict (single image results)
239 | '''
240 | p = self.params
241 | if p.useCats:
242 | gt = self._gts[imgId,catId]
243 | dt = self._dts[imgId,catId]
244 | else:
245 | gt = [_ for cId in p.catIds for _ in self._gts[imgId,cId]]
246 | dt = [_ for cId in p.catIds for _ in self._dts[imgId,cId]]
247 | if len(gt) == 0 and len(dt) ==0:
248 | return None
249 |
250 | for g in gt:
251 | if g['ignore'] or (g['area']aRng[1]):
252 | g['_ignore'] = 1
253 | else:
254 | g['_ignore'] = 0
255 |
256 | # sort dt highest score first, sort gt ignore last
257 | gtind = np.argsort([g['_ignore'] for g in gt], kind='mergesort')
258 | gt = [gt[i] for i in gtind]
259 | dtind = np.argsort([-d['score'] for d in dt], kind='mergesort')
260 | dt = [dt[i] for i in dtind[0:maxDet]]
261 | iscrowd = [int(o['iscrowd']) for o in gt]
262 | # load computed ious
263 | ious = self.ious[imgId, catId][:, gtind] if len(self.ious[imgId, catId]) > 0 else self.ious[imgId, catId]
264 |
265 | T = len(p.iouThrs)
266 | G = len(gt)
267 | D = len(dt)
268 | gtm = np.zeros((T,G))
269 | dtm = np.zeros((T,D))
270 | gtIg = np.array([g['_ignore'] for g in gt])
271 | dtIg = np.zeros((T,D))
272 | if not len(ious)==0:
273 | for tind, t in enumerate(p.iouThrs):
274 | for dind, d in enumerate(dt):
275 | # information about best match so far (m=-1 -> unmatched)
276 | iou = min([t,1-1e-10])
277 | m = -1
278 | for gind, g in enumerate(gt):
279 | # if this gt already matched, and not a crowd, continue
280 | if gtm[tind,gind]>0 and not iscrowd[gind]:
281 | continue
282 | # if dt matched to reg gt, and on ignore gt, stop
283 | if m>-1 and gtIg[m]==0 and gtIg[gind]==1:
284 | break
285 | # continue to next gt unless better match made
286 | if ious[dind,gind] < iou:
287 | continue
288 | # if match successful and best so far, store appropriately
289 | iou=ious[dind,gind]
290 | m=gind
291 | # if match made store id of match for both dt and gt
292 | if m ==-1:
293 | continue
294 | dtIg[tind,dind] = gtIg[m]
295 | dtm[tind,dind] = gt[m]['id']
296 | gtm[tind,m] = d['id']
297 | # set unmatched detections outside of area range to ignore
298 | a = np.array([d['area']aRng[1] for d in dt]).reshape((1, len(dt)))
299 | dtIg = np.logical_or(dtIg, np.logical_and(dtm==0, np.repeat(a,T,0)))
300 | # store results for given image and category
301 | return {
302 | 'image_id': imgId,
303 | 'category_id': catId,
304 | 'aRng': aRng,
305 | 'maxDet': maxDet,
306 | 'dtIds': [d['id'] for d in dt],
307 | 'gtIds': [g['id'] for g in gt],
308 | 'dtMatches': dtm,
309 | 'gtMatches': gtm,
310 | 'dtScores': [d['score'] for d in dt],
311 | 'gtIgnore': gtIg,
312 | 'dtIgnore': dtIg,
313 | }
314 |
315 | def accumulate(self, p = None):
316 | '''
317 | Accumulate per image evaluation results and store the result in self.eval
318 | :param p: input params for evaluation
319 | :return: None
320 | '''
321 | print('Accumulating evaluation results...')
322 | tic = time.time()
323 | if not self.evalImgs:
324 | print('Please run evaluate() first')
325 | # allows input customized parameters
326 | if p is None:
327 | p = self.params
328 | p.catIds = p.catIds if p.useCats == 1 else [-1]
329 | T = len(p.iouThrs)
330 | R = len(p.recThrs)
331 | K = len(p.catIds) if p.useCats else 1
332 | A = len(p.areaRng)
333 | M = len(p.maxDets)
334 | precision = -np.ones((T,R,K,A,M)) # -1 for the precision of absent categories
335 | recall = -np.ones((T,K,A,M))
336 | scores = -np.ones((T,R,K,A,M))
337 |
338 | # create dictionary for future indexing
339 | _pe = self._paramsEval
340 | catIds = _pe.catIds if _pe.useCats else [-1]
341 | setK = set(catIds)
342 | setA = set(map(tuple, _pe.areaRng))
343 | setM = set(_pe.maxDets)
344 | setI = set(_pe.imgIds)
345 | # get inds to evaluate
346 | k_list = [n for n, k in enumerate(p.catIds) if k in setK]
347 | m_list = [m for n, m in enumerate(p.maxDets) if m in setM]
348 | a_list = [n for n, a in enumerate(map(lambda x: tuple(x), p.areaRng)) if a in setA]
349 | i_list = [n for n, i in enumerate(p.imgIds) if i in setI]
350 | I0 = len(_pe.imgIds)
351 | A0 = len(_pe.areaRng)
352 | # retrieve E at each category, area range, and max number of detections
353 | for k, k0 in enumerate(k_list):
354 | Nk = k0*A0*I0
355 | for a, a0 in enumerate(a_list):
356 | Na = a0*I0
357 | for m, maxDet in enumerate(m_list):
358 | E = [self.evalImgs[Nk + Na + i] for i in i_list]
359 | E = [e for e in E if not e is None]
360 | if len(E) == 0:
361 | continue
362 | dtScores = np.concatenate([e['dtScores'][0:maxDet] for e in E])
363 |
364 | # different sorting method generates slightly different results.
365 | # mergesort is used to be consistent as Matlab implementation.
366 | inds = np.argsort(-dtScores, kind='mergesort')
367 | dtScoresSorted = dtScores[inds]
368 |
369 | dtm = np.concatenate([e['dtMatches'][:,0:maxDet] for e in E], axis=1)[:,inds]
370 | dtIg = np.concatenate([e['dtIgnore'][:,0:maxDet] for e in E], axis=1)[:,inds]
371 | gtIg = np.concatenate([e['gtIgnore'] for e in E])
372 | npig = np.count_nonzero(gtIg==0 )
373 | if npig == 0:
374 | continue
375 | tps = np.logical_and( dtm, np.logical_not(dtIg) )
376 | fps = np.logical_and(np.logical_not(dtm), np.logical_not(dtIg) )
377 |
378 | tp_sum = np.cumsum(tps, axis=1).astype(dtype=float)
379 | fp_sum = np.cumsum(fps, axis=1).astype(dtype=float)
380 | for t, (tp, fp) in enumerate(zip(tp_sum, fp_sum)):
381 | tp = np.array(tp)
382 | fp = np.array(fp)
383 | nd = len(tp)
384 | rc = tp / npig
385 | pr = tp / (fp+tp+np.spacing(1))
386 | q = np.zeros((R,))
387 | ss = np.zeros((R,))
388 |
389 | if nd:
390 | recall[t,k,a,m] = rc[-1]
391 | else:
392 | recall[t,k,a,m] = 0
393 |
394 | # numpy is slow without cython optimization for accessing elements
395 | # use python array gets significant speed improvement
396 | pr = pr.tolist(); q = q.tolist()
397 |
398 | for i in range(nd-1, 0, -1):
399 | if pr[i] > pr[i-1]:
400 | pr[i-1] = pr[i]
401 |
402 | inds = np.searchsorted(rc, p.recThrs, side='left')
403 | try:
404 | for ri, pi in enumerate(inds):
405 | q[ri] = pr[pi]
406 | ss[ri] = dtScoresSorted[pi]
407 | except:
408 | pass
409 | precision[t,:,k,a,m] = np.array(q)
410 | scores[t,:,k,a,m] = np.array(ss)
411 | self.eval = {
412 | 'params': p,
413 | 'counts': [T, R, K, A, M],
414 | 'date': datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
415 | 'precision': precision,
416 | 'recall': recall,
417 | 'scores': scores,
418 | }
419 | toc = time.time()
420 | print('DONE (t={:0.2f}s).'.format( toc-tic))
421 |
422 | def summarize(self):
423 | '''
424 | Compute and display summary metrics for evaluation results.
425 | Note this functin can *only* be applied on the default parameter setting
426 | '''
427 | def _summarize( ap=1, iouThr=None, areaRng='all', maxDets=100 ):
428 | p = self.params
429 | iStr = ' {:<18} {} @[ IoU={:<9} | area={:>6s} | maxDets={:>3d} ] = {:0.3f}'
430 | titleStr = 'Average Precision' if ap == 1 else 'Average Recall'
431 | typeStr = '(AP)' if ap==1 else '(AR)'
432 | iouStr = '{:0.2f}:{:0.2f}'.format(p.iouThrs[0], p.iouThrs[-1]) \
433 | if iouThr is None else '{:0.2f}'.format(iouThr)
434 |
435 | aind = [i for i, aRng in enumerate(p.areaRngLbl) if aRng == areaRng]
436 | mind = [i for i, mDet in enumerate(p.maxDets) if mDet == maxDets]
437 | if ap == 1:
438 | # dimension of precision: [TxRxKxAxM]
439 | s = self.eval['precision']
440 | # IoU
441 | if iouThr is not None:
442 | t = np.where(iouThr == p.iouThrs)[0]
443 | s = s[t]
444 | s = s[:,:,:,aind,mind]
445 | else:
446 | # dimension of recall: [TxKxAxM]
447 | s = self.eval['recall']
448 | if iouThr is not None:
449 | t = np.where(iouThr == p.iouThrs)[0]
450 | s = s[t]
451 | s = s[:,:,aind,mind]
452 | if len(s[s>-1])==0:
453 | mean_s = -1
454 | else:
455 | mean_s = np.mean(s[s>-1])
456 | print(iStr.format(titleStr, typeStr, iouStr, areaRng, maxDets, mean_s))
457 | return mean_s
458 | def _summarizeDets():
459 | stats = np.zeros((12,))
460 | stats[0] = _summarize(1)
461 | stats[1] = _summarize(1, iouThr=.5, maxDets=self.params.maxDets[2])
462 | stats[2] = _summarize(1, iouThr=.75, maxDets=self.params.maxDets[2])
463 | stats[3] = _summarize(1, areaRng='small', maxDets=self.params.maxDets[2])
464 | stats[4] = _summarize(1, areaRng='medium', maxDets=self.params.maxDets[2])
465 | stats[5] = _summarize(1, areaRng='large', maxDets=self.params.maxDets[2])
466 | stats[6] = _summarize(0, maxDets=self.params.maxDets[0])
467 | stats[7] = _summarize(0, maxDets=self.params.maxDets[1])
468 | stats[8] = _summarize(0, maxDets=self.params.maxDets[2])
469 | stats[9] = _summarize(0, areaRng='small', maxDets=self.params.maxDets[2])
470 | stats[10] = _summarize(0, areaRng='medium', maxDets=self.params.maxDets[2])
471 | stats[11] = _summarize(0, areaRng='large', maxDets=self.params.maxDets[2])
472 | return stats
473 | def _summarizeKps():
474 | stats = np.zeros((10,))
475 | stats[0] = _summarize(1, maxDets=20)
476 | stats[1] = _summarize(1, maxDets=20, iouThr=.5)
477 | stats[2] = _summarize(1, maxDets=20, iouThr=.75)
478 | stats[3] = _summarize(1, maxDets=20, areaRng='medium')
479 | stats[4] = _summarize(1, maxDets=20, areaRng='large')
480 | stats[5] = _summarize(0, maxDets=20)
481 | stats[6] = _summarize(0, maxDets=20, iouThr=.5)
482 | stats[7] = _summarize(0, maxDets=20, iouThr=.75)
483 | stats[8] = _summarize(0, maxDets=20, areaRng='medium')
484 | stats[9] = _summarize(0, maxDets=20, areaRng='large')
485 | return stats
486 | if not self.eval:
487 | raise Exception('Please run accumulate() first')
488 | iouType = self.params.iouType
489 | if iouType == 'segm' or iouType == 'bbox':
490 | summarize = _summarizeDets
491 | elif iouType == 'keypoints':
492 | summarize = _summarizeKps
493 | self.stats = summarize()
494 |
495 | def __str__(self):
496 | self.summarize()
497 |
498 | class Params:
499 | '''
500 | Params for coco evaluation api
501 | '''
502 | def setDetParams(self):
503 | self.imgIds = []
504 | self.catIds = []
505 | # np.arange causes trouble. the data point on arange is slightly larger than the true value
506 | self.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)
507 | self.recThrs = np.linspace(.0, 1.00, int(np.round((1.00 - .0) / .01)) + 1, endpoint=True)
508 | self.maxDets = [1, 10, 100]
509 | self.areaRng = [[0 ** 2, 1e5 ** 2], [0 ** 2, 32 ** 2], [32 ** 2, 96 ** 2], [96 ** 2, 1e5 ** 2]]
510 | self.areaRngLbl = ['all', 'small', 'medium', 'large']
511 | self.useCats = 1
512 |
513 | def setKpParams(self):
514 | self.imgIds = []
515 | self.catIds = []
516 | # np.arange causes trouble. the data point on arange is slightly larger than the true value
517 | self.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)
518 | self.recThrs = np.linspace(.0, 1.00, int(np.round((1.00 - .0) / .01)) + 1, endpoint=True)
519 | self.maxDets = [20]
520 | self.areaRng = [[0 ** 2, 1e5 ** 2], [32 ** 2, 96 ** 2], [96 ** 2, 1e5 ** 2]]
521 | self.areaRngLbl = ['all', 'medium', 'large']
522 | self.useCats = 1
523 | self.kpt_oks_sigmas = np.array([.26, .25, .25, .35, .35, .79, .79, .72, .72, .62,.62, 1.07, 1.07, .87, .87, .89, .89])/10.0
524 |
525 | def __init__(self, iouType='segm'):
526 | if iouType == 'segm' or iouType == 'bbox':
527 | self.setDetParams()
528 | elif iouType == 'keypoints':
529 | self.setKpParams()
530 | else:
531 | raise Exception('iouType not supported')
532 | self.iouType = iouType
533 | # useSegm is deprecated
534 | self.useSegm = None
535 |
--------------------------------------------------------------------------------
/challenge_iccv_2025/pycocotools/mask.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tsungyi'
2 |
3 | import pycocotools._mask as _mask
4 |
5 | # Interface for manipulating masks stored in RLE format.
6 | #
7 | # RLE is a simple yet efficient format for storing binary masks. RLE
8 | # first divides a vector (or vectorized image) into a series of piecewise
9 | # constant regions and then for each piece simply stores the length of
10 | # that piece. For example, given M=[0 0 1 1 1 0 1] the RLE counts would
11 | # be [2 3 1 1], or for M=[1 1 1 1 1 1 0] the counts would be [0 6 1]
12 | # (note that the odd counts are always the numbers of zeros). Instead of
13 | # storing the counts directly, additional compression is achieved with a
14 | # variable bitrate representation based on a common scheme called LEB128.
15 | #
16 | # Compression is greatest given large piecewise constant regions.
17 | # Specifically, the size of the RLE is proportional to the number of
18 | # *boundaries* in M (or for an image the number of boundaries in the y
19 | # direction). Assuming fairly simple shapes, the RLE representation is
20 | # O(sqrt(n)) where n is number of pixels in the object. Hence space usage
21 | # is substantially lower, especially for large simple objects (large n).
22 | #
23 | # Many common operations on masks can be computed directly using the RLE
24 | # (without need for decoding). This includes computations such as area,
25 | # union, intersection, etc. All of these operations are linear in the
26 | # size of the RLE, in other words they are O(sqrt(n)) where n is the area
27 | # of the object. Computing these operations on the original mask is O(n).
28 | # Thus, using the RLE can result in substantial computational savings.
29 | #
30 | # The following API functions are defined:
31 | # encode - Encode binary masks using RLE.
32 | # decode - Decode binary masks encoded via RLE.
33 | # merge - Compute union or intersection of encoded masks.
34 | # iou - Compute intersection over union between masks.
35 | # area - Compute area of encoded masks.
36 | # toBbox - Get bounding boxes surrounding encoded masks.
37 | # frPyObjects - Convert polygon, bbox, and uncompressed RLE to encoded RLE mask.
38 | #
39 | # Usage:
40 | # Rs = encode( masks )
41 | # masks = decode( Rs )
42 | # R = merge( Rs, intersect=false )
43 | # o = iou( dt, gt, iscrowd )
44 | # a = area( Rs )
45 | # bbs = toBbox( Rs )
46 | # Rs = frPyObjects( [pyObjects], h, w )
47 | #
48 | # In the API the following formats are used:
49 | # Rs - [dict] Run-length encoding of binary masks
50 | # R - dict Run-length encoding of binary mask
51 | # masks - [hxwxn] Binary mask(s) (must have type np.ndarray(dtype=uint8) in column-major order)
52 | # iscrowd - [nx1] list of np.ndarray. 1 indicates corresponding gt image has crowd region to ignore
53 | # bbs - [nx4] Bounding box(es) stored as [x y w h]
54 | # poly - Polygon stored as [[x1 y1 x2 y2...],[x1 y1 ...],...] (2D list)
55 | # dt,gt - May be either bounding boxes or encoded masks
56 | # Both poly and bbs are 0-indexed (bbox=[0 0 1 1] encloses first pixel).
57 | #
58 | # Finally, a note about the intersection over union (iou) computation.
59 | # The standard iou of a ground truth (gt) and detected (dt) object is
60 | # iou(gt,dt) = area(intersect(gt,dt)) / area(union(gt,dt))
61 | # For "crowd" regions, we use a modified criteria. If a gt object is
62 | # marked as "iscrowd", we allow a dt to match any subregion of the gt.
63 | # Choosing gt' in the crowd gt that best matches the dt can be done using
64 | # gt'=intersect(dt,gt). Since by definition union(gt',dt)=dt, computing
65 | # iou(gt,dt,iscrowd) = iou(gt',dt) = area(intersect(gt,dt)) / area(dt)
66 | # For crowd gt regions we use this modified criteria above for the iou.
67 | #
68 | # To compile run "python setup.py build_ext --inplace"
69 | # Please do not contact us for help with compiling.
70 | #
71 | # Microsoft COCO Toolbox. version 2.0
72 | # Data, paper, and tutorials available at: http://mscoco.org/
73 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2015.
74 | # Licensed under the Simplified BSD License [see coco/license.txt]
75 |
76 | iou = _mask.iou
77 | merge = _mask.merge
78 | frPyObjects = _mask.frPyObjects
79 |
80 | def encode(bimask):
81 | if len(bimask.shape) == 3:
82 | return _mask.encode(bimask)
83 | elif len(bimask.shape) == 2:
84 | h, w = bimask.shape
85 | return _mask.encode(bimask.reshape((h, w, 1), order='F'))[0]
86 |
87 | def decode(rleObjs):
88 | if type(rleObjs) == list:
89 | return _mask.decode(rleObjs)
90 | else:
91 | return _mask.decode([rleObjs])[:,:,0]
92 |
93 | def area(rleObjs):
94 | if type(rleObjs) == list:
95 | return _mask.area(rleObjs)
96 | else:
97 | return _mask.area([rleObjs])[0]
98 |
99 | def toBbox(rleObjs):
100 | if type(rleObjs) == list:
101 | return _mask.toBbox(rleObjs)
102 | else:
103 | return _mask.toBbox([rleObjs])[0]
--------------------------------------------------------------------------------
/challenge_iccv_2025/run_evaluation_jetson.py:
--------------------------------------------------------------------------------
1 | import os
2 | import time
3 | import argparse
4 | import cv2
5 | import json
6 | import numpy as np
7 | from ultralytics import YOLO
8 |
9 | from utils import f1_score
10 |
11 | from utils import get_model , preprocess_image , postprocess_result, changeId
12 | import json
13 |
14 | def main():
15 | parser = argparse.ArgumentParser()
16 | parser.add_argument('--image_folder', type=str, default='/data/Fisheye1K_eval/images', help='Path to image folder')
17 | parser.add_argument('--model_path', type=str, default='models/yolo11n_fisheye8k.pt', help='Path to the model')
18 | parser.add_argument('--max_fps', type=float, default=25.0, help='Maximum FPS for evaluation')
19 | parser.add_argument('--output_json', type=str, default='/data/Fisheye1K_eval/predictions.json', help='Output JSON file for predictions')
20 | #parser.add_argument('--ground_truths_path', type=str, default='/data/Fisheye1K_eval/groundtruth.json', help='Path to ground truths JSON file')
21 | args = parser.parse_args()
22 |
23 | image_folder = args.image_folder
24 | model_path = args.model_path
25 |
26 | model = get_model(model_path)
27 |
28 | image_files = sorted([
29 | os.path.join(image_folder, f)
30 | for f in os.listdir(image_folder)
31 | if f.lower().endswith(('.jpg', '.jpeg', '.png'))
32 | ])
33 |
34 | print(f"Found {len(image_files)} images.")
35 |
36 | predictions = []
37 | print('Prediction started')
38 | start_time = time.time()
39 | for image_path in image_files:
40 | img = preprocess_image(image_path)
41 | results = model(img,verbose=False)
42 | results = postprocess_result(results) # [boxes, scores, classes]
43 | predictions.append((image_path,results))
44 | #print(f"Processed {os.path.basename(image_path)}: {len(results[0])} objects detected.")
45 |
46 | end_time = time.time()
47 | elapsed_time = end_time - start_time
48 | print(f"Processed {len(image_files)} images in {elapsed_time:.2f} seconds.")
49 |
50 | predictions_json = []
51 | # Save predictions to JSON
52 | with open(args.output_json, 'w') as f:
53 | json.dump(predictions_json, f, indent=2)
54 |
55 | fps = len(image_files) / elapsed_time
56 | normfps = min(fps, args.max_fps)/args.max_fps
57 |
58 | #f1 = f1_score(args.output_json, args.ground_truths_path)
59 | #harmonic_mean = 2 * f1 * normfps / (f1 + normfps)
60 |
61 | print(f"\n--- Evaluation Complete ---")
62 | print(f"Total inference time: {elapsed_time:.2f} seconds")
63 | print(f"FPS: {fps:.2f}")
64 | print(f"Normalized FPS: {normfps:.4f}")
65 | #print(f"F1-score: {f1:.4f}")
66 | #print(f"Metric (harmonic mean of F1-score and NormalizedFPS): {harmonic_mean:.4f}")
67 |
68 | if __name__ == "__main__":
69 | main()
70 |
--------------------------------------------------------------------------------
/challenge_iccv_2025/run_train_yolo.py:
--------------------------------------------------------------------------------
1 | from ultralytics import YOLO
2 |
3 | # Load a model
4 | model = YOLO("yolo11n.pt") # load a pretrained model (recommended for training)
5 |
6 | # Train the model
7 | results = model.train(data="data/Fisheye8k.yaml",
8 | epochs=100,
9 | batch =32,
10 | imgsz=640,
11 | device="0", # specify the device to use, e.g., "0" for GPU 0
12 | project="/models/", # specify the project directory to save the model
13 | name="yolo11n_fisheye8k", # specify the name of the model
14 | save_period=10, # save the model every 10 epochs
15 | save=True, # save the model after training
16 | ) # train the model with the specified parameters
--------------------------------------------------------------------------------
/challenge_iccv_2025/utils.py:
--------------------------------------------------------------------------------
1 | from ultralytics import YOLO
2 | import os
3 | import cv2
4 | import numpy as np
5 | from pycocotools.coco import COCO
6 | from pycocotools.cocoeval_modified import COCOeval
7 | import json
8 |
9 | def f1_score(predictions_path, ground_truths_path):
10 | coco_gt = COCO(ground_truths_path)
11 |
12 | gt_image_ids = coco_gt.getImgIds()
13 |
14 | with open(predictions_path, 'r') as f:
15 | detection_data = json.load(f)
16 | filtered_detection_data = [
17 | item for item in detection_data if item['image_id'] in gt_image_ids]
18 | with open('./temp.json', 'w') as f:
19 | json.dump(filtered_detection_data, f)
20 | coco_dt = coco_gt.loadRes('./temp.json')
21 | coco_eval = COCOeval(coco_gt, coco_dt, 'bbox')
22 | coco_eval.evaluate()
23 | coco_eval.accumulate()
24 | coco_eval.summarize()
25 |
26 | # Assuming the F1 score is at index 20 in the stats array
27 | return coco_eval.stats[20] # Return the F1 score from the evaluation stats
28 | # return 0.85 # Simulated constant value for demo purposes
29 |
30 | def get_model(model_path):
31 | if not os.path.exists(model_path):
32 | raise FileNotFoundError(f"Model file {model_path} does not exist.")
33 | # You need to implement your model here
34 | model = YOLO(model_path)
35 | model.export(format="engine") # creates 'yolo11n.engine'
36 | engine_path = model_path.replace(".pt",".engine")
37 | # Load the exported TensorRT model
38 | trt_model = YOLO(engine_path)
39 |
40 | return trt_model
41 | def preprocess_image(image_path):
42 | image = cv2.imread(image_path)
43 | if image is None:
44 | raise ValueError("Input image is None.")
45 | # Preprocess the image for your own model
46 |
47 | return image
48 | def postprocess_result(results):
49 | if not results or len(results) == 0:
50 | return []
51 | boxes = results[0].boxes.xyxy.cpu().numpy().tolist() # shape: (N, 4)
52 | scores = results[0].boxes.conf.cpu().numpy().tolist() # shape: (N,)
53 | classes = results[0].boxes.cls.cpu().numpy().tolist() # shape: (N,)
54 |
55 | return [boxes, scores, classes]
56 | def changeId(id):
57 | sceneList = ['M', 'A', 'E', 'N']
58 | cameraId = int(id.split('_')[0].split('camera')[1])
59 | sceneId = sceneList.index(id.split('_')[1])
60 | frameId = int(id.split('_')[2])
61 | imageId = int(str(cameraId)+str(sceneId)+str(frameId))
62 | return imageId
63 |
--------------------------------------------------------------------------------
/challenge_iccv_2025/yolo11n.pt:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/challenge_iccv_2025/yolo11n.pt
--------------------------------------------------------------------------------
/evaluation_Jetson/eval.py:
--------------------------------------------------------------------------------
1 | from pycocotools.coco import COCO
2 | from pycocotools.cocoeval import COCOeval
3 |
4 |
5 | coco_gt = COCO('groundtruth.json')
6 | coco_dt = coco_gt.loadRes(
7 | 'detection_dgx.json')
8 |
9 | print(type(coco_dt))
10 | print(type(coco_gt))
11 |
12 | coco_eval = COCOeval(coco_gt, coco_dt, 'bbox')
13 | coco_eval.evaluate()
14 | coco_eval.accumulate()
15 | coco_eval.summarize()
16 |
--------------------------------------------------------------------------------
/evaluation_Jetson/eval_f1.py:
--------------------------------------------------------------------------------
1 | from pycocotools.coco import COCO
2 | from pycocotools.cocoeval_modified import COCOeval
3 | import json
4 |
5 | coco_gt = COCO('groundtruth.json')
6 |
7 | gt_image_ids = coco_gt.getImgIds()
8 |
9 | print("Total images ", len(gt_image_ids))
10 |
11 |
12 | with open('detections.json', 'r') as f:
13 | detection_data = json.load(f)
14 |
15 | filtered_detection_data = [
16 | item for item in detection_data if item['image_id'] in gt_image_ids]
17 |
18 | with open('./temp.json', 'w') as f:
19 | json.dump(filtered_detection_data, f)
20 |
21 | coco_dt = coco_gt.loadRes('./temp.json')
22 | coco_eval = COCOeval(coco_gt, coco_dt, 'bbox')
23 | coco_eval.evaluate()
24 | coco_eval.accumulate()
25 | coco_eval.summarize()
26 | print('----------------------------------------')
27 | print('AP_0.5-0.95', coco_eval.stats[0])
28 | print('AP_0.5', coco_eval.stats[1])
29 | print('AP_S', coco_eval.stats[3])
30 | print('AP_M', coco_eval.stats[4])
31 | print('AP_L', coco_eval.stats[5])
32 | print('f1_score: ', coco_eval.stats[20])
33 | print('----------------------------------------')
34 |
--------------------------------------------------------------------------------
/evaluation_Jetson/pycocotools/__init__.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tylin'
2 |
--------------------------------------------------------------------------------
/evaluation_Jetson/pycocotools/_mask.cpython-310-aarch64-linux-gnu.so:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Jetson/pycocotools/_mask.cpython-310-aarch64-linux-gnu.so
--------------------------------------------------------------------------------
/evaluation_Jetson/pycocotools/_mask.pyx:
--------------------------------------------------------------------------------
1 | # distutils: language = c
2 | # distutils: sources = ../common/maskApi.c
3 |
4 | #**************************************************************************
5 | # Microsoft COCO Toolbox. version 2.0
6 | # Data, paper, and tutorials available at: http://mscoco.org/
7 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2015.
8 | # Licensed under the Simplified BSD License [see coco/license.txt]
9 | #**************************************************************************
10 |
11 | __author__ = 'tsungyi'
12 |
13 | import sys
14 | PYTHON_VERSION = sys.version_info[0]
15 |
16 | # import both Python-level and C-level symbols of Numpy
17 | # the API uses Numpy to interface C and Python
18 | import numpy as np
19 | cimport numpy as np
20 | from libc.stdlib cimport malloc, free
21 |
22 | # intialized Numpy. must do.
23 | np.import_array()
24 |
25 | # import numpy C function
26 | # we use PyArray_ENABLEFLAGS to make Numpy ndarray responsible to memoery management
27 | cdef extern from "numpy/arrayobject.h":
28 | void PyArray_ENABLEFLAGS(np.ndarray arr, int flags)
29 |
30 | # Declare the prototype of the C functions in MaskApi.h
31 | cdef extern from "maskApi.h":
32 | ctypedef unsigned int uint
33 | ctypedef unsigned long siz
34 | ctypedef unsigned char byte
35 | ctypedef double* BB
36 | ctypedef struct RLE:
37 | siz h,
38 | siz w,
39 | siz m,
40 | uint* cnts,
41 | void rlesInit( RLE **R, siz n )
42 | void rleEncode( RLE *R, const byte *M, siz h, siz w, siz n )
43 | void rleDecode( const RLE *R, byte *mask, siz n )
44 | void rleMerge( const RLE *R, RLE *M, siz n, int intersect )
45 | void rleArea( const RLE *R, siz n, uint *a )
46 | void rleIou( RLE *dt, RLE *gt, siz m, siz n, byte *iscrowd, double *o )
47 | void bbIou( BB dt, BB gt, siz m, siz n, byte *iscrowd, double *o )
48 | void rleToBbox( const RLE *R, BB bb, siz n )
49 | void rleFrBbox( RLE *R, const BB bb, siz h, siz w, siz n )
50 | void rleFrPoly( RLE *R, const double *xy, siz k, siz h, siz w )
51 | char* rleToString( const RLE *R )
52 | void rleFrString( RLE *R, char *s, siz h, siz w )
53 |
54 | # python class to wrap RLE array in C
55 | # the class handles the memory allocation and deallocation
56 | cdef class RLEs:
57 | cdef RLE *_R
58 | cdef siz _n
59 |
60 | def __cinit__(self, siz n =0):
61 | rlesInit(&self._R, n)
62 | self._n = n
63 |
64 | # free the RLE array here
65 | def __dealloc__(self):
66 | if self._R is not NULL:
67 | for i in range(self._n):
68 | free(self._R[i].cnts)
69 | free(self._R)
70 | def __getattr__(self, key):
71 | if key == 'n':
72 | return self._n
73 | raise AttributeError(key)
74 |
75 | # python class to wrap Mask array in C
76 | # the class handles the memory allocation and deallocation
77 | cdef class Masks:
78 | cdef byte *_mask
79 | cdef siz _h
80 | cdef siz _w
81 | cdef siz _n
82 |
83 | def __cinit__(self, h, w, n):
84 | self._mask = malloc(h*w*n* sizeof(byte))
85 | self._h = h
86 | self._w = w
87 | self._n = n
88 | # def __dealloc__(self):
89 | # the memory management of _mask has been passed to np.ndarray
90 | # it doesn't need to be freed here
91 |
92 | # called when passing into np.array() and return an np.ndarray in column-major order
93 | def __array__(self):
94 | cdef np.npy_intp shape[1]
95 | shape[0] = self._h*self._w*self._n
96 | # Create a 1D array, and reshape it to fortran/Matlab column-major array
97 | ndarray = np.PyArray_SimpleNewFromData(1, shape, np.NPY_UINT8, self._mask).reshape((self._h, self._w, self._n), order='F')
98 | # The _mask allocated by Masks is now handled by ndarray
99 | PyArray_ENABLEFLAGS(ndarray, np.NPY_OWNDATA)
100 | return ndarray
101 |
102 | # internal conversion from Python RLEs object to compressed RLE format
103 | def _toString(RLEs Rs):
104 | cdef siz n = Rs.n
105 | cdef bytes py_string
106 | cdef char* c_string
107 | objs = []
108 | for i in range(n):
109 | c_string = rleToString( &Rs._R[i] )
110 | py_string = c_string
111 | objs.append({
112 | 'size': [Rs._R[i].h, Rs._R[i].w],
113 | 'counts': py_string
114 | })
115 | free(c_string)
116 | return objs
117 |
118 | # internal conversion from compressed RLE format to Python RLEs object
119 | def _frString(rleObjs):
120 | cdef siz n = len(rleObjs)
121 | Rs = RLEs(n)
122 | cdef bytes py_string
123 | cdef char* c_string
124 | for i, obj in enumerate(rleObjs):
125 | if PYTHON_VERSION == 2:
126 | py_string = str(obj['counts']).encode('utf8')
127 | elif PYTHON_VERSION == 3:
128 | py_string = str.encode(obj['counts']) if type(obj['counts']) == str else obj['counts']
129 | else:
130 | raise Exception('Python version must be 2 or 3')
131 | c_string = py_string
132 | rleFrString( &Rs._R[i], c_string, obj['size'][0], obj['size'][1] )
133 | return Rs
134 |
135 | # encode mask to RLEs objects
136 | # list of RLE string can be generated by RLEs member function
137 | def encode(np.ndarray[np.uint8_t, ndim=3, mode='fortran'] mask):
138 | h, w, n = mask.shape[0], mask.shape[1], mask.shape[2]
139 | cdef RLEs Rs = RLEs(n)
140 | rleEncode(Rs._R,mask.data,h,w,n)
141 | objs = _toString(Rs)
142 | return objs
143 |
144 | # decode mask from compressed list of RLE string or RLEs object
145 | def decode(rleObjs):
146 | cdef RLEs Rs = _frString(rleObjs)
147 | h, w, n = Rs._R[0].h, Rs._R[0].w, Rs._n
148 | masks = Masks(h, w, n)
149 | rleDecode(Rs._R, masks._mask, n);
150 | return np.array(masks)
151 |
152 | def merge(rleObjs, intersect=0):
153 | cdef RLEs Rs = _frString(rleObjs)
154 | cdef RLEs R = RLEs(1)
155 | rleMerge(Rs._R, R._R, Rs._n, intersect)
156 | obj = _toString(R)[0]
157 | return obj
158 |
159 | def area(rleObjs):
160 | cdef RLEs Rs = _frString(rleObjs)
161 | cdef uint* _a = malloc(Rs._n* sizeof(uint))
162 | rleArea(Rs._R, Rs._n, _a)
163 | cdef np.npy_intp shape[1]
164 | shape[0] = Rs._n
165 | a = np.array((Rs._n, ), dtype=np.uint8)
166 | a = np.PyArray_SimpleNewFromData(1, shape, np.NPY_UINT32, _a)
167 | PyArray_ENABLEFLAGS(a, np.NPY_OWNDATA)
168 | return a
169 |
170 | # iou computation. support function overload (RLEs-RLEs and bbox-bbox).
171 | def iou( dt, gt, pyiscrowd ):
172 | def _preproc(objs):
173 | if len(objs) == 0:
174 | return objs
175 | if type(objs) == np.ndarray:
176 | if len(objs.shape) == 1:
177 | objs = objs.reshape((objs[0], 1))
178 | # check if it's Nx4 bbox
179 | if not len(objs.shape) == 2 or not objs.shape[1] == 4:
180 | raise Exception('numpy ndarray input is only for *bounding boxes* and should have Nx4 dimension')
181 | objs = objs.astype(np.double)
182 | elif type(objs) == list:
183 | # check if list is in box format and convert it to np.ndarray
184 | isbox = np.all(np.array([(len(obj)==4) and ((type(obj)==list) or (type(obj)==np.ndarray)) for obj in objs]))
185 | isrle = np.all(np.array([type(obj) == dict for obj in objs]))
186 | if isbox:
187 | objs = np.array(objs, dtype=np.double)
188 | if len(objs.shape) == 1:
189 | objs = objs.reshape((1,objs.shape[0]))
190 | elif isrle:
191 | objs = _frString(objs)
192 | else:
193 | raise Exception('list input can be bounding box (Nx4) or RLEs ([RLE])')
194 | else:
195 | raise Exception('unrecognized type. The following type: RLEs (rle), np.ndarray (box), and list (box) are supported.')
196 | return objs
197 | def _rleIou(RLEs dt, RLEs gt, np.ndarray[np.uint8_t, ndim=1] iscrowd, siz m, siz n, np.ndarray[np.double_t, ndim=1] _iou):
198 | rleIou( dt._R, gt._R, m, n, iscrowd.data, _iou.data )
199 | def _bbIou(np.ndarray[np.double_t, ndim=2] dt, np.ndarray[np.double_t, ndim=2] gt, np.ndarray[np.uint8_t, ndim=1] iscrowd, siz m, siz n, np.ndarray[np.double_t, ndim=1] _iou):
200 | bbIou( dt.data, gt.data, m, n, iscrowd.data, _iou.data )
201 | def _len(obj):
202 | cdef siz N = 0
203 | if type(obj) == RLEs:
204 | N = obj.n
205 | elif len(obj)==0:
206 | pass
207 | elif type(obj) == np.ndarray:
208 | N = obj.shape[0]
209 | return N
210 | # convert iscrowd to numpy array
211 | cdef np.ndarray[np.uint8_t, ndim=1] iscrowd = np.array(pyiscrowd, dtype=np.uint8)
212 | # simple type checking
213 | cdef siz m, n
214 | dt = _preproc(dt)
215 | gt = _preproc(gt)
216 | m = _len(dt)
217 | n = _len(gt)
218 | if m == 0 or n == 0:
219 | return []
220 | if not type(dt) == type(gt):
221 | raise Exception('The dt and gt should have the same data type, either RLEs, list or np.ndarray')
222 |
223 | # define local variables
224 | cdef double* _iou = 0
225 | cdef np.npy_intp shape[1]
226 | # check type and assign iou function
227 | if type(dt) == RLEs:
228 | _iouFun = _rleIou
229 | elif type(dt) == np.ndarray:
230 | _iouFun = _bbIou
231 | else:
232 | raise Exception('input data type not allowed.')
233 | _iou = malloc(m*n* sizeof(double))
234 | iou = np.zeros((m*n, ), dtype=np.double)
235 | shape[0] = m*n
236 | iou = np.PyArray_SimpleNewFromData(1, shape, np.NPY_DOUBLE, _iou)
237 | PyArray_ENABLEFLAGS(iou, np.NPY_OWNDATA)
238 | _iouFun(dt, gt, iscrowd, m, n, iou)
239 | return iou.reshape((m,n), order='F')
240 |
241 | def toBbox( rleObjs ):
242 | cdef RLEs Rs = _frString(rleObjs)
243 | cdef siz n = Rs.n
244 | cdef BB _bb = malloc(4*n* sizeof(double))
245 | rleToBbox( Rs._R, _bb, n )
246 | cdef np.npy_intp shape[1]
247 | shape[0] = 4*n
248 | bb = np.array((1,4*n), dtype=np.double)
249 | bb = np.PyArray_SimpleNewFromData(1, shape, np.NPY_DOUBLE, _bb).reshape((n, 4))
250 | PyArray_ENABLEFLAGS(bb, np.NPY_OWNDATA)
251 | return bb
252 |
253 | def frBbox(np.ndarray[np.double_t, ndim=2] bb, siz h, siz w ):
254 | cdef siz n = bb.shape[0]
255 | Rs = RLEs(n)
256 | rleFrBbox( Rs._R, bb.data, h, w, n )
257 | objs = _toString(Rs)
258 | return objs
259 |
260 | def frPoly( poly, siz h, siz w ):
261 | cdef np.ndarray[np.double_t, ndim=1] np_poly
262 | n = len(poly)
263 | Rs = RLEs(n)
264 | for i, p in enumerate(poly):
265 | np_poly = np.array(p, dtype=np.double, order='F')
266 | rleFrPoly( &Rs._R[i], np_poly.data, int(len(p)/2), h, w )
267 | objs = _toString(Rs)
268 | return objs
269 |
270 | def frUncompressedRLE(ucRles, siz h, siz w):
271 | cdef np.ndarray[np.uint32_t, ndim=1] cnts
272 | cdef RLE R
273 | cdef uint *data
274 | n = len(ucRles)
275 | objs = []
276 | for i in range(n):
277 | Rs = RLEs(1)
278 | cnts = np.array(ucRles[i]['counts'], dtype=np.uint32)
279 | # time for malloc can be saved here but it's fine
280 | data = malloc(len(cnts)* sizeof(uint))
281 | for j in range(len(cnts)):
282 | data[j] = cnts[j]
283 | R = RLE(ucRles[i]['size'][0], ucRles[i]['size'][1], len(cnts), data)
284 | Rs._R[0] = R
285 | objs.append(_toString(Rs)[0])
286 | return objs
287 |
288 | def frPyObjects(pyobj, h, w):
289 | # encode rle from a list of python objects
290 | if type(pyobj) == np.ndarray:
291 | objs = frBbox(pyobj, h, w)
292 | elif type(pyobj) == list and len(pyobj[0]) == 4:
293 | objs = frBbox(pyobj, h, w)
294 | elif type(pyobj) == list and len(pyobj[0]) > 4:
295 | objs = frPoly(pyobj, h, w)
296 | elif type(pyobj) == list and type(pyobj[0]) == dict \
297 | and 'counts' in pyobj[0] and 'size' in pyobj[0]:
298 | objs = frUncompressedRLE(pyobj, h, w)
299 | # encode rle from single python object
300 | elif type(pyobj) == list and len(pyobj) == 4:
301 | objs = frBbox([pyobj], h, w)[0]
302 | elif type(pyobj) == list and len(pyobj) > 4:
303 | objs = frPoly([pyobj], h, w)[0]
304 | elif type(pyobj) == dict and 'counts' in pyobj and 'size' in pyobj:
305 | objs = frUncompressedRLE([pyobj], h, w)[0]
306 | else:
307 | raise Exception('input type is not supported.')
308 | return objs
309 |
--------------------------------------------------------------------------------
/evaluation_Jetson/pycocotools/coco.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tylin'
2 | __version__ = '2.0'
3 | # Interface for accessing the Microsoft COCO dataset.
4 |
5 | # Microsoft COCO is a large image dataset designed for object detection,
6 | # segmentation, and caption generation. pycocotools is a Python API that
7 | # assists in loading, parsing and visualizing the annotations in COCO.
8 | # Please visit http://mscoco.org/ for more information on COCO, including
9 | # for the data, paper, and tutorials. The exact format of the annotations
10 | # is also described on the COCO website. For example usage of the pycocotools
11 | # please see pycocotools_demo.ipynb. In addition to this API, please download both
12 | # the COCO images and annotations in order to run the demo.
13 |
14 | # An alternative to using the API is to load the annotations directly
15 | # into Python dictionary
16 | # Using the API provides additional utility functions. Note that this API
17 | # supports both *instance* and *caption* annotations. In the case of
18 | # captions not all functions are defined (e.g. categories are undefined).
19 |
20 | # The following API functions are defined:
21 | # COCO - COCO api class that loads COCO annotation file and prepare data structures.
22 | # decodeMask - Decode binary mask M encoded via run-length encoding.
23 | # encodeMask - Encode binary mask M using run-length encoding.
24 | # getAnnIds - Get ann ids that satisfy given filter conditions.
25 | # getCatIds - Get cat ids that satisfy given filter conditions.
26 | # getImgIds - Get img ids that satisfy given filter conditions.
27 | # loadAnns - Load anns with the specified ids.
28 | # loadCats - Load cats with the specified ids.
29 | # loadImgs - Load imgs with the specified ids.
30 | # annToMask - Convert segmentation in an annotation to binary mask.
31 | # showAnns - Display the specified annotations.
32 | # loadRes - Load algorithm results and create API for accessing them.
33 | # download - Download COCO images from mscoco.org server.
34 | # Throughout the API "ann"=annotation, "cat"=category, and "img"=image.
35 | # Help on each functions can be accessed by: "help COCO>function".
36 |
37 | # See also COCO>decodeMask,
38 | # COCO>encodeMask, COCO>getAnnIds, COCO>getCatIds,
39 | # COCO>getImgIds, COCO>loadAnns, COCO>loadCats,
40 | # COCO>loadImgs, COCO>annToMask, COCO>showAnns
41 |
42 | # Microsoft COCO Toolbox. version 2.0
43 | # Data, paper, and tutorials available at: http://mscoco.org/
44 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2014.
45 | # Licensed under the Simplified BSD License [see bsd.txt]
46 |
47 | import json
48 | import time
49 | import numpy as np
50 | import copy
51 | import itertools
52 | from . import mask as maskUtils
53 | import os
54 | from collections import defaultdict
55 | import sys
56 | PYTHON_VERSION = sys.version_info[0]
57 | if PYTHON_VERSION == 2:
58 | from urllib import urlretrieve
59 | elif PYTHON_VERSION == 3:
60 | from urllib.request import urlretrieve
61 |
62 |
63 | def _isArrayLike(obj):
64 | return hasattr(obj, '__iter__') and hasattr(obj, '__len__')
65 |
66 |
67 | class COCO:
68 | def __init__(self, annotation_file=None):
69 | """
70 | Constructor of Microsoft COCO helper class for reading and visualizing annotations.
71 | :param annotation_file (str): location of annotation file
72 | :param image_folder (str): location to the folder that hosts images.
73 | :return:
74 | """
75 | # load dataset
76 | self.dataset,self.anns,self.cats,self.imgs = dict(),dict(),dict(),dict()
77 | self.imgToAnns, self.catToImgs = defaultdict(list), defaultdict(list)
78 | if not annotation_file == None:
79 | print('loading annotations into memory...')
80 | tic = time.time()
81 | with open(annotation_file, 'r') as f:
82 | dataset = json.load(f)
83 | assert type(dataset)==dict, 'annotation file format {} not supported'.format(type(dataset))
84 | print('Done (t={:0.2f}s)'.format(time.time()- tic))
85 | self.dataset = dataset
86 | self.createIndex()
87 |
88 | def createIndex(self):
89 | # create index
90 | print('creating index...')
91 | anns, cats, imgs = {}, {}, {}
92 | imgToAnns,catToImgs = defaultdict(list),defaultdict(list)
93 | if 'annotations' in self.dataset:
94 | for ann in self.dataset['annotations']:
95 | imgToAnns[ann['image_id']].append(ann)
96 | anns[ann['id']] = ann
97 |
98 | if 'images' in self.dataset:
99 | for img in self.dataset['images']:
100 | imgs[img['id']] = img
101 |
102 | if 'categories' in self.dataset:
103 | for cat in self.dataset['categories']:
104 | cats[cat['id']] = cat
105 |
106 | if 'annotations' in self.dataset and 'categories' in self.dataset:
107 | for ann in self.dataset['annotations']:
108 | catToImgs[ann['category_id']].append(ann['image_id'])
109 |
110 | print('index created!')
111 |
112 | # create class members
113 | self.anns = anns
114 | self.imgToAnns = imgToAnns
115 | self.catToImgs = catToImgs
116 | self.imgs = imgs
117 | self.cats = cats
118 |
119 | def info(self):
120 | """
121 | Print information about the annotation file.
122 | :return:
123 | """
124 | for key, value in self.dataset['info'].items():
125 | print('{}: {}'.format(key, value))
126 |
127 | def getAnnIds(self, imgIds=[], catIds=[], areaRng=[], iscrowd=None):
128 | """
129 | Get ann ids that satisfy given filter conditions. default skips that filter
130 | :param imgIds (int array) : get anns for given imgs
131 | catIds (int array) : get anns for given cats
132 | areaRng (float array) : get anns for given area range (e.g. [0 inf])
133 | iscrowd (boolean) : get anns for given crowd label (False or True)
134 | :return: ids (int array) : integer array of ann ids
135 | """
136 | imgIds = imgIds if _isArrayLike(imgIds) else [imgIds]
137 | catIds = catIds if _isArrayLike(catIds) else [catIds]
138 |
139 | if len(imgIds) == len(catIds) == len(areaRng) == 0:
140 | anns = self.dataset['annotations']
141 | else:
142 | if not len(imgIds) == 0:
143 | lists = [self.imgToAnns[imgId] for imgId in imgIds if imgId in self.imgToAnns]
144 | anns = list(itertools.chain.from_iterable(lists))
145 | else:
146 | anns = self.dataset['annotations']
147 | anns = anns if len(catIds) == 0 else [ann for ann in anns if ann['category_id'] in catIds]
148 | anns = anns if len(areaRng) == 0 else [ann for ann in anns if ann['area'] > areaRng[0] and ann['area'] < areaRng[1]]
149 | if not iscrowd == None:
150 | ids = [ann['id'] for ann in anns if ann['iscrowd'] == iscrowd]
151 | else:
152 | ids = [ann['id'] for ann in anns]
153 | return ids
154 |
155 | def getCatIds(self, catNms=[], supNms=[], catIds=[]):
156 | """
157 | filtering parameters. default skips that filter.
158 | :param catNms (str array) : get cats for given cat names
159 | :param supNms (str array) : get cats for given supercategory names
160 | :param catIds (int array) : get cats for given cat ids
161 | :return: ids (int array) : integer array of cat ids
162 | """
163 | catNms = catNms if _isArrayLike(catNms) else [catNms]
164 | supNms = supNms if _isArrayLike(supNms) else [supNms]
165 | catIds = catIds if _isArrayLike(catIds) else [catIds]
166 |
167 | if len(catNms) == len(supNms) == len(catIds) == 0:
168 | cats = self.dataset['categories']
169 | else:
170 | cats = self.dataset['categories']
171 | cats = cats if len(catNms) == 0 else [cat for cat in cats if cat['name'] in catNms]
172 | cats = cats if len(supNms) == 0 else [cat for cat in cats if cat['supercategory'] in supNms]
173 | cats = cats if len(catIds) == 0 else [cat for cat in cats if cat['id'] in catIds]
174 | ids = [cat['id'] for cat in cats]
175 | return ids
176 |
177 | def getImgIds(self, imgIds=[], catIds=[]):
178 | '''
179 | Get img ids that satisfy given filter conditions.
180 | :param imgIds (int array) : get imgs for given ids
181 | :param catIds (int array) : get imgs with all given cats
182 | :return: ids (int array) : integer array of img ids
183 | '''
184 | imgIds = imgIds if _isArrayLike(imgIds) else [imgIds]
185 | catIds = catIds if _isArrayLike(catIds) else [catIds]
186 |
187 | if len(imgIds) == len(catIds) == 0:
188 | ids = self.imgs.keys()
189 | else:
190 | ids = set(imgIds)
191 | for i, catId in enumerate(catIds):
192 | if i == 0 and len(ids) == 0:
193 | ids = set(self.catToImgs[catId])
194 | else:
195 | ids &= set(self.catToImgs[catId])
196 | return list(ids)
197 |
198 | def loadAnns(self, ids=[]):
199 | """
200 | Load anns with the specified ids.
201 | :param ids (int array) : integer ids specifying anns
202 | :return: anns (object array) : loaded ann objects
203 | """
204 | if _isArrayLike(ids):
205 | return [self.anns[id] for id in ids]
206 | elif type(ids) == int:
207 | return [self.anns[ids]]
208 |
209 | def loadCats(self, ids=[]):
210 | """
211 | Load cats with the specified ids.
212 | :param ids (int array) : integer ids specifying cats
213 | :return: cats (object array) : loaded cat objects
214 | """
215 | if _isArrayLike(ids):
216 | return [self.cats[id] for id in ids]
217 | elif type(ids) == int:
218 | return [self.cats[ids]]
219 |
220 | def loadImgs(self, ids=[]):
221 | """
222 | Load anns with the specified ids.
223 | :param ids (int array) : integer ids specifying img
224 | :return: imgs (object array) : loaded img objects
225 | """
226 | if _isArrayLike(ids):
227 | return [self.imgs[id] for id in ids]
228 | elif type(ids) == int:
229 | return [self.imgs[ids]]
230 |
231 | def showAnns(self, anns, draw_bbox=False):
232 | """
233 | Display the specified annotations.
234 | :param anns (array of object): annotations to display
235 | :return: None
236 | """
237 | if len(anns) == 0:
238 | return 0
239 | if 'segmentation' in anns[0] or 'keypoints' in anns[0]:
240 | datasetType = 'instances'
241 | elif 'caption' in anns[0]:
242 | datasetType = 'captions'
243 | else:
244 | raise Exception('datasetType not supported')
245 | if datasetType == 'instances':
246 | import matplotlib.pyplot as plt
247 | from matplotlib.collections import PatchCollection
248 | from matplotlib.patches import Polygon
249 |
250 | ax = plt.gca()
251 | ax.set_autoscale_on(False)
252 | polygons = []
253 | color = []
254 | for ann in anns:
255 | c = (np.random.random((1, 3))*0.6+0.4).tolist()[0]
256 | if 'segmentation' in ann:
257 | if type(ann['segmentation']) == list:
258 | # polygon
259 | for seg in ann['segmentation']:
260 | poly = np.array(seg).reshape((int(len(seg)/2), 2))
261 | polygons.append(Polygon(poly))
262 | color.append(c)
263 | else:
264 | # mask
265 | t = self.imgs[ann['image_id']]
266 | if type(ann['segmentation']['counts']) == list:
267 | rle = maskUtils.frPyObjects([ann['segmentation']], t['height'], t['width'])
268 | else:
269 | rle = [ann['segmentation']]
270 | m = maskUtils.decode(rle)
271 | img = np.ones( (m.shape[0], m.shape[1], 3) )
272 | if ann['iscrowd'] == 1:
273 | color_mask = np.array([2.0,166.0,101.0])/255
274 | if ann['iscrowd'] == 0:
275 | color_mask = np.random.random((1, 3)).tolist()[0]
276 | for i in range(3):
277 | img[:,:,i] = color_mask[i]
278 | ax.imshow(np.dstack( (img, m*0.5) ))
279 | if 'keypoints' in ann and type(ann['keypoints']) == list:
280 | # turn skeleton into zero-based index
281 | sks = np.array(self.loadCats(ann['category_id'])[0]['skeleton'])-1
282 | kp = np.array(ann['keypoints'])
283 | x = kp[0::3]
284 | y = kp[1::3]
285 | v = kp[2::3]
286 | for sk in sks:
287 | if np.all(v[sk]>0):
288 | plt.plot(x[sk],y[sk], linewidth=3, color=c)
289 | plt.plot(x[v>0], y[v>0],'o',markersize=8, markerfacecolor=c, markeredgecolor='k',markeredgewidth=2)
290 | plt.plot(x[v>1], y[v>1],'o',markersize=8, markerfacecolor=c, markeredgecolor=c, markeredgewidth=2)
291 |
292 | if draw_bbox:
293 | [bbox_x, bbox_y, bbox_w, bbox_h] = ann['bbox']
294 | poly = [[bbox_x, bbox_y], [bbox_x, bbox_y+bbox_h], [bbox_x+bbox_w, bbox_y+bbox_h], [bbox_x+bbox_w, bbox_y]]
295 | np_poly = np.array(poly).reshape((4,2))
296 | polygons.append(Polygon(np_poly))
297 | color.append(c)
298 |
299 | p = PatchCollection(polygons, facecolor=color, linewidths=0, alpha=0.4)
300 | ax.add_collection(p)
301 | p = PatchCollection(polygons, facecolor='none', edgecolors=color, linewidths=2)
302 | ax.add_collection(p)
303 | elif datasetType == 'captions':
304 | for ann in anns:
305 | print(ann['caption'])
306 |
307 | def loadRes(self, resFile):
308 | """
309 | Load result file and return a result api object.
310 | :param resFile (str) : file name of result file
311 | :return: res (obj) : result api object
312 | """
313 | res = COCO()
314 | res.dataset['images'] = [img for img in self.dataset['images']]
315 |
316 | print('Loading and preparing results...')
317 | tic = time.time()
318 | if type(resFile) == str or (PYTHON_VERSION == 2 and type(resFile) == unicode):
319 | with open(resFile) as f:
320 | anns = json.load(f)
321 | elif type(resFile) == np.ndarray:
322 | anns = self.loadNumpyAnnotations(resFile)
323 | else:
324 | anns = resFile
325 | assert type(anns) == list, 'results in not an array of objects'
326 | annsImgIds = [ann['image_id'] for ann in anns]
327 | assert set(annsImgIds) == (set(annsImgIds) & set(self.getImgIds())), \
328 | 'Results do not correspond to current coco set'
329 | if 'caption' in anns[0]:
330 | imgIds = set([img['id'] for img in res.dataset['images']]) & set([ann['image_id'] for ann in anns])
331 | res.dataset['images'] = [img for img in res.dataset['images'] if img['id'] in imgIds]
332 | for id, ann in enumerate(anns):
333 | ann['id'] = id+1
334 | elif 'bbox' in anns[0] and not anns[0]['bbox'] == []:
335 | res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])
336 | for id, ann in enumerate(anns):
337 | bb = ann['bbox']
338 | x1, x2, y1, y2 = [bb[0], bb[0]+bb[2], bb[1], bb[1]+bb[3]]
339 | if not 'segmentation' in ann:
340 | ann['segmentation'] = [[x1, y1, x1, y2, x2, y2, x2, y1]]
341 | ann['area'] = bb[2]*bb[3]
342 | ann['id'] = id+1
343 | ann['iscrowd'] = 0
344 | elif 'segmentation' in anns[0]:
345 | res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])
346 | for id, ann in enumerate(anns):
347 | # now only support compressed RLE format as segmentation results
348 | ann['area'] = maskUtils.area(ann['segmentation'])
349 | if not 'bbox' in ann:
350 | ann['bbox'] = maskUtils.toBbox(ann['segmentation'])
351 | ann['id'] = id+1
352 | ann['iscrowd'] = 0
353 | elif 'keypoints' in anns[0]:
354 | res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])
355 | for id, ann in enumerate(anns):
356 | s = ann['keypoints']
357 | x = s[0::3]
358 | y = s[1::3]
359 | x0,x1,y0,y1 = np.min(x), np.max(x), np.min(y), np.max(y)
360 | ann['area'] = (x1-x0)*(y1-y0)
361 | ann['id'] = id + 1
362 | ann['bbox'] = [x0,y0,x1-x0,y1-y0]
363 | print('DONE (t={:0.2f}s)'.format(time.time()- tic))
364 |
365 | res.dataset['annotations'] = anns
366 | res.createIndex()
367 | return res
368 |
369 | def download(self, tarDir = None, imgIds = [] ):
370 | '''
371 | Download COCO images from mscoco.org server.
372 | :param tarDir (str): COCO results directory name
373 | imgIds (list): images to be downloaded
374 | :return:
375 | '''
376 | if tarDir is None:
377 | print('Please specify target directory')
378 | return -1
379 | if len(imgIds) == 0:
380 | imgs = self.imgs.values()
381 | else:
382 | imgs = self.loadImgs(imgIds)
383 | N = len(imgs)
384 | if not os.path.exists(tarDir):
385 | os.makedirs(tarDir)
386 | for i, img in enumerate(imgs):
387 | tic = time.time()
388 | fname = os.path.join(tarDir, img['file_name'])
389 | if not os.path.exists(fname):
390 | urlretrieve(img['coco_url'], fname)
391 | print('downloaded {}/{} images (t={:0.1f}s)'.format(i, N, time.time()- tic))
392 |
393 | def loadNumpyAnnotations(self, data):
394 | """
395 | Convert result data from a numpy array [Nx7] where each row contains {imageID,x1,y1,w,h,score,class}
396 | :param data (numpy.ndarray)
397 | :return: annotations (python nested list)
398 | """
399 | print('Converting ndarray to lists...')
400 | assert(type(data) == np.ndarray)
401 | print(data.shape)
402 | assert(data.shape[1] == 7)
403 | N = data.shape[0]
404 | ann = []
405 | for i in range(N):
406 | if i % 1000000 == 0:
407 | print('{}/{}'.format(i,N))
408 | ann += [{
409 | 'image_id' : int(data[i, 0]),
410 | 'bbox' : [ data[i, 1], data[i, 2], data[i, 3], data[i, 4] ],
411 | 'score' : data[i, 5],
412 | 'category_id': int(data[i, 6]),
413 | }]
414 | return ann
415 |
416 | def annToRLE(self, ann):
417 | """
418 | Convert annotation which can be polygons, uncompressed RLE to RLE.
419 | :return: binary mask (numpy 2D array)
420 | """
421 | t = self.imgs[ann['image_id']]
422 | h, w = t['height'], t['width']
423 | segm = ann['segmentation']
424 | if type(segm) == list:
425 | # polygon -- a single object might consist of multiple parts
426 | # we merge all parts into one mask rle code
427 | rles = maskUtils.frPyObjects(segm, h, w)
428 | rle = maskUtils.merge(rles)
429 | elif type(segm['counts']) == list:
430 | # uncompressed RLE
431 | rle = maskUtils.frPyObjects(segm, h, w)
432 | else:
433 | # rle
434 | rle = ann['segmentation']
435 | return rle
436 |
437 | def annToMask(self, ann):
438 | """
439 | Convert annotation which can be polygons, uncompressed RLE, or RLE to binary mask.
440 | :return: binary mask (numpy 2D array)
441 | """
442 | rle = self.annToRLE(ann)
443 | m = maskUtils.decode(rle)
444 | return m
445 |
--------------------------------------------------------------------------------
/evaluation_Jetson/pycocotools/cocoeval.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tsungyi'
2 |
3 | import numpy as np
4 | import datetime
5 | import time
6 | from collections import defaultdict
7 | from . import mask as maskUtils
8 | import copy
9 |
10 | class COCOeval:
11 | # Interface for evaluating detection on the Microsoft COCO dataset.
12 | #
13 | # The usage for CocoEval is as follows:
14 | # cocoGt=..., cocoDt=... # load dataset and results
15 | # E = CocoEval(cocoGt,cocoDt); # initialize CocoEval object
16 | # E.params.recThrs = ...; # set parameters as desired
17 | # E.evaluate(); # run per image evaluation
18 | # E.accumulate(); # accumulate per image results
19 | # E.summarize(); # display summary metrics of results
20 | # For example usage see evalDemo.m and http://mscoco.org/.
21 | #
22 | # The evaluation parameters are as follows (defaults in brackets):
23 | # imgIds - [all] N img ids to use for evaluation
24 | # catIds - [all] K cat ids to use for evaluation
25 | # iouThrs - [.5:.05:.95] T=10 IoU thresholds for evaluation
26 | # recThrs - [0:.01:1] R=101 recall thresholds for evaluation
27 | # areaRng - [...] A=4 object area ranges for evaluation
28 | # maxDets - [1 10 100] M=3 thresholds on max detections per image
29 | # iouType - ['segm'] set iouType to 'segm', 'bbox' or 'keypoints'
30 | # iouType replaced the now DEPRECATED useSegm parameter.
31 | # useCats - [1] if true use category labels for evaluation
32 | # Note: if useCats=0 category labels are ignored as in proposal scoring.
33 | # Note: multiple areaRngs [Ax2] and maxDets [Mx1] can be specified.
34 | #
35 | # evaluate(): evaluates detections on every image and every category and
36 | # concats the results into the "evalImgs" with fields:
37 | # dtIds - [1xD] id for each of the D detections (dt)
38 | # gtIds - [1xG] id for each of the G ground truths (gt)
39 | # dtMatches - [TxD] matching gt id at each IoU or 0
40 | # gtMatches - [TxG] matching dt id at each IoU or 0
41 | # dtScores - [1xD] confidence of each dt
42 | # gtIgnore - [1xG] ignore flag for each gt
43 | # dtIgnore - [TxD] ignore flag for each dt at each IoU
44 | #
45 | # accumulate(): accumulates the per-image, per-category evaluation
46 | # results in "evalImgs" into the dictionary "eval" with fields:
47 | # params - parameters used for evaluation
48 | # date - date evaluation was performed
49 | # counts - [T,R,K,A,M] parameter dimensions (see above)
50 | # precision - [TxRxKxAxM] precision for every evaluation setting
51 | # recall - [TxKxAxM] max recall for every evaluation setting
52 | # Note: precision and recall==-1 for settings with no gt objects.
53 | #
54 | # See also coco, mask, pycocoDemo, pycocoEvalDemo
55 | #
56 | # Microsoft COCO Toolbox. version 2.0
57 | # Data, paper, and tutorials available at: http://mscoco.org/
58 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2015.
59 | # Licensed under the Simplified BSD License [see coco/license.txt]
60 | def __init__(self, cocoGt=None, cocoDt=None, iouType='segm'):
61 | '''
62 | Initialize CocoEval using coco APIs for gt and dt
63 | :param cocoGt: coco object with ground truth annotations
64 | :param cocoDt: coco object with detection results
65 | :return: None
66 | '''
67 | if not iouType:
68 | print('iouType not specified. use default iouType segm')
69 | self.cocoGt = cocoGt # ground truth COCO API
70 | self.cocoDt = cocoDt # detections COCO API
71 | self.evalImgs = defaultdict(list) # per-image per-category evaluation results [KxAxI] elements
72 | self.eval = {} # accumulated evaluation results
73 | self._gts = defaultdict(list) # gt for evaluation
74 | self._dts = defaultdict(list) # dt for evaluation
75 | self.params = Params(iouType=iouType) # parameters
76 | self._paramsEval = {} # parameters for evaluation
77 | self.stats = [] # result summarization
78 | self.ious = {} # ious between all gts and dts
79 | if not cocoGt is None:
80 | self.params.imgIds = sorted(cocoGt.getImgIds())
81 | self.params.catIds = sorted(cocoGt.getCatIds())
82 |
83 |
84 | def _prepare(self):
85 | '''
86 | Prepare ._gts and ._dts for evaluation based on params
87 | :return: None
88 | '''
89 | def _toMask(anns, coco):
90 | # modify ann['segmentation'] by reference
91 | for ann in anns:
92 | rle = coco.annToRLE(ann)
93 | ann['segmentation'] = rle
94 | p = self.params
95 | if p.useCats:
96 | gts=self.cocoGt.loadAnns(self.cocoGt.getAnnIds(imgIds=p.imgIds, catIds=p.catIds))
97 | dts=self.cocoDt.loadAnns(self.cocoDt.getAnnIds(imgIds=p.imgIds, catIds=p.catIds))
98 | else:
99 | gts=self.cocoGt.loadAnns(self.cocoGt.getAnnIds(imgIds=p.imgIds))
100 | dts=self.cocoDt.loadAnns(self.cocoDt.getAnnIds(imgIds=p.imgIds))
101 |
102 | # convert ground truth to mask if iouType == 'segm'
103 | if p.iouType == 'segm':
104 | _toMask(gts, self.cocoGt)
105 | _toMask(dts, self.cocoDt)
106 | # set ignore flag
107 | for gt in gts:
108 | gt['ignore'] = gt['ignore'] if 'ignore' in gt else 0
109 | gt['ignore'] = 'iscrowd' in gt and gt['iscrowd']
110 | if p.iouType == 'keypoints':
111 | gt['ignore'] = (gt['num_keypoints'] == 0) or gt['ignore']
112 | self._gts = defaultdict(list) # gt for evaluation
113 | self._dts = defaultdict(list) # dt for evaluation
114 | for gt in gts:
115 | self._gts[gt['image_id'], gt['category_id']].append(gt)
116 | for dt in dts:
117 | self._dts[dt['image_id'], dt['category_id']].append(dt)
118 | self.evalImgs = defaultdict(list) # per-image per-category evaluation results
119 | self.eval = {} # accumulated evaluation results
120 |
121 | def evaluate(self):
122 | '''
123 | Run per image evaluation on given images and store results (a list of dict) in self.evalImgs
124 | :return: None
125 | '''
126 | tic = time.time()
127 | print('Running per image evaluation...')
128 | p = self.params
129 | # add backward compatibility if useSegm is specified in params
130 | if not p.useSegm is None:
131 | p.iouType = 'segm' if p.useSegm == 1 else 'bbox'
132 | print('useSegm (deprecated) is not None. Running {} evaluation'.format(p.iouType))
133 | print('Evaluate annotation type *{}*'.format(p.iouType))
134 | p.imgIds = list(np.unique(p.imgIds))
135 | if p.useCats:
136 | p.catIds = list(np.unique(p.catIds))
137 | p.maxDets = sorted(p.maxDets)
138 | self.params=p
139 |
140 | self._prepare()
141 | # loop through images, area range, max detection number
142 | catIds = p.catIds if p.useCats else [-1]
143 |
144 | if p.iouType == 'segm' or p.iouType == 'bbox':
145 | computeIoU = self.computeIoU
146 | elif p.iouType == 'keypoints':
147 | computeIoU = self.computeOks
148 | self.ious = {(imgId, catId): computeIoU(imgId, catId) \
149 | for imgId in p.imgIds
150 | for catId in catIds}
151 |
152 | evaluateImg = self.evaluateImg
153 | maxDet = p.maxDets[-1]
154 | self.evalImgs = [evaluateImg(imgId, catId, areaRng, maxDet)
155 | for catId in catIds
156 | for areaRng in p.areaRng
157 | for imgId in p.imgIds
158 | ]
159 | self._paramsEval = copy.deepcopy(self.params)
160 | toc = time.time()
161 | print('DONE (t={:0.2f}s).'.format(toc-tic))
162 |
163 | def computeIoU(self, imgId, catId):
164 | p = self.params
165 | if p.useCats:
166 | gt = self._gts[imgId,catId]
167 | dt = self._dts[imgId,catId]
168 | else:
169 | gt = [_ for cId in p.catIds for _ in self._gts[imgId,cId]]
170 | dt = [_ for cId in p.catIds for _ in self._dts[imgId,cId]]
171 | if len(gt) == 0 and len(dt) ==0:
172 | return []
173 | inds = np.argsort([-d['score'] for d in dt], kind='mergesort')
174 | dt = [dt[i] for i in inds]
175 | if len(dt) > p.maxDets[-1]:
176 | dt=dt[0:p.maxDets[-1]]
177 |
178 | if p.iouType == 'segm':
179 | g = [g['segmentation'] for g in gt]
180 | d = [d['segmentation'] for d in dt]
181 | elif p.iouType == 'bbox':
182 | g = [g['bbox'] for g in gt]
183 | d = [d['bbox'] for d in dt]
184 | else:
185 | raise Exception('unknown iouType for iou computation')
186 |
187 | # compute iou between each dt and gt region
188 | iscrowd = [int(o['iscrowd']) for o in gt]
189 | ious = maskUtils.iou(d,g,iscrowd)
190 | return ious
191 |
192 | def computeOks(self, imgId, catId):
193 | p = self.params
194 | # dimention here should be Nxm
195 | gts = self._gts[imgId, catId]
196 | dts = self._dts[imgId, catId]
197 | inds = np.argsort([-d['score'] for d in dts], kind='mergesort')
198 | dts = [dts[i] for i in inds]
199 | if len(dts) > p.maxDets[-1]:
200 | dts = dts[0:p.maxDets[-1]]
201 | # if len(gts) == 0 and len(dts) == 0:
202 | if len(gts) == 0 or len(dts) == 0:
203 | return []
204 | ious = np.zeros((len(dts), len(gts)))
205 | sigmas = p.kpt_oks_sigmas
206 | vars = (sigmas * 2)**2
207 | k = len(sigmas)
208 | # compute oks between each detection and ground truth object
209 | for j, gt in enumerate(gts):
210 | # create bounds for ignore regions(double the gt bbox)
211 | g = np.array(gt['keypoints'])
212 | xg = g[0::3]; yg = g[1::3]; vg = g[2::3]
213 | k1 = np.count_nonzero(vg > 0)
214 | bb = gt['bbox']
215 | x0 = bb[0] - bb[2]; x1 = bb[0] + bb[2] * 2
216 | y0 = bb[1] - bb[3]; y1 = bb[1] + bb[3] * 2
217 | for i, dt in enumerate(dts):
218 | d = np.array(dt['keypoints'])
219 | xd = d[0::3]; yd = d[1::3]
220 | if k1>0:
221 | # measure the per-keypoint distance if keypoints visible
222 | dx = xd - xg
223 | dy = yd - yg
224 | else:
225 | # measure minimum distance to keypoints in (x0,y0) & (x1,y1)
226 | z = np.zeros((k))
227 | dx = np.max((z, x0-xd),axis=0)+np.max((z, xd-x1),axis=0)
228 | dy = np.max((z, y0-yd),axis=0)+np.max((z, yd-y1),axis=0)
229 | e = (dx**2 + dy**2) / vars / (gt['area']+np.spacing(1)) / 2
230 | if k1 > 0:
231 | e=e[vg > 0]
232 | ious[i, j] = np.sum(np.exp(-e)) / e.shape[0]
233 | return ious
234 |
235 | def evaluateImg(self, imgId, catId, aRng, maxDet):
236 | '''
237 | perform evaluation for single category and image
238 | :return: dict (single image results)
239 | '''
240 | p = self.params
241 | if p.useCats:
242 | gt = self._gts[imgId,catId]
243 | dt = self._dts[imgId,catId]
244 | else:
245 | gt = [_ for cId in p.catIds for _ in self._gts[imgId,cId]]
246 | dt = [_ for cId in p.catIds for _ in self._dts[imgId,cId]]
247 | if len(gt) == 0 and len(dt) ==0:
248 | return None
249 |
250 | for g in gt:
251 | if g['ignore'] or (g['area']aRng[1]):
252 | g['_ignore'] = 1
253 | else:
254 | g['_ignore'] = 0
255 |
256 | # sort dt highest score first, sort gt ignore last
257 | gtind = np.argsort([g['_ignore'] for g in gt], kind='mergesort')
258 | gt = [gt[i] for i in gtind]
259 | dtind = np.argsort([-d['score'] for d in dt], kind='mergesort')
260 | dt = [dt[i] for i in dtind[0:maxDet]]
261 | iscrowd = [int(o['iscrowd']) for o in gt]
262 | # load computed ious
263 | ious = self.ious[imgId, catId][:, gtind] if len(self.ious[imgId, catId]) > 0 else self.ious[imgId, catId]
264 |
265 | T = len(p.iouThrs)
266 | G = len(gt)
267 | D = len(dt)
268 | gtm = np.zeros((T,G))
269 | dtm = np.zeros((T,D))
270 | gtIg = np.array([g['_ignore'] for g in gt])
271 | dtIg = np.zeros((T,D))
272 | if not len(ious)==0:
273 | for tind, t in enumerate(p.iouThrs):
274 | for dind, d in enumerate(dt):
275 | # information about best match so far (m=-1 -> unmatched)
276 | iou = min([t,1-1e-10])
277 | m = -1
278 | for gind, g in enumerate(gt):
279 | # if this gt already matched, and not a crowd, continue
280 | if gtm[tind,gind]>0 and not iscrowd[gind]:
281 | continue
282 | # if dt matched to reg gt, and on ignore gt, stop
283 | if m>-1 and gtIg[m]==0 and gtIg[gind]==1:
284 | break
285 | # continue to next gt unless better match made
286 | if ious[dind,gind] < iou:
287 | continue
288 | # if match successful and best so far, store appropriately
289 | iou=ious[dind,gind]
290 | m=gind
291 | # if match made store id of match for both dt and gt
292 | if m ==-1:
293 | continue
294 | dtIg[tind,dind] = gtIg[m]
295 | dtm[tind,dind] = gt[m]['id']
296 | gtm[tind,m] = d['id']
297 | # set unmatched detections outside of area range to ignore
298 | a = np.array([d['area']aRng[1] for d in dt]).reshape((1, len(dt)))
299 | dtIg = np.logical_or(dtIg, np.logical_and(dtm==0, np.repeat(a,T,0)))
300 | # store results for given image and category
301 | return {
302 | 'image_id': imgId,
303 | 'category_id': catId,
304 | 'aRng': aRng,
305 | 'maxDet': maxDet,
306 | 'dtIds': [d['id'] for d in dt],
307 | 'gtIds': [g['id'] for g in gt],
308 | 'dtMatches': dtm,
309 | 'gtMatches': gtm,
310 | 'dtScores': [d['score'] for d in dt],
311 | 'gtIgnore': gtIg,
312 | 'dtIgnore': dtIg,
313 | }
314 |
315 | def accumulate(self, p = None):
316 | '''
317 | Accumulate per image evaluation results and store the result in self.eval
318 | :param p: input params for evaluation
319 | :return: None
320 | '''
321 | print('Accumulating evaluation results...')
322 | tic = time.time()
323 | if not self.evalImgs:
324 | print('Please run evaluate() first')
325 | # allows input customized parameters
326 | if p is None:
327 | p = self.params
328 | p.catIds = p.catIds if p.useCats == 1 else [-1]
329 | T = len(p.iouThrs)
330 | R = len(p.recThrs)
331 | K = len(p.catIds) if p.useCats else 1
332 | A = len(p.areaRng)
333 | M = len(p.maxDets)
334 | precision = -np.ones((T,R,K,A,M)) # -1 for the precision of absent categories
335 | recall = -np.ones((T,K,A,M))
336 | scores = -np.ones((T,R,K,A,M))
337 |
338 | # create dictionary for future indexing
339 | _pe = self._paramsEval
340 | catIds = _pe.catIds if _pe.useCats else [-1]
341 | setK = set(catIds)
342 | setA = set(map(tuple, _pe.areaRng))
343 | setM = set(_pe.maxDets)
344 | setI = set(_pe.imgIds)
345 | # get inds to evaluate
346 | k_list = [n for n, k in enumerate(p.catIds) if k in setK]
347 | m_list = [m for n, m in enumerate(p.maxDets) if m in setM]
348 | a_list = [n for n, a in enumerate(map(lambda x: tuple(x), p.areaRng)) if a in setA]
349 | i_list = [n for n, i in enumerate(p.imgIds) if i in setI]
350 | I0 = len(_pe.imgIds)
351 | A0 = len(_pe.areaRng)
352 | # retrieve E at each category, area range, and max number of detections
353 | for k, k0 in enumerate(k_list):
354 | Nk = k0*A0*I0
355 | for a, a0 in enumerate(a_list):
356 | Na = a0*I0
357 | for m, maxDet in enumerate(m_list):
358 | E = [self.evalImgs[Nk + Na + i] for i in i_list]
359 | E = [e for e in E if not e is None]
360 | if len(E) == 0:
361 | continue
362 | dtScores = np.concatenate([e['dtScores'][0:maxDet] for e in E])
363 |
364 | # different sorting method generates slightly different results.
365 | # mergesort is used to be consistent as Matlab implementation.
366 | inds = np.argsort(-dtScores, kind='mergesort')
367 | dtScoresSorted = dtScores[inds]
368 |
369 | dtm = np.concatenate([e['dtMatches'][:,0:maxDet] for e in E], axis=1)[:,inds]
370 | dtIg = np.concatenate([e['dtIgnore'][:,0:maxDet] for e in E], axis=1)[:,inds]
371 | gtIg = np.concatenate([e['gtIgnore'] for e in E])
372 | npig = np.count_nonzero(gtIg==0 )
373 | if npig == 0:
374 | continue
375 | tps = np.logical_and( dtm, np.logical_not(dtIg) )
376 | fps = np.logical_and(np.logical_not(dtm), np.logical_not(dtIg) )
377 |
378 | tp_sum = np.cumsum(tps, axis=1).astype(dtype=float)
379 | fp_sum = np.cumsum(fps, axis=1).astype(dtype=float)
380 | for t, (tp, fp) in enumerate(zip(tp_sum, fp_sum)):
381 | tp = np.array(tp)
382 | fp = np.array(fp)
383 | nd = len(tp)
384 | rc = tp / npig
385 | pr = tp / (fp+tp+np.spacing(1))
386 | q = np.zeros((R,))
387 | ss = np.zeros((R,))
388 |
389 | if nd:
390 | recall[t,k,a,m] = rc[-1]
391 | else:
392 | recall[t,k,a,m] = 0
393 |
394 | # numpy is slow without cython optimization for accessing elements
395 | # use python array gets significant speed improvement
396 | pr = pr.tolist(); q = q.tolist()
397 |
398 | for i in range(nd-1, 0, -1):
399 | if pr[i] > pr[i-1]:
400 | pr[i-1] = pr[i]
401 |
402 | inds = np.searchsorted(rc, p.recThrs, side='left')
403 | try:
404 | for ri, pi in enumerate(inds):
405 | q[ri] = pr[pi]
406 | ss[ri] = dtScoresSorted[pi]
407 | except:
408 | pass
409 | precision[t,:,k,a,m] = np.array(q)
410 | scores[t,:,k,a,m] = np.array(ss)
411 | self.eval = {
412 | 'params': p,
413 | 'counts': [T, R, K, A, M],
414 | 'date': datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
415 | 'precision': precision,
416 | 'recall': recall,
417 | 'scores': scores,
418 | }
419 | toc = time.time()
420 | print('DONE (t={:0.2f}s).'.format( toc-tic))
421 |
422 | def summarize(self):
423 | '''
424 | Compute and display summary metrics for evaluation results.
425 | Note this functin can *only* be applied on the default parameter setting
426 | '''
427 | def _summarize( ap=1, iouThr=None, areaRng='all', maxDets=100 ):
428 | p = self.params
429 | iStr = ' {:<18} {} @[ IoU={:<9} | area={:>6s} | maxDets={:>3d} ] = {:0.3f}'
430 | titleStr = 'Average Precision' if ap == 1 else 'Average Recall'
431 | typeStr = '(AP)' if ap==1 else '(AR)'
432 | iouStr = '{:0.2f}:{:0.2f}'.format(p.iouThrs[0], p.iouThrs[-1]) \
433 | if iouThr is None else '{:0.2f}'.format(iouThr)
434 |
435 | aind = [i for i, aRng in enumerate(p.areaRngLbl) if aRng == areaRng]
436 | mind = [i for i, mDet in enumerate(p.maxDets) if mDet == maxDets]
437 | if ap == 1:
438 | # dimension of precision: [TxRxKxAxM]
439 | s = self.eval['precision']
440 | # IoU
441 | if iouThr is not None:
442 | t = np.where(iouThr == p.iouThrs)[0]
443 | s = s[t]
444 | s = s[:,:,:,aind,mind]
445 | else:
446 | # dimension of recall: [TxKxAxM]
447 | s = self.eval['recall']
448 | if iouThr is not None:
449 | t = np.where(iouThr == p.iouThrs)[0]
450 | s = s[t]
451 | s = s[:,:,aind,mind]
452 | if len(s[s>-1])==0:
453 | mean_s = -1
454 | else:
455 | mean_s = np.mean(s[s>-1])
456 | print(iStr.format(titleStr, typeStr, iouStr, areaRng, maxDets, mean_s))
457 | return mean_s
458 | def _summarizeDets():
459 | stats = np.zeros((12,))
460 | stats[0] = _summarize(1)
461 | stats[1] = _summarize(1, iouThr=.5, maxDets=self.params.maxDets[2])
462 | stats[2] = _summarize(1, iouThr=.75, maxDets=self.params.maxDets[2])
463 | stats[3] = _summarize(1, areaRng='small', maxDets=self.params.maxDets[2])
464 | stats[4] = _summarize(1, areaRng='medium', maxDets=self.params.maxDets[2])
465 | stats[5] = _summarize(1, areaRng='large', maxDets=self.params.maxDets[2])
466 | stats[6] = _summarize(0, maxDets=self.params.maxDets[0])
467 | stats[7] = _summarize(0, maxDets=self.params.maxDets[1])
468 | stats[8] = _summarize(0, maxDets=self.params.maxDets[2])
469 | stats[9] = _summarize(0, areaRng='small', maxDets=self.params.maxDets[2])
470 | stats[10] = _summarize(0, areaRng='medium', maxDets=self.params.maxDets[2])
471 | stats[11] = _summarize(0, areaRng='large', maxDets=self.params.maxDets[2])
472 | return stats
473 | def _summarizeKps():
474 | stats = np.zeros((10,))
475 | stats[0] = _summarize(1, maxDets=20)
476 | stats[1] = _summarize(1, maxDets=20, iouThr=.5)
477 | stats[2] = _summarize(1, maxDets=20, iouThr=.75)
478 | stats[3] = _summarize(1, maxDets=20, areaRng='medium')
479 | stats[4] = _summarize(1, maxDets=20, areaRng='large')
480 | stats[5] = _summarize(0, maxDets=20)
481 | stats[6] = _summarize(0, maxDets=20, iouThr=.5)
482 | stats[7] = _summarize(0, maxDets=20, iouThr=.75)
483 | stats[8] = _summarize(0, maxDets=20, areaRng='medium')
484 | stats[9] = _summarize(0, maxDets=20, areaRng='large')
485 | return stats
486 | if not self.eval:
487 | raise Exception('Please run accumulate() first')
488 | iouType = self.params.iouType
489 | if iouType == 'segm' or iouType == 'bbox':
490 | summarize = _summarizeDets
491 | elif iouType == 'keypoints':
492 | summarize = _summarizeKps
493 | self.stats = summarize()
494 |
495 | def __str__(self):
496 | self.summarize()
497 |
498 | class Params:
499 | '''
500 | Params for coco evaluation api
501 | '''
502 | def setDetParams(self):
503 | self.imgIds = []
504 | self.catIds = []
505 | # np.arange causes trouble. the data point on arange is slightly larger than the true value
506 | self.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)
507 | self.recThrs = np.linspace(.0, 1.00, int(np.round((1.00 - .0) / .01)) + 1, endpoint=True)
508 | self.maxDets = [1, 10, 100]
509 | #self.areaRng = [[0 ** 2, 1e5 ** 2], [0 ** 2, 32 ** 2], [32 ** 2, 96 ** 2], [96 ** 2, 1e5 ** 2]]
510 | #replaced
511 | self.areaRng = [[0 ** 2, 1e5 ** 2], [0 ** 2, 64 ** 2],[64 ** 2, 192 ** 2], [192 ** 2, 1e5 ** 2]]
512 | self.areaRngLbl = ['all', 'small', 'medium', 'large']
513 | self.useCats = 1
514 |
515 | def setKpParams(self):
516 | self.imgIds = []
517 | self.catIds = []
518 | # np.arange causes trouble. the data point on arange is slightly larger than the true value
519 | self.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)
520 | self.recThrs = np.linspace(.0, 1.00, int(np.round((1.00 - .0) / .01)) + 1, endpoint=True)
521 | self.maxDets = [20]
522 | self.areaRng = [[0 ** 2, 1e5 ** 2], [32 ** 2, 96 ** 2], [96 ** 2, 1e5 ** 2]]
523 | self.areaRngLbl = ['all', 'medium', 'large']
524 | self.useCats = 1
525 | self.kpt_oks_sigmas = np.array([.26, .25, .25, .35, .35, .79, .79, .72, .72, .62,.62, 1.07, 1.07, .87, .87, .89, .89])/10.0
526 |
527 | def __init__(self, iouType='segm'):
528 | if iouType == 'segm' or iouType == 'bbox':
529 | self.setDetParams()
530 | elif iouType == 'keypoints':
531 | self.setKpParams()
532 | else:
533 | raise Exception('iouType not supported')
534 | self.iouType = iouType
535 | # useSegm is deprecated
536 | self.useSegm = None
537 |
--------------------------------------------------------------------------------
/evaluation_Jetson/pycocotools/mask.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tsungyi'
2 |
3 | import pycocotools._mask as _mask
4 |
5 | # Interface for manipulating masks stored in RLE format.
6 | #
7 | # RLE is a simple yet efficient format for storing binary masks. RLE
8 | # first divides a vector (or vectorized image) into a series of piecewise
9 | # constant regions and then for each piece simply stores the length of
10 | # that piece. For example, given M=[0 0 1 1 1 0 1] the RLE counts would
11 | # be [2 3 1 1], or for M=[1 1 1 1 1 1 0] the counts would be [0 6 1]
12 | # (note that the odd counts are always the numbers of zeros). Instead of
13 | # storing the counts directly, additional compression is achieved with a
14 | # variable bitrate representation based on a common scheme called LEB128.
15 | #
16 | # Compression is greatest given large piecewise constant regions.
17 | # Specifically, the size of the RLE is proportional to the number of
18 | # *boundaries* in M (or for an image the number of boundaries in the y
19 | # direction). Assuming fairly simple shapes, the RLE representation is
20 | # O(sqrt(n)) where n is number of pixels in the object. Hence space usage
21 | # is substantially lower, especially for large simple objects (large n).
22 | #
23 | # Many common operations on masks can be computed directly using the RLE
24 | # (without need for decoding). This includes computations such as area,
25 | # union, intersection, etc. All of these operations are linear in the
26 | # size of the RLE, in other words they are O(sqrt(n)) where n is the area
27 | # of the object. Computing these operations on the original mask is O(n).
28 | # Thus, using the RLE can result in substantial computational savings.
29 | #
30 | # The following API functions are defined:
31 | # encode - Encode binary masks using RLE.
32 | # decode - Decode binary masks encoded via RLE.
33 | # merge - Compute union or intersection of encoded masks.
34 | # iou - Compute intersection over union between masks.
35 | # area - Compute area of encoded masks.
36 | # toBbox - Get bounding boxes surrounding encoded masks.
37 | # frPyObjects - Convert polygon, bbox, and uncompressed RLE to encoded RLE mask.
38 | #
39 | # Usage:
40 | # Rs = encode( masks )
41 | # masks = decode( Rs )
42 | # R = merge( Rs, intersect=false )
43 | # o = iou( dt, gt, iscrowd )
44 | # a = area( Rs )
45 | # bbs = toBbox( Rs )
46 | # Rs = frPyObjects( [pyObjects], h, w )
47 | #
48 | # In the API the following formats are used:
49 | # Rs - [dict] Run-length encoding of binary masks
50 | # R - dict Run-length encoding of binary mask
51 | # masks - [hxwxn] Binary mask(s) (must have type np.ndarray(dtype=uint8) in column-major order)
52 | # iscrowd - [nx1] list of np.ndarray. 1 indicates corresponding gt image has crowd region to ignore
53 | # bbs - [nx4] Bounding box(es) stored as [x y w h]
54 | # poly - Polygon stored as [[x1 y1 x2 y2...],[x1 y1 ...],...] (2D list)
55 | # dt,gt - May be either bounding boxes or encoded masks
56 | # Both poly and bbs are 0-indexed (bbox=[0 0 1 1] encloses first pixel).
57 | #
58 | # Finally, a note about the intersection over union (iou) computation.
59 | # The standard iou of a ground truth (gt) and detected (dt) object is
60 | # iou(gt,dt) = area(intersect(gt,dt)) / area(union(gt,dt))
61 | # For "crowd" regions, we use a modified criteria. If a gt object is
62 | # marked as "iscrowd", we allow a dt to match any subregion of the gt.
63 | # Choosing gt' in the crowd gt that best matches the dt can be done using
64 | # gt'=intersect(dt,gt). Since by definition union(gt',dt)=dt, computing
65 | # iou(gt,dt,iscrowd) = iou(gt',dt) = area(intersect(gt,dt)) / area(dt)
66 | # For crowd gt regions we use this modified criteria above for the iou.
67 | #
68 | # To compile run "python setup.py build_ext --inplace"
69 | # Please do not contact us for help with compiling.
70 | #
71 | # Microsoft COCO Toolbox. version 2.0
72 | # Data, paper, and tutorials available at: http://mscoco.org/
73 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2015.
74 | # Licensed under the Simplified BSD License [see coco/license.txt]
75 |
76 | iou = _mask.iou
77 | merge = _mask.merge
78 | frPyObjects = _mask.frPyObjects
79 |
80 | def encode(bimask):
81 | if len(bimask.shape) == 3:
82 | return _mask.encode(bimask)
83 | elif len(bimask.shape) == 2:
84 | h, w = bimask.shape
85 | return _mask.encode(bimask.reshape((h, w, 1), order='F'))[0]
86 |
87 | def decode(rleObjs):
88 | if type(rleObjs) == list:
89 | return _mask.decode(rleObjs)
90 | else:
91 | return _mask.decode([rleObjs])[:,:,0]
92 |
93 | def area(rleObjs):
94 | if type(rleObjs) == list:
95 | return _mask.area(rleObjs)
96 | else:
97 | return _mask.area([rleObjs])[0]
98 |
99 | def toBbox(rleObjs):
100 | if type(rleObjs) == list:
101 | return _mask.toBbox(rleObjs)
102 | else:
103 | return _mask.toBbox([rleObjs])[0]
--------------------------------------------------------------------------------
/evaluation_Linux/EVALUATION.md:
--------------------------------------------------------------------------------
1 | # Evaluation Metric for FishEye8k dataset
2 |
--------------------------------------------------------------------------------
/evaluation_Linux/README.md:
--------------------------------------------------------------------------------
1 | # Evaluation Metric for FishEye8k dataset
2 |
3 | Update:
4 |
5 | `2024/03/18`: Change the bug in calculating f1 score.
6 |
7 |
8 |
--------------------------------------------------------------------------------
/evaluation_Linux/eval.py:
--------------------------------------------------------------------------------
1 | from pycocotools.coco import COCO
2 | from pycocotools.cocoeval import COCOeval
3 |
4 |
5 | coco_gt = COCO('groundtruth.json')
6 | coco_dt = coco_gt.loadRes(
7 | 'detection_dgx.json')
8 |
9 | print(type(coco_dt))
10 | print(type(coco_gt))
11 |
12 | coco_eval = COCOeval(coco_gt, coco_dt, 'bbox')
13 | coco_eval.evaluate()
14 | coco_eval.accumulate()
15 | coco_eval.summarize()
16 |
--------------------------------------------------------------------------------
/evaluation_Linux/eval_f1.py:
--------------------------------------------------------------------------------
1 | from pycocotools.coco import COCO
2 | from pycocotools.cocoeval_modified import COCOeval
3 | import json
4 |
5 | coco_gt = COCO('groundtruth.json')
6 |
7 | gt_image_ids = coco_gt.getImgIds()
8 |
9 | print("Total images ", len(gt_image_ids))
10 |
11 |
12 | with open('detections.json', 'r') as f:
13 | detection_data = json.load(f)
14 |
15 | filtered_detection_data = [
16 | item for item in detection_data if item['image_id'] in gt_image_ids]
17 |
18 | with open('./temp.json', 'w') as f:
19 | json.dump(filtered_detection_data, f)
20 |
21 | coco_dt = coco_gt.loadRes('./temp.json')
22 | coco_eval = COCOeval(coco_gt, coco_dt, 'bbox')
23 | coco_eval.evaluate()
24 | coco_eval.accumulate()
25 | coco_eval.summarize()
26 | print('----------------------------------------')
27 | print('AP_0.5-0.95', coco_eval.stats[0])
28 | print('AP_0.5', coco_eval.stats[1])
29 | print('AP_S', coco_eval.stats[3])
30 | print('AP_M', coco_eval.stats[4])
31 | print('AP_L', coco_eval.stats[5])
32 | print('f1_score: ', coco_eval.stats[20])
33 | print('----------------------------------------')
34 |
--------------------------------------------------------------------------------
/evaluation_Linux/pycocotools/__init__.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tylin'
2 |
--------------------------------------------------------------------------------
/evaluation_Linux/pycocotools/__pycache__/__init__.cpython-38.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Linux/pycocotools/__pycache__/__init__.cpython-38.pyc
--------------------------------------------------------------------------------
/evaluation_Linux/pycocotools/__pycache__/coco.cpython-38.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Linux/pycocotools/__pycache__/coco.cpython-38.pyc
--------------------------------------------------------------------------------
/evaluation_Linux/pycocotools/__pycache__/cocoeval.cpython-38.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Linux/pycocotools/__pycache__/cocoeval.cpython-38.pyc
--------------------------------------------------------------------------------
/evaluation_Linux/pycocotools/__pycache__/mask.cpython-38.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Linux/pycocotools/__pycache__/mask.cpython-38.pyc
--------------------------------------------------------------------------------
/evaluation_Linux/pycocotools/_mask.cpython-38-x86_64-linux-gnu.so:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Linux/pycocotools/_mask.cpython-38-x86_64-linux-gnu.so
--------------------------------------------------------------------------------
/evaluation_Linux/pycocotools/coco.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tylin'
2 | __version__ = '2.0'
3 | # Interface for accessing the Microsoft COCO dataset.
4 |
5 | # Microsoft COCO is a large image dataset designed for object detection,
6 | # segmentation, and caption generation. pycocotools is a Python API that
7 | # assists in loading, parsing and visualizing the annotations in COCO.
8 | # Please visit http://mscoco.org/ for more information on COCO, including
9 | # for the data, paper, and tutorials. The exact format of the annotations
10 | # is also described on the COCO website. For example usage of the pycocotools
11 | # please see pycocotools_demo.ipynb. In addition to this API, please download both
12 | # the COCO images and annotations in order to run the demo.
13 |
14 | # An alternative to using the API is to load the annotations directly
15 | # into Python dictionary
16 | # Using the API provides additional utility functions. Note that this API
17 | # supports both *instance* and *caption* annotations. In the case of
18 | # captions not all functions are defined (e.g. categories are undefined).
19 |
20 | # The following API functions are defined:
21 | # COCO - COCO api class that loads COCO annotation file and prepare data structures.
22 | # decodeMask - Decode binary mask M encoded via run-length encoding.
23 | # encodeMask - Encode binary mask M using run-length encoding.
24 | # getAnnIds - Get ann ids that satisfy given filter conditions.
25 | # getCatIds - Get cat ids that satisfy given filter conditions.
26 | # getImgIds - Get img ids that satisfy given filter conditions.
27 | # loadAnns - Load anns with the specified ids.
28 | # loadCats - Load cats with the specified ids.
29 | # loadImgs - Load imgs with the specified ids.
30 | # annToMask - Convert segmentation in an annotation to binary mask.
31 | # showAnns - Display the specified annotations.
32 | # loadRes - Load algorithm results and create API for accessing them.
33 | # download - Download COCO images from mscoco.org server.
34 | # Throughout the API "ann"=annotation, "cat"=category, and "img"=image.
35 | # Help on each functions can be accessed by: "help COCO>function".
36 |
37 | # See also COCO>decodeMask,
38 | # COCO>encodeMask, COCO>getAnnIds, COCO>getCatIds,
39 | # COCO>getImgIds, COCO>loadAnns, COCO>loadCats,
40 | # COCO>loadImgs, COCO>annToMask, COCO>showAnns
41 |
42 | # Microsoft COCO Toolbox. version 2.0
43 | # Data, paper, and tutorials available at: http://mscoco.org/
44 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2014.
45 | # Licensed under the Simplified BSD License [see bsd.txt]
46 |
47 | import json
48 | import time
49 | import numpy as np
50 | import copy
51 | import itertools
52 | from . import mask as maskUtils
53 | import os
54 | from collections import defaultdict
55 | import sys
56 | PYTHON_VERSION = sys.version_info[0]
57 | if PYTHON_VERSION == 2:
58 | from urllib import urlretrieve
59 | elif PYTHON_VERSION == 3:
60 | from urllib.request import urlretrieve
61 |
62 |
63 | def _isArrayLike(obj):
64 | return hasattr(obj, '__iter__') and hasattr(obj, '__len__')
65 |
66 |
67 | class COCO:
68 | def __init__(self, annotation_file=None):
69 | """
70 | Constructor of Microsoft COCO helper class for reading and visualizing annotations.
71 | :param annotation_file (str): location of annotation file
72 | :param image_folder (str): location to the folder that hosts images.
73 | :return:
74 | """
75 | # load dataset
76 | self.dataset,self.anns,self.cats,self.imgs = dict(),dict(),dict(),dict()
77 | self.imgToAnns, self.catToImgs = defaultdict(list), defaultdict(list)
78 | if not annotation_file == None:
79 | print('loading annotations into memory...')
80 | tic = time.time()
81 | with open(annotation_file, 'r') as f:
82 | dataset = json.load(f)
83 | assert type(dataset)==dict, 'annotation file format {} not supported'.format(type(dataset))
84 | print('Done (t={:0.2f}s)'.format(time.time()- tic))
85 | self.dataset = dataset
86 | self.createIndex()
87 |
88 | def createIndex(self):
89 | # create index
90 | print('creating index...')
91 | anns, cats, imgs = {}, {}, {}
92 | imgToAnns,catToImgs = defaultdict(list),defaultdict(list)
93 | if 'annotations' in self.dataset:
94 | for ann in self.dataset['annotations']:
95 | imgToAnns[ann['image_id']].append(ann)
96 | anns[ann['id']] = ann
97 |
98 | if 'images' in self.dataset:
99 | for img in self.dataset['images']:
100 | imgs[img['id']] = img
101 |
102 | if 'categories' in self.dataset:
103 | for cat in self.dataset['categories']:
104 | cats[cat['id']] = cat
105 |
106 | if 'annotations' in self.dataset and 'categories' in self.dataset:
107 | for ann in self.dataset['annotations']:
108 | catToImgs[ann['category_id']].append(ann['image_id'])
109 |
110 | print('index created!')
111 |
112 | # create class members
113 | self.anns = anns
114 | self.imgToAnns = imgToAnns
115 | self.catToImgs = catToImgs
116 | self.imgs = imgs
117 | self.cats = cats
118 |
119 | def info(self):
120 | """
121 | Print information about the annotation file.
122 | :return:
123 | """
124 | for key, value in self.dataset['info'].items():
125 | print('{}: {}'.format(key, value))
126 |
127 | def getAnnIds(self, imgIds=[], catIds=[], areaRng=[], iscrowd=None):
128 | """
129 | Get ann ids that satisfy given filter conditions. default skips that filter
130 | :param imgIds (int array) : get anns for given imgs
131 | catIds (int array) : get anns for given cats
132 | areaRng (float array) : get anns for given area range (e.g. [0 inf])
133 | iscrowd (boolean) : get anns for given crowd label (False or True)
134 | :return: ids (int array) : integer array of ann ids
135 | """
136 | imgIds = imgIds if _isArrayLike(imgIds) else [imgIds]
137 | catIds = catIds if _isArrayLike(catIds) else [catIds]
138 |
139 | if len(imgIds) == len(catIds) == len(areaRng) == 0:
140 | anns = self.dataset['annotations']
141 | else:
142 | if not len(imgIds) == 0:
143 | lists = [self.imgToAnns[imgId] for imgId in imgIds if imgId in self.imgToAnns]
144 | anns = list(itertools.chain.from_iterable(lists))
145 | else:
146 | anns = self.dataset['annotations']
147 | anns = anns if len(catIds) == 0 else [ann for ann in anns if ann['category_id'] in catIds]
148 | anns = anns if len(areaRng) == 0 else [ann for ann in anns if ann['area'] > areaRng[0] and ann['area'] < areaRng[1]]
149 | if not iscrowd == None:
150 | ids = [ann['id'] for ann in anns if ann['iscrowd'] == iscrowd]
151 | else:
152 | ids = [ann['id'] for ann in anns]
153 | return ids
154 |
155 | def getCatIds(self, catNms=[], supNms=[], catIds=[]):
156 | """
157 | filtering parameters. default skips that filter.
158 | :param catNms (str array) : get cats for given cat names
159 | :param supNms (str array) : get cats for given supercategory names
160 | :param catIds (int array) : get cats for given cat ids
161 | :return: ids (int array) : integer array of cat ids
162 | """
163 | catNms = catNms if _isArrayLike(catNms) else [catNms]
164 | supNms = supNms if _isArrayLike(supNms) else [supNms]
165 | catIds = catIds if _isArrayLike(catIds) else [catIds]
166 |
167 | if len(catNms) == len(supNms) == len(catIds) == 0:
168 | cats = self.dataset['categories']
169 | else:
170 | cats = self.dataset['categories']
171 | cats = cats if len(catNms) == 0 else [cat for cat in cats if cat['name'] in catNms]
172 | cats = cats if len(supNms) == 0 else [cat for cat in cats if cat['supercategory'] in supNms]
173 | cats = cats if len(catIds) == 0 else [cat for cat in cats if cat['id'] in catIds]
174 | ids = [cat['id'] for cat in cats]
175 | return ids
176 |
177 | def getImgIds(self, imgIds=[], catIds=[]):
178 | '''
179 | Get img ids that satisfy given filter conditions.
180 | :param imgIds (int array) : get imgs for given ids
181 | :param catIds (int array) : get imgs with all given cats
182 | :return: ids (int array) : integer array of img ids
183 | '''
184 | imgIds = imgIds if _isArrayLike(imgIds) else [imgIds]
185 | catIds = catIds if _isArrayLike(catIds) else [catIds]
186 |
187 | if len(imgIds) == len(catIds) == 0:
188 | ids = self.imgs.keys()
189 | else:
190 | ids = set(imgIds)
191 | for i, catId in enumerate(catIds):
192 | if i == 0 and len(ids) == 0:
193 | ids = set(self.catToImgs[catId])
194 | else:
195 | ids &= set(self.catToImgs[catId])
196 | return list(ids)
197 |
198 | def loadAnns(self, ids=[]):
199 | """
200 | Load anns with the specified ids.
201 | :param ids (int array) : integer ids specifying anns
202 | :return: anns (object array) : loaded ann objects
203 | """
204 | if _isArrayLike(ids):
205 | return [self.anns[id] for id in ids]
206 | elif type(ids) == int:
207 | return [self.anns[ids]]
208 |
209 | def loadCats(self, ids=[]):
210 | """
211 | Load cats with the specified ids.
212 | :param ids (int array) : integer ids specifying cats
213 | :return: cats (object array) : loaded cat objects
214 | """
215 | if _isArrayLike(ids):
216 | return [self.cats[id] for id in ids]
217 | elif type(ids) == int:
218 | return [self.cats[ids]]
219 |
220 | def loadImgs(self, ids=[]):
221 | """
222 | Load anns with the specified ids.
223 | :param ids (int array) : integer ids specifying img
224 | :return: imgs (object array) : loaded img objects
225 | """
226 | if _isArrayLike(ids):
227 | return [self.imgs[id] for id in ids]
228 | elif type(ids) == int:
229 | return [self.imgs[ids]]
230 |
231 | def showAnns(self, anns, draw_bbox=False):
232 | """
233 | Display the specified annotations.
234 | :param anns (array of object): annotations to display
235 | :return: None
236 | """
237 | if len(anns) == 0:
238 | return 0
239 | if 'segmentation' in anns[0] or 'keypoints' in anns[0]:
240 | datasetType = 'instances'
241 | elif 'caption' in anns[0]:
242 | datasetType = 'captions'
243 | else:
244 | raise Exception('datasetType not supported')
245 | if datasetType == 'instances':
246 | import matplotlib.pyplot as plt
247 | from matplotlib.collections import PatchCollection
248 | from matplotlib.patches import Polygon
249 |
250 | ax = plt.gca()
251 | ax.set_autoscale_on(False)
252 | polygons = []
253 | color = []
254 | for ann in anns:
255 | c = (np.random.random((1, 3))*0.6+0.4).tolist()[0]
256 | if 'segmentation' in ann:
257 | if type(ann['segmentation']) == list:
258 | # polygon
259 | for seg in ann['segmentation']:
260 | poly = np.array(seg).reshape((int(len(seg)/2), 2))
261 | polygons.append(Polygon(poly))
262 | color.append(c)
263 | else:
264 | # mask
265 | t = self.imgs[ann['image_id']]
266 | if type(ann['segmentation']['counts']) == list:
267 | rle = maskUtils.frPyObjects([ann['segmentation']], t['height'], t['width'])
268 | else:
269 | rle = [ann['segmentation']]
270 | m = maskUtils.decode(rle)
271 | img = np.ones( (m.shape[0], m.shape[1], 3) )
272 | if ann['iscrowd'] == 1:
273 | color_mask = np.array([2.0,166.0,101.0])/255
274 | if ann['iscrowd'] == 0:
275 | color_mask = np.random.random((1, 3)).tolist()[0]
276 | for i in range(3):
277 | img[:,:,i] = color_mask[i]
278 | ax.imshow(np.dstack( (img, m*0.5) ))
279 | if 'keypoints' in ann and type(ann['keypoints']) == list:
280 | # turn skeleton into zero-based index
281 | sks = np.array(self.loadCats(ann['category_id'])[0]['skeleton'])-1
282 | kp = np.array(ann['keypoints'])
283 | x = kp[0::3]
284 | y = kp[1::3]
285 | v = kp[2::3]
286 | for sk in sks:
287 | if np.all(v[sk]>0):
288 | plt.plot(x[sk],y[sk], linewidth=3, color=c)
289 | plt.plot(x[v>0], y[v>0],'o',markersize=8, markerfacecolor=c, markeredgecolor='k',markeredgewidth=2)
290 | plt.plot(x[v>1], y[v>1],'o',markersize=8, markerfacecolor=c, markeredgecolor=c, markeredgewidth=2)
291 |
292 | if draw_bbox:
293 | [bbox_x, bbox_y, bbox_w, bbox_h] = ann['bbox']
294 | poly = [[bbox_x, bbox_y], [bbox_x, bbox_y+bbox_h], [bbox_x+bbox_w, bbox_y+bbox_h], [bbox_x+bbox_w, bbox_y]]
295 | np_poly = np.array(poly).reshape((4,2))
296 | polygons.append(Polygon(np_poly))
297 | color.append(c)
298 |
299 | p = PatchCollection(polygons, facecolor=color, linewidths=0, alpha=0.4)
300 | ax.add_collection(p)
301 | p = PatchCollection(polygons, facecolor='none', edgecolors=color, linewidths=2)
302 | ax.add_collection(p)
303 | elif datasetType == 'captions':
304 | for ann in anns:
305 | print(ann['caption'])
306 |
307 | def loadRes(self, resFile):
308 | """
309 | Load result file and return a result api object.
310 | :param resFile (str) : file name of result file
311 | :return: res (obj) : result api object
312 | """
313 | res = COCO()
314 | res.dataset['images'] = [img for img in self.dataset['images']]
315 |
316 | print('Loading and preparing results...')
317 | tic = time.time()
318 | if type(resFile) == str or (PYTHON_VERSION == 2 and type(resFile) == unicode):
319 | with open(resFile) as f:
320 | anns = json.load(f)
321 | elif type(resFile) == np.ndarray:
322 | anns = self.loadNumpyAnnotations(resFile)
323 | else:
324 | anns = resFile
325 | assert type(anns) == list, 'results in not an array of objects'
326 | annsImgIds = [ann['image_id'] for ann in anns]
327 | assert set(annsImgIds) == (set(annsImgIds) & set(self.getImgIds())), \
328 | 'Results do not correspond to current coco set'
329 | if 'caption' in anns[0]:
330 | imgIds = set([img['id'] for img in res.dataset['images']]) & set([ann['image_id'] for ann in anns])
331 | res.dataset['images'] = [img for img in res.dataset['images'] if img['id'] in imgIds]
332 | for id, ann in enumerate(anns):
333 | ann['id'] = id+1
334 | elif 'bbox' in anns[0] and not anns[0]['bbox'] == []:
335 | res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])
336 | for id, ann in enumerate(anns):
337 | bb = ann['bbox']
338 | x1, x2, y1, y2 = [bb[0], bb[0]+bb[2], bb[1], bb[1]+bb[3]]
339 | if not 'segmentation' in ann:
340 | ann['segmentation'] = [[x1, y1, x1, y2, x2, y2, x2, y1]]
341 | ann['area'] = bb[2]*bb[3]
342 | ann['id'] = id+1
343 | ann['iscrowd'] = 0
344 | elif 'segmentation' in anns[0]:
345 | res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])
346 | for id, ann in enumerate(anns):
347 | # now only support compressed RLE format as segmentation results
348 | ann['area'] = maskUtils.area(ann['segmentation'])
349 | if not 'bbox' in ann:
350 | ann['bbox'] = maskUtils.toBbox(ann['segmentation'])
351 | ann['id'] = id+1
352 | ann['iscrowd'] = 0
353 | elif 'keypoints' in anns[0]:
354 | res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])
355 | for id, ann in enumerate(anns):
356 | s = ann['keypoints']
357 | x = s[0::3]
358 | y = s[1::3]
359 | x0,x1,y0,y1 = np.min(x), np.max(x), np.min(y), np.max(y)
360 | ann['area'] = (x1-x0)*(y1-y0)
361 | ann['id'] = id + 1
362 | ann['bbox'] = [x0,y0,x1-x0,y1-y0]
363 | print('DONE (t={:0.2f}s)'.format(time.time()- tic))
364 |
365 | res.dataset['annotations'] = anns
366 | res.createIndex()
367 | return res
368 |
369 | def download(self, tarDir = None, imgIds = [] ):
370 | '''
371 | Download COCO images from mscoco.org server.
372 | :param tarDir (str): COCO results directory name
373 | imgIds (list): images to be downloaded
374 | :return:
375 | '''
376 | if tarDir is None:
377 | print('Please specify target directory')
378 | return -1
379 | if len(imgIds) == 0:
380 | imgs = self.imgs.values()
381 | else:
382 | imgs = self.loadImgs(imgIds)
383 | N = len(imgs)
384 | if not os.path.exists(tarDir):
385 | os.makedirs(tarDir)
386 | for i, img in enumerate(imgs):
387 | tic = time.time()
388 | fname = os.path.join(tarDir, img['file_name'])
389 | if not os.path.exists(fname):
390 | urlretrieve(img['coco_url'], fname)
391 | print('downloaded {}/{} images (t={:0.1f}s)'.format(i, N, time.time()- tic))
392 |
393 | def loadNumpyAnnotations(self, data):
394 | """
395 | Convert result data from a numpy array [Nx7] where each row contains {imageID,x1,y1,w,h,score,class}
396 | :param data (numpy.ndarray)
397 | :return: annotations (python nested list)
398 | """
399 | print('Converting ndarray to lists...')
400 | assert(type(data) == np.ndarray)
401 | print(data.shape)
402 | assert(data.shape[1] == 7)
403 | N = data.shape[0]
404 | ann = []
405 | for i in range(N):
406 | if i % 1000000 == 0:
407 | print('{}/{}'.format(i,N))
408 | ann += [{
409 | 'image_id' : int(data[i, 0]),
410 | 'bbox' : [ data[i, 1], data[i, 2], data[i, 3], data[i, 4] ],
411 | 'score' : data[i, 5],
412 | 'category_id': int(data[i, 6]),
413 | }]
414 | return ann
415 |
416 | def annToRLE(self, ann):
417 | """
418 | Convert annotation which can be polygons, uncompressed RLE to RLE.
419 | :return: binary mask (numpy 2D array)
420 | """
421 | t = self.imgs[ann['image_id']]
422 | h, w = t['height'], t['width']
423 | segm = ann['segmentation']
424 | if type(segm) == list:
425 | # polygon -- a single object might consist of multiple parts
426 | # we merge all parts into one mask rle code
427 | rles = maskUtils.frPyObjects(segm, h, w)
428 | rle = maskUtils.merge(rles)
429 | elif type(segm['counts']) == list:
430 | # uncompressed RLE
431 | rle = maskUtils.frPyObjects(segm, h, w)
432 | else:
433 | # rle
434 | rle = ann['segmentation']
435 | return rle
436 |
437 | def annToMask(self, ann):
438 | """
439 | Convert annotation which can be polygons, uncompressed RLE, or RLE to binary mask.
440 | :return: binary mask (numpy 2D array)
441 | """
442 | rle = self.annToRLE(ann)
443 | m = maskUtils.decode(rle)
444 | return m
445 |
--------------------------------------------------------------------------------
/evaluation_Linux/pycocotools/cocoeval.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tsungyi'
2 |
3 | import numpy as np
4 | import datetime
5 | import time
6 | from collections import defaultdict
7 | from . import mask as maskUtils
8 | import copy
9 |
10 | class COCOeval:
11 | # Interface for evaluating detection on the Microsoft COCO dataset.
12 | #
13 | # The usage for CocoEval is as follows:
14 | # cocoGt=..., cocoDt=... # load dataset and results
15 | # E = CocoEval(cocoGt,cocoDt); # initialize CocoEval object
16 | # E.params.recThrs = ...; # set parameters as desired
17 | # E.evaluate(); # run per image evaluation
18 | # E.accumulate(); # accumulate per image results
19 | # E.summarize(); # display summary metrics of results
20 | # For example usage see evalDemo.m and http://mscoco.org/.
21 | #
22 | # The evaluation parameters are as follows (defaults in brackets):
23 | # imgIds - [all] N img ids to use for evaluation
24 | # catIds - [all] K cat ids to use for evaluation
25 | # iouThrs - [.5:.05:.95] T=10 IoU thresholds for evaluation
26 | # recThrs - [0:.01:1] R=101 recall thresholds for evaluation
27 | # areaRng - [...] A=4 object area ranges for evaluation
28 | # maxDets - [1 10 100] M=3 thresholds on max detections per image
29 | # iouType - ['segm'] set iouType to 'segm', 'bbox' or 'keypoints'
30 | # iouType replaced the now DEPRECATED useSegm parameter.
31 | # useCats - [1] if true use category labels for evaluation
32 | # Note: if useCats=0 category labels are ignored as in proposal scoring.
33 | # Note: multiple areaRngs [Ax2] and maxDets [Mx1] can be specified.
34 | #
35 | # evaluate(): evaluates detections on every image and every category and
36 | # concats the results into the "evalImgs" with fields:
37 | # dtIds - [1xD] id for each of the D detections (dt)
38 | # gtIds - [1xG] id for each of the G ground truths (gt)
39 | # dtMatches - [TxD] matching gt id at each IoU or 0
40 | # gtMatches - [TxG] matching dt id at each IoU or 0
41 | # dtScores - [1xD] confidence of each dt
42 | # gtIgnore - [1xG] ignore flag for each gt
43 | # dtIgnore - [TxD] ignore flag for each dt at each IoU
44 | #
45 | # accumulate(): accumulates the per-image, per-category evaluation
46 | # results in "evalImgs" into the dictionary "eval" with fields:
47 | # params - parameters used for evaluation
48 | # date - date evaluation was performed
49 | # counts - [T,R,K,A,M] parameter dimensions (see above)
50 | # precision - [TxRxKxAxM] precision for every evaluation setting
51 | # recall - [TxKxAxM] max recall for every evaluation setting
52 | # Note: precision and recall==-1 for settings with no gt objects.
53 | #
54 | # See also coco, mask, pycocoDemo, pycocoEvalDemo
55 | #
56 | # Microsoft COCO Toolbox. version 2.0
57 | # Data, paper, and tutorials available at: http://mscoco.org/
58 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2015.
59 | # Licensed under the Simplified BSD License [see coco/license.txt]
60 | def __init__(self, cocoGt=None, cocoDt=None, iouType='segm'):
61 | '''
62 | Initialize CocoEval using coco APIs for gt and dt
63 | :param cocoGt: coco object with ground truth annotations
64 | :param cocoDt: coco object with detection results
65 | :return: None
66 | '''
67 | if not iouType:
68 | print('iouType not specified. use default iouType segm')
69 | self.cocoGt = cocoGt # ground truth COCO API
70 | self.cocoDt = cocoDt # detections COCO API
71 | self.evalImgs = defaultdict(list) # per-image per-category evaluation results [KxAxI] elements
72 | self.eval = {} # accumulated evaluation results
73 | self._gts = defaultdict(list) # gt for evaluation
74 | self._dts = defaultdict(list) # dt for evaluation
75 | self.params = Params(iouType=iouType) # parameters
76 | self._paramsEval = {} # parameters for evaluation
77 | self.stats = [] # result summarization
78 | self.ious = {} # ious between all gts and dts
79 | if not cocoGt is None:
80 | self.params.imgIds = sorted(cocoGt.getImgIds())
81 | self.params.catIds = sorted(cocoGt.getCatIds())
82 |
83 |
84 | def _prepare(self):
85 | '''
86 | Prepare ._gts and ._dts for evaluation based on params
87 | :return: None
88 | '''
89 | def _toMask(anns, coco):
90 | # modify ann['segmentation'] by reference
91 | for ann in anns:
92 | rle = coco.annToRLE(ann)
93 | ann['segmentation'] = rle
94 | p = self.params
95 | if p.useCats:
96 | gts=self.cocoGt.loadAnns(self.cocoGt.getAnnIds(imgIds=p.imgIds, catIds=p.catIds))
97 | dts=self.cocoDt.loadAnns(self.cocoDt.getAnnIds(imgIds=p.imgIds, catIds=p.catIds))
98 | else:
99 | gts=self.cocoGt.loadAnns(self.cocoGt.getAnnIds(imgIds=p.imgIds))
100 | dts=self.cocoDt.loadAnns(self.cocoDt.getAnnIds(imgIds=p.imgIds))
101 |
102 | # convert ground truth to mask if iouType == 'segm'
103 | if p.iouType == 'segm':
104 | _toMask(gts, self.cocoGt)
105 | _toMask(dts, self.cocoDt)
106 | # set ignore flag
107 | for gt in gts:
108 | gt['ignore'] = gt['ignore'] if 'ignore' in gt else 0
109 | gt['ignore'] = 'iscrowd' in gt and gt['iscrowd']
110 | if p.iouType == 'keypoints':
111 | gt['ignore'] = (gt['num_keypoints'] == 0) or gt['ignore']
112 | self._gts = defaultdict(list) # gt for evaluation
113 | self._dts = defaultdict(list) # dt for evaluation
114 | for gt in gts:
115 | self._gts[gt['image_id'], gt['category_id']].append(gt)
116 | for dt in dts:
117 | self._dts[dt['image_id'], dt['category_id']].append(dt)
118 | self.evalImgs = defaultdict(list) # per-image per-category evaluation results
119 | self.eval = {} # accumulated evaluation results
120 |
121 | def evaluate(self):
122 | '''
123 | Run per image evaluation on given images and store results (a list of dict) in self.evalImgs
124 | :return: None
125 | '''
126 | tic = time.time()
127 | print('Running per image evaluation...')
128 | p = self.params
129 | # add backward compatibility if useSegm is specified in params
130 | if not p.useSegm is None:
131 | p.iouType = 'segm' if p.useSegm == 1 else 'bbox'
132 | print('useSegm (deprecated) is not None. Running {} evaluation'.format(p.iouType))
133 | print('Evaluate annotation type *{}*'.format(p.iouType))
134 | p.imgIds = list(np.unique(p.imgIds))
135 | if p.useCats:
136 | p.catIds = list(np.unique(p.catIds))
137 | p.maxDets = sorted(p.maxDets)
138 | self.params=p
139 |
140 | self._prepare()
141 | # loop through images, area range, max detection number
142 | catIds = p.catIds if p.useCats else [-1]
143 |
144 | if p.iouType == 'segm' or p.iouType == 'bbox':
145 | computeIoU = self.computeIoU
146 | elif p.iouType == 'keypoints':
147 | computeIoU = self.computeOks
148 | self.ious = {(imgId, catId): computeIoU(imgId, catId) \
149 | for imgId in p.imgIds
150 | for catId in catIds}
151 |
152 | evaluateImg = self.evaluateImg
153 | maxDet = p.maxDets[-1]
154 | self.evalImgs = [evaluateImg(imgId, catId, areaRng, maxDet)
155 | for catId in catIds
156 | for areaRng in p.areaRng
157 | for imgId in p.imgIds
158 | ]
159 | self._paramsEval = copy.deepcopy(self.params)
160 | toc = time.time()
161 | print('DONE (t={:0.2f}s).'.format(toc-tic))
162 |
163 | def computeIoU(self, imgId, catId):
164 | p = self.params
165 | if p.useCats:
166 | gt = self._gts[imgId,catId]
167 | dt = self._dts[imgId,catId]
168 | else:
169 | gt = [_ for cId in p.catIds for _ in self._gts[imgId,cId]]
170 | dt = [_ for cId in p.catIds for _ in self._dts[imgId,cId]]
171 | if len(gt) == 0 and len(dt) ==0:
172 | return []
173 | inds = np.argsort([-d['score'] for d in dt], kind='mergesort')
174 | dt = [dt[i] for i in inds]
175 | if len(dt) > p.maxDets[-1]:
176 | dt=dt[0:p.maxDets[-1]]
177 |
178 | if p.iouType == 'segm':
179 | g = [g['segmentation'] for g in gt]
180 | d = [d['segmentation'] for d in dt]
181 | elif p.iouType == 'bbox':
182 | g = [g['bbox'] for g in gt]
183 | d = [d['bbox'] for d in dt]
184 | else:
185 | raise Exception('unknown iouType for iou computation')
186 |
187 | # compute iou between each dt and gt region
188 | iscrowd = [int(o['iscrowd']) for o in gt]
189 | ious = maskUtils.iou(d,g,iscrowd)
190 | return ious
191 |
192 | def computeOks(self, imgId, catId):
193 | p = self.params
194 | # dimention here should be Nxm
195 | gts = self._gts[imgId, catId]
196 | dts = self._dts[imgId, catId]
197 | inds = np.argsort([-d['score'] for d in dts], kind='mergesort')
198 | dts = [dts[i] for i in inds]
199 | if len(dts) > p.maxDets[-1]:
200 | dts = dts[0:p.maxDets[-1]]
201 | # if len(gts) == 0 and len(dts) == 0:
202 | if len(gts) == 0 or len(dts) == 0:
203 | return []
204 | ious = np.zeros((len(dts), len(gts)))
205 | sigmas = p.kpt_oks_sigmas
206 | vars = (sigmas * 2)**2
207 | k = len(sigmas)
208 | # compute oks between each detection and ground truth object
209 | for j, gt in enumerate(gts):
210 | # create bounds for ignore regions(double the gt bbox)
211 | g = np.array(gt['keypoints'])
212 | xg = g[0::3]; yg = g[1::3]; vg = g[2::3]
213 | k1 = np.count_nonzero(vg > 0)
214 | bb = gt['bbox']
215 | x0 = bb[0] - bb[2]; x1 = bb[0] + bb[2] * 2
216 | y0 = bb[1] - bb[3]; y1 = bb[1] + bb[3] * 2
217 | for i, dt in enumerate(dts):
218 | d = np.array(dt['keypoints'])
219 | xd = d[0::3]; yd = d[1::3]
220 | if k1>0:
221 | # measure the per-keypoint distance if keypoints visible
222 | dx = xd - xg
223 | dy = yd - yg
224 | else:
225 | # measure minimum distance to keypoints in (x0,y0) & (x1,y1)
226 | z = np.zeros((k))
227 | dx = np.max((z, x0-xd),axis=0)+np.max((z, xd-x1),axis=0)
228 | dy = np.max((z, y0-yd),axis=0)+np.max((z, yd-y1),axis=0)
229 | e = (dx**2 + dy**2) / vars / (gt['area']+np.spacing(1)) / 2
230 | if k1 > 0:
231 | e=e[vg > 0]
232 | ious[i, j] = np.sum(np.exp(-e)) / e.shape[0]
233 | return ious
234 |
235 | def evaluateImg(self, imgId, catId, aRng, maxDet):
236 | '''
237 | perform evaluation for single category and image
238 | :return: dict (single image results)
239 | '''
240 | p = self.params
241 | if p.useCats:
242 | gt = self._gts[imgId,catId]
243 | dt = self._dts[imgId,catId]
244 | else:
245 | gt = [_ for cId in p.catIds for _ in self._gts[imgId,cId]]
246 | dt = [_ for cId in p.catIds for _ in self._dts[imgId,cId]]
247 | if len(gt) == 0 and len(dt) ==0:
248 | return None
249 |
250 | for g in gt:
251 | if g['ignore'] or (g['area']aRng[1]):
252 | g['_ignore'] = 1
253 | else:
254 | g['_ignore'] = 0
255 |
256 | # sort dt highest score first, sort gt ignore last
257 | gtind = np.argsort([g['_ignore'] for g in gt], kind='mergesort')
258 | gt = [gt[i] for i in gtind]
259 | dtind = np.argsort([-d['score'] for d in dt], kind='mergesort')
260 | dt = [dt[i] for i in dtind[0:maxDet]]
261 | iscrowd = [int(o['iscrowd']) for o in gt]
262 | # load computed ious
263 | ious = self.ious[imgId, catId][:, gtind] if len(self.ious[imgId, catId]) > 0 else self.ious[imgId, catId]
264 |
265 | T = len(p.iouThrs)
266 | G = len(gt)
267 | D = len(dt)
268 | gtm = np.zeros((T,G))
269 | dtm = np.zeros((T,D))
270 | gtIg = np.array([g['_ignore'] for g in gt])
271 | dtIg = np.zeros((T,D))
272 | if not len(ious)==0:
273 | for tind, t in enumerate(p.iouThrs):
274 | for dind, d in enumerate(dt):
275 | # information about best match so far (m=-1 -> unmatched)
276 | iou = min([t,1-1e-10])
277 | m = -1
278 | for gind, g in enumerate(gt):
279 | # if this gt already matched, and not a crowd, continue
280 | if gtm[tind,gind]>0 and not iscrowd[gind]:
281 | continue
282 | # if dt matched to reg gt, and on ignore gt, stop
283 | if m>-1 and gtIg[m]==0 and gtIg[gind]==1:
284 | break
285 | # continue to next gt unless better match made
286 | if ious[dind,gind] < iou:
287 | continue
288 | # if match successful and best so far, store appropriately
289 | iou=ious[dind,gind]
290 | m=gind
291 | # if match made store id of match for both dt and gt
292 | if m ==-1:
293 | continue
294 | dtIg[tind,dind] = gtIg[m]
295 | dtm[tind,dind] = gt[m]['id']
296 | gtm[tind,m] = d['id']
297 | # set unmatched detections outside of area range to ignore
298 | a = np.array([d['area']aRng[1] for d in dt]).reshape((1, len(dt)))
299 | dtIg = np.logical_or(dtIg, np.logical_and(dtm==0, np.repeat(a,T,0)))
300 | # store results for given image and category
301 | return {
302 | 'image_id': imgId,
303 | 'category_id': catId,
304 | 'aRng': aRng,
305 | 'maxDet': maxDet,
306 | 'dtIds': [d['id'] for d in dt],
307 | 'gtIds': [g['id'] for g in gt],
308 | 'dtMatches': dtm,
309 | 'gtMatches': gtm,
310 | 'dtScores': [d['score'] for d in dt],
311 | 'gtIgnore': gtIg,
312 | 'dtIgnore': dtIg,
313 | }
314 |
315 | def accumulate(self, p = None):
316 | '''
317 | Accumulate per image evaluation results and store the result in self.eval
318 | :param p: input params for evaluation
319 | :return: None
320 | '''
321 | print('Accumulating evaluation results...')
322 | tic = time.time()
323 | if not self.evalImgs:
324 | print('Please run evaluate() first')
325 | # allows input customized parameters
326 | if p is None:
327 | p = self.params
328 | p.catIds = p.catIds if p.useCats == 1 else [-1]
329 | T = len(p.iouThrs)
330 | R = len(p.recThrs)
331 | K = len(p.catIds) if p.useCats else 1
332 | A = len(p.areaRng)
333 | M = len(p.maxDets)
334 | precision = -np.ones((T,R,K,A,M)) # -1 for the precision of absent categories
335 | recall = -np.ones((T,K,A,M))
336 | scores = -np.ones((T,R,K,A,M))
337 |
338 | # create dictionary for future indexing
339 | _pe = self._paramsEval
340 | catIds = _pe.catIds if _pe.useCats else [-1]
341 | setK = set(catIds)
342 | setA = set(map(tuple, _pe.areaRng))
343 | setM = set(_pe.maxDets)
344 | setI = set(_pe.imgIds)
345 | # get inds to evaluate
346 | k_list = [n for n, k in enumerate(p.catIds) if k in setK]
347 | m_list = [m for n, m in enumerate(p.maxDets) if m in setM]
348 | a_list = [n for n, a in enumerate(map(lambda x: tuple(x), p.areaRng)) if a in setA]
349 | i_list = [n for n, i in enumerate(p.imgIds) if i in setI]
350 | I0 = len(_pe.imgIds)
351 | A0 = len(_pe.areaRng)
352 | # retrieve E at each category, area range, and max number of detections
353 | for k, k0 in enumerate(k_list):
354 | Nk = k0*A0*I0
355 | for a, a0 in enumerate(a_list):
356 | Na = a0*I0
357 | for m, maxDet in enumerate(m_list):
358 | E = [self.evalImgs[Nk + Na + i] for i in i_list]
359 | E = [e for e in E if not e is None]
360 | if len(E) == 0:
361 | continue
362 | dtScores = np.concatenate([e['dtScores'][0:maxDet] for e in E])
363 |
364 | # different sorting method generates slightly different results.
365 | # mergesort is used to be consistent as Matlab implementation.
366 | inds = np.argsort(-dtScores, kind='mergesort')
367 | dtScoresSorted = dtScores[inds]
368 |
369 | dtm = np.concatenate([e['dtMatches'][:,0:maxDet] for e in E], axis=1)[:,inds]
370 | dtIg = np.concatenate([e['dtIgnore'][:,0:maxDet] for e in E], axis=1)[:,inds]
371 | gtIg = np.concatenate([e['gtIgnore'] for e in E])
372 | npig = np.count_nonzero(gtIg==0 )
373 | if npig == 0:
374 | continue
375 | tps = np.logical_and( dtm, np.logical_not(dtIg) )
376 | fps = np.logical_and(np.logical_not(dtm), np.logical_not(dtIg) )
377 |
378 | tp_sum = np.cumsum(tps, axis=1).astype(dtype=float)
379 | fp_sum = np.cumsum(fps, axis=1).astype(dtype=float)
380 | for t, (tp, fp) in enumerate(zip(tp_sum, fp_sum)):
381 | tp = np.array(tp)
382 | fp = np.array(fp)
383 | nd = len(tp)
384 | rc = tp / npig
385 | pr = tp / (fp+tp+np.spacing(1))
386 | q = np.zeros((R,))
387 | ss = np.zeros((R,))
388 |
389 | if nd:
390 | recall[t,k,a,m] = rc[-1]
391 | else:
392 | recall[t,k,a,m] = 0
393 |
394 | # numpy is slow without cython optimization for accessing elements
395 | # use python array gets significant speed improvement
396 | pr = pr.tolist(); q = q.tolist()
397 |
398 | for i in range(nd-1, 0, -1):
399 | if pr[i] > pr[i-1]:
400 | pr[i-1] = pr[i]
401 |
402 | inds = np.searchsorted(rc, p.recThrs, side='left')
403 | try:
404 | for ri, pi in enumerate(inds):
405 | q[ri] = pr[pi]
406 | ss[ri] = dtScoresSorted[pi]
407 | except:
408 | pass
409 | precision[t,:,k,a,m] = np.array(q)
410 | scores[t,:,k,a,m] = np.array(ss)
411 | self.eval = {
412 | 'params': p,
413 | 'counts': [T, R, K, A, M],
414 | 'date': datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
415 | 'precision': precision,
416 | 'recall': recall,
417 | 'scores': scores,
418 | }
419 | toc = time.time()
420 | print('DONE (t={:0.2f}s).'.format( toc-tic))
421 |
422 | def summarize(self):
423 | '''
424 | Compute and display summary metrics for evaluation results.
425 | Note this functin can *only* be applied on the default parameter setting
426 | '''
427 | def _summarize( ap=1, iouThr=None, areaRng='all', maxDets=100 ):
428 | p = self.params
429 | iStr = ' {:<18} {} @[ IoU={:<9} | area={:>6s} | maxDets={:>3d} ] = {:0.3f}'
430 | titleStr = 'Average Precision' if ap == 1 else 'Average Recall'
431 | typeStr = '(AP)' if ap==1 else '(AR)'
432 | iouStr = '{:0.2f}:{:0.2f}'.format(p.iouThrs[0], p.iouThrs[-1]) \
433 | if iouThr is None else '{:0.2f}'.format(iouThr)
434 |
435 | aind = [i for i, aRng in enumerate(p.areaRngLbl) if aRng == areaRng]
436 | mind = [i for i, mDet in enumerate(p.maxDets) if mDet == maxDets]
437 | if ap == 1:
438 | # dimension of precision: [TxRxKxAxM]
439 | s = self.eval['precision']
440 | # IoU
441 | if iouThr is not None:
442 | t = np.where(iouThr == p.iouThrs)[0]
443 | s = s[t]
444 | s = s[:,:,:,aind,mind]
445 | else:
446 | # dimension of recall: [TxKxAxM]
447 | s = self.eval['recall']
448 | if iouThr is not None:
449 | t = np.where(iouThr == p.iouThrs)[0]
450 | s = s[t]
451 | s = s[:,:,aind,mind]
452 | if len(s[s>-1])==0:
453 | mean_s = -1
454 | else:
455 | mean_s = np.mean(s[s>-1])
456 | print(iStr.format(titleStr, typeStr, iouStr, areaRng, maxDets, mean_s))
457 | return mean_s
458 | def _summarizeDets():
459 | stats = np.zeros((12,))
460 | stats[0] = _summarize(1)
461 | stats[1] = _summarize(1, iouThr=.5, maxDets=self.params.maxDets[2])
462 | stats[2] = _summarize(1, iouThr=.75, maxDets=self.params.maxDets[2])
463 | stats[3] = _summarize(1, areaRng='small', maxDets=self.params.maxDets[2])
464 | stats[4] = _summarize(1, areaRng='medium', maxDets=self.params.maxDets[2])
465 | stats[5] = _summarize(1, areaRng='large', maxDets=self.params.maxDets[2])
466 | stats[6] = _summarize(0, maxDets=self.params.maxDets[0])
467 | stats[7] = _summarize(0, maxDets=self.params.maxDets[1])
468 | stats[8] = _summarize(0, maxDets=self.params.maxDets[2])
469 | stats[9] = _summarize(0, areaRng='small', maxDets=self.params.maxDets[2])
470 | stats[10] = _summarize(0, areaRng='medium', maxDets=self.params.maxDets[2])
471 | stats[11] = _summarize(0, areaRng='large', maxDets=self.params.maxDets[2])
472 | return stats
473 | def _summarizeKps():
474 | stats = np.zeros((10,))
475 | stats[0] = _summarize(1, maxDets=20)
476 | stats[1] = _summarize(1, maxDets=20, iouThr=.5)
477 | stats[2] = _summarize(1, maxDets=20, iouThr=.75)
478 | stats[3] = _summarize(1, maxDets=20, areaRng='medium')
479 | stats[4] = _summarize(1, maxDets=20, areaRng='large')
480 | stats[5] = _summarize(0, maxDets=20)
481 | stats[6] = _summarize(0, maxDets=20, iouThr=.5)
482 | stats[7] = _summarize(0, maxDets=20, iouThr=.75)
483 | stats[8] = _summarize(0, maxDets=20, areaRng='medium')
484 | stats[9] = _summarize(0, maxDets=20, areaRng='large')
485 | return stats
486 | if not self.eval:
487 | raise Exception('Please run accumulate() first')
488 | iouType = self.params.iouType
489 | if iouType == 'segm' or iouType == 'bbox':
490 | summarize = _summarizeDets
491 | elif iouType == 'keypoints':
492 | summarize = _summarizeKps
493 | self.stats = summarize()
494 |
495 | def __str__(self):
496 | self.summarize()
497 |
498 | class Params:
499 | '''
500 | Params for coco evaluation api
501 | '''
502 | def setDetParams(self):
503 | self.imgIds = []
504 | self.catIds = []
505 | # np.arange causes trouble. the data point on arange is slightly larger than the true value
506 | self.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)
507 | self.recThrs = np.linspace(.0, 1.00, int(np.round((1.00 - .0) / .01)) + 1, endpoint=True)
508 | self.maxDets = [1, 10, 100]
509 | #self.areaRng = [[0 ** 2, 1e5 ** 2], [0 ** 2, 32 ** 2], [32 ** 2, 96 ** 2], [96 ** 2, 1e5 ** 2]]
510 | #replaced
511 | self.areaRng = [[0 ** 2, 1e5 ** 2], [0 ** 2, 64 ** 2],[64 ** 2, 192 ** 2], [192 ** 2, 1e5 ** 2]]
512 | self.areaRngLbl = ['all', 'small', 'medium', 'large']
513 | self.useCats = 1
514 |
515 | def setKpParams(self):
516 | self.imgIds = []
517 | self.catIds = []
518 | # np.arange causes trouble. the data point on arange is slightly larger than the true value
519 | self.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)
520 | self.recThrs = np.linspace(.0, 1.00, int(np.round((1.00 - .0) / .01)) + 1, endpoint=True)
521 | self.maxDets = [20]
522 | self.areaRng = [[0 ** 2, 1e5 ** 2], [32 ** 2, 96 ** 2], [96 ** 2, 1e5 ** 2]]
523 | self.areaRngLbl = ['all', 'medium', 'large']
524 | self.useCats = 1
525 | self.kpt_oks_sigmas = np.array([.26, .25, .25, .35, .35, .79, .79, .72, .72, .62,.62, 1.07, 1.07, .87, .87, .89, .89])/10.0
526 |
527 | def __init__(self, iouType='segm'):
528 | if iouType == 'segm' or iouType == 'bbox':
529 | self.setDetParams()
530 | elif iouType == 'keypoints':
531 | self.setKpParams()
532 | else:
533 | raise Exception('iouType not supported')
534 | self.iouType = iouType
535 | # useSegm is deprecated
536 | self.useSegm = None
537 |
--------------------------------------------------------------------------------
/evaluation_Linux/pycocotools/mask.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tsungyi'
2 |
3 | import pycocotools._mask as _mask
4 |
5 | # Interface for manipulating masks stored in RLE format.
6 | #
7 | # RLE is a simple yet efficient format for storing binary masks. RLE
8 | # first divides a vector (or vectorized image) into a series of piecewise
9 | # constant regions and then for each piece simply stores the length of
10 | # that piece. For example, given M=[0 0 1 1 1 0 1] the RLE counts would
11 | # be [2 3 1 1], or for M=[1 1 1 1 1 1 0] the counts would be [0 6 1]
12 | # (note that the odd counts are always the numbers of zeros). Instead of
13 | # storing the counts directly, additional compression is achieved with a
14 | # variable bitrate representation based on a common scheme called LEB128.
15 | #
16 | # Compression is greatest given large piecewise constant regions.
17 | # Specifically, the size of the RLE is proportional to the number of
18 | # *boundaries* in M (or for an image the number of boundaries in the y
19 | # direction). Assuming fairly simple shapes, the RLE representation is
20 | # O(sqrt(n)) where n is number of pixels in the object. Hence space usage
21 | # is substantially lower, especially for large simple objects (large n).
22 | #
23 | # Many common operations on masks can be computed directly using the RLE
24 | # (without need for decoding). This includes computations such as area,
25 | # union, intersection, etc. All of these operations are linear in the
26 | # size of the RLE, in other words they are O(sqrt(n)) where n is the area
27 | # of the object. Computing these operations on the original mask is O(n).
28 | # Thus, using the RLE can result in substantial computational savings.
29 | #
30 | # The following API functions are defined:
31 | # encode - Encode binary masks using RLE.
32 | # decode - Decode binary masks encoded via RLE.
33 | # merge - Compute union or intersection of encoded masks.
34 | # iou - Compute intersection over union between masks.
35 | # area - Compute area of encoded masks.
36 | # toBbox - Get bounding boxes surrounding encoded masks.
37 | # frPyObjects - Convert polygon, bbox, and uncompressed RLE to encoded RLE mask.
38 | #
39 | # Usage:
40 | # Rs = encode( masks )
41 | # masks = decode( Rs )
42 | # R = merge( Rs, intersect=false )
43 | # o = iou( dt, gt, iscrowd )
44 | # a = area( Rs )
45 | # bbs = toBbox( Rs )
46 | # Rs = frPyObjects( [pyObjects], h, w )
47 | #
48 | # In the API the following formats are used:
49 | # Rs - [dict] Run-length encoding of binary masks
50 | # R - dict Run-length encoding of binary mask
51 | # masks - [hxwxn] Binary mask(s) (must have type np.ndarray(dtype=uint8) in column-major order)
52 | # iscrowd - [nx1] list of np.ndarray. 1 indicates corresponding gt image has crowd region to ignore
53 | # bbs - [nx4] Bounding box(es) stored as [x y w h]
54 | # poly - Polygon stored as [[x1 y1 x2 y2...],[x1 y1 ...],...] (2D list)
55 | # dt,gt - May be either bounding boxes or encoded masks
56 | # Both poly and bbs are 0-indexed (bbox=[0 0 1 1] encloses first pixel).
57 | #
58 | # Finally, a note about the intersection over union (iou) computation.
59 | # The standard iou of a ground truth (gt) and detected (dt) object is
60 | # iou(gt,dt) = area(intersect(gt,dt)) / area(union(gt,dt))
61 | # For "crowd" regions, we use a modified criteria. If a gt object is
62 | # marked as "iscrowd", we allow a dt to match any subregion of the gt.
63 | # Choosing gt' in the crowd gt that best matches the dt can be done using
64 | # gt'=intersect(dt,gt). Since by definition union(gt',dt)=dt, computing
65 | # iou(gt,dt,iscrowd) = iou(gt',dt) = area(intersect(gt,dt)) / area(dt)
66 | # For crowd gt regions we use this modified criteria above for the iou.
67 | #
68 | # To compile run "python setup.py build_ext --inplace"
69 | # Please do not contact us for help with compiling.
70 | #
71 | # Microsoft COCO Toolbox. version 2.0
72 | # Data, paper, and tutorials available at: http://mscoco.org/
73 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2015.
74 | # Licensed under the Simplified BSD License [see coco/license.txt]
75 |
76 | iou = _mask.iou
77 | merge = _mask.merge
78 | frPyObjects = _mask.frPyObjects
79 |
80 | def encode(bimask):
81 | if len(bimask.shape) == 3:
82 | return _mask.encode(bimask)
83 | elif len(bimask.shape) == 2:
84 | h, w = bimask.shape
85 | return _mask.encode(bimask.reshape((h, w, 1), order='F'))[0]
86 |
87 | def decode(rleObjs):
88 | if type(rleObjs) == list:
89 | return _mask.decode(rleObjs)
90 | else:
91 | return _mask.decode([rleObjs])[:,:,0]
92 |
93 | def area(rleObjs):
94 | if type(rleObjs) == list:
95 | return _mask.area(rleObjs)
96 | else:
97 | return _mask.area([rleObjs])[0]
98 |
99 | def toBbox(rleObjs):
100 | if type(rleObjs) == list:
101 | return _mask.toBbox(rleObjs)
102 | else:
103 | return _mask.toBbox([rleObjs])[0]
--------------------------------------------------------------------------------
/evaluation_Windows/EVALUATION.md:
--------------------------------------------------------------------------------
1 | # Evaluation Metric for FishEye8k dataset
2 |
--------------------------------------------------------------------------------
/evaluation_Windows/README.md:
--------------------------------------------------------------------------------
1 | # Evaluation Metric for FishEye8k dataset
2 |
3 | Update:
4 |
5 | `2024/03/18`: Change the bug in calculating f1 score.
6 |
7 |
8 |
--------------------------------------------------------------------------------
/evaluation_Windows/eval.py:
--------------------------------------------------------------------------------
1 | from pycocotools.coco import COCO
2 | from pycocotools.cocoeval import COCOeval
3 |
4 |
5 | coco_gt = COCO('../../../evaluation/Fisheye8K/groundtruth.json')
6 | coco_dt = coco_gt.loadRes(
7 | '../../../evaluation/Fisheye8K/detections.json')
8 |
9 | print(type(coco_dt))
10 | print(type(coco_gt))
11 |
12 | coco_eval = COCOeval(coco_gt, coco_dt, 'bbox')
13 | coco_eval.evaluate()
14 | coco_eval.accumulate()
15 | coco_eval.summarize()
16 |
--------------------------------------------------------------------------------
/evaluation_Windows/eval_f1.py:
--------------------------------------------------------------------------------
1 | from pycocotools.coco import COCO
2 | from pycocotools.cocoeval_modified import COCOeval
3 | import json
4 |
5 | coco_gt = COCO('groundtruth.json')
6 |
7 | gt_image_ids = coco_gt.getImgIds()
8 |
9 | print("Total images ", len(gt_image_ids))
10 |
11 | with open('detections.json', 'r') as f:
12 | detection_data = json.load(f)
13 |
14 | filtered_detection_data = [
15 | item for item in detection_data if item['image_id'] in gt_image_ids]
16 |
17 | with open('./temp.json', 'w') as f:
18 | json.dump(filtered_detection_data, f)
19 |
20 | coco_dt = coco_gt.loadRes('./temp.json')
21 | coco_eval = COCOeval(coco_gt, coco_dt, 'bbox')
22 | coco_eval.evaluate()
23 | coco_eval.accumulate()
24 | coco_eval.summarize()
25 |
26 | print('----------------------------------------')
27 | print('AP_0.5-0.95', coco_eval.stats[0])
28 | print('AP_0.5', coco_eval.stats[1])
29 | print('AP_S', coco_eval.stats[3])
30 | print('AP_M', coco_eval.stats[4])
31 | print('AP_L', coco_eval.stats[5])
32 | print('f1_score: ', coco_eval.stats[20])
33 | print('----------------------------------------')
34 |
--------------------------------------------------------------------------------
/evaluation_Windows/pycocotools/__init__.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tylin'
2 |
--------------------------------------------------------------------------------
/evaluation_Windows/pycocotools/__pycache__/__init__.cpython-310.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Windows/pycocotools/__pycache__/__init__.cpython-310.pyc
--------------------------------------------------------------------------------
/evaluation_Windows/pycocotools/__pycache__/__init__.cpython-39.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Windows/pycocotools/__pycache__/__init__.cpython-39.pyc
--------------------------------------------------------------------------------
/evaluation_Windows/pycocotools/__pycache__/coco.cpython-310.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Windows/pycocotools/__pycache__/coco.cpython-310.pyc
--------------------------------------------------------------------------------
/evaluation_Windows/pycocotools/__pycache__/coco.cpython-39.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Windows/pycocotools/__pycache__/coco.cpython-39.pyc
--------------------------------------------------------------------------------
/evaluation_Windows/pycocotools/__pycache__/cocoeval.cpython-39.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Windows/pycocotools/__pycache__/cocoeval.cpython-39.pyc
--------------------------------------------------------------------------------
/evaluation_Windows/pycocotools/__pycache__/cocoeval_modified.cpython-39.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Windows/pycocotools/__pycache__/cocoeval_modified.cpython-39.pyc
--------------------------------------------------------------------------------
/evaluation_Windows/pycocotools/__pycache__/mask.cpython-39.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Windows/pycocotools/__pycache__/mask.cpython-39.pyc
--------------------------------------------------------------------------------
/evaluation_Windows/pycocotools/_mask.cp39-win_amd64.pyd:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MoyoG/FishEye8K/bfa5002c49f5d2425ba10cad604d71fed05c79fd/evaluation_Windows/pycocotools/_mask.cp39-win_amd64.pyd
--------------------------------------------------------------------------------
/evaluation_Windows/pycocotools/coco.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tylin'
2 | __version__ = '2.0'
3 | # Interface for accessing the Microsoft COCO dataset.
4 |
5 | # Microsoft COCO is a large image dataset designed for object detection,
6 | # segmentation, and caption generation. pycocotools is a Python API that
7 | # assists in loading, parsing and visualizing the annotations in COCO.
8 | # Please visit http://mscoco.org/ for more information on COCO, including
9 | # for the data, paper, and tutorials. The exact format of the annotations
10 | # is also described on the COCO website. For example usage of the pycocotools
11 | # please see pycocotools_demo.ipynb. In addition to this API, please download both
12 | # the COCO images and annotations in order to run the demo.
13 |
14 | # An alternative to using the API is to load the annotations directly
15 | # into Python dictionary
16 | # Using the API provides additional utility functions. Note that this API
17 | # supports both *instance* and *caption* annotations. In the case of
18 | # captions not all functions are defined (e.g. categories are undefined).
19 |
20 | # The following API functions are defined:
21 | # COCO - COCO api class that loads COCO annotation file and prepare data structures.
22 | # decodeMask - Decode binary mask M encoded via run-length encoding.
23 | # encodeMask - Encode binary mask M using run-length encoding.
24 | # getAnnIds - Get ann ids that satisfy given filter conditions.
25 | # getCatIds - Get cat ids that satisfy given filter conditions.
26 | # getImgIds - Get img ids that satisfy given filter conditions.
27 | # loadAnns - Load anns with the specified ids.
28 | # loadCats - Load cats with the specified ids.
29 | # loadImgs - Load imgs with the specified ids.
30 | # annToMask - Convert segmentation in an annotation to binary mask.
31 | # showAnns - Display the specified annotations.
32 | # loadRes - Load algorithm results and create API for accessing them.
33 | # download - Download COCO images from mscoco.org server.
34 | # Throughout the API "ann"=annotation, "cat"=category, and "img"=image.
35 | # Help on each functions can be accessed by: "help COCO>function".
36 |
37 | # See also COCO>decodeMask,
38 | # COCO>encodeMask, COCO>getAnnIds, COCO>getCatIds,
39 | # COCO>getImgIds, COCO>loadAnns, COCO>loadCats,
40 | # COCO>loadImgs, COCO>annToMask, COCO>showAnns
41 |
42 | # Microsoft COCO Toolbox. version 2.0
43 | # Data, paper, and tutorials available at: http://mscoco.org/
44 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2014.
45 | # Licensed under the Simplified BSD License [see bsd.txt]
46 |
47 | import json
48 | import time
49 | import numpy as np
50 | import copy
51 | import itertools
52 | from . import mask as maskUtils
53 | import os
54 | from collections import defaultdict
55 | import sys
56 | PYTHON_VERSION = sys.version_info[0]
57 | if PYTHON_VERSION == 2:
58 | from urllib import urlretrieve
59 | elif PYTHON_VERSION == 3:
60 | from urllib.request import urlretrieve
61 |
62 |
63 | def _isArrayLike(obj):
64 | return hasattr(obj, '__iter__') and hasattr(obj, '__len__')
65 |
66 |
67 | class COCO:
68 | def __init__(self, annotation_file=None):
69 | """
70 | Constructor of Microsoft COCO helper class for reading and visualizing annotations.
71 | :param annotation_file (str): location of annotation file
72 | :param image_folder (str): location to the folder that hosts images.
73 | :return:
74 | """
75 | # load dataset
76 | self.dataset, self.anns, self.cats, self.imgs = dict(), dict(), dict(), dict()
77 | self.imgToAnns, self.catToImgs = defaultdict(list), defaultdict(list)
78 | if not annotation_file == None:
79 | print('loading annotations into memory...')
80 | tic = time.time()
81 | with open(annotation_file, 'r') as f:
82 | dataset = json.load(f)
83 | assert type(dataset) == dict, 'annotation file format {} not supported'.format(
84 | type(dataset))
85 | print('Done (t={:0.2f}s)'.format(time.time() - tic))
86 | self.dataset = dataset
87 | self.createIndex()
88 |
89 | def createIndex(self):
90 | # create index
91 | print('creating index...')
92 | anns, cats, imgs = {}, {}, {}
93 | imgToAnns, catToImgs = defaultdict(list), defaultdict(list)
94 | if 'annotations' in self.dataset:
95 | for ann in self.dataset['annotations']:
96 | imgToAnns[ann['image_id']].append(ann)
97 | anns[ann['id']] = ann
98 |
99 | if 'images' in self.dataset:
100 | for img in self.dataset['images']:
101 | imgs[img['id']] = img
102 |
103 | if 'categories' in self.dataset:
104 | for cat in self.dataset['categories']:
105 | cats[cat['id']] = cat
106 |
107 | if 'annotations' in self.dataset and 'categories' in self.dataset:
108 | for ann in self.dataset['annotations']:
109 | catToImgs[ann['category_id']].append(ann['image_id'])
110 |
111 | print('index created!')
112 |
113 | # create class members
114 | self.anns = anns
115 | self.imgToAnns = imgToAnns
116 | self.catToImgs = catToImgs
117 | self.imgs = imgs
118 | self.cats = cats
119 |
120 | def info(self):
121 | """
122 | Print information about the annotation file.
123 | :return:
124 | """
125 | for key, value in self.dataset['info'].items():
126 | print('{}: {}'.format(key, value))
127 |
128 | def getAnnIds(self, imgIds=[], catIds=[], areaRng=[], iscrowd=None):
129 | """
130 | Get ann ids that satisfy given filter conditions. default skips that filter
131 | :param imgIds (int array) : get anns for given imgs
132 | catIds (int array) : get anns for given cats
133 | areaRng (float array) : get anns for given area range (e.g. [0 inf])
134 | iscrowd (boolean) : get anns for given crowd label (False or True)
135 | :return: ids (int array) : integer array of ann ids
136 | """
137 | imgIds = imgIds if _isArrayLike(imgIds) else [imgIds]
138 | catIds = catIds if _isArrayLike(catIds) else [catIds]
139 |
140 | if len(imgIds) == len(catIds) == len(areaRng) == 0:
141 | anns = self.dataset['annotations']
142 | else:
143 | if not len(imgIds) == 0:
144 | lists = [self.imgToAnns[imgId]
145 | for imgId in imgIds if imgId in self.imgToAnns]
146 | anns = list(itertools.chain.from_iterable(lists))
147 | else:
148 | anns = self.dataset['annotations']
149 | anns = anns if len(catIds) == 0 else [
150 | ann for ann in anns if ann['category_id'] in catIds]
151 | anns = anns if len(areaRng) == 0 else [
152 | ann for ann in anns if ann['area'] > areaRng[0] and ann['area'] < areaRng[1]]
153 | if not iscrowd == None:
154 | ids = [ann['id'] for ann in anns if ann['iscrowd'] == iscrowd]
155 | else:
156 | ids = [ann['id'] for ann in anns]
157 | return ids
158 |
159 | def getCatIds(self, catNms=[], supNms=[], catIds=[]):
160 | """
161 | filtering parameters. default skips that filter.
162 | :param catNms (str array) : get cats for given cat names
163 | :param supNms (str array) : get cats for given supercategory names
164 | :param catIds (int array) : get cats for given cat ids
165 | :return: ids (int array) : integer array of cat ids
166 | """
167 | catNms = catNms if _isArrayLike(catNms) else [catNms]
168 | supNms = supNms if _isArrayLike(supNms) else [supNms]
169 | catIds = catIds if _isArrayLike(catIds) else [catIds]
170 | if len(catNms) == len(supNms) == len(catIds) == 0:
171 | cats = self.dataset['categories']
172 | else:
173 | cats = self.dataset['categories']
174 |
175 | cats = cats if len(catNms) == 0 else [
176 | cat for cat in cats if cat['name'] in catNms]
177 | cats = cats if len(supNms) == 0 else [
178 | cat for cat in cats if cat['supercategory'] in supNms]
179 | cats = cats if len(catIds) == 0 else [
180 | cat for cat in cats if cat['id'] in catIds]
181 | ids = [cat['id'] for cat in cats]
182 | return ids
183 |
184 | def getImgIds(self, imgIds=[], catIds=[]):
185 | '''
186 | Get img ids that satisfy given filter conditions.
187 | :param imgIds (int array) : get imgs for given ids
188 | :param catIds (int array) : get imgs with all given cats
189 | :return: ids (int array) : integer array of img ids
190 | '''
191 | imgIds = imgIds if _isArrayLike(imgIds) else [imgIds]
192 | catIds = catIds if _isArrayLike(catIds) else [catIds]
193 |
194 | if len(imgIds) == len(catIds) == 0:
195 | ids = self.imgs.keys()
196 | else:
197 | ids = set(imgIds)
198 | for i, catId in enumerate(catIds):
199 | if i == 0 and len(ids) == 0:
200 | ids = set(self.catToImgs[catId])
201 | else:
202 | ids &= set(self.catToImgs[catId])
203 | return list(ids)
204 |
205 | def loadAnns(self, ids=[]):
206 | """
207 | Load anns with the specified ids.
208 | :param ids (int array) : integer ids specifying anns
209 | :return: anns (object array) : loaded ann objects
210 | """
211 | if _isArrayLike(ids):
212 | return [self.anns[id] for id in ids]
213 | elif type(ids) == int:
214 | return [self.anns[ids]]
215 |
216 | def loadCats(self, ids=[]):
217 | """
218 | Load cats with the specified ids.
219 | :param ids (int array) : integer ids specifying cats
220 | :return: cats (object array) : loaded cat objects
221 | """
222 | if _isArrayLike(ids):
223 | return [self.cats[id] for id in ids]
224 | elif type(ids) == int:
225 | return [self.cats[ids]]
226 |
227 | def loadImgs(self, ids=[]):
228 | """
229 | Load anns with the specified ids.
230 | :param ids (int array) : integer ids specifying img
231 | :return: imgs (object array) : loaded img objects
232 | """
233 | if _isArrayLike(ids):
234 | return [self.imgs[id] for id in ids]
235 | elif type(ids) == int:
236 | return [self.imgs[ids]]
237 |
238 | def showAnns(self, anns, draw_bbox=False):
239 | """
240 | Display the specified annotations.
241 | :param anns (array of object): annotations to display
242 | :return: None
243 | """
244 | if len(anns) == 0:
245 | return 0
246 | if 'segmentation' in anns[0] or 'keypoints' in anns[0]:
247 | datasetType = 'instances'
248 | elif 'caption' in anns[0]:
249 | datasetType = 'captions'
250 | else:
251 | raise Exception('datasetType not supported')
252 | if datasetType == 'instances':
253 | import matplotlib.pyplot as plt
254 | from matplotlib.collections import PatchCollection
255 | from matplotlib.patches import Polygon
256 |
257 | ax = plt.gca()
258 | ax.set_autoscale_on(False)
259 | polygons = []
260 | color = []
261 | for ann in anns:
262 | c = (np.random.random((1, 3))*0.6+0.4).tolist()[0]
263 | if 'segmentation' in ann:
264 | if type(ann['segmentation']) == list:
265 | # polygon
266 | for seg in ann['segmentation']:
267 | poly = np.array(seg).reshape((int(len(seg)/2), 2))
268 | polygons.append(Polygon(poly))
269 | color.append(c)
270 | else:
271 | # mask
272 | t = self.imgs[ann['image_id']]
273 | if type(ann['segmentation']['counts']) == list:
274 | rle = maskUtils.frPyObjects(
275 | [ann['segmentation']], t['height'], t['width'])
276 | else:
277 | rle = [ann['segmentation']]
278 | m = maskUtils.decode(rle)
279 | img = np.ones((m.shape[0], m.shape[1], 3))
280 | if ann['iscrowd'] == 1:
281 | color_mask = np.array([2.0, 166.0, 101.0])/255
282 | if ann['iscrowd'] == 0:
283 | color_mask = np.random.random((1, 3)).tolist()[0]
284 | for i in range(3):
285 | img[:, :, i] = color_mask[i]
286 | ax.imshow(np.dstack((img, m*0.5)))
287 | if 'keypoints' in ann and type(ann['keypoints']) == list:
288 | # turn skeleton into zero-based index
289 | sks = np.array(self.loadCats(
290 | ann['category_id'])[0]['skeleton'])-1
291 | kp = np.array(ann['keypoints'])
292 | x = kp[0::3]
293 | y = kp[1::3]
294 | v = kp[2::3]
295 | for sk in sks:
296 | if np.all(v[sk] > 0):
297 | plt.plot(x[sk], y[sk], linewidth=3, color=c)
298 | plt.plot(x[v > 0], y[v > 0], 'o', markersize=8,
299 | markerfacecolor=c, markeredgecolor='k', markeredgewidth=2)
300 | plt.plot(x[v > 1], y[v > 1], 'o', markersize=8,
301 | markerfacecolor=c, markeredgecolor=c, markeredgewidth=2)
302 |
303 | if draw_bbox:
304 | [bbox_x, bbox_y, bbox_w, bbox_h] = ann['bbox']
305 | poly = [[bbox_x, bbox_y], [bbox_x, bbox_y+bbox_h],
306 | [bbox_x+bbox_w, bbox_y+bbox_h], [bbox_x+bbox_w, bbox_y]]
307 | np_poly = np.array(poly).reshape((4, 2))
308 | polygons.append(Polygon(np_poly))
309 | color.append(c)
310 |
311 | p = PatchCollection(polygons, facecolor=color,
312 | linewidths=0, alpha=0.4)
313 | ax.add_collection(p)
314 | p = PatchCollection(polygons, facecolor='none',
315 | edgecolors=color, linewidths=2)
316 | ax.add_collection(p)
317 | elif datasetType == 'captions':
318 | for ann in anns:
319 | print(ann['caption'])
320 |
321 | def loadRes(self, resFile):
322 | """
323 | Load result file and return a result api object.
324 | :param resFile (str) : file name of result file
325 | :return: res (obj) : result api object
326 | """
327 | res = COCO()
328 | res.dataset['images'] = [img for img in self.dataset['images']]
329 |
330 | print('Loading and preparing results...')
331 | tic = time.time()
332 | if type(resFile) == str or (PYTHON_VERSION == 2 and type(resFile) == unicode):
333 | with open(resFile) as f:
334 | anns = json.load(f)
335 | elif type(resFile) == np.ndarray:
336 | anns = self.loadNumpyAnnotations(resFile)
337 | else:
338 | anns = resFile
339 | assert type(anns) == list, 'results in not an array of objects'
340 | annsImgIds = [ann['image_id'] for ann in anns]
341 | assert set(annsImgIds) == (set(annsImgIds) & set(self.getImgIds())), \
342 | 'Results do not correspond to current coco set'
343 | if 'caption' in anns[0]:
344 | imgIds = set([img['id'] for img in res.dataset['images']]) & set(
345 | [ann['image_id'] for ann in anns])
346 | res.dataset['images'] = [
347 | img for img in res.dataset['images'] if img['id'] in imgIds]
348 | for id, ann in enumerate(anns):
349 | ann['id'] = id+1
350 | elif 'bbox' in anns[0] and not anns[0]['bbox'] == []:
351 | res.dataset['categories'] = copy.deepcopy(
352 | self.dataset['categories'])
353 | for id, ann in enumerate(anns):
354 | bb = ann['bbox']
355 | x1, x2, y1, y2 = [bb[0], bb[0]+bb[2], bb[1], bb[1]+bb[3]]
356 | if not 'segmentation' in ann:
357 | ann['segmentation'] = [[x1, y1, x1, y2, x2, y2, x2, y1]]
358 | ann['area'] = bb[2]*bb[3]
359 | ann['id'] = id+1
360 | ann['iscrowd'] = 0
361 | elif 'segmentation' in anns[0]:
362 | res.dataset['categories'] = copy.deepcopy(
363 | self.dataset['categories'])
364 | for id, ann in enumerate(anns):
365 | # now only support compressed RLE format as segmentation results
366 | ann['area'] = maskUtils.area(ann['segmentation'])
367 | if not 'bbox' in ann:
368 | ann['bbox'] = maskUtils.toBbox(ann['segmentation'])
369 | ann['id'] = id+1
370 | ann['iscrowd'] = 0
371 | elif 'keypoints' in anns[0]:
372 | res.dataset['categories'] = copy.deepcopy(
373 | self.dataset['categories'])
374 | for id, ann in enumerate(anns):
375 | s = ann['keypoints']
376 | x = s[0::3]
377 | y = s[1::3]
378 | x0, x1, y0, y1 = np.min(x), np.max(x), np.min(y), np.max(y)
379 | ann['area'] = (x1-x0)*(y1-y0)
380 | ann['id'] = id + 1
381 | ann['bbox'] = [x0, y0, x1-x0, y1-y0]
382 | print('DONE (t={:0.2f}s)'.format(time.time() - tic))
383 |
384 | res.dataset['annotations'] = anns
385 | res.createIndex()
386 | return res
387 |
388 | def download(self, tarDir=None, imgIds=[]):
389 | '''
390 | Download COCO images from mscoco.org server.
391 | :param tarDir (str): COCO results directory name
392 | imgIds (list): images to be downloaded
393 | :return:
394 | '''
395 | if tarDir is None:
396 | print('Please specify target directory')
397 | return -1
398 | if len(imgIds) == 0:
399 | imgs = self.imgs.values()
400 | else:
401 | imgs = self.loadImgs(imgIds)
402 | N = len(imgs)
403 | if not os.path.exists(tarDir):
404 | os.makedirs(tarDir)
405 | for i, img in enumerate(imgs):
406 | tic = time.time()
407 | fname = os.path.join(tarDir, img['file_name'])
408 | if not os.path.exists(fname):
409 | urlretrieve(img['coco_url'], fname)
410 | print('downloaded {}/{} images (t={:0.1f}s)'.format(i, N, time.time() - tic))
411 |
412 | def loadNumpyAnnotations(self, data):
413 | """
414 | Convert result data from a numpy array [Nx7] where each row contains {imageID,x1,y1,w,h,score,class}
415 | :param data (numpy.ndarray)
416 | :return: annotations (python nested list)
417 | """
418 | print('Converting ndarray to lists...')
419 | assert (type(data) == np.ndarray)
420 | print(data.shape)
421 | assert (data.shape[1] == 7)
422 | N = data.shape[0]
423 | ann = []
424 | for i in range(N):
425 | if i % 1000000 == 0:
426 | print('{}/{}'.format(i, N))
427 | ann += [{
428 | 'image_id': int(data[i, 0]),
429 | 'bbox': [data[i, 1], data[i, 2], data[i, 3], data[i, 4]],
430 | 'score': data[i, 5],
431 | 'category_id': int(data[i, 6]),
432 | }]
433 | return ann
434 |
435 | def annToRLE(self, ann):
436 | """
437 | Convert annotation which can be polygons, uncompressed RLE to RLE.
438 | :return: binary mask (numpy 2D array)
439 | """
440 | t = self.imgs[ann['image_id']]
441 | h, w = t['height'], t['width']
442 | segm = ann['segmentation']
443 | if type(segm) == list:
444 | # polygon -- a single object might consist of multiple parts
445 | # we merge all parts into one mask rle code
446 | rles = maskUtils.frPyObjects(segm, h, w)
447 | rle = maskUtils.merge(rles)
448 | elif type(segm['counts']) == list:
449 | # uncompressed RLE
450 | rle = maskUtils.frPyObjects(segm, h, w)
451 | else:
452 | # rle
453 | rle = ann['segmentation']
454 | return rle
455 |
456 | def annToMask(self, ann):
457 | """
458 | Convert annotation which can be polygons, uncompressed RLE, or RLE to binary mask.
459 | :return: binary mask (numpy 2D array)
460 | """
461 | rle = self.annToRLE(ann)
462 | m = maskUtils.decode(rle)
463 | return m
464 |
--------------------------------------------------------------------------------
/evaluation_Windows/pycocotools/mask.py:
--------------------------------------------------------------------------------
1 | __author__ = 'tsungyi'
2 |
3 | import pycocotools._mask as _mask
4 |
5 | # Interface for manipulating masks stored in RLE format.
6 | #
7 | # RLE is a simple yet efficient format for storing binary masks. RLE
8 | # first divides a vector (or vectorized image) into a series of piecewise
9 | # constant regions and then for each piece simply stores the length of
10 | # that piece. For example, given M=[0 0 1 1 1 0 1] the RLE counts would
11 | # be [2 3 1 1], or for M=[1 1 1 1 1 1 0] the counts would be [0 6 1]
12 | # (note that the odd counts are always the numbers of zeros). Instead of
13 | # storing the counts directly, additional compression is achieved with a
14 | # variable bitrate representation based on a common scheme called LEB128.
15 | #
16 | # Compression is greatest given large piecewise constant regions.
17 | # Specifically, the size of the RLE is proportional to the number of
18 | # *boundaries* in M (or for an image the number of boundaries in the y
19 | # direction). Assuming fairly simple shapes, the RLE representation is
20 | # O(sqrt(n)) where n is number of pixels in the object. Hence space usage
21 | # is substantially lower, especially for large simple objects (large n).
22 | #
23 | # Many common operations on masks can be computed directly using the RLE
24 | # (without need for decoding). This includes computations such as area,
25 | # union, intersection, etc. All of these operations are linear in the
26 | # size of the RLE, in other words they are O(sqrt(n)) where n is the area
27 | # of the object. Computing these operations on the original mask is O(n).
28 | # Thus, using the RLE can result in substantial computational savings.
29 | #
30 | # The following API functions are defined:
31 | # encode - Encode binary masks using RLE.
32 | # decode - Decode binary masks encoded via RLE.
33 | # merge - Compute union or intersection of encoded masks.
34 | # iou - Compute intersection over union between masks.
35 | # area - Compute area of encoded masks.
36 | # toBbox - Get bounding boxes surrounding encoded masks.
37 | # frPyObjects - Convert polygon, bbox, and uncompressed RLE to encoded RLE mask.
38 | #
39 | # Usage:
40 | # Rs = encode( masks )
41 | # masks = decode( Rs )
42 | # R = merge( Rs, intersect=false )
43 | # o = iou( dt, gt, iscrowd )
44 | # a = area( Rs )
45 | # bbs = toBbox( Rs )
46 | # Rs = frPyObjects( [pyObjects], h, w )
47 | #
48 | # In the API the following formats are used:
49 | # Rs - [dict] Run-length encoding of binary masks
50 | # R - dict Run-length encoding of binary mask
51 | # masks - [hxwxn] Binary mask(s) (must have type np.ndarray(dtype=uint8) in column-major order)
52 | # iscrowd - [nx1] list of np.ndarray. 1 indicates corresponding gt image has crowd region to ignore
53 | # bbs - [nx4] Bounding box(es) stored as [x y w h]
54 | # poly - Polygon stored as [[x1 y1 x2 y2...],[x1 y1 ...],...] (2D list)
55 | # dt,gt - May be either bounding boxes or encoded masks
56 | # Both poly and bbs are 0-indexed (bbox=[0 0 1 1] encloses first pixel).
57 | #
58 | # Finally, a note about the intersection over union (iou) computation.
59 | # The standard iou of a ground truth (gt) and detected (dt) object is
60 | # iou(gt,dt) = area(intersect(gt,dt)) / area(union(gt,dt))
61 | # For "crowd" regions, we use a modified criteria. If a gt object is
62 | # marked as "iscrowd", we allow a dt to match any subregion of the gt.
63 | # Choosing gt' in the crowd gt that best matches the dt can be done using
64 | # gt'=intersect(dt,gt). Since by definition union(gt',dt)=dt, computing
65 | # iou(gt,dt,iscrowd) = iou(gt',dt) = area(intersect(gt,dt)) / area(dt)
66 | # For crowd gt regions we use this modified criteria above for the iou.
67 | #
68 | # To compile run "python setup.py build_ext --inplace"
69 | # Please do not contact us for help with compiling.
70 | #
71 | # Microsoft COCO Toolbox. version 2.0
72 | # Data, paper, and tutorials available at: http://mscoco.org/
73 | # Code written by Piotr Dollar and Tsung-Yi Lin, 2015.
74 | # Licensed under the Simplified BSD License [see coco/license.txt]
75 |
76 | iou = _mask.iou
77 | merge = _mask.merge
78 | frPyObjects = _mask.frPyObjects
79 |
80 | def encode(bimask):
81 | if len(bimask.shape) == 3:
82 | return _mask.encode(bimask)
83 | elif len(bimask.shape) == 2:
84 | h, w = bimask.shape
85 | return _mask.encode(bimask.reshape((h, w, 1), order='F'))[0]
86 |
87 | def decode(rleObjs):
88 | if type(rleObjs) == list:
89 | return _mask.decode(rleObjs)
90 | else:
91 | return _mask.decode([rleObjs])[:,:,0]
92 |
93 | def area(rleObjs):
94 | if type(rleObjs) == list:
95 | return _mask.area(rleObjs)
96 | else:
97 | return _mask.area([rleObjs])[0]
98 |
99 | def toBbox(rleObjs):
100 | if type(rleObjs) == list:
101 | return _mask.toBbox(rleObjs)
102 | else:
103 | return _mask.toBbox([rleObjs])[0]
--------------------------------------------------------------------------------