├── .gitignore ├── LICENSE ├── README.md ├── baselines ├── collect_statistics.m ├── sample_bing_windows.m ├── sample_gaussian.m └── sample_uniform.m ├── compute_recall_candidates_voc07.m ├── convert_to_rcnn.m ├── data ├── ILSVRC2013_val_annotations.mat ├── ILSVRC2013_val_images.mat ├── classes.mat ├── coco2014_save_annotations.m ├── coco2014_val_annotations.mat ├── pascal_voc07_random_sampling_params.mat ├── pascal_voc07_test_annotations.mat ├── pascal_voc07_test_frcn_aps.mat ├── pascal_voc07_test_frcn_noregr_aps.mat ├── pascal_voc07_test_frcn_pretrained_aps.mat ├── pascal_voc07_test_llda_dpm_aps.mat ├── pascal_voc07_test_rcnn_aps.mat ├── pascal_voc07_test_rcnn_aps_old.mat ├── pascal_voc07_test_recall.mat ├── pascal_voc07_test_repeatability_homographies.mat ├── pascal_voc07_test_scores.mat ├── pascal_voc07_trainval_annotations.mat ├── score_histogram_cache.mat └── short_classes.mat ├── detector_wiggling ├── detector_wiggle_from_iou_score.m └── detector_wiggle_rcnn.m ├── figures ├── FRCN_correlation_per_class_colorbars_voc07.pdf ├── FRCN_correlation_per_class_lines_voc07.pdf ├── FRCN_mAP_recall_area0.5_0.8_voc07.pdf ├── FRCN_mAP_recall_area_voc07.pdf ├── FRCN_mAP_recall_area_voc07_edge_boxes.pdf ├── FRCN_mAP_recall_voc07.pdf ├── FRCN_noregr_correlation_per_class_colorbars_voc07.pdf ├── FRCN_noregr_correlation_per_class_lines_voc07.pdf ├── FRCN_noregr_mAP_recall_area_voc07.pdf ├── FRCN_noregr_mAP_recall_area_voc07_edge_boxes.pdf ├── FRCN_noregr_mAP_recall_voc07.pdf ├── LLDA_DPM_correlation_per_class_bars_voc07.pdf ├── LLDA_DPM_correlation_per_class_colorbars_voc07.pdf ├── LLDA_DPM_correlation_per_class_lines_voc07.pdf ├── LLDA_DPM_mAP_recall_area_voc07.pdf ├── LLDA_DPM_mAP_recall_voc07.pdf ├── RCNN_correlation_per_class_bars_voc07.pdf ├── RCNN_correlation_per_class_colorbars_voc07.pdf ├── RCNN_correlation_per_class_lines_voc07.pdf ├── RCNN_mAP_AR_per_class_voc07.pdf ├── RCNN_mAP_recall_area_voc07.pdf ├── RCNN_mAP_recall_area_voc07_edge_boxes.pdf ├── RCNN_mAP_recall_voc07.pdf ├── all_detectors_correlation_per_class_colorbars_voc07.pdf ├── average_recall_all_datasets.pdf ├── candidate_size_histogram.pdf ├── candidate_size_histogram_legend.pdf ├── coco14_num_candidates_average_recall.pdf ├── coco14_num_candidates_recall_0.50.pdf ├── coco14_num_candidates_recall_0.70.pdf ├── coco14_num_candidates_recall_0.75.pdf ├── coco14_num_candidates_recall_0.80.pdf ├── coco14_recall_100.pdf ├── coco14_recall_1000.pdf ├── coco14_recall_10000.pdf ├── coco14_recall_legend.pdf ├── datasets_size_histogram.pdf ├── datasets_size_histogram_legend.pdf ├── detector_scoremaps_pos_scale_voc07.pdf ├── detector_scoremaps_pos_scale_voc07.png ├── detector_scoremaps_voc07.pdf ├── imagenet_num_candidates_average_recall.pdf ├── imagenet_num_candidates_recall_0.50.pdf ├── imagenet_num_candidates_recall_0.70.pdf ├── imagenet_num_candidates_recall_0.75.pdf ├── imagenet_num_candidates_recall_0.80.pdf ├── imagenet_recall_100.pdf ├── imagenet_recall_1000.pdf ├── imagenet_recall_10000.pdf ├── imagenet_recall_legend.pdf ├── num_candidates_average_recall.pdf ├── num_candidates_recall_0.50.pdf ├── num_candidates_recall_0.70.pdf ├── num_candidates_recall_0.75.pdf ├── num_candidates_recall_0.80.pdf ├── recall_100.pdf ├── recall_1000.pdf ├── recall_10000.pdf ├── recall_1000_voc07_edge_boxes.pdf ├── recall_legend.pdf ├── repeatability_blur.pdf ├── repeatability_gt_blur.pdf ├── repeatability_gt_legend.pdf ├── repeatability_gt_saltnpepper.pdf ├── repeatability_jpeg.pdf ├── repeatability_legend.pdf ├── repeatability_light.pdf ├── repeatability_rotate.pdf ├── repeatability_saltnpepper.pdf └── repeatability_scale.pdf ├── get_config.m ├── method_wrappers ├── gop_num_candidates_interpolation_data.mat ├── rantalankila_num_candidates_interpolation_data.mat ├── readme.txt ├── rigor_num_candidates_interpolation_data.mat ├── run_MCG2015.m ├── run_bing.m ├── run_categ_independent.m ├── run_cpmc.m ├── run_edge_boxes.m ├── run_edge_boxes50.m ├── run_edge_boxes90.m ├── run_endres.m ├── run_felsen_candidates.m ├── run_gop.m ├── run_objectness.m ├── run_randomized_prims.m ├── run_rantalankila.m ├── run_rigor.m └── run_selective_search.m ├── plot_correlation_edge_boxes.m ├── plot_correlation_voc07.m ├── plot_detector_wiggle_voc07.m ├── plot_recall_ILSVRC2013.m ├── plot_recall_all_datasets.m ├── plot_recall_coco2014.m ├── plot_recall_voc07.m ├── plot_size_distribution.m ├── plot_size_distribution_datasets.m ├── recall ├── compute_average_recall.m ├── plot_num_candidates_auc.m ├── plot_overlap_recall_curve.m └── save_recall_voc07.m ├── repeatability ├── project_candidates.m ├── repeatability_blur.m ├── repeatability_generate_images.m ├── repeatability_get_config.m ├── repeatability_jpeg.m ├── repeatability_light.m ├── repeatability_matching.m ├── repeatability_plot.m ├── repeatability_rotate.m ├── repeatability_saltnpepper.m ├── repeatability_scale.m └── run_repeatability.m ├── shared ├── closest_candidates.m ├── deduplicate_proposals.m ├── get_candidates.m ├── get_method_configs.m ├── overlap.m ├── plot_legend.m ├── read_candidates_mat.m └── save_candidates_mat.m ├── startup.m └── util ├── legendshrink.m ├── printpdf.m ├── printpng.m └── tic_toc_print.m /.gitignore: -------------------------------------------------------------------------------- 1 | .history.* 2 | *~ 3 | figures/ 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # Detection Proposals 3 | 4 | Evaluation of detection proposal algorithms. The code belongs to the BMVC paper **How good are detection proposals, really?** and an the PAMI paper **What makes for effective detection proposals?**. Have a look at the [Project Page](http://www.mpi-inf.mpg.de/departments/computer-vision-and-multimodal-computing/research/object-recognition-and-scene-understanding/how-good-are-detection-proposals-really/) for more information. 5 | 6 | 7 | Please contact me if you're having trouble with the code! 8 | 9 | 10 | 11 | ## Plot evaluation curves 12 | 13 | 1. Get the data that you want to use from the [Project Page](http://www.mpi-inf.mpg.de/departments/computer-vision-and-multimodal-computing/research/object-recognition-and-scene-understanding/how-good-are-detection-proposals-really/). 14 | 2. Edit `get_config.m` to point to the right locations for images, candidates and so on. 15 | 3. Make sure you either start matlab in the root directory of the code or run `startup.m` manually once. 16 | 4. Run `plot_recall_voc07.m`, curves will be in the figures subdirectory. 17 | 18 | 19 | 20 | ## Benchmark your own method 21 | 22 | 1. Follow **Plot evaluation curves**. 23 | 2. Write a wrapper function that takes an image and the number of proposal boxes and returns the proposals and scores. Proposals are a nx4 matrix, where n is the number of proposals and every row has the format `[x1 y1 x2 y2]` (x and y are 1-based image coordinates). See `method_wrappers/` for examples. 24 | 2. Add your method to `shared/get_method_configs.m` 25 | 3. Run `compute_recall_candidates_voc07.m` passing only the config of your method as an argument. If your method is slow, you probably want to parallelize it in a cluster. 26 | 4. Run `plot_recall_voc07.m`, curves will be in the figures subdirectory. 27 | 28 | You don't have to use `compute_recall_candidates_voc07.m`, but you can have a look to get an idea about how to save the candidates in the right format so, `plot_recall_voc07.m` will be able to read it. 29 | 30 | 31 | ## Format of the precomputed proposals 32 | 33 | For the recall experiments, there is one mat file per image, each storing the 34 | proposals for that image. There are two cases of how proposals are stored. In 35 | the easiest case the proposals can be filtered after we have generated enough 36 | of them, for example because they are sorted or scored. If this is not the 37 | case we need to rerun the proposal method for different number of proposals 38 | that we would like to evaluate. 39 | 40 | In both cases the each mat file contains the variables `boxes`, `scores`, and 41 | `num_candidates`. 42 | 43 | 44 | #### Proposals can be filtered 45 | 46 | `boxes` is a n-by-4 matrix in which each row is of the format `[x1 y1 x2 y2]` 47 | (x and y are 1-based image coordinates). 48 | 49 | `scores` is a column vector of length n, containing scores for each of the 50 | proposals in `boxes`. How these scores are to be used to find the top k 51 | proposals is specified per method in `shared/get_method_configs.m`. `scores` 52 | can also be empty, then the only valid choice for ordering is `none` (proposals 53 | are already sorted) or `random` (proposals should be shuffled before using the 54 | first k). 55 | 56 | `num_candidates` should be empty or n. 57 | 58 | 59 | #### Proposals cannot be filtered 60 | 61 | In this case the proposal method will be called several times with different 62 | `num_candidates` parameter, so we can make a plot for varying number of 63 | proposals. Let's say we run the proposal method d=4 times with the 64 | `num_candidates` = 10, 100, 1000, 10000. These parameters are saved as a column 65 | vector in the `num_candidates` variable in the mat file. 66 | 67 | `boxes` and `scores` are now cells with d elements. Each element of the cells 68 | are as described above (for proposals that can be filtered) and contain 69 | proposals and scores for each of the d reruns of the proposal method. 70 | 71 | #### Implementation 72 | 73 | If you want more details you can have a look at the files 74 | `compute_recall_candidates_voc07.m` around lines 39-65 and at the functions 75 | that are called there (`read_candidates_mat`, `save_candidates_mat`). 76 | -------------------------------------------------------------------------------- /baselines/collect_statistics.m: -------------------------------------------------------------------------------- 1 | function collect_statistics() 2 | % Estimate the parameters for the uniform and gaussian random sampling 3 | % baselines. 4 | % 5 | % Parameters are estimated on the training set and saved in a mat file. 6 | load('data/pascal_voc07_trainval_annotations.mat'); 7 | 8 | uniform_params = []; 9 | [ar] = get_aspect_ratio(pos); 10 | uniform_params.aspect_ratio = select_uniform_parameters(ar); 11 | figure; hist(log2(ar), 100); xlabel('log(aspect ratio)'); 12 | hold on; 13 | plot(log2([uniform_params.aspect_ratio.min uniform_params.aspect_ratio.max]), ... 14 | [100 100], 'r', 'LineWidth', 2); 15 | 16 | [areas, relative_scale] = get_scales(pos); 17 | uniform_params.relative_scale = select_uniform_parameters(relative_scale); 18 | figure; hist(sqrt(areas), 100); xlabel('sqrt(annotation area)'); 19 | figure; hist(sqrt(relative_scale), 100); xlabel('sqrt(relative annotation area)'); 20 | hold on; 21 | plot(sqrt([uniform_params.relative_scale.min uniform_params.relative_scale.max]), ... 22 | [100 100], 'r', 'LineWidth', 2); 23 | 24 | [relative_x, relative_y] = get_positions(pos); 25 | uniform_params.relative_x = select_uniform_parameters(relative_x); 26 | uniform_params.relative_y = select_uniform_parameters(relative_y); 27 | figure; hist(relative_x, 100); xlabel('relative x position'); 28 | hold on; 29 | plot([uniform_params.relative_x.min uniform_params.relative_x.max], ... 30 | [100 100], 'r', 'LineWidth', 2); 31 | figure; hist(relative_y, 100); xlabel('relative y position'); 32 | hold on; 33 | plot([uniform_params.relative_y.min uniform_params.relative_y.max], ... 34 | [100 100], 'r', 'LineWidth', 2); 35 | 36 | gaussian_params = []; 37 | [gaussian_params.mean, gaussian_params.covar] = estimate_multivariate_gaussian( ... 38 | [log2(ar)', sqrt(relative_scale)', relative_x', relative_y']); 39 | gaussian_params.dimensions = {'log2(aspect ratio)', ... 40 | 'sqrt(relative scale)', 'relative_x', 'relative_y'}; 41 | 42 | save('data/pascal_voc07_random_sampling_params.mat', 'uniform_params', 'gaussian_params'); 43 | end 44 | 45 | 46 | function [param] = select_uniform_parameters(values) 47 | coverage = 0.99; % cover 99% of the data 48 | 49 | param = []; 50 | values = sort(values); 51 | n = numel(values); 52 | fraction = (1 - coverage) / 2; 53 | param.min = values(round(fraction * n)); 54 | param.max = values(round((1 - fraction) * n)); 55 | end 56 | 57 | 58 | function [mean_val, covar] = estimate_multivariate_gaussian(values) 59 | % values: one columns is one 'feature' like x position, rows are different 60 | % data points 61 | mean_val = mean(values, 1); 62 | covar = cov(values); 63 | end 64 | 65 | 66 | function pos = add_image_sizes(pos, image_sizes) 67 | images = unique({pos.im}); 68 | img_ids = {image_sizes.img_id}; 69 | pos(1).img_area = []; 70 | pos(1).img_size = []; 71 | for i = 1:numel(images) 72 | [~,img_id,~] = fileparts(images{i}); 73 | mask = strcmp(img_ids, img_id); 74 | assert(sum(mask) == 1); 75 | img_size = image_sizes(mask).size; 76 | img_area = prod(img_size); 77 | 78 | this_image_idxs = find(strcmp({pos.im}, images{i})); 79 | for idx = this_image_idxs(:)' 80 | pos(idx).img_area = img_area; 81 | pos(idx).img_size = img_size; 82 | end 83 | end 84 | end 85 | 86 | function [ar] = get_aspect_ratio(pos) 87 | w = [pos.x2] - [pos.x1] + 1; 88 | h = [pos.y2] - [pos.y1] + 1; 89 | ar = w ./ h; 90 | end 91 | 92 | function [areas, relative_scale] = get_scales(pos) 93 | w = [pos.x2] - [pos.x1] + 1; 94 | h = [pos.y2] - [pos.y1] + 1; 95 | areas = w .* h; 96 | relative_scale = areas ./ [pos.img_area]; 97 | end 98 | 99 | function [relative_x, relative_y] = get_positions(pos) 100 | mid_x = ([pos.x1] + [pos.x2]) / 2; 101 | mid_y = ([pos.y1] + [pos.y2]) / 2; 102 | sizes = reshape([pos.img_size], 2, numel(pos))'; 103 | relative_x = mid_x ./ sizes(:,1)'; 104 | relative_y = mid_y ./ sizes(:,2)'; 105 | end 106 | -------------------------------------------------------------------------------- /baselines/sample_bing_windows.m: -------------------------------------------------------------------------------- 1 | function [ candidates, scores ] = sample_bing_windows( im, num_samples) 2 | %SAMPLE_BING_WINDOWS Will generate equaly distributed windows in space, 3 | %following Bing sizes 4 | % Bing uses 29 specific sizes, this method spread this sizes homogenously 5 | % inside the image 6 | scores = []; 7 | im_wh = [size(im, 2), size(im, 1)]; 8 | 9 | original_bing_window_sizes = [[512 512 ]; [256, 512 ]; [128, 512 ]; [64, 512 ]; ... 10 | [512, 256 ]; [256, 256 ]; [128, 256 ]; [64, 256 ]; [32, 256 ]; ... 11 | [512, 128 ]; [256, 128 ]; [128, 128 ]; [64, 128 ]; [32, 128 ]; [16, 128 ]; ... 12 | [512, 64 ]; [256, 64 ]; [128, 64 ]; [64, 64 ]; [32, 64 ]; [16, 64 ]; ... 13 | [128, 32 ]; [64, 32 ]; [32, 32 ]; [16, 32 ]; ... 14 | [64, 16 ]; [32, 16 ]; [16, 16 ]]; 15 | 16 | %original_bing_window_sizes = [[64, 128]; [32, 128 ]]; 17 | 18 | original_num_window_sizes = size(original_bing_window_sizes, 1); 19 | 20 | bing_window_sizes = []; 21 | 22 | % we filter window sizes to fit inside the image 23 | for i=1:size(original_bing_window_sizes, 1), 24 | window_wh = original_bing_window_sizes(i, :); 25 | 26 | if(sum(window_wh < im_wh) == 2), 27 | bing_window_sizes = [bing_window_sizes; window_wh]; 28 | else 29 | % the window is disregarded 30 | end 31 | 32 | 33 | end 34 | 35 | num_window_sizes = size(bing_window_sizes, 1); 36 | 37 | assert(num_samples > num_window_sizes); 38 | 39 | 40 | candidates = []; 41 | if num_window_sizes ~= original_num_window_sizes, 42 | % we add one candidate that covers the whole image size 43 | candidates = [0, 0, im_wh]; 44 | end 45 | 46 | 47 | use_v0 = false; 48 | 49 | if use_v0, 50 | % will have as many splits in x and y axis (non-square strides) 51 | 52 | sqrt_num_samples_size = floor(sqrt(num_samples / num_window_sizes)); 53 | samples_per_size = sqrt_num_samples_size * sqrt_num_samples_size; 54 | assert(samples_per_size > 3); 55 | extra_samples = num_samples - (samples_per_size * num_window_sizes) - size(candidates, 1); 56 | delta_samples = ((sqrt_num_samples_size + 1) * (sqrt_num_samples_size + 1)) - samples_per_size; 57 | 58 | divisions_per_size = repmat(sqrt_num_samples_size, 1, num_window_sizes); 59 | 60 | start_extra_size_index = 6; % because we like 256x256 (and smaller) 61 | for i = start_extra_size_index:num_window_sizes, 62 | if extra_samples > delta_samples, 63 | divisions_per_size(i) = divisions_per_size(i) + 1; 64 | extra_samples = extra_samples - delta_samples; 65 | else 66 | fprintf('Added %i extra sizes\n', i - 1); 67 | break; 68 | end 69 | end 70 | 71 | 72 | for i = 1:num_window_sizes, 73 | 74 | num_divisions = divisions_per_size(i); 75 | window_wh = bing_window_sizes(i, :); 76 | x1 = 1; 77 | y1 = 1; 78 | x2 = im_wh(1) - window_wh(1); 79 | y2 = im_wh(2) - window_wh(2); 80 | 81 | assert(x2 > 1); 82 | assert(y2 > 1); 83 | 84 | [xx, yy] = meshgrid(linspace(x1, x2, num_divisions), ... 85 | linspace(y1, y2, num_divisions)); 86 | top_left_xy = [xx(:) yy(:)]; 87 | for j = 1:size(top_left_xy, 1), 88 | xy = top_left_xy(j, :); 89 | window = [xy, xy + window_wh]; 90 | candidates = [candidates; window]; 91 | end 92 | 93 | end 94 | 95 | else 96 | % v1 97 | % will use square strides 98 | 99 | 100 | num_samples_per_size = floor(num_samples / num_window_sizes); 101 | 102 | stride_per_size = zeros(1, num_window_sizes); 103 | total_placed_samples = size(candidates, 1); 104 | for i = 1:num_window_sizes, 105 | 106 | window_wh = bing_window_sizes(i, :); 107 | x2 = im_wh(1) - window_wh(1); 108 | y2 = im_wh(2) - window_wh(2); 109 | 110 | assert(x2 > 0); 111 | assert(y2 > 0); 112 | 113 | block_area = (x2 * y2) / num_samples_per_size; 114 | stride = sqrt(block_area); 115 | assert(stride > 0); 116 | 117 | num_samples_placed = compute_num_samples_placed(im_wh, window_wh, stride); 118 | 119 | while(num_samples_placed > num_samples_per_size) 120 | stride = stride + 1; % larger stride, less placements 121 | num_samples_placed = compute_num_samples_placed(im_wh, window_wh, stride); 122 | end 123 | 124 | assert(num_samples_placed <= num_samples_per_size); 125 | 126 | total_placed_samples = total_placed_samples + num_samples_placed; 127 | assert(total_placed_samples <= num_samples); 128 | 129 | stride_per_size(i) = stride; 130 | end 131 | 132 | assert(total_placed_samples <= num_samples); 133 | 134 | sqrt_num_samples_size = floor(sqrt(num_samples / num_window_sizes)); 135 | samples_per_size = sqrt_num_samples_size * sqrt_num_samples_size; 136 | % assert(samples_per_size > 3); 137 | assert(samples_per_size > 0); 138 | extra_samples = num_samples - total_placed_samples; 139 | assert(extra_samples >= 0); 140 | 141 | start_extra_size_index = 6; % because we like 256x256 (and smaller) 142 | for i = start_extra_size_index:num_window_sizes, 143 | 144 | window_wh = bing_window_sizes(i, :); 145 | stride = stride_per_size(i); 146 | 147 | num_samples_placed = compute_num_samples_placed(im_wh, window_wh, stride); 148 | 149 | new_stride = stride * 0.75; 150 | %new_stride = stride - 1; 151 | new_num_samples_placed = compute_num_samples_placed(im_wh, window_wh, new_stride); 152 | 153 | delta_samples = new_num_samples_placed - num_samples_placed; 154 | 155 | if extra_samples > delta_samples, 156 | stride_per_size(i) = new_stride; 157 | total_placed_samples = total_placed_samples + delta_samples; 158 | extra_samples = extra_samples - delta_samples; 159 | else 160 | fprintf('Added %i extra sizes\n', i - start_extra_size_index); 161 | break; 162 | end 163 | end 164 | 165 | if extra_samples > 0, 166 | fprintf('%i extra_samples remaining\n', extra_samples); 167 | end 168 | 169 | for i = 1:num_window_sizes, 170 | 171 | window_wh = bing_window_sizes(i, :); 172 | stride = stride_per_size(i); 173 | x1 = 1; 174 | y1 = 1; 175 | x2 = im_wh(1) - window_wh(1); 176 | y2 = im_wh(2) - window_wh(2); 177 | 178 | x_dots = x1 + (mod(x2 - x1, stride) / 2): stride : x2; 179 | y_dots = y1 + (mod(y2 - y1, stride) / 2): stride : y2; 180 | 181 | assert(x2 > 0); 182 | assert(y2 > 0); 183 | 184 | [xx, yy] = meshgrid(x_dots, y_dots); 185 | top_left_xy = [xx(:) yy(:)]; 186 | for j = 1:size(top_left_xy, 1), 187 | xy = top_left_xy(j, :); 188 | window = [xy, xy + window_wh]; 189 | candidates = [candidates; window]; 190 | end 191 | 192 | end 193 | 194 | assert(size(candidates, 1) == total_placed_samples); 195 | end 196 | 197 | assert(size(candidates, 1) <= num_samples); 198 | end 199 | 200 | 201 | function [ret] = compute_num_samples_placed(im_wh, window_wh, stride) 202 | x1 = 1; 203 | y1 = 1; 204 | x2 = im_wh(1) - window_wh(1); 205 | y2 = im_wh(2) - window_wh(2); 206 | 207 | x_dots = x1 + (mod(x2 - x1, stride) / 2): stride : x2; 208 | y_dots = y1 + (mod(y2 - y1, stride) / 2): stride : y2; 209 | num_samples_placed = size(x_dots, 2) * size(y_dots, 2); 210 | ret = num_samples_placed; 211 | end 212 | 213 | 214 | -------------------------------------------------------------------------------- /baselines/sample_gaussian.m: -------------------------------------------------------------------------------- 1 | function [candidates, scores] = sample_gaussian(im, num_samples, truncate) 2 | if nargin < 3 3 | truncate = true; 4 | end 5 | load('data/pascal_voc07_random_sampling_params.mat'); 6 | 7 | % we need to reject samples that make no sense, to not totally screw up the 8 | % distribution 9 | invalid = true(num_samples, 1); 10 | features = zeros(num_samples, 4); 11 | while any(invalid) 12 | % only resample invalid ones 13 | t_num_samples = sum(invalid); 14 | features(invalid,:) = mvnrnd(gaussian_params.mean, gaussian_params.covar, t_num_samples); 15 | log_aspect_ratio = features(:,1); 16 | sqrt_relative_scale = features(:,2); 17 | relative_x = features(:,3); 18 | relative_y = features(:,4); 19 | invalid = sqrt_relative_scale <= 0 | sqrt_relative_scale > 1 | ... 20 | relative_x < 0 | relative_x > 1 | relative_y < 0 | relative_y > 1; 21 | end 22 | 23 | relative_scale = sqrt_relative_scale .^ 2; 24 | aspect_ratio = 2 .^ log_aspect_ratio; 25 | 26 | im_width = size(im, 2); 27 | im_height = size(im, 1); 28 | image_area = im_width * im_height; 29 | box_area = relative_scale .* image_area; 30 | h = sqrt(box_area ./ aspect_ratio); 31 | w = box_area ./ h; 32 | h = max(h, 5); 33 | w = max(w, 5); 34 | 35 | candidates = zeros(num_samples, 4); 36 | candidates(:,1) = round(relative_x .* im_width - w/2); 37 | candidates(:,2) = round(relative_y .* im_height - h/2); 38 | candidates(:,3) = round(candidates(:,1) + w - 1); 39 | candidates(:,4) = round(candidates(:,2) + h - 1); 40 | 41 | if truncate 42 | offset = max(0, 1 - candidates(:,1)); 43 | candidates(:,1) = candidates(:,1) + offset; 44 | candidates(:,3) = candidates(:,3) + offset; 45 | 46 | offset = max(0, 1 - candidates(:,2)); 47 | candidates(:,2) = candidates(:,2) + offset; 48 | candidates(:,4) = candidates(:,4) + offset; 49 | 50 | candidates(:,3) = min(im_width, candidates(:,3)); 51 | candidates(:,4) = min(im_height, candidates(:,4)); 52 | 53 | assert(all(candidates(:,1) <= candidates(:,3))); 54 | assert(all(candidates(:,2) <= candidates(:,4))); 55 | end 56 | 57 | scores = []; 58 | scores = -1*ones(num_samples, 1, 'single'); 59 | end 60 | -------------------------------------------------------------------------------- /baselines/sample_uniform.m: -------------------------------------------------------------------------------- 1 | function [candidates, scores] = sample_uniform(im, num_samples, truncate) 2 | if nargin < 3 3 | truncate = true; 4 | end 5 | load('data/pascal_voc07_random_sampling_params.mat'); 6 | 7 | mid_x = random('Uniform', ... 8 | uniform_params.relative_x.min, uniform_params.relative_x.max, ... 9 | [num_samples, 1]); 10 | mid_y = random('Uniform', ... 11 | uniform_params.relative_y.min, uniform_params.relative_y.max, ... 12 | [num_samples, 1]); 13 | relative_scale = random('Uniform', ... 14 | sqrt(uniform_params.relative_scale.min), sqrt(uniform_params.relative_scale.max), ... 15 | [num_samples, 1]) .^ 2; 16 | aspect_ratio = 2 .^ random('Uniform', ... 17 | log2(uniform_params.aspect_ratio.min), log2(uniform_params.aspect_ratio.max), ... 18 | [num_samples, 1]); 19 | % aspect ratio = w/h 20 | im_width = size(im, 2); 21 | im_height = size(im, 1); 22 | image_area = im_width * im_height; 23 | box_area = relative_scale .* image_area; 24 | h = sqrt(box_area ./ aspect_ratio); 25 | w = box_area ./ h; 26 | candidates = zeros(num_samples, 4); 27 | candidates(:,1) = round(mid_x .* im_width - w/2); 28 | candidates(:,2) = round(mid_y .* im_height - h/2); 29 | candidates(:,3) = round(candidates(:,1) + w - 1); 30 | candidates(:,4) = round(candidates(:,2) + h - 1); 31 | 32 | if truncate 33 | candidates(:,1) = max(1, candidates(:,1)); 34 | candidates(:,2) = max(1, candidates(:,2)); 35 | candidates(:,3) = min(im_width, candidates(:,3)); 36 | candidates(:,4) = min(im_height, candidates(:,4)); 37 | end 38 | 39 | scores = []; 40 | scores = -1*ones(num_samples, 1, 'single'); 41 | end 42 | -------------------------------------------------------------------------------- /compute_recall_candidates_voc07.m: -------------------------------------------------------------------------------- 1 | function compute_recall_candidates_voc07(method_configs, images) 2 | % Run proposal methods over the pascal test set. 3 | % method_configs are the configs of the methods you want to run, as 4 | % provided by get_method_configs() 5 | % images the pascal image IDs to run on 6 | % 7 | % This method checks is the candidate detections are already there and only 8 | % computes candidates if they are not found. 9 | num_candidates = 10000; 10 | 11 | if nargin < 1 12 | % default to running all methods 13 | method_configs = get_method_configs(); 14 | end 15 | if nargin < 2 16 | % default to running on the full test set 17 | testset = load('data/pascal_voc07_test_annotations.mat'); 18 | % trainval = load('data/pascal_voc07_trainval_annotations.mat'); 19 | data_config = get_config(); 20 | images = cellfun(@(id) sprintf(data_config.pascal_images, id), {testset.impos.im}, ... 21 | 'UniformOutput', false); 22 | clear testset; 23 | end 24 | 25 | 26 | % seed to milliseconds 27 | seed = str2double(datestr(now,'HHMMSSFFF')); 28 | rng(seed); 29 | 30 | num_iterations = numel(images) * numel(method_configs); 31 | for im_i = 1:numel(images) 32 | im = imread(images{im_i}); 33 | if size(im, 3) == 1 34 | im = repmat(im, [1 1 3]); 35 | end 36 | [~,img_id,~] = fileparts(images{im_i}); 37 | for method_i = 1:numel(method_configs) 38 | progress = (im_i-1) * numel(method_configs) + method_i; 39 | method = method_configs(method_i); 40 | fprintf('computing candidates (%s) %d/%d\n', ... 41 | method.name, progress, num_iterations); 42 | 43 | if numel(method.rerun_num_candidates) > 0 44 | n = numel(method.rerun_num_candidates); 45 | try 46 | read_candidates_mat(method.candidate_dir, img_id); 47 | continue; 48 | catch 49 | end 50 | candidates = cell(n, 1); 51 | scores = cell(n, 1); 52 | for cand_idx = 1:n 53 | t_num_candidates = method.rerun_num_candidates(cand_idx); 54 | [candidates{cand_idx}, scores{cand_idx}] = method.extract(im, t_num_candidates); 55 | end 56 | save_candidates_mat(method.candidate_dir, img_id, candidates, scores, method.rerun_num_candidates); 57 | else 58 | try 59 | read_candidates_mat(method.candidate_dir, img_id); 60 | continue; 61 | catch 62 | end 63 | t_num_candidates = num_candidates; 64 | if ~isempty(method.gt_recall_num_candidates) 65 | t_num_candidates = method.gt_recall_num_candidates; 66 | end 67 | [candidates, scores] = method.extract(im, t_num_candidates); 68 | save_candidates_mat(method.candidate_dir, img_id, candidates, scores); 69 | end 70 | end 71 | end 72 | end 73 | -------------------------------------------------------------------------------- /convert_to_rcnn.m: -------------------------------------------------------------------------------- 1 | function convert_to_rcnn() 2 | method_configs = get_method_configs(); 3 | dirs = {'objectness', 'rahtu', 'selective_search_bmvc', ... 4 | 'randomized_prims', 'random_uniform', 'random_gaussian', ... 5 | 'CPMC', 'bing', 'endres', 'rantalankila', 'superpixels', ... 6 | 'sliding_window', 'edge_boxes_70', 'edge_boxes_50', ... 7 | 'MCG', 'edge_boxes_90', 'rigor', 'geodesic', 'edge_boxes_60', ... 8 | 'edge_boxes_80', 'edge_boxes_55', 'edge_boxes_65', 'edge_boxes_75', ... 9 | 'edge_boxes_85', 'edge_boxes_AR'}; 10 | 11 | for i = 1:numel(method_configs) 12 | dest_dir = fullfile('/BS/hosang/work/src/rcnn/data/', dirs{i}); 13 | convert(method_configs(i), dest_dir); 14 | end 15 | end 16 | 17 | function convert(config, dest_dir, num_proposals) 18 | if nargin < 3 19 | num_proposals = 1000; 20 | end 21 | assert(numel(config) == 1); 22 | 23 | if ~exist(dest_dir, 'dir') 24 | mkdir(dest_dir); 25 | end 26 | 27 | fprintf('\nconverting %d boxes of %s...\n', num_proposals, config.name); 28 | test = load('data/pascal_voc07_test_annotations.mat'); 29 | train = load('data/pascal_voc07_trainval_annotations.mat'); 30 | save_to([dest_dir '/voc_2007_test.mat'], config, test, num_proposals); 31 | save_to([dest_dir '/voc_2007_trainval.mat'], config, train, num_proposals); 32 | end 33 | 34 | function save_to(output_filename, config, image_set, num_proposals) 35 | try 36 | load(output_filename, 'images', 'boxes'); 37 | % success 38 | fprintf('already exported\n'); 39 | return; 40 | catch 41 | end 42 | 43 | try 44 | images = {image_set.impos.im}; 45 | boxes = cell(size(images)); 46 | for i = 1:numel(images) 47 | [boxes{i}, ~] = get_candidates(config, images{i}, ... 48 | num_proposals); 49 | % change format from [x1 y1 x2 y2] to [y1 x1 y2 x2] 50 | if ~isempty(boxes{i}) 51 | boxes{i} = boxes{i}(:, [2 1 4 3]); 52 | end 53 | end 54 | save(output_filename, 'images', 'boxes'); 55 | fprintf('wrote %s\n', output_filename); 56 | catch 57 | fprintf('error! probably not computed yet\n'); 58 | end 59 | end 60 | -------------------------------------------------------------------------------- /data/ILSVRC2013_val_annotations.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/ILSVRC2013_val_annotations.mat -------------------------------------------------------------------------------- /data/ILSVRC2013_val_images.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/ILSVRC2013_val_images.mat -------------------------------------------------------------------------------- /data/classes.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/classes.mat -------------------------------------------------------------------------------- /data/coco2014_save_annotations.m: -------------------------------------------------------------------------------- 1 | function save_annotations_to_eval_format() 2 | dataDir='/BS/databases/coco/tools'; dataType='val2014'; 3 | annFile=sprintf('%s/annotations/instances_%s.json',dataDir,dataType); 4 | if(~exist('coco','var')), coco=CocoApi(annFile); end 5 | 6 | imgIds = coco.getImgIds(); 7 | impos = []; 8 | pos = []; 9 | for i = 1:numel(imgIds), imgId = imgIds(i); 10 | if mod(i, 100) == 0 11 | fprintf('%d/%d extracing annotations\n', i, numel(imgIds)); 12 | end 13 | img = coco.loadImgs(imgId); 14 | % filename = sprintf('%s/images/%s/%s',dataDir,dataType,img.file_name); 15 | im_id = sprintf('%s/%s',dataType,img.file_name); 16 | 17 | annIds = coco.getAnnIds('imgIds',imgId,'iscrowd',0); 18 | anns = coco.loadAnns(annIds); 19 | boxes = single(cat(1,anns.bbox)); 20 | if ~isempty(boxes) 21 | % convert from [x y w h] to [x1 y1 x2 y2] 22 | boxes(:,3:4) = boxes(:,1:2) + boxes(:,3:4); 23 | end 24 | 25 | idx = numel(impos)+1; 26 | impos(idx).im = im_id; 27 | impos(idx).img_size = [img.height, img.width]; 28 | impos(idx).img_area = prod(impos(idx).img_size); 29 | impos(idx).boxes = boxes; 30 | 31 | for j = 1:size(boxes,1) 32 | idx = numel(pos)+1; 33 | pos(idx).im = im_id; 34 | pos(idx).x1 = boxes(j,1); 35 | pos(idx).y1 = boxes(j,2); 36 | pos(idx).x2 = boxes(j,3); 37 | pos(idx).y2 = boxes(j,4); 38 | pos(idx).boxes = boxes(j,:); 39 | pos(idx).img_area = impos(end).img_area; 40 | pos(idx).img_size = impos(end).img_size; 41 | end 42 | end 43 | 44 | save(['coco_' dataType '.mat'], 'impos', 'pos'); 45 | end 46 | -------------------------------------------------------------------------------- /data/coco2014_val_annotations.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/coco2014_val_annotations.mat -------------------------------------------------------------------------------- /data/pascal_voc07_random_sampling_params.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/pascal_voc07_random_sampling_params.mat -------------------------------------------------------------------------------- /data/pascal_voc07_test_annotations.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/pascal_voc07_test_annotations.mat -------------------------------------------------------------------------------- /data/pascal_voc07_test_frcn_aps.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/pascal_voc07_test_frcn_aps.mat -------------------------------------------------------------------------------- /data/pascal_voc07_test_frcn_noregr_aps.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/pascal_voc07_test_frcn_noregr_aps.mat -------------------------------------------------------------------------------- /data/pascal_voc07_test_frcn_pretrained_aps.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/pascal_voc07_test_frcn_pretrained_aps.mat -------------------------------------------------------------------------------- /data/pascal_voc07_test_llda_dpm_aps.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/pascal_voc07_test_llda_dpm_aps.mat -------------------------------------------------------------------------------- /data/pascal_voc07_test_rcnn_aps.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/pascal_voc07_test_rcnn_aps.mat -------------------------------------------------------------------------------- /data/pascal_voc07_test_rcnn_aps_old.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/pascal_voc07_test_rcnn_aps_old.mat -------------------------------------------------------------------------------- /data/pascal_voc07_test_recall.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/pascal_voc07_test_recall.mat -------------------------------------------------------------------------------- /data/pascal_voc07_test_repeatability_homographies.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/pascal_voc07_test_repeatability_homographies.mat -------------------------------------------------------------------------------- /data/pascal_voc07_test_scores.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/pascal_voc07_test_scores.mat -------------------------------------------------------------------------------- /data/pascal_voc07_trainval_annotations.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/pascal_voc07_trainval_annotations.mat -------------------------------------------------------------------------------- /data/score_histogram_cache.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/score_histogram_cache.mat -------------------------------------------------------------------------------- /data/short_classes.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/data/short_classes.mat -------------------------------------------------------------------------------- /detector_wiggling/detector_wiggle_from_iou_score.m: -------------------------------------------------------------------------------- 1 | function [overlaps, score, class_ids, avg_cube, avg_cube_per_class] = detector_wiggle_from_iou_score(filename) 2 | ld = load(filename, 'ious', 'scores', 'classes'); 3 | 4 | avg_cube = []; 5 | avg_cube_per_class = []; 6 | score = ld.scores; 7 | overlaps = ld.ious; 8 | class_ids = ld.classes; 9 | end -------------------------------------------------------------------------------- /detector_wiggling/detector_wiggle_rcnn.m: -------------------------------------------------------------------------------- 1 | function [overlaps, score, class_ids, avg_cube, avg_cube_per_class] = detector_wiggle_rcnn(filename) 2 | ld = load(filename); 3 | scoremaps = ld.scoremaps; 4 | 5 | n = numel(scoremaps); 6 | all_overlap_scores = cell(n,1); 7 | for i = 1:n, s = scoremaps(i); 8 | gt = double(s.gt_box); 9 | boxes = double(s.boxes); 10 | invalid = isinf(boxes(:,5)) | isnan(boxes(:,5)); 11 | boxes = boxes(~invalid,:); 12 | 13 | t_overlap = overlap(gt, boxes); 14 | t_class_ids = repmat(s.class, size(t_overlap)); 15 | all_overlap_scores{i} = cat(2, t_overlap, boxes(:,5), double(t_class_ids)); 16 | end 17 | 18 | all_overlap_scores = cat(1, all_overlap_scores{:}); 19 | overlaps = all_overlap_scores(:,1); 20 | score = all_overlap_scores(:,2); 21 | class_ids = all_overlap_scores(:,3); 22 | 23 | avg_cube = []; 24 | avg_cube_per_class = []; 25 | if isfield(scoremaps, 'score_map') && ~isempty(scoremaps(1).score_map) 26 | all_maps = cat(4,scoremaps.score_map); 27 | avg_cube = nanmean(all_maps, 4); 28 | 29 | classes = unique([scoremaps.class]); 30 | avg_cube_per_class = cell(1, numel(classes)); 31 | for c = 1:numel(classes) 32 | t_scoremaps = scoremaps([scoremaps.class] == classes(c)); 33 | avg_cube_per_class{c} = nanmean(cat(4,t_scoremaps.score_map), 4); 34 | end 35 | end 36 | end 37 | -------------------------------------------------------------------------------- /figures/FRCN_correlation_per_class_colorbars_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/FRCN_correlation_per_class_colorbars_voc07.pdf -------------------------------------------------------------------------------- /figures/FRCN_correlation_per_class_lines_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/FRCN_correlation_per_class_lines_voc07.pdf -------------------------------------------------------------------------------- /figures/FRCN_mAP_recall_area0.5_0.8_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/FRCN_mAP_recall_area0.5_0.8_voc07.pdf -------------------------------------------------------------------------------- /figures/FRCN_mAP_recall_area_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/FRCN_mAP_recall_area_voc07.pdf -------------------------------------------------------------------------------- /figures/FRCN_mAP_recall_area_voc07_edge_boxes.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/FRCN_mAP_recall_area_voc07_edge_boxes.pdf -------------------------------------------------------------------------------- /figures/FRCN_mAP_recall_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/FRCN_mAP_recall_voc07.pdf -------------------------------------------------------------------------------- /figures/FRCN_noregr_correlation_per_class_colorbars_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/FRCN_noregr_correlation_per_class_colorbars_voc07.pdf -------------------------------------------------------------------------------- /figures/FRCN_noregr_correlation_per_class_lines_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/FRCN_noregr_correlation_per_class_lines_voc07.pdf -------------------------------------------------------------------------------- /figures/FRCN_noregr_mAP_recall_area_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/FRCN_noregr_mAP_recall_area_voc07.pdf -------------------------------------------------------------------------------- /figures/FRCN_noregr_mAP_recall_area_voc07_edge_boxes.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/FRCN_noregr_mAP_recall_area_voc07_edge_boxes.pdf -------------------------------------------------------------------------------- /figures/FRCN_noregr_mAP_recall_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/FRCN_noregr_mAP_recall_voc07.pdf -------------------------------------------------------------------------------- /figures/LLDA_DPM_correlation_per_class_bars_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/LLDA_DPM_correlation_per_class_bars_voc07.pdf -------------------------------------------------------------------------------- /figures/LLDA_DPM_correlation_per_class_colorbars_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/LLDA_DPM_correlation_per_class_colorbars_voc07.pdf -------------------------------------------------------------------------------- /figures/LLDA_DPM_correlation_per_class_lines_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/LLDA_DPM_correlation_per_class_lines_voc07.pdf -------------------------------------------------------------------------------- /figures/LLDA_DPM_mAP_recall_area_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/LLDA_DPM_mAP_recall_area_voc07.pdf -------------------------------------------------------------------------------- /figures/LLDA_DPM_mAP_recall_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/LLDA_DPM_mAP_recall_voc07.pdf -------------------------------------------------------------------------------- /figures/RCNN_correlation_per_class_bars_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/RCNN_correlation_per_class_bars_voc07.pdf -------------------------------------------------------------------------------- /figures/RCNN_correlation_per_class_colorbars_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/RCNN_correlation_per_class_colorbars_voc07.pdf -------------------------------------------------------------------------------- /figures/RCNN_correlation_per_class_lines_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/RCNN_correlation_per_class_lines_voc07.pdf -------------------------------------------------------------------------------- /figures/RCNN_mAP_AR_per_class_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/RCNN_mAP_AR_per_class_voc07.pdf -------------------------------------------------------------------------------- /figures/RCNN_mAP_recall_area_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/RCNN_mAP_recall_area_voc07.pdf -------------------------------------------------------------------------------- /figures/RCNN_mAP_recall_area_voc07_edge_boxes.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/RCNN_mAP_recall_area_voc07_edge_boxes.pdf -------------------------------------------------------------------------------- /figures/RCNN_mAP_recall_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/RCNN_mAP_recall_voc07.pdf -------------------------------------------------------------------------------- /figures/all_detectors_correlation_per_class_colorbars_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/all_detectors_correlation_per_class_colorbars_voc07.pdf -------------------------------------------------------------------------------- /figures/average_recall_all_datasets.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/average_recall_all_datasets.pdf -------------------------------------------------------------------------------- /figures/candidate_size_histogram.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/candidate_size_histogram.pdf -------------------------------------------------------------------------------- /figures/candidate_size_histogram_legend.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/candidate_size_histogram_legend.pdf -------------------------------------------------------------------------------- /figures/coco14_num_candidates_average_recall.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/coco14_num_candidates_average_recall.pdf -------------------------------------------------------------------------------- /figures/coco14_num_candidates_recall_0.50.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/coco14_num_candidates_recall_0.50.pdf -------------------------------------------------------------------------------- /figures/coco14_num_candidates_recall_0.70.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/coco14_num_candidates_recall_0.70.pdf -------------------------------------------------------------------------------- /figures/coco14_num_candidates_recall_0.75.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/coco14_num_candidates_recall_0.75.pdf -------------------------------------------------------------------------------- /figures/coco14_num_candidates_recall_0.80.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/coco14_num_candidates_recall_0.80.pdf -------------------------------------------------------------------------------- /figures/coco14_recall_100.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/coco14_recall_100.pdf -------------------------------------------------------------------------------- /figures/coco14_recall_1000.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/coco14_recall_1000.pdf -------------------------------------------------------------------------------- /figures/coco14_recall_10000.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/coco14_recall_10000.pdf -------------------------------------------------------------------------------- /figures/coco14_recall_legend.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/coco14_recall_legend.pdf -------------------------------------------------------------------------------- /figures/datasets_size_histogram.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/datasets_size_histogram.pdf -------------------------------------------------------------------------------- /figures/datasets_size_histogram_legend.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/datasets_size_histogram_legend.pdf -------------------------------------------------------------------------------- /figures/detector_scoremaps_pos_scale_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/detector_scoremaps_pos_scale_voc07.pdf -------------------------------------------------------------------------------- /figures/detector_scoremaps_pos_scale_voc07.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/detector_scoremaps_pos_scale_voc07.png -------------------------------------------------------------------------------- /figures/detector_scoremaps_voc07.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/detector_scoremaps_voc07.pdf -------------------------------------------------------------------------------- /figures/imagenet_num_candidates_average_recall.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/imagenet_num_candidates_average_recall.pdf -------------------------------------------------------------------------------- /figures/imagenet_num_candidates_recall_0.50.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/imagenet_num_candidates_recall_0.50.pdf -------------------------------------------------------------------------------- /figures/imagenet_num_candidates_recall_0.70.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/imagenet_num_candidates_recall_0.70.pdf -------------------------------------------------------------------------------- /figures/imagenet_num_candidates_recall_0.75.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/imagenet_num_candidates_recall_0.75.pdf -------------------------------------------------------------------------------- /figures/imagenet_num_candidates_recall_0.80.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/imagenet_num_candidates_recall_0.80.pdf -------------------------------------------------------------------------------- /figures/imagenet_recall_100.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/imagenet_recall_100.pdf -------------------------------------------------------------------------------- /figures/imagenet_recall_1000.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/imagenet_recall_1000.pdf -------------------------------------------------------------------------------- /figures/imagenet_recall_10000.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/imagenet_recall_10000.pdf -------------------------------------------------------------------------------- /figures/imagenet_recall_legend.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/imagenet_recall_legend.pdf -------------------------------------------------------------------------------- /figures/num_candidates_average_recall.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/num_candidates_average_recall.pdf -------------------------------------------------------------------------------- /figures/num_candidates_recall_0.50.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/num_candidates_recall_0.50.pdf -------------------------------------------------------------------------------- /figures/num_candidates_recall_0.70.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/num_candidates_recall_0.70.pdf -------------------------------------------------------------------------------- /figures/num_candidates_recall_0.75.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/num_candidates_recall_0.75.pdf -------------------------------------------------------------------------------- /figures/num_candidates_recall_0.80.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/num_candidates_recall_0.80.pdf -------------------------------------------------------------------------------- /figures/recall_100.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/recall_100.pdf -------------------------------------------------------------------------------- /figures/recall_1000.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/recall_1000.pdf -------------------------------------------------------------------------------- /figures/recall_10000.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/recall_10000.pdf -------------------------------------------------------------------------------- /figures/recall_1000_voc07_edge_boxes.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/recall_1000_voc07_edge_boxes.pdf -------------------------------------------------------------------------------- /figures/recall_legend.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/recall_legend.pdf -------------------------------------------------------------------------------- /figures/repeatability_blur.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/repeatability_blur.pdf -------------------------------------------------------------------------------- /figures/repeatability_gt_blur.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/repeatability_gt_blur.pdf -------------------------------------------------------------------------------- /figures/repeatability_gt_legend.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/repeatability_gt_legend.pdf -------------------------------------------------------------------------------- /figures/repeatability_gt_saltnpepper.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/repeatability_gt_saltnpepper.pdf -------------------------------------------------------------------------------- /figures/repeatability_jpeg.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/repeatability_jpeg.pdf -------------------------------------------------------------------------------- /figures/repeatability_legend.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/repeatability_legend.pdf -------------------------------------------------------------------------------- /figures/repeatability_light.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/repeatability_light.pdf -------------------------------------------------------------------------------- /figures/repeatability_rotate.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/repeatability_rotate.pdf -------------------------------------------------------------------------------- /figures/repeatability_saltnpepper.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/repeatability_saltnpepper.pdf -------------------------------------------------------------------------------- /figures/repeatability_scale.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/figures/repeatability_scale.pdf -------------------------------------------------------------------------------- /get_config.m: -------------------------------------------------------------------------------- 1 | function [config] = get_config() 2 | 3 | config.pascal_images = '/BS/hosang/work/VOC2007/VOCdevkit/VOC2007/JPEGImages/%s.jpg'; 4 | config.transformed_pascal_images = '/var/tmp/transformed_pascal/%s.png'; 5 | 6 | % this is where candidates and all other computed data are stored 7 | % for pascal and imagenet 8 | config.precomputed_candidates = '/BS/candidate_detections/archive00/precomputed'; 9 | % for coco 10 | config.precomputed_candidates_coco = '/BS/candidate_detections/archive00/precomputed-coco'; 11 | end 12 | -------------------------------------------------------------------------------- /method_wrappers/gop_num_candidates_interpolation_data.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/method_wrappers/gop_num_candidates_interpolation_data.mat -------------------------------------------------------------------------------- /method_wrappers/rantalankila_num_candidates_interpolation_data.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/method_wrappers/rantalankila_num_candidates_interpolation_data.mat -------------------------------------------------------------------------------- /method_wrappers/readme.txt: -------------------------------------------------------------------------------- 1 | This are the wrapper scripts for each proposal method that I used. The point 2 | of this code is to provide reproducability of the results and document which 3 | parameters were used for each of the methods. 4 | 5 | This code may also ease using those methods, but I also had to modify the code 6 | of some of those methods to be able to run multiple instances at the same time 7 | in a cluster. You have been warned. :) 8 | -------------------------------------------------------------------------------- /method_wrappers/rigor_num_candidates_interpolation_data.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hosang/detection-proposals/858368afffde5ff4028020fcb1dd4381705ccbfb/method_wrappers/rigor_num_candidates_interpolation_data.mat -------------------------------------------------------------------------------- /method_wrappers/run_MCG2015.m: -------------------------------------------------------------------------------- 1 | function [candidates, score] = run_MCG2015(im, num_candidates) 2 | 3 | if ~isdeployed 4 | old_path = path; 5 | 6 | root_dir = '../mcg/pre-trained'; 7 | addpath(root_dir); 8 | addpath(fullfile(root_dir,'lib')); 9 | addpath(fullfile(root_dir,'scripts')); 10 | addpath(fullfile(root_dir,'datasets')); 11 | addpath(genpath(fullfile(root_dir,'src'))); 12 | end 13 | 14 | tic; 15 | % Test the 'accurate' version, which tackes around 30 seconds in mean 16 | [candidates_mcg, ~] = im2mcg(im,'accurate'); 17 | % flip x and y coordinates 18 | candidates = candidates_mcg.bboxes(:,[2 1 4 3]); 19 | score = candidates_mcg.bboxes_scores; 20 | 21 | toc; 22 | 23 | if ~isdeployed 24 | path(old_path); 25 | end 26 | end 27 | -------------------------------------------------------------------------------- /method_wrappers/run_bing.m: -------------------------------------------------------------------------------- 1 | function [candidates, score] = run_bing(im, num_candidates) 2 | bing_path = '/path/to/bing.sh'; 3 | model_prefix = '/path/to/BING-Objectness/VOC2007/Results/ObjNessB2W8MAXBGR'; 4 | 5 | [ret_code, tmp_dir] = system('mktemp -d'); 6 | assert(ret_code == 0); 7 | tmp_dir = strtrim(tmp_dir); 8 | fprintf('created %s\n', tmp_dir); 9 | 10 | img_filename = fullfile(tmp_dir, 'foo.png'); 11 | imwrite(im, img_filename); 12 | 13 | candidates_file = fullfile(tmp_dir, 'candidates.txt'); 14 | 15 | cmd = sprintf('%s -i %s -o %s -m %s -d %d', ... 16 | bing_path, img_filename, candidates_file, model_prefix, num_candidates); 17 | [ret_code] = system(cmd); 18 | assert(ret_code == 0); 19 | 20 | [score, x1, y1, x2, y2] = textread(candidates_file, '%f %d %d %d %d', 'headerlines', 1); 21 | candidates = [x1 y1 x2 y2]; 22 | 23 | system(sprintf('rm -rf "%s"', tmp_dir)); 24 | end 25 | -------------------------------------------------------------------------------- /method_wrappers/run_categ_independent.m: -------------------------------------------------------------------------------- 1 | function [candidates, scores] = run_categ_independent(im, num_candidates) 2 | if ~isdeployed 3 | old_path = path; 4 | old_pwd = pwd; 5 | cd(fullfile('..', 'category_independen_detection')); 6 | end 7 | 8 | [candidates, scores] = mvg_runObjectDetection(im, num_candidates); 9 | 10 | if ~isdeployed 11 | cd(old_pwd); 12 | path(old_path); 13 | end 14 | end 15 | 16 | -------------------------------------------------------------------------------- /method_wrappers/run_cpmc.m: -------------------------------------------------------------------------------- 1 | function [candidates, scores] = run_cpmc(im, num_candidates) 2 | [ret_code, exp_dir] = system('mktemp -d'); 3 | assert(ret_code == 0); 4 | exp_dir = [strtrim(exp_dir) '/']; 5 | 6 | mkdir(exp_dir, 'PB'); 7 | mkdir(exp_dir, 'WindowsOfInterest/grid_sampler'); 8 | mkdir(exp_dir, 'MySegmentRankers'); 9 | copyfile('/path/to/cpmc_release1/data/MySegmentRankers/attention_model_fewfeats_lambda_10.00_train*.mat', fullfile(exp_dir, 'MySegmentRankers')); 10 | mkdir(exp_dir, 'MyCodebooks'); 11 | copyfile('/path/to/cpmc_release1/data/MyCodebooks/*.mat', fullfile(exp_dir, 'MyCodebooks')); 12 | 13 | fprintf('created %s\n', exp_dir); 14 | img_name = 'foo'; 15 | img_folder = [exp_dir 'JPEGImages/']; 16 | mkdir(img_folder); 17 | img_filename = [img_folder img_name '.jpg']; 18 | imwrite(im, img_filename); 19 | fprintf('wrote %s\n', img_filename); 20 | 21 | if ~isdeployed 22 | old_path = path; 23 | addpath('../cpmc_release1/'); 24 | addpath('../cpmc_release1/code/'); 25 | addpath('../cpmc_release1/external_code/'); 26 | addpath('../cpmc_release1/external_code/paraFmex/'); 27 | addpath('../cpmc_release1/external_code/imrender/vgg/'); 28 | addpath('../cpmc_release1/external_code/immerge/'); 29 | addpath('../cpmc_release1/external_code/color_sift/'); 30 | addpath('../cpmc_release1/external_code/vlfeats/toolbox/kmeans/'); 31 | addpath('../cpmc_release1/external_code/vlfeats/toolbox/kmeans/'); 32 | addpath('../cpmc_release1/external_code/vlfeats/toolbox/mex/mexa64/'); 33 | addpath('../cpmc_release1/external_code/vlfeats/toolbox/mex/mexglx/'); 34 | addpath('../cpmc_release1/external_code/globalPb/lib/'); 35 | addpath('../cpmc_release1/external_code/mpi-chi2-v1_5/'); 36 | end 37 | 38 | [masks, scores] = cpmc(exp_dir, img_name); 39 | [candidates, scores] = masks_to_boxes(masks, scores); 40 | 41 | system(sprintf('rm -rf "%s"', exp_dir)); 42 | fprintf('cleaned up %s\n', exp_dir); 43 | if ~isdeployed 44 | path(old_path); 45 | end 46 | end 47 | 48 | 49 | function [boxes, scores] = masks_to_boxes(masks, scores) 50 | n = size(masks, 3); 51 | boxes = zeros(n, 4); 52 | for i = 1:n 53 | [ys, xs] = find(masks(:,:,i)); 54 | if isempty(ys) || isempty(xs) || any(isnan(xs)) || any(isnan(ys)) 55 | boxes(i,:) = [0 0 0 0]; 56 | else 57 | boxes(i,:) = [min(xs), min(ys), max(xs), max(ys)]; 58 | end 59 | end 60 | invalid = all(boxes == 0, 2); 61 | fprintf('%d invalid, %d left\n', sum(invalid), sum(~invalid)); 62 | boxes = boxes(~invalid,:); 63 | scores = scores(~invalid,:); 64 | end 65 | -------------------------------------------------------------------------------- /method_wrappers/run_edge_boxes.m: -------------------------------------------------------------------------------- 1 | function [candidates, score] = run_edge_boxes(im, num_candidates) 2 | 3 | if ~isdeployed 4 | old_path = path; 5 | 6 | addpath('../edge_box'); 7 | addpath('../edge_box/toolbox/matlab/'); 8 | addpath('../edge_box/toolbox/channels/'); 9 | end 10 | 11 | % Demo for Edge Boxes (please see readme.txt first). 12 | 13 | %% load pre-trained edge detection model (see edgesDemo.m) 14 | model=load('/path/to/edge_box/models/forest/modelBsds.mat'); model=model.model; 15 | 16 | %% set up opts for edgeBoxes (see edgeBoxes.m) 17 | opts = edgeBoxes; 18 | opts.alpha = .65; % step size of sliding window search 19 | opts.beta = .75; % nms threshold for object proposals 20 | opts.minScore = .01; % min score of boxes to detect 21 | opts.maxBoxes = 1e4; % max number of boxes to detect 22 | 23 | %% detect bbs (no visualization code for now) 24 | tic, bbs=edgeBoxes(im,model,opts); toc 25 | candidates = double(bbs(:,1:4)); 26 | candidates(:,3:4) = candidates(:,3:4) + candidates(:,1:2); 27 | score = double(bbs(:,end)); 28 | 29 | if ~isdeployed 30 | path(old_path); 31 | end 32 | end 33 | 34 | -------------------------------------------------------------------------------- /method_wrappers/run_edge_boxes50.m: -------------------------------------------------------------------------------- 1 | function [candidates, score] = run_edge_boxes50(im, num_candidates) 2 | 3 | if ~isdeployed 4 | old_path = path; 5 | 6 | addpath('../edge_box'); 7 | addpath('../edge_box/toolbox/matlab/'); 8 | addpath('../edge_box/toolbox/channels/'); 9 | end 10 | 11 | % Demo for Edge Boxes (please see readme.txt first). 12 | 13 | %% load pre-trained edge detection model (see edgesDemo.m) 14 | model=load('/path/to/edge_box/models/forest/modelBsds'); model=model.model; 15 | 16 | %% set up opts for edgeBoxes (see edgeBoxes.m) 17 | opts = edgeBoxes; 18 | opts.alpha = .65; % step size of sliding window search 19 | opts.beta = .55; % nms threshold for object proposals 20 | opts.minScore = .01; % min score of boxes to detect 21 | opts.maxBoxes = 1e4; % max number of boxes to detect 22 | 23 | %% detect bbs (no visualization code for now) 24 | tic, bbs=edgeBoxes(im,model,opts); toc 25 | candidates = double(bbs(:,1:4)); 26 | candidates(:,3:4) = candidates(:,3:4) + candidates(:,1:2); 27 | score = double(bbs(:,end)); 28 | 29 | if ~isdeployed 30 | path(old_path); 31 | end 32 | end 33 | 34 | -------------------------------------------------------------------------------- /method_wrappers/run_edge_boxes90.m: -------------------------------------------------------------------------------- 1 | function [candidates, score] = run_edge_boxes90(im, num_candidates) 2 | 3 | if ~isdeployed 4 | old_path = path; 5 | 6 | addpath('../edge_box'); 7 | addpath('../edge_box/toolbox/matlab/'); 8 | addpath('../edge_box/toolbox/channels/'); 9 | end 10 | 11 | % Demo for Edge Boxes (please see readme.txt first). 12 | 13 | %% load pre-trained edge detection model (see edgesDemo.m) 14 | model=load('/path/to/edge_box/models/forest/modelBsds'); model=model.model; 15 | 16 | %% set up opts for edgeBoxes (see edgeBoxes.m) 17 | opts = edgeBoxes; 18 | opts.alpha = .85; % step size of sliding window search 19 | opts.beta = .95; % nms threshold for object proposals 20 | opts.minScore = .01; % min score of boxes to detect 21 | opts.maxBoxes = 1e4; % max number of boxes to detect 22 | 23 | %% detect bbs (no visualization code for now) 24 | tic, bbs=edgeBoxes(im,model,opts); toc 25 | candidates = double(bbs(:,1:4)); 26 | candidates(:,3:4) = candidates(:,3:4) + candidates(:,1:2); 27 | score = double(bbs(:,end)); 28 | 29 | if ~isdeployed 30 | path(old_path); 31 | end 32 | end 33 | 34 | -------------------------------------------------------------------------------- /method_wrappers/run_endres.m: -------------------------------------------------------------------------------- 1 | function [candidates, scores] = run_endres(im, num_candidates) 2 | if ~isdeployed 3 | old_path = path; 4 | addpath(genpath('/path/to/Endres/PROP/proposals')); 5 | end 6 | 7 | [proposals, superpixels] = generate_proposals(im); 8 | candidates = masks_to_boxes(proposals, superpixels); 9 | scores = []; 10 | 11 | if ~isdeployed 12 | path(old_path); 13 | end 14 | end 15 | 16 | 17 | function boxes = masks_to_boxes(proposals, superpixels) 18 | n = numel(proposals); 19 | boxes = zeros(n, 4); 20 | for i = 1:n 21 | mask = ismember(superpixels, proposals{i}); 22 | [ys, xs] = find(mask); 23 | boxes(i,:) = [min(xs), min(ys), max(xs), max(ys)]; 24 | end 25 | end 26 | -------------------------------------------------------------------------------- /method_wrappers/run_felsen_candidates.m: -------------------------------------------------------------------------------- 1 | function [proposals, scores] = run_felsen_candidates(im, num_proposals, seed) 2 | if ~isdeployed 3 | old_path = path; 4 | addpath(fullfile('..', 'segmentation_baseline')); 5 | addpath(fullfile('..', 'segmentation_baseline', 'cmex')); 6 | addpath(fullfile('..', 'segmentation_baseline', 'matlab')); 7 | end 8 | 9 | if nargin < 3 10 | % seed to milliseconds 11 | seed = str2double(datestr(now,'HHMMSSFFF')); 12 | end 13 | 14 | configFile = '/path/to/segmentation_baseline/config/rp_4segs.mat'; 15 | configParams = load(configFile); 16 | configParams = configParams.params; 17 | configParams.approxFinalNBoxes = num_proposals; 18 | configParams.q = 1; 19 | configParams.rSeedForRun = seed; 20 | proposals = felsen_candidates(im, configParams); 21 | scores = []; 22 | 23 | if ~isdeployed 24 | path(old_path); 25 | end 26 | end 27 | -------------------------------------------------------------------------------- /method_wrappers/run_gop.m: -------------------------------------------------------------------------------- 1 | function [boxes, scores] = run_gop(im, num_proposals) 2 | 3 | if ~isdeployed 4 | old_path = path; 5 | addpath('/path/to/geodesic/gop_matlab/matlab'); 6 | end 7 | 8 | 9 | interp_data = load('/path/to/gop_num_candidates_interpolation_data.mat'); 10 | interp_data.num_candidates = [-1; interp_data.num_candidates; 1e10]; 11 | interp_data.all_params = [interp_data.all_params(1,:); interp_data.all_params; interp_data.all_params(end,:)]; 12 | params = interp1(interp_data.num_candidates, interp_data.all_params, num_proposals); 13 | assert(~any(isnan(params))); 14 | N_S = params(1); 15 | N_T = params(2); 16 | 17 | boxes = gop_wrapper(im, N_S, N_T); 18 | scores = []; 19 | 20 | if ~isdeployed 21 | path(old_path); 22 | end 23 | end 24 | 25 | 26 | function boxes = gop_wrapper(im, N_S, N_T) 27 | % reimplementation of eval_box.py 28 | gop_mex( 'setDetector', 'MultiScaleStructuredForest("/path/to/geodesic/gop_matlab/data/sf.dat")' ); 29 | data_path = '/path/to/geodesic/gop_matlab/data/'; 30 | max_iou = 0.7; 31 | args = {}; 32 | if N_S > 3 33 | args = [args {'seed', [data_path 'seed_final.dat']}]; 34 | end 35 | if N_T >= 5 36 | max_iou = 0.8; 37 | end 38 | if N_T > 6 39 | max_iou = 0.9; 40 | end 41 | if N_S > 40 42 | args = [args {'unary', N_S, N_T, 'seedUnary()', 'backgroundUnary({0,1,15})', ... 43 | 'unary', 0, N_T, 'seedUnary()', 'backgroundUnary({0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15})', 0.1, 1}]; 44 | elseif N_S > 1 45 | args = [args {'unary', N_S, N_T, 'seedUnary()', 'backgroundUnary({0,15})'}]; 46 | else 47 | args = [args {'unary', N_S, N_T, 'seedUnary()', 'backgroundUnary({15})'}]; 48 | end 49 | 50 | args = [{'max_iou', max_iou}, args]; 51 | p = Proposal(args{:}); 52 | tic(); 53 | os = OverSegmentation( im ); 54 | props = p.propose( os ); 55 | boxes = double(os.maskToBox( props )); 56 | toc(); 57 | end 58 | -------------------------------------------------------------------------------- /method_wrappers/run_objectness.m: -------------------------------------------------------------------------------- 1 | function [candidates, score] = run_objectness(im, num_boxes) 2 | struct = load('/path/to/objectness-release-v2.2/Data/params.mat'); 3 | params = struct.params; 4 | clear 'struct'; 5 | if ~isdeployed 6 | old_path = path; 7 | old_pwd = pwd; 8 | 9 | cd('/path/to/objectness-release-v2.2'); 10 | startup; 11 | else 12 | end 13 | 14 | candidates = runObjectness(im, num_boxes, params); 15 | score = candidates(:,5); 16 | candidates = round(candidates(:,1:4)); 17 | 18 | if ~isdeployed 19 | cd(old_pwd); 20 | path(old_path); 21 | end 22 | end 23 | -------------------------------------------------------------------------------- /method_wrappers/run_randomized_prims.m: -------------------------------------------------------------------------------- 1 | function [proposals, scores] = run_randomized_prims(im, num_proposals, seed) 2 | if ~isdeployed 3 | old_path = path; 4 | addpath(fullfile('..', 'randomized_prims')); 5 | addpath(fullfile('..', 'randomized_prims', 'cmex')); 6 | addpath(fullfile('..', 'randomized_prims', 'matlab')); 7 | end 8 | 9 | if nargin < 3 10 | % seed to milliseconds 11 | seed = str2double(datestr(now,'HHMMSSFFF')); 12 | end 13 | 14 | configFile = '/path/to/randomized_prims/config/rp_4segs.mat'; 15 | configParams = load(configFile); 16 | configParams = configParams.params; 17 | configParams.approxFinalNBoxes = num_proposals; 18 | configParams.q = 1; 19 | configParams.rSeedForRun = seed; 20 | proposals = RP(im, configParams); 21 | scores = []; 22 | 23 | if ~isdeployed 24 | path(old_path); 25 | end 26 | end 27 | -------------------------------------------------------------------------------- /method_wrappers/run_rantalankila.m: -------------------------------------------------------------------------------- 1 | function [candidates, priority] = run_rantalankila(im, num_candidates, varargin) 2 | if ~isdeployed 3 | old_path = path; 4 | addpath(fullfile('..', 'Rantalankila')); 5 | addpath(fullfile('..', 'Rantalankila', 'GCMex')); 6 | addpath(fullfile('..', 'Rantalankila', 'features')); 7 | addpath(fullfile('..', 'Rantalankila', 'vlfeat-0.9.18', 'toolbox', 'mex', 'mexa64')); 8 | 9 | end 10 | 11 | opts = get_opts(); 12 | 13 | interp_data = load('/path/to/all/rantalankila_num_candidates_interpolation_data.mat'); 14 | if num_candidates <= min(interp_data.num_candidates) 15 | opts.gc_branches = 0; 16 | elseif num_candidates >= max(interp_data.num_candidates) 17 | opts.gc_branches = 500; 18 | else 19 | opts.gc_branches = interp1(interp_data.num_candidates, interp_data.gc_branches, num_candidates); 20 | assert(~isnan(opts.gc_branches)); 21 | end 22 | 23 | 24 | 25 | argl = length(varargin); 26 | if mod(argl,2) ~= 0 % string-value pairs 27 | error('Give pairs of extra arguments.'); 28 | end 29 | for argi = 0:((argl/2) - 1) 30 | arg_str = varargin(2*argi + 1); 31 | arg_val = varargin(2*argi + 2); 32 | assert(isfield(opts, arg_str{1})); 33 | opts.(arg_str{1}) = arg_val{1}; 34 | end 35 | 36 | opts.seg_method = 'felz'; 37 | try 38 | [region_parts{1}, orig_sp{1}] = spagglom(im, opts); 39 | catch 40 | fprintf('rantalankila run on felsenzwalb segmentation failed\n'); 41 | end 42 | opts.seg_method = 'slic'; 43 | [region_parts{2}, orig_sp{2}] = spagglom(im, opts); 44 | 45 | candidates = sp_to_candidates(im, region_parts, orig_sp); 46 | priority = []; 47 | 48 | if ~isdeployed 49 | path(old_path); 50 | end 51 | end 52 | 53 | 54 | function bboxes = sp_to_candidates(im, region_parts, orig_sp) 55 | % candidates = []; 56 | % for sus = 1:length(orig_sp) 57 | % ys = double(orig_sp{sus}.pixels(:,1)); 58 | % xs = double(orig_sp{sus}.pixels(:,2)); 59 | % candidates = cat(1, candidates, [min(xs), min(ys), max(xs), max(ys)]); 60 | % end 61 | 62 | [h, w, ~] = size(im); 63 | 64 | % For each sp of orig_sp, find its bounding box 65 | for region_set = 1:length(region_parts) 66 | sp_edges{region_set} = zeros(length(orig_sp{region_set}), 4); 67 | bboxes{region_set} = zeros(length(region_parts{region_set}), 4); 68 | 69 | % find edges of Ri 70 | for ses = 1:length(orig_sp{region_set}) 71 | Ri = false(h, w); % note ',', not '*' as below 72 | Ri(orig_sp{region_set}{ses}.spind) = 1; 73 | Xi = sum(Ri,1); 74 | Yi = sum(Ri,2); 75 | sp_edges{region_set}(ses,:) = [find(Xi,1,'first'), find(Yi,1,'first'), find(Xi,1,'last'), find(Yi,1,'last')]; 76 | end 77 | 78 | % Using the above bounding boxes, solve bounding box of each region 79 | % proposal 80 | for j = 1:length(region_parts{region_set}) 81 | sus = region_parts{region_set}{j}; 82 | if ~isempty(sus) 83 | tg = sp_edges{region_set}(sus,:); 84 | bboxes{region_set}(j,:) = [min(tg(:,1)), min(tg(:,2)), max(tg(:,3)), max(tg(:,4))]; % bounding box is the most extreme values of individual bounding boxes 85 | else 86 | error('what is this?'); 87 | bboxes = []; 88 | end 89 | end 90 | end % for each region set 91 | 92 | bboxes = cat(1, bboxes{:}); 93 | end 94 | 95 | 96 | function [opts] = get_opts() 97 | % Sets options for spagglom.m that are propagated through the functions. Many of the 98 | % options are zero/one flags. The default settings were used to produce the 99 | % results presented in the paper. 100 | 101 | 102 | %% Superpixelation stage 103 | opts.seg_method = 'slic'; % 'felz', 'slic' or 'qshift'. Two first recommended 104 | 105 | % Felzenswalb superpixelation parameters 106 | opts.felz_k = 50; % default 50 107 | opts.felz_sigma = 0.8; % default 0.8 108 | opts.felz_min_area = 150; % default 150 109 | 110 | % SLIC superpixelation parameters 111 | opts.slic_regularizer = 800; % default 800 (uses different definition than SLIC authors) 112 | opts.slic_region_size = 20; % default 20 113 | 114 | %% Features 115 | opts.diagonal_connections = 0; % default 0. Whether only diagonally connected pixels are considered connected. 116 | opts.dsift_step = 2; % default 2. calculate dsfit only every n step (quadratic speedup in n). 117 | 118 | % Histogram features to use 119 | opts.feature_dsift_bow = 1; % 1 % denseSIFT bag-of-words 120 | opts.feature_color_bow = 1; % 2 % color bag-of-words in various color spaces 121 | opts.feature_rgb_raw = 0; % 3 % raw rgb histograms 122 | opts.feature_grad_texture = 0; % 4 % feature used by van de Sande. Has two implementations, see code 123 | opts.feature_lbp = 0; % 6 % Local binary patterns 124 | opts.feature_size = 1; % 8 % size of combined superpixel 125 | opts.features = 1:6; % This will be used in similarity.m to change histogram distances feature-wise 126 | opts.features = opts.features(logical([opts.feature_dsift_bow, opts.feature_color_bow, opts.feature_rgb_raw, opts.feature_grad_texture, opts.feature_lbp ,opts.feature_size])); 127 | 128 | opts.feature_weights = [1,1,2]; % default [1,1,2] with above dsift_bow, color_bow, size enabled and rest disabled 129 | 130 | opts.collect_merged_regions = 1; % default 1. Every time a pair is merged during the greedy pairing algorithm, the new pair is saved as a region 131 | 132 | opts.gc_branches = 15; % default 15. Number of graphcut branches. 133 | 134 | opts.start_phase2 = 0.8; % 0.8 default. Score at which to change features and/or start branching 135 | 136 | % Load precalculated data 137 | opts.load_color_dict = 1; % default 1. 0 means dictionary will be created from the image (very slow). This option is used for both, rgb and lab features. 138 | opts.load_dsift_dict = 1; % default 1. same as above but for dsift 139 | opts.load_dsift_words = 0; % default 0. load precalculated dsift words using precalculated dict. This option overrides opts.load_dsift_dict. 140 | opts.load_init_segs = 0; % default 0. Load initial felz or slic rgb segmentations with "default" parameters (see conditionals in code at spagglom.m) 141 | 142 | if opts.load_dsift_dict 143 | load('/path/to/Rantalankila/dicts/dsift_dict_k500'); 144 | opts.dsift_dict = dsift_dict; 145 | clear dsift_dict; 146 | end 147 | end 148 | -------------------------------------------------------------------------------- /method_wrappers/run_rigor.m: -------------------------------------------------------------------------------- 1 | function [proposals, scores] = run_rigor(im, num_candidates) 2 | 3 | if ~isdeployed 4 | old_path = path; 5 | 6 | addpath('/path/to/rigor'); 7 | end 8 | 9 | 10 | interp_data = load('/path/to/rigor_num_candidates_interpolation_data.mat'); 11 | if num_candidates <= min(interp_data.num_candidates) 12 | num_seeds = 1; 13 | elseif num_candidates >= max(interp_data.num_candidates) 14 | num_seeds = 1024; 15 | else 16 | num_seeds = interp1(interp_data.num_candidates, interp_data.all_num_seeds, num_candidates); 17 | assert(~isnan(opts.gc_branches)); 18 | end 19 | 20 | [masks, seg_obj, total_time] = rigor_obj_segments(im, num_seeds, 'io', true, 'force_recompute', true); 21 | scores = []; 22 | proposals = masks_to_boxes(masks); 23 | proposals = unique(proposals, 'rows'); 24 | 25 | if ~isdeployed 26 | path(old_path); 27 | end 28 | 29 | end 30 | 31 | 32 | function [boxes] = masks_to_boxes(masks) 33 | n = size(masks, 3); 34 | boxes = zeros(n, 4); 35 | for i = 1:n 36 | [ys, xs] = find(masks(:,:,i)); 37 | if isempty(ys) || isempty(xs) || any(isnan(xs)) || any(isnan(ys)) 38 | boxes(i,:) = [0 0 0 0]; 39 | else 40 | boxes(i,:) = [min(xs), min(ys), max(xs), max(ys)]; 41 | end 42 | end 43 | invalid = all(boxes == 0, 2); 44 | fprintf('%d invalid, %d left\n', sum(invalid), sum(~invalid)); 45 | boxes = boxes(~invalid,:); 46 | end 47 | -------------------------------------------------------------------------------- /method_wrappers/run_selective_search.m: -------------------------------------------------------------------------------- 1 | function [candidates, priority] = run_selective_search(im, num_candidates) 2 | if ~isdeployed 3 | old_path = path; 4 | addpath(fullfile('..', 'SelectiveSearchCodeIJCV')); 5 | addpath(fullfile('..', 'SelectiveSearchCodeIJCV', 'Dependencies')); 6 | end 7 | 8 | % 9 | % Parameters. Note that this controls the number of hierarchical 10 | % segmentations which are combined. 11 | colorTypes = {'Hsv', 'Lab', 'RGI', 'H', 'Intensity'}; 12 | % colorType = colorTypes{1}; % Single color space for demo 13 | 14 | % Here you specify which similarity functions to use in merging 15 | simFunctionHandles = {@SSSimColourTextureSizeFillOrig, @SSSimTextureSizeFill, @SSSimBoxFillOrig, @SSSimSize}; 16 | % simFunctionHandles = simFunctionHandles(1:2); % Two different merging strategies 17 | 18 | % Thresholds for the Felzenszwalb and Huttenlocher segmentation algorithm. 19 | % Note that by default, we set minSize = k, and sigma = 0.8. 20 | ks = [50 100 150 300]; % controls size of segments of initial segmentation. 21 | sigma = 0.8; 22 | minBoxWidth = 20; 23 | 24 | boxesT = cell(length(ks)*length(colorTypes),1); priorityT = cell(length(ks)*length(colorTypes),1); 25 | idx = 1; 26 | for j=1:length(ks) 27 | k = ks(j); % Segmentation threshold k 28 | minSize = k; % We set minSize = k 29 | for n = 1:length(colorTypes) 30 | colorType = colorTypes{n}; 31 | [box blobIndIm blobBoxes hierarchy priorityT{idx}] = Image2HierarchicalGrouping(im, sigma, k, minSize, colorType, simFunctionHandles); 32 | boxesT{idx} = [box(:,2), box(:,1), box(:,4), box(:,3)]; 33 | idx = idx + 1; 34 | end 35 | end 36 | 37 | candidates = cat(1, boxesT{:}); % Concatenate boxes from all hierarchies 38 | priority = cat(1, priorityT{:}); % Concatenate priorities 39 | % Do pseudo random sorting as in paper 40 | priority = priority .* rand(size(priority)); 41 | [priority, sortIds] = sort(priority, 'ascend'); 42 | candidates = candidates(sortIds,:); 43 | [candidates, filteredIdx] = FilterBoxesWidth(candidates, minBoxWidth); 44 | priority = priority(filteredIdx); 45 | [candidates, uniqueIdx] = BoxRemoveDuplicates(candidates); 46 | priority = priority(uniqueIdx); 47 | 48 | if size(candidates, 1) > num_candidates 49 | candidates = candidates(1:num_candidates,:); 50 | priority = priority(1:num_candidates); 51 | end 52 | 53 | if ~isdeployed 54 | path(old_path); 55 | end 56 | end 57 | -------------------------------------------------------------------------------- /plot_correlation_edge_boxes.m: -------------------------------------------------------------------------------- 1 | function plot_correlation_edge_boxes() 2 | % Plots for analyzing the correlation between detector performance 3 | % (R-CNN) on the different variants of EdgeBoxes. 4 | % Also plots the recall curves for all the variants. 5 | % 6 | % Bonus feature: Rainbow colormap! Hand-tuned nice colors on the curve! 7 | 8 | methods = get_method_configs(); 9 | 10 | IoUs = 0.5:0.05:0.9; 11 | custom_names = arrayfun(@(x) sprintf('%.2f', x), IoUs, 'UniformOutput', false); 12 | custom_names = [custom_names {'AR'}]; 13 | 14 | % select all edge box variants 15 | method_selection = [13 14 16 19 20:25]; 16 | [~,order] = sort({methods(method_selection).short_name}); 17 | method_selection = method_selection(order); 18 | methods = methods(method_selection); 19 | for i = 1:numel(methods) 20 | methods(i).sort_key = i; 21 | end 22 | 23 | % Recall matrix used for experiments nThresholds x nAlgorithms 24 | ld = load('data/pascal_voc07_test_recall.mat'); 25 | R = ld.recalls(method_selection,:)'; 26 | ARs = ld.ARs(method_selection)'; 27 | 28 | T = ld.iou_thresholds; 29 | 30 | ld = load('data/pascal_voc07_test_rcnn_aps.mat'); 31 | rcnn_AP = ld.aps(method_selection); 32 | ld = load('data/pascal_voc07_test_frcn_aps.mat'); 33 | frcn_AP = ld.aps(method_selection); 34 | ld = load('data/pascal_voc07_test_frcn_noregr_aps.mat'); 35 | frcn_noregr_AP = ld.aps(method_selection); 36 | 37 | % assign distinct colors to the methods 38 | n = numel(methods); 39 | for i = 1:n 40 | methods(i).line_style = '-'; 41 | end 42 | 43 | % average recall 44 | mirror_offset = 0.060; 45 | offsets = zeros(n, 2); 46 | offsets(:,1) = 0.010; 47 | offsets(1,1) = offsets(1,1)-mirror_offset; 48 | offsets(2,1) = offsets(2,1)-mirror_offset; 49 | offsets(6,1) = offsets(6,1)-mirror_offset; 50 | offsets(7,1) = offsets(7,1)-mirror_offset; 51 | methods = plot_weighted_area_color_coded(ARs, ... 52 | rcnn_AP, methods, custom_names, [0.3 0.6 43 57], true, offsets); 53 | hei = 7; wid = 7; 54 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 55 | set(gcf, 'PaperPositionMode','auto'); 56 | printpdf('figures/RCNN_mAP_recall_area_voc07_edge_boxes.pdf'); 57 | 58 | mirror_offset = 0.060; 59 | offsets = zeros(n, 2); 60 | offsets(:,1) = 0.010; 61 | offsets(1:4,1) = offsets(1:4,1)-mirror_offset; 62 | offsets(5,2) = offsets(5,2)+0.5; 63 | offsets(6:9,2) = offsets(6:9,2)-0.5; 64 | plot_weighted_area_color_coded(ARs, ... 65 | frcn_AP, methods, custom_names, [0.3 0.6 45 67], false, offsets); 66 | hei = 7; wid = 7; 67 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 68 | set(gcf, 'PaperPositionMode','auto'); 69 | printpdf('figures/FRCN_mAP_recall_area_voc07_edge_boxes.pdf'); 70 | 71 | mirror_offset = 0.060; 72 | offsets = zeros(n, 2); 73 | offsets(:,1) = 0.010; 74 | offsets(6:9,1) = offsets(6:9,1)-mirror_offset; 75 | offsets(5:6,2) = offsets(5:6,2)+0.5; 76 | offsets(1:4,2) = offsets(1:4,2)-0.6; 77 | offsets(1:4,1) = offsets(1:4,1)-0.006; 78 | plot_weighted_area_color_coded(ARs, ... 79 | frcn_noregr_AP, methods, custom_names, [0.3 0.6 44 61], false, offsets); 80 | hei = 7; wid = 7; 81 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 82 | set(gcf, 'PaperPositionMode','auto'); 83 | printpdf('figures/FRCN_noregr_mAP_recall_area_voc07_edge_boxes.pdf'); 84 | 85 | % plot recall curves for each parameter setting 86 | fh = figure; 87 | plot_overlap_recall_curve({methods.best_voc07_candidates_file}, ... 88 | methods, 1000, fh, true, 'NorthEast', false, ... 89 | custom_names); 90 | [lh, hobj1] = legend(custom_names); 91 | legend boxoff; 92 | 93 | hei = 7; 94 | wid = 7; 95 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 96 | set(gcf, 'PaperPositionMode','auto'); 97 | 98 | % change the length of the line samples in the legend 99 | textobj = findobj(hobj1, 'type', 'line'); 100 | for i = 1:numel(textobj) 101 | lineXData = get(textobj(i), 'XData'); 102 | lineXData(1) = lineXData(1) + 0.3; 103 | set(textobj(i), 'XData', lineXData); 104 | end 105 | 106 | % move the legend more outside 107 | P = get(lh, 'Position'); 108 | P(1) = P(1) + 0.05; 109 | P(2) = P(2) + 0.05; 110 | set(lh, 'Position', P); 111 | 112 | 113 | printpdf('figures/recall_1000_voc07_edge_boxes.pdf') 114 | end 115 | 116 | function [methods] = plot_weighted_area_color_coded(areas, AP, methods, ... 117 | custom_names, axis_lim, reassign_colors, label_offsets) 118 | additional_methods = methods(end); 119 | additional_areas = areas(end); 120 | additional_AP = AP(end); 121 | 122 | methods = methods(1:end-1); 123 | areas = areas(1:end-1); 124 | AP = AP(1:end-1); 125 | 126 | S=corrcoef([areas' AP']); s = S(1,end); 127 | figure; 128 | 129 | x = 0.5:0.05:0.9; 130 | y = cat(1, areas, AP); 131 | xx = 0.5:0.002:0.9; 132 | yy = spline(x, y, xx); 133 | 134 | dists = sqrt(sum((yy(:,2:end) - yy(:,1:end-1)) .^ 2, 1)); 135 | dists = dists .* linspace(6,1,numel(dists)); 136 | dists = cumsum(dists); 137 | 138 | colors = rainbow(dists, max(dists)); 139 | n = numel(xx); 140 | for i = 2:n 141 | plot(yy(1,(i-1):i), yy(2,(i-1):i), 'Color', colors(i-1,:), 'LineWidth', 1.5); 142 | if i == 2; hold on; end 143 | end 144 | 145 | if reassign_colors 146 | for i = 1:numel(methods) 147 | [~,c_idx] = min((yy(1,:) - areas(i)) .^ 2 + (yy(2,:) - AP(i)) .^ 2); 148 | c_idx = max(c_idx - 1, 1); 149 | methods(i).color = colors(c_idx,:); 150 | end 151 | end 152 | hold on; 153 | 154 | p=polyfit(areas,AP,1); line([0 1],[p(2),sum(p)],'Color',[1 1 1]/3); 155 | 156 | for i = 1:numel(methods) 157 | plot(areas(i), AP(i), '.', 'MarkerSize', 20, 'Color', methods(i).color); 158 | end 159 | for i = 1:numel(additional_methods) 160 | plot(additional_areas(i), additional_AP(i), '.', 'MarkerSize', 20, 'Color', 'k'); 161 | additional_methods(i).color = [0 0 0]; 162 | end 163 | grid on; 164 | % move the labels around, so it looks nice 165 | xpos = [areas additional_areas]; 166 | ypos = [AP additional_AP]; 167 | xpos = xpos + label_offsets(:,1)'; 168 | ypos = ypos + label_offsets(:,2)'; 169 | 170 | text(xpos,ypos,custom_names); 171 | xlabel(sprintf('average recall')); axis(axis_lim) 172 | ylabel('mAP'); hold on; 173 | hold off 174 | 175 | % move the title inside of the plot 176 | v = axis; 177 | handle = title(sprintf('correlation=%.3f',s)); 178 | titlepos = [(v(2)-v(1))*0.5+v(1), (v(4)-v(3))*0.91+v(3), 0]; 179 | set(handle, 'Position', titlepos); 180 | 181 | methods = [methods additional_methods]; 182 | end 183 | 184 | function cmap = rainbow(k,n) 185 | rainbow_cmap = [ 186 | 255, 0, 0 187 | 255, 127, 0 188 | 255, 255, 0 189 | 0, 255, 0 190 | 0, 0, 255 191 | 75, 0, 130 192 | 143, 0, 255]/255; 193 | idx = (k/n*size(rainbow_cmap,1)); 194 | idx = max(1, min(size(rainbow_cmap,1), idx)); 195 | 196 | x = (1:size(rainbow_cmap,1))'; 197 | y = rainbow_cmap; 198 | cmap = interp1(x, y, idx); 199 | end 200 | 201 | 202 | -------------------------------------------------------------------------------- /plot_correlation_voc07.m: -------------------------------------------------------------------------------- 1 | function plot_correlation_voc07() 2 | % Plots for analyzing the correlation between detector performance 3 | % (LLDA-DPM and R-CNN) on the different proposal methods and different 4 | % proxy measures based on recall of the proposals. 5 | % 6 | % based on Piotr's blog post: 7 | % https://pdollar.wordpress.com/2014/11/18/evaluating-object-proposals/ 8 | 9 | methods = get_method_configs(); 10 | n_methods = numel(methods); 11 | 12 | method_selection = 1:n_methods; 13 | method_selection([14 16 19:25]) = []; 14 | 15 | % Recall matrix used for experiments nThresholds x nAlgorithms 16 | ld = load('data/pascal_voc07_test_recall.mat'); 17 | R = ld.recalls'; 18 | T = ld.iou_thresholds; 19 | ARs = ld.ARs(:); 20 | ARs_per_class = ld.ARs_per_class'; 21 | 22 | % define algorithm names, IoU thresholds, recall values and AP scores 23 | nms={methods.name}; 24 | 25 | ld = load('data/pascal_voc07_test_llda_dpm_aps.mat'); 26 | axis_lim = [0.5 0.975 0.6 1]; 27 | % the following three lines reproduce Piotr's plots 28 | % whitelist = [8 7 13 9 15 1 2 4 10 3 6 12]; 29 | % AP = ld.aps(:,1)'; AP = AP(whitelist); 30 | % ploteroo(T, R(:,whitelist), AP, nms(whitelist), axis_lim); 31 | llda_dpm_AP = ld.aps(:,2)'; 32 | llda_dpm_AP_per_class = ld.aps_per_class(:,:,2)'; 33 | 34 | plot_correlation_over_recall(T, R(:,method_selection), ... 35 | llda_dpm_AP(method_selection), nms(method_selection), axis_lim); 36 | scale_and_save('figures/LLDA_DPM_mAP_recall_voc07.pdf'); 37 | 38 | ld = load('data/pascal_voc07_test_rcnn_aps.mat'); 39 | rcnn_AP = ld.aps; 40 | rcnn_AP_per_class = ld.aps_per_class'; 41 | 42 | plot_correlation_over_recall(T, R(:,method_selection), ... 43 | rcnn_AP(method_selection), nms(method_selection), axis_lim); 44 | scale_and_save('figures/RCNN_mAP_recall_voc07.pdf'); 45 | 46 | ld = load('data/pascal_voc07_test_frcn_aps.mat'); 47 | frcn_AP = ld.aps; 48 | frcn_AP_per_class = ld.aps_per_class'; 49 | 50 | plot_correlation_over_recall(T, R(:,method_selection), ... 51 | frcn_AP(method_selection), nms(method_selection), axis_lim); 52 | scale_and_save('figures/FRCN_mAP_recall_voc07.pdf'); 53 | 54 | ld = load('data/pascal_voc07_test_frcn_noregr_aps.mat'); 55 | frcn_noregr_AP = ld.aps; 56 | frcn_noregr_AP_per_class = ld.aps_per_class'; 57 | 58 | plot_correlation_over_recall(T, R(:,method_selection), ... 59 | frcn_noregr_AP(method_selection), nms(method_selection), axis_lim); 60 | scale_and_save('figures/FRCN_noregr_mAP_recall_voc07.pdf'); 61 | 62 | % Average recall plots 63 | plot_weighted_area_color_coded(ARs(method_selection), ... 64 | llda_dpm_AP(method_selection), methods(method_selection), [0 0.6 10 38]); 65 | scale_and_save('figures/LLDA_DPM_mAP_recall_area_voc07.pdf'); 66 | 67 | axlim = [0 0.6 12 67]; 68 | plot_weighted_area_color_coded(ARs(method_selection), ... 69 | rcnn_AP(method_selection), methods(method_selection), axlim); 70 | scale_and_save('figures/RCNN_mAP_recall_area_voc07.pdf'); 71 | 72 | plot_weighted_area_color_coded(ARs(method_selection), ... 73 | frcn_AP(method_selection), methods(method_selection), axlim); 74 | scale_and_save('figures/FRCN_mAP_recall_area_voc07.pdf'); 75 | 76 | frcn_AR = mean(R(T>=0.5 & T<=0.7,:), 1); 77 | plot_weighted_area_color_coded(frcn_AR(method_selection)', ... 78 | frcn_AP(method_selection), methods(method_selection), [0 1 12 67]); 79 | scale_and_save('figures/FRCN_mAP_recall_area0.5_0.8_voc07.pdf'); 80 | 81 | plot_weighted_area_color_coded(ARs(method_selection), ... 82 | frcn_noregr_AP(method_selection), methods(method_selection), axlim); 83 | scale_and_save('figures/FRCN_noregr_mAP_recall_area_voc07.pdf'); 84 | 85 | % Average recall plots per class 86 | per_class_method_selection = method_selection; 87 | % plot_AR_per_class(ARs_per_class(:,per_class_method_selection), ... 88 | % llda_dpm_AP_per_class(:,per_class_method_selection), methods(per_class_method_selection), [0 .85 0 65]); 89 | % scale_and_save('figures/LLDA_DPM_correlation_per_class_lines_voc07.pdf', 6, 7); 90 | % plot_AR_per_class(ARs_per_class(:,per_class_method_selection), ... 91 | % rcnn_AP_per_class(:,per_class_method_selection), methods(per_class_method_selection), [0 .85 0 85]); 92 | % scale_and_save('figures/RCNN_correlation_per_class_lines_voc07.pdf', 6, 7); 93 | % plot_AR_per_class(ARs_per_class(:,per_class_method_selection), ... 94 | % frcn_AP_per_class(:,per_class_method_selection), methods(per_class_method_selection), [0 .85 0 90]); 95 | % scale_and_save('figures/FRCN_correlation_per_class_lines_voc07.pdf', 6, 7); 96 | % plot_AR_per_class(ARs_per_class(:,per_class_method_selection), ... 97 | % frcn_noregr_AP_per_class(:,per_class_method_selection), methods(per_class_method_selection), [0 .85 0 90]); 98 | % scale_and_save('figures/FRCN_noregr_correlation_per_class_lines_voc07.pdf', 6, 7); 99 | 100 | % plot_correlation_per_class(ARs_per_class(:,per_class_method_selection), ... 101 | % llda_dpm_AP_per_class(:,per_class_method_selection), true); 102 | % scale_and_save('figures/LLDA_DPM_correlation_per_class_colorbars_voc07.pdf', 6, 12); 103 | % plot_correlation_per_class(ARs_per_class(:,per_class_method_selection), ... 104 | % rcnn_AP_per_class(:,per_class_method_selection), true); 105 | % scale_and_save('figures/RCNN_correlation_per_class_colorbars_voc07.pdf', 6, 12); 106 | % plot_correlation_per_class(ARs_per_class(:,per_class_method_selection), ... 107 | % frcn_AP_per_class(:,per_class_method_selection), true); 108 | % scale_and_save('figures/FRCN_correlation_per_class_colorbars_voc07.pdf', 6, 12); 109 | % plot_correlation_per_class(ARs_per_class(:,per_class_method_selection), ... 110 | % frcn_noregr_AP_per_class(:,per_class_method_selection), true); 111 | % scale_and_save('figures/FRCN_noregr_correlation_per_class_colorbars_voc07.pdf', 6, 12); 112 | 113 | plot_correlation_per_class_multi(ARs_per_class(:,per_class_method_selection), ... 114 | cat(1, llda_dpm_AP_per_class(:,per_class_method_selection), ... 115 | rcnn_AP_per_class(:,per_class_method_selection), ... 116 | frcn_noregr_AP_per_class(:,per_class_method_selection), ... 117 | frcn_AP_per_class(:,per_class_method_selection)), ... 118 | {'LM-LLDA bbpred', 'R-CNN', 'Fast R-CNN', 'Fast R-CNN bbpred'}, ... 119 | 'bars'); 120 | scale_and_save('figures/all_detectors_correlation_per_class_colorbars_voc07.pdf', 7, 16); 121 | 122 | 123 | % plot_correlation_per_class(ARs_per_class(:,per_class_method_selection), ... 124 | % llda_dpm_AP_per_class(:,per_class_method_selection), false); 125 | % scale_and_save('figures/LLDA_DPM_correlation_per_class_bars_voc07.pdf', 6, 12); 126 | % plot_correlation_per_class(ARs_per_class(:,per_class_method_selection), ... 127 | % rcnn_AP_per_class(:,per_class_method_selection), false); 128 | % scale_and_save('figures/RCNN_correlation_per_class_bars_voc07.pdf', 6, 12); 129 | end 130 | 131 | function scale_and_save(output_filename, hei, wid) 132 | s = 0.13; 133 | if nargin < 2, hei = 40*s; end 134 | if nargin < 3, wid = 45*s; end 135 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 136 | set(gcf, 'PaperPositionMode','auto'); 137 | printpdf(output_filename); 138 | end 139 | 140 | function plot_weighted_area(areas, AP, nms, axis_lim) 141 | S=corrcoef([areas' AP']); s = S(1,end); 142 | figure; plot(areas, AP, 'dr'); grid on; text(areas+.015,AP,nms); 143 | xlabel(sprintf('weighted area under recall')); axis(axis_lim) 144 | title(sprintf('correlation=%.3f',s)); ylabel('mAP'); hold on; 145 | p=polyfit(areas,AP,1); line([0 1],[p(2),sum(p)],'Color',[1 1 1]/3); hold off 146 | end 147 | 148 | function colors = class_colors() 149 | colors = [[74,136,55], 150 | [127,137,221], 151 | [209,79,44], 152 | [95,40,42], 153 | [209,68,117], 154 | [218,145,59], 155 | [117,171,189], 156 | [205,132,115], 157 | [54,70,41], 158 | [198,147,179], 159 | [204,104,203], 160 | [114,198,70], 161 | [64,74,86], 162 | [121,60,105], 163 | [86,117,53], 164 | [121,113,53], 165 | [111,182,140], 166 | [138,75,38], 167 | [189,175,70], 168 | [82,92,146]]/255; 169 | end 170 | 171 | function plot_correlation_per_class_multi(AR_per_class, AP_per_class, legend_strs, type) 172 | colors = [ 173 | 24,98,169 174 | 236,178,0 175 | 30,129,43 176 | 211,118,166 177 | ]./255; 178 | % 255, 197, 108 179 | % 110, 197, 233 180 | % 0, 58, 111 181 | % 255, 89, 89 182 | 183 | load('data/short_classes.mat'); 184 | n = numel(classes); 185 | n_detectors = numel(legend_strs); 186 | 187 | S=corrcoef([AR_per_class' AP_per_class']); 188 | Spd = cell(1, n_detectors); 189 | for i = 1:n_detectors 190 | Spd{i} = diag(S((i*n+1):((i+1)*n), 1:n)); 191 | end 192 | Spd = cat(2, Spd{:}); 193 | 194 | figure; 195 | if strcmp(type, 'lines') 196 | for i = 1:n_detectors 197 | plot(1:n, Spd(:,i), '.-', 'Color', colors(i,:)); 198 | if i == 1; hold on; end 199 | end 200 | 201 | legend(legend_strs); 202 | elseif strcmp(type, 'bars') 203 | hb = bar(1:n, Spd); 204 | for i = 1:n_detectors 205 | set(hb(i), 'FaceColor', colors(i,:), 'EdgeColor', colors(i,:)); 206 | end 207 | 208 | set(gca,'XTick',(1:numel(classes))-0.3,'XTickLabel',classes); 209 | set(gca,'XTickLabelRotation',60); 210 | xlim([0,21]); 211 | ylim([0.6 1]); 212 | ylabel('correlation with AP'); 213 | 214 | legend(legend_strs, 'Location', 'SouthEast'); 215 | else 216 | error('unknown type'); 217 | end 218 | end 219 | 220 | function plot_correlation_per_class(AR_per_class, AP_per_class, colorful) 221 | colors = class_colors(); 222 | load('data/short_classes.mat'); 223 | 224 | n = numel(classes); 225 | S=corrcoef([AR_per_class' AP_per_class']); S=diag(S(n+1:end,1:n)); 226 | figure; hold on; 227 | if colorful 228 | for i = 1:n 229 | bar(i, S(i), 'FaceColor', colors(i,:), 'EdgeColor', colors(i,:)); 230 | hold on; 231 | end 232 | else 233 | bar(S, 'FaceColor', [31,120,180]/256, 'EdgeColor', [31,120,180]/256); 234 | end 235 | % bar(S); 236 | set(gca,'XTick',(1:numel(classes))-0.3,'XTickLabel',classes); 237 | set(gca,'XTickLabelRotation',60); 238 | xlim([0,21]); 239 | ylim([0.6 1]); 240 | ylabel('correlation with AP'); 241 | end 242 | 243 | function plot_AR_per_class(AR_per_class, AP_per_class, methods, axis_lim) 244 | colors = class_colors(); 245 | load('data/short_classes.mat'); 246 | 247 | S=corrcoef([AR_per_class(:) AP_per_class(:)]); s = S(1,end); 248 | figure; 249 | for c = 1:numel(classes), cls = classes{c}; 250 | for i = 1:numel(methods) 251 | plot(AR_per_class(c,i), AP_per_class(c,i), 'o', 'MarkerSize', 3, 'Color', colors(c,:), 'LineWidth', 0.5); 252 | hold on; 253 | end 254 | p=polyfit(AR_per_class(c,:),AP_per_class(c,:),1); line([0 1],[p(2),sum(p)],'Color',colors(c,:)); 255 | end 256 | grid on; 257 | xlabel(sprintf('average recall')); 258 | ylabel('AP'); hold off; 259 | axis(axis_lim); 260 | 261 | % move the title inside of the plot 262 | v = axis; 263 | handle = title(sprintf('correlation=%.3f',s)); 264 | titlepos = [(v(2)-v(1))*0.5+v(1), (v(4)-v(3))*0.89+v(3), 0]; 265 | set(handle, 'Position', titlepos); 266 | end 267 | 268 | function plot_weighted_area_color_coded(areas, AP, methods, axis_lim) 269 | AP = AP(:); 270 | S=corrcoef([areas AP]); s = S(1,end); 271 | figure; 272 | for i = 1:numel(methods) 273 | plot(areas(i), AP(i), '.', 'MarkerSize', 20, 'Color', methods(i).color, 'LineWidth', 1.5); 274 | hold on; 275 | end 276 | grid on; 277 | xlabel(sprintf('average recall')); axis(axis_lim) 278 | % title(sprintf('correlation=%.3f',s)); 279 | ylabel('mAP'); 280 | p=polyfit(areas,AP,1); line([0 1],[p(2),sum(p)],'Color',[1 1 1]/3); hold off 281 | 282 | % move the title inside of the plot 283 | v = axis; 284 | handle = title(sprintf('correlation=%.3f',s)); 285 | titlepos = [(v(2)-v(1))*0.5+v(1), (v(4)-v(3))*0.89+v(3), 0]; 286 | set(handle, 'Position', titlepos); 287 | end 288 | 289 | function ploteroo(T, R, AP, nms, axis_lim) 290 | % plot correlation versus IoU and compute best threshold t 291 | S=corrcoef([R' AP']); S=S(1:end-1,end); [s,t]=max(S); 292 | figure(); plot(T,S,'-or'); xlabel('IoU'); ylabel('corr'); grid on; 293 | % plot AP versus recall at single best threshold t 294 | figure(); R1=R(t,:); plot(R1,AP,'dg'); grid on; text(R1+.015,AP,nms); 295 | xlabel(sprintf('recall at IoU=%.3f',T(t))); axis(axis_lim) 296 | title(sprintf('correlation=%.3f',s)); ylabel('mAP'); hold on; 297 | p=polyfit(R1,AP,1); line([0 1],[p(2),sum(p)],'Color',[1 1 1]/3); hold off 298 | % plot AP versus recall for a series of thresholds 299 | figure; 300 | for t = 1:20 301 | subplot(4,5,t); 302 | R1=R(t,:); plot(R1,AP,'dr'); grid on; %text(R1+.015,AP,nms); 303 | xlabel(sprintf('recall at IoU=%.3f',T(t))); axis(axis_lim) 304 | title(sprintf('correlation=%.3f',S(t))); ylabel('mAP'); hold on; 305 | p=polyfit(R1,AP,1); line([0 1],[p(2),sum(p)],'Color',[1 1 1]/3); hold off 306 | end 307 | % compute correlation against optimal range of thrs(a:b) 308 | n=length(T); S=zeros(n,n); 309 | for a=1:n, for b=a:n, s=corrcoef(sum(R(a:b,:),1),AP); S(a,b)=s(2); end; end 310 | [s,t]=max(S(:)); [a,b]=ind2sub([n n],t); R1=mean(R(a:b,:),1); 311 | figure(); plot(R1,AP,'dg'); grid on; text(R1+.015,AP,nms); 312 | xlabel(sprintf('recall at IoU=%.3f-%.3f',T(a),T(b))); axis(axis_lim) 313 | title(sprintf('correlation=%.3f',s)); ylabel('mAP'); hold on; 314 | p=polyfit(R1,AP,1); line([0 1],[p(2),sum(p)],'Color',[1 1 1]/3); hold off 315 | end 316 | 317 | function plot_correlation_over_recall(T, R, AP, nms, axis_lim) 318 | S=corrcoef([R' AP']); S=S(1:end-1,end); 319 | figure(); xlim([T(1), T(end-1)]); 320 | plot(T,S,'.-', 'Color', [31,120,180]/256, 'MarkerSize', 20); 321 | axis(axis_lim); 322 | xlabel('IoU overlap threshold'); ylabel('correlation with mAP'); grid on; 323 | end 324 | -------------------------------------------------------------------------------- /plot_detector_wiggle_voc07.m: -------------------------------------------------------------------------------- 1 | function plot_detector_wiggle_voc07() 2 | 3 | colormap = [ ... 4 | 83, 136, 173 ; ... 5 | 219, 135, 45 ; ... 6 | 218, 71, 56 ; ... 7 | 135, 130, 174 ; ... 8 | 142, 195, 129 ; ... 9 | 138, 180, 66 ; ... 10 | 223, 200, 51 ; ... 11 | 92, 172, 158 ; ... 12 | ] ./ 256; 13 | 14 | config = []; 15 | config(1).name = 'R-CNN'; 16 | config(1).func = @detector_wiggle_rcnn; 17 | config(1).filename = '/BS/candidate_detections/work/v1/pascal_voc07_test_rcnn_scoremaps-3d.mat'; 18 | config(2).name = 'LM-LLDA'; 19 | config(2).func = @detector_wiggle_rcnn; 20 | config(2).filename = '/BS/candidate_detections/work/v1/pascal_voc07_test_dpm_scoremaps-3d.mat'; 21 | config(3).name = 'LM-LLDA bboxpred'; 22 | config(3).func = @detector_wiggle_rcnn; 23 | config(3).filename = '/BS/candidate_detections/work/v1/pascal_voc07_test_dpm_bboxpred_scoremaps-3d.mat'; 24 | config(4).name = 'Fast R-CNN'; 25 | config(4).func = @detector_wiggle_from_iou_score; 26 | config(4).filename = '/BS/candidate_detections/work/v1/pascal_voc07_test_frcn_iou_score.mat'; 27 | config(5).name = 'Fast R-CNN bboxpred'; 28 | config(5).func = @detector_wiggle_from_iou_score; 29 | config(5).filename = '/BS/candidate_detections/work/v1/pascal_voc07_test_frcn_regr_iou_score.mat'; 30 | 31 | autoscale = true; 32 | 33 | bin_edges = 0.2:0.05:1; 34 | bin_centers = (bin_edges(1:end-1) + bin_edges(2:end)) / 2; 35 | 36 | hists = build_histograms(config, bin_edges); 37 | if autoscale 38 | hists = autoscale_per_class(hists); 39 | end 40 | 41 | plot_histograms(config, bin_centers, bin_edges, hists.avg_scores, hists.class_avg_scores, colormap); 42 | % plot_histograms(config, bin_centers, bin_edges, hists.median_scores, hists.class_median_scores, colormap); 43 | 44 | % figure; 45 | % bar(bin_centers', hists.bin_counts'); 46 | % legend({config.name}, 'Location', 'SouthEast'); 47 | 48 | if false 49 | scale_factors = [0.8182 0.8485 0.8864 0.9242 0.9621 1.0000 1.0455 1.0833 1.1288 1.1742 1.2273]; 50 | x = (-5:5) .* 4*227/128; 51 | y = x; 52 | ticks = [-5 -2 0 2 5] .* 4*227/128; 53 | tick_labels = arrayfun(@(t) sprintf('%.0f', t), ticks(:), 'UniformOutput', false); 54 | method_i = 1; 55 | n_steps = size(hists.avg_cubes{method_i},1); 56 | class_cubes = hists.method_class_avg_cubes(method_i,:); 57 | n_classes = numel(class_cubes); 58 | % normalize per class 59 | for c = 1:n_classes 60 | clims = [min(class_cubes{c}(:)), max(class_cubes{c}(:))]; 61 | class_cubes{c} = (class_cubes{c} - clims(1)) ./ (clims(2) - clims(1)); 62 | end 63 | cube = nanmean(cat(4, class_cubes{:}), 4); 64 | 65 | clims = [min(cube(:)), max(cube(:))]; 66 | 67 | figure; 68 | scale_idxs = [1 3 6 9 11]; 69 | for j = 1:numel(scale_idxs), i = scale_idxs(j); % 1:n_steps 70 | subplot(1,6,j); 71 | imagesc(x, y, cube(:,:,i), clims); 72 | set(gca, 'XTick', ticks, 'XTickLabels', tick_labels); 73 | set(gca, 'YTick', ticks, 'YTickLabels', tick_labels); 74 | axis square 75 | title(sprintf('scale %.2f', scale_factors(i))); 76 | end 77 | subplot(1,6,6); 78 | axis off 79 | cbh = colorbar(); 80 | P = get(cbh, 'Position'); 81 | set(cbh, 'Position', [P(1)-0.01, P(2)+0.1, P(3)+0.01, P(4)-0.2]); 82 | hei = 3.5; wid = 25; 83 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 84 | set(gcf, 'PaperPositionMode','auto'); 85 | printpdf('figures/detector_scoremaps_pos_scale_voc07.pdf'); 86 | printpng('figures/detector_scoremaps_pos_scale_voc07.png'); 87 | 88 | % figure; 89 | % for i = 1:n_steps 90 | % subplot(4,3,i); 91 | % imagesc(reshape(hists.avg_cubes{method_i}(:,i,:), [n_steps n_steps]), clims); 92 | % axis equal 93 | % title(sprintf('x jitter %d', i - floor((n_steps+1)/2))); 94 | % end 95 | % 96 | % figure; 97 | % for i = 1:n_steps 98 | % subplot(4,3,i); 99 | % imagesc(reshape(hists.avg_cubes{method_i}(i,:,:), [n_steps n_steps]), clims); 100 | % axis equal 101 | % title(sprintf('y jitter %d', i - floor((n_steps+1)/2))); 102 | % end 103 | end 104 | end 105 | 106 | function plot_histograms(config, bin_centers, bin_edges, avg_scores, class_avg_scores, colormap) 107 | ld = load('data/classes.mat'); 108 | classes = ld.classes; 109 | 110 | fh_all_classes = figure; hold on; 111 | for i = 1:numel(config) 112 | plot(bin_centers, avg_scores(i,:), '.-', 'LineWidth', 1.5, 'Color', colormap(i,:), 'MarkerSize', 20); 113 | end 114 | legend({config.name}, 'Location', 'SouthEast'); 115 | xlabel('IoU with GT'); ylabel('detector score'); 116 | xlim([min(bin_edges), max(bin_edges)]); 117 | hei = 8; wid = 12; 118 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 119 | set(gcf, 'PaperPositionMode','auto'); 120 | printpdf('figures/detector_scoremaps_voc07.pdf'); 121 | 122 | fh_per_class = figure; 123 | for c = 1:20 124 | for i = 1:numel(config) 125 | figure(fh_per_class); subplot(4,5,c); 126 | plot(bin_centers, class_avg_scores{c}(i,:), '-o', 'LineWidth', 2, 'Color', colormap(i,:)); 127 | hold on; 128 | xlim([min(bin_edges), max(bin_edges)]); 129 | title(classes{c}); 130 | end 131 | end 132 | end 133 | 134 | % function [avg_scores, bin_counts, class_avg_scores, class_bin_counts, avg_cubes] = build_histograms(config, bin_edges) 135 | function hists = build_histograms(config, bin_edges) 136 | histogram_cache_file = 'data/score_histogram_cache.mat'; 137 | try 138 | hists = load(histogram_cache_file); 139 | catch 140 | n_methods = numel(config); 141 | n_classes = 20; 142 | n_bins = numel(bin_edges)-1; 143 | 144 | % allocate 145 | avg_scores = zeros(n_methods, n_bins); 146 | median_scores = zeros(n_methods, n_bins); 147 | bin_counts = zeros(n_methods, n_bins); 148 | class_avg_scores = cell(1, n_classes); 149 | class_median_scores = cell(1, n_classes); 150 | class_bin_counts = cell(1, n_classes); 151 | for c = 1:n_classes 152 | class_avg_scores{c} = zeros(n_methods, n_bins); 153 | class_median_scores{c} = zeros(n_methods, n_bins); 154 | class_bin_counts{c} = zeros(n_methods, n_bins); 155 | end 156 | avg_cubes = cell(n_methods,1); 157 | method_class_avg_cubes = cell(n_methods,n_classes); 158 | 159 | % compute stuff 160 | for i = 1:n_methods, conf = config(i); 161 | fprintf('computing %d/%d %s\n', i, numel(config), conf.name); 162 | [overlap, score, class_ids, avg_cubes{i}, class_avg_cubes] = conf.func(conf.filename); 163 | if ~isempty(class_avg_cubes) 164 | method_class_avg_cubes(i,:) = class_avg_cubes; 165 | end 166 | 167 | [bin_counts(i,:), avg_scores(i,:), median_scores(i,:)] = build_histogram(overlap, score, bin_edges); 168 | 169 | fprintf('per class: '); 170 | for c = 1:n_classes 171 | fprintf('%d ', c); 172 | class_mask = class_ids == c; 173 | class_overlap = overlap(class_mask); 174 | class_score = score(class_mask); 175 | [class_bin_counts{c}(i,:), class_avg_scores{c}(i,:), class_median_scores{c}(i,:)] = build_histogram(class_overlap, class_score, bin_edges); 176 | end 177 | fprintf('\n'); 178 | end 179 | 180 | save(histogram_cache_file, 'avg_scores', 'median_scores', 'bin_counts', ... 181 | 'class_avg_scores', 'class_median_scores', 'class_bin_counts', ... 182 | 'avg_cubes', 'method_class_avg_cubes'); 183 | hists = load(histogram_cache_file); 184 | end 185 | end 186 | 187 | function [hists] = autoscale_per_class(hists) 188 | n_classes = 20; 189 | n_methods = size(hists.avg_scores,1); 190 | n_bins = size(hists.avg_scores,2); 191 | 192 | denom = zeros(n_methods,n_bins); 193 | hists.avg_scores = zeros(n_methods,n_bins); 194 | hists.median_scores = zeros(n_methods,n_bins); 195 | for c = 1:n_classes 196 | hists.class_avg_scores{c} = autoscale_scores(hists.class_avg_scores{c}); 197 | hists.class_median_scores{c} = autoscale_scores(hists.class_median_scores{c}); 198 | 199 | hists.avg_scores = hists.avg_scores + hists.class_avg_scores{c} .* hists.class_bin_counts{c}; 200 | hists.median_scores = hists.median_scores + hists.class_median_scores{c} .* hists.class_bin_counts{c}; 201 | denom = denom + hists.class_bin_counts{c}; 202 | end 203 | hists.avg_scores = hists.avg_scores ./ denom; 204 | hists.median_scores = hists.median_scores ./ denom; 205 | assert(all(denom(:) == hists.bin_counts(:))); 206 | end 207 | 208 | function scaled_scores = autoscale_scores(avg_scores) 209 | minscore = min(avg_scores, [], 2); 210 | maxscore = max(avg_scores, [], 2); 211 | minscore = repmat(minscore, [1, size(avg_scores, 2)]); 212 | maxscore = repmat(maxscore, [1, size(avg_scores, 2)]); 213 | scaled_scores = (avg_scores - minscore) ./ (maxscore - minscore); 214 | end 215 | 216 | function [counts, avg_scores, median_scores] = build_histogram(overlap, score, bin_edges) 217 | % [counts,~,bin_idxs] = histcounts(overlap, bin_edges); 218 | [counts,bin_idxs] = histc(overlap, bin_edges); 219 | counts = counts(1:end-1); 220 | 221 | n_centers = numel(bin_edges)-1; 222 | avg_scores = zeros(1, n_centers); 223 | median_scores = zeros(1, n_centers); 224 | % average over the elements in each bin 225 | for j = 1:n_centers 226 | bin_scores = score(bin_idxs == j); 227 | if ~isempty(bin_scores) 228 | avg_scores(j) = mean(bin_scores); 229 | median_scores(j) = median(bin_scores); 230 | end 231 | end 232 | end 233 | -------------------------------------------------------------------------------- /plot_recall_ILSVRC2013.m: -------------------------------------------------------------------------------- 1 | function plot_recall_ILSVRC2013() 2 | % Plot the recall of ILSVRC2013 validation set ground truth for all methods. 3 | % 4 | % This function requires the proposals to already be saved to disk. It will 5 | % compute a matching between ground truth and proposals (if the result is not 6 | % yet found on disk) and then plot recall curves. The plots are saved to 7 | % figures/. 8 | 9 | val = load('data/ILSVRC2013_val_annotations.mat'); 10 | methods = get_method_configs(); 11 | methods([14 16 19:24]) = []; 12 | 13 | valid_methods = compute_best_candidates(val, methods); 14 | methods = methods(valid_methods); 15 | 16 | plot_legend(methods); 17 | printpdf('figures/imagenet_recall_legend.pdf'); 18 | 19 | fprintf('\n\n'); 20 | for i = 1:numel(methods) 21 | fprintf('Valid data for plots: %s\n', methods(i).name); 22 | end 23 | 24 | fh = figure; 25 | plot_overlap_recall_curve({methods.best_imagenet_candidates_file}, methods, 100, fh, true, 'none'); 26 | hei = 10; 27 | wid = 10; 28 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 29 | set(gcf, 'PaperPositionMode','auto'); 30 | printpdf('figures/imagenet_recall_100.pdf') 31 | 32 | fh = figure; 33 | plot_overlap_recall_curve({methods.best_imagenet_candidates_file}, methods, 1000, fh, false, 'none'); 34 | hei = 10; 35 | wid = 10; 36 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 37 | set(gcf, 'PaperPositionMode','auto'); 38 | printpdf('figures/imagenet_recall_1000.pdf') 39 | 40 | fh = figure; 41 | plot_overlap_recall_curve({methods.best_imagenet_candidates_file}, methods, 10000, fh, false, 'none'); 42 | hei = 10; 43 | wid = 10; 44 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 45 | set(gcf, 'PaperPositionMode','auto'); 46 | printpdf('figures/imagenet_recall_10000.pdf') 47 | 48 | plot_num_candidates_auc({methods.best_imagenet_candidates_file}, methods, 'imagenet_'); 49 | end 50 | 51 | function valid_methods = compute_best_candidates(testset, methods) 52 | num_annotations = numel(testset.pos); 53 | candidates_thresholds = round(10 .^ (0:0.5:4)); 54 | num_candidates_thresholds = numel(candidates_thresholds); 55 | 56 | valid_methods = false(numel(methods), 1); 57 | for method_idx = 1:numel(methods) 58 | method = methods(method_idx); 59 | fprintf('%s\n', method.name); 60 | try 61 | load(method.best_imagenet_candidates_file, 'best_candidates'); 62 | valid_methods(method_idx) = true; 63 | continue; 64 | catch 65 | end 66 | 67 | % preallocate 68 | best_candidates = []; 69 | best_candidates(num_candidates_thresholds).candidates_threshold = []; 70 | best_candidates(num_candidates_thresholds).best_candidates = []; 71 | for i = 1:num_candidates_thresholds 72 | best_candidates(i).candidates_threshold = candidates_thresholds(i); 73 | best_candidates(i).best_candidates.candidates = zeros(num_annotations, 4); 74 | best_candidates(i).best_candidates.iou = zeros(num_annotations, 1); 75 | best_candidates(i).image_statistics(numel(testset.impos)).num_candidates = 0; 76 | end 77 | 78 | files_missing = false; 79 | pos_range_start = 1; 80 | for j = 1:numel(testset.impos) 81 | tic_toc_print('evalutating %s: %d/%d\n', method.name, j, numel(testset.impos)); 82 | pos_range_end = pos_range_start + size(testset.impos(j).boxes, 1) - 1; 83 | assert(pos_range_end <= num_annotations); 84 | 85 | % tic_toc_print('sampling candidates for image %d/%d\n', j, numel(testset.impos)); 86 | [~,img_id,~] = fileparts(testset.impos(j).im); 87 | 88 | for i = 1:num_candidates_thresholds 89 | try 90 | [candidates, scores] = get_candidates(method, img_id, ... 91 | candidates_thresholds(i)); 92 | catch 93 | fprintf('%s candidates for %s missing\n', method.name, img_id); 94 | files_missing = true; 95 | break; 96 | end 97 | if isempty(candidates) 98 | impos_best_ious = zeros(size(testset.impos(j).boxes,1),1); 99 | impos_best_boxes = zeros(size(testset.impos(j).boxes,1),4); 100 | else 101 | [impos_best_ious, impos_best_boxes] = closest_candidates(... 102 | testset.impos(j).boxes, candidates); 103 | end 104 | 105 | best_candidates(i).best_candidates.candidates(pos_range_start:pos_range_end,:) = impos_best_boxes; 106 | best_candidates(i).best_candidates.iou(pos_range_start:pos_range_end) = impos_best_ious; 107 | best_candidates(i).image_statistics(j).num_candidates = size(candidates, 1); 108 | end 109 | if files_missing 110 | break; 111 | end 112 | 113 | pos_range_start = pos_range_end + 1; 114 | end 115 | valid_methods(method_idx) = ~files_missing; 116 | if ~files_missing 117 | save(method.best_imagenet_candidates_file, 'best_candidates'); 118 | end 119 | 120 | end 121 | end 122 | -------------------------------------------------------------------------------- /plot_recall_all_datasets.m: -------------------------------------------------------------------------------- 1 | function plot_recall_all_datasets() 2 | % Plot the average recall of PASCAL, ImageNet, and COCO for all methods. 3 | % 4 | % This function requires the proposals to already be saved to disk. It will 5 | % compute a matching between ground truth and proposals (if the result is not 6 | % yet found on disk) and then plot recall curves. The plots are saved to 7 | % figures/. 8 | 9 | methods = get_method_configs(); 10 | methods([14 16 19:25]) = []; 11 | 12 | pascal = {methods.best_voc07_candidates_file}; 13 | imagenet = {methods.best_imagenet_candidates_file}; 14 | coco = {methods.best_coco14_candidates_file}; 15 | 16 | valid_methods = check_for_data(cat(2, pascal(:), imagenet(:), coco(:))); 17 | if ~any(valid_methods) 18 | fprintf('no methods have the recall matchings computed, run the plot scripts for each of the datasets, to get the required data\n'); 19 | return 20 | end 21 | methods = methods(valid_methods); 22 | pascal = {methods.best_voc07_candidates_file}; 23 | imagenet = {methods.best_imagenet_candidates_file}; 24 | coco = {methods.best_coco14_candidates_file}; 25 | 26 | % plot_legend(methods); 27 | % printpdf('figures/imagenet_recall_legend.pdf'); 28 | 29 | fprintf('\n\n'); 30 | for i = 1:numel(methods) 31 | fprintf('Valid data for plots: %s\n', methods(i).name); 32 | end 33 | 34 | dataset_labels = {'PASCAL 2007', 'ImageNet 2013', 'COCO 2014'}; 35 | AR_plots(methods, cat(2, pascal(:), imagenet(:), coco(:)), dataset_labels); 36 | end 37 | 38 | 39 | function AR_plots(methods, iou_files, dataset_labels) 40 | num_candidates = 1000; 41 | 42 | n = size(iou_files, 1); 43 | num_datasets = size(iou_files, 2); 44 | 45 | for i = 1:n 46 | ARs = zeros(1, num_datasets); 47 | for dataset_i = 1:num_datasets, iou_file = iou_files{i,dataset_i}; 48 | try 49 | data = load(iou_file); 50 | 51 | num_experiments = numel(data.best_candidates); 52 | exp_idx = find([data.best_candidates.candidates_threshold] == num_candidates); 53 | assert(numel(exp_idx) == 1); 54 | experiment = data.best_candidates(exp_idx); 55 | [~, ~, ARs(dataset_i)] = compute_average_recall(experiment.best_candidates.iou); 56 | catch 57 | ARs(dataset_i) = nan; 58 | end 59 | end 60 | 61 | line_style = '-'; 62 | if methods(i).is_baseline 63 | line_style = '--'; 64 | end 65 | if ~isempty(methods(i).line_style) 66 | line_style = methods(i).line_style; 67 | end 68 | plot(1:num_datasets, ARs, '.', 'MarkerSize', 20, 'Color', methods(i).color, 'LineWidth', 1.5, 'LineStyle', line_style); 69 | hold on; 70 | end 71 | xlim([0.5, num_datasets + 0.5]); 72 | ylim([0.1, 0.6]); 73 | ylabel('average recall'); 74 | hold off; grid on; 75 | set(gca,'XTick',(1:num_datasets),'XTickLabel',dataset_labels); 76 | set(gca,'XTickLabelRotation',25); 77 | set(gca, 'box','off'); 78 | set(gca,'YTick', 0.1:0.1:0.6); 79 | 80 | hei = 10; 81 | wid = 8; 82 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 83 | set(gcf, 'PaperPositionMode','auto'); 84 | printpdf('figures/average_recall_all_datasets.pdf'); 85 | end 86 | 87 | 88 | function valid = check_for_data(candidate_files) 89 | valid = true(size(candidate_files)); 90 | for i = 1:size(candidate_files,1) 91 | for j = 1:size(candidate_files,2) 92 | try 93 | ld = load(candidate_files{i,j}); 94 | catch 95 | fprintf('file missing: %s\n', candidate_files{i,j}); 96 | valid(i,j) = false; 97 | end 98 | end 99 | end 100 | valid = all(valid,2); 101 | end 102 | -------------------------------------------------------------------------------- /plot_recall_coco2014.m: -------------------------------------------------------------------------------- 1 | function plot_recall_coco2014() 2 | % Plot the recall of COCO 2014 validation set ground truth for all methods. 3 | % 4 | % This function requires the proposals to already be saved to disk. It will 5 | % compute a matching between ground truth and proposals (if the result is not 6 | % yet found on disk) and then plot recall curves. The plots are saved to 7 | % figures/. 8 | 9 | val = load('data/coco2014_val_annotations.mat'); 10 | methods = get_method_configs(); 11 | methods([14 16 19:25]) = []; 12 | [~,order] = sort([methods.sort_key]); 13 | methods = methods(order); 14 | 15 | valid_methods = compute_best_candidates(val, methods); 16 | methods = methods(valid_methods); 17 | 18 | plot_legend(methods); 19 | printpdf('figures/coco14_recall_legend.pdf'); 20 | 21 | fprintf('\n\n'); 22 | for i = 1:numel(methods) 23 | fprintf('Valid data for plots: %s\n', methods(i).name); 24 | end 25 | 26 | fh = figure; 27 | plot_overlap_recall_curve({methods.best_coco14_candidates_file}, methods, 100, fh, true, 'none'); 28 | hei = 10; 29 | wid = 10; 30 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 31 | set(gcf, 'PaperPositionMode','auto'); 32 | printpdf('figures/coco14_recall_100.pdf') 33 | 34 | fh = figure; 35 | plot_overlap_recall_curve({methods.best_coco14_candidates_file}, methods, 1000, fh, false, 'none'); 36 | hei = 10; 37 | wid = 10; 38 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 39 | set(gcf, 'PaperPositionMode','auto'); 40 | printpdf('figures/coco14_recall_1000.pdf') 41 | 42 | fh = figure; 43 | plot_overlap_recall_curve({methods.best_coco14_candidates_file}, methods, 10000, fh, false, 'none'); 44 | hei = 10; 45 | wid = 10; 46 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 47 | set(gcf, 'PaperPositionMode','auto'); 48 | printpdf('figures/coco14_recall_10000.pdf') 49 | 50 | plot_num_candidates_auc({methods.best_coco14_candidates_file}, methods, 'coco14_'); 51 | end 52 | 53 | function valid_methods = compute_best_candidates(testset, methods) 54 | num_annotations = sum(cellfun(@(x) size(x,1), {testset.impos.boxes})); 55 | candidates_thresholds = round(10 .^ (0:0.5:4)); 56 | num_candidates_thresholds = numel(candidates_thresholds); 57 | 58 | valid_methods = false(numel(methods), 1); 59 | for method_idx = 1:numel(methods) 60 | method = methods(method_idx); 61 | fprintf('%s\n', method.name); 62 | try 63 | load(method.best_coco14_candidates_file, 'best_candidates'); 64 | valid_methods(method_idx) = true; 65 | continue; 66 | catch 67 | end 68 | 69 | % preallocate 70 | best_candidates = []; 71 | best_candidates(num_candidates_thresholds).candidates_threshold = []; 72 | best_candidates(num_candidates_thresholds).best_candidates = []; 73 | for i = 1:num_candidates_thresholds 74 | best_candidates(i).candidates_threshold = candidates_thresholds(i); 75 | best_candidates(i).best_candidates.candidates = zeros(num_annotations, 4); 76 | best_candidates(i).best_candidates.iou = zeros(num_annotations, 1); 77 | best_candidates(i).image_statistics(numel(testset.impos)).num_candidates = 0; 78 | end 79 | 80 | files_missing = false; 81 | pos_range_start = 1; 82 | for j = 1:numel(testset.impos) 83 | tic_toc_print('evalutating %s: %d/%d\n', method.name, j, numel(testset.impos)); 84 | pos_range_end = pos_range_start + size(testset.impos(j).boxes, 1) - 1; 85 | assert(pos_range_end <= num_annotations); 86 | 87 | % tic_toc_print('sampling candidates for image %d/%d\n', j, numel(testset.impos)); 88 | [~,img_id,~] = fileparts(testset.impos(j).im); 89 | 90 | for i = 1:num_candidates_thresholds 91 | try 92 | [candidates, scores] = get_candidates(method, img_id, ... 93 | candidates_thresholds(i), true, [], method.candidate_dir_coco, true); 94 | catch 95 | fprintf('%s candidates for %s missing\n', method.name, img_id); 96 | files_missing = true; 97 | break; 98 | end 99 | if isempty(candidates) 100 | impos_best_ious = zeros(size(testset.impos(j).boxes,1),1); 101 | impos_best_boxes = zeros(size(testset.impos(j).boxes,1),4); 102 | else 103 | [impos_best_ious, impos_best_boxes] = closest_candidates(... 104 | testset.impos(j).boxes, candidates); 105 | end 106 | 107 | best_candidates(i).best_candidates.candidates(pos_range_start:pos_range_end,:) = impos_best_boxes; 108 | best_candidates(i).best_candidates.iou(pos_range_start:pos_range_end) = impos_best_ious; 109 | best_candidates(i).image_statistics(j).num_candidates = size(candidates, 1); 110 | end 111 | if files_missing 112 | break; 113 | end 114 | 115 | pos_range_start = pos_range_end + 1; 116 | end 117 | valid_methods(method_idx) = ~files_missing; 118 | if ~files_missing 119 | save(method.best_coco14_candidates_file, 'best_candidates'); 120 | end 121 | 122 | end 123 | end 124 | -------------------------------------------------------------------------------- /plot_recall_voc07.m: -------------------------------------------------------------------------------- 1 | function plot_recall_voc07() 2 | % Plot the recall of pascal test set ground truth for all methods. 3 | % 4 | % This function requires the proposals to already be saved to disk. It will 5 | % compute a matching between ground truth and proposals (if the result is not 6 | % yet found on disk) and then plot recall curves. The plots are saved to 7 | % figures/. 8 | 9 | testset = load('data/pascal_voc07_test_annotations.mat'); 10 | methods = get_method_configs(); 11 | methods([14 16 19:25]) = []; 12 | 13 | compute_best_candidates(testset, methods); 14 | 15 | plot_legend(methods); 16 | printpdf('figures/recall_legend.pdf'); 17 | 18 | fh = figure; 19 | plot_overlap_recall_curve({methods.best_voc07_candidates_file}, methods, 100, fh, true, 'none'); 20 | hei = 10; 21 | wid = 10; 22 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 23 | set(gcf, 'PaperPositionMode','auto'); 24 | printpdf('figures/recall_100.pdf') 25 | 26 | fh = figure; 27 | plot_overlap_recall_curve({methods.best_voc07_candidates_file}, methods, 1000, fh, false, 'none'); 28 | hei = 10; 29 | wid = 10; 30 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 31 | set(gcf, 'PaperPositionMode','auto'); 32 | printpdf('figures/recall_1000.pdf') 33 | 34 | fh = figure; 35 | plot_overlap_recall_curve({methods.best_voc07_candidates_file}, methods, 10000, fh, false, 'none'); 36 | hei = 10; 37 | wid = 10; 38 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 39 | set(gcf, 'PaperPositionMode','auto'); 40 | printpdf('figures/recall_10000.pdf') 41 | 42 | plot_num_candidates_auc({methods.best_voc07_candidates_file}, methods); 43 | end 44 | 45 | function compute_best_candidates(testset, methods) 46 | num_annotations = numel(testset.pos); 47 | candidates_thresholds = round(10 .^ (0:0.5:4)); 48 | num_candidates_thresholds = numel(candidates_thresholds); 49 | 50 | for method_idx = 1:numel(methods) 51 | method = methods(method_idx); 52 | try 53 | load(method.best_voc07_candidates_file, 'best_candidates'); 54 | continue 55 | catch 56 | end 57 | 58 | % preallocate 59 | best_candidates = []; 60 | best_candidates(num_candidates_thresholds).candidates_threshold = []; 61 | best_candidates(num_candidates_thresholds).best_candidates = []; 62 | for i = 1:num_candidates_thresholds 63 | best_candidates(i).candidates_threshold = candidates_thresholds(i); 64 | best_candidates(i).best_candidates.candidates = zeros(num_annotations, 4); 65 | best_candidates(i).best_candidates.iou = zeros(num_annotations, 1); 66 | best_candidates(i).image_statistics(numel(testset.impos)).num_candidates = 0; 67 | end 68 | 69 | pos_range_start = 1; 70 | for j = 1:numel(testset.impos) 71 | tic_toc_print('evalutating %s: %d/%d\n', method.name, j, numel(testset.impos)); 72 | pos_range_end = pos_range_start + size(testset.impos(j).boxes, 1) - 1; 73 | assert(pos_range_end <= num_annotations); 74 | 75 | tic_toc_print('sampling candidates for image %d/%d\n', j, numel(testset.impos)); 76 | [~,img_id,~] = fileparts(testset.impos(j).im); 77 | 78 | for i = 1:num_candidates_thresholds 79 | [candidates, scores] = get_candidates(method, img_id, ... 80 | candidates_thresholds(i)); 81 | if isempty(candidates) 82 | impos_best_ious = zeros(size(testset.impos(j).boxes, 1), 1); 83 | impos_best_boxes = zeros(size(testset.impos(j).boxes, 1), 4); 84 | else 85 | [impos_best_ious, impos_best_boxes] = closest_candidates(... 86 | testset.impos(j).boxes, candidates); 87 | end 88 | 89 | best_candidates(i).best_candidates.candidates(pos_range_start:pos_range_end,:) = impos_best_boxes; 90 | best_candidates(i).best_candidates.iou(pos_range_start:pos_range_end) = impos_best_ious; 91 | best_candidates(i).image_statistics(j).num_candidates = size(candidates, 1); 92 | end 93 | 94 | pos_range_start = pos_range_end + 1; 95 | end 96 | 97 | save(method.best_voc07_candidates_file, 'best_candidates'); 98 | end 99 | end 100 | -------------------------------------------------------------------------------- /plot_size_distribution.m: -------------------------------------------------------------------------------- 1 | function plot_size_distribution() 2 | num_candidates = 10000; 3 | 4 | box_size_bins = 0:0.1:1; 5 | box_size_bin_centers = (box_size_bins(1:(end-1)) + box_size_bins(2:end)) / 2; 6 | box_size_bins(1) = -inf; 7 | box_size_bins(end) = inf; 8 | num_bins = numel(box_size_bins) - 1; 9 | 10 | load('data/pascal_voc07_test_annotations.mat'); 11 | images = {impos.im}; 12 | image_sizes = {impos.img_size}; 13 | 14 | 15 | methods = get_method_configs(); 16 | methods([14 16 19:25]) = []; 17 | 18 | handles = []; 19 | 20 | histograms = zeros(numel(methods), num_bins); 21 | legend_labels = cell(numel(methods), 1); 22 | valid = true(numel(methods), 1); 23 | for method_i = 1:numel(methods) 24 | method = methods(method_i); 25 | fprintf('method: %s\n', method.name); 26 | legend_labels{method_i} = method.name; 27 | cache_file = fullfile(method.candidate_dir, 'size_hist.mat'); 28 | try 29 | data = load(cache_file); 30 | h = data.h; 31 | catch 32 | h = zeros(1, num_bins); 33 | for im_i = 1:numel(images) 34 | tic_toc_print('%s %d/%d\n', method.name, im_i, numel(images)); 35 | [~,img_id,~] = fileparts(images{im_i}); 36 | try 37 | [candidates] = get_candidates(method, img_id, ... 38 | num_candidates); 39 | catch 40 | valid(method_i) = false; 41 | fprintf('candidates for img_id %s are missing!\n', img_id); 42 | break; 43 | end 44 | if isempty(candidates) 45 | continue; 46 | end 47 | t_h = get_size_statistics(candidates, image_sizes{im_i}, box_size_bins); 48 | h = h + t_h(:)'; 49 | end 50 | if valid(method_i) 51 | save(cache_file, 'h'); 52 | end 53 | end 54 | if valid(method_i) 55 | histograms(method_i,:) = h; 56 | end 57 | end 58 | 59 | % normalize histograms 60 | histograms = histograms ./ repmat(sum(histograms, 2), [1, num_bins]); 61 | 62 | histograms = histograms(valid,:); 63 | legend_labels = legend_labels(valid); 64 | methods = methods(valid); 65 | 66 | [~,method_order] = sort([methods.sort_key]); 67 | methods = methods(method_order); 68 | histograms = histograms(method_order,:); 69 | legend_labels = legend_labels(method_order); 70 | figure; hold on; 71 | for i = 1:numel(methods) 72 | line_style = '-'; 73 | if methods(i).is_baseline 74 | line_style = '--'; 75 | end 76 | if ~isempty(methods(i).line_style) 77 | line_style = methods(i).line_style; 78 | end 79 | handles(end+1) = plot(box_size_bin_centers, histograms(i,:), '.', 'LineWidth', 1.5, 'Color', methods(i).color, 'MarkerSize', 10, 'LineStyle', line_style); 80 | end 81 | 82 | gt_w = [pos.x2] - [pos.x1] + 1; 83 | gt_h = [pos.y2] - [pos.y1] + 1; 84 | areas = gt_w .* gt_h; 85 | for i = 1:numel(pos) 86 | areas(i) = sqrt(areas(i) / prod(pos(i).img_size)); 87 | end 88 | [gt_h,arg_hist] = histc(areas, box_size_bins); 89 | assert(min(arg_hist) >= 1); 90 | assert(max(arg_hist) <= numel(box_size_bins) - 1); 91 | gt_h = gt_h(1:end-1)'; 92 | gt_h = gt_h / sum(gt_h); 93 | handles(end+1) = plot(box_size_bin_centers, gt_h, '.-', 'LineWidth', 1.5, 'Color', 'black', 'MarkerSize', 10); 94 | legend_labels{end+1} = 'Ground truth VOC 2007'; 95 | 96 | val = load('data/ILSVRC2013_val_annotations.mat'); 97 | pos = val.pos; 98 | gt_w = [pos.x2] - [pos.x1] + 1; 99 | gt_h = [pos.y2] - [pos.y1] + 1; 100 | areas = gt_w .* gt_h; 101 | for i = 1:numel(pos) 102 | areas(i) = sqrt(areas(i) / prod(pos(i).img_size)); 103 | end 104 | [gt_h,arg_hist] = histc(areas, box_size_bins); 105 | assert(min(arg_hist) >= 1); 106 | assert(max(arg_hist) <= numel(box_size_bins) - 1); 107 | gt_h = gt_h(1:end-1)'; 108 | gt_h = gt_h / sum(gt_h); 109 | handles(end+1) = plot(box_size_bin_centers, gt_h, '.--', 'LineWidth', 1.5, 'Color', 'black', 'MarkerSize', 10); 110 | legend_labels{end+1} = 'Ground truth ILSVRC 2013'; 111 | 112 | val = load('data/coco2014_val_annotations.mat'); 113 | pos = val.pos; 114 | gt_w = [pos.x2] - [pos.x1] + 1; 115 | gt_h = [pos.y2] - [pos.y1] + 1; 116 | areas = gt_w .* gt_h; 117 | for i = 1:numel(pos) 118 | areas(i) = sqrt(areas(i) / prod(pos(i).img_size)); 119 | end 120 | [gt_h,arg_hist] = histc(areas, box_size_bins); 121 | assert(min(arg_hist) >= 1); 122 | assert(max(arg_hist) <= numel(box_size_bins) - 1); 123 | gt_h = gt_h(1:end-1)'; 124 | gt_h = gt_h / sum(gt_h); 125 | handles(end+1) = plot(box_size_bin_centers, gt_h, '^-', 'LineWidth', 1.5, 'Color', 'black', 'MarkerSize', 5); 126 | legend_labels{end+1} = 'Ground truth COCO 2014'; 127 | 128 | xlabel('sqrt(relative candidate size)'); 129 | ylabel('frequency'); 130 | hei = 10; 131 | wid = 10; 132 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 133 | set(gcf, 'PaperPositionMode','auto'); 134 | printpdf('figures/candidate_size_histogram.pdf') 135 | 136 | 137 | legend(legend_labels); 138 | legend boxoff; 139 | for i = 1:numel(handles) 140 | set(handles(i), 'visible', 'off'); 141 | end 142 | set(gca, 'visible', 'off'); 143 | printpdf('figures/candidate_size_histogram_legend.pdf') 144 | 145 | end 146 | 147 | function h = get_size_statistics(candidates, im_size, box_size_bins) 148 | boxes = candidates; 149 | w = boxes(:,3) - boxes(:,1) + 1; 150 | h = boxes(:,4) - boxes(:,2) + 1; 151 | areas = sqrt(w .* h ./ prod(im_size)); 152 | [h,arg_hist] = histc(areas, box_size_bins); 153 | 154 | assert(min(arg_hist) >= 1); 155 | assert(max(arg_hist) <= numel(box_size_bins) - 1); 156 | 157 | assert(h(end) == 0); 158 | h = h(1:end-1)'; 159 | end 160 | -------------------------------------------------------------------------------- /plot_size_distribution_datasets.m: -------------------------------------------------------------------------------- 1 | function plot_size_distribution_datasets() 2 | colormap = [ 3 | 228, 229, 97 4 | 163, 163, 163 5 | 218, 71, 56 6 | 219, 135, 45 7 | 145, 92, 146 8 | 83, 136, 173 9 | 106,61,154 10 | 225, 119, 174 11 | 142, 195, 129 12 | 51,160,44 13 | 223, 200, 51 14 | 92, 172, 158 15 | 177,89,40 16 | 177,89,40 17 | 188, 128, 189 18 | 177,89,40 19 | 251,154,153 20 | 31,120,180]./255; 21 | 22 | box_size_bins = 0:10:600; 23 | box_size_bin_centers = (box_size_bins(1:(end-1)) + box_size_bins(2:end)) / 2; 24 | box_size_bins(1) = -inf; 25 | box_size_bins(end) = inf; 26 | num_bins = numel(box_size_bins) - 1; 27 | 28 | handles = []; 29 | legend_labels = {}; 30 | 31 | figure; hold on; 32 | 33 | load('data/pascal_voc07_test_annotations.mat'); 34 | gt_w = [pos.x2] - [pos.x1] + 1; 35 | gt_h = [pos.y2] - [pos.y1] + 1; 36 | areas = sqrt(gt_w .* gt_h); 37 | [gt_h,arg_hist] = histc(areas, box_size_bins); 38 | assert(min(arg_hist) >= 1); 39 | assert(max(arg_hist) <= numel(box_size_bins) - 1); 40 | gt_h = gt_h(1:end-1)'; 41 | gt_h = gt_h / sum(gt_h); 42 | handles(end+1) = plot(box_size_bin_centers, gt_h, '-', ... 43 | 'LineWidth', 1.5, 'Color', colormap(1,:), 'MarkerSize', 10); 44 | legend_labels{end+1} = 'VOC test 2007'; 45 | 46 | val = load('data/ILSVRC2013_val_annotations.mat'); 47 | pos = val.pos; 48 | gt_w = [pos.x2] - [pos.x1] + 1; 49 | gt_h = [pos.y2] - [pos.y1] + 1; 50 | areas = sqrt(gt_w .* gt_h); 51 | [gt_h,arg_hist] = histc(areas, box_size_bins); 52 | assert(min(arg_hist) >= 1); 53 | assert(max(arg_hist) <= numel(box_size_bins) - 1); 54 | gt_h = gt_h(1:end-1)'; 55 | gt_h = gt_h / sum(gt_h); 56 | handles(end+1) = plot(box_size_bin_centers, gt_h, '-', ... 57 | 'LineWidth', 1.5, 'Color', colormap(2,:), 'MarkerSize', 10); 58 | legend_labels{end+1} = 'ILSVRC val 2013'; 59 | 60 | val = load('data/coco2014_val_annotations.mat'); 61 | pos = val.pos; 62 | gt_w = [pos.x2] - [pos.x1] + 1; 63 | gt_h = [pos.y2] - [pos.y1] + 1; 64 | areas = sqrt(gt_w .* gt_h); 65 | [gt_h,arg_hist] = histc(areas, box_size_bins); 66 | assert(min(arg_hist) >= 1); 67 | assert(max(arg_hist) <= numel(box_size_bins) - 1); 68 | gt_h = gt_h(1:end-1)'; 69 | gt_h = gt_h / sum(gt_h); 70 | handles(end+1) = plot(box_size_bin_centers, gt_h, '-', ... 71 | 'LineWidth', 1.5, 'Color', colormap(3,:), 'MarkerSize', 10); 72 | legend_labels{end+1} = 'COCO val 2014'; 73 | 74 | legend(legend_labels, 'Location', 'northeast'); 75 | legend boxoff; 76 | xlabel('sqrt(annotation area)'); 77 | ylabel('frequency'); 78 | xlim([0, 400]); 79 | hei = 10; 80 | wid = 8; 81 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 82 | set(gcf, 'PaperPositionMode','auto'); 83 | printpdf('figures/datasets_size_histogram.pdf') 84 | 85 | 86 | end 87 | 88 | function h = get_size_statistics(candidates, im_size, box_size_bins) 89 | boxes = candidates; 90 | w = boxes(:,3) - boxes(:,1) + 1; 91 | h = boxes(:,4) - boxes(:,2) + 1; 92 | areas = sqrt(w .* h ./ prod(im_size)); 93 | [h,arg_hist] = histc(areas, box_size_bins); 94 | 95 | assert(min(arg_hist) >= 1); 96 | assert(max(arg_hist) <= numel(box_size_bins) - 1); 97 | 98 | assert(h(end) == 0); 99 | h = h(1:end-1)'; 100 | end 101 | -------------------------------------------------------------------------------- /recall/compute_average_recall.m: -------------------------------------------------------------------------------- 1 | function [overlap, recall, AR] = compute_average_recall(unsorted_overlaps) 2 | all_overlaps = sort(unsorted_overlaps(:)', 'ascend'); 3 | num_pos = numel(all_overlaps); 4 | dx = 0.001; 5 | 6 | overlap = 0:dx:1; 7 | overlap(end) = 1; 8 | recall = zeros(length(overlap), 1); 9 | for i = 1:length(overlap) 10 | recall(i) = sum(all_overlaps >= overlap(i)) / num_pos; 11 | end 12 | 13 | good_recall = recall(overlap >= 0.5); 14 | AR = 2 * dx * trapz(good_recall); 15 | end 16 | -------------------------------------------------------------------------------- /recall/plot_num_candidates_auc.m: -------------------------------------------------------------------------------- 1 | function plot_num_candidates_auc(iou_files, methods, output_file_prefix) 2 | if nargin < 3 3 | output_file_prefix = ''; 4 | end 5 | 6 | [~,method_order] = sort([methods.sort_key]); 7 | methods = methods(method_order); 8 | iou_files = iou_files(method_order); 9 | 10 | labels = {methods.short_name}; 11 | assert(numel(iou_files) == numel(labels)); 12 | n = numel(iou_files); 13 | 14 | figure; 15 | for i = 1:n 16 | data = load(iou_files{i}); 17 | num_experiments = numel(data.best_candidates); 18 | x = zeros(num_experiments, 1); 19 | y = zeros(num_experiments, 1); 20 | for exp_idx = 1:num_experiments 21 | experiment = data.best_candidates(exp_idx); 22 | [~, ~, AR] = compute_average_recall(experiment.best_candidates.iou); 23 | x(exp_idx) = mean([experiment.image_statistics.num_candidates]); 24 | y(exp_idx) = AR; 25 | end 26 | line_style = '-'; 27 | if methods(i).is_baseline 28 | line_style = '--'; 29 | end 30 | if ~isempty(methods(i).line_style) 31 | line_style = methods(i).line_style; 32 | end 33 | semilogx(x, y, 'Color', methods(i).color, 'LineWidth', 1.5, 'LineStyle', line_style); 34 | hold on; grid on; 35 | end 36 | xlim([10, 10000]); 37 | ylim([0 1]); 38 | xlabel('# proposals'); ylabel('average recall'); 39 | hei = 10; 40 | wid = 10; 41 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 42 | set(gcf, 'PaperPositionMode','auto'); 43 | printpdf(sprintf('figures/%snum_candidates_average_recall.pdf', output_file_prefix)); 44 | 45 | % fixed threshold 46 | % legend_locations = {'SouthEast', 'NorthWest', 'NorthWest'}; 47 | thresholds = [0.5 0.7 0.75 0.8]; 48 | for threshold_i = 1:numel(thresholds) 49 | threshold = thresholds(threshold_i); 50 | figure; 51 | for i = 1:n 52 | data = load(iou_files{i}); 53 | num_experiments = numel(data.best_candidates); 54 | x = zeros(num_experiments, 1); 55 | y = zeros(num_experiments, 1); 56 | for exp_idx = 1:num_experiments 57 | experiment = data.best_candidates(exp_idx); 58 | recall = sum(experiment.best_candidates.iou >= threshold) / numel(experiment.best_candidates.iou); 59 | x(exp_idx) = mean([experiment.image_statistics.num_candidates]); 60 | y(exp_idx) = recall; 61 | end 62 | line_style = '-'; 63 | if methods(i).is_baseline 64 | line_style = '--'; 65 | end 66 | if ~isempty(methods(i).line_style) 67 | line_style = methods(i).line_style; 68 | end 69 | semilogx(x, y, 'Color', methods(i).color, 'LineWidth', 1.5, 'LineStyle', line_style); 70 | hold on; grid on; 71 | end 72 | xlim([10, 10000]); 73 | ylim([0 1]); 74 | xlabel('# proposals'); ylabel(sprintf('recall at IoU threshold %.2f', threshold)); 75 | % legend(labels, 'Location', legend_locations{threshold_i}); 76 | % legendshrink(0.5); 77 | % legend boxoff; 78 | % legend(labels, 'Location', 'SouthEast'); 79 | hei = 10; 80 | wid = 10; 81 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 82 | set(gcf, 'PaperPositionMode','auto'); 83 | printpdf(sprintf('figures/%snum_candidates_recall_%.2f.pdf', output_file_prefix, threshold)); 84 | end 85 | end 86 | -------------------------------------------------------------------------------- /recall/plot_overlap_recall_curve.m: -------------------------------------------------------------------------------- 1 | function plot_overlap_recall_curve(iou_files, methods, num_candidates, fh, ... 2 | names_in_plot, legend_location, use_long_labels, custom_legend) 3 | 4 | if nargin < 7 5 | use_long_labels = false; 6 | end 7 | if nargin < 8 8 | custom_legend = []; 9 | end 10 | 11 | [~,method_order] = sort([methods.sort_key]); 12 | methods = methods(method_order); 13 | iou_files = iou_files(method_order); 14 | if ~isempty(custom_legend) 15 | custom_legend = custom_legend(method_order); 16 | end 17 | 18 | labels = {methods.short_name}; 19 | long_labels = {methods.name}; 20 | assert(numel(iou_files) == numel(labels)); 21 | n = numel(iou_files); 22 | 23 | num_pos = zeros(n, 1); 24 | 25 | figure(fh); 26 | for i = 1:n 27 | data = load(iou_files{i}); 28 | thresh_idx = find( ... 29 | [data.best_candidates.candidates_threshold] <= num_candidates, 1, 'last'); 30 | experiment = data.best_candidates(thresh_idx); 31 | [overlaps, recall, auc] = compute_average_recall(experiment.best_candidates.iou); 32 | 33 | display_auc = auc * 100; 34 | % round to first decimal 35 | display_auc = round(display_auc * 10) / 10; 36 | display_num_candidates = mean([experiment.image_statistics.num_candidates]); 37 | display_num_candidates = round(display_num_candidates * 10) / 10; 38 | number_str = sprintf('%g (%g)', display_auc, display_num_candidates); 39 | if names_in_plot 40 | labels{i} = sprintf('%s %s', labels{i}, number_str); 41 | long_labels{i} = sprintf('%s %s', long_labels{i}, number_str); 42 | else 43 | labels{i} = number_str; 44 | long_labels{i} = number_str; 45 | end 46 | num_pos(i) = numel(overlaps); 47 | line_style = '-'; 48 | if methods(i).is_baseline 49 | line_style = '--'; 50 | end 51 | if ~isempty(methods(i).line_style) 52 | line_style = methods(i).line_style; 53 | end 54 | plot(overlaps, recall, 'Color', methods(i).color, 'LineWidth', 1.5, 'LineStyle', line_style); 55 | hold on; 56 | end 57 | grid on; 58 | xlabel('IoU overlap threshold'); 59 | ylabel('recall'); 60 | xlim([0.5, 1]); 61 | ylim([0, 1]); 62 | if ~strcmp(legend_location, 'none') 63 | if ~isempty(custom_legend) 64 | lgnd = legend(custom_legend, 'Location', legend_location); 65 | elseif use_long_labels 66 | lgnd = legend(long_labels, 'Location', legend_location); 67 | else 68 | lgnd = legend(labels, 'Location', legend_location); 69 | end 70 | % set(lgnd, 'color','none'); 71 | % legendshrink(0.5); 72 | legend boxoff; 73 | end 74 | end 75 | -------------------------------------------------------------------------------- /recall/save_recall_voc07.m: -------------------------------------------------------------------------------- 1 | function save_recall_voc07() 2 | methods = get_method_configs(); 3 | iou_files = {methods.best_voc07_candidates_file}; 4 | n_methods = numel(iou_files); 5 | num_candidates = 1000; 6 | num_classes = 20; 7 | 8 | iou_thresholds = .5:.025:1; 9 | 10 | testset = load('data/pascal_voc07_test_annotations.mat'); 11 | 12 | recalls = zeros(n_methods, numel(iou_thresholds)); 13 | ARs = zeros(n_methods,1); 14 | ARs_per_class = zeros(n_methods, num_classes); 15 | 16 | for i = 1:n_methods 17 | fprintf('%s ', methods(i).name); 18 | tic; 19 | data = load(iou_files{i}); 20 | thresh_idx = find( ... 21 | [data.best_candidates.candidates_threshold] <= num_candidates, 1, 'last'); 22 | experiment = data.best_candidates(thresh_idx); 23 | [overlaps, recall, ARs(i)] = compute_average_recall(experiment.best_candidates.iou); 24 | 25 | for j = 1:numel(iou_thresholds) 26 | [~,min_idx] = min(abs(overlaps - iou_thresholds(j))); 27 | recalls(i,j) = recall(min_idx); 28 | end 29 | 30 | for c = 1:num_classes 31 | masks = get_class_masks(testset.per_class{c}.impos, testset.impos); 32 | iou = filter_best_ious(experiment.best_candidates.iou, masks); 33 | [~, ~, ARs_per_class(i,c)] = compute_average_recall(iou); 34 | end 35 | toc; 36 | end 37 | 38 | save('data/pascal_voc07_test_recall.mat', 'recalls', 'methods', ... 39 | 'iou_thresholds', 'ARs', 'ARs_per_class'); 40 | fprintf('done, all is well\n'); 41 | end 42 | 43 | 44 | function iou = filter_best_ious(iou, masks) 45 | flat_mask = cat(1, masks{:}); 46 | assert(numel(iou) == numel(flat_mask)); 47 | iou = iou(flat_mask); 48 | end 49 | 50 | 51 | function masks = get_class_masks(class_gt, gt) 52 | gt_ids = {gt.im}; 53 | n = numel(gt_ids); 54 | masks = cell(n,1); 55 | 56 | for i = 1:n 57 | masks{i} = false(size(gt(i).boxes, 1), 1); 58 | end 59 | 60 | for i = 1:numel(class_gt), im_id = class_gt(i).im; 61 | gt_idx = find(strcmp(gt_ids, im_id)); 62 | assert(numel(gt_idx)==1); 63 | gt_boxes = gt(gt_idx).boxes; 64 | 65 | boxes = class_gt(i).boxes; 66 | % sanity check 67 | found = ismember(boxes, gt_boxes, 'rows'); 68 | assert(all(found)); 69 | 70 | masks{gt_idx} = ismember(gt_boxes, boxes, 'rows'); 71 | end 72 | end -------------------------------------------------------------------------------- /repeatability/project_candidates.m: -------------------------------------------------------------------------------- 1 | function [candidates, exact] = project_candidates(candidates, H) 2 | n = size(candidates, 1); 3 | 4 | % project all 4 coordinates of the bbox 5 | % 1 ---- 2 6 | % | | 7 | % 4 ---- 3 8 | points = [cat(1, ... 9 | candidates(:,1:2), candidates(:,[3,2]), ... 10 | candidates(:,3:4), candidates(:,[1,4])), ... 11 | ones(4*n, 1)]; 12 | projected_points = points * H'; 13 | projected_points = projected_points ./ repmat(projected_points(:,3), [1 3]); 14 | % p1 = projected_points(1:n, 1:2); 15 | % p2 = projected_points((n+1):(2*n), 1:2); 16 | % p3 = projected_points((2*n+1):(3*n), 1:2); 17 | % p4 = projected_points((3*n+1):end, 1:2); 18 | xs = reshape(projected_points(:,1), [n 4]); 19 | ys = reshape(projected_points(:,2), [n 4]); 20 | exact = [xs(:,1), ys(:,1), xs(:,2), ys(:,2), xs(:,3), ys(:,3), xs(:,4), ys(:,4)]; 21 | 22 | xs = sort(xs, 2); 23 | ys = sort(ys, 2); 24 | candidates(:,1) = mean(xs(:,1:2), 2); 25 | candidates(:,3) = mean(xs(:,3:4), 2); 26 | candidates(:,2) = mean(ys(:,1:2), 2); 27 | candidates(:,4) = mean(ys(:,3:4), 2); 28 | end -------------------------------------------------------------------------------- /repeatability/repeatability_blur.m: -------------------------------------------------------------------------------- 1 | function [im2, H] = repeatability_blur(im, sigma) 2 | hsize = ceil(20 * sigma); 3 | if sigma == 0 || hsize < 3 4 | im2 = im; 5 | else 6 | filter = fspecial('gaussian', hsize, sigma); 7 | im2 = imfilter(im, filter, 'symmetric', 'same'); 8 | end 9 | H = eye(3); 10 | end 11 | -------------------------------------------------------------------------------- /repeatability/repeatability_generate_images.m: -------------------------------------------------------------------------------- 1 | function repeatability_generate_images() 2 | load('data/pascal_voc07_test_annotations.mat', 'impos'); 3 | config = get_config(); 4 | rep_config = repeatability_get_config(); 5 | 6 | homographies = []; 7 | fields = fieldnames(rep_config); 8 | for field_i = 1:numel(fields) 9 | homographies.(fields{field_i}) = zeros(numel(impos), 3, 3); 10 | end 11 | 12 | for im_i=1:numel(impos) 13 | tic_toc_print('image %d/%d\n', im_i, numel(impos)); 14 | orig_img_id = impos(im_i).im; 15 | im = imread(sprintf(config.pascal_images, orig_img_id)); 16 | 17 | fields = fieldnames(rep_config); 18 | for field_i = 1:numel(fields) 19 | %tic_toc_print('running %s\n', fields{field_i}); 20 | 21 | sub_config = rep_config.(fields{field_i}); 22 | for i = 1:numel(sub_config.params), param = sub_config.params(i); 23 | % skip parameters that are not plotted 24 | if isfield(sub_config, 'display_points') 25 | if ~ismember(i, sub_config.display_points) 26 | continue; 27 | end 28 | end 29 | [transformed_im, H] = sub_config.func(im, param); 30 | homographies.(fields{field_i})(im_i,:,:) = H; 31 | 32 | img_id = sprintf(sub_config.img_id, orig_img_id, i); 33 | imwrite(transformed_im, sprintf(config.transformed_pascal_images, img_id)); 34 | end 35 | end 36 | end 37 | 38 | save('data/pascal_voc07_test_repeatability_homographies.mat', 'homographies'); 39 | end 40 | -------------------------------------------------------------------------------- /repeatability/repeatability_get_config.m: -------------------------------------------------------------------------------- 1 | function [conf] = repeatability_get_config() 2 | conf.scale.params = [2 .^ (-1:0.2:1), 0.9, 0.95, 0.99, 1.01, 1.05, 1.1]; 3 | conf.scale.func = @repeatability_scale; 4 | conf.scale.img_id = '%s_scale%d'; 5 | conf.scale.plot_x_transform = @log2; 6 | conf.scale.xlabel = 'log_2(scale)'; 7 | conf.scale.id_value = 0; 8 | conf.scale.legend_location = 'SouthWest'; 9 | conf.scale.name_version = 'short_name'; 10 | conf.scale.big_dots_for_id = true; 11 | 12 | degrees = [-20:5:20, -1, -0.5, 0.5, 1, 0]; 13 | empties = cell(numel(degrees), 1); 14 | conf.rotate.params = struct('angle', empties, 'max_angles', empties); 15 | for i = 1:numel(degrees) 16 | conf.rotate.params(i).angle = degrees(i); 17 | conf.rotate.params(i).max_angles = [min(degrees) max(degrees)]; 18 | end 19 | conf.rotate.display_param = 'angle'; 20 | conf.rotate.func = @repeatability_rotate; 21 | conf.rotate.img_id = '%s_rotate%d'; 22 | conf.rotate.ref_params_idx = numel(degrees); 23 | conf.rotate.ref_params = conf.rotate.params(conf.rotate.ref_params_idx); 24 | conf.rotate.display_points = 1:(numel(degrees)-1); 25 | conf.rotate.xlabel = 'rotation in degree'; 26 | conf.rotate.id_value = 0; 27 | conf.rotate.legend_location = 'SouthWest'; 28 | conf.rotate.name_version = 'short_name'; 29 | conf.rotate.big_dots_for_id = true; 30 | 31 | conf.light.params = [50:20:160, 100, 99, 101]; 32 | conf.light.func = @repeatability_light; 33 | conf.light.img_id = '%s_light%d'; 34 | conf.light.xlabel = 'lighting in %'; 35 | conf.light.id_value = 100; 36 | conf.light.legend_location = 'SouthWest'; 37 | conf.light.name_version = 'short_name'; 38 | 39 | conf.jpeg.params = [5 10 20 50 80 100 99 120]; 40 | conf.jpeg.func = @repeatability_jpeg; 41 | conf.jpeg.img_id = '%s_jpeg%d'; 42 | conf.jpeg.xlabel = 'quality of compression in %'; 43 | conf.jpeg.id_value = 120; 44 | conf.jpeg.xticks = [5 20 40 60 80 100 120]; 45 | conf.jpeg.xticklabels = {'5', '20', '40', '60', '80', '100', 'lossless'}; 46 | conf.jpeg.legend_location = 'SouthEast'; 47 | conf.jpeg.name_version = 'short_name'; 48 | conf.jpeg.xreverse = true; 49 | 50 | conf.blur.params = [0.3 0.6 1 2 3 0 0.2 4 8]; 51 | conf.blur.display_points = [3 4 5 6 8 9]; 52 | conf.blur.func = @repeatability_blur; 53 | conf.blur.img_id = '%s_blur%d'; 54 | conf.blur.xlabel = 'sigma in pixels'; 55 | conf.blur.id_value = 0; 56 | conf.blur.legend_location = 'SouthWest'; 57 | conf.blur.name_version = 'short_name'; 58 | 59 | function y = mylog(x) 60 | oor = x <= 0; 61 | y = x; 62 | y(oor) = -1; 63 | y(~oor) = log10(x(~oor)); 64 | end 65 | conf.saltnpepper.params = [0 1 10 100 1000]; 66 | conf.saltnpepper.func = @repeatability_saltnpepper; 67 | conf.saltnpepper.img_id = '%s_snp%d'; 68 | conf.saltnpepper.id_value = -1; 69 | conf.saltnpepper.legend_location = 'SouthWest'; 70 | conf.saltnpepper.name_version = 'short_name'; 71 | conf.saltnpepper.plot_x_transform = @mylog; 72 | conf.saltnpepper.xlabel = 'log_{10}(number of pixels)'; 73 | conf.saltnpepper.xticks = [-1 0 1 2 3]; 74 | conf.saltnpepper.xticklabels = {'none', '0', '1', '2', '3'}; 75 | end 76 | -------------------------------------------------------------------------------- /repeatability/repeatability_jpeg.m: -------------------------------------------------------------------------------- 1 | function [im2,H] = repeatability_jpeg(im, compression) 2 | if compression > 100 3 | tmp_filename = [tempname() '.jpg']; 4 | imwrite(im, tmp_filename, 'jpeg', 'Mode', 'lossless'); 5 | im2 = imread(tmp_filename); 6 | if size(im2,3) == 1 7 | im2 = repmat(im2, [1 1 3]); 8 | end 9 | assert(all(size(im) == size(im2))); 10 | delete(tmp_filename); 11 | else 12 | tmp_filename = [tempname() '.jpg']; 13 | imwrite(im, tmp_filename, 'jpeg', 'Mode', 'lossy', 'Quality', compression); 14 | im2 = imread(tmp_filename); 15 | if size(im2,3) == 1 16 | im2 = repmat(im2, [1 1 3]); 17 | end 18 | assert(all(size(im) == size(im2))); 19 | delete(tmp_filename); 20 | end 21 | 22 | H = eye(3); 23 | end 24 | -------------------------------------------------------------------------------- /repeatability/repeatability_light.m: -------------------------------------------------------------------------------- 1 | function [im2, H] = repeatability_light(im, light_percent) 2 | 3 | tmp_filename = tempname(); 4 | tmp_in_filename = [tmp_filename '.in.png']; 5 | tmp_out_filename = [tmp_filename '.out.png']; 6 | imwrite(im, tmp_in_filename, 'png'); 7 | system(sprintf('convert "%s" -set option:modulate:colorspace hsb -modulate %f "%s"', ... 8 | tmp_in_filename, light_percent, tmp_out_filename)); 9 | im2 = imread(tmp_out_filename); 10 | if size(im2,3) == 1 11 | im2 = repmat(im2, [1 1 3]); 12 | end 13 | assert(all(size(im) == size(im2))); 14 | delete(tmp_in_filename); 15 | delete(tmp_out_filename); 16 | 17 | H = eye(3); 18 | end 19 | -------------------------------------------------------------------------------- /repeatability/repeatability_matching.m: -------------------------------------------------------------------------------- 1 | function repeatability_matching(index, method_idx) 2 | % Computes the matching between proposals on the repeatability set. 3 | 4 | num_candidates = 1000; 5 | 6 | load('voc07/cache/all_with_imsize_test.mat'); 7 | images = {impos.im}; 8 | if nargin >= 1 9 | fprintf('running on image %s/%d\n', mat2str(index), numel(images)); 10 | images = images(index); 11 | end 12 | 13 | config = repeatability_get_config(); 14 | methods = get_method_configs(); 15 | methods = methods(method_idx); 16 | 17 | run_matching(images, config, methods, num_candidates); 18 | end 19 | 20 | function [matchings] = run_matching(images, config, methods, num_candidates) 21 | fields = fieldnames(config); 22 | num_methods = numel(methods); 23 | matchings = cell(num_methods, 1); 24 | % preallocate 25 | for method_i = 1:num_methods 26 | for field_i = 1:numel(fields) 27 | matchings{method_i}.matchings(numel(images)).(fields{field_i}) = []; 28 | end 29 | end 30 | 31 | for im_i = 1:numel(images) 32 | tic_toc_print('run matching image %d/%d\n', im_i, numel(images)); 33 | im = imread(images{im_i}); 34 | if size(im, 3) == 1 35 | im = repmat(im, [1 1 3]); 36 | end 37 | [~,orig_img_id,~] = fileparts(images{im_i}); 38 | subdir = orig_img_id(1:5); 39 | for method_i = 1:num_methods 40 | fprintf('method: %s\n', methods(method_i).name); 41 | 42 | for field_i = 1:numel(fields) 43 | fprintf('task: %s\n', fields{field_i}); 44 | 45 | if numel(matchings{method_i}.matchings(im_i).(fields{field_i})) ~= numel(config.(fields{field_i}).params) 46 | t_matchings = ... 47 | matching_subtask(im, orig_img_id, config.(fields{field_i}), ... 48 | methods(method_i), num_candidates); 49 | matchings{method_i}.matchings(im_i).(fields{field_i}) = t_matchings(1,:); 50 | end 51 | end 52 | 53 | matfile = fullfile(methods(method_i).repeatability_candidate_dir, ... 54 | subdir, sprintf('%s_matchings.mat', orig_img_id)); 55 | fprintf('writing %s\n', matfile); 56 | matching = matchings{method_i}.matchings(im_i); 57 | save(matfile, 'matching'); 58 | end 59 | 60 | end 61 | % for im_i = 1:numel(images) 62 | % tic_toc_print('run matching image %d/%d\n', im_i, numel(images)); 63 | % im = imread(images{im_i}); 64 | % if size(im, 3) == 1 65 | % im = repmat(im, [1 1 3]); 66 | % end 67 | % [~,orig_img_id,~] = fileparts(images{im_i}); 68 | % fields = fieldnames(config); 69 | % for field_i = 1:numel(fields) 70 | % t_matchings = ... 71 | % matching_subtask(im, orig_img_id, config.(fields{field_i}), methods, num_candidates); 72 | % for method_i = 1:num_methods 73 | % matchings{method_i}.matchings(im_i).(fields{field_i}) = t_matchings(method_i,:); 74 | % end 75 | % end 76 | % end 77 | end 78 | 79 | function [matchings] = matching_subtask(orig_im, orig_img_id, sub_config, methods, num_candidates) 80 | if ~isfield(sub_config, 'ref_params_idx') || isempty(sub_config.ref_params_idx) 81 | ref_img_id = [orig_img_id '_reference']; 82 | ref_im = orig_im; 83 | else 84 | ref_img_id = sprintf(sub_config.img_id, orig_img_id, sub_config.ref_params_idx); 85 | ref_im = sub_config.func(orig_im, sub_config.ref_params); 86 | end 87 | 88 | num_parameters = numel(sub_config.params); 89 | num_methods = numel(methods); 90 | 91 | matchings = []; 92 | matchings(num_methods, num_parameters).iou = []; 93 | matchings(num_methods, num_parameters).reference_boxes = []; 94 | matchings(num_methods, num_parameters).matched_boxes = []; 95 | matchings(num_methods, num_parameters).im_size = []; 96 | matchings(num_methods, num_parameters).ref_im_size = []; 97 | 98 | ref_im_size = [size(ref_im, 2), size(ref_im, 1)]; 99 | for method_i = 1:num_methods 100 | method = methods(method_i); 101 | 102 | for param_i = 1:num_parameters 103 | param = sub_config.params(param_i); 104 | img_id = sprintf(sub_config.img_id, orig_img_id, param_i); 105 | [im,H] = sub_config.func(orig_im, param); 106 | Hinv = inv(H); 107 | im_size = [size(im, 2), size(im, 1)]; 108 | 109 | [candidates] = get_candidates(method, img_id, num_candidates, true, 5, method.repeatability_candidate_dir); 110 | % [candidates] = read_candidates_mat(method.repeatability_candidate_dir, img_id, 5); 111 | % assert(size(candidates, 1) <= num_candidates); 112 | ref_candidates = get_candidates(method, ref_img_id, num_candidates, true, 5, method.repeatability_candidate_dir); 113 | % [ref_candidates] = read_candidates_mat(method.repeatability_candidate_dir, ref_img_id, 5); 114 | % assert(size(ref_candidates, 1) <= num_candidates); 115 | 116 | if isempty(candidates) || isempty(ref_candidates) 117 | best_iou_per_cand = []; 118 | matched_candidates = []; 119 | else 120 | proj_candidates = project_candidates(candidates, Hinv); 121 | proj_ref_candidates = project_candidates(ref_candidates, H); 122 | 123 | proj_candidates_centers = [mean(proj_candidates(:,[1 3]), 2), mean(proj_candidates(:,[2 4]), 2)]; 124 | orig_valid = (proj_candidates_centers(:,1) >= 1) & (proj_candidates_centers(:,2) >= 1) ... 125 | & (proj_candidates_centers(:,1) <= ref_im_size(1)) & (proj_candidates_centers(:,2) <= ref_im_size(2)); 126 | proj_ref_candidates_centers = [mean(proj_ref_candidates(:,[1 3]), 2), mean(proj_ref_candidates(:,[2 4]), 2)]; 127 | ref_valid = (proj_ref_candidates_centers(:,1) >= 1) & (proj_ref_candidates_centers(:,2) >= 1) ... 128 | & (proj_ref_candidates_centers(:,1) <= im_size(1)) & (proj_ref_candidates_centers(:,2) <= im_size(2)); 129 | 130 | fprintf('filtered out (transformed/reference): %d/%d, left %d/%d\n', ... 131 | sum(~[orig_valid]), sum(~[ref_valid]), sum(orig_valid), sum(ref_valid)); 132 | proj_candidates = proj_candidates(orig_valid,:); 133 | ref_candidates = ref_candidates(ref_valid,:); 134 | 135 | [best_iou_per_cand, matched_candidates] = closest_candidates(ref_candidates, proj_candidates); 136 | end 137 | 138 | matchings(method_i, param_i).iou = best_iou_per_cand; 139 | matchings(method_i, param_i).reference_boxes = ref_candidates; 140 | matchings(method_i, param_i).matched_boxes = matched_candidates; 141 | matchings(method_i, param_i).ref_im_size = ref_im_size; 142 | matchings(method_i, param_i).im_size = im_size; 143 | end 144 | end 145 | end 146 | 147 | -------------------------------------------------------------------------------- /repeatability/repeatability_plot.m: -------------------------------------------------------------------------------- 1 | function repeatability_plot(method_selection, only_caching, per_method_plot) 2 | % Given the proposals on the repeatability dataset have already been 3 | % extracted and the matching is finished, this function does the evaluation 4 | % of the results and plots them. 5 | % 6 | % The evaluation needs a lot of memory, so watch out. Once the evaluation 7 | % is done, the curve is subsampled and saved to disk, so any further 8 | % plotting is fast and doesn't need so much memory anymore. 9 | % 10 | % method_selection: the methods that you want to evalute and plot 11 | % 12 | % only_caching: only run the evaluation and save results to disk, don't 13 | % do the plotting yet 14 | % 15 | % per_method_plot: plots more detailed plots 16 | 17 | if nargin < 2 18 | only_caching = false; 19 | end 20 | if nargin < 3 21 | per_method_plot = false; 22 | end 23 | weighted = false; 24 | methods = get_method_configs(); 25 | if nargin > 0 26 | methods = methods(method_selection); 27 | else 28 | methods([7 9 14 16 19:25]) = []; 29 | end 30 | 31 | load('data/pascal_voc07_test_annotations.mat'); 32 | images = {impos.im}; 33 | 34 | box_size_bins = 0:0.1:1; 35 | box_size_bin_centers = (box_size_bins(1:(end-1)) + box_size_bins(2:end)) / 2; 36 | box_size_bins(1) = -inf; 37 | box_size_bins(end) = inf; 38 | num_bins = numel(box_size_bins) - 1; 39 | exp_config = repeatability_get_config(); 40 | 41 | files_missing = false(size(methods)); 42 | method_repeatability = cell(numel(methods), 1); 43 | for method_i = 1:numel(methods) 44 | method = methods(method_i); 45 | fprintf('method: %s\n', method.name); 46 | try 47 | load(method.repeatability_matching_file); 48 | catch 49 | empties = cell(numel(images), 1); 50 | matching = struct('scale', empties, 'rotate', empties, 'light', empties, 'jpeg', empties, 'blur', empties, 'saltnpepper', empties); 51 | for im_i = 1:numel(images) 52 | [~,img_id,~] = fileparts(images{im_i}); 53 | subdir = img_id(1:5); 54 | matfile = fullfile(methods(method_i).repeatability_candidate_dir, ... 55 | subdir, sprintf('%s_matchings.mat', img_id)); 56 | % fixfile(matfile); 57 | try 58 | data = load(matfile); 59 | catch 60 | fprintf('couldn''t find file %s\n', matfile); 61 | files_missing(method_i) = true; 62 | break; 63 | end 64 | matching(im_i) = data.matching; 65 | clear data; 66 | end 67 | if files_missing(method_i) 68 | continue; 69 | end 70 | disp('computing statistics'); 71 | bins = get_bin_statistics(matching, box_size_bins); 72 | transformations = fieldnames(matching); 73 | num_transformations = numel(transformations); 74 | clear matching; 75 | 76 | disp('computing recall curves'); 77 | for trafo_i = 1:num_transformations 78 | num_params = size(bins{trafo_i}, 2); 79 | for param_i = 1:num_params 80 | for bin_i = 1:num_bins 81 | if numel(bins{trafo_i}(bin_i,param_i).values) > 500 82 | bins{trafo_i}(bin_i,param_i).values = sort(bins{trafo_i}(bin_i,param_i).values(:)', 'ascend'); 83 | n = numel(bins{trafo_i}(bin_i,param_i).values); 84 | step_size = n/500; 85 | bins{trafo_i}(bin_i,param_i).values = bins{trafo_i}(bin_i,param_i).values(round(1:step_size:n)); 86 | end 87 | [bins{trafo_i}(bin_i,param_i).overlap, bins{trafo_i}(bin_i,param_i).recall, ... 88 | bins{trafo_i}(bin_i,param_i).area_under_recall] = compute_average_recall_entire_curve(... 89 | bins{trafo_i}(bin_i,param_i).values); 90 | % if trafo_i == 3 && param_i == 7 91 | % figure(12); hold on; 92 | % dash = '-'; 93 | % plot(bins{trafo_i}(bin_i,param_i).overlap, bins{trafo_i}(bin_i,param_i).recall, dash, 'Color', cmap(bin_i,:), 'LineWidth', 2); 94 | % % title(mat2str(bins{trafo_i}(bin_i,param_i).area_under_recall)); 95 | % end 96 | end 97 | end 98 | end 99 | 100 | save(method.repeatability_matching_file, 'bins', '-v7.3'); 101 | end 102 | if only_caching 103 | continue; 104 | end 105 | 106 | transformations = fieldnames(exp_config); 107 | num_transformations = numel(transformations); 108 | 109 | % remove the stuff that won't be displayed 110 | for trafo_i = 1:num_transformations 111 | if isfield(exp_config.(transformations{trafo_i}), 'display_points') 112 | param_idxs = exp_config.(transformations{trafo_i}).display_points; 113 | bins{trafo_i} = bins{trafo_i}(:,param_idxs); 114 | end 115 | end 116 | 117 | % method_bins{method_i} = bins; 118 | 119 | for trafo_i = 1:num_transformations 120 | num_params = size(bins{trafo_i}, 2); 121 | trafo = transformations{trafo_i}; 122 | trafo_aur_bars = zeros(num_params, numel(box_size_bin_centers)); 123 | trafo_num_boxes_bars = zeros(size(trafo_aur_bars)); 124 | labels = cell(num_params, 1); 125 | param_vals = zeros(num_params, 1); 126 | for param_i = 1:num_params 127 | if isfield(exp_config.(transformations{trafo_i}), 'display_points') 128 | param_idxs = exp_config.(transformations{trafo_i}).display_points; 129 | else 130 | param_idxs = 1:numel(exp_config.(trafo).params); 131 | end 132 | if isstruct(exp_config.(trafo).params(param_idxs(param_i))) 133 | param_val = exp_config.(trafo).params(param_idxs(param_i)).(exp_config.(trafo).display_param); 134 | else 135 | param_val = exp_config.(trafo).params(param_idxs(param_i)); 136 | end 137 | labels{param_i} = sprintf('%f', param_val); 138 | trafo_num_boxes_bars(param_i,:) = cellfun(@numel, {bins{trafo_i}(:,param_i).values}); 139 | 140 | trafo_aur_bars(param_i,:) = [bins{trafo_i}(:,param_i).area_under_recall]; 141 | param_vals(param_i) = param_val; 142 | end 143 | 144 | if per_method_plot 145 | [~,order] = sort(param_vals); 146 | figure; hold on; 147 | bar(box_size_bin_centers, trafo_num_boxes_bars(order,:)'); 148 | title(sprintf('%s: number of boxes, %s', method.name, trafo)); 149 | legend(labels(order)); 150 | 151 | figure; hold on; 152 | bar(box_size_bin_centers, trafo_aur_bars(order,:)'); 153 | title(sprintf('%s: area under recall curve, %s', method.name, trafo)); 154 | legend(labels(order), 'Location', 'SouthEast'); 155 | end 156 | 157 | if weighted 158 | weights = trafo_num_boxes_bars ./ repmat(sum(trafo_num_boxes_bars, 2), [1, numel(box_size_bin_centers)]); 159 | method_repeatability{method_i}.trafo(trafo_i).area_under_recall = sum(trafo_aur_bars .* weights, 2); 160 | else 161 | method_repeatability{method_i}.trafo(trafo_i).area_under_recall = mean(trafo_aur_bars, 2); 162 | end 163 | method_repeatability{method_i}.trafo(trafo_i).labels = labels; 164 | method_repeatability{method_i}.trafo(trafo_i).param_vals = param_vals; 165 | method_repeatability{method_i}.trafo(trafo_i).name = trafo; 166 | end 167 | end 168 | if only_caching 169 | return; 170 | end 171 | 172 | methods(files_missing) = [];; 173 | method_repeatability(files_missing) = []; 174 | 175 | [~,method_order] = sort([methods.sort_key]); 176 | methods = methods(method_order); 177 | method_repeatability = method_repeatability(method_order); 178 | 179 | plot_legend(methods); 180 | printpdf('figures/repeatability_legend.pdf'); 181 | 182 | transformations = fieldnames(exp_config); 183 | num_transformations = numel(transformations); 184 | symbols = {'.', 'o', 's', '*', 'd'}; 185 | if ~per_method_plot 186 | num_trafo = numel(method_repeatability{1}.trafo); 187 | for trafo_i = 1:num_trafo 188 | trafo_config = exp_config.(transformations{trafo_i}); 189 | num_params = numel(method_repeatability{1}.trafo(trafo_i).param_vals); 190 | areas = zeros(numel(methods), num_params); 191 | for method_i = 1:numel(methods) 192 | areas(method_i, :) = method_repeatability{method_i}.trafo(trafo_i).area_under_recall; 193 | assert(all(method_repeatability{method_i}.trafo(trafo_i).param_vals == method_repeatability{1}.trafo(trafo_i).param_vals)); 194 | end 195 | figure; hold on; grid on; 196 | x = method_repeatability{1}.trafo(trafo_i).param_vals; 197 | if isfield(trafo_config, 'plot_x_transform') 198 | x = trafo_config.plot_x_transform(x); 199 | end 200 | [x, order] = sort(x); 201 | areas = areas(:,order); 202 | assert(all(all(areas <= 1))); 203 | 204 | plot([trafo_config.id_value trafo_config.id_value], ylim, 'LineStyle','--', 'Color', 'black'); 205 | 206 | % make some of the markers fatter if they overlap 207 | if isfield(trafo_config, 'big_dots_for_id') && trafo_config.big_dots_for_id 208 | marker_sizes = ones(numel(methods), 1) * 80; 209 | x_pos = trafo_config.id_value; 210 | for method_i = 1:numel(methods) 211 | y_pos = areas(method_i,(x == x_pos)); 212 | d = abs(y_pos - areas(1:method_i-1,(x == x_pos))); 213 | min_size = min(marker_sizes(find(d < 0.05))-8); 214 | if isempty(min_size) 215 | min_size = marker_sizes(method_i); 216 | end 217 | marker_sizes(method_i) = min_size; 218 | end 219 | for method_i = 1:numel(methods) 220 | % symbols{mod(method_i-1, numel(symbols))+1}, ... 221 | plot(x_pos, areas(method_i,(x == x_pos)), '.', ... 222 | 'LineWidth', 1.5, ... 223 | 'MarkerSize', marker_sizes(method_i), ... 224 | 'Color', methods(method_i).color); 225 | end 226 | end 227 | 228 | plot_handles = zeros(numel(methods),1); 229 | for method_i = 1:numel(methods) 230 | line_style = '-'; 231 | if methods(method_i).is_baseline 232 | line_style = '--'; 233 | end 234 | plot_handles(method_i) = plot(x, areas(method_i,:)', '.', ... 235 | 'LineWidth', 1.5, 'MarkerSize', 10, ... 236 | 'Color', methods(method_i).color, 'LineStyle', line_style); 237 | end 238 | % if isfield(trafo_config, 'legend_location') 239 | % legend_labels = cell(numel(methods), 1); 240 | % for method_i = 1:numel(methods) 241 | % legend_labels{method_i} = methods(method_i).(trafo_config.name_version); 242 | % end 243 | % [lgnd,hObj] = legend(plot_handles, legend_labels, ... 244 | % 'Location', trafo_config.legend_location); 245 | % % legendshrink(0.5); 246 | % legend boxoff; 247 | % if isfield(trafo_config, 'legend_offset') 248 | % lpos = get(lgnd, 'pos'); 249 | % lpos = lpos + trafo_config.legend_offset; 250 | % set(lgnd, 'pos', lpos); 251 | % end 252 | % end 253 | % legend({methods.name}); 254 | % title(method_repeatability{1}.trafo(trafo_i).name); 255 | ylabel('repeatability'); 256 | xlabel(trafo_config.xlabel); 257 | 258 | left = min(x); 259 | right = max(x); 260 | if left == trafo_config.id_value 261 | left = left - 0.2; 262 | end 263 | if right == trafo_config.id_value 264 | right = right * 1.08; 265 | end 266 | xlim([left, right]); 267 | ylim([0.1 1.01]); 268 | 269 | if isfield(trafo_config, 'xticks') 270 | xticks = trafo_config.xticks; 271 | xticklabels = trafo_config.xticklabels; 272 | set(gca, 'XTick', xticks); 273 | set(gca, 'XTickLabel', xticklabels); 274 | end 275 | if isfield(trafo_config, 'xreverse') && trafo_config.xreverse 276 | set(gca,'XDir','reverse'); 277 | end 278 | 279 | hei = 10; 280 | wid = 10; 281 | set(gcf, 'Units','centimeters', 'Position',[0 0 wid hei]); 282 | set(gcf, 'PaperPositionMode','auto'); 283 | printpdf(sprintf('figures/repeatability_%s.pdf', transformations{trafo_i})); 284 | end 285 | end 286 | 287 | % save('repeatability_method_bins.mat', 'method_bins'); 288 | end 289 | 290 | 291 | 292 | 293 | function [bins] = get_bin_statistics(matchings, box_size_bins) 294 | num_imgs = numel(matchings); 295 | num_bins = numel(box_size_bins) - 1; 296 | % bins = []; 297 | % bins(num_bins,5).values = []; 298 | % bins(num_bins).count = 0; 299 | % matchings = matchings_info.matchings; 300 | 301 | transformations = fieldnames(matchings); 302 | num_transformations = numel(transformations); 303 | bins = cell(num_transformations,1); 304 | for trafo_i = 1:num_transformations 305 | transformation_field = matchings(1).(transformations{trafo_i}); 306 | num_params = numel(transformation_field); 307 | bins{trafo_i}(num_bins,num_params).values = []; 308 | for param_i = 1:num_params 309 | for bin_i = 1:num_bins 310 | bins{trafo_i}(bin_i,param_i).values_per_im = cell(num_imgs,1); 311 | end 312 | end 313 | end 314 | 315 | for im_i = 1:num_imgs 316 | for trafo_i = 1:num_transformations 317 | transformation_field = matchings(im_i).(transformations{trafo_i}); 318 | num_params = numel(transformation_field); 319 | for param_i = 1:num_params 320 | ious = transformation_field(param_i).iou; 321 | if isempty(ious) 322 | continue; 323 | end 324 | boxes = transformation_field(param_i).reference_boxes; 325 | if isempty(boxes) 326 | continue 327 | end 328 | im_size = transformation_field(param_i).ref_im_size; 329 | w = boxes(:,3) - boxes(:,1) + 1; 330 | h = boxes(:,4) - boxes(:,2) + 1; 331 | areas = sqrt(w .* h ./ prod(im_size)); 332 | [~,arg_hist] = histc(areas, box_size_bins); 333 | assert(sum(arg_hist == 0) == 0); 334 | assert(min(arg_hist) >= 1); 335 | assert(max(arg_hist) <= numel(box_size_bins) - 1); 336 | for bin_i = 1:num_bins 337 | bins{trafo_i}(bin_i,param_i).values_per_im{im_i} = ious(arg_hist == bin_i); 338 | % bins{trafo_i}(bin_i,param_i).values = ... 339 | % [bins{trafo_i}(bin_i,param_i).values; ious(arg_hist == bin_i)]; 340 | % bins(bin_i).count = bins(bin_i).count + sum(arg_hist == bin_i); 341 | end 342 | end 343 | end 344 | end 345 | 346 | for trafo_i = 1:num_transformations 347 | transformation_field = matchings(im_i).(transformations{trafo_i}); 348 | num_params = numel(transformation_field); 349 | for param_i = 1:num_params 350 | for bin_i = 1:num_bins 351 | bins{trafo_i}(bin_i,param_i).values = cat(1, bins{trafo_i}(bin_i,param_i).values_per_im{:}); 352 | bins{trafo_i}(bin_i,param_i).values_per_im = []; 353 | end 354 | end 355 | end 356 | 357 | end 358 | 359 | 360 | function [overlap, recall, area] = compute_average_recall_entire_curve(unsorted_overlaps) 361 | overlap = sort(unsorted_overlaps(:)', 'ascend'); 362 | num_pos = numel(overlap); 363 | if max(overlap) < 1 364 | overlap = [0, overlap, max(overlap)+0.001]; 365 | recall = [1, (num_pos:-1:1)/num_pos, 0]; 366 | else 367 | overlap = [0, overlap]; 368 | recall = [1, (num_pos:-1:1)/num_pos]; 369 | end 370 | 371 | dx = overlap(2:end) - overlap(1:end-1); 372 | y = (recall(1:end-1) + recall(2:end)) / 2; 373 | area = sum(dx .* y); 374 | end 375 | -------------------------------------------------------------------------------- /repeatability/repeatability_rotate.m: -------------------------------------------------------------------------------- 1 | function [im, H] = repeatability_rotate(im_ref, params) 2 | angle_degree = params.angle; 3 | 4 | im_size = []; 5 | for max_angle_degree = params.max_angles 6 | [im_max_rot, ~] = rotate(im_ref, max_angle_degree); 7 | if isempty(im_size) 8 | im_size = [size(im_max_rot, 2), size(im_max_rot, 1)]; 9 | else 10 | im_size = min([im_size; size(im_max_rot, 2), size(im_max_rot, 1)], [], 1); 11 | end 12 | end 13 | 14 | [im, H] = rotate(im_ref, angle_degree); 15 | 16 | % because reference and rotated image will be cropped to im_size, 17 | % we need to offset that additional cropping in H 18 | 19 | ref_size_diff = [size(im_ref, 2), size(im_ref, 1)] - im_size; 20 | assert(all(ref_size_diff >= 0)); 21 | Htrans_ref = eye(3); 22 | Htrans_ref(1:2,3) = floor(ref_size_diff/2)'; 23 | 24 | rot_im_size = [size(im, 2), size(im, 1)]; 25 | rot_size_diff = rot_im_size - im_size; 26 | rot_offset = floor(rot_size_diff / 2); 27 | assert(all(rot_size_diff >= 0)); 28 | Htrans_rot = eye(3); 29 | Htrans_rot(1:2,3) = -rot_offset'; 30 | x_y_min = rot_offset + 1; 31 | x_y_max = rot_im_size - ceil(rot_size_diff / 2); 32 | im = im(x_y_min(2):x_y_max(2), x_y_min(1):x_y_max(1), :); 33 | 34 | % H * x_reference = x_rotated 35 | H = Htrans_rot * H * Htrans_ref; 36 | 37 | end 38 | 39 | 40 | function [im, H] = rotate(im_ref, angle_degree) 41 | angle_rad = angle_degree / 180 * pi; 42 | im_rot = imrotate(im_ref, angle_degree, 'bilinear', 'crop'); 43 | 44 | center = [size(im_ref, 2), size(im_ref, 1)] / 2; 45 | 46 | Htrans1 = eye(3); 47 | Htrans1(1:2,3) = (center' + 1); 48 | Htrans2 = eye(3); 49 | Htrans2(1:2,3) = -(center' + 1); 50 | 51 | Hrot = eye(3); 52 | Hrot(1,1) = cos(-angle_rad); 53 | Hrot(1,2) = -sin(-angle_rad); 54 | Hrot(2,1) = sin(-angle_rad); 55 | Hrot(2,2) = cos(-angle_rad); 56 | 57 | % H * x_reference = x_rotated 58 | H = Htrans1 * Hrot * Htrans2; 59 | 60 | % crop so there is no 'made up' image content 61 | % upper left, upper right, lower left, lower right 62 | x_ref_max = size(im_ref, 2); 63 | y_ref_max = size(im_ref, 1); 64 | corners_ref = [1, x_ref_max, x_ref_max, 1; ... 65 | 1, 1, y_ref_max, y_ref_max; ... 66 | 1, 1, 1, 1]; 67 | proj_corners_ref = H * corners_ref; 68 | proj_corners_ref = proj_corners_ref(1:2,:) ./ repmat(proj_corners_ref(3,:), [2 1]); 69 | % x_min = ceil(max(proj_corners_ref(1,1), proj_corners_ref(1,3))); 70 | % y_min = ceil(max(proj_corners_ref(2,1), proj_corners_ref(2,2))); 71 | % x_max = floor(min(proj_corners_ref(1,2), proj_corners_ref(1,4))); 72 | % y_max = floor(min(proj_corners_ref(2,3), proj_corners_ref(2,4))); 73 | [x_min, y_min, x_max, y_max] = get_biggest_rect(proj_corners_ref', im_rot); 74 | im = im_rot(y_min:y_max, x_min:x_max,:); 75 | 76 | Htrans3 = eye(3); 77 | Htrans3(1,3) = -(x_min - 1); 78 | Htrans3(2,3) = -(y_min - 1); 79 | H = Htrans3 * H; 80 | end 81 | 82 | function [x_min, y_min, x_max, y_max] = get_biggest_rect(proj_corners, im_ref) 83 | diag1 = [1, 1; size(im_ref, 2), size(im_ref, 1)]; 84 | diag2 = [size(im_ref, 2), 1; 1, size(im_ref, 1)]; 85 | 86 | points = [... 87 | intersect_lines(proj_corners(1:2,:), diag1); ... 88 | intersect_lines(proj_corners(1:2,:), diag2); ... 89 | intersect_lines(proj_corners(2:3,:), diag1); ... 90 | intersect_lines(proj_corners(2:3,:), diag2); ... 91 | intersect_lines(proj_corners(3:4,:), diag1); ... 92 | intersect_lines(proj_corners(3:4,:), diag2); ... 93 | intersect_lines(proj_corners([1 4],:), diag1); ... 94 | intersect_lines(proj_corners([1 4],:), diag2)]; 95 | points = unique(points, 'rows'); 96 | bottom_right = [size(im_ref, 2), size(im_ref, 1)]; 97 | dists = sum((repmat(bottom_right/2, [size(points,1), 1]) - points) .^ 2, 2); 98 | [~,idx] = sort(dists); 99 | closest_points = points(idx(1:4),:); 100 | sorted_x = sort(closest_points(:,1)); 101 | sorted_y = sort(closest_points(:,2)); 102 | x_min = ceil(max(sorted_x(1:2))); 103 | x_max = floor(min(sorted_x(3:4))); 104 | y_min = ceil(max(sorted_y(1:2))); 105 | y_max = floor(min(sorted_y(3:4))); 106 | 107 | % figure; hold on; 108 | % lines = [proj_corners; proj_corners(1,:)]; 109 | % line(lines(:,1), lines(:,2), ... 110 | % 'LineWidth', 3, 'Color', 'r'); 111 | % line(diag1(:,1), diag1(:,2), ... 112 | % 'LineWidth', 3, 'Color', 'b'); 113 | % line(diag2(:,1), diag2(:,2), ... 114 | % 'LineWidth', 3, 'Color', 'b'); 115 | % line(closest_points(:,1), closest_points(:,2), ... 116 | % 'LineWidth', 3, 'Color', 'k'); 117 | % plot(points(:,1), points(:,2), 'x'); 118 | % rectangle('Position', [x_min, y_min, (x_max - x_min + 1), (y_max - y_min + 1)]); 119 | % axis equal; 120 | end 121 | 122 | function [p_int, intersect] = intersect_lines(ps1, ps2) 123 | % ps1 = [x1 y1; x2 y2]; 124 | 125 | % float delta = A1*B2 - A2*B1; 126 | % if(delta == 0) 127 | % throw new ArgumentException("Lines are parallel"); 128 | % 129 | % float x = (B2*C1 - B1*C2)/delta; 130 | % float y = (A1*C2 - A2*C1)/delta; 131 | 132 | % A = y2-y1; B = x1-x2; C = A*x1+B*y1 133 | % figure; 134 | % hold on; 135 | % line(ps1(:,1), ps1(:,2), 'Color', 'r'); 136 | % line(ps2(:,1), ps2(:,2), 'Color', 'k'); 137 | A1 = ps1(2,2) - ps1(1,2); 138 | B1 = ps1(1,1) - ps1(2,1); 139 | C1 = A1 * ps1(1,1) + B1 * ps1(1,2); 140 | A2 = ps2(2,2) - ps2(1,2); 141 | B2 = ps2(1,1) - ps2(2,1); 142 | C2 = A2 * ps2(1,1) + B2 * ps2(1,2); 143 | 144 | delta = A1 * B2 - A2 * B1; 145 | intersect = delta == 0; 146 | if intersect 147 | return; 148 | end 149 | 150 | p_int = [(B2 * C1 - B1 * C2) / delta, (A1 * C2 - A2 * C1) / delta]; 151 | % plot(p_int(:,1), p_int(:,2), 'x'); 152 | end 153 | 154 | -------------------------------------------------------------------------------- /repeatability/repeatability_saltnpepper.m: -------------------------------------------------------------------------------- 1 | function [im2, H] = repeatability_saltnpepper(im, n_pixels) 2 | seed = sum(im(:)); 3 | s = RandStream('mt19937ar', 'Seed', seed); 4 | 5 | [h,w,~] = size(im); 6 | num_missing = n_pixels; 7 | coords = []; 8 | while num_missing > 0 9 | x = randi(s, [1 w], num_missing, 1); 10 | y = randi(s, [1 h], num_missing, 1); 11 | coords = unique([coords; [x y]], 'rows'); 12 | num_missing = n_pixels - size(coords, 1); 13 | end 14 | 15 | im2 = im; 16 | H = eye(3); 17 | if isempty(coords) 18 | return; 19 | end 20 | 21 | im_gray = rgb2gray(im); 22 | coord_inds = sub2ind(size(im_gray), coords(:,2), coords(:,1)); 23 | is_dark = im_gray(coord_inds) < 128; 24 | chan_stride = h * w; 25 | for ch = 1:3 26 | % dark goes bright 27 | im2(coord_inds(is_dark) + (ch - 1) * chan_stride) = 255; 28 | % bright goes dark 29 | im2(coord_inds(~is_dark) + (ch - 1) * chan_stride) = 0; 30 | end 31 | end 32 | -------------------------------------------------------------------------------- /repeatability/repeatability_scale.m: -------------------------------------------------------------------------------- 1 | function [im, H] = repeatability_scale(im, scale_factor) 2 | im = imresize(im, scale_factor); 3 | H = eye(3); 4 | H(1,1) = scale_factor; 5 | H(2,2) = scale_factor; 6 | % because indices are 1 based: 7 | H(1,3) = -scale_factor + 1; 8 | H(2,3) = -scale_factor + 1; 9 | end 10 | -------------------------------------------------------------------------------- /repeatability/run_repeatability.m: -------------------------------------------------------------------------------- 1 | function run_repeatability(index, method_idx) 2 | num_candidates = 1000; 3 | 4 | load('data/pascal_voc07_test_annotations.mat'); 5 | images = {impos.im}; 6 | % testset = get_small_test(); 7 | % images = unique({testset.im}); 8 | if nargin >= 1 9 | index = index(index <= numel(images)); 10 | fprintf('running on image %s/%d\n', mat2str(index), numel(images)); 11 | images = images(index); 12 | end 13 | clear 'testset'; 14 | 15 | config = repeatability_get_config(); 16 | methods = get_method_configs(); 17 | if nargin >= 2 18 | methods = methods(method_idx); 19 | end 20 | 21 | % seed to milliseconds 22 | seed = str2double(datestr(now,'HHMMSSFFF')); 23 | rng(seed); 24 | 25 | for im_i = 1:numel(images) 26 | tic_toc_print('run repeatability image %d/%d\n', im_i, numel(images)); 27 | fprintf('%s\n', images{im_i}); 28 | im = imread(images{im_i}); 29 | if size(im, 3) == 1 30 | im = repmat(im, [1 1 3]); 31 | end 32 | [~,orig_img_id,~] = fileparts(images{im_i}); 33 | reference(im, orig_img_id, methods, num_candidates); 34 | fields = fieldnames(config); 35 | for field_i = 1:numel(fields) 36 | fprintf('running %s\n', fields{field_i}); 37 | subtask(im, orig_img_id, config.(fields{field_i}), methods, num_candidates); 38 | end 39 | end 40 | end 41 | 42 | 43 | function reference(im, orig_img_id, methods, num_candidates) 44 | ref_im_id = [orig_img_id '_reference']; 45 | for method_i = 1:numel(methods) 46 | method = methods(method_i); 47 | if numel(method.repeatability_num_candidates) > 0 48 | t_num_candidates = method.repeatability_num_candidates; 49 | else 50 | t_num_candidates = num_candidates; 51 | end 52 | try % if the candidates are already there 53 | read_candidates_mat(method.repeatability_candidate_dir, ref_im_id, 5); 54 | continue; 55 | catch 56 | end 57 | fprintf('running %s\n', method.name); 58 | [candidates, scores] = method.extract(im, t_num_candidates); 59 | save_candidates_mat(method.repeatability_candidate_dir, ref_im_id, candidates, scores, [], 5); 60 | end 61 | end 62 | 63 | function subtask(im, orig_img_id, sub_config, methods, num_candidates) 64 | for i = 1:numel(sub_config.params) 65 | param = sub_config.params(i); 66 | transformed_im = sub_config.func(im, param); 67 | img_id = sprintf(sub_config.img_id, orig_img_id, i); 68 | 69 | for method_i = 1:numel(methods) 70 | method = methods(method_i); 71 | if numel(method.repeatability_num_candidates) > 0 72 | t_num_candidates = method.repeatability_num_candidates; 73 | else 74 | t_num_candidates = num_candidates; 75 | end 76 | try % if the candidates are already there 77 | read_candidates_mat(method.repeatability_candidate_dir, img_id, 5); 78 | continue; 79 | catch 80 | end 81 | fprintf('running %s\n', method.name); 82 | [candidates, scores] = method.extract(transformed_im, t_num_candidates); 83 | save_candidates_mat(method.repeatability_candidate_dir, img_id, candidates, scores, [], 5); 84 | end 85 | end 86 | end 87 | 88 | -------------------------------------------------------------------------------- /shared/closest_candidates.m: -------------------------------------------------------------------------------- 1 | function [best_overlap,best_boxes] = closest_candidates(gt_boxes, candidates) 2 | % do a matching between gt_boxes and candidates 3 | 4 | num_gt_boxes = size(gt_boxes, 1); 5 | num_candidates = size(candidates, 1); 6 | 7 | iou_matrix = zeros(num_gt_boxes, num_candidates); 8 | for i = 1:num_gt_boxes 9 | iou = overlap(gt_boxes(i,:), candidates); 10 | iou_matrix(i,:) = iou'; 11 | end 12 | 13 | best_overlap = zeros(num_gt_boxes, 1); 14 | best_boxes = -ones(num_gt_boxes, 4); 15 | 16 | [best_overlap,best_boxes] = greedy_matching(iou_matrix, gt_boxes, candidates); 17 | end 18 | 19 | function [best_overlap,best_boxes] = greedy_matching(iou_matrix, gt_boxes, candidates) 20 | [n, m] = size(iou_matrix); 21 | assert(n == size(gt_boxes, 1)); 22 | assert(m == size(candidates, 1)); 23 | if n > m 24 | gt_matching = greedy_matching_rowwise(iou_matrix'); 25 | candidate_matching = (1:m)'; 26 | else 27 | gt_matching = (1:n)'; 28 | candidate_matching = greedy_matching_rowwise(iou_matrix); 29 | end 30 | 31 | best_overlap = zeros(n, 1); 32 | best_boxes = zeros(n, 4); 33 | for pair_idx = 1:numel(gt_matching) 34 | gt_idx = gt_matching(pair_idx); 35 | candidate_idx = candidate_matching(pair_idx); 36 | 37 | best_overlap(gt_idx) = iou_matrix(gt_idx, candidate_idx); 38 | best_boxes(gt_idx,:) = candidates(candidate_idx, :); 39 | end 40 | end 41 | 42 | function [matching, objective] = greedy_matching_rowwise(iou_matrix) 43 | assert(size(iou_matrix, 1) <= size(iou_matrix, 2)); 44 | n = size(iou_matrix, 1); 45 | matching = zeros(n, 1); 46 | objective = 0; 47 | for i = 1:n 48 | % find max element int matrix 49 | [max_per_row, max_col_per_row] = max(iou_matrix, [], 2); 50 | [max_iou,row] = max(max_per_row); 51 | if max_iou == -inf 52 | break 53 | end 54 | 55 | objective = objective + max_iou; 56 | col = max_col_per_row(row); 57 | matching(row) = col; 58 | iou_matrix(row,:) = -inf; 59 | iou_matrix(:,col) = -inf; 60 | end 61 | end 62 | 63 | -------------------------------------------------------------------------------- /shared/deduplicate_proposals.m: -------------------------------------------------------------------------------- 1 | function [proposals, scores] = deduplicate_proposals(proposals, scores) 2 | % Be sure to call this AFTER the proposals are in the right order. 3 | % We cannot just order by score, because not all proposal methods are 4 | % sorted by the same rule. 5 | 6 | if isempty(proposals) 7 | return; 8 | end 9 | 10 | rproposals = proposals; 11 | rproposals(:,1:2) = round(rproposals(:,1:2)); 12 | rproposals(:,3:4) = ceil(rproposals(:,3:4)); 13 | [~, idxs] = unique(rproposals, 'rows', 'stable'); 14 | proposals = proposals(idxs,:); 15 | if ~isempty(scores) 16 | scores = scores(idxs); 17 | end 18 | 19 | end 20 | -------------------------------------------------------------------------------- /shared/get_candidates.m: -------------------------------------------------------------------------------- 1 | function [candidates, scores] = get_candidates(method_config, img_id, num_candidates, ... 2 | allow_filtering, subdirlen, candidate_dir, coco) 3 | 4 | if nargin < 4 5 | allow_filtering = true; 6 | end 7 | if nargin < 5 8 | subdirlen = 4; 9 | end 10 | if nargin < 6 11 | candidate_dir = method_config.candidate_dir; 12 | end 13 | if nargin < 7 14 | coco = false; 15 | end 16 | 17 | [candidates, scores, rerun_num_candidates] = read_candidates_mat(candidate_dir, img_id, subdirlen, coco); 18 | if iscell(candidates) && iscell(scores) 19 | % error('this shouldn''t be used'); 20 | % we have candidates from multiple runs, with different num_candidates 21 | % parameters 22 | % assert(numel(rerun_num_candidates) == numel(candidates)); 23 | % assert(numel(scores) == numel(candidates)); 24 | % assert(all(rerun_num_candidates(1:(end-1)) <= rerun_num_candidates(2:end))); 25 | % idx = find(rerun_num_candidates <= num_candidates, 1, 'last'); 26 | [~,idx] = min(abs(rerun_num_candidates - num_candidates)); 27 | candidates = candidates{idx}; 28 | scores = scores{idx}; 29 | end 30 | 31 | 32 | if allow_filtering 33 | if strcmp(method_config.order, 'none') 34 | % nothing to do 35 | elseif strcmp(method_config.order, 'biggest') 36 | w = candidates(:,3) - candidates(:,1) + 1; 37 | h = candidates(:,4) - candidates(:,2) + 1; 38 | areas = w .* h; 39 | [~,order] = sort(areas, 'descend'); 40 | candidates = candidates(order,:); 41 | scores = scores(order,:); 42 | elseif strcmp(method_config.order, 'smallest') 43 | w = candidates(:,3) - candidates(:,1) + 1; 44 | h = candidates(:,4) - candidates(:,2) + 1; 45 | areas = w .* h; 46 | [~,order] = sort(areas, 'ascend'); 47 | candidates = candidates(order,:); 48 | scores = scores(order,:); 49 | elseif strcmp(method_config.order, 'random') 50 | s = RandStream('mt19937ar','Seed',0); 51 | perm = randperm(s, size(candidates,1)); 52 | candidates = candidates(perm,:); 53 | if numel(scores) > 0 54 | scores = scores(perm); 55 | end 56 | else 57 | [scores, argsort] = sort(scores, method_config.order); 58 | candidates = candidates(argsort,:); 59 | end 60 | 61 | % remove duplicates on the fly 62 | [candidates, scores] = deduplicate_proposals(candidates, scores); 63 | 64 | num_candidates = min(num_candidates, size(candidates, 1)); 65 | candidates = candidates(1:num_candidates,:); 66 | if numel(scores) > 0 67 | scores = scores(1:num_candidates,:); 68 | end 69 | else 70 | error('this shouldn''t be used'); 71 | end 72 | end 73 | -------------------------------------------------------------------------------- /shared/get_method_configs.m: -------------------------------------------------------------------------------- 1 | function methods = get_method_configs() 2 | % Get properties of all proposal methods. For examples, this includes paths to 3 | % candidate files, intermediate files, and also information about how to sort 4 | % candidates. 5 | 6 | % If you want to add your own method, add it at the bottom. 7 | 8 | colormap = [ 9 | 228, 229, 97 10 | 163, 163, 163 11 | 218, 71, 56 12 | 219, 135, 45 13 | 145, 92, 146 14 | 83, 136, 173 15 | 106,61,154 16 | 225, 119, 174 17 | 142, 195, 129 18 | 51,160,44 19 | 223, 200, 51 20 | 92, 172, 158 21 | 177,89,40 22 | 177,89,40 23 | 188, 128, 189 24 | 177,89,40 25 | 251,154,153 26 | 31,120,180 27 | 177,89,40 28 | 177,89,40 29 | 177,89,40 30 | 177,89,40 31 | 177,89,40 32 | 177,89,40 33 | 177,89,40 34 | ] ./ 256; 35 | 36 | config = get_config(); 37 | 38 | methods = []; 39 | 40 | i = numel(methods) + 1; 41 | methods(i).name = 'Objectness'; 42 | methods(i).short_name = 'O'; 43 | methods = add_paths(methods(i), config, 'objectness/'); 44 | methods(i).order = 'descend'; 45 | methods(i).extract = @run_objectness; 46 | methods(i).num_candidates = true; 47 | methods(i).color = colormap(i,:); 48 | methods(i).is_baseline = false; 49 | 50 | i = numel(methods) + 1; 51 | methods(i).name = 'Rahtu'; 52 | methods(i).short_name = 'R1'; 53 | methods(i) = add_paths(methods(i), config, 'categ_indep_detection/'); 54 | methods(i).order = 'none'; 55 | methods(i).extract = @run_categ_independent; 56 | methods(i).num_candidates = true; 57 | methods(i).color = colormap(i,:); 58 | methods(i).is_baseline = false; 59 | 60 | i = numel(methods) + 1; 61 | methods(i).name = 'SelectiveSearch'; 62 | methods(i).short_name = 'SS'; 63 | methods(i) = add_paths(methods(i), config, 'selective_search/'); 64 | methods(i).order = 'ascend'; 65 | methods(i).extract = @run_selective_search; 66 | methods(i).num_candidates = true; 67 | methods(i).color = colormap(i,:); 68 | methods(i).is_baseline = false; 69 | 70 | i = numel(methods) + 1; 71 | methods(i).name = 'RandomizedPrims'; 72 | methods(i).short_name = 'RP'; 73 | methods(i) = add_paths(methods(i), config, 'randomized_prims/'); 74 | methods(i).gt_recall_num_candidates = 20000; 75 | methods(i).rerun_num_candidates = []; 76 | methods(i).repeatability_num_candidates = 5000; 77 | methods(i).order = 'none'; 78 | methods(i).extract = @run_randomized_prims; 79 | methods(i).num_candidates = true; 80 | methods(i).color = colormap(i,:); 81 | methods(i).is_baseline = false; 82 | 83 | i = numel(methods) + 1; 84 | methods(i).name = 'Uniform'; 85 | methods(i).short_name = 'U'; 86 | methods(i) = add_paths(methods(i), config, 'random_uniform/'); 87 | methods(i).order = 'random'; 88 | methods(i).extract = @sample_uniform; 89 | methods(i).num_candidates = true; 90 | methods(i).color = colormap(i,:); 91 | methods(i).is_baseline = true; 92 | 93 | i = numel(methods) + 1; 94 | methods(i).name = 'Gaussian'; 95 | methods(i).short_name = 'G'; 96 | methods(i) = add_paths(methods(i), config, 'random_gaussian/'); 97 | methods(i).order = 'random'; 98 | methods(i).extract = @sample_gaussian; 99 | methods(i).num_candidates = true; 100 | methods(i).color = colormap(i,:); 101 | methods(i).is_baseline = true; 102 | 103 | i = numel(methods) + 1; 104 | methods(i).name = 'CPMC'; 105 | methods(i).short_name = 'C'; 106 | methods(i) = add_paths(methods(i), config, 'CPMC/'); 107 | methods(i).order = 'descend'; 108 | methods(i).extract = @run_cpmc; 109 | methods(i).num_candidates = false; 110 | methods(i).color = colormap(i,:); 111 | methods(i).is_baseline = false; 112 | 113 | i = numel(methods) + 1; 114 | methods(i).name = 'Bing'; 115 | methods(i).short_name = 'B'; 116 | methods(i) = add_paths(methods(i), config, 'BING/'); 117 | methods(i).order = 'descend'; 118 | methods(i).extract = @run_bing; 119 | methods(i).num_candidates = true; 120 | methods(i).color = colormap(i,:); 121 | methods(i).is_baseline = false; 122 | 123 | i = numel(methods) + 1; 124 | methods(i).name = 'Endres'; 125 | methods(i).short_name = 'E'; 126 | methods(i) = add_paths(methods(i), config, 'endres/'); 127 | methods(i).order = 'none'; 128 | methods(i).extract = @run_prop; 129 | methods(i).num_candidates = false; 130 | methods(i).color = colormap(i,:); 131 | methods(i).is_baseline = false; 132 | 133 | i = numel(methods) + 1; 134 | methods(i).name = 'Rantalankila'; 135 | methods(i).short_name = 'R4'; 136 | methods(i) = add_paths(methods(i), config, 'Rantalankila/'); 137 | methods(i).order = 'none'; 138 | methods(i).extract = @run_rantalankila; 139 | methods(i).num_candidates = true; 140 | methods(i).rerun_num_candidates = [100 1000 10000]; 141 | methods(i).color = colormap(i,:); 142 | methods(i).is_baseline = false; 143 | 144 | i = numel(methods) + 1; 145 | methods(i).name = 'Superpixels'; 146 | methods(i).short_name = 'SP'; 147 | methods(i) = add_paths(methods(i), config, 'segmentation_baseline/'); 148 | methods(i).order = 'none'; 149 | methods(i).extract = @run_felsen_candidates; 150 | methods(i).num_candidates = false; 151 | methods(i).color = colormap(i,:); 152 | methods(i).is_baseline = true; 153 | 154 | i = numel(methods) + 1; 155 | methods(i).name = 'Sliding window'; 156 | methods(i).short_name = 'SW'; 157 | methods(i) = add_paths(methods(i), config, 'sliding_window/'); 158 | methods(i).order = 'none'; 159 | methods(i).extract = @sample_bing_windows; 160 | methods(i).num_candidates = true; 161 | methods(i).rerun_num_candidates = ceil(10 .^ (2:0.5:4)); 162 | methods(i).color = colormap(i,:); 163 | methods(i).is_baseline = true; 164 | 165 | i = numel(methods) + 1; 166 | methods(i).name = 'EdgeBoxes'; 167 | methods(i).short_name = 'EB70'; 168 | methods(i) = add_paths(methods(i), config, 'edge_boxes_70/'); 169 | methods(i).order = 'descend'; 170 | methods(i).extract = @run_edge_boxes; 171 | methods(i).num_candidates = true; 172 | methods(i).color = colormap(i,:); 173 | methods(i).is_baseline = false; 174 | 175 | i = numel(methods) + 1; 176 | methods(i).name = 'EdgeBoxes50'; 177 | methods(i).short_name = 'EB50'; 178 | methods(i) = add_paths(methods(i), config, 'edge_boxes_50/'); 179 | methods(i).order = 'descend'; 180 | methods(i).extract = @run_edge_boxes50; 181 | methods(i).num_candidates = true; 182 | methods(i).color = colormap(i,:); 183 | methods(i).is_baseline = false; 184 | methods(i).line_style = ':'; 185 | 186 | i = numel(methods) + 1; 187 | methods(i).name = 'MCG'; 188 | methods(i).short_name = 'M'; 189 | methods(i) = add_paths(methods(i), config, 'MCG2015/'); 190 | methods(i).order = 'none'; 191 | methods(i).extract = @run_MCG2015; 192 | methods(i).num_candidates = true; 193 | methods(i).color = colormap(i,:); 194 | methods(i).is_baseline = false; 195 | 196 | i = numel(methods) + 1; 197 | methods(i).name = 'EdgeBoxes90'; 198 | methods(i).short_name = 'EB90'; 199 | methods(i) = add_paths(methods(i), config, 'edge_boxes_90/'); 200 | methods(i).order = 'descend'; 201 | methods(i).extract = @run_edge_boxes90; 202 | methods(i).num_candidates = true; 203 | methods(i).color = colormap(i,:); 204 | methods(i).is_baseline = false; 205 | methods(i).line_style = '-.'; 206 | 207 | i = numel(methods) + 1; 208 | methods(i).name = 'Rigor'; 209 | methods(i).short_name = 'Ri'; 210 | methods(i) = add_paths(methods(i), config, 'rigor/'); 211 | methods(i).order = 'none'; 212 | methods(i).extract = @run_rigor; 213 | methods(i).num_candidates = true; 214 | methods(i).color = colormap(i,:); 215 | methods(i).is_baseline = false; 216 | methods(i).rerun_num_candidates = [10 100 1000 2000 10000]; 217 | 218 | i = numel(methods) + 1; 219 | methods(i).name = 'Geodesic'; 220 | methods(i).short_name = 'G'; 221 | methods(i) = add_paths(methods(i), config, 'geodesic/'); 222 | methods(i).order = 'none'; 223 | methods(i).extract = @run_gop; 224 | methods(i).num_candidates = false; 225 | methods(i).color = colormap(i,:); 226 | methods(i).is_baseline = false; 227 | methods(i).rerun_num_candidates = [10 100 1000 2000 10000]; 228 | 229 | i = numel(methods) + 1; 230 | methods(i).name = 'EdgeBoxes60'; 231 | methods(i).short_name = 'EB60'; 232 | methods(i) = add_paths(methods(i), config, 'edge_boxes_60/'); 233 | methods(i).order = 'descend'; 234 | methods(i).extract = @run_edge_boxes60; 235 | methods(i).num_candidates = true; 236 | methods(i).color = colormap(i,:); 237 | methods(i).is_baseline = false; 238 | 239 | i = numel(methods) + 1; 240 | methods(i).name = 'EdgeBoxes80'; 241 | methods(i).short_name = 'EB80'; 242 | methods(i) = add_paths(methods(i), config, 'edge_boxes_80/'); 243 | methods(i).order = 'descend'; 244 | methods(i).extract = @run_edge_boxes80; 245 | methods(i).num_candidates = true; 246 | methods(i).color = colormap(i,:); 247 | methods(i).is_baseline = false; 248 | 249 | i = numel(methods) + 1; 250 | methods(i).name = 'EdgeBoxes55'; 251 | methods(i).short_name = 'EB55'; 252 | methods(i) = add_paths(methods(i), config, 'edge_boxes_55/'); 253 | methods(i).order = 'descend'; 254 | methods(i).extract = @(im,num) run_edge_boxes(im, num, 0.65, 0.60); 255 | methods(i).num_candidates = true; 256 | methods(i).color = colormap(i,:); 257 | methods(i).is_baseline = false; 258 | 259 | i = numel(methods) + 1; 260 | methods(i).name = 'EdgeBoxes65'; 261 | methods(i).short_name = 'EB65'; 262 | methods(i) = add_paths(methods(i), config, 'edge_boxes_65/'); 263 | methods(i).order = 'descend'; 264 | methods(i).extract = @(im,num) run_edge_boxes(im, num, 0.65, 0.70); 265 | methods(i).num_candidates = true; 266 | methods(i).color = colormap(i,:); 267 | methods(i).is_baseline = false; 268 | 269 | i = numel(methods) + 1; 270 | methods(i).name = 'EdgeBoxes75'; 271 | methods(i).short_name = 'EB75'; 272 | methods(i) = add_paths(methods(i), config, 'edge_boxes_75/'); 273 | methods(i).order = 'descend'; 274 | methods(i).extract = @(im,num) run_edge_boxes(im, num, 0.70, 0.80); 275 | methods(i).num_candidates = true; 276 | methods(i).color = colormap(i,:); 277 | methods(i).is_baseline = false; 278 | 279 | i = numel(methods) + 1; 280 | methods(i).name = 'EdgeBoxes85'; 281 | methods(i).short_name = 'EB85'; 282 | methods(i) = add_paths(methods(i), config, 'edge_boxes_85/'); 283 | methods(i).order = 'descend'; 284 | methods(i).extract = @(im,num) run_edge_boxes(im, num, 0.80, 0.90); 285 | methods(i).num_candidates = true; 286 | methods(i).color = colormap(i,:); 287 | methods(i).is_baseline = false; 288 | 289 | i = numel(methods) + 1; 290 | methods(i).name = 'Edge Boxes AR'; 291 | methods(i).short_name = 'EBAR'; 292 | methods(i) = add_paths(methods(i), config, 'edge_boxes_AR/'); 293 | methods(i).order = 'descend'; 294 | methods(i).extract = @run_edge_boxes_AR; 295 | methods(i).num_candidates = true; 296 | methods(i).color = colormap(i,:); 297 | methods(i).is_baseline = false; 298 | 299 | % add your own method here: 300 | if false 301 | i = numel(methods) + 1; 302 | methods(i).name = 'The full name of your method'; 303 | methods(i).short_name = 'a very short version of the name'; 304 | methods(i) = add_paths(methods(i), config, 'ours-wip/'); 305 | % This specifies how to order candidates so that the first n, are the best n 306 | % candidates. For example we run a method for 10000 candidates and then take 307 | % the first 10, instead of rerunning for 10 candidates. Valid orderings are: 308 | % none: candidates are already sorted, do nothing 309 | % ascend/descend: sort by score descending or ascending 310 | % random: random order 311 | % biggest/smallest: sort by size of the bounding boxes 312 | methods(i).order = 'descend'; 313 | % A function pointer to a method that runs your proposal detector. 314 | methods(i).extract = @run_edge_boxes90; 315 | % If your method supports sorting this should be empty. If your method has to 316 | % be rerun for every number of candidates we want, specify the number of 317 | % candidates here: 318 | methods(i).rerun_num_candidates = []; % ceil(10 .^ (2:0.5:4)); 319 | % Specifies whether or not your method takes the desired number of candidates 320 | % as an input. 321 | % TODO(hosang): Is this actually used anywhere? 322 | methods(i).num_candidates = true; 323 | % color for drawing 324 | methods(i).color = colormap(i,:); 325 | % This should be false. Is used for drawing baselines dashed. 326 | methods(i).is_baseline = false; 327 | end 328 | 329 | % do the sorting dance 330 | sort_keys = [num2cell([methods.is_baseline])', {methods.name}']; 331 | for i = 1:numel(methods) 332 | sort_keys{i,1} = sprintf('%d', sort_keys{i,1}); 333 | end 334 | [~,idx] = sortrows(sort_keys); 335 | for i = 1:numel(methods) 336 | methods(idx(i)).sort_key = i; 337 | end 338 | end 339 | 340 | function method = add_paths(method, config, method_subdir) 341 | method.candidate_dir = fullfile(config.precomputed_candidates, method_subdir, 'mat'); 342 | method.repeatability_candidate_dir = fullfile(config.precomputed_candidates, method_subdir, 'repeatability_mat'); 343 | method.best_voc07_candidates_file = fullfile(config.precomputed_candidates, method_subdir, 'best_candidates.mat'); 344 | method.best_imagenet_candidates_file = fullfile(config.precomputed_candidates, method_subdir, 'best_candidates_imagenet.mat'); 345 | method.repeatability_matching_file = fullfile(config.precomputed_candidates, method_subdir, 'repeatability_matching.mat'); 346 | method.candidate_dir_coco = fullfile(config.precomputed_candidates_coco, method_subdir, 'mat'); 347 | method.best_coco14_candidates_file = fullfile(config.precomputed_candidates_coco, method_subdir, 'best_candidates_coco14.mat'); 348 | end -------------------------------------------------------------------------------- /shared/overlap.m: -------------------------------------------------------------------------------- 1 | function [iou] = overlap(box, boxes) 2 | % compute intersection over union between a single box and a set of boxes 3 | 4 | n_boxes = size(boxes, 1); 5 | iou = zeros(n_boxes, 1); 6 | 7 | % intersection bbox 8 | bi = [max(box(1),boxes(:,1)) max(box(2),boxes(:,2)) ... 9 | min(box(3),boxes(:,3)) min(box(4),boxes(:,4))]; 10 | 11 | iw = bi(:,3) - bi(:,1) + 1; 12 | ih = bi(:,4) - bi(:,2) + 1; 13 | 14 | not_empty = iw > 0 & ih > 0; 15 | if any(not_empty) 16 | intersection = iw(not_empty) .* ih(not_empty); 17 | % compute overlap as area of intersection / area of union 18 | union = (boxes(not_empty,3) - boxes(not_empty,1) + 1) .* ... 19 | (boxes(not_empty,4) - boxes(not_empty,2) + 1) + ... 20 | (box(3) - box(1) + 1) * (box(4) - box(2) + 1) - ... 21 | intersection; 22 | iou(not_empty) = intersection ./ union; 23 | end 24 | 25 | end -------------------------------------------------------------------------------- /shared/plot_legend.m: -------------------------------------------------------------------------------- 1 | function plot_legend(method_configs) 2 | 3 | figure; hold on; 4 | n_methods = numel(method_configs); 5 | [~,order] = sort([method_configs.sort_key]); 6 | method_configs = method_configs(order); 7 | 8 | x = 1:10; 9 | y = 1:10; 10 | handles = zeros([n_methods, 1]); 11 | for i = 1:n_methods 12 | style = '-'; 13 | if method_configs(i).is_baseline 14 | style = '--'; 15 | end 16 | if ~isempty(method_configs(i).line_style) 17 | style = method_configs(i).line_style; 18 | end 19 | handles(i) = plot(x, y, 'Color', method_configs(i).color, ... 20 | 'LineWidth', 1.5, 'LineStyle', style); 21 | end 22 | lh = legend({method_configs.name}); 23 | legend boxoff; 24 | for i = 1:n_methods 25 | set(handles(i), 'visible', 'off'); 26 | end 27 | set(gca, 'visible', 'off'); 28 | end -------------------------------------------------------------------------------- /shared/read_candidates_mat.m: -------------------------------------------------------------------------------- 1 | function [boxes, scores, num_candidates] = read_candidates_mat(dirname, img_id, subdirlen, coco) 2 | if nargin < 3 3 | subdirlen = 4; 4 | end 5 | if nargin < 4 6 | coco = false; 7 | end 8 | 9 | if coco 10 | subdirlen1 = 14; 11 | subdir1 = img_id(1:subdirlen1); 12 | subdirlen2 = 22; 13 | subdir2 = img_id(1:subdirlen2); 14 | subdir = fullfile(subdir1, subdir2); 15 | else 16 | subdir = img_id(1:subdirlen); 17 | end 18 | matfile = fullfile(dirname, subdir, sprintf('%s.mat', img_id)); 19 | 20 | % default value 21 | num_candidates = 10000; 22 | 23 | load(matfile); 24 | end -------------------------------------------------------------------------------- /shared/save_candidates_mat.m: -------------------------------------------------------------------------------- 1 | function save_candidates_mat(dirname, img_id, boxes, scores, num_candidates, subdirlen, coco) 2 | % Save candidates to disk. 3 | 4 | if nargin < 5 5 | num_candidates = []; 6 | end 7 | if nargin < 6 8 | subdirlen = 4; 9 | end 10 | if nargin < 7 11 | coco = false; 12 | end 13 | 14 | if coco 15 | % full directories turn out to be slow 16 | subdirlen1 = 14; 17 | subdir1 = img_id(1:subdirlen1); 18 | subdirlen2 = 22; 19 | subdir2 = img_id(1:subdirlen2); 20 | subdir = fullfile(subdir1, subdir2); 21 | else 22 | subdir = img_id(1:subdirlen); 23 | end 24 | 25 | path = fullfile(dirname, subdir); 26 | if ~exist(path, 'dir') 27 | mkdir(path); 28 | end 29 | matfile = fullfile(dirname, subdir, sprintf('%s.mat', img_id)); 30 | save(matfile, 'boxes', 'scores', 'num_candidates'); 31 | end 32 | -------------------------------------------------------------------------------- /startup.m: -------------------------------------------------------------------------------- 1 | 2 | if ~isdeployed() 3 | proposal_root_path = fileparts(mfilename('fullpath')); 4 | 5 | addpath(fullfile(proposal_root_path, 'repeatability')); 6 | addpath(fullfile(proposal_root_path, 'util')); 7 | addpath(fullfile(proposal_root_path, 'shared')); 8 | addpath(fullfile(proposal_root_path, 'recall')); 9 | addpath(fullfile(proposal_root_path, 'baselines')); 10 | addpath(fullfile(proposal_root_path, 'detector_wiggling')); 11 | end 12 | -------------------------------------------------------------------------------- /util/legendshrink.m: -------------------------------------------------------------------------------- 1 | function legendshrink(s,align,lg) 2 | % LEGENDSHRINK Shrink the legend lines 3 | % 4 | % This is important for small plots because the size of the lines is 5 | % constant irrespective of the physical size of the figure. Only works 6 | % for vertical legends, for now. 7 | % 8 | % LEGENDSHRINK(S): adjusts the length of the lines in the legend by 9 | % scaling factor S < 1. (S = 0.6 if omitted.) All legends in the current 10 | % figure are affected. 11 | % 12 | % LEGENDSHRING(S,ALIGN): as above aligning the legend text as specified: 13 | % ALIGN = 'left' | 'right' | 'centre' | 'best' (default) 14 | % Default ('best') aligning adapts the alignment based on the 'location' 15 | % of the legend. The first letter can be used as a shorthand for each. 16 | % 17 | % LEGENDSHRINK(S,ALIGN,LG): as above for legend handle(s) LG. 18 | % 19 | % If S or ALIGN are [] then default values are used. 20 | % 21 | % EXAMPLE 22 | % figure; hold on 23 | % plot(1:10,'.-'); 24 | % plot(10:-1:1,'o-'); 25 | % legend({'one' 'two'},'location','north') 26 | % legendshrink 27 | % 28 | % 29 | % Please report bugs and feature requests for 30 | % this package at the development repository: 31 | % 32 | % 33 | % LEGENDSHRINK v0.2 2009/22/06 Will Robertson 34 | % Licence appended. 35 | 36 | 37 | %% Input parsing 38 | if nargin < 1 || isempty(s), s = 0.6; end 39 | if nargin < 2 || isempty(align), align = 'best'; end 40 | if nargin < 3 41 | % all legends in the current figure: 42 | lg = findobj(gcf,'Tag','legend'); 43 | end 44 | 45 | if isempty(lg) 46 | warning('LEGENDSHRINK:nolegend',... 47 | 'There is no legend to shrink. Exiting gracefully.'); 48 | return 49 | elseif length(lg) > 1 50 | % If there are multiple legends then re-call the function for each 51 | % individual legend handle: 52 | for ii = 1:length(lg) 53 | legendshrink(s,align,lg(ii)) 54 | end 55 | return 56 | end 57 | 58 | %% Get parameters 59 | % Will break if children are added to the legend axis. Damn. 60 | 61 | lch = get(lg,'Children'); 62 | % this allows interaction with labelplot.m: 63 | cch = findobj(lch,'-not','Tag','legendlabel'); 64 | cll = findobj(lch,'Tag','legendlabel'); 65 | 66 | orientation = get(lg,'Orientation'); 67 | legendloc = get(lg,'Location'); 68 | orientvertical = strcmp(orientation,'vertical'); 69 | 70 | %% Resizing and aligning 71 | 72 | if orientvertical 73 | if align(1)=='l' 74 | legendtrim = @legendtrimleft; 75 | elseif align(1)=='r' 76 | legendtrim = @legendtrimright; 77 | elseif align(1)=='c' 78 | legendtrim = @legendtrimcentre; 79 | else 80 | % Want to ensure that the legend is trimmed away from the side that's being 81 | % aligned to the figure; e.g., if the legend is inside the figure on the 82 | % right, then the left side needs to be trimmed. And vice versa. Centre it 83 | % if unsure. (Perhaps there should be an option to always do this.) 84 | if ~isempty(regexpi(legendloc,'WestOutside$')) || ... 85 | ~isempty(regexpi(legendloc,'East$')) 86 | legendtrim = @legendtrimleft; 87 | elseif ~isempty(regexpi(legendloc,'EastOutside$')) || ... 88 | ~isempty(regexpi(legendloc,'West$')) 89 | legendtrim = @legendtrimright; 90 | else 91 | legendtrim = @legendtrimcentre; 92 | end 93 | end 94 | else 95 | warning('LEGENDSHRINK:horizontal',... 96 | 'There is yet no implemented method for shrinking horizontal legends. Exiting gracefully.'); 97 | legendtrim = @legendtrimabort; 98 | end 99 | 100 | %% Loop through and adjust the legend children 101 | 102 | % hack to get things working for now: 103 | cch_lines = findobj(cch,'Type','Line'); 104 | cch_min = find(cch==cch_lines(1)); 105 | cch = cch(cch_min:end); 106 | cch_max = 3*floor(length(cch)/3); 107 | cch = cch(1:cch_max); 108 | 109 | for ii = 2:3:length(cch) 110 | % ii-1 == marker handle (line handle) 111 | % ii == line handle 112 | % ii+1 == text handle 113 | linepos = get(cch(ii),'XData'); 114 | textpos = get(cch(ii+1),'Position'); 115 | linewidth = linepos(2)-linepos(1); 116 | 117 | [newlinepos newtextpos] = legendtrim(linepos,textpos); 118 | 119 | set(cch(ii-1),'XData', mean(newlinepos)); 120 | set(cch(ii), 'XData', newlinepos); 121 | set(cch(ii+1),'Position',newtextpos); 122 | end 123 | 124 | %% Adjust the legend title, if any 125 | % This is provided by labelplot.m (same author) 126 | 127 | if ~isempty(cll) 128 | llpos = get(cll,'Position'); 129 | if isequal(legendtrim,@legendtrimleft) 130 | llpos(1) = llpos(1)+s*linewidth/2; 131 | elseif isequal(legendtrim,@legendtrimright) 132 | llpos(1) = llpos(1)-s*linewidth/2; 133 | end 134 | set(cll,'Position',llpos); 135 | end 136 | 137 | %% subfunctions 138 | 139 | function [newlinepos newtextpos] = legendtrimleft(linepos,textpos) 140 | newlinepos = linepos; 141 | newtextpos = textpos; 142 | newlinepos(1) = linepos(2)-s*linewidth; 143 | end 144 | function [newlinepos newtextpos] = legendtrimright(linepos,textpos) 145 | newlinepos = linepos; 146 | newtextpos = textpos; 147 | newlinepos(2) = linepos(1)+s*linewidth; 148 | newtextpos(1) = textpos(1)-(linepos(2)-newlinepos(2)); 149 | end 150 | function [newlinepos newtextpos] = legendtrimcentre(linepos,textpos) 151 | newlinepos = linepos; 152 | newtextpos = textpos; 153 | newtextpos(1) = textpos(1)-(1-s)*linewidth/2; 154 | newlinepos(1) = linepos(1)+(1-s)*linewidth/2; 155 | newlinepos(2) = newlinepos(1)+s*linewidth; 156 | end 157 | function [newlinepos newtextpos] = legendtrimabort(linepos,textpos) 158 | newlinepos = linepos; 159 | newtextpos = textpos; 160 | end 161 | 162 | end 163 | 164 | % Copyright (c) 2007-2009, Will Robertson, wspr 81 at gmail dot com 165 | % All rights reserved. 166 | % 167 | % Distributed under the BSD licence in accordance with the wishes of the 168 | % Matlab File Exchange. 169 | % 170 | % Redistribution and use in source and binary forms, with or without 171 | % modification, are permitted provided that the following conditions are met: 172 | % * Redistributions of source code must retain the above copyright 173 | % notice, this list of conditions and the following disclaimer. 174 | % * Redistributions in binary form must reproduce the above copyright 175 | % notice, this list of conditions and the following disclaimer in the 176 | % documentation and/or other materials provided with the distribution. 177 | % 178 | % THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER ''AS IS'' AND ANY 179 | % EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 180 | % WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 181 | % DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE FOR ANY 182 | % DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 183 | % (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 184 | % LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND 185 | % ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 186 | % (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 187 | % THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 188 | -------------------------------------------------------------------------------- /util/printpdf.m: -------------------------------------------------------------------------------- 1 | function printpdf(fname) 2 | %PRINTPDF Prints the current figure into a pdf document 3 | 4 | % set(gca, 'LooseInset', get(gca, 'TightInset')); 5 | fname = [regexprep(fname, '^(.*)\.pdf$', '$1'), '.eps']; 6 | print('-depsc', fname); 7 | if ~system(['epstopdf ', fname]) 8 | system(['rm ', fname]); 9 | end 10 | -------------------------------------------------------------------------------- /util/printpng.m: -------------------------------------------------------------------------------- 1 | function printpng(fname) 2 | %PRINTPDF Prints the current figure into a png document 3 | 4 | set(gca, 'LooseInset', get(gca, 'TightInset')); 5 | print('-dpng', '-r200', fname); 6 | -------------------------------------------------------------------------------- /util/tic_toc_print.m: -------------------------------------------------------------------------------- 1 | function tic_toc_print(fmt, varargin) 2 | % Print only after 1 second has passed since the last print. 3 | % Arguments are the same as for fprintf. 4 | 5 | persistent th; 6 | 7 | if isempty(th) 8 | fprintf(fmt, varargin{:}); 9 | drawnow; 10 | th = tic(); 11 | end 12 | 13 | if toc(th) > 1 14 | fprintf(fmt, varargin{:}); 15 | drawnow; 16 | th = tic(); 17 | end 18 | --------------------------------------------------------------------------------