├── README.md
├── __init__.py
├── config.py
├── data
    ├── annotations.json
    ├── resnet101_predict.json
    └── valid_target.json
├── docs
    ├── 交通信号类型名称定义.pdf
    └── 基于RetinaNet的交通标注检测.pdf
├── encoder.py
├── evaluate
    ├── __init__.py
    ├── anno_func.py
    ├── anno_process.ipynb
    ├── crop_image.ipynb
    ├── data_process.ipynb
    └── eval_check.ipynb
├── generate_gt.py
├── loss.py
├── main.py
├── preprocessing
    ├── __init__.py
    ├── annotations.py
    ├── bbox.py
    ├── datasets.py
    ├── errors.py
    └── transforms.py
├── resnet.py
├── retinanet.py
├── samples
    ├── group1-296_0_1.jpg
    ├── group1-6_0_1.jpg
    ├── group3-162_0_1.jpg
    ├── group5-109_2_1.jpg
    └── group7-3_0_1.jpg
├── test.py
├── train.py
└── utils.py


/README.md:
--------------------------------------------------------------------------------
 1 | # Traffic-Sign-Detection-with-RetinaNet
 2 | ## Introduction
 3 | 
 4 | 在这个项目里，我们实现了基于[RetinaNet](https://arxiv.org/pdf/1708.02002.pdf) 的交通标志检测算法。该项目主要代码参考自：[Simultaneous-Traffic-Sign-Detection-and-Classification-with-RetinaNet](https://github.com/CJHMPower/Simultaneous-Traffic-Sign-Detection-and-Classification-with-RetinaNet)。我们对数据集和代码进行了部分更改，使其能够在pytorch1.0版本下正常运行，并实现了一个五类的交通标志检测任务。
 5 | 
 6 | ## Dataset
 7 | 
 8 | ### CVTS数据集
 9 | 
10 | Computer Vision Traffic Sign，CVTS是计算机视觉课程上所有人手工标注得到的交通标志数据集。一共有警告、禁令、指示、道路、交通灯五大类别，每个大类别又包含若干个小类别，共77个小类别。其部分类别示例如下图所示：
11 | 
12 | ![](https://ws1.sinaimg.cn/large/a92fa7d4gy1fyxslc95x7j20b606vjti.jpg)
13 | 
14 | **数据清理：**
15 | 
16 | CVTS数据集共包含2620张图片，但由于标注质量参差不齐，存在漏标、误标、检测框未对齐等多种情况，原始数据无法直接用以网络训练。因此，我们对CVTS数据集进行了清理，去除掉图片质量和标注质量较差的数据。经过清理之后，得到的数据集共有1250张，分别包含了754个红绿灯标志、623个禁令标志、345个指路标志以及244个警告标志。显然，清理后的数据集存在着不同类别上的数据量不平衡的问题。
17 | 
18 | **数据增强：**
19 | 
20 | 为了确保网络在每个类别上的表现均衡，我们对清理后的数据集进行了数据增强，确保每个类别的图片量都在1000张以上。经过数据增强后，得到的数据集包含6333张图片，数据量大致确保了在小型网络上能够得到相对较好的结果。
21 | 
22 | ### Tsinghua_Tencent_100K数据集
23 | 
24 | [Tsinghua_Tencent_100K](https://cg.cs.tsinghua.edu.cn/traffic-sign/)是清华大学在2016年发布的交通标志数据集。该数据集是从1,000,000张腾讯街景图片中选取的300,000张交通标志图片构成。在去除掉出现次数少于100次的交通标志数据后，得到的数据集共有37212张图片，共包含指示标志、禁令标志、警告标志3个大类，其中又细分为42个小类。其交通标志示例下图所示。
25 | 
26 | ![](https://ws1.sinaimg.cn/large/a92fa7d4gy1fyxsmx6zuqj20eg06qmyy.jpg)
27 | 
28 | **类别转换：**
29 | 
30 | 为了将Tsinghua_Tencent_100K数据集应用到我们的五类交通标志检测任务中，我们对该数据集的类别进行了转换，将以**i**开头的类别转换为**s**（指示标志），以**p**开头的类别转换为**z**（禁令标志），以**w**开头的类别转换为**j**（警告标志）。
31 | 
32 | **数据合并：**
33 | 
34 | Tsinghua_Tencent_100K的交通标志数据具有很好的标注质量，能够有效得提高算法的检测准确度，因此我们从该数据集的三大类中随机采样1000张图片，和CVTS数据集合并起来，从而构成了我们最后实验中所使用的数据集：*CVTS-TT100K*。
35 | 
36 | CVTS-TT100K数据集共包含9333张图片，包含了警告标志、禁令标志、指示标志、道路标志和交通灯五个类别。我们从CVTS-TT100K中随机采样1000张图片作为测试集，剩下的8333张图片作为训练集。
37 | 
38 | ## Model
39 | 
40 | - 代码中所采用的数据集：[CVTS-TT100K](https://pan.baidu.com/s/1CCHLA0IqVwCXz8yGcA_zKQ)
41 | - 在CVTS-TT100K和resnet101上训练得到的模型：[resnet101_8K.pth](https://pan.baidu.com/s/1dRaFdobiQ74G2Kns1rnO6A)
42 | - 在Tsinghua_Tencent_100K和resnet152训练得到的模型：[resnet152_40K.pth](https://pan.baidu.com/s/11Pe5uVupfDtMuEXBW338fg)
43 | 
44 | ## Usage
45 | 
46 | 利用预训练模型对 /samples 文件夹下的图片进行检测，并将检测结果输出在 /result 文件夹下：
47 | 
48 | ```python
49 | python test.py -m demo
50 | ```
51 | 
52 | 利用预训练模型测试在验证集上的表现，并在预测结果（json文件）输出到 /data 文件夹下：
53 | 
54 | ```python
55 | python test.py -m valid
56 | ```
57 | 
58 | 训练网络：
59 | 
60 | ```python
61 | python train.py -exp model
62 | ```
63 | 
64 | evaluate 文件下有着许多实用代码：
65 | 
66 | - crop_image：对数据集进行resize和增强操作；
67 | - eval_check.ipynb：测试模型在多个指标上的表现；
68 | - anno_process.ipynb：用于处理annotations各种函数；
69 | - data_process.ipynb：用于处理数据集的各种函数。
70 | 
71 | ## Performance
72 | 
73 | 不同类别和不同边框大小下的accuracy和recall：
74 | 
75 | ![](https://ws1.sinaimg.cn/large/a92fa7d4gy1fyxsrcbl3wj20zh0jswh2.jpg)
76 | 
77 | 不同检测框范围下的 acc-recall 曲线：
78 | 
79 | ![](https://ws1.sinaimg.cn/large/a92fa7d4gy1fyxss06k1rj20jr0jg0w4.jpg)
80 | 
81 | ground truth 和预测的检测框大小分布直方图：
82 | 
83 | ![](https://ws1.sinaimg.cn/large/a92fa7d4gy1fyxssk1j40j20fe068dg4.jpg)
84 | 
85 | ## Demo
86 | 
87 |  CVTS-TT100K的部分检测结果示例：
88 | 
89 | ![](https://ws1.sinaimg.cn/large/a92fa7d4gy1fyxsteh2juj20fe0fe49k.jpg)
90 | 
91 | ## References
92 | 
93 | - Zhu, Zhe, et al. "Traffic-sign detection and classification in the wild." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.
94 | - Li, Yuming, et al. "TAD16K: An enhanced benchmark for autonomous driving." Image Processing (ICIP), 2017 IEEE International Conference on. IEEE, 2017.
95 | - Lin, Tsung-Yi, et al. "Focal loss for dense object detection." *IEEE transactions on pattern analysis and machine intelligence* (2018).
96 | - Lin, Tsung-Yi, et al. "Feature Pyramid Networks for Object Detection." CVPR. Vol. 1. No. 2. 2017.
97 | - Long, Jonathan, Evan Shelhamer, and Trevor Darrell. "Fully convolutional networks for semantic segmentation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015. 
98 | 


--------------------------------------------------------------------------------
/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chenshen03/Traffic-Sign-Detection-with-RetinaNet/f34f6194b80fcd16b12ded9ac8b17ba4fb2fbdb0/__init__.py


--------------------------------------------------------------------------------
/config.py:
--------------------------------------------------------------------------------
 1 | import os
 2 | 
 3 | root = 'data'
 4 | train_dir='train'
 5 | test_dir='test'
 6 | image_dir = os.path.join(root, train_dir)
 7 | 
 8 | annotation_file=os.path.join(root,'annotations.json')
 9 | 
10 | train_imageset_fn=os.listdir(image_dir)
11 | test_imageset_fn=os.listdir(os.path.join(root,test_dir))
12 | 
13 | train_imageset_fn=list(map(lambda x:os.path.join(train_dir,x),train_imageset_fn))
14 | 
15 | val_image_dir=os.path.join(root, test_dir)
16 | val_imageset_fn = list(map(lambda x:os.path.join(test_dir,x),test_imageset_fn))
17 | image_ext = '.jpg'
18 | 
19 | backbone = 'resnet101'
20 | classes = ['s', 'z', 'j', 'l', 'd']
21 | 
22 | # TODO change with dataset
23 | # mean, std = (0.499, 0.523, 0.532), (0.200, 0.202, 0.224)
24 | mean, std = (0.485, 0.456, 0.406), (0.229, 0.224, 0.225)
25 | 
26 | scale =None
27 | 
28 | batch_size = 8
29 | lr = 1e-4
30 | momentum = 0.9
31 | weight_decay =0.0002
32 | num_epochs = 70
33 | lr_decay_epochs = [10]
34 | num_workers = 8
35 | width,height=512,512
36 | eval_while_training = True
37 | eval_every = 2
38 | 


--------------------------------------------------------------------------------
/docs/交通信号类型名称定义.pdf:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chenshen03/Traffic-Sign-Detection-with-RetinaNet/f34f6194b80fcd16b12ded9ac8b17ba4fb2fbdb0/docs/交通信号类型名称定义.pdf


--------------------------------------------------------------------------------
/docs/基于RetinaNet的交通标注检测.pdf:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chenshen03/Traffic-Sign-Detection-with-RetinaNet/f34f6194b80fcd16b12ded9ac8b17ba4fb2fbdb0/docs/基于RetinaNet的交通标注检测.pdf


--------------------------------------------------------------------------------
/encoder.py:
--------------------------------------------------------------------------------
  1 | import torch
  2 | from math import sqrt
  3 | from utils import box_iou, box_nms, change_box_order, meshgrid
  4 | from torch.autograd import Variable
  5 | import numpy as np
  6 | 
  7 | import torch.nn.functional as F
  8 | 
  9 |    #this tensor is to scale the loc_loss to the similar ratio to the cls_loss
 10 | class DataEncoder:
 11 |     
 12 |     def __init__(self):
 13 |         self.anchor_areas = [16*16,32*32,64*64,128*128,256*256]
 14 |         self.aspect_ratios = [0.8,1,1.25]
 15 |         self.scale_ratios = [0.7,1,1.42]
 16 |         self.num_levels = len(self.anchor_areas)
 17 |         self.num_anchors = len(self.aspect_ratios) * len(self.scale_ratios)
 18 |         self.anchor_edges = self.calc_anchor_edges()
 19 |         self.std=torch.Tensor([0.1,0.1,0.2,0.2]) 
 20 | 
 21 |     def calc_anchor_edges(self):
 22 |         anchor_edges = []
 23 |         for area in self.anchor_areas:
 24 |             for ar in self.aspect_ratios:
 25 |                 if ar < 1:
 26 |                     height = sqrt(area)
 27 |                     width = height * ar
 28 |                 else:
 29 |                     width = sqrt(area)
 30 |                     height = width / ar
 31 |                 for sr in self.scale_ratios:
 32 |                     anchor_height = height * sr
 33 |                     anchor_width = width * sr
 34 |                     anchor_edges.append((anchor_width, anchor_height))
 35 |         
 36 |         return torch.Tensor(anchor_edges).view(self.num_levels, self.num_anchors, 2)
 37 |     
 38 |     def get_anchor_boxes(self, input_size):
 39 |         
 40 |         fm_sizes = [(input_size / pow(2, i + 3)).ceil() for i in range(self.num_levels)]
 41 |         
 42 |         boxes = []
 43 |         for i in range(self.num_levels):
 44 |             fm_size = fm_sizes[i]
 45 |             grid_size = (input_size/fm_size).floor()
 46 |             fm_w, fm_h = int(fm_size[0]), int(fm_size[1])
 47 |             xy = meshgrid(fm_w, fm_h) + 0.5  # [fm_h * fm_w, 2]
 48 |             # TODO float and long
 49 |             xy = xy.float()
 50 |             xy = (xy * grid_size).view(fm_w, fm_h, 1, 2).expand(fm_w, fm_h, 9, 2)
 51 |             wh = self.anchor_edges[i].view(1, 1, 9, 2).expand(fm_w, fm_h,9, 2)
 52 |             box = torch.cat([xy, wh], 3)  # [x, y, w, h]
 53 |             boxes.append(box.view(-1, 4))
 54 |         return torch.cat(boxes, 0)
 55 |         
 56 |     
 57 | 
 58 |     def encode(self, boxes, labels, input_size):
 59 |         if isinstance(input_size, int):
 60 |             input_size = torch.Tensor([input_size, input_size])
 61 |         else:
 62 |             input_size = torch.Tensor(input_size)
 63 |         
 64 |         anchor_boxes = self.get_anchor_boxes(input_size)
 65 |         boxes = change_box_order(boxes, 'xyxy2xywh')
 66 |         boxes = boxes.float()
 67 |         ious = box_iou(anchor_boxes, boxes, order='xywh')
 68 |         max_ious, max_ids = ious.max(1)
 69 |         boxes = boxes[max_ids]
 70 |         loc_xy = (boxes[:, :2] - anchor_boxes[:, :2]) / anchor_boxes[:, 2:]
 71 |         loc_wh = torch.log(boxes[:, 2:] / anchor_boxes[:, 2:])
 72 |         
 73 |         loc_targets = torch.cat([loc_xy, loc_wh], 1)
 74 |         loc_targets=loc_targets/self.std
 75 |         cls_targets = 1 + labels[max_ids]
 76 |         
 77 |         cls_targets[max_ious < 0.4] = 0
 78 |         cls_targets[(max_ious >= 0.4) & (max_ious < 0.5)] = -1
 79 |         
 80 |         return loc_targets, cls_targets
 81 |  
 82 |     def softmax(score):
 83 |         for i in range(score.size()[0]):
 84 |             score[i]=F.softmax(score[i])
 85 |         return score
 86 |         
 87 |     def decode(self, loc_preds, cls_preds, input_size):
 88 |         CLS_THRESH = 0.05
 89 |         NMS_THRESH = 0.4
 90 | 
 91 |         if isinstance(input_size, int):
 92 |             input_size = torch.Tensor([input_size, input_size])
 93 |         else:
 94 |             input_size = torch.Tensor(input_size)
 95 | 
 96 |         anchor_boxes = self.get_anchor_boxes(input_size)
 97 |         std=Variable(self.std).cuda()
 98 |         loc_preds=loc_preds*std
 99 |         loc_xy = loc_preds.data.cpu()[:, :2]
100 |         loc_wh = loc_preds.data.cpu()[:, 2:]
101 |         xy = loc_xy * anchor_boxes[:, 2:] + anchor_boxes[:, :2]
102 |         wh = loc_wh.exp() * anchor_boxes[:, 2:]
103 |         boxes = torch.cat([xy, wh], 1)
104 |         boxes = change_box_order(boxes, 'xywh2xyxy')
105 |         cls_preds=F.softmax(cls_preds,1)
106 |         score, labels = cls_preds.max(1)
107 |         ids =  (labels > 0)&(score>CLS_THRESH)
108 |         ids = ids.nonzero().squeeze()
109 |         if len(ids.size())==0:
110 |             return None, None,None
111 |         ids=ids.data.cpu()
112 | 
113 |         keep = box_nms(boxes.cpu()[ids], score.data.cpu()[ids], threshold=NMS_THRESH)
114 |         return boxes.cpu()[ids][keep],labels.data.cpu()[ids][keep],score.data.cpu()[ids][keep]


--------------------------------------------------------------------------------
/evaluate/__init__.py:
--------------------------------------------------------------------------------
1 | 
2 | 


--------------------------------------------------------------------------------
/evaluate/anno_func.py:
--------------------------------------------------------------------------------
  1 | import json
  2 | import pylab as pl
  3 | import random
  4 | import numpy as np
  5 | #import cv2
  6 | import copy
  7 | 
  8 | type42 = "i2,i4,i5,il100,il60,il80,ip,p10,p11,p12,p19,p23,p26,p27,p3,p5,p6,pg,ph4,ph4.5,ph5,pl100,pl120,pl20,pl30,pl40,pl5,pl50,pl60,pl70,pl80,pm20,pm30,pm55,pn,pne,pr40,w13,w32,w55,w57,w59"
  9 | type42 = type42.split(',')
 10 | 
 11 | type3 = "z,j,s"
 12 | type3 = type3.split(',')
 13 | 
 14 | type5 = "z,j,s,l,d"
 15 | type5 = type5.split(',')
 16 | 
 17 | 
 18 | def rect_cross(rect1, rect2):
 19 |     rect = [max(rect1[0], rect2[0]),
 20 |             max(rect1[1], rect2[1]),
 21 |             min(rect1[2], rect2[2]),
 22 |             min(rect1[3], rect2[3])]
 23 |     rect[2] = max(rect[2], rect[0])
 24 |     rect[3] = max(rect[3], rect[1])
 25 |     return rect
 26 | 
 27 | def rect_area(rect):
 28 |     return float(max(0.0, (rect[2]-rect[0])*(rect[3]-rect[1])))
 29 | 
 30 | def calc_cover(rect1, rect2):
 31 |     crect = rect_cross(rect1, rect2)
 32 |     return rect_area(crect) / rect_area(rect2)
 33 | 
 34 | def calc_iou(rect1, rect2):
 35 |     crect = rect_cross(rect1, rect2)
 36 |     ac = rect_area(crect)
 37 |     a1 = rect_area(rect1)
 38 |     a2 = rect_area(rect2)
 39 |     return ac / (a1+a2-ac)
 40 | 
 41 | def get_refine_rects(annos, raw_rects, minscore=20):
 42 |     cover_th = 0.5
 43 |     refine_rects = {}
 44 | 
 45 |     for imgid in raw_rects.keys():
 46 |         v = raw_rects[imgid]
 47 |         tv = copy.deepcopy(sorted(v, key=lambda x:-x[2]))
 48 |         nv = []
 49 |         for obj in tv:
 50 |             rect = obj[1]
 51 |             rect[2]+=rect[0]
 52 |             rect[3]+=rect[1]
 53 |             if rect_area(rect) == 0: continue
 54 |             if obj[2] < minscore: continue
 55 |             cover_area = 0
 56 |             for obj2 in nv:
 57 |                 cover_area += calc_cover(obj2[1], rect)
 58 |             if cover_area < cover_th:
 59 |                 nv.append(obj)
 60 |         refine_rects[imgid] = nv
 61 |     results = {}
 62 |     for imgid, v in refine_rects.items():
 63 |         objs = []
 64 |         for obj in v:
 65 |             mobj = {"bbox":dict(zip(["xmin","ymin","xmax","ymax"], obj[1])), 
 66 |                     "category":annos['types'][int(obj[0]-1)], "score":obj[2]}
 67 |             objs.append(mobj)
 68 |         results[imgid] = {"objects":objs}
 69 |     results_annos = {"imgs":results}
 70 |     return results_annos
 71 | 
 72 | def box_long_size(box):
 73 |     return max(box['xmax']-box['xmin'], box['ymax']-box['ymin'])
 74 | 
 75 | def eval_annos(annos_gd, annos_rt, iou=0.5, imgids=None, check_type=True, types=None, minscore=40, minboxsize=0, maxboxsize=512, match_same=True):
 76 |     ac_n, ac_c = 0,0
 77 |     rc_n, rc_c = 0,0
 78 |     if imgids==None:
 79 |         imgids = annos_rt['imgs'].keys()
 80 |     if types!=None:
 81 |         types = { t:0 for t in types }
 82 |     miss = {"imgs":{}}
 83 |     wrong = {"imgs":{}}
 84 |     right = {"imgs":{}}
 85 |     
 86 |     for imgid in imgids:
 87 |         v = annos_rt['imgs'][imgid]
 88 |         vg = annos_gd['imgs'][imgid]
 89 |         convert = lambda objs: [ [ obj['bbox'][key] for key in ['xmin','ymin','xmax','ymax']] for obj in objs]
 90 |         objs_g = vg["objects"]
 91 |         objs_r = v["objects"]
 92 |         bg = convert(objs_g)
 93 |         br = convert(objs_r)
 94 |         
 95 |         match_g = [-1]*len(bg)
 96 |         match_r = [-1]*len(br)
 97 |         if types!=None:
 98 |             for i in range(len(match_g)):
 99 |                 if not types.has_key(objs_g[i]['category']):
100 |                     match_g[i] = -2
101 |             for i in range(len(match_r)):
102 |                 if not types.has_key(objs_r[i]['category']):
103 |                     match_r[i] = -2
104 |         for i in range(len(match_r)):
105 |             if objs_r[i].has_key('score') and objs_r[i]['score']<minscore:
106 |                 match_r[i] = -2
107 |         matches = []
108 |         for i,boxg in enumerate(bg):
109 |             for j,boxr in enumerate(br):
110 |                 if match_g[i] == -2 or match_r[j] == -2:
111 |                     continue
112 |                 #if match_same and objs_g[i]['category'] != objs_r[j]['category']: continue
113 |                 tiou = calc_iou(boxg, boxr)
114 |                 if tiou>iou:
115 |                     matches.append((tiou, i, j))
116 |         matches = sorted(matches, key=lambda x:-x[0])
117 |         for tiou, i, j in matches:
118 |             if match_g[i] == -1 and match_r[j] == -1:
119 |                 match_g[i] = j
120 |                 match_r[j] = i
121 |                 
122 |         for i in range(len(match_g)):
123 |             boxsize = box_long_size(objs_g[i]['bbox'])
124 |             erase = False
125 |             if not (boxsize>=minboxsize and boxsize<maxboxsize):
126 |                 erase = True
127 |             #if types!=None and not types.has_key(objs_g[i]['category']):
128 |             #    erase = True
129 |             if erase:
130 |                 if match_g[i] >= 0:
131 |                     match_r[match_g[i]] = -2
132 |                 match_g[i] = -2
133 |         
134 |         for i in range(len(match_r)):
135 |             boxsize = box_long_size(objs_r[i]['bbox'])
136 |             if match_r[i] != -1: continue
137 |             if not (boxsize>=minboxsize and boxsize<maxboxsize):
138 |                 match_r[i] = -2
139 |                     
140 |         miss["imgs"][imgid] = {"objects":[]}
141 |         wrong["imgs"][imgid] = {"objects":[]}
142 |         right["imgs"][imgid] = {"objects":[]}
143 |         miss_objs = miss["imgs"][imgid]["objects"]
144 |         wrong_objs = wrong["imgs"][imgid]["objects"]
145 |         right_objs = right["imgs"][imgid]["objects"]
146 |         
147 |         tt = 0
148 |         for i in range(len(match_g)):
149 |             if match_g[i] == -1:
150 |                 miss_objs.append(objs_g[i])
151 |         for i in range(len(match_r)):
152 |             if match_r[i] == -1:
153 |                 obj = copy.deepcopy(objs_r[i])
154 |                 obj['correct_catelog'] = 'none'
155 |                 wrong_objs.append(obj)
156 |             elif match_r[i] != -2:
157 |                 j = match_r[i]
158 |                 obj = copy.deepcopy(objs_r[i])
159 |                 if not check_type or objs_g[j]['category'] == objs_r[i]['category']:
160 |                     right_objs.append(objs_r[i])
161 |                     tt+=1
162 |                 else:
163 |                     obj['correct_catelog'] = objs_g[j]['category']
164 |                     wrong_objs.append(obj)
165 |                     
166 |         
167 |         rc_n += len(objs_g) - match_g.count(-2)
168 |         ac_n += len(objs_r) - match_r.count(-2)
169 |         
170 |         ac_c += tt
171 |         rc_c += tt
172 |     if types==None:
173 |         styps = "all"
174 |     elif len(types)==1:
175 |         styps = types.keys()[0]
176 |     elif not check_type or len(types)==0:
177 |         styps = "none"
178 |     else:
179 |         styps = "[%s, ...total %s...]"%(types.keys()[0], len(types))
180 |     report = "iou:%s, size:[%s,%s), types:%s, accuracy:%s, recall:%s"% (
181 |         iou, minboxsize, maxboxsize, styps, 1 if ac_n==0 else ac_c*1.0/ac_n, 1 if rc_n==0 else rc_c*1.0/rc_n)
182 |     summury = {
183 |         "iou":iou,
184 |         "accuracy":1 if ac_n==0 else ac_c*1.0/ac_n,
185 |         "recall":1 if rc_n==0 else rc_c*1.0/rc_n,
186 |         "miss":miss,
187 |         "wrong":wrong,
188 |         "right":right,
189 |         "report":report
190 |     }
191 |     return summury
192 | 


--------------------------------------------------------------------------------
/evaluate/anno_process.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "code",
  5 |    "execution_count": 2,
  6 |    "metadata": {
  7 |     "slideshow": {
  8 |      "slide_type": "-"
  9 |     }
 10 |    },
 11 |    "outputs": [],
 12 |    "source": [
 13 |     "import numpy  as no\n",
 14 |     "import os\n",
 15 |     "import json"
 16 |    ]
 17 |   },
 18 |   {
 19 |    "cell_type": "code",
 20 |    "execution_count": 6,
 21 |    "metadata": {},
 22 |    "outputs": [],
 23 |    "source": [
 24 |     "group_path = 'group4'\n",
 25 |     "gt_path = os.path.join('./data_origin/newgt', group_path)"
 26 |    ]
 27 |   },
 28 |   {
 29 |    "cell_type": "code",
 30 |    "execution_count": 7,
 31 |    "metadata": {},
 32 |    "outputs": [],
 33 |    "source": [
 34 |     "annos={}\n",
 35 |     "annos['imgs']={}\n",
 36 |     "\n",
 37 |     "for file in os.listdir(gt_path):\n",
 38 |     "    imgid = file.split('.')[0]\n",
 39 |     "    file_name = imgid + '.jpg'\n",
 40 |     "    path = group_path + '/' + file_name\n",
 41 |     "    annos['imgs'][imgid] = {}\n",
 42 |     "    annos['imgs'][imgid]['id'] = imgid\n",
 43 |     "    annos['imgs'][imgid]['path'] = path\n",
 44 |     "    annos['imgs'][imgid]['objects'] = []\n",
 45 |     "    with open(os.path.join(gt_path, file), 'r') as f:\n",
 46 |     "        lines = f.readlines()\n",
 47 |     "    for line in lines:\n",
 48 |     "        line = line.strip().split(' ')\n",
 49 |     "        category = line[1][0]\n",
 50 |     "\n",
 51 |     "        bbox={}\n",
 52 |     "        bbox['xmin'] = float(line[2])\n",
 53 |     "        bbox['ymin'] = float(line[3])\n",
 54 |     "        bbox['xmax'] = float(line[4])\n",
 55 |     "        bbox['ymax'] = float(line[5])\n",
 56 |     "\n",
 57 |     "        annos['imgs'][imgid]['objects'].append({'bbox':bbox, 'category':category})\n",
 58 |     "#         print(line)"
 59 |    ]
 60 |   },
 61 |   {
 62 |    "cell_type": "code",
 63 |    "execution_count": 8,
 64 |    "metadata": {},
 65 |    "outputs": [],
 66 |    "source": [
 67 |     "output_path = './data_cleared/annos/annos_' + group_path + '.json'\n",
 68 |     "with open(output_path, 'w') as f:\n",
 69 |     "    json.dump(annos, f)\n",
 70 |     "    f.close()"
 71 |    ]
 72 |   },
 73 |   {
 74 |    "cell_type": "markdown",
 75 |    "metadata": {},
 76 |    "source": [
 77 |     "# Resize\n",
 78 |     "将annotation的检测框resize到512*512"
 79 |    ]
 80 |   },
 81 |   {
 82 |    "cell_type": "code",
 83 |    "execution_count": null,
 84 |    "metadata": {},
 85 |    "outputs": [],
 86 |    "source": [
 87 |     "annos={}\n",
 88 |     "annos['imgs']={}\n",
 89 |     "\n",
 90 |     "for file in os.listdir(gt_path):\n",
 91 |     "    imgid = file.split('.')[0]\n",
 92 |     "    annos['imgs'][imgid] = {}\n",
 93 |     "    annos['imgs'][imgid]['id'] = imgid\n",
 94 |     "    annos['imgs'][imgid]['path'] = os.path.join(group_path, file)\n",
 95 |     "    annos['imgs'][imgid]['objects'] = []\n",
 96 |     "    with open(os.path.join(gt_path, file), 'r') as f:\n",
 97 |     "        lines = f.readlines()\n",
 98 |     "    for line in lines:\n",
 99 |     "        line = line.strip().split(' ')\n",
100 |     "        category = line[1][0]\n",
101 |     "\n",
102 |     "        bbox={}\n",
103 |     "        bbox['xmin'] = float(line[2]) / (1280/512)\n",
104 |     "        bbox['xmax'] = float(line[4]) / (1280/512)\n",
105 |     "        bbox['ymin'] = float(line[3]) / (1024/512)\n",
106 |     "        bbox['ymax'] = float(line[5]) / (1024/512)\n",
107 |     "\n",
108 |     "        annos['imgs'][imgid]['objects'].append({'bbox':bbox, 'category':category})\n",
109 |     "#         print(line)"
110 |    ]
111 |   },
112 |   {
113 |    "cell_type": "code",
114 |    "execution_count": null,
115 |    "metadata": {},
116 |    "outputs": [],
117 |    "source": [
118 |     "output_path = 'annos_' + group_path + '.json'\n",
119 |     "with open(output_path, 'w') as f:\n",
120 |     "    json.dump(annos, f)\n",
121 |     "    f.close()"
122 |    ]
123 |   },
124 |   {
125 |    "cell_type": "markdown",
126 |    "metadata": {},
127 |    "source": [
128 |     "# Generate annotations txt\n",
129 |     "生成rezise后对应的groud_truth的TXT文件"
130 |    ]
131 |   },
132 |   {
133 |    "cell_type": "code",
134 |    "execution_count": 17,
135 |    "metadata": {},
136 |    "outputs": [],
137 |    "source": [
138 |     "input_dir = './data_origin/newgt'\n",
139 |     "output_dir = 'data_resize/newgt'\n",
140 |     "group_path = 'group1'\n",
141 |     "gt_path = os.path.join(input_dir, group_path)\n",
142 |     "output_path = os.path.join(output_dir, group_path)\n",
143 |     "\n",
144 |     "if not os.path.exists(output_path):\n",
145 |     "    os.mkdir(output_path)\n",
146 |     "\n",
147 |     "for file in os.listdir(gt_path):\n",
148 |     "    with open(os.path.join(gt_path, file), 'r') as fin:\n",
149 |     "        lines = fin.readlines()\n",
150 |     "    with open(os.path.join(output_path, file), 'w') as fout:\n",
151 |     "        for line in lines:\n",
152 |     "            line = line.strip().split(' ')\n",
153 |     "            category = line[1][0]\n",
154 |     "            x_min = str(float(line[2]) / (1280/512))\n",
155 |     "            x_max = str(float(line[4]) / (1280/512))\n",
156 |     "            y_min = str(float(line[3]) / (1024/512))\n",
157 |     "            y_max = str(float(line[5]) / (1024/512))\n",
158 |     "            \n",
159 |     "            res = file+' '+category+\" \"+x_min+' '+y_min+' '+x_max+' '+y_max+'\\n'\n",
160 |     "            fout.write(res)"
161 |    ]
162 |   },
163 |   {
164 |    "cell_type": "markdown",
165 |    "metadata": {},
166 |    "source": [
167 |     "# Counter\n",
168 |     "统计annotations下每个类别的数目"
169 |    ]
170 |   },
171 |   {
172 |    "cell_type": "code",
173 |    "execution_count": 13,
174 |    "metadata": {},
175 |    "outputs": [],
176 |    "source": [
177 |     "import os\n",
178 |     "import json\n",
179 |     "from collections import Counter"
180 |    ]
181 |   },
182 |   {
183 |    "cell_type": "code",
184 |    "execution_count": 27,
185 |    "metadata": {},
186 |    "outputs": [],
187 |    "source": [
188 |     "datadir='../data'\n",
189 |     "filedir=os.path.join(datadir, 'annotations.json')\n",
190 |     "annos=json.loads(open(filedir).read())"
191 |    ]
192 |   },
193 |   {
194 |    "cell_type": "code",
195 |    "execution_count": 28,
196 |    "metadata": {},
197 |    "outputs": [],
198 |    "source": [
199 |     "class_counter=Counter()\n",
200 |     "classes=set()\n",
201 |     "for imgid in annos['imgs']:\n",
202 |     "    for boxes in annos['imgs'][imgid]['objects']:\n",
203 |     "        class_counter[boxes['category']]+=1\n",
204 |     "        classes.add(boxes['category'])\n",
205 |     "classes=list(classes)"
206 |    ]
207 |   },
208 |   {
209 |    "cell_type": "code",
210 |    "execution_count": 29,
211 |    "metadata": {},
212 |    "outputs": [
213 |     {
214 |      "name": "stdout",
215 |      "output_type": "stream",
216 |      "text": [
217 |       "Counter({'z': 9990, 'd': 5798, 's': 4313, 'j': 3097, 'l': 683})\n",
218 |       "['z', 's', 'j', 'd', 'l']\n"
219 |      ]
220 |     }
221 |    ],
222 |    "source": [
223 |     "print(class_counter)\n",
224 |     "print(classes)"
225 |    ]
226 |   },
227 |   {
228 |    "cell_type": "markdown",
229 |    "metadata": {},
230 |    "source": [
231 |     "# Merge\n",
232 |     "将两个annotation合并起来"
233 |    ]
234 |   },
235 |   {
236 |    "cell_type": "code",
237 |    "execution_count": 17,
238 |    "metadata": {},
239 |    "outputs": [],
240 |    "source": [
241 |     "import json\n",
242 |     "import os"
243 |    ]
244 |   },
245 |   {
246 |    "cell_type": "code",
247 |    "execution_count": 18,
248 |    "metadata": {},
249 |    "outputs": [],
250 |    "source": [
251 |     "anno1_path = '../data/annotation_aug.json'\n",
252 |     "anno2_path = '../data/annotation_sample.json'"
253 |    ]
254 |   },
255 |   {
256 |    "cell_type": "code",
257 |    "execution_count": 23,
258 |    "metadata": {},
259 |    "outputs": [
260 |     {
261 |      "name": "stdout",
262 |      "output_type": "stream",
263 |      "text": [
264 |       "7832\n",
265 |       "3000\n"
266 |      ]
267 |     }
268 |    ],
269 |    "source": [
270 |     "a1 = json.load(open(anno1_path, 'r'))\n",
271 |     "a2 = json.load(open(anno2_path, 'r'))\n",
272 |     "\n",
273 |     "print(len(a1['imgs']))\n",
274 |     "print(len(a2['imgs']))"
275 |    ]
276 |   },
277 |   {
278 |    "cell_type": "code",
279 |    "execution_count": 24,
280 |    "metadata": {},
281 |    "outputs": [
282 |     {
283 |      "name": "stdout",
284 |      "output_type": "stream",
285 |      "text": [
286 |       "10832\n"
287 |      ]
288 |     }
289 |    ],
290 |    "source": [
291 |     "a2['imgs'].update(a1['imgs'])\n",
292 |     "\n",
293 |     "print(len(a2['imgs']))"
294 |    ]
295 |   },
296 |   {
297 |    "cell_type": "code",
298 |    "execution_count": 26,
299 |    "metadata": {},
300 |    "outputs": [],
301 |    "source": [
302 |     "json.dump(a2, open('../data/annotations.json', 'w'))"
303 |    ]
304 |   },
305 |   {
306 |    "cell_type": "markdown",
307 |    "metadata": {},
308 |    "source": [
309 |     "# Transform\n",
310 |     "将Tsinghua_tencent_100K的标注转换成三类"
311 |    ]
312 |   },
313 |   {
314 |    "cell_type": "code",
315 |    "execution_count": null,
316 |    "metadata": {},
317 |    "outputs": [],
318 |    "source": [
319 |     "import json\n",
320 |     "import os"
321 |    ]
322 |   },
323 |   {
324 |    "cell_type": "code",
325 |    "execution_count": null,
326 |    "metadata": {},
327 |    "outputs": [],
328 |    "source": [
329 |     "datadir='../data'\n",
330 |     "sign_map = {'i':'s', 'p':'z', 'w':'j'}\n",
331 |     "anno_path = 'annotation.json'\n",
332 |     "target_path = 'annotation_3class.json'"
333 |    ]
334 |   },
335 |   {
336 |    "cell_type": "code",
337 |    "execution_count": null,
338 |    "metadata": {},
339 |    "outputs": [],
340 |    "source": [
341 |     "anno_file=os.path.join(datadir, anno_path)\n",
342 |     "annos=json.loads(open(anno_file).read())"
343 |    ]
344 |   },
345 |   {
346 |    "cell_type": "code",
347 |    "execution_count": null,
348 |    "metadata": {},
349 |    "outputs": [],
350 |    "source": [
351 |     "for id in annos['imgs']:\n",
352 |     "    for bi, box in enumerate(annos['imgs'][id]['objects']):\n",
353 |     "        category = box['category']\n",
354 |     "        new_category = sign_map[category[0]]\n",
355 |     "        annos['imgs'][id]['objects'][bi]['category'] = new_category"
356 |    ]
357 |   },
358 |   {
359 |    "cell_type": "code",
360 |    "execution_count": null,
361 |    "metadata": {},
362 |    "outputs": [],
363 |    "source": [
364 |     "with open(os.path.join(datadir, target_path), 'w') as f:\n",
365 |     "    json.dump(annos, f)\n",
366 |     "    f.close()"
367 |    ]
368 |   }
369 |  ],
370 |  "metadata": {
371 |   "kernelspec": {
372 |    "display_name": "Python 3",
373 |    "language": "python",
374 |    "name": "python3"
375 |   },
376 |   "language_info": {
377 |    "codemirror_mode": {
378 |     "name": "ipython",
379 |     "version": 3
380 |    },
381 |    "file_extension": ".py",
382 |    "mimetype": "text/x-python",
383 |    "name": "python",
384 |    "nbconvert_exporter": "python",
385 |    "pygments_lexer": "ipython3",
386 |    "version": "3.7.1"
387 |   },
388 |   "toc-autonumbering": false,
389 |   "toc-showcode": false,
390 |   "toc-showmarkdowntxt": false,
391 |   "toc-showtags": false
392 |  },
393 |  "nbformat": 4,
394 |  "nbformat_minor": 2
395 | }
396 | 


--------------------------------------------------------------------------------
/evaluate/crop_image.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "code",
  5 |    "execution_count": 1,
  6 |    "metadata": {
  7 |     "collapsed": false
  8 |    },
  9 |    "outputs": [],
 10 |    "source": [
 11 |     "import sys\n",
 12 |     "sys.path.append('../')\n",
 13 |     "import tensorlayer as tl\n",
 14 |     "import json\n",
 15 |     "import pylab as pl\n",
 16 |     "import random\n",
 17 |     "import numpy as np\n",
 18 |     "import os\n",
 19 |     "import cv2\n",
 20 |     "import anno_func\n",
 21 |     "%matplotlib inline\n",
 22 |     "from preprocessing.bbox import BoundingBox\n",
 23 |     "from preprocessing.errors import UnsupportedExtensionError, UnsupportedFormatError\n",
 24 |     "from PIL import Image\n",
 25 |     "from collections import Counter\n",
 26 |     "from tqdm import tqdm\n",
 27 |     "from PIL import ImageFile\n",
 28 |     "ImageFile.LOAD_TRUNCATED_IMAGES = True"
 29 |    ]
 30 |   },
 31 |   {
 32 |    "cell_type": "code",
 33 |    "execution_count": 2,
 34 |    "metadata": {
 35 |     "collapsed": false
 36 |    },
 37 |    "outputs": [],
 38 |    "source": [
 39 |     "datadir='../data'\n",
 40 |     "filedir=os.path.join(datadir, 'annotation_type5.json')\n",
 41 |     "annos=json.loads(open(filedir).read())"
 42 |    ]
 43 |   },
 44 |   {
 45 |    "cell_type": "code",
 46 |    "execution_count": 3,
 47 |    "metadata": {
 48 |     "collapsed": true
 49 |    },
 50 |    "outputs": [],
 51 |    "source": [
 52 |     "class_counter=Counter()\n",
 53 |     "classes=set()\n",
 54 |     "for imgid in annos['imgs']:\n",
 55 |     "    for boxes in annos['imgs'][imgid]['objects']:\n",
 56 |     "        class_counter[boxes['category']]+=1\n",
 57 |     "        classes.add(boxes['category'])\n",
 58 |     "classes=list(classes)"
 59 |    ]
 60 |   },
 61 |   {
 62 |    "cell_type": "code",
 63 |    "execution_count": 4,
 64 |    "metadata": {},
 65 |    "outputs": [
 66 |     {
 67 |      "name": "stdout",
 68 |      "output_type": "stream",
 69 |      "text": [
 70 |       "Counter({'d': 754, 'z': 623, 'l': 345, 's': 250, 'j': 244})\n",
 71 |       "['s', 'l', 'z', 'd', 'j']\n"
 72 |      ]
 73 |     }
 74 |    ],
 75 |    "source": [
 76 |     "print(class_counter)\n",
 77 |     "print(classes)"
 78 |    ]
 79 |   },
 80 |   {
 81 |    "cell_type": "code",
 82 |    "execution_count": 5,
 83 |    "metadata": {
 84 |     "collapsed": false
 85 |    },
 86 |    "outputs": [],
 87 |    "source": [
 88 |     "new_annos={}\n",
 89 |     "new_annos['imgs']={}\n",
 90 |     "new_annos['types']=anno_func.type5"
 91 |    ]
 92 |   },
 93 |   {
 94 |    "cell_type": "code",
 95 |    "execution_count": 6,
 96 |    "metadata": {
 97 |     "collapsed": false
 98 |    },
 99 |    "outputs": [],
100 |    "source": [
101 |     "def transform_box(box, new_xmin, new_ymin):\n",
102 |     "    box_dict={}\n",
103 |     "    box_dict['bbox']={'xmin':box['bbox']['xmin']-new_xmin, 'xmax':box['bbox']['xmax']-new_xmin,\n",
104 |     "                     'ymin':box['bbox']['ymin']-new_ymin, 'ymax':box['bbox']['ymax']-new_ymin}\n",
105 |     "    box_dict['category']=box['category']\n",
106 |     "    return box_dict\n",
107 |     "\n",
108 |     "def overlap(box, old_width, old_height,new_xmin, new_ymin, new_width, new_height,tol):\n",
109 |     "    xmin=max(box['bbox']['xmin'],0)\n",
110 |     "    ymin=max(box['bbox']['ymin'],0)\n",
111 |     "    xmax=min(box['bbox']['xmax'],old_width)\n",
112 |     "    ymax=min(box['bbox']['ymax'],old_height)\n",
113 |     "    return xmin>=new_xmin-tol and ymin>=new_ymin-tol and xmax-tol<=new_width+new_xmin and ymax-tol<=new_height+new_ymin\n",
114 |     "    \n",
115 |     "def crop_image(id,old_width=1280, old_height=1024, new_width=512, new_height=512,note='_1',cropped_dir='cropped_train',tol=8,aug=False):\n",
116 |     "    global new_annos\n",
117 |     "    \n",
118 |     "    \n",
119 |     "    ids=0\n",
120 |     "    if id not in annos['imgs']: return\n",
121 |     "    for box in annos['imgs'][id]['objects']:\n",
122 |     "        objects=[]\n",
123 |     "        \n",
124 |     "        label=box['category']\n",
125 |     "        if label not in classes:\n",
126 |     "            continue\n",
127 |     "        \n",
128 |     "        xmin=box['bbox']['xmin']\n",
129 |     "        ymin=box['bbox']['ymin']\n",
130 |     "        xmax=box['bbox']['xmax']\n",
131 |     "        ymax=box['bbox']['ymax']\n",
132 |     "        if(xmax-xmin>128 or ymax-ymin>128):\n",
133 |     "            continue\n",
134 |     "        new_id=id+'_'+str(ids)+note\n",
135 |     "        new_annos['imgs'][new_id]={}\n",
136 |     "        new_annos['imgs'][new_id]['id']=new_id\n",
137 |     "        path=id+'.jpg'\n",
138 |     "        new_xmin=np.random.randint(int(max(0,xmax-new_width)), max(0,int(xmin))+1)\n",
139 |     "        new_ymin=np.random.randint(int(max(0,ymax-new_height)),max(0,int(ymin))+1)\n",
140 |     "        type_path = 'train' if cropped_dir=='cropped_train' else 'test'\n",
141 |     "        image=Image.open(os.path.join(datadir, type_path, path))\n",
142 |     "        bottom=new_ymin+new_height\n",
143 |     "        right=new_xmin+new_width\n",
144 |     "        \n",
145 |     "        if right>=old_width:\n",
146 |     "            right=old_width\n",
147 |     "            new_xmin=right-new_width\n",
148 |     "        if bottom>=old_height:\n",
149 |     "            bottom=old_height\n",
150 |     "            new_ymin=bottom-new_height\n",
151 |     "        \n",
152 |     "        new_img=np.asarray(image.crop((new_xmin, new_ymin, right,bottom)))\n",
153 |     "        if aug:\n",
154 |     "            new_img=img = tl.prepro.illumination(new_img, gamma=(0.5, 1.5), \n",
155 |     "             contrast=(0.5, 1.5), saturation=(0.5, 1.5), is_random=True)\n",
156 |     "        \n",
157 |     "        new_img_path=new_id+'.jpg'\n",
158 |     "        tl.visualize.save_image(new_img, os.path.join(datadir, cropped_dir, new_img_path))\n",
159 |     "        \n",
160 |     "        for obox in annos['imgs'][id]['objects']:\n",
161 |     "            if obox['category'] not in classes:\n",
162 |     "                continue\n",
163 |     "            if overlap(obox, old_width, old_height,new_xmin, new_ymin, new_width, new_height,tol):\n",
164 |     "                box_dict=transform_box(obox, new_xmin, new_ymin)\n",
165 |     "                objects.append(box_dict)\n",
166 |     "       \n",
167 |     "        \n",
168 |     "        \n",
169 |     "        new_annos['imgs'][new_id]['objects']=objects\n",
170 |     "        new_annos['imgs'][new_id]['path']=os.path.join(cropped_dir,new_img_path)\n",
171 |     "        ids+=1"
172 |    ]
173 |   },
174 |   {
175 |    "cell_type": "code",
176 |    "execution_count": 7,
177 |    "metadata": {
178 |     "collapsed": false
179 |    },
180 |    "outputs": [],
181 |    "source": [
182 |     "train_dir=os.path.join(datadir,'train')\n",
183 |     "test_dir=os.path.join(datadir,'test')\n",
184 |     "train_id_list=list(map(lambda x:x.split('.')[0],os.listdir(train_dir)))\n",
185 |     "test_id_list=list(map(lambda x:x.split('.')[0],os.listdir(test_dir)))\n",
186 |     "cropped_dir_train='cropped_train'\n",
187 |     "cropped_dir_test='cropped_test'\n",
188 |     "train_new_path=os.path.join(datadir, cropped_dir_train)\n",
189 |     "test_new_path=os.path.join(datadir, cropped_dir_test)\n",
190 |     "if not os.path.exists(train_new_path):\n",
191 |     "    os.mkdir(train_new_path)\n",
192 |     "if not os.path.exists(test_new_path):\n",
193 |     "    os.mkdir(test_new_path)"
194 |    ]
195 |   },
196 |   {
197 |    "cell_type": "code",
198 |    "execution_count": 8,
199 |    "metadata": {
200 |     "collapsed": true
201 |    },
202 |    "outputs": [
203 |     {
204 |      "name": "stderr",
205 |      "output_type": "stream",
206 |      "text": [
207 |       "100%|██████████| 250/250 [00:22<00:00, 15.35it/s]\n"
208 |      ]
209 |     }
210 |    ],
211 |    "source": [
212 |     "for id in tqdm(test_id_list):\n",
213 |     "    crop_image(id,1280,1024,512,512,'_1',cropped_dir=cropped_dir_test)"
214 |    ]
215 |   },
216 |   {
217 |    "cell_type": "code",
218 |    "execution_count": 8,
219 |    "metadata": {
220 |     "collapsed": false
221 |    },
222 |    "outputs": [
223 |     {
224 |      "name": "stderr",
225 |      "output_type": "stream",
226 |      "text": [
227 |       "100%|██████████| 1250/1250 [01:50<00:00, 10.95it/s]\n"
228 |      ]
229 |     }
230 |    ],
231 |    "source": [
232 |     "for id in tqdm(train_id_list):\n",
233 |     "    crop_image(id,1280,1024,512,512,'_1',cropped_dir=cropped_dir_train)"
234 |    ]
235 |   },
236 |   {
237 |    "cell_type": "code",
238 |    "execution_count": 9,
239 |    "metadata": {
240 |     "collapsed": true
241 |    },
242 |    "outputs": [],
243 |    "source": [
244 |     "def Data_augmentation(num):\n",
245 |     "    keys=list(new_annos['imgs'].keys())\n",
246 |     "    for imgid in tqdm(keys):\n",
247 |     "        if new_annos['imgs'][imgid]['path'].split('/')[0]==cropped_dir_test:\n",
248 |     "            continue\n",
249 |     "        for box in new_annos['imgs'][imgid]['objects']:\n",
250 |     "            category=box['category']\n",
251 |     "            if class_counter[category]<num:\n",
252 |     "                org_id=imgid.split('_')[0]\n",
253 |     "                n=num//class_counter[category]+1\n",
254 |     "                for k in range(n):\n",
255 |     "                    crop_image(org_id, 1280,1024,512,512, note='_'+str(k+2),cropped_dir=cropped_dir_train, aug=True)"
256 |    ]
257 |   },
258 |   {
259 |    "cell_type": "code",
260 |    "execution_count": 10,
261 |    "metadata": {},
262 |    "outputs": [
263 |     {
264 |      "name": "stderr",
265 |      "output_type": "stream",
266 |      "text": [
267 |       "100%|██████████| 1964/1964 [58:22<00:00,  1.55s/it]\n"
268 |      ]
269 |     }
270 |    ],
271 |    "source": [
272 |     "Data_augmentation(1000)"
273 |    ]
274 |   },
275 |   {
276 |    "cell_type": "code",
277 |    "execution_count": 11,
278 |    "metadata": {
279 |     "collapsed": true
280 |    },
281 |    "outputs": [],
282 |    "source": [
283 |     "datadir = '../data'\n",
284 |     "output_path = 'annotation_aug.json'\n",
285 |     "\n",
286 |     "with open(os.path.join(datadir, output_path), 'w') as f:\n",
287 |     "    json.dump(new_annos, f)\n",
288 |     "    f.close()"
289 |    ]
290 |   },
291 |   {
292 |    "cell_type": "code",
293 |    "execution_count": 41,
294 |    "metadata": {},
295 |    "outputs": [],
296 |    "source": [
297 |     "anno_file=os.path.join(datadir,'annotation_aug.json')\n",
298 |     "new_annos=json.loads(open(anno_file).read())"
299 |    ]
300 |   },
301 |   {
302 |    "cell_type": "code",
303 |    "execution_count": null,
304 |    "metadata": {},
305 |    "outputs": [],
306 |    "source": []
307 |   }
308 |  ],
309 |  "metadata": {
310 |   "anaconda-cloud": {},
311 |   "kernelspec": {
312 |    "display_name": "py36",
313 |    "language": "python",
314 |    "name": "py36"
315 |   },
316 |   "language_info": {
317 |    "codemirror_mode": {
318 |     "name": "ipython",
319 |     "version": 3
320 |    },
321 |    "file_extension": ".py",
322 |    "mimetype": "text/x-python",
323 |    "name": "python",
324 |    "nbconvert_exporter": "python",
325 |    "pygments_lexer": "ipython3",
326 |    "version": "3.6.2"
327 |   }
328 |  },
329 |  "nbformat": 4,
330 |  "nbformat_minor": 2
331 | }
332 | 


--------------------------------------------------------------------------------
/evaluate/data_process.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "code",
  5 |    "execution_count": null,
  6 |    "metadata": {},
  7 |    "outputs": [],
  8 |    "source": [
  9 |     "import os\n",
 10 |     "import cv2\n",
 11 |     "from PIL import Image\n",
 12 |     "import numpy as np\n",
 13 |     "import json\n",
 14 |     "import matplotlib.pyplot as plt"
 15 |    ]
 16 |   },
 17 |   {
 18 |    "cell_type": "markdown",
 19 |    "metadata": {},
 20 |    "source": [
 21 |     "# Clear\n",
 22 |     "清理不符合要求的数据，并删除对应的图片和ground_truth文件。"
 23 |    ]
 24 |   },
 25 |   {
 26 |    "cell_type": "code",
 27 |    "execution_count": null,
 28 |    "metadata": {},
 29 |    "outputs": [],
 30 |    "source": [
 31 |     "group_path = 'group1'\n",
 32 |     "\n",
 33 |     "input_path = os.path.join('./data_origin/picwithgt', group_path)\n",
 34 |     "target_gt_path = os.path.join('./data_origin/newgt', group_path)\n",
 35 |     "target_pic_path = os.path.join('./data_origin/newpic', group_path)"
 36 |    ]
 37 |   },
 38 |   {
 39 |    "cell_type": "code",
 40 |    "execution_count": null,
 41 |    "metadata": {},
 42 |    "outputs": [],
 43 |    "source": [
 44 |     "files = []\n",
 45 |     "for file in os.listdir(input_path): \n",
 46 |     "    file_name = file.split('-')[1].split('.')[0]\n",
 47 |     "    files.append(file_name)\n",
 48 |     "\n",
 49 |     "print(len(files))"
 50 |    ]
 51 |   },
 52 |   {
 53 |    "cell_type": "code",
 54 |    "execution_count": null,
 55 |    "metadata": {},
 56 |    "outputs": [],
 57 |    "source": [
 58 |     "num = 0\n",
 59 |     "for file in os.listdir(target_gt_path): \n",
 60 |     "    file_name = file.split('-')[1].split('.')[0]\n",
 61 |     "    if file_name not in files:\n",
 62 |     "        os.remove(os.path.join(target_gt_path, file))\n",
 63 |     "        print(file)\n",
 64 |     "        num += 1\n",
 65 |     "print(num)"
 66 |    ]
 67 |   },
 68 |   {
 69 |    "cell_type": "code",
 70 |    "execution_count": null,
 71 |    "metadata": {},
 72 |    "outputs": [],
 73 |    "source": [
 74 |     "num = 0\n",
 75 |     "for file in os.listdir(target_pic_path): \n",
 76 |     "    file_name = file.split('-')[1].split('.')[0]\n",
 77 |     "    if file_name not in files:\n",
 78 |     "        os.remove(os.path.join(target_pic_path, file))\n",
 79 |     "        print(file)\n",
 80 |     "        num += 1\n",
 81 |     "print(num)"
 82 |    ]
 83 |   },
 84 |   {
 85 |    "cell_type": "markdown",
 86 |    "metadata": {},
 87 |    "source": [
 88 |     "# Images process\n",
 89 |     "- 原始数据中存在着部分图片的分辨率为1920*1080.\n",
 90 |     "- 部分数据虽然分辨率为1280*1024，但是四周存在着黑框，对训练会有影响。\n",
 91 |     "- 将原始图片以及检测框的ground_truth缩放到512*512大小"
 92 |    ]
 93 |   },
 94 |   {
 95 |    "cell_type": "code",
 96 |    "execution_count": null,
 97 |    "metadata": {},
 98 |    "outputs": [],
 99 |    "source": [
100 |     "TARGET_W = 1280\n",
101 |     "TARGET_H = 1024\n",
102 |     "\n",
103 |     "pic_train_path = '../data/train'\n",
104 |     "pic_test_path = '../data/test'\n",
105 |     "\n",
106 |     "annos_path = '../data/annotation_type5.json'\n",
107 |     "annos = json.load(open(annos_path, 'r'))"
108 |    ]
109 |   },
110 |   {
111 |    "cell_type": "code",
112 |    "execution_count": null,
113 |    "metadata": {},
114 |    "outputs": [],
115 |    "source": [
116 |     "def annos_resize(imgid, old_W, old_H, new_W=1280, new_H=1024):\n",
117 |     "    print(imgid, old_W, old_H)\n",
118 |     "    x_ratio = old_W / new_W\n",
119 |     "    y_ratio = old_H / new_H\n",
120 |     "    objects = annos['imgs'][imgid]['objects']\n",
121 |     "    for bid, bboxs in enumerate(objects):\n",
122 |     "        bbox = bboxs['bbox']\n",
123 |     "        print('origin: ', bbox)\n",
124 |     "        bbox['xmin'] = bbox['xmin'] / x_ratio\n",
125 |     "        bbox['xmax'] = bbox['xmax'] / x_ratio\n",
126 |     "        bbox['ymin'] = bbox['ymin'] / y_ratio\n",
127 |     "        bbox['ymax'] = bbox['ymax'] / y_ratio\n",
128 |     "        print('resize: ', bbox)\n",
129 |     "        annos['imgs'][imgid]['objects'][bid]['bbox'] = bbox\n",
130 |     "        "
131 |    ]
132 |   },
133 |   {
134 |    "cell_type": "code",
135 |    "execution_count": null,
136 |    "metadata": {},
137 |    "outputs": [],
138 |    "source": [
139 |     "def data_clear(path):\n",
140 |     "    num = 0\n",
141 |     "    for file in os.listdir(path):\n",
142 |     "        file_path = os.path.join(path, file)\n",
143 |     "#         img = cv2.imread(file_path)\n",
144 |     "        img = Image.open(file_path)\n",
145 |     "        img = np.array(img)\n",
146 |     "        if img.ndim == 2:\n",
147 |     "            print(file)\n",
148 |     "            os.remove(file_path)\n",
149 |     "            num += 1\n",
150 |     "    print('total resize num: ', num)"
151 |    ]
152 |   },
153 |   {
154 |    "cell_type": "code",
155 |    "execution_count": null,
156 |    "metadata": {},
157 |    "outputs": [],
158 |    "source": [
159 |     "data_clear(pic_train_path)"
160 |    ]
161 |   },
162 |   {
163 |    "cell_type": "code",
164 |    "execution_count": null,
165 |    "metadata": {},
166 |    "outputs": [],
167 |    "source": [
168 |     "data_clear(pic_test_path)"
169 |    ]
170 |   },
171 |   {
172 |    "cell_type": "code",
173 |    "execution_count": null,
174 |    "metadata": {},
175 |    "outputs": [],
176 |    "source": [
177 |     "file_path = os.path.join(pic_train_path, 'group4-31_1_2.jpg')\n",
178 |     "# img = cv2.imread(file_path)\n",
179 |     "img = Image.open(file_path)\n",
180 |     "img = np.array(img)\n",
181 |     "\n",
182 |     "print(img.ndim)\n",
183 |     "plt.imshow(img)"
184 |    ]
185 |   },
186 |   {
187 |    "cell_type": "markdown",
188 |    "metadata": {},
189 |    "source": [
190 |     "# Check\n",
191 |     "测试resize之后的检测框框是否正常"
192 |    ]
193 |   },
194 |   {
195 |    "cell_type": "code",
196 |    "execution_count": null,
197 |    "metadata": {},
198 |    "outputs": [],
199 |    "source": [
200 |     "imgid = 'group1-282'\n",
201 |     "img = cv2.imread('./data_tmp/train/'+imgid+'.jpg')\n",
202 |     "\n",
203 |     "objects = annos['imgs'][imgid]['objects']\n",
204 |     "for bid, bboxs in enumerate(objects):\n",
205 |     "    bbox = bboxs['bbox']\n",
206 |     "    category = bboxs['category']\n",
207 |     "    x_min = int(bbox['xmin'])\n",
208 |     "    x_max = int(bbox['xmax'])\n",
209 |     "    y_min = int(bbox['ymin'])\n",
210 |     "    y_max = int(bbox['ymax'])\n",
211 |     "    print(category, x_min, x_max, y_min, y_max)\n",
212 |     "    cv2.rectangle(img, (x_min, y_min), (x_max, y_max), (0, 255, 0), 2)\n",
213 |     "    cv2.putText(img, category, (int(x_min), int(y_min - 6)), cv2.FONT_HERSHEY_COMPLEX_SMALL, 0.8, (255, 255, 255))\n",
214 |     "\n",
215 |     "plt.imshow(img)\n",
216 |     "plt.show()"
217 |    ]
218 |   },
219 |   {
220 |    "cell_type": "markdown",
221 |    "metadata": {},
222 |    "source": [
223 |     "# Sample\n",
224 |     "从Tsinghua_tencent_100K中，每个类别随机选择1000张图片加入训练集中"
225 |    ]
226 |   },
227 |   {
228 |    "cell_type": "code",
229 |    "execution_count": null,
230 |    "metadata": {},
231 |    "outputs": [],
232 |    "source": [
233 |     "import numpy as np\n",
234 |     "import os\n",
235 |     "import json\n",
236 |     "import shutil"
237 |    ]
238 |   },
239 |   {
240 |    "cell_type": "code",
241 |    "execution_count": null,
242 |    "metadata": {},
243 |    "outputs": [],
244 |    "source": [
245 |     "annos_path = '../data/type3/annotation_3class.json'\n",
246 |     "train_path = '../data/type3/train'\n",
247 |     "pic_output_path = '../data/type3/samples'\n",
248 |     "annos_output_path = '../data/type3/annotation_sample.json'"
249 |    ]
250 |   },
251 |   {
252 |    "cell_type": "code",
253 |    "execution_count": null,
254 |    "metadata": {},
255 |    "outputs": [],
256 |    "source": [
257 |     "annos = json.load(open(annos_path, 'r'))\n",
258 |     "if os.path.exists(pic_output_path):\n",
259 |     "    shutil.rmtree(pic_output_path)\n",
260 |     "    os.mkdir(pic_output_path)"
261 |    ]
262 |   },
263 |   {
264 |    "cell_type": "code",
265 |    "execution_count": null,
266 |    "metadata": {},
267 |    "outputs": [],
268 |    "source": [
269 |     "cnt = {'z':0, 'j':0, 's':0}\n",
270 |     "num = 1000\n",
271 |     "\n",
272 |     "annos_new={}\n",
273 |     "annos_new['imgs']={}\n",
274 |     "\n",
275 |     "for file in os.listdir(train_path):\n",
276 |     "    imgid = file.split('.')[0]\n",
277 |     "    objects = annos['imgs'][imgid]['objects']\n",
278 |     "    for bid, bboxs in enumerate(objects):\n",
279 |     "        category = bboxs['category']\n",
280 |     "        if cnt[category] < num:\n",
281 |     "            print(file)\n",
282 |     "            cnt[category] += 1\n",
283 |     "            shutil.copy(train_path+'/'+file, pic_output_path+'/')\n",
284 |     "            annos_new['imgs'][imgid] = annos['imgs'][imgid]\n",
285 |     "            break"
286 |    ]
287 |   },
288 |   {
289 |    "cell_type": "code",
290 |    "execution_count": null,
291 |    "metadata": {},
292 |    "outputs": [],
293 |    "source": [
294 |     "cnt"
295 |    ]
296 |   },
297 |   {
298 |    "cell_type": "code",
299 |    "execution_count": null,
300 |    "metadata": {},
301 |    "outputs": [],
302 |    "source": [
303 |     "with open(annos_output_path, 'w') as f:\n",
304 |     "    json.dump(annos_new, f)\n",
305 |     "    f.close()"
306 |    ]
307 |   },
308 |   {
309 |    "cell_type": "markdown",
310 |    "metadata": {
311 |     "toc-hr-collapsed": true
312 |    },
313 |    "source": [
314 |     "# Data Set Partitioning\n",
315 |     "从数据集中随机划分1000张到测试集"
316 |    ]
317 |   },
318 |   {
319 |    "cell_type": "code",
320 |    "execution_count": null,
321 |    "metadata": {},
322 |    "outputs": [],
323 |    "source": [
324 |     "import os\n",
325 |     "import json\n",
326 |     "import random\n",
327 |     "import shutil"
328 |    ]
329 |   },
330 |   {
331 |    "cell_type": "code",
332 |    "execution_count": null,
333 |    "metadata": {},
334 |    "outputs": [],
335 |    "source": [
336 |     "ipic_dist = {}\n",
337 |     "train_path = '../data/train/'\n",
338 |     "test_path = '../data/test/'\n",
339 |     "data_num = 0\n",
340 |     "test_num = 1000\n",
341 |     "for i, ipic in enumerate(os.listdir(test_path)):\n",
342 |     "    ipic_dist[i] = ipic\n",
343 |     "    data_num += 1\n",
344 |     "#     print(i, ipic)\n",
345 |     "# print(data_num)"
346 |    ]
347 |   },
348 |   {
349 |    "cell_type": "code",
350 |    "execution_count": null,
351 |    "metadata": {},
352 |    "outputs": [],
353 |    "source": [
354 |     "rand_list = []\n",
355 |     "for i in range(test_num):\n",
356 |     "    ran = random.randint(0, data_num-1)\n",
357 |     "    while ran in rand_list:\n",
358 |     "        ran = random.randint(0, data_num-1)\n",
359 |     "    rand_list.append(ran)"
360 |    ]
361 |   },
362 |   {
363 |    "cell_type": "code",
364 |    "execution_count": null,
365 |    "metadata": {},
366 |    "outputs": [],
367 |    "source": [
368 |     "for i in rand_list:\n",
369 |     "    try:\n",
370 |     "        shutil.move(train_path + ipic_dist[i], test_path + ipic_dist[i])\n",
371 |     "        print(ipic_dist[i])\n",
372 |     "    except:\n",
373 |     "        pass"
374 |    ]
375 |   },
376 |   {
377 |    "cell_type": "markdown",
378 |    "metadata": {},
379 |    "source": [
380 |     "# Resize\n",
381 |     "将所有测试集resize到512*512大小"
382 |    ]
383 |   },
384 |   {
385 |    "cell_type": "code",
386 |    "execution_count": 21,
387 |    "metadata": {},
388 |    "outputs": [],
389 |    "source": [
390 |     "import os\n",
391 |     "import cv2\n",
392 |     "import numpy as np\n",
393 |     "from PIL import Image"
394 |    ]
395 |   },
396 |   {
397 |    "cell_type": "code",
398 |    "execution_count": 22,
399 |    "metadata": {},
400 |    "outputs": [],
401 |    "source": [
402 |     "WIDTH = 512\n",
403 |     "HEIGHT = 512\n",
404 |     "\n",
405 |     "test_path = '../data/final_test'\n",
406 |     "output_path = '../data/final_test_resize'\n",
407 |     "\n",
408 |     "assert os.path.exists(test_path)\n",
409 |     "if not os.path.exists(output_path):\n",
410 |     "    os.mkdir(output_path)"
411 |    ]
412 |   },
413 |   {
414 |    "cell_type": "code",
415 |    "execution_count": 24,
416 |    "metadata": {},
417 |    "outputs": [
418 |     {
419 |      "name": "stdout",
420 |      "output_type": "stream",
421 |      "text": [
422 |       "group3-470.png 1174 938\n",
423 |       "group3-494.png 1174 938\n",
424 |       "group3-486.png 1174 938\n",
425 |       "group3-474.png 1174 938\n",
426 |       "group3-479.png 1174 938\n"
427 |      ]
428 |     }
429 |    ],
430 |    "source": [
431 |     "for file in os.listdir(test_path):\n",
432 |     "#     img = Image.open(os.path.join(test_path, file))\n",
433 |     "    img = cv2.imread(os.path.join(test_path, file))\n",
434 |     "    height, width = img.shape[:2]\n",
435 |     "    print(file, width, height)\n",
436 |     "    if width != WIDTH or height != HEIGHT:\n",
437 |     "        img = cv2.resize(img, (WIDTH, HEIGHT))\n",
438 |     "        cv2.imwrite(os.path.join(output_path, file), img)"
439 |    ]
440 |   },
441 |   {
442 |    "cell_type": "code",
443 |    "execution_count": null,
444 |    "metadata": {},
445 |    "outputs": [],
446 |    "source": []
447 |   }
448 |  ],
449 |  "metadata": {
450 |   "kernelspec": {
451 |    "display_name": "Python 3",
452 |    "language": "python",
453 |    "name": "python3"
454 |   },
455 |   "language_info": {
456 |    "codemirror_mode": {
457 |     "name": "ipython",
458 |     "version": 3
459 |    },
460 |    "file_extension": ".py",
461 |    "mimetype": "text/x-python",
462 |    "name": "python",
463 |    "nbconvert_exporter": "python",
464 |    "pygments_lexer": "ipython3",
465 |    "version": "3.7.1"
466 |   }
467 |  },
468 |  "nbformat": 4,
469 |  "nbformat_minor": 2
470 | }
471 | 


--------------------------------------------------------------------------------
/evaluate/eval_check.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "code",
  5 |    "execution_count": 1,
  6 |    "metadata": {
  7 |     "slideshow": {
  8 |      "slide_type": "-"
  9 |     }
 10 |    },
 11 |    "outputs": [],
 12 |    "source": [
 13 |     "import numpy as np\n",
 14 |     "import matplotlib.pyplot as plt\n",
 15 |     "import pylab as pl\n",
 16 |     "import os\n",
 17 |     "import sys\n",
 18 |     "import time\n",
 19 |     "import anno_func, json\n",
 20 |     "import random\n",
 21 |     "import copy\n",
 22 |     "from collections import Counter\n",
 23 |     "from math import fabs\n",
 24 |     "from PIL import Image\n",
 25 |     "import pandas as pd\n",
 26 |     "from IPython.display import display\n",
 27 |     "\n",
 28 |     "%matplotlib inline"
 29 |    ]
 30 |   },
 31 |   {
 32 |    "cell_type": "code",
 33 |    "execution_count": 2,
 34 |    "metadata": {},
 35 |    "outputs": [],
 36 |    "source": [
 37 |     "datadir = \"../data\"\n",
 38 |     "\n",
 39 |     "filedir = os.path.join(datadir,\"valid_target.json\")\n",
 40 |     "annos = json.loads(open(filedir).read())\n",
 41 |     "\n",
 42 |     "result_anno_file=os.path.join(datadir,\"resnet101_predict.json\")\n",
 43 |     "results_annos=json.loads(open(result_anno_file).read())"
 44 |    ]
 45 |   },
 46 |   {
 47 |    "cell_type": "code",
 48 |    "execution_count": 3,
 49 |    "metadata": {},
 50 |    "outputs": [
 51 |     {
 52 |      "name": "stdout",
 53 |      "output_type": "stream",
 54 |      "text": [
 55 |       "1000\n",
 56 |       "iou:0.5, size:[0,512), types:[d, ...total 5...], accuracy:0.865326633166, recall:0.787740164684\n"
 57 |      ]
 58 |     }
 59 |    ],
 60 |    "source": [
 61 |     "print (len(results_annos['imgs']))\n",
 62 |     "sm = anno_func.eval_annos(annos, results_annos, iou=0.5,types=anno_func.type5,minscore=50,check_type=True,minboxsize=0, maxboxsize=512)\n",
 63 |     "print sm['report']"
 64 |    ]
 65 |   },
 66 |   {
 67 |    "cell_type": "code",
 68 |    "execution_count": 4,
 69 |    "metadata": {},
 70 |    "outputs": [
 71 |     {
 72 |      "data": {
 73 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHRdJREFUeJzt3XuUHWWd7vHvQ7hf5RIjl4QEjWhwHNQMR2XGYcQLghoc\nXRAvCIpGHEDxyMKAzoBzZAzL29E5w2jkkggMGFEgXgbBHIXDjApBg0AAySIBEhISRAVEkctz/qi3\nk02ndvfupPelu5/PWr266q3aVb9du3r/+n3fqrdkm4iIiP626HYAERHRm5IgIiKiVhJERETUSoKI\niIhaSRAREVErCSIiImolQXSBpLMkXdymbR8n6YaG+cck7TdM2z5D0nllerIkS9pymLY9qcQ6bji2\nN4T9TpB0vaRHJX2hZvk8SZ/pZEz99v9uSdcM8zbXf46jiaRDJK3swn5XSHpdp/fbCcPyxx3PJumx\nhtntgSeAp8v8hzoZi+0dB1tH0iHAxbb3GWRb/zJccUlaAXzA9o/Ktu8DBo21DWYBDwE7uwdvCrJ9\nCXDJMG9z2D7HkUSSgam2l23GNuYBK21/atgC62GpQbSB7R37foD7gLc0lA3rH3unDFdNoQftCyzt\nxeQw0o20c2akxdsJSRDds7Wkb5SmjdslTe9bIGkvSd+WtE7SckkfabYRSbtLWijpEUk3As/vt9yS\nXlCmD5e0tOxzlaRTJe0A/CewV2nieazs/yxJl0u6WNIjwHFNmsbeL+kBSaslndqw32c1zTRW/yVd\nBEwCvlv2d1r/JqsSw0JJD0taJumDDds6S9KCZsev5hi9WtJNkn5ffr+6L0bgWOC0EkezZoI9JF1b\n9nWdpH1b2PZuklZKekuZ37G8j/c2ifE4SfeUfSyX9O6G8hvKdF+cfT9PlveApF0knV8+h1WSPtOs\nua7xc2w47sdKuk/SQ5I+OcCx3F3Sd8v5dlPZT2OTpiWdKOlu4O6BjlFZ9qzmmaHEJmm7cp79VtJS\n4K8GiPv6MnlLOXZH952Tkj4haQ1wofo10Ta8pxdImgW8mw3ny3cbVjtQ0q/Ke/ympG2bxTKi2M5P\nG3+AFcDr+pWdBfwJOBwYB3wW+FlZtgVwM/BPwNbAfsA9wBubbP8yYAGwA/ASYBVwQ8NyAy8o06uB\nvynTuwIvL9OHUFWb+8f4JHBkiWm7UnZxWT65bPvSsu+/ANb1vVdgHvCZhu09ax/9j0vD9rYs89cD\n5wLbAgeWbb92sONXc3x2A34LHEPVpPrOMr97XZw1r58HPAq8BtgG+HLf8W1h228A1gDPBb4OXN5k\nHzsAjwD7l/k9gQPK9HGNn2fDayYCDwBvKvNXAF8r23oucCPwoSb7q/scv14+47+kahJ98QDn22VU\nTafTgPvZ+Hy7thyb7Vo4Rv3Pg5ZjA+YA/6/sYyJwG/3O436xr/9baDgnnwLOKZ/tdnXHm2f/DW10\nvpT3cCOwV4nlDuCEbn/3DMdPahDdc4PtH9h+GriI6uSH6r+g8bb/2fafbd9D9Qcys/8Gyn+Ibwf+\nyfYfbN8GzB9gn08C0yTtbPu3tn8xSIw/tX2l7Wds/7HJOp8u+74VuJDqC2CzSJoIHAx8wvafbC8B\nzgMa//tudvz6OwK42/ZFtp+yfSlwJ/CWIYT0fdvX234C+CTwqhLjgNu2fQ3wLWARVTIbqP/pGeAl\nkrazvdr27c1WlLQdcCXwZdv/KWlC2f4p5bNYC3yJmnNmAJ+2/UfbtwC3UHM8G863M20/bnsp9efb\nZ20/XM6Z4Tj+zWI7Cji77Ot+4CtD2GafZ8r7eWKAc7wVX7H9gO2Hge9S/VMz4iVBdM+ahunHgW1L\n88q+VM09v+v7Ac4AJtRsYzzVf2X3N5TdO8A+3071RXJvaSp51SAx3j/I8v7r3Ev1X9Tm2gt42Paj\n/ba9d8N8s+NXt63+x6T/tgaz/j3afgx4uGy3lW3PparZzbP9m7qN2/4DcDRwArBa0vclvWiAeM4H\n7rJ9TpnfF9iqvLbvnPkaVU2iVf2PZ90FA3XnW9050lg2HMe/WWx70fq538w623/ahNf118rxG3GS\nIHrP/cBy289p+NnJ9uE1666jqiJPbCib1GzDtm+yPYPqi+NKqqYpqKrQtS9pId7++36gTP+Bqhmi\nz/OGsO0HgN0k7dRv26taiKduW/v2Kxvqtta/R0k7UjUjPDDYtst/3HOBbwD/oNIXVMf2D22/nqp5\n6U6qWuNGJM0GXggc31B8P1XTyx4N58zOtg8YwntsRd/51ni128Sa9Ro/28GO/2DnyUBW0+K5P4D+\n5+Gz4pE0lPN21EmC6D03Ao+WjrPtJI2T9BJJG3XAleaV7wBnSdpe0jSqTteNSNpa1TX1u9h+kqrN\n+5my+EFgd0m7bEK8/1j2fQDwPuCbpXwJcHjprH0ecEq/1z1I1b+ykdJc8N/AZyVtK+mlVF+Im3Lv\nyA+AF0p6l6QtJR1N1Xb+vSFs43BJfy1pa+B/UfV33N/Cts+g+kJ5P/A54Bt1Hceq7sWYoeqCgSeA\nx9jw2TSu9ybgI8DbGptDbK8GrgG+IGlnSVtIer6kvx3CexxUzfn2Ip7d7FdnsGO0BJgpaStVFxq8\nYwghLQBOl7SrpH2AkwdZv+k51+AW4ABJB5aO5rM2YRujRhJEjyl/hG+masNcTnWN/nlAsy/vk6iq\ns2uoOtAuHGDzxwArVF2VdALVFRnYvpOqs/me0kQxlGai64BlVO3sny/t7lD1C9xC1YF3DRsSR5/P\nAp8q+zuVjb2TqpPyAaoO2DNd7pkYitKs82bg48BvgNOAN9t+aAib+Q/gTKqmpVcA7xls25JeAfxP\n4L3lMz2HKlnMrtn+FmXdB8o+/hb4cM16R1M189yhDVcyfbUsey/VRQ1LqTqBL6eqjQy3k6jOxTVU\nn/GlVEmtVgvH/x+prrz7LfBpqmPdqk9TNSstpzrHLhpk/bOA+eWcO6pJvL8G/hn4EdVVWDf0W+V8\nqn6830m6cgixjkiyx1SNKSKGkaRzgOfZrq25xsiWGkREtEzSiyS9VJWDqJr+ruh2XNEeuXMwIoZi\nJ6pmpb2o2uO/AFzV1YiibdLEFBERtdLEFBERtUZ0E9Mee+zhyZMndzuMiIgR5eabb37I9vjB1hvR\nCWLy5MksXry422FERIwoklq66zxNTBERUSsJIiIiaiVBRERErSSIiIiolQQRERG1kiAiIqJWEkRE\nRNRKgoiIiFpJEBERUWtE30ndiybP/v766RVzjuhiJBERmyc1iIiIqJUEERERtZIgIiKiVhJERETU\nSoKIiIhaSRAREVErCSIiImolQURERK0kiIiIqNW2BCHpAklrJd1Ws+zjkixpj4ay0yUtk3SXpDe2\nK66IiGhNO2sQ84DD+hdKmgi8AbivoWwaMBM4oLzmXEnj2hhbREQMom0Jwvb1wMM1i74EnAa4oWwG\ncJntJ2wvB5YBB7UrtoiIGFxH+yAkzQBW2b6l36K9gfsb5leWsoiI6JKOjeYqaXvgDKrmpc3Zzixg\nFsCkSZOGIbKIiKjTyRrE84EpwC2SVgD7AL+Q9DxgFTCxYd19StlGbM+1Pd329PHjx7c55IiIsatj\nCcL2rbafa3uy7clUzUgvt70GWAjMlLSNpCnAVODGTsUWEREba+dlrpcCPwX2l7RS0vHN1rV9O7AA\nWApcDZxo++l2xRYREYNrWx+E7XcOsnxyv/mzgbPbFU9ERAxN7qSOiIhaSRAREVErCSIiImolQURE\nRK0kiIiIqJUEERERtZIgIiKiVhJERETUSoKIiIhaSRAREVGrY8N9j0WTZ39//fSKOUd0MZKIiKFL\nDSIiImolQURERK0kiIiIqJUEERERtZIgIiKiVhJERETUSoKIiIhaSRAREVGrbQlC0gWS1kq6raHs\nc5LulPQrSVdIek7DstMlLZN0l6Q3tiuuiIhoTTtrEPOAw/qVXQu8xPZLgV8DpwNImgbMBA4orzlX\n0rg2xhYREYNoW4KwfT3wcL+ya2w/VWZ/BuxTpmcAl9l+wvZyYBlwULtii4iIwXVzLKb3A98s03tT\nJYw+K0vZRiTNAmYBTJo0qZ3xtaxxzKWIiNGiK53Ukj4JPAVcMtTX2p5re7rt6ePHjx/+4CIiAuhC\nDULSccCbgUNtuxSvAiY2rLZPKYuIiC7paA1C0mHAacBbbT/esGghMFPSNpKmAFOBGzsZW0REPFvb\nahCSLgUOAfaQtBI4k+qqpW2AayUB/Mz2CbZvl7QAWErV9HSi7afbFVtERAyubQnC9jtris8fYP2z\ngbPbFU9ERAxN7qSOiIhaSRAREVErCSIiImolQURERK0kiIiIqJUEERERtZIgIiKiVjcH6xvRMkBf\nRIx2qUFEREStJIiIiKiVBBEREbWSICIiolYSRERE1EqCiIiIWkkQERFRKwkiIiJqJUFEREStJIiI\niKiVBBEREbXaliAkXSBpraTbGsp2k3StpLvL710blp0uaZmkuyS9sV1xRUREa9pZg5gHHNavbDaw\nyPZUYFGZR9I0YCZwQHnNuZLGtTG2iIgYRNsShO3rgYf7Fc8A5pfp+cCRDeWX2X7C9nJgGXBQu2KL\niIjBdXq47wm2V5fpNcCEMr038LOG9VaWso1ImgXMApg0aVKbwhx+jcODr5hzRBcjiYhoTdc6qW0b\n8Ca8bq7t6banjx8/vg2RRUQEdD5BPChpT4Dye20pXwVMbFhvn1IWERFd0ukEsRA4tkwfC1zVUD5T\n0jaSpgBTgRs7HFtERDRoWx+EpEuBQ4A9JK0EzgTmAAskHQ/cCxwFYPt2SQuApcBTwIm2n25XbBER\nMbi2JQjb72yy6NAm658NnN2ueCIiYmgGbWKStHsnAomIiN7SSh/EzyR9S9LhktT2iCIioie0kiBe\nCMwFjgHulvQvkl7Y3rAiIqLbBk0Qrlxb+hQ+SHX10Y2SrpP0qrZHGBERXTFoJ3Xpg3gPVQ3iQeBk\nqstSDwS+BUxpZ4AREdEdrVzF9FPgIuBI2ysbyhdL+mp7woqIiG5rJUHsX4bF2Ijtc4Y5noiI6BGt\ndFJfI+k5fTOSdpX0wzbGFBERPaCVBDHe9u/6Zmz/Fnhu+0KKiIhe0EqCeFrS+nG1Je3LJozCGhER\nI0srfRCfBG6QdB0g4G8oz2OIiIjRa9AEYftqSS8HXlmKTrH9UHvDioiIbmt1sL5tqB4fuiUwTVLf\nI0UjImKUauVGuXOAo4HbgWdKsYEkiIiIUayVGsSRVPdCPNHuYMaiPKs6InpVK1cx3QNs1e5AIiKi\nt7RSg3gcWCJpEbC+FmH7I22LKiIiuq6VBLGw/MQwaWxWiojoVa1c5jpf0nbAJNt3DcdOJX0M+ABV\nZ/etwPuA7YFvApOBFcBR5a7tiIjoglYeOfoWYAlwdZk/UNIm1ygk7Q18BJhu+yXAOGAmMBtYZHsq\nsKjMR0REl7TSSX0WcBDwOwDbS4D9NnO/WwLbSdqSqubwADADmF+Wz6e6eioiIrqklT6IJ23/vt/j\nqJ9ptvJgbK+S9HngPuCPwDW2r5E0wfbqstoaYELd6yXNogz1MWnSpLpVhlUuQ42IsaqVGsTtkt4F\njJM0VdK/Av+9qTuUtCtVbWEKsBewg6T3NK5Tnj/R7BkUc21Ptz19/PjxmxpGREQMopUEcTJwANUl\nrpcCjwCnbMY+Xwcst73O9pPAd4BXAw9K2hOg/F67GfuIiIjN1MpVTI9Tjej6yWHa533AKyVtT9XE\ndCiwGPgDcCwwp/y+apj2FxERm6CVsZh+TE1zj+3XbsoObf9c0uXAL4CngF8Cc4EdgQWSjgfuBY7a\nlO1HRMTwaKWT+tSG6W2Bt1N9sW8y22cCZ/YrfoKqNhERET2glSamm/sV/ZekG9sUT0RE9IhWmph2\na5jdAngFsEvbIoqIiJ7QShPTzVR9EKJqWloOHN/OoCIiovtaaWKa0olAorWb8nLjXkR0SitNTH8/\n0HLb3xm+cKJORn+NiG5opYnpeKob2f5vmf87qjup11E1PY2ZBJEv6ogYS1pJEFsB0/rGSSp3Oc+z\n/b62RhYREV3VylAbExsG0QN4EGj/KHkREdFVrdQgFkn6IdU4TABHAz9qX0gREdELWrmK6SRJbwNe\nU4rm2r6ivWFFRES3tVKDgGrcpEdt/0jS9pJ2sv1oOwOLiIjuauWRox8ELge+Vor2Bq5sZ1AREdF9\nrXRSnwgcTPUcCGzfDTy3nUFFRET3tZIgnrD9576Z8hzp2qe9RUTE6NFKgrhO0hnAdpJeD3wL+G57\nw4qIiG5rJUHMprpr+lbgQ8APgE+1M6iIiOi+Aa9ikjQO+IbtdwNf70xIERHRCwasQdh+GthX0tYd\niiciInpEK/dB3EP1FLmFwB/6Cm1/sW1RRURE1zWtQUi6qEy+FfheWXenhp9NJuk5ki6XdKekOyS9\nStJukq6VdHf5vevm7CMiIjbPQDWIV0jaC7gP+Ndh3u+Xgattv6M0X20PnAEssj1H0myqzvFPDPN+\nIyKiRQMliK8Ci4ApwOKGclHdB7HfpuxQ0i5U4zodB1DusfizpBnAIWW1+cBPGMMJIs+eiIhua9rE\nZPsrtl8MXGh7v4afKbY3KTkUU6gum71Q0i8lnSdpB2BCw7Dia4AJdS+WNEvSYkmL161btxlhRETE\nQAa9D8L2h4d5n1sCLwf+3fbLqDq+Z/fbp2lyt7btuban254+fvz4YQ4tIiL6tHKj3HBbCay0/fMy\nfzlVwniwPK2u76l1a7sQW0REFB1PELbXAPdL2r8UHQosBRYCx5ayY4GrOh1bRERs0OrzIIbbycAl\n5Qqme4D3USWrBZKOB+4FjupSbBERQZcShO0lwPSaRYd2OpbRovGqpxVzjuhiJBExWnSjDyIiIkaA\nbjUx9bTcgxARkRpEREQ0kQQRERG1kiAiIqJWEkRERNRKgoiIiFpJEBERUSsJIiIiaiVBRERErSSI\niIiolTupR7mM0RQRmyoJYhTKUCERMRySIEawJIKIaKf0QURERK0kiIiIqJUmpjEkHdYRMRSpQURE\nRK2uJQhJ4yT9UtL3yvxukq6VdHf5vWu3YouIiO7WID4K3NEwPxtYZHsqsKjMR0REl3QlQUjaBzgC\nOK+heAYwv0zPB47sdFwREbFBt2oQ/xs4DXimoWyC7dVleg0woeNRRUTEeh1PEJLeDKy1fXOzdWwb\ncJPXz5K0WNLidevWtSvMiIgxrxuXuR4MvFXS4cC2wM6SLgYelLSn7dWS9gTW1r3Y9lxgLsD06dNr\nk0gMXS6BjYj+Op4gbJ8OnA4g6RDgVNvvkfQ54FhgTvl9VadjG0syTEdEDKaX7oOYA7xe0t3A68p8\nRER0SVfvpLb9E+AnZfo3wKHdjCciIjbopRpERET0kCSIiIiolQQRERG1kiAiIqJWEkRERNRKgoiI\niFpJEBERUSsJIiIiaiVBRERErSSIiIiolQQRERG1ujoWU4xczUaDzVDhEaNHahAREVErCSIiImql\niSla1spDhvJkuojRIzWIiIiolQQRERG10sQUG0kzUURAahAREdFExxOEpImSfixpqaTbJX20lO8m\n6VpJd5ffu3Y6toiI2KAbNYingI/bnga8EjhR0jRgNrDI9lRgUZmPiIgu6XiCsL3a9i/K9KPAHcDe\nwAxgflltPnBkp2OLiIgNutpJLWky8DLg58AE26vLojXAhCavmQXMApg0aVL7g4xNls7uiJGta53U\nknYEvg2cYvuRxmW2Dbjudbbn2p5ue/r48eM7EGlExNjUlRqEpK2oksMltr9Tih+UtKft1ZL2BNZ2\nMqZW7hIei3JcIsaujicISQLOB+6w/cWGRQuBY4E55fdVnY4t2iejv0aMPN2oQRwMHAPcKmlJKTuD\nKjEskHQ8cC9wVBdiiw5LP0VE7+p4grB9A6Amiw/tZCwREdFc7qSOiIhaGYspRp00W0UMjySIGLGS\nCCLaK01MERFRKzWI6BmpEUT0liSIiM2UxBajVZqYIiKiVmoQ0ZOa/VeeoT8iOic1iIiIqJUEERER\ntZIgIiKi1pjug0h79sjQyueUzzJi+I3pBBGj3+ZegppLWGMsSxNTRETUSg0iokWpTcRYkwQR0aDV\nvoz0ecRYkAQRY0YeexoxNOmDiIiIWqlBxJjXK81F6eOIXtNzCULSYcCXgXHAebbndDmkiE3S7i/8\noTaZDTWeJKzoqQQhaRzwb8DrgZXATZIW2l7a3cgiWtPsS3tzaintruFsSiLoheTR7j6lXniP/XU6\npl7rgzgIWGb7Htt/Bi4DZnQ5poiIMUm2ux3DepLeARxm+wNl/hjgf9g+qWGdWcCsMrs/cFfHA+1N\newAPdTuIHpNj8mw5Hhsbq8dkX9vjB1upp5qYWmF7LjC323H0GkmLbU/vdhy9JMfk2XI8NpZjMrBe\na2JaBUxsmN+nlEVERIf1WoK4CZgqaYqkrYGZwMIuxxQRMSb1VBOT7acknQT8kOoy1wts397lsEaK\nNLttLMfk2XI8NpZjMoCe6qSOiIje0WtNTBER0SOSICIiolYSxAglaYWkWyUtkbS4lO0m6VpJd5ff\nu3Y7znaRdIGktZJuayhr+v4lnS5pmaS7JL2xO1G3V5NjcpakVeU8WSLp8IZlo/qYSJoo6ceSlkq6\nXdJHS/mYPk+GIgliZPs72wc2XMc9G1hkeyqwqMyPVvOAw/qV1b5/SdOorog7oLzm3DKsy2gzj42P\nCcCXynlyoO0fwJg5Jk8BH7c9DXglcGJ532P9PGlZEsToMgOYX6bnA0d2MZa2sn098HC/4mbvfwZw\nme0nbC8HllEN6zKqNDkmzYz6Y2J7te1flOlHgTuAvRnj58lQJEGMXAZ+JOnmMvwIwATbq8v0GmBC\nd0Lrmmbvf2/g/ob1VpayseJkSb8qTVB9zSlj6phImgy8DPg5OU9algQxcv217QOBN1FVnV/TuNDV\n9ctj9hrmsf7+G/w7sB9wILAa+EJ3w+k8STsC3wZOsf1I47KcJwNLghihbK8qv9cCV1BVhR+UtCdA\n+b22exF2RbP3P2aHcLH9oO2nbT8DfJ0NTSZj4phI2ooqOVxi+zulOOdJi5IgRiBJO0jaqW8aeANw\nG9WwJMeW1Y4FrupOhF3T7P0vBGZK2kbSFGAqcGMX4uu4vi/C4m1U5wmMgWMiScD5wB22v9iwKOdJ\ni3pqqI1o2QTgiur8Z0vgP2xfLekmYIGk44F7gaO6GGNbSboUOATYQ9JK4ExgDjXv3/btkhYAS6mu\nbDnR9tNdCbyNmhyTQyQdSNWMsgL4EIyZY3IwcAxwq6QlpewMxvh5MhQZaiMiImqliSkiImolQURE\nRK0kiIiIqJUEERERtZIgIiKiVhJEBNVQDI2joLZ5XydIem8n9hWxOXIfRESH2f5qt2OIaEVqEBEb\nbCnpEkl3SLpc0vYAkg6V9Mvy/I0Lyp22u5RnBuxf1rlU0gf7b1DSnPI8gl9J+nwpO0vSqZL2anhO\nwxJJT0vaV9J4Sd+WdFP5ObizhyGikgQRscH+wLm2Xww8AvyDpG2pnrNwtO2/oKp1f9j274GTgHmS\nZgK72v5648Yk7U41vMUBtl8KfKZxue0H+p7TQDVO0rdt3wt8meoZDn8FvB04r31vOaK5JIiIDe63\n/V9l+mLgr6mSxnLbvy7l84HXANi+FrgV+DfgAzXb+z3wJ+B8SX8PPF6301JD+CDw/lL0OuD/lOEh\nFgI7lxFJIzoqfRARG/Qfd2bAcWgkbQG8mOqLf1eq5wdseLH9lKSDgEOBd1DVOF7bbxt7Ug0o91bb\nj5XiLYBX2v7TJr6PiGGRGkTEBpMkvapMvwu4AbgLmCzpBaX8GOC6Mv0xqqeUvQu4sAwtvV75r3+X\n8pjPjwF/2W/5VsC3gE801FAArgFObljvwGF4bxFDlsH6Ilj/xLGrgcXAK6hG9DzG9uOSDgU+T1Xj\nvgn4MDAZuBI4yPajkr4IPGr7zIZt7kk1lPS2gIDP254v6SzgsbKtHwJ3NoRyOPBnqmarF5d9Xm/7\nhLa88YgBJEFEREStNDFFREStJIiIiKiVBBEREbWSICIiolYSRERE1EqCiIiIWkkQERFR6/8DIEnw\nNWbrJ9QAAAAASUVORK5CYII=\n",
 74 |       "text/plain": [
 75 |        "<matplotlib.figure.Figure at 0x7f73b3fa8f10>"
 76 |       ]
 77 |      },
 78 |      "metadata": {},
 79 |      "output_type": "display_data"
 80 |     },
 81 |     {
 82 |      "data": {
 83 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHmlJREFUeJzt3XmUHXWd9/H3h7DKJpAWwxpQZARHo0bGlUFxwaAiOg+L\nDoKiEUWURzkawUfxGRlhBvA444JBMGyCCKIZQdkehYMbBAwQNtmCJISkERUQRRI+zx/16+TS1O2+\nnfTte2/353XOPV31q7pV31q6v/37VdWvZJuIiIjB1up0ABER0Z2SICIiolYSRERE1EqCiIiIWkkQ\nERFRKwkiIiJqJUF0IUnHSjq7Tcs+RNI1DeOPSdpxlJZ9tKRvl+Gpkixp7VFa9nYl1kmjsbwRrHdL\nSVdLelTSSTXT50j60ljGNGj975V02Sgvc+Vx7Dajdf62Y7+NR6PyyxsjI+mxhtFnAU8AK8r4h8cy\nFtsbDTePpD2As21vM8yy/n204pK0EPig7SvKsn8PDBtrG8wEHgI2cRc+NGT7HOCcUV7mqB3Hdmvx\n/J0K3AusY3t5+d6o77fxKDWIDrC90cAH+D3w9oaynjxpR6um0IW2B27txuTQ68bxOTNuJEF0r3Ul\nnVmaNm6RNH1ggqStJF0oqV/SvZI+3mwhkraQNFfSI5KuBZ43aLolPb8Mz5B0a1nnYklHSdoQ+Amw\nVanOP1bWf6ykCySdLekR4JAmTWMfkPSApCWSjmpY79OaZiTtIWlRGT4L2A74n7K+Tw9usioxzJX0\nsKS7JH2oYVnHSjq/2f6r2UevlnSdpD+Xn68eiBE4GPh0ieONTRYxWdLlZV1XSdq+hWVvLmmRpLeX\n8Y3KdryvSYyHSLqnrONeSe9tKL+mDA/EOfB5smwDkjaVdFo5DoslfalZc13jcWzY7wdL+r2khyQd\nM8S+nCPplCH2hyUdLulO4M5S9g9l/ocl3SFpv4b5R3L+biDpJEn3lf19jaQNgKvL7H8q++VVemZT\nVe1xKtN+LunfJP2ibNNlkiY32wfjiu18OvgBFgJvHFR2LPA3YAYwCfgy8OsybS3geuDzwLrAjsA9\nwFuaLP884HxgQ+BFwGLgmobpBp5fhpcAryvDmwEvK8N7AItqYnwSeGeJaYNSdnaZPrUs+9yy7n8E\n+ge2FZgDfKlheU9bx+D90rC8tcv41cA3gPWBaWXZbxhu/9Xsn82BPwIHUTW5HljGt6iLs+b7c4BH\ngd2B9YCvDuzfFpb9ZuBB4DnAqcAFTdaxIfAIsHMZnwLsWoYPaTyeDd/ZFngAeGsZvwj4VlnWc4Br\ngQ83WV/dcTy1HOOXUDWJvnCk+6PhfLu87JsNSjz3A+8v++ilVE16u6zG+ft14OfA1uW4v7rE8LRz\nZ/B+a+E4/Ry4G3hBifnnwPGd/tsxFp/UILrXNbYvsb0COIvqFxPgFUCf7f9r+++276H65T1g8ALK\nf4jvBj5v+y+2FwBnDLHOJ4FdJG1i+4+2bxgmxl/Z/qHtp2z/tck8Xyzrvhn4DtUv3xqRtC3wGuAz\ntv9mez7wbaDxv+9m+2+wvYE7bZ9le7ntc4HbgbePIKSLbV9t+wngGOBVJcYhl237MuD7wJVUyWyo\n609PAS+StIHtJbZvaTZj+a/5h8BXbf9E0pZl+UeWY7EM+Ao158wQvmj7r7ZvBG6k+f6E5vtjwJdt\nP1zOmbcBC21/p+yj3wIXAv9rJOevpLWADwCfsL3Y9grbvywxDKeVc+A7tn9XYj6f6p+ScS8Jons9\n2DD8OLB+aV7Znqq5508DH+BoYMuaZfRR/Ud0f0PZfUOs891Uf0juK00DrxomxvuHmT54nvuArVr4\nznC2Ah62/eigZW/dMN5s/9Uta/A+Gbys4azcRtuPAQ+X5bay7NlU/xnPsf2HuoXb/guwP3AYsETS\nxZL+YYh4TgPusH1CGd8eWKd8d+Cc+RZVTaJVg/fnUBeHm+2PZ0wvsf3ToPP5vcBzGdn5O5mqNnn3\nMNtRp5XjNJLtHzeSIHrP/cC9tp/d8NnY9oyaefuB5VTNDQO2a7Zg29fZ3ofqD8cPqf5Tgqp6XvuV\nFuIdvO4HyvBfqO7gGvDcESz7AWBzSRsPWvbiFuKpW9b2g8pGuqyV2yhpI6omiweGW3b5D3k2cCbw\n0YG29Dq2L7X9Jqrmpdupao3PIGkWVVPIoQ3F91M1C01uOGc2sb3rCLZxJJrtjwGNx/Z+4KpB5/NG\ntj/CyM7fh6iaFZ9XM22483Q0zoFxKQmi91wLPCrpM+Wi3CRJL5L0isEzluaVHwDHSnqWpF2oLro+\ng6R1Vd0bvqntJ6navJ8qk5cCW0jadDXi/T9l3btStTN/r5TPB2aUi7XPBY4c9L2lVNdXnsH2/cAv\ngS9LWl/Si6n+IK7OsyOXAC+Q9B5Ja0vaH9gF+PEIljFD0mslrQv8G9X1jvtbWPbRVH+8PgD8J3Bm\n3YVjVc9i7KPqhoEngMdYdWwa53sr8HFg38YmP9tLgMuAkyRtImktSc+T9M8j2MaRaLY/6vyYah8d\nJGmd8nmFpBeO5Py1/RRwOnCyqhsYJpWL0etRJZqnaHI+MTrnwLiUBNFjyi/N26jaQO+l+s/p20Cz\nP94fo6oOP0h1AfE7Qyz+IGChqruSDqOq6mP7dqqLzfeUZoCRNBNdBdxF1c5+Yml3h+q6wI1UF6Mv\nY1XiGPBl4HNlfUfxTAdSXXx8gOoC7BdcnpkYidKs8zbgU8AfgE8Db7P90AgW813gC1RNKS8H/nW4\nZUt6OfBJ4H3lmJ5AlSxm1Sx/rTLvA2Ud/wx8pGa+/amaZW7TqjuZTinT3kd1U8OtVBdgL6CqjbRD\n7f6oU5oJ30x1PeQBqvP0BKqLyzCy8/co4GbgurLuE4C1bD8OHAf8opxPrxwUw2icA+OS7NzeHRGj\nQ9VttYtsf67TscSaSw0iIiJqJUFEREStNDFFRESt1CAiIqJWT3eWNXnyZE+dOrXTYURE9JTrr7/+\nIdt9w83XtgRRHq0/k+oJXwOzbX9V0uZUtzROpbrFcT/bfyzf+SzV/ewrgI/bvnSodUydOpV58+a1\naxMiIsYlSUP1qLBSO5uYlgOfsr0L8Erg8PKgyyzgSts7Ud0bPwugTDsA2BXYC/hG3UNDERExNtqW\nIEqHYjeU4UeB26j6NtmHVR1unUHVGyil/DzbT9i+l+rhqt3aFV9ERAxtTC5Sq3qj00uB3wBblkf/\noXo6cqCTua15eqdci6jpME3STEnzJM3r7+9vW8wRERNd2xNE6azrQqquhh9pnObqHtsR3Wdre7bt\n6ban9/UNe40lIiJWU1sThKR1qJLDObZ/UIqXSppSpk8BlpXyxTy918ZtSG+KEREd07YEIUlU/dLf\nZvvkhklzWdUj48HAjxrKD5C0nqQdgJ2oei6NiIgOaOdzEK+h6h30ZknzS9nRwPHA+ZIOpXopx34A\ntm+RdD5Vb5PLgcNLL5cREdEBbUsQtq8B1GTynk2+cxxVt7wREdFh6WojIiJq9XRXG91u6qyLVw4v\nPH7vDkYSETFyqUFEREStJIiIiKiVBBEREbWSICIiolYSRERE1EqCiIiIWkkQERFRKwkiIiJqJUFE\nREStJIiIiKiVBBEREbWSICIiolYSRERE1EqCiIiIWkkQERFRKwkiIiJqtS1BSDpd0jJJCxrKvidp\nfvksHHhXtaSpkv7aMO2UdsUVERGtaecb5eYAXwPOHCiwvf/AsKSTgD83zH+37WltjCciIkagbQnC\n9tWSptZNkyRgP+AN7Vp/RESsmU5dg3gdsNT2nQ1lO5Tmpaskva7ZFyXNlDRP0rz+/v72RxoRMUF1\nKkEcCJzbML4E2K40MX0S+K6kTeq+aHu27em2p/f19Y1BqBERE9OYJwhJawPvAr43UGb7Cdt/KMPX\nA3cDLxjr2CIiYpVO1CDeCNxue9FAgaQ+SZPK8I7ATsA9HYgtIiKKdt7mei7wK2BnSYskHVomHcDT\nm5cAdgduKre9XgAcZvvhdsUWERHDa+ddTAc2KT+kpuxC4MJ2xRIRESOXJ6kjIqJWEkRERNRKgoiI\niFpJEBERUSsJIiIiaiVBRERErSSIiIiolQQRERG1kiAiIqJWO18YNGFMnXXxyuGFx+/dwUgiIkZP\nahAREVErCSIiImolQURERK0kiIiIqJUEERERtXIX02pqvHMpImI8Sg0iIiJqJUFEREStdr6T+nRJ\nyyQtaCg7VtJiSfPLZ0bDtM9KukvSHZLe0q64IiKiNe2sQcwB9qop/4rtaeVzCYCkXYADgF3Ld74h\naVIbY4uIiGG0LUHYvhp4uMXZ9wHOs/2E7XuBu4Dd2hVbREQMrxPXII6QdFNpgtqslG0N3N8wz6JS\n9gySZkqaJ2lef39/u2ONiJiwxjpBfBPYEZgGLAFOGukCbM+2Pd329L6+vtGOLyIiijFNELaX2l5h\n+yngVFY1Iy0Gtm2YdZtSFhERHTKmCULSlIbRfYGBO5zmAgdIWk/SDsBOwLVjGVtERDxd256klnQu\nsAcwWdIi4AvAHpKmAQYWAh8GsH2LpPOBW4HlwOG2V7QrtnbKE9YRMV60LUHYPrCm+LQh5j8OOK5d\n8URExMjkSeqIiKiVBBEREbWSICIiolYSRERE1EqCiIiIWnlhUIc13ha78Pi9OxhJRMTTpQYRERG1\nkiAiIqJWEkRERNRKgoiIiFpJEBERUSsJIiIiaiVBRERErSSIiIiolQQRERG1kiAiIqJWEkRERNRq\nW4KQdLqkZZIWNJT9p6TbJd0k6SJJzy7lUyX9VdL88jmlXXFFRERr2lmDmAPsNajscuBFtl8M/A74\nbMO0u21PK5/D2hhXRES0oG0JwvbVwMODyi6zvbyM/hrYpl3rj4iINdPJaxAfAH7SML5DaV66StLr\nmn1J0kxJ8yTN6+/vb3+UERETVEcShKRjgOXAOaVoCbCd7WnAJ4HvStqk7ru2Z9uebnt6X1/f2AQc\nETEBDZsgJG0xmiuUdAjwNuC9tg1g+wnbfyjD1wN3Ay8YzfVGRMTItFKD+LWk70uaIUlrsjJJewGf\nBt5h+/GG8j5Jk8rwjsBOwD1rsq6IiFgzrSSIFwCzgYOAOyX9u6Rh/7uXdC7wK2BnSYskHQp8DdgY\nuHzQ7ay7AzdJmg9cABxm++HaBUdExJgY9p3UpRnocqo/6q8HzgY+KulGYJbtXzX53oE1xac1mfdC\n4MKWo46IiLYbNkGUaxD/SlWDWAocAcwFpgHfB3ZoZ4AREdEZwyYIqmais4B32l7UUD4vTzxHRIxf\nrSSInQfuNhrM9gmjHE9ERHSJVi5SXzbQZxKApM0kXdrGmCIiogu0UoPos/2ngRHbf5T0nDbGNGFN\nnXXxyuGFx+/dwUgiIlqrQayQtN3AiKTtgdomp4iIGD9aqUEcA1wj6SpAwOuAmW2Nqovkv/qImKha\neQ7ip5JeBryyFB1p+6H2hhUREZ3WSg0CYD2qrrvXBnaRNNCdd7QoNZGI6DWtPCh3ArA/cAvwVCk2\nkAQRETGOtVKDeCfVsxBPtDuYiIjoHq3cxXQPsE67A4mIiO7SSg3icWC+pCuBlbUI2x9vW1QREdFx\nrSSIueUTERETSCu3uZ4haQOqV4LeMQYxda3GO5EiIsa7Vl45+nZgPvDTMj5NUmoUERHjXCsXqY8F\ndgP+BGB7PrBjG2OKiIgu0EqCeNL2nweVPVU7Z0REjButJIhbJL0HmCRpJ0n/DfxyuC9JOl3SMkkL\nGso2l3S5pDvLz80apn1W0l2S7pD0ltXamoiIGDWtJIgjgF2pbnE9F3gEOLKF780B9hpUNgu40vZO\nwJVlHEm7AAeU9ewFfEPSpBbWERERbdLKXUyPU/XoesxIFmz7aklTBxXvA+xRhs8Afg58ppSfV57W\nvlfSXVTXPX41knX2itwNFRG9oJW+mH5GzfsfbL9hNda3pe0lZfhBYMsyvDXw64b5FpWyunhmUrob\n32677epmiYiIUdDKg3JHNQyvD7wbWL6mK7ZtSSN+8ZDt2cBsgOnTp4/bFxel99eI6LRWmpiuH1T0\nC0nXrub6lkqaYnuJpCnAslK+GNi2Yb5tSllERHRIKw/Kbd7wmVzuMNp0Ndc3Fzi4DB8M/Kih/ABJ\n60naAdgJWN0kFBERo6CVJqbrqa5BiKpp6V7g0OG+JOlcqgvSkyUtAr4AHA+cL+lQ4D5gPwDbt0g6\nH7i1rONw2ytGvDURETFqWmli2mF1Fmz7wCaT9mwy/3HAcauzroiIGH2t3MX0rqGm2/7B6IUTERHd\nopUmpkOBVwP/r4y/nupJ6n6qpqckiIiIcaiVBLEOsMvA8wvl7qM5tt/f1sgiIqKjWulqY9uGh9sA\nlgJ5Qi0iYpxrpQZxpaRLqfphAtgfuKJ9IUVERDdo5S6mj0naF9i9FM22fVF7w4qIiE5rpQYBcAPw\nqO0rJD1L0sa2H21nYBER0VmtPEn9IeAC4FulaGvgh+0MKiIiOq+Vi9SHA6+heg8Etu8EntPOoCIi\novNaSRBP2P77wIiktanp/jsiIsaXVhLEVZKOBjaQ9Cbg+8D/tDesiIjotFYSxCyqp6ZvBj4MXAJ8\nrp1BRURE5w15F1N5L/SZtt8LnDo2IUVERDcYsgZRutzeXtK6YxRPRER0iVaeg7iH6i1yc4G/DBTa\nPrltUUVERMc1rUFIOqsMvgP4cZl344ZPRESMY0PVIF4uaSvg98B/j1E8ERHRJYZKEKcAVwI7APMa\nykX1HMSObYwrIiI6rGmCsP1fwH9J+qbtj4zWCiXtDHyvoWhH4PPAs4EPUd1SC3C07UtGa70RETEy\nrfTmOmrJoSzvDmAarLyNdjFwEfB+4Cu2TxzN9UVExOpp5UG5dtoTuNv2fR2OIyIiBul0gjiAVS8i\nAjhC0k2STpe0Wd0XJM2UNE/SvP7+/rpZIiJiFHQsQZSH795B1bcTwDeprkdMA5YAJ9V9z/Zs29Nt\nT+/r6xuTWCMiJqJO1iDeCtxgeymA7aW2V9h+iqpbj906GFtExITXyQRxIA3NS5KmNEzbF1gw5hFF\nRMRKrb5ydFRJ2hB4E1XvsAP+Q9I0qmcsFg6aFhERY6wjCcL2X4AtBpUd1IlYIiKiXqfvYoqIiC7V\nkRpEjMzUWRevHF54/N4djCQiJpIkiB6WxBER7ZQmpoiIqJUEERERtZIgIiKiVhJERETUykXqGo0X\nfyMiJqrUICIiolYSRERE1EqCiIiIWkkQERFRKwkiIiJqJUFERESt3ObaY3ILbkSMldQgIiKiVhJE\nRETU6tQrRxcCjwIrgOW2p0vaHPgeMJXqlaP72f5jJ+KLiIjO1iBeb3ua7ellfBZwpe2dgCvLeERE\ndEg3NTHtA5xRhs8A3tnBWCIiJrxOJQgDV0i6XtLMUral7SVl+EFgy7ovSpopaZ6kef39/WMRa0TE\nhNSp21xfa3uxpOcAl0u6vXGibUty3RdtzwZmA0yfPr12noiIWHMdqUHYXlx+LgMuAnYDlkqaAlB+\nLutEbBERURnzBCFpQ0kbDwwDbwYWAHOBg8tsBwM/GuvYIiJilU40MW0JXCRpYP3ftf1TSdcB50s6\nFLgP2K8DsUVERDHmCcL2PcBLasr/AOw51vFERES9brrNNSIiukg66xuHGjv0W3j83h2MJCJ6WRLE\nOJdkERGrK01MERFRKwkiIiJqpYlpnMiLhCJitCVBBJBrFRHxTGliioiIWkkQERFRKwkiIiJqJUFE\nREStJIiIiKiVu5gmqNwWGxHDSQ0iIiJqpQYRz5BnIiICUoOIiIgmkiAiIqJWmpgmkFyYjoiRGPMa\nhKRtJf1M0q2SbpH0iVJ+rKTFkuaXz4yxji0iIlbpRA1iOfAp2zdI2hi4XtLlZdpXbJ/YgZgiImKQ\nMU8QtpcAS8rwo5JuA7Ye6zgiImJoHb1ILWkq8FLgN6XoCEk3STpd0mZNvjNT0jxJ8/r7+8co0oiI\niadjCULSRsCFwJG2HwG+CewITKOqYZxU9z3bs21Ptz29r69vzOKNiJhoOpIgJK1DlRzOsf0DANtL\nba+w/RRwKrBbJ2KLiIhKJ+5iEnAacJvtkxvKpzTMti+wYKxji4iIVTpxF9NrgIOAmyXNL2VHAwdK\nmgYYWAh8eCyDyjMCw0sXHBETSyfuYroGUM2kS8Y6loiIaC5dbURERK10tRGjKs1QEeNHEkSsllyz\niRj/kiBiTKRmEdF7cg0iIiJqJUFEREStJIiIiKg1oa9B5ELr8LKPIiauCZ0gor2SXCJ6W5qYIiKi\nVhJERETUShNTdFSz5yPy3ERE5yVBRNfINYuI7pImpoiIqJUaRHS9NENFdEYSRIw7SRwRoyNNTBER\nUSs1iBhza3IxerQuZKeWETG8rqtBSNpL0h2S7pI0q9PxRERMVF1Vg5A0Cfg68CZgEXCdpLm2b+1s\nZNHtWqlZrM48I61djLRmkppMdLOuShDAbsBdtu8BkHQesA+QBBGrZU0Tx5rcQdVsuSNNHCP97niT\nJLrKWO8L2W77Slol6V+AvWx/sIwfBPyT7Y81zDMTmFlGdwbuaLK4ycBDbQx3LIyHbYBsR7fJdnSX\nTmzH9rb7hpup22oQw7I9G5g93HyS5tmePgYhtc142AbIdnSbbEd36ebt6LaL1IuBbRvGtyllEREx\nxrotQVwH7CRpB0nrAgcAczscU0TEhNRVTUy2l0v6GHApMAk43fYtq7m4YZuhesB42AbIdnSbbEd3\n6drt6KqL1BER0T26rYkpIiK6RBJERETUGncJope76pC0UNLNkuZLmlfKNpd0uaQ7y8/NOh3nYJJO\nl7RM0oKGsqZxS/psOT53SHpLZ6J+pibbcaykxeWYzJc0o2Fat27HtpJ+JulWSbdI+kQp75ljMsQ2\n9NTxkLS+pGsl3Vi244ulvDeOhe1x86G6sH03sCOwLnAjsEun4xpB/AuByYPK/gOYVYZnASd0Os6a\nuHcHXgYsGC5uYJdyXNYDdijHa1Knt2GI7TgWOKpm3m7ejinAy8rwxsDvSrw9c0yG2IaeOh6AgI3K\n8DrAb4BX9sqxGG81iJVdddj+OzDQVUcv2wc4owyfAbyzg7HUsn018PCg4mZx7wOcZ/sJ2/cCd1Ed\nt45rsh3NdPN2LLF9Qxl+FLgN2JoeOiZDbEMzXbcNAK48VkbXKR/TI8divCWIrYH7G8YXMfRJ1W0M\nXCHp+tKlCMCWtpeU4QeBLTsT2og1i7sXj9ERkm4qTVADTQE9sR2SpgIvpfrPtSePyaBtgB47HpIm\nSZoPLAMut90zx2K8JYhe91rb04C3AodL2r1xoqs6aM/dl9yrcRffpGqynAYsAU7qbDitk7QRcCFw\npO1HGqf1yjGp2YaeOx62V5Tf622A3SS9aND0rj0W4y1B9HRXHbYXl5/LgIuoqpZLJU0BKD+XdS7C\nEWkWd08dI9tLyy/4U8CprKrud/V2SFqH6g/rObZ/UIp76pjUbUOvHg8A238CfgbsRY8ci/GWIHq2\nqw5JG0raeGAYeDOwgCr+g8tsBwM/6kyEI9Ys7rnAAZLWk7QDsBNwbQfia8nAL3GxL9UxgS7eDkkC\nTgNus31yw6SeOSbNtqHXjoekPknPLsMbUL3r5nZ65Vh0+ir/aH+AGVR3PNwNHNPpeEYQ945Udy/c\nCNwyEDuwBXAlcCdwBbB5p2Otif1cqur+k1RtpocOFTdwTDk+dwBv7XT8w2zHWcDNwE1Uv7xTemA7\nXkvVZHETML98ZvTSMRliG3rqeAAvBn5b4l0AfL6U98SxSFcbERFRa7w1MUVExChJgoiIiFpJEBER\nUSsJIiIiaiVBRERErSSICKruHBp7cW3zug6T9L6xWFfEmuiqV45GTAS2T+l0DBGtSA0iYpW1JZ0j\n6TZJF0h6FoCkPSX9VtW7Ok4vT7luWvrr37nMc66kDw1eoKTjyzsNbpJ0Yik7VtJRkrZqeK/BfEkr\nJG1fnr69UNJ15fOasd0NEZUkiIhVdga+YfuFwCPARyWtD8wB9rf9j1S17o/Y/jPwMWCOpAOAzWyf\n2rgwSVtQdQexq+0XA19qnG77AdvTXHXkdipwoe37gK8CX7H9CuDdwLfbt8kRzSVBRKxyv+1flOGz\nqbp72Bm41/bvSvkZVC8WwvblVN0+fB34YM3y/gz8DThN0ruAx+tWWmoIHwI+UIreCHytdBE9F9ik\n9GoaMaZyDSJilcH9zgzZD42ktYAXUv3h34yq/6ZVX7aXS9oN2BP4F6oaxxsGLWMKVad07/CqF8us\nBbzS9t9WczsiRkVqEBGrbCfpVWX4PcA1VB2mTZX0/FJ+EHBVGf7fVG86ew/wndI99Urlv/5NbV9S\n5n3JoOnrAN8HPtNQQwG4DDiiYb5po7BtESOWzvoiWPnWsp8C84CXA7cCB9l+XNKewIlUNe7rgI8A\nU4EfArvZflTSycCjtr/QsMwpVN04r0/1buITbZ8h6VjgsbKsS6m6fx4wA/g7VbPVC8s6r7Z9WFs2\nPGIISRAREVErTUwREVErCSIiImolQURERK0kiIiIqJUEERERtZIgIiKiVhJERETU+v8bnGeaIIt2\nGAAAAABJRU5ErkJggg==\n",
 84 |       "text/plain": [
 85 |        "<matplotlib.figure.Figure at 0x7f73e69d0190>"
 86 |       ]
 87 |      },
 88 |      "metadata": {},
 89 |      "output_type": "display_data"
 90 |     }
 91 |    ],
 92 |    "source": [
 93 |     "# ground truth\n",
 94 |     "sizes = [ anno_func.box_long_size(obj['bbox']) for k,img in results_annos['imgs'].items() for obj in img['objects']]\n",
 95 |     "pl.figure()\n",
 96 |     "pl.title(\"The distribution of box size in ground truth\")\n",
 97 |     "pl.xlabel(\"box size\")\n",
 98 |     "pl.ylabel(\"frequency\")\n",
 99 |     "_ = pl.hist(sizes, bins=100)\n",
100 |     "# prediction\n",
101 |     "sizes = [ anno_func.box_long_size(obj['bbox']) for k,img in annos['imgs'].items() for obj in img['objects']]\n",
102 |     "pl.figure()\n",
103 |     "pl.title(\"The distribution of box size in prediction\")\n",
104 |     "pl.xlabel(\"box size\")\n",
105 |     "pl.ylabel(\"frequency\")\n",
106 |     "_ = pl.hist(sizes, bins=100)"
107 |    ]
108 |   },
109 |   {
110 |    "cell_type": "code",
111 |    "execution_count": 5,
112 |    "metadata": {},
113 |    "outputs": [
114 |     {
115 |      "name": "stdout",
116 |      "output_type": "stream",
117 |      "text": [
118 |       "iou:0.5, size:[0,400), types:[d, ...total 5...], accuracy:0.865326633166, recall:0.787740164684\n",
119 |       "iou:0.5, size:[0,32), types:[d, ...total 5...], accuracy:0.759689922481, recall:0.689170182841\n",
120 |       "iou:0.5, size:[32,96), types:[d, ...total 5...], accuracy:0.918744971842, recall:0.839088905217\n",
121 |       "iou:0.5, size:[96,400), types:[d, ...total 5...], accuracy:0.882352941176, recall:0.789473684211\n",
122 |       "iou:0.5, size:[0,512), types:z, accuracy:0.847457627119, recall:0.841219768665\n",
123 |       "iou:0.5, size:[0,512), types:j, accuracy:0.929012345679, recall:0.893175074184\n",
124 |       "iou:0.5, size:[0,512), types:s, accuracy:0.906329113924, recall:0.8463356974\n",
125 |       "iou:0.5, size:[0,512), types:l, accuracy:0.870967741935, recall:0.870967741935\n",
126 |       "iou:0.5, size:[0,512), types:d, accuracy:0.788679245283, recall:0.506053268765\n"
127 |      ]
128 |     }
129 |    ],
130 |    "source": [
131 |     "reload(anno_func)\n",
132 |     "df=pd.DataFrame(index=['Precision','Recall'])\n",
133 |     "#test_annos = res_annos2\n",
134 |     "#minscore=0.5\n",
135 |     "test_annos = results_annos\n",
136 |     "minscore=50\n",
137 |     "\n",
138 |     "sm = anno_func.eval_annos(annos, test_annos, iou=0.5, check_type=True, types=anno_func.type5,\n",
139 |     "                         minboxsize=0,maxboxsize=400,minscore=minscore)\n",
140 |     "print sm['report']\n",
141 |     "sm = anno_func.eval_annos(annos, test_annos, iou=0.5, check_type=True, types=anno_func.type5,\n",
142 |     "                         minboxsize=0,maxboxsize=32,minscore=minscore)\n",
143 |     "print sm['report']\n",
144 |     "sm = anno_func.eval_annos(annos, test_annos, iou=0.5, check_type=True, types=anno_func.type5,\n",
145 |     "                         minboxsize=32,maxboxsize=96,minscore=minscore)\n",
146 |     "print sm['report']\n",
147 |     "sm = anno_func.eval_annos(annos, test_annos, iou=0.5, check_type=True, types=anno_func.type5,\n",
148 |     "                         minboxsize=96,maxboxsize=400,minscore=minscore)\n",
149 |     "print sm['report']\n",
150 |     "\n",
151 |     "for tp in anno_func.type5:\n",
152 |     "    sm = anno_func.eval_annos(annos, test_annos, iou=0.5, check_type=True, types=[tp],minscore=minscore)\n",
153 |     "    print(sm['report'])\n",
154 |     "    "
155 |    ]
156 |   },
157 |   {
158 |    "cell_type": "code",
159 |    "execution_count": 6,
160 |    "metadata": {},
161 |    "outputs": [
162 |     {
163 |      "name": "stdout",
164 |      "output_type": "stream",
165 |      "text": [
166 |       "99 99.9891757965 iou:0.5, size:[0,400), types:[d, ...total 5...], accuracy:1.0, recall:0.000457456541629\n"
167 |      ]
168 |     },
169 |     {
170 |      "data": {
171 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfUAAAGDCAYAAAAyM4nNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8XXWd//HXJ3vT7HvbJG26JSmlLSHdWEtSLaCACzqA\no4PCDxhEnfn9BkFl1BllZFxGR9lkENxGOopsIgrTjb3QlrZASbpv6ZKbtE26pNluvr8/7m1IS5e0\nyc25y/v5eOTxyDn3m3M/OZTHO+dzv+d7zDmHiIiIRL44rwsQERGRwaFQFxERiRIKdRERkSihUBcR\nEYkSCnUREZEooVAXERGJEgp1kQhnZl83s4eH4H3mmdlToX6f47zvXDM7aGY9ZjZ3qN9fJJIo1EUi\nnHPu35xzNw7BW90N3HNkw8zGmNliM2szs/qTBa6Z/aOZbTKz/Wa208x+bGYJwdcKzOyx4P5WM3vV\nzGYe+Vnn3ALnXBqwLZS/nEg0UKiLyCmZ2XQg0zm3tM/ux4CVQC7wDeBxM8s/wSGeAaY75zKAycBU\n4MvB19KAZcC5QA7wK+DPZpY26L+ISJRTqItECDO7w8x2mNkBM1trZrXB/d82s98Gv7832Ko+8tVt\nZt8OvjbSzP5oZk1mttnMvnyStzvWZcCLfWqZCFQB33LOHXbO/RF4G/jk8X7YObfRObfnyI8DPcD4\n4GubnHP/4Zzb5ZzzO+ceApKA8tOoT0RQqItEBDMrB24jcLWbDswDthw7zjl3m3MuLdiuvgDYBzxt\nZnHAn4DVwCigFvgHM5sXPP4FZtZykhLOBtb22T4L2OScO9Bn3+rg/hP9DteZ2X6gmcCV+s9PMG4a\ngVDfcJJ6ROQ4FOoikcEPJAOTzCzRObfFObfxRIODbfCngC8551YC04F859y/Ouc6nXObgP8CrgFw\nzr3inMs6yftnAX0DPA1oPWbMfiD9RAdwzv0u2H6fCDwINB6n7gzgN8C/OOeOPb6InIJCXSQCOOc2\nAP8AfBvwmdl8Mxt5vLFmlgg8DvzOOTc/uHs0MNLMWo58AV8HCvtZwj6ODuyDQMYxYzI5OvhP9Lus\nB9YA9x9T9zAC3YSlzrnv9bMuEelDoS4SIYJXuhcQCGgH/PsJhv6MwFXzXX32bQc2O+ey+nylO+cu\n7+fbv03gCvuINcBYM+sb9FOD+/sjARh3ZMPMkgl0FhqAm/t5DBE5hkJdJAKYWbmZ1QTDrx04TGCy\n2bHjbgYuBj7jnOv7+pvAgeBku2FmFm9mk4Oz2vvjueBxAXDOrQNWAd8ysxQz+wSBz93/GKxjjJk5\nMxsT3L7RzAqC308CvgYsDG4f6SwcBv7umLpF5DQo1EUiQzKBe8Sbgd1AAYFgPNa1wFhgZ58Z8F93\nzvmBjwLTgM3B4zxMoGWOmV1oZgdP9ObOubeA1r73jxP4PL6aQGv+e8DVzrmm4GslwFZgR3D7fOAd\nMztE4A+E5wi0/wHOC9b2YaClT90X9u/UiMgR5pzzugYRiQBm9mHgVufcx/ox9i6gyTl33Bnup/m+\ntQQ6AMnA5c65xQM9pki0UqiLiIhECbXfRUREooRCXUREJEoo1EVERKKEQl1ERCRKJHhdwOnKy8tz\nY8aM8boMERGRIbNixYpm59yJnoLYK+JCfcyYMSxfvtzrMkRERIaMmW3tzzi130VERKKEQl1ERCRK\nKNRFRESihEJdREQkSijURUREooRCXUREJEoo1EVERKKEQl1ERCRKKNRFRESiRMhC3cweMTOfmb17\ngtfNzH5qZhvM7G0zqwpVLSIiIrEglFfqvwQuPcnrlwETgl83AQ+EsBYREZGoF7JQd869BOw9yZCr\ngF+7gKVAlpmNCFU9x3O4089f393FoY7uoXxbERGRkPDygS6jgO19thuC+3YdO9DMbiJwNU9paemg\nFfDqhmZu+e1bJMXHMWtcLnMrC6ipKKA4O3XQ3kNERGSoRMRT2pxzDwEPAVRXV7vBOu7F5fk89n9m\nsbCukYX1Pr759Bq++fQaKorSqa0soKaikGklWcTH2WC9pYiISMh4Geo7gJI+28XBfUMmMT6O2eNy\nmT0ul7s+OolNTQdZWOdjYX0jD764ifsWbyR3eBJzyguYW1nAhRPzSUuOiL+DREQkBnmZUM8At5nZ\nfGAm0Oqc+0DrfSiNzU9jbH4a/+eisbS2dbFknY9F9T4W1DXyx7caSIw3Zo3NpbaigNrKQkpy1KYX\nEZHwYc4NWjf76AObPQbMAfKARuBbQCKAc+5BMzPgXgIz5NuAzzvnlp/quNXV1W758lMOG1Td/h6W\nb93XG/Cbmg4BMLEwjdrKQuZWFjCtJFttehERCQkzW+Gcqz7luFCFeqh4EerH2tx8KPA5fJ2PZVv2\n0t3jyBmexJzyfOZWFnLhhDzSUxI9rVFERKKHQn2ItB7u4qV1TSysa2Tx2iZaD3eRGG/MLMultrKA\n2opCSnPVphcRkTOnUPdAt7+Ht7a1sLCukQV1jWwMtuknFATa9LWVBVSVqk0vIiKnR6EeBrY0H2Jh\nvY9F9Y28sSnQps9OTWROeQG1lQVcNDGfDLXpRUTkFBTqYWZ/e6BNv6jOx+K1Pva1dZEQZ8woy+md\nbDc6d7jXZYqISBhSqIcxf4/jrW37AvfE1zWy3ncQgHH5w5lbWUhtZSFVpVkkxOsheiIiolCPKNv2\ntLGwPjCb/o3Ne+jyO7JSE5kzMZ/aykIumphP5jC16UVEYpVCPUIdaO/i5fXNLKhrZMnaJvYe6iQh\nzpg+Jicwm76ykLI8telFRGKJQj0K+Hscq7bvY0GwTb+uMdCmHxts09dUFFA9OlttehGRKKdQj0Lb\n97b1Pnxm6aZAmz4jJaF3Nv2ciQVkpqpNLyISbRTqUe5gRzcvr2tiYb2PxfU+9hzqJD7OqB6dHZxs\nV8DY/DSvyxQRkUGgUI8hgTZ9C4uCk+3qdx8AoCxveO/DZ6rHZJOoNr2ISERSqMew7XvbWLzWx4I6\nH0s37qHT30NGSgIXBx8he/HEfLJSk7wuU0RE+kmhLkCgTf/K+ubg2vQ+mg8G2vTnjs5mbmUBNRWF\njMsfTuCheSIiEo4U6vIBPT2O1Q0tLKwLPEL2SJt+TG4qNRWBVe2ml+WoTS8iEmYU6nJKO1oOsyg4\nm/61jXvo7O4hPTmBi8rzmRucTZ89XG16ERGvKdTltBzq6OaVDc0sqvOxsN5H88EO4gzOHZ0deMJc\nRQHjC9LUphcR8YBCXc5YT4/j7R2tLKprZEGdj/d27QegNCe19xnxM8pySEpQm15EZCgo1GXQ7Gw5\nzKL6wKp2r/Zt00/Mp6aigEsqCshRm15EJGQU6hISbZ3dvLphT+/Kdk0HAm36qtJsaioLmFtZyAS1\n6UVEBpVCXUKup8fx7s7W3rXp1+wMtOlLcoZRWxFY1W5mWa7a9CIiA6RQlyG3u7WdhfWNLKrz8cqG\nZjq6e0hLTuDCCXnUVhZySXk+uWnJXpcpIhJxFOriqcOdfl7b2MyCOh+L6htp3N+BGZxTkhWYTV9Z\nQHlhutr0IiL9oFCXsOGcY83O/SyoC6xN/86OVgCKs4dRW1FATWUhs8bmkJwQ73GlIiLhSaEuYatx\nf3vvbPpXNjTT3tXD8KR4LpyQT01lATUVBeSpTS8i0kuhLhGhvatPm77Ox+797ZjBtJKs3ifMVRSp\nTS8isU2hLhHnSJt+YfBz+NUNgTb9qKxh1FQUUFtZwKyxuaQkqk0vIrFFoS4Rz3ekTV/v45X1zRzu\n8pOaFM8F4/OYW1nIJRUF5KerTS8i0U+hLlGlvcvP65uCi97U+djV2g7A1JIs5lYUUFNZwKQRGWrT\ni0hUUqhL1HLOUbfrAAvrGllQ72P19hYARmamUBNcm372OLXpRSR6KNQlZvgOtLOkvokFwdn0bZ1+\nhiXGc8GEvMAtcxUFFGSkeF2miMgZU6hLTGrv8rN00x4WBpeu3XmkTV+cSU1w6dqzRqpNLyKRRaEu\nMc85R/3uA70Pn1m1vQXnoCgjJfjwmQLOG5enNr2IhD2Fusgxmg50sHht4H74l9c3cajTT0piHBeM\nD6xNX1NRQKHa9CIShhTqIifR0e3njU17A5Pt6nzsaDkMwNmjMqkNTrabPEptehEJDwp1kX5yzrGu\n8WBwbfpGVgbb9IUZyYHP4SsKOH98HsOS1KYXEW8o1EXO0J6DHSxe28TCukZeWhdo0ycnBNr0R26Z\nK8pUm15Eho5CXWQQdHT7eXPzXhbW+VhQ10jDvkCbfvKoDGoqCplbWcDkkZnExalNLyKho1AXGWTO\nOdb7Am36RXU+3tq2jx4HBenJwbXpC7lAbXoRCQGFukiI7T3UyeJ6H4vqfby4romDHd0kJ8Rx3rhc\naisD98SPyBzmdZkiEgUU6iJDqLO7h2Vb9gYn2/nYtrcNgEkjMphbWUBNZSFTRqlNLyJnRqEu4hHn\nHBubDrIguKrdiq2BNn1+ejI15YGHz1w4IY/UpASvSxWRCKFQFwkT+w51smSdjwV1Pl5a28SBjm6S\njrTpKwJX8aOy1KYXkRNTqIuEoS5/D8s27w1cxdc3snVPoE1fOSKD2ooCaisLmFqcpTa9iBxFoS4S\n5gJt+kO9a9Mv37KXHgd5aUlcUh6YTX/hhDyGJ6tNLxLrFOoiEaalrZMla5tYWO9jyVofB9q7SYqP\nY9a43MBku4oCirNTvS5TRDygUBeJYF3+HpZv2dd7Fb+5+RAAFUXp1FYWUFNRyLSSLOLVpheJCQp1\nkSiyqelg76p2y7fuw9/jyB2exCUVBdRWFHDhxHzS1KYXiVoKdZEo1drWxZJ1PhbWBdr0+4Nt+plj\nc4KT7QopyVGbXiSaKNRFYkC3v4flW99v029qCrTpywvTqaksYG5lAdNKstWmF4lwCnWRGLS5OTib\nvs7Hm1v24u9x5AxPYk55PnODs+nTUxK9LlNETlNYhLqZXQr8JxAPPOycu+eY1zOB3wKlQALwQ+fc\noyc7pkJdpH9aD3fx4romFtU1snhtE62Hu0iMN2aW5VJbWcBctelFIobnoW5m8cA64ENAA7AMuNY5\n916fMV8HMp1zd5hZPrAWKHLOdZ7ouAp1kdPX7e9hxdZ9LKoPTLbbGGzTTyhI6334TFWp2vQi4aq/\noR7K6bIzgA3OuU3BguYDVwHv9RnjgHQzMyAN2At0h7AmkZiUEB/HzLG5zByby9cur2RL8yEW1gfW\npn/45U08+OJGslMTuSS4Nv1FE/PJUJteJOKEMtRHAdv7bDcAM48Zcy/wDLATSAf+xjnXE8KaRAQY\nkzecGy4o44YLytjf3sVL65pYWOdj0VofT6zcQUKcMXNsDjUVhcytLGB07nCvSxaRfvD6xtZ5wCqg\nBhgH/K+Zveyc2993kJndBNwEUFpaOuRFikSzjJREPjplJB+dMhJ/j+OtbftYUNfIojof33n2Pb7z\n7HuML0jrvV2uqjSLhPg4r8sWkeMI5Wfqs4FvO+fmBbe/BuCc+16fMX8G7nHOvRzcXgTc6Zx780TH\n1WfqIkNn655DgSv4eh9vbN5Dl9+RlZrInIn51FYWcnG52vQiQyEcPlNfBkwwszJgB3ANcN0xY7YB\ntcDLZlYIlAObQliTiJyG0bnD+cIFZXzhgjIOtHfx0rpmFtY3srjex1OrdpIQZ0wfk9M7m35Mntr0\nIl4K9S1tlwM/IXBL2yPOubvN7BYA59yDZjYS+CUwAjACV+2/PdkxdaUu4j1/j2Pltn29k+3WNR4E\nYGz+cOZWFlJTUUD16Gy16UUGiee3tIWKQl0k/Gzf29a7qt3STYE2feawROaU51NTUcCciQVkpqpN\nL3KmFOoi4omDHd28vK6JBXU+Fq/1sfdQJ/FxxvQx2dRWBO6JH5uf5nWZIhFFoS4invP3OFZtb2Fh\nXSOL6n3U7z4AwNi84dQEZ9NXj8kmUW16kZNSqItI2Nm+t41F9b5Am37jHjr9PWSkJHBxeeDhM2rT\nixyfQl1EwtrBjm5eWR9Y9GbxWh/NBwNt+nNHZzO3MnAVPzZvOIEFJ0Vim0JdRCJGT49jVUMLi+oC\na9MfadOPyU3tXZt++pgcteklZinURSRi7Wg5zKK6RhbU+Xg92KZPT0ng4on51Abb9NnDk7wuU2TI\nKNRFJCoc6ujmlQ3Nwcl2TTQf7CDOoHp0DjWVgc/ix+WnqU0vUU2hLiJRp6fH8faO1sA98XU+3tsV\neEzE6NxUaioCq9pNH5NDUoLa9BJdFOoiEvV2thxmYb2PRXWNvLpxD53dPaQnJ3BRsE1/Sbna9BId\nFOoiElPaOrt5ZX1z7y1zTQcCbfqq0mxqKwOPkB1foDa9RCaFuojErJ4exzs7WnvXpl+zM9CmL8kZ\nRm1FIXMrC5lRpja9RA6FuohI0K7Ww4Er+Dofr25opqO7h7TkBC6amEdNRSGXlOeTm5bsdZkiJ6RQ\nFxE5jsOdfl7dEHiE7MI6H74DHViwTX9kst3EQrXpJbwo1EVETqGnx7Fm534WBNemf2dHKwDF2cOo\nDa5NP3NsDskJ8R5XKrFOoS4icpp2t7azqN7HovpGXtnQTHtXD8OT4rlwQnA2fUUBeWrTiwcU6iIi\nA3C4089rG5uDt8z52L2/HTOYVpLF3ODSteWF6WrTy5BQqIuIDBLnAm36hXU+FtY38nZDoE0/KmsY\ntcGHz8xSm15CSKEuIhIivv2BNv2COh+vbGiivauH1KR4LpyQR21FIZdUFJCfrja9DB6FuojIEGjv\n8vP6xj29k+12tQba9FOLs3on21WOUJteBkahLiIyxJxzvLfrSJvex+rtLQCMzEyhJtimnz02l5RE\ntenl9CjURUQ85jvQzuLgojcvr2/mcJefYYnxXDAhj7nB2fQF6SlelykRQKEuIhJG2rv8vL5pD4vq\nAkvX7mxtB2BqcSa1wdn0k0ZkqE0vx6VQFxEJU8456nYdYFF9IwvqfKxuaME5GJGZ0ruq3exxatPL\n+xTqIiIRoulAB4vXBq7gX17fTFtnoE1//vi8wC1zFQUUZKhNH8sU6iIiEaij28/STXtZWBdYm35H\ny2EAphRn9l7FnzVSbfpYo1AXEYlwzjnWNh4IzKava2Tl9kCbvjAjmZqKwDPizx+fpzZ9DFCoi4hE\nmeaDHSyu97Go3sdL65o41OknJTGO88fl9T58JiUxnqKMFOLjdCUfTRTqIiJRrKPbzxub9gZXtmuk\nYd/h3tfy0pL40KQiPjplBOeNy1WrPgoo1EVEYoRzjvW+g6ze3kKnv4fXNu5hSb2PQ51+PjplBN/9\n2GSyUpO8LlMGoL+hnjAUxYiISOiYGRML05lYmA7AZ2aOpr3Lz8Mvb+InC9azbMtevn55JVdMGUmc\n2vJRLc7rAkREZPClJMZzW80Enrz1fLJTk/jK/FVc89BSHl/RwK7Ww6c+gEQkhbqISBQ7uziT5758\nId+/egp1u/fzT39YzT8/tcbrsiREFOoiIlEuLs74dHUJr3+tlo+cPYIX1/nYvrfN67IkBBTqIiIx\nIi05ga9/pJI4My77z5f5+Ysb6fb3eF2WDCKFuohIDBmVNYynvng+M8ty+N5f6pl9zyL+6Q+rWVTf\nSGe3Aj7S6ZY2EZEY5JxjYZ2Pp1fv5MW1Pva3d5M5LJGaigKmj8khJTGOxPg45lYWMixJK9Z5Tbe0\niYjICZkZcycVMndSIZ3dPby8vok/v72LF9c18eTKHb3jfvSpqXzy3GIPK5XToVAXEYlxSQlxwWe6\nF9LT49jRcpg9hzr52H2v8pd3d5OVmsi5o7O1gE0EUKiLiEivuDijJCeVEZkpXDghjyVrA8vQAowv\nSKN6dDZXThvJeePyPK5UjkehLiIiH5AQH8dvbpjJ4U4/qxtaWLF1H8u37OXZYIv+9a/Vel2iHIdC\nXURETmhYUjyzxuYya2wuAPct3sAPnl9L4/52CjNSPK5OjqVQFxGRfisKBvnMf1tIevLREfLxqlH8\n61WTvShLghTqIiLSb5+oGsWYvFReWtfMgfbu3v3Pr9nNym0tHlYmoFAXEZHTYGacOzqHc0fnHLU/\nPg5+8cpm1u4+QHlRukfViVaUExGRAbt1znjSkhP47p/fI9IWNYsmCnURERmw7OFJ/MPciby8vpmn\nVu049Q9ISCjURURkUHxu9mhmjMnhriffZe+hTq/LiUkKdRERGRQJ8XHccVk5hzr9LN20x+tyYpJC\nXUREBs3Zo7LIS0vmt0u3el1KTFKoi4jIoElKiOOWi8fy2sY9rNnZ6nU5MUehLiIig+riifkAbPAd\n9LiS2BPSUDezS81srZltMLM7TzBmjpmtMrM1ZvZiKOsREZHQG5U9jOFJ8fzg+bWs3q4FaYZSyELd\nzOKB+4DLgEnAtWY26ZgxWcD9wJXOubOAT4WqHhERGRqpSQn85saZOAefevB1XtvQ7HVJMSOUV+oz\ngA3OuU3OuU5gPnDVMWOuA55wzm0DcM75QliPiIgMkarSbJ790gWMyUvlpt+s0OfrQySUoT4K2N5n\nuyG4r6+JQLaZLTGzFWb2ueMdyMxuMrPlZra8qakpROWKiMhgyh6exK++MIOMlASuf3QZ2/a0eV1S\n1PN6olwCcC7wEWAe8M9mNvHYQc65h5xz1c656vz8/KGuUUREztCIzGH86gsz6OzuYe6PX+Qr81fy\n2oZmenq0lGwohDLUdwAlfbaLg/v6agCed84dcs41Ay8BU0NYk4iIDLEJhek89cXzuXZ6CYvrfVz3\n8Bt8/P5X6ej2e11a1AllqC8DJphZmZklAdcAzxwz5mngAjNLMLNUYCZQF8KaRETEA2V5w/mXqybz\n5jfm8q9XncXqhlYeenGT12VFnZCFunOuG7gNeJ5AUP/eObfGzG4xs1uCY+qAvwJvA28CDzvn3g1V\nTSIi4q2UxHg+N3sMl59dxL2LN7B9rz5nH0wWaY/Iq66udsuXL/e6DBERGYCdLYeZ+x8vMrMsh0eu\nn46ZeV1SWDOzFc656lON83qinIiIxKCRWcP4vx+ayOK1TTz3zm6vy4kaCnUREfHE9eeNYfKoDL71\nzBq14QeJQl1ERDyREB/Hf3x6Gp3dfq57eCm/WbpV4T5ACV4XICIisWtiYTq/+sIM/vF/VvHPTwXm\nSY8vSGPOxHxuvHAsRZkpHlcYWRTqIiLiqXNKs1n8T3PY1HyIJWubWLLWx69e38Kanft57KZZXpcX\nUdR+FxERz5kZ4/LTuOGCMn5zw0zuuLSC1zft4a1t+7wuLaIo1EVEJOxcO6OUrNRE7l+8wetSIopC\nXUREws7w5AQ+f14ZC+p8rGs84HU5EUOhLiIiYemzs0eTlBDHr1/f4nUpEUOhLiIiYSlneBJXTBnJ\nE2/tYH97l9flRASFuoiIhK2/O280bZ1+nljR4HUpEUGhLiIiYWtKcRZVpVn8ZOF6NvgOel1O2FOo\ni4hIWPvx30wjIc74u0fepHF/u9flhLWTLj5jZn8CTvgYN+fclYNekYiISB+jc4fzyPXTueahpVz/\n6DL+5+ZZZKQkel1WWDrVinI/HJIqRERETmJKcRYP/O253PDLZdzymxU8+vnpJCfEe11W2DlpqDvn\nXhyqQkRERE7m4on5/Psnp/D//rCar/3xHX706al6DvsxTtV+f4eTt9+nDHpFIiIiJ/DJc4vZvq+N\nnyxYz8Xl+Vw1bZTXJYWVU7XfPzokVYiIiPTTbZeM58V1TXzz6TXMLMvVk9z6OOnsd+fc1pN9DVWR\nIiIiR7z/HPYe7vjj2zh3woZyzOnXLW1mNsvMlpnZQTPrNDO/me0PdXEiIiLHU5Y3nK9fXsGL65r4\n3ZvbvC4nbPT3PvV7gWuB9cAw4EbgvlAVJSIicip/O2s0F07I47vP1rGj5bDX5YSFfi8+45zbAMQ7\n5/zOuUeBS0NXloiIyMmZGXd/7GwOd/l5dvVOr8sJC/0N9TYzSwJWmdn3zewfT+NnRUREQqI0N5VJ\nIzJYWO/zupSw0N9g/mxw7G3AIaAE+GSoihIREemv2soCVmzdR0tbp9eleK6/od4MdDrn9jvn/gW4\nHVCvQ0REPFdTUYC/x/HiuiavS/Fcf0N9IZDaZ3sYsGDwyxERETk9U4uzyB2exMI6teD7G+opzrne\nZ94Fv089yXgREZEhERdnXFJRwJK1Prr9PV6X46n+hvohM6s6smFm5wK6f0BERMJCbUUB+9u7WbF1\nn9eleOpUy8Qe8Q/AH8xsJ2BAEfA3IatKRETkNFwwIY+k+Dj+8u5uZo7N9bocz/TrSt05twyoAP4e\nuAWodM6tCGVhIiIi/ZWeksiHzirkyZU7aO/ye12OZ/q7TGwqcAfwFefcu8AYM9PDXkREJGxcN6OU\n1sNd/PXd3V6X4pn+fqb+KNAJzA5u7wC+G5KKREREzsDssbmU5qTyWAyvBd/fUB/nnPs+0AXgnGsj\n8Nm6iIhIWIiLM66ZUcIbm/eysengqX8gCvU31DvNbBjgAMxsHNARsqpERETOwNXnFgPw/JrYbMGf\ncva7mRnwIPBXoMTM/hs4H7g+tKWJiIicnoL0FMzgcGdsTpY7Zag755yZ3Q7MAWYRaLt/xTnXHOLa\nRERETltiXBxdfud1GZ7o733qbwFjnXN/DmUxIiIiAxUfZ/h7YnNluf6G+kzgM2a2lcBT2ozARfyU\nkFUmIiJyBhLiTVfqpzAvpFWIiIgMkoQ4w9+jUD8h59zWUBciIiIyGBLi4+iO0fZ7f29pExERiQgJ\ncUZ3jLbfFeoiIhJVEuKN7hhtvyvURUQkqiQnxLNtbxs9MRjsCnUREYkqn5s9mhVb9/HAixu9LmXI\nKdRFRCSqfHbWaK6cOpIfvbCWV9bH1jppCnUREYkqZsY9nzyb8QVpfHn+Sna1Hva6pCGjUBcRkaiT\nmpTAA397Lh1dfm7977fo7I6NW9wU6iIiEpXG5afxg09NZeW2Fu7+83telzMkFOoiIhK1Lj97BDde\nUMavXt/Kqxui//N1hbqIiES1f5pXzqisYfzbc3VRf5tbSEPdzC41s7VmtsHM7jzJuOlm1m1mV4ey\nHhERiT0pifHcPq+cNTv388zqnV6XE1IhC3UziwfuAy4DJgHXmtmkE4z7d+CFUNUiIiKx7cqpI5k8\nKoMfPL8RuJjjAAAVm0lEQVSW9i6/1+WETCiv1GcAG5xzm5xzncB84KrjjPsS8EfAF8JaREQkhsXF\nGV+/rJIdLYf59etbvC4nZEIZ6qOA7X22G4L7epnZKODjwAMnO5CZ3WRmy81seVNT06AXKiIi0e+8\n8XnMKc/n3kUbaGnr9LqckPB6otxPgDuccye9gdA595Bzrto5V52fnz9EpYmISLS587IKDnZ0c++i\nDV6XEhKhDPUdQEmf7eLgvr6qgflmtgW4GrjfzD4WwppERCSGVRRl8MmqYn69dCu7W9u9LmfQhTLU\nlwETzKzMzJKAa4Bn+g5wzpU558Y458YAjwO3OueeCmFNIiIS475cO4GeHsf9S6Lvaj1koe6c6wZu\nA54H6oDfO+fWmNktZnZLqN5XRETkZEpyUvlUdTHz39zOzpboWhc+pJ+pO+eec85NdM6Nc87dHdz3\noHPuweOMvd4593go6xEREQH44iXjcUTf1brXE+VERESGXHF2Kp+qLuF/lm2nYV+b1+UMGoW6iIjE\npC9eMh6A+xZv9LiSwaNQFxGRmDQqaxjXTC/lD8u3s31vdFytK9RFRCRm3XrJOOLMoua+dYW6iIjE\nrBGZw7h2RgmPv9XAtj2Rf7WuUBcRkZh26yXjiY8zfrZovdelDJhCXUREYlphRgofnlTIqxuavS5l\nwBTqIiIS81KT4nFeFzEIFOoiIiJRQqEuIiISJRTqIiIiUUKhLiIiEiUU6iIiIlFCoS4iIhIlFOoi\nIiJRQqEuIiIxLzUpgZa2Lg51dHtdyoAo1EVEJOZdMXUEh7v8PPv2Tq9LGRCFuoiIxLyq0mwmFqbx\nuze2eV3KgCjURUQk5pkZ180oZXVDK+/uaPW6nDOmUBcREQE+fk4xyQlxPPZm5F6tK9RFRESAzNRE\nPjplJE+v2hmxE+YU6iIiIkHXzSzhYEc3f1odmRPmFOoiIiJBVaXZlBem87sIbcEr1EVERILMjGtn\nlPB2hE6YU6iLiIj08fGqwIS5SLxaV6iLiIj0kTksOGFu5Y6ImzCnUBcRETnGdTNLOdTp55kImzCn\nUBcRETlGVWkW5YXpEXfPukJdRETkGGbGdTNLI27CnEJdRETkOD52zihSEiNrwpxCXURE5Dj6Tphr\n7/J7XU6/KNRFREROYN5ZRRzq9PNOhLTgFeoiIiInUFWaBcCKrfs8rqR/FOoiIiInkJuWzJjcVN5S\nqIuIiES+qtJs3trWgnPO61JOSaEuIiJyElWjs2k+2MH2vYe9LuWUFOoiIiInUVWaDcBb28K/Ba9Q\nFxEROYnyonSGJ8Ur1EVERCJdfJwxrTQrImbAK9RFREROoao0m/rdB8L+qW0KdRERkVOoGp2Nv8ex\nuqHF61JOSqEuIiJyClUlgclyK7cp1EVERCJaZmoi4/KHh/0iNAp1ERGRfjh3dDZvbdsX1ovQKNRF\nRET6oao0m31tXWxuPuR1KSekUBcREemHqtFHFqEJ38/VFeoiIiL9MD4/jfSUhLC+X12hLiIi0g9x\nccY5pdmsDOOV5RTqIiIi/VRVmsXaxgMcaO/yupTjUqiLiIj007mjs3EOVm0Pz8/VQxrqZnapma01\nsw1mdudxXv+Mmb1tZu+Y2WtmNjWU9YiIiAzEtJIszOCtrTEW6mYWD9wHXAZMAq41s0nHDNsMXOyc\nOxv4DvBQqOoREREZqPSURCYWpLMiTD9XD+WV+gxgg3Nuk3OuE5gPXNV3gHPuNefckTOzFCgOYT0i\nIiIDVjU6MFmupyf8FqEJZaiPArb32W4I7juRG4C/hLAeERGRAasqzeJAezcbmw56XcoHhMVEOTO7\nhECo33GC128ys+VmtrypqWloixMREenj3OAiNOF4v3ooQ30HUNJnuzi47yhmNgV4GLjKObfneAdy\nzj3knKt2zlXn5+eHpFgREZH+KMsbTnZqIm+F4efqoQz1ZcAEMyszsyTgGuCZvgPMrBR4Avisc25d\nCGsREREZFGaBRWjCcbnYkIW6c64buA14HqgDfu+cW2Nmt5jZLcFh3wRygfvNbJWZLQ9VPSIiIoPl\nnJIsNvgOcqij2+tSjpIQyoM7554Dnjtm34N9vr8RuDGUNYiIiAy2kpxUAHa1tjO+IM3jat4XFhPl\nREREIklRZgoAu1vbPa7kaAp1ERGR0zQiGOq7Wg97XMnRFOoiIiKnqTBDV+oiIiJRISUxnuzURHbv\nV6iLiIhEvKLMYbpSFxERiQYjMlPYpVAXERGJfEWZKWq/i4iIRIOijBT2HuqkvcvvdSm9FOoiIiJn\n4Mi96r79HR5X8j6FuoiIyBkIx3vVFeoiIiJn4Eioh9Pn6gp1ERGRM1CUOQwIrwVoFOoiIiJnIC05\ngbTkhLC6rU2hLiIicoaKMlN0pS4iIhINRmSmsEufqYuIiES+oowUGnWlLiIiEvmKMlPwHWin29/j\ndSmAQl1EROSMFWWm0OOg6WB4LECjUBcRETlD7y9AEx4teIW6iIjIGSrKCNyrHi6fqyvURUREzlCR\nrtRFRESiQ3ZqIkkJcWGzVKxCXURE5AyZWeBedV2pi4iIRL5wulddoS4iIjIARZkp7NofHo9fVaiL\niIgMQFFmCo2tHfT0OK9LUaiLiIgMxIiMFDr9Pext6/S6FIW6iIjIQITTc9UV6iIiIgNw5F51hbqI\niEiE610qNgzuVVeoi4iIDEBeWjLxccbuVu9nwCvURUREBiA+zihMT2Z3q/dPalOoi4iIDFBhZgq7\nw+BedYW6iIjIAIXLUrEKdRERkQEqyhjG7tZ2nPN2ARqFuoiIyACNyEyhrdPPgY5uT+tQqIuIiAxQ\nYZjcq65QFxERGaDee9UV6iIiIpGtKOPIlbq3M+AV6iIiIgNU2Bvq3t6rrlAXEREZoKSEOPLSkjy/\nV12hLiIiMgiKwuBedYW6iIjIIDhyr7qXFOoiIiKDYERmCrs9flKbQl1ERGQQFGWm0NLWxeFOv2c1\nKNRFREQGQe9tbR5erSd49s6DqKuri4aGBtrbvV9MX04tJSWF4uJiEhMTvS5FRGTQvL8AzWHK8oZ7\nUkNUhHpDQwPp6emMGTMGM/O6HDkJ5xx79uyhoaGBsrIyr8sRERk0RcFQb/TwSj0q2u/t7e3k5uYq\n0COAmZGbm6uuiohEnaIwWCo2KkIdUKBHEP23EpFolJqUQEZKgqe3tUVNqHstPj6eadOmMXnyZK64\n4gpaWlpOOr6lpYX777+/d3vnzp1cffXVZ/z+c+bMobq6und7+fLlzJkz56Q/s2XLFn73u9+d8PU/\n/OEPnHXWWcTFxbF8+fKjXvve977H+PHjKS8v5/nnn+/d/41vfIOSkhLS0tLO7BcREYlgIzKHRe+V\nupldamZrzWyDmd15nNfNzH4afP1tM6sKZT2hNGzYMFatWsW7775LTk4O991330nHHxvqI0eO5PHH\nHx9QDT6fj7/85S/9Hn+qUJ88eTJPPPEEF1100VH733vvPebPn8+aNWv461//yq233orfH7iF44or\nruDNN988s19ARCTCFWWmROdn6mYWD9wHXAZMAq41s0nHDLsMmBD8ugl4IFT1DKXZs2ezY8eO3u0f\n/OAHTJ8+nSlTpvCtb30LgDvvvJONGzcybdo0br/9drZs2cLkyZMB+OUvf8knPvEJLr30UiZMmMBX\nv/rV3mP9/d//PdXV1Zx11lm9xzri9ttv5+677/5APX6/n9tvv723hp///Oe9Nbz88stMmzaNH//4\nxx/4ucrKSsrLyz+w/+mnn+aaa64hOTmZsrIyxo8f3xvks2bNYsSIEad7ykREosIIj5eKDeXs9xnA\nBufcJgAzmw9cBbzXZ8xVwK+dcw5YamZZZjbCObfrTN/0X/60hvd27h9I3R8waWQG37rirH6N9fv9\nLFy4kBtuuAGAF154gfXr1/Pmm2/inOPKK6/kpZde4p577uHdd99l1apVQOCqua9Vq1axcuVKkpOT\nKS8v50tf+hIlJSXcfffd5OTk4Pf7qa2t5e2332bKlClA4I+JJ598ksWLF5Oent57rF/84hdkZmay\nbNkyOjo6OP/88/nwhz/MPffcww9/+EOeffbZ0zofO3bsYNasWb3bxcXFR/0RIyISqwozUmg+2EFn\ndw9JCUP/CXco33EUsL3PdkNw3+mOwcxuMrPlZra8qalp0AsdDIcPH2batGkUFRXR2NjIhz70ISAQ\n6i+88ALnnHMOVVVV1NfXs379+lMer7a2lszMTFJSUpg0aRJbt24F4Pe//z1VVVWcc845rFmzhvfe\ne++on7vrrrv47ne/e9S+F154gV//+tdMmzaNmTNnsmfPnn7VICIip6dyRAZzJuZ7tqpcRNyn7px7\nCHgIoLq62p1sbH+vqAfbkc/U29ramDdvHvfddx9f/vKXcc7xta99jZtvvvmo8cdemR8rOTm59/v4\n+Hi6u7vZvHkzP/zhD1m2bBnZ2dlcf/31H7g1rKamhrvuuoulS5f27nPO8bOf/Yx58+YdNXbJkiVH\nbX/+859n5cqVjBw5kueee+6EtY0aNYrt29//W6yhoYFRoz7wt5iISMy5dHIRl04u8uz9Q3mlvgMo\n6bNdHNx3umMiSmpqKj/96U/50Y9+RHd3N/PmzeORRx7h4MGDQKB17fP5SE9P58CBA6d17P379zN8\n+HAyMzNpbGw84aS4u+66i+9///u92/PmzeOBBx6gq6sLgHXr1nHo0KEP1PDoo4+yatWqkwY6wJVX\nXsn8+fPp6Ohg8+bNrF+/nhkzZpzW7yIiIoMvlKG+DJhgZmVmlgRcAzxzzJhngM8FZ8HPAloH8nl6\nuDjnnHOYMmUKjz32GB/+8Ie57rrrmD17NmeffTZXX301Bw4cIDc3l/PPP5/Jkydz++239+u4U6dO\n5ZxzzqGiooLrrruO888//7jjLr/8cvLz83u3b7zxRiZNmkRVVRWTJ0/m5ptvpru7mylTphAfH8/U\nqVOPO1HuySefpLi4mNdff52PfOQjvVf6Z511Fp/+9KeZNGkSl156Kffddx/x8fEAfPWrX6W4uJi2\ntjaKi4v59re/fZpnT0REzpQF5qiF6OBmlwM/AeKBR5xzd5vZLQDOuQctsArJvcClQBvweefc8hMe\nkED7/dh7puvq6qisrAzFryAhov9mIiL9Z2YrnHPVpxoX0s/UnXPPAc8ds+/BPt874IuhrEFERCRW\naEU5ERGRKKFQFxERiRJRE+qhnBsgg0v/rUREQiMqQj0lJYU9e/YoLCLAkeepp6SkeF2KiEjUiYjF\nZ06luLiYhoYGwnW1OTlaSkoKxcXFXpchIhJ1oiLUExMTKSsr87oMERERT0VF+11EREQU6iIiIlFD\noS4iIhIlQrpMbCiYWROwdRAPmQc0D+LxYpXO48DpHA6czuHA6RwOXCjO4WjnXP6pBkVcqA82M1ve\nn/V05eR0HgdO53DgdA4HTudw4Lw8h2q/i4iIRAmFuoiISJRQqMNDXhcQJXQeB07ncOB0DgdO53Dg\nPDuHMf+ZuoiISLTQlbqIiEiUiJlQN7NLzWytmW0wszuP87qZ2U+Dr79tZlVe1BnO+nEOPxM8d++Y\n2WtmNtWLOsPZqc5hn3HTzazbzK4eyvoiRX/Oo5nNMbNVZrbGzF4c6hrDXT/+f840sz+Z2ergOfy8\nF3WGKzN7xMx8ZvbuCV73JlOcc1H/BcQDG4GxQBKwGph0zJjLgb8ABswC3vC67nD66uc5PA/IDn5/\nmc7h6Z/DPuMWAc8BV3tdd7h99fPfYhbwHlAa3C7wuu5w+urnOfw68O/B7/OBvUCS17WHyxdwEVAF\nvHuC1z3JlFi5Up8BbHDObXLOdQLzgauOGXMV8GsXsBTIMrMRQ11oGDvlOXTOveac2xfcXAroUWxH\n68+/Q4AvAX8EfENZXATpz3m8DnjCObcNwDmnc3m0/pxDB6SbmQFpBEK9e2jLDF/OuZcInJMT8SRT\nYiXURwHb+2w3BPed7phYdrrn5wYCf6XK+055Ds1sFPBx4IEhrCvS9Off4kQg28yWmNkKM/vckFUX\nGfpzDu8FKoGdwDvAV5xzPUNTXlTwJFOi4tGrEl7M7BICoX6B17VEoJ8AdzjnegIXSHKGEoBzgVpg\nGPC6mS11zq3ztqyIMg9YBdQA44D/NbOXnXP7vS1LTiZWQn0HUNJnuzi473THxLJ+nR8zmwI8DFzm\nnNszRLVFiv6cw2pgfjDQ84DLzazbOffU0JQYEfpzHhuAPc65Q8AhM3sJmAoo1AP6cw4/D9zjAh8Q\nbzCzzUAF8ObQlBjxPMmUWGm/LwMmmFmZmSUB1wDPHDPmGeBzwRmLs4BW59yuoS40jJ3yHJpZKfAE\n8FldER3XKc+hc67MOTfGOTcGeBy4VYH+Af35//lp4AIzSzCzVGAmUDfEdYaz/pzDbQQ6HZhZIVAO\nbBrSKiObJ5kSE1fqzrluM7sNeJ7ArM9HnHNrzOyW4OsPEphpfDmwAWgj8FeqBPXzHH4TyAXuD15p\ndjs9GKJXP8+hnEJ/zqNzrs7M/gq8DfQADzvnjnvrUSzq57/F7wC/NLN3CMzgvsM5p6e3BZnZY8Ac\nIM/MGoBvAYngbaZoRTkREZEoESvtdxERkainUBcREYkSCnUREZEooVAXERGJEgp1ERGRKKFQFxER\niRIKdRE5I2YWE+tciEQShbpIFDKzp4IPMlljZjcF911qZm8Fn4+9MLgvzcweNbN3gs98/mRw/8E+\nx7razH4Z/P6XZvagmb0BfN/MZpjZ62a20sxeM7Py4Lh4M/uhmb0bPO6XzKzGzJ7qc9wPmdmTQ3dW\nRKKf/tIWiU5fcM7tNbNhwDIzexr4L+Ai59xmM8sJjvtnAstXng1gZtn9OHYxcJ5zzm9mGcCFwRXK\n5gL/BnwSuAkYA0wLvpYD7COw2mC+c66JwApbjwzerywiCnWR6PRlM/t48PsSAiH7knNuM4Bz7shz\noOcSWPeb4P59/Tj2H5xz/uD3mcCvzGwCgedvJ/Y57oPOue6+72dmvwH+1sweBWYDeiSqyCBSqItE\nGTObQyBUZzvn2sxsCYFHaFacxmH6rh+dcsxrh/p8/x1gsXPu42Y2BlhyiuM+CvwJaCfwx0H3adQk\nIqegz9RFok8msC8Y6BXALALBfJGZlQH0ab//L/DFIz/Yp/3eaGaVZhYHfJwTy+T9x0le32f//wI3\nH5lMd+T9nHM7gZ3AXQQCXkQGkUJdJPr8FUgwszrgHmAp0ESgBf+Ema0G/ic49rtAdnBC22rgkuD+\nO4FngdeAkz0u8vvA98xsJUd3/h4m8OjOt4PHva7Pa/8NbHfO6VGoIoNMT2kTkSFlZvcCK51zv/C6\nFpFoo1AXkSFjZisIfCb/Iedch9f1iEQbhbqIiEiU0GfqIiIiUUKhLiIiEiUU6iIiIlFCoS4iIhIl\nFOoiIiJRQqEuIiISJf4/dyGg8vN1M+MAAAAASUVORK5CYII=\n",
172 |       "text/plain": [
173 |        "<matplotlib.figure.Figure at 0x7f73b08dced0>"
174 |       ]
175 |      },
176 |      "metadata": {},
177 |      "output_type": "display_data"
178 |     },
179 |     {
180 |      "data": {
181 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfUAAAGDCAYAAAAyM4nNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl0m/Wd7/HP15It2ZbjJXFWOyRACIQQkhAgLG3TMiWB\nFhhaZgboBlMOdIF27rlDS2eYKXNbppTS0zktFMqwTadDc7tAWW5YurB0gZJQAiGsgRDiJGR1Fsdx\nvH3vH5IdyZZteZElPX6/zslB0vNE+uYh8PHvp9/z+5q7CwAAFL6iXBcAAABGBqEOAEBAEOoAAAQE\noQ4AQEAQ6gAABAShDgBAQBDqQIExs38ysztG4XOWmtmvsv05g2Vm95jZATNryHUtQL4h1IEC4+7/\n7u6XjcJHXS/phq4nZvaEmW03s71m9qKZnZd07CNm9gcz221m75nZHWZW0dcbm9mpZvacme0zs5fM\n7PQex2vN7F4z22NmjWb2P13H3P0SSWeN6J8UCAhCHUAvZnaipEp3fzbp5X+QVOfu4yRdLuknZjYl\ncaxS0jclTZV0jKRpkr7Tx3vXSHoocbxK0o2SHjKz6qTT7pP0nqTpkiZKummE/mhAoBHqQJ4ys6+a\n2abEaPZ1Mzsj8fp1ZvaTxOObzawp6Ve7mV2XODbVzH6ZGF2vN7MvDeLjz5L0VPIL7v6iux/seiqp\nWFJ94ti97v6ouze7e6Ok/5R0Wh/vfaqkre7+c3fvcPefSNou6WOJus9MvO/V7r7H3dvc/YVB1A6M\nWYQ6kIfMbLakKyWd6O4VkpZKeqfnee5+pbvH3D0m6XRJjZIeMLMixUfDLyo+aj5D0j+Y2dLE+59u\nZrv7KeE4Sa+nqethM2uR9GdJT0pa1cfvf7+ktRn8UbvfWtLcxOPFic/+LzPbaWYrzewDg3gvYMwi\n1IH81CEpImmOmRW7+zvu/lZfJ5tZraRfSboqMao9UVKtu/8fd29197cVHz1fKEnu/gd3r+rn86sk\n7ev5ort/VFKFpLMlPe7unWlq+bCkz0j61z7e+xlJU8zsQjMrNrPPSDpCUlnieJ2kMyU9IWmypO8q\n/oPKhH7qBSBCHchL7r5O8e+wr5O0zcyWm9nUdOeaWbGkX0i6192XJ14+TNLUxMK13YlR+T9JmpRh\nCY2Kh3e62trc/RFJZ5rZuT1qWSzpXkkXuPsbffz+nZL+WtL/lrRV0jJJv5HUtZr9gKR33P3OxGct\nl7RRfU/nA0gg1IE8lfie+nTFA9olfbuPU38gaa+ka5Ne2yhpvbtXJf2qcPezM/z4lyQdNcA5YcVH\n2JIkM1sg6UFJf+/uv+3vN7r7U+5+orvXSPqUpKMlPZf02T3bR9JOEsgAoQ7kITObbWYfMrOIpBbF\nR6/pprqvkPQBSZ/oMRX+nKR9icV2pWYWMrO5iVXtmViReN+uzznazM5KvFexmX1S8e/Nn0ocnyvp\nUcWn/x9KU+d1ZvZk0vMFifcZp/jK9o3u/lji8P2Sqs3sM4m6L1B8Sv6PGdYOjFmEOpCfIorfI75D\n8Vu7Jkr6WprzLpJ0uKTNSSvg/8ndOyR9VNJ8SesT73OH4reeyczeZ2ZNfX24u/9F0h4zOznxkinx\nVYDiK9W/LOnvEudJ8an0Wkl3JtWRvFCuXqmh/JVETRslTZF0ftJn75J0rqR/lLRH0jWSznP3HX3V\nCyDO3JnVAtBb4tayL7j7X4/Ae62WdEbi+/Thvtedkv5G0jZ3P3K47wcECaEOAEBAMP0OAEBAEOoA\nAAQEoQ4AQEAQ6gAABEQ41wUM1oQJE3zGjBm5LgMAgFHz/PPP73D32oHOK7hQnzFjhlat6quHBAAA\nwWNmGzI5j+l3AAACglAHACAgCHUAAAKCUAcAICAIdQAAAoJQBwAgIAh1AAACglAHACAgCHUAAAIi\na6FuZneZ2TYze7mP42Zm3zezdWb2kpktzFYtAACMBdkcqd8jaVk/x8+SNCvx63JJt2axFgAAAi9r\noe7uT0va1c8p50n6scc9K6nKzKZkq550dje36nevbdUbW/epubV9ND8aAIARl8uGLtMkbUx63pB4\nbUvPE83scsVH85o+ffqIFfBSwx79/T2HmsNMiJVoWnWZ6qtLVV9TprrqUtVXx/85rbpUkXBoxD4b\nAICRVhBd2tz9dkm3S9KiRYt8pN73hMOqdd8XTtXGXc1qaDyghsZmbdx1QGs27dFja99TW8ehjzKT\nJlVE40FfEw/+uuoy1dXEg39KZVThEOsOAQC5k8tQ3ySpPul5XeK1UVMeCWvh9GotnF7d61hHp2vr\n3pbuwN+YCPyGxmY9t36XHlh9QJ1JP16EikxTKqPdo/vukX5NmeqryzSxIqKiIhvFPx0AYKzJZag/\nKOlKM1su6WRJe9y919R7roSKTFOrSjW1qlQnpzne1tGpLbtbtLGxuXuEH398QE+9sV3b9h1MOb8k\nVKRp1aWqS4zw62sS/0wE//jyEpkR+gCAoctaqJvZTyUtkTTBzBokfV1SsSS5+22SVkg6W9I6Sc2S\nLs1WLdlQHCrS9PFlmj6+LO3xlrYObdp9IGWk35AY6T+2+T3t2t+acn5pcah7ZH9otN8V/GWqLCse\njT8WAKCAZS3U3f2iAY67pC9m6/NzLVoc0hG1MR1RG0t7vOlguzY1doV+szY2HvoBYOU7u7SvJXU1\nfkU03D2y7xrp1yd9p18eKYjlEQCALCIJciQWCWv25ArNnlyR9vie5raUqf2u4H9n5379/s0dOtDW\nkXJ+TXnJodX6SVP7dYnV+9FiVu4DQNAR6nmqsqxYlWWVmjutstcxd9fO/a3xaf1dzd3f5W/c1axX\ntuzVr1/ZqtaOzpTfM7EikrJwL/nxlKqoilm5DwAFj1AvQGamCbGIJsQiml9f1et4Z6dr276DidF9\n0kh/1wE9v6FRD7+0RR1JS/eLTJpSGb8Xvz7NIr5J46IKsXIfAPIeoR5ARUWmyZVRTa6MatGMml7H\n2zs6tWVPS/cIvyFpMd8f1+3Q1n0t8qTb9YpD8TsBkkf4yav4a2MRVu4DQB4g1MegcKgoPvVek37l\n/sH2Dm3e3fMe/fjj37y6VTuaUlfuR8JFvVbuJy/mqyorJvQBYBQQ6uglEg5p5oRyzZxQnvb4gdaO\n7qn97u/1dx1Qw+5mvfDubu050JZyfiwS7h7Z99yRr76mVBVRbtcDgJFAqGPQSktCmjWpQrMmpV+5\nv7elTQ1Jm/F03bbX0NisZ97aof2tqSv3K0uLD92i12MxX111mUpLWLkPAJkg1DHixkWLNWdqseZM\nHdfrmLursbmtx6168cdvbN2n3722TQfbU1fuT4hF+tyYZ1pVqUrCrNwHAIlQxygzM9WUl6imvETz\n6tKv3N/RdFAbuxvsHPpe/8WNu/XImi1q70xttDN5XDTpHv3U7no02gEwlhDqyCtFRaaJ46KaOC6q\nEw5L32jnvb0tatiVugvfxsZmPfv2Tm1ZvSll5X64yDSlKqq6qt678NXXlKk2RqMdAMFBqKOghIpM\n06pKNa2PRjut7Z3asudA2o15nnh9u7b3bLQTLlJdVeIe/V4b85SqhkY7AAoIoY5AKQkX6bDx5Tps\nfPqV+y1tHfF78xPb7ibfo//ymi1qbE5duV9WEkqZzk+9R79MlaWs3AeQPwh1jCnR4pCOnBjTkRP7\nbrTT0GMXvq7R/nPrd2nfwd6NdtLtwtd1+x6NdgCMJv6PAySJRcI6evI4HT05/cr9vQfaUzbj6Xr8\n9vb9euqN7WppS125X1NeEr8nP2XlfvzxtCoa7QAYWYQ6kCEzG7DRzo6m1kNT+0kj/lc279Wv1/Zu\ntDNpXKTHCP/Qjnw02gEwWIQ6MELMTLUVEdVWRLRgeu+V+52drq37WlJ34Uvcp79qQ6Me6qPRTrru\nenXVpTTaAdALoQ6MkqIi05TKUk2pLNWJaRrttHV06r2uRjs9duT7w5vpG+1Mqzq03W5dj9Cn0Q4w\n9hDqQJ4oTm60c0Tv4wfbO7Sp8UDSd/mHVvE/vnardu5PbbQTLS46FPQpi/nijytLabQDBA2hDhSI\nSDikw2tjOrw2/cr95tb2Q7fr7UrdmOcvGxq1tyV15X5Xo536Hrvwdf1gEWPlPlBw+K8WCIiykrCO\nmlSho/potLPnQOqe+10/ALy7s1l/XLdDzT0a7VSVFfe6Xa8uqcMeK/eB/EOoA2NEZWmxKksrdezU\n9Cv3G5vbeu3Ct7HxgF57b59+8+o2taZptNNXd72pNNoBcoJQB5DSaOf4+v4a7fTemGf1xt1a0aPR\nTlF3o534Xvs9b9ubUlnKyn0gCwh1AANKbbTT+3h7R2e80U6PJjsNuw7o2bd2asve3o12plaV9l7E\nl/gnjXaAoSHUAQxbONS10r5Miw8f3+t4a3unNu8+tHI/eaT/u9e3pW+0U917hN+1I191GSv3gXQI\ndQBZVxIu0owJ5Zoxob9GO4ea7CTvyLemYXevRjvlJaFe9+V3jfTra8o0LkqjHYxNhDqAnIs32qnQ\nkRPTr9zf19KWWK3fu6Xun9fvUlOPRjvjouG0u/B1/bOshP/1IZj4mw0g71VEi3XMlGIdMyV9o509\nB9pStt3terxue5OefGNbr0Y748tLUm7PS75tb1p1qSJhbtdDYSLUARQ0M1NVWYmqykp0XF3fjXaS\nu+t13af/8qY9emzte2rr8JTfM2lcJKWjXn1iFX99dZmmVEYVptEO8hShDiDQkhvtLEzTaKej07Vt\nX0uvXfg27mrWc+t36YHVB5R0t55CRabJ46JJ9+gf+i6/rrpUkyqirNxHzhDqAMa0UFKjnZNm9tNo\nJ83GPE+/uV1b9/ZYuR8q0rTq0u7Fe4c25ok/nxArYeU+soZQB4B+pDTaSaOlrUObdx/Qxl736Dfr\n8c3v9Wq0U1ocSgR++pa6NNrBcBDqADAM0eL+G+3sP5jcaKc5pcve82ka7VREwqqrSdNdLzHdX06j\nHfSDvx0AkEXlkbBmT67Q7Ml9N9pJXsDX9XjDzv36w5s7dKAttdFOdVlxjwV8qV32aLQzthHqAJBD\nlaXFqpxWqbnT0q/c37W/NWUznq7v9V/bsk+/eWWbWjtSb9errYik3YWvq9FOMSv3A41QB4A8ZWYa\nH4tofCyi+X002tnedPDQtH7SYr6/vNuoh1/aoo50jXZ6fJff1VZ38rgojXYKHKEOAAWqqMg0aVxU\nk8ZFtWhG7+NdjXaSR/gNiR8A/vTWDr23tyWl0U5xKLXRzqEFfPHgr62IsIgvzxHqABBQyY12TlHv\nRjsH2zu0ZXdLakvdxFT/b17dph1NqbfrRZIb7XTvwnfoMY12co9QB4AxKhIO9dto50BrhzbtTv0u\nv2uK/8WG3dqdptFOaoOd1B35aLSTfYQ6ACCt0pLMGu1s7NFZr6GxWc+8tVP7W1NX7leWFsdH9VWp\nu/B17cxXWsLK/eEi1AEAQzJQo53dzW0pI/yue/Tf3LZPT7y+TQfbU1fuT4iV9NiF79DjqVVRGu1k\ngFAHAIw4M1N1eYmqy0s0r673yn33rpX7hxrsdAX/mjSNdsykSRXJe+6XptyjT6OdOEIdADDqzEwT\nK6KaWBHVCYelb7SzdW9Lr134Ghqb9ef1u/SrNI12plRGVV9dprPnTdEnTpo+JhvrmLsPfFYeWbRo\nka9atSrXZQAAcqito7N75X7yxjwPrN7cfc6S2bWaWBHRVR+a1efe/YXCzJ5390UDncdIHQBQcIpD\nRZo+vkzTx6eG9fXnH6ebf7dO67Y1aeveFv1x3Q6t2bRX933+1DGxEI9QBwAERiwS1jVnHd39/IHV\nm/Tl5av142fe0RUfOCJ3hY0SVhUAAALro/Om6uSZNbrp8de1Zc+BXJeTdYQ6ACCwQkWm688/Tm0d\nrqvufUENjc25LimrCHUAQKAdOTGm686Zo1UbGvW+G5/QT57dkOuSsoZQBwAE3iWnzdSv/9f7NXN8\nub79yGt6a3tTrkvKCkIdADAmzJpUoR9+cqHMpE/e8eeUtrRBkdVQN7NlZva6ma0zs2vSHK80s4fM\n7EUzW2tml2azHgDA2Hb05HG67txjtWVPi9Zs2pPrckZc1kLdzEKSbpF0lqQ5ki4yszk9TvuipFfc\n/XhJSyR918xKslUTAABLZk+UmfTQi5sHPrnAZHOkfpKkde7+tru3Slou6bwe57ikCos34I1J2iWp\nPYs1AQDGuJryEp1x9CTd9cf1emzte7kuZ0RlM9SnSdqY9Lwh8VqymyUdI2mzpDWSvuzunQIAIIt+\ncNECzaur0peXv6CV7+zKdTkjJtcL5ZZKWi1pqqT5km42s149/MzscjNbZWartm/fPto1AgACprQk\npDs/s0hTKkv1iTv+rEdfDsaIPZuhvklSfdLzusRryS6VdJ/HrZO0XtLRPc6Ru9/u7ovcfVFtbW3W\nCgYAjB0TYhH98vOn6pjJFbr6Fy9q1/7WXJc0bNkM9ZWSZpnZzMTitwslPdjjnHclnSFJZjZJ0mxJ\nb2exJgAAutWUl+imvzleB1o79I8/f1Gt7YX9DXDWQt3d2yVdKekxSa9K+pm7rzWzz5nZ5xKnfUPS\nqWa2RtJvJX3V3XdkqyYAAHqaNalC1517rH732jZd/YsXc13OsGS1S5u7r5C0osdrtyU93izpzGzW\nAADAQD65+DC9uHG3fvmXBt3wsXkF26Y11wvlAADIC3+9YJo6XXrqjcJdkE2oAwAg6aSZNYqEi/T8\nhsK9xY1QBwBAUnGoSFOrSrV5T0uuSxkyQh0AgISa8hI1FvCtbYQ6AAAJRZbrCoaHUAcAICAIdQAA\nEiqixVqzaY/+uK4wt0wh1AEASLjunGM1eVxUn77rOT368pZclzNohDoAAAnTx5fpvi+cquk1Zfrv\nZzfkupxBI9QBAEhSES3WB46q1fMbGgtuL3hCHQCAHhYfPl4tbZ164d3GXJcyKIQ6AAA9nHrkeIWL\nTE8W2JaxhDoAAD2MixbrhMOq9eTrhDoAAAVvyeyJenXLXr29vSnXpWSMUAcAII0LTqhTaXFI3/vN\nm7kuJWOEOgAAadRWRPTZ02fqoRc369Ute3NdTkYIdQAA+nDZ+2aqvCSkW598K9elZIRQBwCgD1Vl\nJfrk4sP08Eub9c6O/bkuZ0CEOgAA/fjs+2YqHCrSj57O/9E6oQ4AQD8mVkT18YV1uu8vm9TS1pHr\ncvpFqAMAMIAzj52kg+2dem79rlyX0i9CHQCAASyeOV4l4SI9lec7zBHqAAAMoLQkpJNn1hDqAAAE\nwQeOqtW6bU1qaGzOdSl9ItQBAMjAB4+eKEl6Io/3gyfUAQDIwOETynXY+DI9+dq2XJfSJ0IdAIAM\nmJmWHFWrP761Q+0dnbkuJy1CHQCADNVVl6mlrVMt7YQ6AAAFrThkkqQ2Qh0AgMJWHI7HZhvT7wAA\nFLbiUDw2Wwl1AAAKW0moa6TuOa4kPUIdAIAMdY/U+U4dAIDC1r1Qjul3AAAKW9dIvb2T6XcAAApa\nqCg+Uu/oZKQOAEBBCydCvZ2FcgAAFLaukTrT7wAAFLhwiFAHACAQwkXx2OQ7dQAAClyI79QBAAiG\naHFIktTc2pHjStIj1AEAyNCUyqgkafOeAzmuJD1CHQCADJVHwqoqK9bm3YQ6AAAFb1pVqTY1EuoA\nABS8qVWl2sRIHQCAwtc1UnfPvxXwhDoAAINQV12q/a0d2nugPdel9EKoAwAwCFOrSiVJDbubc1xJ\nb4Q6AACDMC0R6vm4WI5QBwBgEKrKiiVJ+1rG2PS7mS0zs9fNbJ2ZXdPHOUvMbLWZrTWzp7JZDwAA\nw2WyXJfQp3C23tjMQpJukfRhSQ2SVprZg+7+StI5VZJ+KGmZu79rZhOzVQ8AAEGXzZH6SZLWufvb\n7t4qabmk83qcc7Gk+9z9XUly921ZrAcAgEDLZqhPk7Qx6XlD4rVkR0mqNrMnzex5M/t0ujcys8vN\nbJWZrdq+fXuWygUAoLDleqFcWNIJkj4iaamkfzGzo3qe5O63u/sid19UW1s72jUCAFAQsvaduqRN\nkuqTntclXkvWIGmnu++XtN/MnpZ0vKQ3slgXAACBlM2R+kpJs8xsppmVSLpQ0oM9znlA0ulmFjaz\nMkknS3o1izUBABBYWRupu3u7mV0p6TFJIUl3uftaM/tc4vht7v6qmT0q6SVJnZLucPeXs1UTAABB\nls3pd7n7Ckkrerx2W4/n35H0nWzWAQDAWJDrhXIAAGCEEOoAAAQEoQ4AQEAQ6gAABAShDgBAQBDq\nAAAEBKEOAMAQdLrnuoReCHUAAAahtCQkSWpp68hxJb0R6gAADEIsEt+3rekgoQ4AQEGLFhepyKT9\nB9tzXUovhDoAAINgZiqPhNVEqAMAUPgqCHUAAIKhPBLOy+n3fru0mdlDkvpcs+/u5454RQAA5Ll8\nnX4fqPXqTaNSBQAABSRWiKHu7k+NViEAABSKWCSsbftacl1GLwNNv69R/9Pv80a8IgAA8lz8O/X8\nu099oOn3j45KFQAAFJBYJFSQ0+8bRqsQAAAKRddCOXeXmeW6nG4Z3dJmZovNbKWZNZlZq5l1mNne\nbBcHAEA+ikXD6uh0HWzvzHUpKTK9T/1mSRdJelNSqaTLJN2SraIAAMhnh/Z/z68p+Iw3n3H3dZJC\n7t7h7ndLWpa9sgAAyF/lJfFQz7cNaAZaKNel2cxKJK02sxslbRG70QEAxqjyxEh9X0t+hXqmwfyp\nxLlXStovqV7Sx7NVFAAA+axr+r1QR+o7JLW6e4ukfzOzkKRI9soCACB/xaKJUG/Nr1DPdKT+W0ll\nSc9LJf1m5MsBACD/xSIhSVJTnm1Ak2moR929qetJ4nFZP+cDABBYXd+pNxXod+r7zWxh1xMzO0HS\ngeyUBABAfisv8O/U/0HSz81ssySTNFnS32WtKgAA8ljXLW35dp96RqHu7ivN7GhJsxMvve7ubdkr\nCwCA/BUqMpUWh9RciAvlzKxM0lclfdndX5Y0w8xo9gIAGLOKTPI++5jmRqbfqd8tqVXSKYnnmyR9\nMysVAQCAIck01I9w9xsltUmSuzcr/t06AADIE5mGequZlUpySTKzIyQdzFpVAABg0AZcKGfxRrG3\nSXpUUr2Z/Y+k0yRdkt3SAADAYAwY6u7uZna1pCWSFis+7f5ld9+R5doAAMAgZHqf+l8kHe7u/y+b\nxQAAgKHLNNRPlvQJM9ugeJc2U3wQPy9rlQEAgEHJNNSXZrUKAAAwbJnuKLch24UAAIDhyfSWNgAA\nkOcIdQAAAoJQBwAgIAh1AAACglAHACAgCHUAAAKCUAcAICAIdQAAAoJQBwAgIAh1AAACIquhbmbL\nzOx1M1tnZtf0c96JZtZuZhdksx4AAIIsa6FuZiFJt0g6S9IcSReZ2Zw+zvu2pMezVQsAAGNBNkfq\nJ0la5+5vu3urpOWSzktz3lWSfilpWxZrAQAg8LIZ6tMkbUx63pB4rZuZTZN0vqRb+3sjM7vczFaZ\n2art27ePeKEAAARBrhfK/Yekr7p7Z38nufvt7r7I3RfV1taOUmkAABSWjPqpD9EmSfVJz+sSryVb\nJGm5mUnSBElnm1m7u/8qi3UBABBI2Qz1lZJmmdlMxcP8QkkXJ5/g7jO7HpvZPZIeJtABABiarIW6\nu7eb2ZWSHpMUknSXu681s88ljt+Wrc8GAGAsyuZIXe6+QtKKHq+lDXN3vySbtQAAEHS5XigHAABG\nCKEOAEBAEOoAAAQEoQ4AQEAQ6gAABAShDgBAQBDqAAAEBKEOAMAQhIpMbR39ti4ZdYQ6AABDUBEt\nVtPBjlyXkYJQBwBgCMojIe0/2J7rMlIQ6gAADEF5JKwmQh0AgMIXI9QBAAiGWCTM9DsAAEHA9DsA\nAAHB9DsAAAHRNf3u7rkupRuhDgDAEJRHwup06UBb/tyrTqgDADAEsWhYkvJqCp5QBwBgCGKRkCRp\nfx7tKkeoAwAwBOUliZF6CyN1AAAKGtPvAAAERCwSD/V82oCGUAcAYAjKI4zUAQAIhApCHQCAYChn\n+h0AgGAoKwnJjJE6AAAFz8wUK8mv/d8JdQAAhqg8z9qvEuoAAAxRLMpIHQCAQIj3VGebWAAACl4s\nEmL6HQCAIIhFwuz9DgBAEMSn3wl1AAAKXiwS1v5WQh0AgILXNf3u7rkuRRKhDgDAkJVHwmrvdB1s\n78x1KZIIdQAAhizf2q8S6gAADFEszzq1EeoAAAxRvvVUJ9QBABii7pF6ntyrTqgDADBEsWjiO/U8\nua2NUAcAYIhikZAk5c3+74Q6AABDVM70OwAAwcAtbQAABER5CavfAQAIhKIiU1lJiFAHACAIYpEw\n0+8AAARBLI/arxLqAAAMQz71VCfUAQAYhjEz/W5my8zsdTNbZ2bXpDn+CTN7yczWmNmfzOz4bNYD\nAMBIi4/UA775jJmFJN0i6SxJcyRdZGZzepy2XtIH3P04Sd+QdHu26gEAIBtikZCaDrblugxJ2R2p\nnyRpnbu/7e6tkpZLOi/5BHf/k7s3Jp4+K6kui/UAADDiYtGw9gd9pC5pmqSNSc8bEq/15bOSHsli\nPQAAjLh8WigXznUBkmRmH1Q81E/v4/jlki6XpOnTp49iZQAA9K8iElZre6da2ztVEs7t+vNsfvom\nSfVJz+sSr6Uws3mS7pB0nrvvTPdG7n67uy9y90W1tbVZKRYAgKEoz6P937MZ6islzTKzmWZWIulC\nSQ8mn2Bm0yXdJ+lT7v5GFmsBACAruju15UGoZ2363d3bzexKSY9JCkm6y93XmtnnEsdvk/SvksZL\n+qGZSVK7uy/KVk0AAIy0irEQ6pLk7iskrejx2m1Jjy+TdFk2awAAIJvGyvQ7AACBl0/T74Q6AADD\nUBEl1AEACASm3wEACIhYSddIPfe7yhHqAAAMQ3kkJElqamGkDgBAQQuHihQtLtL+VkIdAICCF8uT\n/d8JdQAAhikWCTP9DgBAEJRHwqx+BwAgCMojYe0j1AEAKHwVjNQBAAgGpt8BAAiIcla/AwAQDBVR\nQh0AgEAoLwmrpa1T7R2dOa2DUAcAYJi6tordn+P93wl1AACGqbv9ao63iiXUAQAYpnxpv0qoAwAw\nTF2hvi/HGyQZAAAOKUlEQVTHW8US6gAADFMFI3UAAIKB6XcAAAIi1jX9TqgDAFDYYozUAQAIBqbf\nAQAIiJJwkUpCRUy/AwAQBLFo7ju1EeoAAIyA8kiIbWIBAAiCWKSYzWcAAAiCWCTE9DsAAEFQHglr\nPw1dAAAofLFIWE1MvwMAUPhikbCamH4HAKDwlUe4pQ0AgECIRcLa39qhzk7PWQ2EOgAAI6B7//cc\nLpYj1AEAGAGH9n/P3QY0hDoAACMgFo2HetPBtpzVQKgDADACYpGQJKmJkToAAIWtvCT37VcJdQAA\nRkDX9Hsu938n1AEAGAHdq98ZqQMAUNi6Vr/nclc5Qh0AgBEQI9QBAAiGSLhI4SJj+h0AgEJnZirP\ncVMXQh0AgBGS605t4Zx98ghqa2tTQ0ODWlpacl0KMhCNRlVXV6fi4uJclwIAIyqW405tgQj1hoYG\nVVRUaMaMGTKzXJeDfri7du7cqYaGBs2cOTPX5QDAiCqPhJh+H66WlhaNHz+eQC8AZqbx48czqwIg\nkGLRYraJHQkEeuHg3xWAoIpFQqx+D4JQKKT58+dr7ty5Ouecc7R79+5+z9+9e7d++MMfdj/fvHmz\nLrjggiF//pIlS7Ro0aLu56tWrdKSJUv6/T3vvPOO7r333j6P//znP9exxx6roqIirVq1KuXYt771\nLR155JGaPXu2Hnvsse7X//mf/1n19fWKxWJD+4MAQAErLwmrKajbxJrZMjN73czWmdk1aY6bmX0/\ncfwlM1uYzXqyqbS0VKtXr9bLL7+smpoa3XLLLf2e3zPUp06dql/84hfDqmHbtm165JFHMj5/oFCf\nO3eu7rvvPr3//e9Pef2VV17R8uXLtXbtWj366KP6whe+oI6O+HTTOeeco+eee25ofwAAKHCxaG4X\nymUt1M0sJOkWSWdJmiPpIjOb0+O0syTNSvy6XNKt2apnNJ1yyinatGlT9/PvfOc7OvHEEzVv3jx9\n/etflyRdc801euuttzR//nxdffXVeueddzR37lxJ0j333KOPfexjWrZsmWbNmqWvfOUr3e/1+c9/\nXosWLdKxxx7b/V5drr76al1//fW96uno6NDVV1/dXcOPfvSj7hp+//vfa/78+fre977X6/cdc8wx\nmj17dq/XH3jgAV144YWKRCKaOXOmjjzyyO4gX7x4saZMmTLYSwYAgRCLhNXU2i53z8nnZ3P1+0mS\n1rn725JkZsslnSfplaRzzpP0Y4//6Z81syozm+LuW4b6of/20Fq9snnvcOruZc7Ucfr6OcdmdG5H\nR4d++9vf6rOf/awk6fHHH9ebb76p5557Tu6uc889V08//bRuuOEGvfzyy1q9erWk+Kg52erVq/XC\nCy8oEolo9uzZuuqqq1RfX6/rr79eNTU16ujo0BlnnKGXXnpJ8+bNkxT/YeL+++/XE088oYqKiu73\nuvPOO1VZWamVK1fq4MGDOu2003TmmWfqhhtu0E033aSHH354UNdj06ZNWrx4cffzurq6lB9iAGCs\nKo+E5S41t3Z07wU/mrI5/T5N0sak5w2J1wZ7jszscjNbZWartm/fPuKFjoQDBw5o/vz5mjx5srZu\n3aoPf/jDkuKh/vjjj2vBggVauHChXnvtNb355psDvt8ZZ5yhyspKRaNRzZkzRxs2bJAk/exnP9PC\nhQu1YMECrV27Vq+88krK77v22mv1zW9+M+W1xx9/XD/+8Y81f/58nXzyydq5c2dGNQAABmfG+DK9\nb9YE5Wo9cEHcp+7ut0u6XZIWLVrU75xGpiPqkdb1nXpzc7OWLl2qW265RV/60pfk7vra176mK664\nIuX8niPzniKRSPfjUCik9vZ2rV+/XjfddJNWrlyp6upqXXLJJb1uDfvQhz6ka6+9Vs8++2z3a+6u\nH/zgB1q6dGnKuU8++WTK80svvVQvvPCCpk6dqhUrVvRZ27Rp07Rx46GfxRoaGjRtWq+fxQBgzFk2\nd4qWzc3dV5DZHKlvklSf9Lwu8dpgzykoZWVl+v73v6/vfve7am9v19KlS3XXXXepqalJUnzqetu2\nbaqoqNC+ffsG9d579+5VeXm5KisrtXXr1j4XxV177bW68cYbu58vXbpUt956q9ra2iRJb7zxhvbv\n39+rhrvvvlurV6/uN9Al6dxzz9Xy5ct18OBBrV+/Xm+++aZOOumkQf1ZAAAjL5uhvlLSLDObaWYl\nki6U9GCPcx6U9OnEKvjFkvYM5/v0fLFgwQLNmzdPP/3pT3XmmWfq4osv1imnnKLjjjtOF1xwgfbt\n26fx48frtNNO09y5c3X11Vdn9L7HH3+8FixYoKOPPloXX3yxTjvttLTnnX322aqtre1+ftlll2nO\nnDlauHCh5s6dqyuuuELt7e2aN2+eQqGQjj/++LQL5e6//37V1dXpmWee0Uc+8pHukf6xxx6rv/3b\nv9WcOXO0bNky3XLLLQqFQpKkr3zlK6qrq1Nzc7Pq6up03XXXDfLqAQCGyrK5Qs/Mzpb0H5JCku5y\n9+vN7HOS5O63WXwXkpslLZPULOlSd1/V5xsqPv3e857pV199Vcccc0w2/gjIEv6dAUDmzOx5d180\n0HlZ/U7d3VdIWtHjtduSHrukL2azBgAAxgp2lAMAICAIdQAAAiIwoZ6r3XswePy7AoDsCESoR6NR\n7dy5k7AoAF391KPRaK5LAYDAKYjNZwZSV1enhoYG5etuc0gVjUZVV1eX6zIAIHACEerFxcWaOXNm\nrssAACCnAjH9DgAACHUAAAKDUAcAICCyuk1sNpjZdkkbRvAtJ0jaMYLvN1ZxHYePazh8XMPh4xoO\nXzau4WHuXjvQSQUX6iPNzFZlsp8u+sd1HD6u4fBxDYePazh8ubyGTL8DABAQhDoAAAFBqEu357qA\ngOA6Dh/XcPi4hsPHNRy+nF3DMf+dOgAAQcFIHQCAgBgzoW5my8zsdTNbZ2bXpDluZvb9xPGXzGxh\nLurMZxlcw08krt0aM/uTmR2fizrz2UDXMOm8E82s3cwuGM36CkUm19HMlpjZajNba2ZPjXaN+S6D\n/54rzewhM3sxcQ0vzUWd+crM7jKzbWb2ch/Hc5Mp7h74X5JCkt6SdLikEkkvSprT45yzJT0iySQt\nlvTnXNedT78yvIanSqpOPD6Lazj4a5h03u8krZB0Qa7rzrdfGf5drJL0iqTpiecTc113Pv3K8Br+\nk6RvJx7XStolqSTXtefLL0nvl7RQ0st9HM9JpoyVkfpJkta5+9vu3ippuaTzepxznqQfe9yzkqrM\nbMpoF5rHBryG7v4nd29MPH1WEq3YUmXy91CSrpL0S0nbRrO4ApLJdbxY0n3u/q4kuTvXMlUm19Al\nVZiZSYopHurto1tm/nL3pxW/Jn3JSaaMlVCfJmlj0vOGxGuDPWcsG+z1+aziP6XikAGvoZlNk3S+\npFtHsa5Ck8nfxaMkVZvZk2b2vJl9etSqKwyZXMObJR0jabOkNZK+7O6do1NeIOQkUwLRehX5xcw+\nqHion57rWgrQf0j6qrt3xgdIGKKwpBMknSGpVNIzZvasu7+R27IKylJJqyV9SNIRkn5tZr939725\nLQv9GSuhvklSfdLzusRrgz1nLMvo+pjZPEl3SDrL3XeOUm2FIpNruEjS8kSgT5B0tpm1u/uvRqfE\ngpDJdWyQtNPd90vab2ZPSzpeEqEel8k1vFTSDR7/gnidma2XdLSk50anxIKXk0wZK9PvKyXNMrOZ\nZlYi6UJJD/Y450FJn06sWFwsaY+7bxntQvPYgNfQzKZLuk/SpxgRpTXgNXT3me4+w91nSPqFpC8Q\n6L1k8t/zA5JON7OwmZVJOlnSq6NcZz7L5Bq+q/hMh8xskqTZkt4e1SoLW04yZUyM1N293cyulPSY\n4qs+73L3tWb2ucTx2xRfaXy2pHWSmhX/KRUJGV7Df5U0XtIPEyPNdqcxRLcMryEGkMl1dPdXzexR\nSS9J6pR0h7unvfVoLMrw7+I3JN1jZmsUX8H9VXene1uCmf1U0hJJE8ysQdLXJRVLuc0UdpQDACAg\nxsr0OwAAgUeoAwAQEIQ6AAABQagDABAQhDoAAAFBqAMAEBCEOoAhMbMxsc8FUEgIdSCAzOxXiUYm\na83s8sRry8zsL4n+2L9NvBYzs7vNbE2i5/PHE683Jb3XBWZ2T+LxPWZ2m5n9WdKNZnaSmT1jZi+Y\n2Z/MbHbivJCZ3WRmLyfe9yoz+5CZ/SrpfT9sZveP3lUBgo+ftIFg+nt332VmpZJWmtkDkv5T0vvd\nfb2Z1STO+xfFt688TpLMrDqD966TdKq7d5jZOEnvS+xQ9leS/l3SxyVdLmmGpPmJYzWSGhXfbbDW\n3bcrvsPWXSP3RwZAqAPB9CUzOz/xuF7xkH3a3ddLkrt39YH+K8X3/Vbi9cYM3vvn7t6ReFwp6b/M\nbJbi/beLk973NndvT/48M/tvSZ80s7slnSKJlqjACCLUgYAxsyWKh+op7t5sZk8q3kLz6EG8TfL+\n0dEex/YnPf6GpCfc/XwzmyHpyQHe925JD0lqUfyHg/ZB1ARgAHynDgRPpaTGRKAfLWmx4sH8fjOb\nKUlJ0++/lvTFrt+YNP2+1cyOMbMiSeerb5U61E7ykqTXfy3piq7FdF2f5+6bJW2WdK3iAQ9gBBHq\nQPA8KilsZq9KukHSs5K2Kz4Ff5+ZvSjp/ybO/aak6sSCthclfTDx+jWSHpb0J0n9tYu8UdK3zOwF\npc783aF4686XEu97cdKx/5G00d1phQqMMLq0ARhVZnazpBfc/c5c1wIEDaEOYNSY2fOKfyf/YXc/\nmOt6gKAh1AEACAi+UwcAICAIdQAAAoJQBwAgIAh1AAACglAHACAgCHUAAALi/wM6LLNVjaWgvQAA\nAABJRU5ErkJggg==\n",
182 |       "text/plain": [
183 |        "<matplotlib.figure.Figure at 0x7f73b412efd0>"
184 |       ]
185 |      },
186 |      "metadata": {},
187 |      "output_type": "display_data"
188 |     },
189 |     {
190 |      "data": {
191 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfUAAAGDCAYAAAAyM4nNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuUXXV9sPHnm8mdmdwvJJlAAkmAEEOA4RpEhEoCFiiW\ntoitlWoBLVjX26Jo8dVWrRRxtVVRpAjUt0IURG6NgLUiKrcECZcESQIBc2MSAiSZ3Cf5vX+cM5Mz\nk7mcSebMmdnzfNaa1Tn77Jz5zpauZ/Y+++wdKSUkSVLP16fcA0iSpM5h1CVJygijLklSRhh1SZIy\nwqhLkpQRRl2SpIww6lI3FxGfi4hbuuDnzImIe0v9czpTRAyIiLqI2BURXy73PFK5GXWpm0sp/XNK\n6WNd8KO+AlzX8CAiTo2IpyNic0Q8HxGnFa4cEaMj4o6I2BgRb0fED9r7ARHxnohIzQMcEZdExOsR\nsSUi7o2IEQXPDYiIWyNiU0S8ERH/p+G5lNKOlFIl0O7PlnoDoy6JiDgBGJpSejL/eATwAPA1YBhw\nPfBARAwv+Gf3AG8AhwBjgBva+Rn9gH8Hnmq2/Gjgu8BfAGOBrcC3C1b5IjAVOBR4L/DpiJi7P7+n\nlHVGXeomIuIzEbE6v2f8ckSclV/+xYj4r/z338ofbm74qo+IL+afGx8RP46I9RGxIiI+2YEffw7w\ny4LHpwK1KaW7Ukq7U0r/BawHPpD/WWcDE4GrU0obU0q7UkrPtvMz/g54BPhds+UfAh5IKT2WUqoD\nPg98ICKq8s//JfCllNLbKaWXgJuBj3Tgd5N6DaMudQMRcQRwJXBCSqkKmAO81ny9lNKVKaXK/CHn\n04C3gfsiog+5PevngAnAWcCnImJO/vVPi4h32hjhXcDL7Y0JzMh/f3J+/f+MiA0RsSAi3tPG73co\n8FfAP7Xw9NH5uRt+x1eAHcC0/JGBcYXP578/up1ZpV7JqEvdw25gADA9IvqllF7Lx61FETEauBe4\nKr+HfAIwOqX0TymlnSmlV4H/AC4GSCn9OqU0rI2fPwzYXPD4CWBcRFwcEf0i4i+Bw4HB+eergbOB\nXwAHA18n98fFqFZe/xvA5/N74s1VAhubLdsEVOWfo9nzDc9JasaoS91ASmk58Cly7x+vi4h5ETG+\npXXz703fDdyRUpqXX3woMD4i3mn4Aj5H7j3qYrxNQShTShuAPyJ3yLwWmAv8D7Aqv8o24LWU0vfy\nh97nASuB2S3Mex5QlVL6YSs/uw4Y0mzZUHJ/ZDT8ETCkheckNWPUpW4ipXRHSuk0coFOwL+0suo3\nye2tXluwbCWwIqU0rOCrKqV0bpE//nlgWrN5fplSOiGlNILcSWxHAk8XrN/8Fo+t3fLxLKAmf+b6\nG8CfkXtr4L7884uBYxpWjojDgf7A0pTS28Dawufz3y8u8veSehWjLnUDEXFERJwZEQOA7eT2hPe0\nsN7lwHuAD6WUCp9/GticP9luUERURMSM/FntxZiff93Cn3Vs/tD7EHJntq9MKT2cf/onwPCI+Mv8\nz7qI3CH53+T/7Rcj4tH8up8n9wfDrPzX/eTeGrg0//wPgPMi4t0RcRDwJeCelFLD3vj3gWsjYnhE\nHAX8NXB7kb+X1KsYdal7GEDuM+JvkvuY2Bjgsy2s90HgMGBNwRnwn0sp7Qb+kFw0V+Rf5xZyh6rJ\nB7Ol97MBSCn9FtgYEScVLP50/nVWkjtZ7cKC9d8Czgf+ntz73dcAF6SU3syvMpF84FNKm1NKbzR8\nkfuDZUv+NUgpLQauIBf3dcBBwCcK5vgC8ArwOvAocH1K6aHWfhepN4uUWjtiJqk3yX9M7RMppT/q\nhNdaBJyVf2++ZPJHNmqBfuRi/4+l/HlSd2fUJUnKCA+/S5KUEUZdkqSMMOqSJGWEUZckKSP6lnuA\njho1alSaNGlSuceQJKnLPPPMM2+mlEa3t16Pi/qkSZNYuHBhuceQJKnLRMTrxazn4XdJkjLCqEuS\nlBFGXZKkjDDqkiRlhFGXJCkjjLokSRlh1CVJygijLklSRhh1SZIyomRRj4hbI2JdRLzYyvMREd+I\niOUR8XxEHFeqWSRJ6g1Kuad+OzC3jefPAabmvy4DvlPCWSRJyrySRT2l9BjwVhurXAB8P+U8CQyL\niHGlmqclG7ft4ucv1bLyra3s2ZO68kdLktTpynlDlwnAyoLHq/LL1jZfMSIuI7c3zyGHHNJpAyxa\n+Q4f/c/czWEG9atgyphKpo6tZOqYKqaNrWTa2ComDBtEnz7RaT9TkqRS6RF3aUsp3QzcDFBTU9Np\nu9QnTBrOjz9+Cktr61hau5nl6+r4zfI3uee3qxvXKYz9tLG52E8dY+wlSd1POaO+GphY8Lg6v6zL\nDO7fl+MPHcHxh45osnzj1l0sX7+5MfbLaveN/eD+udhPGWPsJUndQzmjfj9wZUTMA04CNqaU9jn0\nXg5DB/drNfbL1m1m2br2Yz91TFV+797YS5K6RsmiHhF3AmcAoyJiFfAFoB9ASukmYD5wLrAc2Apc\nWqpZOsvQwf2omTSCmkktx35pbV0u+rV1/GrZen7821WN6xTGftrYve/dG3tJUmeJlHrWWd81NTVp\n4cKF5R6jKIWxb3jPfmntZtZt3tG4zuD+FUwdU8mUgpPzpoypNPaSpEYR8UxKqaa99XrEiXI9VWt7\n9u9s3cmydXUsa3jPft3mFvfsm8d+6thKxg819pKklhn1Mhg2uD8nTBrBCUXE/rFWYj91bBVTx+yN\n/YRhg4gw9pLUmxn1bqS92DecnLds3WZ+uXQ9dz+zN/YHNbxnX3AmvrGXpN7FqPcABxz7xr36vXv4\nxl6Ssseo92Btxb7wTPylta3HflrDVfTGVjFtbBXjhw409pLUQxn1DBo2uD8nTh7BiZObxv7tLXv3\n7BvOxP/Fy+u5q5XYTxtbxZT8SXrGXpK6P6Peiww/qP3YL6vNXVynvdg37N0be0nqPoy6io790tp9\nY185oG/+ojp7Yz9tbBXjjL0kdTmjrla1Fful+T36vbFf12LsC8/EN/aSVFpeUU6d5q0tO3ORX1fH\n8tq9l819s25n4zqFsW+4ep6xl6S2FXtFOaOukiuM/bKCj98Vxr5qQF+mjC08jJ/76J2xlySjrh7g\nrX0O4+fOym8p9tPGVBV89K6Sg4cYe0m9h1FXj9VS7JfV1rFhi7GX1DsZdWXOhrodTU7Oa7i4Tlux\nbzgj39hL6sm8S5syZ2TlAEZWDuDkw0Y2Wd5S7P/npVp+uHBl4zpVA/vmboRTEPtpY6sYO2SAsZeU\nGUZdPV5bsV9aW8fygnva/6yV2BeeiW/sJfVURl2ZNbJyAKdUDuCUw1uOfeG18R9ZUsu8BS3HvvA2\nt8ZeUndm1NXrtBb7N+t2NH7cruHkvJZiPy0f+cLb3Bp7Sd2BUZfyRlUOYFQRsV9aW8fDi99oEvsh\nA/s2Rn7KmKrGi+uMqTL2krqOUZfa0VbsC+9lv7S2jodefIO3txp7SeVh1KX91BD7Uw8f1bgspcSG\nhs/ZFxn7qWOqGj96Z+wlHQijLnWiiGg19m/W7Wxyct6ydbnY39ks9o23tjX2kjrIqEtdICIYXTWA\n0VWtxD4f+YY9/J+2Gvu9Z+JPG1vJaGMvqYBRl8qoSeyntBz7vZfMreOnL67lzq27GtcbOqjfPmfi\nG3up9zLqUjdUbOxzt7nNx/7pprFvfnLe1DHGXso6oy71IG3Ffn3dDpbn369vuM3t/BeKiP3YSkZX\nGnspC4y6lAERwZiqgYypGthi7JfV1jW5p31rsS98z97YSz2PUZcyrDD2s1uJfeFtbv/7+bVs3LY3\n9sMGF7xnn/+/xl7qvoy61Au1GfvNOxrPxG+4Ic5/P7+WO5rFftqYqvxtbvM3xDH2UtkZdUmNIoIx\nQwYyZkjLsV9acEGdZbWbefC5NWzaXt+4XkPsc5+z33tDnFGV/Y291AWMuqR2Fcb+tKktx77wMP4D\nzWI/fHC/Jveybzikb+ylzmXUJe234mOfu6hOi7FvdnLe1DHGXtpfRl1Sp2sr9us2F56gV3zsp42t\nYuRBxl5qi1GX1GUigrFDBjK2ldg3vxHO/c+tYXMLsW/4jP2UfPRHVQ4ox68jdTtGXVLZFcb+3VNH\nNy4vjH3DmfhLa+u4b1HT2I84qH8+8MZevZtRl9RttRX72k07mpyJv2xdy7HPnZTXcIJe7lC+sVdW\nGXVJPU5EcPDQgRw8tP3YL63dzH3PrmHzjn1j3/Q2t5WMNPbq4Yy6pMxoL/ZL85Ffnr+4zr3Prm4S\n+5GNh/Grmlwj39irpzDqkjKvMPanTzvw2DecmW/s1d0YdUm9Vluxf2PT9r0fvcufkd9S7AsP3+fO\nzK9ixEH9y/HrSEZdkpqLCMYNHcS4oYNajH3jyXm1dSxtI/aFV88z9uoKRl2SilQY+/cUEft7frua\nuoLYj6rcexi/8OI6xl6dxahL0gEqNvYN18dvKfYNH7crvM2tsVdHGXVJKpG2Yr924/bGG+C0F/tp\nYyuZko/9tLFVDDf2aoVRl6QuFhGMHzaI8cNajn3hmfhLa+v4cRuxLzyMb+xl1CWpmyiM/RlHjGlc\nXhj7wmvj7xv7AfnA7z05b+qYSmPfixh1Serm2or9mo3b956clz+M31Lsp42tbHImvrHPJqMuST1U\nRDBh2CAmFBH7pevquPuZVWzZubtxvYbYF94EZ9rYSoYNNvY9lVGXpIxpL/a5w/gNH72r466FK5vE\nfnTVgBavjW/suz+jLkm9RGHs37ufsZ+Wj3zhxXWMffdR0qhHxFzg34EK4JaU0nXNnh8K/BdwSH6W\nG1JKt5VyJklSU23FfvU72wo+elfHsiJjP21MFUMH9yvHr9OrRUqpNC8cUQEsBd4HrAIWAB9MKS0p\nWOdzwNCU0mciYjTwMnBwSmlna69bU1OTFi5cWJKZJUnt27MnsWbjtiZn4jfc035rK7FvOJRv7PdP\nRDyTUqppb71S7qmfCCxPKb2aH2gecAGwpGCdBFRFRACVwFtAffMXkiR1H336BNXDB1M9fDDvPXLv\nnn1h7BvOxF9Wu5kfLVzZJPZjqgYUvFdf1Rh+Y3/gShn1CcDKgsergJOarfMt4H5gDVAF/FlKaU8J\nZ5IklUixsV9aW8fydS3HvuFM/POOGcfxh44ox6/Ro5X7RLk5wCLgTOBw4GcR8auU0qbClSLiMuAy\ngEMOOaTLh5Qk7b+2Yr/6nW1Nrp63fN1m7njq97y4eiN3f/zUMk7dM5Uy6quBiQWPq/PLCl0KXJdy\nb+wvj4gVwJHA04UrpZRuBm6G3HvqJZtYktRl+vQJJo4YzMQRTWP/4VufZtO2XWWcrOfqU8LXXgBM\njYjJEdEfuJjcofZCvwfOAoiIscARwKslnEmSpMwq2Z56Sqk+Iq4EHib3kbZbU0qLI+KK/PM3AV8C\nbo+IF4AAPpNSerNUM0mSlGUlfU89pTQfmN9s2U0F368Bzi7lDJKknmfz9l38Zvm++3i5M+eryjBR\nz1DuE+UkSWqickAFr6zfwodueWqf5/r2CZ7/4tkM7m++WuJWkSR1K1/9wEw+curkfZbPf2Ettz/+\nGrvqE3hl2hYZdUlStzJ0UD9OnLzvZ9RfXL0RgKvvfo5+ffvQt0/w5ycfygmT/Dx7A6MuSeoRjpk4\njCMPruKV9XUAvLVlJw8+v5a/O3saV5x+OH36RJknLD+jLknqEY4/dDgPfer0xsebt+/imnte4PqH\nXuapV9/inz/wLqoG7s3aQf37UtHLQl+yG7qUijd0kSQ1SCnxg6d+zz89uISd9U2vMv6eaaP5z786\nsUyTda7ucEMXSZJKKiL3vvqJk0fw2NL1jct//NvVrHp7axknKw+jLknq8XJ3e9v7+fVnV77D79Zu\nauNfZFMpLxMrSZK6kFGXJCkjjLokKZM2bqtn49bedbc3oy5JypwLZ01g47adXHDjr1m+bnO5x+ky\nRl2SlDl/MH0sd/71ydTtqOfCGx/nFy+vK/dIXcKoS5IyqWbSCO678jQmDB/EVXc8S0+7Lsv+MOqS\npMyaMGwQc2ccTN2O+nKP0iWMuiRJGWHUJUnKCKMuSVJGGHVJkjLCqEuSlBFGXZKkjDDqkiRlhFGX\nJCkjjLokSRlh1CVJygijLklSRhh1SVKm9avIpW7Lzt1lnqT0jLokKdOOqR4GwILX3irzJKVn1CVJ\nmXb8ocPpX9GHJ1/ZUO5RSs6oS5IybVD/CmYdMozHjbokST3fqYeP5MU1G9m4dVe5Rykpoy5JyrxT\nDx9FSvDkimzvrRt1SVLmzZo4jIH9+vBExg/BG3VJUub179uHEyaNMOqSJGXBMdXDeLl2M3v2pHKP\nUjJGXZLUKzRchCbLsv8bSpLUSxh1SZIywqhLkpQRRl2SpIww6pIkZYRRlyT1Kpu315d7hJIx6pKk\nXuG4Q4dR0Sc49xu/4slXs3kRGqMuSeoV3j11ND/++Kn0qwg++B9P8i8P/Y6d9XvKPVanMuqSpF5j\n1sRh/Pcn383FJ0zkO4++wifvfLbcI3Uqoy5J6lUOGtCXr35gJn/3vmk8tPgNfr3szXKP1GmMuiSp\nV7rsPYdxyIjBfOnBJdTvzsZheKMuSeqVBvSt4HPnHsnLtZv54cKV5R6nUxh1SVKvNefogzlp8gi+\n/shSNm7bVe5xDphRlyT1WhHB5/9wOm9v3cm3/ndZucc5YEZdktSrzZgwlD85vprbH3+NFW9uKfc4\nB6SkUY+IuRHxckQsj4hrWlnnjIhYFBGLI+KXpZxHkqSW/P3ZR9C/og//PP+lco9yQEoW9YioAG4E\nzgGmAx+MiOnN1hkGfBs4P6V0NPAnpZpHkqTWjBkykL8+/TB+tqSWNe9sK/c4+62Ue+onAstTSq+m\nlHYC84ALmq1zCXBPSun3ACmldSWcR5KkVk0edRAA23ftLvMk+6+UUZ8AFH5GYFV+WaFpwPCIeDQi\nnomID7f0QhFxWUQsjIiF69evL9G4kiT1bOU+Ua4vcDzwfmAO8PmImNZ8pZTSzSmlmpRSzejRo7t6\nRkmSeoS+JXzt1cDEgsfV+WWFVgEbUkpbgC0R8RhwDLC0hHNJkpRJpdxTXwBMjYjJEdEfuBi4v9k6\n9wGnRUTfiBgMnAT07FMPJUkqk5LtqaeU6iPiSuBhoAK4NaW0OCKuyD9/U0rppYh4CHge2APcklJ6\nsVQzSZKUZaU8/E5KaT4wv9mym5o9/hrwtVLOIUlSb1DuE+UkSVInMeqSJGWEUZckKSOMuiRJGWHU\nJUnKCKMuSVJGGHVJkjLCqEuSlBFGXZKkjDDqkiRlhFGXJKnAnpTKPcJ+M+qSJAETRwwGYMWbW8s8\nyf4z6pIkAUceXEUELFmzqdyj7Lc279IWEQ8ArR6HSCmd3+kTSZJUBoP792XyqINYsnZjuUfZb+3d\nevWGLplCkqRuYPq4ITy36p1yj7Hf2ox6SumXXTWIJEnlNn38EB58fi0bt+1i6KB+5R6nw9o7/P4C\nbR9+n9npE0mSVCZHjRsCwO/WbuKkw0aWeZqOa+/w+x92yRSSJHUDR+ej/lIWo55Ser2rBpEkqdxG\nVw1gVGV/lqztmWfAF/WRtog4OSIWRERdROyMiN0R0TN/Y0mSWhERHDVuSLajDnwL+CCwDBgEfAy4\nsVRDSZJULtPHD2HpG3Xs2r2n3KN0WNEXn0kpLQcqUkq7U0q3AXNLN5YkSeUxfdwQdu7ewyvr68o9\nSoe1d6Jcg60R0R9YFBHXA2vxanSSpAyaXnCy3JEHDynzNB1TbJj/Ir/ulcAWYCLwx6UaSpKkcpk8\n6iAG9O3TIy8XW+ye+pvAzpTSduAfI6ICGFC6sSRJKo++FX048uCqHnmyXLF76j8HBhc8HgT8T+eP\nI0lS+U0fP4QlazaRethtWIuN+sCUUuMZA/nvB7exviRJPdZhoyp5e+suNm2vL/coHVJs1LdExHEN\nDyLieGBbaUaSJKm8KvpE7puetaNe9HvqnwLuiog1QAAHA39WsqkkSVKHFRX1lNKCiDgSOCK/6OWU\n0q7SjSVJkjqq2MvEDgY+A/xtSulFYFJEeLMXSZK6kWLfU78N2Amckn+8GvhySSaSJEn7pdioH55S\nuh7YBZBS2kruvXVJktRNFBv1nRExiPx5gBFxOLCjZFNJkqQOa/dEuYgI4CbgIWBiRPwAmA18pLSj\nSZKkjmg36imlFBFXA2cAJ5M77P63KaU3SzybJEnqgGI/p/5b4LCU0n+XchhJkrT/io36ScCHIuJ1\ncndpC3I78TNLNpkkSeqQYqM+p6RTSJKkA1bsFeVeL/UgkiTpwBT7kTZJktTNGXVJkjLCqEuSlBFG\nXZKkjDDqkiRlhFGXJCkjjLokSRlh1CVJygijLklSRhh1SZIyoqRRj4i5EfFyRCyPiGvaWO+EiKiP\niItKOY8kSVlWsqhHRAVwI3AOMB34YERMb2W9fwEeKdUskiT1BqXcUz8RWJ5SejWltBOYB1zQwnpX\nAT8G1pVwFkmSija4fwUAb23dWeZJOqaUUZ8ArCx4vCq/rFFETAAuBL7T1gtFxGURsTAiFq5fv77T\nB5UkqVDNpBEAPPHKhjJP0jHlPlHu34DPpJT2tLVSSunmlFJNSqlm9OjRXTSaJKm3Onz0QYwdMoDf\nvPJmuUfpkKLup76fVgMTCx5X55cVqgHmRQTAKODciKhPKd1bwrkkSWpTRDD78FE8unQ9e/Yk+vSJ\nco9UlFLuqS8ApkbE5IjoD1wM3F+4QkppckppUkppEnA38AmDLknqDmZPGcVbW3by0hubyj1K0UoW\n9ZRSPXAl8DDwEvCjlNLiiLgiIq4o1c+VJKkzzJ4yCoDHl/ec99VLefidlNJ8YH6zZTe1su5HSjmL\nJEkdcfDQgRw2+iB+88qb/PXph5V7nKKU+0Q5SZK6rdmHj+LpFW+xs77N87m7DaMuSVIrZk8Zxdad\nu3lu1TvlHqUoRl2SpFaccthI+gT8ZnnP+GibUZckqRVDB/djxoShRl2SpCw49fBRPPv7d9iyo77c\no7TLqEuS1IbZU0ZSvyfx9GtvlXuUdhl1SZLacMKkEfTv24fHe8AheKMuSVIbBvar4PhDhvObHnAR\nGqMuSVI7Zk8ZyZK1m3huZff+aJtRlySpHRceV834oQP5k+8+wd3PrCr3OK0y6pIktWPCsEE8cNVp\n1Bw6nL+/6zmuvfeFbnmVOaMuSVIRRlYO4Pt/dSKXn34Y//Xk77n45id4Y+P2co/VhFGXJKlIfSv6\n8Nlzj+LGS47jd29s5g+/+WueerX7nEBn1CVJ6qD3zxzHfX8zmyED+3LJLU9x669XkFIq91hGXZKk\n/TF1bBX3XjmbM48cwz89uIRP/XAR23buLutMRl2SpP00ZGA/vvvnx/P3Z0/j/ufWcOG3f8PrG7aU\nbR6jLknSAejTJ7jyzKncfumJrN24nYtueoJdu8tzZrxRlySpE7xn2miufO8U1m/ewbZd5TkMb9Ql\nSeokEeX9+UZdkqSMMOqSJGWEUZckKSOMuiRJGWHUJUnKCKMuSVJGGHVJkjLCqEuSlBFGXZKkjDDq\nkiRlhFGXJCkjjLokSRlh1CVJygijLklSRhh1SZIywqhLkpQRRl2SpIww6pIkZYRRlyQpI4y6JEkZ\nYdQlScoIoy5JUkYYdUmSMsKoS5KUEUZdkqSMMOqSJGWEUZckKSOMuiRJGWHUJUnKCKMuSVJGGHVJ\nkjKipFGPiLkR8XJELI+Ia1p4/kMR8XxEvBARj0fEMaWcR5KkLCtZ1COiArgROAeYDnwwIqY3W20F\n8J6U0ruALwE3l2oeSZKyrpR76icCy1NKr6aUdgLzgAsKV0gpPZ5Sejv/8EmguoTzSJKUaaWM+gRg\nZcHjVfllrfko8NMSziNJUqb1LfcAABHxXnJRP62V5y8DLgM45JBDunAySZJ6jlLuqa8GJhY8rs4v\nayIiZgK3ABeklDa09EIppZtTSjUppZrRo0eXZFhJknq6UkZ9ATA1IiZHRH/gYuD+whUi4hDgHuAv\nUkpLSziLJEmZV7LD7yml+oi4EngYqABuTSktjogr8s/fBPxfYCTw7YgAqE8p1ZRqJkmSsqyk76mn\nlOYD85stu6ng+48BHyvlDJIk9RZeUU6SpIww6pIkZYRRlyQpI4y6JEkZYdQlScoIoy5JUkYYdUmS\nMsKoS5KUEUZdkqSMMOqSJGWEUZckKSOMuiRJGWHUJUnKCKMuSVJGGHVJkjLCqEuSlBFGXZKkjDDq\nkiRlhFGXJCkjjLokSRlh1CVJygijLklSRhh1SZIywqhLkpQRRl2SpIww6pIkZYRRlyQpI4y6JEkZ\nYdQlScoIoy5JUkYYdUmSMsKoS5KUEUZdkqSMMOqSJGWEUZckKSOMuiRJGWHUJUnKCKMuSVJGGHVJ\nkjLCqEuSlBFGXZKkjDDqkiRlhFGXJCkjjLokSRlh1CVJygijLklSRvQt9wCdYdeuXaxatYrt27eX\nexQVYeDAgVRXV9OvX79yjyJJmZKJqK9atYqqqiomTZpERJR7HLUhpcSGDRtYtWoVkydPLvc4kpQp\nmTj8vn37dkaOHGnQe4CIYOTIkR5VkaQSyETUAYPeg/i/lSSVRmaiXm4VFRXMmjWLGTNmcN555/HO\nO++0uf4777zDt7/97cbHa9as4aKLLtrvn3/GGWdQU1PT+HjhwoWcccYZbf6b1157jTvuuKPV5++6\n6y6OPvpo+vTpw8KFC5s899WvfpUpU6ZwxBFH8PDDDzcu/4d/+AcmTpxIZWXl/v0ikqT9VtKoR8Tc\niHg5IpZHxDUtPB8R8Y38889HxHGlnKeUBg0axKJFi3jxxRcZMWIEN954Y5vrN4/6+PHjufvuuw9o\nhnXr1vHTn/606PXbi/qMGTO45557OP3005ssX7JkCfPmzWPx4sU89NBDfOITn2D37t0AnHfeeTz9\n9NP79wtIkg5IyaIeERXAjcA5wHTggxExvdlq5wBT81+XAd8p1Txd6ZRTTmH16tWNj7/2ta9xwgkn\nMHPmTL7whS8AcM011/DKK68wa9Ysrr76al577TVmzJgBwO23384HPvAB5s6dy9SpU/n0pz/d+Fof\n//jHqamNolOrAAAK60lEQVSp4eijj258rQZXX301X/nKV/aZZ/fu3Vx99dWNM3z3u99tnOFXv/oV\ns2bN4l//9V/3+XdHHXUURxxxxD7L77vvPi6++GIGDBjA5MmTmTJlSmPITz75ZMaNG9fRTSZJ6gSl\nPPv9RGB5SulVgIiYB1wALClY5wLg+ymlBDwZEcMiYlxKae3+/tB/fGAxS9ZsOpC59zF9/BC+cN7R\nRa27e/dufv7zn/PRj34UgEceeYRly5bx9NNPk1Li/PPP57HHHuO6667jxRdfZNGiRUBur7nQokWL\nePbZZxkwYABHHHEEV111FRMnTuQrX/kKI0aMYPfu3Zx11lk8//zzzJw5E8j9MfGTn/yEX/ziF1RV\nVTW+1ve+9z2GDh3KggUL2LFjB7Nnz+bss8/muuuu44YbbuDBBx/s0PZYvXo1J598cuPj6urqJn/E\nSJLKo5SH3ycAKwser8ov6+g6RMRlEbEwIhauX7++0wftDNu2bWPWrFkcfPDB1NbW8r73vQ/IRf2R\nRx7h2GOP5bjjjuN3v/sdy5Yta/f1zjrrLIYOHcrAgQOZPn06r7/+OgA/+tGPOO644zj22GNZvHgx\nS5YsafLvrr32Wr785S83WfbII4/w/e9/n1mzZnHSSSexYcOGomaQJHVM9fBBvHvqKPr2Kc8JwT3i\nc+oppZuBmwFqampSW+sWu0fd2RreU9+6dStz5szhxhtv5JOf/CQpJT772c9y+eWXN1m/+Z55cwMG\nDGj8vqKigvr6elasWMENN9zAggULGD58OB/5yEf2+WjYmWeeybXXXsuTTz7ZuCylxDe/+U3mzJnT\nZN1HH320yeNLL72UZ599lvHjxzN//vxWZ5swYQIrV+79W2zVqlVMmLDP32KS1OvMnTGOuTPK9xZk\nKffUVwMTCx5X55d1dJ0eZfDgwXzjG9/g61//OvX19cyZM4dbb72Vuro6IHfoet26dVRVVbF58+YO\nvfamTZs46KCDGDp0KLW1ta2eFHfttddy/fXXNz6eM2cO3/nOd9i1axcAS5cuZcuWLfvMcNttt7Fo\n0aI2gw5w/vnnM2/ePHbs2MGKFStYtmwZJ554Yod+F0lS5ytl1BcAUyNickT0By4G7m+2zv3Ah/Nn\nwZ8MbDyQ99O7i2OPPZaZM2dy5513cvbZZ3PJJZdwyimn8K53vYuLLrqIzZs3M3LkSGbPns2MGTO4\n+uqri3rdY445hmOPPZYjjzySSy65hNmzZ7e43rnnnsvo0aMbH3/sYx9j+vTpHHfcccyYMYPLL7+c\n+vp6Zs6cSUVFBcccc0yLJ8r95Cc/obq6mieeeIL3v//9jXv6Rx99NH/6p3/K9OnTmTt3LjfeeCMV\nFRUAfPrTn6a6upqtW7dSXV3NF7/4xQ5uPUnS/orcOWolevGIc4F/AyqAW1NKX4mIKwBSSjdF7iok\n3wLmAluBS1NKC1t9QXKH35t/Zvqll17iqKOOKsWvoBLxfzNJKl5EPJNSqmlvvZK+p55Smg/Mb7bs\npoLvE/A3pZxBkqTewivKSZKUEUZdkqSMyEzUS3lugDqX/1tJUmlkIuoDBw5kw4YNxqIHaLif+sCB\nA8s9iiRlTo+4+Ex7qqurWbVqFd31anNqauDAgVRXV5d7DEnKnExEvV+/fkyePLncY0iSVFaZOPwu\nSZKMuiRJmWHUJUnKiJJeJrYUImI98HonvuQo4M1OfL3eyu144NyGB85teODchgeuFNvw0JTS6PZW\n6nFR72wRsbCY6+mqbW7HA+c2PHBuwwPnNjxw5dyGHn6XJCkjjLokSRlh1OHmcg+QEW7HA+c2PHBu\nwwPnNjxwZduGvf49dUmSssI9dUmSMqLXRD0i5kbEyxGxPCKuaeH5iIhv5J9/PiKOK8ec3VkR2/BD\n+W33QkQ8HhHHlGPO7qy9bViw3gkRUR8RF3XlfD1FMdsxIs6IiEURsTgiftnVM3Z3Rfz/89CIeCAi\nnstvw0vLMWd3FRG3RsS6iHixlefL05SUUua/gArgFeAwoD/wHDC92TrnAj8FAjgZeKrcc3enryK3\n4anA8Pz357gNO74NC9b7X2A+cFG55+5uX0X+tzgMWAIckn88ptxzd6evIrfh54B/yX8/GngL6F/u\n2bvLF3A6cBzwYivPl6UpvWVP/URgeUrp1ZTSTmAecEGzdS4Avp9yngSGRcS4rh60G2t3G6aUHk8p\nvZ1/+CTgrdiaKua/Q4CrgB8D67pyuB6kmO14CXBPSun3ACklt2VTxWzDBFRFRACV5KJe37Vjdl8p\npcfIbZPWlKUpvSXqE4CVBY9X5Zd1dJ3erKPb56Pk/krVXu1uw4iYAFwIfKcL5+ppivlvcRowPCIe\njYhnIuLDXTZdz1DMNvwWcBSwBngB+NuU0p6uGS8TytKUTNx6Vd1LRLyXXNRPK/csPdC/AZ9JKe3J\n7SBpP/UFjgfOAgYBT0TEkymlpeUdq0eZAywCzgQOB34WEb9KKW0q71hqS2+J+mpgYsHj6vyyjq7T\nmxW1fSJiJnALcE5KaUMXzdZTFLMNa4B5+aCPAs6NiPqU0r1dM2KPUMx2XAVsSCltAbZExGPAMYBR\nzylmG14KXJdybxAvj4gVwJHA010zYo9Xlqb0lsPvC4CpETE5IvoDFwP3N1vnfuDD+TMWTwY2ppTW\ndvWg3Vi72zAiDgHuAf7CPaIWtbsNU0qTU0qTUkqTgLuBTxj0fRTz/8/3AadFRN+IGAycBLzUxXN2\nZ8Vsw9+TO9JBRIwFjgBe7dIpe7ayNKVX7KmnlOoj4krgYXJnfd6aUlocEVfkn7+J3JnG5wLLga3k\n/kpVXpHb8P8CI4Fv5/c065M3hmhU5DZUO4rZjimllyLiIeB5YA9wS0qpxY8e9UZF/rf4JeD2iHiB\n3Bncn0kpefe2vIi4EzgDGBURq4AvAP2gvE3xinKSJGVEbzn8LklS5hl1SZIywqhLkpQRRl2SpIww\n6pIkZYRRlyQpI4y6pP0SEb3iOhdST2LUpQyKiHvzNzJZHBGX5ZfNjYjf5u+P/fP8ssqIuC0iXsjf\n8/mP88vrCl7rooi4Pf/97RFxU0Q8BVwfESdGxBMR8WxEPB4RR+TXq4iIGyLixfzrXhURZ0bEvQWv\n+76I+EnXbRUp+/xLW8qmv0opvRURg4AFEXEf8B/A6SmlFRExIr/e58ldvvJdABExvIjXrgZOTSnt\njoghwLvzVyj7A+CfgT8GLgMmAbPyz40A3iZ3tcHRKaX15K6wdWvn/cqSjLqUTZ+MiAvz308kF9nH\nUkorAFJKDfeB/gNy1/0mv/ztIl77rpTS7vz3Q4H/jIip5O6/3a/gdW9KKdUX/ryI+H/An0fEbcAp\ngLdElTqRUZcyJiLOIBfVU1JKWyPiUXK30DyyAy9TeP3ogc2e21Lw/ZeAX6SULoyIScCj7bzubcAD\nwHZyfxzUd2AmSe3wPXUpe4YCb+eDfiRwMrkwnx4RkwEKDr//DPibhn9YcPi9NiKOiog+wIW0bih7\nbyf5kYLlPwMubziZruHnpZTWAGuAa8kFXlInMupS9jwE9I2Il4DrgCeB9eQOwd8TEc8BP8yv+2Vg\neP6EtueA9+aXXwM8CDwOtHW7yOuBr0bEszQ98ncLuVt3Pp9/3UsKnvsBsDKl5K1QpU7mXdokdamI\n+BbwbErpe+WeRcoaoy6py0TEM+Tek39fSmlHueeRssaoS5KUEb6nLklSRhh1SZIywqhLkpQRRl2S\npIww6pIkZYRRlyQpI/4/xA4KCgmPukwAAAAASUVORK5CYII=\n",
192 |       "text/plain": [
193 |        "<matplotlib.figure.Figure at 0x7f73b0a2cf10>"
194 |       ]
195 |      },
196 |      "metadata": {},
197 |      "output_type": "display_data"
198 |     },
199 |     {
200 |      "data": {
201 |       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfUAAAGDCAYAAAAyM4nNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmUXXWd7/33t4akKklVZR4rIQEyEgKEMIkDLY0J2Eir\nPDbiba9efXBotXv1XSitdOt91Cut9GonFGkFmueqXEVA9EFBaRD7CkKAMCaBSAJUZkJCKnOq6vf8\ncU5VTipVqZOkTp2h3q+1alF7n80+39qEfOr329+zf5FSQpIklb+qYhcgSZL6h6EuSVKFMNQlSaoQ\nhrokSRXCUJckqUIY6pIkVQhDXSozEfHZiPj+ALzP4oi4s9Dvc6Qi4uaI2B0RLcWuRSo1hrpUZlJK\n/zOl9OEBeKsvA9d0bkTE9Ii4PyJ2RcSKiPjzvk4QEUMiYnn3AO7rXBFxeUS8FBE7I+LOiBjd+VpK\n6QPAhcf800kVyFCXdIiIOANoSik9nLP7x8ATwBjgc8BtETGuj1NdCWzuYX+v54qIk4DvAX8NTAB2\nAd85+p9GGjwMdalERcRnImJtRLRGxMqIOD+7/wsR8b+y3387InbkfLVFxBeyr02OiJ9FxOaIWB0R\nnzqCt78Q+F1OLbOAhcDnU0q7U0o/A54C3n2Y+mcA/wX4Srf9fZ3rfcAvUkoPppR2AP8IvCsiGo6g\nfmlQMtSlEhQRs4FPAGeklBqAxcCa7sellD6RUhqRUhoBvBHYCvw8IqqAXwBPAlOA84G/i4jF2fO/\nMSK2HaaEk4GVOdsnAS+mlFpz9j2Z3d+bbwGfBXZ329/XuU7Kbnf+jH8C9gKzDvNekjDUpVLVDgwF\n5kVEbUppTTbcepSdur4T+GRK6QngDGBcSun/SSntSym9CPwbcBlASuk/U0ojD/P+I4Hc0B0BvN7t\nmO1Aj6PniHgnUJ1SuqOHl/s61xG9l6QDaopdgKRDpZRWRcTfAV8AToqIe4C/Tymt635sRNQCtwE/\nSindmt19HDC522i8Gvh9niVs5eAQ3QE0djumiYODv7Oe4cBXgYt6OXdf58r7vSQdzJG6VKJSSj9K\nKb2RTEAn4J97OfRbZEayV+fsewVYnVIamfPVkFLqLWi7e4qDp7ufBY7vdl/7lOz+7mYC04HfR8QG\n4HZgUkRsiIjpeZzr2ew2ABFxAjAEeD7P2qVBy1CXSlBEzI6It0bEUGAPmfvSHT0c9xHgLcD7Ukq5\nrz8CtGab7eojojoi5me72vNxd/a8AKSUngeWAZ+PiLqIeBeZ++4/y9YxPSJSNrSfAaYCp2a/Pgxs\nzH7/Sl/nAn4IXBwRb8qO+r8I3N7tHrykHhjqUmkaSuYz4q8CG4DxwD/0cNx7geOBdTkd8J9NKbUD\nf0EmSFdnz/N9MtPYZANzR29vnlJ6HHg9Is7K2X0ZsIjM1PxXgEtTSp0fV5sKvASsTSm1pZQ2dH4B\nrwEd2e32vs6VUnoW+CiZcN8EDAc+ntdVkwa5SCkVuwZJJSgi3gZ8PKX0l3kcezWwOaX0vQGo6wfA\n/wVsSimdWOj3k8qJoS5JUoVw+l2SpAphqEuSVCEMdUmSKoShLklShSi7J8qNHTs2TZ8+vdhlSJI0\nYB577LFXU0p9rYpYfqE+ffp0li5dWuwyJEkaMBHxUj7HOf0uSVKFMNQlSaoQhrokSRXCUJckqUIY\n6pIkVQhDXZKkCmGoS5JUIQx1SZIqhKEuSVKFKFioR8SNEbEpIp7p5fWIiG9GxKqIeCoiFhaqFkmS\nBoNCjtRvBpYc5vULgZnZryuA7xawFkmSKl7BQj2l9CDw2mEOuQS4JWU8DIyMiEmFqqcnr+/ez33L\nN9KydRcppYF8a0mS+l0xF3SZArySs92S3be++4ERcQWZ0TzTpk3rtwKeeHkrH/r3zOIwDXU1zJnY\nwJyJjcye2MDcSQ3MmtBAQ11tv72fJEmFVBartKWUbgBuAFi0aFG/DanPmjGGn33sHJavb2XlhlZW\nbNjOnU+spXVvW9cxzaPqmTOxkbmTDgT+jLHDqa6K/ipDkqR+UcxQXwtMzdluzu4bMPVDqjn9uNGc\nftzorn0pJdZu250N+VaWr9/Oyg2t3L9yE+0dmd8nhtZUMWtCA3MmNmRH9Y3MmdjAmBFDB7J8SZIO\nUsxQvwv4RETcCpwFvJ5SOmTqfaBFBM2jhtE8ahjnz53QtX/P/nZWbdrBig2trNywnRUbWrl/5WZ+\n+lhL1zFjRwzNjugPjOpPHD+CutrqYvwokqRBpmChHhE/Bs4DxkZEC/B5oBYgpXQ9cDdwEbAK2AV8\nsFC19Ie62mrmT2li/pSmg/a/umMvK7Mj+kzgt3LLQy+xt60DgOqq4Pixw5mTHc3PmdjAnEmNTG6q\nI8IpfElS/4ly6/petGhRWrp0abHLOKy29g7WbNnFig3bs4GfuV/fsnV31zG5jXlzsqN7G/MkST2J\niMdSSov6Oq4sGuXKTU11FSeOH8GJ40fwFwsO7G/ds5/nNx4I+ZUbWjONeQ8faMybOrqe2RMONObN\nmdTA9DE25kmS+maoD6CGutpeG/NWrG9l5cYD0/i9NeblTuPbmCdJymWoF1luY96fz+u5MW/F+u2s\n3HhoY964hqEH7tNnR/Unjh/B0Bob8yRpMDLUS1RvjXmbW/d2faZ+Rfaf//7QS+yzMU+SBj1DvcyM\naxjKuIahvHHm2K59uY15K9ZnPl//xMtb+cWT67qOObQxL/ORuxFD/SMgSZXCv9ErQL6NeSvW99yY\nN2di40GBb2OeJJUnQ72C9dWYd2AKv5X7lm8k25dnY54klSlDfZDJtzGvpyfm2ZgnSaXNUBfQf415\ncyc1MHuijXmSVAyGug4rv8a87T025s3NNuPZmCdJA8O/YXXEemvM275nP89n79F3Bv4dT6xlRw+N\neXMnZkb0NuZJUv8x1NVvGutqWTR9NIum996Ytzy76E33xrzZExuYPSHTmDc3u6StjXmSdGQMdRVU\nX415nevVdz4at6fGvLmTGrOBb2OeJB2Ooa6iyKcxb/n6VlZu3M7Nf1hzUGPeCeOGZ6bubcyTpIMY\n6iopvTfm7cx+3C4T+I+/1HNj3pxJmal7G/MkDUb+jaeSl2nMa+DE8Q09NuYt71z0ZkMrtz++lh17\nDzTmTRs9jNkTG5ibfZDO7Ik25kmqXIa6ylZvjXktW3cfmMLPBn5uY15d7YEn5s3OduLPmdTI6OFD\nivSTSFL/iJRSsWs4IosWLUpLly4tdhkqMz015i1fv50tO/d1HZPbmDcn24FvY56kUhARj6WUFvV1\nnCN1DQr5Nuat2NBzY17nPfq52QfpTLIxT1IJMtQ1qB2uMW/5+tauwH/spa3cldOY11hXc8gytjbm\nSSo2/waSusltzLv4lAP7uzfmrehqzHup65hpo4cdWPQmO41/nI15kgaIoS7l6XCNeSs2tLIypzHv\nt7005nWtXW9jnqQCsFFOKoDcxrwV2Ufjdm/MG98w9KD16udMbOSE8cNtzJN0CBvlpCI6XGPeig3b\nsyGfbcz7P1vY155pzKupCo7PNuZl7tfbmCcpf4a6NIAyjXnjeNPMcV37chvzOgO/r8a8OZMyC+AM\ntzFPUg7/RpCK7ODGvMld+1/fvZ/nN2aXsu2rMS9nGt/GPGnwMtSlEtVUX8sZ00dzRi+NeSvWb2fF\nxjwa87KjexvzpMpno5xUAfbsb+eFjTtYsSEzol+xYTsr1rf22JjXuV69jXlS+bBRThpE6mqrObm5\niZObe27My6xulwn7mw7TmDc3+yAdG/Ok8mSoSxWst8a81a/uPGhE32NjXteo3sY8qVz4f6g0yNRU\nVzFzQgMzJ/TSmLe+cwq/lZ8dpjGvcxrfxjypdBjqkoCeG/M6OhJrt+0+qAN/xYZDG/NmTzhwn97G\nPKl4bJSTdMQ6G/OWb9h+0Cp3r/XSmJeZvrcxTzpaNspJKpieGvNSSmzekV3Kdn1rV+B3b8w7YdyI\nzKg+25g3Z1IDExttzJP6g6EuqV9EBOMb6hjfUHdQY97+9g7WvLqT5dlFb2zMkwrH/2MkFVRtTmMe\nvTTmLc8uenPbYy3s3NfedcxxY4Yxe0JDzjR+I9NGD7MxT+qFoS6pKA7XmLd8fee9+sw0fk+NeXOy\nn6nvnMYfZWOeZKOcpNKX25i3Yn0rKzce2pg3oXEosyfamKfKZKOcpIqRb2PeivWt3PSnQxvz5kzK\nfOTOxjxVOkNdUlnKpzFvRXYaf+marfx82YHGvKb62mzIH1jhbpaNeaoA/gmWVFFyG/Pe0a0xb2W2\nA78z8HtqzJuT7cC3MU/lyFCXNCg01ddy5ozRnDmj58a8FdkO/OUbtvOb53puzMudxrcxT6XIRjlJ\n6qZ7Y17nkrbdG/PmTGzMPgs/u5TtuBEMqakqYuWqVDbKSdJROlxjXm7Ir1jfykO9NOblBr6NeRoo\nhrok5SG3Me/Ns3pvzFvRS2PenIkNXSvc2ZinQvFPlCQdg/5ozDswqrcxT8fGUJekAsinMa9zGj+3\nMa++tppZExuYM6HhoGl8G/OUDxvlJKnIdu9r54VNrdn79X005k1q6Brd25g3eNgoJ0llon5INQua\nR7KgeWTXvkMa87KBf9jGvGzg25g3eBnqklSCDteYt/rVndnu+8yI/tHVr/XYmDd3UnbRm4mZz9cP\nG+Jf+ZWuoP+FI2IJ8A2gGvh+Sumabq83Af8LmJat5dqU0k2FrEmSylltdRWzJmS65w9qzNu1n5Ub\nM6P65eszDXo/XfpKV2NeBEwbfaAxb+6kzJPzbMyrLAUL9YioBq4DLgBagEcj4q6U0nM5h/0N8FxK\n6eKIGAesjIgfppT29XBKSVIvmob13JjXsnX3gc/VZ6fxbcyrXIUcqZ8JrEopvQgQEbcClwC5oZ6A\nhsjc/BkBvAa0FbAmSRo0qqqCaWOGMW3MMN520sSu/T015t373Ab+99JXuo7Jbcybm1273sa80lfI\nUJ8CvJKz3QKc1e2YbwN3AeuABuCvUkodBaxJkga9XhvzWvceNKJf3kNj3onjR3QtetMZ+BMah9qY\nVyKK3TWxGFgGvBU4AfhNRPw+pbQ996CIuAK4AmDatGkDXqQkVbqIYHxjHeMbe27My1305pHVr3Fn\nL415nU15NuYVRyGv+Fpgas52c3Zfrg8C16TMh+VXRcRqYA7wSO5BKaUbgBsg8zn1glUsSTpIbmPe\nJTn7uzfmreihMe+40cOy3feZxrw52ca8KhvzCqaQof4oMDMiZpAJ88uAy7sd8zJwPvD7iJgAzAZe\nLGBNkqR+cLjGvOUbtrMyZxr/3uc2kro15g0fUs37z5nOBfMm2H3fjwoW6imltoj4BHAPmY+03ZhS\nejYiPpp9/Xrgi8DNEfE0EMBnUkqvFqomSVLh5DbmLe6lMW/5hu3cv2ITT76yiz/8aQvTRg/j7y+Y\nxV8smERNtU14x8rHxEqSBtzetnbuW76J6+5fxbPrtjNyWC1vmTWOs2aM4Z2nTaF+SHWxSywp+T4m\n1lCXJBXNvrYOfvPcRv5jxSZ+u3wjr+/ez4LmJm776Bv8+FwOQ12SVFba2ju468l1/P1PnuSM6aP4\n0l+ezOyJDcUuqyTkG+r+GiRJKgk11VW8a2EzX/+rU1m5oZULv/Eg/3jnM7Tu2V/s0sqGHyKUJJWU\nvzxtCm+ZNY5v3PcCtzy0hhc2tfKx807kzTPH+pCbPjhSlySVnFHDh/CFd5zENe9ewCOrX+O/3vgI\nn73jGTZt31Ps0kqa99QlSSVt5942Lr3+IZavzzxsdNaEEdzy385iYlNdkSsbON5TlyRVhOFDa/j5\n35zL7R9/A1e/fS5rXt3Fe//tYV55bVexSys5hrokqeQNqali4bRRfPhNx3PhyRNZ/epO/uzaB/jb\nW59wSj6HoS5JKivfuOw0/s9Vb+X950zn7qfX850H/lTskkqGoS5JKjtTRtbzTxfPY9aEBp5q2Ua5\n9YcViqEuSSpb717YzOMvb+MnS18pdiklwVCXJJWtD7xhOuccP4bP3fEMj655rdjlFJ2hLkkqW1VV\nwffefzoTm+r4zM+eYm9be7FLKipDXZJU1hrravnyO0/mxc07uepnT9PRMXjvrxvqkqSy95ZZ4/jv\nF8zijifW8sNHXi52OUVjqEuSKsIn3noiC5qb+Pc/rKGtvaPY5RSFoS5JqggRwcfecgKrNu3gG/e9\nUOxyisJQlyRVjAtPnsSlpzfzrf9Yxfd+N/geSuPSq5KkinLNu05mb1sHX/nVCl7fvZ///rbZVFcN\njiVbDXVJUkWpqa7iX99zCiOGVvOdB/7EmBFD+dAbZxS7rAHh9LskqeLUVFfxlXct4OzjR/NvD77I\nvrbB0ThnqEuSKtZH33ICG7bv4ZdPrSt2KQPCUJckVay3zBrHlJH13P30hmKXMiAMdUlSxYoIzp87\nnv9ctZk9+yv/EbKGuiSpop06dSR79newbtvuYpdScIa6JKmiNdbVAtC6p63IlRSeoS5JqmjjGoYC\ncO29K1m1aUeRqyksQ12SVNEWNDfx2YvmsOzlbSz++oNce8/KYpdUMIa6JKmiRQRXvPkEHrjyPJbM\nn8i371/F8xtbi11WQRjqkqRBYcyIoXzxkvkMqanilofWFLucgjDUJUmDxujhQ7jklMnc/vhaXt+9\nv9jl9DtDXZI0qPzXN0xn1752bnuspdil9DtDXZI0qMyf0sTpx43ilofW0N6Ril1OvzLUJUmDzv/9\nphm8tGUXP136SrFL6VeGuiRp0Fl80kQWHTeKa+9dSeueyrm3bqhLkgadiOCfLp7Hqzv28e37VxW7\nnH5jqEuSBqUFzSO59PRmbvzP1by4uTKeNGeoS5IGrU8vmU1dTTVf+MVzpFT+TXOGuiRp0BrfUMff\nXTCLB5/fzG+Xbyp2OcfMUJckDWrvP+c4jhszjO///sVil3LMDHVJ0qBWW13FexZN5Y+rX2PNqzuL\nXc4xMdQlSYPeuxc2UxXw08fK+3PrhrokadCb2FTHebPHc9tjLbS1dxS7nKNmqEuSBLxn0VQ2bt/L\ngy9sLnYpR81QlyQJOH/ueEYOq+UXT64vdilHzVCXJIlMw9wFcyfw2+Ub2ddWnlPwhrokSVlL5k+k\ndU8bD724pdilHBVDXZKkrHNPHMvwIdX8+pnynII31CVJyqqrrebcE8fy8IuvFbuUo2KoS5KUY+6k\nRtZs2cnufe3FLuWIGeqSJOWYM7GBlGDVpvJbua2goR4RSyJiZUSsioirejnmvIhYFhHPRsTvClmP\nJEl9mTWxAYAVG7YXuZIjV1OoE0dENXAdcAHQAjwaEXellJ7LOWYk8B1gSUrp5YgYX6h6JEnKx/Qx\nwxlaU8XKDa3FLuWIFXKkfiawKqX0YkppH3ArcEm3Yy4Hbk8pvQyQUir/de8kSWWtuiqYOWEEKwz1\ng0wBcp+M35Ldl2sWMCoiHoiIxyLi/T2dKCKuiIilEbF08+byfXyfJKk8nDxlJE+2bKOjIxW7lCNS\n7Ea5GuB04O3AYuAfI2JW94NSSjeklBallBaNGzduoGuUJA0yC6eNpHVPG6s2l1ezXCFDfS0wNWe7\nObsvVwtwT0ppZ0rpVeBB4JQC1iRJUp8WHjcKgMdf2lrkSo5MIUP9UWBmRMyIiCHAZcBd3Y75OfDG\niKiJiGHAWcDyAtYkSVKfjh87nJHDann85fIK9YJ1v6eU2iLiE8A9QDVwY0rp2Yj4aPb161NKyyPi\n18BTQAfw/ZTSM4WqSZKkfEQEp00dyeMvbyt2KUekYKEOkFK6G7i7277ru21/DfhaIeuQJOlInX7c\nKO5fuZnXd+2naVhtscvJS7Eb5SRJKkmnTcvcV39qbfmM1g11SZJ6MGPscABeeW13kSvJn6EuSVIP\nxjcMpSpg/euGuiRJZa2muooJjXWs27an2KXkzVCXJKkXE5vqHKlLklQJJjfVs+F1R+qSJJW9SU11\nrHt9NymVxzPgDXVJknoxaWQ9e/Z3sG3X/mKXkhdDXZKkXkxuqgNgXZncVzfUJUnqxQnjRwDw3Lrt\nRa4kP4a6JEm9OHHcCEYNq+XhF18rdil5MdQlSepFVVVw1owx/HH1lmKXkhdDXZKkwzjr+NG0bN1N\ny9ZdxS6lT4ddpS0ifgH02sefUnpHv1ckSVIJOWvGGAD++OJrNJ8+rMjVHF5fS69eOyBVSJJUouZM\nbKCpvpY/rt7Cu09vLnY5h3XYUE8p/W6gCpEkqRRVVQVnzhjNH1eXfrNcX9PvT3P46fcF/V6RJEkl\n5qwZo/nNcxtZ//puJjXVF7ucXvU1/f4XA1KFJEkl7OzjD9xX/8vTphS5mt71Nf3+0kAVIklSqZo7\nqZHhQ6pZ9sq2kg71vD7SFhFnR8SjEbEjIvZFRHtElMfjdSRJOkbVVUFDXS2797UXu5TDyvdz6t8G\n3gu8ANQDHwauK1RRkiSVmuqqYH9HR7HLOKy8Hz6TUloFVKeU2lNKNwFLCleWJEmlpaY6aO8o7SVY\n+2qU67QrIoYAyyLiq8B6fBqdJGkQqa4K2ko81PMN5r/OHvsJYCcwFXh3oYqSJKnU1FZV0d5e2qGe\n70j9VWBfSmkP8D8iohoYWriyJEkqLZU0Ur8PyH3gbT3w2/4vR5Kk0pS5p14ZjXJ1KaUdnRvZ70v7\nqfaSJPWjShqp74yIhZ0bEXE6sLswJUmSVHpqqiqn+/3vgJ9GxDoggInAXxWsKkmSSkxNVVXJj9Tz\nCvWU0qMRMQeYnd21MqW0v3BlSZJUWmqqg337KuCeekQMAz4D/G1K6RlgekS42IskadCorgra2isg\n1IGbgH3AOdnttcCXClKRJEklqKaCGuVOSCl9FdgPkFLaRebeuiRJg0JdbTW7KmRBl30RUQ8kgIg4\nAdhbsKokSSoxk0fWs27bblIq3dF6n6EeEQFcD/wamBoRPyTzMJpPF7g2SZJKxuSmOva2dbBl575i\nl9KrPrvfU0opIq4EzgPOJjPt/rcppVcLXJskSSVjyqjMM9fWbt3N2BGl+aT0fD+n/jhwfErp/ytk\nMZIklarJI+sAWLdtN6dMHVnkanqWb6ifBbwvIl4is0pbkBnELyhYZZIklZDmkdmR+rbSfaBqvqG+\nuKBVSJJU4hrraxg+pLr8Qz2l9FKhC5EkqZRFRFcHfKnK9yNtkiQNelNG1Zf0SN1QlyQpT5mR+p5i\nl9ErQ12SpDxNGVnPazv3sWtfW7FL6ZGhLklSnqaMrAco2dG6oS5JUp4md4V6ad5XN9QlScpT5wNo\nSrVZzlCXJClPExvrqApH6pIklb2a6iomNtaxdquhLklS2Svlz6ob6pIkHYHJIwdpqEfEkohYGRGr\nIuKqwxx3RkS0RcSlhaxHkqRjNXlkPRte30N7Ryp2KYcoWKhHRDVwHXAhMA94b0TM6+W4fwbuLVQt\nkiT1lykj62nrSGxu3VvsUg5RyJH6mcCqlNKLKaV9wK3AJT0c90ngZ8CmAtYiSVK/6HwATSlOwRcy\n1KcAr+Rst2T3dYmIKcA7ge8e7kQRcUVELI2IpZs3b+73QiVJyldDXWaB0x17S+9RscVulPs68JmU\nUsfhDkop3ZBSWpRSWjRu3LgBKk2SpEPV1VYDsGd/e5ErOVRe66kfpbXA1Jzt5uy+XIuAWyMCYCxw\nUUS0pZTuLGBdkiQdtbrazHh4sIX6o8DMiJhBJswvAy7PPSClNKPz+4i4GfilgS5JKmVDazIj9b37\nDzvJXBQFC/WUUltEfAK4B6gGbkwpPRsRH82+fn2h3luSpELpmn5vG1wjdVJKdwN3d9vXY5inlD5Q\nyFokSeoPpTz9XuxGOUmSysqBRrnSm3431CVJOgK11VVUV4UjdUmSKkFdTZUjdUmSKkFdbXVJNsoZ\n6pIkHaG62mqn3yVJqgR1tVUl+Tl1Q12SpCPkSF2SpArhPXVJkipEXa3d75IkVYS6GqffJUmqCN5T\nlySpQgx1+l2SpMrgSF2SpArhPXVJkipEXW0Ve9qcfpckqezV1VbT3pHY315awW6oS5J0hOpqM/FZ\nalPwhrokSUeorrYaoOQ64A11SZKOUF1NZ6g7UpckqawNzU6/7y2x578b6pIkHSGn3yVJqhAHQt2R\nuiRJZa2uprP73ZG6JEllzZG6JEkVoivUbZSTJKm8HXj4jNPvkiSVtfrsSH230++SJJW3hrpaAFr3\n7C9yJQcz1CVJOkJ1tVUMqa7i9d2GuiRJZS0iaKyvZbuhLklS+Wusr2H77rZil3EQQ12SpKPQVF/r\n9LskSZXAUJckqUIY6pIkVYjGulq2+5E2SZLKX1O2+72jIxW7lC6GuiRJR6GpvpaOBDv2lU4HvKEu\nSdJRaKrPPFXu9V2lMwVvqEuSdBQaO0O9hJrlDHVJko5CY30NQEk1yxnqkiQdhc7p91J6VKyhLknS\nUWhy+l2SpMpgqEuSVCGGD6mhKiipRV0MdUmSjkJVVWb5VUfqkiRVgFJ7/ruhLknSUTLUJUmqEE31\npbWoi6EuSdJRaqxzpC5JUkVozK7UVioKGuoRsSQiVkbEqoi4qofX3xcRT0XE0xHxh4g4pZD1SJLU\nnzrvqadUGsuvFizUI6IauA64EJgHvDci5nU7bDXwlpTSycAXgRsKVY8kSf2tqb6W/e2J3fvbi10K\nUNiR+pnAqpTSiymlfcCtwCW5B6SU/pBS2prdfBhoLmA9kiT1q65FXUrkATSFDPUpwCs52y3Zfb35\nEPCrAtYjSVK/KrVHxdYUuwCAiPgzMqH+xl5evwK4AmDatGkDWJkkSb0rtVAv5Eh9LTA1Z7s5u+8g\nEbEA+D5wSUppS08nSindkFJalFJaNG7cuIIUK0nSkRpMof4oMDMiZkTEEOAy4K7cAyJiGnA78Ncp\npecLWIskSf2u1NZUL9j0e0qpLSI+AdwDVAM3ppSejYiPZl+/HvgnYAzwnYgAaEspLSpUTZIk9afG\nutIaqRf0nnpK6W7g7m77rs/5/sPAhwtZgyRJhdI4iKbfJUmqaNVVQcPQGkNdkqRKUEqPijXUJUk6\nBo0ltFKboS5J0jFoqnf6XZKkitC5qEspMNQlSToGhrokSRWiqb52UCzoIklSxWusq2X3/nb2tXUU\nuxRDXZLeF2HZAAAOZUlEQVSkY9E0rHQeQGOoS5J0DEppURdDXZKkY9D5qNhS+Ky6oS5J0jEopUVd\nDHVJko5BKS2/aqhLknQMvKcuSVKF6Ar1XYa6JEllbUhNFfW11TbKSZJUCRpLZFEXQ12SpGNUKs9/\nN9QlSTpGhrokSRWiVBZ1MdQlSTpGjXWO1CVJqgiN9bU+fEaSpErQVF9L69422jtSUesw1CVJOkad\nD6BpLfJn1Q11SZKOUak8KtZQlyTpGDUa6pIkVQZH6pIkVQhDXZKkCnFgTfXiPoDGUJck6Rg11tcA\njtQlSSp79bXV1FaHoS5JUrmLiJJY1MVQlySpHzTW17Ldh89IklT+GuuK//x3Q12SpH7g9LskSRXC\nUJckqUIY6pIkVYim7JrqKRVv+VVDXZKkftBYX0NHgh17i/dUOUNdkqR+UArPfzfUJUnqB4a6JEkV\norEEFnUx1CVJ6geNdY7UJUmqCAeWXzXUJUkqa03DHKlLklQRRgypoSoo6qIuNUV75360f/9+Wlpa\n2LNnT7FLUR7q6upobm6mtra22KVIUr+pqgoai/xUuYoI9ZaWFhoaGpg+fToRUexydBgpJbZs2UJL\nSwszZswodjmS1K8a64ob6hUx/b5nzx7GjBljoJeBiGDMmDHOqkiqSMV+/ntFhDpgoJcR/1tJqlSG\neoWorq7m1FNPZf78+Vx88cVs27btsMdv27aN73znO13b69at49JLLz3q9z/vvPNYtGhR1/bSpUs5\n77zzDvvvrFmzhh/96Ee9vv7Tn/6Uk046iaqqKpYuXXrQa1/5ylc48cQTmT17Nvfcc0/X/s997nNM\nnTqVESNGHN0PIkllrHNRl2IpaKhHxJKIWBkRqyLiqh5ej4j4Zvb1pyJiYSHrKaT6+nqWLVvGM888\nw+jRo7nuuusOe3z3UJ88eTK33XbbMdWwadMmfvWrX+V9fF+hPn/+fG6//Xbe/OY3H7T/ueee49Zb\nb+XZZ5/l17/+NR//+Mdpb28H4OKLL+aRRx45uh9AkspcY30Nr1fiE+Uiohq4DrgQmAe8NyLmdTvs\nQmBm9usK4LuFqmcgnXPOOaxdu7Zr+2tf+xpnnHEGCxYs4POf/zwAV111FX/605849dRTufLKK1mz\nZg3z588H4Oabb+Zd73oXS5YsYebMmXz605/uOtfHPvYxFi1axEknndR1rk5XXnklX/7ylw+pp729\nnSuvvLKrhu9973tdNfz+97/n1FNP5V//9V8P+ffmzp3L7NmzD9n/85//nMsuu4yhQ4cyY8YMTjzx\nxK4gP/vss5k0adKRXjJJqgiNRV5+tZDd72cCq1JKLwJExK3AJcBzOcdcAtySMj/9wxExMiImpZTW\nH+2b/o9fPMtz67YfS92HmDe5kc9ffFJex7a3t3PffffxoQ99CIB7772XF154gUceeYSUEu94xzt4\n8MEHueaaa3jmmWdYtmwZkBk151q2bBlPPPEEQ4cOZfbs2Xzyk59k6tSpfPnLX2b06NG0t7dz/vnn\n89RTT7FgwQIg88vEHXfcwf33309DQ0PXuX7wgx/Q1NTEo48+yt69ezn33HN529vexjXXXMO1117L\nL3/5yyO6HmvXruXss8/u2m5ubj7olxhJGqya6mvZ197Bnv0d1A+pHvD3L+T0+xTglZztluy+Iz2G\niLgiIpZGxNLNmzf3e6H9Yffu3Zx66qlMnDiRjRs3csEFFwCZUL/33ns57bTTWLhwIStWrOCFF17o\n83znn38+TU1N1NXVMW/ePF566SUAfvKTn7Bw4UJOO+00nn32WZ577rmD/r2rr76aL33pSwftu/fe\ne7nllls49dRTOeuss9iyZUteNUiSjszxY4fzppljSVTeSL3fpJRuAG4AWLRo0WGvVL4j6v7WeU99\n165dLF68mOuuu45PfepTpJT4h3/4Bz7ykY8cdHz3kXl3Q4cO7fq+urqatrY2Vq9ezbXXXsujjz7K\nqFGj+MAHPnDIR8Pe+ta3cvXVV/Pwww937Usp8a1vfYvFixcfdOwDDzxw0PYHP/hBnnjiCSZPnszd\nd9/da21TpkzhlVcO/C7W0tLClCmH/C4mSYPOkvmTWDK/eLcgCzlSXwtMzdluzu470mPKyrBhw/jm\nN7/Jv/zLv9DW1sbixYu58cYb2bFjB5CZut60aRMNDQ20trYe0bm3b9/O8OHDaWpqYuPGjb02xV19\n9dV89atf7dpevHgx3/3ud9m/P9OR+fzzz7Nz585DarjppptYtmzZYQMd4B3veAe33nore/fuZfXq\n1bzwwguceeaZR/SzSJL6XyFD/VFgZkTMiIghwGXAXd2OuQt4f7YL/mzg9WO5n14qTjvtNBYsWMCP\nf/xj3va2t3H55ZdzzjnncPLJJ3PppZfS2trKmDFjOPfcc5k/fz5XXnllXuc95ZRTOO2005gzZw6X\nX3455557bo/HXXTRRYwbN65r+8Mf/jDz5s1j4cKFzJ8/n4985CO0tbWxYMECqqurOeWUU3pslLvj\njjtobm7moYce4u1vf3vXSP+kk07iPe95D/PmzWPJkiVcd911VFdn7h19+tOfprm5mV27dtHc3MwX\nvvCFI7x6kqSjFYXs0IuIi4CvA9XAjSmlL0fERwFSStdH5ikk3waWALuAD6aUlvZ6QjLT790/M718\n+XLmzp1biB9BBeJ/M0nKX0Q8llJa1NdxBb2nnlK6G7i7277rc75PwN8UsgZJkgYLnygnSVKFMNQl\nSaoQFRPqxXp6j46c/60kqTAqItTr6urYsmWLYVEGOtdTr6urK3YpklRxyuLhM31pbm6mpaWFUn3a\nnA5WV1dHc3NzscuQpIpTEaFeW1vLjBkzil2GJElFVRHT75IkyVCXJKliGOqSJFWIgj4mthAiYjPw\nUj+ecizwaj+eb7DyOh47r+Gx8xoeO6/hsSvENTwupTSur4PKLtT7W0Qszed5ujo8r+Ox8xoeO6/h\nsfMaHrtiXkOn3yVJqhCGuiRJFcJQhxuKXUCF8DoeO6/hsfMaHjuv4bEr2jUc9PfUJUmqFI7UJUmq\nEIMm1CNiSUSsjIhVEXFVD69HRHwz+/pTEbGwGHWWsjyu4fuy1+7piPhDRJxSjDpLWV/XMOe4MyKi\nLSIuHcj6ykU+1zEizouIZRHxbET8bqBrLHV5/P/cFBG/iIgns9fwg8Wos1RFxI0RsSkinunl9eJk\nSkqp4r+AauBPwPHAEOBJYF63Yy4CfgUEcDbwx2LXXUpfeV7DNwCjst9f6DU88muYc9x/AHcDlxa7\n7lL7yvPP4kjgOWBadnt8sesupa88r+FngX/Ofj8OeA0YUuzaS+ULeDOwEHiml9eLkimDZaR+JrAq\npfRiSmkfcCtwSbdjLgFuSRkPAyMjYtJAF1rC+ryGKaU/pJS2ZjcfBlyK7WD5/DkE+CTwM2DTQBZX\nRvK5jpcDt6eUXgZIKXktD5bPNUxAQ0QEMIJMqLcNbJmlK6X0IJlr0puiZMpgCfUpwCs52y3ZfUd6\nzGB2pNfnQ2R+S9UBfV7DiJgCvBP47gDWVW7y+bM4CxgVEQ9ExGMR8f4Bq6485HMNvw3MBdYBTwN/\nm1LqGJjyKkJRMqUill5VaYmIPyMT6m8sdi1l6OvAZ1JKHZkBko5SDXA6cD5QDzwUEQ+nlJ4vblll\nZTGwDHgrcALwm4j4fUppe3HL0uEMllBfC0zN2W7O7jvSYwazvK5PRCwAvg9cmFLaMkC1lYt8ruEi\n4NZsoI8FLoqItpTSnQNTYlnI5zq2AFtSSjuBnRHxIHAKYKhn5HMNPwhckzI3iFdFxGpgDvDIwJRY\n9oqSKYNl+v1RYGZEzIiIIcBlwF3djrkLeH+2Y/Fs4PWU0vqBLrSE9XkNI2IacDvw146IetTnNUwp\nzUgpTU8pTQduAz5uoB8in/+ffw68MSJqImIYcBawfIDrLGX5XMOXycx0EBETgNnAiwNaZXkrSqYM\nipF6SqktIj4B3EOm6/PGlNKzEfHR7OvXk+k0vghYBewi81uqsvK8hv8EjAG+kx1ptiUXhuiS5zVU\nH/K5jiml5RHxa+ApoAP4fkqpx48eDUZ5/ln8InBzRDxNpoP7MyklV2/LiogfA+cBYyOiBfg8UAvF\nzRSfKCdJUoUYLNPvkiRVPENdkqQKYahLklQhDHVJkiqEoS5JUoUw1CVJqhCGuqSjEhGD4jkXUjkx\n1KUKFBF3ZhcyeTYirsjuWxIRj2fXx74vu29ERNwUEU9n13x+d3b/jpxzXRoRN2e/vzkiro+IPwJf\njYgzI+KhiHgiIv4QEbOzx1VHxLUR8Uz2vJ+MiLdGxJ05570gIu4YuKsiVT5/05Yq039LKb0WEfXA\noxHxc+DfgDenlFZHxOjscf9I5vGVJwNExKg8zt0MvCGl1B4RjcCbsk8o+3PgfwLvBq4ApgOnZl8b\nDWwl87TBcSmlzWSesHVj//3Ikgx1qTJ9KiLemf1+KpmQfTCltBogpdS5DvSfk3nuN9n9W/M4909T\nSu3Z75uAf4+ImWTW367NOe/1KaW23PeLiP8X+C8RcRNwDuCSqFI/MtSlChMR55EJ1XNSSrsi4gEy\nS2jOOYLT5D4/uq7baztzvv8icH9K6Z0RMR14oI/z3gT8AthD5peDtiOoSVIfvKcuVZ4mYGs20OcA\nZ5MJ5jdHxAyAnOn33wB/0/kv5ky/b4yIuRFRBbyT3jVxYDnJD+Ts/w3wkc5mus73SymtA9YBV5MJ\neEn9yFCXKs+vgZqIWA5cAzwMbCYzBX97RDwJ/O/ssV8CRmUb2p4E/iy7/yrgl8AfgMMtF/lV4CsR\n8QQHz/x9n8zSnU9lz3t5zms/BF5JKbkUqtTPXKVN0oCKiG8DT6SUflDsWqRKY6hLGjAR8RiZe/IX\npJT2FrseqdIY6pIkVQjvqUuSVCEMdUmSKoShLklShTDUJUmqEIa6JEkVwlCXJKlC/P+U5ArnhAQ+\nrQAAAABJRU5ErkJggg==\n",
202 |       "text/plain": [
203 |        "<matplotlib.figure.Figure at 0x7f73b09e3190>"
204 |       ]
205 |      },
206 |      "metadata": {},
207 |      "output_type": "display_data"
208 |     }
209 |    ],
210 |    "source": [
211 |     "reload(anno_func)\n",
212 |     "def get_acc_res(results_annos, annos,**argv):\n",
213 |     "    scs = [ obj['score'] for k,img in results_annos['imgs'].items() for obj in img['objects']]\n",
214 |     "    scs = sorted(scs)\n",
215 |     "    accs = [0]\n",
216 |     "    recs = [1]\n",
217 |     "    for i, score in enumerate(np.linspace(0, scs[-1], 100)):\n",
218 |     "        sm = anno_func.eval_annos(annos, results_annos, iou=0.5, check_type=True, types=anno_func.type5, minscore=score, **argv)\n",
219 |     "        print \"\\r%s %s %s\" % (i, score, sm['report']), \n",
220 |     "        sys.stdout.flush()\n",
221 |     "        accs.append(sm['accuracy'])\n",
222 |     "        if len(accs)>=2 and accs[-1]<accs[-2]:\n",
223 |     "            accs[-1] = accs[-2]\n",
224 |     "        recs.append(sm['recall'])\n",
225 |     "    accs.append(1)\n",
226 |     "    recs.append(0)\n",
227 |     "    return accs, recs\n",
228 |     "sizes = [0,32,96,400]\n",
229 |     "ac_rc = []\n",
230 |     "\n",
231 |     "for i in range(4):\n",
232 |     "    if i==3:\n",
233 |     "        l=sizes[0]\n",
234 |     "        r=sizes[-1]\n",
235 |     "    else:\n",
236 |     "        l=sizes[i]\n",
237 |     "        r=sizes[i+1]\n",
238 |     "    acc0, recs0 = get_acc_res(results_annos, annos, minboxsize=l, maxboxsize=r)\n",
239 |     "    ac_rc.append([acc0, recs0])\n",
240 |     "    \n",
241 |     "    pl.figure(figsize=(8,6))\n",
242 |     "    pl.plot(acc0, recs0, label='RetinaNet-101')\n",
243 |     "    pl.xlabel(\"accuracy\")\n",
244 |     "    pl.ylabel(\"recall\")\n",
245 |     "    pl.title(\"size: (%s,%s]\"%(l,r))\n",
246 |     "\n",
247 |     "    pl.legend(bbox_to_anchor=(0, 0), loc=\"lower left\")"
248 |    ]
249 |   },
250 |   {
251 |    "cell_type": "code",
252 |    "execution_count": 7,
253 |    "metadata": {},
254 |    "outputs": [],
255 |    "source": [
256 |     "def compute_AP(acc, res):\n",
257 |     "    AP=0\n",
258 |     "    for i in range(1,len(acc)):\n",
259 |     "        AP+=fabs(res[i]-res[i-1])*0.5*(acc[i-1]+acc[i])\n",
260 |     "    return AP"
261 |    ]
262 |   },
263 |   {
264 |    "cell_type": "code",
265 |    "execution_count": 8,
266 |    "metadata": {},
267 |    "outputs": [
268 |     {
269 |      "name": "stdout",
270 |      "output_type": "stream",
271 |      "text": [
272 |       "('RetinaNet-101', 'MAP: ', 0.8586124015556745)\n"
273 |      ]
274 |     }
275 |    ],
276 |    "source": [
277 |     "MAP0=compute_AP(acc0, recs0)\n",
278 |     "print('RetinaNet-101', 'MAP: ',MAP0)"
279 |    ]
280 |   },
281 |   {
282 |    "cell_type": "code",
283 |    "execution_count": 20,
284 |    "metadata": {},
285 |    "outputs": [],
286 |    "source": [
287 |     "# df=pd.read_excel('../data/AP.xlsx')\n",
288 |     "# display(df)"
289 |    ]
290 |   }
291 |  ],
292 |  "metadata": {
293 |   "anaconda-cloud": {},
294 |   "kernelspec": {
295 |    "display_name": "py27",
296 |    "language": "python",
297 |    "name": "py27"
298 |   },
299 |   "language_info": {
300 |    "codemirror_mode": {
301 |     "name": "ipython",
302 |     "version": 2
303 |    },
304 |    "file_extension": ".py",
305 |    "mimetype": "text/x-python",
306 |    "name": "python",
307 |    "nbconvert_exporter": "python",
308 |    "pygments_lexer": "ipython2",
309 |    "version": "2.7.13"
310 |   }
311 |  },
312 |  "nbformat": 4,
313 |  "nbformat_minor": 2
314 | }
315 | 


--------------------------------------------------------------------------------
/generate_gt.py:
--------------------------------------------------------------------------------
 1 | #!/usr/bin/env python
 2 | # -*- coding: utf-8 -*-
 3 | # @Author: CHEN Shen
 4 | 
 5 | import json
 6 | import os
 7 | import shutil
 8 | 
 9 | 
10 | def gene_gt(annos_path, output_path, transform=False, image_ext='jpg'):
11 |     """
12 |     利用预测得到的annotations，生成每张图片对应的ground_truth文件
13 |     :param annos_path: annotation路径
14 |     :param output_path: 输出路径
15 |     :param transform: 是否将ground truth转换到1280*1024的情况
16 |     :param image_ext: 图片的扩展名
17 |     """
18 |     assert os.path.exists(annos_path)
19 |     if os.path.exists(output_path):
20 |         shutil.rmtree(output_path)
21 |     os.mkdir(output_path)
22 | 
23 |     anns = json.load(open(annos_path, 'r'))
24 |     for imgid in anns['imgs']:
25 |         img_name = imgid + image_ext
26 |         gt_name = imgid + '.txt'
27 |         with open(os.path.join(output_path, gt_name), 'w') as fout:
28 |             print imgid
29 |             for bboxs in anns['imgs'][imgid]['objects']:
30 |                 res = ""
31 |                 category = bboxs['category']
32 |                 bbox = bboxs['bbox']
33 |                 x_min = bbox['xmin']
34 |                 x_max = bbox['xmax']
35 |                 y_min = bbox['ymin']
36 |                 y_max = bbox['ymax']
37 |                 if transform == True:
38 |                     x_min *= (1280/512)
39 |                     x_max *= (1280/512)
40 |                     y_min *= (1024/512)
41 |                     y_max *= (1024/512)
42 |                 res += (img_name+" "+category+" "+str(x_min)+" "+str(y_min)+" "+str(x_max)+" "+str(y_max)+"\n")
43 |                 fout.write(res)
44 | 
45 | 
46 | if __name__ == '__main__':
47 |     annos_path = './data/valid_target.json'
48 |     output_path = './result/ground_truth'
49 |     gene_gt(annos_path, output_path, True)
50 | 


--------------------------------------------------------------------------------
/loss.py:
--------------------------------------------------------------------------------
 1 | from __future__ import print_function
 2 | 
 3 | import torch
 4 | import torch.nn as nn
 5 | import torch.nn.functional as F
 6 | from torch.autograd import Variable
 7 | import numpy as np
 8 | 
 9 | def one_hot(index, classes):
10 |    
11 |     size = index.size() + (classes,)
12 |     view = index.size() + (1,)
13 |     mask = torch.Tensor(*size).fill_(0)
14 |     index = index.view(*view)
15 |     ones = 1.
16 |     if isinstance(index, Variable):
17 |         ones = Variable(torch.Tensor(index.size()).fill_(1))
18 |         mask = Variable(mask, volatile=index.volatile)
19 |     return mask.scatter_(1, index, ones)
20 | 
21 | def one_hot_embedding(labels, num_classes):
22 |     
23 |     y = torch.eye(num_classes)  # [D,D]
24 |     return y[labels]            # [N,D]
25 |     
26 | class FocalLoss(nn.Module):
27 |     def __init__(self, num_classes):
28 |         super(FocalLoss, self).__init__()
29 |         self.num_classes = num_classes
30 | 
31 |     def focal_loss(self, x, y):
32 |         y = one_hot(y.cpu(), x.size(-1)).cuda()
33 |         logit = F.softmax(x)
34 |         logit = logit.clamp(1e-7, 1. - 1e-7)
35 |         loss = -1 * y.float() * torch.log(logit)
36 |         loss =loss * (1 - logit) ** 2
37 |         return loss.sum()
38 |     
39 |         
40 |         
41 |     def forward(self, loc_preds, loc_targets, cls_preds, cls_targets,pos,verbose):
42 |         batch_size, num_boxes = cls_targets.size()
43 |         num_pos = pos.data.long().sum()
44 |         mask = pos.unsqueeze(2).expand_as(loc_preds)
45 |         masked_loc_preds = loc_preds[mask].view(-1,4)
46 |         masked_loc_targets = loc_targets[mask].view(-1,4)
47 |         loc_loss = F.smooth_l1_loss(masked_loc_preds, masked_loc_targets,size_average=False)      
48 |         pos_neg = cls_targets > -1
49 |         mask = pos_neg.unsqueeze(2).expand_as(cls_preds)
50 |         masked_cls_preds = cls_preds[mask].view(-1, self.num_classes+1)
51 |         cls_loss =self.focal_loss(masked_cls_preds, cls_targets[pos_neg])
52 |         if verbose:
53 |             # TODO solved error: invalid index of a 0-dim tensor
54 |             try:
55 |                 loc_loss_data = loc_loss.data[0]
56 |                 cls_loss_data = cls_loss.data[0]
57 |             except:
58 |                 loc_loss_data = loc_loss.item()
59 |                 cls_loss_data = cls_loss.item()
60 |             print('loc_loss: %.5f | cls_loss: %.4f' %((loc_loss_data/num_pos), cls_loss_data/(num_pos)), end=' | ')
61 |         
62 |         loc_loss /= num_pos
63 |         cls_loss /= num_pos
64 |         loss = loc_loss+cls_loss
65 |         return loss, loc_loss, cls_loss


--------------------------------------------------------------------------------
/main.py:
--------------------------------------------------------------------------------
 1 | #!/usr/bin/env python
 2 | # -*- coding: utf-8 -*-
 3 | # @Author: CHEN Shen
 4 | 
 5 | import os
 6 | from preprocessing.datasets import VocLikeDataset
 7 | from encoder import DataEncoder
 8 | import config as cfg
 9 | from utils import get_mean_and_std
10 | import preprocessing.transforms as transforms
11 | 
12 | 
13 | if __name__ == '__main__':
14 |     # cmd = "python2 test.py -m demo --backbone resnet101"
15 |     # cmd = "python2 test.py -m valid --backbone resnet101"
16 |     cmd = "python2 train.py --exp model -r"
17 |     os.system(cmd)


--------------------------------------------------------------------------------
/preprocessing/__init__.py:
--------------------------------------------------------------------------------
1 | 
2 | 


--------------------------------------------------------------------------------
/preprocessing/annotations.py:
--------------------------------------------------------------------------------
 1 | import os
 2 | import config as cfg
 3 | from preprocessing.bbox import BoundingBox
 4 | from preprocessing.errors import UnsupportedExtensionError, UnsupportedFormatError
 5 | import json
 6 | 
 7 | class AnnotationDir:
 8 |     def __init__(self, anno_file, labels):
 9 |         self.anno_file = anno_file
10 |         self.labels = labels
11 |         self.ann_dict = self.build_annotations()
12 |         
13 | 
14 |     def build_annotations(self):
15 |         box_dict = {}
16 |         annos = json.loads(open(self.anno_file).read())
17 | 
18 |         for id in annos['imgs']:
19 |             boxes = []
20 |             for sign_dict in annos['imgs'][id]['objects']:
21 |                 label = sign_dict['category']
22 |                 # TODO solve error: u'po' is not in list
23 |                 if label not in self.labels:
24 |                     continue
25 |                 left = int(sign_dict['bbox']['xmin'])
26 |                 right = int(sign_dict['bbox']['xmax'])
27 |                 top = int(sign_dict['bbox']['ymin'])
28 |                 bottom = int(sign_dict['bbox']['ymax'])
29 |                 box = BoundingBox(left, top, right, bottom, cfg.width,cfg.height,self.labels.index(label))
30 |                 boxes.append(box)
31 |             if len(boxes) > 0:
32 |                 box_dict[annos['imgs'][id]['path'].split('/')[-1]] = boxes
33 |         return box_dict
34 | 
35 | 
36 |     def get_boxes(self, fn):
37 |         
38 |         return self.ann_dict[fn]
39 | 


--------------------------------------------------------------------------------
/preprocessing/bbox.py:
--------------------------------------------------------------------------------
 1 | class BoundingBox:
 2 |     def __init__(self, left, top, right, bottom, image_width, image_height, label):
 3 |         self.left = left
 4 |         self.top = top
 5 |         self.right = right
 6 |         self.bottom = bottom
 7 |         self.width = right - left
 8 |         self.height = bottom - top
 9 |         self.image_width = image_width
10 |         self.image_height = image_height
11 |         self.label = label
12 | 
13 |     def __repr__(self):
14 |         return '(x1: {}, y1: {}, x2: {}, y2: {} ({}))'.format(self.left, self.top, self.right, self.bottom, self.label)
15 | 
16 |     def flip(self):
17 |         left = self.image_width - self.right
18 |         top = self.image_height - self.bottom
19 |         right = self.image_width - self.left
20 |         bottom = self.image_height - self.top
21 |         self.left = left
22 |         self.top = top
23 |         self.right = right
24 |         self.bottom = bottom
25 |         return self
26 | 
27 |     def resize(self, width, height):
28 |         width_ratio = width / float(self.image_width)
29 |         height_ratio = height / float(self.image_height)
30 |         self.left = int(self.left * width_ratio)
31 |         self.top = int(self.top * height_ratio)
32 |         self.right = int(self.right * width_ratio)
33 |         self.bottom = int(self.bottom * height_ratio)
34 |         return self


--------------------------------------------------------------------------------
/preprocessing/datasets.py:
--------------------------------------------------------------------------------
 1 | import os
 2 | import random
 3 | from PIL import Image
 4 | import matplotlib.pyplot as plt
 5 | import torch
 6 | from torch.utils.data.dataset import Dataset
 7 | 
 8 | from preprocessing.annotations import AnnotationDir
 9 | 
10 | import config as cfg
11 | 
12 | class VocLikeDataset(Dataset):
13 |     
14 |     def __init__(self, image_dir, annotation_file, imageset_fn, image_ext, classes, encoder, transform=None, test=False):
15 |         self.image_dir_path = image_dir
16 |         self.image_ext = image_ext
17 |         self.filenames=imageset_fn
18 |         if not test:
19 |             self.annotation_dir = AnnotationDir(annotation_file,classes)
20 | 
21 |         self.encoder = encoder
22 |         self.transform = transform
23 |         self.test = test
24 |         
25 | 
26 |     def __getitem__(self, index):
27 |         
28 |         image_fn = self.filenames[index]
29 |         fn=os.path.join(self.image_dir_path.split('/')[-1],image_fn)
30 |         image_path = os.path.join(self.image_dir_path, image_fn)
31 |         image = Image.open(os.path.join('data', image_fn))
32 |         example={}
33 |         example['image']=image
34 |         if not self.test:
35 |             boxes = self.annotation_dir.get_boxes(image_fn.split('/')[-1])
36 |             example['boxes']=boxes
37 |        
38 |         if self.transform:
39 |             example = self.transform(example)
40 |         return example
41 | 
42 |     def __len__(self):
43 |         return len(self.filenames)
44 | 
45 |     def collate_fn(self, batch):
46 |         imgs = [example['image'] for example in batch]
47 |         if not self.test:
48 |             boxes  = [example['boxes'] for example in batch]
49 |             labels = [example['labels'] for example in batch]
50 |         img_sizes = [img.size()[1:] for img in imgs]
51 |         
52 |         max_h = max([im.size(1) for im in imgs])
53 |         max_w = max([im.size(2) for im in imgs])
54 |         num_imgs = len(imgs)
55 |         inputs = torch.zeros(num_imgs, 3, max_h, max_w)
56 | 
57 |         loc_targets = []
58 |         cls_targets = []
59 |         for i in range(num_imgs):
60 |             im = imgs[i]
61 |             imh, imw = im.size(1), im.size(2)
62 |             inputs[i,:,:imh,:imw] = im
63 |             if not self.test:
64 |                 loc_target, cls_target = self.encoder.encode(boxes[i], labels[i], input_size=[cfg.width,cfg.height])
65 |                 loc_targets.append(loc_target)
66 |                 cls_targets.append(cls_target)
67 |         if not self.test:
68 |             return inputs, torch.stack(loc_targets), torch.stack(cls_targets)
69 |         return inputs
70 | 
71 | 
72 |     # TODO use for compute mean and std.
73 |     def load(self, index):
74 |         import cv2
75 |         image_fn = self.filenames[index]
76 |         image = cv2.imread(os.path.join('data', image_fn))
77 |         return image
78 | 
79 | 


--------------------------------------------------------------------------------
/preprocessing/errors.py:
--------------------------------------------------------------------------------
 1 | class UnsupportedExtensionError(Exception):
 2 |     def __init__(self, ext):
 3 |         message = '{} is not a known file extension'.format(ext)
 4 |         super(UnsupportedExtensionError, self).__init__(message)
 5 | 
 6 | 
 7 | class UnsupportedFormatError(Exception):
 8 |     def __init__(self, fmt):
 9 |         message = '{} is not a known annotation format'.format(fmt)
10 |         super(UnsupportedFormatError, self).__init__(message)
11 | 


--------------------------------------------------------------------------------
/preprocessing/transforms.py:
--------------------------------------------------------------------------------
 1 | import numpy as np
 2 | import random
 3 | from PIL import Image
 4 | 
 5 | import torch
 6 | 
 7 | 
 8 | class Compose:
 9 |     def __init__(self, transforms):
10 |         self.transforms = transforms
11 | 
12 |     def __call__(self, example):
13 |         for t in self.transforms:
14 |             example = t(example)
15 |         return example
16 | 
17 | 
18 | class Normalize:
19 |     def __init__(self, mean, std):
20 |         self.mean = mean
21 |         self.std = std
22 | 
23 |     def __call__(self, example):
24 |         image, boxes, labels = example['image'], example['boxes'], example['labels']
25 |         for t, m, s in zip(image, self.mean, self.std):
26 |             t.sub_(m).div_(s)
27 |         return {'image': image, 'boxes': boxes, 'labels': labels}
28 | 
29 | 
30 | class Scale:
31 |     def __init__(self, output_size):
32 |         assert isinstance(output_size, (int, tuple))
33 |         self.output_size = output_size
34 | 
35 |     def __call__(self, example):
36 |         image, boxes = example['image'], example['boxes']
37 |        
38 |         width, height = image.size
39 |         if isinstance(self.output_size, int):
40 |             if width < height:
41 |                 new_width, new_height = width / height * self.output_size, self.output_size
42 |             else:
43 |                 new_width, new_height = self.output_size, height / width * self.output_size
44 |         else:
45 |             new_width, new_height = self.output_size
46 |         new_width, new_height = int(new_width), int(new_height)
47 |         image=image.resize((new_width, new_height))
48 |         boxes=[box.resize(new_width, new_height) for box in boxes]
49 |         
50 |         return {'image': image, 'boxes': boxes}
51 | 
52 | 
53 | class RandomHorizontalFlip:
54 |     def __call__(self, example):
55 |         image, boxes = example['image'], example['boxes']
56 |         if random.random() < 0.5:
57 |             image = image.transpose(Image.FLIP_LEFT_RIGHT)
58 |             boxes = [box.flip() for box in boxes]
59 |         return {'image': image, 'boxes': boxes}
60 | 
61 | 
62 | class ToTensor:
63 |     def __call__(self, example):
64 |         image, boxes = example['image'], example['boxes']
65 |         image = np.array(image).transpose((2, 0, 1))
66 |         labels = np.array([box.label for box in boxes])
67 |         boxes = np.array([[box.left, box.top, box.right, box.bottom] for box in boxes])
68 |         image, boxes, labels = torch.from_numpy(image), torch.from_numpy(boxes), torch.from_numpy(labels)
69 |         if isinstance(image, torch.ByteTensor):
70 |             image = image.float().div(255)
71 |         return {'image': image, 'boxes': boxes, 'labels': labels}
72 | 
73 | 
74 | class Unnormalize:
75 |     def __init__(self, mean, std):
76 |         self.mean = mean
77 |         self.std = std
78 | 
79 |     def __call__(self, tensor):
80 |         for t, m, s in zip(tensor, self.mean, self.std):
81 |             t.mul_(s).add_(m)
82 |         return tensor
83 | 


--------------------------------------------------------------------------------
/resnet.py:
--------------------------------------------------------------------------------
  1 | import torch.utils.model_zoo as model_zoo
  2 | from torchvision.models.resnet import BasicBlock, Bottleneck, ResNet
  3 | 
  4 | 
  5 | __all__ = ['resnet18', 'resnet34', 'resnet50', 'resnet101', 'resnet152']
  6 | 
  7 | 
  8 | model_urls = {
  9 |     'resnet18': 'https://download.pytorch.org/models/resnet18-5c106cde.pth',
 10 |     'resnet34': 'https://download.pytorch.org/models/resnet34-333f7ec4.pth',
 11 |     'resnet50': 'https://download.pytorch.org/models/resnet50-19c8e357.pth',
 12 |     'resnet101': 'https://download.pytorch.org/models/resnet101-5d3b4d8f.pth',
 13 |     'resnet152': 'https://download.pytorch.org/models/resnet152-b121ed2d.pth',
 14 | }
 15 | 
 16 | 
 17 | class BasicBlockFeatures(BasicBlock):
 18 |     def forward(self, x):
 19 |         if isinstance(x, tuple):
 20 |             x = x[0]
 21 | 
 22 |         residual = x
 23 | 
 24 |         out = self.conv1(x)
 25 |         out = self.bn1(out)
 26 |         out = self.relu(out)
 27 | 
 28 |         out = self.conv2(out)
 29 |         conv2_rep = out
 30 |         out = self.bn2(out)
 31 | 
 32 |         if self.downsample is not None:
 33 |             residual = self.downsample(x)
 34 | 
 35 |         out += residual
 36 |         out = self.relu(out)
 37 | 
 38 |         return out, conv2_rep
 39 | 
 40 | 
 41 | class BottleneckFeatures(Bottleneck):
 42 |     def forward(self, x):
 43 |         if isinstance(x, tuple):
 44 |             x = x[0]
 45 | 
 46 |         residual = x
 47 | 
 48 |         out = self.conv1(x)
 49 |         out = self.bn1(out)
 50 |         out = self.relu(out)
 51 | 
 52 |         out = self.conv2(out)
 53 |         out = self.bn2(out)
 54 |         out = self.relu(out)
 55 | 
 56 |         out = self.conv3(out)
 57 |         conv3_rep = out
 58 |         out = self.bn3(out)
 59 | 
 60 |         if self.downsample is not None:
 61 |             residual = self.downsample(x)
 62 | 
 63 |         out += residual
 64 |         out = self.relu(out)
 65 | 
 66 |         return out, conv3_rep
 67 | 
 68 |         
 69 | class ResNetFeatures(ResNet):
 70 |     def forward(self, x):
 71 |         x = self.conv1(x)
 72 |         x = self.bn1(x)
 73 |         x = self.relu(x)
 74 |         x = self.maxpool(x)
 75 | 
 76 |         x, c2 = self.layer1(x)
 77 |         x, c3 = self.layer2(x)
 78 |         x, c4 = self.layer3(x)
 79 |         x, c5 = self.layer4(x)
 80 | 
 81 |         return c2, c3, c4, c5
 82 |         
 83 | 
 84 | 
 85 | def resnet18(pretrained=False, **kwargs):
 86 |     model = ResNetFeatures(BasicBlockFeatures, [2, 2, 2, 2], **kwargs)
 87 | 
 88 |     if pretrained:
 89 |         model.load_state_dict(model_zoo.load_url(model_urls['resnet18']))
 90 | 
 91 |     return model
 92 | 
 93 | 
 94 | def resnet34(pretrained=False, **kwargs):
 95 |     model = ResNetFeatures(BasicBlockFeatures, [3, 4, 6, 3], **kwargs)
 96 | 
 97 |     if pretrained:
 98 |         model.load_state_dict(model_zoo.load_url(model_urls['resnet34']))
 99 | 
100 |     return model
101 | 
102 | 
103 | def resnet50(pretrained=False, **kwargs):
104 |     model = ResNetFeatures(BottleneckFeatures, [3, 4, 6, 3], **kwargs)
105 | 
106 |     if pretrained:
107 |         model.load_state_dict(model_zoo.load_url(model_urls['resnet50']))
108 | 
109 |     return model
110 | 
111 | 
112 | def resnet101(pretrained=False, **kwargs):
113 |     model = ResNetFeatures(BottleneckFeatures, [3, 4, 23, 3], **kwargs)
114 | 
115 |     if pretrained:
116 |         model.load_state_dict(model_zoo.load_url(model_urls['resnet101']))
117 | 
118 |     return model
119 | 
120 | 
121 | def resnet152(pretrained=False, **kwargs):
122 |     model = ResNetFeatures(BottleneckFeatures, [3, 8, 36, 3], **kwargs)
123 | 
124 |     if pretrained:
125 |         model.load_state_dict(model_zoo.load_url(model_urls['resnet152']))
126 | 
127 |     return model
128 | 


--------------------------------------------------------------------------------
/retinanet.py:
--------------------------------------------------------------------------------
  1 | import math
  2 | 
  3 | import torch
  4 | import torch.nn as nn
  5 | import torch.nn.functional as F
  6 | from torch.autograd import Variable
  7 | 
  8 | from resnet import *
  9 | torch.backends.cudnn.enabled = False
 10 | 
 11 | 
 12 | def classification_layer_init(tensor, pi=0.01):
 13 |     fill_constant = - math.log((1 - pi) / pi)
 14 |     if isinstance(tensor, Variable):
 15 |         classification_layer_init(tensor.data)
 16 |     return tensor.fill_(fill_constant)
 17 | 
 18 | def init_conv_weights(layer):
 19 |     nn.init.normal(layer.weight.data, std=0.01)
 20 |     nn.init.constant(layer.bias.data, val=0)
 21 |     return layer
 22 | 
 23 | def conv1x1(in_channels, out_channels, **kwargs):
 24 |     layer = nn.Conv2d(in_channels, out_channels, kernel_size=1, **kwargs)
 25 |     layer = init_conv_weights(layer)
 26 |     return layer
 27 | 
 28 | def conv3x3(in_channels, out_channels, **kwargs):
 29 |     layer = nn.Conv2d(in_channels, out_channels, kernel_size=3, **kwargs)
 30 |     layer = init_conv_weights(layer)
 31 |     return layer
 32 | 
 33 | 
 34 |             
 35 | def upsample(feature, sample_feature, scale_factor=2):
 36 |     out_channels=sample_feature.size()[1:]
 37 |     return F.upsample(feature,scale_factor=scale_factor)
 38 |     
 39 | class GroupNorm(nn.Module):
 40 |     def __init__(self, num_features, num_groups=32, eps=1e-5):
 41 |         super(GroupNorm, self).__init__()
 42 |         self.weight = nn.Parameter(torch.ones(1,num_features,1,1))
 43 |         self.bias = nn.Parameter(torch.zeros(1,num_features,1,1))
 44 |         self.num_groups = num_groups
 45 |         self.eps = eps
 46 | 
 47 |     def forward(self, x):
 48 |         N,C,H,W = x.size()
 49 |         G = self.num_groups
 50 |         assert C % G == 0
 51 | 
 52 |         x = x.view(N,G,-1)
 53 |         mean = x.mean(-1, keepdim=True)
 54 |         var = x.var(-1, keepdim=True)
 55 | 
 56 |         x = (x-mean) / (var+self.eps).sqrt()
 57 |         x = x.view(N,C,H,W)
 58 |         return x * self.weight + self.bias
 59 | 
 60 | class FeaturePyramid(nn.Module):
 61 |     def __init__(self, resnet):
 62 |         super(FeaturePyramid, self).__init__()
 63 | 
 64 |         self.resnet = resnet
 65 | 
 66 |         self.pyramid_transformation_3 = conv1x1(512, 256)
 67 |         self.pyramid_transformation_4 = conv1x1(1024, 256)
 68 |         self.pyramid_transformation_5 = conv1x1(2048, 256)
 69 | 
 70 |         self.pyramid_transformation_6 = conv3x3(2048, 256, padding=1, stride=2)
 71 |         self.pyramid_transformation_7 = conv3x3(256, 256, padding=1, stride=2)
 72 | 
 73 |         self.upsample_transform_1 = conv3x3(256, 256, padding=1)
 74 |         self.upsample_transform_2 = conv3x3(256, 256, padding=1)
 75 |         self.dropout=nn.Dropout(p=0.5)
 76 |         
 77 | 
 78 |     def forward(self, x):
 79 |         _, resnet_feature_3, resnet_feature_4, resnet_feature_5 = self.resnet(x)
 80 | 
 81 |         resnet_feature_3=self.dropout(resnet_feature_3)
 82 |         resnet_faeture_4=self.dropout(resnet_feature_4)
 83 |         resnet_feature_5=self.dropout(resnet_feature_5)
 84 |         
 85 |         pyramid_feature_6 = self.pyramid_transformation_6(resnet_feature_5)
 86 |         pyramid_feature_7 = self.pyramid_transformation_7(F.relu(pyramid_feature_6))
 87 | 
 88 |         pyramid_feature_5 = self.pyramid_transformation_5(resnet_feature_5)
 89 | 
 90 |         pyramid_feature_4 = self.pyramid_transformation_4(resnet_feature_4)
 91 |         upsampled_feature_5 = upsample(pyramid_feature_5, pyramid_feature_4)
 92 |         pyramid_feature_4 = self.upsample_transform_1(torch.add(upsampled_feature_5, pyramid_feature_4))
 93 |         
 94 |         pyramid_feature_3 = self.pyramid_transformation_3(resnet_feature_3)
 95 |         upsampled_feature_4 = upsample(pyramid_feature_4, pyramid_feature_3)
 96 |         pyramid_feature_3 = self.upsample_transform_2(torch.add(upsampled_feature_4, pyramid_feature_3))
 97 | 
 98 |         return pyramid_feature_3, pyramid_feature_4, pyramid_feature_5, pyramid_feature_6, pyramid_feature_7
 99 | 
100 | 
101 | class SubNet(nn.Module):
102 |     def __init__(self, k, anchors=9, depth=4, cls=False, activation=F.relu):
103 |         super(SubNet, self).__init__()
104 |         self.anchors = anchors
105 |         self.activation = activation
106 |         self.base = nn.ModuleList([conv3x3(256, 256, padding=1) for _ in range(depth)])
107 |         self.output = nn.Conv2d(256, k * anchors, kernel_size=3, stride=1,padding=1)
108 |         self.dropout=nn.Dropout(p=0.5)
109 |         self.bn=nn.BatchNorm2d(256)
110 |         self.gn=GroupNorm(256, 32)
111 |       
112 |         init_conv_weights(self.output)
113 |         
114 | 
115 |     def forward(self, x):
116 |         for layer in self.base:
117 |             x = layer(x)
118 |             
119 |             x=self.gn(x)
120 |             x=self.activation(x)
121 |             
122 |             x=self.dropout(x)
123 |         x = self.output(x)
124 |         x = x.permute(0, 2, 3, 1).contiguous().view(x.size(0), x.size(2) * x.size(3) * self.anchors, -1)
125 |         return x
126 | 
127 | 
128 | class RetinaNet(nn.Module):
129 |     backbones = {
130 |         'resnet18': resnet18,
131 |         'resnet34': resnet34,
132 |         'resnet50': resnet50,
133 |         'resnet101': resnet101,
134 |         'resnet152': resnet152
135 |     }
136 | 
137 |     def __init__(self, backbone='resnet101', num_classes=1, pretrained=True):
138 |         super(RetinaNet, self).__init__()
139 |         
140 |         self.resnet = RetinaNet.backbones[backbone](pretrained=pretrained)
141 |         self.feature_pyramid = FeaturePyramid(self.resnet)
142 |         
143 |         self.subnet_classes = SubNet(num_classes+1,cls=True)
144 |         self.subnet_boxes = SubNet(4)
145 | 
146 |     def forward(self, x):
147 |         pyramid_features = self.feature_pyramid(x)
148 |         class_predictions = [self.subnet_classes(p) for p in pyramid_features]
149 |         bbox_predictions = [self.subnet_boxes(p) for p in pyramid_features]
150 |         return torch.cat(bbox_predictions, 1), torch.cat(class_predictions, 1)
151 | 
152 | 
153 | 
154 | 


--------------------------------------------------------------------------------
/samples/group1-296_0_1.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chenshen03/Traffic-Sign-Detection-with-RetinaNet/f34f6194b80fcd16b12ded9ac8b17ba4fb2fbdb0/samples/group1-296_0_1.jpg


--------------------------------------------------------------------------------
/samples/group1-6_0_1.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chenshen03/Traffic-Sign-Detection-with-RetinaNet/f34f6194b80fcd16b12ded9ac8b17ba4fb2fbdb0/samples/group1-6_0_1.jpg


--------------------------------------------------------------------------------
/samples/group3-162_0_1.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chenshen03/Traffic-Sign-Detection-with-RetinaNet/f34f6194b80fcd16b12ded9ac8b17ba4fb2fbdb0/samples/group3-162_0_1.jpg


--------------------------------------------------------------------------------
/samples/group5-109_2_1.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chenshen03/Traffic-Sign-Detection-with-RetinaNet/f34f6194b80fcd16b12ded9ac8b17ba4fb2fbdb0/samples/group5-109_2_1.jpg


--------------------------------------------------------------------------------
/samples/group7-3_0_1.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chenshen03/Traffic-Sign-Detection-with-RetinaNet/f34f6194b80fcd16b12ded9ac8b17ba4fb2fbdb0/samples/group7-3_0_1.jpg


--------------------------------------------------------------------------------
/test.py:
--------------------------------------------------------------------------------
  1 | import numpy as np
  2 | import os
  3 | import cv2
  4 | from PIL import Image
  5 | from utils import box_iou, box_nms, change_box_order, meshgrid
  6 | import torch
  7 | import torch.backends.cudnn as cudnn
  8 | import torch.nn as nn
  9 | import torch.nn.functional as F
 10 | import preprocessing.transforms as transforms
 11 | from encoder import DataEncoder
 12 | from loss import *
 13 | from retinanet import RetinaNet
 14 | from preprocessing.datasets import VocLikeDataset
 15 | import matplotlib.pyplot as plt
 16 | import config as cfg
 17 | from tqdm import tqdm
 18 | from evaluate import anno_func
 19 | import json
 20 | from optparse import OptionParser
 21 | 
 22 | 
 23 | def load_model(backbone):
 24 |     print('loading model...')
 25 |     model= torch.load(os.path.join('model', 'restnet101_8K.pth'))
 26 |     net=RetinaNet(backbone=backbone,num_classes=len(cfg.classes))
 27 |     net=torch.nn.DataParallel(net, device_ids=range(torch.cuda.device_count()))
 28 |     net.cuda()
 29 |     cudnn.benchmark = True
 30 |     net.load_state_dict(model['net'])
 31 |     return net
 32 | 
 33 | def vis(img, boxes, labels, classes,color):
 34 |     img=img.copy()
 35 |     for box,label in zip(boxes,labels):
 36 |         cv2.rectangle(img, (max(0,int(box[0])),max(0,int(box[1]))),(min(511,int(box[2])),min(511,int(box[3]))),color,2)
 37 |         ss=cfg.classes[label-1]
 38 |         cv2.putText(img, ss, (int(box[0]),int(box[1]-10)), 0, 0.6, color, 2)
 39 |     return img
 40 | 
 41 | 
 42 | def eval_valid(net, valloader, anno_file,image_dir):
 43 |     net.eval()
 44 |     annos_pred={}
 45 |     annos_pred['imgs']={}
 46 |     annos=json.loads(open(anno_file).read())
 47 |     annos_target={}
 48 |     annos_target['imgs']={}
 49 |     
 50 |     for batch_idx, (inputs, loc_targets, cls_targets) in tqdm(enumerate(valloader)):
 51 |         inputs = Variable(inputs.cuda())
 52 |         loc_targets = Variable(loc_targets.cuda())
 53 |         cls_targets = Variable(cls_targets.cuda())
 54 |         loc_preds, cls_preds = net(inputs)
 55 |         for i in range(loc_preds.size()[0]):
 56 |             imgid=cfg.val_imageset_fn[batch_idx*batch_size+i].split('/')[-1][:-4]
 57 |             annos_target['imgs'][imgid]=annos['imgs'][imgid]
 58 |             boxes,labels,score=DataEncoder().decode(loc_preds[i], cls_preds[i], input_size=512)
 59 |             annos_pred['imgs'][imgid]={}
 60 |             rpath=os.path.join(image_dir,imgid+'.jpg')
 61 |             annos_pred['imgs'][imgid]['path']=rpath
 62 |             annos_pred['imgs'][imgid]['objects']=[]
 63 |             if boxes is None:
 64 |                 continue
 65 |             for i,box in enumerate(boxes):
 66 |                 # TODO solve error: tensor() is not JSON serializable
 67 |                 bbox={}
 68 |                 bbox['xmin']=float(box[0])
 69 |                 bbox['xmax']=float(box[2])
 70 |                 bbox['ymin']=float(box[1])
 71 |                 bbox['ymax']=float(box[3])
 72 | 
 73 |                 annos_pred['imgs'][imgid]['objects'].append({'score':100*float(score[i]),'bbox':bbox,'category':cfg.classes[labels[i]-1]})
 74 |     
 75 |     print('Test done, evaluating result...')
 76 |     
 77 |     
 78 |     with open(os.path.join(datadir,predict_dir),'w') as f:
 79 |         json_str=json.dumps(annos_pred)
 80 |         json.dump(annos_pred,f)
 81 |         f.close()
 82 |     with open(os.path.join(datadir,target_dir),'w') as f:
 83 |         json_str=json.dumps(annos_target)
 84 |         json.dump(annos_target,f)
 85 |         f.close()
 86 |     
 87 | def test_image(net, imgid_path,file_name):
 88 |     img=Image.open(os.path.join(imgid_path,file_name))
 89 |     width, height=img.size
 90 |     if width!=cfg.width or height!=cfg.height:
 91 |         img=cv2.resize(img,(cfg.width, cfg.height))
 92 |     img=np.asarray(img)
 93 |     image = img.transpose((2, 0, 1))
 94 |     image=torch.from_numpy(image)
 95 |     if isinstance(image, torch.ByteTensor):
 96 |         image = image.float().div(255)
 97 |     for t, m, s in zip(image, cfg.mean, cfg.std):
 98 |         t.sub_(m).div_(s)
 99 |     net.eval()
100 |     image=Variable(image.resize_(1,3,cfg.width,cfg.height))
101 |     loc_pred, cls_pred=net(image)
102 |     boxes,labels,score=DataEncoder().decode(loc_pred[0], cls_pred[0], input_size=(cfg.width,cfg.height))
103 |     if boxes is None:
104 |         new_img=img
105 |     else:
106 |         new_img=vis(img, boxes, labels, cfg.classes, (0,0,255))
107 |     return new_img
108 |     
109 |     
110 | if __name__ == '__main__':
111 |     parser = OptionParser()
112 |     parser.add_option('-m', '--mode', dest='mode',default='demo',
113 | 		help='Operating mode, could be demo or valid, demo mode will provide visulization results for images in samples/')
114 |     
115 |     parser.add_option('--backbone','--backbone',dest='backbone',default='resnet50',
116 |     help='Backbone pretrained model, could be resnet50, resnet101 or resnet152')
117 |     
118 |     options, args = parser.parse_args()
119 |     mode = options.mode
120 |     backbone=options.backbone
121 |     if backbone not in ['resnet50', 'resnet101', 'resnet152']:
122 |         assert ValueError('Invalid backbone: %s' % backbone)
123 |     net=load_model(backbone)
124 |     if mode=='valid':
125 |         datadir=cfg.root
126 |         batch_size=2
127 |         # anno_file=os.path.join(datadir,'annotation.json')
128 |         anno_file=cfg.annotation_file
129 |         target_dir='valid_target.json'
130 |         predict_dir=backbone+'_predict.json'
131 |         val_transform = transforms.Compose([
132 |         transforms.ToTensor(),transforms.Normalize(cfg.mean, cfg.std)])
133 |         valset = VocLikeDataset(image_dir=cfg.val_image_dir, annotation_file=cfg.annotation_file, imageset_fn=cfg.val_imageset_fn,
134 |                         image_ext=cfg.image_ext, classes=cfg.classes, encoder=DataEncoder(), transform=val_transform)
135 |         valloader = torch.utils.data.DataLoader(valset, batch_size=batch_size, shuffle=False,
136 |                                         num_workers=cfg.num_workers, collate_fn=valset.collate_fn)
137 |         eval_valid(net, valloader, anno_file, cfg.test_dir)
138 |         filedir=os.path.join(datadir, target_dir)
139 |         annos = json.loads(open(filedir).read())
140 |         result_anno_file=os.path.join(datadir,predict_dir)
141 |         results_annos1 = json.loads(open(result_anno_file).read())
142 |         print (len(results_annos1['imgs']))
143 |         sm = anno_func.eval_annos(annos, results_annos1, iou=0.5,types=anno_func.type5,minscore=50,check_type=True)
144 |         print sm['report']
145 | 
146 |     elif mode=='demo':
147 |         image_dir='samples'
148 |         img_list=os.listdir(image_dir)
149 |         for fname in img_list:
150 |             print(fname)
151 |             new_img=test_image(net, image_dir, fname)
152 |             plt.imsave('./result/'+fname, new_img)
153 |             # plt.imshow(new_img)
154 |             # plt.show()
155 |     else:
156 |         assert ValueError('Invalid mode: %s' % mode)
157 | 


--------------------------------------------------------------------------------
/train.py:
--------------------------------------------------------------------------------
  1 | import argparse
  2 | import os
  3 | import sys
  4 | import numpy as np
  5 | import torch
  6 | import torch.backends.cudnn as cudnn
  7 | import torch.nn as nn
  8 | import torch.nn.functional as F
  9 | import torch.optim as optim
 10 | from torch.autograd import Variable
 11 | 
 12 | import preprocessing.transforms as transforms
 13 | from encoder import DataEncoder
 14 | from loss import FocalLoss
 15 | from retinanet import RetinaNet
 16 | from preprocessing.datasets import VocLikeDataset
 17 | 
 18 | from tensorboardX import SummaryWriter
 19 | 
 20 | 
 21 | parser = argparse.ArgumentParser(description='PyTorch RetinaNet Training')
 22 | parser.add_argument('--exp', required=True, help='experiment name')
 23 | parser.add_argument('--resume', '-r', action='store_true', help='resume from checkpoint')
 24 | args = parser.parse_args()
 25 | 
 26 | #sys.path.insert(0, os.path.join('exps', 'voc'))
 27 | import config as cfg
 28 | 
 29 | assert torch.cuda.is_available(), 'Error: CUDA not found!'
 30 | best_loss = float('inf')
 31 | start_epoch = 0
 32 | lr = cfg.lr
 33 | 
 34 | print('Preparing data..')
 35 | 
 36 | train_transform_list = [transforms.ToTensor(),transforms.Normalize(cfg.mean, cfg.std)]
 37 | if cfg.scale is not None:
 38 |     train_transform_list.insert(0,transforms.Scale(cfg.scale))
 39 | train_transform = transforms.Compose(train_transform_list)
 40 | val_transform = transforms.Compose([
 41 |     transforms.ToTensor(),transforms.Normalize(cfg.mean, cfg.std)
 42 | ])
 43 | 
 44 | trainset = VocLikeDataset(image_dir=cfg.image_dir, annotation_file=cfg.annotation_file,imageset_fn=cfg.train_imageset_fn,
 45 |                           image_ext=cfg.image_ext, classes=cfg.classes, encoder=DataEncoder(), transform=train_transform)
 46 | valset = VocLikeDataset(image_dir=cfg.image_dir, annotation_file=cfg.annotation_file, imageset_fn=cfg.val_imageset_fn,
 47 |                         image_ext=cfg.image_ext, classes=cfg.classes, encoder=DataEncoder(), transform=val_transform)
 48 | 
 49 | valloader = torch.utils.data.DataLoader(valset, batch_size=4, shuffle=False,
 50 |                                         num_workers=cfg.num_workers, collate_fn=valset.collate_fn)
 51 | 
 52 | print('Building model...')
 53 | net = RetinaNet(backbone=cfg.backbone, num_classes=len(cfg.classes))
 54 | net = torch.nn.DataParallel(net, device_ids=range(torch.cuda.device_count()))
 55 | net.cuda()
 56 | cudnn.benchmark = True
 57 | 
 58 | if args.resume:
 59 |     print('Resuming from checkpoint..')
 60 |     checkpoint = torch.load(os.path.join('ckpts', args.exp, '30_ckpt.pth'))
 61 |     net.load_state_dict(checkpoint['net'])
 62 |     
 63 |     start_epoch = checkpoint['epoch']
 64 |     lr = cfg.lr
 65 | 
 66 | 
 67 | criterion = FocalLoss(len(cfg.classes))
 68 | 
 69 | optimizer = optim.Adam(net.parameters(), lr=cfg.lr,weight_decay=cfg.weight_decay)
 70 | 
 71 | writer = SummaryWriter(log_dir='logs')
 72 | # input_data = torch.rand(cfg.batch_size, 3, cfg.width, cfg.height)
 73 | # writer.add_graph(net, input_data)
 74 | 
 75 | def train(epoch):
 76 |     trainloader = torch.utils.data.DataLoader(trainset, batch_size=cfg.batch_size, shuffle=True,
 77 |                                           num_workers=cfg.num_workers, collate_fn=trainset.collate_fn)
 78 |     print('\nTrain Epoch: %d' % epoch)
 79 |     net.train()
 80 |     train_loss = 0
 81 |     loc_losses = 0
 82 |     cls_losses = 0
 83 |     
 84 |     for batch_idx, (inputs, loc_targets, cls_targets) in enumerate(trainloader):
 85 |         inputs = Variable(inputs.cuda())
 86 |         
 87 |         loc_targets = Variable(loc_targets.cuda())
 88 |         cls_targets = Variable(cls_targets.cuda())
 89 |         
 90 |         optimizer.zero_grad()
 91 |         
 92 |         loc_preds, cls_preds = net(inputs)
 93 |         
 94 |         pos = cls_targets > 0
 95 |         num_pos = pos.data.long().sum()
 96 |         if num_pos==0:
 97 |             print('zero num positive')
 98 |             continue
 99 |         loss, loc_loss, cls_loss = criterion(loc_preds, loc_targets, cls_preds, cls_targets, pos,batch_idx%2==0)
100 |         
101 |         
102 |         loss.backward()
103 |         nn.utils.clip_grad_norm(net.parameters(), max_norm=1.0)
104 |         optimizer.step()
105 | 
106 |         # TODO: solve error: invalid index of a 0-dim tensor
107 |         try:
108 |             loss_data = loss.data[0]
109 |         except:
110 |             loss_data = loss.item()
111 |     
112 |         train_loss += loss_data
113 |         loc_losses += loc_loss
114 |         cls_losses += cls_loss
115 |         
116 |         if batch_idx%2==0:
117 |             print('train_loss: %.3f | avg_loss: %.4f' % (loss_data, train_loss/(batch_idx+1)))
118 | 
119 |     writer.add_scalar("data/loc_loss", loc_losses/batch_idx, epoch)
120 |     writer.add_scalar("data/cls_loss", cls_losses/batch_idx, epoch)
121 |     writer.add_scalar("data/avg_loss", train_loss/batch_idx, epoch)
122 | 
123 |     writer.add_scalars("data/train_loss", {'loc_loss': loc_losses/batch_idx,
124 |                                            'cls_loss': cls_losses/batch_idx,
125 |                                            'avg_loss': train_loss/batch_idx}, 
126 |                                            epoch)
127 |                               
128 |     save_checkpoint(train_loss,epoch, len(trainloader))
129 |     
130 |     
131 | 
132 | def val(epoch):
133 |     net.eval()
134 |     val_loss = 0
135 |     for batch_idx, (inputs, loc_targets, cls_targets) in enumerate(valloader):
136 |         inputs = Variable(inputs.cuda())
137 |         loc_targets = Variable(loc_targets.cuda())
138 |         cls_targets = Variable(cls_targets.cuda())
139 | 
140 |         loc_preds, cls_preds = net(inputs)
141 |         pos = cls_targets > 0
142 |         
143 |         loss = criterion(loc_preds, loc_targets, cls_preds, cls_targets,pos,batch_idx%10==0)
144 |         # TODO: invalid index of a 0-dim tensor
145 |         try:
146 |             loss_data0 = loss.data[0]
147 |         except:
148 |             loss_data0 = loss.item()
149 |         val_loss += loss_data0
150 |         if batch_idx%10==0:
151 |             print('val_loss: %.4f | avg_loss: %.4f' % (loss_data0, val_loss / (batch_idx + 1)))
152 | 
153 | 
154 | def save_checkpoint(loss, epoch, n):
155 |     global best_loss
156 |     loss /= n
157 |     if loss < best_loss:
158 |         print('Saving..')
159 |         state = {
160 |             'net': net.state_dict(),
161 |             'loss': loss,
162 |             'epoch': epoch,
163 |             'lr': lr
164 |         }
165 |         ckpt_path = os.path.join('ckpts', args.exp)
166 |         if not os.path.isdir(ckpt_path):
167 |             os.makedirs(ckpt_path)
168 |         torch.save(state, os.path.join(ckpt_path, str(epoch)+'_ckpt.pth'))
169 |         best_loss = loss
170 | 
171 | for epoch in range(start_epoch + 1, start_epoch + cfg.num_epochs + 1):
172 |     if epoch in cfg.lr_decay_epochs:
173 |         lr *= 0.1
174 |         print('learning rate decay to: ', lr)
175 |         for param_group in optimizer.param_groups:
176 |             param_group['lr'] = lr
177 |     train(epoch)
178 |     if cfg.eval_while_training and epoch % cfg.eval_every == 0:
179 |         val(epoch)
180 | 
181 | writer.export_scalars_to_json('./data/all_scalars.json')
182 | writer.close()


--------------------------------------------------------------------------------
/utils.py:
--------------------------------------------------------------------------------
  1 | import os
  2 | import sys
  3 | import time
  4 | import math
  5 | 
  6 | import torch
  7 | import torch.nn as nn
  8 | import numpy as np
  9 | 
 10 | def get_mean_and_std(dataset, max_load=100000):
 11 |     '''Compute the mean and std value of dataset.'''
 12 |     mean = torch.zeros(3)
 13 |     std = torch.zeros(3)
 14 |     print('==> Computing mean and std..')
 15 |     N = min(max_load, len(dataset))
 16 |     for i in range(N):
 17 |         # im,_,_ = dataset.load(i)
 18 |         # for j in range(3):
 19 |         #     mean[j] += im[:,j,:,:].mean()
 20 |         #     std[j] += im[:,j,:,:].std()
 21 |         print(i)
 22 |         im = dataset.load(i)
 23 |         for j in range(3):
 24 |             mean[j] += im[:,:,j].mean()
 25 |             std[j] += im[:,:,j].std()
 26 |     mean.div_(N)
 27 |     std.div_(N)
 28 |     return mean, std
 29 | 
 30 | def meshgrid(x, y, swap_dims=False):
 31 |     '''Return meshgrid in range x & y.
 32 | 
 33 |     Args:
 34 |       x: (int) first dim range.
 35 |       y: (int) second dim range.
 36 |       swap_dims: (bool) swap dims.
 37 | 
 38 |     Returns:
 39 |       (tensor) meshgrid, sized [x*y,2]
 40 | 
 41 |     Example:
 42 |     >> meshgrid(3,2)
 43 |     0  0
 44 |     0  1
 45 |     1  0
 46 |     1  1
 47 |     2  0
 48 |     2  1
 49 |     [torch.FloatTensor of size 6x2]
 50 |     '''
 51 |     a = torch.arange(0,x)
 52 |     b = torch.arange(0,y)
 53 |     xx = a.view(-1,1).repeat(1,y).view(-1,1)
 54 |     yy = b.repeat(x,1).view(-1,1)
 55 |     return torch.cat([yy,xx],1) if swap_dims else torch.cat([xx,yy],1)
 56 | 
 57 | def change_box_order(boxes, order):
 58 |     assert order in ['xyxy2xywh','xywh2xyxy']
 59 |     a = boxes[:,:2]
 60 |     b = boxes[:,2:]
 61 |     if order == 'xyxy2xywh':
 62 |         return torch.cat([(a+b)/2,b-a], 1)
 63 |     return torch.cat([a-b/2,a+b/2], 1)
 64 | 
 65 | def box_iou(box1, box2, order='xyxy'):
 66 |     if order == 'xywh':
 67 |         box1 = change_box_order(box1, 'xywh2xyxy')
 68 |         box2 = change_box_order(box2, 'xywh2xyxy')
 69 | 
 70 |     N = box1.size(0)
 71 |     M = box2.size(0)
 72 | 
 73 |     lt = torch.max(
 74 |         box1[:,:2].unsqueeze(1).expand(N,M,2),  # [N,2] -> [N,1,2] -> [N,M,2]
 75 |         box2[:,:2].unsqueeze(0).expand(N,M,2),  # [M,2] -> [1,M,2] -> [N,M,2]
 76 |     )
 77 | 
 78 |     rb = torch.min(
 79 |         box1[:,2:].unsqueeze(1).expand(N,M,2),  # [N,2] -> [N,1,2] -> [N,M,2]
 80 |         box2[:,2:].unsqueeze(0).expand(N,M,2),  # [M,2] -> [1,M,2] -> [N,M,2]
 81 |     )
 82 | 
 83 |     wh = (rb-lt).clamp(min=0)  # [N,M,2]
 84 |     inter = wh[:,:,0] * wh[:,:,1]  # [N,M]
 85 | 
 86 |     area1 = (box1[:,2]-box1[:,0]) * (box1[:,3]-box1[:,1])  # [N,]
 87 |     area2 = (box2[:,2]-box2[:,0]) * (box2[:,3]-box2[:,1])  # [M,]
 88 |     area1 = area1.unsqueeze(1).expand_as(inter)  # [N,] -> [N,1] -> [N,M]
 89 |     area2 = area2.unsqueeze(0).expand_as(inter)  # [M,] -> [1,M] -> [N,M]
 90 | 
 91 |     iou = inter / (area1 + area2 - inter)
 92 |     return iou
 93 | 
 94 | def box_nms(bboxes, scores, threshold=0.5):
 95 |     x1 = bboxes[:,0]
 96 |     y1 = bboxes[:,1]
 97 |     x2 = bboxes[:,2]
 98 |     y2 = bboxes[:,3]
 99 | 
100 |     areas = (x2-x1) * (y2-y1)
101 |     _, order = scores.sort(0, descending=True)
102 | 
103 |     keep = []
104 |     while order.numel() > 0:
105 |         # TODO solve error: invalid index of a 0-dim tensor.
106 |         try:
107 |             i = order[0]
108 |         except:
109 |             i = order.item()
110 |         keep.append(i)
111 | 
112 |         if order.numel() == 1:
113 |             break
114 | 
115 |         xx1 = x1[order[1:]].clamp(min=x1[i])
116 |         yy1 = y1[order[1:]].clamp(min=y1[i])
117 |         xx2 = x2[order[1:]].clamp(max=x2[i])
118 |         yy2 = y2[order[1:]].clamp(max=y2[i])
119 | 
120 |         w = (xx2-xx1).clamp(min=0)
121 |         h = (yy2-yy1).clamp(min=0)
122 |         inter = w*h
123 |         ovr = inter / areas[order[1:]].clamp(max=areas[i])
124 |         ids = (ovr<=threshold).nonzero().squeeze()
125 |         if ids.numel() == 0:
126 |             break
127 |         order = order[ids+1]
128 |     return torch.LongTensor(keep)
129 | 


--------------------------------------------------------------------------------