├── .gitignore ├── LICENSE ├── README.md ├── images └── architecture.png ├── models └── AAM │ ├── config.yml │ └── train │ └── VoxCeleb.SpeakerVerification.VoxCeleb2.train │ ├── specs.yml │ ├── validate_equal_error_rate │ └── VoxCeleb.SpeakerVerification.VoxCeleb1_X.development │ │ └── params.yml │ └── weights │ └── 0560.pt └── reproduce.ipynb /.gitignore: -------------------------------------------------------------------------------- 1 | .ipynb_checkpoints/ 2 | models/AAM/train/VoxCeleb.SpeakerVerification.VoxCeleb2.train/validate_equal_error_rate/VoxCeleb.SpeakerVerification.VoxCeleb1_X.development/apply/ 3 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 UniversitéParis-Saclay 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # A Comparison of Metric Learning Loss Functions for End-to-End Speaker Verification 2 | 3 | This is the companion repository for the paper [A Comparison of Metric Learning Loss Functions for End-to-End Speaker Verification](https://arxiv.org/abs/2003.14021), published at the [SLSP 2020](https://irdta.eu/slsp2020/) conference. It hosts our best model trained with additive angular margin loss, and contains instructions for reproducing our results and using the model. 4 | 5 | ## Architecture 6 | 7 | ![Architecture](images/architecture.png?raw=true "Architecture") 8 | 9 | The architecture of our model consists of [SincNet](https://arxiv.org/abs/1808.00158) for feature extraction followed by [x-vector](https://www.danielpovey.com/files/2018_icassp_xvectors.pdf). 10 | 11 | ## Training 12 | 13 | You can train our model from scratch using the configuration file `config.yml` that we provide. All you need to do is run the following commands in your terminal: 14 | 15 | ```console 16 | $ export EXP=models/AAM # Replace with the new path to config.yml 17 | $ export PROTOCOL=VoxCeleb.SpeakerVerification.VoxCeleb2 18 | $ pyannote-audio emb train --parallel=10 --gpu --to=1000 $EXP $PROTOCOL 19 | ``` 20 | 21 | Note that you may need to change parameters based on your setup. 22 | 23 | ## Evaluation 24 | 25 | We provide a step-by-step guide on reproducing our equal error rates alongside their 95% confidence intervals. The guide first evaluates the pretrained model using raw cosine distances, and then improves it with adaptive s-norm score normalization. 26 | 27 | If you want to reproduce our results, check out [this notebook](https://github.com/juanmc2005/SpeakerEmbeddingLossComparison/blob/master/reproduce.ipynb) 28 | 29 | ## Fine-tuning 30 | 31 | You can fine-tune our model to your dataset with the following commands: 32 | 33 | ```console 34 | $ export WEIGHTS=models/AAM/train/VoxCeleb.SpeakerVerification.VoxCeleb2.train/weights/0560.pt 35 | $ export EXP= 36 | $ export PROTOCOL= 37 | $ pyannote-audio emb train --pretrained $WEIGHTS --gpu --to=1000 $EXP $PROTOCOL 38 | ``` 39 | 40 | ## Inference in Python 41 | 42 | The default pyannote model for speaker embedding on `torch.hub` is our AAM loss model trained on variable length audio chunks. If you want to use the model right away, you can do so easily in a Python script: 43 | 44 | ```python 45 | # load pretrained model from torch.hub 46 | import torch 47 | model = torch.hub.load('pyannote/pyannote-audio', 'emb') 48 | 49 | # extract embeddings for the whole files 50 | emb1 = model({'audio': '/path/to/file1.wav'}) 51 | emb2 = model({'audio': '/path/to/file2.wav'}) 52 | 53 | # compute distance between embeddings 54 | from scipy.spatial.distance import cdist 55 | import numpy as np 56 | distance = np.mean(cdist(emb1, emb2, metric='cosine')) 57 | ``` 58 | 59 | You can also replace the call to `torch.hub.load` with a pyannote `Pretrained` instance pointing to the model in this repo: 60 | 61 | ```python 62 | from pyannote.audio.features import Pretrained 63 | model = Pretrained( 64 | 'models/AAM/train/VoxCeleb.SpeakerVerification.VoxCeleb2.train/validate_equal_error_rate/' 65 | 'VoxCeleb.SpeakerVerification.VoxCeleb1_X.development', step=0.0333) 66 | 67 | print(f'Embeddings of {model.sliding_window.duration:g}s duration and of dimension {model.dimension:d}, ' 68 | f'extracted every {1000 * model.sliding_window.step:g}ms') 69 | ``` 70 | 71 | ## Some Compatibility Notes 72 | 73 | This project depends on the [pyannote-audio](https://github.com/pyannote/pyannote-audio) toolkit, so make sure you install it before running any code. 74 | 75 | Under normal circumstances, everything should work with the newest version of pyannote. However, given that pyannote is constantly evolving, some compatibility issues may appear. To make sure these don't happen, you can install the version at [this commit](https://github.com/pyannote/pyannote-audio/commit/562b74e4e2c7c2e97ad8eaabd1c95015c7e41e16) from the `develop` branch. 76 | 77 | ## Citation 78 | 79 | If our work has been useful to you, please cite our paper: 80 | 81 | ```bibtex 82 | @InProceedings{10.1007/978-3-030-59430-5_11, 83 | author="Coria, Juan M. 84 | and Bredin, Herv{\'e} 85 | and Ghannay, Sahar 86 | and Rosset, Sophie", 87 | editor="Espinosa-Anke, Luis 88 | and Mart{\'i}n-Vide, Carlos 89 | and Spasi{\'{c}}, Irena", 90 | title="{A Comparison of Metric Learning Loss Functions for End-To-End Speaker Verification}", 91 | booktitle="Statistical Language and Speech Processing", 92 | year="2020", 93 | publisher="Springer International Publishing", 94 | address="Cham", 95 | pages="137--148", 96 | isbn="978-3-030-59430-5" 97 | } 98 | ``` 99 | -------------------------------------------------------------------------------- /images/architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/juanmc2005/SpeakerEmbeddingLossComparison/6ab248a59673523e797950dc9c8517a5457c6cd3/images/architecture.png -------------------------------------------------------------------------------- /models/AAM/config.yml: -------------------------------------------------------------------------------- 1 | feature_extraction: 2 | name: pyannote.audio.features.RawAudio 3 | params: 4 | sample_rate: 16000 5 | 6 | data_augmentation: 7 | name: AddNoise 8 | params: 9 | snr_min: 10 10 | snr_max: 20 11 | collection: MUSAN.Collection.BackgroundNoise 12 | 13 | architecture: 14 | name: pyannote.audio.models.SincTDNN 15 | params: 16 | sincnet: 17 | stride: [5, 1, 1] 18 | waveform_normalize: True 19 | instance_normalize: True 20 | tdnn: 21 | embedding_dim: 512 22 | embedding: 23 | batch_normalize: False 24 | unit_normalize: False 25 | 26 | task: 27 | name: AdditiveAngularMarginLoss 28 | params: 29 | margin: 0.05 30 | s: 10 31 | duration: 3.0 32 | per_fold: 128 33 | per_label: 1 34 | per_epoch: 5 35 | label_min_duration: 30 36 | 37 | 38 | scheduler: 39 | name: ConstantScheduler 40 | params: 41 | learning_rate: 0.01 42 | -------------------------------------------------------------------------------- /models/AAM/train/VoxCeleb.SpeakerVerification.VoxCeleb2.train/specs.yml: -------------------------------------------------------------------------------- 1 | X: 2 | dimension: 1 3 | task: chunk-wise representation learning 4 | y: 5 | classes: 6 | - id00012 7 | - id00015 8 | - id00016 9 | - id00018 10 | - id00019 11 | - id00020 12 | - id00021 13 | - id00022 14 | - id00024 15 | - id00025 16 | - id00026 17 | - id00027 18 | - id00028 19 | - id00029 20 | - id00032 21 | - id00033 22 | - id00035 23 | - id00036 24 | - id00039 25 | - id00040 26 | - id00042 27 | - id00043 28 | - id00044 29 | - id00046 30 | - id00047 31 | - id00049 32 | - id00050 33 | - id00051 34 | - id00052 35 | - id00053 36 | - id00055 37 | - id00056 38 | - id00058 39 | - id00059 40 | - id00060 41 | - id00062 42 | - id00063 43 | - id00064 44 | - id00066 45 | - id00067 46 | - id00068 47 | - id00069 48 | - id00070 49 | - id00071 50 | - id00073 51 | - id00074 52 | - id00075 53 | - id00076 54 | - id00078 55 | - id00079 56 | - id00080 57 | - id00082 58 | - id00083 59 | - id00084 60 | - id00085 61 | - id00086 62 | - id00087 63 | - id00088 64 | - id00089 65 | - id00090 66 | - id00091 67 | - id00092 68 | - id00096 69 | - id00097 70 | - id00098 71 | - id00099 72 | - id00100 73 | - id00103 74 | - id00104 75 | - id00105 76 | - id00109 77 | - id00111 78 | - id00114 79 | - id00116 80 | - id00117 81 | - id00126 82 | - id00127 83 | - id00128 84 | - id00129 85 | - id00134 86 | - id00137 87 | - id00144 88 | - id00145 89 | - id00146 90 | - id00149 91 | - id00151 92 | - id00155 93 | - id00159 94 | - id00161 95 | - id00163 96 | - id00164 97 | - id00165 98 | - id00166 99 | - id00167 100 | - id00168 101 | - id00169 102 | - id00170 103 | - id00171 104 | - id00172 105 | - id00173 106 | - id00175 107 | - id00176 108 | - id00177 109 | - id00179 110 | - id00180 111 | - id00181 112 | - id00183 113 | - id00184 114 | - id00185 115 | - id00186 116 | - id00187 117 | - id00188 118 | - id00189 119 | - id00190 120 | - id00191 121 | - id00192 122 | - id00195 123 | - id00197 124 | - id00198 125 | - id00201 126 | - id00202 127 | - id00203 128 | - id00206 129 | - id00220 130 | - id00221 131 | - id00222 132 | - id00223 133 | - id00225 134 | - id00231 135 | - id00232 136 | - id00234 137 | - id00235 138 | - id00236 139 | - id00238 140 | - id00239 141 | - id00241 142 | - id00242 143 | - id00243 144 | - id00244 145 | - id00245 146 | - id00246 147 | - id00249 148 | - id00251 149 | - id00253 150 | - id00254 151 | - id00255 152 | - id00257 153 | - id00258 154 | - id00259 155 | - id00261 156 | - id00262 157 | - id00264 158 | - id00265 159 | - id00266 160 | - id00268 161 | - id00269 162 | - id00270 163 | - id00271 164 | - id00272 165 | - id00274 166 | - id00275 167 | - id00276 168 | - id00277 169 | - id00279 170 | - id00281 171 | - id00282 172 | - id00283 173 | - id00285 174 | - id00286 175 | - id00287 176 | - id00288 177 | - id00289 178 | - id00290 179 | - id00291 180 | - id00292 181 | - id00296 182 | - id00297 183 | - id00298 184 | - id00299 185 | - id00300 186 | - id00301 187 | - id00302 188 | - id00303 189 | - id00304 190 | - id00305 191 | - id00306 192 | - id00309 193 | - id00311 194 | - id00312 195 | - id00313 196 | - id00314 197 | - id00316 198 | - id00317 199 | - id00318 200 | - id00319 201 | - id00320 202 | - id00322 203 | - id00323 204 | - id00324 205 | - id00325 206 | - id00328 207 | - id00329 208 | - id00330 209 | - id00331 210 | - id00332 211 | - id00336 212 | - id00341 213 | - id00342 214 | - id00343 215 | - id00344 216 | - id00345 217 | - id00346 218 | - id00347 219 | - id00348 220 | - id00350 221 | - id00351 222 | - id00352 223 | - id00353 224 | - id00354 225 | - id00355 226 | - id00356 227 | - id00357 228 | - id00358 229 | - id00359 230 | - id00361 231 | - id00363 232 | - id00365 233 | - id00366 234 | - id00368 235 | - id00371 236 | - id00372 237 | - id00373 238 | - id00374 239 | - id00376 240 | - id00377 241 | - id00378 242 | - id00379 243 | - id00380 244 | - id00381 245 | - id00383 246 | - id00385 247 | - id00386 248 | - id00387 249 | - id00388 250 | - id00389 251 | - id00390 252 | - id00391 253 | - id00393 254 | - id00395 255 | - id00397 256 | - id00398 257 | - id00399 258 | - id00401 259 | - id00402 260 | - id00403 261 | - id00405 262 | - id00406 263 | - id00407 264 | - id00409 265 | - id00411 266 | - id00412 267 | - id00413 268 | - id00414 269 | - id00415 270 | - id00416 271 | - id00417 272 | - id00418 273 | - id00420 274 | - id00421 275 | - id00422 276 | - id00423 277 | - id00425 278 | - id00426 279 | - id00427 280 | - id00428 281 | - id00429 282 | - id00430 283 | - id00431 284 | - id00432 285 | - id00434 286 | - id00435 287 | - id00436 288 | - id00437 289 | - id00438 290 | - id00444 291 | - id00445 292 | - id00448 293 | - id00449 294 | - id00450 295 | - id00456 296 | - id00457 297 | - id00458 298 | - id00459 299 | - id00460 300 | - id00461 301 | - id00462 302 | - id00463 303 | - id00464 304 | - id00465 305 | - id00467 306 | - id00468 307 | - id00469 308 | - id00472 309 | - id00473 310 | - id00475 311 | - id00476 312 | - id00478 313 | - id00479 314 | - id00481 315 | - id00482 316 | - id00483 317 | - id00484 318 | - id00486 319 | - id00488 320 | - id00489 321 | - id00490 322 | - id00491 323 | - id00492 324 | - id00494 325 | - id00495 326 | - id00496 327 | - id00498 328 | - id00500 329 | - id00501 330 | - id00502 331 | - id00503 332 | - id00504 333 | - id00505 334 | - id00506 335 | - id00507 336 | - id00508 337 | - id00509 338 | - id00510 339 | - id00512 340 | - id00513 341 | - id00514 342 | - id00515 343 | - id00516 344 | - id00517 345 | - id00518 346 | - id00519 347 | - id00520 348 | - id00521 349 | - id00522 350 | - id00524 351 | - id00525 352 | - id00526 353 | - id00527 354 | - id00528 355 | - id00529 356 | - id00530 357 | - id00531 358 | - id00534 359 | - id00535 360 | - id00539 361 | - id00542 362 | - id00543 363 | - id00544 364 | - id00545 365 | - id00546 366 | - id00548 367 | - id00550 368 | - id00552 369 | - id00553 370 | - id00554 371 | - id00555 372 | - id00557 373 | - id00558 374 | - id00559 375 | - id00560 376 | - id00561 377 | - id00566 378 | - id00567 379 | - id00568 380 | - id00569 381 | - id00570 382 | - id00571 383 | - id00573 384 | - id00574 385 | - id00575 386 | - id00577 387 | - id00578 388 | - id00579 389 | - id00581 390 | - id00582 391 | - id00584 392 | - id00585 393 | - id00586 394 | - id00587 395 | - id00588 396 | - id00589 397 | - id00591 398 | - id00592 399 | - id00593 400 | - id00594 401 | - id00596 402 | - id00597 403 | - id00607 404 | - id00608 405 | - id00610 406 | - id00611 407 | - id00612 408 | - id00613 409 | - id00614 410 | - id00615 411 | - id00616 412 | - id00617 413 | - id00618 414 | - id00620 415 | - id00621 416 | - id00622 417 | - id00623 418 | - id00624 419 | - id00628 420 | - id00629 421 | - id00631 422 | - id00632 423 | - id00633 424 | - id00634 425 | - id00636 426 | - id00637 427 | - id00638 428 | - id00641 429 | - id00642 430 | - id00643 431 | - id00644 432 | - id00645 433 | - id00647 434 | - id00648 435 | - id00649 436 | - id00650 437 | - id00651 438 | - id00652 439 | - id00653 440 | - id00654 441 | - id00655 442 | - id00656 443 | - id00657 444 | - id00659 445 | - id00663 446 | - id00664 447 | - id00665 448 | - id00666 449 | - id00667 450 | - id00668 451 | - id00669 452 | - id00670 453 | - id00672 454 | - id00673 455 | - id00674 456 | - id00675 457 | - id00676 458 | - id00677 459 | - id00678 460 | - id00680 461 | - id00681 462 | - id00685 463 | - id00686 464 | - id00687 465 | - id00690 466 | - id00691 467 | - id00692 468 | - id00693 469 | - id00694 470 | - id00695 471 | - id00696 472 | - id00699 473 | - id00700 474 | - id00701 475 | - id00707 476 | - id00708 477 | - id00709 478 | - id00710 479 | - id00711 480 | - id00712 481 | - id00714 482 | - id00715 483 | - id00717 484 | - id00721 485 | - id00723 486 | - id00724 487 | - id00725 488 | - id00726 489 | - id00727 490 | - id00729 491 | - id00730 492 | - id00731 493 | - id00732 494 | - id00735 495 | - id00736 496 | - id00739 497 | - id00740 498 | - id00741 499 | - id00742 500 | - id00745 501 | - id00746 502 | - id00747 503 | - id00750 504 | - id00752 505 | - id00753 506 | - id00754 507 | - id00755 508 | - id00758 509 | - id00759 510 | - id00760 511 | - id00761 512 | - id00762 513 | - id00763 514 | - id00765 515 | - id00766 516 | - id00769 517 | - id00770 518 | - id00773 519 | - id00774 520 | - id00775 521 | - id00776 522 | - id00777 523 | - id00778 524 | - id00779 525 | - id00780 526 | - id00781 527 | - id00782 528 | - id00783 529 | - id00784 530 | - id00785 531 | - id00786 532 | - id00787 533 | - id00788 534 | - id00792 535 | - id00794 536 | - id00796 537 | - id00797 538 | - id00799 539 | - id00801 540 | - id00802 541 | - id00803 542 | - id00804 543 | - id00805 544 | - id00806 545 | - id00809 546 | - id00810 547 | - id00811 548 | - id00813 549 | - id00814 550 | - id00815 551 | - id00816 552 | - id00818 553 | - id00820 554 | - id00823 555 | - id00824 556 | - id00825 557 | - id00826 558 | - id00827 559 | - id00828 560 | - id00829 561 | - id00830 562 | - id00831 563 | - id00832 564 | - id00833 565 | - id00835 566 | - id00841 567 | - id00842 568 | - id00843 569 | - id00844 570 | - id00846 571 | - id00848 572 | - id00851 573 | - id00852 574 | - id00857 575 | - id00859 576 | - id00860 577 | - id00863 578 | - id00865 579 | - id00867 580 | - id00868 581 | - id00869 582 | - id00875 583 | - id00876 584 | - id00879 585 | - id00880 586 | - id00881 587 | - id00882 588 | - id00883 589 | - id00884 590 | - id00885 591 | - id00886 592 | - id00888 593 | - id00890 594 | - id00891 595 | - id00892 596 | - id00893 597 | - id00894 598 | - id00895 599 | - id00897 600 | - id00898 601 | - id00899 602 | - id00901 603 | - id00902 604 | - id00903 605 | - id00904 606 | - id00905 607 | - id00906 608 | - id00907 609 | - id00908 610 | - id00909 611 | - id00910 612 | - id00911 613 | - id00917 614 | - id00918 615 | - id00919 616 | - id00920 617 | - id00922 618 | - id00923 619 | - id00924 620 | - id00925 621 | - id00927 622 | - id00928 623 | - id00929 624 | - id00930 625 | - id00931 626 | - id00933 627 | - id00934 628 | - id00935 629 | - id00937 630 | - id00938 631 | - id00939 632 | - id00941 633 | - id00942 634 | - id00943 635 | - id00944 636 | - id00945 637 | - id00946 638 | - id00947 639 | - id00948 640 | - id00959 641 | - id00960 642 | - id00963 643 | - id00964 644 | - id00965 645 | - id00966 646 | - id00967 647 | - id00969 648 | - id00971 649 | - id00974 650 | - id00975 651 | - id00976 652 | - id00978 653 | - id00979 654 | - id00980 655 | - id00981 656 | - id00982 657 | - id00983 658 | - id00984 659 | - id00985 660 | - id00987 661 | - id00989 662 | - id00990 663 | - id00991 664 | - id00992 665 | - id00993 666 | - id00994 667 | - id00995 668 | - id00996 669 | - id00997 670 | - id00999 671 | - id01001 672 | - id01002 673 | - id01003 674 | - id01004 675 | - id01005 676 | - id01006 677 | - id01007 678 | - id01008 679 | - id01010 680 | - id01011 681 | - id01014 682 | - id01015 683 | - id01016 684 | - id01017 685 | - id01018 686 | - id01021 687 | - id01023 688 | - id01024 689 | - id01025 690 | - id01026 691 | - id01028 692 | - id01029 693 | - id01030 694 | - id01031 695 | - id01035 696 | - id01036 697 | - id01037 698 | - id01038 699 | - id01039 700 | - id01040 701 | - id01042 702 | - id01043 703 | - id01044 704 | - id01045 705 | - id01047 706 | - id01048 707 | - id01049 708 | - id01050 709 | - id01051 710 | - id01052 711 | - id01053 712 | - id01054 713 | - id01056 714 | - id01057 715 | - id01058 716 | - id01060 717 | - id01061 718 | - id01062 719 | - id01063 720 | - id01065 721 | - id01067 722 | - id01068 723 | - id01069 724 | - id01070 725 | - id01071 726 | - id01073 727 | - id01074 728 | - id01075 729 | - id01076 730 | - id01078 731 | - id01080 732 | - id01081 733 | - id01083 734 | - id01084 735 | - id01085 736 | - id01090 737 | - id01091 738 | - id01092 739 | - id01093 740 | - id01094 741 | - id01095 742 | - id01096 743 | - id01097 744 | - id01098 745 | - id01099 746 | - id01100 747 | - id01101 748 | - id01102 749 | - id01103 750 | - id01104 751 | - id01105 752 | - id01108 753 | - id01109 754 | - id01110 755 | - id01111 756 | - id01112 757 | - id01113 758 | - id01115 759 | - id01116 760 | - id01117 761 | - id01119 762 | - id01120 763 | - id01121 764 | - id01122 765 | - id01123 766 | - id01124 767 | - id01125 768 | - id01126 769 | - id01129 770 | - id01130 771 | - id01132 772 | - id01139 773 | - id01140 774 | - id01141 775 | - id01142 776 | - id01149 777 | - id01150 778 | - id01154 779 | - id01155 780 | - id01156 781 | - id01157 782 | - id01158 783 | - id01159 784 | - id01160 785 | - id01161 786 | - id01162 787 | - id01163 788 | - id01164 789 | - id01165 790 | - id01166 791 | - id01167 792 | - id01168 793 | - id01169 794 | - id01170 795 | - id01171 796 | - id01172 797 | - id01173 798 | - id01174 799 | - id01175 800 | - id01178 801 | - id01179 802 | - id01182 803 | - id01183 804 | - id01184 805 | - id01185 806 | - id01187 807 | - id01188 808 | - id01189 809 | - id01190 810 | - id01191 811 | - id01192 812 | - id01193 813 | - id01196 814 | - id01199 815 | - id01200 816 | - id01201 817 | - id01203 818 | - id01204 819 | - id01205 820 | - id01206 821 | - id01207 822 | - id01209 823 | - id01210 824 | - id01211 825 | - id01212 826 | - id01213 827 | - id01214 828 | - id01215 829 | - id01216 830 | - id01217 831 | - id01218 832 | - id01220 833 | - id01221 834 | - id01222 835 | - id01223 836 | - id01225 837 | - id01227 838 | - id01229 839 | - id01230 840 | - id01231 841 | - id01232 842 | - id01234 843 | - id01235 844 | - id01236 845 | - id01237 846 | - id01238 847 | - id01239 848 | - id01240 849 | - id01241 850 | - id01243 851 | - id01244 852 | - id01245 853 | - id01246 854 | - id01248 855 | - id01249 856 | - id01251 857 | - id01252 858 | - id01253 859 | - id01254 860 | - id01255 861 | - id01256 862 | - id01257 863 | - id01258 864 | - id01261 865 | - id01262 866 | - id01263 867 | - id01264 868 | - id01265 869 | - id01266 870 | - id01268 871 | - id01269 872 | - id01270 873 | - id01271 874 | - id01272 875 | - id01273 876 | - id01274 877 | - id01275 878 | - id01276 879 | - id01279 880 | - id01280 881 | - id01281 882 | - id01284 883 | - id01285 884 | - id01286 885 | - id01288 886 | - id01289 887 | - id01290 888 | - id01291 889 | - id01292 890 | - id01293 891 | - id01294 892 | - id01295 893 | - id01296 894 | - id01300 895 | - id01301 896 | - id01303 897 | - id01304 898 | - id01305 899 | - id01306 900 | - id01309 901 | - id01313 902 | - id01314 903 | - id01315 904 | - id01316 905 | - id01317 906 | - id01323 907 | - id01325 908 | - id01326 909 | - id01327 910 | - id01328 911 | - id01329 912 | - id01330 913 | - id01331 914 | - id01332 915 | - id01334 916 | - id01335 917 | - id01337 918 | - id01338 919 | - id01339 920 | - id01340 921 | - id01341 922 | - id01342 923 | - id01344 924 | - id01346 925 | - id01347 926 | - id01348 927 | - id01349 928 | - id01350 929 | - id01351 930 | - id01352 931 | - id01353 932 | - id01354 933 | - id01355 934 | - id01356 935 | - id01358 936 | - id01359 937 | - id01361 938 | - id01362 939 | - id01363 940 | - id01364 941 | - id01365 942 | - id01367 943 | - id01368 944 | - id01369 945 | - id01370 946 | - id01371 947 | - id01372 948 | - id01373 949 | - id01374 950 | - id01376 951 | - id01379 952 | - id01380 953 | - id01384 954 | - id01385 955 | - id01386 956 | - id01387 957 | - id01388 958 | - id01389 959 | - id01391 960 | - id01392 961 | - id01394 962 | - id01395 963 | - id01396 964 | - id01397 965 | - id01399 966 | - id01402 967 | - id01403 968 | - id01404 969 | - id01405 970 | - id01406 971 | - id01407 972 | - id01408 973 | - id01409 974 | - id01410 975 | - id01411 976 | - id01413 977 | - id01418 978 | - id01420 979 | - id01422 980 | - id01423 981 | - id01424 982 | - id01425 983 | - id01426 984 | - id01429 985 | - id01430 986 | - id01431 987 | - id01432 988 | - id01433 989 | - id01434 990 | - id01436 991 | - id01438 992 | - id01439 993 | - id01445 994 | - id01447 995 | - id01448 996 | - id01449 997 | - id01450 998 | - id01451 999 | - id01452 1000 | - id01455 1001 | - id01456 1002 | - id01457 1003 | - id01458 1004 | - id01459 1005 | - id01461 1006 | - id01464 1007 | - id01466 1008 | - id01468 1009 | - id01470 1010 | - id01471 1011 | - id01476 1012 | - id01487 1013 | - id01488 1014 | - id01489 1015 | - id01490 1016 | - id01491 1017 | - id01492 1018 | - id01493 1019 | - id01497 1020 | - id01498 1021 | - id01499 1022 | - id01500 1023 | - id01501 1024 | - id01502 1025 | - id01503 1026 | - id01504 1027 | - id01505 1028 | - id01508 1029 | - id01510 1030 | - id01511 1031 | - id01512 1032 | - id01513 1033 | - id01514 1034 | - id01515 1035 | - id01516 1036 | - id01521 1037 | - id01522 1038 | - id01523 1039 | - id01524 1040 | - id01527 1041 | - id01528 1042 | - id01529 1043 | - id01530 1044 | - id01531 1045 | - id01532 1046 | - id01533 1047 | - id01534 1048 | - id01535 1049 | - id01536 1050 | - id01539 1051 | - id01540 1052 | - id01542 1053 | - id01543 1054 | - id01544 1055 | - id01545 1056 | - id01546 1057 | - id01547 1058 | - id01548 1059 | - id01549 1060 | - id01550 1061 | - id01551 1062 | - id01552 1063 | - id01554 1064 | - id01555 1065 | - id01556 1066 | - id01557 1067 | - id01558 1068 | - id01560 1069 | - id01561 1070 | - id01564 1071 | - id01566 1072 | - id01568 1073 | - id01569 1074 | - id01570 1075 | - id01571 1076 | - id01572 1077 | - id01574 1078 | - id01578 1079 | - id01579 1080 | - id01580 1081 | - id01581 1082 | - id01582 1083 | - id01583 1084 | - id01584 1085 | - id01585 1086 | - id01589 1087 | - id01590 1088 | - id01591 1089 | - id01592 1090 | - id01594 1091 | - id01595 1092 | - id01596 1093 | - id01597 1094 | - id01598 1095 | - id01599 1096 | - id01605 1097 | - id01608 1098 | - id01609 1099 | - id01610 1100 | - id01611 1101 | - id01613 1102 | - id01614 1103 | - id01615 1104 | - id01617 1105 | - id01619 1106 | - id01620 1107 | - id01624 1108 | - id01625 1109 | - id01626 1110 | - id01627 1111 | - id01628 1112 | - id01629 1113 | - id01630 1114 | - id01633 1115 | - id01634 1116 | - id01637 1117 | - id01639 1118 | - id01640 1119 | - id01641 1120 | - id01643 1121 | - id01646 1122 | - id01648 1123 | - id01649 1124 | - id01650 1125 | - id01652 1126 | - id01653 1127 | - id01654 1128 | - id01655 1129 | - id01656 1130 | - id01657 1131 | - id01658 1132 | - id01659 1133 | - id01660 1134 | - id01661 1135 | - id01662 1136 | - id01663 1137 | - id01664 1138 | - id01666 1139 | - id01667 1140 | - id01668 1141 | - id01669 1142 | - id01671 1143 | - id01672 1144 | - id01674 1145 | - id01676 1146 | - id01677 1147 | - id01678 1148 | - id01680 1149 | - id01681 1150 | - id01682 1151 | - id01683 1152 | - id01684 1153 | - id01685 1154 | - id01686 1155 | - id01687 1156 | - id01688 1157 | - id01689 1158 | - id01690 1159 | - id01691 1160 | - id01692 1161 | - id01693 1162 | - id01694 1163 | - id01695 1164 | - id01696 1165 | - id01697 1166 | - id01699 1167 | - id01708 1168 | - id01710 1169 | - id01711 1170 | - id01713 1171 | - id01716 1172 | - id01717 1173 | - id01718 1174 | - id01719 1175 | - id01720 1176 | - id01721 1177 | - id01722 1178 | - id01723 1179 | - id01725 1180 | - id01726 1181 | - id01727 1182 | - id01728 1183 | - id01729 1184 | - id01733 1185 | - id01734 1186 | - id01735 1187 | - id01736 1188 | - id01738 1189 | - id01739 1190 | - id01741 1191 | - id01742 1192 | - id01743 1193 | - id01745 1194 | - id01746 1195 | - id01748 1196 | - id01749 1197 | - id01750 1198 | - id01751 1199 | - id01753 1200 | - id01754 1201 | - id01755 1202 | - id01756 1203 | - id01758 1204 | - id01759 1205 | - id01760 1206 | - id01761 1207 | - id01762 1208 | - id01763 1209 | - id01764 1210 | - id01765 1211 | - id01766 1212 | - id01767 1213 | - id01768 1214 | - id01769 1215 | - id01770 1216 | - id01772 1217 | - id01773 1218 | - id01774 1219 | - id01775 1220 | - id01776 1221 | - id01777 1222 | - id01778 1223 | - id01779 1224 | - id01780 1225 | - id01781 1226 | - id01782 1227 | - id01783 1228 | - id01785 1229 | - id01786 1230 | - id01787 1231 | - id01788 1232 | - id01792 1233 | - id01793 1234 | - id01794 1235 | - id01795 1236 | - id01796 1237 | - id01797 1238 | - id01798 1239 | - id01799 1240 | - id01800 1241 | - id01801 1242 | - id01802 1243 | - id01803 1244 | - id01805 1245 | - id01806 1246 | - id01808 1247 | - id01809 1248 | - id01821 1249 | - id01823 1250 | - id01826 1251 | - id01827 1252 | - id01829 1253 | - id01830 1254 | - id01832 1255 | - id01833 1256 | - id01835 1257 | - id01836 1258 | - id01837 1259 | - id01838 1260 | - id01840 1261 | - id01841 1262 | - id01842 1263 | - id01848 1264 | - id01849 1265 | - id01852 1266 | - id01853 1267 | - id01854 1268 | - id01856 1269 | - id01857 1270 | - id01858 1271 | - id01859 1272 | - id01861 1273 | - id01862 1274 | - id01864 1275 | - id01866 1276 | - id01868 1277 | - id01869 1278 | - id01870 1279 | - id01873 1280 | - id01874 1281 | - id01875 1282 | - id01876 1283 | - id01877 1284 | - id01878 1285 | - id01879 1286 | - id01880 1287 | - id01881 1288 | - id01883 1289 | - id01885 1290 | - id01886 1291 | - id01887 1292 | - id01888 1293 | - id01889 1294 | - id01890 1295 | - id01891 1296 | - id01893 1297 | - id01894 1298 | - id01895 1299 | - id01898 1300 | - id01899 1301 | - id01900 1302 | - id01904 1303 | - id01905 1304 | - id01906 1305 | - id01907 1306 | - id01908 1307 | - id01909 1308 | - id01910 1309 | - id01912 1310 | - id01913 1311 | - id01915 1312 | - id01916 1313 | - id01918 1314 | - id01919 1315 | - id01920 1316 | - id01921 1317 | - id01925 1318 | - id01927 1319 | - id01928 1320 | - id01931 1321 | - id01932 1322 | - id01933 1323 | - id01934 1324 | - id01937 1325 | - id01939 1326 | - id01941 1327 | - id01945 1328 | - id01946 1329 | - id01947 1330 | - id01948 1331 | - id01949 1332 | - id01951 1333 | - id01953 1334 | - id01955 1335 | - id01959 1336 | - id01960 1337 | - id01961 1338 | - id01962 1339 | - id01963 1340 | - id01964 1341 | - id01965 1342 | - id01966 1343 | - id01967 1344 | - id01969 1345 | - id01971 1346 | - id01972 1347 | - id01973 1348 | - id01974 1349 | - id01975 1350 | - id01976 1351 | - id01979 1352 | - id01981 1353 | - id01982 1354 | - id01985 1355 | - id01986 1356 | - id01987 1357 | - id01988 1358 | - id01990 1359 | - id01992 1360 | - id01993 1361 | - id01994 1362 | - id01995 1363 | - id01996 1364 | - id01997 1365 | - id01999 1366 | - id02000 1367 | - id02001 1368 | - id02004 1369 | - id02005 1370 | - id02006 1371 | - id02007 1372 | - id02008 1373 | - id02009 1374 | - id02010 1375 | - id02011 1376 | - id02013 1377 | - id02014 1378 | - id02015 1379 | - id02016 1380 | - id02017 1381 | - id02018 1382 | - id02020 1383 | - id02021 1384 | - id02022 1385 | - id02023 1386 | - id02024 1387 | - id02030 1388 | - id02032 1389 | - id02033 1390 | - id02034 1391 | - id02035 1392 | - id02036 1393 | - id02037 1394 | - id02038 1395 | - id02039 1396 | - id02040 1397 | - id02041 1398 | - id02043 1399 | - id02045 1400 | - id02047 1401 | - id02048 1402 | - id02049 1403 | - id02051 1404 | - id02052 1405 | - id02053 1406 | - id02055 1407 | - id02058 1408 | - id02059 1409 | - id02060 1410 | - id02061 1411 | - id02062 1412 | - id02067 1413 | - id02068 1414 | - id02070 1415 | - id02071 1416 | - id02072 1417 | - id02073 1418 | - id02074 1419 | - id02075 1420 | - id02077 1421 | - id02078 1422 | - id02079 1423 | - id02080 1424 | - id02081 1425 | - id02083 1426 | - id02084 1427 | - id02085 1428 | - id02087 1429 | - id02088 1430 | - id02089 1431 | - id02090 1432 | - id02091 1433 | - id02092 1434 | - id02093 1435 | - id02094 1436 | - id02095 1437 | - id02096 1438 | - id02097 1439 | - id02098 1440 | - id02099 1441 | - id02100 1442 | - id02101 1443 | - id02102 1444 | - id02103 1445 | - id02104 1446 | - id02105 1447 | - id02106 1448 | - id02108 1449 | - id02109 1450 | - id02110 1451 | - id02111 1452 | - id02112 1453 | - id02113 1454 | - id02114 1455 | - id02115 1456 | - id02116 1457 | - id02119 1458 | - id02120 1459 | - id02121 1460 | - id02122 1461 | - id02123 1462 | - id02124 1463 | - id02133 1464 | - id02134 1465 | - id02139 1466 | - id02140 1467 | - id02141 1468 | - id02147 1469 | - id02148 1470 | - id02149 1471 | - id02150 1472 | - id02151 1473 | - id02152 1474 | - id02153 1475 | - id02155 1476 | - id02156 1477 | - id02157 1478 | - id02158 1479 | - id02159 1480 | - id02160 1481 | - id02162 1482 | - id02163 1483 | - id02164 1484 | - id02166 1485 | - id02168 1486 | - id02172 1487 | - id02173 1488 | - id02174 1489 | - id02175 1490 | - id02176 1491 | - id02177 1492 | - id02178 1493 | - id02179 1494 | - id02180 1495 | - id02183 1496 | - id02185 1497 | - id02186 1498 | - id02187 1499 | - id02188 1500 | - id02189 1501 | - id02191 1502 | - id02193 1503 | - id02194 1504 | - id02196 1505 | - id02197 1506 | - id02201 1507 | - id02202 1508 | - id02203 1509 | - id02204 1510 | - id02205 1511 | - id02206 1512 | - id02208 1513 | - id02209 1514 | - id02210 1515 | - id02211 1516 | - id02212 1517 | - id02213 1518 | - id02214 1519 | - id02215 1520 | - id02216 1521 | - id02219 1522 | - id02220 1523 | - id02222 1524 | - id02223 1525 | - id02226 1526 | - id02228 1527 | - id02229 1528 | - id02230 1529 | - id02231 1530 | - id02233 1531 | - id02234 1532 | - id02235 1533 | - id02236 1534 | - id02239 1535 | - id02240 1536 | - id02242 1537 | - id02243 1538 | - id02245 1539 | - id02247 1540 | - id02249 1541 | - id02252 1542 | - id02253 1543 | - id02254 1544 | - id02255 1545 | - id02257 1546 | - id02258 1547 | - id02260 1548 | - id02262 1549 | - id02263 1550 | - id02264 1551 | - id02266 1552 | - id02267 1553 | - id02268 1554 | - id02270 1555 | - id02271 1556 | - id02273 1557 | - id02275 1558 | - id02276 1559 | - id02281 1560 | - id02282 1561 | - id02287 1562 | - id02288 1563 | - id02289 1564 | - id02291 1565 | - id02292 1566 | - id02293 1567 | - id02294 1568 | - id02295 1569 | - id02296 1570 | - id02297 1571 | - id02298 1572 | - id02299 1573 | - id02300 1574 | - id02301 1575 | - id02302 1576 | - id02303 1577 | - id02304 1578 | - id02306 1579 | - id02310 1580 | - id02311 1581 | - id02312 1582 | - id02314 1583 | - id02315 1584 | - id02316 1585 | - id02318 1586 | - id02322 1587 | - id02323 1588 | - id02325 1589 | - id02327 1590 | - id02329 1591 | - id02331 1592 | - id02332 1593 | - id02338 1594 | - id02339 1595 | - id02340 1596 | - id02341 1597 | - id02342 1598 | - id02343 1599 | - id02344 1600 | - id02346 1601 | - id02348 1602 | - id02349 1603 | - id02350 1604 | - id02351 1605 | - id02353 1606 | - id02355 1607 | - id02356 1608 | - id02357 1609 | - id02358 1610 | - id02359 1611 | - id02361 1612 | - id02362 1613 | - id02363 1614 | - id02364 1615 | - id02365 1616 | - id02367 1617 | - id02368 1618 | - id02381 1619 | - id02384 1620 | - id02388 1621 | - id02389 1622 | - id02390 1623 | - id02391 1624 | - id02392 1625 | - id02395 1626 | - id02409 1627 | - id02410 1628 | - id02411 1629 | - id02413 1630 | - id02414 1631 | - id02416 1632 | - id02418 1633 | - id02420 1634 | - id02421 1635 | - id02426 1636 | - id02432 1637 | - id02436 1638 | - id02437 1639 | - id02438 1640 | - id02439 1641 | - id02440 1642 | - id02441 1643 | - id02444 1644 | - id02446 1645 | - id02448 1646 | - id02449 1647 | - id02450 1648 | - id02452 1649 | - id02453 1650 | - id02455 1651 | - id02457 1652 | - id02458 1653 | - id02459 1654 | - id02460 1655 | - id02461 1656 | - id02462 1657 | - id02463 1658 | - id02464 1659 | - id02466 1660 | - id02467 1661 | - id02469 1662 | - id02471 1663 | - id02472 1664 | - id02473 1665 | - id02475 1666 | - id02477 1667 | - id02479 1668 | - id02482 1669 | - id02484 1670 | - id02485 1671 | - id02486 1672 | - id02488 1673 | - id02489 1674 | - id02490 1675 | - id02493 1676 | - id02494 1677 | - id02495 1678 | - id02496 1679 | - id02497 1680 | - id02499 1681 | - id02501 1682 | - id02502 1683 | - id02505 1684 | - id02508 1685 | - id02509 1686 | - id02511 1687 | - id02513 1688 | - id02515 1689 | - id02530 1690 | - id02531 1691 | - id02532 1692 | - id02534 1693 | - id02535 1694 | - id02537 1695 | - id02538 1696 | - id02539 1697 | - id02540 1698 | - id02544 1699 | - id02545 1700 | - id02546 1701 | - id02547 1702 | - id02553 1703 | - id02554 1704 | - id02555 1705 | - id02558 1706 | - id02561 1707 | - id02562 1708 | - id02563 1709 | - id02565 1710 | - id02566 1711 | - id02567 1712 | - id02568 1713 | - id02569 1714 | - id02570 1715 | - id02572 1716 | - id02573 1717 | - id02574 1718 | - id02575 1719 | - id02578 1720 | - id02579 1721 | - id02580 1722 | - id02583 1723 | - id02584 1724 | - id02585 1725 | - id02586 1726 | - id02587 1727 | - id02590 1728 | - id02591 1729 | - id02592 1730 | - id02593 1731 | - id02596 1732 | - id02597 1733 | - id02598 1734 | - id02599 1735 | - id02607 1736 | - id02608 1737 | - id02609 1738 | - id02613 1739 | - id02615 1740 | - id02617 1741 | - id02619 1742 | - id02622 1743 | - id02623 1744 | - id02625 1745 | - id02627 1746 | - id02628 1747 | - id02629 1748 | - id02631 1749 | - id02633 1750 | - id02634 1751 | - id02636 1752 | - id02638 1753 | - id02639 1754 | - id02640 1755 | - id02641 1756 | - id02642 1757 | - id02643 1758 | - id02647 1759 | - id02650 1760 | - id02653 1761 | - id02654 1762 | - id02655 1763 | - id02656 1764 | - id02658 1765 | - id02659 1766 | - id02660 1767 | - id02661 1768 | - id02662 1769 | - id02665 1770 | - id02666 1771 | - id02667 1772 | - id02668 1773 | - id02669 1774 | - id02672 1775 | - id02673 1776 | - id02680 1777 | - id02683 1778 | - id02684 1779 | - id02687 1780 | - id02693 1781 | - id02695 1782 | - id02696 1783 | - id02698 1784 | - id02700 1785 | - id02702 1786 | - id02703 1787 | - id02705 1788 | - id02706 1789 | - id02707 1790 | - id02710 1791 | - id02711 1792 | - id02713 1793 | - id02714 1794 | - id02715 1795 | - id02719 1796 | - id02720 1797 | - id02721 1798 | - id02722 1799 | - id02724 1800 | - id02728 1801 | - id02729 1802 | - id02730 1803 | - id02731 1804 | - id02733 1805 | - id02734 1806 | - id02735 1807 | - id02736 1808 | - id02738 1809 | - id02739 1810 | - id02740 1811 | - id02746 1812 | - id02747 1813 | - id02751 1814 | - id02759 1815 | - id02760 1816 | - id02761 1817 | - id02764 1818 | - id02765 1819 | - id02766 1820 | - id02780 1821 | - id02781 1822 | - id02783 1823 | - id02784 1824 | - id02786 1825 | - id02787 1826 | - id02790 1827 | - id02791 1828 | - id02793 1829 | - id02795 1830 | - id02805 1831 | - id02806 1832 | - id02807 1833 | - id02808 1834 | - id02809 1835 | - id02810 1836 | - id02812 1837 | - id02815 1838 | - id02816 1839 | - id02817 1840 | - id02818 1841 | - id02820 1842 | - id02821 1843 | - id02823 1844 | - id02824 1845 | - id02825 1846 | - id02827 1847 | - id02829 1848 | - id02830 1849 | - id02832 1850 | - id02833 1851 | - id02836 1852 | - id02837 1853 | - id02838 1854 | - id02839 1855 | - id02840 1856 | - id02844 1857 | - id02845 1858 | - id02846 1859 | - id02847 1860 | - id02848 1861 | - id02851 1862 | - id02853 1863 | - id02854 1864 | - id02857 1865 | - id02858 1866 | - id02863 1867 | - id02864 1868 | - id02865 1869 | - id02866 1870 | - id02869 1871 | - id02871 1872 | - id02872 1873 | - id02873 1874 | - id02874 1875 | - id02875 1876 | - id02876 1877 | - id02877 1878 | - id02878 1879 | - id02879 1880 | - id02880 1881 | - id02881 1882 | - id02882 1883 | - id02883 1884 | - id02884 1885 | - id02886 1886 | - id02887 1887 | - id02888 1888 | - id02889 1889 | - id02890 1890 | - id02891 1891 | - id02892 1892 | - id02893 1893 | - id02896 1894 | - id02897 1895 | - id02898 1896 | - id02899 1897 | - id02900 1898 | - id02901 1899 | - id02902 1900 | - id02903 1901 | - id02904 1902 | - id02905 1903 | - id02908 1904 | - id02909 1905 | - id02911 1906 | - id02912 1907 | - id02914 1908 | - id02916 1909 | - id02917 1910 | - id02918 1911 | - id02920 1912 | - id02924 1913 | - id02926 1914 | - id02928 1915 | - id02935 1916 | - id02936 1917 | - id02939 1918 | - id02940 1919 | - id02942 1920 | - id02943 1921 | - id02944 1922 | - id02945 1923 | - id02946 1924 | - id02948 1925 | - id02950 1926 | - id02952 1927 | - id02953 1928 | - id02954 1929 | - id02955 1930 | - id02956 1931 | - id02957 1932 | - id02958 1933 | - id02959 1934 | - id02961 1935 | - id02963 1936 | - id02964 1937 | - id02965 1938 | - id02978 1939 | - id02979 1940 | - id02980 1941 | - id02981 1942 | - id02982 1943 | - id02983 1944 | - id02984 1945 | - id02986 1946 | - id02987 1947 | - id02988 1948 | - id02989 1949 | - id02991 1950 | - id02993 1951 | - id02995 1952 | - id02996 1953 | - id02998 1954 | - id02999 1955 | - id03000 1956 | - id03001 1957 | - id03002 1958 | - id03003 1959 | - id03004 1960 | - id03005 1961 | - id03006 1962 | - id03007 1963 | - id03009 1964 | - id03013 1965 | - id03014 1966 | - id03015 1967 | - id03016 1968 | - id03017 1969 | - id03018 1970 | - id03020 1971 | - id03026 1972 | - id03027 1973 | - id03028 1974 | - id03029 1975 | - id03031 1976 | - id03037 1977 | - id03038 1978 | - id03039 1979 | - id03040 1980 | - id03042 1981 | - id03043 1982 | - id03056 1983 | - id03057 1984 | - id03059 1985 | - id03060 1986 | - id03064 1987 | - id03066 1988 | - id03067 1989 | - id03070 1990 | - id03071 1991 | - id03073 1992 | - id03074 1993 | - id03075 1994 | - id03077 1995 | - id03078 1996 | - id03079 1997 | - id03082 1998 | - id03084 1999 | - id03087 2000 | - id03088 2001 | - id03089 2002 | - id03090 2003 | - id03091 2004 | - id03092 2005 | - id03098 2006 | - id03099 2007 | - id03100 2008 | - id03101 2009 | - id03102 2010 | - id03103 2011 | - id03107 2012 | - id03109 2013 | - id03110 2014 | - id03111 2015 | - id03112 2016 | - id03115 2017 | - id03118 2018 | - id03119 2019 | - id03123 2020 | - id03124 2021 | - id03125 2022 | - id03129 2023 | - id03133 2024 | - id03134 2025 | - id03136 2026 | - id03137 2027 | - id03145 2028 | - id03146 2029 | - id03147 2030 | - id03148 2031 | - id03149 2032 | - id03150 2033 | - id03158 2034 | - id03168 2035 | - id03174 2036 | - id03175 2037 | - id03176 2038 | - id03177 2039 | - id03179 2040 | - id03180 2041 | - id03182 2042 | - id03183 2043 | - id03184 2044 | - id03187 2045 | - id03188 2046 | - id03189 2047 | - id03190 2048 | - id03192 2049 | - id03193 2050 | - id03194 2051 | - id03195 2052 | - id03196 2053 | - id03198 2054 | - id03199 2055 | - id03200 2056 | - id03201 2057 | - id03202 2058 | - id03203 2059 | - id03205 2060 | - id03207 2061 | - id03209 2062 | - id03210 2063 | - id03211 2064 | - id03212 2065 | - id03213 2066 | - id03214 2067 | - id03215 2068 | - id03216 2069 | - id03220 2070 | - id03221 2071 | - id03223 2072 | - id03226 2073 | - id03227 2074 | - id03228 2075 | - id03229 2076 | - id03230 2077 | - id03232 2078 | - id03236 2079 | - id03238 2080 | - id03240 2081 | - id03241 2082 | - id03242 2083 | - id03243 2084 | - id03244 2085 | - id03245 2086 | - id03246 2087 | - id03247 2088 | - id03248 2089 | - id03249 2090 | - id03250 2091 | - id03253 2092 | - id03258 2093 | - id03260 2094 | - id03261 2095 | - id03262 2096 | - id03263 2097 | - id03264 2098 | - id03265 2099 | - id03266 2100 | - id03267 2101 | - id03269 2102 | - id03272 2103 | - id03273 2104 | - id03274 2105 | - id03279 2106 | - id03281 2107 | - id03283 2108 | - id03284 2109 | - id03285 2110 | - id03286 2111 | - id03287 2112 | - id03288 2113 | - id03289 2114 | - id03290 2115 | - id03291 2116 | - id03292 2117 | - id03293 2118 | - id03295 2119 | - id03298 2120 | - id03299 2121 | - id03300 2122 | - id03303 2123 | - id03304 2124 | - id03306 2125 | - id03307 2126 | - id03312 2127 | - id03313 2128 | - id03314 2129 | - id03315 2130 | - id03316 2131 | - id03317 2132 | - id03318 2133 | - id03319 2134 | - id03320 2135 | - id03323 2136 | - id03324 2137 | - id03325 2138 | - id03326 2139 | - id03327 2140 | - id03328 2141 | - id03329 2142 | - id03331 2143 | - id03332 2144 | - id03333 2145 | - id03334 2146 | - id03335 2147 | - id03336 2148 | - id03337 2149 | - id03338 2150 | - id03339 2151 | - id03340 2152 | - id03341 2153 | - id03342 2154 | - id03343 2155 | - id03344 2156 | - id03345 2157 | - id03346 2158 | - id03348 2159 | - id03349 2160 | - id03350 2161 | - id03351 2162 | - id03352 2163 | - id03353 2164 | - id03354 2165 | - id03360 2166 | - id03361 2167 | - id03363 2168 | - id03364 2169 | - id03366 2170 | - id03371 2171 | - id03372 2172 | - id03373 2173 | - id03374 2174 | - id03375 2175 | - id03377 2176 | - id03379 2177 | - id03381 2178 | - id03383 2179 | - id03384 2180 | - id03386 2181 | - id03387 2182 | - id03388 2183 | - id03389 2184 | - id03390 2185 | - id03398 2186 | - id03399 2187 | - id03400 2188 | - id03401 2189 | - id03402 2190 | - id03405 2191 | - id03409 2192 | - id03410 2193 | - id03412 2194 | - id03413 2195 | - id03414 2196 | - id03417 2197 | - id03422 2198 | - id03424 2199 | - id03425 2200 | - id03426 2201 | - id03427 2202 | - id03428 2203 | - id03429 2204 | - id03430 2205 | - id03432 2206 | - id03433 2207 | - id03434 2208 | - id03435 2209 | - id03436 2210 | - id03439 2211 | - id03443 2212 | - id03444 2213 | - id03445 2214 | - id03446 2215 | - id03448 2216 | - id03449 2217 | - id03452 2218 | - id03453 2219 | - id03454 2220 | - id03455 2221 | - id03457 2222 | - id03458 2223 | - id03459 2224 | - id03460 2225 | - id03461 2226 | - id03462 2227 | - id03463 2228 | - id03467 2229 | - id03471 2230 | - id03477 2231 | - id03479 2232 | - id03480 2233 | - id03483 2234 | - id03485 2235 | - id03486 2236 | - id03487 2237 | - id03488 2238 | - id03489 2239 | - id03493 2240 | - id03495 2241 | - id03497 2242 | - id03499 2243 | - id03501 2244 | - id03502 2245 | - id03503 2246 | - id03506 2247 | - id03508 2248 | - id03509 2249 | - id03511 2250 | - id03512 2251 | - id03514 2252 | - id03515 2253 | - id03517 2254 | - id03518 2255 | - id03519 2256 | - id03520 2257 | - id03521 2258 | - id03522 2259 | - id03523 2260 | - id03525 2261 | - id03526 2262 | - id03527 2263 | - id03528 2264 | - id03529 2265 | - id03530 2266 | - id03531 2267 | - id03532 2268 | - id03534 2269 | - id03535 2270 | - id03537 2271 | - id03538 2272 | - id03540 2273 | - id03543 2274 | - id03546 2275 | - id03548 2276 | - id03550 2277 | - id03556 2278 | - id03557 2279 | - id03559 2280 | - id03560 2281 | - id03569 2282 | - id03570 2283 | - id03572 2284 | - id03573 2285 | - id03575 2286 | - id03577 2287 | - id03578 2288 | - id03580 2289 | - id03581 2290 | - id03582 2291 | - id03584 2292 | - id03587 2293 | - id03588 2294 | - id03589 2295 | - id03592 2296 | - id03597 2297 | - id03599 2298 | - id03605 2299 | - id03606 2300 | - id03609 2301 | - id03613 2302 | - id03614 2303 | - id03615 2304 | - id03617 2305 | - id03618 2306 | - id03619 2307 | - id03620 2308 | - id03621 2309 | - id03622 2310 | - id03623 2311 | - id03624 2312 | - id03625 2313 | - id03628 2314 | - id03631 2315 | - id03632 2316 | - id03633 2317 | - id03634 2318 | - id03639 2319 | - id03640 2320 | - id03643 2321 | - id03644 2322 | - id03645 2323 | - id03647 2324 | - id03649 2325 | - id03650 2326 | - id03651 2327 | - id03652 2328 | - id03654 2329 | - id03656 2330 | - id03658 2331 | - id03659 2332 | - id03661 2333 | - id03663 2334 | - id03664 2335 | - id03665 2336 | - id03667 2337 | - id03668 2338 | - id03669 2339 | - id03670 2340 | - id03671 2341 | - id03672 2342 | - id03673 2343 | - id03675 2344 | - id03678 2345 | - id03679 2346 | - id03680 2347 | - id03681 2348 | - id03682 2349 | - id03683 2350 | - id03684 2351 | - id03685 2352 | - id03686 2353 | - id03687 2354 | - id03688 2355 | - id03689 2356 | - id03690 2357 | - id03692 2358 | - id03693 2359 | - id03694 2360 | - id03696 2361 | - id03699 2362 | - id03701 2363 | - id03702 2364 | - id03703 2365 | - id03704 2366 | - id03705 2367 | - id03706 2368 | - id03707 2369 | - id03709 2370 | - id03710 2371 | - id03711 2372 | - id03713 2373 | - id03714 2374 | - id03715 2375 | - id03716 2376 | - id03717 2377 | - id03719 2378 | - id03720 2379 | - id03721 2380 | - id03722 2381 | - id03723 2382 | - id03724 2383 | - id03725 2384 | - id03727 2385 | - id03728 2386 | - id03729 2387 | - id03730 2388 | - id03731 2389 | - id03732 2390 | - id03733 2391 | - id03735 2392 | - id03736 2393 | - id03740 2394 | - id03741 2395 | - id03743 2396 | - id03744 2397 | - id03745 2398 | - id03746 2399 | - id03747 2400 | - id03748 2401 | - id03749 2402 | - id03750 2403 | - id03751 2404 | - id03752 2405 | - id03753 2406 | - id03754 2407 | - id03755 2408 | - id03756 2409 | - id03757 2410 | - id03758 2411 | - id03759 2412 | - id03760 2413 | - id03761 2414 | - id03762 2415 | - id03763 2416 | - id03764 2417 | - id03765 2418 | - id03767 2419 | - id03768 2420 | - id03769 2421 | - id03770 2422 | - id03772 2423 | - id03773 2424 | - id03774 2425 | - id03775 2426 | - id03777 2427 | - id03778 2428 | - id03779 2429 | - id03780 2430 | - id03781 2431 | - id03782 2432 | - id03784 2433 | - id03785 2434 | - id03786 2435 | - id03788 2436 | - id03790 2437 | - id03791 2438 | - id03795 2439 | - id03796 2440 | - id03797 2441 | - id03800 2442 | - id03803 2443 | - id03804 2444 | - id03810 2445 | - id03811 2446 | - id03812 2447 | - id03813 2448 | - id03815 2449 | - id03816 2450 | - id03817 2451 | - id03818 2452 | - id03819 2453 | - id03820 2454 | - id03821 2455 | - id03822 2456 | - id03823 2457 | - id03824 2458 | - id03825 2459 | - id03827 2460 | - id03828 2461 | - id03829 2462 | - id03830 2463 | - id03837 2464 | - id03838 2465 | - id03840 2466 | - id03841 2467 | - id03842 2468 | - id03843 2469 | - id03844 2470 | - id03845 2471 | - id03846 2472 | - id03847 2473 | - id03848 2474 | - id03852 2475 | - id03853 2476 | - id03854 2477 | - id03855 2478 | - id03856 2479 | - id03858 2480 | - id03859 2481 | - id03860 2482 | - id03861 2483 | - id03863 2484 | - id03864 2485 | - id03865 2486 | - id03866 2487 | - id03867 2488 | - id03868 2489 | - id03869 2490 | - id03870 2491 | - id03871 2492 | - id03873 2493 | - id03876 2494 | - id03877 2495 | - id03878 2496 | - id03884 2497 | - id03885 2498 | - id03887 2499 | - id03888 2500 | - id03889 2501 | - id03890 2502 | - id03891 2503 | - id03892 2504 | - id03893 2505 | - id03894 2506 | - id03895 2507 | - id03896 2508 | - id03897 2509 | - id03898 2510 | - id03899 2511 | - id03900 2512 | - id03902 2513 | - id03903 2514 | - id03904 2515 | - id03905 2516 | - id03906 2517 | - id03907 2518 | - id03909 2519 | - id03910 2520 | - id03912 2521 | - id03913 2522 | - id03914 2523 | - id03915 2524 | - id03916 2525 | - id03917 2526 | - id03918 2527 | - id03919 2528 | - id03920 2529 | - id03921 2530 | - id03922 2531 | - id03923 2532 | - id03924 2533 | - id03927 2534 | - id03928 2535 | - id03929 2536 | - id03930 2537 | - id03931 2538 | - id03932 2539 | - id03934 2540 | - id03935 2541 | - id03936 2542 | - id03938 2543 | - id03939 2544 | - id03940 2545 | - id03941 2546 | - id03942 2547 | - id03943 2548 | - id03945 2549 | - id03946 2550 | - id03947 2551 | - id03949 2552 | - id03950 2553 | - id03951 2554 | - id03952 2555 | - id03953 2556 | - id03954 2557 | - id03955 2558 | - id03956 2559 | - id03958 2560 | - id03961 2561 | - id03962 2562 | - id03963 2563 | - id03965 2564 | - id03966 2565 | - id03967 2566 | - id03968 2567 | - id03970 2568 | - id03971 2569 | - id03972 2570 | - id03974 2571 | - id03976 2572 | - id03977 2573 | - id03983 2574 | - id03984 2575 | - id03985 2576 | - id03986 2577 | - id03987 2578 | - id03989 2579 | - id03991 2580 | - id03992 2581 | - id03993 2582 | - id03994 2583 | - id03996 2584 | - id03998 2585 | - id03999 2586 | - id04005 2587 | - id04007 2588 | - id04009 2589 | - id04010 2590 | - id04012 2591 | - id04013 2592 | - id04014 2593 | - id04015 2594 | - id04016 2595 | - id04017 2596 | - id04018 2597 | - id04019 2598 | - id04021 2599 | - id04022 2600 | - id04024 2601 | - id04025 2602 | - id04026 2603 | - id04027 2604 | - id04028 2605 | - id04029 2606 | - id04031 2607 | - id04032 2608 | - id04033 2609 | - id04034 2610 | - id04035 2611 | - id04036 2612 | - id04037 2613 | - id04038 2614 | - id04039 2615 | - id04042 2616 | - id04044 2617 | - id04045 2618 | - id04046 2619 | - id04047 2620 | - id04048 2621 | - id04049 2622 | - id04050 2623 | - id04051 2624 | - id04052 2625 | - id04053 2626 | - id04054 2627 | - id04055 2628 | - id04057 2629 | - id04058 2630 | - id04062 2631 | - id04063 2632 | - id04064 2633 | - id04065 2634 | - id04066 2635 | - id04070 2636 | - id04073 2637 | - id04074 2638 | - id04076 2639 | - id04077 2640 | - id04079 2641 | - id04080 2642 | - id04081 2643 | - id04082 2644 | - id04084 2645 | - id04085 2646 | - id04086 2647 | - id04089 2648 | - id04091 2649 | - id04092 2650 | - id04093 2651 | - id04095 2652 | - id04096 2653 | - id04097 2654 | - id04098 2655 | - id04100 2656 | - id04102 2657 | - id04103 2658 | - id04105 2659 | - id04106 2660 | - id04107 2661 | - id04108 2662 | - id04109 2663 | - id04111 2664 | - id04114 2665 | - id04115 2666 | - id04116 2667 | - id04117 2668 | - id04118 2669 | - id04121 2670 | - id04122 2671 | - id04125 2672 | - id04126 2673 | - id04127 2674 | - id04128 2675 | - id04129 2676 | - id04130 2677 | - id04132 2678 | - id04133 2679 | - id04134 2680 | - id04136 2681 | - id04137 2682 | - id04140 2683 | - id04144 2684 | - id04145 2685 | - id04146 2686 | - id04147 2687 | - id04148 2688 | - id04149 2689 | - id04150 2690 | - id04151 2691 | - id04152 2692 | - id04153 2693 | - id04154 2694 | - id04155 2695 | - id04157 2696 | - id04159 2697 | - id04160 2698 | - id04161 2699 | - id04162 2700 | - id04163 2701 | - id04164 2702 | - id04165 2703 | - id04166 2704 | - id04167 2705 | - id04168 2706 | - id04169 2707 | - id04171 2708 | - id04173 2709 | - id04174 2710 | - id04175 2711 | - id04177 2712 | - id04178 2713 | - id04179 2714 | - id04180 2715 | - id04181 2716 | - id04182 2717 | - id04183 2718 | - id04184 2719 | - id04186 2720 | - id04187 2721 | - id04188 2722 | - id04189 2723 | - id04190 2724 | - id04191 2725 | - id04192 2726 | - id04193 2727 | - id04194 2728 | - id04195 2729 | - id04196 2730 | - id04197 2731 | - id04198 2732 | - id04199 2733 | - id04202 2734 | - id04203 2735 | - id04204 2736 | - id04205 2737 | - id04206 2738 | - id04207 2739 | - id04215 2740 | - id04216 2741 | - id04219 2742 | - id04221 2743 | - id04222 2744 | - id04223 2745 | - id04224 2746 | - id04225 2747 | - id04227 2748 | - id04228 2749 | - id04229 2750 | - id04230 2751 | - id04231 2752 | - id04234 2753 | - id04235 2754 | - id04236 2755 | - id04237 2756 | - id04238 2757 | - id04239 2758 | - id04240 2759 | - id04243 2760 | - id04244 2761 | - id04245 2762 | - id04246 2763 | - id04247 2764 | - id04249 2765 | - id04250 2766 | - id04251 2767 | - id04254 2768 | - id04255 2769 | - id04256 2770 | - id04257 2771 | - id04258 2772 | - id04259 2773 | - id04260 2774 | - id04261 2775 | - id04262 2776 | - id04263 2777 | - id04264 2778 | - id04265 2779 | - id04266 2780 | - id04267 2781 | - id04268 2782 | - id04269 2783 | - id04270 2784 | - id04271 2785 | - id04272 2786 | - id04273 2787 | - id04274 2788 | - id04275 2789 | - id04278 2790 | - id04279 2791 | - id04280 2792 | - id04282 2793 | - id04284 2794 | - id04285 2795 | - id04287 2796 | - id04288 2797 | - id04289 2798 | - id04292 2799 | - id04293 2800 | - id04294 2801 | - id04296 2802 | - id04297 2803 | - id04298 2804 | - id04300 2805 | - id04301 2806 | - id04302 2807 | - id04303 2808 | - id04304 2809 | - id04305 2810 | - id04309 2811 | - id04311 2812 | - id04312 2813 | - id04313 2814 | - id04314 2815 | - id04316 2816 | - id04318 2817 | - id04319 2818 | - id04320 2819 | - id04321 2820 | - id04322 2821 | - id04325 2822 | - id04326 2823 | - id04332 2824 | - id04333 2825 | - id04334 2826 | - id04335 2827 | - id04336 2828 | - id04340 2829 | - id04341 2830 | - id04342 2831 | - id04343 2832 | - id04344 2833 | - id04345 2834 | - id04346 2835 | - id04349 2836 | - id04350 2837 | - id04352 2838 | - id04353 2839 | - id04355 2840 | - id04356 2841 | - id04357 2842 | - id04358 2843 | - id04365 2844 | - id04367 2845 | - id04368 2846 | - id04370 2847 | - id04372 2848 | - id04373 2849 | - id04374 2850 | - id04375 2851 | - id04376 2852 | - id04377 2853 | - id04378 2854 | - id04379 2855 | - id04380 2856 | - id04383 2857 | - id04386 2858 | - id04387 2859 | - id04388 2860 | - id04392 2861 | - id04393 2862 | - id04396 2863 | - id04397 2864 | - id04398 2865 | - id04399 2866 | - id04402 2867 | - id04404 2868 | - id04405 2869 | - id04406 2870 | - id04407 2871 | - id04408 2872 | - id04409 2873 | - id04413 2874 | - id04414 2875 | - id04415 2876 | - id04416 2877 | - id04417 2878 | - id04418 2879 | - id04419 2880 | - id04420 2881 | - id04421 2882 | - id04422 2883 | - id04423 2884 | - id04424 2885 | - id04425 2886 | - id04427 2887 | - id04428 2888 | - id04429 2889 | - id04430 2890 | - id04432 2891 | - id04433 2892 | - id04434 2893 | - id04436 2894 | - id04437 2895 | - id04441 2896 | - id04442 2897 | - id04443 2898 | - id04444 2899 | - id04445 2900 | - id04446 2901 | - id04447 2902 | - id04449 2903 | - id04450 2904 | - id04451 2905 | - id04452 2906 | - id04453 2907 | - id04455 2908 | - id04456 2909 | - id04458 2910 | - id04459 2911 | - id04460 2912 | - id04462 2913 | - id04463 2914 | - id04474 2915 | - id04475 2916 | - id04476 2917 | - id04477 2918 | - id04479 2919 | - id04481 2920 | - id04482 2921 | - id04483 2922 | - id04484 2923 | - id04485 2924 | - id04486 2925 | - id04487 2926 | - id04488 2927 | - id04489 2928 | - id04490 2929 | - id04491 2930 | - id04493 2931 | - id04494 2932 | - id04495 2933 | - id04496 2934 | - id04497 2935 | - id04498 2936 | - id04499 2937 | - id04501 2938 | - id04502 2939 | - id04503 2940 | - id04504 2941 | - id04505 2942 | - id04506 2943 | - id04508 2944 | - id04509 2945 | - id04510 2946 | - id04511 2947 | - id04512 2948 | - id04513 2949 | - id04514 2950 | - id04516 2951 | - id04517 2952 | - id04518 2953 | - id04519 2954 | - id04520 2955 | - id04522 2956 | - id04523 2957 | - id04524 2958 | - id04525 2959 | - id04526 2960 | - id04527 2961 | - id04528 2962 | - id04529 2963 | - id04530 2964 | - id04531 2965 | - id04534 2966 | - id04537 2967 | - id04538 2968 | - id04540 2969 | - id04542 2970 | - id04543 2971 | - id04545 2972 | - id04547 2973 | - id04549 2974 | - id04551 2975 | - id04552 2976 | - id04554 2977 | - id04555 2978 | - id04556 2979 | - id04557 2980 | - id04559 2981 | - id04560 2982 | - id04561 2983 | - id04562 2984 | - id04563 2985 | - id04564 2986 | - id04565 2987 | - id04566 2988 | - id04567 2989 | - id04569 2990 | - id04571 2991 | - id04573 2992 | - id04576 2993 | - id04578 2994 | - id04579 2995 | - id04580 2996 | - id04582 2997 | - id04583 2998 | - id04584 2999 | - id04585 3000 | - id04586 3001 | - id04588 3002 | - id04589 3003 | - id04590 3004 | - id04593 3005 | - id04597 3006 | - id04599 3007 | - id04600 3008 | - id04601 3009 | - id04602 3010 | - id04606 3011 | - id04608 3012 | - id04609 3013 | - id04610 3014 | - id04613 3015 | - id04614 3016 | - id04615 3017 | - id04616 3018 | - id04617 3019 | - id04618 3020 | - id04619 3021 | - id04621 3022 | - id04623 3023 | - id04624 3024 | - id04625 3025 | - id04626 3026 | - id04628 3027 | - id04629 3028 | - id04635 3029 | - id04637 3030 | - id04638 3031 | - id04639 3032 | - id04640 3033 | - id04641 3034 | - id04642 3035 | - id04643 3036 | - id04644 3037 | - id04645 3038 | - id04646 3039 | - id04647 3040 | - id04648 3041 | - id04649 3042 | - id04650 3043 | - id04651 3044 | - id04652 3045 | - id04653 3046 | - id04654 3047 | - id04655 3048 | - id04658 3049 | - id04660 3050 | - id04662 3051 | - id04665 3052 | - id04666 3053 | - id04667 3054 | - id04668 3055 | - id04675 3056 | - id04676 3057 | - id04677 3058 | - id04684 3059 | - id04686 3060 | - id04687 3061 | - id04689 3062 | - id04690 3063 | - id04691 3064 | - id04692 3065 | - id04693 3066 | - id04694 3067 | - id04695 3068 | - id04696 3069 | - id04697 3070 | - id04698 3071 | - id04699 3072 | - id04701 3073 | - id04703 3074 | - id04704 3075 | - id04705 3076 | - id04706 3077 | - id04708 3078 | - id04709 3079 | - id04710 3080 | - id04712 3081 | - id04713 3082 | - id04714 3083 | - id04715 3084 | - id04716 3085 | - id04717 3086 | - id04718 3087 | - id04720 3088 | - id04721 3089 | - id04723 3090 | - id04725 3091 | - id04726 3092 | - id04727 3093 | - id04728 3094 | - id04730 3095 | - id04731 3096 | - id04733 3097 | - id04734 3098 | - id04736 3099 | - id04737 3100 | - id04738 3101 | - id04739 3102 | - id04740 3103 | - id04741 3104 | - id04742 3105 | - id04743 3106 | - id04744 3107 | - id04745 3108 | - id04746 3109 | - id04747 3110 | - id04748 3111 | - id04750 3112 | - id04751 3113 | - id04752 3114 | - id04753 3115 | - id04754 3116 | - id04755 3117 | - id04756 3118 | - id04757 3119 | - id04758 3120 | - id04759 3121 | - id04761 3122 | - id04762 3123 | - id04763 3124 | - id04770 3125 | - id04774 3126 | - id04776 3127 | - id04778 3128 | - id04780 3129 | - id04781 3130 | - id04782 3131 | - id04783 3132 | - id04784 3133 | - id04787 3134 | - id04788 3135 | - id04789 3136 | - id04800 3137 | - id04801 3138 | - id04802 3139 | - id04803 3140 | - id04814 3141 | - id04818 3142 | - id04820 3143 | - id04821 3144 | - id04822 3145 | - id04823 3146 | - id04825 3147 | - id04826 3148 | - id04827 3149 | - id04830 3150 | - id04834 3151 | - id04835 3152 | - id04836 3153 | - id04837 3154 | - id04838 3155 | - id04839 3156 | - id04841 3157 | - id04842 3158 | - id04844 3159 | - id04845 3160 | - id04846 3161 | - id04847 3162 | - id04848 3163 | - id04851 3164 | - id04852 3165 | - id04854 3166 | - id04855 3167 | - id04856 3168 | - id04857 3169 | - id04858 3170 | - id04863 3171 | - id04866 3172 | - id04869 3173 | - id04872 3174 | - id04873 3175 | - id04875 3176 | - id04876 3177 | - id04877 3178 | - id04884 3179 | - id04885 3180 | - id04886 3181 | - id04887 3182 | - id04888 3183 | - id04889 3184 | - id04890 3185 | - id04893 3186 | - id04894 3187 | - id04895 3188 | - id04896 3189 | - id04897 3190 | - id04898 3191 | - id04899 3192 | - id04900 3193 | - id04901 3194 | - id04902 3195 | - id04903 3196 | - id04904 3197 | - id04905 3198 | - id04906 3199 | - id04907 3200 | - id04908 3201 | - id04909 3202 | - id04910 3203 | - id04911 3204 | - id04914 3205 | - id04915 3206 | - id04917 3207 | - id04918 3208 | - id04926 3209 | - id04927 3210 | - id04928 3211 | - id04930 3212 | - id04933 3213 | - id04934 3214 | - id04935 3215 | - id04938 3216 | - id04939 3217 | - id04940 3218 | - id04941 3219 | - id04943 3220 | - id04945 3221 | - id04946 3222 | - id04948 3223 | - id04951 3224 | - id04953 3225 | - id04956 3226 | - id04957 3227 | - id04958 3228 | - id04959 3229 | - id04961 3230 | - id04962 3231 | - id04963 3232 | - id04964 3233 | - id04967 3234 | - id04968 3235 | - id04969 3236 | - id04970 3237 | - id04971 3238 | - id04972 3239 | - id04973 3240 | - id04974 3241 | - id04976 3242 | - id04977 3243 | - id04978 3244 | - id04979 3245 | - id04980 3246 | - id04981 3247 | - id04982 3248 | - id04983 3249 | - id04984 3250 | - id04985 3251 | - id04986 3252 | - id04987 3253 | - id04988 3254 | - id04990 3255 | - id04991 3256 | - id04993 3257 | - id04994 3258 | - id04995 3259 | - id04996 3260 | - id04997 3261 | - id04998 3262 | - id04999 3263 | - id05000 3264 | - id05003 3265 | - id05004 3266 | - id05008 3267 | - id05009 3268 | - id05010 3269 | - id05011 3270 | - id05012 3271 | - id05013 3272 | - id05014 3273 | - id05016 3274 | - id05018 3275 | - id05019 3276 | - id05020 3277 | - id05021 3278 | - id05022 3279 | - id05023 3280 | - id05025 3281 | - id05027 3282 | - id05028 3283 | - id05029 3284 | - id05030 3285 | - id05031 3286 | - id05032 3287 | - id05034 3288 | - id05035 3289 | - id05038 3290 | - id05039 3291 | - id05040 3292 | - id05041 3293 | - id05042 3294 | - id05044 3295 | - id05048 3296 | - id05049 3297 | - id05052 3298 | - id05054 3299 | - id05056 3300 | - id05057 3301 | - id05058 3302 | - id05060 3303 | - id05061 3304 | - id05062 3305 | - id05063 3306 | - id05066 3307 | - id05067 3308 | - id05075 3309 | - id05076 3310 | - id05079 3311 | - id05082 3312 | - id05087 3313 | - id05088 3314 | - id05090 3315 | - id05091 3316 | - id05092 3317 | - id05093 3318 | - id05094 3319 | - id05095 3320 | - id05096 3321 | - id05099 3322 | - id05100 3323 | - id05101 3324 | - id05102 3325 | - id05103 3326 | - id05104 3327 | - id05105 3328 | - id05106 3329 | - id05107 3330 | - id05108 3331 | - id05121 3332 | - id05123 3333 | - id05125 3334 | - id05129 3335 | - id05130 3336 | - id05131 3337 | - id05132 3338 | - id05133 3339 | - id05134 3340 | - id05135 3341 | - id05136 3342 | - id05137 3343 | - id05139 3344 | - id05141 3345 | - id05145 3346 | - id05147 3347 | - id05148 3348 | - id05149 3349 | - id05150 3350 | - id05152 3351 | - id05153 3352 | - id05154 3353 | - id05159 3354 | - id05160 3355 | - id05161 3356 | - id05162 3357 | - id05165 3358 | - id05166 3359 | - id05167 3360 | - id05168 3361 | - id05169 3362 | - id05170 3363 | - id05171 3364 | - id05172 3365 | - id05173 3366 | - id05174 3367 | - id05178 3368 | - id05179 3369 | - id05181 3370 | - id05182 3371 | - id05186 3372 | - id05187 3373 | - id05188 3374 | - id05189 3375 | - id05190 3376 | - id05191 3377 | - id05194 3378 | - id05196 3379 | - id05197 3380 | - id05198 3381 | - id05199 3382 | - id05200 3383 | - id05201 3384 | - id05203 3385 | - id05204 3386 | - id05205 3387 | - id05206 3388 | - id05207 3389 | - id05208 3390 | - id05209 3391 | - id05210 3392 | - id05211 3393 | - id05212 3394 | - id05213 3395 | - id05218 3396 | - id05220 3397 | - id05221 3398 | - id05223 3399 | - id05224 3400 | - id05226 3401 | - id05227 3402 | - id05228 3403 | - id05229 3404 | - id05231 3405 | - id05235 3406 | - id05236 3407 | - id05237 3408 | - id05239 3409 | - id05240 3410 | - id05241 3411 | - id05242 3412 | - id05244 3413 | - id05245 3414 | - id05246 3415 | - id05248 3416 | - id05249 3417 | - id05250 3418 | - id05251 3419 | - id05252 3420 | - id05253 3421 | - id05255 3422 | - id05256 3423 | - id05257 3424 | - id05258 3425 | - id05260 3426 | - id05261 3427 | - id05262 3428 | - id05263 3429 | - id05264 3430 | - id05265 3431 | - id05266 3432 | - id05268 3433 | - id05269 3434 | - id05270 3435 | - id05271 3436 | - id05272 3437 | - id05273 3438 | - id05274 3439 | - id05275 3440 | - id05276 3441 | - id05277 3442 | - id05278 3443 | - id05280 3444 | - id05282 3445 | - id05283 3446 | - id05284 3447 | - id05286 3448 | - id05287 3449 | - id05288 3450 | - id05289 3451 | - id05290 3452 | - id05291 3453 | - id05292 3454 | - id05293 3455 | - id05295 3456 | - id05296 3457 | - id05297 3458 | - id05300 3459 | - id05301 3460 | - id05303 3461 | - id05306 3462 | - id05308 3463 | - id05310 3464 | - id05312 3465 | - id05313 3466 | - id05314 3467 | - id05315 3468 | - id05316 3469 | - id05317 3470 | - id05319 3471 | - id05321 3472 | - id05323 3473 | - id05324 3474 | - id05328 3475 | - id05332 3476 | - id05333 3477 | - id05334 3478 | - id05335 3479 | - id05339 3480 | - id05341 3481 | - id05342 3482 | - id05343 3483 | - id05344 3484 | - id05345 3485 | - id05346 3486 | - id05349 3487 | - id05350 3488 | - id05351 3489 | - id05352 3490 | - id05353 3491 | - id05354 3492 | - id05357 3493 | - id05358 3494 | - id05369 3495 | - id05370 3496 | - id05371 3497 | - id05372 3498 | - id05373 3499 | - id05375 3500 | - id05376 3501 | - id05378 3502 | - id05379 3503 | - id05380 3504 | - id05381 3505 | - id05383 3506 | - id05384 3507 | - id05387 3508 | - id05388 3509 | - id05392 3510 | - id05398 3511 | - id05399 3512 | - id05401 3513 | - id05402 3514 | - id05403 3515 | - id05404 3516 | - id05405 3517 | - id05408 3518 | - id05421 3519 | - id05422 3520 | - id05423 3521 | - id05424 3522 | - id05426 3523 | - id05427 3524 | - id05428 3525 | - id05429 3526 | - id05430 3527 | - id05431 3528 | - id05432 3529 | - id05433 3530 | - id05434 3531 | - id05435 3532 | - id05438 3533 | - id05441 3534 | - id05442 3535 | - id05443 3536 | - id05444 3537 | - id05445 3538 | - id05448 3539 | - id05449 3540 | - id05450 3541 | - id05451 3542 | - id05453 3543 | - id05454 3544 | - id05456 3545 | - id05457 3546 | - id05460 3547 | - id05465 3548 | - id05466 3549 | - id05467 3550 | - id05468 3551 | - id05469 3552 | - id05470 3553 | - id05471 3554 | - id05472 3555 | - id05474 3556 | - id05476 3557 | - id05477 3558 | - id05478 3559 | - id05479 3560 | - id05480 3561 | - id05481 3562 | - id05483 3563 | - id05486 3564 | - id05487 3565 | - id05490 3566 | - id05491 3567 | - id05492 3568 | - id05493 3569 | - id05494 3570 | - id05496 3571 | - id05502 3572 | - id05503 3573 | - id05504 3574 | - id05505 3575 | - id05506 3576 | - id05507 3577 | - id05508 3578 | - id05509 3579 | - id05510 3580 | - id05512 3581 | - id05513 3582 | - id05515 3583 | - id05516 3584 | - id05517 3585 | - id05518 3586 | - id05519 3587 | - id05520 3588 | - id05521 3589 | - id05522 3590 | - id05524 3591 | - id05526 3592 | - id05527 3593 | - id05528 3594 | - id05529 3595 | - id05534 3596 | - id05535 3597 | - id05536 3598 | - id05538 3599 | - id05541 3600 | - id05545 3601 | - id05546 3602 | - id05547 3603 | - id05549 3604 | - id05551 3605 | - id05552 3606 | - id05553 3607 | - id05554 3608 | - id05555 3609 | - id05556 3610 | - id05557 3611 | - id05558 3612 | - id05559 3613 | - id05561 3614 | - id05562 3615 | - id05563 3616 | - id05564 3617 | - id05567 3618 | - id05568 3619 | - id05570 3620 | - id05573 3621 | - id05574 3622 | - id05575 3623 | - id05576 3624 | - id05577 3625 | - id05579 3626 | - id05581 3627 | - id05583 3628 | - id05584 3629 | - id05585 3630 | - id05586 3631 | - id05588 3632 | - id05591 3633 | - id05607 3634 | - id05609 3635 | - id05610 3636 | - id05612 3637 | - id05613 3638 | - id05614 3639 | - id05615 3640 | - id05616 3641 | - id05619 3642 | - id05620 3643 | - id05621 3644 | - id05622 3645 | - id05623 3646 | - id05624 3647 | - id05625 3648 | - id05627 3649 | - id05628 3650 | - id05631 3651 | - id05632 3652 | - id05633 3653 | - id05634 3654 | - id05639 3655 | - id05640 3656 | - id05641 3657 | - id05642 3658 | - id05643 3659 | - id05644 3660 | - id05645 3661 | - id05647 3662 | - id05648 3663 | - id05649 3664 | - id05657 3665 | - id05658 3666 | - id05660 3667 | - id05663 3668 | - id05664 3669 | - id05665 3670 | - id05666 3671 | - id05667 3672 | - id05668 3673 | - id05670 3674 | - id05671 3675 | - id05672 3676 | - id05677 3677 | - id05678 3678 | - id05681 3679 | - id05684 3680 | - id05685 3681 | - id05686 3682 | - id05687 3683 | - id05688 3684 | - id05689 3685 | - id05690 3686 | - id05691 3687 | - id05692 3688 | - id05693 3689 | - id05694 3690 | - id05695 3691 | - id05696 3692 | - id05697 3693 | - id05698 3694 | - id05699 3695 | - id05700 3696 | - id05703 3697 | - id05704 3698 | - id05706 3699 | - id05707 3700 | - id05708 3701 | - id05710 3702 | - id05715 3703 | - id05716 3704 | - id05718 3705 | - id05719 3706 | - id05721 3707 | - id05722 3708 | - id05723 3709 | - id05724 3710 | - id05725 3711 | - id05726 3712 | - id05727 3713 | - id05728 3714 | - id05729 3715 | - id05731 3716 | - id05733 3717 | - id05734 3718 | - id05735 3719 | - id05737 3720 | - id05738 3721 | - id05739 3722 | - id05740 3723 | - id05741 3724 | - id05742 3725 | - id05743 3726 | - id05745 3727 | - id05746 3728 | - id05747 3729 | - id05748 3730 | - id05749 3731 | - id05750 3732 | - id05751 3733 | - id05752 3734 | - id05754 3735 | - id05756 3736 | - id05757 3737 | - id05758 3738 | - id05759 3739 | - id05760 3740 | - id05762 3741 | - id05764 3742 | - id05765 3743 | - id05766 3744 | - id05767 3745 | - id05768 3746 | - id05769 3747 | - id05770 3748 | - id05771 3749 | - id05772 3750 | - id05774 3751 | - id05776 3752 | - id05777 3753 | - id05779 3754 | - id05783 3755 | - id05784 3756 | - id05788 3757 | - id05792 3758 | - id05793 3759 | - id05796 3760 | - id05797 3761 | - id05798 3762 | - id05799 3763 | - id05801 3764 | - id05804 3765 | - id05805 3766 | - id05806 3767 | - id05808 3768 | - id05810 3769 | - id05811 3770 | - id05812 3771 | - id05813 3772 | - id05814 3773 | - id05815 3774 | - id05827 3775 | - id05828 3776 | - id05829 3777 | - id05830 3778 | - id05831 3779 | - id05832 3780 | - id05833 3781 | - id05834 3782 | - id05835 3783 | - id05836 3784 | - id05837 3785 | - id05838 3786 | - id05841 3787 | - id05843 3788 | - id05844 3789 | - id05845 3790 | - id05847 3791 | - id05849 3792 | - id05851 3793 | - id05852 3794 | - id05853 3795 | - id05856 3796 | - id05858 3797 | - id05860 3798 | - id05861 3799 | - id05864 3800 | - id05868 3801 | - id05869 3802 | - id05870 3803 | - id05871 3804 | - id05873 3805 | - id05874 3806 | - id05875 3807 | - id05876 3808 | - id05877 3809 | - id05878 3810 | - id05879 3811 | - id05880 3812 | - id05881 3813 | - id05882 3814 | - id05887 3815 | - id05890 3816 | - id05891 3817 | - id05892 3818 | - id05893 3819 | - id05894 3820 | - id05895 3821 | - id05904 3822 | - id05905 3823 | - id05906 3824 | - id05907 3825 | - id05908 3826 | - id05913 3827 | - id05914 3828 | - id05915 3829 | - id05916 3830 | - id05917 3831 | - id05919 3832 | - id05920 3833 | - id05921 3834 | - id05922 3835 | - id05923 3836 | - id05926 3837 | - id05927 3838 | - id05930 3839 | - id05931 3840 | - id05932 3841 | - id05934 3842 | - id05935 3843 | - id05936 3844 | - id05938 3845 | - id05939 3846 | - id05940 3847 | - id05942 3848 | - id05943 3849 | - id05944 3850 | - id05945 3851 | - id05946 3852 | - id05951 3853 | - id05952 3854 | - id05953 3855 | - id05954 3856 | - id05956 3857 | - id05957 3858 | - id05958 3859 | - id05959 3860 | - id05960 3861 | - id05961 3862 | - id05962 3863 | - id05963 3864 | - id05964 3865 | - id05966 3866 | - id05969 3867 | - id05970 3868 | - id05976 3869 | - id05977 3870 | - id05978 3871 | - id05980 3872 | - id05981 3873 | - id05982 3874 | - id05983 3875 | - id05984 3876 | - id05985 3877 | - id05986 3878 | - id05987 3879 | - id05988 3880 | - id05989 3881 | - id05990 3882 | - id05991 3883 | - id05993 3884 | - id05994 3885 | - id05996 3886 | - id05997 3887 | - id05998 3888 | - id06000 3889 | - id06002 3890 | - id06003 3891 | - id06004 3892 | - id06005 3893 | - id06006 3894 | - id06007 3895 | - id06008 3896 | - id06010 3897 | - id06011 3898 | - id06012 3899 | - id06013 3900 | - id06014 3901 | - id06015 3902 | - id06016 3903 | - id06017 3904 | - id06018 3905 | - id06019 3906 | - id06020 3907 | - id06021 3908 | - id06022 3909 | - id06023 3910 | - id06024 3911 | - id06025 3912 | - id06026 3913 | - id06027 3914 | - id06028 3915 | - id06029 3916 | - id06038 3917 | - id06040 3918 | - id06042 3919 | - id06044 3920 | - id06045 3921 | - id06046 3922 | - id06049 3923 | - id06050 3924 | - id06051 3925 | - id06052 3926 | - id06054 3927 | - id06055 3928 | - id06056 3929 | - id06058 3930 | - id06059 3931 | - id06060 3932 | - id06061 3933 | - id06062 3934 | - id06064 3935 | - id06065 3936 | - id06066 3937 | - id06067 3938 | - id06068 3939 | - id06070 3940 | - id06071 3941 | - id06073 3942 | - id06081 3943 | - id06084 3944 | - id06086 3945 | - id06087 3946 | - id06092 3947 | - id06095 3948 | - id06096 3949 | - id06097 3950 | - id06103 3951 | - id06105 3952 | - id06106 3953 | - id06107 3954 | - id06108 3955 | - id06109 3956 | - id06110 3957 | - id06111 3958 | - id06112 3959 | - id06113 3960 | - id06114 3961 | - id06115 3962 | - id06116 3963 | - id06117 3964 | - id06118 3965 | - id06119 3966 | - id06120 3967 | - id06121 3968 | - id06122 3969 | - id06124 3970 | - id06128 3971 | - id06130 3972 | - id06131 3973 | - id06133 3974 | - id06134 3975 | - id06135 3976 | - id06138 3977 | - id06139 3978 | - id06140 3979 | - id06142 3980 | - id06143 3981 | - id06145 3982 | - id06146 3983 | - id06148 3984 | - id06152 3985 | - id06154 3986 | - id06155 3987 | - id06156 3988 | - id06158 3989 | - id06159 3990 | - id06161 3991 | - id06162 3992 | - id06163 3993 | - id06164 3994 | - id06165 3995 | - id06166 3996 | - id06168 3997 | - id06172 3998 | - id06173 3999 | - id06174 4000 | - id06176 4001 | - id06178 4002 | - id06183 4003 | - id06184 4004 | - id06185 4005 | - id06188 4006 | - id06189 4007 | - id06191 4008 | - id06194 4009 | - id06195 4010 | - id06196 4011 | - id06197 4012 | - id06199 4013 | - id06200 4014 | - id06202 4015 | - id06203 4016 | - id06206 4017 | - id06207 4018 | - id06210 4019 | - id06211 4020 | - id06212 4021 | - id06213 4022 | - id06214 4023 | - id06215 4024 | - id06216 4025 | - id06217 4026 | - id06218 4027 | - id06220 4028 | - id06221 4029 | - id06222 4030 | - id06225 4031 | - id06226 4032 | - id06227 4033 | - id06228 4034 | - id06229 4035 | - id06231 4036 | - id06232 4037 | - id06233 4038 | - id06234 4039 | - id06235 4040 | - id06236 4041 | - id06237 4042 | - id06238 4043 | - id06239 4044 | - id06242 4045 | - id06244 4046 | - id06245 4047 | - id06246 4048 | - id06247 4049 | - id06248 4050 | - id06249 4051 | - id06250 4052 | - id06251 4053 | - id06252 4054 | - id06254 4055 | - id06255 4056 | - id06256 4057 | - id06258 4058 | - id06261 4059 | - id06262 4060 | - id06263 4061 | - id06264 4062 | - id06265 4063 | - id06267 4064 | - id06268 4065 | - id06269 4066 | - id06270 4067 | - id06274 4068 | - id06275 4069 | - id06276 4070 | - id06277 4071 | - id06278 4072 | - id06279 4073 | - id06280 4074 | - id06288 4075 | - id06289 4076 | - id06292 4077 | - id06293 4078 | - id06295 4079 | - id06297 4080 | - id06298 4081 | - id06299 4082 | - id06301 4083 | - id06302 4084 | - id06303 4085 | - id06304 4086 | - id06307 4087 | - id06311 4088 | - id06319 4089 | - id06320 4090 | - id06321 4091 | - id06322 4092 | - id06323 4093 | - id06324 4094 | - id06328 4095 | - id06331 4096 | - id06332 4097 | - id06333 4098 | - id06334 4099 | - id06335 4100 | - id06336 4101 | - id06338 4102 | - id06339 4103 | - id06340 4104 | - id06341 4105 | - id06342 4106 | - id06343 4107 | - id06344 4108 | - id06346 4109 | - id06347 4110 | - id06349 4111 | - id06350 4112 | - id06351 4113 | - id06353 4114 | - id06354 4115 | - id06355 4116 | - id06356 4117 | - id06357 4118 | - id06358 4119 | - id06360 4120 | - id06361 4121 | - id06362 4122 | - id06364 4123 | - id06368 4124 | - id06370 4125 | - id06371 4126 | - id06375 4127 | - id06376 4128 | - id06377 4129 | - id06378 4130 | - id06379 4131 | - id06380 4132 | - id06381 4133 | - id06382 4134 | - id06383 4135 | - id06384 4136 | - id06385 4137 | - id06386 4138 | - id06388 4139 | - id06389 4140 | - id06390 4141 | - id06392 4142 | - id06393 4143 | - id06394 4144 | - id06395 4145 | - id06396 4146 | - id06397 4147 | - id06398 4148 | - id06399 4149 | - id06400 4150 | - id06403 4151 | - id06404 4152 | - id06405 4153 | - id06406 4154 | - id06408 4155 | - id06409 4156 | - id06411 4157 | - id06412 4158 | - id06413 4159 | - id06414 4160 | - id06415 4161 | - id06416 4162 | - id06417 4163 | - id06418 4164 | - id06422 4165 | - id06423 4166 | - id06424 4167 | - id06425 4168 | - id06426 4169 | - id06427 4170 | - id06428 4171 | - id06429 4172 | - id06430 4173 | - id06431 4174 | - id06432 4175 | - id06434 4176 | - id06435 4177 | - id06436 4178 | - id06437 4179 | - id06438 4180 | - id06439 4181 | - id06440 4182 | - id06443 4183 | - id06445 4184 | - id06446 4185 | - id06447 4186 | - id06448 4187 | - id06449 4188 | - id06450 4189 | - id06453 4190 | - id06454 4191 | - id06456 4192 | - id06457 4193 | - id06459 4194 | - id06462 4195 | - id06465 4196 | - id06467 4197 | - id06468 4198 | - id06469 4199 | - id06470 4200 | - id06471 4201 | - id06472 4202 | - id06474 4203 | - id06476 4204 | - id06477 4205 | - id06478 4206 | - id06479 4207 | - id06480 4208 | - id06481 4209 | - id06482 4210 | - id06483 4211 | - id06485 4212 | - id06486 4213 | - id06487 4214 | - id06489 4215 | - id06490 4216 | - id06491 4217 | - id06493 4218 | - id06494 4219 | - id06495 4220 | - id06496 4221 | - id06497 4222 | - id06498 4223 | - id06499 4224 | - id06500 4225 | - id06502 4226 | - id06503 4227 | - id06504 4228 | - id06505 4229 | - id06506 4230 | - id06507 4231 | - id06509 4232 | - id06511 4233 | - id06516 4234 | - id06517 4235 | - id06518 4236 | - id06519 4237 | - id06520 4238 | - id06524 4239 | - id06526 4240 | - id06528 4241 | - id06529 4242 | - id06530 4243 | - id06531 4244 | - id06535 4245 | - id06536 4246 | - id06537 4247 | - id06538 4248 | - id06539 4249 | - id06541 4250 | - id06544 4251 | - id06547 4252 | - id06548 4253 | - id06550 4254 | - id06551 4255 | - id06552 4256 | - id06553 4257 | - id06554 4258 | - id06555 4259 | - id06556 4260 | - id06557 4261 | - id06559 4262 | - id06560 4263 | - id06574 4264 | - id06575 4265 | - id06576 4266 | - id06577 4267 | - id06578 4268 | - id06581 4269 | - id06582 4270 | - id06583 4271 | - id06584 4272 | - id06586 4273 | - id06588 4274 | - id06589 4275 | - id06591 4276 | - id06593 4277 | - id06594 4278 | - id06595 4279 | - id06596 4280 | - id06597 4281 | - id06598 4282 | - id06599 4283 | - id06607 4284 | - id06608 4285 | - id06609 4286 | - id06612 4287 | - id06613 4288 | - id06614 4289 | - id06616 4290 | - id06617 4291 | - id06619 4292 | - id06621 4293 | - id06622 4294 | - id06624 4295 | - id06625 4296 | - id06631 4297 | - id06633 4298 | - id06637 4299 | - id06640 4300 | - id06642 4301 | - id06643 4302 | - id06645 4303 | - id06648 4304 | - id06649 4305 | - id06650 4306 | - id06651 4307 | - id06661 4308 | - id06662 4309 | - id06663 4310 | - id06672 4311 | - id06673 4312 | - id06674 4313 | - id06676 4314 | - id06683 4315 | - id06684 4316 | - id06685 4317 | - id06686 4318 | - id06687 4319 | - id06688 4320 | - id06689 4321 | - id06691 4322 | - id06694 4323 | - id06695 4324 | - id06696 4325 | - id06697 4326 | - id06698 4327 | - id06701 4328 | - id06702 4329 | - id06703 4330 | - id06704 4331 | - id06706 4332 | - id06708 4333 | - id06709 4334 | - id06713 4335 | - id06714 4336 | - id06715 4337 | - id06716 4338 | - id06717 4339 | - id06721 4340 | - id06722 4341 | - id06723 4342 | - id06724 4343 | - id06725 4344 | - id06726 4345 | - id06730 4346 | - id06732 4347 | - id06741 4348 | - id06743 4349 | - id06744 4350 | - id06745 4351 | - id06747 4352 | - id06748 4353 | - id06749 4354 | - id06750 4355 | - id06752 4356 | - id06753 4357 | - id06754 4358 | - id06757 4359 | - id06758 4360 | - id06759 4361 | - id06760 4362 | - id06761 4363 | - id06763 4364 | - id06765 4365 | - id06767 4366 | - id06768 4367 | - id06771 4368 | - id06772 4369 | - id06775 4370 | - id06776 4371 | - id06779 4372 | - id06782 4373 | - id06784 4374 | - id06791 4375 | - id06792 4376 | - id06793 4377 | - id06794 4378 | - id06795 4379 | - id06797 4380 | - id06798 4381 | - id06799 4382 | - id06807 4383 | - id06809 4384 | - id06810 4385 | - id06812 4386 | - id06813 4387 | - id06815 4388 | - id06817 4389 | - id06818 4390 | - id06819 4391 | - id06820 4392 | - id06821 4393 | - id06822 4394 | - id06823 4395 | - id06828 4396 | - id06829 4397 | - id06830 4398 | - id06833 4399 | - id06835 4400 | - id06836 4401 | - id06837 4402 | - id06838 4403 | - id06839 4404 | - id06840 4405 | - id06841 4406 | - id06842 4407 | - id06843 4408 | - id06844 4409 | - id06845 4410 | - id06846 4411 | - id06847 4412 | - id06848 4413 | - id06849 4414 | - id06850 4415 | - id06851 4416 | - id06852 4417 | - id06853 4418 | - id06854 4419 | - id06855 4420 | - id06856 4421 | - id06857 4422 | - id06858 4423 | - id06860 4424 | - id06861 4425 | - id06862 4426 | - id06863 4427 | - id06864 4428 | - id06866 4429 | - id06867 4430 | - id06869 4431 | - id06870 4432 | - id06874 4433 | - id06875 4434 | - id06876 4435 | - id06878 4436 | - id06879 4437 | - id06881 4438 | - id06883 4439 | - id06887 4440 | - id06888 4441 | - id06889 4442 | - id06890 4443 | - id06892 4444 | - id06893 4445 | - id06896 4446 | - id06900 4447 | - id06905 4448 | - id06906 4449 | - id06907 4450 | - id06908 4451 | - id06909 4452 | - id06910 4453 | - id06911 4454 | - id06912 4455 | - id06914 4456 | - id06915 4457 | - id06916 4458 | - id06918 4459 | - id06920 4460 | - id06922 4461 | - id06926 4462 | - id06927 4463 | - id06928 4464 | - id06929 4465 | - id06930 4466 | - id06931 4467 | - id06932 4468 | - id06934 4469 | - id06935 4470 | - id06936 4471 | - id06937 4472 | - id06938 4473 | - id06939 4474 | - id06940 4475 | - id06941 4476 | - id06942 4477 | - id06944 4478 | - id06945 4479 | - id06946 4480 | - id06948 4481 | - id06949 4482 | - id06950 4483 | - id06951 4484 | - id06952 4485 | - id06953 4486 | - id06954 4487 | - id06957 4488 | - id06958 4489 | - id06960 4490 | - id06961 4491 | - id06962 4492 | - id06963 4493 | - id06964 4494 | - id06966 4495 | - id06967 4496 | - id06968 4497 | - id06969 4498 | - id06970 4499 | - id06971 4500 | - id06972 4501 | - id06973 4502 | - id06974 4503 | - id06976 4504 | - id06982 4505 | - id06983 4506 | - id06984 4507 | - id06985 4508 | - id06986 4509 | - id06987 4510 | - id06988 4511 | - id06989 4512 | - id06990 4513 | - id06991 4514 | - id06994 4515 | - id06995 4516 | - id06996 4517 | - id06997 4518 | - id06998 4519 | - id06999 4520 | - id07001 4521 | - id07002 4522 | - id07003 4523 | - id07004 4524 | - id07005 4525 | - id07007 4526 | - id07008 4527 | - id07009 4528 | - id07010 4529 | - id07011 4530 | - id07012 4531 | - id07015 4532 | - id07017 4533 | - id07019 4534 | - id07020 4535 | - id07022 4536 | - id07026 4537 | - id07031 4538 | - id07032 4539 | - id07034 4540 | - id07036 4541 | - id07037 4542 | - id07039 4543 | - id07042 4544 | - id07043 4545 | - id07044 4546 | - id07045 4547 | - id07046 4548 | - id07048 4549 | - id07049 4550 | - id07050 4551 | - id07051 4552 | - id07052 4553 | - id07053 4554 | - id07054 4555 | - id07055 4556 | - id07056 4557 | - id07057 4558 | - id07058 4559 | - id07059 4560 | - id07060 4561 | - id07062 4562 | - id07063 4563 | - id07064 4564 | - id07065 4565 | - id07066 4566 | - id07071 4567 | - id07072 4568 | - id07074 4569 | - id07076 4570 | - id07077 4571 | - id07078 4572 | - id07080 4573 | - id07084 4574 | - id07085 4575 | - id07086 4576 | - id07087 4577 | - id07088 4578 | - id07089 4579 | - id07092 4580 | - id07093 4581 | - id07094 4582 | - id07095 4583 | - id07097 4584 | - id07098 4585 | - id07099 4586 | - id07100 4587 | - id07102 4588 | - id07103 4589 | - id07104 4590 | - id07105 4591 | - id07107 4592 | - id07108 4593 | - id07109 4594 | - id07112 4595 | - id07113 4596 | - id07114 4597 | - id07115 4598 | - id07116 4599 | - id07117 4600 | - id07118 4601 | - id07120 4602 | - id07121 4603 | - id07122 4604 | - id07123 4605 | - id07124 4606 | - id07128 4607 | - id07130 4608 | - id07131 4609 | - id07132 4610 | - id07133 4611 | - id07134 4612 | - id07135 4613 | - id07136 4614 | - id07137 4615 | - id07140 4616 | - id07145 4617 | - id07146 4618 | - id07148 4619 | - id07151 4620 | - id07153 4621 | - id07154 4622 | - id07155 4623 | - id07158 4624 | - id07160 4625 | - id07161 4626 | - id07162 4627 | - id07163 4628 | - id07164 4629 | - id07165 4630 | - id07166 4631 | - id07167 4632 | - id07168 4633 | - id07169 4634 | - id07170 4635 | - id07171 4636 | - id07173 4637 | - id07175 4638 | - id07176 4639 | - id07177 4640 | - id07178 4641 | - id07179 4642 | - id07180 4643 | - id07181 4644 | - id07182 4645 | - id07183 4646 | - id07185 4647 | - id07186 4648 | - id07187 4649 | - id07188 4650 | - id07189 4651 | - id07191 4652 | - id07192 4653 | - id07194 4654 | - id07195 4655 | - id07196 4656 | - id07197 4657 | - id07198 4658 | - id07199 4659 | - id07200 4660 | - id07202 4661 | - id07204 4662 | - id07205 4663 | - id07206 4664 | - id07207 4665 | - id07208 4666 | - id07209 4667 | - id07210 4668 | - id07212 4669 | - id07213 4670 | - id07214 4671 | - id07215 4672 | - id07217 4673 | - id07218 4674 | - id07219 4675 | - id07220 4676 | - id07221 4677 | - id07223 4678 | - id07227 4679 | - id07228 4680 | - id07229 4681 | - id07230 4682 | - id07232 4683 | - id07233 4684 | - id07234 4685 | - id07235 4686 | - id07236 4687 | - id07238 4688 | - id07240 4689 | - id07241 4690 | - id07242 4691 | - id07243 4692 | - id07244 4693 | - id07246 4694 | - id07247 4695 | - id07250 4696 | - id07251 4697 | - id07253 4698 | - id07254 4699 | - id07255 4700 | - id07256 4701 | - id07258 4702 | - id07259 4703 | - id07262 4704 | - id07263 4705 | - id07264 4706 | - id07265 4707 | - id07268 4708 | - id07269 4709 | - id07272 4710 | - id07273 4711 | - id07275 4712 | - id07276 4713 | - id07277 4714 | - id07278 4715 | - id07279 4716 | - id07283 4717 | - id07284 4718 | - id07285 4719 | - id07288 4720 | - id07290 4721 | - id07292 4722 | - id07293 4723 | - id07294 4724 | - id07295 4725 | - id07296 4726 | - id07297 4727 | - id07299 4728 | - id07303 4729 | - id07305 4730 | - id07306 4731 | - id07308 4732 | - id07311 4733 | - id07314 4734 | - id07316 4735 | - id07320 4736 | - id07321 4737 | - id07322 4738 | - id07332 4739 | - id07333 4740 | - id07334 4741 | - id07335 4742 | - id07337 4743 | - id07338 4744 | - id07341 4745 | - id07342 4746 | - id07343 4747 | - id07344 4748 | - id07345 4749 | - id07346 4750 | - id07349 4751 | - id07350 4752 | - id07351 4753 | - id07352 4754 | - id07353 4755 | - id07355 4756 | - id07356 4757 | - id07357 4758 | - id07358 4759 | - id07359 4760 | - id07360 4761 | - id07361 4762 | - id07362 4763 | - id07363 4764 | - id07366 4765 | - id07367 4766 | - id07368 4767 | - id07369 4768 | - id07370 4769 | - id07371 4770 | - id07374 4771 | - id07375 4772 | - id07376 4773 | - id07377 4774 | - id07383 4775 | - id07384 4776 | - id07385 4777 | - id07388 4778 | - id07389 4779 | - id07392 4780 | - id07393 4781 | - id07394 4782 | - id07395 4783 | - id07397 4784 | - id07398 4785 | - id07399 4786 | - id07400 4787 | - id07401 4788 | - id07402 4789 | - id07403 4790 | - id07404 4791 | - id07405 4792 | - id07407 4793 | - id07408 4794 | - id07409 4795 | - id07411 4796 | - id07412 4797 | - id07413 4798 | - id07415 4799 | - id07416 4800 | - id07417 4801 | - id07418 4802 | - id07419 4803 | - id07421 4804 | - id07422 4805 | - id07423 4806 | - id07427 4807 | - id07428 4808 | - id07429 4809 | - id07430 4810 | - id07431 4811 | - id07432 4812 | - id07433 4813 | - id07434 4814 | - id07437 4815 | - id07438 4816 | - id07439 4817 | - id07441 4818 | - id07442 4819 | - id07445 4820 | - id07446 4821 | - id07447 4822 | - id07448 4823 | - id07452 4824 | - id07453 4825 | - id07454 4826 | - id07455 4827 | - id07456 4828 | - id07457 4829 | - id07459 4830 | - id07460 4831 | - id07461 4832 | - id07462 4833 | - id07463 4834 | - id07465 4835 | - id07466 4836 | - id07468 4837 | - id07469 4838 | - id07470 4839 | - id07471 4840 | - id07473 4841 | - id07474 4842 | - id07477 4843 | - id07478 4844 | - id07483 4845 | - id07484 4846 | - id07489 4847 | - id07491 4848 | - id07492 4849 | - id07493 4850 | - id07495 4851 | - id07496 4852 | - id07497 4853 | - id07498 4854 | - id07500 4855 | - id07502 4856 | - id07506 4857 | - id07507 4858 | - id07508 4859 | - id07510 4860 | - id07511 4861 | - id07512 4862 | - id07517 4863 | - id07519 4864 | - id07520 4865 | - id07522 4866 | - id07524 4867 | - id07525 4868 | - id07526 4869 | - id07527 4870 | - id07528 4871 | - id07529 4872 | - id07531 4873 | - id07532 4874 | - id07533 4875 | - id07534 4876 | - id07535 4877 | - id07536 4878 | - id07537 4879 | - id07538 4880 | - id07539 4881 | - id07540 4882 | - id07541 4883 | - id07543 4884 | - id07545 4885 | - id07546 4886 | - id07547 4887 | - id07548 4888 | - id07549 4889 | - id07550 4890 | - id07551 4891 | - id07552 4892 | - id07553 4893 | - id07554 4894 | - id07555 4895 | - id07556 4896 | - id07558 4897 | - id07560 4898 | - id07561 4899 | - id07562 4900 | - id07564 4901 | - id07565 4902 | - id07566 4903 | - id07567 4904 | - id07568 4905 | - id07569 4906 | - id07570 4907 | - id07571 4908 | - id07572 4909 | - id07574 4910 | - id07575 4911 | - id07576 4912 | - id07577 4913 | - id07578 4914 | - id07579 4915 | - id07580 4916 | - id07581 4917 | - id07582 4918 | - id07583 4919 | - id07585 4920 | - id07586 4921 | - id07587 4922 | - id07588 4923 | - id07589 4924 | - id07590 4925 | - id07591 4926 | - id07592 4927 | - id07593 4928 | - id07598 4929 | - id07599 4930 | - id07600 4931 | - id07611 4932 | - id07612 4933 | - id07613 4934 | - id07614 4935 | - id07615 4936 | - id07616 4937 | - id07617 4938 | - id07618 4939 | - id07622 4940 | - id07623 4941 | - id07624 4942 | - id07625 4943 | - id07626 4944 | - id07627 4945 | - id07628 4946 | - id07630 4947 | - id07631 4948 | - id07635 4949 | - id07636 4950 | - id07637 4951 | - id07638 4952 | - id07639 4953 | - id07640 4954 | - id07641 4955 | - id07642 4956 | - id07643 4957 | - id07644 4958 | - id07646 4959 | - id07647 4960 | - id07648 4961 | - id07649 4962 | - id07652 4963 | - id07656 4964 | - id07658 4965 | - id07660 4966 | - id07661 4967 | - id07662 4968 | - id07664 4969 | - id07666 4970 | - id07669 4971 | - id07670 4972 | - id07671 4973 | - id07672 4974 | - id07673 4975 | - id07675 4976 | - id07678 4977 | - id07679 4978 | - id07680 4979 | - id07681 4980 | - id07682 4981 | - id07683 4982 | - id07685 4983 | - id07686 4984 | - id07688 4985 | - id07689 4986 | - id07691 4987 | - id07692 4988 | - id07693 4989 | - id07694 4990 | - id07695 4991 | - id07696 4992 | - id07697 4993 | - id07700 4994 | - id07701 4995 | - id07702 4996 | - id07703 4997 | - id07704 4998 | - id07705 4999 | - id07706 5000 | - id07707 5001 | - id07708 5002 | - id07711 5003 | - id07713 5004 | - id07714 5005 | - id07715 5006 | - id07716 5007 | - id07717 5008 | - id07719 5009 | - id07720 5010 | - id07721 5011 | - id07723 5012 | - id07724 5013 | - id07725 5014 | - id07726 5015 | - id07727 5016 | - id07728 5017 | - id07730 5018 | - id07732 5019 | - id07733 5020 | - id07734 5021 | - id07735 5022 | - id07736 5023 | - id07737 5024 | - id07738 5025 | - id07739 5026 | - id07740 5027 | - id07741 5028 | - id07742 5029 | - id07744 5030 | - id07745 5031 | - id07747 5032 | - id07748 5033 | - id07749 5034 | - id07750 5035 | - id07751 5036 | - id07752 5037 | - id07753 5038 | - id07755 5039 | - id07756 5040 | - id07758 5041 | - id07759 5042 | - id07760 5043 | - id07761 5044 | - id07764 5045 | - id07768 5046 | - id07769 5047 | - id07770 5048 | - id07771 5049 | - id07773 5050 | - id07774 5051 | - id07775 5052 | - id07776 5053 | - id07780 5054 | - id07783 5055 | - id07785 5056 | - id07786 5057 | - id07798 5058 | - id07799 5059 | - id07800 5060 | - id07801 5061 | - id07803 5062 | - id07805 5063 | - id07807 5064 | - id07809 5065 | - id07811 5066 | - id07812 5067 | - id07814 5068 | - id07816 5069 | - id07818 5070 | - id07819 5071 | - id07820 5072 | - id07824 5073 | - id07825 5074 | - id07826 5075 | - id07827 5076 | - id07828 5077 | - id07829 5078 | - id07831 5079 | - id07832 5080 | - id07833 5081 | - id07834 5082 | - id07835 5083 | - id07837 5084 | - id07838 5085 | - id07839 5086 | - id07840 5087 | - id07841 5088 | - id07844 5089 | - id07845 5090 | - id07846 5091 | - id07847 5092 | - id07848 5093 | - id07849 5094 | - id07850 5095 | - id07851 5096 | - id07852 5097 | - id07853 5098 | - id07856 5099 | - id07857 5100 | - id07858 5101 | - id07860 5102 | - id07861 5103 | - id07862 5104 | - id07863 5105 | - id07864 5106 | - id07866 5107 | - id07869 5108 | - id07870 5109 | - id07875 5110 | - id07876 5111 | - id07882 5112 | - id07883 5113 | - id07884 5114 | - id07885 5115 | - id07886 5116 | - id07887 5117 | - id07894 5118 | - id07895 5119 | - id07898 5120 | - id07901 5121 | - id07902 5122 | - id07910 5123 | - id07911 5124 | - id07912 5125 | - id07914 5126 | - id07916 5127 | - id07917 5128 | - id07926 5129 | - id07927 5130 | - id07928 5131 | - id07941 5132 | - id07943 5133 | - id07944 5134 | - id07947 5135 | - id07948 5136 | - id07949 5137 | - id07950 5138 | - id07951 5139 | - id07953 5140 | - id07954 5141 | - id07955 5142 | - id07956 5143 | - id07957 5144 | - id07958 5145 | - id07959 5146 | - id07960 5147 | - id07962 5148 | - id07963 5149 | - id07964 5150 | - id07967 5151 | - id07968 5152 | - id07970 5153 | - id07971 5154 | - id07972 5155 | - id07973 5156 | - id07974 5157 | - id07975 5158 | - id07976 5159 | - id07977 5160 | - id07978 5161 | - id07979 5162 | - id07980 5163 | - id07981 5164 | - id07982 5165 | - id07983 5166 | - id07984 5167 | - id07985 5168 | - id07987 5169 | - id07988 5170 | - id07989 5171 | - id07990 5172 | - id07991 5173 | - id07992 5174 | - id07993 5175 | - id07995 5176 | - id07996 5177 | - id07997 5178 | - id07998 5179 | - id07999 5180 | - id08000 5181 | - id08001 5182 | - id08002 5183 | - id08003 5184 | - id08012 5185 | - id08013 5186 | - id08014 5187 | - id08015 5188 | - id08016 5189 | - id08018 5190 | - id08019 5191 | - id08020 5192 | - id08022 5193 | - id08023 5194 | - id08024 5195 | - id08025 5196 | - id08026 5197 | - id08027 5198 | - id08028 5199 | - id08029 5200 | - id08030 5201 | - id08037 5202 | - id08039 5203 | - id08040 5204 | - id08041 5205 | - id08042 5206 | - id08043 5207 | - id08044 5208 | - id08046 5209 | - id08047 5210 | - id08048 5211 | - id08049 5212 | - id08052 5213 | - id08053 5214 | - id08054 5215 | - id08064 5216 | - id08065 5217 | - id08066 5218 | - id08067 5219 | - id08068 5220 | - id08071 5221 | - id08073 5222 | - id08074 5223 | - id08078 5224 | - id08079 5225 | - id08080 5226 | - id08081 5227 | - id08086 5228 | - id08088 5229 | - id08089 5230 | - id08093 5231 | - id08094 5232 | - id08095 5233 | - id08096 5234 | - id08097 5235 | - id08098 5236 | - id08099 5237 | - id08106 5238 | - id08108 5239 | - id08110 5240 | - id08113 5241 | - id08114 5242 | - id08115 5243 | - id08119 5244 | - id08120 5245 | - id08121 5246 | - id08122 5247 | - id08123 5248 | - id08124 5249 | - id08125 5250 | - id08126 5251 | - id08127 5252 | - id08128 5253 | - id08129 5254 | - id08130 5255 | - id08132 5256 | - id08134 5257 | - id08137 5258 | - id08138 5259 | - id08139 5260 | - id08142 5261 | - id08143 5262 | - id08144 5263 | - id08145 5264 | - id08146 5265 | - id08147 5266 | - id08148 5267 | - id08154 5268 | - id08155 5269 | - id08158 5270 | - id08159 5271 | - id08160 5272 | - id08162 5273 | - id08163 5274 | - id08165 5275 | - id08166 5276 | - id08167 5277 | - id08169 5278 | - id08170 5279 | - id08171 5280 | - id08172 5281 | - id08173 5282 | - id08175 5283 | - id08176 5284 | - id08178 5285 | - id08179 5286 | - id08180 5287 | - id08183 5288 | - id08184 5289 | - id08185 5290 | - id08186 5291 | - id08187 5292 | - id08188 5293 | - id08190 5294 | - id08191 5295 | - id08192 5296 | - id08193 5297 | - id08194 5298 | - id08195 5299 | - id08198 5300 | - id08200 5301 | - id08201 5302 | - id08202 5303 | - id08203 5304 | - id08204 5305 | - id08206 5306 | - id08209 5307 | - id08210 5308 | - id08211 5309 | - id08212 5310 | - id08213 5311 | - id08215 5312 | - id08217 5313 | - id08218 5314 | - id08219 5315 | - id08220 5316 | - id08221 5317 | - id08222 5318 | - id08230 5319 | - id08231 5320 | - id08232 5321 | - id08234 5322 | - id08235 5323 | - id08236 5324 | - id08238 5325 | - id08239 5326 | - id08240 5327 | - id08241 5328 | - id08242 5329 | - id08243 5330 | - id08244 5331 | - id08245 5332 | - id08246 5333 | - id08247 5334 | - id08249 5335 | - id08250 5336 | - id08251 5337 | - id08253 5338 | - id08258 5339 | - id08259 5340 | - id08260 5341 | - id08261 5342 | - id08263 5343 | - id08264 5344 | - id08266 5345 | - id08267 5346 | - id08271 5347 | - id08273 5348 | - id08274 5349 | - id08277 5350 | - id08278 5351 | - id08279 5352 | - id08280 5353 | - id08281 5354 | - id08282 5355 | - id08283 5356 | - id08284 5357 | - id08285 5358 | - id08286 5359 | - id08287 5360 | - id08288 5361 | - id08289 5362 | - id08290 5363 | - id08291 5364 | - id08292 5365 | - id08293 5366 | - id08294 5367 | - id08295 5368 | - id08297 5369 | - id08298 5370 | - id08299 5371 | - id08300 5372 | - id08302 5373 | - id08303 5374 | - id08305 5375 | - id08306 5376 | - id08307 5377 | - id08309 5378 | - id08311 5379 | - id08312 5380 | - id08313 5381 | - id08314 5382 | - id08315 5383 | - id08317 5384 | - id08318 5385 | - id08320 5386 | - id08321 5387 | - id08323 5388 | - id08324 5389 | - id08327 5390 | - id08329 5391 | - id08330 5392 | - id08331 5393 | - id08333 5394 | - id08334 5395 | - id08335 5396 | - id08337 5397 | - id08338 5398 | - id08339 5399 | - id08340 5400 | - id08343 5401 | - id08344 5402 | - id08347 5403 | - id08348 5404 | - id08349 5405 | - id08350 5406 | - id08351 5407 | - id08352 5408 | - id08353 5409 | - id08354 5410 | - id08355 5411 | - id08356 5412 | - id08358 5413 | - id08360 5414 | - id08362 5415 | - id08364 5416 | - id08365 5417 | - id08368 5418 | - id08370 5419 | - id08372 5420 | - id08373 5421 | - id08375 5422 | - id08376 5423 | - id08377 5424 | - id08380 5425 | - id08381 5426 | - id08382 5427 | - id08394 5428 | - id08395 5429 | - id08396 5430 | - id08397 5431 | - id08398 5432 | - id08399 5433 | - id08400 5434 | - id08402 5435 | - id08403 5436 | - id08404 5437 | - id08405 5438 | - id08406 5439 | - id08408 5440 | - id08411 5441 | - id08414 5442 | - id08415 5443 | - id08416 5444 | - id08417 5445 | - id08418 5446 | - id08419 5447 | - id08420 5448 | - id08421 5449 | - id08422 5450 | - id08423 5451 | - id08424 5452 | - id08426 5453 | - id08432 5454 | - id08433 5455 | - id08434 5456 | - id08437 5457 | - id08438 5458 | - id08440 5459 | - id08441 5460 | - id08442 5461 | - id08443 5462 | - id08445 5463 | - id08446 5464 | - id08451 5465 | - id08453 5466 | - id08454 5467 | - id08455 5468 | - id08457 5469 | - id08458 5470 | - id08459 5471 | - id08462 5472 | - id08463 5473 | - id08464 5474 | - id08465 5475 | - id08467 5476 | - id08468 5477 | - id08470 5478 | - id08471 5479 | - id08472 5480 | - id08474 5481 | - id08475 5482 | - id08476 5483 | - id08477 5484 | - id08478 5485 | - id08480 5486 | - id08481 5487 | - id08482 5488 | - id08483 5489 | - id08489 5490 | - id08490 5491 | - id08492 5492 | - id08494 5493 | - id08496 5494 | - id08497 5495 | - id08498 5496 | - id08499 5497 | - id08500 5498 | - id08501 5499 | - id08502 5500 | - id08510 5501 | - id08511 5502 | - id08512 5503 | - id08513 5504 | - id08515 5505 | - id08516 5506 | - id08517 5507 | - id08518 5508 | - id08520 5509 | - id08521 5510 | - id08522 5511 | - id08523 5512 | - id08524 5513 | - id08525 5514 | - id08527 5515 | - id08528 5516 | - id08529 5517 | - id08530 5518 | - id08531 5519 | - id08532 5520 | - id08533 5521 | - id08534 5522 | - id08535 5523 | - id08536 5524 | - id08538 5525 | - id08540 5526 | - id08543 5527 | - id08544 5528 | - id08545 5529 | - id08546 5530 | - id08547 5531 | - id08549 5532 | - id08551 5533 | - id08553 5534 | - id08554 5535 | - id08555 5536 | - id08556 5537 | - id08563 5538 | - id08564 5539 | - id08565 5540 | - id08566 5541 | - id08567 5542 | - id08568 5543 | - id08569 5544 | - id08570 5545 | - id08572 5546 | - id08573 5547 | - id08574 5548 | - id08577 5549 | - id08578 5550 | - id08580 5551 | - id08581 5552 | - id08583 5553 | - id08584 5554 | - id08585 5555 | - id08586 5556 | - id08587 5557 | - id08588 5558 | - id08589 5559 | - id08590 5560 | - id08592 5561 | - id08593 5562 | - id08595 5563 | - id08597 5564 | - id08598 5565 | - id08599 5566 | - id08600 5567 | - id08601 5568 | - id08602 5569 | - id08603 5570 | - id08604 5571 | - id08605 5572 | - id08606 5573 | - id08607 5574 | - id08608 5575 | - id08609 5576 | - id08610 5577 | - id08611 5578 | - id08612 5579 | - id08613 5580 | - id08614 5581 | - id08615 5582 | - id08616 5583 | - id08617 5584 | - id08618 5585 | - id08622 5586 | - id08623 5587 | - id08624 5588 | - id08625 5589 | - id08626 5590 | - id08627 5591 | - id08628 5592 | - id08629 5593 | - id08633 5594 | - id08634 5595 | - id08635 5596 | - id08637 5597 | - id08638 5598 | - id08639 5599 | - id08640 5600 | - id08641 5601 | - id08642 5602 | - id08643 5603 | - id08645 5604 | - id08646 5605 | - id08647 5606 | - id08648 5607 | - id08649 5608 | - id08650 5609 | - id08651 5610 | - id08652 5611 | - id08653 5612 | - id08655 5613 | - id08656 5614 | - id08657 5615 | - id08658 5616 | - id08659 5617 | - id08660 5618 | - id08661 5619 | - id08663 5620 | - id08665 5621 | - id08666 5622 | - id08667 5623 | - id08668 5624 | - id08669 5625 | - id08670 5626 | - id08671 5627 | - id08672 5628 | - id08674 5629 | - id08676 5630 | - id08677 5631 | - id08678 5632 | - id08679 5633 | - id08680 5634 | - id08681 5635 | - id08683 5636 | - id08684 5637 | - id08685 5638 | - id08686 5639 | - id08687 5640 | - id08688 5641 | - id08689 5642 | - id08691 5643 | - id08692 5644 | - id08693 5645 | - id08694 5646 | - id08695 5647 | - id08698 5648 | - id08699 5649 | - id08700 5650 | - id08702 5651 | - id08703 5652 | - id08704 5653 | - id08705 5654 | - id08706 5655 | - id08707 5656 | - id08708 5657 | - id08709 5658 | - id08710 5659 | - id08711 5660 | - id08713 5661 | - id08714 5662 | - id08715 5663 | - id08716 5664 | - id08719 5665 | - id08721 5666 | - id08722 5667 | - id08723 5668 | - id08724 5669 | - id08725 5670 | - id08726 5671 | - id08727 5672 | - id08728 5673 | - id08730 5674 | - id08731 5675 | - id08732 5676 | - id08733 5677 | - id08736 5678 | - id08737 5679 | - id08738 5680 | - id08739 5681 | - id08740 5682 | - id08741 5683 | - id08742 5684 | - id08743 5685 | - id08745 5686 | - id08746 5687 | - id08747 5688 | - id08748 5689 | - id08749 5690 | - id08750 5691 | - id08751 5692 | - id08754 5693 | - id08755 5694 | - id08756 5695 | - id08757 5696 | - id08758 5697 | - id08760 5698 | - id08761 5699 | - id08762 5700 | - id08764 5701 | - id08765 5702 | - id08766 5703 | - id08768 5704 | - id08769 5705 | - id08771 5706 | - id08773 5707 | - id08774 5708 | - id08775 5709 | - id08776 5710 | - id08781 5711 | - id08782 5712 | - id08783 5713 | - id08784 5714 | - id08785 5715 | - id08786 5716 | - id08788 5717 | - id08789 5718 | - id08790 5719 | - id08792 5720 | - id08794 5721 | - id08795 5722 | - id08797 5723 | - id08799 5724 | - id08800 5725 | - id08801 5726 | - id08802 5727 | - id08803 5728 | - id08804 5729 | - id08805 5730 | - id08806 5731 | - id08808 5732 | - id08809 5733 | - id08810 5734 | - id08811 5735 | - id08819 5736 | - id08820 5737 | - id08821 5738 | - id08822 5739 | - id08823 5740 | - id08826 5741 | - id08830 5742 | - id08831 5743 | - id08832 5744 | - id08834 5745 | - id08837 5746 | - id08839 5747 | - id08840 5748 | - id08850 5749 | - id08853 5750 | - id08860 5751 | - id08862 5752 | - id08863 5753 | - id08864 5754 | - id08866 5755 | - id08868 5756 | - id08870 5757 | - id08871 5758 | - id08872 5759 | - id08873 5760 | - id08875 5761 | - id08876 5762 | - id08877 5763 | - id08878 5764 | - id08879 5765 | - id08881 5766 | - id08882 5767 | - id08883 5768 | - id08884 5769 | - id08885 5770 | - id08887 5771 | - id08890 5772 | - id08891 5773 | - id08892 5774 | - id08896 5775 | - id08898 5776 | - id08900 5777 | - id08901 5778 | - id08902 5779 | - id08903 5780 | - id08904 5781 | - id08906 5782 | - id08907 5783 | - id08909 5784 | - id08910 5785 | - id08912 5786 | - id08913 5787 | - id08915 5788 | - id08916 5789 | - id08917 5790 | - id08919 5791 | - id08920 5792 | - id08923 5793 | - id08925 5794 | - id08926 5795 | - id08927 5796 | - id08928 5797 | - id08929 5798 | - id08930 5799 | - id08931 5800 | - id08933 5801 | - id08936 5802 | - id08937 5803 | - id08938 5804 | - id08940 5805 | - id08941 5806 | - id08942 5807 | - id08944 5808 | - id08945 5809 | - id08950 5810 | - id08953 5811 | - id08954 5812 | - id08955 5813 | - id08967 5814 | - id08968 5815 | - id08969 5816 | - id08971 5817 | - id08972 5818 | - id08973 5819 | - id08974 5820 | - id08975 5821 | - id08976 5822 | - id08977 5823 | - id08980 5824 | - id08981 5825 | - id08982 5826 | - id08983 5827 | - id08984 5828 | - id08992 5829 | - id08993 5830 | - id08994 5831 | - id08996 5832 | - id08997 5833 | - id08998 5834 | - id08999 5835 | - id09000 5836 | - id09003 5837 | - id09004 5838 | - id09005 5839 | - id09006 5840 | - id09007 5841 | - id09008 5842 | - id09009 5843 | - id09010 5844 | - id09012 5845 | - id09013 5846 | - id09015 5847 | - id09016 5848 | - id09018 5849 | - id09020 5850 | - id09021 5851 | - id09022 5852 | - id09023 5853 | - id09024 5854 | - id09025 5855 | - id09027 5856 | - id09028 5857 | - id09029 5858 | - id09030 5859 | - id09031 5860 | - id09032 5861 | - id09033 5862 | - id09034 5863 | - id09035 5864 | - id09036 5865 | - id09038 5866 | - id09039 5867 | - id09040 5868 | - id09044 5869 | - id09046 5870 | - id09047 5871 | - id09048 5872 | - id09049 5873 | - id09051 5874 | - id09052 5875 | - id09053 5876 | - id09054 5877 | - id09055 5878 | - id09056 5879 | - id09057 5880 | - id09059 5881 | - id09061 5882 | - id09062 5883 | - id09063 5884 | - id09064 5885 | - id09065 5886 | - id09066 5887 | - id09067 5888 | - id09068 5889 | - id09069 5890 | - id09070 5891 | - id09071 5892 | - id09074 5893 | - id09075 5894 | - id09076 5895 | - id09078 5896 | - id09079 5897 | - id09081 5898 | - id09082 5899 | - id09083 5900 | - id09087 5901 | - id09088 5902 | - id09091 5903 | - id09092 5904 | - id09094 5905 | - id09095 5906 | - id09098 5907 | - id09099 5908 | - id09100 5909 | - id09108 5910 | - id09109 5911 | - id09110 5912 | - id09111 5913 | - id09112 5914 | - id09116 5915 | - id09119 5916 | - id09121 5917 | - id09123 5918 | - id09124 5919 | - id09125 5920 | - id09138 5921 | - id09139 5922 | - id09143 5923 | - id09144 5924 | - id09145 5925 | - id09146 5926 | - id09147 5927 | - id09149 5928 | - id09152 5929 | - id09153 5930 | - id09154 5931 | - id09155 5932 | - id09156 5933 | - id09159 5934 | - id09160 5935 | - id09161 5936 | - id09162 5937 | - id09163 5938 | - id09164 5939 | - id09171 5940 | - id09172 5941 | - id09174 5942 | - id09175 5943 | - id09179 5944 | - id09181 5945 | - id09185 5946 | - id09186 5947 | - id09187 5948 | - id09188 5949 | - id09189 5950 | - id09190 5951 | - id09192 5952 | - id09193 5953 | - id09194 5954 | - id09197 5955 | - id09198 5956 | - id09199 5957 | - id09200 5958 | - id09201 5959 | - id09202 5960 | - id09203 5961 | - id09204 5962 | - id09205 5963 | - id09207 5964 | - id09210 5965 | - id09211 5966 | - id09212 5967 | - id09213 5968 | - id09214 5969 | - id09215 5970 | - id09216 5971 | - id09217 5972 | - id09218 5973 | - id09220 5974 | - id09222 5975 | - id09224 5976 | - id09225 5977 | - id09226 5978 | - id09227 5979 | - id09229 5980 | - id09230 5981 | - id09231 5982 | - id09232 5983 | - id09234 5984 | - id09235 5985 | - id09236 5986 | - id09237 5987 | - id09238 5988 | - id09240 5989 | - id09242 5990 | - id09243 5991 | - id09245 5992 | - id09252 5993 | - id09255 5994 | - id09262 5995 | - id09263 5996 | - id09268 5997 | - id09269 5998 | - id09271 5999 | - id09272 6000 | -------------------------------------------------------------------------------- /models/AAM/train/VoxCeleb.SpeakerVerification.VoxCeleb2.train/validate_equal_error_rate/VoxCeleb.SpeakerVerification.VoxCeleb1_X.development/params.yml: -------------------------------------------------------------------------------- 1 | epoch: 560 2 | equal_error_rate: 0.038853658536585386 3 | -------------------------------------------------------------------------------- /models/AAM/train/VoxCeleb.SpeakerVerification.VoxCeleb2.train/weights/0560.pt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/juanmc2005/SpeakerEmbeddingLossComparison/6ab248a59673523e797950dc9c8517a5457c6cd3/models/AAM/train/VoxCeleb.SpeakerVerification.VoxCeleb2.train/weights/0560.pt -------------------------------------------------------------------------------- /reproduce.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# A Comparison of Metric Learning Loss Functions for End-to-End Speaker Verification" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "This notebook contains the code to reproduce the equal error rate of the additive angular margin loss model from the paper.\n", 15 | "\n", 16 | "Before you begin, make sure you have installed [pyannote-audio](https://github.com/pyannote/pyannote-audio) and [pyannote.db.voxceleb](https://github.com/pyannote/pyannote-db-voxceleb)." 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "metadata": {}, 22 | "source": [ 23 | "## Preparation" 24 | ] 25 | }, 26 | { 27 | "cell_type": "markdown", 28 | "metadata": {}, 29 | "source": [ 30 | "### Dataset" 31 | ] 32 | }, 33 | { 34 | "cell_type": "markdown", 35 | "metadata": {}, 36 | "source": [ 37 | "First of all, we apply the pretrained model to generate `VoxCeleb1` embeddings. Audio chunk duration is 3s as indicated in `config.yml`, and the step of the sliding window is ~100ms (3 * 0.0333).\n", 38 | "\n", 39 | "To do so, you need to execute the following commands in your terminal:" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": null, 45 | "metadata": {}, 46 | "outputs": [], 47 | "source": [ 48 | "$ pyannote-audio emb apply --gpu --step=0.0333 --batch=128 --subset=test models/AAM/train/VoxCeleb.SpeakerVerification.VoxCeleb2.train/validate_equal_error_rate/VoxCeleb.SpeakerVerification.VoxCeleb1_X.development VoxCeleb.SpeakerVerification.VoxCeleb1_X\n", 49 | "$ pyannote-audio emb apply --gpu --step=0.0333 --batch=128 --subset=train models/AAM/train/VoxCeleb.SpeakerVerification.VoxCeleb2.train/validate_equal_error_rate/VoxCeleb.SpeakerVerification.VoxCeleb1_X.development VoxCeleb.SpeakerVerification.VoxCeleb1_X" 50 | ] 51 | }, 52 | { 53 | "cell_type": "markdown", 54 | "metadata": {}, 55 | "source": [ 56 | "Notice that you need to remove `--gpu` if your machine doesn't have one. These commands may take some time to execute, especially the second one. However, once all embeddings are calculated, you will be free to run this notebook many times and modify it without waiting for hours to obtain results.\n", 57 | "\n", 58 | "In this notebook we will be using the `Test` subset to calculate the EER, and `Train` to normalize similarity scores with adaptive s-norm.\n", 59 | "\n", 60 | "If you want to know more about the `apply` method, you can check out [pyannote's tutorials](https://github.com/pyannote/pyannote-audio/tree/develop/tutorials/models/speaker_embedding#application)" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "metadata": {}, 66 | "source": [ 67 | "### Code" 68 | ] 69 | }, 70 | { 71 | "cell_type": "markdown", 72 | "metadata": {}, 73 | "source": [ 74 | "Make sure all the needed libraries are installed:\n", 75 | "- `numpy` for obvious reasons\n", 76 | "- `xarray` to facilitate score normalization\n", 77 | "- `feerci` to calculate EER and its confidence interval\n", 78 | "- `pyannote.audio` to use pretrained models\n", 79 | "- `pyannote.database` to access `VoxCeleb1`\n", 80 | "- `tqdm` to show nice progress bars" 81 | ] 82 | }, 83 | { 84 | "cell_type": "code", 85 | "execution_count": 1, 86 | "metadata": {}, 87 | "outputs": [], 88 | "source": [ 89 | "import numpy as np\n", 90 | "from xarray import DataArray\n", 91 | "from feerci import feerci\n", 92 | "from pyannote.core.utils.distance import cdist\n", 93 | "from pyannote.audio.features import Precomputed\n", 94 | "from pyannote.audio.applications.speaker_embedding import SpeakerEmbedding\n", 95 | "from pyannote.database import get_protocol\n", 96 | "from tqdm import tqdm" 97 | ] 98 | }, 99 | { 100 | "cell_type": "markdown", 101 | "metadata": {}, 102 | "source": [ 103 | "We also initialize the database with the preprocessors needed, and we define some useful functions." 104 | ] 105 | }, 106 | { 107 | "cell_type": "code", 108 | "execution_count": 2, 109 | "metadata": {}, 110 | "outputs": [], 111 | "source": [ 112 | "# We use the VoxCeleb1_X protocol, with a train and dev set\n", 113 | "# resulting from splitting the original development set\n", 114 | "protocol = get_protocol('VoxCeleb.SpeakerVerification.VoxCeleb1_X')\n", 115 | "\n", 116 | "\n", 117 | "# A function to crop the embeddings from a file\n", 118 | "def get_embedding(file, pretrained, mean=False):\n", 119 | " emb = []\n", 120 | " for f in file.files():\n", 121 | " if 'try_with' in f:\n", 122 | " segments = f['try_with']\n", 123 | " else:\n", 124 | " segments = f['annotation'].get_timeline()\n", 125 | " for segment in segments:\n", 126 | " for mode in ['center', 'loose']:\n", 127 | " e = pretrained.crop(f, segment, mode=mode)\n", 128 | " if len(e) > 0:\n", 129 | " break\n", 130 | " emb.append(e)\n", 131 | " emb = np.vstack(emb)\n", 132 | " if mean:\n", 133 | " emb = np.mean(emb, axis=0, keepdims=True)\n", 134 | " return emb\n", 135 | "\n", 136 | "\n", 137 | "# A function to calculate the EER on a subset of VoxCeleb1_X\n", 138 | "def run_experiment(distance, subset):\n", 139 | " total = 37720 if subset == 'test' else None\n", 140 | " y_pred, y_true = [], []\n", 141 | " for trial in tqdm(getattr(protocol, f'{subset}_trial')(), total=total):\n", 142 | " file1 = trial['file1']\n", 143 | " hash1 = get_hash(file1)\n", 144 | " file2 = trial['file2']\n", 145 | " hash2 = get_hash(file2)\n", 146 | " y_pred.append(distance.data[index1[hash1], index2[hash2]])\n", 147 | " y_true.append(trial['reference'])\n", 148 | " y_pred = np.array(y_pred)\n", 149 | " y_true = np.array(y_true)\n", 150 | " eer, ci_lower, ci_upper, _ = feerci(-y_pred[y_true == 0],\n", 151 | " -y_pred[y_true == 1],\n", 152 | " is_sorted=False)\n", 153 | " return {\n", 154 | " 'eer': eer,\n", 155 | " 'ci_lower': ci_lower,\n", 156 | " 'ci_upper': ci_upper,\n", 157 | " 'y_true': y_true,\n", 158 | " 'y_pred': y_pred}" 159 | ] 160 | }, 161 | { 162 | "cell_type": "markdown", 163 | "metadata": {}, 164 | "source": [ 165 | "## Loading the Pretrained Model" 166 | ] 167 | }, 168 | { 169 | "cell_type": "code", 170 | "execution_count": 3, 171 | "metadata": {}, 172 | "outputs": [ 173 | { 174 | "name": "stdout", 175 | "output_type": "stream", 176 | "text": [ 177 | "Embeddings of 3s duration and of dimension 512, extracted every 99.9ms\n" 178 | ] 179 | } 180 | ], 181 | "source": [ 182 | "# Load the precomputed embeddings calculated at the beginning of the notebook\n", 183 | "model = Precomputed(\n", 184 | " 'models/AAM/train/VoxCeleb.SpeakerVerification.VoxCeleb2.train/validate_equal_error_rate/'\n", 185 | " 'VoxCeleb.SpeakerVerification.VoxCeleb1_X.development/apply/0560/',\n", 186 | " use_memmap=False)\n", 187 | "\n", 188 | "print(f'Embeddings of {model.sliding_window.duration:g}s duration and of dimension {model.dimension:d}, '\n", 189 | " f'extracted every {1000 * model.sliding_window.step:g}ms')" 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "metadata": {}, 195 | "source": [ 196 | "## Evaluating with Raw Distances" 197 | ] 198 | }, 199 | { 200 | "cell_type": "markdown", 201 | "metadata": {}, 202 | "source": [ 203 | "### Get embeddings" 204 | ] 205 | }, 206 | { 207 | "cell_type": "code", 208 | "execution_count": 4, 209 | "metadata": {}, 210 | "outputs": [ 211 | { 212 | "name": "stderr", 213 | "output_type": "stream", 214 | "text": [ 215 | "100%|██████████| 37720/37720 [00:30<00:00, 1218.67it/s]\n" 216 | ] 217 | } 218 | ], 219 | "source": [ 220 | "get_hash = lambda file: SpeakerEmbedding.get_hash(file)\n", 221 | "\n", 222 | "# hash to embedding mapping\n", 223 | "cache1 = dict()\n", 224 | "cache2 = dict()\n", 225 | "\n", 226 | "# hash to index mapping\n", 227 | "index1 = dict()\n", 228 | "index2 = dict()\n", 229 | "\n", 230 | "n_file1 = 0\n", 231 | "n_file2 = 0\n", 232 | "\n", 233 | "# Get embeddings for every trial in Test\n", 234 | "for trial in tqdm(protocol.test_trial(), total=37720):\n", 235 | " \n", 236 | " file1 = trial['file1']\n", 237 | " hash1 = get_hash(file1)\n", 238 | " if hash1 not in cache1:\n", 239 | " cache1[hash1] = get_embedding(file1, model, mean=True)\n", 240 | " index1[hash1] = n_file1\n", 241 | " n_file1 += 1\n", 242 | " \n", 243 | " file2 = trial['file2']\n", 244 | " hash2 = get_hash(file2)\n", 245 | " if hash2 not in cache2:\n", 246 | " cache2[hash2] = get_embedding(file2, model, mean=True)\n", 247 | " index2[hash2] = n_file2\n", 248 | " n_file2 += 1\n", 249 | "\n", 250 | "hashes1 = list(cache1.keys())\n", 251 | "hashes2 = list(cache2.keys())\n", 252 | "emb1 = np.vstack(list(cache1.values()))\n", 253 | "emb2 = np.vstack(list(cache2.values()))" 254 | ] 255 | }, 256 | { 257 | "cell_type": "markdown", 258 | "metadata": {}, 259 | "source": [ 260 | "### Calculate cosine distances" 261 | ] 262 | }, 263 | { 264 | "cell_type": "code", 265 | "execution_count": 5, 266 | "metadata": {}, 267 | "outputs": [], 268 | "source": [ 269 | "distance = DataArray(\n", 270 | " cdist(emb1, emb2, metric='cosine'),\n", 271 | " dims=('file1', 'file2'),\n", 272 | " coords=(hashes1, hashes2))" 273 | ] 274 | }, 275 | { 276 | "cell_type": "markdown", 277 | "metadata": {}, 278 | "source": [ 279 | "### Calculate EER on VoxCeleb1 Test" 280 | ] 281 | }, 282 | { 283 | "cell_type": "code", 284 | "execution_count": 6, 285 | "metadata": {}, 286 | "outputs": [ 287 | { 288 | "name": "stderr", 289 | "output_type": "stream", 290 | "text": [ 291 | "100%|██████████| 37720/37720 [00:14<00:00, 2603.12it/s]\n" 292 | ] 293 | }, 294 | { 295 | "name": "stdout", 296 | "output_type": "stream", 297 | "text": [ 298 | "EER with raw distances: 3.94 in [3.74, 4.14]\n" 299 | ] 300 | } 301 | ], 302 | "source": [ 303 | "raw_results = run_experiment(distance, 'test')\n", 304 | "print(f\"EER with raw distances: {100 * raw_results['eer']:.2f} in \"\n", 305 | " f\"[{100 * raw_results['ci_lower']:.2f}, {100 * raw_results['ci_upper']:.2f}]\")" 306 | ] 307 | }, 308 | { 309 | "cell_type": "markdown", 310 | "metadata": {}, 311 | "source": [ 312 | "## Evaluating with adaptive s-norm" 313 | ] 314 | }, 315 | { 316 | "cell_type": "markdown", 317 | "metadata": {}, 318 | "source": [ 319 | "Here we improve the above EER with adaptive s-norm score normalization." 320 | ] 321 | }, 322 | { 323 | "cell_type": "markdown", 324 | "metadata": {}, 325 | "source": [ 326 | "### Create a cohort set" 327 | ] 328 | }, 329 | { 330 | "cell_type": "code", 331 | "execution_count": 7, 332 | "metadata": {}, 333 | "outputs": [ 334 | { 335 | "name": "stderr", 336 | "output_type": "stream", 337 | "text": [ 338 | "100%|██████████| 143506/143506 [04:52<00:00, 491.02it/s]\n" 339 | ] 340 | } 341 | ], 342 | "source": [ 343 | "# Get cohort embeddings from VoxCeleb1_X.train\n", 344 | "cohort_embedding = dict()\n", 345 | "for cohort_file in tqdm(protocol.train(), total=143506):\n", 346 | " speaker = cohort_file['annotation'].argmax()\n", 347 | " embedding = get_embedding(cohort_file, model, mean=False)\n", 348 | " cohort_embedding.setdefault(speaker, []).append(embedding)\n", 349 | "\n", 350 | "# The cohort consists of the mean embedding for each speaker\n", 351 | "cohort_speakers = list(cohort_embedding.keys())\n", 352 | "cohort = np.vstack([np.mean(np.vstack(cohort_embedding[speaker]), axis=0, keepdims=True) \n", 353 | " for speaker in cohort_speakers])" 354 | ] 355 | }, 356 | { 357 | "cell_type": "markdown", 358 | "metadata": {}, 359 | "source": [ 360 | "### Calculate raw trial scores" 361 | ] 362 | }, 363 | { 364 | "cell_type": "code", 365 | "execution_count": 8, 366 | "metadata": {}, 367 | "outputs": [], 368 | "source": [ 369 | "# Calculate the distances between each trial embedding (file1 and file2) and the cohort\n", 370 | "distance1 = DataArray(\n", 371 | " cdist(emb1, cohort, metric='cosine'),\n", 372 | " dims=('file1', 'cohort'),\n", 373 | " coords=(hashes1, cohort_speakers))\n", 374 | "\n", 375 | "distance2 = DataArray(\n", 376 | " cdist(emb2, cohort, metric='cosine'),\n", 377 | " dims=('file2', 'cohort'),\n", 378 | " coords=(hashes2, cohort_speakers))" 379 | ] 380 | }, 381 | { 382 | "cell_type": "markdown", 383 | "metadata": {}, 384 | "source": [ 385 | "### Normalize scores w.r.t the N most similar cohort embeddings\n", 386 | "\n", 387 | "N=400 for us. We have previously tuned this value on `VoxCeleb1 Dev`" 388 | ] 389 | }, 390 | { 391 | "cell_type": "code", 392 | "execution_count": 9, 393 | "metadata": {}, 394 | "outputs": [], 395 | "source": [ 396 | "# This is our N\n", 397 | "COHORT_SIZE = 400\n", 398 | "\n", 399 | "# Calculate mean and std of N most similar cohort embeddings for file1\n", 400 | "data1 = np.partition(distance1.data, COHORT_SIZE)[:, :COHORT_SIZE]\n", 401 | "mz = np.mean(data1, axis=1) \n", 402 | "sz = np.std(data1, axis=1)\n", 403 | "mz = DataArray(mz, dims=('file1',), coords=(hashes1,))\n", 404 | "sz = DataArray(sz, dims=('file1',), coords=(hashes1,))\n", 405 | "\n", 406 | "# Calculate mean and std of N most similar cohort embeddings for file2\n", 407 | "data2 = np.partition(distance2.data, COHORT_SIZE)[:, :COHORT_SIZE]\n", 408 | "mt = np.mean(data2, axis=1) \n", 409 | "st = np.std(data2, axis=1)\n", 410 | "mt = DataArray(mt, dims=('file2',), coords=(hashes2,))\n", 411 | "st = DataArray(st, dims=('file2',), coords=(hashes2,))\n", 412 | "\n", 413 | "# Normalize\n", 414 | "distance_z = (distance - mz) / sz\n", 415 | "distance_t = (distance - mt) / st\n", 416 | "distance_s = 0.5 * (distance_z + distance_t)" 417 | ] 418 | }, 419 | { 420 | "cell_type": "markdown", 421 | "metadata": {}, 422 | "source": [ 423 | "### Calculate EER on VoxCeleb1 Test" 424 | ] 425 | }, 426 | { 427 | "cell_type": "code", 428 | "execution_count": 10, 429 | "metadata": {}, 430 | "outputs": [ 431 | { 432 | "name": "stderr", 433 | "output_type": "stream", 434 | "text": [ 435 | "100%|██████████| 37720/37720 [00:13<00:00, 2835.68it/s]\n" 436 | ] 437 | }, 438 | { 439 | "name": "stdout", 440 | "output_type": "stream", 441 | "text": [ 442 | "EER with adaptive s-norm: 3.56 in[3.35, 3.79]\n" 443 | ] 444 | } 445 | ], 446 | "source": [ 447 | "# Calculate the DET curve on test and print the EER value\n", 448 | "ada_snorm_results = run_experiment(distance_s, 'test')\n", 449 | "print(f\"EER with adaptive s-norm: {100 * ada_snorm_results['eer']:.2f} in \"\n", 450 | " f\"[{100 * ada_snorm_results['ci_lower']:.2f}, {100 * ada_snorm_results['ci_upper']:.2f}]\")" 451 | ] 452 | }, 453 | { 454 | "cell_type": "markdown", 455 | "metadata": {}, 456 | "source": [ 457 | "That's all! If you have any questions or suggestions, feel free to open an issue." 458 | ] 459 | }, 460 | { 461 | "cell_type": "code", 462 | "execution_count": null, 463 | "metadata": {}, 464 | "outputs": [], 465 | "source": [] 466 | } 467 | ], 468 | "metadata": { 469 | "kernelspec": { 470 | "display_name": "Python 3", 471 | "language": "python", 472 | "name": "python3" 473 | }, 474 | "language_info": { 475 | "codemirror_mode": { 476 | "name": "ipython", 477 | "version": 3 478 | }, 479 | "file_extension": ".py", 480 | "mimetype": "text/x-python", 481 | "name": "python", 482 | "nbconvert_exporter": "python", 483 | "pygments_lexer": "ipython3", 484 | "version": "3.7.6" 485 | } 486 | }, 487 | "nbformat": 4, 488 | "nbformat_minor": 4 489 | } 490 | --------------------------------------------------------------------------------