├── .gitignore ├── Neural Net Arch Genealogy.txt ├── Neural_Net_Arch_Genealogy.dot ├── Neural_Net_Arch_Genealogy.png ├── Neural_Net_Arch_Genealogy.svg ├── README.md ├── txt2dot.py └── txt2md.py /.gitignore: -------------------------------------------------------------------------------- 1 | .idea 2 | 3 | -------------------------------------------------------------------------------- /Neural Net Arch Genealogy.txt: -------------------------------------------------------------------------------- 1 | Neural Net Arch Genealogy 2 | Reinforcement Learning Algorithms 3 | [A3C, '16.02.06](https://arxiv.org/abs/1602.01783) 4 | [DARLA, '17.07.26](https://arxiv.org/pdf/1707.08475.pdf) 5 | [ACTKR, '17.08.17](https://arxiv.org/pdf/1708.05144.pdf) 6 | [c51, '17.10.27](https://arxiv.org/pdf/1710.10044.pdf) 7 | CNN 8 | [AlexNet, '12.12](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf) 9 | [VggNet, '14.09](https://arxiv.org/pdf/1409.1556.pdf) 10 | [GoogLeNet, '14.09](https://arxiv.org/abs/1409.4842) 11 | [ResNet, '15.12](https://arxiv.org/pdf/1512.03385v1.pdf) 12 | [DenseNet, '16.08](https://arxiv.org/pdf/1608.06993.pdf) 13 | [SENet: Squeeze-and-Excitation Networks, '17.09](https://arxiv.org/abs/1709.01507) 14 | Object Detection 15 | [R-CNN](https://arxiv.org/pdf/1311.2524.pdf) 16 | [Fast R-CNN](https://arxiv.org/pdf/1504.08083.pdf) 17 | [Faster R-CNN](https://arxiv.org/pdf/1506.01497.pdf) 18 | [Mask R-CNN](https://arxiv.org/pdf/1703.06870.pdf) 19 | [YOLO](https://arxiv.org/pdf/1506.02640.pdf) 20 | [SSD](https://arxiv.org/pdf/1512.02325.pdf) 21 | [R-FCN](https://arxiv.org/pdf/1605.06409.pdf) 22 | Semantic Segmentation 23 | [FCN](https://arxiv.org/pdf/1411.4038.pdf) 24 | [DeconvNet](https://arxiv.org/pdf/1505.04366.pdf) 25 | [DeepLab](https://arxiv.org/pdf/1606.00915.pdf) 26 | [U-Net](https://arxiv.org/pdf/1505.04597.pdf) 27 | Super-resolution 28 | [MemNet](https://arxiv.org/abs/1708.02209) 29 | [FSRCNN](https://arxiv.org/1608.00367) 30 | [SRCNN](https://arxiv.org/abs/1501.00092) 31 | [VDSR](https://arxiv.org/abs/1511.04587) 32 | [DRCN](https://arxiv.org/abs/1511.04491) 33 | [LabSRN](https://arxiv.org/abs/1704.03915) 34 | [EDSR](https://arxiv.org/abs/1707.02921) 35 | TTS 36 | [Wavenet, '16.09.12](https://arxiv.org/abs/1609.03499) 37 | Generative Models 38 | Autoregressive models 39 | [MADE, '15.02.12](https://arxiv.org/pdf/1502.03509.pdf) 40 | [PixelRNN, '16.01.25](https://arxiv.org/pdf/1601.06759.pdf) 41 | [NADE, '16.05.07](https://arxiv.org/pdf/1605.02226.pdf) 42 | [PixelCNN, '16.06.16](https://arxiv.org/pdf/1606.05328.pdf) 43 | [PixelCNN++, '17.01.19](https://arxiv.org/pdf/1701.05517.pdf) 44 | Latent variable models 45 | [VAE, '13.12.20](https://arxiv.org/pdf/1312.6114.pdf) 46 | [CVAE, '14.06.20](https://arxiv.org/pdf/1406.5298.pdf) 47 | [AAE, '15.11.18](https://arxiv.org/pdf/1511.05644.pdf) 48 | [AVB, '17.01.17](https://arxiv.org/pdf/1701.04722.pdf) 49 | [VQ-VAE, '17.11.2](https://arxiv.org/abs/1711.00937) 50 | [GAN, '14.06.10](https://arxiv.org/pdf/1406.2661.pdf) 51 | Variants 52 | [CGAN, '14.11.06](https://arxiv.org/pdf/1411.1784.pdf) 53 | [DCGAN, '15.11.19](https://arxiv.org/pdf/1511.06434.pdf) 54 | [infoGAN, '16.06.12](https://arxiv.org/pdf/1704.00028.pdf) 55 | [EBGAN, '16.09.11](https://arxiv.org/pdf/1609.03126.pdf) 56 | [ACGAN, '16.10.30](https://arxiv.org/pdf/1610.09585.pdf) 57 | [WGAN, '17.01.26](https://arxiv.org/pdf/1701.07875.pdf) 58 | [BEGAN, '17.02.27](https://arxiv.org/pdf/1702.08431.pdf) 59 | [WGAN-GP, '17.03.31](https://arxiv.org/pdf/1704.00028.pdf) 60 | [TripleGAN, '17.03.07](https://arxiv.org/pdf/1703.02291.pdf) 61 | Applications 62 | [Pix2Pix, '16.11.21](https://arxiv.org/pdf/1611.07004v1.pdf) 63 | [PPGN, '16.11.30](https://arxiv.org/pdf/1612.00005.pdf) 64 | [StackGAN, '16.12.10](https://arxiv.org/pdf/1612.03242.pdf) 65 | RNN 66 | [LSTM, '97.11](http://www.mitpressjournals.org/doi/10.1162/neco.1997.9.8.1735) 67 | [GRU, 14.11](https://arxiv.org/abs/1412.3555) 68 | [ACT: Adaptive Computation Time, '17.05](https://arxiv.org/abs/1603.08983) 69 | [S2S: RNN Encoder-Decoder, '14.06](https://arxiv.org/abs/1406.1078) 70 | [Attention: Jointly Learning to Align, '14.09](https://arxiv.org/abs/1409.0473) 71 | [Effective Approaches to Attention, Luong et al. '15.08](https://arxiv.org/abs/1508.04025) 72 | [DCN: Dynamic Coattention Networks, '16.08](https://arxiv.org/abs/1611.01604), [DCN+, '17.08](https://arxiv.org/abs/1711.00106) 73 | Self-attention 74 | NLP 75 | [Transformer: Attention Is All You Need, '17.06](https://arxiv.org/abs/1706.03762) 76 | [GPT1: Improving Language Understanding by Generative Pre-Training, '18.06](https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf) 77 | [BERT: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, '18.10](https://arxiv.org/abs/1810.04805) 78 | [GPT2: Language Models are Unsupervised Multitask Learners, '19.01](https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf) 79 | [GPT3: Language Models are Few-Shot Learners, '20.05](https://arxiv.org/abs/2005.14165) 80 | Vision 81 | [Image-GPT: Generative Pretraining from Pixels, '20.06](https://cdn.openai.com/papers/Generative_Pretraining_from_Pixels_V2.pdf) 82 | [Vision transformer: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, '20.09](https://arxiv.org/abs/2010.11929) 83 | [DeiT: Training data-efficient image transformers & distillation through attention, '20.12](https://arxiv.org/abs/2012.12877) 84 | Speech 85 | Multimodal 86 | [DALL-E, '21.01](https://openai.com/blog/dall-e/) 87 | [CLIP: Learning Transferable Visual Models From Natural Language Supervision](https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language_Supervision.pdf) 88 | 89 | [Capsule Net, '17.10](https://arxiv.org/abs/1710.09829) 90 | Memory Networks 91 | Neural Programming 92 | [Neural Turing Machine,'14.10](https://arxiv.org/pdf/1410.5401.pdf) 93 | [Neural Random-Access Machines,'16.02](https://arxiv.org/pdf/1511.06392.pdf) 94 | [Hierarchical Attentive Memory, '16.02](https://arxiv.org/abs/1602.03218) 95 | [Neural GPUs Learn Algorithms, '16.03](https://arxiv.org/pdf/1511.08228.pdf) 96 | [Neural Programmer,'16.08](https://arxiv.org/pdf/1511.04834.pdf) 97 | [Neural Module Networks, '16.06](https://www.cv-foundation.org/openaccess/content_cvpr_2016/html/Andreas_Neural_Module_Networks_CVPR_2016_paper.html) 98 | [Hybrid Computing, '16.10](https://www.nature.com/nature/journal/v538/n7626/full/nature20101.html) 99 | [Memory Networks,'14.10](https://arxiv.org/pdf/1410.3916.pdf) 100 | [End-to-End Memory Network,'15.03](https://arxiv.org/pdf/1503.08895.pdf) 101 | [DMN: Dynamic Memory Network, '16.03](https://arxiv.org/pdf/1506.07285.pdf), [DMN+, '16.04 ](https://arxiv.org/pdf/1603.01417.pdf) 102 | -------------------------------------------------------------------------------- /Neural_Net_Arch_Genealogy.dot: -------------------------------------------------------------------------------- 1 | 2 | digraph "Neural_Net_Arch_Genealogy" { 3 | rankdir = LR; 4 | overlap = scale; 5 | 6 | "Neural Net Arch Genealogy" -> "Reinforcement Learning Algorithms"; 7 | "Reinforcement Learning Algorithms" -> "A3C"; 8 | "Reinforcement Learning Algorithms" -> "DARLA"; 9 | "Reinforcement Learning Algorithms" -> "ACTKR"; 10 | "Reinforcement Learning Algorithms" -> "c51"; 11 | "Neural Net Arch Genealogy" -> "CNN"; 12 | "CNN" -> "AlexNet"; 13 | "CNN" -> "VggNet"; 14 | "CNN" -> "GoogLeNet"; 15 | "CNN" -> "ResNet"; 16 | "CNN" -> "DenseNet"; 17 | "CNN" -> "SENet"; 18 | "CNN" -> "Object Detection"; 19 | "Object Detection" -> "R-CNN"; 20 | "Object Detection" -> "Fast R-CNN"; 21 | "Object Detection" -> "Faster R-CNN"; 22 | "Object Detection" -> "Mask R-CNN"; 23 | "Object Detection" -> "YOLO"; 24 | "Object Detection" -> "SSD"; 25 | "Object Detection" -> "R-FCN"; 26 | "CNN" -> "Semantic Segmentation"; 27 | "Semantic Segmentation" -> "FCN"; 28 | "Semantic Segmentation" -> "DeconvNet"; 29 | "Semantic Segmentation" -> "DeepLab"; 30 | "Semantic Segmentation" -> "U-Net"; 31 | "CNN" -> "Super-resolution"; 32 | "Super-resolution" -> "MemNet"; 33 | "Super-resolution" -> "FSRCNN"; 34 | "Super-resolution" -> "SRCNN"; 35 | "Super-resolution" -> "VDSR"; 36 | "Super-resolution" -> "DRCN"; 37 | "Super-resolution" -> "LabSRN"; 38 | "Super-resolution" -> "EDSR"; 39 | "CNN" -> "TTS"; 40 | "TTS" -> "Wavenet"; 41 | "Neural Net Arch Genealogy" -> "Generative Models"; 42 | "Generative Models" -> "Autoregressive models"; 43 | "Autoregressive models" -> "MADE"; 44 | "Autoregressive models" -> "PixelRNN"; 45 | "Autoregressive models" -> "NADE"; 46 | "Autoregressive models" -> "PixelCNN"; 47 | "Autoregressive models" -> "PixelCNN"; 48 | "Generative Models" -> "Latent variable models"; 49 | "Latent variable models" -> "VAE"; 50 | "VAE" -> "CVAE"; 51 | "VAE" -> "AAE"; 52 | "VAE" -> "AVB"; 53 | "VAE" -> "VQ-VAE"; 54 | "Latent variable models" -> "GAN"; 55 | "GAN" -> "Variants"; 56 | "Variants" -> "CGAN"; 57 | "Variants" -> "DCGAN"; 58 | "Variants" -> "infoGAN"; 59 | "Variants" -> "EBGAN"; 60 | "Variants" -> "ACGAN"; 61 | "Variants" -> "WGAN"; 62 | "Variants" -> "BEGAN"; 63 | "Variants" -> "WGAN-GP"; 64 | "Variants" -> "TripleGAN"; 65 | "GAN" -> "Applications"; 66 | "Applications" -> "Pix2Pix"; 67 | "Applications" -> "PPGN"; 68 | "Applications" -> "StackGAN"; 69 | "Neural Net Arch Genealogy" -> "RNN"; 70 | "RNN" -> "LSTM"; 71 | "RNN" -> "GRU"; 72 | "RNN" -> "ACT"; 73 | "RNN" -> "S2S"; 74 | "S2S" -> "Attention"; 75 | "Attention" -> "Effective Approaches to Attention"; 76 | "Attention" -> "DCN"; 77 | "Attention" -> "Transformer"; 78 | "Neural Net Arch Genealogy" -> "Capsule Net"; 79 | "Neural Net Arch Genealogy" -> "Memory Networks"; 80 | "Memory Networks" -> "Neural Programming"; 81 | "Neural Programming" -> "Neural Turing Machine"; 82 | "Neural Programming" -> "Neural Random-Access Machines"; 83 | "Neural Programming" -> "Hierarchical Attentive Memory"; 84 | "Neural Programming" -> "Neural GPUs Learn Algorithms"; 85 | "Neural Programming" -> "Neural Programmer"; 86 | "Neural Programming" -> "Neural Module Networks"; 87 | "Neural Programming" -> "Hybrid Computing"; 88 | "Memory Networks" -> "Memory Networks"; 89 | "Memory Networks" -> "End-to-End Memory Network"; 90 | "Memory Networks" -> "DMN"; 91 | } 92 | 93 | -------------------------------------------------------------------------------- /Neural_Net_Arch_Genealogy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hunkim/deep_architecture_genealogy/30bd312ddcbd4d17cac5b8d6b8f18f44c06a3c07/Neural_Net_Arch_Genealogy.png -------------------------------------------------------------------------------- /Neural_Net_Arch_Genealogy.svg: -------------------------------------------------------------------------------- 1 | 2 | 4 | 6 | 7 | 9 | 10 | Neural_Net_Arch_Genealogy 11 | 12 | 13 | 14 | Neural Net Arch Genealogy 15 | 16 | Neural Net Arch Genealogy 17 | 18 | 19 | 20 | Reinforcement Learning Algorithms 21 | 22 | Reinforcement Learning Algorithms 23 | 24 | 25 | 26 | Neural Net Arch Genealogy->Reinforcement Learning Algorithms 27 | 28 | 29 | 30 | 31 | 32 | CNN 33 | 34 | CNN 35 | 36 | 37 | 38 | Neural Net Arch Genealogy->CNN 39 | 40 | 41 | 42 | 43 | 44 | Generative Models 45 | 46 | Generative Models 47 | 48 | 49 | 50 | Neural Net Arch Genealogy->Generative Models 51 | 52 | 53 | 54 | 55 | 56 | RNN 57 | 58 | RNN 59 | 60 | 61 | 62 | Neural Net Arch Genealogy->RNN 63 | 64 | 65 | 66 | 67 | 68 | Capsule Net 69 | 70 | Capsule Net 71 | 72 | 73 | 74 | Neural Net Arch Genealogy->Capsule Net 75 | 76 | 77 | 78 | 79 | 80 | Memory Networks 81 | 82 | Memory Networks 83 | 84 | 85 | 86 | Neural Net Arch Genealogy->Memory Networks 87 | 88 | 89 | 90 | 91 | 92 | A3C 93 | 94 | A3C 95 | 96 | 97 | 98 | Reinforcement Learning Algorithms->A3C 99 | 100 | 101 | 102 | 103 | 104 | DARLA 105 | 106 | DARLA 107 | 108 | 109 | 110 | Reinforcement Learning Algorithms->DARLA 111 | 112 | 113 | 114 | 115 | 116 | ACTKR 117 | 118 | ACTKR 119 | 120 | 121 | 122 | Reinforcement Learning Algorithms->ACTKR 123 | 124 | 125 | 126 | 127 | 128 | c51 129 | 130 | c51 131 | 132 | 133 | 134 | Reinforcement Learning Algorithms->c51 135 | 136 | 137 | 138 | 139 | 140 | AlexNet 141 | 142 | AlexNet 143 | 144 | 145 | 146 | CNN->AlexNet 147 | 148 | 149 | 150 | 151 | 152 | VggNet 153 | 154 | VggNet 155 | 156 | 157 | 158 | CNN->VggNet 159 | 160 | 161 | 162 | 163 | 164 | GoogLeNet 165 | 166 | GoogLeNet 167 | 168 | 169 | 170 | CNN->GoogLeNet 171 | 172 | 173 | 174 | 175 | 176 | ResNet 177 | 178 | ResNet 179 | 180 | 181 | 182 | CNN->ResNet 183 | 184 | 185 | 186 | 187 | 188 | DenseNet 189 | 190 | DenseNet 191 | 192 | 193 | 194 | CNN->DenseNet 195 | 196 | 197 | 198 | 199 | 200 | SENet 201 | 202 | SENet 203 | 204 | 205 | 206 | CNN->SENet 207 | 208 | 209 | 210 | 211 | 212 | Object Detection 213 | 214 | Object Detection 215 | 216 | 217 | 218 | CNN->Object Detection 219 | 220 | 221 | 222 | 223 | 224 | Semantic Segmentation 225 | 226 | Semantic Segmentation 227 | 228 | 229 | 230 | CNN->Semantic Segmentation 231 | 232 | 233 | 234 | 235 | 236 | Super-resolution 237 | 238 | Super-resolution 239 | 240 | 241 | 242 | CNN->Super-resolution 243 | 244 | 245 | 246 | 247 | 248 | TTS 249 | 250 | TTS 251 | 252 | 253 | 254 | CNN->TTS 255 | 256 | 257 | 258 | 259 | 260 | R-CNN 261 | 262 | R-CNN 263 | 264 | 265 | 266 | Object Detection->R-CNN 267 | 268 | 269 | 270 | 271 | 272 | Fast R-CNN 273 | 274 | Fast R-CNN 275 | 276 | 277 | 278 | Object Detection->Fast R-CNN 279 | 280 | 281 | 282 | 283 | 284 | Faster R-CNN 285 | 286 | Faster R-CNN 287 | 288 | 289 | 290 | Object Detection->Faster R-CNN 291 | 292 | 293 | 294 | 295 | 296 | Mask R-CNN 297 | 298 | Mask R-CNN 299 | 300 | 301 | 302 | Object Detection->Mask R-CNN 303 | 304 | 305 | 306 | 307 | 308 | YOLO 309 | 310 | YOLO 311 | 312 | 313 | 314 | Object Detection->YOLO 315 | 316 | 317 | 318 | 319 | 320 | SSD 321 | 322 | SSD 323 | 324 | 325 | 326 | Object Detection->SSD 327 | 328 | 329 | 330 | 331 | 332 | R-FCN 333 | 334 | R-FCN 335 | 336 | 337 | 338 | Object Detection->R-FCN 339 | 340 | 341 | 342 | 343 | 344 | FCN 345 | 346 | FCN 347 | 348 | 349 | 350 | Semantic Segmentation->FCN 351 | 352 | 353 | 354 | 355 | 356 | DeconvNet 357 | 358 | DeconvNet 359 | 360 | 361 | 362 | Semantic Segmentation->DeconvNet 363 | 364 | 365 | 366 | 367 | 368 | DeepLab 369 | 370 | DeepLab 371 | 372 | 373 | 374 | Semantic Segmentation->DeepLab 375 | 376 | 377 | 378 | 379 | 380 | U-Net 381 | 382 | U-Net 383 | 384 | 385 | 386 | Semantic Segmentation->U-Net 387 | 388 | 389 | 390 | 391 | 392 | MemNet 393 | 394 | MemNet 395 | 396 | 397 | 398 | Super-resolution->MemNet 399 | 400 | 401 | 402 | 403 | 404 | FSRCNN 405 | 406 | FSRCNN 407 | 408 | 409 | 410 | Super-resolution->FSRCNN 411 | 412 | 413 | 414 | 415 | 416 | SRCNN 417 | 418 | SRCNN 419 | 420 | 421 | 422 | Super-resolution->SRCNN 423 | 424 | 425 | 426 | 427 | 428 | VDSR 429 | 430 | VDSR 431 | 432 | 433 | 434 | Super-resolution->VDSR 435 | 436 | 437 | 438 | 439 | 440 | DRCN 441 | 442 | DRCN 443 | 444 | 445 | 446 | Super-resolution->DRCN 447 | 448 | 449 | 450 | 451 | 452 | LabSRN 453 | 454 | LabSRN 455 | 456 | 457 | 458 | Super-resolution->LabSRN 459 | 460 | 461 | 462 | 463 | 464 | EDSR 465 | 466 | EDSR 467 | 468 | 469 | 470 | Super-resolution->EDSR 471 | 472 | 473 | 474 | 475 | 476 | Wavenet 477 | 478 | Wavenet 479 | 480 | 481 | 482 | TTS->Wavenet 483 | 484 | 485 | 486 | 487 | 488 | Autoregressive models 489 | 490 | Autoregressive models 491 | 492 | 493 | 494 | Generative Models->Autoregressive models 495 | 496 | 497 | 498 | 499 | 500 | Latent variable models 501 | 502 | Latent variable models 503 | 504 | 505 | 506 | Generative Models->Latent variable models 507 | 508 | 509 | 510 | 511 | 512 | MADE 513 | 514 | MADE 515 | 516 | 517 | 518 | Autoregressive models->MADE 519 | 520 | 521 | 522 | 523 | 524 | PixelRNN 525 | 526 | PixelRNN 527 | 528 | 529 | 530 | Autoregressive models->PixelRNN 531 | 532 | 533 | 534 | 535 | 536 | NADE 537 | 538 | NADE 539 | 540 | 541 | 542 | Autoregressive models->NADE 543 | 544 | 545 | 546 | 547 | 548 | PixelCNN 549 | 550 | PixelCNN 551 | 552 | 553 | 554 | Autoregressive models->PixelCNN 555 | 556 | 557 | 558 | 559 | 560 | Autoregressive models->PixelCNN 561 | 562 | 563 | 564 | 565 | 566 | VAE 567 | 568 | VAE 569 | 570 | 571 | 572 | Latent variable models->VAE 573 | 574 | 575 | 576 | 577 | 578 | GAN 579 | 580 | GAN 581 | 582 | 583 | 584 | Latent variable models->GAN 585 | 586 | 587 | 588 | 589 | 590 | CVAE 591 | 592 | CVAE 593 | 594 | 595 | 596 | VAE->CVAE 597 | 598 | 599 | 600 | 601 | 602 | AAE 603 | 604 | AAE 605 | 606 | 607 | 608 | VAE->AAE 609 | 610 | 611 | 612 | 613 | 614 | AVB 615 | 616 | AVB 617 | 618 | 619 | 620 | VAE->AVB 621 | 622 | 623 | 624 | 625 | 626 | VQ-VAE 627 | 628 | VQ-VAE 629 | 630 | 631 | 632 | VAE->VQ-VAE 633 | 634 | 635 | 636 | 637 | 638 | Variants 639 | 640 | Variants 641 | 642 | 643 | 644 | GAN->Variants 645 | 646 | 647 | 648 | 649 | 650 | Applications 651 | 652 | Applications 653 | 654 | 655 | 656 | GAN->Applications 657 | 658 | 659 | 660 | 661 | 662 | CGAN 663 | 664 | CGAN 665 | 666 | 667 | 668 | Variants->CGAN 669 | 670 | 671 | 672 | 673 | 674 | DCGAN 675 | 676 | DCGAN 677 | 678 | 679 | 680 | Variants->DCGAN 681 | 682 | 683 | 684 | 685 | 686 | infoGAN 687 | 688 | infoGAN 689 | 690 | 691 | 692 | Variants->infoGAN 693 | 694 | 695 | 696 | 697 | 698 | EBGAN 699 | 700 | EBGAN 701 | 702 | 703 | 704 | Variants->EBGAN 705 | 706 | 707 | 708 | 709 | 710 | ACGAN 711 | 712 | ACGAN 713 | 714 | 715 | 716 | Variants->ACGAN 717 | 718 | 719 | 720 | 721 | 722 | WGAN 723 | 724 | WGAN 725 | 726 | 727 | 728 | Variants->WGAN 729 | 730 | 731 | 732 | 733 | 734 | BEGAN 735 | 736 | BEGAN 737 | 738 | 739 | 740 | Variants->BEGAN 741 | 742 | 743 | 744 | 745 | 746 | WGAN-GP 747 | 748 | WGAN-GP 749 | 750 | 751 | 752 | Variants->WGAN-GP 753 | 754 | 755 | 756 | 757 | 758 | TripleGAN 759 | 760 | TripleGAN 761 | 762 | 763 | 764 | Variants->TripleGAN 765 | 766 | 767 | 768 | 769 | 770 | Pix2Pix 771 | 772 | Pix2Pix 773 | 774 | 775 | 776 | Applications->Pix2Pix 777 | 778 | 779 | 780 | 781 | 782 | PPGN 783 | 784 | PPGN 785 | 786 | 787 | 788 | Applications->PPGN 789 | 790 | 791 | 792 | 793 | 794 | StackGAN 795 | 796 | StackGAN 797 | 798 | 799 | 800 | Applications->StackGAN 801 | 802 | 803 | 804 | 805 | 806 | LSTM 807 | 808 | LSTM 809 | 810 | 811 | 812 | RNN->LSTM 813 | 814 | 815 | 816 | 817 | 818 | GRU 819 | 820 | GRU 821 | 822 | 823 | 824 | RNN->GRU 825 | 826 | 827 | 828 | 829 | 830 | ACT 831 | 832 | ACT 833 | 834 | 835 | 836 | RNN->ACT 837 | 838 | 839 | 840 | 841 | 842 | S2S 843 | 844 | S2S 845 | 846 | 847 | 848 | RNN->S2S 849 | 850 | 851 | 852 | 853 | 854 | Attention 855 | 856 | Attention 857 | 858 | 859 | 860 | S2S->Attention 861 | 862 | 863 | 864 | 865 | 866 | Effective Approaches to Attention 867 | 868 | Effective Approaches to Attention 869 | 870 | 871 | 872 | Attention->Effective Approaches to Attention 873 | 874 | 875 | 876 | 877 | 878 | DCN 879 | 880 | DCN 881 | 882 | 883 | 884 | Attention->DCN 885 | 886 | 887 | 888 | 889 | 890 | Transformer 891 | 892 | Transformer 893 | 894 | 895 | 896 | Attention->Transformer 897 | 898 | 899 | 900 | 901 | 902 | Memory Networks->Memory Networks 903 | 904 | 905 | 906 | 907 | 908 | Neural Programming 909 | 910 | Neural Programming 911 | 912 | 913 | 914 | Memory Networks->Neural Programming 915 | 916 | 917 | 918 | 919 | 920 | End-to-End Memory Network 921 | 922 | End-to-End Memory Network 923 | 924 | 925 | 926 | Memory Networks->End-to-End Memory Network 927 | 928 | 929 | 930 | 931 | 932 | DMN 933 | 934 | DMN 935 | 936 | 937 | 938 | Memory Networks->DMN 939 | 940 | 941 | 942 | 943 | 944 | Neural Turing Machine 945 | 946 | Neural Turing Machine 947 | 948 | 949 | 950 | Neural Programming->Neural Turing Machine 951 | 952 | 953 | 954 | 955 | 956 | Neural Random-Access Machines 957 | 958 | Neural Random-Access Machines 959 | 960 | 961 | 962 | Neural Programming->Neural Random-Access Machines 963 | 964 | 965 | 966 | 967 | 968 | Hierarchical Attentive Memory 969 | 970 | Hierarchical Attentive Memory 971 | 972 | 973 | 974 | Neural Programming->Hierarchical Attentive Memory 975 | 976 | 977 | 978 | 979 | 980 | Neural GPUs Learn Algorithms 981 | 982 | Neural GPUs Learn Algorithms 983 | 984 | 985 | 986 | Neural Programming->Neural GPUs Learn Algorithms 987 | 988 | 989 | 990 | 991 | 992 | Neural Programmer 993 | 994 | Neural Programmer 995 | 996 | 997 | 998 | Neural Programming->Neural Programmer 999 | 1000 | 1001 | 1002 | 1003 | 1004 | Neural Module Networks 1005 | 1006 | Neural Module Networks 1007 | 1008 | 1009 | 1010 | Neural Programming->Neural Module Networks 1011 | 1012 | 1013 | 1014 | 1015 | 1016 | Hybrid Computing 1017 | 1018 | Hybrid Computing 1019 | 1020 | 1021 | 1022 | Neural Programming->Hybrid Computing 1023 | 1024 | 1025 | 1026 | 1027 | 1028 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Deep Architecture Genealogy 2 | There are so many new models and architectures. If you find something interesting and worth paying attention to, please send us a pull requests (PR) and write issues. 3 | `README.md` is automatically generated. Please send PRs on the `Neural Net Arch Genealogy.txt` file. 4 | ## Mindmap Coggle Link 5 | https://coggle.it/diagram/Wf5mYoJbsgABUF9P 6 | ![https://coggle.it/diagram/Wf5mYoJbsgABUF9P](Neural_Net_Arch_Genealogy.png) 7 | ## Text Version 8 | This is automatically generated. Please send a PR on the `Neural Net Arch Genealogy.txt` file. 9 | * Reinforcement Learning Algorithms 10 | * [A3C, '16.02.06](https://arxiv.org/abs/1602.01783) 11 | * [DARLA, '17.07.26](https://arxiv.org/pdf/1707.08475.pdf) 12 | * [ACTKR, '17.08.17](https://arxiv.org/pdf/1708.05144.pdf) 13 | * [c51, '17.10.27](https://arxiv.org/pdf/1710.10044.pdf) 14 | * CNN 15 | * [AlexNet, '12.12](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf) 16 | * [VggNet, '14.09](https://arxiv.org/pdf/1409.1556.pdf) 17 | * [GoogLeNet, '14.09](https://arxiv.org/abs/1409.4842) 18 | * [ResNet, '15.12](https://arxiv.org/pdf/1512.03385v1.pdf) 19 | * [DenseNet, '16.08](https://arxiv.org/pdf/1608.06993.pdf) 20 | * [SENet: Squeeze-and-Excitation Networks, '17.09](https://arxiv.org/abs/1709.01507) 21 | * Object Detection 22 | * [R-CNN](https://arxiv.org/pdf/1311.2524.pdf) 23 | * [Fast R-CNN](https://arxiv.org/pdf/1504.08083.pdf) 24 | * [Faster R-CNN](https://arxiv.org/pdf/1506.01497.pdf) 25 | * [Mask R-CNN](https://arxiv.org/pdf/1703.06870.pdf) 26 | * [YOLO](https://arxiv.org/pdf/1506.02640.pdf) 27 | * [SSD](https://arxiv.org/pdf/1512.02325.pdf) 28 | * [R-FCN](https://arxiv.org/pdf/1605.06409.pdf) 29 | * Semantic Segmentation 30 | * [FCN](https://arxiv.org/pdf/1411.4038.pdf) 31 | * [DeconvNet](https://arxiv.org/pdf/1505.04366.pdf) 32 | * [DeepLab](https://arxiv.org/pdf/1606.00915.pdf) 33 | * [U-Net](https://arxiv.org/pdf/1505.04597.pdf) 34 | * Super-resolution 35 | * [MemNet](https://arxiv.org/abs/1708.02209) 36 | * [FSRCNN](https://arxiv.org/1608.00367) 37 | * [SRCNN](https://arxiv.org/abs/1501.00092) 38 | * [VDSR](https://arxiv.org/abs/1511.04587) 39 | * [DRCN](https://arxiv.org/abs/1511.04491) 40 | * [LabSRN](https://arxiv.org/abs/1704.03915) 41 | * [EDSR](https://arxiv.org/abs/1707.02921) 42 | * TTS 43 | * [Wavenet, '16.09.12](https://arxiv.org/abs/1609.03499) 44 | * Generative Models 45 | * Autoregressive models 46 | * [MADE, '15.02.12](https://arxiv.org/pdf/1502.03509.pdf) 47 | * [PixelRNN, '16.01.25](https://arxiv.org/pdf/1601.06759.pdf) 48 | * [NADE, '16.05.07](https://arxiv.org/pdf/1605.02226.pdf) 49 | * [PixelCNN, '16.06.16](https://arxiv.org/pdf/1606.05328.pdf) 50 | * [PixelCNN++, '17.01.19](https://arxiv.org/pdf/1701.05517.pdf) 51 | * Latent variable models 52 | * [VAE, '13.12.20](https://arxiv.org/pdf/1312.6114.pdf) 53 | * [CVAE, '14.06.20](https://arxiv.org/pdf/1406.5298.pdf) 54 | * [AAE, '15.11.18](https://arxiv.org/pdf/1511.05644.pdf) 55 | * [AVB, '17.01.17](https://arxiv.org/pdf/1701.04722.pdf) 56 | * [VQ-VAE, '17.11.2](https://arxiv.org/abs/1711.00937) 57 | * [GAN, '14.06.10](https://arxiv.org/pdf/1406.2661.pdf) 58 | * Variants 59 | * [CGAN, '14.11.06](https://arxiv.org/pdf/1411.1784.pdf) 60 | * [DCGAN, '15.11.19](https://arxiv.org/pdf/1511.06434.pdf) 61 | * [infoGAN, '16.06.12](https://arxiv.org/pdf/1704.00028.pdf) 62 | * [EBGAN, '16.09.11](https://arxiv.org/pdf/1609.03126.pdf) 63 | * [ACGAN, '16.10.30](https://arxiv.org/pdf/1610.09585.pdf) 64 | * [WGAN, '17.01.26](https://arxiv.org/pdf/1701.07875.pdf) 65 | * [BEGAN, '17.02.27](https://arxiv.org/pdf/1702.08431.pdf) 66 | * [WGAN-GP, '17.03.31](https://arxiv.org/pdf/1704.00028.pdf) 67 | * [TripleGAN, '17.03.07](https://arxiv.org/pdf/1703.02291.pdf) 68 | * Applications 69 | * [Pix2Pix, '16.11.21](https://arxiv.org/pdf/1611.07004v1.pdf) 70 | * [PPGN, '16.11.30](https://arxiv.org/pdf/1612.00005.pdf) 71 | * [StackGAN, '16.12.10](https://arxiv.org/pdf/1612.03242.pdf) 72 | * RNN 73 | * [LSTM, '97.11](http://www.mitpressjournals.org/doi/10.1162/neco.1997.9.8.1735) 74 | * [GRU, 14.11](https://arxiv.org/abs/1412.3555) 75 | * [ACT: Adaptive Computation Time, '17.05](https://arxiv.org/abs/1603.08983) 76 | * [S2S: RNN Encoder-Decoder, '14.06](https://arxiv.org/abs/1406.1078) 77 | * [Attention: Jointly Learning to Align, '14.09](https://arxiv.org/abs/1409.0473) 78 | * [Effective Approaches to Attention, Luong et al. '15.08](https://arxiv.org/abs/1508.04025) 79 | * [DCN: Dynamic Coattention Networks, '16.08](https://arxiv.org/abs/1611.01604), [DCN+, '17.08](https://arxiv.org/abs/1711.00106) 80 | * [Transformer: Attention Is All You Need, '17.06](https://arxiv.org/abs/1706.03762) 81 | * [Capsule Net, '17.10](https://arxiv.org/abs/1710.09829) 82 | * Memory Networks 83 | * Neural Programming 84 | * [Neural Turing Machine,'14.10](https://arxiv.org/pdf/1410.5401.pdf) 85 | * [Neural Random-Access Machines,'16.02](https://arxiv.org/pdf/1511.06392.pdf) 86 | * [Hierarchical Attentive Memory, '16.02](https://arxiv.org/abs/1602.03218) 87 | * [Neural GPUs Learn Algorithms, '16.03](https://arxiv.org/pdf/1511.08228.pdf) 88 | * [Neural Programmer,'16.08](https://arxiv.org/pdf/1511.04834.pdf) 89 | * [Neural Module Networks, '16.06](https://www.cv-foundation.org/openaccess/content_cvpr_2016/html/Andreas_Neural_Module_Networks_CVPR_2016_paper.html) 90 | * [Hybrid Computing, '16.10](https://www.nature.com/nature/journal/v538/n7626/full/nature20101.html) 91 | * [Memory Networks,'14.10](https://arxiv.org/pdf/1410.3916.pdf) 92 | * [End-to-End Memory Network,'15.03](https://arxiv.org/pdf/1503.08895.pdf) 93 | * [DMN: Dynamic Memory Network, '16.03](https://arxiv.org/pdf/1506.07285.pdf), [DMN+, '16.04 ](https://arxiv.org/pdf/1603.01417.pdf) 94 | ## Contributions 95 | Your pull requests and issues are always welcome. -------------------------------------------------------------------------------- /txt2dot.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # usage: 3 | # python3 txt2dot.py 'Neural Net Arch Genealogy.txt' > Neural_Net_Arch_Genealogy.dot 4 | # then, if graphviz is installed in your system: 5 | # dot -Tsvg Neural_Net_Arch_Genealogy.dot > Neural_Net_Arch_Genealogy.svg 6 | 7 | import re, fileinput 8 | 9 | head= ''' 10 | digraph "Neural_Net_Arch_Genealogy" { 11 | rankdir = LR; 12 | overlap = scale; 13 | ''' 14 | 15 | tail = '}\n' 16 | 17 | ancestors = [''] * 9 18 | 19 | print(head) 20 | for line in fileinput.input(): 21 | name = re.match('^(\t*)(\S.*\S)', line) 22 | n = len(name.group(1)) 23 | m2 = re.match('\[(\w[-\w\s]+)(.*?)\s*\]', name.group(2)) 24 | if m2: 25 | name = m2.group(1) 26 | else: 27 | name = name.group(2) 28 | if n>0: 29 | print(' "{}" -> "{}";'.format(ancestors[n-1], name)) 30 | ancestors[n] = name 31 | 32 | print(tail) 33 | 34 | -------------------------------------------------------------------------------- /txt2md.py: -------------------------------------------------------------------------------- 1 | # Simple txt to README.md 2 | # Just download txt and png after updating from 3 | # https://coggle.it/diagram/Wf5mYoJbsgABUF9P and 4 | # run this script 5 | head='# Deep Architecture Genealogy\n' \ 6 | 'There are so many new models and architectures. ' \ 7 | 'If you find something interesting and worth paying attention to, ' \ 8 | 'please send us a pull requests (PR) and write issues.\n' \ 9 | '`README.md` is automatically generated. Please send PRs on the `Neural Net Arch Genealogy.txt` file.\n' \ 10 | '## Mindmap Coggle Link\n' \ 11 | 'https://coggle.it/diagram/Wf5mYoJbsgABUF9P\n' \ 12 | '![https://coggle.it/diagram/Wf5mYoJbsgABUF9P](Neural_Net_Arch_Genealogy.png)\n' \ 13 | '## Text Version\n' \ 14 | 'This is automatically generated. Please send a PR on the `Neural Net Arch Genealogy.txt` file.\n' 15 | 16 | tail = '\n## Contributions\nYour pull requests and issues are always welcome.' \ 17 | 18 | with open('Neural Net Arch Genealogy.txt') as fin, open('README.md', 'w') as fout: 19 | fout.write(head) 20 | for line in fin: 21 | tab_count = line.count('\t') 22 | if not tab_count: 23 | continue 24 | 25 | spaces = [' ' for i in range(tab_count-1)] 26 | fout.write(''.join(spaces) + '* ' + line.replace('\t', '')) 27 | 28 | fout.write(tail) 29 | --------------------------------------------------------------------------------