├── Bayesian ├── Deep Learning │ └── Droupout.md ├── Deeplearning │ └── Droupout.md └── InverseProblem │ └── Basic.md ├── DeeplearningModels ├── Bath normalization │ └── BN.md ├── Read me.md ├── ResidualLearning │ ├── Readme.md │ └── Resnet.pdf └── papers.md ├── Diffusion Maps ├── Diffusion maps.md ├── HarmonicOnManifold │ └── GraphCNN.md └── Inferring interaction rules from observations of evolutive systems I: The variational approach.md ├── LICENSE.md ├── Optimization ├── DynamicSysmtemModelling │ └── HJ.md ├── Lowrank │ └── Readme.md └── Opt For DL.md ├── Other Topics └── readme.md ├── README.md ├── computer vision ├── Object Detection.md ├── Style Transfer.md └── denoising.md ├── generative models ├── GAN.md └── readme.md └── learningtheory ├── approximation(NN).md └── dl.md /Bayesian/Deep Learning/Droupout.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /Bayesian/Deeplearning/Droupout.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /Bayesian/InverseProblem/Basic.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /DeeplearningModels/Bath normalization/BN.md: -------------------------------------------------------------------------------- 1 | 1. - [x] Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift 2 | 1. - [ ] Riemannian approach to batch normalization 3 | 1. - [ ] On the Effects of Batch and Weight Normalization in Generative Adversarial Networks 4 | 1. - [ ] Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks 5 | 1. - [x] Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models 6 | 1. - [ ] Streaming Normalization: Towards Simpler and More Biologically-plausible Normalizations for Online and Recurrent Learning 7 | 1. - [ ] Revisiting Batch Normalization For Practical Domain Adaptation 8 | -------------------------------------------------------------------------------- /DeeplearningModels/Read me.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /DeeplearningModels/ResidualLearning/Readme.md: -------------------------------------------------------------------------------- 1 | # Deep Residual Learning 2 | 3 | ## Resnet.pdf 4 | 5 | > Yiping Lu's Talk at Bin Dong's Group 6 | 7 | K He,X Zhang,S Ren,J Sun (2015,MSRA) 8 | Deep Residual Learning for Image Recognition 9 | 10 | K He,X Zhang,S Ren,J Sun (2015,MSRA) 11 | Identity Mappings in Deep Residual Networks 12 | 13 | RK Srivastava,K Gre↵,J Schmidhuber (2015) 14 | Highway Networks 15 | 16 | Yunjin Chen, Wei Yu and Thomas Pock 17 | On learning optimized reaction di↵usion processes for e↵ective image restoration 18 | 19 | Qianli Liao, Tomaso Poggio 20 | Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex 21 | 22 | 23 | G Huang,Z Liu, KQ Weinberger 24 | Densely Connected Convolutional Networks 25 | 26 | 27 | M Abdi, S Nahavandi 28 | Multi-Residual Networks: Improving the Speed and Accuracy of Residual Networks 29 | 30 | 31 | G Larsson, M Maire, G Shakhnarovich 32 | FractalNet:Ultra-Deep Neural Networks without Residuals 33 | 34 | Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, Kilian Weinberger 35 | Deep Networks with Stochastic Depth 36 | 37 | 38 | Kaiming He,Xiangyu Zhang,Shaoqing Ren,Jian Sun 39 | Delving Deep into Rectifiers:Surpassing Human-Level Performance on ImageNet 40 | 41 | 42 | E Haber, L Ruthotto, E Holtham 43 | Learning across scales - A multiscale method for Convolution Neural Network 44 | 45 | E Haber, L Ruthotto 46 | Stable Architectures for Deep Neural Networks 47 | -------------------------------------------------------------------------------- /DeeplearningModels/ResidualLearning/Resnet.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/2prime/Paper-Reading/f729d9b96d3d732a097db80c99b468f0ffa32e98/DeeplearningModels/ResidualLearning/Resnet.pdf -------------------------------------------------------------------------------- /DeeplearningModels/papers.md: -------------------------------------------------------------------------------- 1 | 1. - [x] Krizhevsky, Alex, et al. "**Imagenet classification with deep convolutional neural networks**." Advances in neural information processing systems. 2012. [[pdf]](http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks) (**AlexNet**) 2 | 3 | 1. - [ ] Zeiler, Matthew D., and R. Fergus. "**Visualizing and Understanding Convolutional Networks**." arXiv:1311.2901 (2013). [[pdf]](https://arxiv.org/abs/1311.2901) (**ZFNet**) 4 | 5 | 1. - [ ] Simonyan, Karen, and Andrew Zisserman. "**Very deep convolutional networks for large-scale image recognition**." arXiv preprint arXiv:1409.1556 (2014). [[pdf]](https://arxiv.org/pdf/1409.1556) (**VGGNet**) 6 | 7 | 1. - [ ] Szegedy, Christian, et al. "**Going deeper with convolutions**." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015. [[pdf]](https://arxiv.org/abs/1409.4842) (**GoogLeNet/Inception**) 8 | 9 | 1. - [x] Srivastava, Rupesh Kumar, K. Greff, and J. Schmidhuber. "**Highway Networks**." Computer Science (2015). [[pdf]](https://arxiv.org/abs/1505.00387) (**Highway Networks**) 10 | 1. - [x] GregorK, LecunY. Learning fast approximations of sparse coding ICML 2010 11 | 1. - [x] KaimingHe,XiangyuZhang,ShaoqingRen,JianSun Delving Deep into Rectifiers:SurpassingHuman-Level Performance on ImageNet Classification ICCV2015 12 | 1. - [x] K He,XZhang,SRen,JSun Deep Residual Learning for Image Recognition CVPR 2016 13 | 1. - [x] K He,XZhang,SRen,JSun Identity Mappings in Deep Residual Networks ECCV 2016 14 | 1. - [x] VeitA, Wilber M, BelongieS. Residual Networks Behave Like Ensembles of Relatively Shallow Networks. NIPS2016 15 | 1. - [x] RK Srivastava,KGre,JSchmidhuberHighway Networks 16 | 1. - [x] GreffK, Srivastava R K, SchmidhuberJ. Highway and Residual Networks learn Unrolled Iterative Estimation. ICLR2017. 17 | 1. - [x] QianliLiao, Tomaso PoggioBridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex 18 | 1. - [x] G Huang,ZLiu, KQ WeinbergeDensely Connected Convolutional Networks CVPR2017 19 | 1. - [x] G Larsson, M Maire, G ShakhnarovichFractalNet:Ultra-DeepNeural Networks without Residuals ICLR2017 20 | 1. - [x] Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, Kilian Weinberger Deep Networks with Stochastic Depth ECCV2016 21 | 1. - [x] XieS, GirshickR, DollárP, et al. Aggregated Residual Transformations for Deep Neural Networks. CVPR2016. 22 | 1. - [x] GregorK, LecunY. Learning fast approximations of sparse coding ICML 2010 23 | 1. - [x] BalduzziD, FreanM, Leary L, et al. The Shattered Gradients Problem: If resnetsare the answer, then what is the question? ICML2017. 24 | 1. - [x] ZagoruykoS, KomodakisN. DiracNets: Training Very Deep Neural Networks Without Skip-Connections[J]. 2017. 25 | 1. - [x] Improving training of deep neural networks via Singular Value Bounding 26 | 1. - [x] Liming Zhao Jingdong Wang Xi Li Zhuowen Tu Wenjun Zeng. Deep Convolutional Neural Networks with Merge-and-Run Mappings 27 | 1. - [x] TheReversibleResidualNetwork: BackpropagationWithoutStoringActivations 28 | 1. - [ ] MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications 29 | 1. - [ ] Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation 30 | 1. - [ ] MULTI-SCALE DENSE NETWORKS FOR RESOURCE EFFICIENT IMAGE CLASSIFICATION 31 | 1. - [ ] CRESCENDONET: A NEW DEEP CONVOLUTIONAL NEURAL NETWORK WITH ENSEMBLE BEHAVIOR 32 | 1. - [ ] CondenseNet: An Efficient DenseNet using Learned Group Convolutions 33 | -------------------------------------------------------------------------------- /Diffusion Maps/Diffusion maps.md: -------------------------------------------------------------------------------- 1 | # Diffusion Maps 2 | 3 | > A General manifold learning framework by R.R.Coifman 4 | 5 | Idea: Building the bridge between random walk and Diffusion on manifold 6 | 7 | ### Related Papers: 8 | - Nadler B, Lafon S, Coifman R R, et al. Diffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck operators 9 | - Coifman R R, Lafon S, Lee A B, et al. Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps 10 | - Nadler B, Lafon S, Coifman R R, et al. Diffusion maps, spectral clustering and reaction coordinates of dynamical systems 11 | - Coifman R R, Lafon S. Diffusion maps 12 | 13 | $M=D^{-1}L$ adjoint to $D^{1/2}MD^{-1/2}$(symmetric) 14 | 15 | left and right eignvectors are bi-orhtnormal 16 | 17 | Generate Operator: $H_f=lim (T_f-I)/eps$ 18 | 19 | -------------------------------------------------------------------------------- /Diffusion Maps/HarmonicOnManifold/GraphCNN.md: -------------------------------------------------------------------------------- 1 | 2 | #### Reference: 3 | - https://github.com/mdeff/cnn_graph 4 | - 3d deep learning tutorial in cvpr2017:http://3ddl.stanford.edu/ 5 | - geometric deep learning tutorial in cvpr2017:http://geometricdeeplearning.com/ 6 | 7 | **Review Paper** 8 | M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, P. Vandergheynst, Geometric deep learning: going beyond Euclidean data, IEEE Signal Processing Magazine 2017 (Review paper) http://cims.nyu.edu/~bruna/Media/graph_cnn_ieee.pdf 9 | 10 | **Paper list** 11 | R. Levie*, F. Monti*, X. Bresson, M. M. Bronstein, CayleyNets: Graph convolutional neural networks with complex rational spectral filters, 2017 (CayleyNet framework) 12 | 13 | F. Monti, X. Bresson, M. M. Bronstein, Geometric matrix completion with recurrent multi-graph neural networks, NIPS 2017 (CNNs on multiple graphs) [CODE] 14 | 15 | Y. Seo, M. Defferrard , P. Vandergheynst, X. Bresson, Structured Sequence Modeling with Graph Convolutional Recurrent Networks , 2016 (recurrent single graph CNN) 16 | 17 | L. Yi, H. Su, X. Guo, L. Guibas, SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation , CVPR 2017 (spectral transformer networks) 18 | 19 | F. Monti*, D. Boscaini*, J. Masci, E. Rodolà, J. Svoboda, M. M. Bronstein, Geometric deep learning on graphs and manifolds using mixture model CNNs, CVPR 2017 (MoNet framework) [CODE] [VIDEO] 20 | 21 | T. Kipf, M. Welling, Semi-supervised Classification with Graph Convolutional Networks, ICLR 2017 (simplification of ChebNet) [CODE] 22 | 23 | M. Defferrard, X. Bresson, P. Vandergheynst, Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering, NIPS 2017 (ChebNet framework) [CODE] 24 | 25 | D. Boscaini, J. Masci, E. Rodolà, M. M. Bronstein, Learning shape correspondence with anisotropic convolutional neural networks, NIPS 2016 (Anisotropic CNN framework) 26 | 27 | J. Masci, D. Boscaini, M. M. Bronstein, P. Vandergheynst, Geodesic convolutional neural networks on Riemannian manifolds, 3dRR 2015 (Geodesic CNN framework) 28 | 29 | D. Duvenaud, D. Maclaurin, J. Aguilera-Iparraguirre, R. Gomez-Bombarelli, T. Hirzel, A. Aspuru-Guzik, R. P. Adams, Convolutional Networks on Graphs for Learning Molecular Fingerprints, NIPS 2015 (molecular fingerprints using graph CNNs) 30 | 31 | J. Atwood, D. Towsley, Diffusion-Convolutional Neural Networks, 2015 32 | 33 | M. Henaff, J. Bruna, Y. LeCun: Deep Convolutional Networks on Graph-Structured Data, 2015 34 | 35 | J. Bruna, W. Zaremba, A. Szlam, Y. LeCun, Spectral Networks and Deep Locally Connected Networks on Graphs, ICLR 2014 (spectral CNN on graphs) 36 | 37 | F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, G. Monfardini, The graph neural network model, Trans. Neural Networks 20(1):61-80, 2009 (first neural networks on graphs) 38 | 39 | -------------------------------------------------------------------------------- /Diffusion Maps/Inferring interaction rules from observations of evolutive systems I: The variational approach.md: -------------------------------------------------------------------------------- 1 | # Inferring interaction rules from observations of evolutive systems I: The variational approach 2 | 3 | 4 | [arxiv link] 5 | 6 | @article{Bongini2017Inferring, 7 | title={Inferring interaction rules from observations of evolutive systems I: The variational approach}, 8 | author={Bongini, Mattia and Fornasier, Massimo and Hansen, Markus and Maggioni, Mauro}, 9 | journal={Mathematical Models & Methods in Applied Sciences}, 10 | year={2017}, 11 | } 12 | 13 | 14 | A gradient flow is $\dot x(t)\in -\partial_x J(x(t),t)$ controled by some function [a] 15 | 16 | Using optimal control to get the best [a] 17 | 18 | Preliminaries: 19 | - Optimal Transport 20 | - Wasserstein Distance 21 | - Mean-field Equation 22 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) [2017-2019] [2prime Lab] 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Optimization/DynamicSysmtemModelling/HJ.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /Optimization/Lowrank/Readme.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /Optimization/Opt For DL.md: -------------------------------------------------------------------------------- 1 | 1. - [ ] Riemannian approach to batch normalization 2 | 1. - [ ] The Marginal Value of Adaptive Gradient Methods in Machine Learning 3 | 1. - [ ] Entropy-SGD: Biasing Gradient Descent Into Wide Valleys 4 | 1. - [ ] Train faster, generalize better: Stability of stochastic gradient descent 5 | 1. - [x] Generalization Bounds of SGLD for Non-convex Learning: Two Theoretical Viewpoints 6 | 1. - [ ] A Variational Analysis of Stochastic Gradient Algorithms 7 | -------------------------------------------------------------------------------- /Other Topics/readme.md: -------------------------------------------------------------------------------- 1 | Deep Residual Networks and Weight Initialization 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Paper-Reading 2 | Paper Reading Reviews From 2prime. link 3 | 4 |
5 | 6 | **Attention:** Didn't upadted from 2017.10. *Out of date!* 7 | 8 | **New version will come out soon, with application in low level computer vision(denoising, compression, optical flow, stereo and etc.)** 9 | 10 | ## Topics: 11 | - Machine learning 12 | - Deep Learning Models 13 | - UQ 14 | - Dynamic Systems 15 | - optimization 16 | - Bayesian Stat 17 | - etc. 18 | 19 | 20 | Reference： 21 | 1. -[ ] https://github.com/WarBean/Unsupervised-Learning-Research 22 | 1. -[ ] https://github.com/joanbruna/stat212b 23 | -------------------------------------------------------------------------------- /computer vision/Object Detection.md: -------------------------------------------------------------------------------- 1 | 1. - [x] Girshick, Ross, et al. "**Rich feature hierarchies for accurate object detection and semantic segmentation**." Proceedings of the IEEE conference on computer vision and pattern recognition. (2014). [[pdf]](http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Girshick_Rich_Feature_Hierarchies_2014_CVPR_paper) (**RCNN**) 2 | 3 | 1. - [ ] He, K., et al. "**Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition**." IEEE Transactions on Pattern Analysis & Machine Intelligence. (2014). [[pdf]](https://arxiv.org/abs/1406.4729) (**SPPNet**) 4 | 5 | 1. - [x] Girshick, Ross. "**Fast r-cnn**." Proceedings of the IEEE International Conference on Computer Vision. 2015. [[pdf]](https://pdfs.semanticscholar.org/8f67/64a59f0d17081f2a2a9d06f4ed1cdea1a0ad) (**Fast RCNN**) 6 | 7 | 1. - [x] Ren, Shaoqing, et al. "**Faster R-CNN: Towards real-time object detection with region proposal networks**." Advances in neural information processing systems. 2015. [[pdf]](https://arxiv.org/pdf/1506.01497.pdf) (**Faster RCNN**) 8 | 9 | 1. - [ ] Redmon, Joseph, et al. "**You only look once: Unified, real-time object detection**." arXiv preprint arXiv:1506.02640 (2015). [[pdf]](http://homes.cs.washington.edu/~ali/papers/YOLO) (**YOLO**) 10 | 11 | 1. - [x] Liu, Wei, et al. "**SSD: Single Shot MultiBox Detector**." arXiv preprint arXiv:1512.02325 (2015). [[pdf]](http://arxiv.org/pdf/1512.02325)(**SSD**) 12 | 13 | 1. - [x] Hong, Sanghoon, et al. "PVANet: Lightweight Deep Neural Networks for Real-time Object Detection." arXiv:1611.08588 (2016). [[pdf]](https://arxiv.org/abs/1611.08588)(**PVANet**) 14 | 15 | 1. - [ ] Dai, Jifeng, et al. "**R-FCN: Object Detection via Region-based Fully Convolutional Networks**." arXiv preprint arXiv:1605.06409 (2016). [[pdf]](https://arxiv.org/abs/1605.06409) (**R-FCN**) 16 | 17 | 1. - [x] Lin, Tsung Yi, et al. "**Feature Pyramid Networks for Object Detection**." (2016). [[pdf]](https://arxiv.org/abs/1612.03144) (**FPN**) 18 | 19 | 1. - [ ] He, Gkioxari, et al. "**Mask R-CNN**." arXiv preprint arXiv:1703.06870 (2017). [[pdf]](https://arxiv.org/abs/1703.06870) (**Mask RCNN**) 20 | 21 | 1. - [x] Lin, Tsung Yi, et al. "**Focal Loss for Dense Object Detection**." (2017). [[pdf]](https://arxiv.org/abs/1708.02002) (**Focal Loss**) 22 | 23 | **Review** 24 | 25 | 1. - [x] Huang, Jonathan, et al. "**Speed/accuracy trade-offs for modern convolutional object detectors**." (2016).. [[pdf]](https://arxiv.org/abs/1611.10012) 26 | -------------------------------------------------------------------------------- /computer vision/Style Transfer.md: -------------------------------------------------------------------------------- 1 | 1. - [x] Texture synthesis using convolutional neural networks 2 | 1. - [ ] A neural algorithm of artistic style 3 | 1. - [ ] Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis 4 | 1. - [ ] Texture networks: Feed-forward synthesis of textures and stylized images 5 | 1. - [ ] A Learned Representation For Artistic Style 6 | 1. - [ ] Fast Patch-based Style Transfer of Arbitrary Style 7 | 1. - [ ] Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization 8 | 1. - [ ] Visual Attribute Transfer through Deep Image Analogy 9 | 1. - [x] Demystifying Neural Sytle Transfer 10 | 11 | ### Review 12 | 1. - [ ] Neural Style Transfer: A Review arXiv:1703.06870 13 | -------------------------------------------------------------------------------- /computer vision/denoising.md: -------------------------------------------------------------------------------- 1 | # Image Denoising State-of-the-art 2 | 3 | A curated list of image denoising resources and a benchmark for image denoising approaches. 4 | 5 | ## State-of-the-art algorithms 6 | #### Filter 7 | * BD3M [[Web]](http://www.cs.tut.fi/~foi/GCF-BM3D/) [[Code]](http://www.cs.tut.fi/~foi/GCF-BM3D/BM3D.zip) [[PDF]](http://www.cs.tut.fi/~foi/GCF-BM3D/SPIE08_deblurring.pdf) 8 | * Image restoration by sparse 3D transform-domain collaborative filtering (SPIE Electronic Imaging 2008), Dabov et al. 9 | * Activity-tuned Image Filtering [[PDF]](https://arxiv.org/pdf/1707.02637.pdf) 10 | * Local Activity-tuned Image Filtering for Noise Removal and Image Smoothing (Arxiv 2017), Lijun Zhao, Jie Liang, Huihui Bai, Lili Meng, Anhong Wang, and Yao Zhao. 11 | 12 | #### Sparse Coding 13 | * KSVD [[Web]](http://www.cs.technion.ac.il/~ronrubin/software.html) [[Code]](https://github.com/jbhuang0604/SelfSimSR/tree/master/Lib/KSVD) [[PDF]](http://www.egr.msu.edu/~aviyente/elad06.pdf) 14 | * Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries (TIP2006), Elad et al. 15 | * SAINT [[Web]](http://see.xidian.edu.cn/faculty/wsdong/wsdong_Publication.htm) [Code] [[PDF]](http://see.xidian.edu.cn/faculty/wsdong/Papers/Journal/TIP_LASSC.pdf) 16 | * Nonlocal image restoration with bilateral variance estimation: a low-rank approach (TIP2013), Dong et al. 17 | * NCSR [[Web]](http://www4.comp.polyu.edu.hk/~cslzhang/NCSR.htm) [[Code]](http://www4.comp.polyu.edu.hk/~cslzhang/code/NCSR.rar) [[PDF]](http://www4.comp.polyu.edu.hk/~cslzhang/paper/NCSR_TIP_final.pdf) 18 | * Nonlocally Centralized Sparse Representation for Image Restoration (TIP2012), Dong et al. 19 | * LSSC [[Web]](http://www.di.ens.fr/~fbach/) [Code] [[PDF]](http://www.di.ens.fr/~fbach/iccv09_mairal.pdf) 20 | * Non-local Sparse Models for Image Restoration (ICCV2009), Mairal et al. 21 | 22 | #### Effective Prior 23 | * EPLL [[Web]](https://people.csail.mit.edu/danielzoran/) [[Code]](https://people.csail.mit.edu/danielzoran/epllcode.zip) [[PDF]](http://people.ee.duke.edu/~lcarin/EPLICCVCameraReady.pdf) 24 | * From Learning Models of Natural Image Patches to Whole Image Restoration (ICCV2011), Zoran et al. 25 | * Bayesian Hyperprior [[PDF]](https://arxiv.org/pdf/1706.03261.pdf) 26 | * A Bayesian Hyperprior Approach for Joint Image Denoising and Interpolation with an Application to HDR Imaging, Cecilia Aguerrebere, Andres Almansa, Julie Delon, Yann Gousseau and Pablo Muse. 27 | * External Prior Guided [[PDF]](https://arxiv.org/pdf/1705.04505.pdf) 28 | * External Prior Guided Internal Prior Learning for Real Noisy Image Denoising, Jun Xu, Lei Zhang, and David Zhang. 29 | * Multi-Layer Image Representation [[PDF]](https://arxiv.org/pdf/1707.02194.pdf) 30 | * A Multi-Layer Image Representation Using Regularized Residual Quantization: Application to Compression and Denoising, Sohrab Ferdowsi, Slava Voloshynovskiy, Dimche Kostadinov. 31 | * A Faster Patch Ordering [[PDF]](https://arxiv.org/ftp/arxiv/papers/1704/1704.08090.pdf) 32 | * A Faster Patch Ordering Method for Image Denoising, Badre Munir. 33 | 34 | #### Low Rank 35 | * WNNM [[Web]](https://sites.google.com/site/shuhanggu/home) [[Code]](http://www4.comp.polyu.edu.hk/~cslzhang/code/WNNM_code.zip) [[PDF]](https://pdfs.semanticscholar.org/6d55/6272625b672ba54b5ab3d9e6474088a4b78f.pdf) 36 | * Weighted Nuclear Norm Minimization with Application to Image Denoising (CVPR2014), Gu et al. 37 | * Multi-channel Weighted Nuclear Norm [[PDF]](https://arxiv.org/pdf/1705.09912.pdf) 38 | * Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising (Arxiv 2017), Jun Xu, Lei Zhang, David Zhang, and Xiangchu Feng. 39 | 40 | #### Deep Learning 41 | * TNRD [[Web]](http://www.icg.tugraz.at/Members/Chenyunjin/about-yunjin-chen) [[Code]](https://www.dropbox.com/s/8j6b880m6ddxtee/TNRD-Codes.zip?dl=0) [[PDF]](https://arxiv.org/pdf/1508.02848.pdf) 42 | * Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration (TPAMI2016), Chen et al. 43 | * DnCNN [[Web]](https://github.com/cszn/DnCNN) [[PDF]](https://arxiv.org/pdf/1608.03981v1.pdf) 44 | * Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising (TIP2017), Zhang et al. 45 | * DAAM [[Web]](https://arxiv.org/abs/1612.06508) [[PDF]](https://arxiv.org/pdf/1612.06508.pdf) 46 | * Deeply Aggregated Alternating Minimization for Image Restoration (Arxiv2016), Youngjung Kim et al. 47 | * Adversirial Denoising [[PDF]](https://arxiv.org/pdf/1708.00159.pdf) 48 | * Image Denoising via CNNs: An Adversarial Approach (Arxiv2017), Nithish Divakar, R. Venkatesh Babu. 49 | * Unrolled Optimization Deep Priors [[PDF]](https://arxiv.org/pdf/1705.08041.pdf) 50 | * Unrolled Optimization with Deep Priors (Arxiv2017), Steven Diamond, Vincent Sitzmann, Felix Heide, Gordon Wetzstein. 51 | * Wider Network [[PDF]](https://arxiv.org/pdf/1707.05414.pdf) 52 | * Going Wider with Convolution for Image Denoising (Arxiv2017), Peng Liu, Ruogu Fang. 53 | * Recurrent Inference Machines [[PDF]](https://arxiv.org/pdf/1706.04008.pdf) 54 | * Recurrent Inference Machines for Solving Inverse Problems, Patrick Putzky, Max Welling. 55 | * Learning Pixel-Distribution Prior 56 | * Learning Pixel-Distribution Prior with Wider Convolution for Image Denoising (Arxiv2017), Peng Liu, Ruogu Fang. 57 | 58 | #### Combined with High-Level Tasks 59 | * Meets High-level Tasks [[PDF]](https://arxiv.org/pdf/1706.04284.pdf) 60 | * When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach, Ding Liu (Arxiv2017), Bihan Wen, Xianming Liu, Thomas S. Huang. 61 | * Class-Specific Denoising 62 | * Class-Specific Poisson Denoising By Patch-Based Importance Sampling (Arxiv2017), Milad Niknejad, Jose M. Bioucas-Dias, Mario A. T. Figueiredo. 63 | 64 | #### Benchmark 65 | * Benchmark [[PDF]](https://arxiv.org/pdf/1707.01313.pdf) 66 | * Benchmarking Denoising Algorithms with Real Photographs (Arxiv2017), Tobias Plotz, Stefan Roth. 67 | -------------------------------------------------------------------------------- /generative models/GAN.md: -------------------------------------------------------------------------------- 1 | # AdversarialNetsPapers 2 | The classical Papers about adversarial nets 3 | 4 | The First paper 5 | -------------------------------------------- 6 | :white_check_mark: [Generative Adversarial Nets] [[Paper]](https://arxiv.org/abs/1406.2661) 7 | [[Code]](https://github.com/goodfeli/adversarial)(the first paper about it) 8 | 9 | ## Unclassified 10 | 11 | :white_check_mark: [Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks] [[Paper]](https://arxiv.org/abs/1506.05751)[[Code]](https://github.com/facebook/eyescream) 12 | 13 | :white_check_mark: [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks] [[Paper]](https://arxiv.org/abs/1511.06434)[[Code]](https://github.com/jacobgil/keras-dcgan)(Gan with convolutional networks)(ICLR) 14 | 15 | :white_check_mark: [Adversarial Autoencoders] [[Paper]](http://arxiv.org/abs/1511.05644)[[Code]](https://github.com/musyoku/adversarial-autoencoder) 16 | 17 | 18 | :white_check_mark: [Generating Images with Perceptual Similarity Metrics based on Deep Networks] [[Paper]](https://arxiv.org/pdf/1602.02644v2.pdf) 19 | 20 | 21 | :white_check_mark: [Generating images with recurrent adversarial networks] [[Paper]](https://arxiv.org/abs/1602.05110)[[Code]](https://github.com/ofirnachum/sequence_gan) 22 | 23 | :white_check_mark: [Generative Visual Manipulation on the Natural Image Manifold] [[Paper]](https://people.eecs.berkeley.edu/~junyanz/projects/gvm/eccv16_gvm.pdf)[[Code]](https://github.com/junyanz/iGAN) 24 | 25 | :white_check_mark: [Generative Adversarial Text to Image Synthesis] [[Paper]](https://arxiv.org/abs/1605.05396)[[Code]](https://github.com/reedscot/icml2016)[[code]](https://github.com/paarthneekhara/text-to-image) 26 | 27 | 28 | :white_check_mark: [Learning What and Where to Draw] [[Paper]](http://www.scottreed.info/files/nips2016.pdf)[[Code]](https://github.com/reedscot/nips2016) 29 | 30 | :white_check_mark: [Adversarial Training for Sketch Retrieval] [[Paper]](http://link.springer.com/chapter/10.1007/978-3-319-46604-0_55) 31 | 32 | :white_check_mark: [Generative Image Modeling using Style and Structure Adversarial Networks] [[Paper]](https://arxiv.org/pdf/1603.05631.pdf)[[Code]](https://github.com/xiaolonw/ss-gan) 33 | 34 | :white_check_mark: [Generative Adversarial Networks as Variational Training of Energy Based Models] [[Paper]](http://www.mathpubs.com/detail/1611.01799v1/Generative-Adversarial-Networks-as-Variational-Training-of-Energy-Based-Models)(ICLR 2017) 35 | 36 | :white_check_mark: [Adversarial Training Methods for Semi-Supervised Text Classification] [[Paper]](https://arxiv.org/abs/1605.07725)[[Note]](https://github.com/dennybritz/deeplearning-papernotes/blob/master/notes/adversarial-text-classification.md)( Ian Goodfellow Paper) 37 | 38 | :white_check_mark: [Learning from Simulated and Unsupervised Images through Adversarial Training] [[Paper]](https://arxiv.org/abs/1612.07828)[[code]](https://github.com/carpedm20/simulated-unsupervised-tensorflow)（Apple paper） 39 | 40 | :white_check_mark: [Synthesizing the preferred inputs for neurons in neural networks via deep generator networks] [[Paper]](https://arxiv.org/pdf/1605.09304v5.pdf)[[Code]](https://github.com/Evolving-AI-Lab/synthesizing) 41 | 42 | :white_check_mark: [SalGAN: Visual Saliency Prediction with Generative Adversarial Networks] [[Paper]](https://arxiv.org/abs/1701.01081)[[Code]](https://github.com/imatge-upc/saliency-salgan-2017) 43 | 44 | 45 | :white_check_mark: [Adversarial Feature Learning] [[Paper]](https://arxiv.org/abs/1605.09782) 46 | 47 | ## Ensemble 48 | 49 | :white_check_mark: [AdaGAN: Boosting Generative Models] [[Paper]](https://arxiv.org/abs/1701.02386)[[Code]]（Google Brain） 50 | 51 | ## Clustering 52 | 53 | :white_check_mark: [Unsupervised Learning Using Generative Adversarial Training And Clustering] [[Paper]](https://openreview.net/forum?id=SJ8BZTjeg¬eId=SJ8BZTjeg)[[Code]](https://github.com/VittalP/UnsupGAN)(ICLR) 54 | :white_check_mark: [Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks] [[Paper]](https://arxiv.org/abs/1511.06390)(ICLR) 55 | 56 | ## Image Inpainting 57 | 58 | :white_check_mark: [Semantic Image Inpainting with Perceptual and Contextual Losses] [[Paper]](https://arxiv.org/abs/1607.07539)[[Code]](https://github.com/bamos/dcgan-completion.tensorflow) 59 | 60 | :white_check_mark: [Context Encoders: Feature Learning by Inpainting] [[Paper]](https://arxiv.org/abs/1604.07379)[[Code]](https://github.com/jazzsaxmafia/Inpainting) 61 | 62 | :white_check_mark: [Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks] [[Paper]](https://arxiv.org/abs/1611.06430v1) 63 | 64 | ## Joint Probability 65 | 66 | :white_check_mark: [Adversarially Learned Inference][[Paper]](https://arxiv.org/abs/1606.00704)[[Code]](https://github.com/IshmaelBelghazi/ALI) 67 | 68 | ## Super-Resolution 69 | 70 | :white_check_mark: [Image super-resolution through deep learning ][[Code]](https://github.com/david-gpu/srez)(Just for face dataset) 71 | 72 | :white_check_mark: [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network] [[Paper]](https://arxiv.org/abs/1609.04802)[[Code]](https://github.com/leehomyc/Photo-Realistic-Super-Resoluton)（Using Deep residual network） 73 | 74 | :white_check_mark: [EnhanceGAN] [[Docs]](https://medium.com/@richardherbert/faces-from-noise-super-enhancing-8x8-images-with-enhancegan-ebda015bb5e0#.io6pskvin)[[Code]] 75 | 76 | 77 | ## Disocclusion 78 | 79 | :white_check_mark: [Robust LSTM-Autoencoders for Face De-Occlusion in the Wild] [[Paper]](https://arxiv.org/abs/1612.08534) 80 | 81 | ## Semantic Segmentation 82 | 83 | :white_check_mark: [Semantic Segmentation using Adversarial Networks] [[Paper]](https://arxiv.org/abs/1611.08408)（soumith's paper） 84 | 85 | ## Object Detection 86 | 87 | :white_check_mark: [Perceptual generative adversarial networks for small object detection] [[Paper]]（Submitted） 88 | 89 | :white_check_mark: [A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection] [[Paper]](http://abhinavsh.info/papers/pdfs/adversarial_object_detection.pdf)(CVPR2017) 90 | 91 | ## RNN 92 | 93 | :white_check_mark: [C-RNN-GAN: Continuous recurrent neural networks with adversarial training] [[Paper]](https://arxiv.org/abs/1611.09904)[[Code]](https://github.com/olofmogren/c-rnn-gan) 94 | 95 | ## Conditional adversarial 96 | 97 | :white_check_mark: [Conditional Generative Adversarial Nets] [[Paper]](https://arxiv.org/abs/1411.1784)[[Code]](https://github.com/zhangqianhui/Conditional-Gans) 98 | 99 | :white_check_mark: [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets] [[Paper]](https://arxiv.org/abs/1606.03657)[[Code]](https://github.com/buriburisuri/supervised_infogan) 100 | 101 | :white_check_mark: [Image-to-image translation using conditional adversarial nets] [[Paper]](https://arxiv.org/pdf/1611.07004v1.pdf)[[Code]](https://github.com/phillipi/pix2pix)[[Code]](https://github.com/yenchenlin/pix2pix-tensorflow) 102 | 103 | :white_check_mark: [Conditional Image Synthesis With Auxiliary Classifier GANs] [[Paper]](https://arxiv.org/abs/1610.09585)[[Code]](https://github.com/buriburisuri/ac-gan)(GoogleBrain ICLR 2017) 104 | 105 | :white_check_mark: [Pixel-Level Domain Transfer] [[Paper]](https://arxiv.org/pdf/1603.07442v2.pdf)[[Code]](https://github.com/fxia22/pldtgan) 106 | 107 | :white_check_mark: [Invertible Conditional GANs for image editing] [[Paper]](https://arxiv.org/abs/1611.06355)[[Code]](https://github.com/Guim3/IcGAN) 108 | 109 | :white_check_mark: [Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space] [[Paper]](https://arxiv.org/abs/1612.00005v1)[[Code]](https://github.com/Evolving-AI-Lab/ppgn) 110 | 111 | :white_check_mark: [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks] [[Paper]](https://arxiv.org/pdf/1612.03242v1.pdf)[[Code]](https://github.com/hanzhanggit/StackGAN) 112 | 113 | :white_check_mark: [Unsupervised Image-to-Image Translation with Generative Adversarial Networks] [[Paper]](https://arxiv.org/pdf/1701.02676.pdf) 114 | 115 | :white_check_mark: [Learning to Discover Cross-Domain Relations with Generative Adversarial Networks] [[Paper]](https://arxiv.org/abs/1703.05192)[[Code]](https://github.com/carpedm20/DiscoGAN-pytorch) 116 | 117 | ## Video Prediction 118 | 119 | :white_check_mark: [Deep multi-scale video prediction beyond mean square error] [[Paper]](https://arxiv.org/abs/1511.05440)[[Code]](https://github.com/dyelax/Adversarial_Video_Generation)(Yann LeCun's paper) 120 | 121 | :white_check_mark: [Unsupervised Learning for Physical Interaction through Video Prediction] [[Paper]](https://arxiv.org/abs/1605.07157)(Ian Goodfellow's paper) 122 | 123 | :white_check_mark: [Generating Videos with Scene Dynamics] [[Paper]](https://arxiv.org/abs/1609.02612)[[Web]](http://web.mit.edu/vondrick/tinyvideo/)[[Code]](https://github.com/cvondrick/videogan) 124 | 125 | ## Texture Synthesis & style transfer 126 | 127 | :white_check_mark: [Precomputed real-time texture synthesis with markovian generative adversarial networks] [[Paper]](https://arxiv.org/abs/1604.04382)[[Code]](https://github.com/chuanli11/MGANs)(ECCV 2016) 128 | 129 | 130 | ## GAN Theory 131 | 132 | :white_check_mark: [Energy-based generative adversarial network] [[Paper]](https://arxiv.org/pdf/1609.03126v2.pdf)[[Code]](https://github.com/buriburisuri/ebgan)(Lecun paper) 133 | 134 | :white_check_mark: [Improved Techniques for Training GANs] [[Paper]](https://arxiv.org/abs/1606.03498)[[Code]](https://github.com/openai/improved-gan)(Goodfellow's paper) 135 | 136 | :white_check_mark: [Mode RegularizedGenerative Adversarial Networks] [[Paper]](https://openreview.net/pdf?id=HJKkY35le)(Yoshua Bengio , ICLR 2017) 137 | 138 | :white_check_mark: [Improving Generative Adversarial Networks with Denoising Feature Matching] [[Paper]](https://openreview.net/pdf?id=S1X7nhsxl)[[Code]](https://github.com/hvy/chainer-gan-denoising-feature-matching)(Yoshua Bengio , ICLR 2017) 139 | 140 | :white_check_mark: [Sampling Generative Networks] [[Paper]](https://arxiv.org/abs/1609.04468)[[Code]](https://github.com/dribnet/plat) 141 | 142 | :white_check_mark: [Mode Regularized Generative Adversarial Networkss] [[Paper]](https://arxiv.org/abs/1612.02136)( Yoshua Bengio's paper) 143 | 144 | :white_check_mark: [How to train Gans] [[Docu]](https://github.com/soumith/ganhacks#authors) 145 | 146 | :white_check_mark: [Towards Principled Methods for Training Generative Adversarial Networks] [[Paper]](http://openreview.net/forum?id=Hk4_qw5xe)(ICLR 2017) 147 | 148 | :white_check_mark: [Unrolled Generative Adversarial Networks] [[Paper]](https://arxiv.org/abs/1611.02163)[[Code]](https://github.com/poolio/unrolled_gan) 149 | 150 | :white_check_mark: [Wasserstein GAN] [[Paper]](https://arxiv.org/abs/1701.07875)[[Code]](https://github.com/martinarjovsky/WassersteinGAN) 151 | 152 | :white_check_mark: [Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities] [[Paper]](https://arxiv.org/abs/1701.06264)[[Code]](https://github.com/guojunq/lsgan)(The same as WGan) 153 | 154 | :white_check_mark: [Towards Principled Methods for Training Generative Adversarial Networks] [[Paper]](https://arxiv.org/abs/1701.04862) 155 | 156 | 157 | ## 3D 158 | 159 | :white_check_mark: [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling] [[Paper]](https://arxiv.org/abs/1610.07584)[[Web]](http://3dgan.csail.mit.edu/)[[code]](https://github.com/zck119/3dgan-release)(2016 NIPS) 160 | 161 | ## Face Generative and Editing 162 | 163 | :white_check_mark: [Autoencoding beyond pixels using a learned similarity metric] [[Paper]](https://arxiv.org/abs/1512.09300)[[code]](https://github.com/andersbll/autoencoding_beyond_pixels) 164 | 165 | :white_check_mark: [Coupled Generative Adversarial Networks] [[Paper]](http://mingyuliu.net/)[[Caffe Code]](https://github.com/mingyuliutw/CoGAN)[[Tensorflow Code]](https://github.com/andrewliao11/CoGAN-tensorflow)（NIPS） 166 | 167 | :white_check_mark: [Invertible Conditional GANs for image editing] [[Paper]](https://drive.google.com/file/d/0B48XS5sLi1OlRkRIbkZWUmdoQmM/view)[[Code]](https://github.com/Guim3/IcGAN) 168 | 169 | :white_check_mark: [Learning Residual Images for Face Attribute Manipulation] [[Paper]](https://arxiv.org/abs/1612.05363) 170 | 171 | :white_check_mark: [Neural Photo Editing with Introspective Adversarial Networks] [[Paper]](https://arxiv.org/abs/1609.07093)[[Code]](https://github.com/ajbrock/Neural-Photo-Editor)(ICLR 2017) 172 | 173 | ## For discrete distributions 174 | 175 | :white_check_mark: [Maximum-Likelihood Augmented Discrete Generative Adversarial Networks] [[Paper]](https://arxiv.org/abs/1702.07983v1) 176 | 177 | :white_check_mark: [Boundary-Seeking Generative Adversarial Networks] [[Paper]](https://arxiv.org/abs/1702.08431) 178 | 179 | :white_check_mark: [GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution] [[Paper]](https://arxiv.org/abs/1611.04051) 180 | 181 | # Project 182 | 183 | :white_check_mark: [cleverhans] [[Code]](https://github.com/openai/cleverhans)(A library for benchmarking vulnerability to adversarial examples) 184 | 185 | :white_check_mark: [reset-cppn-gan-tensorflow] [[Code]](https://github.com/hardmaru/resnet-cppn-gan-tensorflow)(Using Residual Generative Adversarial Networks and Variational Auto-encoder techniques to produce high resolution images) 186 | 187 | :white_check_mark: [HyperGAN] [[Code]](https://github.com/255bits/HyperGAN)(Open source GAN focused on scale and usability) 188 | 189 | # Blogs 190 | | Author | Address | 191 | |:----:|:---:| 192 | | **inFERENCe** | [Adversarial network](http://www.inference.vc/) | 193 | | **inFERENCe** | [InfoGan](http://www.inference.vc/infogan-variational-bound-on-mutual-information-twice/) | 194 | | **distill** | [Deconvolution and Image Generation](http://distill.pub/2016/deconv-checkerboard/) | 195 | | **yingzhenli** | [Gan theory](http://www.yingzhenli.net/home/blog/?p=421http://www.yingzhenli.net/home/blog/?p=421) | 196 | | **OpenAI** | [Generative model](https://openai.com/blog/generative-models/) | 197 | 198 | 199 | # Other 200 | 201 | :white_check_mark: [1] http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf (NIPS Goodfellow Slides)[[Chinese Trans]](http://c.m.163.com/news/a/C7UE2MLT0511AQHO.html?spss=newsapp&spsw=1)[[details]](https://arxiv.org/pdf/1701.00160v1.pdf) 202 | 203 | :white_check_mark: [2] [[PDF]](https://drive.google.com/file/d/0BxKBnD5y2M8NbzBUbXRwUDBZOVU/view)(NIPS Lecun Slides) 204 | 205 | 206 | 207 | -------------------------------------------------------------------------------- /generative models/readme.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /learningtheory/approximation(NN).md: -------------------------------------------------------------------------------- 1 | 1. - [ ] How regularization affects the critical points in linear networks 2 | 1. - [ ] Norm-Based Capacity Control in Neural Networks 3 | 1. - [ ] Exploring Generalization in Deep Learning 4 | 1. - [ ] Convexified Convolutional Neural Networks 5 | 1. - [ ] The Expressive Power of Neural Networks: A View from the Width 6 | 1. - [ ] Approximation and estimation bounds for artificial neural networks. 7 | 1. - [ ] On the Expressive Power of Deep Learning: A Tensor Analysis. 8 | 1. - [ ] The Power of Depth for Feedforward Neural Networks 9 | 1. - [ ] Why deep neural networks for funtion approximation? 10 | 1. - [ ] Benefits of depth in neural networks 11 | 1. - [ ] Error bounds for approximations with deep ReLU networks 12 | 1. - [ ] Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity 13 | 1. - [ ] Breaking the curse of dimensionality with convex neural networks. 14 | 1. - [ ] Learning polynomials with neural networks. 15 | 1. - [ ] Provable bounds for learning some deep representations. 16 | 1. - [ ] The Loss Surfaces of Multilayer Networks 17 | 1. - [ ] Short and Deep: Sketching and Neural Networks 18 | 1. - [ ] On the energy landscape of deep networks 19 | -------------------------------------------------------------------------------- /learningtheory/dl.md: -------------------------------------------------------------------------------- 1 | 1. - [ ] ON LARGE-BATCH TRAINING FOR DEEP LEARNING: GENERALIZATION GAP AND SHARP MINIMA 2 | 1. - [ ] Mixing Complexity and its Applications to Neural Networks 3 | 1. - [ ] Opening the black box of Deep Neural Networks via Information 4 | 1. - [ ] On the Emergence of Invariance and Disentangling in Deep Representations 5 | 1. - [ ] Visual Representations: Defining Properties and Deep Approximations 6 | 7 | --------------------------------------------------------------------------------