├── _data ├── options.yml └── social.yml ├── .gitignore ├── flow ├── iaf.PNG ├── iaf.png ├── maf.PNG ├── maf.png ├── flow-graphical.PNG ├── flow-graphical.png └── index.md ├── gan ├── gan.png ├── cyclegan_gendisc.png ├── index.md └── index.tex ├── vae ├── klgap.png └── vae.png ├── docs ├── flow │ ├── iaf.PNG │ ├── maf.PNG │ └── flow-graphical.PNG ├── gan │ ├── gan.png │ ├── cyclegan_gendisc.png │ └── index.tex ├── vae │ ├── klgap.png │ └── vae.png ├── fonts │ ├── icomoon.eot │ ├── icomoon.ttf │ ├── icomoon.woff │ ├── et-book │ │ ├── et-book-bold-line-figures │ │ │ ├── et-book-bold-line-figures.eot │ │ │ ├── et-book-bold-line-figures.ttf │ │ │ └── et-book-bold-line-figures.woff │ │ ├── et-book-roman-line-figures │ │ │ ├── et-book-roman-line-figures.eot │ │ │ ├── et-book-roman-line-figures.ttf │ │ │ └── et-book-roman-line-figures.woff │ │ ├── et-book-roman-old-style-figures │ │ │ ├── et-book-roman-old-style-figures.eot │ │ │ ├── et-book-roman-old-style-figures.ttf │ │ │ └── et-book-roman-old-style-figures.woff │ │ ├── et-book-semi-bold-old-style-figures │ │ │ ├── et-book-semi-bold-old-style-figures.eot │ │ │ ├── et-book-semi-bold-old-style-figures.ttf │ │ │ └── et-book-semi-bold-old-style-figures.woff │ │ └── et-book-display-italic-old-style-figures │ │ │ ├── et-book-display-italic-old-style-figures.eot │ │ │ ├── et-book-display-italic-old-style-figures.ttf │ │ │ └── et-book-display-italic-old-style-figures.woff │ ├── et-bembo │ │ ├── et-bembo-bold-line-figures │ │ │ ├── et-bembo-bold-line-figures.eot │ │ │ ├── et-bembo-bold-line-figures.ttf │ │ │ └── et-bembo-bold-line-figures.woff │ │ ├── et-bembo-roman-line-figures │ │ │ ├── et-bembo-roman-line-figures.eot │ │ │ ├── et-bembo-roman-line-figures.ttf │ │ │ └── et-bembo-roman-line-figures.woff │ │ ├── et-bembo-roman-old-style-figures │ │ │ ├── et-bembo-roman-old-style-figures.eot │ │ │ ├── et-bembo-roman-old-style-figures.ttf │ │ │ └── et-bembo-roman-old-style-figures.woff │ │ ├── et-bembo-semi-bold-old-style-figures │ │ │ ├── et-bembo-semi-bold-old-style-figures.eot │ │ │ ├── et-bembo-semi-bold-old-style-figures.ttf │ │ │ └── et-bembo-semi-bold-old-style-figures.woff │ │ └── et-bembo-display-italic-old-style-figures │ │ │ ├── et-bembo-display-italic-old-style-figures.eot │ │ │ ├── et-bembo-display-italic-old-style-figures.ttf │ │ │ └── et-bembo-display-italic-old-style-figures.woff │ └── icomoon.svg ├── autoregressive │ ├── nade.png │ ├── fvsbn.png │ ├── autoregressive.png │ └── index.tex ├── introduction │ ├── learning.png │ ├── learning_1.png │ ├── learning_2.png │ └── index.html ├── Makefile ├── LICENSE ├── index.html └── css │ └── tufte.orginal.css ├── fonts ├── icomoon.eot ├── icomoon.ttf ├── icomoon.woff ├── et-book │ ├── et-book-bold-line-figures │ │ ├── et-book-bold-line-figures.eot │ │ ├── et-book-bold-line-figures.ttf │ │ └── et-book-bold-line-figures.woff │ ├── et-book-roman-line-figures │ │ ├── et-book-roman-line-figures.eot │ │ ├── et-book-roman-line-figures.ttf │ │ └── et-book-roman-line-figures.woff │ ├── et-book-roman-old-style-figures │ │ ├── et-book-roman-old-style-figures.eot │ │ ├── et-book-roman-old-style-figures.ttf │ │ └── et-book-roman-old-style-figures.woff │ ├── et-book-semi-bold-old-style-figures │ │ ├── et-book-semi-bold-old-style-figures.eot │ │ ├── et-book-semi-bold-old-style-figures.ttf │ │ └── et-book-semi-bold-old-style-figures.woff │ └── et-book-display-italic-old-style-figures │ │ ├── et-book-display-italic-old-style-figures.eot │ │ ├── et-book-display-italic-old-style-figures.ttf │ │ └── et-book-display-italic-old-style-figures.woff ├── et-bembo │ ├── et-bembo-bold-line-figures │ │ ├── et-bembo-bold-line-figures.eot │ │ ├── et-bembo-bold-line-figures.ttf │ │ └── et-bembo-bold-line-figures.woff │ ├── et-bembo-roman-line-figures │ │ ├── et-bembo-roman-line-figures.eot │ │ ├── et-bembo-roman-line-figures.ttf │ │ └── et-bembo-roman-line-figures.woff │ ├── et-bembo-roman-old-style-figures │ │ ├── et-bembo-roman-old-style-figures.eot │ │ ├── et-bembo-roman-old-style-figures.ttf │ │ └── et-bembo-roman-old-style-figures.woff │ ├── et-bembo-semi-bold-old-style-figures │ │ ├── et-bembo-semi-bold-old-style-figures.eot │ │ ├── et-bembo-semi-bold-old-style-figures.ttf │ │ └── et-bembo-semi-bold-old-style-figures.woff │ └── et-bembo-display-italic-old-style-figures │ │ ├── et-bembo-display-italic-old-style-figures.eot │ │ ├── et-bembo-display-italic-old-style-figures.ttf │ │ └── et-bembo-display-italic-old-style-figures.woff └── icomoon.svg ├── autoregressive ├── nade.png ├── fvsbn.png ├── autoregressive.png ├── index.tex └── index.md ├── introduction ├── learning.png ├── learning_1.png ├── learning_2.png └── index.md ├── _layouts ├── page.html ├── default.html ├── full-width.html └── post.html ├── _includes ├── print-footer.html ├── header.html ├── footer.html └── head.html ├── Makefile ├── _plugins ├── marginnote.rb ├── sidenote.rb ├── newthought.rb ├── mathjaxtag.rb ├── main_column_img.rb ├── fullwidth.rb └── margin_figure.rb ├── _config.yml ├── _sass ├── _settings.scss ├── _syntax-highlighting.scss └── _fonts.scss ├── LICENSE ├── index.md ├── README.md └── css ├── tufte.css └── tufte.orginal.css /_data/options.yml: -------------------------------------------------------------------------------- 1 | mathjax: true 2 | lato_font_load: true -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | _site 2 | .sass-cache 3 | .DS\_Store 4 | config.codekit 5 | -------------------------------------------------------------------------------- /flow/iaf.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/flow/iaf.PNG -------------------------------------------------------------------------------- /flow/iaf.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/flow/iaf.png -------------------------------------------------------------------------------- /flow/maf.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/flow/maf.PNG -------------------------------------------------------------------------------- /flow/maf.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/flow/maf.png -------------------------------------------------------------------------------- /gan/gan.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/gan/gan.png -------------------------------------------------------------------------------- /vae/klgap.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/vae/klgap.png -------------------------------------------------------------------------------- /vae/vae.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/vae/vae.png -------------------------------------------------------------------------------- /docs/flow/iaf.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/flow/iaf.PNG -------------------------------------------------------------------------------- /docs/flow/maf.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/flow/maf.PNG -------------------------------------------------------------------------------- /docs/gan/gan.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/gan/gan.png -------------------------------------------------------------------------------- /docs/vae/klgap.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/vae/klgap.png -------------------------------------------------------------------------------- /docs/vae/vae.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/vae/vae.png -------------------------------------------------------------------------------- /fonts/icomoon.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/icomoon.eot -------------------------------------------------------------------------------- /fonts/icomoon.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/icomoon.ttf -------------------------------------------------------------------------------- /fonts/icomoon.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/icomoon.woff -------------------------------------------------------------------------------- /autoregressive/nade.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/autoregressive/nade.png -------------------------------------------------------------------------------- /docs/fonts/icomoon.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/icomoon.eot -------------------------------------------------------------------------------- /docs/fonts/icomoon.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/icomoon.ttf -------------------------------------------------------------------------------- /docs/fonts/icomoon.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/icomoon.woff -------------------------------------------------------------------------------- /flow/flow-graphical.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/flow/flow-graphical.PNG -------------------------------------------------------------------------------- /flow/flow-graphical.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/flow/flow-graphical.png -------------------------------------------------------------------------------- /autoregressive/fvsbn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/autoregressive/fvsbn.png -------------------------------------------------------------------------------- /gan/cyclegan_gendisc.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/gan/cyclegan_gendisc.png -------------------------------------------------------------------------------- /introduction/learning.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/introduction/learning.png -------------------------------------------------------------------------------- /docs/autoregressive/nade.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/autoregressive/nade.png -------------------------------------------------------------------------------- /docs/flow/flow-graphical.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/flow/flow-graphical.PNG -------------------------------------------------------------------------------- /introduction/learning_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/introduction/learning_1.png -------------------------------------------------------------------------------- /introduction/learning_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/introduction/learning_2.png -------------------------------------------------------------------------------- /docs/autoregressive/fvsbn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/autoregressive/fvsbn.png -------------------------------------------------------------------------------- /docs/gan/cyclegan_gendisc.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/gan/cyclegan_gendisc.png -------------------------------------------------------------------------------- /docs/introduction/learning.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/introduction/learning.png -------------------------------------------------------------------------------- /autoregressive/autoregressive.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/autoregressive/autoregressive.png -------------------------------------------------------------------------------- /docs/introduction/learning_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/introduction/learning_1.png -------------------------------------------------------------------------------- /docs/introduction/learning_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/introduction/learning_2.png -------------------------------------------------------------------------------- /docs/autoregressive/autoregressive.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/autoregressive/autoregressive.png -------------------------------------------------------------------------------- /_layouts/page.html: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | --- 4 |

{{ page.title | capitalize }}

5 |

{{ page.date | date: "%B %-d, %Y" }}

6 | 7 | 8 | {{ content }} 9 | -------------------------------------------------------------------------------- /fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.eot -------------------------------------------------------------------------------- /fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.ttf -------------------------------------------------------------------------------- /fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.woff -------------------------------------------------------------------------------- /_includes/print-footer.html: -------------------------------------------------------------------------------- 1 | {% if page.date %}{{ page.title }} - {{ page.date | date: "%B %-d, %Y" }} - {{site.author}}{% else %}{{ page.title }} - {{site.author}}{% endif %} -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.eot -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.ttf -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.woff -------------------------------------------------------------------------------- /fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.eot -------------------------------------------------------------------------------- /fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.ttf -------------------------------------------------------------------------------- /fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.woff -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.eot -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.ttf -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.woff -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.eot -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.ttf -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.woff -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.eot -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.ttf -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.woff -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.eot -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.ttf -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.woff -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.eot -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.ttf -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.woff -------------------------------------------------------------------------------- /fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.eot -------------------------------------------------------------------------------- /fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.ttf -------------------------------------------------------------------------------- /fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.woff -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.eot -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.ttf -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.woff -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.eot -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.ttf -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.woff -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.eot -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.ttf -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.woff -------------------------------------------------------------------------------- /fonts/et-book/et-book-semi-bold-old-style-figures/et-book-semi-bold-old-style-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-semi-bold-old-style-figures/et-book-semi-bold-old-style-figures.eot -------------------------------------------------------------------------------- /fonts/et-book/et-book-semi-bold-old-style-figures/et-book-semi-bold-old-style-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-semi-bold-old-style-figures/et-book-semi-bold-old-style-figures.ttf -------------------------------------------------------------------------------- /fonts/et-book/et-book-semi-bold-old-style-figures/et-book-semi-bold-old-style-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-semi-bold-old-style-figures/et-book-semi-bold-old-style-figures.woff -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-semi-bold-old-style-figures/et-bembo-semi-bold-old-style-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-semi-bold-old-style-figures/et-bembo-semi-bold-old-style-figures.eot -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-semi-bold-old-style-figures/et-bembo-semi-bold-old-style-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-semi-bold-old-style-figures/et-bembo-semi-bold-old-style-figures.ttf -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-semi-bold-old-style-figures/et-bembo-semi-bold-old-style-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-semi-bold-old-style-figures/et-bembo-semi-bold-old-style-figures.woff -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-semi-bold-old-style-figures/et-book-semi-bold-old-style-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-semi-bold-old-style-figures/et-book-semi-bold-old-style-figures.eot -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-semi-bold-old-style-figures/et-book-semi-bold-old-style-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-semi-bold-old-style-figures/et-book-semi-bold-old-style-figures.ttf -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-semi-bold-old-style-figures/et-book-semi-bold-old-style-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-semi-bold-old-style-figures/et-book-semi-bold-old-style-figures.woff -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-semi-bold-old-style-figures/et-bembo-semi-bold-old-style-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-semi-bold-old-style-figures/et-bembo-semi-bold-old-style-figures.eot -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-semi-bold-old-style-figures/et-bembo-semi-bold-old-style-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-semi-bold-old-style-figures/et-bembo-semi-bold-old-style-figures.ttf -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-semi-bold-old-style-figures/et-bembo-semi-bold-old-style-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-semi-bold-old-style-figures/et-bembo-semi-bold-old-style-figures.woff -------------------------------------------------------------------------------- /fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.eot -------------------------------------------------------------------------------- /fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.ttf -------------------------------------------------------------------------------- /fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.woff -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.eot -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.ttf -------------------------------------------------------------------------------- /fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.woff -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.eot -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.ttf -------------------------------------------------------------------------------- /docs/fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.woff -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.eot -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.ttf -------------------------------------------------------------------------------- /docs/fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deepgenerativemodels/notes/HEAD/docs/fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.woff -------------------------------------------------------------------------------- /_layouts/default.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | {% include head.html %} 4 | 5 | {% include header.html %} 6 |
7 | {{ content }} 8 |
9 | {% include print-footer.html %} 10 | {% include footer.html %} 11 | 12 | 13 | -------------------------------------------------------------------------------- /_includes/header.html: -------------------------------------------------------------------------------- 1 | 2 |
3 | 8 |
9 | -------------------------------------------------------------------------------- /_layouts/full-width.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | {% include head.html %} 4 | 5 | {% include header.html %} 6 |
7 | {{ content }} 8 |
9 | {% include print-footer.html %} 10 | {% include footer.html %} 11 | 12 | 13 | -------------------------------------------------------------------------------- /_data/social.yml: -------------------------------------------------------------------------------- 1 | - link: //www.twitter.com/twitter_handle 2 | icon: icon-twitter 3 | - link: //plus.google.com/+googlePlusName 4 | icon: icon-googleplus 5 | - link: //github.com/GithubHandle 6 | icon: icon-github 7 | - link: //www.flickr.com/photos/FlickrUserID 8 | icon: icon-flickr 9 | - link: /feed 10 | icon: icon-feed -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | TEMPDIR := $(shell mktemp -d -t tmp.XXX) 2 | 3 | publish: 4 | echo 'hmmm' 5 | cp -r ./_site/* $(TEMPDIR) 6 | cd $(TEMPDIR) && \ 7 | ls -a && \ 8 | git init && \ 9 | git add . && \ 10 | git commit -m 'publish site' && \ 11 | git remote add origin https://github.com/deepgenerativemodels/notes.git && \ 12 | git push origin master:refs/heads/gh-pages --force 13 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | TEMPDIR := $(shell mktemp -d -t tmp.XXX) 2 | 3 | publish: 4 | echo 'hmmm' 5 | cp -r ./_site/* $(TEMPDIR) 6 | cd $(TEMPDIR) && \ 7 | ls -a && \ 8 | git init && \ 9 | git add . && \ 10 | git commit -m 'publish site' && \ 11 | git remote add origin https://github.com/deepgenerativemodels/notes.git && \ 12 | git push origin master:refs/heads/gh-pages --force 13 | -------------------------------------------------------------------------------- /_plugins/marginnote.rb: -------------------------------------------------------------------------------- 1 | module Jekyll 2 | class RenderMarginNoteTag < Liquid::Tag 3 | 4 | require "shellwords" 5 | 6 | def initialize(tag_name, text, tokens) 7 | super 8 | @text = text.shellsplit 9 | end 10 | 11 | def render(context) 12 | "#{@text[1]} " 13 | end 14 | end 15 | end 16 | 17 | Liquid::Template.register_tag('marginnote', Jekyll::RenderMarginNoteTag) 18 | 19 | -------------------------------------------------------------------------------- /_plugins/sidenote.rb: -------------------------------------------------------------------------------- 1 | module Jekyll 2 | class RenderSideNoteTag < Liquid::Tag 3 | 4 | require "shellwords" 5 | 6 | def initialize(tag_name, text, tokens) 7 | super 8 | @text = text.shellsplit 9 | end 10 | 11 | def render(context) 12 | "#{@text[1]} " 13 | end 14 | end 15 | end 16 | 17 | Liquid::Template.register_tag('sidenote', Jekyll::RenderSideNoteTag) 18 | 19 | -------------------------------------------------------------------------------- /_plugins/newthought.rb: -------------------------------------------------------------------------------- 1 | ## Newthought tag will render anything in the tag with small caps 2 | ## Usage {% newthought Your text string here} will render a span 3 | ## YOUR TEXT STRING HERE (sort of, you know, small caps) if you are using the tufte.css file 4 | 5 | module Jekyll 6 | class RenderNewThoughtTag < Liquid::Tag 7 | 8 | require "shellwords" 9 | 10 | def initialize(tag_name, text, tokens) 11 | super 12 | @text = text.shellsplit 13 | end 14 | 15 | 16 | def render(context) 17 | "#{@text[0]} " 18 | end 19 | end 20 | end 21 | 22 | Liquid::Template.register_tag('newthought', Jekyll::RenderNewThoughtTag) -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | baseurl: /notes 2 | title: Deep Generative Models 3 | subtitle: Lecture notes 4 | author: Aditya Grover 5 | simple_search: http://google.com/search 6 | description: Lecture notes for Deep Generative Models. 7 | name: notes 8 | markdown_ext: "markdown,mkdown,mkdn,mkd,md" 9 | permalink: /articles/:short_year/:title 10 | timezone: America/New_York 11 | excerpt_separator: # you can specify your own separator, of course. 12 | exclude: ['Gemfile', 'Gemfile.lock', 'Rakefile', 'README.md'] 13 | destination: docs 14 | google_analytics: UA-129020129-1 15 | post: 16 | template: _post.txt 17 | extension: md 18 | page: 19 | template: _page.txt 20 | extension: md 21 | editor: gvim 22 | git: 23 | branch: master 24 | transfer: 25 | command: rsync 26 | settings: -av 27 | source: _site/ -------------------------------------------------------------------------------- /_includes/footer.html: -------------------------------------------------------------------------------- 1 | 16 | -------------------------------------------------------------------------------- /_plugins/mathjaxtag.rb: -------------------------------------------------------------------------------- 1 | module Jekyll 2 | class MathJaxBlockTag < Liquid::Tag 3 | def render(context) 4 | '
' 15 | end 16 | end 17 | class MathJaxEndInlineTag < Liquid::Tag 18 | def render(context) 19 | '' 20 | end 21 | end 22 | end 23 | 24 | Liquid::Template.register_tag('math', Jekyll::MathJaxBlockTag) 25 | Liquid::Template.register_tag('m', Jekyll::MathJaxInlineTag) 26 | Liquid::Template.register_tag('endmath', Jekyll::MathJaxEndBlockTag) 27 | Liquid::Template.register_tag('em', Jekyll::MathJaxEndInlineTag) -------------------------------------------------------------------------------- /_layouts/post.html: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | --- 4 |

{{ page.title | capitalize }}

5 |

{{ page.date | date: "%B %-d, %Y" }}

6 | 7 | 8 | 32 | 33 | 34 | {{ content }} 35 | 36 | -------------------------------------------------------------------------------- /_plugins/main_column_img.rb: -------------------------------------------------------------------------------- 1 | ## Liquid tag 'maincolumn-figure' used to add image data that fits within the 2 | ## main column area of the layout 3 | ## Usage {% maincolumn 'path/to/image' 'This is the caption' %} 4 | # 5 | module Jekyll 6 | class RenderMainColumnTag < Liquid::Tag 7 | 8 | require "shellwords" 9 | 10 | def initialize(tag_name, text, tokens) 11 | super 12 | @text = text.shellsplit 13 | end 14 | 15 | def render(context) 16 | baseurl = context.registers[:site].config['baseurl'] 17 | if @text[0].start_with?('http://', 'https://','//') 18 | "
#{@text[1]}
" 19 | else 20 | "
#{@text[1]}
" 21 | end 22 | end 23 | end 24 | end 25 | 26 | Liquid::Template.register_tag('maincolumn', Jekyll::RenderMainColumnTag) 27 | -------------------------------------------------------------------------------- /_plugins/fullwidth.rb: -------------------------------------------------------------------------------- 1 | ## This has a fairly harmless hack that wraps the img tag in a div to prevent it from being 2 | ## wrapped in a paragraph tag instead, which would totally fuck things up layout-wise 3 | ## Usage {% fullwidth 'path/to/image' 'caption goes here in quotes' %} 4 | # 5 | module Jekyll 6 | class RenderFullWidthTag < Liquid::Tag 7 | 8 | require "shellwords" 9 | 10 | def initialize(tag_name, text, tokens) 11 | super 12 | @text = text.shellsplit 13 | end 14 | 15 | def render(context) 16 | baseurl = context.registers[:site].config['baseurl'] 17 | if @text[0].start_with?('http://', 'https://','//') 18 | "
"+ 19 | "
#{@text[1]}
" 20 | else 21 | "
"+ 22 | "
#{@text[1]}
" 23 | end 24 | end 25 | end 26 | end 27 | 28 | Liquid::Template.register_tag('fullwidth', Jekyll::RenderFullWidthTag) 29 | -------------------------------------------------------------------------------- /_sass/_settings.scss: -------------------------------------------------------------------------------- 1 | /* This file contains all the constants for colors and font styles */ 2 | 3 | $body-font: et-book, Palatino, "Palatino Linotype", "Palatino LT STD", "Book Antiqua", Georgia, serif; 4 | // $body-font: ETBembo, Palatino, "Palatino Linotype", "Palatino LT STD", "Book Antiqua", Georgia, serif; 5 | // Note that Gill Sans is the top of the stack and corresponds to what is used in Tufte's books 6 | // However, it is not a free font, so if it is not present on the computer that is viewing the webpage 7 | // The free Google 'Lato' font is used instead. It is similar. 8 | $sans-font: "Gill Sans", "Gill Sans MT", "Lato", Calibri, sans-serif; 9 | $code-font: Consolas, "Liberation Mono", Menlo, Courier, monospace; 10 | $url-font: "Lucida Console", "Lucida Sans Typewriter", Monaco, "Bitstream Vera Sans Mono", monospace; 11 | $text-color: #111; 12 | $bg-color: #fffff8; 13 | $contrast-color: #a00000; 14 | $border-color: #333333; 15 | $link-style: underline; // choices are 'color' or 'underline'. Default is color using $contrast-color set above 16 | 17 | 18 | 19 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2018 Aditya Grover, Stefano Ermon 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /docs/LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2018 Aditya Grover, Stefano Ermon 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /index.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: post 3 | title: Contents 4 | --- 5 | These notes form a concise introductory course on deep generative models. 6 | They are based on Stanford [CS236](https://deepgenerativemodels.github.io/), taught by [Stefano Ermon](http://cs.stanford.edu/~ermon/) and [Aditya Grover](http://aditya-grover.github.io/), and have been written by [Aditya Grover](http://aditya-grover.github.io/), with the [help](https://github.com/deepgenerativemodels/notes/commits/master) of many students and course staff. 7 | {% marginnote 'mn-id-whatever' 'The notes are still **under construction**! 8 | Since these notes are brand new, you will find several typos. If you do, please let us know, or submit a pull request with your fixes to our [Github repository](https://github.com/deepgenerativemodels/notes).'%} 9 | You too may help make these notes better by submitting your improvements to us via [Github](https://github.com/deepgenerativemodels/notes). 10 | 11 | 12 | 1. [Introduction](introduction/) 13 | 14 | 2. [Autoregressive Models](autoregressive/) 15 | 16 | 3. [Variational Autoencoders](vae/) 17 | 18 | 4. [Normalizing Flow Models](flow/) 19 | 20 | 5. [Generative Adversarial Networks](gan/) 21 | 22 | -------------------------------------------------------------------------------- /_plugins/margin_figure.rb: -------------------------------------------------------------------------------- 1 | ## Liquid tag 'maincolumn' used to add image data that fits within the main 2 | ## column area of the layout 3 | ## Usage {% marginfigure 'margin-id-whatever' 'path/to/image' 'This is the caption' %} 4 | # 5 | module Jekyll 6 | class RenderMarginFigureTag < Liquid::Tag 7 | 8 | require "shellwords" 9 | 10 | def initialize(tag_name, text, tokens) 11 | super 12 | @text = text.shellsplit 13 | end 14 | 15 | def render(context) 16 | baseurl = context.registers[:site].config['baseurl'] 17 | if @text[1].start_with?('http://', 'https://', '//') 18 | ""+ 19 | ""+ 20 | "
#{@text[2]}" 21 | else 22 | ""+ 23 | ""+ 24 | "
#{@text[2]}" 25 | end 26 | end 27 | end 28 | end 29 | 30 | Liquid::Template.register_tag('marginfigure', Jekyll::RenderMarginFigureTag) 31 | -------------------------------------------------------------------------------- /_includes/head.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | {% if page.title %}{{ page.title }}{% else %}{{ site.title }}{% endif %} 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | {% if site.data.options.lato_font_load %} 15 | 16 | {% endif %} 17 | 18 | {% if site.data.options.mathjax %} 19 | 20 | {% endif %} 21 | 22 | 32 | 33 | 34 | 35 | 36 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Notes on Deep Generative Models 2 | 3 | These notes form a concise introductory course on deep generative models. They are based on Stanford [CS236](https://deepgenerativemodels.github.io/), taught by [Aditya Grover](http://aditya-grover.github.io/) and [Stefano Ermon](http://cs.stanford.edu/~ermon/), and have been written by [Aditya Grover](http://aditya-grover.github.io/), with the [help](https://github.com/deepgenerativemodels/notes/commits/master) of many students and course staff. 4 | 5 | The compiled version is available [here](https://deepgenerativemodels.github.io/notes/index.html). 6 | 7 | ## Contributing 8 | 9 | This material is under construction! Although we have written up most of it, you will probably find several typos. If you do, please let us know, or submit a pull request with your fixes via Github. 10 | 11 | 12 | The notes are written in Markdown and are compiled into HTML using Jekyll. Please add your changes directly to the Markdown source code. In order to install jekyll, you can follow the instructions posted on their website (https://jekyllrb.com/docs/installation/). 13 | 14 | Note that jekyll is only supported on GNU/Linux, Unix, or macOS. Thus, if you run Windows 10 on your local machine, you will have to install Bash on Ubuntu on Windows. Windows gives instructions on how to do that here and Jekyll's website offers helpful instructions on how to proceed through the rest of the process. 15 | 16 | To compile Markdown to HTML (i.e. after you have made changes to markdown and want them to be accessible to students viewing the docs), 17 | run the following commands from the root of your cloned version of the https://github.com/deepgenerativemodels/notes repo: 18 | 1) rm -r docs/ 19 | 2) jekyll serve # This should create a folder called _site. Note: This creates a running server; press Ctrl-C to stop the server before proceeding 20 | 3) mv _site docs # Change the name of the _site folder to "docs". This won't work if the server is still running. 21 | 4) git add file_names 22 | 5) git commit -am "your commit message describing what you did" 23 | 6) git push origin master 24 | 25 | Note that if you cloned the ermongroup/cs228-notes repo directly onto your local machine (instead of forking it) then you may see an error like "remote: Permission to ermongroup/cs228-notes.git denied to userjanedoe". If that is the case, then you need to fork their repo first. Then, if your github profile were userjanedoe, you would need to first push your local updates to your forked repo like so: 26 | 27 | git push https://github.com/deepgenerativemodels/notes.git master 28 | 29 | And then you could go and submit the pull request through the GitHub website. 30 | -------------------------------------------------------------------------------- /_sass/_syntax-highlighting.scss: -------------------------------------------------------------------------------- 1 | /** 2 | * Syntax highlighting styles 3 | */ 4 | $spacing-unit: 30px; 5 | %vertical-rhythm { 6 | margin-bottom: $spacing-unit / 2; 7 | } 8 | 9 | .highlight { 10 | background: #fffff8; 11 | @extend %vertical-rhythm; 12 | 13 | .c { color: #998; font-style: italic } // Comment 14 | .err { color: #a61717; background-color: #e3d2d2 } // Error 15 | .k { font-weight: bold } // Keyword 16 | .o { font-weight: bold } // Operator 17 | .cm { color: #998; font-style: italic } // Comment.Multiline 18 | .cp { color: #999; font-weight: bold } // Comment.Preproc 19 | .c1 { color: #998; font-style: italic } // Comment.Single 20 | .cs { color: #999; font-weight: bold; font-style: italic } // Comment.Special 21 | .gd { color: #000; background-color: #fdd } // Generic.Deleted 22 | .gd .x { color: #000; background-color: #faa } // Generic.Deleted.Specific 23 | .ge { font-style: italic } // Generic.Emph 24 | .gr { color: #a00 } // Generic.Error 25 | .gh { color: #999 } // Generic.Heading 26 | .gi { color: #000; background-color: #dfd } // Generic.Inserted 27 | .gi .x { color: #000; background-color: #afa } // Generic.Inserted.Specific 28 | .go { color: #888 } // Generic.Output 29 | .gp { color: #555 } // Generic.Prompt 30 | .gs { font-weight: bold } // Generic.Strong 31 | .gu { color: #aaa } // Generic.Subheading 32 | .gt { color: #a00 } // Generic.Traceback 33 | .kc { font-weight: bold } // Keyword.Constant 34 | .kd { font-weight: bold } // Keyword.Declaration 35 | .kp { font-weight: bold } // Keyword.Pseudo 36 | .kr { font-weight: bold } // Keyword.Reserved 37 | .kt { color: #458; font-weight: bold } // Keyword.Type 38 | .m { color: #099 } // Literal.Number 39 | .s { color: #d14 } // Literal.String 40 | .na { color: #008080 } // Name.Attribute 41 | .nb { color: #0086B3 } // Name.Builtin 42 | .nc { color: #458; font-weight: bold } // Name.Class 43 | .no { color: #008080 } // Name.Constant 44 | .ni { color: #800080 } // Name.Entity 45 | .ne { color: #900; font-weight: bold } // Name.Exception 46 | .nf { color: #900; font-weight: bold } // Name.Function 47 | .nn { color: #555 } // Name.Namespace 48 | .nt { color: #000080 } // Name.Tag 49 | .nv { color: #008080 } // Name.Variable 50 | .ow { font-weight: bold } // Operator.Word 51 | .w { color: #bbb } // Text.Whitespace 52 | .mf { color: #099 } // Literal.Number.Float 53 | .mh { color: #099 } // Literal.Number.Hex 54 | .mi { color: #099 } // Literal.Number.Integer 55 | .mo { color: #099 } // Literal.Number.Oct 56 | .sb { color: #d14 } // Literal.String.Backtick 57 | .sc { color: #d14 } // Literal.String.Char 58 | .sd { color: #d14 } // Literal.String.Doc 59 | .s2 { color: #d14 } // Literal.String.Double 60 | .se { color: #d14 } // Literal.String.Escape 61 | .sh { color: #d14 } // Literal.String.Heredoc 62 | .si { color: #d14 } // Literal.String.Interpol 63 | .sx { color: #d14 } // Literal.String.Other 64 | .sr { color: #009926 } // Literal.String.Regex 65 | .s1 { color: #d14 } // Literal.String.Single 66 | .ss { color: #990073 } // Literal.String.Symbol 67 | .bp { color: #999 } // Name.Builtin.Pseudo 68 | .vc { color: #008080 } // Name.Variable.Class 69 | .vg { color: #008080 } // Name.Variable.Global 70 | .vi { color: #008080 } // Name.Variable.Instance 71 | .il { color: #099 } // Literal.Number.Integer.Long 72 | } 73 | -------------------------------------------------------------------------------- /_sass/_fonts.scss: -------------------------------------------------------------------------------- 1 | // Font imports file. If you don't want these fonts, comment out these and add your own into the fonts directory 2 | // and point the src attribute to the file. 3 | // 4 | 5 | @charset "UTF-8"; 6 | // 7 | // @font-face { 8 | // font-family: ETBembo; 9 | // src: url("../fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.eot"); 10 | // src: url("../fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.eot?#iefix") format("embedded-opentype"), url("../fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.woff") format("woff"), url("../fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.ttf") format("truetype"), url("../fonts/et-bembo/et-bembo-roman-line-figures/et-bembo-roman-line-figures.svg#etbemboromanosf") format("svg"); 11 | // font-weight: normal; 12 | // font-style: normal 13 | // } 14 | // 15 | // @font-face { 16 | // font-family: ETBembo; 17 | // src: url("../fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.eot"); 18 | // src: url("../fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.eot?#iefix") format("embedded-opentype"), url("../fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.woff") format("woff"), url("../fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.ttf") format("truetype"), url("../fonts/et-bembo/et-bembo-display-italic-old-style-figures/et-bembo-display-italic-old-style-figures.svg#etbemboromanosf") format("svg"); 19 | // font-weight: normal; 20 | // font-style: italic 21 | // } 22 | // 23 | // @font-face { 24 | // font-family: ETBembo; 25 | // src: url("../fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.eot"); 26 | // src: url("../fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.eot?#iefix") format("embedded-opentype"), url("../fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.woff") format("woff"), url("../fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.ttf") format("truetype"), url("../fonts/et-bembo/et-bembo-bold-line-figures/et-bembo-bold-line-figures.svg#etbemboromanosf") format("svg"); 27 | // font-weight: bold; 28 | // font-style: normal 29 | // } 30 | // 31 | // @font-face { 32 | // font-family: ETBemboRomanOldStyle; 33 | // src: url("../fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.eot"); 34 | // src: url("../fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.eot?#iefix") format("embedded-opentype"), url("../fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.woff") format("woff"), url("../fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.ttf") format("truetype"), url("../fonts/et-bembo/et-bembo-roman-old-style-figures/et-bembo-roman-old-style-figures.svg#etbemboromanosf") format("svg"); 35 | // font-weight: normal; 36 | // font-style: normal; 37 | // } 38 | 39 | 40 | @font-face { 41 | font-family: "et-book"; 42 | src: url("../fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.eot"); 43 | src: url("../fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.eot?#iefix") format("embedded-opentype"), url("../fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.woff") format("woff"), url("../fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.ttf") format("truetype"), url("../fonts/et-book/et-book-roman-line-figures/et-book-roman-line-figures.svg#etbookromanosf") format("svg"); 44 | font-weight: normal; 45 | font-style: normal 46 | } 47 | 48 | @font-face { 49 | font-family: "et-book"; 50 | src: url("../fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.eot"); 51 | src: url("../fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.eot?#iefix") format("embedded-opentype"), url("../fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.woff") format("woff"), url("../fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.ttf") format("truetype"), url("../fonts/et-book/et-book-display-italic-old-style-figures/et-book-display-italic-old-style-figures.svg#etbookromanosf") format("svg"); 52 | font-weight: normal; 53 | font-style: italic 54 | } 55 | 56 | @font-face { 57 | font-family: "et-book"; 58 | src: url("../fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.eot"); 59 | src: url("../fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.eot?#iefix") format("embedded-opentype"), url("../fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.woff") format("woff"), url("../fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.ttf") format("truetype"), url("../fonts/et-book/et-book-bold-line-figures/et-book-bold-line-figures.svg#etbookromanosf") format("svg"); 60 | font-weight: bold; 61 | font-style: normal 62 | } 63 | 64 | @font-face { 65 | font-family: "et-book-roman-old-style"; 66 | src: url("../fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.eot"); 67 | src: url("../fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.eot?#iefix") format("embedded-opentype"), url("../fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.woff") format("woff"), url("../fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.ttf") format("truetype"), url("../fonts/et-book/et-book-roman-old-style-figures/et-book-roman-old-style-figures.svg#etbookromanosf") format("svg"); 68 | font-weight: normal; 69 | font-style: normal; 70 | } 71 | -------------------------------------------------------------------------------- /docs/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | Contents 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 |
42 | 47 |
48 | 49 |
50 |

Contents

51 |

52 | 53 | 54 | 78 | 79 | 80 |

These notes form a concise introductory course on deep generative models. 81 | They are based on Stanford CS236, taught by Stefano Ermon and Aditya Grover, and have been written by Aditya Grover, with the help of many students and course staff. 82 | The notes are still under construction! 83 | Since these notes are brand new, you will find several typos. If you do, please let us know, or submit a pull request with your fixes to our Github repository. 84 | You too may help make these notes better by submitting your improvements to us via Github.

85 | 86 |
    87 |
  1. 88 |

    Introduction

    89 |
    2. 90 |
  2. 91 |

    Autoregressive Models

    92 |
    3. 93 |
  3. 94 |

    Variational Autoencoders

    95 |
    4. 96 |
  4. 97 |

    Normalizing Flow Models

    98 |
    5. 99 |
  5. 100 |

    Generative Adversarial Networks

    101 |
    6. 102 |
103 | 104 | 105 | 106 | 107 |
108 | Contents - Aditya Grover 109 | 140 | 141 | 142 | 143 | -------------------------------------------------------------------------------- /introduction/index.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: post 3 | title: Introduction 4 | --- 5 | 6 | Intelligent agents are constantly generating, acquiring, and processing 7 | data. This data could be in the form of *images* that we capture on our 8 | phones, *text* messages we share with our friends, *graphs* that model 9 | interactions on social media, *videos* that record important events, 10 | etc. Natural agents excel at discovering patterns, extracting 11 | knowledge, and performing complex reasoning based on the data they observe. How 12 | can we build artificial learning systems to do the same? 13 | 14 | In this course, we will study generative models that view the world under the lens of probability. 15 | In such a worldview, we can think of any kind of 16 | observed data, say $$\mathcal{D}$$, as a finite set of samples from an 17 | underlying distribution, say $$p_{\mathrm{data}}$$. At its very core, the 18 | goal of any generative model is then to approximate this data 19 | distribution given access to the dataset $$\mathcal{D}$$. The hope is that 20 | if we are able to *learn* a good generative model, we can use the 21 | learned model for downstream *inference*. 22 | 23 | Learning 24 | -------- 25 | 26 | We will be primarily interested in parametric approximations to the data 27 | distribution, which summarize all the information about the dataset $$\mathcal{D}$$ in 28 | a finite set of parameters. In contrast with non-parametric models, 29 | parametric models scale more efficiently with large datasets but are 30 | limited in the family of distributions they can represent. 31 | 32 | In the parametric setting, we can think of the task of learning a 33 | generative model as picking the parameters within a family of model 34 | distributions that minimizes some notion of distance[^1] between the 35 | model distribution and the data distribution. 36 | 37 | drawing 38 | 39 | 40 | drawing 41 | 43 | 44 | 45 | For instance, we might be given access to a dataset of dog images $$\mathcal{D}$$ and 46 | our goal is to learn the parameters of a generative model $$\theta$$ within a model family $$\mathcal{M}$$ such that 47 | the model distribution $$p_\theta$$ is close to the data distribution over dogs 48 | $$p_{\mathrm{data}}$$. Mathematically, we can specify our goal as the 49 | following optimization problem: $$$$\begin{equation} 50 | \min_{\theta\in \mathcal{M}}d(p_{\mathrm{data}}, p_{\theta}) 51 | \label{eq:learning_gm} 52 | \tag{1} 53 | \end{equation}$$$$where $$p_{\mathrm{data}}$$ is accessed via the dataset 54 | $$\mathcal{D}$$ and $$d(\cdot)$$ is a notion of distance between probability distributions. 55 | 56 | As we navigate through this course, it is interesting to take note of 57 | the difficulty of the problem at hand. A typical image from a modern 58 | phone camera has a resolution of approximately $$700 \times 1400$$ pixels. 59 | Each pixel has three channels: R(ed), G(reen) and B(lue) and each 60 | channel can take a value between 0 to 255. Hence, the number of possible 61 | images is given by $$256^{700 \times 1400 \times 3}\approx 10 ^{800000}$$. 62 | In contrast, ImageNet, one of the largest publicly available datasets, 63 | consists of only about 15 million images. Hence, learning a generative 64 | model with such a limited dataset is a highly underdetermined problem. 65 | 66 | Fortunately, the real world is highly structured and automatically 67 | discovering the underlying structure is key to learning generative 68 | models. For example, we can hope to learn some basic artifacts about 69 | dogs even with just a few images: two eyes, two ears, fur etc. Instead 70 | of incorporating this prior knowledge explicitly, we will hope the model 71 | learns the underlying structure directly from data. There is no free 72 | lunch however, and indeed successful learning of generative models will 73 | involve instantiating the optimization problem in 74 | $$(\ref{eq:learning_gm})$$ in a suitable way. In this course, we will be 75 | primarily interested in the following questions: 76 | 77 | * What is the representation for the model family $$\mathcal{M}$$? 78 | * What is the objective function $$d(\cdot)$$? 79 | * What is the optimization procedure for minimizing $$d(\cdot)$$? 80 | 81 | In the next few set of lectures, we will take a deeper dive into certain 82 | families of generative models. For each model family, we will note how 83 | the representation is closely tied with the choice of learning objective 84 | and the optimization procedure. 85 | 86 | Inference 87 | --------- 88 | 89 | For a discriminative model such as logistic regression, the fundamental 90 | inference task is to predict a label for any given datapoint. Generative 91 | models, on the other hand, learn a joint distribution over the entire 92 | data.[^2] 93 | 94 | While the range of applications to which generative models have been 95 | used continue to grow, we can identify three fundamental inference 96 | queries for evaluating a generative model.: 97 | 98 | 1. *Density estimation:* Given a datapoint $$\mathbf{x}$$, what is the 99 | probability assigned by the model, i.e., $$p_\theta(\mathbf{x})$$? 100 | 101 | 2. *Sampling:* How can we *generate* novel data from the model 102 | distribution, i.e., 103 | $$\mathbf{x}_{\mathrm{new}} \sim p_\theta(\mathbf{x})$$? 104 | 105 | 3. *Unsupervised representation learning:* How can we learn meaningful 106 | feature representations for a datapoint $$\mathbf{x}$$? 107 | 108 | Going back to our example of learning a generative model over dog 109 | images, we can intuitively expect a good generative model to work as 110 | follows. For density estimation, we expect $$p_\theta(\mathbf{x})$$ to be 111 | high for dog images and low otherwise. Alluding to the name *generative 112 | model*, sampling involves generating novel images of dogs beyond the 113 | ones we observe in our dataset. Finally, representation learning can 114 | help discover high-level structure in the data such as the breed of 115 | dogs. 116 | 117 | In light of the above inference tasks, we note two caveats. First, 118 | quantitative evaluation of generative models on these tasks is itself 119 | non-trivial (in particular, sampling and representation learning) and an 120 | area of active research. Some quantitative metrics exist, but these 121 | metrics often fail to reflect desirable qualitative attributes in the 122 | generated samples and the learned representations. Secondly, not all 123 | model families permit efficient and accurate inference on all these 124 | tasks. Indeed, the trade-offs in the inference capabilities of the 125 | current generative models have led to the development of very diverse approaches as 126 | we shall see in this course. 127 | 128 | 129 | ## Footnotes 130 | 131 | [^1]: As we shall see later, functions that do not satisfy all 132 | properties of a distance metric are also used in practice, e.g., KL 133 | divergence. 134 | 135 | [^2]: Technically, a probabilistic discriminative model is also a 136 | generative model of the labels conditioned on the data. However, the 137 | usage of the term generative models is typically reserved for high 138 | dimensional data. 139 | -------------------------------------------------------------------------------- /flow/index.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: post 3 | title: Normalizing Flow Models 4 | --- 5 | 6 | We continue our study over another type of likelihood based generative models. As before, we assume we are given access to a dataset $$\mathcal{D}$$ of $$n$$-dimensional datapoints $$\mathbf{x}$$. So far we have learned two types of likelihood based generative models: 7 | 8 | 1. Autoregressive Models: $$p_\theta(\mathbf{x}) = \prod_{i=1}^{N} p_\theta(x_i \vert \mathbf{x}_{ 2 | 3 | 4 | Generated by IcoMoon 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | -------------------------------------------------------------------------------- /docs/fonts/icomoon.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | Generated by IcoMoon 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | -------------------------------------------------------------------------------- /docs/css/tufte.orginal.css: -------------------------------------------------------------------------------- 1 | /***************************************************************************** 2 | /* 3 | /* Tufte Jekyll blog theme 4 | /* Based on Tufte CSS by Dave Liepmann ( https://github.com/edwardtufte/tufte- 5 | /* 6 | /* The README.md will show you how to set up your site along with other goodie 7 | /*****************************************************************************/ 8 | 9 | // Imports to create final 10 | 11 | @import "../_sass/fonts"; 12 | @import "../_sass/settings"; 13 | @import "../_sass/syntax-highlighting"; 14 | 15 | /* Tufte CSS styles */ 16 | html { font-size: 15px; } 17 | 18 | body { width: 87.5%; 19 | margin-left: auto; 20 | margin-right: auto; 21 | padding-left: 12.5%; 22 | font-family: et-book, Palatino, "Palatino Linotype", "Palatino LT STD", "Book Antiqua", Georgia, serif; 23 | background-color: #fffff8; 24 | color: #111; 25 | max-width: 1400px; 26 | counter-reset: sidenote-counter; } 27 | 28 | h1 { font-weight: 400; 29 | margin-top: 4rem; 30 | margin-bottom: 1.5rem; 31 | font-size: 3.2rem; 32 | line-height: 1; } 33 | 34 | h2 { font-style: italic; 35 | font-weight: 400; 36 | margin-top: 2.1rem; 37 | margin-bottom: 0; 38 | font-size: 2.2rem; 39 | line-height: 1; } 40 | 41 | h3 { font-style: italic; 42 | font-weight: 400; 43 | font-size: 1.7rem; 44 | margin-top: 2rem; 45 | margin-bottom: 0; 46 | line-height: 1; } 47 | 48 | p.subtitle { font-style: italic; 49 | margin-top: 1rem; 50 | margin-bottom: 1rem; 51 | font-size: 1.8rem; 52 | display: block; 53 | line-height: 1; } 54 | 55 | .numeral { font-family: et-book-roman-old-style; } 56 | 57 | .danger { color: red; } 58 | 59 | article { position: relative; 60 | padding: 5rem 0rem; } 61 | 62 | section { padding-top: 1rem; 63 | padding-bottom: 1rem; } 64 | 65 | p, ol, ul { font-size: 1.4rem; } 66 | 67 | p { line-height: 2rem; 68 | margin-top: 1.4rem; 69 | margin-bottom: 1.4rem; 70 | padding-right: 0; 71 | vertical-align: baseline; } 72 | 73 | /* Chapter Epigraphs */ 74 | div.epigraph { margin: 5em 0; } 75 | 76 | div.epigraph > blockquote { margin-top: 3em; 77 | margin-bottom: 3em; } 78 | 79 | div.epigraph > blockquote, div.epigraph > blockquote > p { font-style: italic; } 80 | 81 | div.epigraph > blockquote > footer { font-style: normal; } 82 | 83 | div.epigraph > blockquote > footer > cite { font-style: italic; } 84 | 85 | /* end chapter epigraphs styles */ 86 | 87 | blockquote { font-size: 1.4rem; } 88 | 89 | blockquote p { width: 50%; } 90 | 91 | blockquote footer { width: 50%; 92 | font-size: 1.1rem; 93 | text-align: right; } 94 | 95 | ol, ul { width: 45%; 96 | -webkit-padding-start: 5%; 97 | -webkit-padding-end: 5%; } 98 | 99 | li { padding: 0.5rem 0; } 100 | 101 | figure { padding: 0; 102 | border: 0; 103 | font-size: 100%; 104 | font: inherit; 105 | vertical-align: baseline; 106 | max-width: 55%; 107 | -webkit-margin-start: 0; 108 | -webkit-margin-end: 0; 109 | margin: 0 0 3em 0; } 110 | 111 | figcaption { float: right; 112 | clear: right; 113 | margin-right: -48%; 114 | margin-top: 0; 115 | margin-bottom: 0; 116 | font-size: 1.1rem; 117 | line-height: 1.6; 118 | vertical-align: baseline; 119 | position: relative; 120 | max-width: 40%; } 121 | 122 | figure.fullwidth figcaption { margin-right: 24%; } 123 | 124 | /* Links: replicate underline that clears descenders */ 125 | a:link, a:visited { color: inherit; } 126 | 127 | a:link { text-decoration: none; 128 | background: -webkit-linear-gradient(#fffff8, #fffff8), -webkit-linear-gradient(#fffff8, #fffff8), -webkit-linear-gradient(#333, #333); 129 | background: linear-gradient(#fffff8, #fffff8), linear-gradient(#fffff8, #fffff8), linear-gradient(#333, #333); 130 | -webkit-background-size: 0.05em 1px, 0.05em 1px, 1px 1px; 131 | -moz-background-size: 0.05em 1px, 0.05em 1px, 1px 1px; 132 | background-size: 0.05em 1px, 0.05em 1px, 1px 1px; 133 | background-repeat: no-repeat, no-repeat, repeat-x; 134 | text-shadow: 0.03em 0 #fffff8, -0.03em 0 #fffff8, 0 0.03em #fffff8, 0 -0.03em #fffff8, 0.06em 0 #fffff8, -0.06em 0 #fffff8, 0.09em 0 #fffff8, -0.09em 0 #fffff8, 0.12em 0 #fffff8, -0.12em 0 #fffff8, 0.15em 0 #fffff8, -0.15em 0 #fffff8; 135 | background-position: 0% 93%, 100% 93%, 0% 93%; } 136 | 137 | @media screen and (-webkit-min-device-pixel-ratio: 0) { a:link { background-position-y: 87%, 87%, 87%; } } 138 | 139 | a:link::selection { text-shadow: 0.03em 0 #b4d5fe, -0.03em 0 #b4d5fe, 0 0.03em #b4d5fe, 0 -0.03em #b4d5fe, 0.06em 0 #b4d5fe, -0.06em 0 #b4d5fe, 0.09em 0 #b4d5fe, -0.09em 0 #b4d5fe, 0.12em 0 #b4d5fe, -0.12em 0 #b4d5fe, 0.15em 0 #b4d5fe, -0.15em 0 #b4d5fe; 140 | background: #b4d5fe; } 141 | 142 | a:link::-moz-selection { text-shadow: 0.03em 0 #b4d5fe, -0.03em 0 #b4d5fe, 0 0.03em #b4d5fe, 0 -0.03em #b4d5fe, 0.06em 0 #b4d5fe, -0.06em 0 #b4d5fe, 0.09em 0 #b4d5fe, -0.09em 0 #b4d5fe, 0.12em 0 #b4d5fe, -0.12em 0 #b4d5fe, 0.15em 0 #b4d5fe, -0.15em 0 #b4d5fe; 143 | background: #b4d5fe; } 144 | 145 | /* Sidenotes, margin notes, figures, captions */ 146 | img { max-width: 100%; } 147 | 148 | .sidenote, .marginnote { float: right; 149 | clear: right; 150 | margin-right: -60%; 151 | width: 50%; 152 | margin-top: 0; 153 | margin-bottom: 0; 154 | font-size: 1.1rem; 155 | line-height: 1.3; 156 | vertical-align: baseline; 157 | position: relative; } 158 | 159 | .table-caption { float:right; 160 | clear:right; 161 | margin-right: -60%; 162 | width: 50%; 163 | margin-top: 0; 164 | margin-bottom: 0; 165 | font-size: 1.0rem; 166 | line-height: 1.6; } 167 | 168 | .sidenote-number { counter-increment: sidenote-counter; } 169 | 170 | .sidenote-number:after, .sidenote:before { content: counter(sidenote-counter) " "; 171 | font-family: et-book-roman-old-style; 172 | position: relative; 173 | vertical-align: baseline; } 174 | 175 | .sidenote-number:after { content: counter(sidenote-counter); 176 | font-size: 1rem; 177 | top: -0.5rem; 178 | left: 0.1rem; } 179 | 180 | .sidenote:before { content: counter(sidenote-counter) " "; 181 | top: -0.5rem; } 182 | 183 | p, footer, table, div.table-wrapper-small, div.supertable-wrapper > p, div.booktabs-wrapper { width: 55%; } 184 | 185 | div.fullwidth, table.fullwidth { width: 100%; } 186 | 187 | div.table-wrapper { overflow-x: auto; 188 | font-family: "Trebuchet MS", "Gill Sans", "Gill Sans MT", sans-serif; } 189 | 190 | @media screen and (max-width: 760px) { p, footer { width: 90%; } 191 | pre.code { width: 87.5%; } 192 | ul { width: 85%; } 193 | figure { max-width: 90%; } 194 | figcaption, figure.fullwidth figcaption { margin-right: 0%; 195 | max-width: none; } 196 | blockquote p, blockquote footer { width: 90%; }} 197 | 198 | .sans { font-family: "Gill Sans", "Gill Sans MT", Calibri, sans-serif; 199 | letter-spacing: .03em; } 200 | 201 | .code { font-family: Consolas, "Liberation Mono", Menlo, Courier, monospace; 202 | font-size: 1.125rem; 203 | line-height: 1.6; } 204 | 205 | h1 .code, h2 .code, h3 .code { font-size: 0.80em; } 206 | 207 | .marginnote .code, .sidenote .code { font-size: 1rem; } 208 | 209 | pre.code { width: 52.5%; 210 | padding-left: 2.5%; 211 | overflow-x: auto; } 212 | 213 | .fullwidth { max-width: 90%; 214 | clear:both; } 215 | 216 | span.newthought { font-variant: small-caps; 217 | font-size: 1.2em; } 218 | 219 | input.margin-toggle { display: none; } 220 | 221 | label.sidenote-number { display: inline; } 222 | 223 | label.margin-toggle:not(.sidenote-number) { display: none; } 224 | 225 | @media (max-width: 760px) { label.margin-toggle:not(.sidenote-number) { display: inline; } 226 | .sidenote, .marginnote { display: none; } 227 | .margin-toggle:checked + .sidenote, 228 | .margin-toggle:checked + .marginnote { display: block; 229 | float: left; 230 | left: 1rem; 231 | clear: both; 232 | width: 95%; 233 | margin: 1rem 2.5%; 234 | vertical-align: baseline; 235 | position: relative; } 236 | label { cursor: pointer; } 237 | pre.code { width: 90%; 238 | padding: 0; } 239 | .table-caption { display: block; 240 | float: right; 241 | clear: both; 242 | width: 98%; 243 | margin-top: 1rem; 244 | margin-bottom: 0.5rem; 245 | margin-left: 1%; 246 | margin-right: 1%; 247 | vertical-align: baseline; 248 | position: relative; } 249 | div.table-wrapper, table, table.booktabs { width: 85%; } 250 | div.table-wrapper { border-right: 1px solid #efefef; } 251 | img { width: 100%; } } 252 | -------------------------------------------------------------------------------- /css/tufte.css: -------------------------------------------------------------------------------- 1 | --- 2 | # this ensures Jekyll reads the file to be transformed into CSS later 3 | # only Main files contain this front matter, not partials. 4 | nav_exclude: true 5 | --- 6 | /***************************************************************************** 7 | /* 8 | /* Tufte Jekyll blog theme 9 | /* Based on Tufte CSS by Dave Liepmann ( https://github.com/edwardtufte/tufte- 10 | /* 11 | /* The README.md will show you how to set up your site along with other goodie 12 | /*****************************************************************************/ 13 | 14 | // Imports to create final 15 | 16 | @import "../_sass/fonts"; 17 | @import "../_sass/settings"; 18 | @import "../_sass/syntax-highlighting"; 19 | 20 | /* Tufte CSS styles */ 21 | html { font-size: 15px; } 22 | 23 | body { width: 87.5%; 24 | margin-left: auto; 25 | margin-right: auto; 26 | padding-left: 12.5%; 27 | font-family: et-book, Palatino, "Palatino Linotype", "Palatino LT STD", "Book Antiqua", Georgia, serif; 28 | background-color: #fffff8; 29 | color: #111; 30 | max-width: 1400px; 31 | counter-reset: sidenote-counter; } 32 | 33 | h1 { font-weight: 400; 34 | margin-top: 4rem; 35 | margin-bottom: 1.5rem; 36 | font-size: 3.2rem; 37 | line-height: 1; } 38 | 39 | h2 { font-style: italic; 40 | font-weight: 400; 41 | margin-top: 2.1rem; 42 | margin-bottom: 0; 43 | font-size: 2.2rem; 44 | line-height: 1; } 45 | 46 | h3 { font-style: italic; 47 | font-weight: 400; 48 | font-size: 1.7rem; 49 | margin-top: 2rem; 50 | margin-bottom: 0; 51 | line-height: 1; } 52 | 53 | p.subtitle { font-style: italic; 54 | margin-top: 1rem; 55 | margin-bottom: 1rem; 56 | font-size: 1.8rem; 57 | display: block; 58 | line-height: 1; } 59 | 60 | .numeral { font-family: et-book-roman-old-style; } 61 | 62 | .danger { color: red; } 63 | 64 | article { position: relative; 65 | padding: 5rem 0rem; } 66 | 67 | section { padding-top: 1rem; 68 | padding-bottom: 1rem; } 69 | 70 | p, ol, ul { font-size: 1.4rem; } 71 | 72 | p { line-height: 2rem; 73 | margin-top: 1.4rem; 74 | margin-bottom: 1.4rem; 75 | padding-right: 0; 76 | vertical-align: baseline; } 77 | 78 | /* Chapter Epigraphs */ 79 | div.epigraph { margin: 5em 0; } 80 | 81 | div.epigraph > blockquote { margin-top: 3em; 82 | margin-bottom: 3em; } 83 | 84 | div.epigraph > blockquote, div.epigraph > blockquote > p { font-style: italic; } 85 | 86 | div.epigraph > blockquote > footer { font-style: normal; } 87 | 88 | div.epigraph > blockquote > footer > cite { font-style: italic; } 89 | 90 | /* end chapter epigraphs styles */ 91 | 92 | blockquote { font-size: 1.4rem; } 93 | 94 | blockquote p { width: 50%; } 95 | 96 | blockquote footer { width: 50%; 97 | font-size: 1.1rem; 98 | text-align: right; } 99 | 100 | ol, ul { width: 45%; 101 | -webkit-padding-start: 5%; 102 | -webkit-padding-end: 5%; } 103 | 104 | li { padding: 0.5rem 0; } 105 | 106 | figure { padding: 0; 107 | border: 0; 108 | font-size: 100%; 109 | font: inherit; 110 | vertical-align: baseline; 111 | max-width: 55%; 112 | -webkit-margin-start: 0; 113 | -webkit-margin-end: 0; 114 | margin: 0 0 3em 0; } 115 | 116 | figcaption { float: right; 117 | clear: right; 118 | margin-right: -48%; 119 | margin-top: 0; 120 | margin-bottom: 0; 121 | font-size: 1.1rem; 122 | line-height: 1.6; 123 | vertical-align: baseline; 124 | position: relative; 125 | max-width: 40%; } 126 | 127 | figure.fullwidth figcaption { margin-right: 24%; } 128 | 129 | /* Links: replicate underline that clears descenders */ 130 | a:link, a:visited { color: inherit; } 131 | 132 | a:link { text-decoration: none; 133 | background: -webkit-linear-gradient(#fffff8, #fffff8), -webkit-linear-gradient(#fffff8, #fffff8), -webkit-linear-gradient(#333, #333); 134 | background: linear-gradient(#fffff8, #fffff8), linear-gradient(#fffff8, #fffff8), linear-gradient(#333, #333); 135 | -webkit-background-size: 0.05em 1px, 0.05em 1px, 1px 1px; 136 | -moz-background-size: 0.05em 1px, 0.05em 1px, 1px 1px; 137 | background-size: 0.05em 1px, 0.05em 1px, 1px 1px; 138 | background-repeat: no-repeat, no-repeat, repeat-x; 139 | text-shadow: 0.03em 0 #fffff8, -0.03em 0 #fffff8, 0 0.03em #fffff8, 0 -0.03em #fffff8, 0.06em 0 #fffff8, -0.06em 0 #fffff8, 0.09em 0 #fffff8, -0.09em 0 #fffff8, 0.12em 0 #fffff8, -0.12em 0 #fffff8, 0.15em 0 #fffff8, -0.15em 0 #fffff8; 140 | background-position: 0% 93%, 100% 93%, 0% 93%; } 141 | 142 | @media screen and (-webkit-min-device-pixel-ratio: 0) { a:link { background-position-y: 87%, 87%, 87%; } } 143 | 144 | a:link::selection { text-shadow: 0.03em 0 #b4d5fe, -0.03em 0 #b4d5fe, 0 0.03em #b4d5fe, 0 -0.03em #b4d5fe, 0.06em 0 #b4d5fe, -0.06em 0 #b4d5fe, 0.09em 0 #b4d5fe, -0.09em 0 #b4d5fe, 0.12em 0 #b4d5fe, -0.12em 0 #b4d5fe, 0.15em 0 #b4d5fe, -0.15em 0 #b4d5fe; 145 | background: #b4d5fe; } 146 | 147 | a:link::-moz-selection { text-shadow: 0.03em 0 #b4d5fe, -0.03em 0 #b4d5fe, 0 0.03em #b4d5fe, 0 -0.03em #b4d5fe, 0.06em 0 #b4d5fe, -0.06em 0 #b4d5fe, 0.09em 0 #b4d5fe, -0.09em 0 #b4d5fe, 0.12em 0 #b4d5fe, -0.12em 0 #b4d5fe, 0.15em 0 #b4d5fe, -0.15em 0 #b4d5fe; 148 | background: #b4d5fe; } 149 | 150 | /* Sidenotes, margin notes, figures, captions */ 151 | img { max-width: 100%; } 152 | 153 | .sidenote, .marginnote { float: right; 154 | clear: right; 155 | margin-right: -60%; 156 | width: 50%; 157 | margin-top: 0; 158 | margin-bottom: 0; 159 | font-size: 1.1rem; 160 | line-height: 1.3; 161 | vertical-align: baseline; 162 | position: relative; } 163 | 164 | .table-caption { float:right; 165 | clear:right; 166 | margin-right: -60%; 167 | width: 50%; 168 | margin-top: 0; 169 | margin-bottom: 0; 170 | font-size: 1.0rem; 171 | line-height: 1.6; } 172 | 173 | .sidenote-number { counter-increment: sidenote-counter; } 174 | 175 | .sidenote-number:after, .sidenote:before { content: counter(sidenote-counter) " "; 176 | font-family: et-book-roman-old-style; 177 | position: relative; 178 | vertical-align: baseline; } 179 | 180 | .sidenote-number:after { content: counter(sidenote-counter); 181 | font-size: 1rem; 182 | top: -0.5rem; 183 | left: 0.1rem; } 184 | 185 | .sidenote:before { content: counter(sidenote-counter) " "; 186 | top: -0.5rem; } 187 | 188 | p, footer, table, div.table-wrapper-small, div.supertable-wrapper > p, div.booktabs-wrapper { width: 55%; } 189 | 190 | div.fullwidth, table.fullwidth { width: 100%; } 191 | 192 | div.table-wrapper { overflow-x: auto; 193 | font-family: "Trebuchet MS", "Gill Sans", "Gill Sans MT", sans-serif; } 194 | 195 | @media screen and (max-width: 760px) { p, footer { width: 90%; } 196 | pre.code { width: 87.5%; } 197 | ul { width: 85%; } 198 | figure { max-width: 90%; } 199 | figcaption, figure.fullwidth figcaption { margin-right: 0%; 200 | max-width: none; } 201 | blockquote p, blockquote footer { width: 90%; }} 202 | 203 | .sans { font-family: "Gill Sans", "Gill Sans MT", Calibri, sans-serif; 204 | letter-spacing: .03em; } 205 | 206 | .code { font-family: Consolas, "Liberation Mono", Menlo, Courier, monospace; 207 | font-size: 1.125rem; 208 | line-height: 1.6; } 209 | 210 | h1 .code, h2 .code, h3 .code { font-size: 0.80em; } 211 | 212 | .marginnote .code, .sidenote .code { font-size: 1rem; } 213 | 214 | pre.code { width: 52.5%; 215 | padding-left: 2.5%; 216 | overflow-x: auto; } 217 | 218 | .fullwidth { max-width: 90%; 219 | clear:both; } 220 | 221 | span.newthought { font-variant: small-caps; 222 | font-size: 1.2em; } 223 | 224 | input.margin-toggle { display: none; } 225 | 226 | label.sidenote-number { display: inline; } 227 | 228 | label.margin-toggle:not(.sidenote-number) { display: none; } 229 | 230 | @media (max-width: 760px) { label.margin-toggle:not(.sidenote-number) { display: inline; } 231 | .sidenote, .marginnote { display: none; } 232 | .margin-toggle:checked + .sidenote, 233 | .margin-toggle:checked + .marginnote { display: block; 234 | float: left; 235 | left: 1rem; 236 | clear: both; 237 | width: 95%; 238 | margin: 1rem 2.5%; 239 | vertical-align: baseline; 240 | position: relative; } 241 | label { cursor: pointer; } 242 | pre.code { width: 90%; 243 | padding: 0; } 244 | .table-caption { display: block; 245 | float: right; 246 | clear: both; 247 | width: 98%; 248 | margin-top: 1rem; 249 | margin-bottom: 0.5rem; 250 | margin-left: 1%; 251 | margin-right: 1%; 252 | vertical-align: baseline; 253 | position: relative; } 254 | div.table-wrapper, table, table.booktabs { width: 85%; } 255 | div.table-wrapper { border-right: 1px solid #efefef; } 256 | img { width: 100%; } } 257 | -------------------------------------------------------------------------------- /css/tufte.orginal.css: -------------------------------------------------------------------------------- 1 | --- 2 | # this ensures Jekyll reads the file to be transformed into CSS later 3 | # only Main files contain this front matter, not partials. 4 | nav_exclude: true 5 | --- 6 | /***************************************************************************** 7 | /* 8 | /* Tufte Jekyll blog theme 9 | /* Based on Tufte CSS by Dave Liepmann ( https://github.com/edwardtufte/tufte- 10 | /* 11 | /* The README.md will show you how to set up your site along with other goodie 12 | /*****************************************************************************/ 13 | 14 | // Imports to create final 15 | 16 | @import "../_sass/fonts"; 17 | @import "../_sass/settings"; 18 | @import "../_sass/syntax-highlighting"; 19 | 20 | /* Tufte CSS styles */ 21 | html { font-size: 15px; } 22 | 23 | body { width: 87.5%; 24 | margin-left: auto; 25 | margin-right: auto; 26 | padding-left: 12.5%; 27 | font-family: et-book, Palatino, "Palatino Linotype", "Palatino LT STD", "Book Antiqua", Georgia, serif; 28 | background-color: #fffff8; 29 | color: #111; 30 | max-width: 1400px; 31 | counter-reset: sidenote-counter; } 32 | 33 | h1 { font-weight: 400; 34 | margin-top: 4rem; 35 | margin-bottom: 1.5rem; 36 | font-size: 3.2rem; 37 | line-height: 1; } 38 | 39 | h2 { font-style: italic; 40 | font-weight: 400; 41 | margin-top: 2.1rem; 42 | margin-bottom: 0; 43 | font-size: 2.2rem; 44 | line-height: 1; } 45 | 46 | h3 { font-style: italic; 47 | font-weight: 400; 48 | font-size: 1.7rem; 49 | margin-top: 2rem; 50 | margin-bottom: 0; 51 | line-height: 1; } 52 | 53 | p.subtitle { font-style: italic; 54 | margin-top: 1rem; 55 | margin-bottom: 1rem; 56 | font-size: 1.8rem; 57 | display: block; 58 | line-height: 1; } 59 | 60 | .numeral { font-family: et-book-roman-old-style; } 61 | 62 | .danger { color: red; } 63 | 64 | article { position: relative; 65 | padding: 5rem 0rem; } 66 | 67 | section { padding-top: 1rem; 68 | padding-bottom: 1rem; } 69 | 70 | p, ol, ul { font-size: 1.4rem; } 71 | 72 | p { line-height: 2rem; 73 | margin-top: 1.4rem; 74 | margin-bottom: 1.4rem; 75 | padding-right: 0; 76 | vertical-align: baseline; } 77 | 78 | /* Chapter Epigraphs */ 79 | div.epigraph { margin: 5em 0; } 80 | 81 | div.epigraph > blockquote { margin-top: 3em; 82 | margin-bottom: 3em; } 83 | 84 | div.epigraph > blockquote, div.epigraph > blockquote > p { font-style: italic; } 85 | 86 | div.epigraph > blockquote > footer { font-style: normal; } 87 | 88 | div.epigraph > blockquote > footer > cite { font-style: italic; } 89 | 90 | /* end chapter epigraphs styles */ 91 | 92 | blockquote { font-size: 1.4rem; } 93 | 94 | blockquote p { width: 50%; } 95 | 96 | blockquote footer { width: 50%; 97 | font-size: 1.1rem; 98 | text-align: right; } 99 | 100 | ol, ul { width: 45%; 101 | -webkit-padding-start: 5%; 102 | -webkit-padding-end: 5%; } 103 | 104 | li { padding: 0.5rem 0; } 105 | 106 | figure { padding: 0; 107 | border: 0; 108 | font-size: 100%; 109 | font: inherit; 110 | vertical-align: baseline; 111 | max-width: 55%; 112 | -webkit-margin-start: 0; 113 | -webkit-margin-end: 0; 114 | margin: 0 0 3em 0; } 115 | 116 | figcaption { float: right; 117 | clear: right; 118 | margin-right: -48%; 119 | margin-top: 0; 120 | margin-bottom: 0; 121 | font-size: 1.1rem; 122 | line-height: 1.6; 123 | vertical-align: baseline; 124 | position: relative; 125 | max-width: 40%; } 126 | 127 | figure.fullwidth figcaption { margin-right: 24%; } 128 | 129 | /* Links: replicate underline that clears descenders */ 130 | a:link, a:visited { color: inherit; } 131 | 132 | a:link { text-decoration: none; 133 | background: -webkit-linear-gradient(#fffff8, #fffff8), -webkit-linear-gradient(#fffff8, #fffff8), -webkit-linear-gradient(#333, #333); 134 | background: linear-gradient(#fffff8, #fffff8), linear-gradient(#fffff8, #fffff8), linear-gradient(#333, #333); 135 | -webkit-background-size: 0.05em 1px, 0.05em 1px, 1px 1px; 136 | -moz-background-size: 0.05em 1px, 0.05em 1px, 1px 1px; 137 | background-size: 0.05em 1px, 0.05em 1px, 1px 1px; 138 | background-repeat: no-repeat, no-repeat, repeat-x; 139 | text-shadow: 0.03em 0 #fffff8, -0.03em 0 #fffff8, 0 0.03em #fffff8, 0 -0.03em #fffff8, 0.06em 0 #fffff8, -0.06em 0 #fffff8, 0.09em 0 #fffff8, -0.09em 0 #fffff8, 0.12em 0 #fffff8, -0.12em 0 #fffff8, 0.15em 0 #fffff8, -0.15em 0 #fffff8; 140 | background-position: 0% 93%, 100% 93%, 0% 93%; } 141 | 142 | @media screen and (-webkit-min-device-pixel-ratio: 0) { a:link { background-position-y: 87%, 87%, 87%; } } 143 | 144 | a:link::selection { text-shadow: 0.03em 0 #b4d5fe, -0.03em 0 #b4d5fe, 0 0.03em #b4d5fe, 0 -0.03em #b4d5fe, 0.06em 0 #b4d5fe, -0.06em 0 #b4d5fe, 0.09em 0 #b4d5fe, -0.09em 0 #b4d5fe, 0.12em 0 #b4d5fe, -0.12em 0 #b4d5fe, 0.15em 0 #b4d5fe, -0.15em 0 #b4d5fe; 145 | background: #b4d5fe; } 146 | 147 | a:link::-moz-selection { text-shadow: 0.03em 0 #b4d5fe, -0.03em 0 #b4d5fe, 0 0.03em #b4d5fe, 0 -0.03em #b4d5fe, 0.06em 0 #b4d5fe, -0.06em 0 #b4d5fe, 0.09em 0 #b4d5fe, -0.09em 0 #b4d5fe, 0.12em 0 #b4d5fe, -0.12em 0 #b4d5fe, 0.15em 0 #b4d5fe, -0.15em 0 #b4d5fe; 148 | background: #b4d5fe; } 149 | 150 | /* Sidenotes, margin notes, figures, captions */ 151 | img { max-width: 100%; } 152 | 153 | .sidenote, .marginnote { float: right; 154 | clear: right; 155 | margin-right: -60%; 156 | width: 50%; 157 | margin-top: 0; 158 | margin-bottom: 0; 159 | font-size: 1.1rem; 160 | line-height: 1.3; 161 | vertical-align: baseline; 162 | position: relative; } 163 | 164 | .table-caption { float:right; 165 | clear:right; 166 | margin-right: -60%; 167 | width: 50%; 168 | margin-top: 0; 169 | margin-bottom: 0; 170 | font-size: 1.0rem; 171 | line-height: 1.6; } 172 | 173 | .sidenote-number { counter-increment: sidenote-counter; } 174 | 175 | .sidenote-number:after, .sidenote:before { content: counter(sidenote-counter) " "; 176 | font-family: et-book-roman-old-style; 177 | position: relative; 178 | vertical-align: baseline; } 179 | 180 | .sidenote-number:after { content: counter(sidenote-counter); 181 | font-size: 1rem; 182 | top: -0.5rem; 183 | left: 0.1rem; } 184 | 185 | .sidenote:before { content: counter(sidenote-counter) " "; 186 | top: -0.5rem; } 187 | 188 | p, footer, table, div.table-wrapper-small, div.supertable-wrapper > p, div.booktabs-wrapper { width: 55%; } 189 | 190 | div.fullwidth, table.fullwidth { width: 100%; } 191 | 192 | div.table-wrapper { overflow-x: auto; 193 | font-family: "Trebuchet MS", "Gill Sans", "Gill Sans MT", sans-serif; } 194 | 195 | @media screen and (max-width: 760px) { p, footer { width: 90%; } 196 | pre.code { width: 87.5%; } 197 | ul { width: 85%; } 198 | figure { max-width: 90%; } 199 | figcaption, figure.fullwidth figcaption { margin-right: 0%; 200 | max-width: none; } 201 | blockquote p, blockquote footer { width: 90%; }} 202 | 203 | .sans { font-family: "Gill Sans", "Gill Sans MT", Calibri, sans-serif; 204 | letter-spacing: .03em; } 205 | 206 | .code { font-family: Consolas, "Liberation Mono", Menlo, Courier, monospace; 207 | font-size: 1.125rem; 208 | line-height: 1.6; } 209 | 210 | h1 .code, h2 .code, h3 .code { font-size: 0.80em; } 211 | 212 | .marginnote .code, .sidenote .code { font-size: 1rem; } 213 | 214 | pre.code { width: 52.5%; 215 | padding-left: 2.5%; 216 | overflow-x: auto; } 217 | 218 | .fullwidth { max-width: 90%; 219 | clear:both; } 220 | 221 | span.newthought { font-variant: small-caps; 222 | font-size: 1.2em; } 223 | 224 | input.margin-toggle { display: none; } 225 | 226 | label.sidenote-number { display: inline; } 227 | 228 | label.margin-toggle:not(.sidenote-number) { display: none; } 229 | 230 | @media (max-width: 760px) { label.margin-toggle:not(.sidenote-number) { display: inline; } 231 | .sidenote, .marginnote { display: none; } 232 | .margin-toggle:checked + .sidenote, 233 | .margin-toggle:checked + .marginnote { display: block; 234 | float: left; 235 | left: 1rem; 236 | clear: both; 237 | width: 95%; 238 | margin: 1rem 2.5%; 239 | vertical-align: baseline; 240 | position: relative; } 241 | label { cursor: pointer; } 242 | pre.code { width: 90%; 243 | padding: 0; } 244 | .table-caption { display: block; 245 | float: right; 246 | clear: both; 247 | width: 98%; 248 | margin-top: 1rem; 249 | margin-bottom: 0.5rem; 250 | margin-left: 1%; 251 | margin-right: 1%; 252 | vertical-align: baseline; 253 | position: relative; } 254 | div.table-wrapper, table, table.booktabs { width: 85%; } 255 | div.table-wrapper { border-right: 1px solid #efefef; } 256 | img { width: 100%; } } 257 | -------------------------------------------------------------------------------- /gan/index.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: post 3 | title: Generative Adversarial Networks 4 | --- 5 | 6 | We now move onto another family of generative models called generative adversarial networks (GANs). GANs are unique from all the other model families that we have seen so far, such as autoregressive models, VAEs, and normalizing flow models, because we do not train them using maximum likelihood. 7 | 8 | 9 | Likelihood-free learning 10 | ============== 11 | 12 | Why not? In fact, it is not so clear that better likelihood numbers necessarily correspond to higher sample quality. We know that the *optimal generative model* will give us the best sample quality and highest test log-likelihood. However, models with high test log-likelihoods can still yield poor samples, and vice versa. To see why, consider pathological cases in which our model is comprised almost entirely of noise, or our model simply memorizes the training set. Therefore, we turn to *likelihood-free training* with the hope that optimizing a different objective will allow us to disentangle our desiderata of obtaining high likelihoods as well as high-quality samples. 13 | 14 | Recall that maximum likelihood required us to evaluate the likelihood of the data under our model $$p_\theta$$. A natural way to set up a likelihood-free objective is to consider the *two-sample test*, a statistical test that determines whether or not a finite set of samples from two distributions are from the same distribution *using only samples from $$P$$ and $$Q$$*. Concretely, given $$S_1 = \{\mathbf{x} \sim P\}$$ and $$S_2 = \{\mathbf{x} \sim Q\}$$, we compute a test statistic $$T$$ according to the difference in $$S_1$$ and $$S_2$$ that, when less than a threshold $$\alpha$$, accepts the null hypothesis that $$P = Q$$. 15 | 16 | Analogously, we have in our generative modeling setup access to our training set $$S_1 = \mathcal{D} = \{\mathbf{x} \sim p_{\textrm{data}} \}$$ and $$S_2 = \{\mathbf{x} \sim p_{\theta} \}$$. The key idea is to train the model to minimize a *two-sample test objective* between $$S_1$$ and $$S_2$$. But this objective becomes extremely difficult to work with in high dimensions, so we choose to optimize a surrogate objective that instead *maximizes some distance* between $$S_1$$ and $$S_2$$. 17 | 18 | GAN Objective 19 | ============== 20 | 21 | We thus arrive at the generative adversarial network formulation. There are two components in a GAN: (1) a generator and (2) a discriminator. The generator $$G_\theta$$ is a directed latent variable model that deterministically generates samples $$\mathbf{x}$$ from $$\mathbf{z}$$, and the discriminator $$D_\phi$$ is a function whose job is to distinguish samples from the real dataset and the generator. The image below is a graphical model of $$G_\theta$$ and $$D_\phi$$. $$\mathbf{x}$$ denotes samples (either from data or generator), $$\mathbf{z}$$ denotes our noise vector, and $$\mathbf{y}$$ denotes the discriminator's prediction about $$\mathbf{x}$$. 22 | 23 |
24 |
drawing
25 | 28 |
29 | 30 | 31 | The generator and discriminator both play a two player minimax game, where the generator minimizes a two-sample test objective ($$p_{\textrm{data}} = p_\theta$$) and the discriminator maximizes the objective ($$p_{\textrm{data}} \neq p_\theta$$). Intuitively, the generator tries to fool the discriminator to the best of its ability by generating samples that look indistinguishable from $$p_{\textrm{data}}$$. 32 | 33 | Formally, the GAN objective can be written as: 34 | 35 | {% math %} 36 | \min_{\theta} \max_{\phi} V(G_\theta, D_\phi) = \mathbb{E}_{\mathbf{x} \sim \textbf{p}_{\textrm{data}}}[\log D_\phi(\textbf{x})] + 37 | \mathbb{E}_{\mathbf{z} \sim p(\textbf{z})}[\log (1-D_\phi(G_\theta(\textbf{z})))] 38 | {% endmath %} 39 | 40 | Let's unpack this expression. We know that the discriminator is maximizing this function with respect to its parameters $$\phi$$, where given a fixed generator $$G_\theta$$ it is performing binary classification: it assigns probability 1 to data points from the training set $$\mathbf{x} \sim p_{\textrm{data}}$$, and assigns probability 0 to generated samples $$\mathbf{x} \sim p_G$$. In this setup, the optimal discriminator is: 41 | 42 | {% math %} 43 | D^*_{G}(\mathbf{x}) = \frac{p_{\textrm{data}}(\mathbf{x})}{p_{\textrm{data}}(\mathbf{x}) + p_G(\mathbf{x})} 44 | {% endmath %} 45 | 46 | On the other hand, the generator minimizes this objective for a fixed discriminator $$D_\phi$$. And after performing some algebra, plugging in the optimal discriminator $$D^*_G(\cdot)$$ into the overall objective $$V(G_\theta, D^*_G(\mathbf{x}))$$ gives us: 47 | 48 | {% math %} 49 | 2D_{\textrm{JSD}}[p_{\textrm{data}}, p_G] - \log 4 50 | {% endmath %} 51 | 52 | The $$D_{\textrm{JSD}}$$ term is the *Jenson-Shannon Divergence*, which is also known as the symmetric form of the KL divergence: 53 | 54 | {% math %} 55 | D_{\textrm{JSD}}[p, q] = \frac{1}{2} \left( D_{\textrm{KL}}\left[p, \frac{p+q}{2} \right] + D_{\textrm{KL}}\left[q, \frac{p+q}{2} \right] \right) 56 | {% endmath %} 57 | 58 | The JSD satisfies all properties of the KL, and has the additional perk that $$D_{\textrm{JSD}}[p,q] = D_{\textrm{JSD}}[q,p]$$. With this distance metric, the optimal generator for the GAN objective becomes $$p_G = p_{\textrm{data}}$$, and the optimal objective value that we can achieve with optimal generators and discriminators $$G^*(\cdot)$$ and $$D^*_{G^*}(\mathbf{x})$$ is $$-\log 4$$. 59 | 60 | 61 | GAN training algorithm 62 | ============== 63 | 64 | Thus, the way in which we train a GAN is as follows: 65 | 66 | For epochs $$1, \ldots, N$$ do: 67 | 1. Sample minibatch of size $$m$$ from data: $$\mathbf{x}^{(1)}, \ldots, \mathbf{x}^{(m)} \sim \mathcal{D}$$ 68 | 2. Sample minibatch of size $$m$$ of noise: $$\mathbf{z}^{(1)}, \ldots, \mathbf{z}^{(m)} \sim p_z$$ 69 | 3. Take a gradient *descent* step on the generator parameters $$\theta$$: 70 | {% math %} 71 | \triangledown_\theta V(G_\theta, D_\phi) = \frac{1}{m} \triangledown_\theta \sum_{i=1}^m \log \left(1 - D_\phi(G_\theta(\mathbf{z}^{(i)})) \right) 72 | {% endmath %} 73 | 4. Take a gradient *ascent* step on the discriminator parameters $$\phi$$: 74 | {% math %} 75 | \triangledown_\phi V(G_\theta, D_\phi) = \frac{1}{m} \triangledown_\phi \sum_{i=1}^m \left[\log D_\phi(\mathbf{x}^{(i)}) + \log (1 - D_\phi(G_\theta(\mathbf{z}^{(i)}))) \right] 76 | {% endmath %} 77 | 78 | 79 | Challenges 80 | ============== 81 | 82 | Although GANs have been successfully applied to several domains and tasks, working with them in practice is challenging because of their: (1) unstable optimization procedure, (2) potential for mode collapse, (3) difficulty in evaluation. 83 | 84 | During optimization, the generator and discriminator loss often continue to oscillate without converging to a clear stopping point. Due to the lack of a robust stopping criteria, it is difficult to know when exactly the GAN has finished training. Additionally, the generator of a GAN can often get stuck producing one of a few types of samples over and over again (mode collapse). Most fixes to these challenges are empirically driven, and there has been a significant amount of work put into developing new architectures, regularization schemes, and noise perturbations in an attempt to circumvent these issues. Soumith Chintala has a nice [link](https://github.com/soumith/ganhacks) outlining various tricks of the trade to stabilize GAN training. 85 | 86 | 87 | Selected GANs 88 | ============== 89 | 90 | Next, we focus our attention to a few select types of GAN architectures and explore them in more detail. 91 | 92 | ### f-GAN 93 | The [f-GAN](https://arxiv.org/abs/1606.00709) optimizes the variant of the two-sample test objective that we have discussed so far, but using a very general notion of distance: the $$f divergence$$. Given two densities $$p$$ and $$q$$, the $$f$$-divergence can be written as: 94 | 95 | {% math %} 96 | D_f(p,q) = \mathbb{E}_{\mathbf{x}\sim q}\left[f \left(\frac{p(\mathbf{x})}{q(\mathbf{x})} \right) \right] 97 | {% endmath %} 98 | where $$f$$ is any convex[^1], lower-semicontinuous[^2] function with $$f(1) = 0$$. Several of the distance "metrics" that we have seen so far fall under the class of f-divergences, such as KL, Jenson-Shannon, and total variation. 99 | 100 | To set up the f-GAN objective, we borrow two commonly used tools from convex optimization[^3]: the Fenchel conjugate and duality. Specifically, we obtain a lower bound to any f-divergence via its Fenchel conjugate: 101 | 102 | {% math %} 103 | D_f(p,q) \geq \sup_{T \in \mathcal{T}} \left(\mathbb{E}_{x \sim p}[T(\mathbf{x})] - \mathbb{E}_{x \sim q}[f^*(T(\mathbf{x}))] \right) 104 | {% endmath %} 105 | 106 | Therefore we can choose any f-divergence that we desire, let $$p = p_{\textrm{data}}$$ and $$q = p_G$$, parameterize $$T$$ by $$\phi$$ and $$G$$ by $$\theta$$, and obtain the following fGAN objective: 107 | 108 | {% math %} 109 | \min_\theta \max_\phi F(\theta,\phi) = \mathbb{E}_{x \sim p_{\textrm{data}}}[T_\phi(\mathbf{x})] - \mathbb{E}_{x \sim p_{G_\theta}}[f^*(T_\phi(\mathbf{x}))] 110 | {% endmath %} 111 | 112 | Intuitively, we can think about this objective as the generator trying to minimize the divergence estimate, while the discriminator tries to tighten the lower bound. 113 | 114 | ### BiGAN 115 | We won't worry too much about the [BiGAN](https://arxiv.org/abs/1605.09782) in these notes. However, we can think about this model as one that allows us to infer latent representations even within a GAN framework. 116 | 117 | ### CycleGAN 118 | [CycleGAN](https://arxiv.org/abs/1703.10593) is a type of GAN that allows us to do unsupervised image-to-image translation, from two domains $$\mathcal{X} \leftrightarrow \mathcal{Y}$$. 119 | 120 | Specifically, we learn two conditional generative models: $$G: \mathcal{X} \leftrightarrow \mathcal{Y}$$ and $$F: \mathcal{Y} \leftrightarrow \mathcal{X}$$. There is a discriminator $$D_\mathcal{Y}$$ associated with $$G$$ that compares the true $$Y$$ with the generated samples $$\hat{Y} = G(X)$$. Similarly, there is another discriminator $$D_\mathcal{X}$$ associated with $$F$$ that compares the true $$X$$ with the generated samples $$\hat{X} = F(Y)$$. The figure below illustrates the CycleGAN setup: 121 | 122 |
123 |
drawing
124 | 127 |
128 | 129 | CycleGAN enforces a property known as *cycle consistency*, which states that if we can go from $$X$$ to $$\hat{Y}$$ via $$G$$, then we should also be able to go from $$\hat{Y}$$ to $$X$$ via $$F$$. The overall loss function can be written as: 130 | 131 | {% math %} 132 | \min_{F, G, D_\mathcal{X}, D_\mathcal{Y}} \mathcal{L}_{GAN}(G, D_\mathcal{Y}, X, Y) + \mathcal{L}_{GAN}(F, D_\mathcal{X}, X, Y) + \lambda \left(\mathbb{E}_X [||F(G(X)) - X||_1] + \mathbb{E}_Y [||G(F(Y)) - Y||_1] \right) 133 | {% endmath %} 134 | 135 | Footnotes 136 | ============== 137 | [^1]: In this context, convex means a line joining any two points that lies above the function. 138 | [^2]: The function value at any point $$\mathbf{x}_0$$ is close to or greater than $$f(\mathbf{x}_0)$$. 139 | [^3]: This [book](http://web.stanford.edu/~boyd/cvxbook/) is an excellent resource to learn more about these topics. 140 | -------------------------------------------------------------------------------- /docs/autoregressive/index.tex: -------------------------------------------------------------------------------- 1 | \section{Autoregressive Models} 2 | 3 | We begin our study with the autoregressive generative models. As before, we assume we are given access to a dataset $\mathcal{D}$ of $n$-dimensional datapoints $\mathbf{x}$. For simplicity, we assume the datapoints are binary, i.e., $\mathbf{x} \in \{0,1\}^n$. 4 | 5 | \section{Representation} 6 | 7 | By the chain rule of probability, we can factorize the joint distribution over the $n$-dimensions as: 8 | \[ 9 | \begin{equation} 10 | p(\mathbf{x}) = \prod\limits_{i=1}^{n}p(x_i \vert x_1, x_2, \ldots, x_{i-1}) = \prod\limits_{i=1}^{n} p(x_i \vert \mathbf{x}_{