├── .gitignore ├── README.md ├── figure ├── context.png ├── deltaproduct.jpg ├── fast_weight.tex ├── fla-repo.png ├── fla-speed.png ├── gdn-speed.png ├── hebbian.png ├── hybrid_twitter.png ├── io-aware-la.tex ├── mqar.tex ├── parallel_output_linearattn.tex ├── rnn.png ├── rwkv7.png ├── state_passing_deltanet.tex ├── state_passing_linearattn.tex ├── state_tracking.png └── tc0.png ├── intro.tex ├── linear_attn.tex ├── main.bib ├── main.pdf ├── main.tex ├── pdf_history └── 20250221.pdf └── section ├── beyond.tex ├── delta_rule.tex ├── intro.tex ├── linear_attn.tex └── linear_attn_with_decay.tex /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/.gitignore -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/README.md -------------------------------------------------------------------------------- /figure/context.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/context.png -------------------------------------------------------------------------------- /figure/deltaproduct.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/deltaproduct.jpg -------------------------------------------------------------------------------- /figure/fast_weight.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/fast_weight.tex -------------------------------------------------------------------------------- /figure/fla-repo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/fla-repo.png -------------------------------------------------------------------------------- /figure/fla-speed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/fla-speed.png -------------------------------------------------------------------------------- /figure/gdn-speed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/gdn-speed.png -------------------------------------------------------------------------------- /figure/hebbian.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/hebbian.png -------------------------------------------------------------------------------- /figure/hybrid_twitter.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/hybrid_twitter.png -------------------------------------------------------------------------------- /figure/io-aware-la.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/io-aware-la.tex -------------------------------------------------------------------------------- /figure/mqar.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/mqar.tex -------------------------------------------------------------------------------- /figure/parallel_output_linearattn.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/parallel_output_linearattn.tex -------------------------------------------------------------------------------- /figure/rnn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/rnn.png -------------------------------------------------------------------------------- /figure/rwkv7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/rwkv7.png -------------------------------------------------------------------------------- /figure/state_passing_deltanet.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/state_passing_deltanet.tex -------------------------------------------------------------------------------- /figure/state_passing_linearattn.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/state_passing_linearattn.tex -------------------------------------------------------------------------------- /figure/state_tracking.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/state_tracking.png -------------------------------------------------------------------------------- /figure/tc0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/figure/tc0.png -------------------------------------------------------------------------------- /intro.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/intro.tex -------------------------------------------------------------------------------- /linear_attn.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/linear_attn.tex -------------------------------------------------------------------------------- /main.bib: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/main.bib -------------------------------------------------------------------------------- /main.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/main.pdf -------------------------------------------------------------------------------- /main.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/main.tex -------------------------------------------------------------------------------- /pdf_history/20250221.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/pdf_history/20250221.pdf -------------------------------------------------------------------------------- /section/beyond.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/section/beyond.tex -------------------------------------------------------------------------------- /section/delta_rule.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/section/delta_rule.tex -------------------------------------------------------------------------------- /section/intro.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/section/intro.tex -------------------------------------------------------------------------------- /section/linear_attn.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/section/linear_attn.tex -------------------------------------------------------------------------------- /section/linear_attn_with_decay.tex: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sustcsonglin/linear-attention-and-beyond-slides/HEAD/section/linear_attn_with_decay.tex --------------------------------------------------------------------------------