├── assets ├── discrete.png └── continuous.png └── Readme.md /assets/discrete.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AoiDragon/Awesome-Text-Diffusion-Models/HEAD/assets/discrete.png -------------------------------------------------------------------------------- /assets/continuous.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AoiDragon/Awesome-Text-Diffusion-Models/HEAD/assets/continuous.png -------------------------------------------------------------------------------- /Readme.md: -------------------------------------------------------------------------------- 1 | # Awesome-Text-Diffusion-Models 2 | 3 | > A collection of papers related to text diffusion models. 4 | > 5 | > The organization of papers refer to our survey ['Diffusion Models for Non-autoregressive Text Generation: A Survey'](https://arxiv.org/abs/2303.06574) , which is accepted by IJCAI 2023 survey track. 6 | > 7 | > If you find our survey useful for your research, please cite the following paper: 8 | 9 | ``` 10 | @article{li2023diffusion, 11 | title={Diffusion Models for Non-autoregressive Text Generation: A Survey}, 12 | author={Li, Yifan and Zhou, Kun and Zhao, Wayne Xin and Wen, Ji-Rong}, 13 | journal={arXiv preprint arXiv:2303.06574}, 14 | year={2023} 15 | } 16 | ``` 17 | 18 | ## Continuous Text Diffusion Model 19 | 20 | ![avatar](/assets/continuous.png) 21 | 22 | 1. **Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions**\ 23 | *Emiel Hoogeboom et al.* NeurIPS 2021. [[Paper](https://arxiv.org/abs/2102.05379)] 24 | 25 | 2. **Diffusion-LM Improves Controllable Text Generation** \ 26 | *Xiang Lisa Li et al.* NeurIPS 2022. [[Paper](https://arxiv.org/abs/2205.14217)] [[Code](https://github.com/XiangLi1999/Diffusion-LM)] 27 | 28 | 3. **Composable Text Controls in Latent Space with ODEs** \ 29 | *Guangyi Liu et al.* arxiv 2022. [[Paper](https://arxiv.org/abs/2208.00638)] [[Code](https://github.com/guangyliu/LatentOps)] 30 | 31 | 4. **DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models** \ 32 | *Shansan Gong et al.* ICLR 2023. [[Paper](https://arxiv.org/abs/2210.08933)] [[Code](https://github.com/Shark-NLP/DiffuSeq)] 33 | 34 | 5. **SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control** \ 35 | *Xiaochuang Han et al.* arxiv 2022.[[Paper](https://arxiv.org/abs/2210.17432)] [[Code](https://github.com/xhan77/ssd-lm)] 36 | 37 | 6. **Self-conditioned Embedding Diffusion for Text Generation** \ 38 | *Robin Strudel et al.* arxiv 2022. [[Paper](https://arxiv.org/abs/2211.04236)] 39 | 40 | 7. **Continuous diffusion for categorical data** \ 41 | *Sander Dieleman et al.* arxiv 2022. [[Paper](https://arxiv.org/abs/2211.15089)] 42 | 43 | 8. **Difformer: Empowering Diffusion Model on Embedding Space for Text Generation** \ 44 | *Zhujin Gao et al.* arxiv 2022. [[Paper](https://arxiv.org/abs/2212.09412)] 45 | 46 | 9. **Latent Diffusion for Language Generation** \ 47 | *Justin Lovelace et al.* arxiv 2022. [[Paper](https://arxiv.org/pdf/2212.09462)] [[Code](https://github.com/justinlovelace/latent-diffusion-for-language)] 48 | 49 | 10. **SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers** \ 50 | *Hongyi Yuan et al.* arxiv 2022. [[Paper](https://arxiv.org/abs/2212.10325)] [[Code](https://github.com/Yuanhy1997/SeqDiffuSeq)] 51 | 52 | 11. **Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise** \ 53 | *Zhenghao Lin et al.* arxiv 2022. [[Paper](https://arxiv.org/abs/2212.11685)] [[Code](https://github.com/microsoft/ProphetNet/tree/master/GENIE)] 54 | 55 | 12. **A Reparameterized Discrete Diffusion Model for Text Generation** \ 56 | *Lin Zheng et al.* arxiv 2023. [[Paper](https://arxiv.org/abs/2302.05737)] [[Code](https://github.com/HKUNLP/reparam-discrete-diffusion)] 57 | 58 | 13. **DINOISER: Diffused Conditional Sequence Learning by Manipulating Noises** \ 59 | *Jiasheng Ye et al.* arxiv 2023. [[Paper](https://arxiv.org/abs/2302.10025)] [[Code](https://github.com/yegcjs/DINOISER)] 60 | 61 | 14. **GlyphDiffusion: Text Generation as Image Generation** \ 62 | *Junyi Li et al.* arxiv 2023. [[Paper](https://arxiv.org/abs/2304.12519)] 63 | 64 | 15. **DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM** \ 65 | *Weijie Xu et al.* Findings of EMNLP 2023. [[Paper](https://arxiv.org/abs/2310.15296)] 66 | 67 | ## Discrete Text Diffusion Models 68 | 69 | ![2](/assets/discrete.png) 70 | 71 | 1. **Structured Denoising Diffusion Models in Discrete State-Spaces** \ 72 | *Jacob Austin et al.* NeurIPS 2021. [[Paper](https://arxiv.org/abs/2107.03006)] 73 | 74 | 2. **DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models** \ 75 | *Zhengfu He et al.* arXiv 2022. [[Paper](https://arxiv.org/abs/2211.15029)] [[Code](https://github.com/Hzfinfdu/Diffusion-BERT)] 76 | 77 | 3. **Diff-Glat: Diffusion Glancing Transformer for Parallel Sequence to Sequence Learning** \ 78 | *Lihua Qian et al.* arxiv 2022. [[Paper](https://arxiv.org/abs/2212.10240)] 79 | 80 | 4. **Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation** \ 81 | *Kun Zhou et al.* arxiv 2023. [[Paper](https://arxiv.org/abs/2305.04044)] 82 | 83 | 84 | --------------------------------------------------------------------------------