├── LICENSE └── README.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 ai-agi 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # LLMs-Enhanced-Long-Text-Generation-Survey 2 | 3 | Long Form NLG Generation Based on Large Language Models 4 | 5 | # **Resource** 6 | 7 | ## A. Task Perspective 8 | 9 | ### _A.1 Long Form Open Domain Dialogue_ 10 | 11 | 1. MemoChat: Tuning LLMs to Use Memos for Consistent Long-Range Open-Domain Conversation. _Junru Lu, Siyu An, Mingbao Lin, Gabriele Pergola, Yulan He, Di Yin, Xing Sun and Yunsheng Wu._ [\[pdf\]](https://arxiv.org/pdf/2308.08239.pdf). `arXiv Aug 16, 2023`. 12 | 2. Prompted LLMs as Chatbot Modules for Long Open-domain Conversation. _Gibbeum Lee，Volker Hartmann， Jongho Park，Dimitris Papailiopoulos and Kangwook Lee._ [\[pdf\]](https://aclanthology.org/2023.findings-acl.277.pdf). `Findings of ACL 2023`. 13 | 14 | ### _A.2 Long Dialogue Summarization_ 15 | 16 | 1. An Exploratory Study on Long Dialogue Summarization: What Works and What’s Next.Yusen Zhang, Ansong Ni, Tao Yu, Rui Zhang, Chenguang Zhu, Budhaditya Deb, Asli Celikyilmaz, Ahmed Hassan Awadallah, Dragomir Radev, [\[pdf\]](https://arxiv.org/pdf/2109.04609.pdf)`Arxiv 10 Sep 2021` 17 | 2. Improving Long Dialogue Summarization with Semantic Graph Representation.Yilun Hua, Zhaoyuan Deng, Kathleen McKeown, [\[pdf\]](https://pdfs.semanticscholar.org/4f75/d77de71d6c61cc2f732849a02cf4ff2f3282.pdf)`ACL July 9-14 2023` 18 | 3. DIALOGLM: Pre-trained Model for Long Dialogue Understanding and Summarization.Ming Zhong, Yang Liu, Yichong Xu, Chenguang Zhu, Michael Zeng, [\[pdf\]](https://arxiv.org/pdf/2109.02492.pdf)`Arxiv 6 Jan 2022` 19 | 4. Negative Guided Abstractive Dialogue Summarization.Junpeng Liu, Yanyan Zou, Yuxuan Xi, Shengjie Li, Mian Ma, Zhuoye Ding, Bo Long, [\[pdf\]](https://www.isca-speech.org/archive/pdfs/interspeech_2022/liu22r_interspeech.pdf)`Interspeech 18-22 Sept 2022` 20 | 5. Improving Abstractive Dialogue Summarization with Graph Structures and Topic Words.Lulu Zhao, Weiran Xu, Jun Guo, [\[pdf\]](https://aclanthology.org/2020.coling-main.39/)`ICCL Dec 8-13 2020` 21 | 6. Improving Long Dialogue Summarization with Semantic Graph Representation._Bobby Yilun Hua, Zhaoyuan Deng and K. McKeown_.[\[pdf\]](https://pdfs.semanticscholar.org/4f75/d77de71d6c61cc2f732849a02cf4ff2f3282.pdf)`ACL July 2023` 22 | 23 | ### _A.3 Long Document Summarization_ 24 | 25 | 1. Two-Stage Movie Script Summarization: An Efficient Method For Low-Resource Long Document Summarization.Dongqi Pu, Xudong Hong, Pin-Jie Lin, Ernie Chang, Vera Demberg, [\[pdf\]](https://aclanthology.org/2022.creativesumm-1.9.pdf).`ACL July 2020` 26 | 2. Efficient Attentions for Long Document Summarization.Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, Lu Wang, [\[pdf\]](https://aclanthology.org/2021.naacl-main.112.pdf)`NAACL June 6-11 2021`. 27 | 3. HEGEL: Hypergraph Transformer for Long Document Summarization.Haopeng Zhang, Xiao Liu, Jiawei Zhang, [\[pdf\]](https://aclanthology.org/2022.emnlp-main.692.pdf)`NLP Dec 7-11 2022` 28 | 4. Long Document Summarization with Top-down and Bottom-up Inference.Bo Pang, Erik Nijkamp, Wojciech Kryscinski, Silvio Savarese, Yingbo Zhou, Caiming Xiong, [\[pdf\]](https://aclanthology.org/2023.findings-eacl.94.pdf)`EACL May 2-6 2023` 29 | 5. Globalizing BERT-based Transformer Architectures for Long Document.Quentin Grail, Julien Perez, [\[pdf\]](https://aclanthology.org/2021.eacl-main.154/)`EACL April 19-23 2021` 30 | 6. PLSGA：阶段式长文本摘要生成方法._方缙, 李宝安, 游新冬 and 吕学强_.[\[pdf\]](https://kns.cnki.net/kcms2/article/abstract?v=z-q19lQZUWFmmUoqQWhR6VMffqBnwCKFrAciK1CfhSjuK90QS8IY7tkSt1vIfMiMwfmYgJ9pX0mm33l7Fu1IDLK2lZ3ol5-mqAyNmJZXz2iDDemeX05A-btaRGuW1EINVzU_mPW7CO74YhgSffR2YBNqSIjXD2wF&uniplatform=NZKPT&language=CHS)`CNKI Nov 2023` 31 | 7. 关于中文长文本的自动文本摘要算法研究._李永星_.[\[pdf\]](https://kns.cnki.net/kcms2/article/abstract?v=z-q19lQZUWFWm3VAJd8LjBgJswv_7te-SqBC2MNxUcAt8jR-t_XatSFw-G7mdcxm0JQWiiihJpG1YPMx1EuzFX2OO12BlaM3qgn_GorNJV1vF2XokkgsCljPLXkJJqvURwmts5g0XpUKNCsn1IffMAUyfeW7R5SjSGDJeepKaS0=&uniplatform=NZKPT&language=CHS)`CNKI Jan 2023` 32 | 8. V2P: Vision-to-Prompt based Multi-Modal Product Summary Generation._Xuemeng Song, Liqiang Jing, Dengtian Lin, Zhongzhou Zhao, Haiqing Chen and Liqiang Nie_.[\[doi.org\]](https://dl.acm.org/doi/10.1145/3477495.3532076)`ACM 7 July 2022` 33 | 34 | ### _A.4 Long-Form Narrative Text_ 35 | 36 | 1. EIPE-text: Evaluation-Guided Iterative Plan Extraction for Long-Form Narrative Text Generation.Wang You, WenshanWu, Yaobo Liang, Shaoguang Mao, Chenfei Wu, Maosong Cao, Yuzhe Cai, Yiduo Guo, Yan Xia, Furu Wei, Nan Duan, [\[pdf\]](https://arxiv.org/pdf/2310.08185.pdf).`Arxiv 12 Oct 2023`. 37 | 38 | ### _A.5 Story_ 39 | 40 | 1. A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation.Jian Guan, Fei Huang, Zhihao Zhao, Xiaoyan Zhu, Minlie Huang, [\[pdf\]](https://aclanthology.org/2020.tacl-1.7.pdf)`TACL 1 Jan 2020` 41 | 2. Open-ended Long Text Generation via Masked Language Modeling.Xiaobo Liang, Zecheng Tang, Juntao Li, Min Zhang, [\[pdf\]](https://aclanthology.org/2023.acl-long.13.pdf)`ACL July 9-14 2023` 42 | 3. Improving Chinese Story Generation via Awareness of Syntactic Dependencies and Semantics.Henglin Huang, Chen Tang, Tyler Loakman, Frank Guerin, Chenghua Lin, [\[pdf\]](https://aclanthology.org/2022.aacl-short.23.pdf)`AACL 19 Oct 2022` 43 | 4. NGEP: A Graph-based Event Planning Framework for Story Generation._Chen Tang, Zhihao Zhang, Tyler Loakman, Chenghua Lin and Frank Guerin_.[\[pdf\]](https://aclanthology.org/2022.aacl-short.24.pdf)`ACL 2022` 44 | 5. Improving Chinese Story Generation via Awareness of Syntactic Dependencies and Semantics._Henglin Huang, Chen Tang, Tyler Loakman, Frank Guerin1 and Chenghua Lin_.[\[pdf\]](https://aclanthology.org/2022.aacl-short.23.pdf)`ACL 2022` 45 | 6. Make-A-Story: Visual Memory Conditioned Consistent Story Generation._Tanzila Rahman, Hsin-Ying Lee, Jian Ren, S. Tulyakov, Shweta Mahajan, L. Sigal_.[\[pdf\]](https://arxiv.org/pdf/2211.13319.pdf)`arXiv 6 May 2023` 46 | 7. EtriCA: Event-Triggered Context-Aware Story Generation Augmented by Cross Attention._Chen Tang, Chenghua Lin, Hen-Hsen Huang, Frank Guerin and Zhihao Zhang_.[\[pdf\]](https://arxiv.org/pdf/2210.12463.pdf)`arXiv 22 Oct 2022` 47 | 8. GraphPlan: Story Generation by Planning with Event Graph._Hong Chen, Raphael Shu, Hiroya Takamura, Hideki Nakayama_.[\[pdf\]](https://aclanthology.org/2021.inlg-1.42.pdf)`INLG Sept 2021` 48 | 9. Controllable Story Generation with External Knowledge Using Large-Scale Language Models._Peng Xu, M. Patwary, M. Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar and Bryan Catanzaro_.[\[pdf\]](https://aclanthology.org/2020.emnlp-main.226.pdf)`NLP 2 Oct 2020` 49 | 10. A Temporal Variational Model for Story Generation._David Wilmot and Frank Keller_.[\[pdf\]](https://arxiv.org/pdf/2109.06807.pdf)`arXiv 14 Sept 2021` 50 | 11. Chinese Story Generation with FastText Transformer Network._Jhe-Wei Lin, Yunwen Gao and Rong-Guey Chang_.[\[doi.org\]](https://ieeexplore.ieee.org/document/8669087)`IEEE Feb 2019` 51 | 12. Improving Pacing in Long-Form Story Planning._Yichen Wang, Kevin Yang, Xiaoming Liu and Dan Klein_.[\[pdf\]](https://arxiv.org/pdf/2311.04459.pdf)`arXiv 8 Nov 2023` 52 | 53 | ### _A.6 Reviews_ 54 | 55 | 1. Towards coherent and cohesive long-form text generation.Woon Sang Cho, Pengchuan Zhang, Yizhe Zhang, Xiujun Li, Michel Galley, Chris Brockett, Mengdi Wang, Jianfeng Gao, [\[pdf\]](https://aclanthology.org/W19-2401.pdf)`ACL 1 Nov 2018` 56 | 57 | ### _A.7 Steganography_ 58 | 59 | 1. Generative Steganography Based on Long Readable Text Generation. _Yi Cao, Zhili Zhou, Chinmay Chakraborty, Meimin Wang, Q.M.Jonathan Wu, Xingming Sun and Keping Yu_.[\[pdf\]](https://ieeexplore.ieee.org/document/9778236?denied=)`IEEE 19 May 2022` 60 | 61 | ### _A.8 Table-to-Text_ 62 | 63 | 1. Two-Level Model for Table-to-Text Generation._Juan Cao, Junpeng Gong and Pengzhou Zhang_.[\[pdf\]](https://dl.acm.org/doi/10.1145/3364908.3365287)`ACM Sept 2019` 64 | 2. Three-stage Logical Table-to-Text Generation based on Type Control._Weiwei Shi, Yubo Liu, Jie Wu and Jianming Liao_.[\[pdf\]](https://dl.acm.org/doi/10.1145/3579654.3579667)`ACM Dec 2022` 65 | 66 | ### _A.9 Patent_ 67 | 68 | 1. Controlling Patent Text Generation by Structural Metadata._Jieh-Sheng Lee_.[\[pdf\]](https://dl.acm.org/doi/10.1145/3340531.3418503)`ACM Oct 2020` 69 | 70 | ### _A.10 Multilingual abstract_ 71 | 72 | 1. XWikiGen: Cross-lingual Summarization for Encyclopedic Text Generation in Low Resource Languages._Dhaval Taunk, Shivprasad Sagare, Anupam Patil, Shivansh Subramanian and Manish Gupta_.[\[pdf\]](https://dl.acm.org/doi/10.1145/3543507.3583405)`ACM Apr 2023` 73 | 2. Long-Document Cross-Lingual Summarization._Shaohui Zheng, Zhixu Li, Jiaan Wang, Jianfeng Qu, An Liu, Lei Zhao and Zhigang Chen_.[\[pdf\]](https://dl.acm.org/doi/10.1145/3539597.3570479)`ACM Feb 2023` 74 | 3. mLongT5: A Multilingual and Efficient Text-To-Text Transformer for Longer Sequences._David C. Uthus, Santiago Ontan'on, J. Ainslie and Mandy Guo_.[\[pdf\]](https://arxiv.org/pdf/2305.11129.pdf)`arXiv 26 Oct 2023` 75 | 76 | ### _A.11 Persuasive Text_ 77 | 78 | 1. PersuAIDE! An Adaptive Persuasive Text Generation System for Fashion Domain._Vitobha Munigala, Abhijit Mishra, Srikanth G. Tamilselvam, Shreya Khare, Riddhiman Dasgupta and Anush Sankaran_.[\[pdf\]](https://dl.acm.org/doi/10.1145/3184558.3186345)`ACM Apr 2018` 79 | 80 | ### _A.12 Hierarchical Topic-to-Essay_ 81 | 82 | 1. Transformer-based Hierarchical Topic-to-Essay Generation._Wangbo He and Yuan Rao_.[\[pdf\]](https://www.sciencedirect.com/science/article/pii/S1877050922005920)`ScienceDirect May 2022` 83 | 84 | ### _A.13 Expository Text_ 85 | 86 | 1. Expository Text Generation: Imitate, Retrieve, Paraphrase._Nishant Balepur, Jie Huang and K. Chang_.[\[pdf\]](https://arxiv.org/pdf/2305.03276.pdf)`arXiv 2023` 87 | 88 | ### _A.14 Open-ended Text_ 89 | 90 | 1. KNN-LM Does Not Improve Open-ended Text Generation._Shufan Wang, Yixiao Song, Andrew Drozdov, Aparna Garimella, Varun Manjunatha and Mohit Iyyer_.[\[pdf\]](https://arxiv.org/pdf/2305.14625.pdf)`arXiv 2023` 91 | 2. Towards Informative Open-ended Text Generation with Dynamic Knowledge Triples._Zixuan Ren, Yang Zhao and Chengqing Zong_.[\[pdf\]](https://aclanthology.org/2023.findings-emnlp.210.pdf)`EMNLP 2023` 92 | 3. Open-ended Long Text Generation via Masked Language Modeling._Xiaobo Liang, Zecheng Tang, Juntao Li and Min Zhang_.[\[pdf\]](https://aclanthology.org/2023.acl-long.13.pdf)`ACL July 2023` 93 | 94 | ### _A.15 Poetry_ 95 | 96 | 1. ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models._Jonas Belouadi and Steffen Eger_.[\[pdf\]](https://aclanthology.org/2023.acl-long.406.pdf)`ACL 20 Dec 2022` 97 | 2. Chinese poetry generation model with UniLM._Zhangmin Ling and Lin Zhang_[\[doi.org\]](https://ieeexplore.ieee.org/document/9712702)`IEEE Jan 2022` 98 | 3. Modern French Poetry Generation with RoBERTa and GPT-2._Mika Hämäläinen, Khalid Alnajjar and T. Poibeau_.[\[pdf\]](https://arxiv.org/pdf/2212.02911.pdf)`arXiv Dec 2022` 99 | 4. Ancient poetry generation with an unsupervised method._Zhanjun Zhang, Haoyu Zhang, Qian Wan, Xiangyu Jia, Zhe Zhang and Jie Liu_.[\[pdf\]](https://link.springer.com/content/pdf/10.1007/s00521-021-06571-w.pdf)`Springer 12 Mar 2022` 100 | 5. A Sentiment and Style Controllable Approach for Chinese Poetry Generation._Yizhan Shao, Tong Shao, Minghao Wang, Peng Wang and Jie Gao_.[\[doi.org\]](https://dl.acm.org/doi/10.1145/3459637.3481964)`ACM 30 Oct 2021` 101 | 6. SP-GPT2: Semantics Improvement in Vietnamese Poetry Generation._Tuan-Duy H. Nguyen, H. Pham, T. Bui, Tan-Minh Nguyen, D. Luong and Phong Nguyen.[\[pdf\]](https://arxiv.org/pdf/2110.15723.pdf)`arXiv Oct 2021` 102 | 7. Classical Chinese Poetry Generation based on Transformer-XL._Jianli Zhao and H. Lee_.[\[doi.org\]](https://ieeexplore.ieee.org/document/9544316)`IEEE Aug 2021` 103 | 8. AfriKI: Machine-in-the-Loop Afrikaans Poetry Generation._Imke van Heerden and Anil Bas_.[\[pdf\]](https://aclanthology.org/2021.hcinlp-1.12.pdf)`HCINLP 30 March 2021` 104 | 9. Lingxi: A Diversity-aware Chinese Modern Poetry Generation System._Xinran Zhang, Maosong Sun, Jiafeng Liu, Xiaobing Li_.[\[pdf\]](https://aclanthology.org/2023.acl-demo.6.pdf)`ACL 27 Aug 2021` 105 | 10. Jiuge: A Human-Machine Collaborative Chinese Classical Poetry Generation System._Zhipeng Guo, Xiaoyuan Yi, Maosong Sun, Wenhao Li, Cheng Yang, Jian-na Liang, Huimin Chen, Yuhui Zhang and Ruoyu Li_.[\[pdf\]](https://nlp.csai.tsinghua.edu.cn/~chm/publications/acl2019_jiugedemo.pdf)`ACL 1 July 2019` 106 | 11. Compose Like Humans: Jointly Improving the Coherence and Novelty for Modern Chinese Poetry Generation._Lei Shen, Xiaoyu Guo and Meng Chen_.[\[pdf\]](https://arxiv.org/pdf/2005.01556.pdf)`arXiv 4 May 2020` 107 | 12. Image Inspired Poetry Generation in XiaoIce._Wen-Feng Cheng, Chao-Chung Wu, Ruihua Song, Jianlong Fu, Xing Xie and Jian-Yun Nie_.[\[pdf\]](https://arxiv.org/pdf/1808.03090.pdf)`arXiv 9 Aug 2018` 108 | 13. TPoet: Topic-Enhanced Chinese Poetry Generation._Liang Yang, Zhexu Shen, Fengqing Zhou, Hongfei Lin and Junpeng Li_.[\[doi.org\]](https://dl.acm.org/doi/10.1145/3593805)`ACM 19 June 2023` 109 | 14. GPT-based Generation for Classical Chinese Poetry._Yi Liao, Yasheng Wang, Qun Liu and Xin Jiang_.[\[pdf\]](https://arxiv.org/pdf/1907.00151.pdf)`arXiv 29 June 2019` 110 | 15. Automatic Generation Method of Ancient Poetry Based on LSTM._Hanshuang Zhang and Zhi Zhang_.[\[doi.org\]](https://ieeexplore.ieee.org/document/9248260)`IEEE Nov 2020` 111 | 112 | ### _A.16 Script_ 113 | 114 | 1. GPT-2-based Human-in-the-loop Theatre Play Script Generation._Rudolf Rosa, Patrícia Schmidtová, Ondrej Dusek, Tomáš Musil, D. Mareček, Saad Obaid, Marie Nováková, Klára Vosecká and Josef Doležal_.[\[pdf\]](https://aclanthology.org/2022.wnu-1.4.pdf)`WNU 2022` 115 | 2. The Film Script Generation Analysis Based on the Fiction Book Text Using Machine Learning._Danylo Ivanchyshyn, V. Vysotska and S. Albota_[\[doi.org\]](https://ieeexplore.ieee.org/document/9648818)`IEEE Sept 2021` 116 | 3. Leveraging Narrative to Generate Movie Script._Yutao Zhu, Ruihua Song, J. Nie, Pan Du, Zhicheng Dou and Jin Zhou_.[\[doi.org\]](https://dl.acm.org/doi/10.1145/3507356)`ACM 9 March 2022` 117 | 4. "Kurosawa": A Script Writer's Assistant._Prerak Gandhi, Vishal Pramanik and P. Bhattacharyya_.[\[pdf\]](https://arxiv.org/pdf/2308.03122.pdf)`arXiv 6 Aug 2023` 118 | 119 | ### _A.17 News_ 120 | 121 | 1. Fact-Enhanced Synthetic News Generation._Kai Shu, Yichuan Li, Kaize Ding and Huan Liu_.[\[pdf\]](https://arxiv.org/pdf/2012.04778.pdf)`arXiv 13 Dec 2020` 122 | 2. Topic-Preserving Synthetic News Generation: An Adversarial Deep Reinforcement Learning Approach._Ahmadreza Mosallanezhad, Kai Shu and Huan Liu_.[\[pdf\]](https://arxiv.org/pdf/2010.16324.pdf)`arXiv 30 Oct 2020` 123 | 3. Data-Driven News Generation for Automated Journalism._Leo Leppänen, Myriam Munezero, Mark Granroth-Wilding and Hannu (TT) Toivonen_.[\[pdf\]](https://www.cs.helsinki.fi/u/htoivone/pubs/leppanenetal_inlg_2017.pdf)`NLG 1 Sept 2017` 124 | 4. SHEG: summarization and headline generation of news articles using deep learning._R. Singh, Sonia Khetarpaul, R. Gorantla, Sai Giridhar Allada_.[\[pdf\]](https://link.springer.com/content/pdf/10.1007/s00521-020-05188-9.pdf)`Springer 23 July 2020` 125 | 126 | ### _A.18 Paper_ 127 | 128 | 1. Neural Academic Paper Generation._Samet Demir, Uras Mutlu and Özgür Özdemir_.[\[pdf\]](https://arxiv.org/pdf/1912.01982.pdf)`arXiv 2 Dec 2019` 129 | 130 | ### _A.19 Advertising Slogan_ 131 | 132 | 1. Japanese Advertising Slogan Generator using Case Frame and Word Vector._Kango Iwama and Yoshinobu Kano_.[\[pdf\]](https://aclanthology.org/W18-6526.pdf)`INLG Nov 2018` 133 | 2. Smart Generation System of Personalized Advertising Copy and Its Application to Advertising Practice and Research._Shasha Deng, Chee‐Wee Tan, Weijun Wang and Yu Pan_.[\[pdf\]](https://research-api.cbs.dk/ws/portalfiles/portal/61444279/chee_wee_tan_et_al_smart_generation_system_of_personalized_advertising_copy_acceptedversion.pdf)`Journal of Advertising 8 Aug 2019` 134 | 3. Scenario-based Multi-product Advertising Copywriting Generation for E-Commerce._Xueying Zhang, Kai Shen, Chi Zhang, Xiaochuan Fan, Yun Xiao, Zhen He, Bo Long and Lingfei Wu_.[\[pdf\]](https://arxiv.org/pdf/2205.10530.pdf)`arXiv 21 May 2022` 135 | 136 | ### _A.20 Email_ 137 | 138 | 1. Template-based Contact Email Generation for Job Recommendation._Qiuchi Li and C. Lioma_.[\[pdf\]](https://aclanthology.org/2022.gem-1.15.pdf)`IEEE 6 Dec 2022` 139 | 2. Automated email Generation for Targeted Attacks using Natural Language._Avisha Das and Rakesh M. Verma_.[\[pdf\]](https://arxiv.org/pdf/1908.06893.pdf)`arXiv 19 Aug 2019` 140 | 141 | ### _A.21 Description_ 142 | 143 | 1. Towards Knowledge-Based Personalized Product Description Generation in E-commerce._Qibin Chen, Junyang Lin, Yichang Zhang, Hongxia Yang, Jingren Zhou and Jie Tang_.[\[pdf\]](https://arxiv.org/pdf/1903.12457.pdf)`arXiv 5 Jun 2019` 144 | 2. Probing Product Description Generation via Posterior Distillation._Haolan Zhan, Hainan Zhang, Hongshen Chen, Lei Shen, Zhuoye Ding, Yongjun Bao, Weipeng P. Yan and Yanyan Lan_.[\[pdf\]](https://arxiv.org/pdf/2103.01594.pdf)`arXiv 2 Mar 2021` 145 | 3. Automatic Generation of Pattern-controlled Product Description in E-commerce._Zhang Tao, Jin Zhang, Chengfu Huo and Weijun Ren_.[\[doi.org\]](https://dl.acm.org/doi/10.1145/3308558.3313407)`ACM 13 May 2019` 146 | 4. Description Generation for Points of Interest._Meng Zhou, Jingbo Zhou, Yanjie Fu, Z. Ren, Xiaoli Wang and Hui Xiong_.[\[doi.org\]](https://ieeexplore.ieee.org/document/9458894)`IEEE Apr 2021` 147 | 148 | ### _A.22 Code_ 149 | 150 | ## B. Constraints Perspective 151 | 152 | 1. Fixed global memory for controllable long text generation. _Zheng Chen, Zhejun Liu._ [\[pdf\]](https://dl.acm.org/doi/abs/10.1007/s10489-022-04197-6). `Applied Intelligence, 2022`. 153 | 2. Critic-Guided Decoding for Controlled Text Generation.Minbeom Kim, Hwanhee Lee, Kang Min Yoo, Joonsuk Park, Hwaran Lee, Kyomin Jung, [\[pdf\]](https://arxiv.org/pdf/2212.10938.pdf)`Arxiv 21 Dec 2022` 154 | 3. MDM: Meta diffusion model for hard-constrained text generation._Wenjun Ke, Yikai Guo, Qi Liu, Wanyi Chen, Peng Wang, Haoran Luo and Zhizhao Luo_.[\[pdf\]](https://www.sciencedirect.com/science/article/abs/pii/S0950705123008973)`ScienceDirect Nov 2023` 155 | 156 | ## C. Technique (Method) Perspective 157 | 158 | ### C.1 Data Augmentation 159 | 160 | 1. Data augmentation in natural language processing: a novel text generation approach for long and short text classifers.Markus Bayer, Marc‑André Kaufhold, Björn Buchhold, Marcel Keller, Jörg Dallmeyer and Christian Reuter.[\[pdf\]](https://link.springer.com/content/pdf/10.1007/s13042-022-01553-3.pdf?pdf=button) `International Journal of Machine Learning and Cybernetics (2023)`. 161 | 2. Augmented Language Models: a Survey Grégoire Mialon， Roberto Dessì， Maria Lomel [\[pdf\]](https://arxiv.org/pdf/2302.07842.pdf) `Arxiv Feb 15 2023` 162 | 163 | ### C.2 Detector 164 | 165 | 1. A Survey on LLM-generated Text Detection: Necessity, Methods, and Future Directions.Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Derek F. Wong Senior Member, IEEE and Lidia S. Chao Member, IEEE. [\[pdf\]](https://arxiv.org/pdf/2310.14724.pdf) `Arxiv Oct 24, 2023`. 166 | 167 | ### C.3 Instruction Tuning 168 | 169 | 1. LongForm: Optimizing Instruction Tuning for Long Text Generation with Corpus Extraction.Abdullatif Köksal, Timo Schick, Anna Korhonen, Hinrich Schütze [\[pdf\]](https://arxiv.org/abs/2304.08460) `Arxiv *17 Apr 2023*`. 170 | 171 | 172 | ### C.4 Adversarial Training 173 | 174 | 1. Long Text Generation via Adversarial Training with Leaked Information.Jiaxian Guo, Sidi Lu, Han Cai, Weinan Zhang, Yong Yu, Jun Wang, [\[pdf\]](https://arxiv.org/pdf/1709.08624.pdf)`Arxiv 8 Dec 2017`. 175 | 2. Improving Adversarial Text Generation by Modeling the Distant Future.Ruiyi Zhang, Changyou Chen, Zhe Gan, Wenlin Wang, Dinghan Shen, GuoyinWang, Zheng Wen, Lawrence Carin, [\[pdf\]](https://aclanthology.org/2020.acl-main.227.pdf)`ACL 4 May 2020`. 176 | 3. Diversity regularized autoencoders for text generation._Hyeseon Ko, Junhyuk Lee, Jinhong Kim, Jongwuk Lee and Hyunjung Shim_.[\[pdf\]](https://dl.acm.org/doi/10.1145/3341105.3373998)`ACM Mar 2020` 177 | 4. 融合自注意力机制的长文本生成对抗网络模型._夏鸿斌， 肖奕飞 and 刘渊_.[\[pdf\]](http://fcst.ceaj.org/CN/10.3778/j.issn.1673-9418.2104038)`计算机科学与探索 2022 16(7)` 178 | 5. Feature-aware conditional GAN for category text generation._Xinze Li, Kezhi Mao, Fanfan Lin and Zijian Feng_.[\[pdf\]](https://www.sciencedirect.com/science/article/abs/pii/S0925231223004757)`ScienceDirect May 2023` 179 | 6. WordIllusion: An adversarial text generation algorithm based on human cognitive system._Haoran Fu, Chundong Wang, Jiaqi Sun, Yumeng Zhao, Hao Lin, Junqing Sun and Baixue Zhang_.[\[pdf\]](https://www.sciencedirect.com/science/article/abs/pii/S1389041723001134)`ScienceDirect Oct 2023` 180 | 181 | ### C.5 Task-adaptive Tokenization 182 | 183 | 1. Enhancing Long-form Text Generation in Mental Health with Task-adaptive Tokenization.Siyang Liu, Naihao Deng, Sahand Sabour, Yilin Jia, Minlie Huang, Rada Mihalcea, [\[pdf\]](https://arxiv.org/pdf/2310.05317.pdf)`Arxiv 23 Oct 2023`. 184 | 185 | ### C.6 Graph-based 186 | 187 | 1. Graph-based Multi-hop Reasoning for Long Text Generation.Liang Zhao, Jingjing Xu, Junyang Lin, Yichang Zhang, Hongxia Yang, Xu Sun, [\[pdf\]](https://arxiv.org/pdf/2009.13282.pdf)`Arxiv 28 Sep 2020`. 188 | 2. Text Generation from Knowledge Graphs with Graph Transformers.Rik Koncel-Kedziorski, Dhanush Bekal, Yi Luan, Mirella Lapata, and Hannaneh Hajishirzi, [\[pdf\]](https://aclanthology.org/N19-1238.pdf)`ACL 1 Apr 2019`. 189 | 3. GGP: A Graph-based Grouping Planner for Explicit Control of Long Text Generation. _Xuming Lin, Shaobo Cui, Zhongzhou Zhao, Wei Zhou, Ji Zhang and Haiqing Chen_.[\[pdf\]](https://dl.acm.org/doi/10.1145/3459637.3482111)`ACM Oct 2021` 190 | 4. NGEP: A Graph-based Event Planning Framework for Story Generation._Chen Tang, Zhihao Zhang, Tyler Loakman, Chenghua Lin and Frank Guerin_.[\[pdf\]](https://aclanthology.org/2022.aacl-short.24.pdf)`ACL 2022` 191 | 192 | ### C.7 Active Learning 193 | 194 | 1. Active Learning for Natural Language Generation.Yotam Perlitz, Ariel Gera, Michal Shmueli-Scheuer, Dafna Sheinwald, Noam Slonim, Liat Ein-Dor, [\[pdf\]](https://arxiv.org/pdf/2305.15040.pdf)`Arxiv 17 Oct 2023`. 195 | 196 | ### C.8 Model Criticism 197 | 198 | 1. Model Criticism for Long-Form Text Generation.Yuntian Deng, Volodymyr Kuleshov, Alexander M. Rush, [\[pdf\]](https://aclanthology.org/2022.emnlp-main.815.pdf)`ACL 16 Oct 2022`. 199 | 200 | ### C.9 Planning 201 | 202 | 1. DYPLOC: Dynamic Planning of Content Using Mixed Language Models for Text Generation. _Xinyu Hua, Ashwin Sreevatsa and Lu Wang_. [\[pdf\]](https://aclanthology.org/2021.acl-long.501.pdf). `ACL 2021`. 203 | 2. Improving Text Generation via Neural Discourse Planning._Alexander Chernyavskiy_.[\[pdf\]](https://dl.acm.org/doi/10.1145/3488560.3502214)`ACM Feb 2022` 204 | 3. Knowledge-based Review Generation by Coherence Enhanced Text Planning._Junyi Li, Wayne Xin Zhao, Zhicheng Wei, Nicholas Jing Yuan and Ji-Rong Wen_.[\[pdf\]](https://dl.acm.org/doi/10.1145/3404835.3462865)`ACM July 2021` 205 | 206 | ### C.10 Text Embedding for Long Input Tokens 207 | 1. Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documents.Michael Günther, Jackmin Ong, Isabelle Mohr, Alaeddine Abdessalem, Tanguy Abel, 208 | Mohammad Kalim Akram, Susana Guzman, Georgios Mastrapas, Saba Sturua, Bo Wang, Maximilian Werk, Nan Wang, Han Xiao.[\[pdf\]](https://arxiv.org/pdf/2310.19923.pdf)`ArXiv 30 Oct 2023`. 209 | 210 | ### C.11 Diffusion 211 | 212 | 1. AR-DIFFUSION: Auto-Regressive Diffusion Model for Text Generation. _Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, Zhongyu Wei, Jian Guo, Nan Duan and Weizhu Chen_. [\[pdf\]](https://arxiv.org/pdf/2305.09515.pdf). `NeurIPS 2023`. 213 | 214 | ## D. Model Perspective 215 | 216 | ### D.1 Language Models 217 | 218 | 1. A Survey of Large Language Models. _Wayne Xin Zhao, Kun Zhou and Junyi Li_. [\[pdf\]](https://arxiv.org/abs/2303.18223). `arXiv 31 Mar, 2023`. 219 | 2. Long and Diverse Text Generation with Planning-based Hierarchical Variational Model.Zhihong Shao, Minlie Huang, Jiangtao Wen, Wenfei Xu, Xiaoyan Zhu, [\[pdf\]](https://arxiv.org/pdf/1908.06605.pdf)`Arxiv *25 Aug 2019*`. 220 | 3. Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence.Jian Guan, Xiaoxi Mao, Changjie Fan, Zitao Liu, Wenbiao Ding, and Minlie Huang, [\[pdf\]](https://arxiv.org/pdf/2105.08963.pdf)`Arxiv 19 May 2021`. 221 | 4. Coherent Long Text Generation by Contrastive Soft Prompt.Guandan Chen, Jiashu Pu, Yadong Xi, Rongsheng Zhang, [\[pdf\]](https://aclanthology.org/2022.gem-1.42.pdf)`ACL 7 Dec 2022`. 222 | 5. LONGLORA: EFFICIENT FINE-TUNING OF LONG CONTEXT LARGE LANGUAGE MODELS. _Yukang Chen, Shengju Qian, Zhijian Liu, Haotian Tang, Song Han and Jiaya Jia_.[\[pdf\]](https://browse.arxiv.org/pdf/2309.12307.pdf)`arXiv 5 Dec 2023` 223 | 224 | ### D.2 Pretrained Models 225 | 226 | 1. Progressive Generation of Long Text with Pretrained Language Models.Bowen Tan, Zichao Yang, Maruan Al-Shedivat, Eric P. Xing, Zhiting Hu, [\[pdf\]](https://aclanthology.org/2021.naacl-main.341.pdf)`NAACL 11 Jun 2021`. 227 | 2. A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation.Jian Guan, Fei Huang, Zhihao Zhao, Xiaoyan Zhu, Minlie Huang, [\[pdf\]](https://aclanthology.org/2020.tacl-1.7.pdf)`TACL 1 Jan 2020`. 228 | 3. DIALOGLM: Pre-trained Model for Long Dialogue Understanding and Summarization.Ming Zhong, Yang Liu, Yichong Xu, Chenguang Zhu, Michael Zeng, [\[pdf\]](https://arxiv.org/pdf/2109.02492.pdf)`Arxiv 6 Jan 2022`. 229 | 4. Adapting Pretrained Text-to-Text Models for Long Text Sequences._Wenhan Xiong, Anchit Gupta, Shubham Toshniwal, Yashar Mehdad and Wen-tau Yih_.[\[pdf\]](https://arxiv.org/pdf/2209.10052.pdf)`arXiv 2022` 230 | 231 | ### D.3 Combination of RNN and Transformer 232 | 233 | 1. RECURRENTGPT:Interactive Generation of (Arbitrarily) Long Text.Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui Tiannan Wang, Zhenxin Xiao, Yifan Hou, Ryan Cotterell, Mrinmaya Sachan, ETH Zürich, [\[pdf\]](https://arxiv.org/pdf/2305.13304.pdf)`Arxiv 22 May 2023` 234 | 235 | ### D.4 Transformer 236 | 237 | 1. Text Generation from Knowledge Graphs with Graph Transformers.Rik Koncel-Kedziorski, Dhanush Bekal, Yi Luan, Mirella Lapata, and Hannaneh Hajishirzi, [\[pdf\]](https://aclanthology.org/N19-1238.pdf)`ACL 1 Apr 2019`. 238 | 2. LongT5 Efficient Text-To-Text Transformer for Long Sequences.Mandy Guo, Joshua Ainslie, David Uthus, Santiago Ontañón, Jianmo Ni, Yun-Hsuan Sung, Yinfei Yang, [\[pdf\]](https://arxiv.org/pdf/2112.07916.pdf)`Arxiv 3 May 2022`. 239 | 3. DISCODVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer.Haozhe Ji, Minlie Huang, [\[pdf\]](https://aclanthology.org/2021.emnlp-main.347.pdf)`ACL 12 Oct 2021`. 240 | 4. Generating Long Sequences with Sparse Transformers._Rewon Child, Scott Gray, Alec Radford and Ilya Sutskever_.[\[pdf\]](https://arxiv.org/pdf/1904.10509.pdf)`arXiv 2019` 241 | 5. Segmented Recurrent Transformer: An Efficient Sequence-to-Sequence Model._Yinghan Long, Sayeed Shafayet Chowdhury and Kaushik Roy_.[\[pdf\]](https://arxiv.org/pdf/2305.16340.pdf)`arXiv 23 Oct 2023` 242 | 6. Fixed global memory for controllable long text generation._Zheng Chen and Zhejun Liu_.[\[pdf\]](https://link.springer.com/content/pdf/10.1007/s10489-022-04197-6.pdf)`Springer 20 Oct 2022` 243 | 244 | 245 | ### D.5 RNN (LSTM) 246 | 247 | 1. Research on Text Generation Based on LSTM.Lifen Li, Tianyu Zhang, [\[pdf\]](http://www.icj-e.org/download/ICJE-7-5-525-535.pdf)`International Core Journal of Engineering Volume 7 Issue 5, 2021`. 248 | 2. A method of automatic text summarisation based on long short-term memory.Wei Fang, TianXiao Jiang, Ke Jiang, Feihong Zhang, Yewen Ding and Jack Sheng, [\[pdf\]](https://www.inderscienceonline.com/doi/epdf/10.1504/IJCSE.2020.107243)`International Journal of Computational Science and Engineering 4 May 2020`. 249 | 250 | ### D.6 CNN 251 | 252 | 1. A Hybrid Convolutional Variational Autoencoder for Text Generation. _Stanislau Semeniuta, Aliaksei Severyn and Erhardt Barth_. [\[pdf\]](https://aclanthology.org/D17-1066.pdf). `EMNLP 2017`. 253 | 254 | ### D.7 Different semantic granularity 255 | 256 | 1. 面向不同语义粒度约束的文本生成方法研究._潘囿丞_[\[pdf\]](https://kns.cnki.net/kcms2/article/abstract?v=TzO8JwpG6uil4nMnfSwaMPP0HMSYaLhlcR7C8aJoic0emFvpbLrPABWYeSDOTLgIWOYT1YqQ3rHTl1bjeFqRcBEJ6Fhae9SHNtqmuT0F0iovkSTbqlwkMz0rAwIDi2VJUwTd_kE97TsN_Y9AI8va69eJG2H6YmMg&uniplatform=NZKPT&language=CHS)`CNKI July 2022` 257 | 258 | 259 | ## E. Reading Papers 260 | 1. Long Text Generation Challenge. _Nikolay Mikhaylovskiy_. [\[pdf\]](https://arxiv.org/pdf/2306.02334.pdf).`arXiv 4 June 2023`. 261 | 262 | ## F. Long Text Generation Evaluation Methods and Metrics 263 | 1. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. _Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer and Hannaneh Hajishirzi_. [\[pdf\]](https://arxiv.org/pdf/2305.14251.pdf). [\[code\]](https://github.com/shmsw25/FActScore). `EMNLP 2023`. 264 | 265 | ## G. Research Hotpot, Trend and Challenge of Long Text Generation 266 | 267 | ### G.1 The Combination of Agent and Long Text Generation 268 | 269 | ### G.2 Long Text Generation with LLMs (above 13B) 270 | 271 | ### G.2 Hallucination in Long Text Generation 272 | 273 | ### G.3 Model Lossless Compression for Long Text Generation 274 | i. Quantization 275 | ii. Distillation 276 | iii. Pruning 277 | 278 | ### G.4 Efficient Parameters Learning for Long Text Generation 279 | i. LoRA 280 | ii. Flash Attention 281 | iii. Sparse Attenation 282 | iv. KV Quantization & Storage 283 | 284 | ### G.5 Context Size Extension (Input Token Length) for Long Text Generation 285 | 286 | ### G.6 Reasoning in Long Text Generation 287 | 288 | ### G.7 Chain-of-Thought in Long Text Generation 289 | i. Reflect CoT 290 | ii. Tree-of-Thought 291 | iii. Graph-of-Thought 292 | 293 | ### G.8 Long Text Contraint Generation 294 | i. Multi-Styles Long Text Generation 295 | ii. Multi-Tasks Long Text Generation 296 | iii. Controllable Long Text Generation 297 | 298 | ### G.9 Interactive Long Text Generation 299 | i. Editable Long Text Generation 300 | ii. Chat to Generate Long Text 301 | 302 | ### G.10 Mutli-modal Text Generation 303 | i. Image to Long Text Generation 304 | ii. Video to Long Text Generation 305 | 306 | ### G.11 Align with Human for Long Text Generation 307 | i. RLHF 308 | ii RLAIF 309 | iii. RLMoEF 310 | 311 | ### G.12 RAG Enhanced Long Text Generation 312 | 313 | # Project Maintainers & Contributors 314 | * Junwen Zhang ([@Fendi](https://github.com/ai-agi)) 315 | * Yuanhao Lou ([@Yuanhao](https://github.com/zju22)) 316 | * Shuang Chen ([@Chen Shuang](https://csfufu.life)) 317 | 318 | # Contact 319 | * Junwen Zhang: junwenzhang@zju.edu.cn 320 | --------------------------------------------------------------------------------