├── .DS_Store ├── .gitignore ├── LICENSE ├── README.md └── assets ├── .DS_Store ├── etods.png ├── fully.png └── texts ├── camrest.csv ├── multiwoz20.csv ├── multiwoz21.csv ├── multiwoz21e2e.csv ├── paperlist.csv └── smd.csv /.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wbopan/Awesome-EToDs-Survey/667bf90c58e8a7d827c36e0edd4c685c0e459be7/.DS_Store -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | ignore/ -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 BMPixel 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Awesome-EToDs-Survey 2 | 3 | 🔥 **Collection of papers, benchmarks and newest trends in the domain of End-to-end ToDs** [[ArXiv]](https://arxiv.org/abs/2311.09008) 4 | 5 | 6 | 🌟 **Any contributions via PRs, issues, emails or other methods are greatly appreciated.** 7 | 8 | 🔮 **Interactive paperlist&benchmark website is also available on [etods.net](https://etods.net)** 9 | 10 | ## Intro: What is the End-to-end ToDs (EToDs)? 11 | 12 | In contrast to traditional pipelined task-oriented dialogue (ToDs) systems, EToDs are able to directly map user utterances to system actions without any intermediate representations. EToDs are usually trained in an end-to-end manner, which means that the model is trained to directly map user utterances to system actions without any intermediate representations. 13 | 14 | In this repo, we further categorize EToDs into three types, as shown in the following figure. 15 | 16 | - Modularly end-to-end task-oriented dialogue without pretraining models 17 | - Modularly end-to-end task-oriented dialogue with pretraining models 18 | - Fully end-to-end task-oriented dialogue 19 | 20 |
21 | 22 | ## Table of Content (ToC) 23 | 24 | ## 1. Modularly EToD 25 | 26 | Modularly EToDs are systems that generate responses using modularized components which are trained in an end-to-end manner, differing from fully EToDs by their non-differentiable API call knowledge base retrieval 27 | 28 | ### 1.1 Modularly EToD w/o PLM 29 | 30 | Modularly EToD without Pretrained Language Models (PLMs) primarily focuses on optimizing dialogue systems with either supervised learning or reinforcement learning techniques 31 | 32 | - [2020] **A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learning .** *Zhang et al EMNLP.* [[paper]](https://aclanthology.org/2020.emnlp-main.740/) [[code]](https://github.com/thu-spmi/LABES) 33 | - [2020] **Attention over Parameters for Dialogue Systems .** *Madotto et al NeurIPS.* [[paper]](https://arxiv.org/abs/2001.01871) 34 | - [2020] **LAVA: Latent Action Spaces via Variational Auto-encoding for Dialogue Policy Optimization .** *Lubis et al COLING.* [[paper]](https://aclanthology.org/2020.coling-main.41/) 35 | - [2020] **SUMBT+LaRL: Effective Multi-Domain End-to-End Neural Task-Oriented Dialog System .** *Lee et al IEEE.* [[paper]](https://arxiv.org/abs/2009.10447) 36 | - [2020] **UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues .** *Le et al EMNLP.* [[paper]](https://virtual.2020.emnlp.org/paper_main.1012.html) [[code]](https://github.com/henryhungle/UniConv) 37 | - [2019] **A Modular Task-oriented Dialogue System Using a Neural Mixture-of-Experts .** *Pei et al WCIS.* [[paper]](https://arxiv.org/abs/1907.05346) 38 | - [2019] **Flexibly-Structured Model for Task-Oriented Dialogues.** *Shu et al SIGDIAL.* [[paper]](https://aclanthology.org/W19-5922/) [[code]](https://github.com/uber-research/FSDM) 39 | - [2019] **Incremental Learning from Scratch for Task-Oriented Dialogue Systems.** *Wang et al ACL.* [[paper]](https://aclanthology.org/P19-1361.pdf) [[code]](https://github.com/Leechikara/Incremental-Dialogue-System) 40 | - [2019] **Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use.** *Rajendran et al TACL.* [[paper]](https://aclanthology.org/Q19-1024.pdf) [[code]](https://github.com/IBM/modified-bAbI-dialog-tasks) 41 | - [2019] **MOSS: End-to-End Dialog System Framework with Modular Supervision .** *Liang et al AAAI.* [[paper]](https://ojs.aaai.org/index.php/AAAI/article/view/6349/6205) 42 | - [2019] **Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Models .** *Zhao et al NAACL.* [[paper]](https://aclanthology.org/N19-1123.pdf) [[code]](https://github.com/snakeztc/NeuralDialog-LaRL) 43 | - [2019] **Structured Fusion Networks for Dialog.** *Mehri et al SIGDIAL.* [[paper]](https://aclanthology.org/W19-5921) 44 | - [2019] **Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context.** *Zhang et al AAAI.* [[paper]](https://arxiv.org/abs/1911.10484) [[code]](https://github.com/thu-spmi/damd-multiwoz) 45 | - [2018] **Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems .** *Liu et al NAACL.* [[paper]](https://aclanthology.org/N18-1187/) 46 | - [2018] **End-to-End Learning of Task-Oriented Dialogs .** *Liu and Lane NAACL.* [[paper]](https://aclanthology.org/N18-4010/) 47 | - [2018] **Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures.** *Lei et al ACL.* [[paper]](https://aclanthology.org/P18-1133/) [[code]](https://github.com/WING-NUS/sequicity) 48 | - [2017] **A Copy-Augmented Sequence-to-Sequence Architecture Gives Good Performance on Task-Oriented Dialogue .** *Eric and Manning EACL.* [[paper]](https://aclanthology.org/E17-2075/) 49 | - [2017] **An End-to-End Trainable Neural Network Model with Belief Tracking for Task-Oriented Dialog .** *Liu and Lane InterSpeech.* [[paper]](https://www.isca-speech.org/archive_v0/Interspeech_2017/pdfs/1326.PDF) 50 | - [2017] **End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learning.** *Liu et al Arxiv.* [[paper]](https://arxiv.org/abs/1711.10712) 51 | - [2017] **End-to-End Task-Completion Neural Dialogue Systems .** *Li et al IJCNLP.* [[paper]](https://aclanthology.org/I17-1074.pdf) [[code]](https://github.com/MiuLab/TC-Bot) 52 | - [2017] **Generative Encoder-Decoder Models for Task-Oriented Spoken Dialog Systems with Chatting Capability.** *Zhao et al SIGDIAL.* [[paper]](https://aclanthology.org/W17-5505/) 53 | - [2017] **Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning .** *Williams et al ACL.* [[paper]](https://aclanthology.org/P17-1062/) 54 | - [2016] **A network- based end-to-end trainable task-oriented dialogue system.** *Wen et al EACL.* [[paper]](https://aclanthology.org/E17-1042.pdf) [[code]](NDM) 55 | - [2016] **Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning.** *Zhao and Eskenazi SIGDIAL.* [[paper]](https://aclanthology.org/W16-3601/) [[code]](https://github.com/snakeztc/NeuralDialog-DM) 56 | 57 | ### 1.2 Modularly EToD w/ PLM 58 | 59 | Modularly EToD with PLM incorporates Pretrained Language Models using either decoder-only PLMs like GPT-2, which takes dialogue context, belief state, and database state as input to generate system responses, or encoder-decoder PLMs​ 60 | 61 | - [2023] **A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding.** *Pan et al ArXiv.* [[paper]](https://api.semanticscholar.org/CorpusID:258049061) 62 | - [2023] **ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?.** *Heck et al ArXiv.* [[paper]](https://api.semanticscholar.org/CorpusID:259063822) 63 | - [2023] **Are Large Language Models All You Need for Task-Oriented Dialogue?.** *Hudevcek and Dusek SIGDIAL.* [[paper]](https://api.semanticscholar.org/CorpusID:258108409) 64 | - [2022] **Autoregressive Entity Generation for End-to-End Task-Oriented Dialog.** *Huang et al COLING.* [[paper]](https://aclanthology.org/2022.coling-1.25.pdf) 65 | - [2022] **BORT: Back and Denoising Reconstruction for End-to-End Task-Oriented Dialog.** *Sun et al NAACL.* [[paper]](https://aclanthology.org/2022.findings-naacl.166.pdf) [[code]](https://github.com/jd-ai-research-nlp/bort) 66 | - [2022] **SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation.** *He et al SIGIR.* [[paper]](https://arxiv.org/pdf/2209.06664) [[code]](https://github.com/alibabaresearch/damo-convai) 67 | - [2022] **Task-Oriented Dialogue System as Natural Language Generation .** *Wang et al SIGIR.* [[paper]](https://arxiv.org/pdf/2108.13679) [[code]](https://github.com/victorwz/tod_as_nlg) 68 | - [2022] **Q-TOD: A Query-driven Task-oriented Dialogue System.** *Tian et al EMNLP.* [[paper]](https://arxiv.org/abs/2210.07564) [[code]](https://github.com/PaddlePaddle/Knover/tree/develop/projects/Q-TOD) 69 | - [2021] **[CASPI] Causal-aware Safe Policy Improvement for Task-oriented Dialogue.** *Ramachandran et al ACL.* [[paper]](https://aclanthology.org/2022.acl-long.8.pdf) [[code]](https://github.com/salesforce/CASPI) 70 | - [2021] **AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Models .** *Kulha_nek et al EMNLP.* [[paper]](https://aclanthology.org/2021.nlp4convai-1.19.pdf) [[code]](https://github.com/ufal/augpt) 71 | - [2021] **GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection .** *He et al AAAI.* [[paper]](https://ojs.aaai.org/index.php/AAAI/article/download/21320/version/19607/21069) [[code]](https://github.com/siat-nlp/GALAXY) 72 | - [2021] **Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System .** *Su et al AAAI.* [[paper]](https://aclanthology.org/2022.acl-long.319.pdf) [[code]](https://github.com/awslabs/pptod) 73 | - [2021] **Improving End-to-End Task-Oriented Dialog System with A Simple Auxiliary Task.** *Lee Findings.* [[paper]](https://aclanthology.org/2021.findings-emnlp.112) 74 | - [2021] **A Co-Interactive Transformer for Joint Slot Filling and Intent Detection.** *Qin et al ICASSP.* [[paper]](https://doi.org/10.1109/ICASSP39728.2021.9414110) [[code]](https://github.com/kangbrilliant/DCA-Net) 75 | - [2020] **A Simple Language Model for Task-Oriented Dialogue.** *Hosseini-Asl et al NeurIPS.* [[paper]](https://proceedings.neurips.cc/paper/2020/file/e946209592563be0f01c844ab2170f0c-Paper.pdf) [[code]](https://github.com/salesforce/simpletod) 76 | - [2020] **End-to-End Neural Pipeline for Goal-Oriented Dialogue Systems using GPT-2.** *Ham et al ACL.* [[paper]](https://aclanthology.org/2020.acl-main.54.pdf) 77 | - [2020] **End-to-End Trainable Non-Collaborative Dialog System .** *Li et al AAAI.* [[paper]](https://ojs.aaai.org/index.php/AAAI/article/view/6345/6201) [[code]](https://gitlab.com/ucdavisnlp/antiscam) 78 | - [2020] **MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems .** *Lin et al EMNLP.* [[paper]](https://aclanthology.org/2020.emnlp-main.273.pdf) [[code]](https://github.com/zlinao/MinTL) 79 | - [2020] **Soloist : BuildingTask Bots at Scale with Transfer Learning and Machine Teaching.** *Peng et al TACL.* [[paper]](https://aclanthology.org/2021.tacl-1.49.pdf) [[code]](https://aka.ms/soloist. ) 80 | - [2020] **UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2.** *Yang et al AAAI.* [[paper]](https://arxiv.org/pdf/2012.03539.pdf) [[code]](https://github.com/TonyNemo/UBAR-MultiWOZ) 81 | - [2020] **AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling.** *Qin et al EMNLP.* [[paper]](https://aclanthology.org/2020.findings-emnlp.163) [[code]](https://github.com/LooperXX/AGIF) 82 | - [2019] **Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models .** *Wu et al EACL.* [[paper]](https://aclanthology.org/2021.eacl-main.110.pdf) [[code]](https://github.com/qywu/ARDM ) 83 | - [2019] **Hello, It's GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems.** *Budzianowski et al ACL.* [[paper]](https://aclanthology.org/D19-5602.pdf) 84 | 85 | ## 2. Fully EToD 86 | 87 | Fully EToD systems incorporate knowledge bases directly, using neural networks to query the knowledge base in a differentiable manner, which allows for end-to-end training without the need for intermediate modular annotations 88 | 89 |
90 | 91 | ### 2.1 Entity Triplet Representation 92 | 93 | Entity Triplet Representation in Fully EToD stores knowledge base entities in a triplet format (subject, relation, object), which is summed up through word embeddings, offering a widely used method for representing knowledge base entities 94 | 95 | - [2021] **Intention Reasoning Network for Multi-Domain End-to-end Task-Oriented Dialogue.** *Ma et al EMNLP.* [[paper]](https://aclanthology.org/2021.emnlp-main.174) 96 | - [2020] **Dual Dynamic Memory Network for End-to-End Multi-turn Task-oriented Dialog Systems .** *Wang et al COLING.* [[paper]](https://aclanthology.org/2020.coling-main.362.pdf) [[code]](https://github.com/siat-nlp/ddmn) 97 | - [2020] **Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog.** *Qin et al ACL.* [[paper]](https://aclanthology.org/2020.acl-main.565.pdf) [[code]](https://github.com/Looper--/DF-Net) 98 | - [2019] **Disentangling Language and Knowledge in Task-Oriented Dialogs.** *Raghu et al NAACL.* [[paper]](https://aclanthology.org/N19-1126/) [[code]](https://github.com/dair-iitd/BossNet) 99 | - [2019] **Global-to-local Memory Pointer Networks for Task-Oriented Dialogue.** *Wu et al ICLR.* [[paper]](https://ar-iv.org/abs/1901.04713) [[code]](https://github.com/jasonwu0731/GLMP) 100 | - [2019] **A Working Memory Model for Task-oriented Dialog Response Generation.** *Chen tal ACL.* [[paper]](https://aclanthology.org/P19-1258) 101 | - [2018] **Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems .** *Madotto et al ACL.* [[paper]](https://aclanthology.org/P18-1136/) [[code]](https://github.com/HLTCHKUST/Mem2Seq) 102 | 103 | ### 2.2 Row-level Representation 104 | 105 | Row-level Representation in Fully EToD addresses the limitations of triplet representation by considering the relationships across entities within the same row of a knowledge base, allowing for a more nuanced retrieval of relevant KB rows and columns 106 | 107 | - [2022] **A Hierarchical Memory Model for Task-Oriented Dialogue System.** *Zeng et al IEICE.* [[paper]](https://www.jstage.jst.go.jp/article/transinf/E105.D/8/E105.D_2022EDP7001/_article) [[code]](https://github.com/zengyazy/HM2Seq) 108 | - [2021] **Constraint based Knowledge Base Distillation in End-to-End Task Oriented Dialogs.** *Raghu et al IJCNLP.* [[paper]](https://aclanthology.org/2021.findings-acl.448) [[code]](https://github.com/dair-iitd/CDNet) 109 | - [2019] **Entity-Consistent End-to-end Task-Oriented Dialogue System with KB Retriever .** *Qin et al EMNLP.* [[paper]](https://aclanthology.org/D19-1013.pdf) [[code]](https://github.com/yizhen20133868/Retriever-Dialogue) 110 | - [2018] **Multi-Level Memory for Task Oriented Dialogs .** *Reddy et al NAACL.* [[paper]](https://aclanthology.org/N19-1375/) [[code]](https://github.com/DineshRaghu/multi-level-memory-network) 111 | - [2018] **Sequence-to-Sequence Learning for Task-oriented Dialogue with Dialogue State Representation .** *Wen et al COLING.* [[paper]](http://ir.hit.edu.cn/~car/papers/coling18-hywen.pdf) 112 | - [2017] **Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access .** *Dhingra et al ACL.* [[paper]](https://ar-iv.org/abs/1609.00777) [[code]](https://github.com/MiuLab/KB-InfoBot) 113 | 114 | ### 2.3 Graph Representation 115 | 116 | Graph Representation in Fully EToD aims to enhance the conte-tualization of entity embeddings within a knowledge base by densely linking entities to related slot titles in the dialogue history, utilizing graph-based reasoning or attention mechanisms for a more integrated understanding 117 | 118 | - [2023] **Multi-Grained Knowledge Retrieval for End-to-End Task-Oriented Dialog.** *Wan et al Ar-iv.* [[paper]](https://ar-iv.org/pdf/2305.10149.pdf) [[code]](https://github.com/18907305772/MAKER) 119 | - [2022] **DialoKG: Knowledge-Structure Aware Task-Oriented Dialogue Generation.** *Rony et al NAACL.* [[paper]](https://aclanthology.org/2022.findings-naacl.195.pdf) [[code]](https://github.com/rashad101/DialoKG ) 120 | - [2022] **GraphMemDialog: Optimizing End-to-End Task-Oriented Dialog Systems Using Graph Memory Networks.** *Wu et al AAAI.* [[paper]](https://ojs.aaai.org/inde-.php/AAAI/article/view/21403/21152) 121 | - [2020] **Conte-tualize Knowledge Bases with Transformer for End-to-end Task-Oriented Dialogue Systems .** *Gou et al EMNLP.* [[paper]](https://aclanthology.org/2021.emnlp-main.353.pdf) 122 | - [2020] **FG2SEQ: EFFECTIVELY ENCODING KNOWLEDGE FOR END-TO-END TASK-ORIENTED DIALOG .** *He et al ICASSP .* [[paper]](https://ieee-plore.ieee.org/document/9053667) [[code]](https://github.com/scoyer/FG2Seq) 123 | - [2020] **GraphDialog: Integrating Graph Knowledge into End-to-End Task-Oriented Dialogue Systems .** *Yang et al EMNLP.* [[paper]](https://aclanthology.org/2020.emnlp-main.147.pdf) [[code]](https://github.com/shiquanyang/GraphDialog) 124 | 125 | ## Benchmark 126 | 127 | ## Modularly EToD on MultiWOZ2.0 and MultiWOZ2.1 128 | 129 | ### MultiWOZ2.0 130 | 131 | | Model | Year | Type | Inform | Success | BLEU | Combined | 132 | |----------------|------|------------|--------|---------|------|----------| 133 | | SFN+RL | 2019 | Modularly | 73.8 | 58.6 | 16.9 | 83 | 134 | | DAMD | 2019 | Modularly | 76.3 | 60.4 | 16.6 | 85 | 135 | | LAVA | 2020 | Modularly | 91.8 | 81.8 | 12 | 98.8 | 136 | | SimpleToD | 2020 | Pretrain | 84.4 | 70.1 | 15 | 92.3 | 137 | | MinTL-BART | 2020 | Pretrain | 84.9 | 74.9 | 17.9 | 97.8 | 138 | | UBAR | 2020 | Pretrain | 95.4 | 80.7 | 17 | 105.1 | 139 | | AuGPT | 2021 | Pretrain | 83.1 | 70.1 | 17.2 | 93.8 | 140 | | SOLOIST | 2021 | Pretrain | 85.5 | 72.9 | 16.5 | 95.7 | 141 | | MTToD | 2021 | Pretrain | 91 | 82.6 | 21.6 | 108.3 | 142 | | PPTOD | 2021 | Pretrain | 89.2 | 79.4 | 18.6 | 102.9 | 143 | | SimpleToD-ACN | 2022 | Pretrain | 85.8 | 72.1 | 15.5 | 94.5 | 144 | | BORT | 2022 | Pretrain | 93.8 | 85.8 | 18.5 | 108.3 | 145 | | GALA-Y | 2022 | Pretrain | 94.4 | 85.3 | 20 | 110.3 | 146 | 147 | ### MultiWOZ2.1 148 | 149 | | Model | Time | Type | Inform | Success | BLEU | Combined | 150 | |----------------|------|------------|--------|---------|------|----------| 151 | | MD-Sequicity | 2018 | Modularly | 66.4 | 45.3 | 15.5 | 71.4 | 152 | | SFN+RL | 2019 | Modularly | 73.8 | 58.6 | 16.9 | 83 | 153 | | DAMD | 2019 | Modularly | 76.4 | 60.4 | 16.6 | 85 | 154 | | UniConv | 2020 | Modularly | 72.6 | 62.9 | 19.8 | 87.6 | 155 | | LABES-S2S | 2020 | Modularly | 78.1 | 67.1 | 18.1 | 90.7 | 156 | | SimpleToD | 2020 | Pretrain | 85 | 70.5 | 15.2 | 93 | 157 | | UBAR | 2020 | Pretrain | 95.7 | 81.8 | 16.5 | 105.3 | 158 | | AuGPT | 2021 | Pretrain | 83.5 | 67.3 | 17.2 | 92.6 | 159 | | SOLOIST | 2021 | Pretrain | 85.5 | 72.9 | 16.5 | 95.7 | 160 | | MTToD | 2022 | Pretrain | 91 | 82.1 | 21 | 107.5 | 161 | | PPTOD | 2021 | Pretrain | 87.1 | 79.1 | 19.2 | 102.3 | 162 | | GALA-Y | 2022 | Pretrain | 95.3 | 86.2 | 20 | 110.8 | 163 | | SPACE3 | 2022 | Pretrain | 95.3 | 88 | 19.3 | 111 | 164 | | BORT | 2022 | Pretrain | 93.8 | 85.8 | 18.5 | 108.3 | 165 | 166 | ## Modularly EToD on CamRest6762 167 | 168 | | Model | Time | Match | Success | BLEU | Combined | 169 | |----------------|------|-------|---------|------|----------| 170 | | NDM | 2016 | 90.4 | 83.2 | 21.2 | 108 | 171 | | MD-Sequicity | 2018 | 92.7 | 85.4 | 25.3 | 114.35 | 172 | | FSDM | 2019 | 93.5 | 86.2 | 25.8 | 115.65 | 173 | | MOSS | 2019 | 95.1 | 86 | 25.9 | 116.45 | 174 | | LABES-S2S | 2020 | 96.4 | 82.3 | 25.6 | 114.95 | 175 | | ARDM | 2021 | - | 86.2 | 25.4 | - | 176 | | SOLOIST | 2020 | - | 87.1 | 25.5 | - | 177 | | BORT | 2022 | - | 89.7 | 25.9 | - | 178 | | SPACE3 | 2022 | 97.7 | 88.2 | 23.7 | 116.67 | 179 | 180 | ## Fully EToD on SMD 181 | 182 | | Model | Time | KB Type | BLEU | Ent_F1 | Sch_F1 | Wea_F1 | Nav_F1 | 183 | |----------------|------|--------|------|--------|--------|--------|--------| 184 | | KVRet | 2017 | Triplet| 13.2 | 48 | 62.9 | 53.3 | 44.5 | 185 | | Mem2Seq | 2018 | Triplet| 12.6 | 33.4 | 49.3 | 32.8 | 20 | 186 | | GLMP | 2019 | Triplet| 14.8 | 60 | 69.6 | 62.6 | 53 | 187 | | BossNet | 2019 | Triplet| 8.3 | 35.9 | 50.2 | 34.5 | 21.6 | 188 | | KB-Transformer | 2019 | Triplet| 13.9 | 37.1 | 51.2 | 48.2 | 23.3 | 189 | | DDMN | 2020 | Triplet| 17.7 | 55.6 | 65 | 58.7 | 47.2 | 190 | | DFNet | 2020 | Triplet| 14.4 | 62.7 | 73.1 | 57.6 | 57.9 | 191 | | DSR | 2018 | Row | 12.7 | 51.9 | 52.1 | 50.4 | 52 | 192 | | MLM | 2018 | Row | 15.6 | 55.5 | 67.4 | 54.8 | 45.1 | 193 | | KB-retriever | 2019 | Row | 13.9 | 53.7 | 55.6 | 52.2 | 54.5 | 194 | | HM2Seq | 2022 | Row | 14.6 | 63.1 | 73.9 | 64.4 | 56.2 | 195 | | Fg2Seq | 2020 | Graph | 16.8 | 61.1 | 73.3 | 57.4 | 56.1 | 196 | | GraphDialog | 2020 | Graph | 13.7 | 60.7 | 72.8 | 55.2 | 54.2 | 197 | | GraphMemDialog | 2021 | Graph | 18.8 | 64.5 | 75.9 | 62.3 | 56.3 | 198 | | GPT2KE | 2021 | Graph | 17.4 | 59.8 | 72.6 | 57.7 | 53.5 | 199 | | COMET | 2021 | Graph | 17.3 | 63.6 | 77.6 | 58.3 | 56 | 200 | | DialoKG | 2022 | Graph | 20 | 65.9 | - | - | - | 201 | | MAKER | 2023 | Graph | 25.91| 71.30 | - | - | - | 202 | 203 | ## Fully EToD on MultiWOZ2.1 204 | 205 | | Model | Time | KB Type | BLEU | Ent_F1 | Res_F1 | Att_F1 | Hot_F1 | 206 | |----------------|------|--------|------|--------|--------|--------|--------| 207 | | Mem2Seq | 2018 | Triplet| 6.6 | 21.6 | 22.4 | 22 | 21 | 208 | | MLM | 2018 | Row | 9.2 | 27.8 | 29.8 | 27.4 | 25.2 | 209 | | DSR | 2018 | Row | 9.1 | 30 | 33.4 | 28 | 27.1 | 210 | | BossNet | 2019 | Triplet| 5.7 | 25.3 | 26.2 | 24.8 | 23.4 | 211 | | GLMP | 2019 | Triplet| 6.9 | 32.4 | 38.4 | 24.4 | 28.1 | 212 | | DDMN | 2020 | Triplet| 12.4 | 31.4 | 30.6 | 32.9 | 30.6 | 213 | | DFNet | 2020 | Triplet| 9.4 | 35.1 | 40.9 | 28.1 | 30.6 | 214 | | Fg2Seq | 2020 | Graph | 13.5 | 36 | 40.4 | 41.7 | 30.9 | 215 | | GraphMemDialog | 2021 | Graph | 14.9 | 40.2 | 42.8 | 48.8 | 36.4 | 216 | | MAKER | 2023 | Graph | 18.77| 54.72 | - | - | - | 217 | 218 | ## Citation 219 | If you find this repository useful, please cite our paper: 220 | 221 | ``` 222 | @misc{qin2023endtoend, 223 | title={End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions}, 224 | author={Libo Qin and Wenbo Pan and Qiguang Chen and Lizi Liao and Zhou Yu and Yue Zhang and Wanxiang Che and Min Li}, 225 | year={2023}, 226 | eprint={2311.09008}, 227 | archivePrefix={arXiv}, 228 | primaryClass={cs.CL} 229 | } 230 | ``` 231 | 232 | ## Project Maintainers & Contributors 233 | 234 | Wenbo Pan (pixelwenbo@gmail.com), Qiguang Chen (qgchen@ir.hit.edu.cn) and Libo Qin (lbqin@csu.edu.cn) 235 | 236 | 237 | 240 | 241 | -------------------------------------------------------------------------------- /assets/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wbopan/Awesome-EToDs-Survey/667bf90c58e8a7d827c36e0edd4c685c0e459be7/assets/.DS_Store -------------------------------------------------------------------------------- /assets/etods.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wbopan/Awesome-EToDs-Survey/667bf90c58e8a7d827c36e0edd4c685c0e459be7/assets/etods.png -------------------------------------------------------------------------------- /assets/fully.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wbopan/Awesome-EToDs-Survey/667bf90c58e8a7d827c36e0edd4c685c0e459be7/assets/fully.png -------------------------------------------------------------------------------- /assets/texts/camrest.csv: -------------------------------------------------------------------------------- 1 | Model,Time,Match,Success,BLEU,Combined 2 | NDM,2016,90.4,83.2,21.2,108 3 | MD-Sequicity,2018,92.7,85.4,25.3,114.35 4 | FSDM,2019,93.5,86.2,25.8,115.65 5 | MOSS,2019,95.1,86,25.9,116.45 6 | LABES-S2S,2020,96.4,82.3,25.6,114.95 7 | ARDM,2021,-,86.2,25.4,- 8 | SOLOIST,2020,-,87.1,25.5,- 9 | BORT,2022,-,89.7,25.9,- 10 | SPACE3,2022,97.7,88.2,23.7,116.67 -------------------------------------------------------------------------------- /assets/texts/multiwoz20.csv: -------------------------------------------------------------------------------- 1 | Model,Time,Type,Link,Inform,Success,BLEU,Combined 2 | SFN+RL,2019,Modularly,,73.8,58.6,16.9,83 3 | DAMD,2019,Modularly,,76.3,60.4,16.6,85 4 | LAVA,2020,Modularly,,91.8,81.8,12,98.8 5 | SimpleToD,2020,Pretrain,,84.4,70.1,15,92.3 6 | MinTL-BART,2020,Pretrain,,84.9,74.9,17.9,97.8 7 | UBAR,2020,Pretrain,,95.4,80.7,17,105.1 8 | AuGPT,2021,Pretrain,,83.1,70.1,17.2,93.8 9 | SOLOIST,2021,Pretrain,,85.5,72.9,16.5,95.7 10 | MTToD,2021,Pretrain,,91,82.6,21.6,108.3 11 | PPTOD,2021,Pretrain,,89.2,79.4,18.6,102.9 12 | SimpleToD-ACN,2022,Pretrain,,85.8,72.1,15.5,94.5 13 | BORT,2022,Pretrain,,93.8,85.8,18.5,108.3 14 | GALAXY,2022,Pretrain,,94.4,85.3,20.5,110.4 15 | SPACE3,2022,Pretrain,,95.3,88,19.3,111 -------------------------------------------------------------------------------- /assets/texts/multiwoz21.csv: -------------------------------------------------------------------------------- 1 | Model,Time,Type,Paperlink,Codelink,Inform,Success,BLEU,Combined 2 | MD-Sequicity,2018,Modularly,,,66.4,45.3,15.5,71.4 3 | SFN+RL,2019,Modularly,,,73.8,58.6,16.9,83 4 | DAMD,2019,Modularly,,,76.4,60.4,16.6,85 5 | UniConv,2020,Modularly,,,72.6,62.9,19.8,87.6 6 | LABES-S2S,2020,Modularly,,,78.1,67.1,18.1,90.7 7 | SimpleToD,2020,Pretrain,,,85,70.5,15.2,93 8 | UBAR,2020,Pretrain,,,95.7,81.8,16.5,105.3 9 | AuGPT,2021,Pretrain,,,83.5,67.3,17.2,92.6 10 | SOLOIST,2021,Pretrain,,,85.5,72.9,16.5,95.7 11 | MTToD,2022,Pretrain,,,91,82.1,21,107.5 12 | PPTOD,2021,Pretrain,,,87.1,79.1,19.2,102.3 13 | GALAXY,2022,Pretrain,,,95.3,86.2,20,110.8 14 | SPACE3,2022,Pretrain,He et al., 2022a,95.3,88.0,19.3,111.0 15 | BORT,2022,Pretrain,Sun et al., 2022,93.8,85.8,18.5,108.3 16 | -------------------------------------------------------------------------------- /assets/texts/multiwoz21e2e.csv: -------------------------------------------------------------------------------- 1 | Model,Time,Type,Paperlink,CodeLink,BLEU,Ent_F1,Res_F1,Att_F1,Hot_F1 2 | Mem2Seq,2018,Triplet,,,6.6,21.6,22.4,22,21 3 | MLM,2018,Row,,,9.2,27.8,29.8,27.4,25.2 4 | DSR,2018,Row,,,9.1,30,33.4,28,27.1 5 | BossNet,2019,Triplet,,,5.7,25.3,26.2,24.8,23.4 6 | GLMP,2019,Triplet,,,6.9,32.4,38.4,24.4,28.1 7 | DDMN,2020,Triplet,,,12.4,31.4,30.6,32.9,30.6 8 | DFNet,2020,Triplet,,,9.4,35.1,40.9,28.1,30.6 9 | Fg2Seq,2020,Graph,,,13.5,36,40.4,41.7,30.9 10 | GraphMemDialog,2021,Graph,,,14.9,40.2,42.8,48.8,36.4 11 | MAKER,2023,Graph,,,18.77,54.72,x,x,x -------------------------------------------------------------------------------- /assets/texts/paperlist.csv: -------------------------------------------------------------------------------- 1 | Title,author,paperlink,codelink,nickname,conference,type,time 2 | Multi-Grained Knowledge Retrieval for End-to-End Task-Oriented Dialog,Wan et al,https://arxiv.org/pdf/2305.10149.pdf,https://github.com/18907305772/MAKER,MAKER,Arxiv,fully,2023 3 | Towards Complex Scenarios: Building End-to-End Task-Oriented Dialogue System across Multiple Knowledge Bases,Libo Qin et al,https://doi.org/10.1609/aaai.v37i11.26581,https://github.com/RaleLee/KoK-HAN,KoK-HAN,AAAI,fully,2023 4 | DialoKG: Knowledge-Structure Aware Task-Oriented Dialogue Generation,Rony et al,https://aclanthology.org/2022.findings-naacl.195.pdf,https://github.com/rashad101/DialoKG ,DialoKG,NAACL,fully,2022 5 | GraphMemDialog: Optimizing End-to-End Task-Oriented Dialog Systems Using Graph Memory Networks,Wu et al,https://ojs.aaai.org/index.php/AAAI/article/view/21403/21152,,GraphMemDialog,AAAI,fully,2022 6 | A Hierarchical Memory Model for Task-Oriented Dialogue System,Zeng et al,https://www.jstage.jst.go.jp/article/transinf/E105.D/8/E105.D_2022EDP7001/_article,https://github.com/zengyazy/HM2Seq,HM2Seq,IEICE,fully,2022 7 | UniDU: Towards A Unified Generative Dialogue Understanding Framework,Zhi Chen et al,https://aclanthology.org/2022.sigdial-1.43,,UniDU,SIGDIAL,fully,2022 8 | Intention Reasoning Network for Multi-Domain End-to-end Task-Oriented Dialogue,Ma et al,https://aclanthology.org/2021.emnlp-main.174.pdf,,IR-Net,EMNLP,fully,2021 9 | UniMF: A Unified Framework to Incorporate Multimodal Knowledge Bases into End-to-End Task-Dialogue Systems ,Yang et al,https://www.ijcai.org/proceedings/2021/0548.pdf,https://github.com/ruizhang-ai/MMDialKB,UniMF,IJCAI,fully,2021 10 | Intention Reasoning Network for Multi-Domain End-to-end Task-Oriented Dialogue,Ma et al,https://aclanthology.org/2021.emnlp-main.174,,IR-Net,EMNLP,fully,2021 11 | Constraint based Knowledge Base Distillation in End-to-End Task Oriented Dialogs,Raghu et al,https://aclanthology.org/2021.findings-acl.448,https://github.com/dair-iitd/CDNet,CDNet,IJCNLP,fully,2021 12 | Exploring Auxiliary Reasoning Tasks for Task-oriented Dialog Systems with Meta Cooperative Learning,Qin et al,https://ojs.aaai.org/index.php/AAAI/article/view/17615,https://github.com/siat-nlp/MCL,MCL,AAAI,fully,2021 13 | Contextualize Knowledge Bases with Transformer for End-to-end Task-Oriented Dialogue Systems ,Gou et al,https://aclanthology.org/2021.emnlp-main.353.pdf,,COMET,EMNLP,fully,2020 14 | Dual Dynamic Memory Network for End-to-End Multi-turn Task-oriented Dialog Systems ,Wang et al,https://aclanthology.org/2020.coling-main.362.pdf,https://github.com/siat-nlp/ddmn,DDMN,COLING,fully,2020 15 | Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog,Qin et al,https://aclanthology.org/2020.acl-main.565.pdf,https://github.com/LooperXX/DF-Net,DFNet,ACL,fully,2020 16 | FG2SEQ: EFFECTIVELY ENCODING KNOWLEDGE FOR END-TO-END TASK-ORIENTED DIALOG ,He et al,https://ieeexplore.ieee.org/document/9053667,https://github.com/scoyer/FG2Seq,FG2Seq,ICASSP ,fully,2020 17 | GraphDialog: Integrating Graph Knowledge into End-to-End Task-Oriented Dialogue Systems ,Yang et al,https://aclanthology.org/2020.emnlp-main.147.pdf,https://github.com/shiquanyang/GraphDialog,GraphDialog,EMNLP,fully,2020 18 | Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems,Madotto et al,https://aclanthology.org/2020.findings-emnlp.215.pdf,https://github.com/HLTCHKUST/ke-dialogue,GPT2KE,EMNLP,fully,2020 19 | Learning Low-Resource End-To-End Goal-Oriented Dialog for Fast and Reliable System Deployment,Dai et al,https://aclanthology.org/2020.acl-main.57.pdf,,MDS,ACL,fully,2020 20 | End-to-End Task-Oriented Dialog System Through Template Slot Value Generation,Hong et al,https://www.isca-speech.org/archive/interspeech_2020/hong20b_interspeech.html,,-,InterSpeech,fully,2020 21 | Amalgamating Knowledge from Two Teachers for Task-oriented Dialogue System with Adversarial Training,He et al,https://aclanthology.org/2020.emnlp-main.281,https://github.com/siat-nlp/TTOS,TTOS,EMNLP,fully,2020 22 | Disentangling Language and Knowledge in Task-Oriented Dialogs,Raghu et al,https://aclanthology.org/N19-1126/,https://github.com/dair-iitd/BossNet,BossNet,NAACL,fully,2019 23 | End-to-End latent-variable task-oriented dialogue system with exact log-likelihood optimization ,Xu et al,https://dl.acm.org/doi/abs/10.1007/s11280-019-00688-8,,GPT-Adapter-CopyNet ,WWW,fully,2019 24 | Entity-Consistent End-to-end Task-Oriented Dialogue System with KB Retriever ,Qin et al,https://aclanthology.org/D19-1013.pdf,https://github.com/yizhen20133868/Retriever-Dialogue,KB-Retriever,EMNLP,fully,2019 25 | GECOR: An End-to-End Generative Ellipsis and Co-reference Resolution Model for Task-Oriented Dialogue,Quan et al,https://aclanthology.org/D19-1462.pdf,,GECOR,EMNLP,fully,2019 26 | Global-to-local Memory Pointer Networks for Task-Oriented Dialogue,Wu et al,https://arxiv.org/abs/1901.04713,https://github.com/jasonwu0731/GLMP,GLMP,ICLR,fully,2019 27 | KB-Transformer: Incorporating Knowledge into End-to-End Task-Oriented Dialog Systems ,E et al,https://ieeexplore.ieee.org/document/9044058,,KB-transformer,SKG,fully,2019 28 | Learning Personalized End-to-End Goal-Oriented Dialog ,Luo et al,https://aaai.org/ojs/index.php/AAAI/article/view/4654/4532,,PERSONALIZED MEMN2N ,AAAI,fully,2019 29 | A Working Memory Model for Task-oriented Dialog Response Generation,Chen tal,https://aclanthology.org/P19-1258,,WMM2Seq,ACL,fully,2019 30 | Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems ,Madotto et al,https://aclanthology.org/P18-1136/,Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems ,Mem2seq,ACL,fully,2018 31 | Multi-Level Memory for Task Oriented Dialogs ,Reddy et al,https://aclanthology.org/N19-1375/,https://github.com/DineshRaghu/multi-level-memory-network,MLM,NAACL,fully,2018 32 | Sequence-to-Sequence Learning for Task-oriented Dialogue with Dialogue State Representation ,Wen et al,http://ir.hit.edu.cn/~car/papers/coling18-hywen.pdf,,DSR,COLING,fully,2018 33 | Key-Value Retrieval Networks for Task-Oriented Dialogue ,Eric et al,https://www-nlp.stanford.edu/pubs/eric2017kvret.pdf,https://nlp.stanford.edu/blog/a-new-multi-turn-multi- domain-task-oriented-dialogue-dataset/ ,KVRet,ACL,fully,2017 34 | Latent Intention Dialogue Models,Wen et al,https://proceedings.mlr.press/v70/wen17a.html,https://github.com/shawnwun/NNDIAL ,LIDM,ICML,fully,2017 35 | Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access ,Dhingra et al,https://arxiv.org/abs/1609.00777,https://github.com/MiuLab/KB-InfoBot,KB-InfoBot,ACL,fully,2017 36 | LEARNING END-TO-END GOAL-ORIENTED DIALOG ,Bordes et al,https://arxiv.org/abs/1605.07683,,MemNN,ICLR,fully,2016 37 | A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learning ,Zhang et al,https://aclanthology.org/2020.emnlp-main.740/,https://github.com/thu-spmi/LABES,LABES-S2S,EMNLP,modularly,2020 38 | Attention over Parameters for Dialogue Systems ,Madotto et al,https://arxiv.org/abs/2001.01871,,AoP,NeurIPS,modularly,2020 39 | LAVA: Latent Action Spaces via Variational Auto-encoding for Dialogue Policy Optimization ,Lubis et al,https://aclanthology.org/2020.coling-main.41/,,LAVA,COLING,modularly,2020 40 | SUMBT+LaRL: Effective Multi-Domain End-to-End Neural Task-Oriented Dialog System ,Lee et al,https://arxiv.org/abs/2009.10447,,SUMBT+LaRL,IEEE,modularly,2020 41 | UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues ,Le et al,https://virtual.2020.emnlp.org/paper_main.1012.html,https://github.com/henryhungle/UniConv,UniConv,EMNLP,modularly,2020 42 | A Modular Task-oriented Dialogue System Using a Neural Mixture-of-Experts ,Pei et al,https://arxiv.org/abs/1907.05346,,TokenMoE,WCIS,modularly,2019 43 | Flexibly-Structured Model for Task-Oriented Dialogues,Shu et al,https://aclanthology.org/W19-5922/,https://github.com/uber-research/FSDM,FSDM,SIGDIAL,modularly,2019 44 | Incremental Learning from Scratch for Task-Oriented Dialogue Systems,Wang et al,https://aclanthology.org/P19-1361.pdf,https://github.com/Leechikara/Incremental-Dialogue-System,IDS,ACL,modularly,2019 45 | Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use, Rajendran et al,https://aclanthology.org/Q19-1024.pdf,https://github.com/IBM/modified-bAbI-dialog-tasks,-,TACL,modularly,2019 46 | MOSS: End-to-End Dialog System Framework with Modular Supervision ,Liang et al,https://ojs.aaai.org/index.php/AAAI/article/view/6349/6205,,Moss,AAAI,modularly,2019 47 | Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Models ,Zhao et al,https://aclanthology.org/N19-1123.pdf,https://github.com/snakeztc/NeuralDialog-LaRL,LARL,NAACL,modularly,2019 48 | Structured Fusion Networks for Dialog,Mehri et al,https://aclanthology.org/W19-5921,,SFN+RL,SIGDIAL,modularly,2019 49 | Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context,Zhang et al,https://arxiv.org/abs/1911.10484,https://github.com/thu-spmi/damd-multiwoz,DAMD,AAAI,modularly,2019 50 | Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems ,Liu et al,https://aclanthology.org/N18-1187/,,-,NAACL,modularly,2018 51 | End-to-End Learning of Task-Oriented Dialogs ,Liu and Lane,https://aclanthology.org/N18-4010/,,-,NAACL,modularly,2018 52 | Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures,Lei et al,https://aclanthology.org/P18-1133/,https://github.com/WING-NUS/sequicity,MD-Sequicity,ACL,modularly,2018 53 | A Copy-Augmented Sequence-to-Sequence Architecture Gives Good Performance on Task-Oriented Dialogue ,Eric and Manning,https://aclanthology.org/E17-2075/,,-,EACL,modularly,2017 54 | An End-to-End Trainable Neural Network Model with Belief Tracking for Task-Oriented Dialog ,Liu and Lane,https://www.isca-speech.org/archive_v0/Interspeech_2017/pdfs/1326.PDF,,-,InterSpeech,modularly,2017 55 | End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learning,Liu et al,https://arxiv.org/abs/1711.10712,,-,Arxiv,modularly,2017 56 | End-to-End Task-Completion Neural Dialogue Systems ,Li et al,https://aclanthology.org/I17-1074.pdf,https://github.com/MiuLab/TC-Bot,-,IJCNLP,modularly,2017 57 | Generative Encoder-Decoder Models for Task-Oriented Spoken Dialog Systems with Chatting Capability,Zhao et al,https://aclanthology.org/W17-5505/,,-,SIGDIAL,modularly,2017 58 | Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning ,Williams et al,https://aclanthology.org/P17-1062/,,HCNs,ACL,modularly,2017 59 | A network- based end-to-end trainable task-oriented dialogue system,Wen et al,https://aclanthology.org/E17-1042.pdf,NDM,NDM,EACL,modularly,2016 60 | Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning,Zhao and Eskenazi,https://aclanthology.org/W16-3601/,https://github.com/snakeztc/NeuralDialog-DM,DRQN,SIGDIAL,modularly,2016 61 | A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding,Wenbo Pan et al,https://api.semanticscholar.org/CorpusID:258049061,,-,ArXiv,pretrain,2023 62 | ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?,Michael Heck et al,https://api.semanticscholar.org/CorpusID:259063822,,-,ArXiv,pretrain,2023 63 | Are Large Language Models All You Need for Task-Oriented Dialogue?,Vojtvech Hudevcek and Ondrej Dusek,https://api.semanticscholar.org/CorpusID:258108409,,-,SIGDIAL,pretrain,2023 64 | Autoregressive Entity Generation for End-to-End Task-Oriented Dialog,Huang et al,https://aclanthology.org/2022.coling-1.25.pdf,,ECO,COLING,pretrain,2022 65 | BORT: Back and Denoising Reconstruction for End-to-End Task-Oriented Dialog,Sun et al,https://aclanthology.org/2022.findings-naacl.166.pdf,https://github.com/jd-ai-research-nlp/bort,BORT,NAACL,pretrain,2022 66 | SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation,He et al,https://arxiv.org/pdf/2209.06664,https://github.com/alibabaresearch/damo-convai,SPACE3,SIGIR,pretrain,2022 67 | Task-Oriented Dialogue System as Natural Language Generation ,Wang et al,https://arxiv.org/pdf/2108.13679,https://github.com/victorwz/tod_as_nlg,SimpleToD-ACN,SIGIR,pretrain,2022 68 | Q-TOD: A Query-driven Task-oriented Dialogue System,Tian et al,https://arxiv.org/abs/2210.07564,https://github.com/PaddlePaddle/Knover/tree/ develop/projects/Q-TOD,Q-ToD,EMNLP,pretrain,2022 69 | [CASPI] Causal-aware Safe Policy Improvement for Task-oriented Dialogue,Ramachandran et al ,https://aclanthology.org/2022.acl-long.8.pdf,https://github.com/salesforce/CASPI,CASPI,ACL,pretrain,2021 70 | AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Models ,Kulha_nek et al,https://aclanthology.org/2021.nlp4convai-1.19.pdf,https://github.com/ufal/augpt,AuGPT,EMNLP,pretrain,2021 71 | GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection ,He et al,https://ojs.aaai.org/index.php/AAAI/article/download/21320/version/19607/21069,https://github.com/siat-nlp/GALAXY,GALAXY,AAAI,pretrain,2021 72 | Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System ,Su et al,https://aclanthology.org/2022.acl-long.319.pdf,https://github.com/awslabs/pptod,PPTOD,AAAI,pretrain,2021 73 | Improving End-to-End Task-Oriented Dialog System with A Simple Auxiliary Task,Lee,https://aclanthology.org/2021.findings-emnlp.112,,MTToD,Findings,pretrain,2021 74 | A Co-Interactive Transformer for Joint Slot Filling and Intent Detection,Libo Qin et al,https://doi.org/10.1109/ICASSP39728.2021.9414110,https://github. com/kangbrilliant/DCA-Net,DCANet,ICASSP,pretrain,2021 75 | A Simple Language Model for Task-Oriented Dialogue,Hosseini-Asl et al,https://proceedings.neurips.cc/paper/2020/file/e946209592563be0f01c844ab2170f0c-Paper.pdf,https://github.com/salesforce/simpletod,SimpleTOD,NeurIPS,pretrain,2020 76 | End-to-End Neural Pipeline for Goal-Oriented Dialogue Systems using GPT-2,Ham et al,https://aclanthology.org/2020.acl-main.54.pdf,,NeuralPipeline,ACL,pretrain,2020 77 | End-to-End Trainable Non-Collaborative Dialog System ,Li et al,https://ojs.aaai.org/index.php/AAAI/article/view/6345/6201,https://gitlab.com/ucdavisnlp/antiscam,MISSA,AAAI,pretrain,2020 78 | MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems ,Lin et al,https://aclanthology.org/2020.emnlp-main.273.pdf,https://github.com/zlinao/MinTL,MINTL-BART,EMNLP,pretrain,2020 79 | Soloist : BuildingTask Bots at Scale with Transfer Learning and Machine Teaching,Peng et al,https://aclanthology.org/2021.tacl-1.49.pdf,https://aka.ms/soloist. ,SOLOIST,TACL,pretrain,2020 80 | UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2,Yang et al,https://arxiv.org/pdf/2012.03539.pdf,https://github.com/TonyNemo/UBAR-MultiWOZ,UBAR,AAAI,pretrain,2020 81 | AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling,Libo Qin et al,https://aclanthology.org/2020.findings-emnlp.163,https://github.com/LooperXX/AGIF,AGIF,EMNLP,pretrain,2020 82 | Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models ,Wu et al,https://aclanthology.org/2021.eacl-main.110.pdf,https://github.com/qywu/ARDM ,ARDM,EACL,pretrain,2019 83 | "Hello, It's GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems",Budzianowski et al,https://aclanthology.org/D19-5602.pdf,,-,ACL,pretrain,2019 -------------------------------------------------------------------------------- /assets/texts/smd.csv: -------------------------------------------------------------------------------- 1 | Model,Time,Type,BLEU,Ent_F1,Sch_F1,Wea_F1,Nav_F1 2 | KVRet,2017,Triplet,13.2,48,62.9,53.3,44.5 3 | Mem2Seq,2018,Triplet,12.6,33.4,49.3,32.8,20 4 | GLMP,2019,Triplet,14.8,60,69.6,62.6,53 5 | BossNet,2019,Triplet,8.3,35.9,50.2,34.5,21.6 6 | KB-Transformer,2019,Triplet,13.9,37.1,51.2,48.2,23.3 7 | DDMN,2020,Triplet,17.7,55.6,65,58.7,47.2 8 | DFNet,2020,Triplet,14.4,62.7,73.1,57.6,57.9 9 | DSR,2018,Row,12.7,51.9,52.1,50.4,52 10 | MLM,2018,Row,15.6,55.5,67.4,54.8,45.1 11 | KB-retriever,2019,Row,13.9,53.7,55.6,52.2,54.5 12 | HM2Seq,2022,Row,14.6,63.1,73.9,64.4,56.2 13 | Fg2Seq,2020,Graph,16.8,61.1,73.3,57.4,56.1 14 | GraphDialog,2020,Graph,13.7,60.7,72.8,55.2,54.2 15 | GraphMemDialog,2021,Graph,18.8,64.5,75.9,62.3,56.3 16 | GPT2KE,2021,Graph,17.4,59.8,72.6,57.7,53.5 17 | COMET,2021,Graph,17.3,63.6,77.6,58.3,56 18 | DialoKG,2022,Graph,20,65.9,x,x ,x 19 | MAKER,2023,Graph,25.91,71.30,x,x,x --------------------------------------------------------------------------------