├── assets └── TutorialSIGMOD23.pdf ├── LICENSE ├── .gitignore └── README.md /assets/TutorialSIGMOD23.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/madelonhulsebos/neural-table-representations-tutorial-2023/HEAD/assets/TutorialSIGMOD23.pdf -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Madelon Hulsebos 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | pip-wheel-metadata/ 24 | share/python-wheels/ 25 | *.egg-info/ 26 | .installed.cfg 27 | *.egg 28 | MANIFEST 29 | 30 | # PyInstaller 31 | # Usually these files are written by a python script from a template 32 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 33 | *.manifest 34 | *.spec 35 | 36 | # Installer logs 37 | pip-log.txt 38 | pip-delete-this-directory.txt 39 | 40 | # Unit test / coverage reports 41 | htmlcov/ 42 | .tox/ 43 | .nox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | *.py,cover 51 | .hypothesis/ 52 | .pytest_cache/ 53 | 54 | # Translations 55 | *.mo 56 | *.pot 57 | 58 | # Django stuff: 59 | *.log 60 | local_settings.py 61 | db.sqlite3 62 | db.sqlite3-journal 63 | 64 | # Flask stuff: 65 | instance/ 66 | .webassets-cache 67 | 68 | # Scrapy stuff: 69 | .scrapy 70 | 71 | # Sphinx documentation 72 | docs/_build/ 73 | 74 | # PyBuilder 75 | target/ 76 | 77 | # Jupyter Notebook 78 | .ipynb_checkpoints 79 | 80 | # IPython 81 | profile_default/ 82 | ipython_config.py 83 | 84 | # pyenv 85 | .python-version 86 | 87 | # pipenv 88 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 89 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 90 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 91 | # install all needed dependencies. 92 | #Pipfile.lock 93 | 94 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow 95 | __pypackages__/ 96 | 97 | # Celery stuff 98 | celerybeat-schedule 99 | celerybeat.pid 100 | 101 | # SageMath parsed files 102 | *.sage.py 103 | 104 | # Environments 105 | .env 106 | .venv 107 | env/ 108 | venv/ 109 | ENV/ 110 | env.bak/ 111 | venv.bak/ 112 | 113 | # Spyder project settings 114 | .spyderproject 115 | .spyproject 116 | 117 | # Rope project settings 118 | .ropeproject 119 | 120 | # mkdocs documentation 121 | /site 122 | 123 | # mypy 124 | .mypy_cache/ 125 | .dmypy.json 126 | dmypy.json 127 | 128 | # Pyre type checker 129 | .pyre/ 130 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Models and Practice of Neural Table Representations 2 | 3 | In this repository, you can find more information and up to date material for the tutorial on Models and Practice of Neural Table Representations. This tutorial will be given on 23 June 2023 at SIGMOD 2023 in Seattle. More details will follow. 4 | 5 | **Abstract**: In the last few years, the natural language processing community 6 | witnessed advances in neural representations of free-form text with 7 | transformer-based language models (LMs). Given the importance 8 | of knowledge available in relational tables, recent research efforts 9 | extend LMs by developing neural representations for tabular data. 10 | In this tutorial, we present these proposals with three main goals. 11 | First, we aim at introducing the potentials and limitations of current 12 | models to a database audience. Second, we want the attendees 13 | to see the benefit of such line of work in a large variety of data 14 | applications. Third, we would like to empower the audience with a 15 | new set of tools and to inspire them to tackle some of the important 16 | directions for neural table representations, including model and 17 | system design, evaluation, application and deployment. To achieve 18 | these goals, the tutorial is organized in two parts. The first part 19 | covers the background for neural table representations, including a 20 | survey of the most important systems. The second part is designed 21 | as a hands-on session, where attendees will use their laptop to 22 | explore this new framework and test neural models involving text 23 | and tabular data. 24 | 25 | 26 | ## Contents 27 | - [Tutorial overview](#tutorial-overview) 28 | - [Logistics](#logistics) 29 | - [Material and sources](#code-and-sources) 30 | 31 | 32 | 33 | ## Tutorial overview 34 | The tutorial will take 3 hours in total. The first part (1.5 hours) 35 | covers the background for neural table representations. 36 | The second part (1.5 hours) is designed as a hands-on session, 37 | where attendees will use their laptop to explore this new framework and 38 | test neural models involving text and tabular data. 39 | 40 | In part 1: 41 | - Motivation & background on Transformers and Language models. 42 | - Survey of the key neural table models and applications. 43 | - Overview of open challenges and future directions. 44 | 45 | In part 2a: 46 | - Input formats and preprocessing (serialization, tokenization, numeric) 47 | - Understanding model architecture 48 | - Using fine-tuned models to answer questions / verify statements 49 | 50 | In part 2b: 51 | - Understand self-supervised pre-training using raw tables without additional labels 52 | - Understand different types of fine-tuning strategies for downstream tasks: 53 | - Token and sequence Classification: classification based on table, column, row or cell representations (e.g. column type prediction, fact verification) 54 | - Extractive QA: predict the start and end positions of the answer span (e.g. table QA) 55 | 56 | 57 | 58 | ## Logistics 59 | 60 | Location: Seattle (at SIGMOD 2023). 61 | 62 | Time: 1:30-5:00, 23 June. 63 | 64 | Presenters: Madelon Hulsebos, Xiang Deng, Huan Sun, and Paolo Papotti. 65 | 66 | 67 | 68 | ## Material: 69 | 70 | - Slides part 1 (by Paolo Papotti): [Tutorial slides download](assets/TutorialSIGMOD23.pdf) 71 | 72 | - Google Colab Notebook for part 2a (by Madelon Hulsebos): [https://colab.research.google.com/drive/1trjqxE6YGvFcERyimITkZD98TRGbCWx1?usp=sharing](https://colab.research.google.com/drive/1trjqxE6YGvFcERyimITkZD98TRGbCWx1?usp=sharing) 73 | - Google Colab Notebook for part 2b (by Xiang Deng): [https://colab.research.google.com/drive/1WTg-YnfNVX4M0P0m1mEJYDQVkWr4HKXl](https://colab.research.google.com/drive/1WTg-YnfNVX4M0P0m1mEJYDQVkWr4HKXl) 74 | 75 | 76 | 77 | ## Citation: 78 | 79 | ``` 80 | @inproceedings{hulsebos2023models, 81 | title={Models and Practice of Neural Table Representations}, 82 | author={Hulsebos, Madelon and Deng, Xiang and Sun, Huan and Papotti, Paolo}, 83 | booktitle={Companion of the 2023 International Conference on Management of Data}, 84 | pages={83--89}, 85 | year={2023} 86 | } 87 | --------------------------------------------------------------------------------