├── .gitignore ├── LICENSE ├── MetaDataFormat.txt └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | .private/ 2 | __pycache__ 3 | *.pyc 4 | *.swp 5 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (C) 2020 Brad Jascob 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /MetaDataFormat.txt: -------------------------------------------------------------------------------- 1 | Meta file must be named amrlib_meta.json and reside in the top level of the model directory. 2 | 3 | The following is considered a minimal example of the json file... 4 | } 5 | "model_type":"stog", 6 | "version":"0.1.0", 7 | "date":"2022-02-16", 8 | "inference_module":".parse_xfm.inference", 9 | "inference_class":"Inference", 10 | "model_fn":"pytorch_model.bin", 11 | "base_model":"facebook/bart-large", 12 | "kwargs":{} 13 | } 14 | 15 | At this time only "inference_module" and "inference_class" are required for all models. 16 | "model_fn" is not required for T5 based models but may be for other model types. 17 | "kwargs" will be passed directly to the instantiation of the inference class. 18 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # amrlib models 2 | This repository contains [releases](https://github.com/bjascob/amrlib-models/releases) of models 3 | for the [amrlib](https://github.com/bjascob/amrlib) library. For information on how to download 4 | and install see [ReadTheDocs Installation Instructions](https://amrlib.readthedocs.io/en/latest/install/) 5 | or follow the instructions below. 6 | 7 | Note that because these models are large binary files, they are not directly tracked with git. 8 | Instead the are provided for download as `.tar.gz` files. 9 | 10 | ## Installation 11 | Download the pre-trained models from the [releases](https://github.com/bjascob/amrlib-models/releases) 12 | directory and extract in them in `amrlib/data`. Set a link to the directory (or rename it) as either 13 | `model_stog` (sentence to graph) for the parse style models or `model_gtos` (graph to sentence) for 14 | the generate style models. 15 | 16 | If you're unsure where amrlib is installed you can do... 17 | ``` 18 | pip3 show amrlib 19 | ``` 20 | or 21 | ``` 22 | python3 23 | >>> import amrlib 24 | >>> amrlib.__file__ 25 | ``` 26 | 27 | ## Sentence to Graph Models 28 | 29 | | Name | Version | Date | Size | Score | Speed ||| 30 | | ---- | ------- | ---- | ---- | ----- | --- | --- |---| 31 | | parse_xfm_bart_large | 0.1.0 | 2022-02-16 | 1.4GB | 83.7 SMATCH | 17/sec| [![][i]][i-parse_xfm_bart_large] | [![][dl]][d-parse_xfm_bart_large] 32 | | parse_xfm_bart_base | 0.1.0 | 2022-02-16 | 492 MB | 82.3 SMATCH | 31/sec| [![][i]][i-parse_xfm_bart_base] | [![][dl]][d-parse_xfm_bart_base] 33 | | parse_spring | 0.1.0 | 2021-11-25 | 1.5GB | 83.5 SMATCH | 14/sec| [![][i]][i-parse_spring] | [![][dl]][d-parse_spring] 34 | | parse_t5 | 0.2.0 | 2020-11-27 | 785MB | 81.9 SMATCH | 11/sec| [![][i]][i-parse_t5] | [![][dl]][d-parse_t5] 35 | | parse_gsii | 0.1.0 | 2020-08-30 | 787MB | 76.8 SMATCH | 28/sec| | [![][dl]][d-parse_gsii] 36 | 37 | All models are trained and scored on AMR-3 (LDC2020T02) using `num_beams=4` for `parse_xfm_x` and `num_beams=5` for `parse_spring`. 38 | Note that AMR-3 is a more difficult test set than the older AMR-2 set and generally scores a bit lower for similar models. All scores are 39 | without adding the :wiki tags. However, when using BLINK, scores typically stay approximately the same since the wikification process itself 40 | scores in the low to mid 80s on smatch. 41 | 42 | Speed is the inference speed on the AMR-3 test set (1898 graphs) using an RTX3090 with `num_beams=1` and `batch_size=32`. The units are sentences/second. 43 | 44 | #### Attribution: 45 | * The parse_spring code is from the [SPRING model](https://github.com/SapienzaNLP/spring). Note that the author's license for their code is "Attribution-NonCommercial-ShareAlike 4.0 International". Details on the model can be found in this [paper](https://ojs.aaai.org/index.php/AAAI/article/view/17489). 46 | 47 | * The parse_gsii model comes from [jcyk/AMR-gs](https://github.com/jcyk/AMR-gs), the details of which can be found in this 48 | [paper](https://arxiv.org/abs/2004.05572). 49 | 50 | * All other models were developed as part of [amrlib](https://github.com/bjascob/amrlib). 51 | 52 | [d-parse_xfm_bart_large]: https://github.com/bjascob/amrlib-models/releases/download/parse_xfm_bart_large-v0_1_0/model_parse_xfm_bart_large-v0_1_0.tar.gz 53 | [d-parse_xfm_bart_base]: https://github.com/bjascob/amrlib-models/releases/download/parse_xfm_bart_base-v0_1_0/model_parse_xfm_bart_base-v0_1_0.tar.gz 54 | [d-parse_spring]: https://github.com/bjascob/amrlib-models/releases/download/model_parse_spring-v0_1_0/model_parse_spring-v0_1_0.tar.gz 55 | [d-parse_t5]: https://github.com/bjascob/amrlib-models/releases/download/model_parse_t5-v0_2_0/model_parse_t5-v0_2_0.tar.gz 56 | [d-parse_gsii]: https://u.pcloud.link/publink/show?code=XZD2z0XZOqRtS2mNMHhMG4UhXOCNO4yzeaLk 57 | 58 | [i-parse_xfm_bart_large]: https://github.com/bjascob/amrlib-models/releases/parse_xfm_bart_large-v0_1_0 59 | [i-parse_xfm_bart_base]: https://github.com/bjascob/amrlib-models/releases/parse_xfm_bart_base-v0_1_0 60 | [i-parse_spring]: https://github.com/bjascob/amrlib-models/releases/model_parse_spring-v0_1_0 61 | [i-parse_t5]: https://github.com/bjascob/amrlib-models/releases/model_parse_t5-v0_2_0 62 | 63 | 64 | ## Graph to Sentence Models 65 | 66 | | Name | Version | Date | Size | Score | | | 67 | | ---- | ------- | ---- | ---- | ----- | --- | --- | 68 | | generate_t5wtense | 0.1.0 | 2020-12-30 | 787MB | 54/44 BLEU | [![][i]][i-model_generate_t5wtense-v0_1_0] | [![][dl]][model_generate_t5wtense-v0_1_0] 69 | 70 | The generate_t5wtense gives a **54 BLEU** with tense tags or **44 BLEU** with un-tagged LDC2020T02. 71 | Note that the model is only scored with graphs that fit in the T5 model's 512 token limit. If including clipped 72 | graphs, scores will be more like 52/43 BLEU. 73 | Details on using this type of model for generation can be found in this [paper](https://arxiv.org/abs/2007.08426). 74 | 75 | Additionally, there is a training config file for a T5-large based model in the [amrlib/config directory here](https://github.com/bjascob/amrlib/blob/master/configs/model_generate_xfm_t5_large_wTT.json). This model scores about 2 BLEU points higher than generate_t5wtense-v0_1_0 76 | if you take the time to train it yourself. The training configuration has not been full optimized so you may be able to increase 77 | this if you try different hyperparameters or possibly a different version of the base pretrained T5 model. 78 | 79 | [model_generate_t5wtense-v0_1_0]: https://github.com/bjascob/amrlib-models/releases/download/model_generate_t5wtense-v0_1_0/model_generate_t5wtense-v0_1_0.tar.gz 80 | 81 | [i-model_generate_t5wtense-v0_1_0]: https://github.com/bjascob/amrlib-models/releases/model_generate_t5wtense-v0_1_0 82 | 83 | 84 | [dl]: http://i.imgur.com/gQvPgr0.png 85 | [i]: http://i.imgur.com/OpLOcKn.png 86 | 87 | 88 | ## Issues and bug reports 89 | To report an issue with a model please use the [amrlib issues](https://github.com/bjascob/amrlib/issues) list. 90 | --------------------------------------------------------------------------------