├── COMP6v1 ├── ANI-MD │ └── ani_md_bench.h5 ├── DrugBank │ └── drugbank_testset.h5 ├── GDB07to09 │ ├── gdb11_07_test500.h5 │ ├── gdb11_08_test500.h5 │ └── gdb11_09_test500.h5 ├── GDB10to13 │ ├── gdb11_10_test500.h5 │ ├── gdb11_11_test500.h5 │ ├── gdb13_12_test1000.h5 │ └── gdb13_13_test1000.h5 ├── Tripeptides │ └── tripeptide_full.h5 └── s66x8 │ └── s66x8_wb97x6-31gd.h5 ├── LICENSE └── README.md /COMP6v1/ANI-MD/ani_md_bench.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/isayev/COMP6/79f41c156f8e19506fe951b5513b98e7c534a503/COMP6v1/ANI-MD/ani_md_bench.h5 -------------------------------------------------------------------------------- /COMP6v1/DrugBank/drugbank_testset.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/isayev/COMP6/79f41c156f8e19506fe951b5513b98e7c534a503/COMP6v1/DrugBank/drugbank_testset.h5 -------------------------------------------------------------------------------- /COMP6v1/GDB07to09/gdb11_07_test500.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/isayev/COMP6/79f41c156f8e19506fe951b5513b98e7c534a503/COMP6v1/GDB07to09/gdb11_07_test500.h5 -------------------------------------------------------------------------------- /COMP6v1/GDB07to09/gdb11_08_test500.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/isayev/COMP6/79f41c156f8e19506fe951b5513b98e7c534a503/COMP6v1/GDB07to09/gdb11_08_test500.h5 -------------------------------------------------------------------------------- /COMP6v1/GDB07to09/gdb11_09_test500.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/isayev/COMP6/79f41c156f8e19506fe951b5513b98e7c534a503/COMP6v1/GDB07to09/gdb11_09_test500.h5 -------------------------------------------------------------------------------- /COMP6v1/GDB10to13/gdb11_10_test500.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/isayev/COMP6/79f41c156f8e19506fe951b5513b98e7c534a503/COMP6v1/GDB10to13/gdb11_10_test500.h5 -------------------------------------------------------------------------------- /COMP6v1/GDB10to13/gdb11_11_test500.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/isayev/COMP6/79f41c156f8e19506fe951b5513b98e7c534a503/COMP6v1/GDB10to13/gdb11_11_test500.h5 -------------------------------------------------------------------------------- /COMP6v1/GDB10to13/gdb13_12_test1000.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/isayev/COMP6/79f41c156f8e19506fe951b5513b98e7c534a503/COMP6v1/GDB10to13/gdb13_12_test1000.h5 -------------------------------------------------------------------------------- /COMP6v1/GDB10to13/gdb13_13_test1000.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/isayev/COMP6/79f41c156f8e19506fe951b5513b98e7c534a503/COMP6v1/GDB10to13/gdb13_13_test1000.h5 -------------------------------------------------------------------------------- /COMP6v1/Tripeptides/tripeptide_full.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/isayev/COMP6/79f41c156f8e19506fe951b5513b98e7c534a503/COMP6v1/Tripeptides/tripeptide_full.h5 -------------------------------------------------------------------------------- /COMP6v1/s66x8/s66x8_wb97x6-31gd.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/isayev/COMP6/79f41c156f8e19506fe951b5513b98e7c534a503/COMP6v1/s66x8/s66x8_wb97x6-31gd.h5 -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Justin S. Smith, Olexandr Isayev, Adrian E. Roitberg 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## COmprehensive Machine-learning Potential (COMP6) Benchmark Suite 2 | This repository contains the COMP6 benchmark for evaluating the extensibility of machine-learning based molecular potentials. 3 | 4 | ##### If you use the COMP6 benchmark please cite this paper: 5 | 6 | ### Active learning-based (ANI-1x): 7 | Justin S. Smith, Ben Nebgen, Nicholas Lubbers, Olexandr Isayev, Adrian E. Roitberg. *Less is more: sampling chemical space with active learning*. The Journal of Chemical Physics 148, 241733 (2018), (https://aip.scitation.org/doi/abs/10.1063/1.5023802) 8 | 9 | ## Usage 10 | Please read the README.md in the repository linked below for instructions on how to extract the COMP6 HDF5 (extention \*.h5) files. 11 | https://github.com/isayev/ANI1_dataset 12 | 13 | The following paper contains a description of the file format: 14 | https://www.nature.com/articles/sdata2017193 15 | 16 | ## COMP6 Benchmark Results: 17 | #### These results represent the errors (MAE/RMSE) over the entire benchmark using a single ML potential (column 1). Please read https://aip.scitation.org/doi/abs/10.1063/1.5023802 Section IID for a detailed description of the error metrics. 18 | #### Please contact Justin S. Smith at jussmith48@gmail.com if you'd like to add your results from the COMP6 benchmark. 19 | ### Complete COMP6 benchmark results: 20 | | Potential | Energy | Relative Energy | Force | 21 | | ------------------ | ---------------- | ---------------- | ---------------- | 22 | | ANI-1x1 | 1.93/3.37 | 1.85/2.95 | 3.09/5.29 | 23 | | ANI-11 | 5.01/16.9 | 3.01/6.97 | 3.70/7.13 | 24 | 25 | Units: kcal/mol and kcal/mol/A (errors are NOT per atom) 26 | Error key: MAE/RMSE 27 | 28 | ##### 1) https://aip.scitation.org/doi/abs/10.1063/1.5023802 29 | 30 | ## Related work 31 | 32 | ### ANAKIN-ME ML Potential Method: 33 | Justin S. Smith, Olexandr Isayev, Adrian E. Roitberg. *ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost*. Chemical Science, 2017, DOI: [10.1039/C6SC05720A](http://pubs.rsc.org/en/content/articlelanding/2017/sc/c6sc05720a) 34 | 35 | ### Original ANI-1 data: 36 | Justin S. Smith, Olexandr Isayev, Adrian E. Roitberg. ANI-1, A data set of 20 million calculated off-equilibrium conformations for organic molecules. Scientific Data, 4, Article number: 170193, DOI: 10.1038/sdata.2017.193 https://www.nature.com/articles/sdata2017193 37 | 38 | ### Active learning and transfer learning-based (ANI-1ccx): 39 | Justin S. Smith, Benjamin T. Nebgen, Roman Zubatyuk, Nicholas Lubbers, Christian Devereux, Kipton Barros, Sergei Tretiak, Olexandr Isayev, Adrian Roitberg. Outsmarting Quantum Chemistry Through Transfer Learning. ChemRxiv, 2018, DOI: [https://doi.org/10.26434/chemrxiv.6744440.v1] 40 | --------------------------------------------------------------------------------