├── LICENSE ├── README.md ├── examples ├── Batch_Example.ipynb ├── generic_batch.py ├── generic_batch_file.py ├── mm_interactive.py ├── mti_interactive.py ├── sr_interactive.py └── sr_interactive_file.py ├── pyproject.toml ├── requirements.txt ├── setup.cfg └── src └── skr_web_api ├── __init__.py └── casauth.py /LICENSE: -------------------------------------------------------------------------------- 1 | Terms and Conditions for Use of MetaMapLite 2 | 3 | Informational Notice: 4 | 5 | This software, “MetaMapLite” was developed and funded by the National 6 | Library of Medicine, part of the National Institutes of Health, and 7 | agency of the United States Department of Health and Human Services, 8 | which is making the software available to the public for any 9 | commercial or non-commercial purpose under the following open-source 10 | BSD license. 11 | 12 | NOTE: Users of the data distributed with MetaMapLite are 13 | responsible for compliance with the UMLS Metathesaurus License 14 | Agreement which requires you to respect the copyrights of the 15 | constituent vocabularies and to file a brief annual report on your use 16 | of the UMLS. You also must have activated a UMLS Terminology Services 17 | (UTS) account. 18 | 19 | LICENSE: 20 | 21 | Government Usage Rights Notice: The U.S. Government retains unlimited, 22 | royalty-free usage rights to this software, but not ownership, as 23 | provided by Federal law. 24 | 25 | Redistribution and use in source and binary forms, with or without 26 | modification, are permitted provided that the following conditions are 27 | met: 28 | 29 | * Redistributions of source code must retain this Informational Notice. 30 | * Redistributions in binary form must reproduce this Informational 31 | Notice, this list of conditions and the following disclaimer in the 32 | documentation and/or other materials provided with the distribution. 33 | * Neither the names of the National Library of Medicine, the National 34 | Institutes of Health, nor the names of any of the software 35 | developers may be used to endorse or promote products derived from 36 | this software without specific prior written permission. The 37 | U.S. Government retains an unlimited, royalty-free right to use, 38 | distribute or modify the software. 39 | * Please acknowledge NLM as the source of the MetaMapLite software by 40 | including the phrase “Courtesy of the U.S. National Library of 41 | Medicine” or “Source: U.S. National Library of Medicine.” 42 | 43 | 44 | THIS SOFTWARE IS PROVIDED BY THE U.S. GOVERNMENT AND CONTRIBUTORS "AS 45 | IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 46 | LIMITEDTO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A 47 | PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 48 | U.S. GOVERNMENT 49 | 50 | OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 51 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 52 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 53 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 54 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 55 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 56 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 57 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Indexing Initiative Web API: Python Implementation 2 | 3 | This Python-based API for the Indexing Initiative Scheduler facility 4 | was created to provide users with the ability to programmatically 5 | submit jobs to the Scheduler Batch and Interactive facilities instead 6 | of using the web-based interface. 7 | 8 | Three programs are accessible via the Web API: MetaMap, the NLM 9 | Medical Text Indexer (MTI), and SemRep. The functionality in these 10 | programs includes: automatic indexing of MEDLINE citations, 11 | concept-based query expansion, analysis of complex Metathesaurus 12 | strings, accurate identification of the terminology and relationships 13 | in anatomical documents, and the extraction of chemical binding 14 | relations from biomedical text. 15 | 16 | See Indexing Initiative's Web API page 17 | (https://ii.nlm.nih.gov/Web_API/index.shtml) for more information. 18 | 19 | # Prerequisites 20 | 21 | + To access either the the Interactive Mode or Batch Mode facilities, 22 | you must have accepted the terms of the 23 | [UMLS Metathesaurus License Agreement] 24 | (https://uts.nlm.nih.gov/license.html), which requires you to 25 | respect the copyrights of the constituent vocabularies and to file a 26 | brief annual report on your use of the UMLS. You also must have 27 | activated a [UMLS Terminology Services (UTS) account] 28 | (https://uts.nlm.nih.gov/home.html). See 29 | [UTS Account Information page] 30 | (http://skr.nlm.nih.gov/Help/umlsks.shtml) for information on how we 31 | use UTS authentication. 32 | 33 | + The SKR Web API requires at least Python 3.8 and two packages: 34 | Requests (https://docs.python-requests.org/en/latest/) and 35 | Requests-HTML (https://github.com/psf/requests-html) to run. It's 36 | possible that it will work with earlier versions of Python 3 but 37 | that hasn't been tested. 38 | 39 | # Building and Installing the API 40 | 41 | Installing prerequisites using pip: 42 | 43 | python3 -m pip install requests requests-html 44 | python3 -m pip install wheel 45 | python3 -m pip install --upgrade pip 46 | python3 -m pip install --upgrade build 47 | 48 | Building the wheel package from sources after cloning the repository: 49 | 50 | git clone https://github.com/lhncbc/skr_web_python_api.git 51 | cd skr_web_python_api 52 | python3 -m build 53 | 54 | Installing the wheel package into your virtual environment: 55 | 56 | python3 -m pip install dist/skr_web_api-0.1-py3-none-any.whl 57 | 58 | Note: If you don't have write access to the installed Python then you 59 | may need to use virtual environment created using venv 60 | (https://docs.python.org/3/library/venv.html) or Miniconda 61 | (https://docs.conda.io/en/latest/miniconda.html). 62 | 63 | 64 | # Usage 65 | 66 | To run the examples, you'll need a UTS API key which available in your 67 | UTS profile. 68 | 69 | ## A Simple Batch Example 70 | 71 | A simple example of submitting file to generic batch with validation 72 | to be processed by SemRep: 73 | 74 | >>> email = 'username@address' 75 | >>> apikey = '' 76 | >>> inputfilename = 'doc.txt' 77 | >>> inst = Submission(email, apikey) 78 | >>> inst.init_generic_batch("semrep", "-D") 79 | >>> inst.set_batch_file(inputfilename) 80 | >>> response = inst.submit() 81 | >>> print('response status: {}'.format(response.status_code)) 82 | response status: 200 83 | >>> print('content: {}'.format(response.content)) 84 | ... output omitted ... 85 | 86 | ## Interactive Example 87 | 88 | An example of processing a string using interactive MetaMap service: 89 | 90 | >>> email = 'username@address' 91 | >>> apikey = '' 92 | >>> inputfilename = 'doc.txt' 93 | >>> inst = Submission(email, apikey) 94 | >>> inputtext = "A spinal tap was performed and oligoclonal bands were \ 95 | detected in the cerebrospinal fluid.\n" 96 | >>> inst.init_mm_interactive(inputtext) 97 | >>> response = inst.submit() 98 | >>> print('response status: {}'.format(response.status_code)) 99 | response status: 200 100 | >>> print('content:\n {}'.format(response.content.decode())) 101 | content: 102 | /dmzfiler/II_Group/MetaMap2020/public_mm/bin/SKRrun.20 /dmzfiler/II_Group/MetaMap2020/public_mm/bin/metamap20.BINARY.Linux --lexicon db -Z 2020AB -N 103 | USER|MMI|5.18|Diagnostic lumbar puncture|C0553794|[diap]|["Spinal Tap"-tx-1-"spinal tap"-noun-0]|TX|2/10| 104 | USER|MMI|5.18|Oligoclonal Bands|C4048246|[lbtr]|["Oligoclonal Bands"-tx-1-"oligoclonal bands"-noun-0]|TX|31/17| 105 | USER|MMI|5.18|Oligoclonal Bands (protein)|C0069426|[aapp,imft]|["Oligoclonal Bands"-tx-1-"oligoclonal bands"-noun-0]|TX|31/17| 106 | USER|MMI|5.18|Oligoclonal protein measurement|C0202205|[lbpr]|["Oligoclonal Bands"-tx-1-"oligoclonal bands"-noun-0]|TX|31/17| 107 | USER|MMI|5.18|Performed|C0884358|[ftcn]|["PERFORMED"-tx-1-"performed"-verb-0]|TX|17/9| 108 | USER|MMI|5.18|Spinal Puncture|C0037943|[hlca]|["Spinal Tap"-tx-1-"spinal tap"-noun-0]|TX|2/10| 109 | USER|MMI|3.61|Cerebrospinal Fluid|C0007806|[bdsu]|["Spinal Fluid, Cerebro"-tx-1-"cerebrospinal fluid"-noun-0]|TX|70/19| 110 | USER|MMI|3.61|In Cerebrospinal Fluid|C0007807|[ftcn]|["cerebrospinal fluid"-tx-1-"cerebrospinal fluid"-noun-0]|TX|70/19| 111 | USER|MMI|3.56|Detected (finding)|C0442726|[fndg]|["Detected"-tx-1-"detected"-verb-0]|TX|54/8| 112 | USER|MMI|3.56|Detection|C1511790|[topp]|["Detected"-tx-1-"detected"-verb-0]|TX|54/8| 113 | >>> 114 | 115 | ## Example programs 116 | 117 | ### Generic Batch Example 118 | 119 | The Python source file __generic_batch_file.py__, in the __examples__ 120 | directory, contains source for example of submitting file to generic 121 | batch with validation. 122 | 123 | Usage: 124 | 125 | usage: generic_batch_file.py [-h] [-e EMAIL] [-a APIKEY] inputfile 126 | 127 | 128 | Example of use: 129 | 130 | python examples/generic_batch_file.py -e user@host -a 1234-5678-9ABC-DEF1 \ 131 | ~/queries/iiquery 132 | 133 | 134 | ### Interactive example sources 135 | 136 | 137 | The directory __examples__ also contains examples of using the 138 | MetaMap, MTI, and SemRep interactive services. 139 | 140 | MetaMap: 141 | 142 | python examples/mm_interactive.py -e $EMAIL -a $UTS_API_KEY 143 | 144 | MTI: 145 | 146 | python examples/mti_interactive.py -e $EMAIL -a $UTS_API_KEY 147 | 148 | SemRep: 149 | 150 | python examples/sr_interactive.py -e $EMAIL -a $UTS_API_KEY 151 | -------------------------------------------------------------------------------- /examples/Batch_Example.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "05c3928c", 6 | "metadata": {}, 7 | "source": [ 8 | "# Example of submitting a batch run using SKR Web Python API " 9 | ] 10 | }, 11 | { 12 | "cell_type": "code", 13 | "execution_count": 1, 14 | "id": "9e477e98", 15 | "metadata": {}, 16 | "outputs": [], 17 | "source": [ 18 | "import os\n", 19 | "from skr_web_api import Submission" 20 | ] 21 | }, 22 | { 23 | "cell_type": "markdown", 24 | "id": "ac5ec434", 25 | "metadata": {}, 26 | "source": [ 27 | "The program below expects user's email to be in the environment variable \"EMAIL\" and the UTS API key to be in the variable \"UTS_API_KEY\". The program then sets the name of filename to \"input.txt\" with content of the file supplied by the parameter \"inputtext\". If parameter inputtext is not supplied then the API attempts to open the file inputfilename. The program then submits the job to the SKR service." 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "execution_count": 2, 33 | "id": "df5fb3b1", 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "email = os.environ['EMAIL']\n", 38 | "apikey = os.environ['UTS_API_KEY']\n", 39 | "inputfilename = 'input.txt'\n", 40 | "inst = Submission(email, apikey)\n", 41 | "inst.init_generic_batch(\"metamap -N\", \"\")\n", 42 | "inst.set_batch_file(inputfilename, inputtext=\"A spinal tap was performed and oligoclonal bands were \\\n", 43 | "detected in the cerebrospinal fluid.\\n\")\n", 44 | "response = inst.submit()" 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "id": "395a2525", 50 | "metadata": {}, 51 | "source": [ 52 | "When the response is returned the following displays the response code:" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": 3, 58 | "id": "804f8dd4", 59 | "metadata": {}, 60 | "outputs": [ 61 | { 62 | "data": { 63 | "text/plain": [ 64 | "200" 65 | ] 66 | }, 67 | "execution_count": 3, 68 | "metadata": {}, 69 | "output_type": "execute_result" 70 | } 71 | ], 72 | "source": [ 73 | "response.status_code" 74 | ] 75 | }, 76 | { 77 | "cell_type": "markdown", 78 | "id": "37b16e5c", 79 | "metadata": {}, 80 | "source": [ 81 | "If the response status code is 200, then the returned data is supplied in the content attribute of the response:" 82 | ] 83 | }, 84 | { 85 | "cell_type": "code", 86 | "execution_count": 4, 87 | "id": "a04b9ee8", 88 | "metadata": {}, 89 | "outputs": [ 90 | { 91 | "name": "stdout", 92 | "output_type": "stream", 93 | "text": [ 94 | "text_000N_67799|MMI|5.18|Diagnostic lumbar puncture|C0553794|[diap]|[\"Spinal Tap\"-tx-1-\"spinal tap\"-noun-0]|TX|2/10|\n", 95 | "text_000N_67799|MMI|5.18|Oligoclonal Bands|C4048246|[lbtr]|[\"Oligoclonal Bands\"-tx-1-\"oligoclonal bands\"-noun-0]|TX|31/17|\n", 96 | "text_000N_67799|MMI|5.18|Oligoclonal Bands (protein)|C0069426|[aapp,imft]|[\"Oligoclonal Bands\"-tx-1-\"oligoclonal bands\"-noun-0]|TX|31/17|\n", 97 | "text_000N_67799|MMI|5.18|Performed|C0884358|[ftcn]|[\"Performed\"-tx-1-\"performed\"-verb-0]|TX|17/9|\n", 98 | "text_000N_67799|MMI|5.18|Spinal Puncture|C0037943|[hlca]|[\"Spinal Tap\"-tx-1-\"spinal tap\"-noun-0]|TX|2/10|\n", 99 | "text_000N_67799|MMI|3.61|Cerebrospinal Fluid|C0007806|[bdsu]|[\"CEREBROSPINAL FLUID\"-tx-1-\"cerebrospinal fluid\"-noun-0]|TX|70/19|\n", 100 | "text_000N_67799|MMI|3.61|In Cerebrospinal Fluid|C0007807|[ftcn]|[\"cerebrospinal fluid\"-tx-1-\"cerebrospinal fluid\"-noun-0]|TX|70/19|\n", 101 | "text_000N_67799|MMI|3.56|Detected Finding|C0442726|[fndg]|[\"Detected\"-tx-1-\"detected\"-verb-0]|TX|54/8|\n", 102 | "text_000N_67799|MMI|3.56|Detection|C1511790|[topp]|[\"Detected\"-tx-1-\"detected\"-verb-0]|TX|54/8|\n", 103 | "\n" 104 | ] 105 | } 106 | ], 107 | "source": [ 108 | "print(response.content.decode('utf-8').replace('NOT DONE LOOP\\n', ''))" 109 | ] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "execution_count": null, 114 | "id": "120d511b", 115 | "metadata": {}, 116 | "outputs": [], 117 | "source": [] 118 | } 119 | ], 120 | "metadata": { 121 | "kernelspec": { 122 | "display_name": "pyston_env", 123 | "language": "python", 124 | "name": "pyston_env" 125 | }, 126 | "language_info": { 127 | "codemirror_mode": { 128 | "name": "ipython", 129 | "version": 3 130 | }, 131 | "file_extension": ".py", 132 | "mimetype": "text/x-python", 133 | "name": "python", 134 | "nbconvert_exporter": "python", 135 | "pygments_lexer": "ipython3", 136 | "version": "3.11.0" 137 | } 138 | }, 139 | "nbformat": 4, 140 | "nbformat_minor": 5 141 | } 142 | -------------------------------------------------------------------------------- /examples/generic_batch.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | from skr_web_api import Submission, BATCH_VALIDATION_URL 3 | 4 | 5 | def unescape_string(textstring): 6 | """Remove leading backslashes from text string""" 7 | return textstring.replace('\\', '') 8 | 9 | if __name__ == '__main__': 10 | parser = argparse.ArgumentParser(description="test cas auth") 11 | parser.add_argument('-e', '--email', help='Email address') 12 | parser.add_argument('-a', '--apikey', help='UTS api key') 13 | parser.add_argument('-c', '--cmd', default='semrep', 14 | help='batch command') 15 | parser.add_argument('-r', '--cmdargs', 16 | default='-D', 17 | help='batch command arguments') 18 | parser.add_argument('-s', '--serviceurl', 19 | default=BATCH_VALIDATION_URL, 20 | help='url of service') 21 | args = parser.parse_args() 22 | inputtext = "A spinal tap was performed and oligoclonal bands were \ 23 | detected in the cerebrospinal fluid.\n" 24 | print("cmd: {}".format(args.cmd)) 25 | print("cmdargs: {}".format(unescape_string(args.cmdargs))) 26 | inst = Submission(args.email, args.apikey) 27 | if args.serviceurl: 28 | inst.set_serviceurl(args.serviceurl) 29 | inst.init_generic_batch(args.cmd, unescape_string(args.cmdargs)) 30 | inst.set_batch_file('USERINPUT', inputtext) 31 | response = inst.submit() 32 | print('response status: {}'.format(response.status_code)) 33 | print('content: {}'.format(response.content.decode())) 34 | -------------------------------------------------------------------------------- /examples/generic_batch_file.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | from skr_web_api import Submission 3 | 4 | 5 | def unescape_string(textstring): 6 | """Remove leading backslashes from text string""" 7 | return textstring.replace('\\', '') 8 | 9 | if __name__ == '__main__': 10 | serverurl = "https://utslogin.nlm.nih.gov/cas/v1/tickets" 11 | tgtserverurl = "https://utslogin.nlm.nih.gov/cas/v1/api-key" 12 | serviceurl = \ 13 | 'https://ii.nlm.nih.gov/cgi-bin/II/UTS_Required/API_batchValidationII.pl' 14 | parser = argparse.ArgumentParser(description="test cas auth") 15 | parser.add_argument('-e', '--email', help='Email address') 16 | parser.add_argument('-a', '--apikey', help='UTS api key') 17 | parser.add_argument('-c', '--cmd', default='semrep', 18 | help='batch command') 19 | parser.add_argument('-r', '--cmdargs', 20 | default='-D', 21 | help='batch command arguments') 22 | parser.add_argument('inputfile', help='inputfile') 23 | parser.add_argument('-s', '--serviceurl', 24 | default=serviceurl, 25 | help='url of service') 26 | args = parser.parse_args() 27 | 28 | print("cmd: {}".format(args.cmd)) 29 | print("cmdargs: {}".format(unescape_string(args.cmdargs))) 30 | inst = Submission(args.email, args.apikey) 31 | if args.serviceurl: 32 | inst.set_serviceurl(args.serviceurl) 33 | inst.init_generic_batch(args.cmd, unescape_string(args.cmdargs)) 34 | inst.set_batch_file(args.inputfile) 35 | response = inst.submit() 36 | print('response status: {}'.format(response.status_code)) 37 | print('content: {}'.format(response.content.decode())) 38 | -------------------------------------------------------------------------------- /examples/mm_interactive.py: -------------------------------------------------------------------------------- 1 | """ MetaMap interactive using embedded string """ 2 | import argparse 3 | from skr_web_api import Submission, METAMAP_INTERACTIVE_URL 4 | 5 | if __name__ == '__main__': 6 | parser = argparse.ArgumentParser(description="test cas auth") 7 | parser.add_argument('-s', '--serviceurl', 8 | default=METAMAP_INTERACTIVE_URL, 9 | help='url of service') 10 | parser.add_argument('-e', '--email', help='Email address') 11 | parser.add_argument('-a', '--apikey', help='UTS api key') 12 | args = parser.parse_args() 13 | inputtext = "A spinal tap was performed and oligoclonal bands were \ 14 | detected in the cerebrospinal fluid.\n" 15 | inst = Submission(args.email, args.apikey) 16 | if args.serviceurl: 17 | inst.set_serviceurl(args.serviceurl) 18 | inst.init_mm_interactive(inputtext, args='-N') 19 | response = inst.submit() 20 | print('response status: {}'.format(response.status_code)) 21 | print('content: {}'.format(response.content.decode())) 22 | -------------------------------------------------------------------------------- /examples/mti_interactive.py: -------------------------------------------------------------------------------- 1 | """ MTI interactive """ 2 | import argparse 3 | import os 4 | from skr_web_api import Submission, SEMREP_INTERACTIVE_URL 5 | 6 | if __name__ == '__main__': 7 | parser = argparse.ArgumentParser(description="test cas auth") 8 | parser.add_argument('-s', '--serviceurl', 9 | default=SEMREP_INTERACTIVE_URL, 10 | help='url of service') 11 | parser.add_argument('-e', '--email', help='Email address') 12 | parser.add_argument('-a', '--apikey', help='UTS api key') 13 | args = parser.parse_args() 14 | if args.email is None and 'EMAIL' in os.environ: 15 | args.email = os.environ['EMAIL'] 16 | if args.apikey is None and 'UTS_API_KEY' in os.environ: 17 | args.apikey = os.environ['UTS_API_KEY'] 18 | 19 | inputtext = "A spinal tap was performed and oligoclonal bands \ 20 | were detected in the cerebrospinal fluid.\n" 21 | 22 | inst = Submission(args.email, args.apikey) 23 | if args.serviceurl: 24 | inst.set_serviceurl(args.serviceurl) 25 | inst.init_mti_interactive(inputtext, args='-opt1L_DCMS') 26 | response = inst.submit() 27 | print('response status: {}'.format(response.status_code)) 28 | print('content: {}'.format(response.content.decode())) 29 | -------------------------------------------------------------------------------- /examples/sr_interactive.py: -------------------------------------------------------------------------------- 1 | """ SemRep interactive 2 | """ 3 | import argparse 4 | import os 5 | from skr_web_api import Submission, SEMREP_INTERACTIVE_URL 6 | 7 | if __name__ == '__main__': 8 | parser = argparse.ArgumentParser(description="test cas auth") 9 | parser.add_argument('-s', '--serviceurl', 10 | default=SEMREP_INTERACTIVE_URL, 11 | help='url of service') 12 | parser.add_argument('-e', '--email', help='Email address') 13 | parser.add_argument('-a', '--apikey', help='UTS api key') 14 | args = parser.parse_args() 15 | if args.email is None and 'EMAIL' in os.environ: 16 | args.email = os.environ['EMAIL'] 17 | if args.apikey is None and 'UTS_API_KEY' in os.environ: 18 | args.apikey = os.environ['UTS_API_KEY'] 19 | 20 | inputtext = "A spinal tap was performed and oligoclonal bands \ 21 | were detected in the cerebrospinal fluid.\n" 22 | 23 | inst = Submission(args.email, args.apikey) 24 | if args.serviceurl: 25 | inst.set_serviceurl(args.serviceurl) 26 | inst.init_sr_interactive(inputtext, args='-D') 27 | response = inst.submit() 28 | print('response status: {}'.format(response.status_code)) 29 | print('content: {}'.format(response.content.decode())) 30 | -------------------------------------------------------------------------------- /examples/sr_interactive_file.py: -------------------------------------------------------------------------------- 1 | """ SemRep interactive using input file """ 2 | import argparse 3 | import os 4 | from skr_web_api import Submission, SEMREP_INTERACTIVE_URL 5 | 6 | if __name__ == '__main__': 7 | parser = argparse.ArgumentParser(description="test cas auth") 8 | parser.add_argument('-s', '--serviceurl', 9 | default=SEMREP_INTERACTIVE_URL, 10 | help='url of service') 11 | parser.add_argument('-e', '--email', help='Email address') 12 | parser.add_argument('-a', '--apikey', help='UTS api key') 13 | parser.add_argument('inputfile', help='inputfile') 14 | args = parser.parse_args() 15 | if args.email is None and 'EMAIL' in os.environ: 16 | args.email = os.environ['EMAIL'] 17 | if args.apikey is None and 'UTS_API_KEY' in os.environ: 18 | args.apikey = os.environ['UTS_API_KEY'] 19 | 20 | with open(args.inputfile) as chan: 21 | inputtext = chan.read() 22 | 23 | inst = Submission(args.email, args.apikey) 24 | if args.serviceurl: 25 | inst.set_serviceurl(args.serviceurl) 26 | inst.init_sr_interactive(inputtext, args='-D') 27 | response = inst.submit() 28 | print('response status: {}'.format(response.status_code)) 29 | print('content: {}'.format(response.content.decode())) 30 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [build-system] 2 | requires = ["setuptools>=42"] 3 | build-backend = "setuptools.build_meta" 4 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | requests>=2.27.1 2 | requests-html>=0.10.0 3 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | name = skr_web_api 3 | version = 0.1 4 | description = SKR_Web_API implementation 5 | author = Willie Rogers 6 | author_email = wjrogers@mail.nih.gov 7 | url = https://lhncbc.nlm.nih.gov/ii/tools/Web_API_Access.html 8 | classifiers = 9 | Programming Language :: Python :: 3 10 | License :: Public Domain 11 | Operating System :: OS Independent 12 | 13 | [options] 14 | package_dir = 15 | = src 16 | packages = find: 17 | python_requires = >=3.6 18 | 19 | [options.packages.find] 20 | where = src 21 | -------------------------------------------------------------------------------- /src/skr_web_api/__init__.py: -------------------------------------------------------------------------------- 1 | import os.path 2 | import requests 3 | from skr_web_api.casauth import get_ticket 4 | 5 | CAS_SERVERURL = "https://utslogin.nlm.nih.gov/cas/v1" 6 | II_SKR_SERVERURL = 'https://ii.nlm.nih.gov/cgi-bin/II/UTS_Required' 7 | BATCH_VALIDATION_URL = II_SKR_SERVERURL + '/API_batchValidationII.pl' 8 | SEMREP_INTERACTIVE_URL = II_SKR_SERVERURL + "/API_SR_interactive.pl" 9 | METAMAP_INTERACTIVE_URL = II_SKR_SERVERURL + "/API_MM_interactive.pl" 10 | MTI_INTERACTIVE_URL = II_SKR_SERVERURL + "/API_MTI_interactive.pl" 11 | 12 | 13 | class Submission: 14 | def __init__(self, email=None, apikey=None): 15 | self.form = {} 16 | self.form['COMMAND_ARGS'] = "" 17 | self.form['Email_Address'] = email 18 | self.apikey = apikey 19 | self.files = {} 20 | self.casserverurl = CAS_SERVERURL 21 | # ticket granting ticket server 22 | self.tgtserverurl = CAS_SERVERURL + "/api-key" 23 | # service ticket server 24 | self.stserverurl = CAS_SERVERURL + "/tickets" 25 | 26 | def set_casserverurl(self, cas_serverurl): 27 | """ set CAS server url""" 28 | self.casserverurl = cas_serverurl 29 | 30 | def set_stserverurl(self, cas_serverurl): 31 | """ set service ticket server url""" 32 | self.stserverurl = cas_serverurl 33 | 34 | def set_tgtserverurl(self, tgt_serverurl): 35 | """ set ticket granting ticket server url""" 36 | self.tgtserverurl = tgt_serverurl 37 | 38 | def set_serviceurl(self, serviceurl): 39 | """ set target service url """ 40 | self.serviceurl = serviceurl 41 | 42 | def init_generic_batch(self, command, command_args): 43 | self.serviceurl = BATCH_VALIDATION_URL 44 | self.form["Batch_Command"] = '{} {}'.format(command, command_args) 45 | self.form['COMMAND_ARGS'] = command_args 46 | self.form['RUN_PROG'] = 'GENERIC_V' 47 | self.form['SKR_API'] = 'true' 48 | 49 | def init_sr_interactive(self, inputtext, args='-D'): 50 | self.serviceurl = SEMREP_INTERACTIVE_URL 51 | self.form['APIText'] = inputtext 52 | self.form['COMMAND_ARGS'] = args 53 | 54 | def init_mm_interactive(self, inputtext, args='-N', ksource='2020AB'): 55 | self.serviceurl = METAMAP_INTERACTIVE_URL 56 | self.form['APIText'] = inputtext 57 | self.form['COMMAND_ARGS'] = args 58 | self.form['KSOURCE'] = ksource 59 | 60 | def init_mti_interactive(self, inputtext, args='-opt1L_DCMS'): 61 | self.serviceurl = MTI_INTERACTIVE_URL 62 | self.form['APIText'] = inputtext 63 | self.form['COMMAND_ARGS'] = args 64 | 65 | def set_mm_ksource(self, ksrelease): 66 | """ set UMLS Knowledge source release (e.g.: 2020AB, etc.) """ 67 | self.form['KSOURCE'] = ksrelease 68 | 69 | def set_command_args(self, args): 70 | """set arguments for command """ 71 | self.form["Batch_Command"] = '{} {}'.format( 72 | self.form["Batch_Command"], args) 73 | self.form['COMMAND_ARGS'] = args 74 | 75 | def set_batch_file(self, inputfilename, inputtext=None): 76 | """ set input file """ 77 | if inputtext is None: 78 | with open(inputfilename) as chan: 79 | inputtext = chan.read() 80 | inputfilename = os.path.basename(inputfilename) 81 | self.files['UpLoad_File'] = (inputfilename, inputtext, 82 | 'text/plain', {'Expires': '0'}) 83 | 84 | def set_email(self, email): 85 | self.form['Email_Address'] = email 86 | 87 | def set_apikey(self, apikey): 88 | self.apikey = apikey 89 | 90 | def submit(self): 91 | serviceticket = get_ticket(self.casserverurl, self.apikey, 92 | self.serviceurl) 93 | params = {'ticket': serviceticket} 94 | headers = {'Accept': 'application/json'} 95 | s = requests.Session() 96 | response = s.post(self.serviceurl, 97 | self.form, files=self.files, 98 | headers=headers, params=params, 99 | allow_redirects=False) 100 | # handle the redirect manually 101 | if response.status_code == 302: 102 | newurl = s.get_redirect_target(response) 103 | response = s.post(newurl, 104 | self.form, files=self.files, 105 | headers=headers, params=params, 106 | allow_redirects=False) 107 | return response 108 | -------------------------------------------------------------------------------- /src/skr_web_api/casauth.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import requests 3 | from requests_html import HTML 4 | 5 | 6 | def get_service_ticket(serverurl, ticket_granting_ticket, serviceurl): 7 | """ Obtain a Single-Use Proxy Ticket (also known as service ticket). 8 | Request for a Service Ticket: 9 | 10 | POST /cas/v1/tickets/{TGT id} HTTP/1.0 11 | 12 | data: 13 | service={form encoded parameter for the service url} 14 | 15 | Sucessful Response: 16 | 17 | 200 OK 18 | 19 | ST-1-FFDFHDSJKHSDFJKSDHFJKRUEYREWUIFSD2132 20 | 21 | @param serverurl authentication server 22 | @param ticketGrantingTicket a Proxy Granting Ticket. 23 | @param serviceurl url of service with protected resources 24 | @return authentication ticket for service. """ 25 | resp = requests.post("{}/{}".format(serverurl, ticket_granting_ticket), 26 | {"service": serviceurl}) 27 | if resp.status_code == 200: 28 | return resp.content 29 | return 'Error: status: {}'.format(resp.content) 30 | 31 | 32 | def extract_tgt_ticket(htmlcontent): 33 | "Extract ticket granting ticket from HTML." 34 | # print('htmlcontent: {}'.format(htmlcontent)) 35 | html = HTML(html=htmlcontent) 36 | # get form element 37 | elements = html.xpath("//form") 38 | # print('html response: {}'.format(etree.tostring(html.lxml).decode())) 39 | # print('action attribure: {}'.format(elements[0].attrs['action'])) 40 | # extract ticket granting ticket out of 'action' attribute 41 | if elements != []: 42 | return elements[0].attrs['action'].split('/')[-1] 43 | else: 44 | return "form element missing from ticket granting ticket response" 45 | 46 | 47 | def get_ticket_granting_ticket(tgtserverurl, apikey): 48 | """ Obtain a Proxy Granting Ticket. 49 | Response for a Ticket Granting Ticket Resource 50 | 51 | POST /cas/v1/api-key HTTP/1.0 52 | 53 | data: 54 | apikey 55 | 56 | Successful Response: 57 | 201 Created 58 | 59 | Location: http://serviceurl/cas/v1/tickets/{TGT id} 60 | 61 | Parameters: 62 | serverurl: authentication server 63 | apikey: UTS profile API key 64 | 65 | Returns: 66 | a Proxy Granting Ticket. 67 | """ 68 | response = requests.post(tgtserverurl, {'apikey': apikey}, 69 | headers={'Accept': 'test/plain'}) 70 | return extract_tgt_ticket(response.content) 71 | 72 | 73 | def get_ticket(cas_serverurl, apikey, serviceurl): 74 | """Obtain a Single-Use Proxy Ticket from Central Authentication 75 | Server (CAS). 76 | 77 | Parameters: 78 | stserverurl: service ticket server 79 | tgtserverurl: ticket granting ticket server 80 | apikey: UTS profile API key 81 | serviceurl: url of service with protected resources 82 | 83 | Returns: 84 | authentication ticket for service. 85 | 86 | """ 87 | if cas_serverurl is None: 88 | print("cas server url must not be None") 89 | if apikey is None: 90 | print("api key must not be null") 91 | if serviceurl is None: 92 | print("service must not be null") 93 | # set ticket granting ticket server url 94 | tgtserverurl = cas_serverurl + "/api-key" 95 | # set service ticket server url 96 | stserverurl = cas_serverurl + "/tickets" 97 | tgt = get_ticket_granting_ticket(tgtserverurl, apikey) 98 | return get_service_ticket(stserverurl, tgt, serviceurl) 99 | 100 | 101 | def get_protected_document(service_url, serviceticket): 102 | """ get document protected by CAS Authentication. """ 103 | url = '%s?ticket=%s' % (service_url, serviceticket) 104 | params = {'ticket': serviceticket} 105 | s = requests.Session() 106 | response = s.get(url, params=params) 107 | # handle the redirect manually 108 | if response.status_code == 302: 109 | newurl = s.get_redirect_target(response) 110 | response = s.get(newurl, params=params, allow_redirects=False) 111 | return response 112 | 113 | 114 | def get_document(cas_baseurl, apikey, service_url): 115 | """ get document protected by CAS Authentication. """ 116 | serviceticket = get_ticket(cas_baseurl, apikey, service_url) 117 | return get_protected_document(service_url, serviceticket) 118 | 119 | if __name__ == '__main__': 120 | parser = argparse.ArgumentParser(description="test cas auth") 121 | parser.add_argument('-s', '--serviceurl', 122 | help='url of service') 123 | parser.add_argument('apikey', help='UTS api key') 124 | args = parser.parse_args() 125 | stserverurl = "https://utslogin.nlm.nih.gov/cas/v1/tickets" 126 | tgtserverurl = "https://utslogin.nlm.nih.gov/cas/v1/api-key" 127 | ticket = get_ticket(stserverurl, tgtserverurl, 128 | args.apikey, args.serviceurl) 129 | --------------------------------------------------------------------------------