├── eodms_rapi ├── __version__.py ├── __init__.py ├── query_error.py ├── rapi_requests.py └── geo.py ├── requirements.txt ├── docs ├── download.rst ├── eodms_rapi.rst ├── initialization.rst ├── Makefile ├── make.bat ├── about.rst ├── conf.py ├── installation.rst ├── index.rst ├── order.rst ├── examples.rst └── search-rapi.rst ├── .readthedocs.yml ├── setup.cfg ├── .gitignore ├── LICENSE ├── setup.py ├── .github └── workflows │ ├── check_version.yml │ └── test_pyeodmsrapi.yml ├── README.md └── test └── test_pyeodmsrapi.py /eodms_rapi/__version__.py: -------------------------------------------------------------------------------- 1 | __version__ = "1.10.4" -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | requests>=2.23.0 2 | tqdm>=4.59.0 3 | dateparser>=1.0.0 4 | lxml>=5.2.1 5 | py-eodms-rapi -------------------------------------------------------------------------------- /docs/download.rst: -------------------------------------------------------------------------------- 1 | 2 | Download Images 3 | =============== 4 | 5 | The *download* method of the EODMSRAPI requires: 6 | 7 | 8 | * Either the **order results** from the *order* method or a list of **Order Item IDs**. 9 | * A **local destination path** where the images will be downloaded. 10 | 11 | .. code-block:: python 12 | 13 | dest = "C:\\TEMP" 14 | dn_res = rapi.download(order_res, dest) 15 | -------------------------------------------------------------------------------- /docs/eodms_rapi.rst: -------------------------------------------------------------------------------- 1 | eodms\_rapi package 2 | =================== 3 | 4 | 5 | eodms\_rapi.eodms module 6 | ------------------------ 7 | 8 | .. automodule:: eodms_rapi.eodms 9 | :members: 10 | :undoc-members: 11 | :show-inheritance: 12 | 13 | eodms\_rapi.geo module 14 | ---------------------- 15 | 16 | .. automodule:: eodms_rapi.geo 17 | :members: 18 | :undoc-members: 19 | :show-inheritance: 20 | -------------------------------------------------------------------------------- /eodms_rapi/__init__.py: -------------------------------------------------------------------------------- 1 | __title__ = 'py-eodms-rapi' 2 | __name__ = 'eodms_rapi' 3 | __author__ = 'Kevin Ballantyne' 4 | __copyright__ = 'Copyright (c) His Majesty the King in Right of Canada, ' \ 5 | 'as represented by the Minister of Natural Resources, 2025' 6 | __license__ = 'MIT License' 7 | __description__ = 'A Python package to access the EODMS RAPI service.' 8 | __maintainer__ = 'Kevin Ballantyne' 9 | __email__ = 'eodms-sgdot@nrcan-rncan.gc.ca' 10 | 11 | from .__version__ import __version__ 12 | from .eodms import EODMSRAPI 13 | from .eodms import QueryError 14 | from .geo import EODMSGeo -------------------------------------------------------------------------------- /.readthedocs.yml: -------------------------------------------------------------------------------- 1 | # .readthedocs.yml 2 | # Read the Docs configuration file 3 | # See https://docs.readthedocs.io/en/stable/config-file/v2.html for details 4 | 5 | # Required 6 | version: 2 7 | 8 | # Build documentation in the docs/ directory with Sphinx 9 | sphinx: 10 | configuration: docs/conf.py 11 | builder: html 12 | fail_on_warning: false 13 | 14 | # Optionally build your docs in additional formats such as PDF and ePub 15 | formats: all 16 | 17 | # Optionally set the version of Python and requirements required to build your docs 18 | python: 19 | version: 3.7 20 | install: 21 | - requirements: requirements.txt -------------------------------------------------------------------------------- /docs/initialization.rst: -------------------------------------------------------------------------------- 1 | 2 | Initializing the EODMSRAPI 3 | ================================================== 4 | 5 | The EODMSRAPI class is the object which contains the methods and functions used to access the EODMS REST API service. 6 | 7 | .. code-block:: python 8 | 9 | from eodms_rapi import EODMSRAPI 10 | 11 | Initialization of the EODMSRAPI requires entry of a password from a valid EODMS account. 12 | 13 | .. note:: 14 | If you do not have an EODMS account, please visit https://www.eodms-sgdot.nrcan-rncan.gc.ca/index-en.html and click the **Register (Required to Order)** link under **Account**. 15 | 16 | .. code-block:: python 17 | 18 | rapi = EODMSRAPI('eodms-username', 'eodms-password') 19 | 20 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Minimal makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line, and also 5 | # from the environment for the first two. 6 | SPHINXOPTS ?= 7 | SPHINXBUILD ?= sphinx-build 8 | SOURCEDIR = . 9 | BUILDDIR = _build 10 | 11 | # Put it first so that "make" without argument is like "make help". 12 | help: 13 | @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 14 | 15 | .PHONY: help Makefile 16 | 17 | # Catch-all target: route all unknown targets to Sphinx using the new 18 | # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). 19 | %: Makefile 20 | @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 21 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | name = py-eodms-rapi 3 | version = 1.10.4 4 | author = Kevin Ballantyne (Natural Resources Canada) 5 | author_email = kevin.ballantyne@nrcan-rncan.gc.ca 6 | description = EODMS RAPI Client is a Python3 package used to access the REST API service provided by the Earth Observation Data Management System (EODMS) from Natural Resources Canada. 7 | description_file = README.md 8 | long_description = file: README.md 9 | long_description_content_type = text/markdown 10 | url = https://py-eodms-rapi.readthedocs.io/en/latest/ 11 | project_urls = 12 | Source = https://github.com/eodms-sgdot/py-eodms-rapi 13 | Bug Tracker = https://github.com/eodms-sgdot/py-eodms-rapi/issues 14 | classifiers = 15 | Programming Language :: Python :: 3 16 | License :: OSI Approved :: MIT License 17 | Operating System :: OS Independent -------------------------------------------------------------------------------- /docs/make.bat: -------------------------------------------------------------------------------- 1 | @ECHO OFF 2 | 3 | pushd %~dp0 4 | 5 | REM Command file for Sphinx documentation 6 | 7 | if "%SPHINXBUILD%" == "" ( 8 | set SPHINXBUILD=sphinx-build 9 | ) 10 | set SOURCEDIR=. 11 | set BUILDDIR=_build 12 | 13 | if "%1" == "" goto help 14 | 15 | %SPHINXBUILD% >NUL 2>NUL 16 | if errorlevel 9009 ( 17 | echo. 18 | echo.The 'sphinx-build' command was not found. Make sure you have Sphinx 19 | echo.installed, then set the SPHINXBUILD environment variable to point 20 | echo.to the full path of the 'sphinx-build' executable. Alternatively you 21 | echo.may add the Sphinx directory to PATH. 22 | echo. 23 | echo.If you don't have Sphinx installed, grab it from 24 | echo.http://sphinx-doc.org/ 25 | exit /b 1 26 | ) 27 | 28 | %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% 29 | goto end 30 | 31 | :help 32 | %SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% 33 | 34 | :end 35 | popd 36 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | eodms_rapi/__pycache__/ 4 | *.py[cod] 5 | *$py.class 6 | 7 | # Distribution / packaging 8 | _deploy_testpypi.bat 9 | _deploy_pypi.bat 10 | _build.bat 11 | .Python 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | dist_backup/ 16 | old_dist/ 17 | downloads/ 18 | eggs/ 19 | .eggs/ 20 | lib/ 21 | lib64/ 22 | parts/ 23 | sdist/ 24 | var/ 25 | wheels/ 26 | share/python-wheels/ 27 | *.egg-info/ 28 | .installed.cfg 29 | *.egg 30 | MANIFEST 31 | 32 | # Sphinx documentation 33 | docs/_build/ 34 | docs/_pyexamples 35 | docs/build/ 36 | docs/source/ 37 | docs/make.bat 38 | docs/Makefile 39 | 40 | # Environments 41 | .env 42 | .venv 43 | env/ 44 | venv/ 45 | ENV/ 46 | env.bak/ 47 | venv.bak/ 48 | 49 | # Tests 50 | test/test_pyeodmsrapi.bat 51 | test/__pycache__ 52 | test/*.bat 53 | test/files/ 54 | test/venv/ 55 | 56 | # user specific overrides 57 | _update_repository.bat 58 | eodms_rapi/backup*/ 59 | eodms_rapi/*.bak 60 | old_docs/ 61 | open_env.ps1 62 | *.bak -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) His Majesty the King in Right of Canada, as 4 | represented by the Minister of Natural Resources, 2025. 5 | 6 | Permission is hereby granted, free of charge, to any person obtaining a 7 | copy of this software and associated documentation files (the "Software"), 8 | to deal in the Software without restriction, including without limitation 9 | the rights to use, copy, modify, merge, publish, distribute, sublicense, 10 | and/or sell copies of the Software, and to permit persons to whom the 11 | Software is furnished to do so, subject to the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be included in 14 | all copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 17 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 18 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 19 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 20 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 21 | FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 22 | DEALINGS IN THE SOFTWARE. -------------------------------------------------------------------------------- /eodms_rapi/query_error.py: -------------------------------------------------------------------------------- 1 | class QueryError: 2 | """ 3 | The QueryError class is used to store error information for a query. 4 | """ 5 | 6 | def __init__(self, msgs): 7 | """ 8 | Initializer for QueryError object which stores an error message. 9 | 10 | :param msgs: The error message to print. 11 | :type msgs: str 12 | """ 13 | 14 | self.msgs = msgs 15 | 16 | def get_msgs(self, as_str=False): 17 | """ 18 | Gets the messages stored with the QueryError. 19 | 20 | :param as_str: Determines whether to return a string or a list of 21 | messages. 22 | :type as_str: boolean 23 | 24 | :return: Either a string or a list of messages. 25 | :rtype: str or list 26 | """ 27 | 28 | if as_str: 29 | if isinstance(self.msgs, list): 30 | return ' '.join(filter(None, self.msgs)) 31 | 32 | return self.msgs 33 | 34 | def _set_msgs(self, msgs): 35 | """ 36 | Sets the messages stored with the QueryError. 37 | 38 | :param msgs: Can either be a string or a list of messages. 39 | :type msgs: str or list 40 | 41 | """ 42 | self.msgs = msgs -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import find_packages, setup 2 | 3 | setup( 4 | name='py-eodms-rapi', 5 | version='1.10.4', 6 | author='Kevin Ballantyne (Natural Resources Canada)', 7 | author_email='kevin.ballantyne@nrcan-rncan.gc.ca', 8 | packages=find_packages(), 9 | include_package_data=True, 10 | url='https://py-eodms-rapi.readthedocs.io/en/latest/', 11 | license='LICENSE', 12 | description='EODMS RAPI Client is a Python3 package used to access the ' \ 13 | 'REST API service provided by the Earth Observation Data ' \ 14 | 'Management System (EODMS) from Natural Resources Canada.', 15 | long_description=open('README.md').read(), 16 | long_description_content_type="text/markdown", 17 | install_requires=[ 18 | "dateparser", 19 | "requests", 20 | "tqdm", 21 | "geomet", 22 | "lxml", 23 | ], 24 | project_urls={ 25 | "Source": "https://github.com/eodms-sgdot/py-eodms-rapi", 26 | "Bug Tracker": "https://github.com/eodms-sgdot/py-eodms-rapi/issues", 27 | }, 28 | classifiers=[ 29 | "Programming Language :: Python :: 3", 30 | "License :: OSI Approved :: MIT License", 31 | "Operating System :: OS Independent", 32 | ], 33 | python_requires='>=3.6', 34 | ) -------------------------------------------------------------------------------- /docs/about.rst: -------------------------------------------------------------------------------- 1 | 2 | About 3 | ===== 4 | 5 | Acknowledgements 6 | ---------------- 7 | 8 | Some code in this package is based off the `EODMS API Client `_ designed by Mike Brady. 9 | 10 | Contact 11 | ------- 12 | 13 | If you have any questions or require support, please contact the EODMS Support Team at `eodms-sgdot@nrcan-rncan.gc.ca `_. 14 | 15 | License 16 | ------- 17 | 18 | MIT License 19 | 20 | Copyright (c) His Majesty the King in Right of Canada, as 21 | represented by the Minister of Natural Resources, 2024. 22 | 23 | Permission is hereby granted, free of charge, to any person obtaining a 24 | copy of this software and associated documentation files (the "Software"), 25 | to deal in the Software without restriction, including without limitation 26 | the rights to use, copy, modify, merge, publish, distribute, sublicense, 27 | and/or sell copies of the Software, and to permit persons to whom the 28 | Software is furnished to do so, subject to the following conditions: 29 | 30 | The above copyright notice and this permission notice shall be included in 31 | all copies or substantial portions of the Software. 32 | 33 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 34 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 35 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 36 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 37 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 38 | FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 39 | DEALINGS IN THE SOFTWARE. 40 | -------------------------------------------------------------------------------- /.github/workflows/check_version.yml: -------------------------------------------------------------------------------- 1 | 2 | name: py-eodms-rapi Version Check 3 | 4 | # Controls when the workflow will run 5 | on: 6 | 7 | # Allows you to run this workflow manually from the Actions tab 8 | workflow_dispatch: 9 | 10 | # A workflow run is made up of one or more jobs that can run sequentially or in parallel 11 | jobs: 12 | # This workflow contains a single job called "build" 13 | build: 14 | # The type of runner that the job will run on 15 | runs-on: ubuntu-latest 16 | 17 | # Steps represent a sequence of tasks that will be executed as part of the job 18 | steps: 19 | # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it 20 | - uses: actions/checkout@v3 21 | with: 22 | ref: 'main' 23 | 24 | # List files 25 | - name: List files 26 | run: ls 27 | 28 | # Install package from setup.py 29 | - name: Install package from setup.py 30 | run: | 31 | sudo pip install -e . 32 | sudo pip install -r requirements.txt 33 | 34 | # Runs a single command using the runners shell 35 | - name: Run py-eodms-rapi Version Check 36 | # continue-on-error: true 37 | run: | 38 | SETUP_VERSION=`python setup.py --version` 39 | CFG_VERSION=`python -c 'from setuptools.config.setupcfg import read_configuration as c; print(c("setup.cfg")["metadata"]["version"])'` 40 | PY_VERSION=`python -c 'from eodms_rapi import __version__; print(__version__)'` 41 | 42 | if [[ "$SETUP_VERSION" == "$CFG_VERSION" ]] && [[ "$CFG_VERSION" == "$PY_VERSION" ]]; then 43 | echo "All versions are equal" 44 | else 45 | echo "At least one version is not equal" 46 | echo "setup.py: ${SETUP_VERSION}" 47 | echo "setup.cfg: ${CFG_VERSION}" 48 | echo "__version__.py: ${PY_VERSION}" 49 | exit 1 50 | fi 51 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | EODMS RAPI Client 2 | ================= 3 | 4 | ## Overview 5 | 6 | EODMS RAPI Client is a Python3 package used to access the REST API service provided by the [Earth Observation Data Management System (EODMS)](https://www.eodms-sgdot.nrcan-rncan.gc.ca/index-en.html) from Natural Resources Canada. 7 | 8 | ## Installation 9 | 10 | The package can be installed using pip: 11 | 12 | ```bash 13 | pip install py-eodms-rapi -U 14 | ``` 15 | 16 | ## Basic Usage 17 | 18 | Here are a few examples on how to use the EODMS RAPI Client (```EODMSRAPI```). For full documentation, visit [https://py-eodms-rapi.readthedocs.io/en/latest/](https://py-eodms-rapi.readthedocs.io/en/latest/) 19 | 20 | ### Search for Images 21 | 22 | ```python 23 | from eodms_rapi import EODMSRAPI 24 | 25 | # Create the EODMSRAPI object 26 | rapi = EODMSRAPI('eodms-username', 'eodms-password') 27 | 28 | # Add a point to the search 29 | feat = [('intersects', "POINT (-96.47 62.4)")] 30 | 31 | # Create a dictionary of query filters for the search 32 | filters = {'Beam Mnemonic': ('=', ['16M11', '16M13']), 33 | 'Incidence Angle': ('>=', '35')} 34 | 35 | # Set a date range for the search 36 | dates = [{"start": "20200513_120000", "end": "20200613_150000"}] 37 | 38 | # Submit the search to the EODMSRAPI, specifying the Collection 39 | rapi.search("RCMImageProducts", filters=filters, features=feat, dates=dates) 40 | 41 | # Get the results from the search 42 | res = rapi.get_results('full') 43 | 44 | # Print results 45 | rapi.print_results() 46 | ``` 47 | 48 | ### Order and Download Images 49 | 50 | Using the results from the previous example: 51 | 52 | ```python 53 | # Submit an order using results 54 | order_res = rapi.order(res) 55 | 56 | # Specify a folder location to download the images 57 | dest = "C:\\temp\\py_eodms_rapi" 58 | 59 | # Download the images from the order 60 | dn_res = rapi.download(order_res, dest) 61 | ``` 62 | 63 | ## Acknowledgements 64 | 65 | Some code in this package is based off the [EODMS API Client](https://pypi.org/project/eodms-api-client/) designed by Mike Brady. 66 | 67 | ## Contact 68 | 69 | If you have any questions or require support, please contact the EODMS Support Team at eodms-sgdot@nrcan-rncan.gc.ca. 70 | 71 | ## License 72 | 73 | MIT License 74 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | # Configuration file for the Sphinx documentation builder. 2 | # 3 | # This file only contains a selection of the most common options. For a full 4 | # list see the documentation: 5 | # https://www.sphinx-doc.org/en/master/usage/configuration.html 6 | 7 | # -- Path setup -------------------------------------------------------------- 8 | 9 | # If extensions (or modules to document with autodoc) are in another directory, 10 | # add these directories to sys.path here. If the directory is relative to the 11 | # documentation root, use os.path.abspath to make it absolute, like shown here. 12 | # 13 | import os 14 | import sys 15 | import re 16 | sys.path.insert(0, os.path.abspath('..')) 17 | 18 | # -- Project information ----------------------------------------------------- 19 | 20 | project = 'py-eodms-rapi' 21 | copyright = '2022, Kevin Ballantyne (Natural Resources Canada)' 22 | author = 'Kevin Ballantyne (Natural Resources Canada)' 23 | 24 | # The full version, including alpha/beta/rc tags 25 | with open(os.path.join('..', 'eodms_rapi', '__init__.py'), 'r') as f: 26 | init_txt = f.read() 27 | release = re.search(r"__version__ = \s*'([\d.*]+)'", init_txt).group(1) 28 | 29 | 30 | # -- General configuration --------------------------------------------------- 31 | 32 | # Add any Sphinx extension module names here, as strings. They can be 33 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom 34 | # ones. 35 | # extensions = [] #'myst_parser', 'sphinx.ext.githubpages'] 36 | 37 | extensions = [ 38 | 'sphinx.ext.autodoc', 39 | 'sphinx.ext.autosummary', 40 | 'sphinx.ext.coverage', 41 | 'sphinx.ext.napoleon', 42 | 'sphinx.ext.viewcode' 43 | ] 44 | 45 | # Add any paths that contain templates here, relative to this directory. 46 | templates_path = ['_templates'] 47 | 48 | # List of patterns, relative to source directory, that match files and 49 | # directories to ignore when looking for source files. 50 | # This pattern also affects html_static_path and html_extra_path. 51 | exclude_patterns = [] 52 | 53 | 54 | # -- Options for HTML output ------------------------------------------------- 55 | 56 | pygments_theme = 'sphinx' 57 | 58 | # The theme to use for HTML and HTML Help pages. See the documentation for 59 | # a list of builtin themes. 60 | # 61 | html_theme = 'sphinx_rtd_theme' 62 | 63 | # Add any paths that contain custom static files (such as style sheets) here, 64 | # relative to this directory. They are copied after the builtin static files, 65 | # so a file named "default.css" will overwrite the builtin "default.css". 66 | html_static_path = ['_static'] 67 | -------------------------------------------------------------------------------- /docs/installation.rst: -------------------------------------------------------------------------------- 1 | 2 | Installation & Quick Guide 3 | ================================================== 4 | 5 | Installation 6 | ------------ 7 | 8 | The package is installed using the pip command 9 | 10 | .. code-block:: bash 11 | 12 | pip install pg-eodms-rapi 13 | 14 | The installation will also add the following packages: 15 | 16 | 17 | * `dateparser `_ 18 | * `Requests `_ 19 | * `tqdm `_ 20 | * `geomet `_ 21 | 22 | The package does not require the installation of the GDAL package. However, GDAL has to be installed if you wish to use ESRI Shapefiles. 23 | 24 | Initializing the EODMSRAPI 25 | -------------------------- 26 | 27 | The EODMSRAPI class is the object which contains the methods and functions used to access the EODMS REST API service. 28 | 29 | .. code-block:: python 30 | 31 | from eodms_rapi import EODMSRAPI 32 | 33 | Initialization of the EODMSRAPI requires entry of a password from a valid EODMS account. 34 | 35 | .. note:: 36 | If you do not have an EODMS account, please visit https://www.eodms-sgdot.nrcan-rncan.gc.ca/index-en.html and click the **Register (Required to Order)** link under **Account**. 37 | 38 | .. code-block:: python 39 | 40 | rapi = EODMSRAPI('eodms-username', 'eodms-password') 41 | 42 | Example Code 43 | ------------ 44 | 45 | An example to search, order and download RCM images: 46 | 47 | .. code-block:: python 48 | 49 | from eodms_rapi import EODMSRAPI 50 | 51 | # Initialize EODMSRAPI using your EODMS account credentials 52 | rapi = EODMSRAPI('eodms-username', 'eodms-password') 53 | 54 | # Set a polygon of geographic centre of Canada using GeoJSON 55 | feat = [('INTERSECTS', {"type":"Polygon", "coordinates":[[[-95.47,61.4],\ 56 | [-97.47,61.4],[-97.47,63.4],[-95.47,63.4],[-95.47,61.4]]]})] 57 | 58 | # Set date ranges 59 | dates = [{"start": "20190101_000000", "end": "20210621_000000"}] 60 | 61 | # Set search filters 62 | filters = {'Beam Mode Type': ('LIKE', ['%50m%']), 63 | 'Polarization': ('=', 'HH HV'), 64 | 'Incidence Angle': ('>=', 17)} 65 | 66 | # Set the results fields 67 | result_fields = ['ARCHIVE_RCM.PROCESSING_FACILITY', 'RCM.ANTENNA_ORIENTATION'] 68 | 69 | # Submit search 70 | rapi.search("RCMImageProducts", filters, feat, dates, result_fields, 2) 71 | 72 | # Get results 73 | rapi.set_field_convention('upper') 74 | res = rapi.get_results('full') 75 | 76 | # Now order the images 77 | order_res = rapi.order(res) 78 | 79 | # Download images to a specific destination 80 | dest = "C:\\TEMP" 81 | dn_res = rapi.download(order_res, dest) 82 | 83 | # Print results 84 | rapi.print_results(dn_res) 85 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | ======================================= 2 | EODMS RAPI Python Package Documentation 3 | ======================================= 4 | 5 | EODMS RAPI Client is a Python3 package used to access the REST API service provided by the `Earth Observation Data Management System (EODMS) `_ from Natural Resources Canada. 6 | 7 | This package requires Python 3.6 or higher (it was designed using Python 3.7). 8 | 9 | Installation 10 | ------------ 11 | 12 | The package is installed using the pip command 13 | 14 | .. code-block:: bash 15 | 16 | pip install py-eodms-rapi -U 17 | 18 | The installation will also add the following packages: 19 | 20 | 21 | * `dateparser `_ 22 | * `Requests `_ 23 | * `tqdm `_ 24 | * `geomet `_ 25 | 26 | The package does not require the installation of the GDAL package. However, GDAL has to be installed if you wish to use ESRI Shapefiles. 27 | 28 | Example Code 29 | ------------ 30 | 31 | An example to search, order and download RCM images: 32 | 33 | .. code-block:: python 34 | 35 | from eodms_rapi import EODMSRAPI 36 | 37 | # Initialize EODMSRAPI using your EODMS account credentials 38 | rapi = EODMSRAPI('eodms-username', 'eodms-password') 39 | 40 | # Set a polygon of geographic centre of Canada using GeoJSON 41 | feat = [('INTERSECTS', {"type":"Polygon", "coordinates":[[[-95.47,61.4],\ 42 | [-97.47,61.4],[-97.47,63.4],[-95.47,63.4],[-95.47,61.4]]]})] 43 | 44 | # Set date ranges 45 | dates = [{"start": "20190101_000000", "end": "20210621_000000"}] 46 | 47 | # Set search filters 48 | filters = {'Beam Mode Type': ('LIKE', ['%50m%']), 49 | 'Polarization': ('=', 'HH HV'), 50 | 'Incidence Angle': ('>=', 17)} 51 | 52 | # Set the results fields 53 | result_fields = ['ARCHIVE_RCM.PROCESSING_FACILITY', 'RCM.ANTENNA_ORIENTATION'] 54 | 55 | # Submit search 56 | rapi.search("RCMImageProducts", filters, feat, dates, result_fields, 2) 57 | 58 | # Get results 59 | rapi.set_field_convention('upper') 60 | res = rapi.get_results('full') 61 | 62 | # Now order the images 63 | order_res = rapi.order(res) 64 | 65 | # Download images to a specific destination 66 | dest = "C:\\TEMP" 67 | dn_res = rapi.download(order_res, dest) 68 | 69 | # Print results 70 | rapi.print_results(dn_res) 71 | 72 | # Clear search results 73 | rapi.clear_results() 74 | 75 | Contents 76 | -------- 77 | 78 | .. toctree:: 79 | :maxdepth: 3 80 | :caption: User Guide 81 | 82 | initialization 83 | search-rapi 84 | order 85 | download 86 | examples 87 | 88 | .. toctree:: 89 | :maxdepth: 4 90 | :caption: API 91 | 92 | eodms_rapi.rst 93 | 94 | Support 95 | ------- 96 | 97 | If you have any issues or questions, please contact the EODMS Support Team at `eodms-sgdot@nrcan-rncan.gc.ca `_. 98 | 99 | Acknowledgements 100 | ---------------- 101 | 102 | Some code in this package is based off the `EODMS API Client `_ designed by Mike Brady. 103 | 104 | License 105 | ------- 106 | 107 | Copyright (c) His Majesty the King in Right of Canada, as 108 | represented by the Minister of Natural Resources, 2024. 109 | 110 | Licensed under the MIT license 111 | (see LICENSE or ) All files in the 112 | project carrying such notice may not be copied, modified, or distributed 113 | except according to those terms. 114 | -------------------------------------------------------------------------------- /docs/order.rst: -------------------------------------------------------------------------------- 1 | 2 | Order Images 3 | ============ 4 | 5 | To order images using the RAPI, a POST request is submitted containing the following JSON (as an example): 6 | 7 | .. code-block:: json 8 | 9 | { 10 | "destinations": [], 11 | "items": [ 12 | { 13 | "collectionId": "RCMImageProducts", 14 | "recordId": "7822244", 15 | "parameters": [ 16 | { 17 | "packagingFormat": "TAR" 18 | }, 19 | { 20 | "NOTIFICATION_EMAIL_ADDRESS": "example@email.com" 21 | } 22 | ] 23 | } 24 | ] 25 | } 26 | 27 | So, ordering images using the EODMSRAPI requires a list of **results** (items) and optional **priority**\ , **parameters** and **destinations** values. 28 | 29 | Results 30 | ------- 31 | 32 | The **results** parameter can be a list of results returned from a search session or a list of items. The **results** is required. 33 | 34 | Each item must have: ``recordId`` ``collectionId`` 35 | 36 | Priority 37 | -------- 38 | 39 | The **priority** can be a single string entry ("Low", "Medium", "High", or "Urgent") which will be applied to all images or a list of dictionaries containing ``recordId`` and ``priority`` value for each individual image. The **priority** is optional and the default is "Medium". 40 | 41 | Parameters 42 | ---------- 43 | 44 | The **parameters** can be a list of parameter dictionaries which will be applied to all images or a list of dictionaries containing the ``recordId`` and ``parameters``. 45 | 46 | Each item in the ``parameters`` list should be the same as how it appears in the POST request (ex: ``{"packagingFormat": "TAR"}``\ ) 47 | 48 | You can get a list of available parameters by calling the ``get_order_parameters`` method of the EODMSRAPI, submitting arguments **collection** and **recordId**. The **parameters** is optional. 49 | 50 | Destinations 51 | ------------ 52 | 53 | The **destinations** is a list of destination dictionaries containing a set of items. There are 2 types of destinations, "FTP" and "Physical". 54 | 55 | The "FTP" dictionary would look something like this: 56 | 57 | .. code-block:: json 58 | 59 | { 60 | "type": "FTP", 61 | "name": "FTP Name", 62 | "hostname": "ftp://ftpsite.com", 63 | "username": "username", 64 | "password": "password", 65 | "stringValue": "ftp://username@ftpsite.com/downloads", 66 | "path": "downloads", 67 | "canEdit": "false" 68 | } 69 | 70 | The "Physical" dictionary would look like this: 71 | 72 | .. code-block:: json 73 | 74 | { 75 | "type": "Physical", 76 | "name": "Destination Name", 77 | "customerName": "John Doe", 78 | "contactEmail": "example@email.com", 79 | "organization": "Organization Name", 80 | "phone": "555-555-5555", 81 | "addr1": "123 Fake Street", 82 | "addr2": "Optional", 83 | "addr3": "Optional", 84 | "city": "Ottawa", 85 | "stateProv": "Ontario", 86 | "country": "Canada", 87 | "postalCode": "A1A 1A1", 88 | "classification": "Optional" 89 | } 90 | 91 | For more information on the destination items, visit `Directly Accessing the EODMS REST API `_. 92 | 93 | Example 94 | ------- 95 | 96 | Here's an example of how to submit an order to the EODMSRAPI using the previous search session: 97 | 98 | .. code-block:: python 99 | 100 | params = [{"packagingFormat": "TAR"}] 101 | 102 | order_res = rapi.order(res, priority="low", parameters=params) 103 | -------------------------------------------------------------------------------- /docs/examples.rst: -------------------------------------------------------------------------------- 1 | 2 | Examples 3 | ======== 4 | 5 | Search, Order and Download 6 | -------------------------- 7 | 8 | .. code-block:: python 9 | 10 | from eodms_rapi import EODMSRAPI 11 | 12 | # Initialize EODMSRAPI using your EODMS account credentials 13 | rapi = EODMSRAPI('eodms-username', 'eodms-password') 14 | 15 | # Set a polygon of geographic centre of Canada using GeoJSON 16 | feat = [('INTERSECTS', {"type":"Polygon", "coordinates":[[[-95.47,61.4],\ 17 | [-97.47,61.4],[-97.47,63.4],[-95.47,63.4],[-95.47,61.4]]]})] 18 | 19 | # Set date ranges 20 | dates = [{"start": "20190101_000000", "end": "20210621_000000"}] 21 | 22 | # Set search filters 23 | filters = {'Beam Mode Type': ('LIKE', ['%50m%']), 24 | 'Polarization': ('=', 'HH HV'), 25 | 'Incidence Angle': ('>=', 17)} 26 | 27 | # Set the results fields 28 | result_fields = ['ARCHIVE_RCM.PROCESSING_FACILITY', 'RCM.ANTENNA_ORIENTATION'] 29 | 30 | # Submit search 31 | rapi.search("RCMImageProducts", filters, feat, dates, result_fields, 2) 32 | 33 | # Get results 34 | rapi.set_field_convention('upper') 35 | res = rapi.get_results('full') 36 | 37 | # Now order the images 38 | order_res = rapi.order(res) 39 | 40 | # Download images to a specific destination 41 | dest = "C:\\TEMP" 42 | dn_res = rapi.download(order_res, dest) 43 | 44 | # Print results 45 | rapi.print_results(dn_res) 46 | 47 | Get Available Order Parameters for an Image 48 | ------------------------------------------- 49 | 50 | .. code-block:: python 51 | 52 | from eodms_rapi import EODMSRAPI 53 | 54 | # Initialize EODMSRAPI using EODMS account credentials 55 | rapi = EODMSRAPI('username', 'password') 56 | 57 | # Get the order parameters for RCM image with Record ID 7627902 58 | param_res = rapi.get_order_parameters('RCMImageProducts', '7627902') 59 | 60 | # Print the parameters 61 | print(f"param_res: {param_res}") 62 | 63 | Cancel an Existing Order Item 64 | ----------------------------- 65 | 66 | .. code-block:: python 67 | 68 | from eodms_rapi import EODMSRAPI 69 | 70 | # Initialize EODMSRAPI using your EODMS account credentials 71 | rapi = EODMSRAPI('eodms-username', 'eodms-password') 72 | 73 | # Cancel the order item with Order ID 48188 and Order Item ID 289377 74 | delete_res = rapi.cancel_order_item('48188', '289377') 75 | 76 | Get a List of Available Fields for a Collection 77 | ----------------------------------------------- 78 | 79 | .. code-block:: python 80 | 81 | from eodms_rapi import EODMSRAPI 82 | 83 | # Initialize EODMSRAPI using your EODMS account credentials 84 | rapi = EODMSRAPI('eodms-username', 'eodms-password') 85 | 86 | # Get the available field information for RCMImageProducts collection 87 | fields = rapi.get_available_fields('RCMImageProducts') 88 | print(fields) 89 | 90 | >>> {'search': {'Special Handling Required': {'id': 'RCM.SPECIAL_HANDLING_REQUIRED', 'datatype': 'String'}, ...}, 91 | 'results': {'Buyer Id': {'id': 'ARCHIVE_IMAGE.AGENCY_BUYER', 'datatype': 'Integer'}, ...} 92 | } 93 | 94 | # Get a list of available field IDs for RCMImageProducts collection 95 | field_ids = rapi.get_available_fields('RCMImageProducts', name_type='id') 96 | print(field_ids) 97 | 98 | >>> {'search': ['RCM.SPECIAL_HANDLING_REQUIRED', 'ARCHIVE_IMAGE.CLIENT_ORDER_NUMBER', ...], 99 | 'results': ['ARCHIVE_IMAGE.AGENCY_BUYER', 'ARCHIVE_IMAGE.ARCH_VISIBILITY_START', ...] 100 | } 101 | 102 | # Get a list of available field names used to submit searches (rapi.search()) 103 | field_titles = rapi.get_available_fields('RCMImageProducts', name_type='title') 104 | print(field_titles) 105 | 106 | >>> {'search': ['Special Handling Required', 'Client Order Number', 'Order Key', ...], 107 | 'results': ['Buyer Id', 'Archive Visibility Start Date', 'Client Order Item Number', ...] 108 | } 109 | -------------------------------------------------------------------------------- /.github/workflows/test_pyeodmsrapi.yml: -------------------------------------------------------------------------------- 1 | # This is a basic workflow to help you get started with Actions 2 | 3 | name: py-eodms-rapi Tests 4 | 5 | # Controls when the workflow will run 6 | on: 7 | 8 | # Allows you to run this workflow manually from the Actions tab 9 | workflow_dispatch: 10 | 11 | # A workflow run is made up of one or more jobs that can run sequentially or in parallel 12 | jobs: 13 | # This workflow contains a single job called "build" 14 | build: 15 | # The type of runner that the job will run on 16 | runs-on: ubuntu-latest 17 | 18 | # Steps represent a sequence of tasks that will be executed as part of the job 19 | steps: 20 | # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it 21 | - uses: actions/checkout@v3 22 | with: 23 | ref: 'development' 24 | 25 | # List files 26 | - name: List files 27 | run: ls 28 | 29 | # Install package from setup.py 30 | - name: Install package from setup.py 31 | run: | 32 | sudo python setup.py install 33 | 34 | # Runs a single command using the runners shell 35 | - name: Run py-eodms-rapi Test 1 - Search, Order & Download 36 | # continue-on-error: true 37 | env: 38 | EODMS_USER: ${{ secrets.EODMS_USER }} 39 | EODMS_PASSWORD: ${{ secrets.EODMS_PWD }} 40 | run: | 41 | cd test 42 | python -m unittest test_pyeodmsrapi.TestEodmsRapi.test_search 43 | 44 | # Runs a single command using the runners shell 45 | - name: Run py-eodms-rapi Test 2 - Show Order Parameters 46 | # continue-on-error: true 47 | env: 48 | EODMS_USER: ${{ secrets.EODMS_USER }} 49 | EODMS_PASSWORD: ${{ secrets.EODMS_PWD }} 50 | run: | 51 | cd test 52 | python -m unittest test_pyeodmsrapi.TestEodmsRapi.test_orderparameters 53 | 54 | # Runs a single command using the runners shell 55 | - name: Run py-eodms-rapi Test 3 - Delete Order 56 | # continue-on-error: true 57 | env: 58 | EODMS_USER: ${{ secrets.EODMS_USER }} 59 | EODMS_PASSWORD: ${{ secrets.EODMS_PWD }} 60 | run: | 61 | cd test 62 | python -m unittest test_pyeodmsrapi.TestEodmsRapi.test_deleteorder 63 | 64 | # Runs a single command using the runners shell 65 | - name: Run py-eodms-rapi Test 4 - Get Available Fields 66 | # continue-on-error: true 67 | env: 68 | EODMS_USER: ${{ secrets.EODMS_USER }} 69 | EODMS_PASSWORD: ${{ secrets.EODMS_PWD }} 70 | run: | 71 | cd test 72 | python -m unittest test_pyeodmsrapi.TestEodmsRapi.test_availablefields 73 | 74 | # Runs multiple searches 75 | - name: Run py-eodms-rapi Test 5 - Multiple Searches and Clearing Results 76 | # continue-on-error: true 77 | env: 78 | EODMS_USER: ${{ secrets.EODMS_USER }} 79 | EODMS_PASSWORD: ${{ secrets.EODMS_PWD }} 80 | run: | 81 | cd test 82 | python -m unittest test_pyeodmsrapi.TestEodmsRapi.test_multiple_searches 83 | 84 | # Runs with wrong creds 85 | - name: Run py-eodms-rapi Test 6 - Wrong Creds 86 | # continue-on-error: true 87 | env: 88 | EODMS_USER: ${{ secrets.EODMS_USER }} 89 | EODMS_PASSWORD: ${{ secrets.EODMS_PWD }} 90 | run: | 91 | cd test 92 | python -m unittest test_pyeodmsrapi.TestEodmsRapi.test_wrong_creds 93 | 94 | # Runs with wrong creds 95 | - name: Run py-eodms-rapi Test 7 - SAR Toolbox Order 96 | # continue-on-error: true 97 | env: 98 | EODMS_USER: ${{ secrets.EODMS_USER }} 99 | EODMS_PASSWORD: ${{ secrets.EODMS_PWD }} 100 | ORDER_ID: 708364 101 | run: | 102 | cd test 103 | python -m unittest test_pyeodmsrapi.TestEodmsRapi.test_st_orders 104 | -------------------------------------------------------------------------------- /test/test_pyeodmsrapi.py: -------------------------------------------------------------------------------- 1 | ############################################################################## 2 | # MIT License 3 | # 4 | # Copyright (c) His Majesty the King in Right of Canada, as 5 | # represented by the Minister of Natural Resources, 2023. 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a 8 | # copy of this software and associated documentation files (the "Software"), 9 | # to deal in the Software without restriction, including without limitation 10 | # the rights to use, copy, modify, merge, publish, distribute, sublicense, 11 | # and/or sell copies of the Software, and to permit persons to whom the 12 | # Software is furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in 15 | # all copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 22 | # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 23 | # DEALINGS IN THE SOFTWARE. 24 | # 25 | ############################################################################## 26 | 27 | __title__ = 'py-eodms-rapi Tester' 28 | __author__ = 'Kevin Ballantyne' 29 | __copyright__ = 'Copyright (c) His Majesty the King in Right of Canada, ' \ 30 | 'as represented by the Minister of Natural Resources, 2023' 31 | __license__ = 'MIT License' 32 | __description__ = 'Performs various tests of the py-eodms-rapi Python package.' 33 | __email__ = 'eodms-sgdot@nrcan-rncan.gc.ca' 34 | 35 | import os 36 | import sys 37 | import eodms_rapi 38 | 39 | import unittest 40 | 41 | class TestEodmsRapi(unittest.TestCase): 42 | 43 | def test_search(self): 44 | """ 45 | Tests the search, order and download of the py-eodms-rapi 46 | """ 47 | 48 | rapi = eodms_rapi.EODMSRAPI(os.getenv('EODMS_USER'), 49 | os.environ.get('EODMS_PASSWORD')) 50 | 51 | # Set a polygon of geographic centre of Canada using GeoJSON 52 | feat = [ 53 | ('INTERSECTS', {"type": "Polygon", "coordinates": [[[-95.47, 61.4], \ 54 | [-97.47, 61.4], 55 | [-97.47, 63.4], 56 | [-95.47, 63.4], 57 | [-95.47, 58 | 61.4]]]})] 59 | 60 | # Set date ranges 61 | dates = [{"start": "20190101_000000", "end": "20210621_000000"}] 62 | 63 | # Set search filters 64 | filters = {'Beam Mode Type': ('LIKE', ['%50m%']), 65 | 'Polarization': ('=', 'HH HV'), 66 | 'Incidence Angle': ('>=', 17)} 67 | 68 | # Set the results fields 69 | result_fields = ['ARCHIVE_RCM.PROCESSING_FACILITY', 70 | 'RCM.ANTENNA_ORIENTATION'] 71 | 72 | # Submit search 73 | rapi.search("RCMImageProducts", filters, feat, dates, result_fields, 2) 74 | 75 | # Get results 76 | rapi.set_field_convention('upper') 77 | res = rapi.get_results('full') 78 | 79 | # Now order the images 80 | order_res = rapi.order(res) 81 | 82 | # Download images to a specific destination 83 | dest = "files/downloads" 84 | os.makedirs(dest, exist_ok=True) 85 | dn_res = rapi.download(order_res, dest, max_attempts=100) 86 | 87 | # Print results 88 | rapi.print_results(dn_res) 89 | 90 | assert dn_res is not None and not dn_res == '' 91 | 92 | def test_orderparameters(self): 93 | """ 94 | Tests getting the order parameters using py-eodms-rapi 95 | """ 96 | 97 | rapi = eodms_rapi.EODMSRAPI(os.getenv('EODMS_USER'), 98 | os.environ.get('EODMS_PASSWORD')) 99 | 100 | # Get the order parameters for RCM image with Record ID 7627902 101 | param_res = rapi.get_order_parameters('RCMImageProducts', '7627902') 102 | 103 | # Print the parameters 104 | print(f"param_res: {param_res}") 105 | 106 | assert param_res is not None and not param_res == '' 107 | 108 | def test_deleteorder(self): 109 | """ 110 | Tests deleting an order using py-eodms-rapi 111 | """ 112 | 113 | rapi = eodms_rapi.EODMSRAPI(os.getenv('EODMS_USER'), 114 | os.environ.get('EODMS_PASSWORD')) 115 | 116 | orders = rapi.get_orders() 117 | 118 | order_id = None 119 | item_id = None 120 | for o in orders: 121 | if o['status'] == 'AVAILABLE_FOR_DOWNLOAD': 122 | order_id = o['orderId'] 123 | item_id = o['itemId'] 124 | break 125 | 126 | # Delete the order item 127 | if order_id is not None and item_id is not None: 128 | delete_res = rapi.cancel_order_item(order_id, item_id) 129 | 130 | assert delete_res is not None and not delete_res == '' 131 | 132 | def test_availablefields(self): 133 | """ 134 | Tests getting available fields for a collection using py-eodms-rapi 135 | """ 136 | 137 | rapi = eodms_rapi.EODMSRAPI(os.getenv('EODMS_USER'), 138 | os.environ.get('EODMS_PASSWORD')) 139 | 140 | # Get the available field information for RCMImageProducts collection 141 | fields = rapi.get_available_fields('RCMImageProducts') 142 | print(fields) 143 | 144 | # Get a list of available field IDs for RCMImageProducts collection 145 | field_ids = rapi.get_available_fields('RCMImageProducts', 146 | name_type='id') 147 | print(field_ids) 148 | 149 | # Get a list of available field names used to submit searches (rapi.search()) 150 | field_titles = rapi.get_available_fields('RCMImageProducts', 151 | name_type='title') 152 | print(field_titles) 153 | 154 | def test_multiple_searches(self): 155 | 156 | rapi = eodms_rapi.EODMSRAPI(os.getenv('EODMS_USER'), 157 | os.environ.get('EODMS_PASSWORD')) 158 | 159 | # Set search filters 160 | filters = {'Beam Mode Type': ('LIKE', ['%50m%']), 161 | 'Polarization': ('=', 'HH HV'), 162 | 'Incidence Angle': ('>=', 17)} 163 | 164 | # Submit RCMImageProducts search 165 | rapi.search("RCMImageProducts", filters, max_results=2) 166 | 167 | # Submit R1 search 168 | rapi.search("Radarsat1", max_results=2) 169 | 170 | # Get results 171 | rapi.set_field_convention('upper') 172 | res = rapi.get_results('full') 173 | 174 | trim_res = [(r['RECORD_ID'], r['COLLECTION_ID']) for r in res] 175 | print(f"res: {trim_res}") 176 | print(f"Number of results: {len(res)}") 177 | 178 | rapi.clear_results() 179 | 180 | res = rapi.get_results('full') 181 | print(f"Number of results: {len(res)}") 182 | 183 | def test_wrong_creds(self): 184 | rapi = eodms_rapi.EODMSRAPI('dflgkhdfgjkh', 'sdfglkdfhgjkf') 185 | 186 | colls = rapi.get_collections() 187 | 188 | def test_st_orders(self): 189 | 190 | rapi = eodms_rapi.EODMSRAPI(os.getenv('EODMS_USER'), 191 | os.environ.get('EODMS_PASSWORD')) 192 | 193 | order_id = os.environ.get('ORDER_ID') 194 | if order_id is None: 195 | order_id = 708364 196 | 197 | print(f"order_id: {order_id}") 198 | order_res = rapi.get_order(order_id) 199 | dest = "files/downloads" 200 | os.makedirs(dest, exist_ok=True) 201 | dn_res = rapi.download(order_res, dest, max_attempts=100) 202 | 203 | print(f"dn_res: {dn_res}") 204 | 205 | if __name__ == '__main__': 206 | unittest.main() -------------------------------------------------------------------------------- /eodms_rapi/rapi_requests.py: -------------------------------------------------------------------------------- 1 | ############################################################################## 2 | # MIT License 3 | # 4 | # Copyright (c) His Majesty the King in Right of Canada, as 5 | # represented by the Minister of Natural Resources, 2024 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a 8 | # copy of this software and associated documentation files (the "Software"), 9 | # to deal in the Software without restriction, including without limitation 10 | # the rights to use, copy, modify, merge, publish, distribute, sublicense, 11 | # and/or sell copies of the Software, and to permit persons to whom the 12 | # Software is furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in 15 | # all copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 22 | # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 23 | # DEALINGS IN THE SOFTWARE. 24 | # 25 | ############################################################################## 26 | 27 | 28 | 29 | import os 30 | import requests 31 | import traceback 32 | 33 | from tqdm.auto import tqdm 34 | 35 | # from .geo import EODMSGeo 36 | from .query_error import QueryError 37 | 38 | # OTHER_FORMAT = '| %(name)s | %(levelname)s: %(message)s', '%Y-%m-%d %H:%M:%S' 39 | 40 | # logger = logging.getLogger('EODMSRAPI') 41 | 42 | # # Set handler for output to terminal 43 | # logger.setLevel(logging.DEBUG) 44 | # ch = logging.NullHandler() 45 | # formatter = logging.Formatter('| %(name)s | %(asctime)s | %(levelname)s: ' 46 | # '%(message)s', '%Y-%m-%d %H:%M:%S') 47 | # ch.setFormatter(formatter) 48 | # logger.addHandler(ch) 49 | 50 | class RAPIRequests: 51 | """ 52 | The RAPIRequests Class containing the methods which sends requests to the 53 | RAPI. 54 | """ 55 | 56 | def __init__(self, eodms_obj, username=None, password=None, 57 | timeout_query=120.0, timeout_order=180.0, attempts=4, 58 | verify=True): 59 | """ 60 | Initializer for RAPIRequests. 61 | 62 | 63 | """ 64 | 65 | self.eodms = eodms_obj 66 | 67 | # Create session 68 | self._session = None 69 | if username and password: 70 | self._session = requests.Session() 71 | self._session.auth = (username, password) 72 | 73 | self.rapi_root = "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi" 74 | 75 | self.timeout_query = timeout_query 76 | self.timeout_order = timeout_order 77 | self.attempts = attempts 78 | self.verify = verify 79 | 80 | # self.geo = EODMSGeo(self) 81 | 82 | self.logger = eodms_obj.logger 83 | 84 | self._headers = {} 85 | 86 | # print(f"version: {eodms_obj.__version__()}") 87 | 88 | # self._map_fields() 89 | 90 | # self._header = '| EODMSRAPI | ' 91 | 92 | def close_session(self): 93 | 94 | self.eodms.log_msg("Logging out of EODMS", log_indent='\n\n\t', 95 | out_indent='\n') 96 | 97 | self._session.get('https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/logout.jsp') 98 | 99 | def check_http(self, err_msg): 100 | """ 101 | Checks an error message for the HTTP code and returns a more 102 | appropriate message. 103 | 104 | :param err_msg: The error message from the response. 105 | :type err_msg: str 106 | 107 | :return: The new error message (or None is no error). 108 | :rtype: str (or None) 109 | """ 110 | 111 | if err_msg.find('404 Client Error') > -1 or \ 112 | err_msg.find('404 for url') > -1: 113 | msg = f"404 Client Error: Could not find {self.eodms.rapi_url}." 114 | elif err_msg.find('400 Client Error') > -1: 115 | msg = f"400 Client Error: A Bad Request occurred while trying to " \ 116 | f"reach {self.eodms.rapi_url}" 117 | elif err_msg.find('500 Server Error') > -1: 118 | msg = f"500 Server Error: An internal server error has occurred " \ 119 | f"while to access {self.eodms.rapi_url}" 120 | elif err_msg.find('401 Client Error') > -1: 121 | return err_msg 122 | else: 123 | return None 124 | 125 | return msg 126 | 127 | def add_header(self, key, value, append=False, top=True): 128 | """ 129 | Add a value to either existing key or add new key. 130 | 131 | :param key: The key in the header to change. 132 | :type key: str 133 | :param value: The value to add to the header. 134 | :type key: str 135 | :param append: Determines whether to add the value to the header or 136 | replace the existing header with value. 137 | :type append: boolean 138 | """ 139 | 140 | entry = {key: value} 141 | if append: 142 | if self._session: 143 | exist_entry = self._session.headers.get(key) 144 | else: 145 | exist_entry = self._headers.get(key) 146 | 147 | if top: 148 | entry = {key: f"{value} {exist_entry}"} 149 | else: 150 | entry = {key: f"{exist_entry} {value}"} 151 | 152 | if self._session: 153 | self._session.headers.update(entry) 154 | else: 155 | self._headers.update(entry) 156 | 157 | # print(f"Session headers: {self._session.headers}") 158 | # print(f"headers: {self._headers}") 159 | 160 | def get_header(self, url): 161 | 162 | if self._session: 163 | return self._session.head(url) 164 | else: 165 | return requests.head(url, headers=self._headers, verify=self.verify) 166 | 167 | def submit(self, query_url, request_type='get', post_data=None, 168 | timeout=None, record_name=None, quiet=True, as_json=True): 169 | """ 170 | Send a query to the RAPI. 171 | 172 | :param query_url: The query URL. 173 | :type query_url: str 174 | :param timeout: The length of the timeout in seconds. 175 | :type timeout: float 176 | :param record_name: A string used to supply information for the record 177 | in a print statement. 178 | :type record_name: str 179 | :param quiet: Determines whether to ignore log printing. 180 | :type quiet: bool 181 | :param as_json: Determines whether to return results in JSON format. 182 | :type as_json: bool 183 | 184 | :return: The response returned from the RAPI. 185 | :rtype: request.Response 186 | """ 187 | 188 | if timeout is None: 189 | timeout = self.timeout_query 190 | 191 | self.logger.debug(f"RAPI Query URL: {query_url}") 192 | # print(f"RAPI Query URL: {query_url}") 193 | 194 | res = None 195 | attempt = 1 196 | err = None 197 | msg = '' 198 | # Get the entry records from the RAPI using the downlink segment ID 199 | while res is None and attempt <= self.attempts: 200 | # Continue to attempt if timeout occurs 201 | try: 202 | if record_name is None: 203 | msg = f"Sending request to the RAPI (attempt {attempt})..." 204 | else: 205 | msg = f"Sending request to the RAPI for '{record_name}' " \ 206 | f"(attempt {attempt})..." 207 | if not quiet and attempt > 1: 208 | self.logger.debug(f"\n{self.eodms.header}{msg}") 209 | if self._session is None: 210 | if request_type.lower() == 'post': 211 | res = requests.post(query_url, post_data, 212 | headers=self._headers, 213 | timeout=timeout, 214 | verify=self.verify) 215 | elif request_type.lower() == 'put': 216 | res = requests.put(url=query_url, 217 | headers=self._headers, 218 | timeout=timeout, 219 | verify=self.verify) 220 | elif request_type.lower() == 'delete': 221 | res = requests.delete(url=query_url, 222 | headers=self._headers, 223 | timeout=timeout, 224 | verify=self.verify) 225 | else: 226 | res = requests.get(query_url, headers=self._headers, 227 | timeout=timeout, 228 | verify=self.verify) 229 | elif request_type.lower() == 'post': 230 | res = self._session.post(query_url, post_data, 231 | timeout=timeout, 232 | verify=self.verify) 233 | elif request_type.lower() == 'put': 234 | res = self._session.put(url=query_url, 235 | timeout=timeout, 236 | verify=self.verify) 237 | elif request_type.lower() == 'delete': 238 | res = self._session.delete(url=query_url, 239 | timeout=timeout, 240 | verify=self.verify) 241 | else: 242 | # print(f"Cookies: {self._session.cookies}") 243 | res = self._session.get(query_url, timeout=timeout, 244 | verify=self.verify) 245 | res.raise_for_status() 246 | except requests.exceptions.HTTPError as errh: 247 | msg = f"HTTP Error: {errh}" 248 | 249 | out_msg = self.check_http(msg) 250 | 251 | if out_msg is not None: 252 | err = out_msg 253 | query_err = QueryError(err) 254 | 255 | return query_err if self.eodms._check_auth(query_err) \ 256 | else query_err 257 | 258 | if attempt < self.attempts: 259 | msg = f"{msg}; attempting to connect again..." 260 | self.eodms.log_msg(msg, 'warning') 261 | res = None 262 | else: 263 | err = msg 264 | attempt += 1 265 | except requests.exceptions.SSLError as ssl_err: 266 | msg = f"SSL Error: {ssl_err}" 267 | if attempt < self.attempts: 268 | msg = f"{msg}; removing SSL verification and attempting " \ 269 | f"to connect again..." 270 | self.eodms.log_msg(msg, 'warning') 271 | res = None 272 | self.verify = False 273 | else: 274 | err = msg 275 | attempt += 1 276 | except (requests.exceptions.Timeout, 277 | requests.exceptions.ReadTimeout) as errt: 278 | msg = f"Timeout Error: {errt}" 279 | if attempt < self.attempts: 280 | msg = f"{msg}; increasing timeout by a minute and " \ 281 | f"trying again..." 282 | self.eodms.log_msg(msg, 'warning') 283 | res = None 284 | timeout += 60.0 285 | self.timeout_query = timeout 286 | else: 287 | err = msg 288 | attempt += 1 289 | except (requests.exceptions.ConnectionError, 290 | requests.exceptions.RequestException) as req_err: 291 | # print(f"res: {res}") 292 | self.err_msg = f"{req_err.__class__.__name__} Error: {req_err}" 293 | self.err_occurred = True 294 | self.eodms.log_msg(self.err_msg, 'error') 295 | # attempt = self.attempts 296 | return None 297 | except KeyboardInterrupt: 298 | self.err_msg = "Process ended by user." 299 | self.eodms.log_msg(self.err_msg, out_indent='\n') 300 | self.err_occurred = True 301 | return None 302 | except Exception: 303 | msg = f"Unexpected error: {traceback.format_exc()}" 304 | if attempt < self.attempts: 305 | msg = f"{msg}; attempting to connect again..." 306 | self.eodms.log_msg(msg, 'warning') 307 | res = None 308 | else: 309 | err = msg 310 | attempt += 1 311 | 312 | if err is not None: 313 | query_err = QueryError(err) 314 | 315 | return None if self.eodms._check_auth(query_err) else query_err 316 | # If no results from RAPI, return None 317 | if res is None: 318 | return None 319 | 320 | # Check for exceptions that weren't already caught 321 | if not res.ok: 322 | except_err = self._get_exception(res) 323 | 324 | if isinstance(except_err, QueryError): 325 | if self.eodms._check_auth(except_err): 326 | return None 327 | 328 | self.eodms.log_msg(msg, 'warning') 329 | return except_err 330 | 331 | if res.text == '': 332 | return res 333 | 334 | if res.text.find('BRB!') > -1: 335 | self.err_msg = f"There was a problem while attempting to access the " \ 336 | f"EODMS RAPI server. If the problem persists, please " \ 337 | f"contact the EODMS Support Team at {self.eodms._email}." 338 | self.eodms.log_msg(self.err_msg, 'error') 339 | self.err_occurred = True 340 | query_err = QueryError(self.err_msg) 341 | return query_err 342 | 343 | return res.json() if as_json else res 344 | 345 | def set_query_timeout(self, timeout): 346 | """ 347 | Sets the timeout limit for a query to the RAPI. 348 | 349 | :param timeout: The value of the timeout in seconds. 350 | :type timeout: float 351 | 352 | """ 353 | self.timeout_query = float(timeout) 354 | 355 | def set_order_timeout(self, timeout): 356 | """ 357 | Sets the timeout limit for an order to the RAPI. 358 | 359 | :param timeout: The value of the timeout in seconds. 360 | :type timeout: float 361 | 362 | """ 363 | self.timeout_order = float(timeout) 364 | 365 | def set_attempts(self, number): 366 | """ 367 | Sets number of attempts to be made to the RAPI before the script 368 | ends. 369 | 370 | :param number: The value for the number of attempts. 371 | :type number: int 372 | 373 | """ 374 | self.attempts = int(number) 375 | 376 | def download(self, url, fsize=None, dest_fn=None, show_progress=True): 377 | """ 378 | Given a list of remote and local items, download the remote data if 379 | it is not already found locally. 380 | 381 | (Adapted from the eodms-api-client ( 382 | https://pypi.org/project/eodms-api-client/) developed by Mike Brady) 383 | 384 | :param url: The download URL of the image. 385 | :type url: str 386 | :param dest_fn: The local destination filename for the download. 387 | :type dest_fn: str 388 | :param fsize: The total filesize of the image. 389 | :type fsize: int 390 | :param show_progress: Determines whether to show progress while 391 | downloading an image 392 | :type show_progress: bool 393 | :param no_file: Determines whether to download as a file. 394 | :type no_file: bool 395 | """ 396 | 397 | # Use streamed download so we can wrap nicely with tqdm 398 | if show_progress: 399 | with self._session.get(url, stream=True, verify=self.verify) \ 400 | as stream: 401 | try: 402 | with open(dest_fn, 'wb') as pipe: 403 | with tqdm.wrapattr( 404 | pipe, 405 | method='write', 406 | miniters=1, 407 | total=fsize, 408 | desc=f"{self.eodms.header}\ 409 | {os.path.basename(dest_fn)}" 410 | ) as file_out: 411 | for chunk in stream.iter_content(chunk_size=1024): 412 | file_out.write(chunk) 413 | except FileNotFoundError: 414 | pass 415 | 416 | else: 417 | response = self._session.get(url, stream=True, verify=self.verify) 418 | if dest_fn is None: 419 | return response.content 420 | open(dest_fn, "wb").write(response.content) 421 | 422 | return dest_fn 423 | -------------------------------------------------------------------------------- /eodms_rapi/geo.py: -------------------------------------------------------------------------------- 1 | ############################################################################## 2 | # MIT License 3 | # 4 | # Copyright (c) His Majesty the King in Right of Canada, as 5 | # represented by the Minister of Natural Resources, 2023 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a 8 | # copy of this software and associated documentation files (the "Software"), 9 | # to deal in the Software without restriction, including without limitation 10 | # the rights to use, copy, modify, merge, publish, distribute, sublicense, 11 | # and/or sell copies of the Software, and to permit persons to whom the 12 | # Software is furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in 15 | # all copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 22 | # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 23 | # DEALINGS IN THE SOFTWARE. 24 | # 25 | ############################################################################## 26 | 27 | import os 28 | # import sys 29 | import re 30 | from xml.etree import ElementTree 31 | import json 32 | import logging 33 | # import traceback 34 | from geomet import wkt 35 | from warnings import warn 36 | 37 | # import decimal 38 | 39 | try: 40 | import osgeo.ogr as ogr 41 | import osgeo.osr as osr 42 | 43 | GDAL_INSTALLED = True 44 | except ImportError: 45 | try: 46 | import ogr 47 | import osr 48 | 49 | GDAL_INSTALLED = True 50 | except ImportError: 51 | GDAL_INSTALLED = False 52 | 53 | 54 | # try: 55 | # import ogr 56 | # import osr 57 | # GDAL_INSTALLED = True 58 | # except ImportError: 59 | # GDAL_INSTALLED = False 60 | 61 | # try: 62 | # import geojson 63 | # GEOJSON_INSTALLED = True 64 | # except ImportError: 65 | # GEOJSON_INSTALLED = False 66 | 67 | class EODMSGeo: 68 | """ 69 | The Geo class contains all the methods and functions used to perform 70 | geographic processes mainly using OGR. 71 | """ 72 | 73 | def __init__(self, eodmsrapi=None): 74 | """ 75 | Initializer for the Geo object. 76 | 77 | :param eodmsrapi: The parent EODMSRAPI object. 78 | :type eodmsrapi: eodms.EODMSRAPI 79 | 80 | """ 81 | self.aoi = None 82 | 83 | self.logger = logging.getLogger('EODMSRAPI') 84 | 85 | self.wkt_types = ['point', 'linestring', 'polygon', 86 | 'multipoint', 'multilinestring', 'multipolygon'] 87 | self.eodmsrapi = eodmsrapi 88 | self.logger = eodmsrapi.logger 89 | self.feats = None 90 | 91 | ############################################################### 92 | # Backwards compatibility methods 93 | ############################################################### 94 | 95 | def convert_imageGeom(self, coords, output='array'): 96 | warn("Method 'convert_imageGeom' is deprecated. Please use " 97 | "'convert_image_geom'.", DeprecationWarning, stacklevel=2) 98 | return self.convert_image_geom(coords, output) 99 | 100 | def convert_toWKT(self, in_feat, in_type): 101 | warn("Method 'convert_toWKT' is deprecated. Please use " 102 | "'convert_to_wkt'.", DeprecationWarning, stacklevel=2) 103 | return self.convert_to_wkt(in_feat, in_type) 104 | 105 | def convert_toGeoJSON(self, results, output='FeatureCollection'): 106 | warn("Method 'convert_toGeoJSON' is deprecated. Please use " 107 | "'convert_to_geojson'.", DeprecationWarning, stacklevel=2) 108 | return self.convert_to_geojson(results, output) 109 | 110 | ############################################################### 111 | 112 | def _check_ogr(self): 113 | """ 114 | There is another ogr Python package. This will check if it was 115 | imported instead of the proper ogr. 116 | """ 117 | 118 | if ogr.__doc__ is not None and \ 119 | ogr.__doc__.find("Module providing one api for multiple git " 120 | "services") > -1: 121 | if self.eodmsrapi is not None: 122 | msg = "Another package named 'ogr' is installed." 123 | self.eodmsrapi.log_msg(msg, 'warning') 124 | return False 125 | 126 | return True 127 | 128 | def _convert_list(self, in_feat, out='wkt'): 129 | """ 130 | Converts a list to a specified output. 131 | 132 | :param in_feat: The input feature(s). 133 | :type in_feat: list 134 | :param out: The type of output, either 'wkt' or 'json'. 135 | :type out: str 136 | 137 | :return: The converted feature(s) to the specified output. 138 | :rtype: json or str 139 | """ 140 | 141 | pnts = [list(p) for p in in_feat] 142 | 143 | if len(pnts) == 1: 144 | geojson = {"type": "Point", "coordinates": pnts[0]} 145 | else: 146 | geojson = {"type": "Polygon", "coordinates": [pnts]} 147 | 148 | if out == 'json': 149 | return geojson 150 | else: 151 | out_wkt = wkt.dumps(geojson) 152 | 153 | return out_wkt 154 | 155 | def _is_wkt(self, in_feat, show_error=False, return_wkt=False): 156 | """ 157 | Verifies if a string is WKT. 158 | 159 | :param in_feat: Input string containing a WKT. 160 | :type in_feat: str 161 | :param show_error: Determines whether to display the error. 162 | :type show_error: boolean 163 | :param return_wkt: Determines whether to return the converted WKT 164 | if True. 165 | :type return_wkt: boolean 166 | 167 | :return: If the input is a valid WKT, return WKT if return_wkt 168 | is True or return just True; False if not valid. 169 | :rtype: str or boolean 170 | """ 171 | 172 | try: 173 | wkt_val = wkt.loads(in_feat.upper()) 174 | except (ValueError, TypeError) as e: 175 | if show_error: 176 | if self.eodmsrapi is not None: 177 | self.eodmsrapi.log_msg(str(e), 'warning') 178 | return False 179 | 180 | return wkt_val if return_wkt else True 181 | 182 | def _remove_zero_trail(self, in_wkt): 183 | """ 184 | Removes the trailing zeros in coordinates after the decimal in a WKT. 185 | 186 | :param in_wkt: The input WKT. 187 | :type in_wkt: str 188 | 189 | :return: The WKT without trailing zeros after the decimal. 190 | :rtype: str 191 | """ 192 | 193 | numbers = re.findall(r"\d+\.\d+", in_wkt) 194 | 195 | out_wkt = in_wkt 196 | for num in numbers: 197 | flt_num = float(num) 198 | out_wkt = out_wkt.replace(num, str(flt_num)) 199 | 200 | return out_wkt 201 | 202 | def _split_multi(self, feats, in_type='json', out='wkt'): 203 | """ 204 | Splits multi-geometry into several valid geometry for the RAPI. 205 | 206 | :param feats: The input feature(s). 207 | :type feats: json, str or list 208 | :param in_type: Helps determine the type of input (either 'wkt', 209 | 'json' or 'list'). 210 | :type in_type: str 211 | :param out: Determines the type of output (either 'wkt' or 'json'). 212 | :type out: str 213 | 214 | :return: The geometry (or geometries) in the specified output type. 215 | :rtype: json, str 216 | """ 217 | 218 | if in_type == 'wkt': 219 | 220 | # Convert feats to json for easier manipulation 221 | json_geom = self._is_wkt(feats, True, True) 222 | 223 | elif in_type == 'list': 224 | json_geom = self._convert_list(feats, 'json') 225 | else: 226 | json_geom = feats 227 | 228 | geom_type = json_geom.get('type').lower() 229 | if geom_type.find('multi') > -1: 230 | feat_coords = json_geom.get('coordinates') 231 | out_geom = [] 232 | if geom_type == 'multipoint': 233 | for pnt in feat_coords: 234 | geom = {'type': 'Point', 'coordinates': pnt} 235 | out_geom.append(geom) 236 | elif geom_type == 'multilinestring': 237 | for line in feat_coords: 238 | geom = {'type': 'LineString', 'coordinates': line} 239 | out_geom.append(geom) 240 | elif geom_type == 'multipolygon': 241 | for poly in feat_coords: 242 | geom = {'type': 'Polygon', 'coordinates': poly} 243 | out_geom.append(geom) 244 | else: 245 | out_geom = json_geom 246 | 247 | if out == 'wkt': 248 | out_feats = [] 249 | if isinstance(out_geom, list): 250 | for g in out_geom: 251 | wkt_feat = self.convert_to_wkt(g, 'json') 252 | out_feats.append(wkt_feat) 253 | else: 254 | out_feats = self.convert_to_wkt(out_geom, 'json') 255 | else: 256 | out_feats = out_geom 257 | 258 | return out_feats 259 | 260 | def add_geom(self, in_src): 261 | """ 262 | Processes the source and converts it for use in the RAPI. 263 | 264 | :param in_src: The in_src can either be: 265 | 266 | - a filename (ESRI Shapefile, KML, GML or GeoJSON) of multiple 267 | features 268 | - a WKT format string of a single feature 269 | - the 'geometry' entry from a GeoJSON Feature 270 | - a list of coordinates (ex: ``[(x1, y1), (x2, y2), ...]``) 271 | 272 | :type in_src: str 273 | 274 | :return: A string of the WKT of the feature. 275 | :rtype: str 276 | 277 | """ 278 | 279 | if in_src is None: 280 | return None 281 | 282 | # If the source is in JSON format 283 | if self.eodmsrapi is not None: 284 | if self.eodmsrapi.is_json(in_src): 285 | in_src = json.loads(in_src) 286 | 287 | if isinstance(in_src, dict): 288 | # self.feats = self.convert_toWKT(in_src, 'json') 289 | self.feats = self._split_multi(in_src, 'json') 290 | return self.feats 291 | 292 | if isinstance(in_src, list): 293 | # self.feats = self.convert_toWKT(in_src, 'list') 294 | self.feats = self._split_multi(in_src, 'list') 295 | return self.feats 296 | 297 | # If the source is a file 298 | if os.path.isfile(in_src): 299 | self.feats = self.get_features(in_src) 300 | return self.feats 301 | 302 | if os.path.isdir(in_src): 303 | return None 304 | 305 | # print("in_src: %s" % in_src) 306 | 307 | if in_src.find('(') > -1 and in_src.find(')') > -1: 308 | if self._is_wkt(in_src, True): 309 | # Can only be a single WKT object 310 | self.feats = self._split_multi(in_src, 'wkt') 311 | return self.feats 312 | 313 | # If the source is a list of coordinates 314 | print(f"in_src: {in_src}") 315 | if not isinstance(in_src, list): 316 | try: 317 | eval(in_src) 318 | except SyntaxError as err: 319 | self.logger.warning(f"{err}") 320 | return err 321 | 322 | def convert_coords(self, coord_lst, geom_type): 323 | """ 324 | Converts a list of points to GeoJSON format. 325 | 326 | :param coord_lst: A list of points. 327 | :type coord_lst: list 328 | :param geom_type: The type of geometry, either 'Point', 329 | 'LineString' or 'Polygon'. 330 | :type geom_type: str 331 | 332 | :return: A dictionary in the GeoJSON format. 333 | :rtype: dict 334 | 335 | """ 336 | 337 | pnts_array = [] 338 | pnts = None 339 | for c in coord_lst: 340 | pnts = [ 341 | p.strip('\n').strip('\t').split(',') 342 | for p in c.split(' ') 343 | if p.strip('\n').strip('\t') != '' 344 | ] 345 | pnts_array += pnts 346 | 347 | if pnts is None: 348 | return None 349 | 350 | if geom_type == 'LineString': 351 | return { 352 | 'type': 'LineString', 353 | 'coordinates': [[float(p[0]), float(p[1])] for p in pnts], 354 | } 355 | elif geom_type == 'Point': 356 | return { 357 | 'type': 'Point', 358 | 'coordinates': [float(pnts[0][0]), float(pnts[0][1])], 359 | } 360 | else: 361 | return { 362 | 'type': 'Polygon', 363 | 'coordinates': [[[float(p[0]), float(p[1])] for p in pnts]], 364 | } 365 | 366 | def convert_image_geom(self, coords, output='array'): 367 | """ 368 | Converts a list of coordinates from the RAPI to a polygon geometry, 369 | array of points or as WKT. 370 | 371 | :param coords: A list of coordinates from the RAPI results. 372 | :type coords: list 373 | :param output: The type of return, can be 'array', 'wkt' or 'geom'. 374 | :type output: str 375 | 376 | :return: Either a polygon geometry, WKT or array of points. 377 | :rtype: multiple types 378 | 379 | """ 380 | 381 | if isinstance(coords, dict): 382 | 383 | if 'coordinates' in coords.keys(): 384 | val = coords['coordinates'] 385 | level = 0 386 | while isinstance(val, list): 387 | val = val[0] 388 | level += 1 389 | lst_level = level - 2 390 | 391 | if lst_level > -1: 392 | pnt_array = eval("coords['coordinates']" + 393 | '[0]' * lst_level) 394 | else: 395 | pnt_array = coords['coordinates'] 396 | else: 397 | self.logger.warning("No coordinates provided.") 398 | return None 399 | else: 400 | pnt_array = coords[0] 401 | 402 | if GDAL_INSTALLED: 403 | if not self._check_ogr(): 404 | if self.eodmsrapi is not None: 405 | self.eodmsrapi.log_msg("Cannot convert geometry.", 'warning') 406 | return None 407 | 408 | # Create ring 409 | ring = ogr.Geometry(ogr.wkbLinearRing) 410 | # Get the points from the coordinates list 411 | pnt1 = pnt_array[0] 412 | ring.AddPoint(pnt1[0], pnt1[1]) 413 | pnt2 = pnt_array[1] 414 | ring.AddPoint(pnt2[0], pnt2[1]) 415 | pnt3 = pnt_array[2] 416 | ring.AddPoint(pnt3[0], pnt3[1]) 417 | pnt4 = pnt_array[3] 418 | 419 | ring.AddPoint(pnt4[0], pnt4[1]) 420 | ring.AddPoint(pnt1[0], pnt1[1]) 421 | 422 | # Create polygon 423 | poly = ogr.Geometry(ogr.wkbPolygon) 424 | poly.AddGeometry(ring) 425 | 426 | # Send specified output 427 | if output == 'wkt': 428 | return poly.ExportToWkt() 429 | elif output == 'geom': 430 | return poly 431 | else: 432 | return pnt_array 433 | 434 | elif output == 'wkt': 435 | # Convert values in point array to strings 436 | pnt_array = [[str(p[0]), str(p[1])] for p in pnt_array] 437 | 438 | return "POLYGON ((%s))" % ', '.join([' '.join(pnt) 439 | for pnt in pnt_array]) 440 | else: 441 | return pnt_array 442 | 443 | # def convert_fromWKT(self, in_feat): 444 | # """ 445 | # Converts a WKT to a GDAL geometry. 446 | 447 | # :param in_feat: The WKT of the feature. 448 | # :type in_feat: str 449 | 450 | # :return: The polygon geometry of the input WKT. 451 | # :rtype: ogr.Geometry 452 | 453 | # """ 454 | 455 | # if GDAL_INSTALLED: 456 | # out_poly = ogr.CreateGeometryFromWkt(in_feat) 457 | 458 | # return out_poly 459 | 460 | def convert_to_wkt(self, in_feat, in_type): 461 | """ 462 | Converts a feature into WKT format. 463 | 464 | :param in_feat: The input feature, either as a GeoJSON 465 | dictionary or list of points. 466 | :type in_feat: dict or list 467 | :param in_type: The type of the input, whether it's 'json', 'list' 468 | or 'file'. 469 | :type in_type: str 470 | 471 | :return: The input feature converted to WKT. 472 | :rtype: str 473 | 474 | """ 475 | 476 | out_wkt = None 477 | if in_type == 'json': 478 | out_wkt = wkt.dumps(in_feat) 479 | elif in_type == 'list': 480 | out_wkt = self._convert_list(in_feat) 481 | elif in_type == 'file': 482 | return self.get_features(in_feat) 483 | out_wkt = self._remove_zero_trail(out_wkt) 484 | 485 | return out_wkt 486 | 487 | def convert_to_geojson(self, results, output='FeatureCollection'): 488 | """ 489 | Converts RAPI results to GeoJSON geometries. 490 | 491 | :param results: A list of results from the RAPI. 492 | :type results: list 493 | :param output: The output of the results (either 'FeatureCollection' 494 | or 'list' for a list of features in geojson) 495 | :type output: str 496 | 497 | :return: A dictionary of a GeoJSON FeatureCollection. 498 | :rtype: dict 499 | """ 500 | 501 | if isinstance(results, dict): 502 | results = [results] 503 | 504 | features = [] 505 | for rec in results: 506 | if self.eodmsrapi is None: 507 | return None 508 | geom = rec.get(self.eodmsrapi.get_conv('geometry')) 509 | props = self.eodmsrapi.parse_metadata(rec) 510 | 511 | feat = {"type": "Feature", "geometry": geom, "properties": props} 512 | 513 | features.append(feat) 514 | 515 | if output == 'list': 516 | return features 517 | 518 | return {"type": "FeatureCollection", "features": features} 519 | 520 | def process_polygon(self, geom, t_crs): 521 | 522 | # Convert the geometry to WGS84 523 | s_crs = geom.GetSpatialReference() 524 | 525 | if s_crs is None: 526 | s_crs = osr.SpatialReference() 527 | s_crs.ImportFromEPSG(4326) 528 | 529 | # Get the EPSG codes from the spatial references 530 | epsg_scrs = s_crs.GetAttrValue("AUTHORITY", 1) 531 | epsg_tcrs = t_crs.GetAttrValue("AUTHORITY", 1) 532 | 533 | if str(epsg_scrs) != '4326': 534 | if epsg_tcrs is None: 535 | print("\nCannot reproject AOI.") 536 | return None 537 | 538 | if not s_crs.IsSame(t_crs) and epsg_scrs != epsg_tcrs: 539 | # Create the CoordinateTransformation 540 | print("\nReprojecting input AOI...") 541 | coord_trans = osr.CoordinateTransformation(s_crs, t_crs) 542 | geom.Transform(coord_trans) 543 | 544 | # Reverse x and y of transformed geometry 545 | ring = geom.GetGeometryRef(0) 546 | for i in range(ring.GetPointCount()): 547 | ring.SetPoint(i, ring.GetY(i), ring.GetX(i)) 548 | 549 | # Convert multipolygon to polygon (if applicable) 550 | if geom.GetGeometryType() == 6: 551 | geom = geom.UnionCascaded() 552 | 553 | # Convert to WKT 554 | return geom.ExportToWkt() 555 | 556 | def get_features(self, in_src): 557 | """ 558 | Extracts the features from an AOI file. 559 | 560 | :param in_src: The input filename of the AOI file. Can either be 561 | a GML, KML, GeoJSON, or Shapefile. 562 | :type in_src: str 563 | 564 | :return: The AOI in WKT format. 565 | :rtype: str 566 | 567 | """ 568 | 569 | # print() 570 | 571 | out_feats = [] 572 | if GDAL_INSTALLED: 573 | # There is another ogr Python package that might have been imported 574 | # Check if its the wrong ogr 575 | if not self._check_ogr(): 576 | msg = "Cannot import feature using OGR." 577 | if self.eodmsrapi is not None: 578 | self.eodmsrapi.log_msg(msg, 'warning') 579 | return None 580 | 581 | # Determine the OGR driver of the input AOI 582 | if in_src.find('.gml') > -1: 583 | ogr_driver = 'GML' 584 | elif in_src.find('.kml') > -1: 585 | ogr_driver = 'KML' 586 | elif in_src.find('.json') > -1 or in_src.find('.geojson') > -1: 587 | ogr_driver = 'GeoJSON' 588 | elif in_src.find('.shp') > -1: 589 | ogr_driver = 'ESRI Shapefile' 590 | else: 591 | self.logger.error("The AOI file type could not be determined.") 592 | return None 593 | 594 | # Open AOI file and extract AOI 595 | driver = ogr.GetDriverByName(ogr_driver) 596 | ds = driver.Open(in_src, 0) 597 | 598 | # Get the layer from the file 599 | lyr = ds.GetLayer() 600 | 601 | # Set the target spatial reference to WGS84 602 | t_crs = osr.SpatialReference() 603 | t_crs.ImportFromEPSG(4326) 604 | 605 | for feat in lyr: 606 | # Create the geometry 607 | geom = feat.GetGeometryRef() 608 | 609 | if geom.GetGeometryName() == 'MULTIPOLYGON': 610 | out_feats.extend(self.process_polygon(geom_part, t_crs) for geom_part in geom) 611 | else: 612 | out_feats.append(self.process_polygon(geom, t_crs)) 613 | 614 | elif in_src.find('.gml') > -1 or in_src.find('.kml') > -1: 615 | 616 | with open(in_src, 'rt') as f: 617 | tree = ElementTree.parse(f) 618 | root = tree.getroot() 619 | 620 | geom_choices = ['Point', 'LineString', 'Polygon'] 621 | 622 | if in_src.find('.gml') > -1: 623 | for feat in root.findall('.//{http://www.opengis.net/' 624 | 'gml}featureMember'): 625 | 626 | # Get geometry type 627 | geom_type = 'Polygon' 628 | for elem in feat.findall('*'): 629 | tag = elem.tag.replace('{http://ogr.maptools.' 630 | 'org/}', '') 631 | if tag in geom_choices: 632 | geom_type = tag 633 | 634 | coord_lst = [ 635 | coords.text 636 | for coords in root.findall( 637 | './/{http://www.opengis' '.net/gml}coordinates' 638 | ) 639 | ] 640 | json_geom = self.convert_coords(coord_lst, geom_type) 641 | 642 | wkt_feat = self._split_multi(json_geom) 643 | 644 | if isinstance(wkt_feat, list): 645 | out_feats += wkt_feat 646 | else: 647 | out_feats.append(wkt_feat) 648 | else: 649 | for plcmark in root.findall('.//{http://www.opengis.net/' 650 | 'kml/2.2}Placemark'): 651 | 652 | # Get geometry type 653 | geom_type = 'Polygon' 654 | for elem in plcmark.findall('*'): 655 | tag = elem.tag.replace('{http://www.opengis.net/' 656 | 'kml/2.2}', '') 657 | if tag in geom_choices: 658 | geom_type = tag 659 | 660 | coord_lst = [ 661 | coords.text 662 | for coords in plcmark.findall( 663 | './/{http://www.opengis.net/kml/2.2}' 'coordinates' 664 | ) 665 | ] 666 | json_geom = self.convert_coords(coord_lst, geom_type) 667 | 668 | wkt_feat = self._split_multi(json_geom) 669 | 670 | if isinstance(wkt_feat, list): 671 | out_feats += wkt_feat 672 | else: 673 | out_feats.append(wkt_feat) 674 | 675 | elif in_src.find('.json') > -1 or in_src.find('.geojson') > -1: 676 | with open(in_src) as f: 677 | data = json.load(f) 678 | 679 | feats = data['features'] 680 | for f in feats: 681 | 682 | wkt_feat = self._split_multi(f['geometry']) 683 | 684 | if isinstance(wkt_feat, list): 685 | out_feats += wkt_feat 686 | else: 687 | out_feats.append(wkt_feat) 688 | 689 | elif in_src.find('.shp') > -1: 690 | msg = "Could not open shapefile. The GDAL Python Package " \ 691 | "must be installed to use shapefiles." 692 | self.logger.warning(msg) 693 | self.eodmsrapi.log_msg(msg, 'warning') 694 | return None 695 | return out_feats 696 | -------------------------------------------------------------------------------- /docs/search-rapi.rst: -------------------------------------------------------------------------------- 1 | 2 | Submit an Image Search 3 | ====================== 4 | 5 | You can perform a search on the RAPI using the ``search`` method of the EODMSRAPI. 6 | 7 | However, before submitting a search, you'll have to create the items used to filter the results. The ``search`` function requires a **Collection** name and optional **filters**\ , **geometry features**\ , **dates**\ , **result fields** and **maximum results** values. 8 | 9 | Collection 10 | ---------- 11 | 12 | The Collection ID has to be specified when submitting a search. 13 | 14 | To get a list of Collection IDs, use: 15 | 16 | .. code-block:: python 17 | 18 | >>> print(rapi.get_collections(as_list=True)) 19 | | EODMSRAPI | Getting Collection information, please wait... 20 | ['NAPL', 'SGBAirPhotos', 'RCMImageProducts', 'COSMO-SkyMed1', 'Radarsat1', 'Radarsat1RawProducts', 'Radarsat2', 'Radarsat2RawProducts', 'RCMScienceData', 'TerraSarX', 'DMC', 'Gaofen-1', 'GeoEye-1', 'IKONOS', 'IRS', 'PlanetScope', 'QuickBird-2', 'RapidEye', 'SPOT', 'WorldView-1', 'WorldView-2', 'WorldView-3', 'VASP'] 21 | 22 | Geometry Features 23 | ----------------- 24 | 25 | The **geometry features** are a list of tuples with each tuple containing an operator and a specified geometry (\ ``[(, ), ...]``\ ). 26 | 27 | The *operator* can be **contains**\ , **contained by**\ , **crosses**\ , **disjoint with**\ , **intersects**\ , **overlaps**\ , **touches**\ , and **within**\ . 28 | 29 | .. note:: 30 | The *operator* is not case sensitive. However, the *geometry* value(s) should follow the proper formatting and cases for their type (i.e. follow the proper formatting for GeoJSON, WKT, etc.). 31 | 32 | The *geometry* can be: 33 | 34 | +-------------+-------------------------------------------+----------------------------------------------------------------------------------+ 35 | | Type | Info | Example | 36 | +=============+===========================================+==================================================================================+ 37 | | A filename | - | Can be a ESRI Shapefile, KML, GML or | .. code-block:: python | 38 | | | | GeoJSON | | 39 | | | - | Can contain points, lines or | feats = [('contains', 'C:\\TEMP\\test.geojson')] | 40 | | | | polygons and have multiple features | | 41 | +-------------+-------------------------------------------+----------------------------------------------------------------------------------+ 42 | | WKT format | - Can be a point, line or polygon. | .. code-block:: python | 43 | | | | | 44 | | | | feats = [ | 45 | | | | ('intersects', 'POINT (-75.92790414335645721 45.63414106580390239)'), | 46 | | | | ('intersects', 'POINT (-76.04462125987681986 46.23234274318053849)') | 47 | | | | ] | 48 | +-------------+-------------------------------------------+----------------------------------------------------------------------------------+ 49 | | GeoJSON | - | The 'geometry' entry from a GeoJSON | .. code-block:: python | 50 | | | | Feature. | | 51 | | | - | Can be a point, line or polygon. | ('within', { | 52 | | | | "type":"Polygon", | 53 | | | | "coordinates":[ | 54 | | | | [ | 55 | | | | [-75.71484393257714,45.407703298380106], | 56 | | | | [-75.6962772564671,45.40738537380734], | 57 | | | | [-75.69343667852566,45.39264326981817], | 58 | | | | [-75.71826085966613,45.390764097853655], | 59 | | | | [-75.71484393257714,45.407703298380106] | 60 | | | | ] | 61 | | | | ] | 62 | | | | }) | 63 | +-------------+-------------------------------------------+----------------------------------------------------------------------------------+ 64 | | Coordinates | - | A list of coordinates of a polygon | .. code-block:: python | 65 | | | | (ex: ```[(x1, y1), (x2, y2), ...]```) | | 66 | | | - | A single point (ex: ```[(x1, y1)]```) | feats = [ | 67 | | | | ('contains', [ | 68 | | | | (-75.71, 45.41), | 69 | | | | (-75.70, 45.41), | 70 | | | | (-75.69, 45.39), | 71 | | | | (-75.72, 45.39), | 72 | | | | (-75.71, 45.41) | 73 | | | | ] | 74 | | | | ) | 75 | | | | ] | 76 | +-------------+-------------------------------------------+----------------------------------------------------------------------------------+ 77 | 78 | .. note:: 79 | The `GDAL Python package `_ is required if you wish to use shapefiles. 80 | 81 | WKT example to get results for the easternmost and westernmost points of Canada: 82 | 83 | .. code-block:: python 84 | 85 | >>> feats = [('intersects', 'POINT (-141.001944 60.306389)'), ('intersects', 'POINT (-52.619444 47.523611)')] 86 | 87 | Date Range(s) 88 | ------------- 89 | 90 | The **date range** is either: 91 | 92 | - A list of date range dictionaries containing a *start* and *end* key. The date values should be in format *YYYYMMDD_HHMMSS*. 93 | - A date of a previous time interval (ex: '24 hours', '7 days'). Available intervals are 'hour', 'day', 'week', 'month' or 'year' (plural is permitted). 94 | 95 | For example, to search for images between January 1, 2019 at midnight to September 15, 2019 at 3:35:55 PM and in the last 3 days, use: 96 | 97 | .. code-block:: python 98 | 99 | >>> dates = [{"start": "20190101_000000", "end": "20190915_153555"}, "3 days"] 100 | 101 | Query Filter(s) 102 | --------------- 103 | 104 | The **query** variable is a dictionary containing filter titles as keys and tuples with the operator and filter value such as: ``{: (, ), ...}`` 105 | 106 | Example of beam mnemonic filter: ``{'Beam Mnemonic': ('like', ['16M%', '3M11'])}`` 107 | 108 | The *operator* can be one of the following: **=**\ , **<**\ , **>**\ , **<>**\ , **<=**\ , **>=**\ , **like**\ , **starts with**\ , **ends with**\ , or **contains**. 109 | 110 | .. note:: 111 | The *operator* is not case sensitive. However, *fields* and *values* are case sensitive. 112 | 113 | The following example will search for images with **Beam Mnemonic** that equals '3M11' or contains '16M' and with **Incidence Angle** greater than or equal to 45 degrees: 114 | 115 | .. code-block:: python 116 | 117 | >>> filters = {'Beam Mnemonic': ('like', 'SC50%'), 'Incidence Angle': ('<=', '45')} 118 | 119 | Get Available Fields 120 | -------------------- 121 | 122 | You can get a list of available query fields using the ``get_available_fields`` and passing the **Collection ID**. 123 | 124 | There are 3 ways to get the available fields for a Collection using the **\ *name_type*\ ** argument of the ``get_available_fields`` function: 125 | 126 | +-------------+-------------------------------------------+-----------------------------------------------------------------------------+ 127 | | Value | Description | Results | 128 | +=============+===========================================+=============================================================================+ 129 | | empty | | Gets the raw field information from the | .. code-block:: python | 130 | | | | RAPI. | | 131 | | | | print(rapi.get_available_fields('RCMImageProducts')) | 132 | | | | {'search': { | 133 | | | | 'Special Handling Required': { | 134 | | | | 'id': 'RCM.SPECIAL_HANDLING_REQUIRED', | 135 | | | | 'datatype': 'String'}, | 136 | | | | 'Client Order Number': { | 137 | | | | 'id': 'ARCHIVE_IMAGE.CLIENT_ORDER_NUMBER', | 138 | | | | 'datatype': 'String'}, | 139 | | | | ...}, | 140 | | | | 'results': { | 141 | | | | 'Buyer Id': { | 142 | | | | 'id': 'ARCHIVE_IMAGE.AGENCY_BUYER', | 143 | | | | 'datatype': 'Integer'}, | 144 | | | | 'Archive Visibility Start Date': { | 145 | | | | 'id': 'ARCHIVE_IMAGE.ARCH_VISIBILITY_START', | 146 | | | | 'datatype': 'Date'}, | 147 | | | | ...} | 148 | | | | } | 149 | +-------------+-------------------------------------------+-----------------------------------------------------------------------------+ 150 | | **id** | Gets a list of field IDs. | .. code-block:: python | 151 | | | | | 152 | | | | print(rapi.get_available_fields('RCMImageProducts', name_type='id')) | 153 | | | | {'search': [ | 154 | | | | 'RCM.SPECIAL_HANDLING_REQUIRED', | 155 | | | | 'ARCHIVE_IMAGE.CLIENT_ORDER_NUMBER', | 156 | | | | ...], | 157 | | | | 'results': [ | 158 | | | | 'ARCHIVE_IMAGE.AGENCY_BUYER', | 159 | | | | 'ARCHIVE_IMAGE.ARCH_VISIBILITY_START', | 160 | | | | ...] | 161 | | | | } | 162 | +-------------+-------------------------------------------+-----------------------------------------------------------------------------+ 163 | | **title** | | Gets a list of field names (these are | .. code-block:: python | 164 | | | | used when performing a search using the | | 165 | | | | EODMSRAPI). | print(rapi.get_available_fields('RCMImageProducts', name_type='title')) | 166 | | | | {'search': [ | 167 | | | | 'Special Handling Required', | 168 | | | | 'Client Order Number', | 169 | | | | ...], | 170 | | | | 'results': [ | 171 | | | | 'Buyer Id', 'Archive Visibility Start Date', | 172 | | | | ...] | 173 | | | | } | 174 | +-------------+-------------------------------------------+-----------------------------------------------------------------------------+ 175 | 176 | Get Available Field Choices 177 | --------------------------- 178 | 179 | Some fields have specific choices that the user can enter. These values are included in the ``get_available_fields`` *empty* results, however the function ``get_field_choices`` in the EODMSRAPI offers results easier to manipulate. 180 | 181 | The ``get_field_choices`` function requires a **Collection ID** and an optional **field** name or ID. If no field is specified, all fields and choices for the specified Collection will be returned. 182 | 183 | Example of choices for the Polarization field in RCM: 184 | 185 | .. code-block:: python 186 | 187 | >>> rapi.get_field_choices('RCMImageProducts', 'Polarization') 188 | ['CH CV', 'HH', 'HH HV', 'HH HV VH VV', 'HH VV', 'HV', 'VH', 'VH VV', 'VV'] 189 | 190 | Result Fields 191 | ------------- 192 | 193 | The next value to set is the **result fields**. The raw JSON results from the RAPI returns only a select few fields. For example, when searching RCM images, the RAPI only returns metadata for these Field IDs: 194 | 195 | .. code-block:: 196 | 197 | RCM.ORBIT_REL 198 | ARCHIVE_IMAGE.PROCESSING_DATETIME 199 | ARCHIVE_IMAGE.PRODUCT_TYPE 200 | IDX_SENSOR.SENSOR_NAME 201 | RCM.SBEAMFULL 202 | RCM.POLARIZATION 203 | RCM.SPECIAL_HANDLING_REQUIRED_R 204 | CATALOG_IMAGE.START_DATETIME 205 | RELATED_PRODUCTS 206 | RCM.SPECIAL_HANDLING_INSTRUCTIONS 207 | Metadata 208 | RCM.DOWNLINK_SEGMENT_ID 209 | 210 | If you want more fields returned, you can create a list and add Field IDs (found in the 'results' entry of the ``get_available_fields`` method results, in bold below) of fields you'd like included in the results JSON. 211 | 212 | 213 | .. code-block:: python 214 | 215 | >>> print(rapi.get_available_fields('RCMImageProducts')) 216 | {'search': 217 | { 218 | [...] 219 | }, 220 | 'results': 221 | { 222 | 'Buyer Id': {'id': ' \*ARCHIVE_IMAGE.AGENCY_BUYER*\ ', 'datatype': 'Integer'}, 223 | [...] 224 | } 225 | } 226 | 227 | 228 | .. note:: 229 | The **result fields** parameter is not necessary if you use the 'full' option when getting the results after the search; see `Get Results <#get-results>`_ for more information. 230 | 231 | For example, the following will include the Processing Facility and Look Orientation of the images: 232 | 233 | .. code-block:: python 234 | 235 | >>> result_fields = ['ARCHIVE_RCM.PROCESSING_FACILITY', 'RCM.ANTENNA_ORIENTATION'] 236 | 237 | Submit Search 238 | ------------- 239 | 240 | Now submit the search, in this example, setting the **Collection ID** to 'RCMImageProducts' and **max results** to 100: 241 | 242 | .. code-block:: python 243 | 244 | >>> rapi.search("RCMImageProducts", filters=filters, features=feats, dates=dates, result_fields=result_fields, max_results=100) 245 | | EODMSRAPI | Searching for RCMImageProducts images on RAPI 246 | | EODMSRAPI | Querying records within 1 to 1000... 247 | | EODMSRAPI | Number of RCMImageProducts images returned from RAPI: 9 248 | 249 | Get Results 250 | ----------- 251 | 252 | Before getting the results, set the field type to return: 253 | 254 | * 255 | **camel** (default): All field names will be in lower camelcase (ex: fieldName) 256 | * 257 | **upper**\ : Field names will be in upper case with underscore for spaces (ex: FIELD_NAME) 258 | * 259 | **words**\ : Field names will be English words (ex: Field Name) 260 | 261 | .. code-block:: python 262 | 263 | >>> rapi.set_field_convention('upper') 264 | 265 | .. note:: 266 | Changing the field name convention does not apply when using the 'raw' parameter for the ``get_results`` method. 267 | 268 | Now to get the results of your search using the ``get_results`` method. 269 | 270 | There are three options for getting results: 271 | 272 | * 273 | **raw** (default): The raw JSON data results from the RAPI. Only the basic fields and the fields you specified in the result_fields will be returned. 274 | 275 | .. code-block:: python 276 | 277 | >>> print(rapi.get_results('raw')) 278 | [ 279 | { 280 | "recordId": "7822244", 281 | "overviewUrl": "http://was-eodms.compusult.net/wes/images/No_Data_Available.png", 282 | "collectionId": "RCMImageProducts", 283 | "metadata2": [ 284 | { 285 | "id": "RCM.ANTENNA_ORIENTATION", 286 | "value": "Right", 287 | "label": "Look Orientation" 288 | }, 289 | { 290 | "id": "ARCHIVE_IMAGE.PROCESSING_DATETIME", 291 | "value": "2020-11-09 13:49:14 GMT", 292 | "label": "Processing Date" 293 | }, 294 | { 295 | "id": "ARCHIVE_IMAGE.PRODUCT_TYPE", 296 | "value": "GRD", 297 | "label": "Type" 298 | }, 299 | [...] 300 | ], 301 | "rapiOrderUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/order/direct?collection=RCMImageProducts&recordId=7822244&destination=fill_me_in", 302 | "geometry": { 303 | "type": "Polygon", 304 | "coordinates": [ 305 | [ 306 | [ 307 | -111.2061013084167, 308 | 62.4209316874871 309 | ], 310 | [ 311 | -111.2710554014949, 312 | 62.22606212562155 313 | ], 314 | [ 315 | -110.6882156023417, 316 | 62.18309404584561 317 | ], 318 | [ 319 | -110.6194709629304, 320 | 62.3778734605923 321 | ], 322 | [ 323 | -111.2061013084167, 324 | 62.4209316874871 325 | ] 326 | ] 327 | ] 328 | }, 329 | "title": "RCM2_OK1370026_PK1375301_3_16M17_20201109_134014_HH_HV_GRD", 330 | "orderExecuteUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/Client/?entryPoint=preview#?cseq=RCMImageProducts&record=7822244", 331 | "thumbnailUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/getObject?FeatureID=62f0e816-8006-4768-8f32-6ef4008e6895-7822244&ObjectType=Thumbview&collectionId=RCMImageProducts", 332 | "metadataUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/Client/?entryPoint=resultDetails&resultId=7822244&collectionId=RCMImageProducts", 333 | "isGeorectified": "False", 334 | "collectionTitle": "RCM Image Products", 335 | "isOrderable": "True", 336 | "thisRecordUrl": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/record/RCMImageProducts/7822244", 337 | "metadata": [ 338 | [ 339 | "Look Orientation", 340 | "Right" 341 | ], 342 | [...] 343 | ] 344 | }, 345 | [...] 346 | ] 347 | 348 | 349 | * 350 | **full**\ : The full metadata for each image in the results from the RAPI. 351 | 352 | .. note:: 353 | When running the ```get_results``` function for the first time, the 'full' option will require calls to the RAPI to fetch all the metadata for each image. This can take time depending on the number of images returned from the search. 354 | 355 | The following example is the output from the 'full' results returned from the RAPI when using the 'upper' field name convention: 356 | 357 | .. code-block:: python 358 | 359 | >>> print(rapi.get_results('full')) 360 | | EODMSRAPI | Fetching result metadata: 100%|████████████████████████████████████████| 29/29 [00:07<00:00, 3.81item/s] 361 | [ 362 | { 363 | "RECORD_ID": "8572605", 364 | "COLLECTION_ID": "RCMImageProducts", 365 | "GEOMETRY": { 366 | "type": "Polygon", 367 | "coordinates": [ 368 | [ 369 | [ 370 | -75.87136946742638, 371 | 45.53642826726489 372 | ], 373 | [ 374 | -75.88537895138599, 375 | 45.47880111111606 376 | ], 377 | [ 378 | -75.63233378406722, 379 | 45.44847937835439 380 | ], 381 | [ 382 | -75.61805821213746, 383 | 45.50610429149886 384 | ], 385 | [ 386 | -75.87136946742638, 387 | 45.53642826726489 388 | ] 389 | ] 390 | ] 391 | }, 392 | "TITLE": "rcm_20210407_N4549W07575", 393 | "COLLECTION_TITLE": "RCM Image Products", 394 | "IS_ORDERABLE": true, 395 | "THIS_RECORD_URL": "https://www.eodms-sgdot.nrcan-rncan.gc.ca/wes/rapi/record/RCMImageProducts/8572605", 396 | "ABSOLUTE_ORBIT": "9917.0", 397 | "ACQUISITION_END_DATE": "2021-04-07 11:12:05 GMT", 398 | "ACQUISITION_START_DATE": "2021-04-07 11:12:04 GMT", 399 | "ARCHIVE_VISIBILITY_START_DATE": "2021-04-07 11:12:04 GMT", 400 | "BEAM_MNEMONIC": "FSL22", 401 | "BEAM_MODE_DEFINITION_ID": "422", 402 | [...] 403 | "VISIBILITY_RESTRICTION_EXPIRY_DATE": "2021-04-07 11:12:06 GMT", 404 | "WITHIN_ORBIT_TUBE": "true", 405 | "WKT_GEOMETRY": "POLYGON ((-75.8713694674264 45.5364282672649 0,-75.885378951386 45.4788011111161 0,-75.6323337840672 45.4484793783544 0,-75.6180582121375 45.5061042914989 0,-75.8713694674264 45.5364282672649 0))" 406 | }, 407 | [...] 408 | ] 409 | 410 | 411 | * 412 | **geojson**\ : The results will be returned in GeoJSON format. 413 | 414 | .. note: 415 | When running the ```get_results``` function for the first time, the 'geojson' option will require calls to the RAPI to fetch all the metadata for each image. This can take time depending on the number of images returned from the search. 416 | 417 | The following example is the output from the 'geojson' results returned from the RAPI when using the 'upper' field name convention: 418 | 419 | .. code-block:: python 420 | 421 | >>> print(rapi.get_results('geojson')) 422 | | EODMSRAPI | Fetching result metadata: 100%|████████████████████████████████████████| 29/29 [00:07<00:00, 3.86item/s] 423 | { 424 | "type": "FeatureCollection", 425 | "features": [ 426 | { 427 | "type": "Feature", 428 | "geometry": { 429 | "type": "Polygon", 430 | "coordinates": [ 431 | [ 432 | [ 433 | -75.87136946742638, 434 | 45.53642826726489 435 | ], 436 | [ 437 | -75.88537895138599, 438 | 45.47880111111606 439 | ], 440 | [ 441 | -75.63233378406722, 442 | 45.44847937835439 443 | ], 444 | [ 445 | -75.61805821213746, 446 | 45.50610429149886 447 | ], 448 | [ 449 | -75.87136946742638, 450 | 45.53642826726489 451 | ] 452 | ] 453 | ] 454 | }, 455 | "properties": { 456 | "RECORD_ID": "8572605", 457 | "COLLECTION_ID": "RCMImageProducts", 458 | "GEOMETRY": { 459 | "type": "Polygon", 460 | "coordinates": [ 461 | [ 462 | [ 463 | -75.87136946742638, 464 | 45.53642826726489 465 | ], 466 | [ 467 | -75.88537895138599, 468 | 45.47880111111606 469 | ], 470 | [ 471 | -75.63233378406722, 472 | 45.44847937835439 473 | ], 474 | [ 475 | -75.61805821213746, 476 | 45.50610429149886 477 | ], 478 | [ 479 | -75.87136946742638, 480 | 45.53642826726489 481 | ] 482 | ] 483 | ] 484 | }, 485 | [...] 486 | "VISIBILITY_RESTRICTION_EXPIRY_DATE": "2021-04-07 11:12:06 GMT", 487 | "WITHIN_ORBIT_TUBE": "true", 488 | "WKT_GEOMETRY": "POLYGON ((-75.8713694674264 45.5364282672649 0,-75.885378951386 45.4788011111161 0,-75.6323337840672 45.4484793783544 0,-75.6180582121375 45.5061042914989 0,-75.8713694674264 45.5364282672649 0))" 489 | } 490 | }, 491 | [...] 492 | ] 493 | } 494 | 495 | .. code-block:: python 496 | 497 | >>> res = rapi.get_results('full') 498 | | EODMSRAPI | Fetching result metadata: 100%|██████████████████████████████████████████| 9/9 [00:02<00:00, 4.40item/s] 499 | 500 | Print Results 501 | ------------- 502 | 503 | The EODMSRAPI has a ``print_results`` function which will print the results in pretty print. You can pass a specific results from the RAPI to the function. If not, the 'full' results will be printed. 504 | 505 | .. note:: 506 | If you haven't run ``get_results`` prior to ``print_results``\ , the EODMSRAPI will first fetch the full metadata which can some time depending on the number of results. 507 | 508 | .. code-block:: python 509 | 510 | >>> rapi.print_results() 511 | [ 512 | { 513 | "RECORD_ID": "13791752", 514 | "COLLECTION_ID": "RCMImageProducts", 515 | ... 516 | 517 | .. note:: 518 | In Linux, if you get the error ``UnicodeEncodeError: 'ascii' codec can't encode character...``\ , add ``export LC_CTYPE=en_US.UTF-8`` to the "~/.bashrc" file and run ``source ~/.bashrc``. 519 | 520 | Multiple Searches 521 | ----------------- 522 | 523 | As of v1.5.0, you can now perform more than one search and the news results will be added to the existing results. 524 | 525 | .. code-block:: python 526 | 527 | >>> len(res) # This line is not needed, only to show number of results before new search 528 | 9 529 | >>> rapi.search('RCMImageProducts', max_results=4) 530 | 531 | | EODMSRAPI | Searching for RCMImageProducts images on RAPI 532 | | EODMSRAPI | Querying records within 1 to 1000... 533 | | EODMSRAPI | Number of RCMImageProducts images returned from RAPI: 4 534 | >>> res = rapi.get_results('full') 535 | | EODMSRAPI | Fetching result metadata: 100%|████████████████████████████████████████| 13/13 [00:03<00:00, 4.21item/s] 536 | >>> len(res) # This line is not needed, only to show number of results after new search 537 | 13 538 | 539 | Clear Results 540 | ------------- 541 | 542 | Since each search will add to any existing results, it is important to clear the results whenever needed. 543 | 544 | .. code-block:: python 545 | 546 | >>> rapi.clear_results() 547 | 548 | Full Search Code Example 549 | ------------------------ 550 | 551 | .. code-block:: python 552 | 553 | from eodms_rapi import EODMSRAPI 554 | 555 | # Initialize EODMSRAPI using your EODMS account credentials 556 | rapi = EODMSRAPI('eodms-username', 'eodms-password') 557 | 558 | # Set features using the easternmost and westernmost points of Canada in WKT format 559 | feats = [('intersects', 'POINT (-141.001944 60.306389)'), \ 560 | ('intersects', 'POINT (-52.619444 47.523611)')] 561 | 562 | # Set date ranges 563 | dates = [{"start": "20190101_000000", "end": "20190915_153555"}, 564 | {"start": "20201013_120000", "end": "20201113_150000"}] 565 | 566 | # Set search filters 567 | filters = {'Beam Mnemonic': ('like', 'SC50%'), \ 568 | 'Incidence Angle': ('<=', '45')} 569 | 570 | # Set the results fields 571 | result_fields = ['ARCHIVE_RCM.PROCESSING_FACILITY', 'RCM.ANTENNA_ORIENTATION'] 572 | 573 | # Submit search 574 | rapi.search("RCMImageProducts", filters, feats, dates, result_fields, 100) 575 | 576 | # Get results 577 | rapi.set_field_convention('upper') 578 | res = rapi.get_results('full') 579 | 580 | # Print current results 581 | rapi.print_results(res) 582 | 583 | # Perform another search 584 | rapi.search('RCMImageProducts', max_results=4) 585 | res = rapi.get_results('full') 586 | 587 | # Clear results 588 | rapi.clear_results() 589 | --------------------------------------------------------------------------------