├── .gitignore ├── 1-Getting-Started ├── 1-Welcome.md ├── 2-Using-the-Playground.md ├── 3-Using-NASA-Earthdata.md └── 4-Glossary.md ├── 2-Data-How-Tos └── delete.txt ├── 3-Data-Stories ├── Above_Ground_Biomass_Density.ipynb ├── Data_stories_template.ipynb ├── Hurricanes_Wind_and_Sea_Surface_Temperature.ipynb ├── Observing_Seasonal_Ag_Changes.ipynb └── Sea_Level_Rise.ipynb ├── 4-Offboarding └── delete.txt └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | # Editors 2 | .vscode/ 3 | .idea/ 4 | 5 | # Vagrant 6 | .vagrant/ 7 | 8 | # Mac/OSX 9 | .DS_Store 10 | 11 | # Windows 12 | Thumbs.db 13 | 14 | # Source for the following rules: https://raw.githubusercontent.com/github/gitignore/master/Python.gitignore 15 | # Byte-compiled / optimized / DLL files 16 | __pycache__/ 17 | *.py[cod] 18 | *$py.class 19 | 20 | # C extensions 21 | *.so 22 | 23 | # Distribution / packaging 24 | .Python 25 | build/ 26 | develop-eggs/ 27 | dist/ 28 | downloads/ 29 | eggs/ 30 | .eggs/ 31 | lib/ 32 | lib64/ 33 | parts/ 34 | sdist/ 35 | var/ 36 | wheels/ 37 | *.egg-info/ 38 | .installed.cfg 39 | *.egg 40 | MANIFEST 41 | 42 | # PyInstaller 43 | # Usually these files are written by a python script from a template 44 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 45 | *.manifest 46 | *.spec 47 | 48 | # Installer logs 49 | pip-log.txt 50 | pip-delete-this-directory.txt 51 | 52 | # Unit test / coverage reports 53 | htmlcov/ 54 | .tox/ 55 | .nox/ 56 | .coverage 57 | .coverage.* 58 | .cache 59 | nosetests.xml 60 | coverage.xml 61 | *.cover 62 | .hypothesis/ 63 | .pytest_cache/ 64 | 65 | # Translations 66 | *.mo 67 | *.pot 68 | 69 | # Django stuff: 70 | *.log 71 | local_settings.py 72 | db.sqlite3 73 | 74 | # Flask stuff: 75 | instance/ 76 | .webassets-cache 77 | 78 | # Scrapy stuff: 79 | .scrapy 80 | 81 | # Sphinx documentation 82 | docs/_build/ 83 | 84 | # PyBuilder 85 | target/ 86 | 87 | # Jupyter Notebook 88 | .ipynb_checkpoints 89 | 90 | # IPython 91 | profile_default/ 92 | ipython_config.py 93 | 94 | # pyenv 95 | .python-version 96 | 97 | # celery beat schedule file 98 | celerybeat-schedule 99 | 100 | # SageMath parsed files 101 | *.sage.py 102 | 103 | # Environments 104 | .env 105 | .venv 106 | env/ 107 | venv/ 108 | ENV/ 109 | env.bak/ 110 | venv.bak/ 111 | 112 | # Spyder project settings 113 | .spyderproject 114 | .spyproject 115 | 116 | # Rope project settings 117 | .ropeproject 118 | 119 | # mkdocs documentation 120 | /site 121 | 122 | # mypy 123 | .mypy_cache/ 124 | .dmypy.json 125 | dmypy.json -------------------------------------------------------------------------------- /1-Getting-Started/1-Welcome.md: -------------------------------------------------------------------------------- 1 | # Welcome 2 | 3 | ## What is this space? 4 | The Earthdata Cloud Playground is a [JupyterHub|https://jupyter.org/hub] built on top of Amazon Web Services (AWS), the same cloud provider that currently hosts NASA Earthdata. This space allows users an opportunity to trial the "in-place analysis" that allows them to access and analyze NASA Earthdata data collections from the cloud without downloading data. 5 | 6 | The Earthdata Cloud Playground aims to provide an easy path for users to both test their workflows running in the cloud and facilitate a transition to their own separately-funded cloud environment. The Playground is built as a cross-DAAC collaboration with contributions to content coming from DAAC representatives of different disciplines. 7 | 8 | - Scope of space, limitations and benefits of using the Sandbox 9 | - Describe different areas of the JupyterHub: file nav bar on left, terminals, jupyter notebooks, shared areas (if exist), etc. 10 | 11 | ## Where to go for help and provide feedback 12 | - [NASA Earthdata Forum|https://forum.earthdata.nasa.gov/]: Questions and feedback concerning NASA Earthdata collections and services 13 | - [GitHub Issues|https://github.com/nasa/Earthdata-cloud-playground/issues]: Questions, concerns, suggestions and feedback about the content and experience provided in the Playground. 14 | 15 | 16 | Citation? 17 | License? 18 | -------------------------------------------------------------------------------- /1-Getting-Started/2-Using-the-Playground.md: -------------------------------------------------------------------------------- 1 | # Using the Playground 2 | 3 | ## __we may not need this__ How do I get access to the Earthdata Cloud Playground? 4 | 5 | ## Ground rules 6 | This space is meant for small, temporary experiments that help users to get to know cloud-based features and workflows before deciding whether to implement them on their own cloud-hosted resources. Below are some ground rules to ensure prudent and safe use of the Playground: 7 | - Choose the smallest server size to start 8 | - As you progress in your learning, you may find you need to use a larger server size. Please scale appropriately. 9 | - Start with a small, testable portion of your larger workflow. 10 | - Log out when you are finished for your session or not actively using it 11 | - Sub-group 1 met with OpenScienceLab from ASF and one of the things they did to encourage users to do this was: 12 | - Rename the Logout button to "Shutdown and Logout" 13 | 14 | ## How do I use what is provided? 15 | - We recommend starting to explore the __[Data How-Tos|../Data-How-Tos/]__ folder, which offers building blocks for NASA Earthdata search and access in the cloud. 16 | - Then, to see how those building blocks can be used to piece together a scientific workflow in the cloud, explore the __[Data Stories|../Data-Stories/]__ folder. 17 | - If after this exploration you find that a cloud workflow would be beneficial to you, you can find out what options can help you "graduate" from the Playground in the __[Offboarding|../Offboarding/]__ folder. 18 | - And for any background or support information you're looking for, check out the contents of the __[Getting Started|../Getting-Started/]__ folder. 19 | 20 | ## How do I bring in my own notebooks, scripts or code? 21 | There are a few ways to bring your own existing notebooks, scripts, and codes to the Earthdata Cloud Hub. 22 | - Drag and drop a file from your local computer into the Earthdata Hub’s File Browser 23 | - Clone a repository into the Earthdata Hub 24 | 25 | ## How do I upload my own data? 26 | 27 | ## How do I save my code and/or share it with others outside of the Earthdata Cloud Playground? 28 | - Insert Github 101 as a suggested pathway e.g. https://nasa-openscapes.github.io/earthdata-cloud-cookbook/contributing/workflow.html#github-workflow 29 | 30 | ## How do I install a package that’s missing from the default cloud environment? 31 | - You may use conda or pip to install a package that is not currently in the Playground. _However, this installation will not persist between sessions of active use._ You will need to install the package each session. 32 | - Write a request in the NASA Earthdata Cloud Playground GitHub repository's [Issues|https://github.com/nasa/Earthdata-cloud-playground/issues] tab listing which package you require and what necessitates its use for your workflow. A Playground maintainer will consider your request, and if approved, the package will be integrated into the default list of packages available. 33 | 34 | ## How do I create and use a new kernel/environment? 35 | - Using corn? https://nasa-openscapes.github.io/earthdata-cloud-cookbook/environment-setup/ 36 | 37 | ## How do I monitor my cloud cost and compute resources usage in this Hub? 38 | AWS Console? - does this go into offboarding? 39 | 40 | ## How do I log out and shut down my instance once I am done using it? 41 | 42 | 43 | ### Want more tutorials? Here’s our respective Repo’s 44 | -------------------------------------------------------------------------------- /1-Getting-Started/3-Using-NASA-Earthdata.md: -------------------------------------------------------------------------------- 1 | # Working with NASA Earthdata 2 | 3 | ## How to authenticate, search, & access data - earthaccess 4 | 5 | ## How do I … 6 | - Find data programmatically - earthaccess tutorial 7 | - Find data using Earthdata Search - Earthdata Search tutorial 8 | - Access data in the cloud - earthaccess tutorial in the cloud 9 | - Read data - different formats 10 | - Access & subset data - Earthdata Cloud Clinic 11 | 12 | ## How do I utilize the power of cloud computing? 13 | - Spatial subsetting and reading directly into memory 14 | - Spin up Dask cluster 15 | - GDAL transformation 16 | - (Maybe these could focus on the Scaling up part? so Dask, Lambda ..?) 17 | 18 | ## More specific how to’s? 19 | - (opening file formats, reformatting, regridding, dask cluster, gdal transformation) 20 | 21 | Q: where is the limit here? Want to keep things simple. 22 | -------------------------------------------------------------------------------- /1-Getting-Started/4-Glossary.md: -------------------------------------------------------------------------------- 1 | # Glossary 2 | 3 | You will encounter new concepts and terms when accessing data from the cloud archive and using computing resources in the cloud. Find a list of terms and definitions, concepts, and links to cheatsheets below to help you navigate the jargon. 4 | 5 | *__Populate this page with terms and definitions__* 6 | 7 | TODO: Create a list of key terms to define here 8 | 9 | ## Cloud Terminology 101 10 | 11 | Cloud Terminology 101 for those new to commonly used cloud computing terms and phrases. 12 | 13 | 16 | 17 | [Published Google Slide](https://docs.google.com/presentation/d/e/2PACX-1vScDfFwgPMf6mU_UP9qnVs2F4Rtumv7OG4CxgO_YZW8A_VJOI4gGIimcSWbCAMt7rTV1N4kYATjQSCy/pub?start=false&loop=false&delayms=3000){target="_blank"} 18 | 19 | ## Workflow Cheatsheet Terminology 20 | 21 | Terminology cheatsheet to explain terms commonly used in cloud computing and those located on the NASA Earthdata Cloud Cheatsheet. See also [NASA Earthdata Glossary](https://earthdata.nasa.gov/learn/user-resources/glossary). 22 | 23 | 26 | 27 | [Published Google Slide](https://docs.google.com/presentation/d/e/2PACX-1vSekMsRRaREYOA81prJvcGfj1-qqm3VZxNDtaJDNIcohk2364cH-f_R07lDrljjd5fOjyakiXK12K5p/pub?start=false&loop=false&delayms=3000){target="_blank"} -------------------------------------------------------------------------------- /2-Data-How-Tos/delete.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nasa/Earthdata-cloud-playground/f5ad6fc9a93217e087aecdb3f954a48ca38e3636/2-Data-How-Tos/delete.txt -------------------------------------------------------------------------------- /3-Data-Stories/Data_stories_template.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "74ac889e-99ed-41de-9ec7-dffe81d78ac7", 6 | "metadata": {}, 7 | "source": [ 8 | "# Title\n", 9 | "## Overview\n", 10 | "\n", 11 | "Add Summary of data story/background information needed. 2-5 sentences. \n", 12 | "\n", 13 | "## Learning Objectives\n", 14 | "- enter objective \n", 15 | "- enter objective \n", 16 | "- ...\n", 17 | "\n", 18 | "## Requirements\n", 19 | "### 1. Compute environment - This notebook can **only** be run in an **AWS Cloud instance running in the `us-west-2` region**.\n", 20 | "### 2. Earthdata Login\n", 21 | "An Earthdata Login account is required to access data, as well as discover restricted data, from the NASA Earthdata system. Thus, to access NASA data, you need Earthdata Login. Please visit https://urs.earthdata.nasa.gov to register and manage your Earthdata Login account. This account is free to create and only takes a moment to set up.\n", 22 | "### 3. Additional Requirements\n", 23 | "...\n", 24 | "\n", 25 | "### Install Packages" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "execution_count": null, 31 | "id": "811a21bc-cf2f-47d0-8050-992cc2aca10a", 32 | "metadata": {}, 33 | "outputs": [], 34 | "source": [ 35 | "import earthaccess\n", 36 | "import xarray as xr" 37 | ] 38 | }, 39 | { 40 | "cell_type": "markdown", 41 | "id": "65d18415-e44d-4622-a86d-3cb9f3cfd862", 42 | "metadata": {}, 43 | "source": [ 44 | "We recommend authenticating your Earthdata Login (EDL) information using the `earthaccess` python package as follows:" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "id": "bf22b7ed-b1d7-4dc3-8041-984146226387", 51 | "metadata": {}, 52 | "outputs": [], 53 | "source": [ 54 | "auth = earthaccess.login(strategy=\"netrc\") # works if the EDL login already been persisted to a netrc\n", 55 | "if not auth.authenticated:\n", 56 | " # ask for EDL credentials and persist them in a .netrc file\n", 57 | " auth = earthaccess.login(strategy=\"interactive\", persist=True)" 58 | ] 59 | } 60 | ], 61 | "metadata": { 62 | "kernelspec": { 63 | "display_name": "Python 3 (ipykernel)", 64 | "language": "python", 65 | "name": "python3" 66 | }, 67 | "language_info": { 68 | "codemirror_mode": { 69 | "name": "ipython", 70 | "version": 3 71 | }, 72 | "file_extension": ".py", 73 | "mimetype": "text/x-python", 74 | "name": "python", 75 | "nbconvert_exporter": "python", 76 | "pygments_lexer": "ipython3", 77 | "version": "3.9.16" 78 | } 79 | }, 80 | "nbformat": 4, 81 | "nbformat_minor": 5 82 | } 83 | -------------------------------------------------------------------------------- /4-Offboarding/delete.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nasa/Earthdata-cloud-playground/f5ad6fc9a93217e087aecdb3f954a48ca38e3636/4-Offboarding/delete.txt -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Earthdata-cloud-playground 2 | Content will be open and available for use by our user communities, and will be used to help foster open science practices with our external collaborators. 3 | --------------------------------------------------------------------------------