├── .gitignore ├── .ipynb_checkpoints └── README-checkpoint.md ├── LICENSE ├── README.md ├── environment.yml ├── figures └── DC-data_availability.png ├── notebooks ├── .ipynb_checkpoints │ ├── baseline_oi-checkpoint.ipynb │ ├── example_eval_baseline-checkpoint.ipynb │ ├── example_eval_bfn-checkpoint.ipynb │ ├── example_eval_convlstm_ssh-checkpoint.ipynb │ ├── example_eval_convlstm_ssh-sst-checkpoint.ipynb │ ├── example_eval_duacs-checkpoint.ipynb │ ├── example_eval_dymost-checkpoint.ipynb │ ├── example_eval_miost-checkpoint.ipynb │ ├── example_eval_neurost_ssh-sst-checkpoint.ipynb │ └── example_intercomparison-checkpoint.ipynb ├── baseline_oi.ipynb ├── example_data_access_aviso.ipynb ├── example_eval_4dvarnet.ipynb ├── example_eval_4dvarnet_v2022.ipynb ├── example_eval_4dvarqg.ipynb ├── example_eval_baseline.ipynb ├── example_eval_bfn.ipynb ├── example_eval_convlstm_ssh-sst.ipynb ├── example_eval_convlstm_ssh.ipynb ├── example_eval_duacs.ipynb ├── example_eval_dymost.ipynb ├── example_eval_miost.ipynb ├── example_eval_neurost_ssh-sst.ipynb └── example_intercomparison.ipynb ├── quickstart.ipynb └── src ├── .ipynb_checkpoints ├── mod_inout-checkpoint.py ├── mod_interp-checkpoint.py ├── mod_oi-checkpoint.py ├── mod_plot-checkpoint.py ├── mod_spectral-checkpoint.py ├── mod_stats-checkpoint.py └── mod_write-checkpoint.py ├── __pycache__ ├── mod_inout.cpython-312.pyc ├── mod_inout.cpython-37.pyc ├── mod_inout.cpython-38.pyc ├── mod_interp.cpython-312.pyc ├── mod_interp.cpython-37.pyc ├── mod_interp.cpython-38.pyc ├── mod_oi.cpython-37.pyc ├── mod_plot.cpython-312.pyc ├── mod_plot.cpython-37.pyc ├── mod_plot.cpython-38.pyc ├── mod_spectral.cpython-312.pyc ├── mod_spectral.cpython-37.pyc ├── mod_spectral.cpython-38.pyc ├── mod_stats.cpython-312.pyc ├── mod_stats.cpython-37.pyc ├── mod_stats.cpython-38.pyc ├── mod_write.cpython-312.pyc ├── mod_write.cpython-37.pyc └── mod_write.cpython-38.pyc ├── mod_inout.py ├── mod_interp.py ├── mod_oi.py ├── mod_plot.py ├── mod_spectral.py ├── mod_stats.py └── mod_write.py /.gitignore: -------------------------------------------------------------------------------- 1 | # ignoring results directory 2 | /results/ 3 | /data/ 4 | -------------------------------------------------------------------------------- /.ipynb_checkpoints/README-checkpoint.md: -------------------------------------------------------------------------------- 1 | [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.5511905.svg)](https://doi.org/10.5281/zenodo.5511905) 2 | 3 | # SSH Mapping Data Challenge 2021a 4 | 5 | This repository contains codes and sample notebooks for downloading and processing the SSH mapping data challenge. 6 | 7 | The quickstart can be run online by clicking here: 8 | [![Binder](https://binder.pangeo.io/badge_logo.svg)](https://binder.pangeo.io/v2/gh/ocean-data-challenges/2020a_SSH_mapping_NATL60/master?filepath=quickstart.ipynb) 9 | 10 | ## Motivation 11 | 12 | The goal is to investigate how to best reconstruct sequences of Sea Surface Height (SSH) maps from partial satellite altimetry observations. This data challenge follows an _Observation System Experiment_ framework: Satellite observations are from real sea surface height data from altimeter. The practical goal of the challenge is to investigate the best mapping method according to scores described below and in Jupyter notebooks. 13 | 14 | ### Observations 15 | The SSH observations include SARAL/Altika, Jason 2, Jason 3, Sentinel 3A, Haiyang-2A and Cryosat-2 altimeter data. This nadir altimeters constellation was operating during the 20170101-20171231 period. Note that for the mapping the Cryosat-2 altimeter data are not taken in the mapping to perfor the independent assessment of the various reconstructions. 16 | 17 | ### Data sequence and use 18 | 19 | The SSH reconstructions are assessed over the period from 2017-01-01 to 2017-12-31. 20 | 21 | For reconstruction methods that need a spin-up, the **observations** can be used from 2016-12-01 until the beginning of the evaluation period (31 days). This spin-up period is not included in the evaluation. For reconstruction methods that need learning from full fields, the **baseline reconstruction** or **duacs reconstrcution** can be used from 2017-01-01 to 2017-12-31. The altimeter data from Cryosat-2 should never be used so that any reconstruction can be considered uncorrelated to the evaluation period. 22 | 23 | ![Data Sequence](figures/DC-data_availability.png) 24 | 25 | ## Leaderboard 26 | 27 | | Method | µ(RMSE) | σ(RMSE) | λx (km) | Notes | Reference | 28 | |:---------|-----------:|----------:|----------:|:------------------|:-------------------------| 29 | | BASELINE | 0.85 | 0.09 | 140 | Covariances BASELINE OI | [example_eval_baseline.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_baseline.ipynb) | 30 | | DUACS | 0.88 | 0.07 | 152 | Covariances DUACS DT2018 | [example_eval_duacs.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_duacs.ipynb) | 31 | | MIOST | 0.89 | 0.08 | 139 | Multiscale mapping | [example_eval_miost.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_miost.ipynb) | 32 | | DYMOST | 0.89 | **0.06** | 129 | Dynamic mapping | [example_eval_dymost.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_dymost.ipynb) | 33 | | BNF | 0.88 | **0.06** | 122 | BFN mapping | [example_eval_bfn.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_bfn.ipynb) | 34 | | 4DVarNet (v2021) | 0.89 | **0.06** | 122 | 4DVarNet mapping | [example_eval_4DVarnet.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_4dvarnet.ipynb) | 35 | | 4DVarNet (v2022) | 0.89 | 0.09 | 109 | 4DVarNet mapping | [example_eval_4DVarnet_v2022.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_4dvarnet_v2022.ipynb) | 36 | | 4DVarQG | **0.90** | **0.06** | **106** | 4DVarQG mapping | [example_eval_4dvarqg.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_4dvarqg.ipynb) | 37 | | ConvLSTM SSH | **0.90** | **0.06** | 113 | ConvLSTM SSH | [example_eval_convlstm_ssh.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_convlstm_ssh.ipynb) | 38 | | ConvLSTM SSH-SST1 | **0.90** | **0.06** | **100** | ConvLSTM SSH-SST | [example_eval_convlstm_ssh-sst.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_convlstm_ssh-sst.ipynb) | 39 | | NeurOST SSH-SST1 | **0.90** | **0.06** | 114 | NeurOST SSH-SST (trained for global mapping) | [example_eval_neurost_ssh-sst.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_neurost_ssh-sst.ipynb) | 40 | 41 | 1: These methods additionally use L4 SST from the NASA MUR product as input. 42 | 43 | **µ(RMSE)**: average RMSE score. 44 | **σ(RMSE)**: standard deviation of the RMSE score. 45 | **λx**: minimum spatial scale resolved. 46 | 47 | ## Quick start 48 | You can follow the quickstart guide in [this notebook](https://github.com/ocean-data-challenges/2020a_SSH_mapping_NATL60/blob/master/quickstart.ipynb) or launch it directly from binder. 49 | 50 | ## Download the data 51 | The data are hosted on the [AVISO+ website](https://www.aviso.altimetry.fr/en/data/products/ocean-data-challenges/2021a-ssh-mapping-ose.html) and tagged with DOI: 10.24400/527896/a01-2021.005. The website also provides a data handbook. This is the recommended access. This [wiki](https://github.com/ocean-data-challenges/2020a_SSH_mapping_NATL60/wiki/AVISO---account-creation) can help you create an AVISO account to access the data. The data are also temporarily available [here](https://ige-meom-opendap.univ-grenoble-alpes.fr/thredds/catalog/meomopendap/extract/MEOM/OCEAN_DATA_CHALLENGES/2021a-SSH-mapping-OSE/catalog.html). They are presented with the following directory structure: 52 | 53 | ``` 54 | . 55 | |-- dc_obs 56 | | |-- dt_global_alg_phy_l3_20161201-20180131_285-315_23-53.nc 57 | | |-- dt_global_c2_phy_l3_20161201-20180131_285-315_23-53.nc 58 | | |-- dt_global_h2g_phy_l3_20161201-20180131_285-315_23-53.nc 59 | | |-- dt_global_j2g_phy_l3_20161201-20180131_285-315_23-53.nc 60 | | |-- dt_global_j2n_phy_l3_20161201-20180131_285-315_23-53.nc 61 | | |-- dt_global_j3_phy_l3_20161201-20180131_285-315_23-53.nc 62 | | |-- dt_global_s3a_phy_l3_20161201-20180131_285-315_23-53.nc 63 | 64 | |-- dc_maps 65 | | |-- OSE_ssh_mapping_BASELINE.nc 66 | | |-- OSE_ssh_mapping_BFN.nc 67 | | |-- OSE_ssh_mapping_DUACS.nc 68 | | |-- OSE_ssh_mapping_DYMOST.nc 69 | | |-- OSE_ssh_mapping_MIOST.nc 70 | | |-- OSE_ssh_mapping_4dvarNet.nc 71 | | |-- OSE_ssh_mapping_4dvarNet_2022.nc (NOT on AVISO+ yet !!!!) 72 | | |-- mdt.nc 73 | 74 | ``` 75 | 76 | ## Baseline and evaluation 77 | 78 | ### Baseline 79 | The baseline mapping method is optimal interpolation (OI), in the spirit of the present-day standard for DUACS products provided by AVISO. OI is implemented in the [`baseline_oi`](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/baseline_oi.ipynb) Jupyter notebook. The SSH reconstructions are saved as a NetCDF file in the `results` directory. The content of this directory is git-ignored. 80 | 81 | ### Evaluation 82 | 83 | The evaluation of the mapping methods is based on the comparison of the SSH reconstructions with the *independent* Cryosat-2 along-track dataset. It includes two scores, one based on the Root-Mean-Square Error (RMSE), the other based on Fourier wavenumber spectra. The evaluation notebook [`example_data_eval`](https://github.com/ocean-data-challenges/2020a_SSH_mapping_NATL60/blob/master/notebooks/example_data_eval.ipynb) implements the computation of these two scores as they could appear in the leaderboard. The notebook also provides additional, graphical diagnostics based on RMSE and spectra. 84 | 85 | ## Data processing 86 | 87 | Cross-functional modules are gathered in the `src` directory. They include tools for regridding, plots, evaluation, writing and reading NetCDF files. The directory also contains a module that implements the baseline method. 88 | 89 | ## Acknowledgement 90 | 91 | The structure of this data challenge was to a large extent inspired by [WeatherBench](https://github.com/pangeo-data/WeatherBench). 92 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 MEOM-IGE 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.5511905.svg)](https://doi.org/10.5281/zenodo.5511905) 2 | 3 | # SSH Mapping Data Challenge 2021a 4 | 5 | This repository contains codes and sample notebooks for downloading and processing the SSH mapping data challenge. 6 | 7 | The quickstart can be run online by clicking here: 8 | [![Binder](https://binder.pangeo.io/badge_logo.svg)](https://binder.pangeo.io/v2/gh/ocean-data-challenges/2020a_SSH_mapping_NATL60/master?filepath=quickstart.ipynb) 9 | 10 | ## Motivation 11 | 12 | The goal is to investigate how to best reconstruct sequences of Sea Surface Height (SSH) maps from partial satellite altimetry observations. This data challenge follows an _Observation System Experiment_ framework: Satellite observations are from real sea surface height data from altimeter. The practical goal of the challenge is to investigate the best mapping method according to scores described below and in Jupyter notebooks. 13 | 14 | ### Observations 15 | The SSH observations include SARAL/Altika, Jason 2, Jason 3, Sentinel 3A, Haiyang-2A and Cryosat-2 altimeter data. This nadir altimeters constellation was operating during the 20170101-20171231 period. Note that for the mapping the Cryosat-2 altimeter data are not taken in the mapping to perfor the independent assessment of the various reconstructions. 16 | 17 | ### Data sequence and use 18 | 19 | The SSH reconstructions are assessed over the period from 2017-01-01 to 2017-12-31. 20 | 21 | For reconstruction methods that need a spin-up, the **observations** can be used from 2016-12-01 until the beginning of the evaluation period (31 days). This spin-up period is not included in the evaluation. For reconstruction methods that need learning from full fields, the **baseline reconstruction** or **duacs reconstrcution** can be used from 2017-01-01 to 2017-12-31. The altimeter data from Cryosat-2 should never be used so that any reconstruction can be considered uncorrelated to the evaluation period. 22 | 23 | ![Data Sequence](figures/DC-data_availability.png) 24 | 25 | ## Leaderboard 26 | 27 | | Method | µ(RMSE) | σ(RMSE) | λx (km) | Notes | Reference | 28 | |:---------|-----------:|----------:|----------:|:------------------|:-------------------------| 29 | | BASELINE | 0.85 | 0.09 | 140 | Covariances BASELINE OI | [example_eval_baseline.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_baseline.ipynb) | 30 | | DUACS | 0.88 | 0.07 | 152 | Covariances DUACS DT2018 | [example_eval_duacs.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_duacs.ipynb) | 31 | | MIOST | 0.89 | 0.08 | 139 | Multiscale mapping | [example_eval_miost.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_miost.ipynb) | 32 | | DYMOST | 0.89 | **0.06** | 129 | Dynamic mapping | [example_eval_dymost.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_dymost.ipynb) | 33 | | BNF | 0.88 | **0.06** | 122 | BFN mapping | [example_eval_bfn.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_bfn.ipynb) | 34 | | 4DVarNet (v2021) | 0.89 | **0.06** | 122 | 4DVarNet mapping | [example_eval_4DVarnet.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_4dvarnet.ipynb) | 35 | | 4DVarNet (v2022) | 0.89 | 0.09 | 109 | 4DVarNet mapping | [example_eval_4DVarnet_v2022.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_4dvarnet_v2022.ipynb) | 36 | | 4DVarQG | **0.90** | **0.06** | **106** | 4DVarQG mapping | [example_eval_4dvarqg.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_4dvarqg.ipynb) | 37 | | ConvLSTM SSH | **0.90** | **0.06** | 113 | ConvLSTM SSH | [example_eval_convlstm_ssh.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_convlstm_ssh.ipynb) | 38 | | ConvLSTM SSH-SST1 | **0.90** | **0.06** | **100** | ConvLSTM SSH-SST | [example_eval_convlstm_ssh-sst.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_convlstm_ssh-sst.ipynb) | 39 | | NeurOST SSH-SST1 | **0.90** | **0.06** | 114 | NeurOST SSH-SST (trained for global mapping) | [example_eval_neurost_ssh-sst.ipynb](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/example_eval_neurost_ssh-sst.ipynb) | 40 | 41 | 1: These methods additionally use L4 SST from the NASA MUR product as input. 42 | 43 | **µ(RMSE)**: average RMSE score. 44 | **σ(RMSE)**: standard deviation of the RMSE score. 45 | **λx**: minimum spatial scale resolved. 46 | 47 | ## Quick start 48 | You can follow the quickstart guide in [this notebook](https://github.com/ocean-data-challenges/2020a_SSH_mapping_NATL60/blob/master/quickstart.ipynb) or launch it directly from binder. 49 | 50 | ## Download the data 51 | The data are hosted on the [AVISO+ website](https://www.aviso.altimetry.fr/en/data/products/ocean-data-challenges/2021a-ssh-mapping-ose.html) and tagged with DOI: 10.24400/527896/a01-2021.005. The website also provides a data handbook. This is the recommended access. This [wiki](https://github.com/ocean-data-challenges/2020a_SSH_mapping_NATL60/wiki/AVISO---account-creation) can help you create an AVISO account to access the data. The data are also temporarily available [here](https://ige-meom-opendap.univ-grenoble-alpes.fr/thredds/catalog/meomopendap/extract/MEOM/OCEAN_DATA_CHALLENGES/2021a-SSH-mapping-OSE/catalog.html). They are presented with the following directory structure: 52 | 53 | ``` 54 | . 55 | |-- dc_obs 56 | | |-- dt_global_alg_phy_l3_20161201-20180131_285-315_23-53.nc 57 | | |-- dt_global_c2_phy_l3_20161201-20180131_285-315_23-53.nc 58 | | |-- dt_global_h2g_phy_l3_20161201-20180131_285-315_23-53.nc 59 | | |-- dt_global_j2g_phy_l3_20161201-20180131_285-315_23-53.nc 60 | | |-- dt_global_j2n_phy_l3_20161201-20180131_285-315_23-53.nc 61 | | |-- dt_global_j3_phy_l3_20161201-20180131_285-315_23-53.nc 62 | | |-- dt_global_s3a_phy_l3_20161201-20180131_285-315_23-53.nc 63 | 64 | |-- dc_maps 65 | | |-- OSE_ssh_mapping_BASELINE.nc 66 | | |-- OSE_ssh_mapping_BFN.nc 67 | | |-- OSE_ssh_mapping_DUACS.nc 68 | | |-- OSE_ssh_mapping_DYMOST.nc 69 | | |-- OSE_ssh_mapping_MIOST.nc 70 | | |-- OSE_ssh_mapping_4dvarNet.nc 71 | | |-- OSE_ssh_mapping_4dvarNet_2022.nc (NOT on AVISO+ yet !!!!) 72 | | |-- OSE_ssh_mapping_neurost_ssh-sst.nc (Also using SST) 73 | | |-- OSE_ssh_mapping_convlstm_ssh-sst.nc (Also using SST) 74 | | |-- OSE_ssh_mapping_convlstm_ssh.nc 75 | | |-- mdt.nc 76 | 77 | ``` 78 | 79 | ## Baseline and evaluation 80 | 81 | ### Baseline 82 | The baseline mapping method is optimal interpolation (OI), in the spirit of the present-day standard for DUACS products provided by AVISO. OI is implemented in the [`baseline_oi`](https://github.com/ocean-data-challenges/2021a_SSH_mapping_OSE/blob/master/notebooks/baseline_oi.ipynb) Jupyter notebook. The SSH reconstructions are saved as a NetCDF file in the `results` directory. The content of this directory is git-ignored. 83 | 84 | ### Evaluation 85 | 86 | The evaluation of the mapping methods is based on the comparison of the SSH reconstructions with the *independent* Cryosat-2 along-track dataset. It includes two scores, one based on the Root-Mean-Square Error (RMSE), the other based on Fourier wavenumber spectra. The evaluation notebook [`example_data_eval`](https://github.com/ocean-data-challenges/2020a_SSH_mapping_NATL60/blob/master/notebooks/example_data_eval.ipynb) implements the computation of these two scores as they could appear in the leaderboard. The notebook also provides additional, graphical diagnostics based on RMSE and spectra. 87 | 88 | ## Data processing 89 | 90 | Cross-functional modules are gathered in the `src` directory. They include tools for regridding, plots, evaluation, writing and reading NetCDF files. The directory also contains a module that implements the baseline method. 91 | 92 | ## Acknowledgement 93 | 94 | The structure of this data challenge was to a large extent inspired by [WeatherBench](https://github.com/pangeo-data/WeatherBench). 95 | -------------------------------------------------------------------------------- /environment.yml: -------------------------------------------------------------------------------- 1 | name: odc-ssh-mapping 2 | channels: 3 | - menpo 4 | - conda-forge 5 | - pyviz 6 | - anaconda 7 | - defaults 8 | dependencies: 9 | - basemap 10 | - xarray 11 | - zarr 12 | - cython 13 | - numpy 14 | - matplotlib 15 | - gcsfs 16 | - geoviews==1.6.6 17 | - cftime==1.1.2 18 | - holoviews 19 | - bokeh 20 | - dask 21 | - numpy 22 | - netCDF4 23 | - scipy 24 | - pandas 25 | - cartopy 26 | - gcsfs 27 | - hvplot 28 | - pyinterp 29 | - xrft 30 | - pydap 31 | - tabulate 32 | - pip 33 | - pip: 34 | - fasteners==0.15 35 | - monotonic==1.5 36 | - numcodecs==0.6.4 37 | - opencv-python==4.2.0.34 38 | - polygon3==3.0.8 39 | - pyeddytracker==3.0.0 40 | - pyqt5-sip==4.19.18 41 | - pyqtwebengine==5.12.1 42 | -------------------------------------------------------------------------------- /figures/DC-data_availability.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/figures/DC-data_availability.png -------------------------------------------------------------------------------- /notebooks/.ipynb_checkpoints/baseline_oi-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# II- Demo. Optimal Interpolation" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "An example of simulated SSH data access is provided in the \"example_data_access_meom.ipynb\" notebook. Here, an example of a mapping technique based on a simple optimal interpolation is proposed. The notebook is structured as follow: \n", 15 | "\n", 16 | " 1) set optimal interpolation parameters,\n", 17 | " 2) reading of pseudo-observations,\n", 18 | " 3) perform optimal interpolation and,\n", 19 | " 4) save the results (reconstructed SSH field)\n", 20 | "\n", 21 | "\n", 22 | "Here, we assume a vector of observations, noted $y$ defined as:\n", 23 | "\n", 24 | "$$y = H x + \\epsilon $$\n", 25 | "\n", 26 | "where $H$ is a linear observation operator between the reconstruction grid space and the observation space\n", 27 | ", $x$ is the state to estimate and $\\epsilon$ is an independent observation error.\n", 28 | "\n", 29 | "The optimal interpolation consists in estimating an analysed state $x_{a}$ in combining the available observations to approximate the real state $x$:\n", 30 | "\n", 31 | "$$x_{a} = K y $$\n", 32 | "where $K$ is the weigth matrix defined as:\n", 33 | "\n", 34 | "$$ K = BH^T(HBH^T + R)^{-1} $$\n", 35 | "\n", 36 | "$B$ is the covariance matrix of $x$, and $R$ the covariance matrix of the error vector $\\epsilon$ ($^T$ is the transpose operator)" 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": 1, 42 | "metadata": {}, 43 | "outputs": [], 44 | "source": [ 45 | "import xarray as xr\n", 46 | "import numpy\n", 47 | "import warnings\n", 48 | "import logging\n", 49 | "import sys\n", 50 | "import os\n", 51 | "warnings.filterwarnings('ignore')" 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "execution_count": 2, 57 | "metadata": {}, 58 | "outputs": [], 59 | "source": [ 60 | "logger = logging.getLogger()\n", 61 | "logger.setLevel(logging.INFO)" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": 3, 67 | "metadata": {}, 68 | "outputs": [], 69 | "source": [ 70 | "sys.path.append('..')" 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": 4, 76 | "metadata": {}, 77 | "outputs": [], 78 | "source": [ 79 | "from src.mod_oi import *\n", 80 | "from src.mod_inout import *" 81 | ] 82 | }, 83 | { 84 | "cell_type": "markdown", 85 | "metadata": {}, 86 | "source": [ 87 | "### 1) set optimal interpolation parameters" 88 | ] 89 | }, 90 | { 91 | "cell_type": "code", 92 | "execution_count": 5, 93 | "metadata": {}, 94 | "outputs": [], 95 | "source": [ 96 | "# OI Grid\n", 97 | "lon_min = 295. # domain min longitude\n", 98 | "lon_max = 305. # domain max longitude\n", 99 | "lat_min = 33. # domain min latitude\n", 100 | "lat_max = 43. # domain max latitude\n", 101 | "time_min = numpy.datetime64('2017-01-01') # domain min time\n", 102 | "time_max = numpy.datetime64('2017-12-31') # domain max time\n", 103 | "dx = 0.2 # zonal grid spatial step (in degree)\n", 104 | "dy = 0.2 # meridional grid spatial step (in degree)\n", 105 | "dt = numpy.timedelta64(1, 'D') # temporal grid step\n", 106 | "\n", 107 | "glon = numpy.arange(lon_min, lon_max + dx, dx) # output OI longitude grid\n", 108 | "glat = numpy.arange(lat_min, lat_max + dy, dy) # output OI latitude grid\n", 109 | "gtime = numpy.arange(time_min, time_max + dt, dt) # output OI time grid\n", 110 | "\n", 111 | "# OI parameters\n", 112 | "Lx = 1. # Zonal decorrelation scale (in degree)\n", 113 | "Ly = 1. # Meridional decorrelation scale (in degree)\n", 114 | "Lt = 7. # Temporal decorrelation scale (in days)\n", 115 | "noise = 0.05 # Noise level (5%)" 116 | ] 117 | }, 118 | { 119 | "cell_type": "markdown", 120 | "metadata": {}, 121 | "source": [ 122 | "### 2) reading of pseudo-observations + define output folder" 123 | ] 124 | }, 125 | { 126 | "cell_type": "code", 127 | "execution_count": 6, 128 | "metadata": {}, 129 | "outputs": [], 130 | "source": [ 131 | "inputs = ['../inputs/dc_obs/dt_global_alg_phy_l3_20161201-20180131_285-315_23-53.nc', \n", 132 | " '../inputs/dc_obs/dt_global_j3_phy_l3_20161201-20180131_285-315_23-53.nc', \n", 133 | " '../inputs/dc_obs/dt_global_s3a_phy_l3_20161201-20180131_285-315_23-53.nc',\n", 134 | " '../inputs/dc_obs/dt_global_h2g_phy_l3_20161201-20180131_285-315_23-53.nc',\n", 135 | " '../inputs/dc_obs/dt_global_j2g_phy_l3_20161201-20180131_285-315_23-53.nc',\n", 136 | " '../inputs/dc_obs/dt_global_j2n_phy_l3_20161201-20180131_285-315_23-53.nc'] " 137 | ] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "execution_count": 7, 142 | "metadata": {}, 143 | "outputs": [], 144 | "source": [ 145 | "# Define outputs\n", 146 | "output_directory = '../results/'\n", 147 | "if not os.path.exists(output_directory):\n", 148 | " os.mkdir(output_directory) \n", 149 | "output_oi = f'../inputs/dc_maps/OSE_ssh_mapping_BASELINE.nc'" 150 | ] 151 | }, 152 | { 153 | "cell_type": "markdown", 154 | "metadata": {}, 155 | "source": [ 156 | "### 3) perform optimal interpolation" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": 8, 162 | "metadata": {}, 163 | "outputs": [ 164 | { 165 | "name": "stderr", 166 | "output_type": "stream", 167 | "text": [ 168 | "INFO:root: Set OI params...\n", 169 | "INFO:root: Set OI grid...\n", 170 | "INFO:root: Reading observations...\n" 171 | ] 172 | }, 173 | { 174 | "name": "stdout", 175 | "output_type": "stream", 176 | "text": [ 177 | "CPU times: user 2h 6min 18s, sys: 6min 37s, total: 2h 12min 56s\n", 178 | "Wall time: 1h 15min 37s\n" 179 | ] 180 | } 181 | ], 182 | "source": [ 183 | "%%time\n", 184 | "# set OI param & grid\n", 185 | "ds_oi1_param = oi_param(Lx, Ly, Lt, noise)\n", 186 | "ds_oi1_grid = oi_grid(glon, glat, gtime)\n", 187 | "# Read input obs + discard a bit...\n", 188 | "coarsening = {'time': 5}\n", 189 | "ds_oi1_obs = read_obs(inputs, ds_oi1_grid, ds_oi1_param, coarsening)\n", 190 | "# Run OI (take 1h on my laptop)\n", 191 | "for it in range(len(gtime)):\n", 192 | " oi_core(it, ds_oi1_grid, ds_oi1_param, ds_oi1_obs)" 193 | ] 194 | }, 195 | { 196 | "cell_type": "markdown", 197 | "metadata": {}, 198 | "source": [ 199 | "### 4) save the results (reconstructed SSH field)" 200 | ] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": 9, 205 | "metadata": {}, 206 | "outputs": [], 207 | "source": [ 208 | "ds_oi1_grid = reformate_oi_output(ds_oi1_grid, '../inputs/dc_maps/mdt.nc')\n", 209 | "ds_oi1_grid.to_netcdf(output_oi)" 210 | ] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": null, 215 | "metadata": {}, 216 | "outputs": [], 217 | "source": [] 218 | }, 219 | { 220 | "cell_type": "code", 221 | "execution_count": null, 222 | "metadata": {}, 223 | "outputs": [], 224 | "source": [] 225 | } 226 | ], 227 | "metadata": { 228 | "kernelspec": { 229 | "display_name": "Python 3", 230 | "language": "python", 231 | "name": "python3" 232 | }, 233 | "language_info": { 234 | "codemirror_mode": { 235 | "name": "ipython", 236 | "version": 3 237 | }, 238 | "file_extension": ".py", 239 | "mimetype": "text/x-python", 240 | "name": "python", 241 | "nbconvert_exporter": "python", 242 | "pygments_lexer": "ipython3", 243 | "version": "3.7.8" 244 | } 245 | }, 246 | "nbformat": 4, 247 | "nbformat_minor": 4 248 | } 249 | -------------------------------------------------------------------------------- /notebooks/.ipynb_checkpoints/example_eval_convlstm_ssh-checkpoint.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Evaluation DUACS OI: \n", 8 | "\n", 9 | "This notebook presents the evaluation of the SSH reconstructions based on the DUACS OI ([Taburet et al., 2018](https://os.copernicus.org/articles/15/1207/2019/)) and performed for the **\"2021a_SSH_mapping_OSE\" ocean data challenge**. " 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": null, 15 | "metadata": {}, 16 | "outputs": [], 17 | "source": [ 18 | "import os\r\n", 19 | "import sys\r\n", 20 | "sys.path.append('..')\r\n", 21 | "import logging\r\n", 22 | "import pandas as pd" 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": null, 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "from src.mod_inout import *\r\n", 32 | "from src.mod_interp import *\r\n", 33 | "from src.mod_stats import *\r\n", 34 | "from src.mod_spectral import *\r\n", 35 | "from src.mod_plot import *" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "metadata": {}, 42 | "outputs": [], 43 | "source": [ 44 | "logger = logging.getLogger()\r\n", 45 | "logger.setLevel(logging.INFO)" 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": {}, 51 | "source": [ 52 | "### Study Area & Ouput Parameters" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "metadata": {}, 59 | "outputs": [], 60 | "source": [ 61 | "# study area\r\n", 62 | "lon_min = 295.\r\n", 63 | "lon_max = 305.\r\n", 64 | "lat_min = 33.\r\n", 65 | "lat_max = 43.\r\n", 66 | "is_circle = False\r\n", 67 | "time_min = '2017-01-01'\r\n", 68 | "time_max = '2017-12-31'\r\n", 69 | "\r\n", 70 | "# Outputs\r\n", 71 | "bin_lat_step = 1.\r\n", 72 | "bin_lon_step = 1.\r\n", 73 | "bin_time_step = '1D'\r\n", 74 | "output_directory = '../results'\r\n", 75 | "if not os.path.exists(output_directory):\r\n", 76 | " os.mkdir(output_directory)\r\n", 77 | "output_filename = f'{output_directory}/stat_OSE_DUACS_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\r\n", 78 | "output_filename_timeseries = f'{output_directory}/stat_timeseries_OSE_DUACS_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\r\n", 79 | "\r\n", 80 | "# Spectral parameter\r\n", 81 | "# C2 parameter\r\n", 82 | "delta_t = 0.9434 # s\r\n", 83 | "velocity = 6.77 # km/s\r\n", 84 | "delta_x = velocity * delta_t\r\n", 85 | "lenght_scale = 1000 # km\r\n", 86 | "output_filename_spectrum = f'{output_directory}/psd_OSE_DUACS_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'" 87 | ] 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "metadata": {}, 92 | "source": [ 93 | "### Open your AVISO+ session: fill the `````` and `````` items below" 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": null, 99 | "metadata": {}, 100 | "outputs": [], 101 | "source": [ 102 | "my_aviso_session = rq.Session()\r\n", 103 | "my_aviso_session.auth = (\"\", \"\")\r\n", 104 | "url_alongtrack = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-along-track-data'\r\n", 105 | "url_map = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-grid-data'" 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "metadata": {}, 111 | "source": [ 112 | "### Read L3 datasets" 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": null, 118 | "metadata": {}, 119 | "outputs": [], 120 | "source": [ 121 | "# independent along-track\r\n", 122 | "alontrack_independent_dataset = f'{url_alongtrack}/dt_gulfstream_c2_phy_l3_20161201-20180131_285-315_23-53.nc'\r\n", 123 | "# Read along-track\r\n", 124 | "ds_alongtrack = read_l3_dataset_from_aviso(alontrack_independent_dataset, \r\n", 125 | " my_aviso_session, \r\n", 126 | " lon_min=lon_min, \r\n", 127 | " lon_max=lon_max, \r\n", 128 | " lat_min=lat_min, \r\n", 129 | " lat_max=lat_max, \r\n", 130 | " time_min=time_min, \r\n", 131 | " time_max=time_max)\r\n", 132 | "ds_alongtrack" 133 | ] 134 | }, 135 | { 136 | "cell_type": "markdown", 137 | "metadata": {}, 138 | "source": [ 139 | "### Read L4 dataset and interpolate onto along-track positions" 140 | ] 141 | }, 142 | { 143 | "cell_type": "code", 144 | "execution_count": null, 145 | "metadata": {}, 146 | "outputs": [], 147 | "source": [ 148 | "# series of maps to evaluate\r\n", 149 | "gridded_dataset = [f'{url_map}/OSE_ssh_mapping_DUACS.nc', my_aviso_session]\r\n", 150 | "# Interpolate maps onto alongtrack dataset\r\n", 151 | "time_alongtrack, lat_alongtrack, lon_alongtrack, ssh_alongtrack, ssh_map_interp = interp_on_alongtrack(gridded_dataset, \r\n", 152 | " ds_alongtrack,\r\n", 153 | " lon_min=lon_min, \r\n", 154 | " lon_max=lon_max, \r\n", 155 | " lat_min=lat_min, \r\n", 156 | " lat_max=lat_max, \r\n", 157 | " time_min=time_min, \r\n", 158 | " time_max=time_max,\r\n", 159 | " is_circle=is_circle)" 160 | ] 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "metadata": {}, 165 | "source": [ 166 | "### Compute statistical score" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": null, 172 | "metadata": {}, 173 | "outputs": [], 174 | "source": [ 175 | "leaderboard_nrmse, leaderboard_nrmse_std = compute_stats(time_alongtrack, \r\n", 176 | " lat_alongtrack, \r\n", 177 | " lon_alongtrack, \r\n", 178 | " ssh_alongtrack, \r\n", 179 | " ssh_map_interp, \r\n", 180 | " bin_lon_step,\r\n", 181 | " bin_lat_step, \r\n", 182 | " bin_time_step,\r\n", 183 | " output_filename,\r\n", 184 | " output_filename_timeseries)" 185 | ] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": null, 190 | "metadata": {}, 191 | "outputs": [], 192 | "source": [ 193 | "plot_spatial_statistics(output_filename)" 194 | ] 195 | }, 196 | { 197 | "cell_type": "code", 198 | "execution_count": null, 199 | "metadata": {}, 200 | "outputs": [], 201 | "source": [ 202 | "plot_temporal_statistics(output_filename_timeseries)" 203 | ] 204 | }, 205 | { 206 | "cell_type": "markdown", 207 | "metadata": {}, 208 | "source": [ 209 | "### Compute spectral scores" 210 | ] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": null, 215 | "metadata": {}, 216 | "outputs": [], 217 | "source": [ 218 | "compute_spectral_scores(time_alongtrack, \n", 219 | " lat_alongtrack, \n", 220 | " lon_alongtrack, \n", 221 | " ssh_alongtrack, \n", 222 | " ssh_map_interp, \n", 223 | " lenght_scale,\n", 224 | " delta_x,\n", 225 | " delta_t,\n", 226 | " output_filename_spectrum)" 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": null, 232 | "metadata": {}, 233 | "outputs": [], 234 | "source": [ 235 | "leaderboard_psds_score = plot_psd_score(output_filename_spectrum)" 236 | ] 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "metadata": {}, 241 | "source": [ 242 | "### Show leaderboard metrics" 243 | ] 244 | }, 245 | { 246 | "cell_type": "code", 247 | "execution_count": null, 248 | "metadata": {}, 249 | "outputs": [], 250 | "source": [ 251 | "# Print leaderboard\n", 252 | "data = [['DUACS', \n", 253 | " leaderboard_nrmse, \n", 254 | " leaderboard_nrmse_std, \n", 255 | " int(leaderboard_psds_score),\n", 256 | " 'Covariances DUACS',\n", 257 | " 'example_eval_duacs.ipynb']]\n", 258 | "Leaderboard = pd.DataFrame(data, \n", 259 | " columns=['Method', \n", 260 | " \"µ(RMSE) \", \n", 261 | " \"σ(RMSE)\", \n", 262 | " 'λx (km)', \n", 263 | " 'Notes',\n", 264 | " 'Reference'])\n", 265 | "print(\"Summary of the leaderboard metrics:\")\n", 266 | "Leaderboard\n", 267 | "print(Leaderboard.to_markdown())" 268 | ] 269 | }, 270 | { 271 | "cell_type": "code", 272 | "execution_count": null, 273 | "metadata": {}, 274 | "outputs": [], 275 | "source": [] 276 | }, 277 | { 278 | "cell_type": "code", 279 | "execution_count": null, 280 | "metadata": {}, 281 | "outputs": [], 282 | "source": [] 283 | }, 284 | { 285 | "cell_type": "code", 286 | "execution_count": null, 287 | "metadata": {}, 288 | "outputs": [], 289 | "source": [] 290 | } 291 | ], 292 | "metadata": { 293 | "kernelspec": { 294 | "display_name": "conda_env", 295 | "language": "python", 296 | "name": "conda_env" 297 | }, 298 | "language_info": { 299 | "codemirror_mode": { 300 | "name": "ipython", 301 | "version": 3 302 | }, 303 | "file_extension": ".py", 304 | "mimetype": "text/x-python", 305 | "name": "python", 306 | "nbconvert_exporter": "python", 307 | "pygments_lexer": "ipython3", 308 | "version": "3.7.8" 309 | } 310 | }, 311 | "nbformat": 4, 312 | "nbformat_minor": 4 313 | } 314 | -------------------------------------------------------------------------------- /notebooks/baseline_oi.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# II- Demo. Optimal Interpolation" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "An example of simulated SSH data access is provided in the \"example_data_access_meom.ipynb\" notebook. Here, an example of a mapping technique based on a simple optimal interpolation is proposed. The notebook is structured as follow: \n", 15 | "\n", 16 | " 1) set optimal interpolation parameters,\n", 17 | " 2) reading of pseudo-observations,\n", 18 | " 3) perform optimal interpolation and,\n", 19 | " 4) save the results (reconstructed SSH field)\n", 20 | "\n", 21 | "\n", 22 | "Here, we assume a vector of observations, noted $y$ defined as:\n", 23 | "\n", 24 | "$$y = H x + \\epsilon $$\n", 25 | "\n", 26 | "where $H$ is a linear observation operator between the reconstruction grid space and the observation space\n", 27 | ", $x$ is the state to estimate and $\\epsilon$ is an independent observation error.\n", 28 | "\n", 29 | "The optimal interpolation consists in estimating an analysed state $x_{a}$ in combining the available observations to approximate the real state $x$:\n", 30 | "\n", 31 | "$$x_{a} = K y $$\n", 32 | "where $K$ is the weigth matrix defined as:\n", 33 | "\n", 34 | "$$ K = BH^T(HBH^T + R)^{-1} $$\n", 35 | "\n", 36 | "$B$ is the covariance matrix of $x$, and $R$ the covariance matrix of the error vector $\\epsilon$ ($^T$ is the transpose operator)" 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": null, 42 | "metadata": {}, 43 | "outputs": [], 44 | "source": [ 45 | "import xarray as xr\n", 46 | "import numpy\n", 47 | "import warnings\n", 48 | "import logging\n", 49 | "import sys\n", 50 | "import os\n", 51 | "warnings.filterwarnings('ignore')" 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "execution_count": null, 57 | "metadata": {}, 58 | "outputs": [], 59 | "source": [ 60 | "logger = logging.getLogger()\n", 61 | "logger.setLevel(logging.INFO)" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": null, 67 | "metadata": {}, 68 | "outputs": [], 69 | "source": [ 70 | "sys.path.append('..')" 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": null, 76 | "metadata": {}, 77 | "outputs": [], 78 | "source": [ 79 | "from src.mod_oi import *\n", 80 | "from src.mod_inout import *" 81 | ] 82 | }, 83 | { 84 | "cell_type": "markdown", 85 | "metadata": {}, 86 | "source": [ 87 | "### 1) set optimal interpolation parameters" 88 | ] 89 | }, 90 | { 91 | "cell_type": "code", 92 | "execution_count": null, 93 | "metadata": {}, 94 | "outputs": [], 95 | "source": [ 96 | "# OI Grid\n", 97 | "lon_min = 295. # domain min longitude\n", 98 | "lon_max = 305. # domain max longitude\n", 99 | "lat_min = 33. # domain min latitude\n", 100 | "lat_max = 43. # domain max latitude\n", 101 | "time_min = numpy.datetime64('2017-01-01') # domain min time\n", 102 | "time_max = numpy.datetime64('2017-12-31') # domain max time\n", 103 | "dx = 0.2 # zonal grid spatial step (in degree)\n", 104 | "dy = 0.2 # meridional grid spatial step (in degree)\n", 105 | "dt = numpy.timedelta64(1, 'D') # temporal grid step\n", 106 | "\n", 107 | "glon = numpy.arange(lon_min, lon_max + dx, dx) # output OI longitude grid\n", 108 | "glat = numpy.arange(lat_min, lat_max + dy, dy) # output OI latitude grid\n", 109 | "gtime = numpy.arange(time_min, time_max + dt, dt) # output OI time grid\n", 110 | "\n", 111 | "# OI parameters\n", 112 | "Lx = 1. # Zonal decorrelation scale (in degree)\n", 113 | "Ly = 1. # Meridional decorrelation scale (in degree)\n", 114 | "Lt = 7. # Temporal decorrelation scale (in days)\n", 115 | "noise = 0.05 # Noise level (5%)" 116 | ] 117 | }, 118 | { 119 | "cell_type": "markdown", 120 | "metadata": {}, 121 | "source": [ 122 | "### Open your AVISO+ session: fill the `````` and `````` items below" 123 | ] 124 | }, 125 | { 126 | "cell_type": "code", 127 | "execution_count": null, 128 | "metadata": {}, 129 | "outputs": [], 130 | "source": [ 131 | "my_aviso_session = rq.Session()\n", 132 | "my_aviso_session.auth = (\"\", \"\")\n", 133 | "url_alongtrack = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-along-track-data'\n", 134 | "url_map = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-grid-data'" 135 | ] 136 | }, 137 | { 138 | "cell_type": "markdown", 139 | "metadata": {}, 140 | "source": [ 141 | "### 2) reading of pseudo-observations + define output folder" 142 | ] 143 | }, 144 | { 145 | "cell_type": "code", 146 | "execution_count": null, 147 | "metadata": {}, 148 | "outputs": [], 149 | "source": [ 150 | "inputs = [f'{url_alongtrack}/dt_gulfstream_alg_phy_l3_20161201-20180131_285-315_23-53.nc', \n", 151 | " f'{url_alongtrack}/dt_gulfstream_j3_phy_l3_20161201-20180131_285-315_23-53.nc', \n", 152 | " f'{url_alongtrack}/dt_gulfstream_s3a_phy_l3_20161201-20180131_285-315_23-53.nc',\n", 153 | " f'{url_alongtrack}/dt_gulfstream_h2g_phy_l3_20161201-20180131_285-315_23-53.nc',\n", 154 | " f'{url_alongtrack}/dt_gulfstream_j2g_phy_l3_20161201-20180131_285-315_23-53.nc',\n", 155 | " f'{url_alongtrack}/dt_gulfstream_j2n_phy_l3_20161201-20180131_285-315_23-53.nc'] " 156 | ] 157 | }, 158 | { 159 | "cell_type": "code", 160 | "execution_count": null, 161 | "metadata": {}, 162 | "outputs": [], 163 | "source": [ 164 | "# Define outputs\n", 165 | "output_directory = '../results/'\n", 166 | "if not os.path.exists(output_directory):\n", 167 | " os.mkdir(output_directory) \n", 168 | "output_oi = f'{output_directory}/OSE_ssh_mapping_BASELINE.nc'" 169 | ] 170 | }, 171 | { 172 | "cell_type": "markdown", 173 | "metadata": {}, 174 | "source": [ 175 | "### 3) perform optimal interpolation" 176 | ] 177 | }, 178 | { 179 | "cell_type": "code", 180 | "execution_count": null, 181 | "metadata": {}, 182 | "outputs": [], 183 | "source": [ 184 | "%%time\n", 185 | "# set OI param & grid\n", 186 | "ds_oi1_param = oi_param(Lx, Ly, Lt, noise)\n", 187 | "ds_oi1_grid = oi_grid(glon, glat, gtime)\n", 188 | "# Read input obs + discard a bit...\n", 189 | "coarsening = {'time': 5}\n", 190 | "#ds_oi1_obs = read_obs(inputs, ds_oi1_grid, ds_oi1_param, coarsening)\n", 191 | "ds_oi1_obs = read_obs_from_aviso(inputs, my_aviso_session, ds_oi1_grid, ds_oi1_param, coarsening)\n", 192 | "# Run OI (take 1h on my laptop)\n", 193 | "for it in range(len(gtime)):\n", 194 | " oi_core(it, ds_oi1_grid, ds_oi1_param, ds_oi1_obs)" 195 | ] 196 | }, 197 | { 198 | "cell_type": "markdown", 199 | "metadata": {}, 200 | "source": [ 201 | "### 4) save the results (reconstructed SSH field)" 202 | ] 203 | }, 204 | { 205 | "cell_type": "code", 206 | "execution_count": null, 207 | "metadata": {}, 208 | "outputs": [], 209 | "source": [ 210 | "url_ds_mdt = f'{url_map}/mdt.nc'\n", 211 | "ds_oi1_grid = reformate_oi_output(ds_oi1_grid, url_ds_mdt, my_aviso_session)\n", 212 | "ds_oi1_grid.to_netcdf(output_oi)" 213 | ] 214 | }, 215 | { 216 | "cell_type": "code", 217 | "execution_count": null, 218 | "metadata": {}, 219 | "outputs": [], 220 | "source": [] 221 | } 222 | ], 223 | "metadata": { 224 | "kernelspec": { 225 | "display_name": "my_conda_env", 226 | "language": "python", 227 | "name": "python3" 228 | }, 229 | "language_info": { 230 | "codemirror_mode": { 231 | "name": "ipython", 232 | "version": 3 233 | }, 234 | "file_extension": ".py", 235 | "mimetype": "text/x-python", 236 | "name": "python", 237 | "nbconvert_exporter": "python", 238 | "pygments_lexer": "ipython3", 239 | "version": "3.8.10" 240 | } 241 | }, 242 | "nbformat": 4, 243 | "nbformat_minor": 4 244 | } 245 | -------------------------------------------------------------------------------- /notebooks/example_data_access_aviso.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# I- Demo. Data Access from AVISO+ repository" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "This notebook aims at documenting how to access & manipulate the input datasets for one \"ocean data challenge\".\n", 15 | "Real Sea Surface Height (SSH) datasets are available on the AVISO+ opendap server.\n", 16 | "The **2021a-SSH-mapping-OSE-along-track** corresponds to the observations datasets based on the year 2017 nadir altimeter constellation (SARAL/Altika, Cryosat-2, Jason-2, Jason-3, Sentinel-3A, Haiyang-2). These data are distributed by the Copernicus Marine Service ([CMEMS](https://resources.marine.copernicus.eu/?option=com_csw&view=details&product_id=SEALEVEL_GLO_PHY_L3_REP_OBSERVATIONS_008_062)), here only an extraction of the data over the Gulfstream region is provided.\n", 17 | "The example below read the dataset using an xarray interface for accessing OpenDAP datasets with pydap. Alternatively, you may use the ```wget``` command to download the files. Note that users must first create an AVISO+ account to access the data. You can follow [this guide](https://github.com/ocean-data-challenges/2020a_SSH_mapping_NATL60/wiki/AVISO---account-creation) for creating your account... " 18 | ] 19 | }, 20 | { 21 | "cell_type": "code", 22 | "execution_count": null, 23 | "metadata": {}, 24 | "outputs": [], 25 | "source": [ 26 | "import xarray as xr\n", 27 | "import requests as rq\n", 28 | "import sys\n", 29 | "import numpy\n", 30 | "import hvplot.xarray" 31 | ] 32 | }, 33 | { 34 | "cell_type": "markdown", 35 | "metadata": {}, 36 | "source": [ 37 | "### Open your AVISO+ session: fill the `````` and `````` items below" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": null, 43 | "metadata": {}, 44 | "outputs": [], 45 | "source": [ 46 | "my_aviso_session = rq.Session()\n", 47 | "my_aviso_session.auth = (\"\", \"\")" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "execution_count": null, 53 | "metadata": {}, 54 | "outputs": [], 55 | "source": [ 56 | "url_alongtrack = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-along-track-data'\n", 57 | "url_map = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-grid-data'" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": null, 63 | "metadata": {}, 64 | "outputs": [], 65 | "source": [ 66 | "sys.path.append('..')" 67 | ] 68 | }, 69 | { 70 | "cell_type": "code", 71 | "execution_count": null, 72 | "metadata": {}, 73 | "outputs": [], 74 | "source": [ 75 | "from src.mod_plot import *" 76 | ] 77 | }, 78 | { 79 | "cell_type": "markdown", 80 | "metadata": {}, 81 | "source": [ 82 | "### Load SARAL/Altika obs." 83 | ] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": null, 88 | "metadata": {}, 89 | "outputs": [], 90 | "source": [ 91 | "url_ds_alg = f'{url_alongtrack}/dt_gulfstream_alg_phy_l3_20161201-20180131_285-315_23-53.nc'\n", 92 | "store_ds_alg = xr.backends.PydapDataStore.open(url_ds_alg, session=my_aviso_session)" 93 | ] 94 | }, 95 | { 96 | "cell_type": "code", 97 | "execution_count": null, 98 | "metadata": {}, 99 | "outputs": [], 100 | "source": [ 101 | "ds_alg = xr.open_dataset(store_ds_alg)\n", 102 | "ds_alg" 103 | ] 104 | }, 105 | { 106 | "cell_type": "code", 107 | "execution_count": null, 108 | "metadata": {}, 109 | "outputs": [], 110 | "source": [ 111 | "# Alternatiavely, you may use wget:\n", 112 | "#!wget --user '' --password '' 'https://tds.aviso.altimetry.fr/thredds/fileServer/2021a-SSH-mapping-OSE-along-track-data/dt_gulfstream_alg_phy_l3_20161201-20180131_285-315_23-53.nc'" 113 | ] 114 | }, 115 | { 116 | "cell_type": "markdown", 117 | "metadata": {}, 118 | "source": [ 119 | "### Load Jason-3 obs." 120 | ] 121 | }, 122 | { 123 | "cell_type": "code", 124 | "execution_count": null, 125 | "metadata": {}, 126 | "outputs": [], 127 | "source": [ 128 | "url_ds_j3 = f'{url_alongtrack}/dt_gulfstream_j3_phy_l3_20161201-20180131_285-315_23-53.nc'\n", 129 | "store_ds_j3 = xr.backends.PydapDataStore.open(url_ds_j3, session=my_aviso_session)" 130 | ] 131 | }, 132 | { 133 | "cell_type": "code", 134 | "execution_count": null, 135 | "metadata": {}, 136 | "outputs": [], 137 | "source": [ 138 | "ds_j3 = xr.open_dataset(store_ds_j3)\n", 139 | "ds_j3" 140 | ] 141 | }, 142 | { 143 | "cell_type": "markdown", 144 | "metadata": {}, 145 | "source": [ 146 | "### Load Sentinel-3A obs." 147 | ] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "execution_count": null, 152 | "metadata": {}, 153 | "outputs": [], 154 | "source": [ 155 | "url_ds_s3a = f'{url_alongtrack}/dt_gulfstream_s3a_phy_l3_20161201-20180131_285-315_23-53.nc'\n", 156 | "store_ds_s3a = xr.backends.PydapDataStore.open(url_ds_s3a, session=my_aviso_session)" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "metadata": {}, 163 | "outputs": [], 164 | "source": [ 165 | "ds_s3a = xr.open_dataset(store_ds_s3a)\n", 166 | "ds_s3a" 167 | ] 168 | }, 169 | { 170 | "cell_type": "markdown", 171 | "metadata": {}, 172 | "source": [ 173 | "### Load Jason-2 obs." 174 | ] 175 | }, 176 | { 177 | "cell_type": "code", 178 | "execution_count": null, 179 | "metadata": {}, 180 | "outputs": [], 181 | "source": [ 182 | "url_ds_j2g = f'{url_alongtrack}/dt_gulfstream_j2g_phy_l3_20161201-20180131_285-315_23-53.nc'\n", 183 | "store_ds_j2g = xr.backends.PydapDataStore.open(url_ds_j2g, session=my_aviso_session)" 184 | ] 185 | }, 186 | { 187 | "cell_type": "code", 188 | "execution_count": null, 189 | "metadata": {}, 190 | "outputs": [], 191 | "source": [ 192 | "ds_j2g = xr.open_dataset(store_ds_j2g)\n", 193 | "ds_j2g" 194 | ] 195 | }, 196 | { 197 | "cell_type": "code", 198 | "execution_count": null, 199 | "metadata": {}, 200 | "outputs": [], 201 | "source": [ 202 | "### Load Jason-2 obs." 203 | ] 204 | }, 205 | { 206 | "cell_type": "code", 207 | "execution_count": null, 208 | "metadata": {}, 209 | "outputs": [], 210 | "source": [ 211 | "url_ds_j2n = f'{url_alongtrack}/dt_gulfstream_j2n_phy_l3_20161201-20180131_285-315_23-53.nc'\n", 212 | "store_ds_j2n = xr.backends.PydapDataStore.open(url_ds_j2n, session=my_aviso_session)" 213 | ] 214 | }, 215 | { 216 | "cell_type": "code", 217 | "execution_count": null, 218 | "metadata": {}, 219 | "outputs": [], 220 | "source": [ 221 | "ds_j2n = xr.open_dataset(store_ds_j2n)\n", 222 | "ds_j2n" 223 | ] 224 | }, 225 | { 226 | "cell_type": "markdown", 227 | "metadata": {}, 228 | "source": [ 229 | "### Load Cryosat-2 obs." 230 | ] 231 | }, 232 | { 233 | "cell_type": "code", 234 | "execution_count": null, 235 | "metadata": {}, 236 | "outputs": [], 237 | "source": [ 238 | "url_ds_c2 = f'{url_alongtrack}/dt_gulfstream_c2_phy_l3_20161201-20180131_285-315_23-53.nc'\n", 239 | "store_ds_c2 = xr.backends.PydapDataStore.open(url_ds_c2, session=my_aviso_session)" 240 | ] 241 | }, 242 | { 243 | "cell_type": "code", 244 | "execution_count": null, 245 | "metadata": {}, 246 | "outputs": [], 247 | "source": [ 248 | "ds_c2 = xr.open_dataset(store_ds_c2)\n", 249 | "ds_c2" 250 | ] 251 | }, 252 | { 253 | "cell_type": "markdown", 254 | "metadata": {}, 255 | "source": [ 256 | "### Load Haiyang-2 obs." 257 | ] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "execution_count": null, 262 | "metadata": {}, 263 | "outputs": [], 264 | "source": [ 265 | "url_ds_h2g = f'{url_alongtrack}/dt_gulfstream_h2g_phy_l3_20161201-20180131_285-315_23-53.nc'\n", 266 | "store_ds_h2g = xr.backends.PydapDataStore.open(url_ds_h2g, session=my_aviso_session)" 267 | ] 268 | }, 269 | { 270 | "cell_type": "code", 271 | "execution_count": null, 272 | "metadata": {}, 273 | "outputs": [], 274 | "source": [ 275 | "ds_h2g = xr.open_dataset(store_ds_h2g)\n", 276 | "ds_h2g" 277 | ] 278 | }, 279 | { 280 | "cell_type": "markdown", 281 | "metadata": {}, 282 | "source": [ 283 | "### Example of figures" 284 | ] 285 | }, 286 | { 287 | "cell_type": "code", 288 | "execution_count": null, 289 | "metadata": {}, 290 | "outputs": [], 291 | "source": [ 292 | "list_of_dataset = [ds_alg, ds_j2g, ds_j2n, ds_j3, ds_s3a, ds_c2, ds_h2g]\n", 293 | "central_date = numpy.datetime64('2017-10-05')\n", 294 | "delta_t = numpy.timedelta64(5, 'D')" 295 | ] 296 | }, 297 | { 298 | "cell_type": "code", 299 | "execution_count": null, 300 | "metadata": {}, 301 | "outputs": [], 302 | "source": [ 303 | "plot_demo_obs(list_of_dataset, central_date, delta_t)" 304 | ] 305 | }, 306 | { 307 | "cell_type": "code", 308 | "execution_count": null, 309 | "metadata": {}, 310 | "outputs": [], 311 | "source": [] 312 | }, 313 | { 314 | "cell_type": "code", 315 | "execution_count": null, 316 | "metadata": {}, 317 | "outputs": [], 318 | "source": [ 319 | "url_ds_mdt = f'{url_map}/mdt.nc'\n", 320 | "store_ds_mdt = xr.backends.PydapDataStore.open(url_ds_mdt, session=my_aviso_session)\n", 321 | "ds_mdt = xr.open_dataset(store_ds_mdt)\n", 322 | "ds_mdt" 323 | ] 324 | }, 325 | { 326 | "cell_type": "code", 327 | "execution_count": null, 328 | "metadata": {}, 329 | "outputs": [], 330 | "source": [ 331 | "ds_mdt.hvplot.image(width=500, height=400)" 332 | ] 333 | }, 334 | { 335 | "cell_type": "code", 336 | "execution_count": null, 337 | "metadata": {}, 338 | "outputs": [], 339 | "source": [] 340 | } 341 | ], 342 | "metadata": { 343 | "kernelspec": { 344 | "display_name": "conda_env", 345 | "language": "python", 346 | "name": "conda_env" 347 | }, 348 | "language_info": { 349 | "codemirror_mode": { 350 | "name": "ipython", 351 | "version": 3 352 | }, 353 | "file_extension": ".py", 354 | "mimetype": "text/x-python", 355 | "name": "python", 356 | "nbconvert_exporter": "python", 357 | "pygments_lexer": "ipython3", 358 | "version": "3.7.8" 359 | } 360 | }, 361 | "nbformat": 4, 362 | "nbformat_minor": 4 363 | } 364 | -------------------------------------------------------------------------------- /notebooks/example_eval_4dvarnet.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Evaluation 4DVarNet: \n", 8 | "\n", 9 | "This notebook presents the evaluation of the SSH reconstructions based on the 4DVarNet method ([Fablet et al., 2021](https://) and performed for the **\"2021a_SSH_mapping_OSE\" ocean data challenge**. " 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": null, 15 | "metadata": {}, 16 | "outputs": [], 17 | "source": [ 18 | "import os\n", 19 | "import sys\n", 20 | "sys.path.append('..')\n", 21 | "import logging\n", 22 | "import pandas as pd" 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": null, 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "from src.mod_inout import *\n", 32 | "from src.mod_interp import *\n", 33 | "from src.mod_stats import *\n", 34 | "from src.mod_spectral import *\n", 35 | "from src.mod_plot import *" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "metadata": {}, 42 | "outputs": [], 43 | "source": [ 44 | "logger = logging.getLogger()\n", 45 | "logger.setLevel(logging.INFO)" 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": {}, 51 | "source": [ 52 | "### Study Area & Ouput Parameters" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "metadata": {}, 59 | "outputs": [], 60 | "source": [ 61 | "# study area\n", 62 | "lon_min = 295.\n", 63 | "lon_max = 305.\n", 64 | "lat_min = 33.\n", 65 | "lat_max = 43.\n", 66 | "is_circle = False\n", 67 | "time_min = '2017-01-01'\n", 68 | "time_max = '2017-12-31'\n", 69 | "\n", 70 | "# Outputs\n", 71 | "bin_lat_step = 1.\n", 72 | "bin_lon_step = 1.\n", 73 | "bin_time_step = '1D'\n", 74 | "output_directory = '../results'\n", 75 | "if not os.path.exists(output_directory):\n", 76 | " os.mkdir(output_directory)\n", 77 | "output_filename = f'{output_directory}/stat_OSE_4DVARNET_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\n", 78 | "output_filename_timeseries = f'{output_directory}/stat_timeseries_OSE_4DVARNET_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\n", 79 | "\n", 80 | "# Spectral parameter\n", 81 | "# C2 parameter\n", 82 | "delta_t = 0.9434 # s\n", 83 | "velocity = 6.77 # km/s\n", 84 | "delta_x = velocity * delta_t\n", 85 | "lenght_scale = 1000 # km\n", 86 | "output_filename_spectrum = f'{output_directory}/psd_OSE_4DVARNET_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'" 87 | ] 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "metadata": {}, 92 | "source": [ 93 | "### Open your AVISO+ session: fill the `````` and `````` items below" 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": null, 99 | "metadata": {}, 100 | "outputs": [], 101 | "source": [ 102 | "my_aviso_session = rq.Session()\n", 103 | "my_aviso_session.auth = (\"\", \"\")\n", 104 | "url_alongtrack = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-along-track-data'\n", 105 | "url_map = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-grid-data'" 106 | ] 107 | }, 108 | { 109 | "cell_type": "code", 110 | "execution_count": null, 111 | "metadata": {}, 112 | "outputs": [], 113 | "source": [ 114 | "alontrack_independent_dataset = f'{url_alongtrack}/dt_gulfstream_c2_phy_l3_20161201-20180131_285-315_23-53.nc'\n", 115 | "alontrack_independent_dataset" 116 | ] 117 | }, 118 | { 119 | "cell_type": "markdown", 120 | "metadata": {}, 121 | "source": [ 122 | "### Read L3 datasets" 123 | ] 124 | }, 125 | { 126 | "cell_type": "code", 127 | "execution_count": null, 128 | "metadata": {}, 129 | "outputs": [], 130 | "source": [ 131 | "# independent along-track\n", 132 | "alontrack_independent_dataset = f'{url_alongtrack}/dt_gulfstream_c2_phy_l3_20161201-20180131_285-315_23-53.nc'\n", 133 | "# Read along-track\n", 134 | "ds_alongtrack = read_l3_dataset_from_aviso(alontrack_independent_dataset, \n", 135 | " my_aviso_session,\n", 136 | " lon_min=lon_min, \n", 137 | " lon_max=lon_max, \n", 138 | " lat_min=lat_min, \n", 139 | " lat_max=lat_max, \n", 140 | " time_min=time_min, \n", 141 | " time_max=time_max)\n", 142 | "ds_alongtrack" 143 | ] 144 | }, 145 | { 146 | "cell_type": "markdown", 147 | "metadata": {}, 148 | "source": [ 149 | "### Read L4 dataset and interpolate onto along-track positions" 150 | ] 151 | }, 152 | { 153 | "cell_type": "code", 154 | "execution_count": null, 155 | "metadata": {}, 156 | "outputs": [], 157 | "source": [ 158 | "# series of maps to evaluate\n", 159 | "gridded_dataset = [f'{url_map}/OSE_ssh_mapping_4dvarNet.nc', my_aviso_session] \n", 160 | "#gridded_dataset = ['../inputs/dc_maps/OSE_ssh_mapping_4dvarNet.nc', my_aviso_session] \n", 161 | "#gridded_dataset = '../inputs/dc_maps/OSE_ssh_mapping_4dvarNet_v2.nc' #### A COMMENTER LORSQUE LA DONNEE SERA SUR AVISO\n", 162 | "# Interpolate maps onto alongtrack dataset\n", 163 | "time_alongtrack, lat_alongtrack, lon_alongtrack, ssh_alongtrack, ssh_map_interp = interp_on_alongtrack(gridded_dataset, \n", 164 | " ds_alongtrack,\n", 165 | " lon_min=lon_min, \n", 166 | " lon_max=lon_max, \n", 167 | " lat_min=lat_min, \n", 168 | " lat_max=lat_max, \n", 169 | " time_min=time_min, \n", 170 | " time_max=time_max,\n", 171 | " is_circle=is_circle)" 172 | ] 173 | }, 174 | { 175 | "cell_type": "markdown", 176 | "metadata": {}, 177 | "source": [ 178 | "### Compute statistical score" 179 | ] 180 | }, 181 | { 182 | "cell_type": "code", 183 | "execution_count": null, 184 | "metadata": {}, 185 | "outputs": [], 186 | "source": [ 187 | "leaderboard_nrmse, leaderboard_nrmse_std = compute_stats(time_alongtrack, \n", 188 | " lat_alongtrack, \n", 189 | " lon_alongtrack, \n", 190 | " ssh_alongtrack, \n", 191 | " ssh_map_interp, \n", 192 | " bin_lon_step,\n", 193 | " bin_lat_step, \n", 194 | " bin_time_step,\n", 195 | " output_filename,\n", 196 | " output_filename_timeseries)" 197 | ] 198 | }, 199 | { 200 | "cell_type": "code", 201 | "execution_count": null, 202 | "metadata": {}, 203 | "outputs": [], 204 | "source": [ 205 | "plot_spatial_statistics(output_filename)" 206 | ] 207 | }, 208 | { 209 | "cell_type": "code", 210 | "execution_count": null, 211 | "metadata": {}, 212 | "outputs": [], 213 | "source": [ 214 | "plot_temporal_statistics(output_filename_timeseries)" 215 | ] 216 | }, 217 | { 218 | "cell_type": "markdown", 219 | "metadata": {}, 220 | "source": [ 221 | "### Compute spectral scores" 222 | ] 223 | }, 224 | { 225 | "cell_type": "code", 226 | "execution_count": null, 227 | "metadata": {}, 228 | "outputs": [], 229 | "source": [ 230 | "compute_spectral_scores(time_alongtrack, \n", 231 | " lat_alongtrack, \n", 232 | " lon_alongtrack, \n", 233 | " ssh_alongtrack, \n", 234 | " ssh_map_interp, \n", 235 | " lenght_scale,\n", 236 | " delta_x,\n", 237 | " delta_t,\n", 238 | " output_filename_spectrum)" 239 | ] 240 | }, 241 | { 242 | "cell_type": "code", 243 | "execution_count": null, 244 | "metadata": {}, 245 | "outputs": [], 246 | "source": [ 247 | "leaderboard_psds_score = plot_psd_score(output_filename_spectrum)" 248 | ] 249 | }, 250 | { 251 | "cell_type": "code", 252 | "execution_count": null, 253 | "metadata": {}, 254 | "outputs": [], 255 | "source": [ 256 | "# Print leaderboard\n", 257 | "data = [['4DVarNet', \n", 258 | " leaderboard_nrmse, \n", 259 | " leaderboard_nrmse_std, \n", 260 | " int(leaderboard_psds_score),\n", 261 | " '4DVarNet mapping',\n", 262 | " 'example_eval_4DVarnet.ipynb']]\n", 263 | "Leaderboard = pd.DataFrame(data, \n", 264 | " columns=['Method', \n", 265 | " \"µ(RMSE) \", \n", 266 | " \"σ(RMSE)\", \n", 267 | " 'λx (km)', \n", 268 | " 'Notes',\n", 269 | " 'Reference'])\n", 270 | "print(\"Summary of the leaderboard metrics:\")\n", 271 | "Leaderboard\n", 272 | "print(Leaderboard.to_markdown())" 273 | ] 274 | }, 275 | { 276 | "cell_type": "code", 277 | "execution_count": null, 278 | "metadata": {}, 279 | "outputs": [], 280 | "source": [] 281 | }, 282 | { 283 | "cell_type": "code", 284 | "execution_count": null, 285 | "metadata": {}, 286 | "outputs": [], 287 | "source": [] 288 | } 289 | ], 290 | "metadata": { 291 | "kernelspec": { 292 | "display_name": "conda_env", 293 | "language": "python", 294 | "name": "conda_env" 295 | }, 296 | "language_info": { 297 | "codemirror_mode": { 298 | "name": "ipython", 299 | "version": 3 300 | }, 301 | "file_extension": ".py", 302 | "mimetype": "text/x-python", 303 | "name": "python", 304 | "nbconvert_exporter": "python", 305 | "pygments_lexer": "ipython3", 306 | "version": "3.7.8" 307 | } 308 | }, 309 | "nbformat": 4, 310 | "nbformat_minor": 4 311 | } 312 | -------------------------------------------------------------------------------- /notebooks/example_eval_4dvarnet_v2022.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Evaluation 4DVarNet: \n", 8 | "\n", 9 | "This notebook presents the evaluation of the SSH reconstructions based on the 4DVarNet method ([Fablet et al., 2021](https://) and performed for the **\"2021a_SSH_mapping_OSE\" ocean data challenge**. " 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": null, 15 | "metadata": {}, 16 | "outputs": [], 17 | "source": [ 18 | "import os\n", 19 | "import sys\n", 20 | "sys.path.append('..')\n", 21 | "import logging\n", 22 | "import pandas as pd" 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": null, 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "from src.mod_inout import *\n", 32 | "from src.mod_interp import *\n", 33 | "from src.mod_stats import *\n", 34 | "from src.mod_spectral import *\n", 35 | "from src.mod_plot import *" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "metadata": {}, 42 | "outputs": [], 43 | "source": [ 44 | "logger = logging.getLogger()\n", 45 | "logger.setLevel(logging.INFO)" 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": {}, 51 | "source": [ 52 | "### Study Area & Ouput Parameters" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "metadata": {}, 59 | "outputs": [], 60 | "source": [ 61 | "# study area\n", 62 | "lon_min = 295.\n", 63 | "lon_max = 305.\n", 64 | "lat_min = 33.\n", 65 | "lat_max = 43.\n", 66 | "is_circle = False\n", 67 | "time_min = '2017-01-01'\n", 68 | "time_max = '2017-12-31'\n", 69 | "\n", 70 | "# Outputs\n", 71 | "bin_lat_step = 1.\n", 72 | "bin_lon_step = 1.\n", 73 | "bin_time_step = '1D'\n", 74 | "output_directory = '../results'\n", 75 | "if not os.path.exists(output_directory):\n", 76 | " os.mkdir(output_directory)\n", 77 | "output_filename = f'{output_directory}/stat_OSE_4DVARNET_v2022_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\n", 78 | "output_filename_timeseries = f'{output_directory}/stat_timeseries_OSE_4DVARNET_v2022_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\n", 79 | "\n", 80 | "# Spectral parameter\n", 81 | "# C2 parameter\n", 82 | "delta_t = 0.9434 # s\n", 83 | "velocity = 6.77 # km/s\n", 84 | "delta_x = velocity * delta_t\n", 85 | "lenght_scale = 1000 # km\n", 86 | "output_filename_spectrum = f'{output_directory}/psd_OSE_4DVARNET_v2022_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'" 87 | ] 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "metadata": {}, 92 | "source": [ 93 | "### Open your AVISO+ session: fill the `````` and `````` items below" 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": null, 99 | "metadata": {}, 100 | "outputs": [], 101 | "source": [ 102 | "my_aviso_session = rq.Session()\n", 103 | "my_aviso_session.auth = (\"\", \"\")\n", 104 | "url_alongtrack = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-along-track-data'\n", 105 | "url_map = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-grid-data'" 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "metadata": {}, 111 | "source": [ 112 | "### Read L3 datasets" 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": null, 118 | "metadata": {}, 119 | "outputs": [], 120 | "source": [ 121 | "# independent along-track\n", 122 | "alontrack_independent_dataset = f'{url_alongtrack}/dt_gulfstream_c2_phy_l3_20161201-20180131_285-315_23-53.nc'\n", 123 | "# Read along-track\n", 124 | "ds_alongtrack = read_l3_dataset_from_aviso(alontrack_independent_dataset, \n", 125 | " my_aviso_session,\n", 126 | " lon_min=lon_min, \n", 127 | " lon_max=lon_max, \n", 128 | " lat_min=lat_min, \n", 129 | " lat_max=lat_max, \n", 130 | " time_min=time_min, \n", 131 | " time_max=time_max)\n", 132 | "ds_alongtrack" 133 | ] 134 | }, 135 | { 136 | "cell_type": "markdown", 137 | "metadata": {}, 138 | "source": [ 139 | "### Read L4 dataset and interpolate onto along-track positions" 140 | ] 141 | }, 142 | { 143 | "cell_type": "code", 144 | "execution_count": null, 145 | "metadata": {}, 146 | "outputs": [], 147 | "source": [ 148 | "# series of maps to evaluate\n", 149 | "gridded_dataset = [f'{url_map}/OSE_ssh_mapping_4dvarNet_2022.nc', my_aviso_session] \n", 150 | "# Interpolate maps onto alongtrack dataset\n", 151 | "time_alongtrack, lat_alongtrack, lon_alongtrack, ssh_alongtrack, ssh_map_interp = interp_on_alongtrack(gridded_dataset, \n", 152 | " ds_alongtrack,\n", 153 | " lon_min=lon_min, \n", 154 | " lon_max=lon_max, \n", 155 | " lat_min=lat_min, \n", 156 | " lat_max=lat_max, \n", 157 | " time_min=time_min, \n", 158 | " time_max=time_max,\n", 159 | " is_circle=is_circle)" 160 | ] 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "metadata": {}, 165 | "source": [ 166 | "### Compute statistical score" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": null, 172 | "metadata": {}, 173 | "outputs": [], 174 | "source": [ 175 | "leaderboard_nrmse, leaderboard_nrmse_std = compute_stats(time_alongtrack, \n", 176 | " lat_alongtrack, \n", 177 | " lon_alongtrack, \n", 178 | " ssh_alongtrack, \n", 179 | " ssh_map_interp, \n", 180 | " bin_lon_step,\n", 181 | " bin_lat_step, \n", 182 | " bin_time_step,\n", 183 | " output_filename,\n", 184 | " output_filename_timeseries)" 185 | ] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": null, 190 | "metadata": {}, 191 | "outputs": [], 192 | "source": [ 193 | "plot_spatial_statistics(output_filename)" 194 | ] 195 | }, 196 | { 197 | "cell_type": "code", 198 | "execution_count": null, 199 | "metadata": {}, 200 | "outputs": [], 201 | "source": [ 202 | "plot_temporal_statistics(output_filename_timeseries)" 203 | ] 204 | }, 205 | { 206 | "cell_type": "markdown", 207 | "metadata": {}, 208 | "source": [ 209 | "### Compute spectral scores" 210 | ] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": null, 215 | "metadata": {}, 216 | "outputs": [], 217 | "source": [ 218 | "compute_spectral_scores(time_alongtrack, \n", 219 | " lat_alongtrack, \n", 220 | " lon_alongtrack, \n", 221 | " ssh_alongtrack, \n", 222 | " ssh_map_interp, \n", 223 | " lenght_scale,\n", 224 | " delta_x,\n", 225 | " delta_t,\n", 226 | " output_filename_spectrum)" 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": null, 232 | "metadata": {}, 233 | "outputs": [], 234 | "source": [ 235 | "leaderboard_psds_score = plot_psd_score(output_filename_spectrum)" 236 | ] 237 | }, 238 | { 239 | "cell_type": "code", 240 | "execution_count": null, 241 | "metadata": {}, 242 | "outputs": [], 243 | "source": [ 244 | "# Print leaderboard\n", 245 | "data = [['4DVarNet', \n", 246 | " leaderboard_nrmse, \n", 247 | " leaderboard_nrmse_std, \n", 248 | " int(leaderboard_psds_score),\n", 249 | " '4DVarNet mapping',\n", 250 | " 'example_eval_4DVarnet.ipynb']]\n", 251 | "Leaderboard = pd.DataFrame(data, \n", 252 | " columns=['Method', \n", 253 | " \"µ(RMSE) \", \n", 254 | " \"σ(RMSE)\", \n", 255 | " 'λx (km)', \n", 256 | " 'Notes',\n", 257 | " 'Reference'])\n", 258 | "print(\"Summary of the leaderboard metrics:\")\n", 259 | "Leaderboard\n", 260 | "print(Leaderboard.to_markdown())" 261 | ] 262 | }, 263 | { 264 | "cell_type": "code", 265 | "execution_count": null, 266 | "metadata": {}, 267 | "outputs": [], 268 | "source": [] 269 | }, 270 | { 271 | "cell_type": "code", 272 | "execution_count": null, 273 | "metadata": {}, 274 | "outputs": [], 275 | "source": [] 276 | } 277 | ], 278 | "metadata": { 279 | "kernelspec": { 280 | "display_name": "conda_env", 281 | "language": "python", 282 | "name": "conda_env" 283 | }, 284 | "language_info": { 285 | "codemirror_mode": { 286 | "name": "ipython", 287 | "version": 3 288 | }, 289 | "file_extension": ".py", 290 | "mimetype": "text/x-python", 291 | "name": "python", 292 | "nbconvert_exporter": "python", 293 | "pygments_lexer": "ipython3", 294 | "version": "3.9.12" 295 | } 296 | }, 297 | "nbformat": 4, 298 | "nbformat_minor": 4 299 | } 300 | -------------------------------------------------------------------------------- /notebooks/example_eval_baseline.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Evaluation BASELINE OI: \n", 8 | "\n", 9 | "This notebook presents the evaluation of the SSH reconstructions based on the BASELINE OI and performed for the **\"2021a_SSH_mapping_OSE\" ocean data challenge**. " 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": null, 15 | "metadata": {}, 16 | "outputs": [], 17 | "source": [ 18 | "import os\n", 19 | "import sys\n", 20 | "sys.path.append('..')\n", 21 | "import logging\n", 22 | "import pandas as pd" 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": null, 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "from src.mod_inout import *\n", 32 | "from src.mod_interp import *\n", 33 | "from src.mod_stats import *\n", 34 | "from src.mod_spectral import *\n", 35 | "from src.mod_plot import *" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "metadata": {}, 42 | "outputs": [], 43 | "source": [ 44 | "logger = logging.getLogger()\n", 45 | "logger.setLevel(logging.INFO)" 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": {}, 51 | "source": [ 52 | "### Study Area & Ouput Parameters" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "metadata": {}, 59 | "outputs": [], 60 | "source": [ 61 | "# study area\n", 62 | "lon_min = 295.\n", 63 | "lon_max = 305.\n", 64 | "lat_min = 33.\n", 65 | "lat_max = 43.\n", 66 | "is_circle = False\n", 67 | "time_min = '2017-01-01'\n", 68 | "time_max = '2017-12-31'\n", 69 | "\n", 70 | "# Outputs\n", 71 | "bin_lat_step = 1.\n", 72 | "bin_lon_step = 1.\n", 73 | "bin_time_step = '1D'\n", 74 | "output_directory = '../results'\n", 75 | "if not os.path.exists(output_directory):\n", 76 | " os.mkdir(output_directory)\n", 77 | "output_filename = f'{output_directory}/stat_OSE_BASELINE_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\n", 78 | "output_filename_timeseries = f'{output_directory}/stat_timeseries_OSE_BASELINE_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\n", 79 | "\n", 80 | "# Spectral parameter\n", 81 | "# C2 parameter\n", 82 | "delta_t = 0.9434 # s\n", 83 | "velocity = 6.77 # km/s\n", 84 | "delta_x = velocity * delta_t\n", 85 | "lenght_scale = 1000 # sehment length scale in km\n", 86 | "output_filename_spectrum = f'{output_directory}/psd_OSE_BASELINE_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'" 87 | ] 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "metadata": {}, 92 | "source": [ 93 | "### Open your AVISO+ session: fill the `````` and `````` items below" 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": null, 99 | "metadata": {}, 100 | "outputs": [], 101 | "source": [ 102 | "my_aviso_session = rq.Session()\n", 103 | "my_aviso_session.auth = (\"\", \"\")\n", 104 | "url_alongtrack = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-along-track-data'\n", 105 | "url_map = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-grid-data'" 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "metadata": {}, 111 | "source": [ 112 | "### Read L3 datasets" 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": null, 118 | "metadata": {}, 119 | "outputs": [], 120 | "source": [ 121 | "# independent along-track\n", 122 | "alontrack_independent_dataset = f'{url_alongtrack}/dt_gulfstream_c2_phy_l3_20161201-20180131_285-315_23-53.nc'\n", 123 | "# Read along-track\n", 124 | "ds_alongtrack = read_l3_dataset_from_aviso(alontrack_independent_dataset,\n", 125 | " my_aviso_session,\n", 126 | " lon_min=lon_min, \n", 127 | " lon_max=lon_max, \n", 128 | " lat_min=lat_min, \n", 129 | " lat_max=lat_max, \n", 130 | " time_min=time_min, \n", 131 | " time_max=time_max)\n", 132 | "ds_alongtrack" 133 | ] 134 | }, 135 | { 136 | "cell_type": "markdown", 137 | "metadata": {}, 138 | "source": [ 139 | "### Read L4 dataset and interpolate onto along-track positions" 140 | ] 141 | }, 142 | { 143 | "cell_type": "code", 144 | "execution_count": null, 145 | "metadata": {}, 146 | "outputs": [], 147 | "source": [ 148 | "# series of maps to evaluate\n", 149 | "# gridded_dataset = '../results/OSE_ssh_mapping_BASELINE.nc'\n", 150 | "# Alternatively, read data from AVISO server\n", 151 | "gridded_dataset = [f'{url_map}/OSE_ssh_mapping_BASELINE.nc', my_aviso_session] \n", 152 | "# Interpolate maps onto alongtrack dataset\n", 153 | "time_alongtrack, lat_alongtrack, lon_alongtrack, ssh_alongtrack, ssh_map_interp = interp_on_alongtrack(gridded_dataset, \n", 154 | " ds_alongtrack,\n", 155 | " lon_min=lon_min, \n", 156 | " lon_max=lon_max, \n", 157 | " lat_min=lat_min, \n", 158 | " lat_max=lat_max, \n", 159 | " time_min=time_min, \n", 160 | " time_max=time_max,\n", 161 | " is_circle=is_circle)" 162 | ] 163 | }, 164 | { 165 | "cell_type": "markdown", 166 | "metadata": {}, 167 | "source": [ 168 | "### Compute statistical score" 169 | ] 170 | }, 171 | { 172 | "cell_type": "code", 173 | "execution_count": null, 174 | "metadata": {}, 175 | "outputs": [], 176 | "source": [ 177 | "leaderboard_nrmse, leaderboard_nrmse_std = compute_stats(time_alongtrack, \n", 178 | " lat_alongtrack, \n", 179 | " lon_alongtrack, \n", 180 | " ssh_alongtrack, \n", 181 | " ssh_map_interp, \n", 182 | " bin_lon_step,\n", 183 | " bin_lat_step, \n", 184 | " bin_time_step,\n", 185 | " output_filename,\n", 186 | " output_filename_timeseries)" 187 | ] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "execution_count": null, 192 | "metadata": {}, 193 | "outputs": [], 194 | "source": [ 195 | "plot_spatial_statistics(output_filename)" 196 | ] 197 | }, 198 | { 199 | "cell_type": "code", 200 | "execution_count": null, 201 | "metadata": {}, 202 | "outputs": [], 203 | "source": [ 204 | "plot_temporal_statistics(output_filename_timeseries)" 205 | ] 206 | }, 207 | { 208 | "cell_type": "markdown", 209 | "metadata": {}, 210 | "source": [ 211 | "### Compute spectral scores" 212 | ] 213 | }, 214 | { 215 | "cell_type": "code", 216 | "execution_count": null, 217 | "metadata": {}, 218 | "outputs": [], 219 | "source": [ 220 | "compute_spectral_scores(time_alongtrack, \n", 221 | " lat_alongtrack, \n", 222 | " lon_alongtrack, \n", 223 | " ssh_alongtrack, \n", 224 | " ssh_map_interp, \n", 225 | " lenght_scale,\n", 226 | " delta_x,\n", 227 | " delta_t,\n", 228 | " output_filename_spectrum)" 229 | ] 230 | }, 231 | { 232 | "cell_type": "code", 233 | "execution_count": null, 234 | "metadata": {}, 235 | "outputs": [], 236 | "source": [ 237 | "leaderboard_psds_score = plot_psd_score(output_filename_spectrum)" 238 | ] 239 | }, 240 | { 241 | "cell_type": "markdown", 242 | "metadata": {}, 243 | "source": [ 244 | "### Show leaderboard metrics" 245 | ] 246 | }, 247 | { 248 | "cell_type": "code", 249 | "execution_count": null, 250 | "metadata": {}, 251 | "outputs": [], 252 | "source": [ 253 | "# Print leaderboard\n", 254 | "data = [['BASELINE', \n", 255 | " leaderboard_nrmse, \n", 256 | " leaderboard_nrmse_std, \n", 257 | " int(leaderboard_psds_score),\n", 258 | " 'Covariances BASELINE OI',\n", 259 | " 'example_eval_baseline.ipynb']]\n", 260 | "Leaderboard = pd.DataFrame(data, \n", 261 | " columns=['Method', \n", 262 | " \"µ(RMSE) \", \n", 263 | " \"σ(RMSE)\", \n", 264 | " 'λx (km)', \n", 265 | " 'Notes',\n", 266 | " 'Reference'])\n", 267 | "print(\"Summary of the leaderboard metrics:\")\n", 268 | "Leaderboard\n", 269 | "print(Leaderboard.to_markdown())" 270 | ] 271 | }, 272 | { 273 | "cell_type": "code", 274 | "execution_count": null, 275 | "metadata": {}, 276 | "outputs": [], 277 | "source": [] 278 | }, 279 | { 280 | "cell_type": "code", 281 | "execution_count": null, 282 | "metadata": {}, 283 | "outputs": [], 284 | "source": [] 285 | }, 286 | { 287 | "cell_type": "code", 288 | "execution_count": null, 289 | "metadata": {}, 290 | "outputs": [], 291 | "source": [] 292 | } 293 | ], 294 | "metadata": { 295 | "kernelspec": { 296 | "display_name": "conda_env", 297 | "language": "python", 298 | "name": "conda_env" 299 | }, 300 | "language_info": { 301 | "codemirror_mode": { 302 | "name": "ipython", 303 | "version": 3 304 | }, 305 | "file_extension": ".py", 306 | "mimetype": "text/x-python", 307 | "name": "python", 308 | "nbconvert_exporter": "python", 309 | "pygments_lexer": "ipython3", 310 | "version": "3.7.8" 311 | } 312 | }, 313 | "nbformat": 4, 314 | "nbformat_minor": 4 315 | } 316 | -------------------------------------------------------------------------------- /notebooks/example_eval_bfn.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "source": [ 6 | "# Evaluation Back-and-Forth Nudging Mapping (BNF): \n", 7 | "\n", 8 | "This notebook presents the evaluation of the SSH reconstructions based on the Back-and-Forth Nudging a One-Layer Quasigeostrophic Model method ([Le Guillou et al., 2021](https://journals.ametsoc.org/view/journals/atot/38/4/JTECH-D-20-0104.1.xml)) and performed for the **\"2021a_SSH_mapping_OSE\" ocean data challenge**. " 9 | ], 10 | "metadata": {} 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": null, 15 | "source": [ 16 | "import os\r\n", 17 | "import sys\r\n", 18 | "sys.path.append('..')\r\n", 19 | "import logging\r\n", 20 | "import pandas as pd" 21 | ], 22 | "outputs": [], 23 | "metadata": {} 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": null, 28 | "source": [ 29 | "from src.mod_inout import *\r\n", 30 | "from src.mod_interp import *\r\n", 31 | "from src.mod_stats import *\r\n", 32 | "from src.mod_spectral import *\r\n", 33 | "from src.mod_plot import *" 34 | ], 35 | "outputs": [], 36 | "metadata": {} 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "source": [ 42 | "logger = logging.getLogger()\r\n", 43 | "logger.setLevel(logging.INFO)" 44 | ], 45 | "outputs": [], 46 | "metadata": {} 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "source": [ 51 | "### Study Area & Ouput Parameters" 52 | ], 53 | "metadata": {} 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "source": [ 59 | "# study area\r\n", 60 | "lon_min = 295.\r\n", 61 | "lon_max = 305.\r\n", 62 | "lat_min = 33.\r\n", 63 | "lat_max = 43.\r\n", 64 | "is_circle = False\r\n", 65 | "time_min = '2017-01-01'\r\n", 66 | "time_max = '2017-12-31'\r\n", 67 | "\r\n", 68 | "# Outputs\r\n", 69 | "bin_lat_step = 1.\r\n", 70 | "bin_lon_step = 1.\r\n", 71 | "bin_time_step = '1D'\r\n", 72 | "output_directory = '../results'\r\n", 73 | "if not os.path.exists(output_directory):\r\n", 74 | " os.mkdir(output_directory)\r\n", 75 | "output_filename = f'{output_directory}/stat_OSE_BFN_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\r\n", 76 | "output_filename_timeseries = f'{output_directory}/stat_timeseries_OSE_BFN_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\r\n", 77 | "\r\n", 78 | "# Spectral parameter\r\n", 79 | "# C2 parameter\r\n", 80 | "delta_t = 0.9434 # s\r\n", 81 | "velocity = 6.77 # km/s\r\n", 82 | "delta_x = velocity * delta_t\r\n", 83 | "lenght_scale = 1000 # km\r\n", 84 | "output_filename_spectrum = f'{output_directory}/psd_OSE_BFN_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'" 85 | ], 86 | "outputs": [], 87 | "metadata": {} 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "source": [ 92 | "### Open your AVISO+ session: fill the `````` and `````` items below" 93 | ], 94 | "metadata": {} 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": null, 99 | "source": [ 100 | "my_aviso_session = rq.Session()\r\n", 101 | "my_aviso_session.auth = (\"\", \"\")\r\n", 102 | "url_alongtrack = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-along-track-data'\r\n", 103 | "url_map = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-grid-data'" 104 | ], 105 | "outputs": [], 106 | "metadata": {} 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "source": [ 111 | "### Read L3 datasets" 112 | ], 113 | "metadata": {} 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": null, 118 | "source": [ 119 | "# independent along-track\r\n", 120 | "alontrack_independent_dataset = f'{url_alongtrack}/dt_gulfstream_c2_phy_l3_20161201-20180131_285-315_23-53.nc'\r\n", 121 | "# Read along-track\r\n", 122 | "ds_alongtrack = read_l3_dataset_from_aviso(alontrack_independent_dataset, \r\n", 123 | " my_aviso_session,\r\n", 124 | " lon_min=lon_min, \r\n", 125 | " lon_max=lon_max, \r\n", 126 | " lat_min=lat_min, \r\n", 127 | " lat_max=lat_max, \r\n", 128 | " time_min=time_min, \r\n", 129 | " time_max=time_max)\r\n", 130 | "ds_alongtrack" 131 | ], 132 | "outputs": [], 133 | "metadata": {} 134 | }, 135 | { 136 | "cell_type": "code", 137 | "execution_count": null, 138 | "source": [], 139 | "outputs": [], 140 | "metadata": {} 141 | }, 142 | { 143 | "cell_type": "markdown", 144 | "source": [ 145 | "### Read L4 dataset and interpolate onto along-track positions" 146 | ], 147 | "metadata": {} 148 | }, 149 | { 150 | "cell_type": "code", 151 | "execution_count": null, 152 | "source": [ 153 | "# series of maps to evaluate\r\n", 154 | "# gridded_dataset = '../inputs/dc_maps/OSE_ssh_mapping_BFN.nc'\r\n", 155 | "gridded_dataset = [f'{url_map}/OSE_ssh_mapping_BFN.nc', my_aviso_session] \r\n", 156 | "# Interpolate maps onto alongtrack dataset\r\n", 157 | "time_alongtrack, lat_alongtrack, lon_alongtrack, ssh_alongtrack, ssh_map_interp = interp_on_alongtrack(gridded_dataset, \r\n", 158 | " ds_alongtrack,\r\n", 159 | " lon_min=lon_min, \r\n", 160 | " lon_max=lon_max, \r\n", 161 | " lat_min=lat_min, \r\n", 162 | " lat_max=lat_max, \r\n", 163 | " time_min=time_min, \r\n", 164 | " time_max=time_max,\r\n", 165 | " is_circle=is_circle)" 166 | ], 167 | "outputs": [], 168 | "metadata": {} 169 | }, 170 | { 171 | "cell_type": "markdown", 172 | "source": [ 173 | "### Compute statistical score" 174 | ], 175 | "metadata": {} 176 | }, 177 | { 178 | "cell_type": "code", 179 | "execution_count": null, 180 | "source": [ 181 | "leaderboard_nrmse, leaderboard_nrmse_std = compute_stats(time_alongtrack, \r\n", 182 | " lat_alongtrack, \r\n", 183 | " lon_alongtrack, \r\n", 184 | " ssh_alongtrack, \r\n", 185 | " ssh_map_interp, \r\n", 186 | " bin_lon_step,\r\n", 187 | " bin_lat_step, \r\n", 188 | " bin_time_step,\r\n", 189 | " output_filename,\r\n", 190 | " output_filename_timeseries)" 191 | ], 192 | "outputs": [], 193 | "metadata": {} 194 | }, 195 | { 196 | "cell_type": "code", 197 | "execution_count": null, 198 | "source": [ 199 | "plot_spatial_statistics(output_filename)" 200 | ], 201 | "outputs": [], 202 | "metadata": {} 203 | }, 204 | { 205 | "cell_type": "code", 206 | "execution_count": null, 207 | "source": [ 208 | "plot_temporal_statistics(output_filename_timeseries)" 209 | ], 210 | "outputs": [], 211 | "metadata": {} 212 | }, 213 | { 214 | "cell_type": "markdown", 215 | "source": [ 216 | "### Compute spectral scores" 217 | ], 218 | "metadata": {} 219 | }, 220 | { 221 | "cell_type": "code", 222 | "execution_count": null, 223 | "source": [ 224 | "compute_spectral_scores(time_alongtrack, \r\n", 225 | " lat_alongtrack, \r\n", 226 | " lon_alongtrack, \r\n", 227 | " ssh_alongtrack, \r\n", 228 | " ssh_map_interp, \r\n", 229 | " lenght_scale,\r\n", 230 | " delta_x,\r\n", 231 | " delta_t,\r\n", 232 | " output_filename_spectrum)" 233 | ], 234 | "outputs": [], 235 | "metadata": {} 236 | }, 237 | { 238 | "cell_type": "code", 239 | "execution_count": null, 240 | "source": [ 241 | "leaderboard_psds_score = plot_psd_score(output_filename_spectrum)" 242 | ], 243 | "outputs": [], 244 | "metadata": {} 245 | }, 246 | { 247 | "cell_type": "code", 248 | "execution_count": null, 249 | "source": [], 250 | "outputs": [], 251 | "metadata": {} 252 | }, 253 | { 254 | "cell_type": "code", 255 | "execution_count": null, 256 | "source": [ 257 | "# Print leaderboard\r\n", 258 | "data = [['BNF', \r\n", 259 | " leaderboard_nrmse, \r\n", 260 | " leaderboard_nrmse_std, \r\n", 261 | " int(leaderboard_psds_score),\r\n", 262 | " 'BFN mapping',\r\n", 263 | " 'example_eval_bfn.ipynb']]\r\n", 264 | "Leaderboard = pd.DataFrame(data, \r\n", 265 | " columns=['Method', \r\n", 266 | " \"µ(RMSE) \", \r\n", 267 | " \"σ(RMSE)\", \r\n", 268 | " 'λx (km)', \r\n", 269 | " 'Notes',\r\n", 270 | " 'Reference'])\r\n", 271 | "print(\"Summary of the leaderboard metrics:\")\r\n", 272 | "Leaderboard\r\n", 273 | "print(Leaderboard.to_markdown())" 274 | ], 275 | "outputs": [], 276 | "metadata": {} 277 | }, 278 | { 279 | "cell_type": "code", 280 | "execution_count": null, 281 | "source": [], 282 | "outputs": [], 283 | "metadata": {} 284 | }, 285 | { 286 | "cell_type": "code", 287 | "execution_count": null, 288 | "source": [], 289 | "outputs": [], 290 | "metadata": {} 291 | }, 292 | { 293 | "cell_type": "code", 294 | "execution_count": null, 295 | "source": [], 296 | "outputs": [], 297 | "metadata": {} 298 | } 299 | ], 300 | "metadata": { 301 | "kernelspec": { 302 | "display_name": "conda_env", 303 | "language": "python", 304 | "name": "conda_env" 305 | }, 306 | "language_info": { 307 | "codemirror_mode": { 308 | "name": "ipython", 309 | "version": 3 310 | }, 311 | "file_extension": ".py", 312 | "mimetype": "text/x-python", 313 | "name": "python", 314 | "nbconvert_exporter": "python", 315 | "pygments_lexer": "ipython3", 316 | "version": "3.7.8" 317 | } 318 | }, 319 | "nbformat": 4, 320 | "nbformat_minor": 4 321 | } -------------------------------------------------------------------------------- /notebooks/example_eval_duacs.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "source": [ 6 | "# Evaluation DUACS OI: \n", 7 | "\n", 8 | "This notebook presents the evaluation of the SSH reconstructions based on the DUACS OI ([Taburet et al., 2018](https://os.copernicus.org/articles/15/1207/2019/)) and performed for the **\"2021a_SSH_mapping_OSE\" ocean data challenge**. " 9 | ], 10 | "metadata": {} 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": null, 15 | "source": [ 16 | "import os\r\n", 17 | "import sys\r\n", 18 | "sys.path.append('..')\r\n", 19 | "import logging\r\n", 20 | "import pandas as pd" 21 | ], 22 | "outputs": [], 23 | "metadata": {} 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": null, 28 | "source": [ 29 | "from src.mod_inout import *\r\n", 30 | "from src.mod_interp import *\r\n", 31 | "from src.mod_stats import *\r\n", 32 | "from src.mod_spectral import *\r\n", 33 | "from src.mod_plot import *" 34 | ], 35 | "outputs": [], 36 | "metadata": {} 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "source": [ 42 | "logger = logging.getLogger()\r\n", 43 | "logger.setLevel(logging.INFO)" 44 | ], 45 | "outputs": [], 46 | "metadata": {} 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "source": [ 51 | "### Study Area & Ouput Parameters" 52 | ], 53 | "metadata": {} 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "source": [ 59 | "# study area\r\n", 60 | "lon_min = 295.\r\n", 61 | "lon_max = 305.\r\n", 62 | "lat_min = 33.\r\n", 63 | "lat_max = 43.\r\n", 64 | "is_circle = False\r\n", 65 | "time_min = '2017-01-01'\r\n", 66 | "time_max = '2017-12-31'\r\n", 67 | "\r\n", 68 | "# Outputs\r\n", 69 | "bin_lat_step = 1.\r\n", 70 | "bin_lon_step = 1.\r\n", 71 | "bin_time_step = '1D'\r\n", 72 | "output_directory = '../results'\r\n", 73 | "if not os.path.exists(output_directory):\r\n", 74 | " os.mkdir(output_directory)\r\n", 75 | "output_filename = f'{output_directory}/stat_OSE_DUACS_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\r\n", 76 | "output_filename_timeseries = f'{output_directory}/stat_timeseries_OSE_DUACS_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\r\n", 77 | "\r\n", 78 | "# Spectral parameter\r\n", 79 | "# C2 parameter\r\n", 80 | "delta_t = 0.9434 # s\r\n", 81 | "velocity = 6.77 # km/s\r\n", 82 | "delta_x = velocity * delta_t\r\n", 83 | "lenght_scale = 1000 # km\r\n", 84 | "output_filename_spectrum = f'{output_directory}/psd_OSE_DUACS_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'" 85 | ], 86 | "outputs": [], 87 | "metadata": {} 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "source": [ 92 | "### Open your AVISO+ session: fill the `````` and `````` items below" 93 | ], 94 | "metadata": {} 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": null, 99 | "source": [ 100 | "my_aviso_session = rq.Session()\r\n", 101 | "my_aviso_session.auth = (\"\", \"\")\r\n", 102 | "url_alongtrack = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-along-track-data'\r\n", 103 | "url_map = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-grid-data'" 104 | ], 105 | "outputs": [], 106 | "metadata": {} 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "source": [ 111 | "### Read L3 datasets" 112 | ], 113 | "metadata": {} 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": null, 118 | "source": [ 119 | "# independent along-track\r\n", 120 | "alontrack_independent_dataset = f'{url_alongtrack}/dt_gulfstream_c2_phy_l3_20161201-20180131_285-315_23-53.nc'\r\n", 121 | "# Read along-track\r\n", 122 | "ds_alongtrack = read_l3_dataset_from_aviso(alontrack_independent_dataset, \r\n", 123 | " my_aviso_session, \r\n", 124 | " lon_min=lon_min, \r\n", 125 | " lon_max=lon_max, \r\n", 126 | " lat_min=lat_min, \r\n", 127 | " lat_max=lat_max, \r\n", 128 | " time_min=time_min, \r\n", 129 | " time_max=time_max)\r\n", 130 | "ds_alongtrack" 131 | ], 132 | "outputs": [], 133 | "metadata": {} 134 | }, 135 | { 136 | "cell_type": "markdown", 137 | "source": [ 138 | "### Read L4 dataset and interpolate onto along-track positions" 139 | ], 140 | "metadata": {} 141 | }, 142 | { 143 | "cell_type": "code", 144 | "execution_count": null, 145 | "source": [ 146 | "# series of maps to evaluate\r\n", 147 | "gridded_dataset = [f'{url_map}/OSE_ssh_mapping_DUACS.nc', my_aviso_session]\r\n", 148 | "# Interpolate maps onto alongtrack dataset\r\n", 149 | "time_alongtrack, lat_alongtrack, lon_alongtrack, ssh_alongtrack, ssh_map_interp = interp_on_alongtrack(gridded_dataset, \r\n", 150 | " ds_alongtrack,\r\n", 151 | " lon_min=lon_min, \r\n", 152 | " lon_max=lon_max, \r\n", 153 | " lat_min=lat_min, \r\n", 154 | " lat_max=lat_max, \r\n", 155 | " time_min=time_min, \r\n", 156 | " time_max=time_max,\r\n", 157 | " is_circle=is_circle)" 158 | ], 159 | "outputs": [], 160 | "metadata": {} 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "source": [ 165 | "### Compute statistical score" 166 | ], 167 | "metadata": {} 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": null, 172 | "source": [ 173 | "leaderboard_nrmse, leaderboard_nrmse_std = compute_stats(time_alongtrack, \r\n", 174 | " lat_alongtrack, \r\n", 175 | " lon_alongtrack, \r\n", 176 | " ssh_alongtrack, \r\n", 177 | " ssh_map_interp, \r\n", 178 | " bin_lon_step,\r\n", 179 | " bin_lat_step, \r\n", 180 | " bin_time_step,\r\n", 181 | " output_filename,\r\n", 182 | " output_filename_timeseries)" 183 | ], 184 | "outputs": [], 185 | "metadata": {} 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": null, 190 | "source": [ 191 | "plot_spatial_statistics(output_filename)" 192 | ], 193 | "outputs": [], 194 | "metadata": {} 195 | }, 196 | { 197 | "cell_type": "code", 198 | "execution_count": null, 199 | "source": [ 200 | "plot_temporal_statistics(output_filename_timeseries)" 201 | ], 202 | "outputs": [], 203 | "metadata": {} 204 | }, 205 | { 206 | "cell_type": "markdown", 207 | "source": [ 208 | "### Compute spectral scores" 209 | ], 210 | "metadata": {} 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": null, 215 | "source": [ 216 | "compute_spectral_scores(time_alongtrack, \n", 217 | " lat_alongtrack, \n", 218 | " lon_alongtrack, \n", 219 | " ssh_alongtrack, \n", 220 | " ssh_map_interp, \n", 221 | " lenght_scale,\n", 222 | " delta_x,\n", 223 | " delta_t,\n", 224 | " output_filename_spectrum)" 225 | ], 226 | "outputs": [], 227 | "metadata": {} 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": null, 232 | "source": [ 233 | "leaderboard_psds_score = plot_psd_score(output_filename_spectrum)" 234 | ], 235 | "outputs": [], 236 | "metadata": {} 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "source": [ 241 | "### Show leaderboard metrics" 242 | ], 243 | "metadata": {} 244 | }, 245 | { 246 | "cell_type": "code", 247 | "execution_count": null, 248 | "source": [ 249 | "# Print leaderboard\n", 250 | "data = [['DUACS', \n", 251 | " leaderboard_nrmse, \n", 252 | " leaderboard_nrmse_std, \n", 253 | " int(leaderboard_psds_score),\n", 254 | " 'Covariances DUACS',\n", 255 | " 'example_eval_duacs.ipynb']]\n", 256 | "Leaderboard = pd.DataFrame(data, \n", 257 | " columns=['Method', \n", 258 | " \"µ(RMSE) \", \n", 259 | " \"σ(RMSE)\", \n", 260 | " 'λx (km)', \n", 261 | " 'Notes',\n", 262 | " 'Reference'])\n", 263 | "print(\"Summary of the leaderboard metrics:\")\n", 264 | "Leaderboard\n", 265 | "print(Leaderboard.to_markdown())" 266 | ], 267 | "outputs": [], 268 | "metadata": {} 269 | }, 270 | { 271 | "cell_type": "code", 272 | "execution_count": null, 273 | "source": [], 274 | "outputs": [], 275 | "metadata": {} 276 | }, 277 | { 278 | "cell_type": "code", 279 | "execution_count": null, 280 | "source": [], 281 | "outputs": [], 282 | "metadata": {} 283 | }, 284 | { 285 | "cell_type": "code", 286 | "execution_count": null, 287 | "source": [], 288 | "outputs": [], 289 | "metadata": {} 290 | } 291 | ], 292 | "metadata": { 293 | "kernelspec": { 294 | "display_name": "conda_env", 295 | "language": "python", 296 | "name": "conda_env" 297 | }, 298 | "language_info": { 299 | "codemirror_mode": { 300 | "name": "ipython", 301 | "version": 3 302 | }, 303 | "file_extension": ".py", 304 | "mimetype": "text/x-python", 305 | "name": "python", 306 | "nbconvert_exporter": "python", 307 | "pygments_lexer": "ipython3", 308 | "version": "3.7.8" 309 | } 310 | }, 311 | "nbformat": 4, 312 | "nbformat_minor": 4 313 | } -------------------------------------------------------------------------------- /notebooks/example_eval_dymost.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "source": [ 6 | "# Evaluation Dynamic Interpolation (DYMOST): \n", 7 | "\n", 8 | "This notebook presents the evaluation of the SSH reconstructions based on the Dynamic Interpolation method ([Ballarotta et al., 2020](https://journals.ametsoc.org/view/journals/atot/37/9/jtechD200030.xml)) and performed for the **\"2021a_SSH_mapping_OSE\" ocean data challenge**. " 9 | ], 10 | "metadata": {} 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": null, 15 | "source": [ 16 | "import os\r\n", 17 | "import sys\r\n", 18 | "sys.path.append('..')\r\n", 19 | "import logging\r\n", 20 | "import pandas as pd" 21 | ], 22 | "outputs": [], 23 | "metadata": {} 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": null, 28 | "source": [ 29 | "from src.mod_inout import *\r\n", 30 | "from src.mod_interp import *\r\n", 31 | "from src.mod_stats import *\r\n", 32 | "from src.mod_spectral import *\r\n", 33 | "from src.mod_plot import *" 34 | ], 35 | "outputs": [], 36 | "metadata": {} 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "source": [ 42 | "logger = logging.getLogger()\r\n", 43 | "logger.setLevel(logging.INFO)" 44 | ], 45 | "outputs": [], 46 | "metadata": {} 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "source": [ 51 | "### Study Area & Ouput Parameters" 52 | ], 53 | "metadata": {} 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "source": [ 59 | "# study area\r\n", 60 | "lon_min = 295.\r\n", 61 | "lon_max = 305.\r\n", 62 | "lat_min = 33.\r\n", 63 | "lat_max = 43.\r\n", 64 | "is_circle = False\r\n", 65 | "time_min = '2017-01-01'\r\n", 66 | "time_max = '2017-12-31'\r\n", 67 | "\r\n", 68 | "# Outputs\r\n", 69 | "bin_lat_step = 1.\r\n", 70 | "bin_lon_step = 1.\r\n", 71 | "bin_time_step = '1D'\r\n", 72 | "output_directory = '../results'\r\n", 73 | "if not os.path.exists(output_directory):\r\n", 74 | " os.mkdir(output_directory)\r\n", 75 | "output_filename = f'{output_directory}/stat_OSE_DYMOST_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\r\n", 76 | "output_filename_timeseries = f'{output_directory}/stat_timeseries_OSE_DYMOST_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\r\n", 77 | "\r\n", 78 | "# Spectral parameter\r\n", 79 | "# C2 parameter\r\n", 80 | "delta_t = 0.9434 # s\r\n", 81 | "velocity = 6.77 # km/s\r\n", 82 | "delta_x = velocity * delta_t\r\n", 83 | "lenght_scale = 1000 # km\r\n", 84 | "output_filename_spectrum = f'{output_directory}/psd_OSE_DYMOST_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'" 85 | ], 86 | "outputs": [], 87 | "metadata": {} 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "source": [ 92 | "### Open your AVISO+ session: fill the `````` and `````` items below" 93 | ], 94 | "metadata": {} 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": null, 99 | "source": [ 100 | "my_aviso_session = rq.Session()\r\n", 101 | "my_aviso_session.auth = (\"\", \"\")\r\n", 102 | "url_alongtrack = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-along-track-data'\r\n", 103 | "url_map = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-grid-data'" 104 | ], 105 | "outputs": [], 106 | "metadata": {} 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "source": [ 111 | "### Read L3 datasets" 112 | ], 113 | "metadata": {} 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": null, 118 | "source": [ 119 | "# independent along-track\r\n", 120 | "alontrack_independent_dataset = f'{url_alongtrack}/dt_gulfstream_c2_phy_l3_20161201-20180131_285-315_23-53.nc'\r\n", 121 | "# Read along-track\r\n", 122 | "ds_alongtrack = read_l3_dataset_from_aviso(alontrack_independent_dataset, \r\n", 123 | " my_aviso_session, \r\n", 124 | " lon_min=lon_min, \r\n", 125 | " lon_max=lon_max, \r\n", 126 | " lat_min=lat_min, \r\n", 127 | " lat_max=lat_max, \r\n", 128 | " time_min=time_min, \r\n", 129 | " time_max=time_max)\r\n", 130 | "ds_alongtrack" 131 | ], 132 | "outputs": [], 133 | "metadata": {} 134 | }, 135 | { 136 | "cell_type": "markdown", 137 | "source": [ 138 | "### Read L4 dataset and interpolate onto along-track positions" 139 | ], 140 | "metadata": {} 141 | }, 142 | { 143 | "cell_type": "code", 144 | "execution_count": null, 145 | "source": [ 146 | "# series of maps to evaluate\r\n", 147 | "gridded_dataset = [f'{url_map}/OSE_ssh_mapping_DYMOST.nc', my_aviso_session]\r\n", 148 | "# Interpolate maps onto alongtrack dataset\r\n", 149 | "time_alongtrack, lat_alongtrack, lon_alongtrack, ssh_alongtrack, ssh_map_interp = interp_on_alongtrack(gridded_dataset, \r\n", 150 | " ds_alongtrack,\r\n", 151 | " lon_min=lon_min, \r\n", 152 | " lon_max=lon_max, \r\n", 153 | " lat_min=lat_min, \r\n", 154 | " lat_max=lat_max, \r\n", 155 | " time_min=time_min, \r\n", 156 | " time_max=time_max,\r\n", 157 | " is_circle=is_circle)" 158 | ], 159 | "outputs": [], 160 | "metadata": {} 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "source": [ 165 | "### Compute statistical score" 166 | ], 167 | "metadata": {} 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": null, 172 | "source": [ 173 | "leaderboard_nrmse, leaderboard_nrmse_std = compute_stats(time_alongtrack, \r\n", 174 | " lat_alongtrack, \r\n", 175 | " lon_alongtrack, \r\n", 176 | " ssh_alongtrack, \r\n", 177 | " ssh_map_interp, \r\n", 178 | " bin_lon_step,\r\n", 179 | " bin_lat_step, \r\n", 180 | " bin_time_step,\r\n", 181 | " output_filename,\r\n", 182 | " output_filename_timeseries)" 183 | ], 184 | "outputs": [], 185 | "metadata": {} 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": null, 190 | "source": [ 191 | "plot_spatial_statistics(output_filename)" 192 | ], 193 | "outputs": [], 194 | "metadata": {} 195 | }, 196 | { 197 | "cell_type": "code", 198 | "execution_count": null, 199 | "source": [ 200 | "plot_temporal_statistics(output_filename_timeseries)" 201 | ], 202 | "outputs": [], 203 | "metadata": {} 204 | }, 205 | { 206 | "cell_type": "markdown", 207 | "source": [ 208 | "### Compute spectral scores" 209 | ], 210 | "metadata": {} 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": null, 215 | "source": [ 216 | "compute_spectral_scores(time_alongtrack, \n", 217 | " lat_alongtrack, \n", 218 | " lon_alongtrack, \n", 219 | " ssh_alongtrack, \n", 220 | " ssh_map_interp, \n", 221 | " lenght_scale,\n", 222 | " delta_x,\n", 223 | " delta_t,\n", 224 | " output_filename_spectrum)" 225 | ], 226 | "outputs": [], 227 | "metadata": {} 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": null, 232 | "source": [ 233 | "leaderboard_psds_score = plot_psd_score(output_filename_spectrum)" 234 | ], 235 | "outputs": [], 236 | "metadata": {} 237 | }, 238 | { 239 | "cell_type": "code", 240 | "execution_count": null, 241 | "source": [], 242 | "outputs": [], 243 | "metadata": {} 244 | }, 245 | { 246 | "cell_type": "code", 247 | "execution_count": null, 248 | "source": [ 249 | "# Print leaderboard\n", 250 | "data = [['DYMOST', \n", 251 | " leaderboard_nrmse, \n", 252 | " leaderboard_nrmse_std, \n", 253 | " int(leaderboard_psds_score),\n", 254 | " 'Dynamic mapping',\n", 255 | " 'example_eval_dymost.ipynb']]\n", 256 | "Leaderboard = pd.DataFrame(data, \n", 257 | " columns=['Method', \n", 258 | " \"µ(RMSE) \", \n", 259 | " \"σ(RMSE)\", \n", 260 | " 'λx (km)', \n", 261 | " 'Notes',\n", 262 | " 'Reference'])\n", 263 | "print(\"Summary of the leaderboard metrics:\")\n", 264 | "Leaderboard\n", 265 | "print(Leaderboard.to_markdown())" 266 | ], 267 | "outputs": [], 268 | "metadata": {} 269 | }, 270 | { 271 | "cell_type": "code", 272 | "execution_count": null, 273 | "source": [], 274 | "outputs": [], 275 | "metadata": {} 276 | }, 277 | { 278 | "cell_type": "code", 279 | "execution_count": null, 280 | "source": [], 281 | "outputs": [], 282 | "metadata": {} 283 | }, 284 | { 285 | "cell_type": "code", 286 | "execution_count": null, 287 | "source": [], 288 | "outputs": [], 289 | "metadata": {} 290 | } 291 | ], 292 | "metadata": { 293 | "kernelspec": { 294 | "display_name": "conda_env", 295 | "language": "python", 296 | "name": "conda_env" 297 | }, 298 | "language_info": { 299 | "codemirror_mode": { 300 | "name": "ipython", 301 | "version": 3 302 | }, 303 | "file_extension": ".py", 304 | "mimetype": "text/x-python", 305 | "name": "python", 306 | "nbconvert_exporter": "python", 307 | "pygments_lexer": "ipython3", 308 | "version": "3.7.8" 309 | } 310 | }, 311 | "nbformat": 4, 312 | "nbformat_minor": 4 313 | } -------------------------------------------------------------------------------- /notebooks/example_eval_miost.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "source": [ 6 | "# Evaluation Multiscale Interpolation (MIOST): \n", 7 | "\n", 8 | "This notebook presents the evaluation of the SSH reconstructions based on the Dynamic Interpolation method ([Ubelmann et al., 2021](https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2020JC016560)) and performed for the **\"2021a_SSH_mapping_OSE\" ocean data challenge**. " 9 | ], 10 | "metadata": {} 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": null, 15 | "source": [ 16 | "import os\r\n", 17 | "import sys\r\n", 18 | "sys.path.append('..')\r\n", 19 | "import logging\r\n", 20 | "import pandas as pd" 21 | ], 22 | "outputs": [], 23 | "metadata": {} 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": null, 28 | "source": [ 29 | "from src.mod_inout import *\r\n", 30 | "from src.mod_interp import *\r\n", 31 | "from src.mod_stats import *\r\n", 32 | "from src.mod_spectral import *\r\n", 33 | "from src.mod_plot import *" 34 | ], 35 | "outputs": [], 36 | "metadata": {} 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "source": [ 42 | "logger = logging.getLogger()\r\n", 43 | "logger.setLevel(logging.INFO)" 44 | ], 45 | "outputs": [], 46 | "metadata": {} 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "source": [ 51 | "### Study Area & Ouput Parameters" 52 | ], 53 | "metadata": {} 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "source": [ 59 | "# study area\r\n", 60 | "lon_min = 295.\r\n", 61 | "lon_max = 305.\r\n", 62 | "lat_min = 33.\r\n", 63 | "lat_max = 43.\r\n", 64 | "is_circle = False\r\n", 65 | "time_min = '2017-01-01'\r\n", 66 | "time_max = '2017-12-31'\r\n", 67 | "\r\n", 68 | "# Outputs\r\n", 69 | "bin_lat_step = 1.\r\n", 70 | "bin_lon_step = 1.\r\n", 71 | "bin_time_step = '1D'\r\n", 72 | "output_directory = '../results'\r\n", 73 | "if not os.path.exists(output_directory):\r\n", 74 | " os.mkdir(output_directory)\r\n", 75 | "output_filename = f'{output_directory}/stat_OSE_MIOST_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\r\n", 76 | "output_filename_timeseries = f'{output_directory}/stat_timeseries_OSE_MIOST_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'\r\n", 77 | "\r\n", 78 | "# Spectral parameter\r\n", 79 | "# C2 parameter\r\n", 80 | "delta_t = 0.9434 # s\r\n", 81 | "velocity = 6.77 # km/s\r\n", 82 | "delta_x = velocity * delta_t\r\n", 83 | "lenght_scale = 1000 # km\r\n", 84 | "output_filename_spectrum = f'{output_directory}/psd_OSE_MIOST_{time_min}_{time_max}_{lon_min}_{lon_max}_{lat_min}_{lat_max}.nc'" 85 | ], 86 | "outputs": [], 87 | "metadata": {} 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "source": [ 92 | "### Open your AVISO+ session: fill the `````` and `````` items below" 93 | ], 94 | "metadata": {} 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": null, 99 | "source": [ 100 | "my_aviso_session = rq.Session()\r\n", 101 | "my_aviso_session.auth = (\"\", \"\")\r\n", 102 | "url_alongtrack = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-along-track-data'\r\n", 103 | "url_map = 'https://tds.aviso.altimetry.fr/thredds/dodsC/2021a-SSH-mapping-OSE-grid-data'" 104 | ], 105 | "outputs": [], 106 | "metadata": {} 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "source": [ 111 | "### Read L3 datasets" 112 | ], 113 | "metadata": {} 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": null, 118 | "source": [ 119 | "# independent along-track\r\n", 120 | "alontrack_independent_dataset = f'{url_alongtrack}/dt_gulfstream_c2_phy_l3_20161201-20180131_285-315_23-53.nc'\r\n", 121 | "# Read along-track\r\n", 122 | "ds_alongtrack = read_l3_dataset_from_aviso(alontrack_independent_dataset, \r\n", 123 | " my_aviso_session,\r\n", 124 | " lon_min=lon_min, \r\n", 125 | " lon_max=lon_max, \r\n", 126 | " lat_min=lat_min, \r\n", 127 | " lat_max=lat_max, \r\n", 128 | " time_min=time_min, \r\n", 129 | " time_max=time_max)\r\n", 130 | "ds_alongtrack" 131 | ], 132 | "outputs": [], 133 | "metadata": {} 134 | }, 135 | { 136 | "cell_type": "markdown", 137 | "source": [ 138 | "### Read L4 dataset and interpolate onto along-track positions" 139 | ], 140 | "metadata": {} 141 | }, 142 | { 143 | "cell_type": "code", 144 | "execution_count": null, 145 | "source": [ 146 | "# series of maps to evaluate\r\n", 147 | "gridded_dataset = [f'{url_map}/OSE_ssh_mapping_MIOST.nc', my_aviso_session] \r\n", 148 | "# Interpolate maps onto alongtrack dataset\r\n", 149 | "time_alongtrack, lat_alongtrack, lon_alongtrack, ssh_alongtrack, ssh_map_interp = interp_on_alongtrack(gridded_dataset, \r\n", 150 | " ds_alongtrack,\r\n", 151 | " lon_min=lon_min, \r\n", 152 | " lon_max=lon_max, \r\n", 153 | " lat_min=lat_min, \r\n", 154 | " lat_max=lat_max, \r\n", 155 | " time_min=time_min, \r\n", 156 | " time_max=time_max,\r\n", 157 | " is_circle=is_circle)" 158 | ], 159 | "outputs": [], 160 | "metadata": {} 161 | }, 162 | { 163 | "cell_type": "markdown", 164 | "source": [ 165 | "### Compute statistical score" 166 | ], 167 | "metadata": {} 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": null, 172 | "source": [ 173 | "leaderboard_nrmse, leaderboard_nrmse_std = compute_stats(time_alongtrack, \r\n", 174 | " lat_alongtrack, \r\n", 175 | " lon_alongtrack, \r\n", 176 | " ssh_alongtrack, \r\n", 177 | " ssh_map_interp, \r\n", 178 | " bin_lon_step,\r\n", 179 | " bin_lat_step, \r\n", 180 | " bin_time_step,\r\n", 181 | " output_filename,\r\n", 182 | " output_filename_timeseries)" 183 | ], 184 | "outputs": [], 185 | "metadata": {} 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": null, 190 | "source": [ 191 | "plot_spatial_statistics(output_filename)" 192 | ], 193 | "outputs": [], 194 | "metadata": {} 195 | }, 196 | { 197 | "cell_type": "code", 198 | "execution_count": null, 199 | "source": [ 200 | "plot_temporal_statistics(output_filename_timeseries)" 201 | ], 202 | "outputs": [], 203 | "metadata": {} 204 | }, 205 | { 206 | "cell_type": "markdown", 207 | "source": [ 208 | "### Compute spectral scores" 209 | ], 210 | "metadata": {} 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": null, 215 | "source": [ 216 | "compute_spectral_scores(time_alongtrack, \r\n", 217 | " lat_alongtrack, \r\n", 218 | " lon_alongtrack, \r\n", 219 | " ssh_alongtrack, \r\n", 220 | " ssh_map_interp, \r\n", 221 | " lenght_scale,\r\n", 222 | " delta_x,\r\n", 223 | " delta_t,\r\n", 224 | " output_filename_spectrum)" 225 | ], 226 | "outputs": [], 227 | "metadata": {} 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": null, 232 | "source": [ 233 | "leaderboard_psds_score = plot_psd_score(output_filename_spectrum)" 234 | ], 235 | "outputs": [], 236 | "metadata": {} 237 | }, 238 | { 239 | "cell_type": "code", 240 | "execution_count": null, 241 | "source": [], 242 | "outputs": [], 243 | "metadata": {} 244 | }, 245 | { 246 | "cell_type": "code", 247 | "execution_count": null, 248 | "source": [ 249 | "# Print leaderboard\r\n", 250 | "data = [['MIOST', \r\n", 251 | " leaderboard_nrmse, \r\n", 252 | " leaderboard_nrmse_std, \r\n", 253 | " int(leaderboard_psds_score),\r\n", 254 | " 'Multiscale mapping',\r\n", 255 | " 'example_eval_miost.ipynb']]\r\n", 256 | "Leaderboard = pd.DataFrame(data, \r\n", 257 | " columns=['Method', \r\n", 258 | " \"µ(RMSE) \", \r\n", 259 | " \"σ(RMSE)\", \r\n", 260 | " 'λx (km)', \r\n", 261 | " 'Notes',\r\n", 262 | " 'Reference'])\r\n", 263 | "print(\"Summary of the leaderboard metrics:\")\r\n", 264 | "Leaderboard\r\n", 265 | "print(Leaderboard.to_markdown())" 266 | ], 267 | "outputs": [], 268 | "metadata": {} 269 | }, 270 | { 271 | "cell_type": "code", 272 | "execution_count": null, 273 | "source": [], 274 | "outputs": [], 275 | "metadata": {} 276 | }, 277 | { 278 | "cell_type": "code", 279 | "execution_count": null, 280 | "source": [], 281 | "outputs": [], 282 | "metadata": {} 283 | }, 284 | { 285 | "cell_type": "code", 286 | "execution_count": null, 287 | "source": [], 288 | "outputs": [], 289 | "metadata": {} 290 | } 291 | ], 292 | "metadata": { 293 | "kernelspec": { 294 | "display_name": "conda_env", 295 | "language": "python", 296 | "name": "conda_env" 297 | }, 298 | "language_info": { 299 | "codemirror_mode": { 300 | "name": "ipython", 301 | "version": 3 302 | }, 303 | "file_extension": ".py", 304 | "mimetype": "text/x-python", 305 | "name": "python", 306 | "nbconvert_exporter": "python", 307 | "pygments_lexer": "ipython3", 308 | "version": "3.7.8" 309 | } 310 | }, 311 | "nbformat": 4, 312 | "nbformat_minor": 4 313 | } -------------------------------------------------------------------------------- /notebooks/example_intercomparison.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Intercomparison of various altimetry-based SSH reconstruction\n", 8 | "This notebook aims at visualizing simustaneously the key spactral and statistical metrics computed for various SSH mapping reconstructions" 9 | ] 10 | }, 11 | { 12 | "cell_type": "code", 13 | "execution_count": null, 14 | "metadata": {}, 15 | "outputs": [], 16 | "source": [ 17 | "import sys\n", 18 | "sys.path.append('..')\n", 19 | "import hvplot.xarray" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": null, 25 | "metadata": {}, 26 | "outputs": [], 27 | "source": [ 28 | "from src.mod_plot import *" 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": {}, 34 | "source": [ 35 | "### Power spectral densities & Power Spectral Density Score" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "metadata": {}, 42 | "outputs": [], 43 | "source": [ 44 | "list_of_filename = ['../results/psd_OSE_BASELINE_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 45 | " '../results/psd_OSE_BFN_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 46 | " '../results/psd_OSE_DUACS_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 47 | " '../results/psd_OSE_DYMOST_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 48 | " '../results/psd_OSE_MIOST_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 49 | " '../results/psd_OSE_4DVARNET_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 50 | " '../results/psd_OSE_4DVARNET_v2022_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc'\n", 51 | " ]\n", 52 | "list_of_label = ['BASELINE', \n", 53 | " 'BFN',\n", 54 | " 'DUACS',\n", 55 | " 'DYMOST',\n", 56 | " 'MIOST',\n", 57 | " '4DVARNET (v2021)',\n", 58 | " '4DVARNET (v2022)'\n", 59 | " ]" 60 | ] 61 | }, 62 | { 63 | "cell_type": "code", 64 | "execution_count": null, 65 | "metadata": {}, 66 | "outputs": [], 67 | "source": [ 68 | "spectral_score_intercomparison(list_of_filename, list_of_label)" 69 | ] 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "metadata": {}, 74 | "source": [ 75 | "### Timeseries of daily averaged RMSE score\n", 76 | "This figure show the temporal statibility of the SSH reconstructions" 77 | ] 78 | }, 79 | { 80 | "cell_type": "code", 81 | "execution_count": null, 82 | "metadata": {}, 83 | "outputs": [], 84 | "source": [ 85 | "list_of_filename = ['../results/stat_timeseries_OSE_BASELINE_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 86 | " '../results/stat_timeseries_OSE_BFN_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 87 | " '../results/stat_timeseries_OSE_DUACS_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 88 | " '../results/stat_timeseries_OSE_DYMOST_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 89 | " '../results/stat_timeseries_OSE_MIOST_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 90 | " '../results/stat_timeseries_OSE_4DVARNET_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 91 | " '../results/stat_timeseries_OSE_4DVARNET_v2022_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc'\n", 92 | " ]" 93 | ] 94 | }, 95 | { 96 | "cell_type": "code", 97 | "execution_count": null, 98 | "metadata": {}, 99 | "outputs": [], 100 | "source": [ 101 | "intercomparison_temporal_statistics(list_of_filename, list_of_label)" 102 | ] 103 | }, 104 | { 105 | "cell_type": "markdown", 106 | "metadata": {}, 107 | "source": [ 108 | "### Gain / loss of RMSE (expressed in %) relativelive to the baseline OI experiment\n", 109 | "In the figure below, blue means a reduction of the RMSE in the experiments relatively to the BASELINE OI method" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": null, 115 | "metadata": {}, 116 | "outputs": [], 117 | "source": [ 118 | "list_of_filename = ['../results/stat_OSE_BASELINE_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 119 | " '../results/stat_OSE_BFN_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 120 | " '../results/stat_OSE_DUACS_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 121 | " '../results/stat_OSE_DYMOST_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 122 | " '../results/stat_OSE_MIOST_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 123 | " '../results/stat_OSE_4DVARNET_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc',\n", 124 | " '../results/stat_OSE_4DVARNET_v2022_2017-01-01_2017-12-31_295.0_305.0_33.0_43.0.nc'\n", 125 | " ]\n", 126 | "list_of_label = ['BASELINE', \n", 127 | " 'BFN',\n", 128 | " 'DUACS',\n", 129 | " 'DYMOST',\n", 130 | " 'MIOST',\n", 131 | " '4DVARNET (v2021)',\n", 132 | " '4DVARNET (v2022)'\n", 133 | " ]" 134 | ] 135 | }, 136 | { 137 | "cell_type": "code", 138 | "execution_count": null, 139 | "metadata": {}, 140 | "outputs": [], 141 | "source": [ 142 | "intercomparison_spatial_statistics(list_of_filename[0], list_of_filename[1:], list_of_label[1:])" 143 | ] 144 | }, 145 | { 146 | "cell_type": "code", 147 | "execution_count": null, 148 | "metadata": {}, 149 | "outputs": [], 150 | "source": [] 151 | }, 152 | { 153 | "cell_type": "code", 154 | "execution_count": null, 155 | "metadata": {}, 156 | "outputs": [], 157 | "source": [] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "metadata": {}, 163 | "outputs": [], 164 | "source": [] 165 | }, 166 | { 167 | "cell_type": "code", 168 | "execution_count": null, 169 | "metadata": {}, 170 | "outputs": [], 171 | "source": [] 172 | } 173 | ], 174 | "metadata": { 175 | "kernelspec": { 176 | "display_name": "cmems_env", 177 | "language": "python", 178 | "name": "cmems_env" 179 | }, 180 | "language_info": { 181 | "codemirror_mode": { 182 | "name": "ipython", 183 | "version": 3 184 | }, 185 | "file_extension": ".py", 186 | "mimetype": "text/x-python", 187 | "name": "python", 188 | "nbconvert_exporter": "python", 189 | "pygments_lexer": "ipython3", 190 | "version": "3.8.12" 191 | } 192 | }, 193 | "nbformat": 4, 194 | "nbformat_minor": 4 195 | } 196 | -------------------------------------------------------------------------------- /src/.ipynb_checkpoints/mod_inout-checkpoint.py: -------------------------------------------------------------------------------- 1 | import xarray as xr 2 | import numpy as np 3 | #import pandas as pd 4 | import pyinterp 5 | import logging 6 | 7 | def read_l3_dataset(file, 8 | lon_min=0., 9 | lon_max=360., 10 | lat_min=-90, 11 | lat_max=90., 12 | time_min='1900-10-01', 13 | time_max='2100-01-01'): 14 | 15 | ds = xr.open_dataset(file) 16 | ds = ds.sel(time=slice(time_min, time_max), drop=True) 17 | ds = ds.where((ds["latitude"] >= lat_min) & (ds["latitude"] <= lat_max), drop=True) 18 | ds = ds.where((ds["longitude"] >= lon_min%360.) & (ds["longitude"] <= lon_max%360.), drop=True) 19 | 20 | return ds 21 | 22 | 23 | def read_l4_dataset(list_of_file, 24 | lon_min=0., 25 | lon_max=360., 26 | lat_min=-90, 27 | lat_max=90., 28 | time_min='1900-10-01', 29 | time_max='2100-01-01', 30 | is_circle=True): 31 | 32 | 33 | ds = xr.open_mfdataset(list_of_file, concat_dim ='time', combine='nested', parallel=True) 34 | ds = ds.sel(time=slice(time_min, time_max), drop=True) 35 | ds = ds.where((ds["lon"]%360. >= lon_min) & (ds["lon"]%360. <= lon_max), drop=True) 36 | ds = ds.where((ds["lat"] >= lat_min) & (ds["lat"] <= lat_max), drop=True) 37 | 38 | x_axis = pyinterp.Axis(ds["lon"][:]%360., is_circle=is_circle) 39 | y_axis = pyinterp.Axis(ds["lat"][:]) 40 | z_axis = pyinterp.TemporalAxis(ds["time"][:]) 41 | 42 | var = ds['ssh'][:] 43 | var = var.transpose('lon', 'lat', 'time') 44 | 45 | # The undefined values must be set to nan. 46 | try: 47 | var[var.mask] = float("nan") 48 | except AttributeError: 49 | pass 50 | 51 | grid = pyinterp.Grid3D(x_axis, y_axis, z_axis, var.data) 52 | 53 | del ds 54 | 55 | return x_axis, y_axis, z_axis, grid 56 | 57 | 58 | def read_obs(input_file, oi_grid, oi_param, coarsening): 59 | 60 | logging.info(' Reading observations...') 61 | 62 | def preprocess(ds): 63 | return ds.coarsen(coarsening, boundary="trim").mean() 64 | 65 | ds_obs = xr.open_mfdataset(input_file, combine='nested', concat_dim='time', parallel=True, preprocess=preprocess) #.sortby('time') 66 | #ds_obs = ds_obs.coarsen(coarsening, boundary="trim").mean().sortby('time') 67 | ds_obs = ds_obs.sortby('time') 68 | 69 | lon_min = oi_grid.lon.min().values 70 | lon_max = oi_grid.lon.max().values 71 | lat_min = oi_grid.lat.min().values 72 | lat_max = oi_grid.lat.max().values 73 | time_min = oi_grid.time.min().values 74 | time_max = oi_grid.time.max().values 75 | 76 | ds_obs = ds_obs.sel(time=slice(time_min - np.timedelta64(int(2*oi_param.Lt.values), 'D'), 77 | time_max + np.timedelta64(int(2*oi_param.Lt.values), 'D')), drop=True) 78 | 79 | # correct lon if domain is between [-180:180] 80 | if lon_min < 0: 81 | ds_obs['lon'] = xr.where(ds_obs['longitude'] >= 180., ds_obs['longitude']-360., ds_obs['longitude']) 82 | 83 | ds_obs = ds_obs.where((ds_obs['longitude'] >= lon_min - oi_param.Lx.values) & 84 | (ds_obs['longitude'] <= lon_max + oi_param.Lx.values) & 85 | (ds_obs['latitude'] >= lat_min - oi_param.Ly.values) & 86 | (ds_obs['latitude'] <= lat_max + oi_param.Ly.values) , drop=True) 87 | 88 | vtime = (ds_obs['time'].values - time_min) / np.timedelta64(1, 'D') 89 | ds_obs = ds_obs.assign_coords({'time': vtime}) 90 | 91 | ds_obs = ds_obs.dropna(dim='time') 92 | 93 | return ds_obs 94 | 95 | 96 | def reformate_oi_output(ds_oi, mdt_file): 97 | ds_oi = ds_oi.drop(['gtime', 'ng', 'glon2', 'glat2', 'fglon', 'fglat', 'nobs']) 98 | ds_oi = ds_oi.rename({'gssh': 'sla'}) 99 | 100 | mdt = xr.open_dataset(mdt_file) 101 | mdt_interp = mdt.interp(lon=ds_oi.lon, lat=ds_oi.lat) 102 | 103 | ds_oi['ssh'] = ds_oi['sla'] + mdt_interp['mdt'] 104 | 105 | return ds_oi -------------------------------------------------------------------------------- /src/.ipynb_checkpoints/mod_interp-checkpoint.py: -------------------------------------------------------------------------------- 1 | import xarray as xr 2 | import numpy as np 3 | import pyinterp 4 | import netCDF4 5 | 6 | #import sys 7 | #sys.path.append('.') 8 | from src.mod_inout import * 9 | 10 | 11 | def interp_on_alongtrack(gridded_dataset, 12 | ds_alongtrack, 13 | lon_min=0., 14 | lon_max=360., 15 | lat_min=-90, 16 | lat_max=90., 17 | time_min='1900-10-01', 18 | time_max='2100-01-01', 19 | is_circle=True): 20 | 21 | # Interpolate maps onto alongtrack dataset 22 | x_axis, y_axis, z_axis, grid = read_l4_dataset(gridded_dataset, 23 | lon_min=lon_min, 24 | lon_max=lon_max, 25 | lat_min=lat_min, 26 | lat_max=lat_max, 27 | time_min=time_min, 28 | time_max=time_max, 29 | is_circle=is_circle) 30 | 31 | ssh_map_interp = pyinterp.trivariate(grid, 32 | ds_alongtrack["longitude"].values, 33 | ds_alongtrack["latitude"].values, 34 | z_axis.safe_cast(ds_alongtrack.time.values), 35 | bounds_error=False).reshape(ds_alongtrack["longitude"].values.shape) 36 | 37 | ssh_alongtrack = (ds_alongtrack["sla_unfiltered"] + ds_alongtrack["mdt"] - ds_alongtrack["lwe"]).values 38 | lon_alongtrack = ds_alongtrack["longitude"].values 39 | lat_alongtrack = ds_alongtrack["latitude"].values 40 | time_alongtrack = ds_alongtrack["time"].values 41 | 42 | # get and apply mask from map_interp & alongtrack on each dataset 43 | msk1 = np.ma.masked_invalid(ssh_alongtrack).mask 44 | msk2 = np.ma.masked_invalid(ssh_map_interp).mask 45 | msk = msk1 + msk2 46 | 47 | ssh_alongtrack = np.ma.masked_where(msk, ssh_alongtrack).compressed() 48 | lon_alongtrack = np.ma.masked_where(msk, lon_alongtrack).compressed() 49 | lat_alongtrack = np.ma.masked_where(msk, lat_alongtrack).compressed() 50 | time_alongtrack = np.ma.masked_where(msk, time_alongtrack).compressed() 51 | ssh_map_interp = np.ma.masked_where(msk, ssh_map_interp).compressed() 52 | 53 | # select inside value (this is done to insure similar number of point in statistical comparison between methods) 54 | indices = np.where((lon_alongtrack >= lon_min+0.25) & (lon_alongtrack <= lon_max-0.25) & 55 | (lat_alongtrack >= lat_min+0.25) & (lat_alongtrack <= lat_max-0.25))[0] 56 | 57 | return time_alongtrack[indices], lat_alongtrack[indices], lon_alongtrack[indices], ssh_alongtrack[indices], ssh_map_interp[indices] 58 | -------------------------------------------------------------------------------- /src/.ipynb_checkpoints/mod_oi-checkpoint.py: -------------------------------------------------------------------------------- 1 | import xarray as xr 2 | import numpy 3 | import logging 4 | 5 | 6 | def oi_grid(glon, glat, gtime): 7 | """ 8 | 9 | """ 10 | 11 | logging.info(' Set OI grid...') 12 | 13 | nx = len(glon) 14 | ny = len(glat) 15 | nt = len(gtime) 16 | 17 | # define & initialize ssh array 18 | gssh = numpy.empty((nt, ny, nx)) 19 | nobs = numpy.empty(nt) 20 | 21 | # Make 2D grid 22 | glon2, glat2 = numpy.meshgrid(glon, glat) 23 | fglon = glon2.flatten() 24 | fglat = glat2.flatten() 25 | 26 | ng = len(fglat) # number of grid points 27 | vtime = (gtime - gtime[0]) / numpy.timedelta64(1, 'D') 28 | 29 | 30 | ds_oi_grid = xr.Dataset({'gssh' : (('time', 'lat', 'lon'), gssh), 31 | 'glon2' : (('lat', 'lon'), glon2), 32 | 'glat2' : (('lat', 'lon'), glat2), 33 | 'fglon' : (('ng'), fglon), 34 | 'fglat' : (('ng'), fglat), 35 | 'nobs' : (('time'), nobs)}, 36 | coords={'gtime': (vtime).astype(numpy.float), 37 | 'time': gtime, 38 | 'lat': glat, 39 | 'lon': glon, 40 | 'ng': numpy.arange(ng)}) 41 | 42 | return ds_oi_grid 43 | 44 | 45 | def oi_param(Lx, Ly, Lt, noise): 46 | 47 | logging.info(' Set OI params...') 48 | 49 | ds_oi_param = xr.Dataset({'Lx' : Lx, 50 | 'Ly' : Ly, 51 | 'Lt' : Lt, 52 | 'noise' : noise}) 53 | 54 | return ds_oi_param 55 | 56 | 57 | def oi_core(it, ds_oi_grid, ds_oi_param, ds_obs): 58 | 59 | ind1 = numpy.where((numpy.abs(ds_obs.time.values - ds_oi_grid.gtime.values[it]) < 2.*ds_oi_param.Lt.values))[0] 60 | nobs = len(ind1) 61 | print('Processing time-step : ', it, '/', len(ds_oi_grid.gtime.values) - 1, ' nobs = ', nobs, end="\r") 62 | 63 | BHt = numpy.empty((len(ds_oi_grid.ng), nobs)) 64 | HBHt = numpy.empty((nobs, nobs)) 65 | 66 | obs_lon = ds_obs.longitude.values[ind1] 67 | obs_lat = ds_obs.latitude.values[ind1] 68 | obs_time = ds_obs.time.values[ind1] 69 | 70 | fglon = ds_oi_grid.fglon.values 71 | fglat = ds_oi_grid.fglat.values 72 | ftime = ds_oi_grid.gtime.values[it] 73 | 74 | for iobs in range(nobs): 75 | # print(iobs) 76 | 77 | BHt[:,iobs] = numpy.exp(-((ftime - obs_time[iobs])/ds_oi_param.Lt.values)**2 - 78 | ((fglon - obs_lon[iobs])/ds_oi_param.Lx.values)**2 - 79 | ((fglat - obs_lat[iobs])/ds_oi_param.Ly.values)**2) 80 | 81 | HBHt[:,iobs] = numpy.exp(-((obs_time - obs_time[iobs])/ds_oi_param.Lt.values)**2 - 82 | ((obs_lon - obs_lon[iobs])/ds_oi_param.Lx.values)**2 - 83 | ((obs_lat - obs_lat[iobs])/ds_oi_param.Ly.values)**2) 84 | 85 | del obs_lon, obs_lat, obs_time 86 | 87 | R = numpy.diag(numpy.full((nobs), ds_oi_param.noise.values**2)) 88 | 89 | Coo = HBHt + R 90 | Mi = numpy.linalg.inv(Coo) 91 | 92 | sol = numpy.dot(numpy.dot(BHt, Mi), ds_obs.sla_unfiltered.values[ind1]) 93 | 94 | ds_oi_grid.gssh[it, :, :] = sol.reshape(ds_oi_grid.lat.size, ds_oi_grid.lon.size) 95 | ds_oi_grid.nobs[it] = nobs 96 | 97 | 98 | #return None # sol.reshape(ds_oi_grid.glat.size, ds_oi_grid.glon.size), nobs 99 | 100 | -------------------------------------------------------------------------------- /src/.ipynb_checkpoints/mod_plot-checkpoint.py: -------------------------------------------------------------------------------- 1 | import xarray as xr 2 | import numpy as np 3 | import logging 4 | import matplotlib.pylab as plt 5 | from scipy import interpolate 6 | import hvplot.xarray 7 | import cartopy.crs as ccrs 8 | 9 | 10 | def find_wavelength_05_crossing(filename): 11 | 12 | ds = xr.open_dataset(filename) 13 | y = 1./ds.wavenumber 14 | x = (1. - ds.psd_diff/ds.psd_ref) 15 | f = interpolate.interp1d(x, y) 16 | 17 | xnew = 0.5 18 | ynew = f(xnew) 19 | 20 | return ynew 21 | 22 | 23 | 24 | def plot_psd_score(filename): 25 | 26 | ds = xr.open_dataset(filename) 27 | 28 | resolved_scale = find_wavelength_05_crossing(filename) 29 | 30 | plt.figure(figsize=(10, 5)) 31 | ax = plt.subplot(121) 32 | ax.invert_xaxis() 33 | plt.plot((1./ds.wavenumber), ds.psd_ref, label='reference', color='k') 34 | plt.plot((1./ds.wavenumber), ds.psd_study, label='reconstruction', color='lime') 35 | plt.xlabel('wavelength [km]') 36 | plt.ylabel('Power Spectral Density [m$^{2}$/cy/km]') 37 | plt.xscale('log') 38 | plt.yscale('log') 39 | plt.legend(loc='best') 40 | plt.grid(which='both') 41 | 42 | ax = plt.subplot(122) 43 | ax.invert_xaxis() 44 | plt.plot((1./ds.wavenumber), (1. - ds.psd_diff/ds.psd_ref), color='k', lw=2) 45 | plt.xlabel('wavelength [km]') 46 | plt.ylabel('PSD Score [1. - PSD$_{err}$/PSD$_{ref}$]') 47 | plt.xscale('log') 48 | plt.hlines(y=0.5, 49 | xmin=np.ma.min(np.ma.masked_invalid(1./ds.wavenumber)), 50 | xmax=np.ma.max(np.ma.masked_invalid(1./ds.wavenumber)), 51 | color='r', 52 | lw=0.5, 53 | ls='--') 54 | plt.vlines(x=resolved_scale, ymin=0, ymax=1, lw=0.5, color='g') 55 | ax.fill_betweenx((1. - ds.psd_diff/ds.psd_ref), 56 | resolved_scale, 57 | np.ma.max(np.ma.masked_invalid(1./ds.wavenumber)), 58 | color='green', 59 | alpha=0.3, 60 | label=f'resolved scales \n $\lambda$ > {int(resolved_scale)}km') 61 | plt.legend(loc='best') 62 | plt.grid(which='both') 63 | 64 | logging.info(' ') 65 | logging.info(f' Minimum spatial scale resolved = {int(resolved_scale)}km') 66 | 67 | plt.show() 68 | 69 | return resolved_scale 70 | 71 | 72 | def plot_spatial_statistics(filename): 73 | 74 | ds = xr.open_dataset(filename, group='diff') 75 | 76 | figure = ds['rmse'].hvplot.image(x='lon', y='lat', z='rmse', clabel='RMSE [m]', cmap='Reds', coastline=True) 77 | 78 | return figure 79 | 80 | 81 | def plot_temporal_statistics(filename): 82 | 83 | ds1 = xr.open_dataset(filename, group='diff') 84 | ds2 = xr.open_dataset(filename, group='alongtrack') 85 | rmse_score = 1. - ds1['rms']/ds2['rms'] 86 | 87 | rmse_score = rmse_score.dropna(dim='time').where(ds1['count'] > 10, drop=True) 88 | 89 | figure = rmse_score.hvplot.line(ylabel='RMSE SCORE', shared_axes=True, color='r') + ds1['count'].dropna(dim='time').hvplot.step(ylabel='#Obs.', shared_axes=True, color='grey') 90 | 91 | return figure.cols(1) 92 | 93 | 94 | def spectral_score_intercomparison(list_of_filename, list_of_label): 95 | 96 | plt.figure(figsize=(15, 6)) 97 | ax = plt.subplot(121) 98 | ax.invert_xaxis() 99 | ds = xr.open_dataset(list_of_filename[0]) 100 | plt.plot((1./ds.wavenumber), ds.psd_ref, label='reference', color='k') 101 | for cfilename, clabel in zip(list_of_filename, list_of_label): 102 | ds = xr.open_dataset(cfilename) 103 | plt.plot((1./ds.wavenumber), ds.psd_study, label=clabel) 104 | plt.xlabel('wavelength [km]') 105 | plt.ylabel('Power Spectral Density [m$^{2}$/cy/km]') 106 | plt.xscale('log') 107 | plt.yscale('log') 108 | plt.legend(loc='best') 109 | plt.grid(which='both') 110 | plt.xticks([50, 100, 200, 500, 1000], ["50km", "100km", "200km", "500km", "1000km"]) 111 | 112 | ax = plt.subplot(122) 113 | ax.invert_xaxis() 114 | for cfilename, clabel in zip(list_of_filename, list_of_label): 115 | ds = xr.open_dataset(cfilename) 116 | plt.plot((1./ds.wavenumber), (1. - ds.psd_diff/ds.psd_ref), lw=2, label=clabel) 117 | plt.xlabel('wavelength [km]') 118 | plt.ylabel('PSD Score [1. - PSD$_{err}$/PSD$_{ref}$]') 119 | plt.xscale('log') 120 | plt.hlines(y=0.5, 121 | xmin=np.ma.min(np.ma.masked_invalid(1./ds.wavenumber)), 122 | xmax=np.ma.max(np.ma.masked_invalid(1./ds.wavenumber)), 123 | color='r', 124 | lw=0.5, 125 | ls='--') 126 | # plt.vlines(x=resolved_scale, ymin=0, ymax=1, lw=0.5, color='g') 127 | # ax.fill_betweenx((1. - ds.psd_diff/ds.psd_ref), 128 | # resolved_scale, 129 | # np.ma.max(np.ma.masked_invalid(1./ds.wavenumber)), 130 | # color='green', 131 | # alpha=0.3, 132 | # label=f'resolved scales \n $\lambda$ > {int(resolved_scale)}km') 133 | plt.legend(loc='best') 134 | plt.grid(which='both') 135 | plt.xticks([50, 100, 200, 500, 1000], ["50km", "100km", "200km", "500km", "1000km"]) 136 | 137 | plt.show() 138 | 139 | 140 | def intercomparison_temporal_statistics(list_of_filename, list_of_label): 141 | 142 | ds_diff = xr.concat([xr.open_dataset(filename, group='diff') for filename in list_of_filename], dim='experiment') 143 | ds_diff['experiment'] = list_of_label 144 | ds_alongtrack = xr.concat([xr.open_dataset(filename, group='alongtrack') for filename in list_of_filename], dim='experiment') 145 | ds_alongtrack['experiment'] = list_of_label 146 | 147 | rmse_score = 1. - ds_diff['rms']/ds_alongtrack['rms'] 148 | rmse_score = rmse_score.dropna(dim='time').where(ds_diff['count'] > 10, drop=True) 149 | 150 | figure = rmse_score.hvplot.line(x='time', y='rms', by='experiment', ylim=(0, 1), title='RMSE SCORE', shared_axes=True) + ds_diff['count'][0, :].dropna(dim='time').hvplot.step(ylabel='#Obs.', shared_axes=True, color='grey') 151 | 152 | return figure.cols(1) 153 | 154 | 155 | def intercomparison_spatial_statistics(baseline_filename, list_of_filename, list_of_label): 156 | 157 | ds_baseline = xr.open_dataset(baseline_filename, group='diff') 158 | ds = xr.concat([xr.open_dataset(filename, group='diff') for filename in list_of_filename], dim='experiment') 159 | ds['experiment'] = list_of_label 160 | 161 | delta_rmse = 100*(ds - ds_baseline)/ds_baseline 162 | 163 | figure = delta_rmse['rmse'].hvplot.image(x='lon', y='lat', z='rmse', clim=(-20, 20), by='experiment', 164 | subplots=True, projection=ccrs.PlateCarree(), 165 | clabel='[%]', cmap='coolwarm', coastline=True) 166 | 167 | return figure.cols(2) -------------------------------------------------------------------------------- /src/.ipynb_checkpoints/mod_spectral-checkpoint.py: -------------------------------------------------------------------------------- 1 | 2 | import numpy as np 3 | import scipy.signal 4 | import logging 5 | 6 | from src.mod_write import * 7 | 8 | 9 | def compute_segment_alongtrack(time_alongtrack, 10 | lat_alongtrack, 11 | lon_alongtrack, 12 | ssh_alongtrack, 13 | ssh_map_interp, 14 | lenght_scale, 15 | delta_x, 16 | delta_t): 17 | 18 | segment_overlapping = 0.25 19 | max_delta_t_gap = 4 * np.timedelta64(1, 's') # max delta t of 4 seconds to cut tracks 20 | 21 | list_lat_segment = [] 22 | list_lon_segment = [] 23 | list_ssh_alongtrack_segment = [] 24 | list_ssh_map_interp_segment = [] 25 | 26 | # Get number of point to consider for resolution = lenghtscale in km 27 | delta_t_jd = delta_t / (3600 * 24) 28 | npt = int(lenght_scale / delta_x) 29 | 30 | # cut track when diff time longer than 4*delta_t 31 | indi = np.where((np.diff(time_alongtrack) > max_delta_t_gap))[0] 32 | track_segment_lenght = np.insert(np.diff(indi), [0], indi[0]) 33 | 34 | # Long track >= npt 35 | selected_track_segment = np.where(track_segment_lenght >= npt)[0] 36 | 37 | if selected_track_segment.size > 0: 38 | 39 | for track in selected_track_segment: 40 | 41 | if track-1 >= 0: 42 | index_start_selected_track = indi[track-1] 43 | index_end_selected_track = indi[track] 44 | else: 45 | index_start_selected_track = 0 46 | index_end_selected_track = indi[track] 47 | 48 | start_point = index_start_selected_track 49 | end_point = index_end_selected_track 50 | 51 | for sub_segment_point in range(start_point, end_point - npt, int(npt*segment_overlapping)): 52 | 53 | # Near Greenwhich case 54 | if ((lon_alongtrack[sub_segment_point + npt - 1] < 50.) 55 | and (lon_alongtrack[sub_segment_point] > 320.)) \ 56 | or ((lon_alongtrack[sub_segment_point + npt - 1] > 320.) 57 | and (lon_alongtrack[sub_segment_point] < 50.)): 58 | 59 | tmp_lon = np.where(lon_alongtrack[sub_segment_point:sub_segment_point + npt] > 180, 60 | lon_alongtrack[sub_segment_point:sub_segment_point + npt] - 360, 61 | lon_alongtrack[sub_segment_point:sub_segment_point + npt]) 62 | mean_lon_sub_segment = np.median(tmp_lon) 63 | 64 | if mean_lon_sub_segment < 0: 65 | mean_lon_sub_segment = mean_lon_sub_segment + 360. 66 | else: 67 | 68 | mean_lon_sub_segment = np.median(lon_alongtrack[sub_segment_point:sub_segment_point + npt]) 69 | 70 | mean_lat_sub_segment = np.median(lat_alongtrack[sub_segment_point:sub_segment_point + npt]) 71 | 72 | ssh_alongtrack_segment = np.ma.masked_invalid(ssh_alongtrack[sub_segment_point:sub_segment_point + npt]) 73 | 74 | ssh_map_interp_segment = [] 75 | ssh_map_interp_segment = np.ma.masked_invalid(ssh_map_interp[sub_segment_point:sub_segment_point + npt]) 76 | if np.ma.is_masked(ssh_map_interp_segment): 77 | ssh_alongtrack_segment = np.ma.compressed(np.ma.masked_where(np.ma.is_masked(ssh_map_interp_segment), ssh_alongtrack_segment)) 78 | ssh_map_interp_segment = np.ma.compressed(ssh_map_interp_segment) 79 | 80 | if ssh_alongtrack_segment.size > 0: 81 | list_ssh_alongtrack_segment.append(ssh_alongtrack_segment) 82 | list_lon_segment.append(mean_lon_sub_segment) 83 | list_lat_segment.append(mean_lat_sub_segment) 84 | list_ssh_map_interp_segment.append(ssh_map_interp_segment) 85 | 86 | 87 | return list_lon_segment, list_lat_segment, list_ssh_alongtrack_segment, list_ssh_map_interp_segment, npt 88 | 89 | 90 | 91 | 92 | def compute_spectral_scores(time_alongtrack, 93 | lat_alongtrack, 94 | lon_alongtrack, 95 | ssh_alongtrack, 96 | ssh_map_interp, 97 | lenght_scale, 98 | delta_x, 99 | delta_t, 100 | output_filename): 101 | 102 | # make time vector as days since 1950-01-01 103 | #time_alongtrack = (time_alongtrack - np.datetime64('1950-01-01T00:00:00Z')) / np.timedelta64(1, 'D') 104 | 105 | # compute segments 106 | lon_segment, lat_segment, ref_segment, study_segment, npt = compute_segment_alongtrack(time_alongtrack, 107 | lat_alongtrack, 108 | lon_alongtrack, 109 | ssh_alongtrack, 110 | ssh_map_interp, 111 | lenght_scale, 112 | delta_x, 113 | delta_t) 114 | 115 | # Power spectrum density reference field 116 | global_wavenumber, global_psd_ref = scipy.signal.welch(np.asarray(ref_segment).flatten(), 117 | fs=1.0 / delta_x, 118 | nperseg=npt, 119 | scaling='density', 120 | noverlap=0) 121 | 122 | # Power spectrum density study field 123 | _, global_psd_study = scipy.signal.welch(np.asarray(study_segment).flatten(), 124 | fs=1.0 / delta_x, 125 | nperseg=npt, 126 | scaling='density', 127 | noverlap=0) 128 | 129 | # Power spectrum density study field 130 | _, global_psd_diff = scipy.signal.welch(np.asarray(study_segment).flatten()-np.asarray(ref_segment).flatten(), 131 | fs=1.0 / delta_x, 132 | nperseg=npt, 133 | scaling='density', 134 | noverlap=0) 135 | 136 | # Save psd in netcdf file 137 | ds = xr.Dataset({"psd_ref": (["wavenumber"], global_psd_ref), 138 | "psd_study": (["wavenumber"], global_psd_study), 139 | "psd_diff": (["wavenumber"], global_psd_diff), 140 | }, 141 | coords={"wavenumber": (["wavenumber"], global_wavenumber)}, 142 | ) 143 | 144 | ds.to_netcdf(output_filename) 145 | logging.info(f' Results saved in: {output_filename}') 146 | 147 | -------------------------------------------------------------------------------- /src/.ipynb_checkpoints/mod_stats-checkpoint.py: -------------------------------------------------------------------------------- 1 | import pyinterp 2 | import numpy as np 3 | import netCDF4 4 | import logging 5 | 6 | from src.mod_write import * 7 | 8 | def compute_stats(time_alongtrack, 9 | lat_alongtrack, 10 | lon_alongtrack, 11 | ssh_alongtrack, 12 | ssh_map_interp, 13 | bin_lon_step, 14 | bin_lat_step, 15 | bin_time_step, 16 | output_filename, 17 | output_filename_timeseries): 18 | 19 | ncfile = netCDF4.Dataset(output_filename,'w') 20 | 21 | binning = pyinterp.Binning2D( 22 | pyinterp.Axis(np.arange(0, 360, bin_lon_step), is_circle=True), 23 | pyinterp.Axis(np.arange(-90, 90 + bin_lat_step, bin_lat_step))) 24 | 25 | # binning alongtrack 26 | binning.push(lon_alongtrack, lat_alongtrack, ssh_alongtrack, simple=True) 27 | write_stat(ncfile, 'alongtrack', binning) 28 | binning.clear() 29 | 30 | # binning map interp 31 | binning.push(lon_alongtrack, lat_alongtrack, ssh_map_interp, simple=True) 32 | write_stat(ncfile, 'maps', binning) 33 | binning.clear() 34 | 35 | # binning diff sla-msla 36 | binning.push(lon_alongtrack, lat_alongtrack, ssh_alongtrack - ssh_map_interp, simple=True) 37 | write_stat(ncfile, 'diff', binning) 38 | binning.clear() 39 | 40 | # add rmse 41 | diff2 = (ssh_alongtrack - ssh_map_interp)**2 42 | binning.push(lon_alongtrack, lat_alongtrack, diff2, simple=True) 43 | var = ncfile.groups['diff'].createVariable('rmse', binning.variable('mean').dtype, ('lat','lon'), zlib=True) 44 | var[:, :] = np.sqrt(binning.variable('mean')).T 45 | 46 | ncfile.close() 47 | 48 | logging.info(f' Results saved in: {output_filename}') 49 | 50 | # write time series statistics 51 | leaderboard_nrmse, leaderboard_nrmse_std = write_timeserie_stat(ssh_alongtrack, 52 | ssh_map_interp, 53 | time_alongtrack, 54 | bin_time_step, 55 | output_filename_timeseries) 56 | 57 | return leaderboard_nrmse, leaderboard_nrmse_std -------------------------------------------------------------------------------- /src/.ipynb_checkpoints/mod_write-checkpoint.py: -------------------------------------------------------------------------------- 1 | import xarray as xr 2 | import numpy as np 3 | import logging 4 | 5 | 6 | def write_stat(nc, group_name, binning): 7 | 8 | grp = nc.createGroup(group_name) 9 | grp.createDimension('lon', len(binning.x)) 10 | grp.createDimension('lat', len(binning.y)) 11 | 12 | longitude = grp.createVariable('lon', 'f4', 'lon', zlib=True) 13 | longitude[:] = binning.x 14 | latitude = grp.createVariable('lat', 'f4', 'lat', zlib=True) 15 | latitude[:] = binning.y 16 | 17 | stats = ['min', 'max', 'sum', 'sum_of_weights', 'variance', 'mean', 'count', 'kurtosis', 'skewness'] 18 | for variable in stats: 19 | 20 | var = grp.createVariable(variable, binning.variable(variable).dtype, ('lat','lon'), zlib=True) 21 | var[:, :] = binning.variable(variable).T 22 | 23 | 24 | def write_timeserie_stat(ssh_alongtrack, ssh_map_interp, time_vector, freq, output_filename): 25 | 26 | 27 | diff = ssh_alongtrack - ssh_map_interp 28 | # convert data vector and time vector into xarray.Dataarray 29 | da = xr.DataArray(diff, coords=[time_vector], dims="time") 30 | 31 | # resample 32 | da_resample = da.resample(time=freq) 33 | 34 | # compute stats 35 | vmean = da_resample.mean() 36 | vminimum = da_resample.min() 37 | vmaximum = da_resample.max() 38 | vcount = da_resample.count() 39 | vvariance = da_resample.var() 40 | vmedian = da_resample.median() 41 | vrms = np.sqrt(np.square(da).resample(time=freq).mean()) 42 | 43 | rmse = np.copy(vrms) 44 | 45 | # save stat to dataset 46 | ds = xr.Dataset( 47 | { 48 | "mean": (("time"), vmean.values), 49 | "min": (("time"), vminimum.values), 50 | "max": (("time"), vmaximum.values), 51 | "count": (("time"), vcount.values), 52 | "variance": (("time"), vvariance.values), 53 | "median": (("time"), vmedian.values), 54 | "rms": (("time"), vrms.values), 55 | }, 56 | {"time": vmean['time']}, 57 | ) 58 | 59 | ds.to_netcdf(output_filename, group='diff') 60 | 61 | 62 | # convert data vector and time vector into xarray.Dataarray 63 | da = xr.DataArray(ssh_alongtrack, coords=[time_vector], dims="time") 64 | 65 | # resample 66 | da_resample = da.resample(time=freq) 67 | 68 | # compute stats 69 | vmean = da_resample.mean() 70 | vminimum = da_resample.min() 71 | vmaximum = da_resample.max() 72 | vcount = da_resample.count() 73 | vvariance = da_resample.var() 74 | vmedian = da_resample.median() 75 | vrms = np.sqrt(np.square(da).resample(time=freq).mean()) 76 | 77 | rms_alongtrack = np.copy(vrms) 78 | 79 | # save stat to dataset 80 | ds = xr.Dataset( 81 | { 82 | "mean": (("time"), vmean.values), 83 | "min": (("time"), vminimum.values), 84 | "max": (("time"), vmaximum.values), 85 | "count": (("time"), vcount.values), 86 | "variance": (("time"), vvariance.values), 87 | "median": (("time"), vmedian.values), 88 | "rms": (("time"), vrms.values), 89 | }, 90 | {"time": vmean['time']}, 91 | ) 92 | 93 | ds.to_netcdf(output_filename, group='alongtrack', mode='a') 94 | 95 | 96 | # convert data vector and time vector into xarray.Dataarray 97 | da = xr.DataArray(ssh_map_interp, coords=[time_vector], dims="time") 98 | 99 | # resample 100 | da_resample = da.resample(time=freq) 101 | 102 | # compute stats 103 | vmean = da_resample.mean() 104 | vminimum = da_resample.min() 105 | vmaximum = da_resample.max() 106 | vcount = da_resample.count() 107 | vvariance = da_resample.var() 108 | vmedian = da_resample.median() 109 | vrms = np.sqrt(np.square(da).resample(time=freq).mean()) 110 | 111 | # save stat to dataset 112 | ds = xr.Dataset( 113 | { 114 | "mean": (("time"), vmean.values), 115 | "min": (("time"), vminimum.values), 116 | "max": (("time"), vmaximum.values), 117 | "count": (("time"), vcount.values), 118 | "variance": (("time"), vvariance.values), 119 | "median": (("time"), vmedian.values), 120 | "rms": (("time"), vrms.values), 121 | }, 122 | {"time": vmean['time']}, 123 | ) 124 | 125 | ds.to_netcdf(output_filename, group='maps', mode='a') 126 | 127 | logging.info(' ') 128 | logging.info(f' Results saved in: {output_filename}') 129 | 130 | rmse_score = 1. - rmse/rms_alongtrack 131 | # mask score if nb obs < nb_min_obs 132 | nb_min_obs = 10 133 | rmse_score = np.ma.masked_where(vcount.values < nb_min_obs, rmse_score) 134 | 135 | mean_rmse = np.ma.mean(np.ma.masked_invalid(rmse_score)) 136 | std_rmse = np.ma.std(np.ma.masked_invalid(rmse_score)) 137 | 138 | logging.info(' ') 139 | logging.info(f' MEAN RMSE Score = {mean_rmse}') 140 | logging.info(' ') 141 | logging.info(f' STD RMSE Score = {std_rmse}') 142 | 143 | return mean_rmse, std_rmse -------------------------------------------------------------------------------- /src/__pycache__/mod_inout.cpython-312.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_inout.cpython-312.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_inout.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_inout.cpython-37.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_inout.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_inout.cpython-38.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_interp.cpython-312.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_interp.cpython-312.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_interp.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_interp.cpython-37.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_interp.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_interp.cpython-38.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_oi.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_oi.cpython-37.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_plot.cpython-312.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_plot.cpython-312.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_plot.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_plot.cpython-37.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_plot.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_plot.cpython-38.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_spectral.cpython-312.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_spectral.cpython-312.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_spectral.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_spectral.cpython-37.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_spectral.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_spectral.cpython-38.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_stats.cpython-312.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_stats.cpython-312.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_stats.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_stats.cpython-37.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_stats.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_stats.cpython-38.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_write.cpython-312.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_write.cpython-312.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_write.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_write.cpython-37.pyc -------------------------------------------------------------------------------- /src/__pycache__/mod_write.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocean-data-challenges/2021a_SSH_mapping_OSE/e3a22aa69f6b5e9aa423713f736aaedd0c41def6/src/__pycache__/mod_write.cpython-38.pyc -------------------------------------------------------------------------------- /src/mod_inout.py: -------------------------------------------------------------------------------- 1 | import xarray as xr 2 | import numpy as np 3 | #import pandas as pd 4 | import pyinterp 5 | import logging 6 | import requests as rq 7 | 8 | def read_l3_dataset(file, 9 | lon_min=0., 10 | lon_max=360., 11 | lat_min=-90, 12 | lat_max=90., 13 | time_min='1900-10-01', 14 | time_max='2100-01-01'): 15 | 16 | ds = xr.open_dataset(file) 17 | ds = ds.sel(time=slice(time_min, time_max), drop=True) 18 | ds = ds.where((ds["latitude"] >= lat_min) & (ds["latitude"] <= lat_max), drop=True) 19 | ds = ds.where((ds["longitude"] >= lon_min%360.) & (ds["longitude"] <= lon_max%360.), drop=True) 20 | 21 | return ds 22 | 23 | 24 | def read_l3_dataset_from_aviso(url_dataset, 25 | my_aviso_session, 26 | lon_min=0., 27 | lon_max=360., 28 | lat_min=-90, 29 | lat_max=90., 30 | time_min='1900-10-01', 31 | time_max='2100-01-01'): 32 | 33 | # disable logger for library 34 | logging.getLogger("requests").setLevel(logging.WARNING) 35 | logging.getLogger("pydap").setLevel(logging.WARNING) 36 | 37 | store = xr.backends.PydapDataStore.open(url_dataset, session=my_aviso_session) 38 | ds = xr.open_dataset(store) 39 | ds = ds.sel(time=slice(time_min, time_max), drop=True) 40 | ds = ds.where((ds["latitude"] >= lat_min) & (ds["latitude"] <= lat_max), drop=True) 41 | ds = ds.where((ds["longitude"] >= lon_min%360.) & (ds["longitude"] <= lon_max%360.), drop=True) 42 | 43 | return ds 44 | 45 | 46 | def read_l4_dataset(list_of_file, 47 | lon_min=0., 48 | lon_max=360., 49 | lat_min=-90, 50 | lat_max=90., 51 | time_min='1900-10-01', 52 | time_max='2100-01-01', 53 | is_circle=True): 54 | 55 | 56 | ds = xr.open_mfdataset(list_of_file, concat_dim ='time', combine='nested', parallel=True) 57 | ds = ds.sel(time=slice(time_min, time_max), drop=True) 58 | ds = ds.where((ds["lon"]%360. >= lon_min) & (ds["lon"]%360. <= lon_max), drop=True) 59 | ds = ds.where((ds["lat"] >= lat_min) & (ds["lat"] <= lat_max), drop=True) 60 | 61 | x_axis = pyinterp.Axis(ds["lon"][:].values%360., is_circle=is_circle) 62 | y_axis = pyinterp.Axis(ds["lat"][:].values) 63 | z_axis = pyinterp.TemporalAxis(ds["time"][:].values) 64 | 65 | var = ds['ssh'][:] 66 | var = var.transpose('lon', 'lat', 'time') 67 | 68 | # The undefined values must be set to nan. 69 | try: 70 | var[var.mask] = float("nan") 71 | except AttributeError: 72 | pass 73 | 74 | grid = pyinterp.Grid3D(x_axis, y_axis, z_axis, var.data) 75 | 76 | del ds 77 | 78 | return x_axis, y_axis, z_axis, grid 79 | 80 | 81 | def read_l4_dataset_from_aviso(url_dataset, 82 | my_aviso_session, 83 | lon_min=0., 84 | lon_max=360., 85 | lat_min=-90, 86 | lat_max=90., 87 | time_min='1900-10-01', 88 | time_max='2100-01-01', 89 | is_circle=True): 90 | 91 | # disable logger for library 92 | logging.getLogger("requests").setLevel(logging.WARNING) 93 | logging.getLogger("pydap").setLevel(logging.WARNING) 94 | 95 | store = xr.backends.PydapDataStore.open(url_dataset, session=my_aviso_session) 96 | ds = xr.open_dataset(store, chunks={'time': '100MB'}) 97 | ds = ds.sel(time=slice(time_min, time_max), drop=True) 98 | ds = ds.where((ds["lon"]%360. >= lon_min) & (ds["lon"]%360. <= lon_max), drop=True) 99 | ds = ds.where((ds["lat"] >= lat_min) & (ds["lat"] <= lat_max), drop=True) 100 | 101 | x_axis = pyinterp.Axis(ds["lon"][:].values%360., is_circle=is_circle) 102 | y_axis = pyinterp.Axis(ds["lat"][:].values) 103 | z_axis = pyinterp.TemporalAxis(ds["time"][:].values) 104 | 105 | var = ds['ssh'][:] 106 | var = var.transpose('lon', 'lat', 'time') 107 | 108 | # The undefined values must be set to nan. 109 | try: 110 | var[var.mask] = float("nan") 111 | except AttributeError: 112 | pass 113 | 114 | grid = pyinterp.Grid3D(x_axis, y_axis, z_axis, var.data) 115 | 116 | del ds 117 | 118 | return x_axis, y_axis, z_axis, grid 119 | 120 | 121 | def read_obs(input_file, oi_grid, oi_param, coarsening): 122 | 123 | logging.info(' Reading observations...') 124 | 125 | def preprocess(ds): 126 | return ds.coarsen(coarsening, boundary="trim").mean() 127 | 128 | ds_obs = xr.open_mfdataset(input_file, combine='nested', concat_dim='time', parallel=True, preprocess=preprocess) #.sortby('time') 129 | #ds_obs = ds_obs.coarsen(coarsening, boundary="trim").mean().sortby('time') 130 | ds_obs = ds_obs.sortby('time') 131 | 132 | lon_min = oi_grid.lon.min().values 133 | lon_max = oi_grid.lon.max().values 134 | lat_min = oi_grid.lat.min().values 135 | lat_max = oi_grid.lat.max().values 136 | time_min = oi_grid.time.min().values 137 | time_max = oi_grid.time.max().values 138 | 139 | ds_obs = ds_obs.sel(time=slice(time_min - np.timedelta64(int(2*oi_param.Lt.values), 'D'), 140 | time_max + np.timedelta64(int(2*oi_param.Lt.values), 'D')), drop=True) 141 | 142 | # correct lon if domain is between [-180:180] 143 | if lon_min < 0: 144 | ds_obs['lon'] = xr.where(ds_obs['longitude'] >= 180., ds_obs['longitude']-360., ds_obs['longitude']) 145 | 146 | ds_obs = ds_obs.where((ds_obs['longitude'] >= lon_min - oi_param.Lx.values) & 147 | (ds_obs['longitude'] <= lon_max + oi_param.Lx.values) & 148 | (ds_obs['latitude'] >= lat_min - oi_param.Ly.values) & 149 | (ds_obs['latitude'] <= lat_max + oi_param.Ly.values) , drop=True) 150 | 151 | vtime = (ds_obs['time'].values - time_min) / np.timedelta64(1, 'D') 152 | ds_obs = ds_obs.assign_coords({'time': vtime}) 153 | 154 | ds_obs = ds_obs.dropna(dim='time') 155 | 156 | return ds_obs 157 | 158 | 159 | def read_obs_from_aviso(input_file, my_aviso_session, oi_grid, oi_param, coarsening): 160 | 161 | logging.info(' Reading observations from aviso...') 162 | 163 | # disable logger for library 164 | logging.getLogger("requests").setLevel(logging.WARNING) 165 | logging.getLogger("pydap").setLevel(logging.WARNING) 166 | 167 | def preprocess(ds): 168 | return ds.coarsen(coarsening, boundary="trim").mean() 169 | 170 | list_of_dataset = [] 171 | for url_dataset in input_file: 172 | store = xr.backends.PydapDataStore.open(url_dataset, session=my_aviso_session) 173 | ds_store = xr.open_dataset(store) 174 | list_of_dataset.append(ds_store) 175 | 176 | ds_obs = xr.concat(list_of_dataset, dim='time') 177 | #ds_obs = xr.open_mfdataset(input_file, combine='nested', concat_dim='time', parallel=True, preprocess=preprocess) #.sortby('time') 178 | #ds_obs = ds_obs.coarsen(coarsening, boundary="trim").mean().sortby('time') 179 | ds_obs = ds_obs.coarsen(coarsening, boundary="trim").mean() 180 | ds_obs = ds_obs.sortby('time') 181 | 182 | lon_min = oi_grid.lon.min().values 183 | lon_max = oi_grid.lon.max().values 184 | lat_min = oi_grid.lat.min().values 185 | lat_max = oi_grid.lat.max().values 186 | time_min = oi_grid.time.min().values 187 | time_max = oi_grid.time.max().values 188 | 189 | ds_obs = ds_obs.sel(time=slice(time_min - np.timedelta64(int(2*oi_param.Lt.values), 'D'), 190 | time_max + np.timedelta64(int(2*oi_param.Lt.values), 'D')), drop=True) 191 | 192 | # correct lon if domain is between [-180:180] 193 | if lon_min < 0: 194 | ds_obs['longitude'] = xr.where(ds_obs['longitude'] >= 180., ds_obs['longitude']-360., ds_obs['longitude']) 195 | 196 | ds_obs = ds_obs.where((ds_obs['longitude'] >= lon_min - oi_param.Lx.values) & 197 | (ds_obs['longitude'] <= lon_max + oi_param.Lx.values) & 198 | (ds_obs['latitude'] >= lat_min - oi_param.Ly.values) & 199 | (ds_obs['latitude'] <= lat_max + oi_param.Ly.values) , drop=True) 200 | 201 | vtime = (ds_obs['time'].values - time_min) / np.timedelta64(1, 'D') 202 | ds_obs = ds_obs.assign_coords({'time': vtime}) 203 | 204 | ds_obs = ds_obs.dropna(dim='time') 205 | 206 | return ds_obs 207 | 208 | 209 | def reformate_oi_output(ds_oi, url_ds_mdt, my_aviso_session): 210 | ds_oi = ds_oi.drop(['gtime', 'ng', 'glon2', 'glat2', 'fglon', 'fglat', 'nobs']) 211 | ds_oi = ds_oi.rename({'gssh': 'sla'}) 212 | 213 | 214 | store_ds_mdt = xr.backends.PydapDataStore.open(url_ds_mdt, session=my_aviso_session) 215 | mdt = xr.open_dataset(store_ds_mdt) 216 | 217 | # mdt = xr.open_dataset(mdt_file) 218 | mdt_interp = mdt.interp(lon=ds_oi.lon, lat=ds_oi.lat) 219 | 220 | ds_oi['ssh'] = ds_oi['sla'] + mdt_interp['mdt'] 221 | 222 | return ds_oi -------------------------------------------------------------------------------- /src/mod_interp.py: -------------------------------------------------------------------------------- 1 | import xarray as xr 2 | import numpy as np 3 | import pyinterp 4 | import netCDF4 5 | 6 | #import sys 7 | #sys.path.append('.') 8 | from src.mod_inout import * 9 | 10 | 11 | def interp_on_alongtrack(gridded_dataset, 12 | ds_alongtrack, 13 | lon_min=0., 14 | lon_max=360., 15 | lat_min=-90, 16 | lat_max=90., 17 | time_min='1900-10-01', 18 | time_max='2100-01-01', 19 | is_circle=True): 20 | 21 | # Interpolate maps onto alongtrack dataset 22 | if isinstance(gridded_dataset, str): 23 | x_axis, y_axis, z_axis, grid = read_l4_dataset(gridded_dataset, 24 | lon_min=lon_min, 25 | lon_max=lon_max, 26 | lat_min=lat_min, 27 | lat_max=lat_max, 28 | time_min=time_min, 29 | time_max=time_max, 30 | is_circle=is_circle) 31 | elif isinstance(gridded_dataset, list): 32 | 33 | x_axis, y_axis, z_axis, grid = read_l4_dataset_from_aviso(gridded_dataset[0], 34 | gridded_dataset[1], 35 | lon_min=lon_min, 36 | lon_max=lon_max, 37 | lat_min=lat_min, 38 | lat_max=lat_max, 39 | time_min=time_min, 40 | time_max=time_max, 41 | is_circle=is_circle) 42 | 43 | ssh_map_interp = pyinterp.trivariate(grid, 44 | ds_alongtrack["longitude"].values, 45 | ds_alongtrack["latitude"].values, 46 | z_axis.safe_cast(ds_alongtrack.time.values), 47 | bounds_error=False).reshape(ds_alongtrack["longitude"].values.shape) 48 | 49 | ssh_alongtrack = (ds_alongtrack["sla_unfiltered"] + ds_alongtrack["mdt"] - ds_alongtrack["lwe"]).values 50 | lon_alongtrack = ds_alongtrack["longitude"].values 51 | lat_alongtrack = ds_alongtrack["latitude"].values 52 | time_alongtrack = ds_alongtrack["time"].values 53 | 54 | # get and apply mask from map_interp & alongtrack on each dataset 55 | msk1 = np.ma.masked_invalid(ssh_alongtrack).mask 56 | msk2 = np.ma.masked_invalid(ssh_map_interp).mask 57 | msk = msk1 + msk2 58 | 59 | ssh_alongtrack = np.ma.masked_where(msk, ssh_alongtrack).compressed() 60 | lon_alongtrack = np.ma.masked_where(msk, lon_alongtrack).compressed() 61 | lat_alongtrack = np.ma.masked_where(msk, lat_alongtrack).compressed() 62 | time_alongtrack = np.ma.masked_where(msk, time_alongtrack).compressed() 63 | ssh_map_interp = np.ma.masked_where(msk, ssh_map_interp).compressed() 64 | 65 | # select inside value (this is done to insure similar number of point in statistical comparison between methods) 66 | indices = np.where((lon_alongtrack >= lon_min+0.25) & (lon_alongtrack <= lon_max-0.25) & 67 | (lat_alongtrack >= lat_min+0.25) & (lat_alongtrack <= lat_max-0.25))[0] 68 | 69 | return time_alongtrack[indices], lat_alongtrack[indices], lon_alongtrack[indices], ssh_alongtrack[indices], ssh_map_interp[indices] 70 | -------------------------------------------------------------------------------- /src/mod_oi.py: -------------------------------------------------------------------------------- 1 | import xarray as xr 2 | import numpy 3 | import logging 4 | 5 | 6 | def oi_grid(glon, glat, gtime): 7 | """ 8 | 9 | """ 10 | 11 | logging.info(' Set OI grid...') 12 | 13 | nx = len(glon) 14 | ny = len(glat) 15 | nt = len(gtime) 16 | 17 | # define & initialize ssh array 18 | gssh = numpy.empty((nt, ny, nx)) 19 | nobs = numpy.empty(nt) 20 | 21 | # Make 2D grid 22 | glon2, glat2 = numpy.meshgrid(glon, glat) 23 | fglon = glon2.flatten() 24 | fglat = glat2.flatten() 25 | 26 | ng = len(fglat) # number of grid points 27 | vtime = (gtime - gtime[0]) / numpy.timedelta64(1, 'D') 28 | 29 | 30 | ds_oi_grid = xr.Dataset({'gssh' : (('time', 'lat', 'lon'), gssh), 31 | 'glon2' : (('lat', 'lon'), glon2), 32 | 'glat2' : (('lat', 'lon'), glat2), 33 | 'fglon' : (('ng'), fglon), 34 | 'fglat' : (('ng'), fglat), 35 | 'nobs' : (('time'), nobs)}, 36 | coords={'gtime': (vtime).astype(numpy.float), 37 | 'time': gtime, 38 | 'lat': glat, 39 | 'lon': glon, 40 | 'ng': numpy.arange(ng)}) 41 | 42 | return ds_oi_grid 43 | 44 | 45 | def oi_param(Lx, Ly, Lt, noise): 46 | 47 | logging.info(' Set OI params...') 48 | 49 | ds_oi_param = xr.Dataset({'Lx' : Lx, 50 | 'Ly' : Ly, 51 | 'Lt' : Lt, 52 | 'noise' : noise}) 53 | 54 | return ds_oi_param 55 | 56 | 57 | def oi_core(it, ds_oi_grid, ds_oi_param, ds_obs): 58 | 59 | ind1 = numpy.where((numpy.abs(ds_obs.time.values - ds_oi_grid.gtime.values[it]) < 2.*ds_oi_param.Lt.values))[0] 60 | nobs = len(ind1) 61 | print('Processing time-step : ', it, '/', len(ds_oi_grid.gtime.values) - 1, ' nobs = ', nobs, end="\r") 62 | 63 | BHt = numpy.empty((len(ds_oi_grid.ng), nobs)) 64 | HBHt = numpy.empty((nobs, nobs)) 65 | 66 | obs_lon = ds_obs.longitude.values[ind1] 67 | obs_lat = ds_obs.latitude.values[ind1] 68 | obs_time = ds_obs.time.values[ind1] 69 | 70 | fglon = ds_oi_grid.fglon.values 71 | fglat = ds_oi_grid.fglat.values 72 | ftime = ds_oi_grid.gtime.values[it] 73 | 74 | for iobs in range(nobs): 75 | # print(iobs) 76 | 77 | BHt[:,iobs] = numpy.exp(-((ftime - obs_time[iobs])/ds_oi_param.Lt.values)**2 - 78 | ((fglon - obs_lon[iobs])/ds_oi_param.Lx.values)**2 - 79 | ((fglat - obs_lat[iobs])/ds_oi_param.Ly.values)**2) 80 | 81 | HBHt[:,iobs] = numpy.exp(-((obs_time - obs_time[iobs])/ds_oi_param.Lt.values)**2 - 82 | ((obs_lon - obs_lon[iobs])/ds_oi_param.Lx.values)**2 - 83 | ((obs_lat - obs_lat[iobs])/ds_oi_param.Ly.values)**2) 84 | 85 | del obs_lon, obs_lat, obs_time 86 | 87 | R = numpy.diag(numpy.full((nobs), ds_oi_param.noise.values**2)) 88 | 89 | Coo = HBHt + R 90 | Mi = numpy.linalg.inv(Coo) 91 | 92 | sol = numpy.dot(numpy.dot(BHt, Mi), ds_obs.sla_unfiltered.values[ind1]) 93 | 94 | ds_oi_grid.gssh[it, :, :] = sol.reshape(ds_oi_grid.lat.size, ds_oi_grid.lon.size) 95 | ds_oi_grid.nobs[it] = nobs 96 | 97 | 98 | #return None # sol.reshape(ds_oi_grid.glat.size, ds_oi_grid.glon.size), nobs 99 | 100 | -------------------------------------------------------------------------------- /src/mod_plot.py: -------------------------------------------------------------------------------- 1 | import xarray as xr 2 | import numpy as np 3 | import logging 4 | import matplotlib.pylab as plt 5 | from scipy import interpolate 6 | import hvplot.xarray 7 | import cartopy.crs as ccrs 8 | from matplotlib.patches import Rectangle 9 | 10 | def find_wavelength_05_crossing(filename): 11 | 12 | ds = xr.open_dataset(filename) 13 | y = 1./ds.wavenumber 14 | x = (1. - ds.psd_diff/ds.psd_ref) 15 | f = interpolate.interp1d(x, y) 16 | 17 | xnew = 0.5 18 | ynew = f(xnew) 19 | 20 | return ynew 21 | 22 | 23 | 24 | def plot_psd_score(filename): 25 | 26 | ds = xr.open_dataset(filename) 27 | 28 | resolved_scale = find_wavelength_05_crossing(filename) 29 | 30 | plt.figure(figsize=(10, 5)) 31 | ax = plt.subplot(121) 32 | ax.invert_xaxis() 33 | plt.plot((1./ds.wavenumber), ds.psd_ref, label='reference', color='k') 34 | plt.plot((1./ds.wavenumber), ds.psd_study, label='reconstruction', color='lime') 35 | plt.xlabel('wavelength [km]') 36 | plt.ylabel('Power Spectral Density [m$^{2}$/cy/km]') 37 | plt.xscale('log') 38 | plt.yscale('log') 39 | plt.legend(loc='best') 40 | plt.grid(which='both') 41 | 42 | ax = plt.subplot(122) 43 | ax.invert_xaxis() 44 | plt.plot((1./ds.wavenumber), (1. - ds.psd_diff/ds.psd_ref), color='k', lw=2) 45 | plt.xlabel('wavelength [km]') 46 | plt.ylabel('PSD Score [1. - PSD$_{err}$/PSD$_{ref}$]') 47 | plt.xscale('log') 48 | plt.hlines(y=0.5, 49 | xmin=np.ma.min(np.ma.masked_invalid(1./ds.wavenumber)), 50 | xmax=np.ma.max(np.ma.masked_invalid(1./ds.wavenumber)), 51 | color='r', 52 | lw=0.5, 53 | ls='--') 54 | plt.vlines(x=resolved_scale, ymin=0, ymax=1, lw=0.5, color='g') 55 | ax.fill_betweenx((1. - ds.psd_diff/ds.psd_ref), 56 | resolved_scale, 57 | np.ma.max(np.ma.masked_invalid(1./ds.wavenumber)), 58 | color='green', 59 | alpha=0.3, 60 | label=f'resolved scales \n $\lambda$ > {int(resolved_scale)}km') 61 | plt.legend(loc='best') 62 | plt.grid(which='both') 63 | 64 | logging.info(' ') 65 | logging.info(f' Minimum spatial scale resolved = {int(resolved_scale)}km') 66 | 67 | plt.show() 68 | 69 | return resolved_scale 70 | 71 | 72 | def plot_spatial_statistics(filename): 73 | 74 | ds = xr.open_dataset(filename, group='diff') 75 | # try: 76 | # figure = ds['rmse'].hvplot.image(x='lon', y='lat', z='rmse', clabel='RMSE [m]', cmap='Reds', coastline=True) 77 | # except KeyError: 78 | # figure = ds['rmse'].hvplot.image(x='lon', y='lat', z='rmse', clabel='RMSE [m]', cmap='Reds') 79 | 80 | figure = ds['rmse'].hvplot.image(x='lon', y='lat', z='rmse', clabel='RMSE [m]', cmap='Reds') 81 | 82 | return figure 83 | 84 | 85 | def plot_temporal_statistics(filename): 86 | 87 | ds1 = xr.open_dataset(filename, group='diff') 88 | ds2 = xr.open_dataset(filename, group='alongtrack') 89 | rmse_score = 1. - ds1['rms']/ds2['rms'] 90 | 91 | rmse_score = rmse_score.dropna(dim='time').where(ds1['count'] > 10, drop=True) 92 | 93 | figure = rmse_score.hvplot.line(ylabel='RMSE SCORE', shared_axes=True, color='r') + ds1['count'].dropna(dim='time').hvplot.step(ylabel='#Obs.', shared_axes=True, color='grey') 94 | 95 | return figure.cols(1) 96 | 97 | 98 | def spectral_score_intercomparison(list_of_filename, list_of_label): 99 | 100 | plt.figure(figsize=(15, 6)) 101 | ax = plt.subplot(121) 102 | ax.invert_xaxis() 103 | ds = xr.open_dataset(list_of_filename[0]) 104 | plt.plot((1./ds.wavenumber), ds.psd_ref, label='reference', color='k') 105 | for cfilename, clabel in zip(list_of_filename, list_of_label): 106 | ds = xr.open_dataset(cfilename) 107 | plt.plot((1./ds.wavenumber), ds.psd_study, label=clabel) 108 | plt.xlabel('wavelength [km]') 109 | plt.ylabel('Power Spectral Density [m$^{2}$/cy/km]') 110 | plt.xscale('log') 111 | plt.yscale('log') 112 | plt.legend(loc='best') 113 | plt.grid(which='both') 114 | plt.xticks([50, 100, 200, 500, 1000], ["50km", "100km", "200km", "500km", "1000km"]) 115 | 116 | ax = plt.subplot(122) 117 | ax.invert_xaxis() 118 | for cfilename, clabel in zip(list_of_filename, list_of_label): 119 | ds = xr.open_dataset(cfilename) 120 | plt.plot((1./ds.wavenumber), (1. - ds.psd_diff/ds.psd_ref), lw=2, label=clabel) 121 | plt.xlabel('wavelength [km]') 122 | plt.ylabel('PSD Score [1. - PSD$_{err}$/PSD$_{ref}$]') 123 | plt.xscale('log') 124 | plt.hlines(y=0.5, 125 | xmin=np.ma.min(np.ma.masked_invalid(1./ds.wavenumber)), 126 | xmax=np.ma.max(np.ma.masked_invalid(1./ds.wavenumber)), 127 | color='r', 128 | lw=0.5, 129 | ls='--') 130 | # plt.vlines(x=resolved_scale, ymin=0, ymax=1, lw=0.5, color='g') 131 | # ax.fill_betweenx((1. - ds.psd_diff/ds.psd_ref), 132 | # resolved_scale, 133 | # np.ma.max(np.ma.masked_invalid(1./ds.wavenumber)), 134 | # color='green', 135 | # alpha=0.3, 136 | # label=f'resolved scales \n $\lambda$ > {int(resolved_scale)}km') 137 | plt.legend(loc='best') 138 | plt.grid(which='both') 139 | plt.xticks([50, 100, 200, 500, 1000], ["50km", "100km", "200km", "500km", "1000km"]) 140 | 141 | plt.show() 142 | 143 | 144 | def intercomparison_temporal_statistics(list_of_filename, list_of_label): 145 | 146 | ds_diff = xr.concat([xr.open_dataset(filename, group='diff') for filename in list_of_filename], dim='experiment') 147 | ds_diff['experiment'] = list_of_label 148 | ds_alongtrack = xr.concat([xr.open_dataset(filename, group='alongtrack') for filename in list_of_filename], dim='experiment') 149 | ds_alongtrack['experiment'] = list_of_label 150 | 151 | rmse_score = 1. - ds_diff['rms']/ds_alongtrack['rms'] 152 | rmse_score = rmse_score.dropna(dim='time').where(ds_diff['count'] > 10, drop=True) 153 | 154 | figure = rmse_score.hvplot.line(x='time', y='rms', by='experiment', ylim=(0, 1), title='RMSE SCORE', shared_axes=True) + ds_diff['count'][0, :].dropna(dim='time').hvplot.step(ylabel='#Obs.', shared_axes=True, color='grey') 155 | 156 | return figure.cols(1) 157 | 158 | 159 | def intercomparison_spatial_statistics(baseline_filename, list_of_filename, list_of_label): 160 | 161 | ds_baseline = xr.open_dataset(baseline_filename, group='diff') 162 | ds = xr.concat([xr.open_dataset(filename, group='diff') for filename in list_of_filename], dim='experiment') 163 | ds['experiment'] = list_of_label 164 | 165 | delta_rmse = 100*(ds - ds_baseline)/ds_baseline 166 | 167 | figure = delta_rmse['rmse'].hvplot.image(x='lon', y='lat', z='rmse', clim=(-20, 20), by='experiment', 168 | subplots=True, projection=ccrs.PlateCarree(), 169 | clabel='[%]', cmap='coolwarm', coastline=True) 170 | 171 | return figure.cols(2) 172 | 173 | 174 | def hvplot_demo_obs_nadir(list_of_dataset, central_date, delta_t): 175 | 176 | ds_concat_nadirs = xr.concat(list_of_dataset, dim='time') 177 | ds_concat_nadirs = ds_concat_nadirs.sortby(ds_concat_nadirs.time) 178 | ds_concat_nadirs = ds_concat_nadirs.assign_coords({'longitude': ds_concat_nadirs.longitude, 'latitude': ds_concat_nadirs.latitude}) 179 | 180 | ds_concat_nadirs_selection = ds_concat_nadirs.sel(time=slice(central_date - delta_t, central_date + delta_t)).drop( 181 | 'time') 182 | 183 | plot = ds_concat_nadirs_selection.hvplot.scatter(x='longitude', y='latitude', color='sla_unfiltered', 184 | height=300, width=400, cmap = 'gist_stern', datashade=True) 185 | 186 | return plot 187 | 188 | 189 | def plot_demo_obs(list_of_dataset, central_date, delta_t): 190 | 191 | tmin = central_date - delta_t 192 | tmax = central_date + delta_t 193 | 194 | list_of_dataset_sel = [] 195 | for ds in list_of_dataset: 196 | ds_sel = ds.sel(time=slice(tmin, tmax)) 197 | if ds_sel.time.size > 0: 198 | list_of_dataset_sel.append(ds_sel) 199 | 200 | plt.figure(figsize=(10, 10)) 201 | plt.subplot(111) 202 | for ds in list_of_dataset_sel: 203 | plt.scatter(ds.longitude % 360., ds.latitude, c=ds.sla_unfiltered, s=20, cmap='gist_stern') 204 | ax = plt.gca() 205 | ax.add_patch(Rectangle((295, 33), 10, 10, fill=None, alpha=1)) 206 | plt.xlabel('longitude', fontweight='bold') 207 | plt.ylabel('latitude', fontweight='bold') 208 | plt.title(f'SLA @ altimeter track') 209 | plt.colorbar(orientation='horizontal') 210 | plt.text(298, 43.5,'STUDY AREA') 211 | plt.show() 212 | -------------------------------------------------------------------------------- /src/mod_spectral.py: -------------------------------------------------------------------------------- 1 | 2 | import numpy as np 3 | import scipy.signal 4 | import logging 5 | 6 | from src.mod_write import * 7 | 8 | 9 | def compute_segment_alongtrack(time_alongtrack, 10 | lat_alongtrack, 11 | lon_alongtrack, 12 | ssh_alongtrack, 13 | ssh_map_interp, 14 | lenght_scale, 15 | delta_x, 16 | delta_t): 17 | 18 | segment_overlapping = 0.25 19 | max_delta_t_gap = 4 * np.timedelta64(1, 's') # max delta t of 4 seconds to cut tracks 20 | 21 | list_lat_segment = [] 22 | list_lon_segment = [] 23 | list_ssh_alongtrack_segment = [] 24 | list_ssh_map_interp_segment = [] 25 | 26 | # Get number of point to consider for resolution = lenghtscale in km 27 | delta_t_jd = delta_t / (3600 * 24) 28 | npt = int(lenght_scale / delta_x) 29 | 30 | # cut track when diff time longer than 4*delta_t 31 | indi = np.where((np.diff(time_alongtrack) > max_delta_t_gap))[0] 32 | track_segment_lenght = np.insert(np.diff(indi), [0], indi[0]) 33 | 34 | # Long track >= npt 35 | selected_track_segment = np.where(track_segment_lenght >= npt)[0] 36 | 37 | if selected_track_segment.size > 0: 38 | 39 | for track in selected_track_segment: 40 | 41 | if track-1 >= 0: 42 | index_start_selected_track = indi[track-1] 43 | index_end_selected_track = indi[track] 44 | else: 45 | index_start_selected_track = 0 46 | index_end_selected_track = indi[track] 47 | 48 | start_point = index_start_selected_track 49 | end_point = index_end_selected_track 50 | 51 | for sub_segment_point in range(start_point, end_point - npt, int(npt*segment_overlapping)): 52 | 53 | # Near Greenwhich case 54 | if ((lon_alongtrack[sub_segment_point + npt - 1] < 50.) 55 | and (lon_alongtrack[sub_segment_point] > 320.)) \ 56 | or ((lon_alongtrack[sub_segment_point + npt - 1] > 320.) 57 | and (lon_alongtrack[sub_segment_point] < 50.)): 58 | 59 | tmp_lon = np.where(lon_alongtrack[sub_segment_point:sub_segment_point + npt] > 180, 60 | lon_alongtrack[sub_segment_point:sub_segment_point + npt] - 360, 61 | lon_alongtrack[sub_segment_point:sub_segment_point + npt]) 62 | mean_lon_sub_segment = np.median(tmp_lon) 63 | 64 | if mean_lon_sub_segment < 0: 65 | mean_lon_sub_segment = mean_lon_sub_segment + 360. 66 | else: 67 | 68 | mean_lon_sub_segment = np.median(lon_alongtrack[sub_segment_point:sub_segment_point + npt]) 69 | 70 | mean_lat_sub_segment = np.median(lat_alongtrack[sub_segment_point:sub_segment_point + npt]) 71 | 72 | ssh_alongtrack_segment = np.ma.masked_invalid(ssh_alongtrack[sub_segment_point:sub_segment_point + npt]) 73 | 74 | ssh_map_interp_segment = [] 75 | ssh_map_interp_segment = np.ma.masked_invalid(ssh_map_interp[sub_segment_point:sub_segment_point + npt]) 76 | if np.ma.is_masked(ssh_map_interp_segment): 77 | ssh_alongtrack_segment = np.ma.compressed(np.ma.masked_where(np.ma.is_masked(ssh_map_interp_segment), ssh_alongtrack_segment)) 78 | ssh_map_interp_segment = np.ma.compressed(ssh_map_interp_segment) 79 | 80 | if ssh_alongtrack_segment.size > 0: 81 | list_ssh_alongtrack_segment.append(ssh_alongtrack_segment) 82 | list_lon_segment.append(mean_lon_sub_segment) 83 | list_lat_segment.append(mean_lat_sub_segment) 84 | list_ssh_map_interp_segment.append(ssh_map_interp_segment) 85 | 86 | 87 | return list_lon_segment, list_lat_segment, list_ssh_alongtrack_segment, list_ssh_map_interp_segment, npt 88 | 89 | 90 | 91 | 92 | def compute_spectral_scores(time_alongtrack, 93 | lat_alongtrack, 94 | lon_alongtrack, 95 | ssh_alongtrack, 96 | ssh_map_interp, 97 | lenght_scale, 98 | delta_x, 99 | delta_t, 100 | output_filename): 101 | 102 | # make time vector as days since 1950-01-01 103 | #time_alongtrack = (time_alongtrack - np.datetime64('1950-01-01T00:00:00Z')) / np.timedelta64(1, 'D') 104 | 105 | # compute segments 106 | lon_segment, lat_segment, ref_segment, study_segment, npt = compute_segment_alongtrack(time_alongtrack, 107 | lat_alongtrack, 108 | lon_alongtrack, 109 | ssh_alongtrack, 110 | ssh_map_interp, 111 | lenght_scale, 112 | delta_x, 113 | delta_t) 114 | 115 | # Power spectrum density reference field 116 | global_wavenumber, global_psd_ref = scipy.signal.welch(np.asarray(ref_segment).flatten(), 117 | fs=1.0 / delta_x, 118 | nperseg=npt, 119 | scaling='density', 120 | noverlap=0) 121 | 122 | # Power spectrum density study field 123 | _, global_psd_study = scipy.signal.welch(np.asarray(study_segment).flatten(), 124 | fs=1.0 / delta_x, 125 | nperseg=npt, 126 | scaling='density', 127 | noverlap=0) 128 | 129 | # Power spectrum density study field 130 | _, global_psd_diff = scipy.signal.welch(np.asarray(study_segment).flatten()-np.asarray(ref_segment).flatten(), 131 | fs=1.0 / delta_x, 132 | nperseg=npt, 133 | scaling='density', 134 | noverlap=0) 135 | 136 | # Save psd in netcdf file 137 | ds = xr.Dataset({"psd_ref": (["wavenumber"], global_psd_ref), 138 | "psd_study": (["wavenumber"], global_psd_study), 139 | "psd_diff": (["wavenumber"], global_psd_diff), 140 | }, 141 | coords={"wavenumber": (["wavenumber"], global_wavenumber)}, 142 | ) 143 | 144 | ds.to_netcdf(output_filename) 145 | logging.info(f' Results saved in: {output_filename}') 146 | 147 | -------------------------------------------------------------------------------- /src/mod_stats.py: -------------------------------------------------------------------------------- 1 | import pyinterp 2 | import numpy as np 3 | import netCDF4 4 | import logging 5 | 6 | from src.mod_write import * 7 | 8 | def compute_stats(time_alongtrack, 9 | lat_alongtrack, 10 | lon_alongtrack, 11 | ssh_alongtrack, 12 | ssh_map_interp, 13 | bin_lon_step, 14 | bin_lat_step, 15 | bin_time_step, 16 | output_filename, 17 | output_filename_timeseries): 18 | 19 | ncfile = netCDF4.Dataset(output_filename,'w') 20 | 21 | binning = pyinterp.Binning2D( 22 | pyinterp.Axis(np.arange(0, 360, bin_lon_step), is_circle=True), 23 | pyinterp.Axis(np.arange(-90, 90 + bin_lat_step, bin_lat_step))) 24 | 25 | # binning alongtrack 26 | binning.push(lon_alongtrack, lat_alongtrack, ssh_alongtrack, simple=True) 27 | write_stat(ncfile, 'alongtrack', binning) 28 | binning.clear() 29 | 30 | # binning map interp 31 | binning.push(lon_alongtrack, lat_alongtrack, ssh_map_interp, simple=True) 32 | write_stat(ncfile, 'maps', binning) 33 | binning.clear() 34 | 35 | # binning diff sla-msla 36 | binning.push(lon_alongtrack, lat_alongtrack, ssh_alongtrack - ssh_map_interp, simple=True) 37 | write_stat(ncfile, 'diff', binning) 38 | binning.clear() 39 | 40 | # add rmse 41 | diff2 = (ssh_alongtrack - ssh_map_interp)**2 42 | binning.push(lon_alongtrack, lat_alongtrack, diff2, simple=True) 43 | var = ncfile.groups['diff'].createVariable('rmse', binning.variable('mean').dtype, ('lat','lon'), zlib=True) 44 | var[:, :] = np.sqrt(binning.variable('mean')).T 45 | 46 | ncfile.close() 47 | 48 | logging.info(f' Results saved in: {output_filename}') 49 | 50 | # write time series statistics 51 | leaderboard_nrmse, leaderboard_nrmse_std = write_timeserie_stat(ssh_alongtrack, 52 | ssh_map_interp, 53 | time_alongtrack, 54 | bin_time_step, 55 | output_filename_timeseries) 56 | 57 | return leaderboard_nrmse, leaderboard_nrmse_std -------------------------------------------------------------------------------- /src/mod_write.py: -------------------------------------------------------------------------------- 1 | import xarray as xr 2 | import numpy as np 3 | import logging 4 | 5 | 6 | def write_stat(nc, group_name, binning): 7 | 8 | grp = nc.createGroup(group_name) 9 | grp.createDimension('lon', len(binning.x)) 10 | grp.createDimension('lat', len(binning.y)) 11 | 12 | longitude = grp.createVariable('lon', 'f4', 'lon', zlib=True) 13 | longitude[:] = binning.x 14 | latitude = grp.createVariable('lat', 'f4', 'lat', zlib=True) 15 | latitude[:] = binning.y 16 | 17 | stats = ['min', 'max', 'sum', 'sum_of_weights', 'variance', 'mean', 'count', 'kurtosis', 'skewness'] 18 | for variable in stats: 19 | 20 | var = grp.createVariable(variable, binning.variable(variable).dtype, ('lat','lon'), zlib=True) 21 | var[:, :] = binning.variable(variable).T 22 | 23 | 24 | def write_timeserie_stat(ssh_alongtrack, ssh_map_interp, time_vector, freq, output_filename): 25 | 26 | 27 | diff = ssh_alongtrack - ssh_map_interp 28 | # convert data vector and time vector into xarray.Dataarray 29 | da = xr.DataArray(diff, coords=[time_vector], dims="time") 30 | 31 | # resample 32 | da_resample = da.resample(time=freq) 33 | 34 | # compute stats 35 | vmean = da_resample.mean() 36 | vminimum = da_resample.min() 37 | vmaximum = da_resample.max() 38 | vcount = da_resample.count() 39 | vvariance = da_resample.var() 40 | vmedian = da_resample.median() 41 | vrms = np.sqrt(np.square(da).resample(time=freq).mean()) 42 | 43 | rmse = np.copy(vrms) 44 | 45 | # save stat to dataset 46 | ds = xr.Dataset( 47 | { 48 | "mean": (("time"), vmean.values), 49 | "min": (("time"), vminimum.values), 50 | "max": (("time"), vmaximum.values), 51 | "count": (("time"), vcount.values), 52 | "variance": (("time"), vvariance.values), 53 | "median": (("time"), vmedian.values), 54 | "rms": (("time"), vrms.values), 55 | }, 56 | {"time": vmean['time']}, 57 | ) 58 | 59 | ds.to_netcdf(output_filename, group='diff') 60 | 61 | 62 | # convert data vector and time vector into xarray.Dataarray 63 | da = xr.DataArray(ssh_alongtrack, coords=[time_vector], dims="time") 64 | 65 | # resample 66 | da_resample = da.resample(time=freq) 67 | 68 | # compute stats 69 | vmean = da_resample.mean() 70 | vminimum = da_resample.min() 71 | vmaximum = da_resample.max() 72 | vcount = da_resample.count() 73 | vvariance = da_resample.var() 74 | vmedian = da_resample.median() 75 | vrms = np.sqrt(np.square(da).resample(time=freq).mean()) 76 | 77 | rms_alongtrack = np.copy(vrms) 78 | 79 | # save stat to dataset 80 | ds = xr.Dataset( 81 | { 82 | "mean": (("time"), vmean.values), 83 | "min": (("time"), vminimum.values), 84 | "max": (("time"), vmaximum.values), 85 | "count": (("time"), vcount.values), 86 | "variance": (("time"), vvariance.values), 87 | "median": (("time"), vmedian.values), 88 | "rms": (("time"), vrms.values), 89 | }, 90 | {"time": vmean['time']}, 91 | ) 92 | 93 | ds.to_netcdf(output_filename, group='alongtrack', mode='a') 94 | 95 | 96 | # convert data vector and time vector into xarray.Dataarray 97 | da = xr.DataArray(ssh_map_interp, coords=[time_vector], dims="time") 98 | 99 | # resample 100 | da_resample = da.resample(time=freq) 101 | 102 | # compute stats 103 | vmean = da_resample.mean() 104 | vminimum = da_resample.min() 105 | vmaximum = da_resample.max() 106 | vcount = da_resample.count() 107 | vvariance = da_resample.var() 108 | vmedian = da_resample.median() 109 | vrms = np.sqrt(np.square(da).resample(time=freq).mean()) 110 | 111 | # save stat to dataset 112 | ds = xr.Dataset( 113 | { 114 | "mean": (("time"), vmean.values), 115 | "min": (("time"), vminimum.values), 116 | "max": (("time"), vmaximum.values), 117 | "count": (("time"), vcount.values), 118 | "variance": (("time"), vvariance.values), 119 | "median": (("time"), vmedian.values), 120 | "rms": (("time"), vrms.values), 121 | }, 122 | {"time": vmean['time']}, 123 | ) 124 | 125 | ds.to_netcdf(output_filename, group='maps', mode='a') 126 | 127 | logging.info(' ') 128 | logging.info(f' Results saved in: {output_filename}') 129 | 130 | rmse_score = 1. - rmse/rms_alongtrack 131 | # mask score if nb obs < nb_min_obs 132 | nb_min_obs = 10 133 | rmse_score = np.ma.masked_where(vcount.values < nb_min_obs, rmse_score) 134 | 135 | mean_rmse = np.ma.mean(np.ma.masked_invalid(rmse_score)) 136 | std_rmse = np.ma.std(np.ma.masked_invalid(rmse_score)) 137 | 138 | logging.info(' ') 139 | logging.info(f' MEAN RMSE Score = {mean_rmse}') 140 | logging.info(' ') 141 | logging.info(f' STD RMSE Score = {std_rmse}') 142 | 143 | return mean_rmse, std_rmse --------------------------------------------------------------------------------