├── .gitignore ├── images └── logo.png ├── contributing.md ├── LICENSE └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | .vscode 2 | -------------------------------------------------------------------------------- /images/logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DahnJ/Awesome-Zarr/HEAD/images/logo.png -------------------------------------------------------------------------------- /contributing.md: -------------------------------------------------------------------------------- 1 | Contributions are most welcome! 2 | 3 | ## Adding something to an awesome list 4 | 5 | If you have something awesome to contribute to an awesome list, this is how you do it. 6 | 7 | You'll need a [GitHub account](https://github.com/join)! 8 | 9 | 1. Access the [awesome list's GitHub page](https://github.com/DahnJ/Awesome-Zarr). 10 | 2. Click on the `readme.md` file: ![Step 2 Click on Readme.md](https://cloud.githubusercontent.com/assets/170270/9402920/53a7e3ea-480c-11e5-9d81-aecf64be55eb.png) 11 | 3. Now click on the edit icon. ![Step 3 - Click on Edit](https://cloud.githubusercontent.com/assets/170270/9402927/6506af22-480c-11e5-8c18-7ea823530099.png) 12 | 4. You can start editing the text of the file in the in-browser editor. Make sure you follow the guidelines above. You can use [GitHub Flavored Markdown](https://help.github.com/articles/github-flavored-markdown/). ![Step 4 - Edit the file](https://cloud.githubusercontent.com/assets/170270/9402932/7301c3a0-480c-11e5-81f5-7e343b71674f.png) 13 | 5. Say why you're proposing the changes, and then click on "Propose file change". ![Step 5 - Propose Changes](https://cloud.githubusercontent.com/assets/170270/9402937/7dd0652a-480c-11e5-9138-bd14244593d5.png) 14 | 6. Submit the [pull request](https://help.github.com/articles/using-pull-requests/)! -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Creative Commons Legal Code 2 | 3 | CC0 1.0 Universal 4 | 5 | CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE 6 | LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN 7 | ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS 8 | INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES 9 | REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS 10 | PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM 11 | THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED 12 | HEREUNDER. 13 | 14 | Statement of Purpose 15 | 16 | The laws of most jurisdictions throughout the world automatically confer 17 | exclusive Copyright and Related Rights (defined below) upon the creator 18 | and subsequent owner(s) (each and all, an "owner") of an original work of 19 | authorship and/or a database (each, a "Work"). 20 | 21 | Certain owners wish to permanently relinquish those rights to a Work for 22 | the purpose of contributing to a commons of creative, cultural and 23 | scientific works ("Commons") that the public can reliably and without fear 24 | of later claims of infringement build upon, modify, incorporate in other 25 | works, reuse and redistribute as freely as possible in any form whatsoever 26 | and for any purposes, including without limitation commercial purposes. 27 | These owners may contribute to the Commons to promote the ideal of a free 28 | culture and the further production of creative, cultural and scientific 29 | works, or to gain reputation or greater distribution for their Work in 30 | part through the use and efforts of others. 31 | 32 | For these and/or other purposes and motivations, and without any 33 | expectation of additional consideration or compensation, the person 34 | associating CC0 with a Work (the "Affirmer"), to the extent that he or she 35 | is an owner of Copyright and Related Rights in the Work, voluntarily 36 | elects to apply CC0 to the Work and publicly distribute the Work under its 37 | terms, with knowledge of his or her Copyright and Related Rights in the 38 | Work and the meaning and intended legal effect of CC0 on those rights. 39 | 40 | 1. Copyright and Related Rights. A Work made available under CC0 may be 41 | protected by copyright and related or neighboring rights ("Copyright and 42 | Related Rights"). Copyright and Related Rights include, but are not 43 | limited to, the following: 44 | 45 | i. the right to reproduce, adapt, distribute, perform, display, 46 | communicate, and translate a Work; 47 | ii. moral rights retained by the original author(s) and/or performer(s); 48 | iii. publicity and privacy rights pertaining to a person's image or 49 | likeness depicted in a Work; 50 | iv. rights protecting against unfair competition in regards to a Work, 51 | subject to the limitations in paragraph 4(a), below; 52 | v. rights protecting the extraction, dissemination, use and reuse of data 53 | in a Work; 54 | vi. database rights (such as those arising under Directive 96/9/EC of the 55 | European Parliament and of the Council of 11 March 1996 on the legal 56 | protection of databases, and under any national implementation 57 | thereof, including any amended or successor version of such 58 | directive); and 59 | vii. other similar, equivalent or corresponding rights throughout the 60 | world based on applicable law or treaty, and any national 61 | implementations thereof. 62 | 63 | 2. Waiver. To the greatest extent permitted by, but not in contravention 64 | of, applicable law, Affirmer hereby overtly, fully, permanently, 65 | irrevocably and unconditionally waives, abandons, and surrenders all of 66 | Affirmer's Copyright and Related Rights and associated claims and causes 67 | of action, whether now known or unknown (including existing as well as 68 | future claims and causes of action), in the Work (i) in all territories 69 | worldwide, (ii) for the maximum duration provided by applicable law or 70 | treaty (including future time extensions), (iii) in any current or future 71 | medium and for any number of copies, and (iv) for any purpose whatsoever, 72 | including without limitation commercial, advertising or promotional 73 | purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each 74 | member of the public at large and to the detriment of Affirmer's heirs and 75 | successors, fully intending that such Waiver shall not be subject to 76 | revocation, rescission, cancellation, termination, or any other legal or 77 | equitable action to disrupt the quiet enjoyment of the Work by the public 78 | as contemplated by Affirmer's express Statement of Purpose. 79 | 80 | 3. Public License Fallback. Should any part of the Waiver for any reason 81 | be judged legally invalid or ineffective under applicable law, then the 82 | Waiver shall be preserved to the maximum extent permitted taking into 83 | account Affirmer's express Statement of Purpose. In addition, to the 84 | extent the Waiver is so judged Affirmer hereby grants to each affected 85 | person a royalty-free, non transferable, non sublicensable, non exclusive, 86 | irrevocable and unconditional license to exercise Affirmer's Copyright and 87 | Related Rights in the Work (i) in all territories worldwide, (ii) for the 88 | maximum duration provided by applicable law or treaty (including future 89 | time extensions), (iii) in any current or future medium and for any number 90 | of copies, and (iv) for any purpose whatsoever, including without 91 | limitation commercial, advertising or promotional purposes (the 92 | "License"). The License shall be deemed effective as of the date CC0 was 93 | applied by Affirmer to the Work. Should any part of the License for any 94 | reason be judged legally invalid or ineffective under applicable law, such 95 | partial invalidity or ineffectiveness shall not invalidate the remainder 96 | of the License, and in such case Affirmer hereby affirms that he or she 97 | will not (i) exercise any of his or her remaining Copyright and Related 98 | Rights in the Work or (ii) assert any associated claims and causes of 99 | action with respect to the Work, in either case contrary to Affirmer's 100 | express Statement of Purpose. 101 | 102 | 4. Limitations and Disclaimers. 103 | 104 | a. No trademark or patent rights held by Affirmer are waived, abandoned, 105 | surrendered, licensed or otherwise affected by this document. 106 | b. Affirmer offers the Work as-is and makes no representations or 107 | warranties of any kind concerning the Work, express, implied, 108 | statutory or otherwise, including without limitation warranties of 109 | title, merchantability, fitness for a particular purpose, non 110 | infringement, or the absence of latent or other defects, accuracy, or 111 | the present or absence of errors, whether or not discoverable, all to 112 | the greatest extent permissible under applicable law. 113 | c. Affirmer disclaims responsibility for clearing rights of other persons 114 | that may apply to the Work or any use thereof, including without 115 | limitation any person's Copyright and Related Rights in the Work. 116 | Further, Affirmer disclaims responsibility for obtaining any necessary 117 | consents, permissions or other rights required for any use of the 118 | Work. 119 | d. Affirmer understands and acknowledges that Creative Commons is not a 120 | party to this document and has no duty or obligation with respect to 121 | this CC0 or use of the Work. 122 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Zarr 2 | [![Awesome](https://awesome.re/badge.svg)](https://awesome.re) 3 | 4 | drawing 5 | 6 | 7 | [Zarr](https://zarr.dev/) is a cloud-native, chunked, compressed, and hierarchical array data format. 8 | 9 | 10 | # Contents 11 | [Resources](#resources) 12 | - [Existing resources](#existing-resources) 13 | - [Introductory videos](#introductory-videos) 14 | - [Zarr V3](#zarr-v3) 15 | - [Libraries](#libraries) 16 | - [Platforms](#platforms) 17 | - [Articles](#articles) 18 | - [Talks & Videos](#talks--videos) 19 | - [Life sciences](#life-sciences) 20 | 21 | [Topics](#topics) 22 | - [Zarr & other array data formats](#zarr--other-array-data-formats) 23 | - [GeoZarr](#geozarr) 24 | - [Zarr & STAC](#zarr--stac) 25 | 26 | 27 | # Resources 28 | 29 | ## Existing resources 30 | The [Zarr website](https://zarr.dev/) is already an excellent resource for learning about Zarr and its ecosystem. 31 | This list is intended to complement the website with a curated and opinionated list of resources. 32 | 33 | This list focuses on Geo/Earth Sciences, but is not limited to that domain. 34 | 35 | ### Existing lists 36 | Lists 37 | - The Zarr website already contains great lists: [Zarr Implementations](https://zarr.dev/implementations/), [Zarr Datasets](https://zarr.dev/datasets/), [Zarr metadata conventions](https://zarr.dev/conventions/) 38 | - [Zarr tutorials (zarr-developers/tutorials)](https://github.com/zarr-developers/tutorials) 39 | - [Projects using Zarr (zarr-developers/community#19)](https://github.com/zarr-developers/community/issues/19) 40 | - [Beautiful Zarr (zarr-developers/beautiful-zarr)](https://github.com/zarr-developers/beautiful-zarr) 41 | - See playlists & lists in [Talks & Videos](#talks--videos) 42 | 43 | ## Introductory videos 44 | [Introductory talks Youtube playlist](https://www.youtube.com/playlist?list=PLvkeNUPrCU04Xvcph4ErxsRkZq28Oucr7) 45 | 46 | Two excellent and up-to-date introductory talks: 47 | - [Sanket Verma: The Beauty of Zarr](https://www.youtube.com/watch?v=OYaMi9WnQpA&ab_channel=PyData) 48 | - [Ryan Abernathey: State of Zarr](https://www.youtube.com/watch?v=1owtGSkNLQU&ab_channel=Cloud-NativeGeospatialFoundation) 49 | 50 | ## Zarr V3 51 | [Zarr V3](https://zarr-specs.readthedocs.io/en/latest/v3/core/v3.0.html) is the upcoming version of Zarr. It is a major update that will bring many new features and improvements. 52 | 53 | If you're getting into Zarr now, it might be a good idea to start with Zarr V3. 54 | - [Zarr-Python 3 and why you should be excited!](https://www.youtube.com/watch?v=g0nq4TtWuhM&ab_channel=Pangeo) 55 | 56 | For an excellent in-depth overview, see the [ESIP](https://www.esipfed.org/) series of talks 57 | - [2023-03-27 ESIP Cloud Computing Cluster: Zarr - The Next Generation](https://www.youtube.com/watch?v=50_LwbIUXi0&ab_channel=ESIP) 58 | - [2023-04-24 ESIP Cloud Computing Cluster: Next Generation of Zarr Part 2/3 GeoZarr and Zarr Sharding](https://www.youtube.com/watch?v=a4-vmJRQcrg&ab_channel=ESIP) 59 | - [2023-05-22 ESIP CCC: Next Gen Zarr Part 3/3: accumulation proposal, Kerchunk and Pangeo-Forge](https://www.youtube.com/watch?v=ROsHdJI3-yw&ab_channel=ESIP) 60 | 61 | ## Libraries 62 | This list contains libraries that directly relate to Zarr in some way. 63 | 64 | For implementations of Zarr, see [Zarr Implementations](https://zarr.dev/implementations/). 65 | - [kerchunk](https://fsspec.github.io/kerchunk/), see [kerchunk section](#kerchunk) 66 | - [xpublish](https://github.com/xpublish-community/xpublish): Exposing as and consuming Zarr through a REST API 67 | - See also routers at [xpublish-community](https://github.com/xpublish-community), e.g. [xpublish-opendap](https://github.com/xpublish-community/xpublish-opendap) 68 | - [Improving Access to NOAA NOS Model Data with Kerchunk and Xpublish](https://www.youtube.com/watch?v=Bxkg6LJpKyc&ab_channel=Pangeo) 69 | - [ndpyramid](https://github.com/carbonplan/ndpyramid): utility for generating ND array pyramids using Xarray and Zarr 70 | 71 | Storage & I/O 72 | - [Tensorstore](https://google.github.io/tensorstore/) and [xarray-tensorstore](https://github.com/google/xarray-tensorstore/): library for efficiently reading and writing large multi-dimensional arrays, has Zarr API 73 | - [KivkIO](https://github.com/rapidsai/kvikio/): C++ and Python bindings to [cuFile](https://docs.nvidia.com/gpudirect-storage/api-reference-guide/index.html), enabling [GPUDirect Storage](https://developer.nvidia.com/blog/gpudirect-storage/) 74 | - [rechunker](https://github.com/pangeo-data/rechunker): disk-to-disk transformation for chunked arrays 75 | - [xpartition](https://github.com/spencerkclark/xpartition): writing large xarray datasets to Zarr. Works around shortcomings of Dask ([distributed#6360](https://github.com/dask/distributed/issues/6360)) 76 | 77 | ETL 78 | - [Xarray](https://docs.xarray.dev/en/stable/index.html): Zarr is commonly written and accessed through xarray's API. 79 | - Xarray has its own [Zarr Encoding Specification](https://docs.xarray.dev/en/stable/internals/zarr-encoding-spec.html) 80 | - [xarray-beam](https://github.com/google/xarray-beam/): Integration of [xarray](https://docs.xarray.dev/en/stable/index.html) and [Apache Beam](https://beam.apache.org/) built using Zarr. 81 | - [Pangeo-forge](https://pangeo-forge.org/): Open-source data platform for transforming datasets into [analysis-ready cloud-optimized](https://ieeexplore.ieee.org/document/9354557) formats. 82 | - See [Pangeo Forge in 4 minutes](https://www.youtube.com/watch?v=qfAC-emFUxc&ab_channel=CharlesStern) and [Pangeo Forge: Crowdsourcing Open Data in the Cloud- Ryan Abernathey | SciPy 2022](https://www.youtube.com/watch?v=sY20UpYCAEE&ab_channel=Enthought) 83 | 84 | Developer-oriented 85 | - [numcodecs](https://github.com/zarr-developers/numcodecs): Compression and transformation codecs used by Zarr 86 | - [pydantic-zarr](https://github.com/janelia-cellmap/pydantic-zarr): Pydantic models for Zarr objects 87 | - [traverzarr](https://github.com/xaviernogueira/traverzarr): Traversing Zarr JSON as if it's a filesystem 88 | - [zarr_checksum](https://github.com/dandi/zarr_checksum): Calculating checksum information form Zarr 89 | - [zarrdump](https://github.com/oliverwm1/zarrdump): Describe zarr stores from the command line 90 | 91 | Visualization: For tools & libraries for visualization, see [visualization section](#visualization) 92 | 93 | ### Kerchunk 94 | [Kerchunk](https://fsspec.github.io/kerchunk/) allows you to efficiently read chunked data formats such as GRID, NetCDF, COGs by exposing them as a Zarr store. 95 | 96 | Talks and tutorials 97 | - [All you need is Zarr](https://www.youtube.com/watch?v=0bqpxX3Nn_A) 98 | - [2022 ESIP Kerchunk Tutorial](https://github.com/lsterzinger/2022-esip-kerchunk-tutorial) 99 | - [Accessing NetCDF and GRIB file collections as cloud-native virtual datasets using Kerchunk](https://www.youtube.com/watch?v=cHYQuufXkP8&ab_channel=Pangeo) 100 | #### Future of Kerchunk 101 | In the future, Kerchunk will be split into upstream functionality in Zarr itself and a new [VirtualiZarr](https://github.com/TomNicholas/VirtualiZarr) package. 102 | - [Kerchunk JSON references](https://fsspec.github.io/kerchunk/spec.html) will become a part of the [Chunk manifest](https://github.com/zarr-developers/zarr-specs/issues/287) 103 | - For a full overview, see [Upstreaming Kerchunk](https://hackmd.io/t9Myqt0HR7O0nq6wiHWCDA?view) 104 | - [What's Next for Kerchunk](https://www.youtube.com/watch?v=sYgZkqTAgYk&ab_channel=Pangeo) 105 | 106 | ## Platforms 107 | - [Arraylake](https://docs.earthmover.io/): a data lake platform based on Zarr. The company, [Earthmover](https://earthmover.io) was started by core Zarr developers. 108 | 109 | ## Articles 110 | - [NASA IMPACT: Zarr Visualization Report](https://nasa-impact.github.io/zarr-visualization-report/) 111 | - [Earthmover: cloud-native data loaders for machine learning using zarr and xarray](https://earthmover.io/blog/cloud-native-dataloader) 112 | - [Zarr Sprint Recap](https://zarr.dev/blog/zarr-sprint-2024/) relevant overviews 113 | 114 | ## Talks & Videos 115 | Existing lists 116 | - [Zarr Developers playlists](https://www.youtube.com/@zarr_dev/playlists), namely 117 | - [Zarr: Introductory Talks](https://www.youtube.com/playlist?list=PLvkeNUPrCU04Xvcph4ErxsRkZq28Oucr7) 118 | - [Zarr: Zarr: Projects, Uses, Research and Workflows](https://www.youtube.com/playlist?list=PLvkeNUPrCU05qHkZso_T74yoayqLFHzkI) 119 | - [Zarr Talks](https://zarr.dev/blog/zarr-talks/) 120 | - [Introductory videos](#introductory-videos) in this list 121 | 122 | Talks 123 | - [Earthmover Webinar: Building a Planetary Scale Earth Observation Data Cube in Zarr ](https://vimeo.com/935616988/d9de7a97e9) with [code repository](https://github.com/earth-mover/serverless-datacube-demo) and [slides](https://app.hubspot.com/documents/22822350/view/788058450?accessId=9423df) 124 | - [Earthmover Webinar: Analysis-ready Weather Forecast Data Cubes with Zarr 125 | ](https://www.youtube.com/watch?v=JwsrPVJPmag&ab_channel=Earthmover) with [code repository](https://github.com/earth-mover/forecast-datacube-demo) and [slides](https://app.hubspot.com/documents/22822350/view/814797193?accessId=e3acdd) 126 | - [Presentation | Zarr: Community specification of large, cloud-optimised, N-dimensional, typed array storage](https://zenodo.org/records/8221177) 127 | - Presentations for Sanket Verma's talks: [SciPy 2023](https://github.com/MSanKeys963/presentations/tree/main/scipy_2023) and [PyCon DE 2023](https://github.com/MSanKeys963/presentations/tree/main/pycon_de_pydata_berlin_2023) 128 | 129 | ## Life sciences 130 | Zarr has seen great adoption in the life sciences domain. 131 | 132 | - [bdz](https://github.com/openssbd/bdz): Zarr-based format for storing quantitative biosystems dynamics data 133 | - [ome-zarr-py](https://github.com/ome/ome-zarr-py): Implementation of next-generation file format ([NGFF](https://ngff.openmicroscopy.org/)) specifications for storing bioimaging data in the cloud. 134 | - [ez_zarr](https://github.com/fmicompbio/ez_zarr): Easy, high-level access to OME-Zarr filesets 135 | - [hdmf-zarr](https://github.com/hdmf-dev/hdmf-zarr): Zarr I/O backend for [HDMF](https://hdmf.readthedocs.io/en/stable/) 136 | 137 | Talks and resources 138 | - [Zarr | Life Science Lightning Talk | Trevor Manz | Dask Summit 2021](https://www.youtube.com/watch?v=8TlAAZcJnvA&list=PLvkeNUPrCU04Xvcph4ErxsRkZq28Oucr7&index=3&ab_channel=Dask) 139 | - [Accelerating Single-cell Bioinformatics with N-dimensional Arrays in the Cloud | ISMMS](https://www.youtube.com/watch?v=cYSYoiAVoEE) 140 | - [What are next-generation file formats (NGFF)?](https://gerbi-gmb.de/2023/10/02/next-generation-file-formats-for-bioimaging/) 141 | 142 | ### Visualization 143 | Zarr has seen most work on visualization in the bioimaging community: 144 | - List: [Image viewers with OME-Zarr support](https://ngff.openmicroscopy.org/tools/) 145 | - [WEBKNOSSOS](https://webknossos.org/): web-based visualization & annotation tool, supports OME-Zarr 146 | - [Napari](https://napari.org/stable/): interactive viewer 147 | - [Vizarr](https://github.com/hms-dbmi/vizarr): interactive viewer built using [viv](https://github.com/hms-dbmi/viv) (OME-Zarr and OME-TIFF) 148 | - [Neuroglancer](https://github.com/google/neuroglancer): WebGL-based viewer for volumetric data 149 | - [BigDataViewer](https://imagej.net/plugins/bdv/) 150 | 151 | # Topics 152 | ## Zarr & other array data formats 153 | 154 | For a general overview, see 155 | - [Introduction to Cloud-Native Geospatial Formats](https://www.youtube.com/watch?v=zMPFHwW7Ujo&ab_channel=Cloud-NativeGeospatialFoundation) 156 | - [Cloud-Optimized Geospatial Formats Guide](https://guide.cloudnativegeo.org/). 157 | 158 | Essentially all other common array data formats can be exposed as Zarr. See [Kerchunk](#kerchunk). 159 | 160 | ### NetCDF & HDF5 161 | 162 | Zarr, NetCDF, and HDF5 are three separate data formats that nonetheless relate to each other in multiple ways. 163 | - Zarr inherits its hierarchical structure from HDF5. 164 | - Zarr is commonly accessed through [xarray](https://docs.xarray.dev/en/stable/index.html), whose [data models are based on the NetCDF data format](https://docs.xarray.dev/en/stable/getting-started-guide/why-xarray.html#core-data-structures) 165 | - NetCDF4 can [use HDF5 as a backend](https://www.bic.mni.mcgill.ca/users/sean/Docs/netcdf/guide.txn_85.html) 166 | - [NCZarr](https://docs.unidata.ucar.edu/nug/current/nczarr_head.html) is an extension of the Zarr format to map it to a subset of the NetCDF data model. 167 | 168 | Resources 169 | - [A Comparison of HDF5, Zarr, and netCDF4 in Performing Common I/O Operations](https://arxiv.org/abs/2207.09503) 170 | HDF5 171 | - [Pangeo: HDF5 at the speed of Zarr](https://www.youtube.com/watch?v=iRboOFIB74o&ab_channel=Pangeo) 172 | - [Joe Jevnik: Zarr vs. HDF5 | PyData New York 2019](https://www.youtube.com/watch?v=-l445lCPTts&ab_channel=PyData) 173 | 174 | ### COG: Cloud-Optimized GeoTIFF 175 | - [Overview: Zarr - A Cloud Native ND Array Format @ 19:17](https://youtu.be/KiiKvXzhyMs?si=4sEqZ00lcl0WNXUb&t=1157) 176 | - [COG and Zarr for Geospatial Data](https://paper.dropbox.com/doc/COG-and-Zarr-for-Geospatial-Data--CNQFg0s2jhdrXjbNGDQ7Y~qgAg-UyeMJVOn5PqcNJoifLQiV) 177 | 178 | ### N5 179 | Zarr and N5 are two similar array data formats that share common goals and development. 180 | 181 | The Zarr V3 spec aims to provide a common implementation target (sources: [1](https://zarr.dev/zarr/specs/2019/06/19/zarr-v3-update.html), [2](https://zarr-specs.readthedocs.io/en/latest/v3/core/v3.0.html)) 182 | 183 | Links 184 | - [n5](https://github.com/saalfeldlab/n5) 185 | - [zarr.n5](https://zarr.readthedocs.io/en/stable/api/n5.html) 186 | - [z5](https://github.com/constantinpape/z5): C++ and Python interface for datasets in zarr and n5 format 187 | - [Zarr N5 spec diff (zarr-specs#3)](https://github.com/zarr-developers/zarr-specs/issues/3) 188 | 189 | ## GeoZarr 190 | GeoZarr is a proposal for a Zarr-based geospatial data format, being submitted as an [OGC standard](https://www.ogc.org/standards/) 191 | 192 | GeoZarr will define a [metadata convention](https://zarr.dev/conventions/) for Zarr stores that contain geospatial data. 193 | 194 | It will also define the relationship of Zarr with [CF](https://cfconventions.org/) and NetCDF 195 | 196 | Links 197 | - [Specs](https://github.com/zarr-developers/geozarr-spec) 198 | - [Current status of GeoZarr](https://www.youtube.com/watch?v=arjEaBwXRxg&ab_channel=Cloud-NativeGeospatialFoundation) 199 | ## Zarr & STAC 200 | [STAC](https://stacspec.org/en/) provides a common structure for describing and cataloging spatiotemporal assets. 201 | 202 | With its hierarchical structure and key-value metadata support, Zarr's capabilities overlap significantly with STAC. 203 | 204 | The communities have not yet converged on a canonical representation of Zarr datasets through STAC. 205 | 206 | Today, a good example of exposing Zarr in STAC is Planetary Computer 207 | - [Reading Zarr Data](https://planetarycomputer.microsoft.com/docs/quickstarts/reading-zarr-data/) 208 | - STAC collection: [Daymet Annual North America](https://planetarycomputer.microsoft.com/dataset/daymet-annual-na) 209 | - STAC collection: [CIL Global Downscaled Projections for Climate Impacts Research](https://planetarycomputer.microsoft.com/dataset/group/cil-gdpcir) 210 | - [xstac](https://github.com/stac-utils/xstac/tree/main?tab=readme-ov-file): STAC from xarray 211 | - Related STAC extensions: [xarray-assets](https://github.com/stac-extensions/xarray-assets), [datacube](https://github.com/stac-extensions/datacube) 212 | 213 | More discussion & Related links 214 | - [Pangeo: Metadata duplication on STAC zarr collections](https://discourse.pangeo.io/t/metadata-duplication-on-stac-zarr-collections/3193/6) 215 | - [geozarr-spec#32: Integration of Zarr with STAC Catalogs](https://github.com/zarr-developers/geozarr-spec/issues/32) 216 | - [stac-spec#781: Zarr Extension?](https://github.com/radiantearth/stac-spec/issues/781) 217 | - [Tom Augspurper: STAC and Kerchunk](https://tomaugspurger.net/posts/stac-updates/#stac-and-kerchunk) 218 | - [Presentation | Daniel Jahn – STAC vs Zarr](https://cloud.mohr.ws/index.php/s/8oyTgc2kRK6QQcC?dir=undefined&openfile=8487) 219 | - [Arraylake](https://docs.earthmover.io/) a data lake platform that is arguably the first example of a pure Zarr data catalog 220 | 221 | In the future, the [Zarr V3 Spec](#zarr-v3) and [GeoZarr convention](#geozarr) will likely enable greater interoperability between STAC and Zarr. 222 | --------------------------------------------------------------------------------