Zarr is a community project to develop specifications and software for storage of large N-dimensional typed arrays, also commonly known as tensors. A particular focus of Zarr is to provide support for storage using distributed systems like cloud object stores, and to enable efficient I/O for parallel computing applications.
Zarr is motivated by the need for a simple, transparent, open, and community-driven format that supports high-throughput distributed I/O on different storage systems. Zarr data can be stored in any storage system that can be represented as a key-value store, including most commonly POSIX file systems and cloud object storage but also zip files as well as relational and document databases.
See the following GitHub repositories for more information:
- Multi-scale n-dimensional image storage, e.g. in light and electron microscopy
- Geospatial rasters, e.g. following the NetCDF / CF metadata conventions
- Chunk multi-dimensional arrays along any dimension.
- Store arrays in memory, on disk, inside a Zip file, on S3, etc.
- Read and write arrays concurrently from multiple threads or processes.
- Organize arrays into hierarchies via annotatable groups.
Zarr is a Sponsored Project of NumFOCUS, a US 501(c)(3) public charity.
NumFOCUS Sponsored Projects rely on the generous support of corporate sponsors, institutional partners, and individual donors.
Videos of community members talking about Zarr. If you have a video you’d like us to share, let us know!
ZEP stands for Zarr Enhancement Proposal. A ZEP is a design document providing information to the Zarr community, describing a modification or enhancement of the Zarr specifications, a new feature for its processes or environment.
More info on ZEPs is available at: https://zarr.dev/zeps/
We hold bi-weekly ZEPs meetings to propose, discuss, review and finalize discussions around current ZEPs and Zarr Specification. More info available here: https://zarr.dev/zeps/meetings/
Please note ZEPs meetings are different from bi-weekly community meetings. The Zarr Community meets two times every two weeks. Please have a look at our community calendar for exact timings.
Zarr community meeting notes, joining instructions, agenda and calendar are now available at: https://zarr.dev/community-calls
Check out the newly launched Zarr blog here.