Zarr is a community project to develop specifications and software for storage of large N-dimensional typed arrays, also commonly known as tensors. A particular focus of Zarr is to provide support for storage using distributed systems like cloud object stores, and to enable efficient I/O for parallel computing applications.

Recent Updates

  • Zarr 2.11 is released! Check out the blog. 🥂
  • Sanket Verma has officially joined Zarr as the new Community Manager! 🎉 Read the blog post here.

Description

Zarr is motivated by the need for a simple, transparent, open, and community-driven format that supports high-throughput distributed I/O on different storage systems. Zarr data can be stored in any storage system that can be represented as a key-value store, including most commonly POSIX file systems and cloud object storage but also zip files as well as relational and document databases.

See the following GitHub repositories for more information:

Applications

  • Simple and fast serialization of NumPy-like arrays, accessible from languages including Python, C, C++, Rust, Javascript and Java
  • Multi-scale n-dimensional image storage, e.g. in light and electron microscopy
  • Geospatial rasters, e.g. following the NetCDF / CF metadata conventions

Features

  • Chunk multi-dimensional arrays along any dimension.
  • Store arrays in memory, on disk, inside a Zip file, on S3, etc.
  • Read and write arrays concurrently from multiple threads or processes.
  • Organize arrays into hierarchies via annotatable groups.

Sponsorship

Zarr is a Sponsored Project of NumFOCUS, a US 501(c)(3) public charity.

NumFOCUS Sponsored Projects rely on the generous support of corporate sponsors, institutional partners, and individual donors.

Videos

Videos of community members talking about Zarr. If you have a video you’d like us to share, let us know!

ZEPs

ZEP stands for Zarr Enhancement Proposal. A ZEP is a design document providing information to the Zarr community, describing a modification or enhancement of the Zarr specifications, a new feature for its processes or environment.

More info on ZEPs is available at: https://zarr.dev/zeps/

We hold bi-weekly ZEPs meetings to propose, discuss, review and finalize discussions around current ZEPs and Zarr Specification. More info available here: https://zarr.dev/zeps/meetings/

Please note ZEPs meetings are different from bi-weekly community meetings. The Zarr Community meets two times every two weeks. Please have a look at our community calendar for exact timings.

Community meetings

Zarr community meeting notes, joining instructions, agenda and calendar are now available at: https://zarr.dev/community-calls

Blog

Check out the newly launched Zarr blog here.