Zarr is a community project to develop specifications and software for storage of large N-dimensional typed arrays, also commonly known as tensors. A particular focus of Zarr is to provide support for storage using distributed systems like cloud object stores, and to enable efficient I/O for parallel computing applications.

Recent Updates

  • Zarr 2.11 is released! Check out the blog. 🥂
  • Sanket Verma has officially joined Zarr as the new Community Manager! 🎉 Read the blog post here.

Description

Zarr is motivated by the need for a simple, transparent, open, and community-driven format that supports high-throughput distributed I/O on different storage systems. Zarr data can be stored in any storage system that can be represented as a key-value store, including most commonly POSIX file systems and cloud object storage but also zip files as well as relational and document databases.

See the following GitHub repositories for more information:

Applications

  • Simple and fast serialization of NumPy-like arrays, accessible from languages including Python, C, C++, Rust, Javascript and Java
  • Multi-scale n-dimensional image storage, e.g. in light and electron microscopy
  • Geospatial rasters, e.g. following the NetCDF / CF metadata conventions

Features

  • Chunk multi-dimensional arrays along any dimension.
  • Store arrays in memory, on disk, inside a Zip file, on S3, etc.
  • Read and write arrays concurrently from multiple threads or processes.
  • Organize arrays into hierarchies via annotatable groups.

Sponsorship

Zarr is a Sponsored Project of NumFOCUS, a US 501(c)(3) public charity.

NumFOCUS Sponsored Projects rely on the generous support of corporate sponsors, institutional partners, and individual donors.

Videos

Videos of community members talking about Zarr. If you have a video you’d like us to share, let us know!

Community meetings

Zarr community meetings take place every two weeks where maintainers, contributors and the overall general community get together generally for an hour to discuss ongoing issues, features, bugs, processes or anything important for the development and progress of Zarr. The agenda for the meeting is open and anyone can add items to it. The standing agenda for the community meeting is available here.

We’re always looking to discuss new ideas, implementations, research or anything of interest to Zarr. If you think you have something to share with the Zarr community please join us via this link. New contributors looking for tips and guidance are most welcome to join us. We’d be more than happy to reserve a slot for them at the end of the meeting.

A Google calendar (ics) lists the regular community meetings.

Blog

Check out the newly launched Zarr blog here.