Hi Zarr Community! 👋🏻

Recently, I and several community members have been speaking at various conferences and events. There has been an exciting development in the Zarr ecosystem, like finalising V3 specification, submitting new ZEPs, initiating new implementations, etc.

While I’m mostly giving beginner talks on Zarr, which answers how, why, and what, the enthusiastic community members have been talking about other exciting stuff!

In this blog post, I highlight a few talks which were delivered in the past two months. Also, we’re maintaining a playlist on YouTube, which has a more extensive collection of talks from various domains and diverse speakers. Check the playlists: Zarr: Introductory Talks and Zarr: Projects, Uses, Research and Workflows.

PyCon DE and PyData Berlin 2023 🇩🇪

I went to Berlin, Germany, in April to speak at PyCon DE and PyData Berlin 2023. My talk was titled “The Beauty of Zarr”, where I emphasised the inner workings using some near illustrations by Trevor Manz. I highlighted how simple, convenient and hackable it is to use Zarr. After going through various explanations, I focused on some critical issues that Zarr eradicates because of its design and workings, i.e. chunking, compression, cloud-enabled etc.

Towards the end, I prepared a Jupyter notebook where I walked through Zarr 101 code to create, read, write and manipulate arrays. I also converted the Zarr pixelated logo from .png to .zarr format, which was a neat closing for my talk.

The slides and notebook can be accessed here.

Please watch the video here: 👇🏻

ESIP Meetings 🌏

Earth Science Information Partners (ESIP) is a community of data and information technology practitioners working together to coordinate earth science interoperability efforts. ESIP has various collaboration areas. ESIP Collaboration areas are made up of administrative committees and small working groups that are called clusters. Some of them are:

  • Agriculture & Climate
  • Open Science
  • Cloud Computing
  • Soli Ontology and Informatics
  • Data Management Training Clearinghouse
  • Council of Data Facilities

And many more.

The ESIP Cloud Computing Cluster organised a three-part series on Zarr titled “Zarr: The Next Generation” In every part, the Zarr Community members talked about several things ranging from V3 to conventions to ZEPs.

The first part took place on March 27th where:

The video recording of the session can be seen here: 👇🏻

The second part took place on April 24th where:

The video recording of the session can be seen here:

The third part took place on May 22nd where:

The video recording of the session can be seen here:

These meetings covered a great deal of recent developments in the Zarr ecosystem. The ZEPs mentioned above explained the V3 specification, sharding, and a couple of new exciting features the community is working on. The interesting thing to note here is that the ZEP0003 and ZEP0005 are something the community members wrote to support their use-case in their domain. This shows the openness and flexibility of the Zarr open-source community and how we support everyone. Though these ZEPs are still in the draft state, they’ll be finalised soon for adoption.

I will discuss about V3 specification in a separate blog post, so I’d not go into the details here. But it’s worth noticing GeoZarr specification and what Briana presented. GeoZarr is one of the conventions on top of Zarr specification, which support various use cases of the geospatial community on how they store their data and metadata. The GeoZarr SWG (Steering Working Group) has been working quickly despite the roadblocks (as mentioned by Briana). The progress and specification can be seen here.

Conclusion

These are some of the public engagements done by the Zarr Community members in the past months. If you spoke on Zarr recently or in the past and would like me to highlight your talk, please don’t hesitate to contact me. If you’re working on something interesting which involves Zarr and want to share it with the community, please say ‘Hi’ to me!

I’ll be talking to you all soon.

Until next time, peace! ✌🏻

~Sanket Verma