2023-09-20
Attending: Eric Perlman (EP), Sanket Verma (SV), Norman Rzepka (NR), Nathan Zimmerberg (NZ), Gabor Kovacs (GB), Dennis Heimbigner (DH), Ward Fisher (WF), Martin Durrant (MD), Davis Bennett (DB)
TL;DR:
SV kicked off the meeting with intros. NZ had some questions about the current state of V3 implementation. MD wants to push ZEP0003 for technical review and voting. DB has submitted a new ZEP, i.e. ZOM (Zarr Object Model). SV gave important updates from NumFOCUS Summit 2023.
Updates:
- First meeting for Zarr-Python Benchmarking & Peformance team
- ZEP0006 (by Davis Bennett) : https://github.com/zarr-developers/zeps/pull/46
Meeting Minutes:
- Intros (favourite tourist place)
- SV: Himalayas
- NR: Highway One
- EP: New Zealand (South Island)
- GK: Allen Institute works in Light sheet microscopy - favourite tourist place: Pacific North West
- NZ: University of Maryland, PhD in bioinformatics - Shenandoah Park
- WF: Pacific North West and Astoria, Oregon
- DH: Highway One - took a 1.5 days to complete it - but also took many stops
- MD: Canary Islands in the Atlantic
- NZ: Current state of the V3 implementation
- SV: Currently WIP
- Use Zarrita to convert V2 to V3?
- NR: Yes!
- NZ: Codec name change for byte to endian - is it going in?
- NR: Can take place along with a voting
- It simply takes an array and spits out bytes - V3 requires you to have a codec
- NZ: Has written Julia implementation apart Zarr.jl - mostly a prototype implementation
- Trying to merge what’s already in Zarr.jl
- MD: Pangeo meeting - They were wondering when it’ll get finalised
- MD: Done all the things that was asked - now what?
- SV: Will put it up for technical review and voting process
- NR: Sending a PR to Zarrita can help in the acceptance
- DB: ZEP0006 - brief overview
JSON
serialisable reprenstation of a Zarr hierarchy- Providing a foundation for declarative API rather than procedural
- SV: Updates from the NumFOCUS Summit 2023
- Zarr Shards ↔ HDF5 files (https://github.com/zarr-developers/zarr-specs/pull/152#issuecomment-1632034874)
- Xvec using Zarr for storage - https://github.com/xarray-contrib/xvec/issues/48#issuecomment-1716000623
- MD: String storage is kind of ragged structure - also they’d store the strings using variable chunking, i.e. ZEP0003