2024-02-07

Attending: Davis Bennett (DB), Sanket Verma (SV), Josh Moore (JM), Norman Rzepka (NR), Eric Perlman (EP), Gābor Kovācs (GB), Jeremy Maitin-Shepard (JMS)

TL;DR:

Meeting covers Zarr Sprint updates, GSoC participation plans, and SPEC0 endorsement. Discussions include dataset conversion to Zarr V3, proposals for OME-NGFF integration, impending changes in Zarr V3, Java support, chat platform considerations, and collaboration for SciPy 2024 proposal.

Updates:

• Zarr Sprint @ Columbia University, NYC (taking place right now):
  • Agenda and links: https://docs.google.com/document/d/1x62xVWxcjJJHQWdNE5jPHkVzh5k6_HfoifUJ9v-AwPA/edit?usp=sharing
  • Slack: https://join.slack.com/t/cloudnativegeo/shared_invite/zt-235w8flfo-TW5Tpi1sPqQFWeMy~7ROHA
• Participating in GSoC 2024
  • Ideas list: https://github.com/zarr-developers/gsoc/blob/main/2024/ideas-list.md
• SPEC0 endorsement

Meeting Minutes:

• EP: Converted a huge dataset into Zarr - storing the data on GCS
  • DB: Using Zarr V3 and converting to shards
  • NR: If you’re already have in Zarr then it’s easy to convert it into V3/Shards
  • Links:
    • https://images.jax.org/webclient/?show=image-190714
    • https://storage.googleapis.com/jax-public-ngff/data/0.4/public_data/3580/sms_107/2022-07/20/12-04-09.625/3215%20D%20-%202022-06-15%2012.09.50.zarr/0/
    • https://ome.github.io/ome-ngff-validator/?source=https://storage.googleapis.com/jax-public-ngff/data/0.4/public_data/3580/sms_107/2022-07/20/12-04-09.625/3215%20D%20-%202022-06-15%2012.09.50.zarr/0/
  • JM: Some benchmarks
    • https://github.com/ome/bioimage-latency-benchmark
    • https://www.nature.com/articles/s41592-021-01326-w
  • JMS: Neuroglancer viewer: http://tinyurl.com/yzpnxz7u
• JM: NR will write a proposal for Zarr3 in OME-NGFF to test drive the NGFF RFC process
• SV: Endorsing SPEC4
  • DB: Doesn’t see a downside
  • JMS: Zarr-Python might not need nightly wheels but maybe numcodecs do
• JM: Big V3 change is coming in - the diff. is getting bigger day by day
  • DB: Beta release in a month would be ambitious
  • SV: Joe wants to release directly from V3 branch
  • JM: Good, if they’re pre-releases
• GB: Zarr Java supporting V3
  • NR: N5 supports Zarr V2 - trying to get funds to bring Zarr-Java and N5 together
  • GB: Using the N5 approach and dependent on it and can’t switch to V3
  • DB: N5 would like to support V3
• Chat platform discussion: https://github.com/zarr-developers/community/issues/68
  • OSSCi Zulipchat has Zarr stream
  • https://ossci.zulipchat.com
  • EP: Like the idea of having a shared Zulip - don’t want to get logged in several Zulip
  • EP: Hierarchies can be useful for various topics
• SciPy 2024 proposal
  • Looking for collaborators (currently we have: Sanket and Joe)