2023-02-23

Attending: Sanket Verma (SV), Ward Fisher (WF), Ryan Abernathey (RA), Jonathan Striebel (JS)

TL;DR:

Finally, after several meetings, all the essential issues related to the ZEP1 (V3) are resolved! However, some minor tasks remain, which JS will be looking at in the upcoming days. Then, JS asked whether we should ship V3 with OGC and whether chunk key encoding could be an extension. RA mentioned he’s working on an extension ZEP for non-listable stores. We also discussed having a hack week to eliminate the technical gap between the zarr-python’s V3 implementation and the current V3 specification.

Updates:

Meeting Minutes:

  • JS: No items in TODOs and Needs pr in ZEP1 project board - spec is coming to final stage 🎉 - last few days at scalable minds - needs to finish the remaining tasks soon!
  • JS: Should we ship V3 with OGC?
    • RA: We already have Zarr V2 Spec as OGC standard - we can ask for v3 - but its more of take it or leave thing
  • JS: chunk key encoding could be an extension - separate key in the metadata - may prepends 0 - two things should be configurable separately
    • RA: could it be an extension?
    • JS: possibly!
    • RA: we should have it - this could enable to see only metadata when opening a directory without seeing the whole array
    • JMS: separator / would allow multiple possibilities
    • JS: should be backwards compatible, not a breaking feature
  • RA: working on a extension ZEP for non - listable stores -
    • wants to run it by the community first - read only stores - no writes to these stores
    • copied from STAC - link: https://github.com/radiantearth/stac-spec/blob/master/catalog-spec/catalog-spec.md#relation-types
    • provide explicit link between parent and children document - write a store and create links for the store -
    • JMS: new group property, no reason to have parent - because you always know the parent
    • RA: what if someone gives you a middle hierarchy array address? - it’s helpful then
  • RA: Maybe we should not advertise V3 as we don’t have a reference implementation
    • SV: hack week is a good idea to get ride of the existing technical debt
    • JMS: Looking to implement V3 in tensorstore and can help with Zarr-Python too!