23rd March, 2022

Attending: Sanket V., Josh M., Hailey J., Greg L., Dennis H., Jim Pivarski, John K, Jeremy Maitin-Shepard

Agenda:

  • Tornadoes!
  • GSoC and GSoD Updates
    • Several people are showing up looking for good first issues.
    • Feel free to help them out, but SV & JM’ll be chatting wit them this weekend.
    • Anyone who is interested in spearheading GSOD (deadline tomorrow)
  • ZEP feedback: Governance, template,
    • JMS: propose/offer using an existing issue to test the ZEP. (:+1:)
    • e.g. Data type rename issue: https://github.com/zarr-developers/zarr-specs/issues/131
    • Description of some dtypes currently supported in Zarr-Python’s v2 implementation, but are not part of the core v3 spec: https://github.com/zarr-developers/zarr-specs/pull/135
    • Now or later? SV: will ping after another round of changes
  • V3 / awkward arrays: extension mechanism
    • https://github.com/zarr-developers/zarr-specs/issues/62
    • https://github.com/zarr-developers/zarr-specs/issues/89
    • JP: requires 5 1D arrays, an integer with the size, the array names, and some JSON as to the types
      • edits are not supported. memory is shared in python implementation
      • one option might be to have arrow as a subspec of zarr
    • DH: potentially having a different return type.
      • JM: that might could be used for xarray as well
    • JP: looking to warn people about the need for awkward arrays (or even making the storage opaque)
    • JM: e.g. having the extension mechanism allow/enable:
      • mandatory (MUST): throws exception if not installed
      • suggested (SHOULD): which raises a warning
      • optional (MAY): which silently ignores
    • JMS: would see open_with as a better pattern. e.g. choosing a backend without disallowing access to the details
    • DH: this seems to follow the pattern of being “part of something bigger” while still made of parsable units (e.g. dimension arrays, multiscale, etc.)
    • JMS: would be interesting to make use of chunking in the layout. JP: agreed, e.g. to enable parallel writes
  • xarray/nczarr (Josh) - anyone interested from the xarray side? (tabled)