2023-01-25
Attending: Davis Bennett (DB), Sanket Verma (SV), Josh Moore (JM), Ethan Davis (ED) - Unidata, Ward Fisher (WF), Alan Liddell (AL), Brianna Pagán (BP), Erik Welch (EW), Hailey Johnson (HJ), Jeremy Maitin-Shepard (JMS), Isaac Virshup (IV)
TL;DR:
Release update: Zarr Python 2.13.6 along with 2.13.4 and 2.13.5 is out! Check the release notes here. SciPy 2023 CFP is open until 22nd February. If you’re planning to submit a proposal involving Zarr, feel free to reach out to our community manager, and he’ll help you out. The meeting started with DB discussing his recently opened PR #1323. Next, JM gave us updates on AI4Life and initiated a discussion on GeoJSON. After this, we discussed ZEP0004 and Unicode names. Next, BP gave an update on GeoZarr bi-weekly meetings, and lastly, SV asked a general question on visualising N-dimensional arrays.
Updates:
- Zarr-Python 2.13.4 (outreachy updates), 2.13.5 and 2.13.6 is out!
- Josh: sorry, I stuttered.
- Migration to PyData Sphinx theme almost complete; check here: https://github.com/zarr-developers/zarr-python/pull/1242
- SciPy 2023 CFP is open until 22nd February
Open Agenda(add here 👇🏻):
- automated formatting - Davis
- what’s the line in the sand to get it in?
- ping the core devs, getting sharding in
- Declarative hierarchy (Davis)
- Create object then serialize it to disk
- JMS: maybe a layer on top of zarr (multi-formats)
- https://pypi.org/project/h5py-like/0.5.1/
- AI4Life, GeoJSON, etc. (Josh)
- JM: “GeoJSON is JSON. Zarr has JSON. Can you put GeoJSON in Zarr?”
- IV: something more like GeoArrow
- ED: cfconventions (trajectories, etc.) for bio?
- probably not
- IV: https://cfconventions.org/cf-conventions/cf-conventions.html#_contiguous_ragged_array_representation
- IV: convention for ragged array (start in Zarr space) - least-common denominator
- IV: shapely support for GeoArrow.
- get_items (Sanket) - Tabled (comments welcome)
- ZEP4 (EW)
- looking to make an embeddable spec for sparse arrays and sparse tensors
- natural fit for a ZEP.
- JM: previously we had discussed an extension, right?
- EW: convention so it can be stored along with other things.
- IV: not super strongly about zarr being aware. but consistent way to structure things.
- ZEP4 is more collecting what people might be doing
- IV: break ZEP4 into a completed set of conventions and then we can identify later.
- https://github.com/zarr-developers/geozarr-spec and https://github.com/zarr-developers/geozarr-spec/issues/2 (Brianna)
- BP: initial meeting last week on a governance group.
- meeting biweekly. invite open. Wednesday mornings 11am, Eastern.
- submitting to OGC approval by the summer.
- Unicode for names (Ethan): https://github.com/zarr-developers/zarr-specs/issues/56
- JMS: very helpful. normalization important.
- tension of zarr not being its own container format
- normalization by default breaks that in a subtle way
- ED: normalize on serialization, on query, etc.
- can see optional, but also advice on whether to do it and why.
- world going to UTF-8
- JMS: very helpful. normalization important.
- Visualization of Zarr arrays
- DB: neuroglancer (client-side website)
- HTTP/S3 accessible (even static fileserver)
- python code that spins up a browser
- http://vitessce.io
- https://imagej.net/plugins/bdv/
- https://www.unidata.ucar.edu/software/idv/
- https://github.com/google/neuroglancer
- https://github.com/bigdataviewer
- DB: neuroglancer (client-side website)