1st June, 2022
Attending: Davis Bennett (DV), Sanket Verma (SV), John A. Kirkham (JK), Trevor Manz (TM), Brianna Pagán (BP), Parth Tripathi (PT), Gregory Lee (GL), Ward Fisher (WF)
- Release 2.12 - Blog and Tweet
- ZEP website: https://zarr.dev/zeps
- ZEP 1 - Alistair and Jonathon working on it, check here. PR here
- David(xtensor), Ward(NetCDF-C & Java), Trevor(zarr.js) and Gregory Lee(zarr-python) added to the ZIC
- GSoC 2022 contributors updates - Shivank and Parth
- TM: https://excalidraw.com/ for making graphic representations, TM used them for the Zarr slides 🖼
fill_valuefor empty chunks:
- DV: How could it implemented to avoid breaking changes?
- DV: Whether there is a clear description about
fill_valuein Zarr Spec V2?
- JK: V3 spec aims to mitigate the issue of
fill_valueand have clear text about it
- DV: Haven’t looked at V3 but if it specifices
fill_valueby default then it’s good
- TM: Not writing
fill_valueis sematically correct, on the implementations side: having to implement something like this is more flexible and easy in JS as compared to Python
- DV: Problems representing numbers as
.jsonmetadata. There should not be constraints to
.jsonas it’s metadata
- TM: Serialising the scalar arrays as containers
- JK: Slightly hard to read metadata with
fill_valueas there are amiguity issues
- JK: How does NetCDF handles
- WF: Couple of different approaches to handle it.
- We do it for standard and compound data types
- In absence of user-input we have a default
- Mostly users need to specify the data type as it’s difficult to come up with it on it’s own
- There’s also a flag tp supress
- As NetCDF is mostly used for data archival and for that we need to maintain data integrity, it’s difficult to know what’s done if there are no defined data types or no
- We can also explicitly write
- Store in metadata
- JK: Is there a concept of not storing it?
fill_valuecan be changed, the default behaviour is to replace old
fill_value, not the best case but yeah, it is what it is!
FSSPECreference ability to record
.json. How data
- SV: Conferences 📣
- BP: Speaking at ESIP(in-person) on July 19th-22nd on cloud processing and how they process internal Zarr & COGs data at NASA. Christine will be talking about Zarr and almost 50% of the content will be focused on Zarr. Link: 1 and 2
- WF: Will be talking about NCZarr in Rocky Mountain(hybrid), current state of NetCDF. Link to follow in the Gitter chat.
- JK: GTC(virtual) happened in spring, NVIDIA is presenting a few things at SciPy
- SV: CFP for EuroSciPy is open until 6/6 AoE, submit here