The Zarr Book
A newcomer's guide to the Zarr ecosystem and data model.
Welcome to The Zarr Book, an introductory guide to Zarr. Zarr is a way of storing large, chunked, compressed, N-dimensional arrays. As you'll soon see, it is also a specification, a community, and a growing family of tools. That breadth is exactly what makes Zarr hard to walk into. Newcomers tend to hit two walls at the same time: an unfamiliar data model built from arrays, groups, codecs, and stores, and a fog of acronyms (ZSC, ZEP, "the spec", conventions, implementations). This book is here to take down both walls, in order, so that each idea has somewhere to stand.
We've tried to make the material broadly accessible. You don't need to have used Zarr before, and you don't need to be a specialist in array storage. If you've worked with arrays in any language, whether NumPy, a dataframe, or even a spreadsheet, you have enough to begin.
Who This Book Is For
This book is for a few kinds of readers, and it tries to serve all of them.
If you are completely new to Zarr and keep nodding along in meetings without quite knowing what people mean by "the spec" or "a store," this book is written for you first. We define the vocabulary early and then use it consistently, so the acronyms stop being noise and start being a map.
If you are a new contributor, perhaps about to work on zarr-python, you'll want a clear mental model of how arrays, chunks, codecs, and stores relate before you read the code. Part I is written with you in mind.
If you already use Zarr through a tool like xarray and want to understand what is happening underneath, this book fills in the layer between "it works" and "I know why."
How to Read This Book
In general, this book assumes you're reading it from front to back. Later chapters build on concepts from earlier ones: the "Start here" chapters give you the words, Part I spends those words on the technical model, and Parts II and III widen back out to the spec, the conventions layered on it, and the community around both. But there is no wrong way to read this book. If you came for the landscape rather than the machinery, skip ahead to Part II — Extending Zarr. If a chapter covers something you know, move on.
A few chapters cover ideas that are genuinely hard to picture, like how an array's chunks become keys in a store. Those chapters include interactive diagrams you can adjust and short quizzes to check your understanding. Use them. Poking at a diagram for a minute often does more than rereading a paragraph.
Finally, this book is the conceptual companion to the zarr-intro notebook. The book teaches the model; the notebook is where you run real code and watch a real store fill up as you write. When you're ready to get your hands dirty, that's where to go next.
Ready? Start with Why we need Zarr.