feat: add openConsolidated and remove lazy attrs

[manzt]

Right now the "lazy" attrs() method is a bit more trouble than it's worth. This PR introduces openConsolidated and requires that loading attrs for v2 must be decided when opening an array:

import * as zarr from "zarrita";

const store = new zarr.FetchStore("http://localhost:8080/data.zarr");
const nodes = await zarr.openConsolidated(store);
nodes; // [zarr.Group, zarr.Array, zarr.Array, ... ]
nodes[1].kind // "array"
nodes[1].path // "/"
nodes[1].shape // [512, 512]
nodes[1].chunks // [100, 100]
nodes[1].attrs // { answer: 42 }

When opening non-consolidated data, you must explicit opt out of loading .zattrs for v2:

let arrWithAttrs = await zarr.open.v2(store, { kind: "array" }); // will try to load .zattrs
let arrWithoutAttrs = await zarr.open.v2(store, { kind: "array", attrs: false }); // no loading of .zattrs

[manzt]

Need to update documentation. Does this look like it exposes the information that you'd need @andersy005? Rather than dealing with the metadata directly, you get a list of objects in the store (which have their associated attrs). The alternative would be to add something like:

import * as zarr from "zarrita";

let store = new FetchStore("https://localhost:8080/data.zarr");
let consolidatedStore = zarr.withConsolidated(store);
consolidatedStore.metadata // { ".zgroup": { ... }, "foo/.zarray": { ... }, }
consolidatedStore.get("/.zgroup"); // reads from in-memory consolidated meta
consolidatedStore.get("/foo/0/0/0"); // reads from original store

let root = await zarr.open.v2(consolidatedStore, { kind: "group" });
let arr = await zarr.open.v2(root.resolve("foo"), { kind: "array"});

but here you'd need to manually manage the consolidated metadata.

[andersy005]

i really like the convenience of having access to the consolidated metadata via this approach

import * as zarr from "zarrita";

const store = new zarr.FetchStore("http://localhost:8080/data.zarr");
const nodes = await zarr.openConsolidated(store);

however, i find the access pattern in the following code less intuitive:

const nodes = await zarr.openConsolidated(store);
nodes; // [zarr.Group, zarr.Array, zarr.Array, ... ]
nodes[1].kind // "array"
nodes[1].path // "/"
nodes[1].shape // [512, 512]
nodes[1].chunks // [100, 100]
nodes[1].attrs // { answer: 42 }

to be less intuitive. in this case, if i know the variable i want to access (e.g., time), does it mean that I would have to loop over the nodes and try finding this node using the path attribute?

[manzt]

to be less intuitive. in this case, if i know the variable i want to access (e.g., time), does it mean that I would have to loop over the nodes and try finding this node using the path attribute?

Really good point. Going to take the weekend to think about a more useful API.

[manzt]

Ok, this is the API I am working on now:

import * as zarr from "zarrita";

let store = new zarr.FetchStore("http://localhost:8080/data.zarr");
let h = await zarr.openConsolidated(store); // ConsolidatedHierarchy<FetchStore>
h.contents;
// Map {
//  "/" => Group,
//  "/foo" => Array,
//  "/bar" => Group,
//  "/bar/baz" => Array,
// }
let grp = h.root(); // Group
let foo = h.open("/foo", { kind: "array" }); // Array
let baz = h.open(grp.resolve("bar/baz"), { kind: "array" }); // Array
let bar = h.open(baz.resolve(".."), { kind: "group" }); // Group

openConsolidated returns a ConsolidatedHierarchy which has:

• contents - a Map of the keys and nodes
• open - a method just like zarr.open to navigate the hierarchy from other nodes (but also absolute paths).
• root - a helper to grab the root node (alias for open("/", { kind: "group" }))

Note that:

h.contents.get("/foo")

and

h.open("/foo")

are essentially equivalent, but the latter is more type safe and lets you navigate with relative paths from an existing node.

let baz = h.open(grp.resolve("bar/baz"), { kind: "array" });
let foo = h.open(baz.resolve("../foo"), { kind: "array" });

[andersy005]

👍🏽 the ConsolidatedHierarchy looks great, @manzt! looking forward to using this.