The ancestor-subzone layout

Use this layout to group cells by ancestor, so each storage chunk covers a complete subtree. It stores data in a two-dimensional array: subzone index within an ancestor's subtree, then the ancestor. One column is one subtree, and its row order is the OGC API-DGGS sub-zone order, here ascending cell id. This gives three useful properties:

an ancestor nobody wrote is a chunk that was never stored, so a land-only global store costs nothing for the ocean and reads back as fill_value;
a column is one file, so production tasks can write columns independently;
a reader gets the tree's own irregular chunking back.

IGEO7 pentagon subtrees contain fewer cells than hexagon subtrees. The layout pads the shorter columns to fit Zarr's uniform chunks, and the reader removes that padding from the cell axis.

Write it with dggwrite(dest, cube; layout = :subzones, ancestor_level = k) or incrementally through subzonestore and dggwrite!. Read it with dggread, which returns a Cells dimension over a lazy DiskArrays view, drops pentagon padding, and publishes subtree chunk boundaries.

Each column is written whole. A cube whose coverage stops inside a subtree raises DGGSFormatError(check = :incomplete_subtree).

DiscreteGlobalGrids.SubzoneLayout Type

julia

SubzoneLayout(system, level, ancestor_level; gridname, capacity)

The shape of an ancestor-subzone store: a level-level cell axis cut into the subtrees of the complete level-ancestor_level grid, one subtree per column.

ncolumns is ncells(levelgrid(system, ancestor_level)), and column i is the ith level-ancestor_level cell in canonical order. The column axis is IMPLICIT: index is the ancestor, and a store may carry the ids as a coordinate array for interop but is not read through it.
capacity is the number of ROWS the store needs, which is the longest column: 7^d for IGEO7, 4^d for a quad-face system. Shorter columns — the twelve pentagon-rooted ones — hold their cells first and fill after (SUBZONE_PADDING).

Both directions of the mapping are O(level) arithmetic on one cell id:

julia

subzoneindex(layout, cell)      # (column, row)
columnindices(layout, i)        # the level grid indices column i holds
columncell(layout, i)           # the ancestor cell itself

Only systems with has_sorted_subtrees can be laid out this way: a column is a contiguous run of the level grid, and a system whose descendants are scattered has no such run.

The default capacity is measured by one pass over the ancestor grid; pass it where it is already known (subzone_count in a store's attributes).

DiscreteGlobalGrids.subzonestore Function

julia

subzonestore(dest, system, level; ancestor_level, layers, kwargs...) -> SubzoneStore
subzonestore(dest) -> SubzoneStore

Requires using Zarr. The methods live in DiscreteGlobalGridsZarrExt, whose docstring is the full keyword reference.

Create — or reopen — an ancestor-subzone store for incremental writing: the group, its arrays and its attributes are stamped once, and the columns are filled afterwards, one dggwrite! at a time. A column is one chunk and therefore one file, and a column write rewrites nothing shared, so tasks writing disjoint columns need no coordination.

See SubzoneLayout for the layout itself and dggwrite's layout = :subzones for the one-shot form.

julia

subzonestore(dest, system, level; ancestor_level, layers,
             capacity = nothing, fill_value = NaN, ancestor_coordinate = true,
             attrs = Dict{String,Any}(), compressor = Zarr.BloscCompressor())
    -> SubzoneStore
subzonestore(dest) -> SubzoneStore

Create an ancestor-subzone store, or reopen one for writing.

The store is a Zarr v2 group of (ancestor, subzone) arrays — (capacity, ncolumns) in Julia's own order, chunked (capacity, 1), so one chunk is one level-ancestor_level subtree. Nothing is written into them here: the columns are filled afterwards by dggwrite!, in any order and from any number of tasks, and a column nobody writes stays a chunk that was never stored and reads back as fill_value.

layers names the data variables and their element types: "elevation" => Float32, an iterable of such pairs, or a NamedTuple/Dict of them.
capacity is the row extent, which defaults to the measured longest subtree (subzone_capacity). Pass it where it is known — one pass over a level-6 ancestor grid is 1.2 million descendant_range calls.
fill_value is what an unwritten cell reads back as, NaN by default, which needs a floating-point layer; a layer of another element type has to be given one.
ancestor_coordinate writes the level-ancestor_level ids as a one-dimensional coordinate array. The column axis is IMPLICIT — the index is the ancestor — and this reader never consults it; it is there so that a generic xdggs reader sees the ancestor axis for the cell axis it is.
attrs are the producer's own group attributes, which the layout's are stamped over.

Reopening reads the layout back out of the attributes and checks it against the arrays, so a mistyped path is refused rather than half-written into. dest is a local directory path or an open writeable Zarr.ZGroup; a remote URL is refused.

DiscreteGlobalGrids.dggwrite! Function

julia

dggwrite!(store::SubzoneStore, ancestor, values; var = the only layer) -> store
dggwrite!(store::SubzoneStore, cube) -> store

Requires using Zarr. The methods live in DiscreteGlobalGridsZarrExt.

Fill columns of a store subzonestore has already created: one ancestor cell's subtree from a vector in ascending cell id, or every complete column of a cube over a cell axis.

values is as long as that ancestor's subtree really is — 7^d for a hexagon and (5*7^d + 1)/6 for a pentagon — and the rest of the column stays fill.

julia

dggwrite!(store, ancestor, values; var = the store's only layer) -> store
dggwrite!(store, cube) -> store

Fill columns of an open SubzoneStore.

The first form writes ONE column: ancestor is a level-ancestor_level cell — or its column index — and values is its subtree in ascending cell id, exactly as long as that subtree really is. A NamedTuple or Dict of vectors writes several layers of the one column. The rest of the column, where the ancestor is one of the twelve pentagons, stays fill.

The second form writes a whole cube: every layer of a DimStack, or a DimArray, over a cell axis at the store's own level. Its coverage has to be whole columns — see subzone_runs — and it need not be all of them, or contiguous.

Only the chunks of the columns named are touched, so two tasks writing disjoint columns are safe on a directory store, and nothing shared is rewritten.

DiscreteGlobalGrids.SubzoneRun Type

julia

SubzoneRun(column, rows, axis)

One contiguous piece of a cube's cell axis that lands inside one column: axis indices of the cube map onto rows of column column, in order.

length(rows) == length(axis) always; a run whose rows is not the whole column is a partially covered subtree, which subzone_runs refuses.

DiscreteGlobalGrids.subzone_capacity Function

julia

subzone_capacity(system, ancestor_level, level) -> Int

The row extent an ancestor-subzone store needs: the size of the LONGEST level-ancestor_level subtree at level.

Measured rather than assumed, because the closed form is the grid's business and not this layer's: aperture 7 gives 7^d for a hexagon and (5*7^d + 1)/6 for a pentagon, a quad-face system gives 4^d throughout, and a system yet to be written gives whatever descendant_range says. The pass is one O(level) range per level-ancestor_level cell — a fifth of a second for the 1 176 494 columns of an IGEO7 level-6 ancestor grid, and it happens once, when a store is created. Every read takes the number out of the store's attributes instead.

DiscreteGlobalGrids.subzone_depth Function

julia

subzone_depth(layout) -> Int

d = level - ancestor_level, the depth of one column's subtree.

DiscreteGlobalGrids.subzone_runs Function

julia

subzone_runs(layout, cells; complete = true) -> Vector{SubzoneRun}

The columns a cube's cell axis covers, and which slice of the cube goes into each.

cells is a CellLookup, a CellVector, or any strictly ascending vector of level-level cell ids. The first two are walked through their INDEX WINDOWS — one step per column touched, and no id is ever materialized, which is what lets a land-only cube of tens of millions of cells be planned in microseconds. A plain vector costs one globalindex per cell.

complete = true refuses a column the axis covers only part of. That is the layout's central restriction and not an implementation limit: a column is one chunk, a chunk is written whole, and a store whose columns were written from partial coverage would read back with data cells indistinguishable from fill. Ancestor-snapped coverage — the way covering and a multi-order query name a region — satisfies it by construction.

DiscreteGlobalGrids.subzone_cellvector Function

julia

subzone_cellvector(layout, columns) -> CellVector

The cell axis a set of columns spells: their subtrees concatenated, in ascending column order. columns is nothing for the whole store — every column of the level, which is the complete level grid and one window.

This is the read side of subzone_runs and its exact inverse: writing a cube's columns and reading them back gives the same axis.

DiscreteGlobalGrids.subzone_columns Function

julia

subzone_columns(layout, ancestors) -> Vector{Int}

The column indices a set of ancestor cells names, ascending and unique. ancestors may hold cells or column indices; nothing means every column.

DiscreteGlobalGrids.columncell Function

julia

columncell(layout, i) -> AbstractCellIndex

The level-ancestor_level cell column i holds the subtree of.

DiscreteGlobalGrids.columnindex Function

julia

columnindex(layout, ancestor) -> Int

The column an ancestor cell occupies: its index in the complete level-ancestor_level grid. Throws for a cell of another level.

DiscreteGlobalGrids.columnindices Function

julia

columnindices(layout, i) -> UnitRange{Int}

The indices of the complete level grid that column i holds — the ancestor's descendant_range, which is what makes a column one contiguous piece of the cell axis.

DiscreteGlobalGrids.columnlength Function

julia

columnlength(layout, i) -> Int

How many cells column i really holds: capacity for a full subtree and less for a pentagon-rooted one, whose remaining rows are fill.

DiscreteGlobalGrids.subzoneindex Function

julia

subzoneindex(layout, cell) -> (column, row)

Where one level-level cell sits in the store: its ancestor's column, and its one-based index in the ancestor's subtree in ascending id order.

O(level) digit arithmetic on the id, so a whole store is addressed without any level-level id vector ever existing.

DiscreteGlobalGrids.columnrow Function

julia

columnrow(layout, p) -> (column, row)

subzoneindex from an INDEX of the complete level grid rather than from a cell id — what a run of the cell axis is walked with.

Attributes

The dggs attribute object carries the grid name and level, with layout fields nested under subzone_layout. It omits zarr_conventions because that field describes the one-dimensional layout.

DiscreteGlobalGrids.subzone_attrs Function

julia

subzone_attrs(layout; variables = String[], coordinate = nothing,
              fill_value = "NaN") -> Dict{String,Any}

The group attributes an ancestor-subzone store carries: a dggs object naming the grid and the level, with everything the layout adds nested under subzone_layout. No zarr_conventions declaration is written — this is not the convention's one-dimensional layout.

coordinate names the array of level-ancestor_level ids where the store carries one. The column axis is implicit either way and is never read through it, so this is interop and provenance rather than structure.

DiscreteGlobalGrids.subzone_layout Function

julia

subzone_layout(attrs; store = "") -> SubzoneLayout

The layout a store's group attributes describe, checked against what this package can read: the layout name, a version it understands, a registered grid name, and a level pair whose column count and row extent are the ones the store declares.

The declared ancestor_count and subzone_count are CHECKED against the arithmetic rather than believed. They are the store's claim about the grid, and a store whose claim disagrees is one written against another grid definition — which would read every column at the wrong offset, silently.

DiscreteGlobalGrids.issubzonestore Function

julia

issubzonestore(attrs) -> Bool

Whether a group's attributes describe an ancestor-subzone store. Never throws: a store this reader does not recognize is read the ordinary way, and a malformed one fails in subzone_layout with its own message.

DiscreteGlobalGrids.subzone_coordinate Function

julia

subzone_coordinate(attrs) -> Union{String,Nothing}

The name of the ancestor-id coordinate array a subzone store carries, where it carries one.

DiscreteGlobalGrids.SUBZONE_LAYOUT Constant

julia

SUBZONE_LAYOUT

The layout string an ancestor-subzone store carries in its dggs attributes, and the value issubzonestore recognizes it by.

DiscreteGlobalGrids.SUBZONE_ORDER Constant

julia

SUBZONE_ORDER

How the values inside one column are ordered: ascending cell id, which on a system with sorted subtrees is ascending index within the ancestor's descendant_range.

DiscreteGlobalGrids.SUBZONE_PADDING Constant

julia

SUBZONE_PADDING

What a column shorter than the array's row extent holds after its cells: the array's own fill_value, at the END of the column. A pentagon's subtree is p(d) = (5*7^d + 1)/6 cells against a hexagon's 7^d, so twelve columns of an IGEO7 store are short; which twelve is derivable from the grid and is not recorded.

DiscreteGlobalGrids.gridnamefor Function

julia

gridnamefor(system) -> String

The canonical store spelling of system, read backwards out of GRID_REFERENCE.

A grid name pins the id packing, so a system with no registered name has no store spelling either and this raises rather than inventing one.