Reading and writing DGGS stores

A DGGS store is a Zarr group containing variables over one cell axis and the metadata needed to interpret that axis. dggwrite writes a DimensionalData DimStack; dggread reopens it as a DimStack with a shared Cells dimension and lazy arrays.

The methods live in the Zarr.jl extension, which using Zarr loads. The extension supplies the store-aware methods documented here.

StoreDescription carries plain data between store attributes and the cube: grid name, level, encoding, array names, and grid parameters. Reading follows attrs → description → axis; writing follows axis → description → attrs. StoreSnapshot supplies metadata-only input to DGGSConvention, while CellEncoding supplies the identifier layout. The grid retains id arithmetic, so encodings work across systems.

Both are registries. A downstream package can add a metadata dialect with register_convention! or an id layout with register_encoding!.

The reader returns a ChunkedCellLookup. It answers At, Contains and Covering through the ChunkManifest, which describes the chunk grid in cells. Arithmetic and range encodings can open without coordinate reads; a foreign dense store reads its ids once at open to validate them.

A stored axis is also a region and answers halo, border, interior and adjacency with the same code as an in-memory axis. It does so through region, which is the axis's compressed CellVector twin, built on the first call and kept. What that conversion costs is the encoding's and not the axis's length: a ranges or implicit store converts by arithmetic alone, because a stored interval is a run of consecutive ranks and a rank plus one is an index; a dense store reads its ids once, in the order that touches each chunk once. Index order is preserved either way, which is what lets a result computed through the twin be written back against the store's own axis with no permutation. Sweeping a store along its own chunk lines is its own page.

Validation is strict. An unknown grid, conflicting metadata, invalid id, or inconsistent length raises DGGSFormatError with the failed check. Supply description = StoreDescription(...) to assert the metadata explicitly when a store has no attributes.

Reading and writing

DiscreteGlobalGrids.dggread Function

dggread(store; vars = All(), lazy = true, validate = :strict,
        conventions = CONVENTION_REGISTRY, description = nothing) -> DimStack
dggread(store, var::Symbol; kwargs...) -> DimArray

Read a DGGS store as a dimensional cube. Requires using Zarr. The stack shares a Cells axis backed by ChunkedCellLookup. Data stay lazy unless lazy=false; the single-variable form returns a DimArray.

store accepts a local path, URL, Zarr.ZGroup, or Zarr.AbstractStore. Public gs:// URLs use HTTPS; s3:// additionally requires using AWSS3. vars selects data variables. Metadata retain source attributes and the detected grid description for a later rewrite.

validate=:strict validates IDs when scanning an axis. A stored chunk manifest can avoid that scan; :scan forces it. :lazy samples IDs instead. A supplied StoreDescription bypasses metadata detection, not mechanical validation. Invalid formats raise DGGSFormatError.

See Reading and writing DGGS stores for formats and examples, and Workflow execution details for validation and metadata rules.

source

dggread(store; vars = All(), lazy = true, validate = :strict,
        conventions = CONVENTION_REGISTRY, description = nothing,
        ancestors = nothing) -> DimStack
dggread(store, var::Symbol; kwargs...) -> DimArray

Zarr implementation of dggread. The generic function documents the common read contract. s3:// additionally requires using AWSS3.

For ancestor-subzone stores, ancestors selects ancestor cells or column indices. The default includes the full level, including unwritten columns that read as fill. The result still has a Cells dimension; its data use the subzone layout lazily. Its metadata contain "layout" instead of ordinary encoding/convention/description entries. Other dimensions use stored coordinates when available.

See Workflow execution details for validation, manifest trust, and the metadata keys retained for a rewrite.

source

DiscreteGlobalGrids.dggwrite Function

dggwrite(dest, stack_or_array; encoding = :auto,
         conventions = DEFAULT_WRITE_CONVENTIONS, chunks = :auto,
         merge = :step, chunk_target = 1_000_000, target = nothing) -> dest

Write a DimArray or DimStack with a cell lookup to a Zarr v2 store. Requires using Zarr. dest is a local directory or writable Zarr.ZGroup; remote URL writing is not supported.

encoding=:auto chooses an eligible ranges encoding or dense IDs. :dense stores each ID; :ranges stores intervals; :implicit requires a complete level. merge=:step joins integer-adjacent IDs. merge=:rank joins consecutive valid cells and requires a rank-aware reader.

target=:xdggs writes a store the Python package xdggs opens with xdggs.decode: the dense encoding, and a grid that xdggs or one of its plugins registers, checked by require_xdggs_readable before writing. See Writing a store for xdggs.

chunks is a cell chunk length or :auto. chunk_target counts all elements per chunk, including non-cell dimensions. Layer metadata become array attributes; group attributes come from metadata["attrs"]. Generated convention keys take precedence, and layer order is normalized alphabetically.

layout=:subzones selects the separate ancestor-subzone writer and requires ancestor_level. See Reading and writing DGGS stores, The ancestor-subzone layout, and Workflow execution details for layout-specific options and metadata rules.

source

dggwrite(dest, stack_or_array; encoding = :auto,
         conventions = DEFAULT_WRITE_CONVENTIONS, chunks = :auto,
         merge = :step, chunk_target = DEFAULT_CHUNK_TARGET,
         target = nothing) -> dest

Zarr v2 implementation of dggwrite, including consolidated metadata. The generic function describes encodings, chunks, and metadata handling. The cell dimension must retain an AbstractCellLookup; a categorical axis created by operations such as reverse is rejected.

target = :xdggs writes the store xdggs opens: the dense coordinate, with the description checked through require_xdggs_readable before a byte is written. An encoding other than :auto or :dense contradicts the target and is an ArgumentError, like every other keyword conflict; a description xdggs cannot read is a DGGSFormatError with check = :not_xdggs_readable.

The writer persists a chunk-manifest sidecar so readers can open the axis without scanning every ID. It never overwrites an existing layer: conflicting array names in an open group raise before attributes are written. Remote URL destinations are not supported.

See Workflow execution details for sidecar, attribute, and round-trip rules.

source

Writing a store for xdggs

xdggs's default convention reads a one-dimensional cell_ids coordinate whose attributes are the fields of its grid-info dataclass, for a grid its registry knows. target = :xdggs writes that layout and checks the store's description against it before the group is created:

using DiscreteGlobalGrids, Zarr
dggwrite("tas.zarr", cube; target = :xdggs)

The store then opens in Python with no arguments beyond the path:

import xarray as xr, xdggs
ds = xr.open_dataset("tas.zarr", engine="zarr").pipe(xdggs.decode)
ds.dggs.cell_centers()

What the target chooses and checks, and why:

Choice	Reason
encoding = :dense	:auto prefers the ranges encoding, which stores (n, 2) intervals and no cell_ids array; xdggs finds nothing to decode there.
the coordinate is cell_ids and carries a level	xdggs.decode looks the coordinate up by that name, and its grid info has no default level.
grid in XDGGS_GRIDS	healpix and h3 ship with xdggs; igeo7 needs the xdggs-dggrid4py plugin from its main branch, whose grid info takes the attributes XdggsConvention writes.

Two properties every dense store from this writer already has matter to xdggs: the coordinate's attributes are grid keys only, because xdggs forwards every attribute but grid_name to its dataclass constructor and a stray units is a TypeError there; and cell_ids has no fill value, because xarray reads a Zarr v2 fill value as a mask and promotes the ids to Float64.

Both write conventions are stamped, so ds.dggs.decode(convention="zarr") opens the same store through the zarr-conventions/dggs group attributes. xdggs.decode strips the grid attributes off the coordinate as it builds its index, so a dataset edited in Python is written back with ds.dggs.encode("xdggs").to_zarr(...).

The chunk manifest sidecar rides along as a data variable on dimensions of its own, which xdggs ignores.

DiscreteGlobalGrids.require_xdggs_readable Function

require_xdggs_readable(d::StoreDescription; store = "", conventions = String[]) -> d

Check that a store described by d opens through xdggs.decode(ds), or raise a DGGSFormatError with check = :not_xdggs_readable saying what stops it.

xdggs's default convention reads the one-dimensional coordinate named cell_ids, one id per cell, and builds its grid info from that coordinate's level and grid name. So d passes when

• its encoding is dense and its coordinate is cell_ids,

• it carries a level, and

• its grid is in XDGGS_GRIDS.

source

DiscreteGlobalGrids.XDGGS_GRIDS Constant

XDGGS_GRIDS

Grid name → the Python package that registers it with xdggs, and so the names require_xdggs_readable accepts. "healpix" and "h3" ship with xdggs itself. "igeo7" ships with the xdggs-dggrid4py plugin from its main branch, whose IGEO7Info takes the coordinate attributes XdggsConvention writes field for field; the plugin's 0.1.3 release on PyPI predates that attribute set and needs a DGGRID binary to import. A downstream grid with an xdggs plugin adds itself here next to its register_grid! entry.

source

DiscreteGlobalGrids.xdggs_ellipsoid_attrs Function

xdggs_ellipsoid_attrs(e::Ellipsoid) -> Union{String, Dict{String,Any}, Nothing}

An ellipsoid as xdggs's parse_ellipsoid reads it: a bare name, or an object spelled radius for a sphere and semimajor_axis plus inverse_flattening for an ellipsoid, with name alongside when there is one. This is the one place the package writes the semimajor_axis spelling, because xdggs's dataclasses take exactly these keys.

nothing for an unnamed ellipsoid that has neither a radius nor the axis pair, which xdggs has no spelling for.

source

The stored axis

DiscreteGlobalGrids.ChunkedLookups.ChunkedCellLookup Type

ChunkedCellLookup(axis::ChunkedCellVector)

The DimensionalData lookup over a STORED cell axis. Pair it with Cells to make a cube axis, exactly as CellLookup is paired:

axis = cellaxis(RangesEncoding(), grid, ranges)
A    = DimensionalData.DimArray(data, Cells(ChunkedCellLookup(axis)))
A[Cells(DimensionalData.At(cell))]
A[Cells(Covering(basin))]

It answers the same selectors as CellLookup — At and Contains on a cell id, Contains on a lon/lat point, Covering on a region — and differs in where the answer comes from: the axis is what a store wrote, so a selector is resolved by the manifest first and by at most one chunk of ids after, never by a scan.

ForwardOrdered is claimed on every axis, and how far it is EARNED depends on where the axis came from. A scanned store proves it: sorted, unique and single-level are verified id by id at open. A store opened on a persisted manifest (cellaxis) does not — there the three are the writer's attestation, and all that is verified per chunk, as chunks are decoded, is the first id, the last id and the length. dggread(store; validate = :scan) declines the attestation and scans.

A SUBSET is no longer a stored axis — indexing or selecting materialises the cells it names. A sorted, unique subset becomes the package's own compressed CellLookup, which every later operation then treats as any other cell axis; one that is neither becomes an Unordered Categorical lookup over the same cells, since a cell axis is sorted by definition and this one is not. Base.reverse is the everyday way to reach the second case.

source

DiscreteGlobalGrids.ChunkedLookups.ChunkedCellVector Type

ChunkedCellVector(grid, source, length)

The stored cell axis as a lazy AbstractVector of typed cell ids at one level.

Semantically it is the id vector the store wrote: length is the number of cells, axis[k] is the kth of them, collect(axis) is the vector itself. What backs it is the encoding's business — closed-form rank/select over stored intervals, over the whole level, or a cached chunk of a stored id array — and axisindex is the inverse in every case.

Build one with cellaxis, never directly.

source

DiscreteGlobalGrids.ChunkedLookups.axisindex Function

axisindex(axis::ChunkedCellVector, id::Integer) -> Union{Int,Nothing}

The index of raw id id in the axis, or nothing when the axis does not hold it. The inverse of rawcell, and the half of the bijection every selector ends at.

Resolution is two-level wherever the ids are stored rather than computed: the manifest names the one chunk id could be in, and only that chunk is read.

source

DiscreteGlobalGrids.ChunkedLookups.ChunkManifest Type

ChunkManifest(axis::ChunkedCellVector, chunklength::Integer)

The chunk grid of a stored array, described in cells: for chunk c, the first and last cell id it holds, how many cells that is, and how many precede it.

manifest.firstids[c]   manifest.lastids[c]
manifest.lengths[c]    manifest.offsets[c]     # cells before chunk c

This is what the lookup owns and what every IO decision is made from — which chunks a selection touches, where a halo's neighbours live, which chunk an id resolves in. Chunking is a property of an ARRAY, not of the axis, so a store whose data variables are chunked differently from its coordinate has one manifest per chunk length; each is built by the same call.

The constructor reads only the axis's chunk boundaries, so on a computed axis (RangesEncoding, ImplicitEncoding) it is closed-form arithmetic over the stored intervals and touches no store at all, however many cells the axis holds. On a dense axis it costs the two boundary reads per chunk that the cache usually already holds.

The final chunk is short whenever the length is not a multiple of chunklength; Zarr pads it on disk and the manifest does not. Every other chunk holds exactly chunklength cells, and the constructor refuses a manifest that says otherwise: chunkof divides by chunklength rather than searching offsets, so a manifest read back from a sidecar under a different chunk length would resolve every index into the wrong chunk.

source

DiscreteGlobalGrids.ChunkedLookups.chunkmanifest Function

chunkmanifest(lk::ChunkedCellLookup, chunklength) -> ChunkManifest
chunkmanifest(axis::ChunkedCellVector, chunklength) -> ChunkManifest

The ChunkManifest of an array chunked in blocks of chunklength cells along this axis. The lookup owns the chunk pattern, so this is where an IO plan starts.

source

DiscreteGlobalGrids.ChunkedLookups.nchunks Method

nchunks(m::ChunkManifest) -> Int

The number of chunks the axis spans.

source

DiscreteGlobalGrids.ChunkedLookups.chunkof Function

chunkof(m::ChunkManifest, k::Integer) -> Int

The chunk holding axis index k.

source

DiscreteGlobalGrids.ChunkedLookups.chunkbounds Function

chunkbounds(m::ChunkManifest, c::Integer) -> UnitRange{Int}

The axis indices chunk c holds.

source

Describing a store

DiscreteGlobalGrids.StoreDescription Type

StoreDescription(; gridname, kwargs...)

The pivot between store attributes and a cell axis. Reading is attrs → description → lookup; writing is lookup → description → attrs.

• gridname: canonical grid name, e.g. "igeo7" or "healpix".

• system: the grid system it resolves to, from GRID_REFERENCE.

• idscheme: how a cell id is packed — :z7int, :nested, :ring, …

• level: the refinement level, or nothing for a variable-sized axis.

• encoding: a CellEncoding instance, from ENCODING_REGISTRY.

• coordinate: name of the array encoding the cell ids, nothing when the axis is implicit in index.

• spatial_dimension: the dimension the data variables share. It is not the coordinate's own dimension: a ranges coordinate is shaped (n, 2) and names neither.

• variables: the data variable names, or nothing when unknown.

• ellipsoid, orientation, geodetic_conversion: grid parameters as read.

• provenance: verbatim source attributes keyed by convention name, enough to regenerate a value-identical store.

Every field but gridname and provenance may be nothing, meaning the store did not say. == compares the semantic fields and deliberately ignores provenance, which differs between a store and its faithful copy.

source

DiscreteGlobalGrids.StoreSnapshot Type

StoreSnapshot(; identifier = "", attrs = Dict{String,Any}(), arrays = ArrayEntry[])

A metadata-only view of one store group: its group attributes and its arrays.

This is the only thing a DGGSConvention ever sees. identifier is the store URL or path, used in DGGSFormatError messages.

Both attrs and each array's attrs are mutable; encode! writes into them. copy deep-copies the attribute dictionaries so a snapshot can be re-stamped without disturbing the one it came from.

Subgroups are out of scope: a snapshot describes one group, and a hierarchy is a sequence of snapshots.

source

DiscreteGlobalGrids.ArrayEntry Type

ArrayEntry(; name, attrs, shape, eltype = Any, dims = String[])

One array of a StoreSnapshot, as metadata only: no chunks, no compressor, no values.

• name: the array's name within its group.

• attrs: the array's attributes, mutable, as read (.zattrs in Zarr v2, attributes in v3). Conventions stamp write attrs into this dictionary.

• shape: the DECLARED shape. Chunk extents may exceed it; every length check is against this.

• eltype: the element type, or Any where it is not known.

• dims: the dimension names, outermost first, empty for a scalar array. In Zarr v2 these come from _ARRAY_DIMENSIONS.

source

DiscreteGlobalGrids.describe_store Function

describe_store(snapshot; conventions = CONVENTION_REGISTRY) -> StoreDescription

Detect, decode and reconcile: the whole read-side metadata path.

Where several conventions fire their descriptions must agree field for field — one may be silent where another speaks, but two answers to one question is the "attrs lie" failure and raises DGGSFormatError rather than a guess. Convention defaults are applied last, so a store's silence is never mistaken for a declaration during reconciliation.

What comes back is usable: a description still missing its encoding or its spatial dimension after every convention and every default has spoken names a store nothing can open, and raises rather than being returned.

source

DiscreteGlobalGrids.Detection Type

Detection(convention, rank, evidence; payload = Dict{Symbol,Any}())

What detect returns when a convention recognizes a snapshot.

rank is :declared when the store names the convention by identifier — the zarr_conventions UUID — and :fingerprint when it was recognized by the shape of its attributes. Declared outranks fingerprint in registry order, so a store that says what it is is decoded by that convention first.

evidence is a short human phrase naming what matched; payload carries whatever the convention's own decode needs, typically the array it found.

source

DiscreteGlobalGrids.Ellipsoid Type

Ellipsoid(; name, semi_major_axis, semi_minor_axis, inverse_flattening, radius)

The reference ellipsoid a store declares, as read. Every field may be nothing: stores supply whichever subset their producer had.

Read accepts both spellings of the axis keys — the schema-valid semi_major_axis/semi_minor_axis and the semimajor_axis/semiminor_axis form that xdggs and every store in the wild actually use — and normalizes to these field names. ellipsoid_attrs writes the schema-valid spelling.

source

DiscreteGlobalGrids.GridOrientation Type

GridOrientation(; vert0_lon, vert0_lat, vert0_azimuth, rotation_pattern)

The icosahedron placement of an ISEA-family grid, as read. Two stores of the same grid name and level but different orientations hold incomparable cell ids and nothing in any convention flags it, so this travels with the description.

Fields carry the DGGRID vertex-0 parameters; nothing means the store said nothing. Convention A spells them dggs_vert0_lon and friends inside the dggs object, convention B spells the longitude igeo7_dggs_vert0_lon on the coordinate.

source

DiscreteGlobalGrids.ellipsoid Function

ellipsoid(desc) -> Ellipsoid

The description's ellipsoid, or DEFAULT_ELLIPSOID where it declared none.

source

DiscreteGlobalGrids.DEFAULT_ELLIPSOID Constant

DEFAULT_ELLIPSOID

The sphere of radius 6 370 997 m that zarr-conventions/dggs prescribes when a store declares no ellipsoid. ellipsoid applies it; a decoded StoreDescription keeps nothing so that a store's silence is never confused with a declaration.

source

DiscreteGlobalGrids.ellipsoid_attrs Function

ellipsoid_attrs(e::Ellipsoid) -> Dict{String,Any}

An ellipsoid as write attributes, in the schema-valid semi_major_axis spelling. This deliberately diverges from the stores in the wild, which emit semimajor_axis and therefore fail the schema they declare.

source

Conventions

A convention is a dialect of store attributes. Several may fire on one store — the published stores are stamped twice on purpose — and where they do, their descriptions must agree field for field.

DiscreteGlobalGrids.DGGSConvention Type

DGGSConvention

A storage convention: a way a store says which grid, level and encoding its cell axis has.

Three verbs, all operating on a StoreSnapshot:

detect(c, snapshot)             -> Union{Detection, Nothing}
decode(c, snapshot, detection)  -> StoreDescription
encode!(c, snapshot, desc)      -> snapshot

Downstream packages subtype this and push! onto CONVENTION_REGISTRY; gridname is the hook for folding a private grid vocabulary in.

source

DiscreteGlobalGrids.ZarrDGGSConvention Type

ZarrDGGSConvention()

zarr-conventions/dggs: a zarr_conventions declaration on the group naming ZARR_DGGS_UUID, plus a dggs object carrying name, refinement_level, spatial_dimension and optionally coordinate, compression, indexing_scheme and ellipsoid.

This is the only convention that can express a non-dense encoding, so on a ranges store it is the one that says so; the flat attrs of XdggsConvention ride along on the same store and describe less.

Group-level declarations only: the spec's array-level override is not read.

source

DiscreteGlobalGrids.XdggsConvention Type

XdggsConvention()

The xdggs dialect: flat grid attributes on the cell-id coordinate — grid_name, level (or an alias), indexing_scheme (or nest), ellipsoid, plus per-plugin extras such as igeo7_dggs_vert0_lon.

The only convention most producers emit, and the only one on stores whose group attributes are empty. It describes a dense 1-D axis and nothing else: on the (n, 2) coordinate of a ranges store it decodes the grid and leaves the encoding to ZarrDGGSConvention.

Writing puts ONLY grid keys on the coordinate. xdggs forwards every attribute but grid_name as a dataclass keyword, so one stray units makes the store unreadable there.

source

DiscreteGlobalGrids.LegacyHealpixConvention Type

LegacyHealpixConvention()

grid_name: "healpix" with the nside/nest pair rather than level/indexing_scheme — the dialect the Ifremer and Grid4Earth demo stores ship.

The trio is also within XdggsConvention's alias set, so both fire on such a store and merge to the same description. This convention is the named extension point for the dialect: it is what provenance and error messages call it, and it is where a healpix-specific reading of the trio dispatches.

source

DiscreteGlobalGrids.DKRZConvention Type

DKRZConvention()

The nextGEMS / DestinE / easy.gems dialect: a crs variable carrying grid_mapping_name: "healpix", healpix_nside and healpix_order.

Two things distinguish it from CF 1.13, which has the same shape. healpix_nside is an nside where CF's refinement_level is an order, and healpix_order takes the value "nest". Most of these stores hold no cell array at all: the array index is the nested index and the axis is implicit. Where a cell array does exist the store is regional and holds global indices, so its presence is the sparsity signal.

Data variables' grid_mapping back-references are known to dangle, so detection scans every array for the grid mapping rather than following one.

source

DiscreteGlobalGrids.CONVENTION_REGISTRY Constant

CONVENTION_REGISTRY

The conventions describe_store tries, in order: UUID-declared before fingerprinted, so a store that names its convention is decoded by that one first. Downstream packages extend it with register_convention!.

source

DiscreteGlobalGrids.DEFAULT_WRITE_CONVENTIONS Constant

DEFAULT_WRITE_CONVENTIONS

What dggwrite stamps by default: zarr-conventions/dggs for the encoding vocabulary a flat coordinate cannot express, and xdggs so the store opens in the ecosystem's own reader. The zarr-conventions/dggs half is written schema-VALID, which the stores in the wild are not.

source

DiscreteGlobalGrids.register_convention! Function

register_convention!(c::DGGSConvention; first = false) -> CONVENTION_REGISTRY

Add c to CONVENTION_REGISTRY, at the end or ahead of everything.

source

A new dialect is a subtype of DGGSConvention with a detect, a decode and — if it is to be written and not only read — an encode!. These names stay qualified: they are generic enough that exporting them would collide with half the ecosystem.

DiscreteGlobalGrids.detect Function

detect(c::DGGSConvention, snapshot) -> Union{Detection, Nothing}

Whether c recognizes snapshot, and on what evidence. nothing means "not mine"; a malformed declaration of c's own convention is a DGGSFormatError rather than a silent nothing.

source

DiscreteGlobalGrids.decode Function

decode(c::DGGSConvention, snapshot, detection) -> StoreDescription

The part of the store's identity c can read. Fields c has no vocabulary for are left nothing so another convention can supply them; see describe_store for how several decodings are reconciled.

source

DiscreteGlobalGrids.encode! Function

encode!(c::DGGSConvention, snapshot, desc) -> snapshot

Stamp desc into snapshot's attribute dictionaries in c's spelling. Read-only conventions do not implement this.

source

DiscreteGlobalGrids.conventionname Function

conventionname(c::DGGSConvention) -> String

Short name used in DGGSFormatError messages and as the provenance key.

source

DiscreteGlobalGrids.gridname Function

gridname(c::DGGSConvention, name, attrs) -> String

Canonical grid name for the raw name a store carries, given the attribute dictionary it came from.

The default normalizes case and whitespace and otherwise trusts the store, so that unregistered names fail loudly in gridreference rather than being decoded as something else. A convention overrides this to fold its own vocabulary in:

DGG.gridname(::MyConvention, name, attrs) =
    name == "ISEA7H_Z7" ? "igeo7" : invoke(DGG.gridname, Tuple{DGG.DGGSConvention,Any,Any}, MyConvention(), name, attrs)

source

Encodings and grid references

An encoding maps cell ids to disk storage; a grid reference maps the store's grid name to a system. Both are lookup tables with registration functions and strict recognition rules. The exported types and tables let downstream packages add entries.

DiscreteGlobalGrids.Encodings.CellEncoding Type

CellEncoding

How a store lays its cell axis out. The three shipped layouts are DenseEncoding, RangesEncoding and ImplicitEncoding; a downstream package adds its own by subtyping this and registering an instance in ENCODING_REGISTRY.

An encoding implements cellaxis (build the axis, and with it the chunk manifest), its own validation, and write_eligible, which encoding = :auto consults. It never touches id arithmetic: everything it needs about the ids themselves is idrank / idselect / idcount_between on the grid.

source

DiscreteGlobalGrids.Encodings.DenseEncoding Type

DenseEncoding()

One stored id per cell. Universal and the interop escape hatch; the axis costs a chunked pass over the id array to verify and to build the manifest from.

source

DiscreteGlobalGrids.Encodings.RangesEncoding Type

RangesEncoding()

An (n, 2) array of INCLUSIVE [start, stop] raw-id intervals at one level. The axis is computed from the intervals by rank/select, so it needs no id storage at all and no data IO: the length, every chunk's first and last id, and every selector are closed-form.

source

DiscreteGlobalGrids.Encodings.ImplicitEncoding Type

ImplicitEncoding()

No stored axis: index k is the cell at rank k - 1 of the level. The whole-level case, as written by the DKRZ-style conventions.

source

DiscreteGlobalGrids.Encodings.ENCODING_REGISTRY Constant

ENCODING_REGISTRY

Store vocabulary ("none", "ranges", "implicit") to CellEncoding instance. Conventions resolve an attribute's string through this table, and keyword symbols are sugar over the same entries, so a downstream encoding becomes usable by adding one pair — register_encoding!.

source

DiscreteGlobalGrids.Encodings.register_encoding! Function

register_encoding!(name::AbstractString, enc::CellEncoding) -> ENCODING_REGISTRY

Register enc under the vocabulary string name, the way a store spells it.

A store's compression attribute, and dggwrite's encoding keyword, are both resolved through ENCODING_REGISTRY, so this is what makes a downstream encoding reachable by name. Reading it also needs cellaxis and writing it the write path's own verbs; an encoding that registers without them is refused by name rather than by MethodError.

source

DiscreteGlobalGrids.GridReference Type

GridReference(name, system, idscheme, schemes)

What a canonical grid name resolves to: the grid system, the id scheme to assume when a store names none, and the schemes that name accepts.

A grid name pins the id packing, not just the tessellation: "igeo7" and "isea7h" are the same cells under different ids, and a reader that treated one as the other would misplace every cell. Unknown names are therefore rejected rather than guessed.

source

DiscreteGlobalGrids.GRID_REFERENCE Constant

GRID_REFERENCE

Canonical grid name → GridReference. Shared by every convention; gridname is what maps a store's own spelling onto a key of this table, and register_grid! is how a downstream grid system gets one.

source

DiscreteGlobalGrids.register_grid! Function

register_grid!(name::AbstractString, ref::GridReference) -> GRID_REFERENCE

Register the canonical store spelling name for a grid system.

Every convention resolves a store's grid name through GRID_REFERENCE, and an unregistered name is refused rather than guessed, so this is what makes a downstream grid system readable and writeable. The name pins the id packing as well as the tessellation: register "isea7h" separately from "igeo7" if the ids differ, rather than aliasing one onto the other.

source

DiscreteGlobalGrids.gridreference Function

gridreference(canonical; store = "", conventions = String[]) -> GridReference

Look canonical up in GRID_REFERENCE, or raise a DGGSFormatError listing every registered name.

source

An encoding builds the axis, declares write eligibility, and supplies its store name. It asks the grid for id arithmetic, so one encoding works across systems and one system works with every encoding.

The Zarr extension uses storedaxis to open an axis and dispatches four write operations for encodings that support output. An incomplete registration raises DGGSFormatError(check = :unsupported_encoding).

DiscreteGlobalGrids.Encodings.cellaxis Function

cellaxis(enc::CellEncoding, grid::AbstractGrid, source; kw...) -> ChunkedCellVector

The stored axis as an AbstractVector of typed cell ids. source is whatever the encoding reads:

encoding	source	cost
RangesEncoding	the (n, 2) inclusive-range array	arithmetic, no IO
ImplicitEncoding	the axis length	arithmetic, no IO
DenseEncoding	the id vector, lazy or not	one chunked pass

Implemented in chunked_lookup.jl, where the axis type lives.

source

DiscreteGlobalGrids.Encodings.write_eligible Function

write_eligible(enc::CellEncoding, grid::AbstractGrid, ids::AbstractVector) -> Bool

Whether ids can be written in enc. This is what encoding = :auto asks: RangesEncoding is eligible exactly when the ids are sorted, unique, and all cells of grid's single level; ImplicitEncoding additionally requires them to be the whole level; DenseEncoding always is, which is what makes :auto total.

ids may be raw integers or typed cell ids.

source

DiscreteGlobalGrids.Encodings.encodingname Function

encodingname(enc::CellEncoding) -> String

The ENCODING_REGISTRY key enc is registered under — what a writer stamps into the store's attributes.

source

DiscreteGlobalGrids.Encodings.idrank Function

idrank(grid::AbstractGrid, id::Integer) -> Int

The number of cells of grid whose raw id is strictly less than id — a COUNT, so it is a zero-based rank and idrank(grid, rawid(c)) + 1 is globalindex(grid, c).

Total on the integer type. id need not name a cell: an id above every cell of the level answers ncells(grid), one below every cell answers 0, and one that is well formed but names nothing — a Z7 phantom on a pentagon's deleted branch — answers where it would sit. That totality is the whole point: a stored [start, stop] interval is counted by subtracting two ranks, and an interval's endpoints are not required to be cells.

grid is a complete level grid; rank is meaningless against a subset, which has localindex instead.

Required of a grid that is to be read from a store, together with idselect and idcount_between.

source

DiscreteGlobalGrids.Encodings.idselect Function

idselect(grid::AbstractGrid, r::Integer) -> Integer

The raw id of the cell at zero-based rank r in grid's canonical order — the inverse of idrank:

idrank(grid, idselect(grid, r)) == r          for r in 0:ncells(grid)-1
idselect(grid, idrank(grid, x)) == x          for every cell id x

Equivalently rawid(cellindex(grid, r + 1)), without constructing the typed id. r outside 0:ncells(grid)-1 throws a BoundsError.

source

DiscreteGlobalGrids.Encodings.idcount_between Function

idcount_between(grid::AbstractGrid, lo::Integer, hi::Integer) -> Int

The number of cells of grid whose raw id lies in the INCLUSIVE interval [lo, hi]; zero when hi < lo. Neither endpoint need name a cell.

This is what makes a ranges store readable without touching a data array: the axis length is the sum of this over the stored intervals, and the same sum prefixed gives the index of every interval's first cell.

The generic implementation is the rank difference, and is correct for any grid that implements idrank.

source

DiscreteGlobalGrids.Encodings.idvalid Function

idvalid(grid::AbstractGrid, id::Integer) -> Bool

Whether id names a cell of this grid's level. Never throws.

The ingest-time check for a dense axis: a Z7 id can be well formed and still name nothing (the twelve pentagons delete one child digit each), and DGGRID itself does not reject those, so a reader must.

source

DiscreteGlobalGrids.Encodings.idcell Function

idcell(grid::AbstractGrid, id::Integer) -> AbstractCellIndex

The typed cell id for a raw stored integer at this grid's level. The inverse of rawid given the level, which the raw integer does not always carry.

source

Errors

One exception type, defined layer-neutrally so that the encoding and lookup layers — which never learn what a store is — can throw it too. The store URL and the conventions that fired are optional context, added at the boundary by the layer that does know them.

DiscreteGlobalGrids.DGGSFormatError Type

DGGSFormatError(; check, store = nothing, conventions = nothing,
                declared = nothing, observed = nothing, detail = "")

Raised when a store cannot be read as it says it can: conventions that contradict each other, a grid name in no registry, an alias supplied twice, a length that does not check out, an id that names no cell.

• check: a symbol naming the check that failed, e.g. :level_disagreement or :unknown_grid_name. This is the field to test against.

• store: the store identifier, so an error from a lazy read still says where it came from.

• conventions: the conventions that fired, in the order they were tried.

• declared / observed: the two values that failed to reconcile.

• detail: one sentence saying what to do about it.

store and conventions are CONTEXT, and optional: the encoding and lookup layers see ids and lengths, not stores, and throw without them. The layer that does know the store adds it with with_store_context; showerror omits whatever is absent.

The policy this serves: vocabulary disagreement is refused rather than guessed.

source

DiscreteGlobalGrids.with_store_context Function

with_store_context(f, store; conventions = nothing)

Run f, and rethrow any DGGSFormatError it raises with store and conventions added.

This is how a store-aware caller — the Zarr extension around an encoding's validation pass — turns a context-free error from a lower layer into one that names the store it came from, without that layer ever learning what a store is.

source

DiscreteGlobalGrids.store_context Function

store_context(e::DGGSFormatError; store = nothing, conventions = nothing)

e with store and conventions filled in where it carries none. Context already on the error wins: the throw site knew more than the boundary does.

source

Index