# Metadata Cataloging for Raster/Vector

Metadata cataloging is where a federated spatial estate becomes discoverable: it is the control plane that turns owned raster and vector products into routable, queryable, compliant assets. This reference sits inside Geospatial Data Mesh Fundamentals and addresses one architectural concern precisely — how enterprise platforms register, validate, and synchronize raster and vector metadata under strict domain-driven constraints. In a federated topology, catalog entries function as immutable infrastructure contracts rather than passive documentation. Platform engineers and GIS data stewards must treat catalog registration as a security boundary, enforcing absolute domain isolation, explicit ownership, and event-driven synchronization. This is the operational consequence of the shift described in Data Mesh vs Traditional GIS Architecture: once a centralized geodatabase no longer holds the canonical record, metadata becomes the primary mechanism for product distribution, lineage tracking, and access routing.

Figure — Metadata is validated at ingestion and registered idempotently, turning the catalog into the discovery and routing control plane.

# Architectural Boundaries & Design Rationale

Architectural isolation begins at the namespace level. Each spatial product domain must be provisioned with a dedicated metadata partition, isolated compute routing, and explicit tenant identifiers. This is not an optimization — it is the failure-mode boundary that prevents one domain’s CRS drift, schema mutation, or topology corruption from cascading into another domain’s discovery layer. The contractual limits that govern how raster tiles and vector features are partitioned across the mesh are defined upstream in Spatial Domain Boundary Design; the catalog enforces those limits at registration time rather than trusting producers to self-police.

Configure domain-specific routing tables that map spatial extents (WKT/GeoJSON envelopes) to catalog partitions using deterministic hashing. A routing proxy intercepts catalog queries, evaluates the requested bounding box against registered domain boundaries, and forwards requests only to authorized partitions. Cross-domain metadata leakage is prevented by enforcing strict network policies and schema-level tenant validation at the ingress layer — the same zero-trust posture that Cross-Domain Routing Strategies applies to query traffic.

This pattern exists to prevent four concrete failure modes:

• Silent cross-domain joins — a consumer queries a bounding box that straddles two domains and unknowingly mixes products with incompatible CRS or freshness SLAs.
• Catalog drift — repeated CI/CD runs re-register the same asset under slightly different identifiers, fracturing discovery.
• Ownership ambiguity — an asset is registered without an enforceable product_owner_id, leaving lineage and incident response with no accountable party.
• Stale discovery — object storage is updated but the catalog is not, so consumers route to artifacts that no longer exist.

# Idempotent Domain Provisioning

Domain boundaries must be provisioned idempotently to prevent drift during pipeline executions. Use transactional DDL with IF NOT EXISTS guards and deterministic hash-based partitioning keys, and bind Row-Level Security to the domain identifier so the database itself enforces isolation.

-- Idempotent domain schema provisioning
CREATE SCHEMA IF NOT EXISTS domain_agriculture_vegetation;
ALTER SCHEMA domain_agriculture_vegetation OWNER TO platform_admin;

-- Enforce Row-Level Security (RLS) bound to domain_id
ALTER TABLE domain_agriculture_vegetation.spatial_metadata ENABLE ROW LEVEL SECURITY;
CREATE POLICY domain_isolation_policy ON domain_agriculture_vegetation.spatial_metadata
    USING (domain_id = current_setting('app.current_domain_id', true));

# Routing Proxy Configuration

Deploy a metadata routing sidecar (Envoy or Linkerd) configured with spatial extent matching rules. Route queries via X-Domain-Id and X-Spatial-Extent headers. The proxy must evaluate bounding box intersections before forwarding to the catalog partition, denying any principal that attempts a cross-domain read.

# Envoy HTTP Router Configuration (Excerpt)
- match:
    headers:
      - name: "x-domain-id"
        string_match: { exact: "domain_agriculture_vegetation" }
    prefix: "/catalog/query"
  route:
    cluster: catalog_partition_ag_veg
    timeout: 3s
  typed_per_filter_config:
    envoy.filters.http.rbac:
      "@type": type.googleapis.com/envoy.extensions.filters.http.rbac.v3.RBACPerRoute
      rbac:
        action: DENY
        policies:
          cross_domain_deny:
            permissions:
              - not_rule:
                  any: true
            principals:
              - any: true

Boundary validation middleware then rejects catalog registrations whose declared CRS, bounding box, or topology rules conflict with the registered domain contract. Middleware executes synchronously during POST /metadata/register and returns 409 Conflict with a structured error payload on mismatch.

# Specification & Contract Reference

Raster and vector datasets require divergent schema definitions that must be codified as versioned contracts. Treat each catalog entry as a data product with explicit SLAs, ownership metadata, and consumption constraints — the same discipline applied in Product Thinking for GIS Datasets. The mandatory contract surface for every registered asset is as follows.

The routing and isolation contract is carried in transport headers and policy inputs rather than the payload:

Codify these fields as JSON Schema (or Protobuf) and run validation gates synchronously during registration and asynchronously during batch reconciliation.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "SpatialProductMetadata",
  "type": "object",
  "required": ["product_id", "domain_id", "crs_epsg", "spatial_resolution", "temporal_coverage", "product_owner_id"],
  "properties": {
    "product_id": { "type": "string", "format": "uuid" },
    "domain_id": { "type": "string", "pattern": "^domain_[a-z0-9_]+$" },
    "crs_epsg": { "type": "integer", "minimum": 1000, "maximum": 99999 },
    "spatial_resolution": { "type": "number", "exclusiveMinimum": 0 },
    "band_configuration": { "type": "array", "items": { "type": "string" } },
    "topology_rule_set": { "type": "string", "enum": ["none", "planar", "network", "3d_solid"] },
    "temporal_coverage": {
      "type": "object",
      "properties": {
        "start": { "type": "string", "format": "date-time" },
        "end": { "type": "string", "format": "date-time" }
      }
    },
    "data_freshness_sla_minutes": { "type": "integer", "minimum": 0 },
    "product_owner_id": { "type": "string", "format": "email" }
  },
  "additionalProperties": false
}

EPSG codes extend beyond four digits — UTM zone codes such as EPSG:32633 are five-digit integers — so maximum is set to 99999 to cover the current registry range. Geometry precision must be carried explicitly in spatial_resolution: a vector product asserting sub-metre topology that is stored at coarser precision will fail planar topology validation downstream. Schema evolution is governed by a monotonic schema_version counter; breaking changes trigger a parallel catalog migration rather than an in-place mutation, and deprecated contracts remain queryable for a defined retention window (e.g. 90 days) to protect downstream consumers. Vector tile producers that publish through the routing layer should align this version contract with Schema Contracts for Vector Tile Data so tile and catalog versions never diverge.

# Production Implementation

Catalog registration must be strictly idempotent and zero-trust by default: duplicate submissions, network retries, and out-of-order event deliveries must resolve to a single canonical state, and no mutation persists without an explicit policy decision. The registration pipeline derives a deterministic idempotency key and performs a conflict-resolving UPSERT, so the same asset registered twice converges instead of forking.

import hashlib
import json

def register_metadata(payload: dict, db) -> dict:
    # Zero-trust: caller identity is verified upstream by the routing proxy;
    # this function trusts only the validated, domain-scoped payload.

    # 1. Deterministic idempotency key — identical assets converge,
    #    differing assets collide and must bump schema_version.
    idempotency_key = hashlib.sha256(
        f"{payload['product_id']}|{payload['crs_epsg']}|{payload['spatial_extent_hash']}".encode()
    ).hexdigest()

    # 2. UPSERT with conflict resolution: retries are safe and side-effect free.
    query = """
        INSERT INTO spatial_metadata (idempotency_key, payload, domain_id, created_at, updated_at)
        VALUES (%s, %s, %s, NOW(), NOW())
        ON CONFLICT (idempotency_key)
        DO UPDATE SET
            payload = EXCLUDED.payload,
            updated_at = NOW()
        RETURNING id, updated_at;
    """
    return db.execute(query, (idempotency_key, json.dumps(payload), payload['domain_id']))

Every mutation is then evaluated against an OPA/Rego policy before persistence. The policy validates tenant ownership (the caller’s domain_id must equal the body’s domain_id), enforces the CRS allowlist, blocks restricted data classifications, and restricts write roles — a default allow = false posture means anything not explicitly permitted is denied.

package catalog.security

import rego.v1

default allow = false

allow if {
    input.method == "POST"
    input.path == ["catalog", "register"]
    input.user.domain_id == input.body.domain_id
    input.user.role in ["data_steward", "platform_engineer"]
    input.body.crs_epsg in [4326, 3857, 32610, 32611, 32612, 32633]
    not "restricted" in input.body.tags
}

Run the policy as a sidecar admission step (opa eval in CI, or the OPA HTTP API at runtime) so that registration, RLS, and routing all enforce the same domain contract independently — no single bypass can leak a product across a boundary.

# Diagnostic Runbook

Platform engineers must maintain deterministic diagnostic pathways for catalog failures. The following steps isolate the most common failure modes in production, from header propagation through policy evaluation to registry sync.

1. Routing proxy rejection (HTTP 403/404). Verify the X-Domain-Id header matches the provisioned schema and check Envoy access logs for RBAC_DENIED events. Confirm the requested extent actually intersects the domain boundary:

   SELECT ST_Intersects(
     ST_MakeEnvelope(xmin, ymin, xmax, ymax, 4326),
     domain_boundary
   ) FROM domain_registry WHERE domain_id = 'domain_agriculture_vegetation';
   

   Remediation: update the routing table or adjust the domain boundary WKT, then reload the sidecar via the admin API or pod restart.

2. Schema validation failure (HTTP 400). Inspect the JSON Schema error payload for missing required fields or additionalProperties violations, and cross-reference the CRS against the enterprise allowlist. Remediation: patch the payload to match the contract; if the contract itself changed, increment schema_version and trigger a parallel migration.

3. Policy evaluation denial (OPA allow = false). Run opa eval -i input.json -d policy.rego "data.catalog.security.allow" with the exact request body. A false result with a matching domain_id usually means a role or tag violation. Remediation: correct the caller role binding or strip the restricted tag; never widen the CRS allowlist to force a pass.

4. RLS permission denial (PostgreSQL error 42501). Confirm app.current_domain_id is set at connection initialization and verify the policy attachment:

   SELECT policyname, cmd, qual FROM pg_policies
   WHERE tablename = 'spatial_metadata';
   

   Remediation: ensure the connection pool middleware injects tenant context before executing queries.

5. Idempotency key collision. Look up the existing idempotency_key in spatial_metadata and compare updated_at timestamps to detect stale retries. Remediation: if the payload matches, return 200 OK with the existing record ID; if it differs, reject with 409 Conflict and require an explicit version bump.

6. Registry / storage drift. When consumers report missing artifacts, reconcile the catalog against object-storage manifests and PostGIS extents. Remediation: re-run the reconciliation job for the affected domain_id and route flagged records to the governance queue, following the same reconciliation cadence as Domain Sync Protocols for Spatial Data.

7. SLA breach on discovery latency. If catalog query p95 exceeds target, check partition fan-out and sidecar timeout (3s above). Remediation: split oversized partitions by spatial extent or raise replica count for the hot partition.

# SLA Targets & Performance Baselines

Catalog SLAs are first-class product guarantees, not best-effort metrics. The following baselines apply per domain partition and feed the alerting rules that govern remediation.

# Governance & Compliance Notes

Catalog integrity requires continuous reconciliation backed by policy-as-code hooks. Schedule asynchronous batch jobs that validate registered metadata against source storage (object-storage manifests, PostGIS geometry extents), flag drift events, and route them to a governance queue for manual or automated remediation. Every catalog mutation must emit an append-only audit event capturing the actor (product_owner_id or service principal), the domain_id, the resolved idempotency_key, the prior and new schema_version, and the policy decision — this audit trail is what lets compliance reconstruct exactly who registered which raster tile or vector feature, when, and under which contract.

Jurisdictional constraints attach at the metadata layer: products tagged with residency or classification labels (e.g. restricted) must be denied registration into partitions that do not satisfy the corresponding boundary, and the Rego allowlist is the single enforcement point for those labels. CRS governance is equally non-negotiable — only the enterprise-approved EPSG set may be registered, because an unapproved projection silently breaks cross-domain spatial joins and undermines every downstream SLA. Retention of deprecated contracts (the 90-day queryable window) is a compliance requirement, not a convenience: it guarantees that audit and rollback can resolve historical lineage. The end state across the mesh is that every raster tile and vector feature remains discoverable, compliant, and securely routed — the operational discipline detailed in Best practices for spatial metadata catalogs keeps it that way over the product lifecycle.

# Related

• Geospatial Data Mesh Fundamentals — parent reference and architectural overview
• Spatial Domain Boundary Design — the boundary contracts the catalog enforces
• Product Thinking for GIS Datasets — ownership and SLA model behind metadata contracts
• Spatial Product Lifecycle Management — versioning, deprecation, and retention
• Best practices for spatial metadata catalogs — operational guidance for this concern
• GeoParquet vs Cloud Optimized GeoTIFF for Mesh Distribution — choosing the output-port format a cataloged product advertises