Enforcing schema contracts for GeoJSON and Shapefiles

In a federated geospatial estate, an unvalidated vector payload is the single most common way one domain corrupts another. When a producer publishes GeoJSON or Shapefile derivatives without strict structural guarantees, the routing plane admits coordinate-order flips, coerced field types, and degenerate topology that surface only after they have poisoned a tile cache. This guide is a concrete operation under Schema Contracts for Vector/Tile Data, itself part of the Federated Ownership & Routing Architecture: it walks through a dual-validation gate that pins GeoJSON against JSON Schema and Shapefiles against a versioned field map, so a payload that clears producer-side CI also clears runtime ingestion. The gate runs before any geometry reaches the worker pool described in Optimizing async execution for spatial joins, so heavy compute never burns on a payload that was malformed at the boundary.

Dual-validation gate for GeoJSON and Shapefile payloads A vector payload enters and splits into two branches. The GeoJSON branch runs a JSON Schema and CRS84 check; the Shapefile branch runs an OGR field-map and geometry-type check. Both converge on a single decision: is the schema, CRS, and field map valid? A pass emits exit code 0 with a signed manifest to the consumer. A contract violation emits exit code 1 — an HTTP 422 with an error_code routed to the dead-letter path. A parser fault emits exit code 2 and is retried as an infrastructure fault. Vector payload GeoJSON or Shapefile GeoJSON branch JSON Schema validation + CRS84 axis-order check Shapefile branch OGR field-map diff + geometry-type check Schema, CRS & field map valid? fail parse error pass exit 1 — HTTP 422 error_code to dead-letter exit 0 — signed manifest to consumer exit 2 — parser fault infrastructure retry

Prerequisites

Requirement Value / Constraint
Validation tools ogrinfo (GDAL 3.6+), python3 -m jsonschema, jq, uuidgen
Orchestration kubectl for sidecar rollout; CI runner with pre-merge hook access
CRS assumption GeoJSON contracts pin urn:ogc:def:crs:OGC:1.3:CRS84 (EPSG:4326, lon/lat axis order); Shapefiles destined for GeoJSON output must carry WGS 84 SRS
Access roles contract-validator service account (read payloads, write manifests); GIS Data Steward for drift reconciliation
Env vars CONTRACT_ROOT, FIELD_MAP_PATH, MAX_FEATURES, STRICT_CRS
Versioned inputs feature-collection.schema.json and field-mapping.yaml committed alongside the domain manifest

The validator is strictly idempotent: it reads input streams without mutation, writes no temporary artifacts, and evaluates schemas in memory, so a retry loop with exponential backoff at the orchestrator level is always safe.

Step-by-Step Implementation

1. Validate GeoJSON against a versioned JSON Schema

The schema enforces geometry/property type strictness and a single canonical CRS. RFC 7946 mandates WGS 84 with longitude/latitude axis order and removes the crs member from GeoJSON objects, so the contract pins urn:ogc:def:crs:OGC:1.3:CRS84 — never urn:ogc:def:crs:EPSG::4326, whose axis order differs.

bash
# Strict contract validation; exit 1 on violation, 2 on parser failure.
python3 -m jsonschema \
  --instance payload.geojson \
  "${CONTRACT_ROOT}/geojson/v1/feature-collection.schema.json"

Verify the payload declares no forbidden CRS member and carries no null geometry before trusting the exit code:

bash
jq -e '(.crs == null) and ([.features[].geometry | select(. == null)] | length == 0)' \
  payload.geojson && echo "geojson structural pre-check: pass"

2. Verify Shapefile schema and geometry type with OGR

GDAL/OGR introspection is the authoritative read for Shapefile .dbf/.shp alignment. Confirm the layer geometry type and feature count match the contract before any field-level checks.

bash
# Geometry type, layer name, and feature count from the OGR driver.
ogrinfo -so -al input.shp | grep -E "Geometry|Layer name|Feature Count"

A Geometry: Polygon line that disagrees with the contract’s declared geometry_type is a hard rejection — do not pass it to topology repair.

3. Pin the field map and reject .dbf drift

Legacy exports coerce numeric fields to strings or truncate column names to 10 characters. The FIELD_MAP_PATH file is version-controlled, and any deviation between the .dbf header and the map triggers a hard failure rather than a silent rename.

bash
# Diff the live .dbf header against the contracted field map.
ogrinfo -so -al input.shp \
  | sed -n '/^[A-Za-z].*:.*([A-Za-z]/p' \
  | awk -F: '{gsub(/ /,"",$1); print $1}' | sort > /tmp/live_fields.txt
yq '.fields | keys | .[]' "${FIELD_MAP_PATH}" | sort > /tmp/contract_fields.txt
diff /tmp/contract_fields.txt /tmp/live_fields.txt && echo "field map: aligned"

4. Confirm CRS alignment for cross-format output

A Shapefile destined for GeoJSON output must reproject to WGS 84 before serialization. Read the SRS WKT directly so a mislabeled projection is caught at the gate, not after a consumer join silently offsets.

bash
ogrinfo -al -so input.shp | grep -A2 "SRS WKT"

If the WKT does not resolve to WGS 84, route the payload to reprojection rather than admitting it — silent projection drift is the failure that Cross-Domain Routing Strategies cannot recover from once it reaches the routing table.

5. Wrap the gate in a deterministic CI/CD script

The gate returns 0 for pass, 1 for contract violation, and 2 for parser/runtime failure, and emits structured JSON keyed by a correlation id so logs correlate across the pipeline.

bash
#!/usr/bin/env bash
set -euo pipefail

INPUT="${1:?Missing input file}"
CORRELATION_ID="$(uuidgen)"

echo "Starting validation: correlation_id=${CORRELATION_ID}"

# Geometry type and feature count check; OGR parse failure is exit 2.
ogrinfo -so -al "${INPUT}" | grep -E "Geometry|Feature Count" || {
  echo '{"level":"ERROR","error_code":"ERR_OGR_PARSE","correlation_id":"'"${CORRELATION_ID}"'"}'
  exit 2
}

echo '{"level":"INFO","status":"pass","correlation_id":"'"${CORRELATION_ID}"'"}'
exit 0

On success the gate emits a signed manifest (checksum.sha256, schema_version.json) that downstream consumers verify before mounting volumes or streaming through API Gateway Mapping for GIS Services. Validated feature counts and CRS alignment then feed the reconciliation in Domain Sync Protocols for Spatial Data.

Configuration Reference

Deploy the following to the validation sidecar or CI runner. Each parameter maps directly to one rejection class.

yaml
# contract-validator-config.yaml
validation:
  geojson:
    schema_path: /etc/contracts/geojson/v1/feature-collection.schema.json
    strict_crs: true
    allowed_crs: ["urn:ogc:def:crs:OGC:1.3:CRS84"]
    max_features: 50000
    reject_null_geometry: true
    enforce_flat_properties: true
  shapefile:
    dbf_encoding: "UTF-8"
    enforce_field_mapping: true
    field_map_path: /etc/contracts/shapefile/v1/field-mapping.yaml
    geometry_type: "POLYGON"
    max_record_length: 254
    require_spatial_index: true
Parameter Format Value Rationale
strict_crs geojson true Rejects any CRS other than allowed_crs
allowed_crs geojson OGC:1.3:CRS84 RFC 7946 lon/lat axis order; not EPSG::4326
reject_null_geometry geojson true Null geometry breaks spatial indexes downstream
enforce_flat_properties geojson true Nested objects are not tile-renderable
max_features geojson 50000 Caps per-payload memory in the validator
dbf_encoding shapefile UTF-8 Prevents mojibake on non-ASCII attribute values
enforce_field_mapping shapefile true Binds .dbf header to a version-controlled map
geometry_type shapefile POLYGON Hard-rejects mixed or wrong geometry layers
max_record_length shapefile 254 DBF character-field ceiling; longer fields are truncated at source

Common Failure Modes & Fixes

  • Symptom: validation aborts with ERR_GEOJSON_NESTED_OBJECT. Root cause: a producer shipped nested property objects, violating enforce_flat_properties: true. Fix: flatten at source or strip with jq '.features[].properties |= with_entries(select(.value | type != "object"))', then bump the contract version.
  • Symptom: ERR_DBF_FIELD_COERCION on a numeric column. Root cause: a legacy export coerced an integer field (e.g. POPULATION) to string. Fix: correct the export type, or add an explicit cast in field-mapping.yaml and re-run step 3.
  • Symptom: ERR_SHP_TRUNCATED_FIELD. Root cause: a DBF column name exceeds 10 characters and was silently truncated by the writer. Fix: rename at source or register an alias in FIELD_MAP_PATH so the live header diffs clean.
  • Symptom: ERR_TOPOLOGY_DEGENERATE. Root cause: a polygon ring has fewer than 4 coordinate pairs or self-intersects. Fix: reject and route to the topology-repair queue; surface offenders with jq '.features[] | select(.geometry.type != "Polygon" or (.geometry.coordinates | length) < 1)' payload.geojson.
  • Symptom: consumer joins are offset by hundreds of metres. Root cause: silent projection drift — a payload labeled CRS84 actually carries EPSG:3857 coordinates. Fix: re-run step 4, reject on SRS mismatch, and quarantine the producer pipeline until the contract version is corrected.

When a batch crosses thresholds, escalate by impact scope: a single contract violation auto-rejects and notifies the domain steward (P3); a batch failure rate above 5% halts ingestion and triggers a drift audit (P2); a cascading serialization breach with tile-cache corruption isolates the routing table and pauses the slicer with kubectl rollout pause deployment/tile-slicer before restoring the last known-good manifest (P1).

FAQ

Why pin OGC:1.3:CRS84 instead of EPSG:4326 in a GeoJSON contract?

RFC 7946 requires WGS 84 with longitude-then-latitude axis order and drops the crs member entirely. The URN urn:ogc:def:crs:OGC:1.3:CRS84 encodes that lon/lat order explicitly, whereas urn:ogc:def:crs:EPSG::4326 formally implies lat/lon. Pinning the EPSG form invites tools that honour authority axis order to swap coordinates, producing a silent offset that no JSON-level check would catch.

How does the gate stay idempotent across retries?

It reads input streams without modification, writes no temporary artifacts, and evaluates schemas purely in memory. The same payload always yields the same exit code and the same payload_hash, so an orchestrator can retry with exponential backoff without ever leaving partial writes in the staging bucket.

What is the difference between exit code 1 and exit code 2?

Exit 1 is a contract violation: the file parsed cleanly but failed an assertion (wrong CRS, coerced field, degenerate ring) and belongs in the dead-letter path with an error_code. Exit 2 is a parser or runtime failure — corrupt .shp, unreadable .dbf, OGR crash — which is an infrastructure fault and should trigger a retry, not a producer-quarantine decision.

Should the validator repair geometry, or only reject it?

Only reject. The gate is a trust boundary, not a transform stage. Mixing repair into validation makes the pass/fail decision non-deterministic and hides the producer defect. Route ERR_TOPOLOGY_DEGENERATE payloads to a dedicated repair queue so the original contract violation stays auditable.

Where should the gate run — CI, ingress, or batch?

All three, on the same config. A pre-commit hook blocks invalid payloads at merge with pull-request annotations; an ingress sidecar intercepts object-storage uploads and returns HTTP 422 before tile slicing; an async batch validator sweeps legacy data lakes on a schedule and emits drift reports when coordinate precision degrades below 1e-6 decimal degrees. Sharing one config keeps producer-side CI and runtime ingestion in agreement.

Standards referenced