Enforcing schema contracts for GeoJSON and Shapefiles
In a federated geospatial estate, an unvalidated vector payload is the single most common way one domain corrupts another. When a producer publishes GeoJSON or Shapefile derivatives without strict structural guarantees, the routing plane admits coordinate-order flips, coerced field types, and degenerate topology that surface only after they have poisoned a tile cache. This guide is a concrete operation under Schema Contracts for Vector/Tile Data, itself part of the Federated Ownership & Routing Architecture: it walks through a dual-validation gate that pins GeoJSON against JSON Schema and Shapefiles against a versioned field map, so a payload that clears producer-side CI also clears runtime ingestion. The gate runs before any geometry reaches the worker pool described in Optimizing async execution for spatial joins, so heavy compute never burns on a payload that was malformed at the boundary.
Prerequisites
| Requirement | Value / Constraint |
|---|---|
| Validation tools | ogrinfo (GDAL 3.6+), python3 -m jsonschema, jq, uuidgen |
| Orchestration | kubectl for sidecar rollout; CI runner with pre-merge hook access |
| CRS assumption | GeoJSON contracts pin urn:ogc:def:crs:OGC:1.3:CRS84 (EPSG:4326, lon/lat axis order); Shapefiles destined for GeoJSON output must carry WGS 84 SRS |
| Access roles | contract-validator service account (read payloads, write manifests); GIS Data Steward for drift reconciliation |
| Env vars | CONTRACT_ROOT, FIELD_MAP_PATH, MAX_FEATURES, STRICT_CRS |
| Versioned inputs | feature-collection.schema.json and field-mapping.yaml committed alongside the domain manifest |
The validator is strictly idempotent: it reads input streams without mutation, writes no temporary artifacts, and evaluates schemas in memory, so a retry loop with exponential backoff at the orchestrator level is always safe.
Step-by-Step Implementation
1. Validate GeoJSON against a versioned JSON Schema
The schema enforces geometry/property type strictness and a single canonical CRS. RFC 7946 mandates WGS 84 with longitude/latitude axis order and removes the crs member from GeoJSON objects, so the contract pins urn:ogc:def:crs:OGC:1.3:CRS84 — never urn:ogc:def:crs:EPSG::4326, whose axis order differs.
# Strict contract validation; exit 1 on violation, 2 on parser failure.
python3 -m jsonschema \
--instance payload.geojson \
"${CONTRACT_ROOT}/geojson/v1/feature-collection.schema.json"
Verify the payload declares no forbidden CRS member and carries no null geometry before trusting the exit code:
jq -e '(.crs == null) and ([.features[].geometry | select(. == null)] | length == 0)' \
payload.geojson && echo "geojson structural pre-check: pass"
2. Verify Shapefile schema and geometry type with OGR
GDAL/OGR introspection is the authoritative read for Shapefile .dbf/.shp alignment. Confirm the layer geometry type and feature count match the contract before any field-level checks.
# Geometry type, layer name, and feature count from the OGR driver.
ogrinfo -so -al input.shp | grep -E "Geometry|Layer name|Feature Count"
A Geometry: Polygon line that disagrees with the contract’s declared geometry_type is a hard rejection — do not pass it to topology repair.
3. Pin the field map and reject .dbf drift
Legacy exports coerce numeric fields to strings or truncate column names to 10 characters. The FIELD_MAP_PATH file is version-controlled, and any deviation between the .dbf header and the map triggers a hard failure rather than a silent rename.
# Diff the live .dbf header against the contracted field map.
ogrinfo -so -al input.shp \
| sed -n '/^[A-Za-z].*:.*([A-Za-z]/p' \
| awk -F: '{gsub(/ /,"",$1); print $1}' | sort > /tmp/live_fields.txt
yq '.fields | keys | .[]' "${FIELD_MAP_PATH}" | sort > /tmp/contract_fields.txt
diff /tmp/contract_fields.txt /tmp/live_fields.txt && echo "field map: aligned"
4. Confirm CRS alignment for cross-format output
A Shapefile destined for GeoJSON output must reproject to WGS 84 before serialization. Read the SRS WKT directly so a mislabeled projection is caught at the gate, not after a consumer join silently offsets.
ogrinfo -al -so input.shp | grep -A2 "SRS WKT"
If the WKT does not resolve to WGS 84, route the payload to reprojection rather than admitting it — silent projection drift is the failure that Cross-Domain Routing Strategies cannot recover from once it reaches the routing table.
5. Wrap the gate in a deterministic CI/CD script
The gate returns 0 for pass, 1 for contract violation, and 2 for parser/runtime failure, and emits structured JSON keyed by a correlation id so logs correlate across the pipeline.
#!/usr/bin/env bash
set -euo pipefail
INPUT="${1:?Missing input file}"
CORRELATION_ID="$(uuidgen)"
echo "Starting validation: correlation_id=${CORRELATION_ID}"
# Geometry type and feature count check; OGR parse failure is exit 2.
ogrinfo -so -al "${INPUT}" | grep -E "Geometry|Feature Count" || {
echo '{"level":"ERROR","error_code":"ERR_OGR_PARSE","correlation_id":"'"${CORRELATION_ID}"'"}'
exit 2
}
echo '{"level":"INFO","status":"pass","correlation_id":"'"${CORRELATION_ID}"'"}'
exit 0
On success the gate emits a signed manifest (checksum.sha256, schema_version.json) that downstream consumers verify before mounting volumes or streaming through API Gateway Mapping for GIS Services. Validated feature counts and CRS alignment then feed the reconciliation in Domain Sync Protocols for Spatial Data.
Configuration Reference
Deploy the following to the validation sidecar or CI runner. Each parameter maps directly to one rejection class.
# contract-validator-config.yaml
validation:
geojson:
schema_path: /etc/contracts/geojson/v1/feature-collection.schema.json
strict_crs: true
allowed_crs: ["urn:ogc:def:crs:OGC:1.3:CRS84"]
max_features: 50000
reject_null_geometry: true
enforce_flat_properties: true
shapefile:
dbf_encoding: "UTF-8"
enforce_field_mapping: true
field_map_path: /etc/contracts/shapefile/v1/field-mapping.yaml
geometry_type: "POLYGON"
max_record_length: 254
require_spatial_index: true
| Parameter | Format | Value | Rationale |
|---|---|---|---|
strict_crs |
geojson | true |
Rejects any CRS other than allowed_crs |
allowed_crs |
geojson | OGC:1.3:CRS84 |
RFC 7946 lon/lat axis order; not EPSG::4326 |
reject_null_geometry |
geojson | true |
Null geometry breaks spatial indexes downstream |
enforce_flat_properties |
geojson | true |
Nested objects are not tile-renderable |
max_features |
geojson | 50000 |
Caps per-payload memory in the validator |
dbf_encoding |
shapefile | UTF-8 |
Prevents mojibake on non-ASCII attribute values |
enforce_field_mapping |
shapefile | true |
Binds .dbf header to a version-controlled map |
geometry_type |
shapefile | POLYGON |
Hard-rejects mixed or wrong geometry layers |
max_record_length |
shapefile | 254 |
DBF character-field ceiling; longer fields are truncated at source |
Common Failure Modes & Fixes
- Symptom: validation aborts with
ERR_GEOJSON_NESTED_OBJECT. Root cause: a producer shipped nested property objects, violatingenforce_flat_properties: true. Fix: flatten at source or strip withjq '.features[].properties |= with_entries(select(.value | type != "object"))', then bump the contract version. - Symptom:
ERR_DBF_FIELD_COERCIONon a numeric column. Root cause: a legacy export coerced an integer field (e.g.POPULATION) to string. Fix: correct the export type, or add an explicit cast infield-mapping.yamland re-run step 3. - Symptom:
ERR_SHP_TRUNCATED_FIELD. Root cause: a DBF column name exceeds 10 characters and was silently truncated by the writer. Fix: rename at source or register an alias inFIELD_MAP_PATHso the live header diffs clean. - Symptom:
ERR_TOPOLOGY_DEGENERATE. Root cause: a polygon ring has fewer than 4 coordinate pairs or self-intersects. Fix: reject and route to the topology-repair queue; surface offenders withjq '.features[] | select(.geometry.type != "Polygon" or (.geometry.coordinates | length) < 1)' payload.geojson. - Symptom: consumer joins are offset by hundreds of metres. Root cause: silent projection drift — a payload labeled CRS84 actually carries
EPSG:3857coordinates. Fix: re-run step 4, reject on SRS mismatch, and quarantine the producer pipeline until the contract version is corrected.
When a batch crosses thresholds, escalate by impact scope: a single contract violation auto-rejects and notifies the domain steward (P3); a batch failure rate above 5% halts ingestion and triggers a drift audit (P2); a cascading serialization breach with tile-cache corruption isolates the routing table and pauses the slicer with kubectl rollout pause deployment/tile-slicer before restoring the last known-good manifest (P1).
FAQ
Why pin OGC:1.3:CRS84 instead of EPSG:4326 in a GeoJSON contract?
RFC 7946 requires WGS 84 with longitude-then-latitude axis order and drops the crs member entirely. The URN urn:ogc:def:crs:OGC:1.3:CRS84 encodes that lon/lat order explicitly, whereas urn:ogc:def:crs:EPSG::4326 formally implies lat/lon. Pinning the EPSG form invites tools that honour authority axis order to swap coordinates, producing a silent offset that no JSON-level check would catch.
How does the gate stay idempotent across retries?
It reads input streams without modification, writes no temporary artifacts, and evaluates schemas purely in memory. The same payload always yields the same exit code and the same payload_hash, so an orchestrator can retry with exponential backoff without ever leaving partial writes in the staging bucket.
What is the difference between exit code 1 and exit code 2?
Exit 1 is a contract violation: the file parsed cleanly but failed an assertion (wrong CRS, coerced field, degenerate ring) and belongs in the dead-letter path with an error_code. Exit 2 is a parser or runtime failure — corrupt .shp, unreadable .dbf, OGR crash — which is an infrastructure fault and should trigger a retry, not a producer-quarantine decision.
Should the validator repair geometry, or only reject it?
Only reject. The gate is a trust boundary, not a transform stage. Mixing repair into validation makes the pass/fail decision non-deterministic and hides the producer defect. Route ERR_TOPOLOGY_DEGENERATE payloads to a dedicated repair queue so the original contract violation stays auditable.
Where should the gate run — CI, ingress, or batch?
All three, on the same config. A pre-commit hook blocks invalid payloads at merge with pull-request annotations; an ingress sidecar intercepts object-storage uploads and returns HTTP 422 before tile slicing; an async batch validator sweeps legacy data lakes on a schedule and emits drift reports when coordinate precision degrades below 1e-6 decimal degrees. Sharing one config keeps producer-side CI and runtime ingestion in agreement.
Related
- Schema Contracts for Vector/Tile Data — parent reference for the contract-first validation model
- Optimizing async execution for spatial joins — the worker pool that consumes only validated payloads
- Cross-Domain Routing Strategies — contract-aware dispatch that depends on a clean CRS at the gate
- Domain Sync Protocols for Spatial Data — reconciling divergent schema versions across domains