Pinning PROJ Transformation Pipelines in CI

PROJ chooses a datum transformation at runtime from the grid files present on the machine, which means the same ogr2ogr command on two workers can produce coordinates metres apart — deterministically, silently, and with both results perfectly valid. This guide removes the choice: pin the transformation explicitly, ship the grids as a versioned image dependency, and add the CI check that fails a build whose declared pipeline cannot be resolved. It enforces the reproducibility requirement in CRS Governance and Reprojection Standards within Geospatial Data Mesh Fundamentals, and pairs with Standardizing on EPSG:4326 Across Domains.

Prerequisites

Requirement Value / Assumption Notes
Tools PROJ ≥ 9.2 with projinfo, GDAL ≥ 3.6, docker projinfo --grid-check reports availability
Grid package A versioned archive of the datum grids the estate uses Never projsync at runtime
CI A pipeline that builds the image and runs the checks below The check must gate, not report
Access roles platform-engineer (image), gis-data-steward (approved transformations) The registry is governed
Environment PROJ_GRID_PACKAGE, SOURCE_CRS, TARGET_CRS Exported before running

Step-by-Step Implementation

1. Enumerate the candidate transformations and their accuracy

Before pinning anything, see what PROJ would choose and what the alternatives cost.

bash
# Every candidate transformation between two CRS, ordered by PROJ's own preference,
# with the accuracy of each and whether its grid is present on this machine.
projinfo -s "$SOURCE_CRS" -t "$TARGET_CRS" --spatial-test intersects --summary
projinfo -s "$SOURCE_CRS" -t "$TARGET_CRS" --grid-check known -o PROJ | head -40

What PROJ falls back to, and what each fallback costs in metresFour transformation outcomes for the same source-to-target pair, in metres of positional error, against a 0.2 metre survey tolerance. The pinned grid-based pipeline lands at 0.05 metres. A 7-parameter Helmert fallback, chosen when the grid is unavailable, lands at about 2. A 3-parameter fallback lands at about 6. An assumed null transformation, applied when no candidate is found and the datums are treated as equivalent, can be over 100 metres out — and every one of these outcomes produces perfectly valid geometry.pinned grid pipeline0.05 m7-param fallback2 m3-param fallback6 massumed null118 mno check detects itsurvey tolerance

Verify the difference between candidates is material rather than theoretical:

bash
# The same point through the best grid-based pipeline and through a Helmert fallback.
echo "459200 6621500" | cs2cs -f "%.8f" \
  "EPSG:32633" "+proj=pipeline +step +inv +proj=utm +zone=33 +ellps=GRS80 \
   +step +proj=hgridshift +grids=national-grid-2024.gsb \
   +step +proj=unitconvert +xy_in=rad +xy_out=deg"
echo "459200 6621500" | cs2cs -f "%.8f" "EPSG:32633" "EPSG:4326"
# A difference in the fifth decimal is roughly a metre. That is the whole problem.

2. Bake the grids into the image and forbid runtime resolution

dockerfile
# Dockerfile — grids are a build input, never fetched at run time.
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.9.0

ARG PROJ_GRID_PACKAGE=proj-grids-2024.2.tar.gz
COPY ${PROJ_GRID_PACKAGE} /tmp/grids.tar.gz
RUN mkdir -p /usr/share/proj && tar -xzf /tmp/grids.tar.gz -C /usr/share/proj \
 && rm /tmp/grids.tar.gz

# PROJ will silently download missing grids over the network unless told not to.
# A pipeline whose grids arrive at run time is not reproducible.
ENV PROJ_NETWORK=OFF
ENV PROJ_DATA=/usr/share/proj

# Fail the build if a transformation the estate declares cannot be resolved from
# the packaged grids — catching at build time what would otherwise be a field offset.
ARG REQUIRED_TRANSFORMS="EPSG:32633>EPSG:4326 EPSG:27700>EPSG:4326"
RUN for t in ${REQUIRED_TRANSFORMS}; do \
      s="${t%%>*}"; d="${t##*>}"; \
      projinfo -s "$s" -t "$d" --grid-check known \
        | grep -q "Grid .* not found" && { echo "MISSING GRID: $s -> $d"; exit 1; } ; \
      echo "resolved: $s -> $d"; \
    done

Verify inside the image rather than on a developer machine, which is where the two diverge:

Three things must be pinned together; any one alone leaves a path openA reproducible reprojection requires the transformation pipeline stated explicitly at the call site, the grid package baked into the image as a versioned build argument, and network resolution disabled so PROJ cannot fetch a missing grid at run time. All three feed the warp that produces the canonical artifact. Each drops onto a rail describing what its omission alone still permits: PROJ choosing per machine, grids differing per host, and a grid arriving over the network at run time.Explicit -ctpipeline statedBaked gridsversioned build argPROJ_NETWORK=OFFno runtime fetchReproducible warpsame result anywhere++→PROJ chooses per machinegrids differ per hostgrid fetched at run timeSilent metre-scale driftvalid geometry, wrong place

bash
docker build -t gis-pipeline:test --build-arg PROJ_GRID_PACKAGE=proj-grids-2024.2.tar.gz .
docker run --rm gis-pipeline:test projinfo -s EPSG:32633 -t EPSG:4326 --grid-check known \
  | grep -c 'available'

3. Pin the pipeline at the call site

-s_srs/-t_srs describe endpoints and leave the route to PROJ. -ct states the route.

bash
ogr2ogr -f PostgreSQL "PG:${PG_DSN}" "$SOURCE_URI" \
  -s_srs "EPSG:32633" -t_srs "EPSG:4326" \
  -ct "+proj=pipeline \
       +step +inv +proj=utm +zone=33 +ellps=GRS80 \
       +step +proj=hgridshift +grids=national-grid-2024.gsb \
       +step +proj=unitconvert +xy_in=rad +xy_out=deg" \
  -lco GEOMETRY_NAME=geom -lco SPATIAL_INDEX=GIST -nlt PROMOTE_TO_MULTI

Verify with a control point whose EPSG:4326 position is independently known — the only check that distinguishes a wrong-but-valid reprojection:

bash
psql -c "SELECT ST_AsText(ST_SnapToGrid(geom, 0.0000001)) FROM staging WHERE ref_code='CTRL-0001';"

4. Record the provenance on every artifact

python
# reprojection_provenance.py — what has to travel with the artifact for a
# positional claim to be defensible later.
import subprocess


def provenance(source_crs: str, target_crs: str, pipeline: str) -> dict:
    proj_version = subprocess.run(["projinfo", "--version"], capture_output=True,
                                  text=True).stdout.strip()
    grid_pkg = subprocess.run(["cat", "/usr/share/proj/PACKAGE_VERSION"],
                              capture_output=True, text=True).stdout.strip()
    return {
        "source_crs": source_crs,
        "target_crs": target_crs,
        "transformation_pipeline": pipeline,   # the exact +proj=pipeline string
        "proj_version": proj_version,
        "grid_package": grid_pkg,
        "network_resolution": "disabled",
    }

What has to be recorded for a positional claim to be defensible laterSix provenance fields against whether they can be reconstructed after the fact. The source CRS as declared and the target CRS are usually recoverable from the manifest. The transformation pipeline actually applied, the grid package version and the PROJ version are all resolved at run time and cannot be reconstructed once the run is gone. Whether network resolution was enabled is likewise unrecoverable. Every one is available at pipeline runtime and costs nothing to persist.Recoverable laterCost to recorddeclared source CRSfrom the manifestnonetarget CRSfrom the manifestnonetransformation pipelinenoone stringgrid package versionnoone stringPROJ versionnoone stringnetwork resolutionnoone flag

Configuration Reference

Setting Value Effect Omission
PROJ_NETWORK OFF Forbids runtime grid download Results depend on network state
PROJ_DATA /usr/share/proj Where the baked grids live PROJ falls back to a partial default set
-ct Explicit +proj=pipeline Pins the route, not just the endpoints PROJ chooses per machine
Grid package version Pinned build arg Reproducibility across rebuilds A rebuild silently changes coordinates
REQUIRED_TRANSFORMS Estate’s declared pairs Build fails on a missing grid A field offset instead
Provenance record On every artifact Defensible positional claims Unreconstructable afterwards

Common Failure Modes & Fixes

Two workers produce coordinates differing by a metre. Root cause: unpinned transformation plus differing grid availability. Fix: -ct plus baked grids plus PROJ_NETWORK=OFF; all three, since any one alone leaves a path open.

The build passes and production still picks a Helmert fallback. Root cause: the grid is present but the pipeline string references it by a name PROJ resolves differently, so the pipeline silently falls back. Fix: projinfo --grid-check known reports availability per candidate; assert on it rather than on the file existing.

Coordinates change after a routine base-image update. Root cause: the base image shipped a different PROJ version whose transformation preferences differ. Fix: pin the base image digest and treat a PROJ upgrade as a pipeline-version bump with re-measurement.

PROJ_NETWORK=OFF breaks a transformation that used to work. Root cause: it was working by downloading a grid at run time. Fix: this is the check succeeding — add the grid to the package rather than re-enabling the network.

Provenance is recorded and the pipeline string is a variable name. Root cause: recording the reference rather than the resolved value. Fix: record the expanded pipeline; a reference resolves differently in a different environment, which is the failure being guarded against.

FAQ

Is a pinned pipeline really better than letting PROJ choose the best one?

Yes, because “best” is evaluated against what is installed rather than against what is correct. PROJ’s selection is genuinely good when every candidate grid is present — it will prefer the most accurate available transformation — but that qualifier is the problem: a worker missing one grid silently gets the next-best route, and nothing in the output says so. Pinning converts a silent degradation into a loud failure, which is the trade worth making for anything whose coordinates carry consequence.

How often should the grid package be updated?

Rarely, and never casually. A new grid release can shift coordinates by centimetres to metres, so updating it is a pipeline-version bump: cache keys invalidate, affected partitions re-run, and published positional accuracy is re-measured against the control set afterwards. That is appropriate work when a national agency issues a corrected grid, and disproportionate as a routine dependency bump. Treat the package like a schema, not like a library.

What about transformations where no grid exists?

Use the best published parametric transformation, record which one, and publish the resulting accuracy honestly. A 7-parameter Helmert at two to three metres is entirely adequate for many products and unacceptable for cadastral work, and the difference is a decision the consumer should be able to make from the metadata. What must not happen is a null transformation applied by default because no grid was found — that is the largest error available and the hardest to detect.

Does this apply to raster reprojection as well?

Identically. gdalwarp accepts -ct and honours the same environment variables, and a warped raster carries no more evidence of its transformation than a vector layer does. The one addition worth making for raster is recording the resampling algorithm alongside the pipeline, since nearest-neighbour and cubic resampling produce visibly different results from the same transformation and neither is inferable from the output.

Should the pipeline string live in the manifest or in the code?

In the manifest, because it is a property of the product rather than of the program that happens to process it. Two pipelines processing the same source must apply the same transformation, and a string embedded in one of them cannot be enforced across the other. Keeping it in the manifest also makes it reviewable by the data steward who owns the approved-transformation registry, rather than by whoever reviews the pipeline code — and those are different people with different expertise.

How do you detect that a transformation was wrong after the fact?

By the control-point check, which is the only method that works. A wrong-but-valid reprojection produces geometry that passes every validity, topology and schema check, so nothing except comparison against an independently-known position will reveal it. That is why the accuracy measurement against a control set is not merely a quality indicator but the estate’s only defence against a silent transformation error — and why a product with no accuracy measurement has no way of knowing whether its coordinates mean what its manifest says.

Related in CRS Governance and Reprojection Standards