Weighted Traffic Shifting for Spatial Domain Migration

Migrating a spatial product from an old owning domain to a new one is a data-correctness problem wearing a traffic-routing costume. Cut over all at once and you discover a CRS drift or a dropped attribute only after consumers have cached wrong geometry. Weighted traffic shifting instead moves a controlled slice — 5%, then 50%, then 100% — and validates geometry and CRS parity between old and new backends at every stage before the weight advances. This operation sits under Cross-Domain Routing Strategies, part of the Federated Ownership & Routing Architecture: it builds directly on the header match surface from Header-Based Routing for Spatial Domains, splitting one matched domain slice across two backends by weight rather than sending it to a single destination.

Figure — Old-domain and new-domain weights across four canary stages; the new domain rises from 0 to 100 percent only after parity holds at each step.

Weight-shift timeline across canary stages Four stacked bars show the split of traffic weight between the old owning domain and the new owning domain. At baseline the old domain holds 100 percent. At the canary stage the new domain takes 5 percent. At the ramp stage the split is 50/50. At cutover the new domain holds 100 percent and the old domain is retired. Each advance happens only after geometry and CRS parity is validated. Old domain weight New domain weight Baseline old 100 / new 0 Canary old 95 / new 5 Ramp old 50 / new 50 Cutover old 0 / new 100 parity gate must pass before each advance

Prerequisites

Requirement Value / Constraint
Service mesh Istio 1.20+ with both old and new domain services registered in the same namespace
CLI tools kubectl, istioctl 1.20+, curl, ogrinfo/ogr2ogr (GDAL 3.6+) for parity checks, jq
CRS assumption Both backends must serve EPSG:4326 vector output; a parity check confirms the new domain did not silently reproject to EPSG:3857
Parity signal Geometry hash and CRS URN compared old vs new per feature collection before each weight advance
Access roles mesh-routing-admin to edit weights; GIS Data Steward sign-off to advance a stage
Env vars MESH_HOST=spatial.mesh.enterprise.internal, DOMAIN=hydrology, OLD_SVC, NEW_SVC

Weight edits are declarative and idempotent: re-applying a stage manifest converges the mesh to that exact split with no drift, so a retried apply during a flaky pipeline never double-shifts traffic. Never advance a stage on a timer alone — the parity gate, not the clock, authorizes the next weight.

Step-by-Step Implementation

1. Establish the baseline: 100% old, 0% new

Split the matched domain slice into two weighted destinations while sending all live traffic to the old owner. This proves the two-destination route is wired correctly before any real traffic moves.

yaml
# weighted-migration.yaml — stage: baseline
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: hydrology-domain-migration
  namespace: spatial
spec:
  hosts: ["spatial.mesh.enterprise.internal"]
  gateways: ["mesh-ingress-gateway"]
  http:
    - name: hydrology-weighted
      match:
        - headers:
            x-spatial-domain:
              exact: "hydrology"        # the migrating domain slice
      route:
        - destination: { host: hydrology-old.spatial.svc.cluster.local, port: { number: 8080 } }
          weight: 100
        - destination: { host: hydrology-new.spatial.svc.cluster.local, port: { number: 8080 } }
          weight: 0

Verify the split is exactly 100/0 in the effective config before proceeding:

bash
kubectl apply -f weighted-migration.yaml
istioctl proxy-config routes deploy/istio-ingressgateway --name http.8080 -o json \
  | jq '.[].virtualHosts[].routes[] | select(.name=="hydrology-weighted") | .route.weightedClusters.clusters[] | {name, weight}'

2. Validate geometry and CRS parity before the canary

Query the same feature collection from both backends and compare geometry and CRS. Only a clean parity result authorizes moving the first 5%.

What the parity gate compares, and what a difference in each column meansFour comparisons run between the old and new domains on the same request. Identical feature counts confirm the query reached the same data. Identical geometry within tolerance confirms the reprojection path matches. The same CRS in the response confirms the contract is honoured. Matching attribute sets confirm the schema survived migration. The final column distinguishes an intended difference — a declared, accepted correction such as a repaired topology — from an unintended one, which is the distinction a gate needs in order to keep enforcing parity everywhere else rather than being switched off.Difference meansIf declaredFeature countquery reached different dataaccepted within named extentsGeometry (tolerance)reprojection path differsaccepted for a datum fixResponse CRScontract brokennever acceptableAttribute setschema driftedaccepted for a MAJOR bump

bash
# Pull the same collection from each backend and compare CRS + geometry hash.
for svc in hydrology-old hydrology-new; do
  curl -s -H "x-spatial-domain: hydrology" -H "x-crs: EPSG:4326" \
    "https://${MESH_HOST}/v1/features/collections/basins/items?limit=500" \
    | ogrinfo -so -al /vsistdin/ | grep -E "Geometry|Feature Count|SRS WKT" > "/tmp/${svc}.parity"
done
diff /tmp/hydrology-old.parity /tmp/hydrology-new.parity && echo "parity: PASS — safe to canary"

3. Shift the canary slice to 5%

With parity confirmed, move 5% of the matched slice to the new domain. Re-apply the manifest with the updated weights.

The ramp, with a parity gate before every increase rather than only at the startFive stages of a weighted shift. The baseline is 100 percent old and 0 percent new. Geometry and CRS parity are validated before any traffic moves. A 5 percent canary runs until extent coverage, not merely elapsed time, crosses its threshold. Parity is re-validated, then the shift goes to 50 percent, re-validated again, and finally to 100 percent. Re-validating at each step is what catches a defect confined to an extent the previous step never served — which a percentage-based canary can easily miss, because tile traffic is heavily concentrated in a few metropolitan areas.0% · baselineall traffic on oldparity gategeometry + CRS5% · canaryuntil extent coverage50% · re-validatedparity again100% · cut overold kept warmCoverage of distinct extents gates each step, not elapsed time or percentage alone

yaml
      route:
        - destination: { host: hydrology-old.spatial.svc.cluster.local, port: { number: 8080 } }
          weight: 95
        - destination: { host: hydrology-new.spatial.svc.cluster.local, port: { number: 8080 } }
          weight: 5

Confirm roughly 1-in-20 requests reach the new backend by sampling the upstream header:

bash
for i in $(seq 1 40); do
  curl -s -o /dev/null -w "%{header_json}" -H "x-spatial-domain: hydrology" \
    "https://${MESH_HOST}/v1/features/collections/basins/items?limit=1" \
    | jq -r '."x-envoy-upstream-service-time" // "new"'
done | grep -c new || echo "sampled canary share"

4. Ramp to 50%, re-validate, then cut over to 100%

Repeat the parity gate from step 2 against live canary traffic, then set weights to 50/50. Hold, re-validate once more under equal load, and only then set the new domain to 100 and the old to 0. After cutover, retire the old destination from the route so a stale weight cannot resurrect it.

yaml
      route:
        - destination: { host: hydrology-new.spatial.svc.cluster.local, port: { number: 8080 } }
          weight: 100                    # old destination removed entirely at cutover

Verify the cutover is total and the old service receives zero traffic:

bash
istioctl proxy-config routes deploy/istio-ingressgateway --name http.8080 -o json \
  | jq '.[].virtualHosts[].routes[] | select(.name=="hydrology-weighted") | .route.cluster'

Configuration Reference

Field Layer Required Purpose / Constraint
match.headers.x-spatial-domain Match Yes Isolates the one migrating domain slice; other domains are untouched
route[].destination.host (old) Route Until cutover The current owner; weight decreases each stage
route[].destination.host (new) Route Yes The target owner; weight increases each stage
route[].weight Route Yes Integer per destination; the two weights must sum to 100
Stage sequence Process Yes 100/0 → 95/5 → 50/50 → 0/100; never skip the parity gate between stages
Parity artifact Process Yes Geometry + SRS WKT diff of old vs new must be empty before advancing
Old destination removal Route At cutover Delete the weight: 0 old destination so it cannot be re-enabled by drift

Common Failure Modes & Fixes

  • Symptom: weights sum to a value other than 100 and traffic distribution is skewed. Root cause: an edited manifest left the two weights at, e.g., 95/10. Fix: Istio normalizes proportionally, masking the error — set the pair to sum exactly to 100 and re-run the step 1 jq weight check.
  • Symptom: parity diff is empty but consumers report offset geometry after cutover. Root cause: the parity query used limit too small to surface a reprojected sub-region. Fix: re-run step 2 across the full extent (or a bbox-tiled sweep), comparing SRS WKT and a full-collection geometry hash, before advancing.
  • Symptom: canary share looks like 0% at 5% weight. Root cause: an upstream cache or keep-alive pinned connections to the old backend. Fix: disable connection reuse for the sample (curl -H "connection: close") and confirm the destination via Envoy access logs, not client timing.
  • Symptom: rollback re-sends traffic to a half-decommissioned old backend. Root cause: the old destination was scaled to zero before weight reached 0. Fix: keep the old backend warm until the 0/100 stage is applied and verified, then decommission; re-apply the previous stage manifest to roll back cleanly.

FAQ

Why validate parity at every stage instead of once up front?

Because the new backend’s behavior can change under real load — a lazy reprojection, a cache warm-up that alters geometry precision, an attribute that only appears in certain collections. Gating each weight advance on a fresh geometry/CRS parity check means a regression is caught while only 5% of traffic is exposed, not after full cutover.

How is a weighted shift rolled back safely?

Re-apply the previous stage’s manifest. Because weight edits are idempotent and declarative, reverting 50/50 back to 95/5 converges the mesh with no partial state. Keep the old destination present and warm until the 0/100 cutover is verified so a rollback always has a healthy target.

Can I run header-based routing and weighted shifting together?

Yes — that is the intended composition. Header-Based Routing for Spatial Domains selects the migrating domain slice with a match, and the weighted route splits only that slice. Every other domain keeps its single-destination route untouched.

What sits behind the parity gate technically?

A geometry hash and CRS comparison of identical requests to both backends, the same class of check enforced at ingest by Enforcing Schema Contracts for GeoJSON and Shapefiles. If ogrinfo reports a different SRS WKT or feature count from the new domain, the gate fails and the weight holds.

Why is a percentage weight not enough on its own for a spatial migration?

Because a random 5% of requests is not a representative 5% of the spatial estate. Traffic to a tile service is concentrated: a handful of metropolitan extents carry the large majority of requests, so a naive percentage canary exercises those extents thoroughly and may never touch a sparsely-queried region at all — which is precisely where a reprojection defect or a missing partition is most likely to be hiding. Pair the weight with extent coverage: track which distinct tile ranges or bounding boxes the canary has actually served from the new domain, and treat the canary as incomplete until coverage crosses a threshold, however long the percentage has been running.

How should the parity gate handle a genuine, intended difference between old and new?

Record it as an accepted difference before the canary starts, not during it. A migration that also corrects data — a repaired topology, a better datum transformation, a resolution change — will produce parity failures that are correct, and a gate with no way to express that either blocks the migration or gets disabled entirely. Neither is acceptable. Declare the expected differences up front as a list of extents or feature classes where divergence is permitted and within what tolerance, so the gate keeps enforcing parity everywhere else and any divergence outside the declared list still stops the shift.