Weighted Traffic Shifting for Spatial Domain Migration

Migrating a spatial product from an old owning domain to a new one is a data-correctness problem wearing a traffic-routing costume. Cut over all at once and you discover a CRS drift or a dropped attribute only after consumers have cached wrong geometry. Weighted traffic shifting instead moves a controlled slice — 5%, then 50%, then 100% — and validates geometry and CRS parity between old and new backends at every stage before the weight advances. This operation sits under Cross-Domain Routing Strategies, part of the Federated Ownership & Routing Architecture: it builds directly on the header match surface from Header-Based Routing for Spatial Domains, splitting one matched domain slice across two backends by weight rather than sending it to a single destination.

Figure — Old-domain and new-domain weights across four canary stages; the new domain rises from 0 to 100 percent only after parity holds at each step.

Weight-shift timeline across canary stages Four stacked bars show the split of traffic weight between the old owning domain and the new owning domain. At baseline the old domain holds 100 percent. At the canary stage the new domain takes 5 percent. At the ramp stage the split is 50/50. At cutover the new domain holds 100 percent and the old domain is retired. Each advance happens only after geometry and CRS parity is validated. Old domain weight New domain weight Baseline old 100 / new 0 Canary old 95 / new 5 Ramp old 50 / new 50 Cutover old 0 / new 100 parity gate must pass before each advance

Prerequisites

Requirement Value / Constraint
Service mesh Istio 1.20+ with both old and new domain services registered in the same namespace
CLI tools kubectl, istioctl 1.20+, curl, ogrinfo/ogr2ogr (GDAL 3.6+) for parity checks, jq
CRS assumption Both backends must serve EPSG:4326 vector output; a parity check confirms the new domain did not silently reproject to EPSG:3857
Parity signal Geometry hash and CRS URN compared old vs new per feature collection before each weight advance
Access roles mesh-routing-admin to edit weights; GIS Data Steward sign-off to advance a stage
Env vars MESH_HOST=spatial.mesh.enterprise.internal, DOMAIN=hydrology, OLD_SVC, NEW_SVC

Weight edits are declarative and idempotent: re-applying a stage manifest converges the mesh to that exact split with no drift, so a retried apply during a flaky pipeline never double-shifts traffic. Never advance a stage on a timer alone — the parity gate, not the clock, authorizes the next weight.

Step-by-Step Implementation

1. Establish the baseline: 100% old, 0% new

Split the matched domain slice into two weighted destinations while sending all live traffic to the old owner. This proves the two-destination route is wired correctly before any real traffic moves.

yaml
# weighted-migration.yaml — stage: baseline
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: hydrology-domain-migration
  namespace: spatial
spec:
  hosts: ["spatial.mesh.enterprise.internal"]
  gateways: ["mesh-ingress-gateway"]
  http:
    - name: hydrology-weighted
      match:
        - headers:
            x-spatial-domain:
              exact: "hydrology"        # the migrating domain slice
      route:
        - destination: { host: hydrology-old.spatial.svc.cluster.local, port: { number: 8080 } }
          weight: 100
        - destination: { host: hydrology-new.spatial.svc.cluster.local, port: { number: 8080 } }
          weight: 0

Verify the split is exactly 100/0 in the effective config before proceeding:

bash
kubectl apply -f weighted-migration.yaml
istioctl proxy-config routes deploy/istio-ingressgateway --name http.8080 -o json \
  | jq '.[].virtualHosts[].routes[] | select(.name=="hydrology-weighted") | .route.weightedClusters.clusters[] | {name, weight}'

2. Validate geometry and CRS parity before the canary

Query the same feature collection from both backends and compare geometry and CRS. Only a clean parity result authorizes moving the first 5%.

bash
# Pull the same collection from each backend and compare CRS + geometry hash.
for svc in hydrology-old hydrology-new; do
  curl -s -H "x-spatial-domain: hydrology" -H "x-crs: EPSG:4326" \
    "https://${MESH_HOST}/v1/features/collections/basins/items?limit=500" \
    | ogrinfo -so -al /vsistdin/ | grep -E "Geometry|Feature Count|SRS WKT" > "/tmp/${svc}.parity"
done
diff /tmp/hydrology-old.parity /tmp/hydrology-new.parity && echo "parity: PASS — safe to canary"

3. Shift the canary slice to 5%

With parity confirmed, move 5% of the matched slice to the new domain. Re-apply the manifest with the updated weights.

yaml
      route:
        - destination: { host: hydrology-old.spatial.svc.cluster.local, port: { number: 8080 } }
          weight: 95
        - destination: { host: hydrology-new.spatial.svc.cluster.local, port: { number: 8080 } }
          weight: 5

Confirm roughly 1-in-20 requests reach the new backend by sampling the upstream header:

bash
for i in $(seq 1 40); do
  curl -s -o /dev/null -w "%{header_json}" -H "x-spatial-domain: hydrology" \
    "https://${MESH_HOST}/v1/features/collections/basins/items?limit=1" \
    | jq -r '."x-envoy-upstream-service-time" // "new"'
done | grep -c new || echo "sampled canary share"

4. Ramp to 50%, re-validate, then cut over to 100%

Repeat the parity gate from step 2 against live canary traffic, then set weights to 50/50. Hold, re-validate once more under equal load, and only then set the new domain to 100 and the old to 0. After cutover, retire the old destination from the route so a stale weight cannot resurrect it.

yaml
      route:
        - destination: { host: hydrology-new.spatial.svc.cluster.local, port: { number: 8080 } }
          weight: 100                    # old destination removed entirely at cutover

Verify the cutover is total and the old service receives zero traffic:

bash
istioctl proxy-config routes deploy/istio-ingressgateway --name http.8080 -o json \
  | jq '.[].virtualHosts[].routes[] | select(.name=="hydrology-weighted") | .route.cluster'

Configuration Reference

Field Layer Required Purpose / Constraint
match.headers.x-spatial-domain Match Yes Isolates the one migrating domain slice; other domains are untouched
route[].destination.host (old) Route Until cutover The current owner; weight decreases each stage
route[].destination.host (new) Route Yes The target owner; weight increases each stage
route[].weight Route Yes Integer per destination; the two weights must sum to 100
Stage sequence Process Yes 100/0 → 95/5 → 50/50 → 0/100; never skip the parity gate between stages
Parity artifact Process Yes Geometry + SRS WKT diff of old vs new must be empty before advancing
Old destination removal Route At cutover Delete the weight: 0 old destination so it cannot be re-enabled by drift

Common Failure Modes & Fixes

  • Symptom: weights sum to a value other than 100 and traffic distribution is skewed. Root cause: an edited manifest left the two weights at, e.g., 95/10. Fix: Istio normalizes proportionally, masking the error — set the pair to sum exactly to 100 and re-run the step 1 jq weight check.
  • Symptom: parity diff is empty but consumers report offset geometry after cutover. Root cause: the parity query used limit too small to surface a reprojected sub-region. Fix: re-run step 2 across the full extent (or a bbox-tiled sweep), comparing SRS WKT and a full-collection geometry hash, before advancing.
  • Symptom: canary share looks like 0% at 5% weight. Root cause: an upstream cache or keep-alive pinned connections to the old backend. Fix: disable connection reuse for the sample (curl -H "connection: close") and confirm the destination via Envoy access logs, not client timing.
  • Symptom: rollback re-sends traffic to a half-decommissioned old backend. Root cause: the old destination was scaled to zero before weight reached 0. Fix: keep the old backend warm until the 0/100 stage is applied and verified, then decommission; re-apply the previous stage manifest to roll back cleanly.

FAQ

Why validate parity at every stage instead of once up front?

Because the new backend’s behavior can change under real load — a lazy reprojection, a cache warm-up that alters geometry precision, an attribute that only appears in certain collections. Gating each weight advance on a fresh geometry/CRS parity check means a regression is caught while only 5% of traffic is exposed, not after full cutover.

How is a weighted shift rolled back safely?

Re-apply the previous stage’s manifest. Because weight edits are idempotent and declarative, reverting 50/50 back to 95/5 converges the mesh with no partial state. Keep the old destination present and warm until the 0/100 cutover is verified so a rollback always has a healthy target.

Can I run header-based routing and weighted shifting together?

Yes — that is the intended composition. Header-Based Routing for Spatial Domains selects the migrating domain slice with a match, and the weighted route splits only that slice. Every other domain keeps its single-destination route untouched.

What sits behind the parity gate technically?

A geometry hash and CRS comparison of identical requests to both backends, the same class of check enforced at ingest by Enforcing Schema Contracts for GeoJSON and Shapefiles. If ogrinfo reports a different SRS WKT or feature count from the new domain, the gate fails and the weight holds.