Purging Tile Caches After a Partition Republish

Purging a whole layer to correct one region evicts the warm working set, converts a small republish into a renderer traffic spike, and takes minutes to recover from. Purging only the affected tiles takes seconds and costs nothing — provided the purge reaches every cache layer, which is the step most often skipped and never noticed until a consumer reports data that changed everywhere except at one edge. This guide computes the exact tile set a republished partition covers, purges it across every layer, and verifies the purge rather than assuming it. It is the invalidation half of Edge Caching and Tile Delivery Topology within Federated Ownership & Routing Architecture, and it is unnecessary for layers using the immutable addressing described there.

Prerequisites

Requirement Value / Assumption Notes
Tools python3 ≥ 3.11, curl, edge CLI with a purge API Purge must be scriptable
Addressing Mutable tile addresses Immutable addressing needs no purge
Partition The republished partition’s bounding box in EPSG:4326 From the publishing DAG run
Matrix set WebMercatorQuad, zoom range from the product manifest Tile maths is grid-specific
Cache layers Origin cache, shared cache, and every edge PoP enumerated An unenumerated layer is an unpurged layer
Environment PRODUCT, PARTITION_BBOX, ZMIN, ZMAX Exported before running

Step-by-Step Implementation

1. Enumerate the tiles the partition actually covers

Purging by prefix is tempting and wrong: a prefix covers a whole zoom level, and the partition covers a fraction of it. The tile maths is small and exact.

python
# tile_range.py — the tiles a bbox covers at each zoom, in WebMercatorQuad.
import math
import sys


def lonlat_to_tile(lon: float, lat: float, z: int) -> tuple[int, int]:
    """WebMercatorQuad tile containing a coordinate. Latitude is clamped to the
    projection's valid range: EPSG:3857 is undefined beyond about ±85.05°."""
    lat = max(-85.05112878, min(85.05112878, lat))
    n = 2 ** z
    x = int((lon + 180.0) / 360.0 * n)
    y = int((1.0 - math.asinh(math.tan(math.radians(lat))) / math.pi) / 2.0 * n)
    return max(0, min(n - 1, x)), max(0, min(n - 1, y))


def tiles_for_bbox(bbox: tuple[float, float, float, float], zmin: int, zmax: int):
    """Every tile intersecting the bbox, zmin..zmax inclusive. Note the y inversion:
    tile y increases southward while latitude increases northward."""
    minx, miny, maxx, maxy = bbox
    for z in range(zmin, zmax + 1):
        x0, y1 = lonlat_to_tile(minx, miny, z)
        x1, y0 = lonlat_to_tile(maxx, maxy, z)
        for x in range(min(x0, x1), max(x0, x1) + 1):
            for y in range(min(y0, y1), max(y0, y1) + 1):
                yield z, x, y


if __name__ == "__main__":
    bbox = tuple(float(v) for v in sys.argv[1].split(","))
    for z, x, y in tiles_for_bbox(bbox, int(sys.argv[2]), int(sys.argv[3])):
        print(f"{z}/{x}/{y}")

Objects evicted by purge scope, for a city-scale partition republishFour purge scopes and the number of cached objects each evicts, on a logarithmic-feeling scale, with the roughly four thousand objects the partition actually covers marked. Purging the enumerated tile range evicts about four thousand. Purging by zoom prefix evicts around 900,000. Purging the whole layer evicts about 14 million and takes the warm working set with it. Doing nothing evicts none and leaves consumers stale until expiry. The scoped purge is two to three orders of magnitude smaller than the alternatives.enumerated tile range4kexactly what changedzoom prefix900kwhole layer1000k≈14,000k, off scalenothing0kstale until expiryactually changed

Verify the count is plausible before purging anything. An unexpectedly large count usually means the bbox is wrong, and purging on it would evict most of the layer:

bash
python3 tile_range.py "$PARTITION_BBOX" "$ZMIN" "$ZMAX" | wc -l
# A city-scale partition at z6–z14 is typically thousands. Millions means a bad bbox.

2. Purge every layer, not merely the edge

Each layer has its own purge API and its own idea of what it holds. A purge that reaches the edge and misses the shared cache leaves the edge to re-fetch the stale object it just discarded.

bash
#!/usr/bin/env bash
# purge_partition.sh — purge one partition's tiles across every cache layer.
# Idempotent: purging an absent object is a no-op, so re-running is always safe.
set -euo pipefail

PRODUCT="${1:?product}"; BBOX="${2:?bbox}"; ZMIN="${3:?zmin}"; ZMAX="${4:?zmax}"

# Order matters: purge from the origin outward, so an edge that re-fetches during
# the purge fetches the new object rather than re-caching the old one.
LAYERS=(origin-cache.internal shared-cache.internal edge.internal)

python3 tile_range.py "$BBOX" "$ZMIN" "$ZMAX" > /tmp/tiles.txt
total=$(wc -l < /tmp/tiles.txt)
echo "purging ${total} tile(s) across ${#LAYERS[@]} layer(s)"

for layer in "${LAYERS[@]}"; do
  # Batch rather than one request per tile; a per-tile purge of 50k tiles is an outage.
  split -l 500 /tmp/tiles.txt /tmp/batch_
  for batch in /tmp/batch_*; do
    jq -Rsc --arg p "$PRODUCT" \
      '{product: $p, tiles: (split("\n") | map(select(length > 0)))}' < "$batch" \
      | curl -sS -X POST "https://${layer}/purge" \
             -H 'content-type: application/json' -d @- > /dev/null
  done
  rm -f /tmp/batch_*
  echo "  purged: ${layer}"
done

Why the purge runs origin-outward rather than edge-firstPurging origin-outward means the origin cache is cleared first, then the shared cache, then the edge. An edge that re-fetches during the purge therefore fetches the new object. Purging edge-first inverts this: the edge is cleared, immediately re-fetches from a shared cache that still holds the old object, and re-caches the stale copy — so the purge appears to succeed and the old data returns minutes later.Purge jobOrigin cacheShared cacheEdgepurge firstthen sharedthen edgeedge re-fetch gets the NEW objectedge-first order: re-caches the OLD object

Verify the purge landed at every layer, by sampling rather than trusting the API’s response:

bash
sample=$(shuf -n 5 /tmp/tiles.txt)
for layer in origin-cache.internal shared-cache.internal edge.internal; do
  for t in $sample; do
    printf '%-24s %-12s ' "$layer" "$t"
    curl -sS -o /dev/null -D - "https://${layer}/${PRODUCT}/${t}.mvt" \
      | grep -iE '^(x-cache|x-spatial-version):' | tr '\n' ' '; echo
  done
done
# Every layer must report a MISS and the new version. A HIT with the old version
# at any layer is an unpurged layer, and it will keep serving stale until expiry.

3. Watch the hit ratio recover

A purge is not free, and the recovery curve tells you whether the scope was right.

Hit-ratio recovery after a scoped purge versus a layer-wide oneFive points after a republish. Immediately before the purge the hit ratio sits at ninety-four percent. A scoped purge dips it to about ninety-one and it recovers within four minutes. A layer-wide purge collapses it to roughly eleven percent and takes over forty minutes to recover, during which the renderer absorbs a miss storm. The recovery curve is the check that tells you whether the purge scope was right, and it is worth watching on the first few republishes after any change to the purge job.t−1m · 94%steady statescoped purgedips to 91%t+4m · 94%recoveredlayer-wide purgecollapses to 11%t+40m · 88%still recoveringThe recovery curve is what tells you whether the purge scope was right

bash
# Hit ratio before, during and after. A scoped purge dips a few points and recovers
# within minutes; a layer-wide purge collapses the ratio and takes the renderer with it.
for i in $(seq 1 12); do
  curl -sS "$PROM/api/v1/query" --data-urlencode \
    'query=sum(rate(tile_cache_hits_total{product="'"$PRODUCT"'"}[2m]))
           / sum(rate(tile_requests_total{product="'"$PRODUCT"'"}[2m]))' \
    | jq -r '"\(now|floor) \(.data.result[0].value[1])"'
  sleep 60
done

4. Wire the purge into the publish, not into an operator’s hands

A purge that depends on someone remembering to run it will be forgotten exactly once, and that once will be the republish that mattered.

python
# publish_hook.py — the purge is a step of the publish, with the bbox from the run.
def on_partition_published(run) -> None:
    """Called by the orchestrator after an atomic publish. The bbox comes from the
    partition definition, so the purge scope can never drift from what changed."""
    if run.manifest["spec"].get("addressing") == "immutable":
        return                                   # nothing to purge; addresses are new
    bbox = run.partition.bbox_4326
    zoom = next(p["zoom"] for p in run.manifest["spec"]["ports"] if p["type"] == "tiles")
    run.exec([
        "./purge_partition.sh", run.product,
        ",".join(str(v) for v in bbox), str(zoom["min"]), str(zoom["max"]),
    ])

Configuration Reference

Parameter Scope Value Effect
Purge scope Publish hook The republished partition’s bbox Layer-wide purge evicts the warm set
Zoom range Manifest ports[].zoom Purging beyond the published range wastes calls
Batch size Purge script 500 tiles Per-tile purge of a large range is itself an outage
Layer order Purge script Origin → shared → edge Reverse order lets an edge re-cache the old object
Verification sample Verify step 5 tiles per layer Trusting the purge API’s response is how layers get missed
Latitude clamp Tile maths ±85.05112878 EPSG:3857 is undefined beyond it

Common Failure Modes & Fixes

Most consumers see new data; some see old. Root cause: one cache layer — usually a regional edge PoP added after the purge script was written — is not in the layer list. Fix: derive the layer list from the edge provider’s PoP inventory rather than hard-coding it, and verify by sampling every layer.

Hit ratio collapses after every republish. Root cause: purging by prefix or by layer rather than by partition. Fix: the tile enumeration above; the difference is typically two orders of magnitude in evicted objects.

Purge takes twenty minutes. Root cause: one HTTP request per tile. Fix: batch, as above. Most purge APIs accept hundreds of paths per call, and the difference is minutes to seconds.

Tiles at the highest zoom are still stale. Root cause: the purge zoom range was hard-coded and the product’s published range was extended. Fix: read the zoom range from the manifest, so the two cannot diverge.

Purge succeeds and the old data returns minutes later. Root cause: the layers were purged edge-first, so an edge re-fetched from a shared cache that still held the old object. Fix: purge origin-outward.

FAQ

Why not just shorten the cache lifetime instead of purging?

Because the two solve different problems and the cheap-looking option is expensive. Shortening s-maxage bounds staleness for every tile all the time, which means paying continuous origin load to handle an event that happens a few times a day. Purging pays nothing until a republish and then pays exactly for what changed. The right configuration usually uses both: a lifetime long enough that ordinary traffic is nearly all cache hits, plus a scoped purge on republish to close the gap immediately rather than waiting for expiry.

How do I purge when the partition’s bbox is not a rectangle?

Purge the bounding rectangle of the partition, accepting that it covers some tiles that did not change. The alternative — enumerating tiles that intersect an arbitrary polygon — is more code, is slower to compute, and saves relatively little because tile grids are coarse compared with most partition shapes. The over-purge is bounded by the difference between the partition’s area and its envelope, and for typical administrative or grid-aligned partitions that is small. Where a partition is genuinely long and thin, splitting it into several rectangles is a better trade than exact polygon enumeration.

Should the purge block the publish from completing?

No — publish atomically first, then purge. The publish is the operation that must be transactional; the purge is a cache correction that can be retried freely because it is idempotent. Blocking publication on a purge means a slow or failing edge API can prevent a validated artifact from becoming available, which is the wrong dependency direction. Run the purge as a post-publish hook, retry it on failure, and alert if it fails repeatedly — consumers will get the new data at expiry regardless, just later than intended.

How can I tell whether a purge is even needed?

By the layer’s addressing. If tile addresses carry the product version, a republish creates entirely new addresses and nothing cached is stale — the old version’s tiles simply age out unused. That is why the publish hook above returns immediately for immutable layers. For mutable layers a purge is always needed, and the fact that it is easy to forget is the strongest argument for moving a layer to immutable addressing whenever its consumers can resolve a version.

Related in Edge Caching and Tile Delivery Topology