# Implementing product thinking for satellite imagery datasets

Transitioning from centralized raster warehouses to domain-scoped data products shifts the primary failure vector from storage capacity to metadata contract enforcement. This guide isolates a single, high-impact operation: configuring a versioned, domain-bound SpatioTemporal Asset Catalog (STAC) ingestion pipeline that treats multispectral satellite imagery as a consumable product rather than a raw archive. It operationalizes the parent topic, Product Thinking for GIS Datasets, under the broader Geospatial Data Mesh Fundamentals paradigm, and pairs naturally with Best Practices for Spatial Metadata Catalogs for the discovery layer. The workflow enforces strict spatial scoping rules, automated schema validation, and deterministic rollback paths so that a Sentinel-2 L2A product carries the same contractual guarantees as any other enterprise dataset.

The pipeline binds every tile to a declared spatial domain, validates it against the STAC specification before publication, and versions it with the site convention v2.1.0-crs:EPSG:4326-res:10m. Below it is decomposed into prerequisites, ordered implementation steps, a configuration reference, the failure modes you will actually hit, and answers to the questions that recur during rollout.

# Prerequisites

# Architectural Baseline: Domain-Scoped Ingestion

In legacy monolithic systems, raster data is ingested into shared buckets with implicit coordinate reference system (CRS) assumptions and deferred cataloging — the pattern dissected in Data Mesh vs Traditional GIS Architecture. Domain-scoped ingestion inverts this by treating each spatial domain as an autonomous product boundary, with explicit contract enforcement at the ingestion edge rather than post-hoc ETL reconciliation.

Spatial Domain Boundary Design dictates that every tile be validated against a deterministic polygon envelope before entering the product catalog. This prevents silent CRS drift and guarantees downstream consumers receive spatially consistent assets. The pipeline applies Scoping Rules for Spatial Products at the ingestion layer, rejecting tiles that fall outside the declared boundary_wkt or lack mandatory projection extensions.

# Step-by-Step Implementation

# 1. Declare the product configuration

The tactical objective is a declarative configuration that binds Sentinel-2 L2A tiles to a specific spatial domain while publishing them as versioned data products. Couple rio-tiler tiling parameters with pystac validation hooks:

# product-config.yaml
spatial_domain:
  crs: "EPSG:4326"
  source_crs: "EPSG:32633"
  boundary_wkt: "POLYGON((-122.5 37.7, -122.3 37.7, -122.3 37.9, -122.5 37.9, -122.5 37.7))"
product_metadata:
  product_id: "sentinel2-msi-l2a"
  version: "v2.1.0-crs:EPSG:4326-res:10m"
  lifecycle_stage: "production"
ingestion:
  tile_size: 256
  resampling: "bilinear"
  output_format: "cloud-optimized-geotiff"
  checksum_algorithm: "sha256"

Verify the file parses and the boundary is a valid polygon before proceeding:

python3 -c "import yaml, shapely.wkt as w; c=yaml.safe_load(open('product-config.yaml')); print(w.loads(c['spatial_domain']['boundary_wkt']).is_valid)"
# expect: True

# 2. Clip and reproject the tile with explicit creation options

Reproject from the source UTM grid to the product CRS while preserving projection metadata. Missing creation options here are the single most common source of downstream contract drift:

gdalwarp \
  -s_srs EPSG:32633 \
  -t_srs EPSG:4326 \
  -te -122.5 37.7 -122.3 37.9 \
  -co "TILED=YES" \
  -co "COMPRESS=DEFLATE" \
  -co "BIGTIFF=IF_SAFER" \
  s3://raw-bucket/tile_123.tif /vsimem/tile_123_clipped.tif

Confirm the CRS survived the clip before cataloging:

rio info /vsimem/tile_123_clipped.tif | jq '.crs'
# expect: "EPSG:4326"  (a null value means the warp dropped projection metadata — see Failure Modes)

# 3. Build and validate the STAC item with a dry run

Idempotency is enforced through pre-flight validation, content-addressable identifiers, and atomic staging. The pipeline never mutates existing catalog entries without an explicit version increment. Use pystac for item creation and validation:

import hashlib
from pathlib import Path

import pystac
from pystac.validation import validate_dict
from shapely.geometry import box, mapping

BOUNDARY = box(-122.5, 37.7, -122.3, 37.9)
PRODUCT_ID = "sentinel2-msi-l2a"
VERSION = "v2.1.0-crs:EPSG:4326-res:10m"


def build_stac_item(tile_path: Path, dry_run: bool = True) -> pystac.Item:
    """Build and optionally publish a STAC item for a COG tile."""
    tile_hash = hashlib.sha256(tile_path.read_bytes()).hexdigest()

    item = pystac.Item(
        id=f"{PRODUCT_ID}-{tile_hash[:8]}",
        geometry=mapping(BOUNDARY),
        bbox=list(BOUNDARY.bounds),
        datetime=None,
        properties={
            "start_datetime": "2024-01-01T00:00:00Z",
            "end_datetime": "2024-03-31T23:59:59Z",
            "platform": "sentinel-2",
            "instruments": ["msi"],
            "version": VERSION,
            "proj:epsg": 4326,
            "checksum:sha256": tile_hash,
        },
    )
    item.add_asset(
        "data",
        pystac.Asset(
            href=str(tile_path),
            media_type=pystac.MediaType.COG,
            roles=["data"],
        ),
    )

    # Validate against the STAC spec before committing.
    validate_dict(item.to_dict())

    if dry_run:
        print(f"DRY RUN: Item {item.id} validated successfully")
        return item

    # Atomic commit: write to the catalog only after validation passes.
    item.normalize_hrefs(f"s3://geo-mesh-prod/sentinel2/{PRODUCT_ID}/")
    print(f"Committed: {item.id}")
    return item

Run with dry_run=True first to trigger schema validation and spatial-envelope intersection checks without writing to the target bucket. Set dry_run=False only after the dry run passes.

# 4. Promote through lifecycle states

Spatial Product Lifecycle Management dictates that assets transition through staging → validation → production. Promotion to production requires all of the following:

1. Successful pystac item validation against the STAC specification and any declared extensions.
2. SHA-256 checksum match between the source tile and the staged COG.
3. Explicit domain-owner sign-off via the governance webhook.

Semantic versioning governs deterministic consumer upgrades. A major increment requires schema-breaking changes (e.g. CRS migration); a minor increment denotes additive metadata or processing improvements; a patch increment covers reprocessing of corrupted tiles without altering the spatial contract. Validate the staged checksum before promotion:

test "$(sha256sum staged.tif | cut -d' ' -f1)" = "$(jq -r '.properties["checksum:sha256"]' item.json)" \
  && echo "CHECKSUM_OK" || echo "CHECKSUM_MISMATCH"

# Configuration Reference

# Common Failure Modes & Fixes

Symptom: pystac raises STACValidationError: Missing required field 'proj:geometry'. Root cause: an upstream gdalwarp call stripped projection metadata during domain clipping, so the item lacks the projection extension fields. Fix: re-run the warp from step 2 with -t_srs set and a GeoTIFF output driver, then populate proj:geometry explicitly with pyproj and the pystac projection extension before submission. Confirm with rio info ... | jq '.crs' returning a non-null EPSG code.

Symptom: rio info ... | jq '.crs' returns null. Root cause: the clipping step bypassed CRS embedding — typically a missing -t_srs or an output written to a non-GeoTIFF driver. Note that -co "PROFILE=GeoTIFF" does not control CRS embedding; what matters is -t_srs plus a GeoTIFF output. Fix: re-clip with explicit -s_srs/-t_srs as shown in step 2 and re-validate.

Symptom: ingestion halts with a boundary_wkt intersection failure. Root cause: the tile falls outside the declared domain envelope, or the source CRS differs from the boundary CRS so the intersection is computed in mismatched units. Fix: reproject the tile bounds into the boundary CRS before testing intersection; quarantine the tile in s3://geo-mesh-quarantine/ if it is genuinely out of domain.

Symptom: CHECKSUM_MISMATCH during promotion. Root cause: the staged COG was re-compressed or re-tiled after the digest was recorded, changing the byte stream. Fix: recompute the digest on the final staged artifact, update checksum:sha256, and re-stage; never edit the COG after the digest is bound to the item.

Symptom: a previously published version needs to be withdrawn. Root cause: a corrupted or non-compliant item reached production. Fix: run the idempotent rollback, which removes the failed version from the catalog index and restores the previous manifest:

import pystac

catalog = pystac.Catalog.from_file("s3://geo-mesh-prod/sentinel2/catalog.json")
failed_item_id = "sentinel2-msi-l2a-abc12345"
item = catalog.get_item(failed_item_id, recursive=True)
if item:
    item.get_parent().remove_item(failed_item_id)
    catalog.normalize_and_save(
        root_href="s3://geo-mesh-prod/sentinel2/",
        catalog_type=pystac.CatalogType.SELF_CONTAINED,
    )
    print(f"PRODUCT_ROLLBACK_COMPLETE: {failed_item_id} removed from catalog")

Validation failures should be routed automatically to the owning tier. A missing optional field is logged and the pipeline proceeds (domain data steward notified); a null CRS or mismatched projection halts the pipeline and quarantines the tile (platform engineer plus GIS steward); a boundary_wkt intersection failure or checksum mismatch rolls back staging, invalidates the cache, and triggers re-clipping (architecture review board plus incident commander). All pipeline executions must emit structured JSON logs for audit trails and drift analysis.

# FAQ

# Related

• Product Thinking for GIS Datasets — the parent topic this operation implements.
• Best Practices for Spatial Metadata Catalogs — the discovery layer that consumes these STAC items.
• Defining Scoping Rules for Enterprise Spatial Products — how the boundary_wkt contract is authored.
• Migrating from Monolithic GIS to Data Mesh — the architectural shift that makes domain-scoped ingestion possible.