Skip to content

Repository files navigation

GeoTIFF to PMTiles

A memory-efficient toolset for working with PMTiles v3 archives: convert GeoTIFF/COG files and transform existing archives (change format, zoom levels, resampling, fill empty tiles).

For more information on the PMTiles format, see the PMTiles documentation.

You can visualize generated PMTiles files at pmtiles.io.

Real satellite and raster test data is downloaded via make test-integration-download into integration/testdata/. Seven datasets are available: Copernicus DEM (float32), Natural Earth (8-bit RGB + TFW), ESA WorldCover S2 RGBNIR / NDVI / SWIR, ESA WorldCover S1 SAR gamma0, and swisstopo SWISSIMAGE DOP10 (EPSG:2056 mosaic).

Features

  • Memory-efficient: Reads COG tiles on-demand via memory-mapped I/O; never loads entire rasters into memory
  • Disk-backed tile store: Tiles are stored in encoded form (5-25x smaller than raw pixels) and continuously spilled to disk via a dedicated I/O goroutine with configurable memory backpressure
  • Native WebP: WebP encoding/decoding via native libwebp (CGo), eliminating WASM overhead for 3-5x faster encodes
  • Multiple encodings: JPEG, PNG, WebP, and Terrarium (for elevation/DEM data)
  • Auto zoom detection: Calculates maximum zoom level from source resolution
  • Auto-detection: Automatically detects data type and configures processing — float GeoTIFFs get Terrarium encoding for elevation/DEM data; multi-band satellite GeoTIFFs with GDAL band descriptions get automatic band ordering and rescale range. Works with Sentinel-2, PlanetScope, Google Earth Engine exports, HLS, and any GDAL-created multi-band GeoTIFF — no manual flags needed
  • Coverage gap detection: Warns about geographic holes in input file coverage
  • Parallel processing: Concurrent tile generation with configurable worker pool and Hilbert-curve batch scheduling for spatial locality
  • PMTiles v3: Writes spec-compliant archives with Hilbert-curve tile ordering
  • COG-aware: Exploits Cloud Optimized GeoTIFF overview levels for lower zoom tiles
  • Resampling methods: Bicubic (4×4 Catmull-Rom), bilinear, nearest-neighbor, Lanczos-3, and mode (most common value) resampling (optimized with precomputed LUTs and batched tile fetches). Mode resampling is ideal for categorical/classified rasters (e.g. land cover) where interpolated values are meaningless.
  • Pyramid downsampling: Lower zoom tiles are generated by downsampling higher zoom tiles, with gray-aware fast path for single-channel data (15x faster)
  • Uniform tile compaction: Single-color tiles stored as 4 bytes each, never spilled to disk
  • Pooled RGBA buffers: sync.Pool reuses 256 KB tile buffers across render/downsample/decode paths, reducing GC pressure
  • Processing provenance: Source metadata and processing steps are recorded in the PMTiles description; transforms stack their parameters above the source history for full traceability

Supported Input

  • GeoTIFF / Cloud Optimized GeoTIFF (COG) files
  • Plain TIFF with TFW (TIFF World File) sidecar for georeferencing
  • Strip-based and tiled TIFF layouts, pixel- or band-interleaved (planar-separate strips: all compressions except JPEG)
  • TIFF compression: JPEG, LZW, Deflate/Zlib, and uncompressed (with predictor support)
  • Sample formats: 8-bit RGB/RGBA, 16-bit uint16 (with linear/log rescaling), Float32/Float64 (for elevation/DEM data)
  • Band reordering and alpha band selection for multi-band GeoTIFFs (e.g. RGBNIR false-color composites)
  • Source CRS: EPSG:2056 (Swiss LV95), EPSG:4326 (WGS84), EPSG:3857 (Web Mercator)
  • Extensible projection interface for adding additional CRS support

Prerequisites

WebP support requires libwebp to be installed:

# macOS
brew install webp

# Debian/Ubuntu
sudo apt-get install libwebp-dev

# Fedora/RHEL
sudo dnf install libwebp-devel

Installation

go build -o geotiff2pmtiles ./cmd/geotiff2pmtiles/
go build -o pmtransform ./cmd/pmtransform/

Or using the Makefile:

make build            # geotiff2pmtiles only
make build-transform  # pmtransform only
make build-all        # both binaries
make example-all      # run every example target

Cross-compilation

Cross-compilation requires a C cross-compiler and libwebp built for the target platform:

make cross-all

Or target a specific platform:

make cross-linux          # Linux amd64
make cross-linux-arm64    # Linux arm64
make cross-darwin         # macOS amd64
make cross-darwin-arm64   # macOS arm64

Usage

geotiff2pmtiles [flags] <input-dir-or-files...> <output.pmtiles>

Flags

Flag Default Description
--format jpeg Tile encoding: jpeg, png, webp, terrarium
--quality 85 JPEG/WebP quality (1-100)
--min-zoom auto Minimum zoom level (default: max_zoom - 6)
--max-zoom auto Maximum zoom level (auto-detected from resolution)
--tile-size 256 Output tile size in pixels
--concurrency NumCPU Number of parallel workers
--resampling bicubic Interpolation method: lanczos, bicubic, bilinear, nearest, mode
--resampling-gamma 1.0 Gamma correction for resampling output encoding (1.0 = disabled, typical 1.5–2.2 for dB-space to RGB)
--mem-limit auto Tile store memory limit in MB before disk spilling (0 = auto ~90% of RAM)
--no-spill false Disable disk spilling (keep all tiles in memory)
--fill-color 0,0,0,0 Substitute transparent/nodata with RGBA color (color transform); also fill missing tile positions. E.g. "0,0,0,255" or "#000000ff" (default: transparent)
--attribution Attribution string for data sources (stored in metadata)
--type baselayer Layer type: baselayer, overlay
--bands 1,2,3 1-indexed band numbers for R,G,B output (e.g. 4,1,2 for NIR-R-G false color)
--alpha-band auto Alpha band: auto (band 4 for 8-bit spp>=4), -1 (none), or 1-indexed band
--rescale auto Rescale mode: auto, linear, log, none (auto requires --rescale-range for 16-bit)
--rescale-range Input value range min,max for rescaling (required for 16-bit data)
--nodata Nodata value: pixels with all bands equal to this integer are transparent (auto-detected from GeoTIFF if not set). When set without --format, output auto-switches from jpeg to webp so transparency is preserved.
--nodata-tolerance 0 Per-band tolerance for --nodata matching. Use 4–8 for borders that come from lossy JPEG sources, where the strict nodata value is smeared by compression.
--nodata-flood false Source-level flood-fill from the COG outer edges through near-nodata pixels. Only the connected component reachable from the image boundary becomes transparent; interior dark pixels (text, shadows, canopy) stay opaque even when tolerance is widened. Pair with a generous --nodata-tolerance (e.g. 40) to clean up JPEG-smeared boundaries. Costs ~W·H/8 bytes RAM per source plus an upfront decode pass (parallelized across --concurrency workers and up to 4 sources at once).
--verbose false Verbose progress output
--version Print version and exit
--cpuprofile Write CPU profile to file
--memprofile Write memory profile to file

Examples

Convert a directory of GeoTIFFs with auto zoom detection (scans subfolders recursively):

./geotiff2pmtiles --verbose integration/testdata/swissimage/ output.pmtiles

Convert specific files with custom zoom range and PNG format:

./geotiff2pmtiles --format png --min-zoom 10 --max-zoom 18 \
  file1.tif file2.tif output.pmtiles

High-quality WebP with bilinear resampling:

./geotiff2pmtiles --format webp --quality 95 --resampling bilinear \
  input/ output.pmtiles

Categorical data (e.g. land cover classification) with mode resampling:

./geotiff2pmtiles --format png --resampling mode \
  landcover/ classification.pmtiles

Fill transparent/nodata areas with a solid color (e.g. black):

./geotiff2pmtiles --fill-color "0,0,0,255" --format png \
  input/ output.pmtiles

Historic JPEG-compressed scan with a black border (output auto-switches to WebP for transparency):

./geotiff2pmtiles --nodata 0 --nodata-tolerance 8 \
  scan.tif output.pmtiles
# Nodata is active; switching output format jpeg → webp so transparency is preserved.

Same source but the boundary still shows JPEG-smear speckles — flood-fill from the image edge with a wide tolerance removes the fringe while preserving interior dark detail:

./geotiff2pmtiles --nodata 0 --nodata-tolerance 40 --nodata-flood \
  scan.tif output.pmtiles
# Building nodata flood masks for 1 source(s)...
# Flood masks built in 710ms

Elevation data (auto-detects float GeoTIFF and selects Terrarium encoding):

./geotiff2pmtiles --verbose dem/ elevation.pmtiles

Multi-band satellite data (auto-detected — no band/rescale flags needed):

./geotiff2pmtiles --format png data2/ rgbnir.pmtiles
# Auto-detected: multispectral-rgbnir (bands 1,2,3, rescale linear [0, 10000])

RGBNIR satellite data with log rescaling and NIR as alpha:

./geotiff2pmtiles --bands 1,2,3 --alpha-band 4 --rescale log \
  --rescale-range 1,10000 --format png data2/ rgbnir.pmtiles

RGBNIR false-color composite (NIR-R-G):

./geotiff2pmtiles --bands 4,1,2 --alpha-band -1 --rescale linear \
  --rescale-range 0,8000 --format webp data2/ falsecolor.pmtiles

NIR as alpha (vegetation opaque, water/urban transparent — useful as overlay):

./geotiff2pmtiles --alpha-band 4 --rescale linear \
  --rescale-range 0,10000 --format png --type overlay data2/ nir-alpha.pmtiles

Convert a plain TIFF with TFW world file (global Natural Earth data):

./geotiff2pmtiles --format webp --max-zoom 6 data_tfw/ output.pmtiles

pmtransform

Transform an existing PMTiles archive: change format, zoom levels, resampling, or fill empty tiles. Always creates a new file — the original is never modified.

pmtransform [flags] <input.pmtiles> <output.pmtiles>

Flags

Flag Default Description
--format keep source Target tile encoding: jpeg, png, webp
--quality 85 JPEG/WebP quality (1-100)
--min-zoom keep source Minimum zoom level
--max-zoom keep source Maximum zoom level
--tile-size keep source Output tile size in pixels (inferred from first decoded tile)
--resampling bicubic Interpolation method: lanczos, bicubic, bilinear, nearest, mode
--rebuild false Force full pyramid rebuild (for resampling changes)
--terrarium auto-detected Treat tiles as terrarium-encoded elevations so rebuild downsamples in elevation space. Auto-detected from archive metadata written by geotiff2pmtiles
--fill-color 0,0,0,0 Substitute transparent/nodata with RGBA color (color transform); also fill missing tile positions. E.g. "0,0,0,255" or "#000000ff" (default: transparent)
--concurrency NumCPU Number of parallel workers
--mem-limit auto Tile store memory limit in MB (0 = auto ~90% of RAM)
--no-spill false Disable disk spilling
--attribution keep source Attribution string for data sources
--type keep source Layer type: baselayer, overlay
--verbose false Verbose progress output
--version Print version and exit

Examples

Convert WebP tiles to PNG format:

./pmtransform --format png input.pmtiles output.pmtiles

Extend zoom range by adding lower zoom levels (rebuilds pyramid):

./pmtransform --min-zoom 8 --verbose input.pmtiles output.pmtiles

Rebuild the entire pyramid with Lanczos resampling:

./pmtransform --rebuild --resampling lanczos input.pmtiles output.pmtiles

Substitute transparent/nodata with black and fill missing tile positions:

./pmtransform --fill-color "0,0,0,255" input.pmtiles output.pmtiles

Remove higher zoom levels (keep only z10-z14):

./pmtransform --min-zoom 10 --max-zoom 14 input.pmtiles output.pmtiles

Utilities

coginfo

Inspect COG file metadata (EPSG, dimensions, pixel size, bounds, overview levels):

go run ./cmd/coginfo/ <file.tif>

debug

Low-level COG debugging (float detection, NoData values, raw IFD info, sample tile bytes):

go run ./cmd/debug/ <file.tif>

checkpmtiles

Validate a PMTiles v3 archive for structural correctness (16 KiB root directory budget, section contiguity, directory integrity). Works with local files and HTTP URLs:

go run ./cmd/checkpmtiles/ output.pmtiles
go run ./cmd/checkpmtiles/ https://example.com/tiles.pmtiles

Architecture

See ARCHITECTURE.md for the full project structure, pipeline description, memory efficiency details, and how to add new projections.

Performance

The pipeline is profiled and optimized for throughput. Key techniques: LUT-accelerated resampling (precomputed Lanczos-3 and bicubic kernel tables), batched tile fetches to minimize cache lookups, native libwebp encoding via CGo, precomputed lon/lat arrays for O(n) Mercator projection, direct pixel buffer writes, and YCbCr fast paths.

Integration Tests

End-to-end tests exercise the full pipeline with synthetic and real satellite data:

make test-integration            # Synthetic tests only (~8s, no download needed)
make test-integration-download   # Download all real satellite data (~1.2 GB total)
make test-integration-all        # Download + run all tests

Seven real-data datasets are used, each exercising a different input type:

Dataset Size EPSG Type Description
copernicus/ ~8 MB 4326 Float32 Copernicus DEM GLO-30 — 30m elevation tile (Swiss Alps)
naturalearth/ ~200 MB 4326 8-bit RGB + TFW Natural Earth hypsometric tints (global, TFW sidecar)
esaworldcover/ ~455 MB 4326 16-bit 4-band ESA WorldCover S2 RGBNIR composite (Sentinel-2)
esaworldcover-ndvi/ ~168 MB 4326 8-bit 3-band ESA WorldCover S2 NDVI percentiles (p10/p50/p90)
esaworldcover-swir/ ~20 MB 4326 8-bit 2-band ESA WorldCover S2 SWIR composite (B11/B12)
esaworldcover-gamma0/ ~346 MB 4326 16-bit 3-band ESA WorldCover S1 SAR gamma0 VV/VH ratio
swissimage/ varies 2056 8-bit RGB swisstopo SWISSIMAGE DOP10 — 10cm orthophoto mosaic (LV95)

Per-dataset test targets:

make test-integration-copernicus           # Float32 DEM → terrarium PNG
make test-integration-naturalearth         # 8-bit RGB + TFW → JPEG
make test-integration-esaworldcover        # 16-bit 4-band RGBNIR → PNG (preset + pipeline tests)
make test-integration-esaworldcover-ndvi   # 8-bit 3-band NDVI → grayscale PNG
make test-integration-esaworldcover-swir   # 8-bit 2-band SWIR → grayscale PNG
make test-integration-esaworldcover-gamma0 # 16-bit 3-band SAR → PNG
make test-integration-swissimage           # 8-bit RGB EPSG:2056 mosaic → JPEG

Profiling

make example-swissimage-profile             # Run with CPU + memory profiling
go tool pprof -http=:8080 dist/cpu.prof    # Interactive flame graph
go tool pprof -http=:8081 dist/mem.prof    # Memory profile

License

MIT

About

No description, website, or topics provided.

Resources

Stars

17 stars

Watchers

3 watching

Forks

Releases

Packages

Contributors

Languages