Portolan CLI
Publish and manage cloud-native geospatial data catalogs.
Portolan is a CLI for publishing and managing cloud-native geospatial data catalogs. It orchestrates format conversion (GeoParquet, COG), versioning, and sync to object storage (S3, GCS, Azure) — no running servers, just static files.
Specification: The Portolan Spec defines catalog structure, format requirements, and versioning rules. When in doubt about requirements, read the spec files directly (core.md, structure.md, versions.md, formats/vector.md, formats/raster.md, formats/pointcloud.md, best-practices.md) for the most up-to-date information.
Key concepts:
- STAC (SpatioTemporal Asset Catalog) — The catalog metadata spec. Portolan is a STAC profile, not a competing spec.
- GeoParquet — Cloud-optimized vector data (columnar, spatial indexing)
- COG (Cloud-Optimized GeoTIFF) — Cloud-optimized raster data (HTTP range requests)
- COPC (Cloud-Optimized Point Cloud) — Cloud-optimized point cloud data
- PMTiles — Cloud-optimized vector/raster tiles for web map rendering
- versions.json — Single source of truth for version history, sync state, and checksums
Installation
bashpipx install portolan-cli # Isolated (recommended) pip install portolan-cli # Or with pip uv pip install portolan-cli # Or with uv
If portolan is not installed, guide the user through installation before proceeding.
Catalog Structure
Portolan catalogs use a flat hierarchy with STAC metadata:
project/ ├── .portolan/ │ ├── config.yaml # Internal config (sentinel file) │ └── state.json # Local sync state ├── catalog.json # STAC Catalog (root metadata) ├── versions.json # Catalog-level version tracking └── {collection_id}/ ├── collection.json # STAC Collection metadata ├── versions.json # Collection-level versioning ├── {data}.parquet # Single-file collection asset ├── {data}.pmtiles # Visualization derivative └── thumbnail.png # Preview image
Single-file collections (one GeoParquet/COG) use collection-level assets directly — no item directory needed.
Partitioned collections (>2GB) use item subdirectories per partition.
All STAC links use relative paths (SELF_CONTAINED) — catalogs are portable across hosting locations.
CLI Commands
portolan init
Initialize a new Portolan catalog.
bashportolan init # Initialize in current directory portolan init --auto # Skip prompts, use defaults portolan init --title "My Catalog" # Set title portolan init /path/to/data --auto # Initialize in specific directory
portolan scan
Scan a directory for geospatial files and potential issues.
bashportolan scan # Scan current directory portolan scan --json # JSON output portolan scan /data/geospatial portolan scan /large/tree --max-depth=2
portolan check
Validate a Portolan catalog or check files for cloud-native status.
bashportolan check # Validate all (metadata + geo-assets) portolan check --metadata # Validate metadata only portolan check --geo-assets # Check geo-assets only portolan check --fix # Fix both metadata and geo-assets
portolan add
Track files in the catalog.
bashportolan add demographics/census.parquet portolan add file1.geojson file2.geojson # Add multiple files portolan add imagery/ # Add all files in directory portolan add . # Add all files in catalog
portolan push
Push local catalog changes to cloud object storage.
bashportolan push s3://mybucket/catalog --collection demographics portolan push gs://mybucket/catalog -c imagery --dry-run portolan push s3://mybucket/catalog portolan push --dry-run # Uses configured remote
portolan pull
Pull updates from a remote catalog.
bashportolan pull s3://mybucket/my-catalog --collection demographics portolan pull s3://mybucket/catalog -c imagery --dry-run portolan pull s3://mybucket/catalog portolan pull s3://mybucket/catalog --workers 4
portolan sync
Sync local catalog with remote storage (pull + push).
bashportolan sync s3://mybucket/catalog --collection demographics portolan sync s3://mybucket/catalog -c imagery --dry-run portolan sync s3://mybucket/catalog -c data --fix --force
portolan clone
Clone a remote catalog to a local directory.
bashportolan clone s3://mybucket/my-catalog portolan clone s3://mybucket/my-catalog . portolan clone s3://mybucket/catalog -c demographics
portolan status
Show local vs remote version state for collections.
bashportolan status # Status for all collections portolan status -c demographics # Status for one collection portolan status --offline # Skip remote check portolan status --json # JSON output for agents
portolan list
List all files in the catalog with tracking status.
bashportolan list # List all files with status portolan list --collection demographics # Filter by collection portolan list --tracked-only # Show only tracked files
portolan info
Show information about a file, collection, or catalog.
bashportolan info demographics/census.parquet # File info portolan info demographics/ # Collection info portolan info # Catalog info portolan info demographics/census.parquet --json
portolan rm
Remove files from tracking.
bashportolan rm --keep imagery/old_data.tif # Safe: untrack only portolan rm --dry-run vectors/ # Preview portolan rm -f demographics/census.parquet # Force delete and untrack
portolan partition
Partition a large GeoParquet file for better query performance.
bashportolan partition buildings.parquet --preview portolan partition buildings.parquet output/ portolan partition buildings.parquet output/ --target-rows 50000
portolan extract
Extract data from external sources into Portolan catalogs.
bashportolan extract arcgis https://services.arcgis.com/.../FeatureServer ./output portolan extract arcgis URL --layers "Census*" --dry-run portolan extract arcgis URL --filter "sdn_*" --resume
portolan metadata
Manage catalog metadata for README generation.
bashportolan metadata init # Create template at catalog root portolan metadata init demographics # Create template for collection portolan metadata validate # Validate metadata.yaml
portolan readme
Generate README.md from STAC metadata and metadata.yaml.
bashportolan readme # Generate at catalog root portolan readme demographics # Generate for collection portolan readme --stdout # Print without writing portolan readme --check # CI mode: exit 1 if stale
portolan stac-geoparquet
Generate items.parquet for efficient STAC queries.
bashportolan stac-geoparquet # Generate for ALL collections portolan stac-geoparquet -c landsat # Generate for one collection portolan stac-geoparquet -c imagery --dry-run # Preview
portolan config
Manage catalog configuration.
bashportolan config set backend iceberg portolan config get remote portolan config list
portolan clean
Remove all Portolan metadata while preserving data files.
bashportolan clean # Remove all metadata portolan clean --dry-run # Preview what would be removed
Common Workflows
Publishing a New Catalog
bashportolan init --title "My Geospatial Data" portolan scan /data/geospatial portolan check --geo-assets --fix # Convert to cloud-native formats portolan add demographics/ portolan push s3://mybucket/my-catalog --collection demographics
Full Sync (Recommended)
Single command: pull -> init -> scan -> check -> push:
bashportolan sync s3://mybucket/my-catalog --collection demographics portolan sync s3://mybucket/my-catalog -c demographics --fix
Versioning
Versions follow Semantic Versioning per collection:
- Major: Breaking changes (schema changes, column removals, CRS changes)
- Minor: New features (new columns, new items)
- Patch: Data updates (same schema, new data)
Each collection's versions.json tracks version history with SHA-256 checksums per asset.
JSON Output
All commands support --json or --format json:
bashportolan scan . --json portolan check --format json portolan --format json init --auto
Consistent envelope: {"success": true, "command": "...", "data": {...}, "errors": []}.
Python API
pythonfrom portolan_cli import Catalog, FormatType, detect_format catalog = Catalog("/path/to/data") format_type = detect_format("data.parquet") # Returns FormatType.GEOPARQUET
Troubleshooting
| Error | Solution |
|---|---|
| "Not inside a Portolan catalog" | Run portolan init or navigate into existing catalog |
| "Catalog already exists" | Remove catalog.json and .portolan/ to reinitialize |
| "Push conflict: remote has newer version" | Run portolan pull first, or --force to overwrite |
| "Pull blocked by uncommitted changes" | Push local changes first, or --force to discard |
| Shapefile missing components | Ensure .shp, .shx, .dbf files are all present |
| Non-cloud-native files | Use portolan check --fix to convert (vectors -> GeoParquet, rasters -> COG) |
