Skip to content

feat: add pgstac export script. - #600

Merged
pantierra merged 2 commits into
mainfrom
feat/pgstac-export-script
Sep 3, 2026
Merged

feat: add pgstac export script.#600
pantierra merged 2 commits into
mainfrom
feat/pgstac-export-script

Conversation

@pantierra

@pantierra pantierra commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Execute eoapi-cli export [OUTPUT_DIR] to export the STAC collections and items from a PgSTAC database to collections.ndjson and items.ndjson in OUTPUT_DIR (defaults to ./stac-export).
This is primarily meant for migrating data between PgSTAC instances: export from an old instance, then load the result into a new one with eoapi-cli ingest OUTPUT_DIR/collections.ndjson OUTPUT_DIR/items.ndjson.

@claude

claude Bot commented Sep 1, 2026

Copy link
Copy Markdown

Claude finished @pantierra's task in 2m 15s —— View job


✅ No blocking issues — safe to merge.

Reviewed scripts/raw/export.sh, scripts/raw/ingest.sh, scripts/data-management.sh, scripts/lib/common.sh, eoapi-cli, and the docs updates. Checked SQL-quoting/escaping in both raw scripts (single-quote doubling for IN (...) lists and JSON payloads, \o path escaping), pipefail propagation from common.sh into the { ... } | psql pipeline in raw/ingest.sh (confirmed set -euo pipefail is inherited via source, so emit_records's exit 1 correctly fails the pipeline), the port-forward discovery/cleanup trap in data-management.sh, and the eoapi-cli command dispatch changes. No correctness bugs or injection issues found — collection IDs and JSON payloads are properly SQL-escaped, and db_user/db_pass are URI-encoded before being embedded in the discovered DSN.

One non-blocking observation:

  • scripts/raw/export.sh / scripts/raw/ingest.sh take --dsn (with embedded password) as a plain CLI argument, which is visible to other local users via ps//proc. The prior ingest.sh avoided this by running entirely inside the pod via kubectl exec. Since this is a deliberate design tradeoff (enabling a Kubernetes-free standalone tool) and DSNs are still typically local/ephemeral (port-forwarded or admin-supplied), this isn't blocking, but consider supporting PGPASSWORD/a --dsn-file option in a follow-up if this is expected to run against production credentials in shared environments.

Simplify (ponytail)

  • scripts/raw/export.sh:76-79psql_quote_path hand-rolls backslash/quote escaping; since OUTPUT_DIR is already canonicalized via cd "$OUTPUT_DIR" && pwd just above, the only realistic risky character is a literal ', so this is a reasonable minimal escape, not flagging further.

💰 Estimated review cost: $0.53 · 2m15s · 24 turns

@pantierra
pantierra force-pushed the feat/pgstac-export-script branch from 828c005 to d834657 Compare September 1, 2026 17:46
@pantierra
pantierra force-pushed the feat/pgstac-export-script branch from d834657 to d1e3567 Compare September 1, 2026 17:55
@pantierra
pantierra merged commit 8990e6b into main Sep 3, 2026
5 checks passed
@pantierra
pantierra deleted the feat/pgstac-export-script branch September 3, 2026 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant