You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Execute eoapi-cli export [OUTPUT_DIR] to export the STAC collections and items from a PgSTAC database to collections.ndjson and items.ndjson in OUTPUT_DIR (defaults to ./stac-export).
This is primarily meant for migrating data between PgSTAC instances: export from an old instance, then load the result into a new one with eoapi-cli ingest OUTPUT_DIR/collections.ndjson OUTPUT_DIR/items.ndjson.
Reviewed scripts/raw/export.sh, scripts/raw/ingest.sh, scripts/data-management.sh, scripts/lib/common.sh, eoapi-cli, and the docs updates. Checked SQL-quoting/escaping in both raw scripts (single-quote doubling for IN (...) lists and JSON payloads, \o path escaping), pipefail propagation from common.sh into the { ... } | psql pipeline in raw/ingest.sh (confirmed set -euo pipefail is inherited via source, so emit_records's exit 1 correctly fails the pipeline), the port-forward discovery/cleanup trap in data-management.sh, and the eoapi-cli command dispatch changes. No correctness bugs or injection issues found — collection IDs and JSON payloads are properly SQL-escaped, and db_user/db_pass are URI-encoded before being embedded in the discovered DSN.
One non-blocking observation:
scripts/raw/export.sh / scripts/raw/ingest.sh take --dsn (with embedded password) as a plain CLI argument, which is visible to other local users via ps//proc. The prior ingest.sh avoided this by running entirely inside the pod via kubectl exec. Since this is a deliberate design tradeoff (enabling a Kubernetes-free standalone tool) and DSNs are still typically local/ephemeral (port-forwarded or admin-supplied), this isn't blocking, but consider supporting PGPASSWORD/a --dsn-file option in a follow-up if this is expected to run against production credentials in shared environments.
Simplify (ponytail)
scripts/raw/export.sh:76-79 — psql_quote_path hand-rolls backslash/quote escaping; since OUTPUT_DIR is already canonicalized via cd "$OUTPUT_DIR" && pwd just above, the only realistic risky character is a literal ', so this is a reasonable minimal escape, not flagging further.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Execute
eoapi-cli export [OUTPUT_DIR]to export the STAC collections and items from a PgSTAC database tocollections.ndjsonanditems.ndjsoninOUTPUT_DIR(defaults to./stac-export).This is primarily meant for migrating data between PgSTAC instances: export from an old instance, then load the result into a new one with
eoapi-cli ingest OUTPUT_DIR/collections.ndjson OUTPUT_DIR/items.ndjson.