Skip to content

source-hyperline: initial release - #5199

Open
Bio-mar wants to merge 2 commits into
estuary:mainfrom
Bio-mar:omar/source-hyperline
Open

Bio-mar wants to merge 2 commits into
estuary:mainfrom
Bio-mar:omar/source-hyperline

Conversation

@Bio-mar

@Bio-mar Bio-mar commented Sep 5, 2026

Copy link
Copy Markdown

Description

New estuary-cdk capture connector for Hyperline (billing / subscription management), covering 19 streams:

  • Incremental + backfill (5): customers, invoices, subscriptions, quotes, customer_credits — one shared window engine over Hyperline's updated_at__gt/gte/lt/lte filter grammar, parametrized across the API's two paging engines (v2 keyset cursor on customers/invoices, take/skip elsewhere). Boundary inclusivity, millisecond tick precision, filter+pagination composition, and page-size enforcement were all live-verified (evidence in bruno/).
  • Incremental + backfill (1): audit_logs — strictly-descending walk-back on happened_at with the provider's keyset token as the backfill PageCursor and a surrogate xxh128 key (events carry no id on the wire; an expired token fails loudly and restarts, never losing data).
  • Full-refresh snapshots (13): products, price_books, price_configurations, features, coupons, promotion_codes, tax_rates, invoicing_entities, users, customer_segments, transactions, wallets, subscription_transitions — no honored filters or sort exist on these endpoints (verified: unknown params are silently ignored), so snapshot is the only viable strategy. transactions' unbounded-growth risk is documented in the bruno collection.

Auth is a static bearer API key; test_-prefixed keys are routed to the sandbox host automatically. The bruno/ OpenCollection carries dated VERIFIED/UNOBSERVABLE findings behind every design decision, plus user-run-only seeding requests to close the item shapes of endpoints that are empty on the test account.

Also includes a one-line CONTRIBUTING.md fix (separate commit): the sops example still referenced the pre-migration helpful-kingdom-273219 KMS key; the repo standard is estuary-theatre's connector-repository key.

⚠️ Maintainer action needed: config re-encryption

I don't have access to Estuary's GCP KMS, so source-hyperline/config.yaml is sops-encrypted with my local age key — CI cannot decrypt it. Please re-encrypt with the connector-repository KMS key (I can supply the API key through a secure channel, or you can swap in an Estuary-owned Hyperline account). Until then test_capture will fail in CI; test_spec/test_discover are credential-free and pass.

Notes for review

  • The test account behind the snapshots/bruno examples holds production data. All saved artifacts were redacted (PII, free text, financial values, addresses) via the collection's after-response script, tests/test_snapshots.py's REDACTED_FIELDS, and a final sweep; the capture snapshot keeps one document per stream.
  • All 3 pytest snapshot tests pass locally (flowctl v0.6.13, preview-next); two consecutive capture runs were byte-identical.
  • subscription_transitions was deliberately classified snapshot rather than incremental: no updated_at exists and its candidate timestamps are nullable + date-granularity, so any range filter silently drops pending rows.

🤖 Generated with Claude Code

Bio-mar and others added 2 commits September 4, 2026 21:22
New estuary-cdk capture connector for Hyperline (billing/subscriptions).
19 streams: customers, invoices, subscriptions, quotes, and
customer_credits as incremental + backfill over the shared updated_at
filter grammar (keyset-cursor and take/skip paging engines); audit_logs
as a descending walk-back incremental with keyset-token backfill and a
surrogate hash key (events carry no id); 13 full-refresh snapshot
streams including subscription_transitions and transactions (no usable
cursor exists on either).

Auth is a static bearer API key; test_-prefixed keys route to the
sandbox host automatically. The bruno/ collection carries the dated
live-verification evidence behind every pagination, boundary, and
filter decision, with PII redacted from all saved examples.

Note for review: config.yaml is sops-encrypted with the contributor's
local age key (no Estuary KMS access) and needs re-encryption with the
connector-repository KMS key before CI can decrypt it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The example still referenced the pre-migration helpful-kingdom-273219
key; every encrypted config in the repo (including all 2025-2026
additions) uses estuary-theatre's connector-repository key.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant