Skip to content

sdk: handle gateway contract v2 (committed slow answers, new receipt headers, non-success finishes) - #12

Merged
capy-ai[bot] merged 1 commit into
mainfrom
capy/handle-gateway-contract-v2
Oct 2, 2026
Merged

capy-ai[bot] merged 1 commit into
mainfrom
capy/handle-gateway-contract-v2

Conversation

@charlespers

Copy link
Copy Markdown
Contributor

This catches both clients (TypeScript and Python, kept in parity) up to what the live gateway ships. The live build is typhoon e5e9eca, per api.conifer.build/healthz.

Committed slow answers (the correctness fix)
Since 2026-09-30, a non-streaming call still running at 285 s is answered with a provisional 200 marked x-conifer-keepalive: committed. Whitespace heartbeats follow, then the real body with conifer_receipt: {status, headers} appended. Before this PR, both clients returned such an answer with an empty receipt, and a committed error came back as a successful completion with no choices. New src/keepalive.ts / python/conifer_sdk/keepalive.py fix this:

  • The real status and receipt are read from conifer_receipt, and that member is stripped from the returned data.
  • A body cut mid-read, or one that ends without conifer_receipt, raises a retryable ConiferConnectionError and is retried under the same idempotency key. That cut is how the gateway ends an uncharged failure.
  • Transport.request() now also returns headers, which chat, embeddings and jobs read the receipt from.

Charged errors are never retried
ConiferError.receipt holds the receipt of an error that carries one, and retryable is false whenever it is set. This uses the gateway's own classifier: response-keepalive's carries_execution_receipt treats any receipt header as "money may have moved". A charged 502 or "retry shortly" 409 would otherwise have been re-sent and paid twice. Stream error frames are excluded, because a stream's head carries identity headers even when nothing was charged. The private beta cost headers aren't in the SDK's list, so a charged beta-router error is not covered. In TypeScript, retryable on ConiferRateLimitError and ConiferUnavailableError widens from true to boolean.

Receipt fields
pricingIdentity (x-conifer-pricing-identity) and pinFallback (x-conifer-receipt-pin-fallback), in snake_case in Python.

Non-success finishes
Completions expose finishReason, providerStopReason and providerStopDetails. A new incompleteReason(completion) (completion.incomplete_reason) flags anything other than stop or tool_calls: that covers length, content_filter, pause_turn, incomplete, a missing value and any unknown value. These turns are flagged, not thrown, like empty and length answers already are. The turn was charged, and a pause_turn is continued by sending its output back. They are never retried and never advance a client-side fallback chain. The MCP tools return incomplete_reason too.

Contract and streams

  • contracts/gateway-contract.json moves to v2, without model_management and the private beta_* fields.
  • STREAM_IDLE_MS changes from 120 000 to 280 000, matching the gateway's new stream idle bound.
  • New tests confirm that : keep-alive comments are skipped and restart the idle clock, and that a late in-band error on a committed stream throws the typed error.

Everything goes under ## [Unreleased] in the CHANGELOG. There is no version bump and nothing is published; releasing is a human step (RELEASING.md).

Verification (no CI evidence yet; live QA not run because it spends money)

  • npm run build && npm test: 246/246 pass (223 before). CONIFER_TEST_COMPILE=1 npm test: 246/246. npm run typecheck: clean.
  • python3 -m pytest -q: 159 passed and 14 subtests passed (138 before). unittest discover: 159 OK. check-changelog: ok.
  • The keepalive paths are tested against stubs and over a real socket, using a raw HTTP/1.1 server that aborts before the terminating chunk. A 1000x-scaled test shows the headers timeout still holds when the commit arrives at 285 s.
  • Mutation checks: reverting each fix made its new tests fail. That covers contract v1, the receipt fields, finish-as-success, the stop fields, committed responses, the charged-receipt rule, the mid-read cut, the headers timer, the idle constant, keep-alive restarting the idle clock, and in-band error frames.

Known gaps

  • ReceiptCollector / SpendBudget read headers only, so they still record nothing for a committed response. Fixing that means reading a cloned body.
  • This repo is the public mirror of conifer-workspace/sdk/, which hasn't absorbed Refuse inexact Helicone quotas and cost ceilings in both clients #11 either. The workspace copy needs this change ported before the next public-split.sh run, or the split will revert it.

Open in Capy

@greptile-apps

greptile-apps Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

[High risk] Updates SDK to handle gateway contract v2 with new response patterns.

Do not merge until the stalled committed-body read is bounded. The incomplete-receipt and Python retry issues are non-blocking follow-ups.

Fix All in Claude CodeFindings

  1. P1 Slow answer can hang forever ▶
  2. P2 Answer can lose its receipt ▶
  3. P2 Cut answer skips retry ▶
Fix with agent prompt
### Issue 1
src/transport.ts:182-184
If the gateway or a proxy sends committed response headers but leaves the body open without sending more data, the TypeScript client can wait indefinitely. The request timer ends at the headers, and the body read has no idle limit. Bound that read before merging so the caller can receive an error or retry.

### Issue 2
src/keepalive.ts:31-34
If a committed answer contains `conifer_receipt: {"status": 200}` but omits `headers`, both clients accept it and return the answer without settled cost or routing details. Require the carried headers object before treating the answer as complete; this is a non-blocking concern with incomplete gateway envelopes.

### Issue 3
python/conifer_sdk/transport.py:50-52
If an EOF-delimited committed answer ends midway through a UTF-8 character, Python raises `UnicodeDecodeError` outside the connection-error retry path. The caller sees a decoding exception after one request instead of a retry under the same key. This is a non-blocking failure for cut responses; treat the decoding error as a cut committed response.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR updates both SDKs for committed slow answers, charged-error receipts, new receipt fields, and unfinished turns. A stalled committed body can still leave the TypeScript caller waiting indefinitely. Both clients can return an answer from an incomplete receipt envelope, and Python can skip a retry when a committed answer is cut mid-character.

Reviews (1) · Last reviewed commit: "sdk: handle gateway contract v2 (committ..."

Comment thread src/transport.ts
Comment on lines +182 to 184
const committed = isCommitted(response.headers);
let cut: unknown;
data = await parseJson(response).catch((cause: unknown) => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Slow answer can hang forever

If the gateway or a proxy sends committed response headers but leaves the body open without sending more data, the TypeScript client can wait indefinitely. The request timer ends at the headers, and the body read has no idle limit. Bound that read before merging so the caller can receive an error or retry.

Knowledge Base Used: SDK client and transport

Artifacts

Local HTTP committed-response reproduction script

  • The authored script runs Transport against a local server in completed-body and open-body modes, providing the executed source.

Completed committed response output

  • The completed-body command resolved with the expected answer and exit code 0, establishing the comparison case.

Open committed response output

  • The open-body command remained pending beyond both configured timeouts and rejected only after caller abort, confirming the hang.

View artifacts

T-Rex Ran code and verified through T-Rex

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/transport.ts
Line: 182-184

Comment:
**Slow answer can hang forever**

If the gateway or a proxy sends committed response headers but leaves the body open without sending more data, the TypeScript client can wait indefinitely. The request timer ends at the headers, and the body read has no idle limit. Bound that read before merging so the caller can receive an error or retry.

**Knowledge Base Used:** [SDK client and transport](https://app.greptile.com/conifer/-/custom-context/knowledge-base/coniferkit/use-conifer/-/docs/sdk-client-transport.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code

Comment thread src/keepalive.ts
Comment on lines +31 to +34
const { status, headers: carried } = member as { status?: unknown; headers?: unknown };
if (typeof status !== "number" || !Number.isInteger(status)) return undefined;
const named = new Map<string, string>();
if (typeof carried === "object" && carried !== null) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Answer can lose its receipt

If a committed answer contains conifer_receipt: {"status": 200} but omits headers, both clients accept it and return the answer without settled cost or routing details. Require the carried headers object before treating the answer as complete; this is a non-blocking concern with incomplete gateway envelopes.

Knowledge Base Used:

Artifacts

Before-and-after verification command

  • Runs both focused client mocks against temporary guarded copies and the checked-out source, showing how the comparison was executed.

TypeScript chat client mock

  • Calls the TypeScript chat client with a mocked committed HTTP 200 OK response that omits receipt headers.

Python chat client mock

  • Calls the Python chat client with the same mocked committed HTTP 200 OK envelope.

Guarded-copy responses reject the missing-header envelope

  • Both executed clients reject the mocked HTTP 200 OK envelope under a temporary header-shape guard, rather than return an answer without a receipt.

Checked-out clients return answers without settled receipts

  • Both executed clients accept the mocked HTTP 200 OK envelope and return the answer with null cost and effective model, confirming the defect.

View artifacts

T-Rex Ran code and verified through T-Rex

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/keepalive.ts
Line: 31-34

Comment:
**Answer can lose its receipt**

If a committed answer contains `conifer_receipt: {"status": 200}` but omits `headers`, both clients accept it and return the answer without settled cost or routing details. Require the carried headers object before treating the answer as complete; this is a non-blocking concern with incomplete gateway envelopes.

**Knowledge Base Used:**
- [SDK receipts and errors](https://app.greptile.com/conifer/-/custom-context/knowledge-base/coniferkit/use-conifer/-/docs/sdk-receipts-and-errors.md)
- [Python client and receipts](https://app.greptile.com/conifer/-/custom-context/knowledge-base/coniferkit/use-conifer/-/docs/python-client-and-receipts.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code

Comment on lines +50 to +52
try:
text = response.read().decode("utf-8")
except (http.client.IncompleteRead, ConnectionError) as cause:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Cut answer skips retry

If an EOF-delimited committed answer ends midway through a UTF-8 character, Python raises UnicodeDecodeError outside the connection-error retry path. The caller sees a decoding exception after one request instead of a retry under the same key. This is a non-blocking failure for cut responses; treat the decoding error as a cut committed response.

Knowledge Base Used: Python client and receipts

Artifacts

Local HTTP reproduction script

  • The authored script sends a committed response cut mid-character, then a valid response; its optional in-memory exception-tuple change tests the proposed fix without modifying tracked code.

Unmodified client run with a cut UTF-8 response

  • The executed request raised UnicodeDecodeError after one POST, showing that the retry was bypassed.

Client run with an in-memory exception catch

  • The same request completed after two POSTs bearing the same idempotency key, showing that the catch enables the existing retry path.

View artifacts

T-Rex Ran code and verified through T-Rex

Prompt To Fix With AI
This is a comment left during a code review.
Path: python/conifer_sdk/transport.py
Line: 50-52

Comment:
**Cut answer skips retry**

If an EOF-delimited committed answer ends midway through a UTF-8 character, Python raises `UnicodeDecodeError` outside the connection-error retry path. The caller sees a decoding exception after one request instead of a retry under the same key. This is a non-blocking failure for cut responses; treat the decoding error as a cut committed response.

**Knowledge Base Used:** [Python client and receipts](https://app.greptile.com/conifer/-/custom-context/knowledge-base/coniferkit/use-conifer/-/docs/python-client-and-receipts.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code

@capy-ai
capy-ai Bot merged commit 7e0eba6 into main Oct 2, 2026
8 checks passed
@conifer-s-parallax
conifer-s-parallax Bot deleted the capy/handle-gateway-contract-v2 branch October 19, 2026 21:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant