Skip to content

Refuse inexact Helicone quotas and cost ceilings in both clients - #11

Merged
charlespers merged 2 commits into
ConiferKit:mainfrom
Abhishek-B-R:fix/ts-python-parity-quota-and-ceiling
Sep 27, 2026
Merged

charlespers merged 2 commits into
ConiferKit:mainfrom
Abhishek-B-R:fix/ts-python-parity-quota-and-ceiling

Conversation

@Abhishek-B-R

@Abhishek-B-R Abhishek-B-R commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

  • TypeScript ceilingFromPolicy refuses a Helicone quota that is not plain digits, as Python already did. Python now also refuses a signed or underscored quota (-10, +10, 1_0), so both accept the same quota syntax. One gap stays: TypeScript also refuses a quota whose ceiling passes Number.MAX_SAFE_INTEGER (about 900,000,000 cents), which Python accepts; the changelog says so. Python also turns a quota longer than its int() digit limit into a portability error instead of a bare ValueError.
  • costCeiling and cost_ceiling refuse a negative ceiling instead of sending it.
  • TypeScript costCeiling refuses a value above Number.MAX_SAFE_INTEGER with the same ConiferPortabilityError it already throws for fractions.
  • Changelog entry under Unreleased.

Why

Two TypeScript helpers broke the twins rule and the money is exact rule.

ceilingFromPolicy read the quota with Number.parseInt, which stops at the first non-digit:

ceilingFromPolicy("1.5;w=60;u=cents");   // 10000000, a 1 cent ceiling
ceilingFromPolicy("10abc;w=60;u=cents"); // 100000000
ceilingFromPolicy("1e3;w=60;u=cents");   // 10000000

Python's ceiling_from_policy raises on all three, but accepts -10, +10 and 1_0 because int() does.

costCeiling only checked Number.isInteger:

costCeiling(1e21);        // "1e+21" goes out in x-conifer-max-cost-nanousd
costCeiling(2 ** 60 + 1); // "1152921504606847000", not 1152921504606846977
costCeiling(-1);          // "-1"

Expected: a quota is accepted only when it is plain digits in both languages, a negative ceiling is refused in both, and TypeScript refuses a ceiling it cannot hold exactly rather than sending a rounded or exponent form value.

One asymmetry is left on purpose: Python still sends a ceiling above 2^53 exactly, while TypeScript now refuses it. Closing that would mean accepting bigint in maxCostNanoUsd, a type change to ChatRequest and EmbeddingsRequest, so I left it for you to decide.

How it was verified

  • npm test and npm run typecheck pass (223 pass, 0 fail; typecheck clean)
  • Python suite passes: python3 -m unittest discover -s tests -t tests from python/, 138 tests OK (pytest is not installed on this machine, so I used the stdlib runner CONTRIBUTING mentions)
  • Behavioral change? A test that fails without this fix comes with it: paired cases in tests/portability.test.ts and tests/client.test.ts, and in python/tests/test_sdk.py. With the source change stashed, 2 TypeScript and 2 Python tests fail.
  • User-visible change? An entry under ## [Unreleased] in CHANGELOG.md (node scripts/check-changelog.mjs ok)

Covering tests at 2ad64d8 (merge sweep 2026-09-27)

  • node --experimental-strip-types --test tests/client.test.ts tests/portability.test.ts: 111 pass, 0 fail
  • cd python && python3 -m unittest tests.test_sdk: 138 tests OK (1 skipped)

Follow-ups (verifier P3s, non-blocking)

  • The CHANGELOG says an over-long Python quota is "now" refused as a portability error, but base already refused it that way.
  • "Both accept the same quota syntax" is not exact. The clients strip different whitespace (\x1c5 versus a BOM prefix), and Python now refuses Unicode-digit quotas that it used to accept. The CHANGELOG mentions neither.
  • Two table rows would kill the surviving mutants: TS quota 900719926 and Python ١٢٣.
  • A test comment says "1.5" became 10 cents; it actually became 1 cent.

The house rules say the two clients are twins and money is exact, and the
TypeScript side broke both. ceilingFromPolicy read the quota with parseInt,
so "1.5;w=60;u=cents" became a 1 cent ceiling and "10abc" or "1e3" were
accepted where Python raised. costCeiling passed any integer number through
String(), so 1e21 went out as "1e+21" and 2 ** 60 + 1 as a rounded amount.

The quota must now be plain digits in both languages, a negative ceiling is
refused in both, and TypeScript refuses a ceiling it cannot hold exactly.
@greptile-apps

greptile-apps Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

No outstanding findings block merging.

Summary

The PR tightens quota parsing and cost-ceiling validation in both clients. Python now wraps oversized-quota conversion errors, and the changelog clarifies where the clients differ. No outstanding findings remain.

Reviews (2) · Last reviewed commit: "sdk: refuse an over-long Helicone quota ..."

Comment thread python/conifer_sdk/portability.py Resolved
Comment thread CHANGELOG.md Resolved
@greptile-apps

This comment has been minimized.

@charlespers

Copy link
Copy Markdown
Contributor

VERIFIER: APPROVE 2ad64d8

Independent verification of the whole PR (git diff origin/main...HEAD, 8 files) at head 2ad64d8. The head contains origin/main (b9e1100, confirmed with merge-base --is-ancestor). I checked out a fresh clone with a scratch HOME, and a worktree of base b9e1100 for comparison.

Covering tests at head

  • node --experimental-strip-types --test tests/client.test.ts tests/portability.test.ts (Node 26.5.1): 111 pass, 0 fail
  • cd python && python3 -m unittest tests.test_sdk: Python 3.9.6 gives 138 run, OK (skipped=1). The skip is the int-digit-limit test, because 3.9 has no such limit. Python 3.11 gives 138 run, OK.
  • node scripts/check-changelog.mjs: ok
  • Revert check: I ran the head's tests against the base code. The TS run had 2 failures: a negative or inexact cost ceiling is refused, not sent and a rate-limit quota must be all digits. The Python run had 2 failures on both 3.9 and 3.11: test_rate_limit_quota_must_be_all_digits and test_a_negative_ceiling_is_refused. test_rate_limit_quota_past_the_int_digit_limit_is_refused passes at base (see F1).

Executed table repro, base -> head (u=cents quota; PortErr = ConiferPortabilityError)

quota TS base TS head Py base Py head
1e3 1 cent (1e7) PortErr PortErr PortErr
0.1 0 ceiling PortErr PortErr PortErr
10.005 10 cents PortErr PortErr PortErr
1_000 1 cent PortErr 1000 cents PortErr
5 5 cents 5 cents 5 cents 5 cents
-1 -1e7 PortErr -1e7 PortErr
`` (empty) PortErr PortErr PortErr PortErr
9007199254740993 9.007e+22 (inexact) PortErr exact int exact int (documented gap)
99999999999999999999 1e+27 PortErr exact int exact int (documented gap)
900719925 (last safe) 9007199250000000 same same same
900719926 9007199260000000 (unsafe) PortErr exact exact (documented gap)
1000, 0, 00010 1e10, 0, 1e8 same same same
١٢٣ (Arabic-Indic) PortErr PortErr 123 cents PortErr
5 (fullwidth) PortErr PortErr 5 cents PortErr
0x10 0 ceiling PortErr PortErr PortErr
10abc 10 cents PortErr PortErr PortErr
+10 10 cents PortErr 10 cents PortErr
\x1c5 PortErr PortErr 5 cents 5 cents
5 5 cents 5 cents PortErr PortErr
"1"*5000 n/a n/a PortErr on 3.11 PortErr on 3.11

costCeiling / cost_ceiling, base -> head:

  • TS: -1 went out as -1 and now raises PortErr. 1e21 went out as the header "1e+21" and now raises PortErr. 2**53 and 2**60 (rounded) were sent and now raise PortErr. 2**53-1 is still sent exactly. 0 and -0 both still send "0". 1.5, NaN and Infinity raise PortErr at both base and head.
  • Python: -1 was sent and now raises PortErr. 10**21 and 2**60 are still sent exactly, because Python ints are exact. True and 1.5 raise PortErr at both.

Money result: at head, no input in either client produces a ceiling above the value the user wrote. Every refusal I observed is ConiferPortabilityError, and no refusal escapes as a bare ValueError or TypeError. Every previously accepted plain-ASCII-digit quota, and every non-negative safe ceiling, is still accepted with the same value.

TS and Python agree on every row except three kinds. The CHANGELOG documents the first (quotas past MAX_SAFE_INTEGER / 1e7) but not the other two (F2).

Mutation checks on the new lines (covering tests; killed = suite fails)

  • Killed: TS unanchored regex; TS isFinite in place of the safe-integer check; parseInt restored; TS negative check dropped; TS < 0 changed to < -1; TS safe-integer check in costCeiling dropped; Python fullmatch changed to match; Python regex gate dropped; Python negative check dropped; Python except ValueError changed to except IndexError (killed on 3.11, survives on 3.9 only because that test is skipped there).
  • Survived:
    • TS isSafeInteger(quota * 10_000_000) changed to isSafeInteger(quota). No test uses a quota between 900719926 and 2^53. End to end this is still safe, because costCeiling refuses the unsafe amount at send time. F3.
    • Python [0-9]+ changed to \d+. No test covers Unicode digits. F3.

Findings

  • F1 (P3, docs): the CHANGELOG says "A quota too long for Python's int() digit limit is now refused as a portability error rather than a bare ValueError." Base already caught it: except (ValueError, IndexError) at portability.py:312. The 5000-digit repro raises PortErr at base on 3.11, and the new test passes at base. The claim is not a regression, but it describes a fix this PR does not make.
  • F2 (P3, parity/docs): the claim that "both accept the same quota syntax" is not quite true.
    • Python str.strip() and JS trim() strip different characters: \x1c5 is accepted by Python and refused by TS, and 5 is the reverse. The value is exact in both, so no money impact.
    • Python now also refuses Unicode-digit quotas (١٢٣, 5) that base accepted exactly. That is an intended tightening for parity, but the CHANGELOG does not mention it.
  • F3 (P3, tests): the two surviving mutants above. Adding the quota 900719926 to the TS table and ١٢٣ to the Python table would kill them.
  • F4 (P3, comment): tests/portability.test.ts says "1.5" became 10 cents. The base repro gives 1 cent (1e7 nanodollars), which matches the CHANGELOG.

There are no P0, P1 or P2 findings, and nothing about money, security or data loss blocks. The P3s are follow-ups under the merge bar.

@charlespers
charlespers merged commit e0ac116 into ConiferKit:main Sep 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants