diff --git a/docs/design.md b/docs/design.md
index bebc72d..c3aecaa 100644
--- a/docs/design.md
+++ b/docs/design.md
@@ -178,9 +178,11 @@ type SettingChange struct {
At *Position
}
-// Size is bytes only. A context operator does not count tokens: the library converts bytes to tokens (§4.2 rule 7).
+// Size is the bytes of a range, and optionally its tokens (§4.2 rule 7).
type Size struct {
- Bytes int // required on rewrite; the sum check compares it with the Body sizes (§6.5)
+ Bytes int // required on rewrite; the sum check compares it with the Body sizes (§6.5)
+ Tokens int // the host's count; 0 = not measured, so the library converts Bytes and marks it Estimated
+ MediaBytes int // the base64 part of Bytes; priced at a fixed 250 bytes a token, apart from the text ratio
}
type MessageSplice struct{ At, Removed, Inserted int } // messages only
@@ -273,6 +275,17 @@ and can be reported as unpriced ([R19](spec.md#r19)).
cache write). The provider bills the forwarded body, so the ratio is taken on it. If the request has no usage, the library uses a fixed fallback ratio. The library
marks every priced result that uses this conversion as `Estimated`. No operator names a tokenizer,
so two operators give the same tokens for the same bytes.
+ Two exceptions keep the ratio true (accountant#11):
+ - **Base64 media stays out of the ratio.** `NewBody` parses each position and counts the `data`
+ of an image or document block whose source has `type: base64`, and the payload of a `data:...;base64,` URL
+ (OpenAI form), in `PosHash.Media`. Text that only quotes "base64" is not media. `Media` above `Bytes` is capped. The ratio uses the text bytes only, over the billed tokens
+ less the media tokens (if the bill is below the media price, checks `media_tokens` and `ratio_range` fail and nothing is priced). Media is priced at a fixed 1 token for 250 bytes. A host that builds a
+ `Body` itself sets `PosHash.Media`. A host marks media inside a `Size` with `Size.MediaBytes`. If a size does not mark it, the library takes the
+ media of the range from the client body for a before-size and from the forwarded body, over the mapped range, for an after-size (the size's bytes beyond the range's text), so an unmarked cut of an image
+ message is not priced at the text ratio.
+ - **The host may pass tokens.** If a host knows the tokens of a range (a tokenizer, or a
+ count_tokens call), it sets `Size.Tokens`. The library uses them as they are, and does not mark
+ that size `Estimated`. `Tokens` 0 means not measured, and the bytes are converted.
8. **The engine reports a change that `apply` writes, under the context operator that decided it.**
`toolschema`, `toolfilter` and `cachesplit` are marker components: their `Reformat()` does
nothing, and `apply` performs the rewrite (`apply/envelope.go`, `apply/prefixsplit.go`). `apply`
@@ -674,6 +687,7 @@ type Body struct {
type PosHash struct {
Hash uint64 // cumulative: equal hashes mean an equal prefix up to this position
Bytes int32
+ Media int32 // the base64 image/document part of Bytes (Anthropic source blocks and data URLs); not in the hash
}
```
@@ -688,7 +702,7 @@ The fingerprint itself is built inside `Compute`. `Cached` comes from the usage,
after the response, so no earlier call can build it. `Compute` keeps the result in
`ThreadState.PrevPrint` for the next request.
-`PosHash` is 16 bytes in memory (8 for the hash, 4 for the size, 4 of padding). It holds hashes and sizes only ([R2](spec.md#r2)).
+`PosHash` is 16 bytes in memory (8 for the hash, 4 for the size, 4 for the media size). It holds hashes and sizes only ([R2](spec.md#r2)).
### 5.2 Classifying a break
@@ -809,8 +823,8 @@ cache break (§6.2), which counts forwarded content between the break and B\*. T
Both hosts send bytes for each forwarded position (`Body`) and for each operation (`Size`). The
library converts both with the request's own ratio (billed prompt tokens ÷ body bytes), as
`pricing.EstimateTokensFromBytes` does today. Results that use the ratio are marked `Estimated`.
-No host sends a token count, so the same tokens in both hosts holds whenever the two hosts receive
-the same bodies and the same operations.
+A host may also send a token count in `Size.Tokens` (§4.2 rule 7). The same tokens in both hosts
+holds whenever the two hosts receive the same bodies, the same operations and send the same token counts.
## 6. The library
@@ -1106,7 +1120,8 @@ type Unpriced struct {
type CheckFailure struct {
Check string // "sum" | "order" | "unique_change" | "spans" | "attribution" |
- // "conservation" | "shared_warm" (§6.5)
+ // "conservation" | "shared_warm" | "bad_usage" | "ratio_range" | "breakpoints" |
+ // "media_tokens" | "size_tokens" (§6.5)
Operator string // "" when the check is not about one operator
Detail string // numbers only, no content
}
@@ -1201,20 +1216,31 @@ check, the operator, the detail, and the request, session and thread IDs). `Prev
4. **Spans:** for a context operation with `Spans`, its `SizeBefore` and `SizeAfter` equal the sums over its spans.
5. **Attribution:** a `break` effect's `ChangeID` is the `ChangeID` of a context operation of the same request, or empty. A `rescue`, `ttlPremium` and `ttlRescue` are exempt: their `ChangeID` is a keep-alive window or a head entry, which names no operation of this request.
6. **Conservation:** the request total of the applied context operations and the cache effects equals the bill difference (§6.3).
- `DryRun` context operations are left out, because their content went upstream unchanged.
+ Each size is compared at the request ratio, so a correct `Size.Tokens` passes (the gap between the host's tokens
+ and the ratio's figure for the same bytes is taken out of the total). `DryRun` context operations are left out,
+ because their content went upstream unchanged.
7. **Shared warm:** `EntryFacts.SharedWarm`, when set, is a position in the forwarded body. If it is not, the check fails
with the section and index, and the keep-alive rescue of that request is not credited ([R54](spec.md#r54)).
A doubtful figure may only lower a saving.
8. **Usage sign:** no count in `Usage` is negative, and `CacheWrite1h` is not larger than `CacheWrite` (it is part of
it, never an addition). If one is, the check `bad_usage` fails.
-9. **Ratio range:** the tokens per forwarded byte, `(Input + CacheRead + CacheWrite) / bytes`, is from 0.05 to 2.0.
+9. **Ratio range:** the tokens per forwarded text byte, `(Input + CacheRead + CacheWrite - media price) / (bytes - media bytes)`,
+ is from 0.05 to 2.0. Base64 media bytes are not in the denominator, and the media price (1 token per 250 bytes) is not in the numerator.
If it is outside that range, the usage does not fit the body, and the check `ratio_range` fails. A ratio of 0
- (a body with bytes and no billed prompt token) and an infinite ratio (billed tokens and an empty body) are outside
- the range too. Without this
+ (a body with text bytes and no billed prompt token) and an infinite ratio (billed tokens and an empty body) are outside
+ the range too. A body of media only has no text bytes: it passes only when the bill equals the media price. A bill above it fails this check, because the extra tokens fit no content. A bill below
+ the media price fails this check as well as check 11. Without this
check the ratio, and so the conservation check (check 6), agree with themselves for any usage.
10. **Breakpoints:** a request has at most 4 forwarded breakpoints (the Anthropic limit), and each TTL (client and
forwarded) is `"5m"`, `"1h"` or empty (empty counts as `5m`). Otherwise the check `breakpoints` fails.
+11. **Media tokens (`media_tokens`):** the fixed price of the forwarded body's media (1 token per 250 bytes) is not above the
+ billed prompt tokens. If it is, the bill cannot hold the media: check 9 (`ratio_range`) fails too, nothing is priced, and
+ every operation goes to `Unpriced` with the reason `ratio_range`.
+12. **Size tokens (`size_tokens`):** a `Size.Tokens` is between 0.05 tokens for each text byte and 2.0 tokens for each byte
+ of its range (the limits of check 9). If it is not, the host's count is a bug: the library ignores it, converts the
+ bytes, and reports the check.
+
If check 8, 9 or 10 fails, the library prices nothing for the request. Each operation goes to `Unpriced` with the
reason `bad_usage`, `ratio_range` or `breakpoints` (the first check that fails, in that order), no cache effect is
added, and the cache state for the next request is unknown, as for a response with no usage (§8). The check name
@@ -1287,8 +1313,9 @@ Where a question has test cases, a host that compares them with its own code mar
- **Q3. Accuracy of the bytes ratio.** The request ratio is an average. Removed content (large JSON
tool output) may have fewer bytes per token than the average request. Option: `Size` takes a
content class (`text`, `json`, `binary`) and the library keeps a ratio for each class. Decide
- after measuring the error on real sessions. An optional `Tokens` field on `Size`, which
- the library would use instead of estimating, is also possible.
+ after measuring the error on real sessions. The optional `Tokens` field on `Size` is done
+ (accountant#11), as is the exclusion of base64 media from the ratio. A content class and an error
+ range for each `PricedOp` are still open.
- **Q4. Where the previous request's hashes live.** The counterfactual needs the previous request's
hashes on the same thread. The current design (D12, wiring document) has the host keep one
diff --git a/docs/spec.md b/docs/spec.md
index a15a124..2d21b79 100644
--- a/docs/spec.md
+++ b/docs/spec.md
@@ -283,8 +283,10 @@ numbers are not used: those requirements were dropped.
### C. Facts for pricing
- **R7.** Each context operation has a position: section and index, in the provider's hash order.
-- **R8.** Sizes are in bytes. A context operator sends no tokens. The library converts bytes to
- tokens and marks the result `Estimated` ([§4.2](design.md#42-rules) rule 7).
+- **R8.** Sizes are in bytes. The library converts bytes to tokens and marks the result `Estimated`
+ ([§4.2](design.md#42-rules) rule 7). A host that knows the tokens of a range may send them in `Size.Tokens`. The
+ library then uses them and does not mark that size `Estimated`. Base64 image and document bytes are not in the
+ conversion ratio.
- **R11.** Model-call usage is the provider's billed usage, read from the call's own response by
the model client the host gives the engine. The engine does not count it (wiring document).
- **R22.** A change has an ID that is the same on every request that carries it (`ChangeID`).
diff --git a/internal/engine/body.go b/internal/engine/body.go
index 8804234..ff00cbc 100644
--- a/internal/engine/body.go
+++ b/internal/engine/body.go
@@ -106,16 +106,55 @@ func lastAtOrBefore(bps []bp, x int) int {
return r
}
-// ratio converts forwarded bytes to tokens with the request's own ratio (§5.3).
+// mediaBytesPerToken is the fixed price of a media block (a base64 image or document): about one
+// token for each 250 bytes of its base64 text. Media tokens do not follow the text ratio, so the
+// request ratio leaves the media bytes out and takes the media tokens out of the billed total
+// first (§4.2 rule 7).
+const mediaBytesPerToken = 250
+
+// cumMedia returns the media bytes of positions 0..i; 0 for i < 0.
+func (b Body) cumMedia(i int) int {
+ n := 0
+ for j := 0; j <= i && j < len(b.Pos); j++ {
+ n += int(max(min(b.Pos[j].Media, b.Pos[j].Bytes), 0)) // a host-built Media outside 0..Bytes is clamped
+ }
+ return n
+}
+
+// ratio converts forwarded text bytes to tokens with the request's own ratio (§5.3). The bytes and
+// the tokens are those of the text only: media is priced apart (mediaBytesPerToken).
type ratio struct{ tokens, bytes int }
+// newRatio builds the ratio of a body that the provider billed for billed prompt tokens.
+// The tokens clamp to 0 when the bill is below the media's fixed price; checks() reports that.
+func newRatio(billed int, b Body) ratio {
+ last := len(b.Pos) - 1
+ media := b.cumMedia(last)
+ return ratio{tokens: max(billed-media/mediaBytesPerToken, 0), bytes: b.cumBytes(last) - media}
+}
+
func (r ratio) of(bytes int) int {
- if r.bytes == 0 {
+ if r.bytes <= 0 {
return 0
}
return int(int64(bytes) * int64(r.tokens) / int64(r.bytes))
}
+// tokensOf converts bytes of which media are media bytes: the text with the ratio, the media with
+// the fixed price.
+func (r ratio) tokensOf(bytes, media int) int {
+ return r.of(bytes-media) + media/mediaBytesPerToken
+}
+
+// upTo is the tokens of positions 0..i of b.
+func (r ratio) upTo(b Body, i int) int { return r.tokensOf(b.cumBytes(i), b.cumMedia(i)) }
+
+// between is the tokens of positions from+1..to of b (the bytes of position from are not counted
+// twice: the same difference as cumBytes(to) - cumBytes(from)).
+func (r ratio) between(b Body, from, to int) int {
+ return r.tokensOf(b.cumBytes(to)-b.cumBytes(from), b.cumMedia(to)-b.cumMedia(from))
+}
+
// extent returns the last candidate position whose estimated prefix fits in tokens, or -1. The
// tolerance absorbs the error of the byte ratio and the 128-token rounding of older OpenAI models.
// With no candidate breakpoint every position is a candidate.
@@ -126,7 +165,7 @@ func extent(b Body, r ratio, tokens int, candidates []bp) int {
limit := tokens + 128 + tokens/50
best := -1
try := func(i int) {
- if i > best && r.of(b.cumBytes(i)) <= limit {
+ if i > best && r.upTo(b, i) <= limit {
best = i
}
}
diff --git a/internal/engine/compute.go b/internal/engine/compute.go
index 06b2eeb..1fe2bd6 100644
--- a/internal/engine/compute.go
+++ b/internal/engine/compute.go
@@ -198,11 +198,11 @@ func isChange(op Operation) bool {
}
func negative(op Operation) bool {
- if op.SizeBefore.Bytes < 0 || op.SizeAfter.Bytes < 0 {
+ if op.SizeBefore.invalid() || op.SizeAfter.invalid() {
return true
}
for _, s := range op.Spans {
- if s.SizeBefore.Bytes < 0 || s.SizeAfter.Bytes < 0 {
+ if s.SizeBefore.invalid() || s.SizeAfter.invalid() {
return true
}
}
@@ -236,6 +236,63 @@ func apply(p Position, s *MessageSplice, into bool) (Position, bool) {
return p, into
}
+// plausible says whether host tokens fit the bytes of the range. media is the range's media bytes.
+func plausible(s Size, media int) bool {
+ text := s.Bytes - media
+ if float64(s.Tokens) > maxTokensPerByte*float64(s.Bytes) {
+ return false
+ }
+ return text <= 0 || float64(s.Tokens) >= minTokensPerByte*float64(text)
+}
+
+// rangeMedia is the media bytes a size covers. The host's MediaBytes wins. When it is 0, the media
+// comes from the body that holds the range: the client body for a "before" size, and the forwarded
+// body, over the range mapped by toFwd, for an "after" size. The media is the size's bytes beyond the
+// range's text, clamped to 0..media. Without this an unmarked size over an image would price its
+// base64 at the text ratio.
+func (c *calcState) rangeMedia(k int, s Size, sp span, isBefore bool) int {
+ if s.MediaBytes > 0 || s.Bytes == 0 {
+ return s.MediaBytes
+ }
+ body, from, to := c.client, sp.from, sp.to
+ if !isBefore {
+ body = c.fwd
+ var ok1, ok2 bool
+ if sp.before.Bytes == 0 && c.items[k].op.MessageSplice != nil { // an inserted item sits in its own slot
+ from, ok1 = c.toFwdSkipping(k, sp.from)
+ to, ok2 = c.toFwdSkipping(k, sp.to)
+ } else {
+ from, ok1 = c.toFwd(k, sp.from)
+ to, ok2 = c.toFwd(k, sp.to)
+ }
+ if !ok1 || !ok2 {
+ return 0
+ }
+ }
+ f, ok1 := body.flat(from)
+ t, ok2 := body.flat(to)
+ if !ok1 || !ok2 || t < f {
+ return 0
+ }
+ media := body.cumMedia(t) - body.cumMedia(f-1)
+ text := body.cumBytes(t) - body.cumBytes(f-1) - media
+ return min(max(s.Bytes-text, 0), media)
+}
+
+// sizeTokens converts one reported size to tokens. A plausible size the host counted (Tokens > 0) is
+// exact. Any other size is converted from its bytes with the request ratio, and so is an estimate. A
+// size of zero bytes is exactly zero tokens (§4.2 rule 7). bad is true when the host's tokens were
+// ignored as implausible.
+func (c *calcState) sizeTokens(s Size, media int) (tokens int, exact, bad bool) {
+ if s.Tokens > 0 {
+ if plausible(s, media) {
+ return s.Tokens, true, false
+ }
+ bad = true
+ }
+ return c.ratio.tokensOf(s.Bytes, media), s.Bytes == 0, bad
+}
+
// calcState is the working state of one calculation.
type calcState struct {
r *Request
@@ -247,6 +304,7 @@ type calcState struct {
invalid string // the reason the usage or breakpoints cannot be trusted; "" when they can (§6.5, checks 8 to 10)
seeded bool // s.PrevPrint is the parent's fingerprint (seededFromParent), not this thread's own
ratio ratio
+ hostGap int // saved tokens that come from Size.Tokens beyond the ratio's figure (check 6)
client, fwd Body
clientBPs, fwdBPs []bp
@@ -278,7 +336,7 @@ func (r *Request) newCalc(s ThreadState, items []*item, usage Usage) *calcState
c.splicers = append(c.splicers, j)
}
}
- c.ratio = ratio{tokens: usage.Input + usage.CacheRead + usage.CacheWrite, bytes: r.Forwarded.cumBytes(len(r.Forwarded.Pos) - 1)}
+ c.ratio = newRatio(usage.Input+usage.CacheRead+usage.CacheWrite, r.Forwarded)
c.invalid = c.validate(usage)
c.clientBPs = c.client.breakpoints(r.ClientBreakpoints)
c.fwdBPs = c.fwd.breakpoints(r.Breakpoints)
@@ -476,7 +534,7 @@ func (c *calcState) pingRead() *Position {
// Only the tokens the ping cached are in that body. Its Input after the prefix (a sweep question,
// a summary request) is billed but is not part of the body, so it would raise the ratio and pull
// the extent back to an earlier message.
- r := ratio{tokens: u.CacheRead + u.CacheWrite, bytes: prev.Forwarded.cumBytes(len(prev.Forwarded.Pos) - 1)}
+ r := newRatio(u.CacheRead+u.CacheWrite, prev.Forwarded)
i := -1
if r.tokens > 0 { // a usage that bills no prompt token proves nothing about the extent
i = extent(prev.Forwarded, r, u.CacheRead+u.CacheWrite, nil)
@@ -518,7 +576,7 @@ func (c *calcState) phaseAndBoundary(res *Result) {
// with no ping, a gap past a minimum proves no expiry (R45)
default:
c.phase = PhaseCold
- if gap-f.TTL <= ttlSkew && c.usage.CacheRead > 0 && c.readReaches(prev.Forwarded.cumBytes(mustFlat(prev.Forwarded, *c.s.Cache.Extent))) {
+ if gap-f.TTL <= ttlSkew && c.usage.CacheRead > 0 && c.readReaches(prev.Forwarded, mustFlat(prev.Forwarded, *c.s.Cache.Extent)) {
// The clock says expired by at most ttlSkew, but the provider billed a read that covers the whole previous
// extent (clock skew, queue time). The read is the evidence; it only lowers a saving (R45).
c.phase, c.prevExt = PhaseHot, mustFlat(prev.Forwarded, *c.s.Cache.Extent)
@@ -583,16 +641,16 @@ func (c *calcState) phaseAndBoundary(res *Result) {
const ttlSkew = 10 * time.Second
// readReaches reports whether Usage.CacheRead covers the first bytes bytes of the previous forwarded
-// body, with the tolerance of extent (128 tokens plus 2%). The bytes are priced at the byte ratio of
-// this request. It is the one tolerance for the TTL-edge rule and for the B* rule (decision 7).
-func (c *calcState) readReaches(bytes int) bool {
- want := c.ratio.of(bytes)
+// body up to position i, with the tolerance of extent (128 tokens plus 2%). The prefix is priced at the
+// byte ratio of this request, its media at the fixed media price. It is the one tolerance for the TTL-edge rule and for the B* rule (decision 7).
+func (c *calcState) readReaches(b Body, i int) bool {
+ want := c.ratio.upTo(b, i)
return want > 0 && want <= c.usage.CacheRead+128+c.usage.CacheRead/50
}
// readCoversExtent reports whether the tokens the provider read reach the previous request's cached extent.
func (c *calcState) readCoversExtent(prev *Fingerprint) bool {
- return c.readReaches(prev.Forwarded.cumBytes(c.prevExt))
+ return c.readReaches(prev.Forwarded, c.prevExt)
}
// readable is the last position up to x that a request with the breakpoints bps can read. The
@@ -657,7 +715,7 @@ func (c *calcState) cacheEffects(res *Result) {
// past the 5m TTL only the entries held at 1h are still alive: a 5m part past the head expired
expect = min(expect, c.headEnd())
}
- predicted := c.ratio.of(c.fwd.cumBytes(expect))
+ predicted := c.ratio.upTo(c.fwd, expect)
low := c.usage.CacheRead+128+predicted/50 < predicted // the one-block tolerance (row 4)
if !low {
return
@@ -734,7 +792,7 @@ func (c *calcState) addBreak(res *Result, from int, changeID string) {
if c.bstarFwd >= 0 {
p := c.fwd.position(c.bstarFwd)
e.To = &p
- if n := c.ratio.of(c.fwd.cumBytes(c.bstarFwd) - c.fwd.cumBytes(from)); n > 0 {
+ if n := c.ratio.between(c.fwd, from, c.bstarFwd); n > 0 {
e.Saved.CacheRead = n
ttl, anyBP, ok := c.writeBP(c.fwd, c.fwdBPs, c.bstarFwd)
if c.noCacheTokens() || (anyBP && !ok) {
@@ -759,7 +817,7 @@ func (c *calcState) rescue(res *Result) {
if c.usage.CacheRead <= 0 {
return
}
- n := min(c.ratio.of(c.fwd.cumBytes(mustFlat(c.fwd, *c.pingRead()))), c.usage.CacheRead)
+ n := min(c.ratio.upTo(c.fwd, mustFlat(c.fwd, *c.pingRead())), c.usage.CacheRead)
if w := c.r.Facts.SharedWarm; w != nil {
i, ok := c.fwd.flat(*w)
if !ok { // a host bug: a doubtful figure may only lower a saving, so credit nothing (R54)
@@ -769,7 +827,7 @@ func (c *calcState) rescue(res *Result) {
}
// a sibling kept this part warm without the ping (R54). Two floors, so n can be 1 token off the single
// conversion: the same tolerance the conservation check allows.
- n -= c.ratio.of(c.fwd.cumBytes(i))
+ n -= c.ratio.upTo(c.fwd, i)
}
if n <= 0 {
return
@@ -805,7 +863,7 @@ func (c *calcState) headTTL(res *Result) {
if head < 0 {
return
}
- n := min(c.ratio.of(c.fwd.cumBytes(head)), c.usage.CacheRead)
+ n := min(c.ratio.upTo(c.fwd, head), c.usage.CacheRead)
e := CacheEffect{Kind: EffectTTLRescue, ChangeID: id, TTL: "1h", Estimated: true}
e.Saved.CacheWrite5m, e.Caused.CacheRead = n, n
c.addEffect(res, e)
@@ -833,9 +891,9 @@ func (c *calcState) split(res *Result) {
if c.s.PrevPrint == nil || c.clientFD >= c.client.sectionEnd(1) {
return
}
- n := min(c.ratio.of(c.fwd.cumBytes(c.fwdFD-1)), c.usage.CacheRead)
+ n := min(c.ratio.upTo(c.fwd, c.fwdFD-1), c.usage.CacheRead)
if c.bstarFwd >= 0 { // the client body would have read up to B* anyway: only the rest is the split's
- n -= c.ratio.of(c.fwd.cumBytes(c.bstarFwd))
+ n -= c.ratio.upTo(c.fwd, c.bstarFwd)
}
if n <= 0 {
return
@@ -883,7 +941,7 @@ func (c *calcState) writeBP(body Body, bps []bp, p int) (ttl string, anyBP, ok b
continue
}
anyBP = true
- if (best < 0 || x.at < best) && (c.r.Facts.MinCacheable <= 0 || c.ratio.of(body.cumBytes(x.at)) >= c.r.Facts.MinCacheable) {
+ if (best < 0 || x.at < best) && (c.r.Facts.MinCacheable <= 0 || c.ratio.upTo(body, x.at) >= c.r.Facts.MinCacheable) {
best, ttl = x.at, x.ttl
}
}
@@ -927,7 +985,7 @@ func (c *calcState) addTier(t *TierTokens, body Body, forwarded bool, p, bstar i
// breakpoints failed check 8, 9 or 10 (§6.5).
func (c *calcState) skipsPricing() bool { return c.noUsage || c.invalid != "" }
-// Limits of check 9 and check 10 (§6.5).
+// Limits of check 9 and check 10 (§6.5). Check 12 (size_tokens) uses the two token limits too.
const (
minTokensPerByte = 0.05
maxTokensPerByte = 2.0
@@ -951,12 +1009,23 @@ func (c *calcState) validate(u Usage) string {
return ""
}
-// badRatio says whether the tokens per forwarded byte are outside 0.05 to 2.0 (check 9). A usage that
+// badRatio says whether the tokens per forwarded text byte are outside 0.05 to 2.0 (check 9). Media
+// bytes are not in the denominator and the media price is not in the numerator (newRatio). A usage that
// bills no prompt token for a body with bytes (tokens per byte 0), or tokens for an empty body (infinite),
// is outside the range too.
func (c *calcState) badRatio() bool {
+ last := len(c.fwd.Pos) - 1
+ if c.fwd.cumBytes(last) == 0 {
+ return true // an empty body billed for tokens, or for none
+ }
+ if c.fwd.cumMedia(last)/mediaBytesPerToken > c.usage.Input+c.usage.CacheRead+c.usage.CacheWrite {
+ return true // the bill is below the media price: check 11 fails too
+ }
t, n := float64(c.ratio.tokens), float64(c.ratio.bytes)
- if t == 0 || n == 0 {
+ if n == 0 {
+ return t != 0 // all media: the bill must be the media price, and no text is left to price
+ }
+ if t == 0 {
return true
}
return t < minTokensPerByte*n || t > maxTokensPerByte*n
@@ -997,6 +1066,7 @@ func (c *calcState) price(res *Result) {
continue
}
p := priced(it.op, it.id)
+ estimated := false // true when a size is converted from bytes (§4.2 rule 7)
switch {
case it.bad != "":
res.Unpriced = append(res.Unpriced, Unpriced{OpID: it.id, Reason: it.bad})
@@ -1007,9 +1077,24 @@ func (c *calcState) price(res *Result) {
case resultUnknown(it.op):
res.Unpriced = append(res.Unpriced, Unpriced{OpID: it.id, Reason: ReasonPingResultUnknown})
case it.op.Kind == KindRewrite:
- p.Estimated = true // the tokens come from the bytes ratio (§4.2 rule 7)
for _, sp := range it.spans {
- d := c.ratio.of(sp.before.Bytes) - c.ratio.of(sp.after.Bytes)
+ bm, am := c.rangeMedia(k, sp.before, sp, true), c.rangeMedia(k, sp.after, sp, false)
+ bt, bx, bbad := c.sizeTokens(sp.before, bm)
+ at, ax, abad := c.sizeTokens(sp.after, am)
+ if bbad || abad {
+ res.Checks = append(res.Checks, CheckFailure{Check: "size_tokens", Operator: it.op.Operator,
+ Detail: fmt.Sprintf("tokens=%d/%d bytes=%d/%d: outside %g..%g tokens a byte, bytes used",
+ sp.before.Tokens, sp.after.Tokens, sp.before.Bytes, sp.after.Bytes, minTokensPerByte, maxTokensPerByte)})
+ }
+ if !bx || !ax {
+ estimated = true
+ }
+ d := bt - at
+ if !it.op.DryRun {
+ // what the host's tokens add to the saving beyond the ratio's figure: the
+ // conservation check compares at the ratio (check 6)
+ c.hostGap += d - (c.ratio.tokensOf(sp.before.Bytes, bm) - c.ratio.tokensOf(sp.after.Bytes, am))
+ }
if d > 0 {
c.addTier(&p.Saved, c.client, false, mustFlat(c.client, sp.from), c.bstar, c.clientBPs, d)
if !it.op.DryRun {
@@ -1031,6 +1116,7 @@ func (c *calcState) price(res *Result) {
}
}
}
+ p.Estimated = p.Estimated || estimated
if it.active {
active = append(active, p)
} else {
@@ -1084,7 +1170,8 @@ func (c *calcState) conservation(res *Result) {
tol++
}
billed := c.usage.Input + c.usage.CacheRead + c.usage.CacheWrite
- want := c.ratio.of(c.client.cumBytes(len(c.client.Pos)-1)) - billed
+ want := c.ratio.upTo(c.client, len(c.client.Pos)-1) - billed
+ net -= c.hostGap // the ratio's figure: host tokens are exact and the bodies are not
if d := want - net; d > tol+1 || d < -(tol+1) {
res.Checks = append(res.Checks, CheckFailure{Check: "conservation",
Detail: fmt.Sprintf("bill difference=%d ops=%d tokens", want, net)})
@@ -1097,6 +1184,10 @@ func tierSum(t TierTokens) int { return t.Input + t.CacheWrite5m + t.CacheWrite1
// are not run here (conservation is, in calc): attribution holds by construction, because the library
// only gives a break the ChangeID of one of the request's operations.
func (c *calcState) checks(res *Result) {
+ if media := c.fwd.cumMedia(len(c.fwd.Pos)-1) / mediaBytesPerToken; !c.noUsage && media > c.usage.Input+c.usage.CacheRead+c.usage.CacheWrite {
+ res.Checks = append(res.Checks, CheckFailure{Check: "media_tokens",
+ Detail: fmt.Sprintf("media=%d tokens billed=%d: the bill is below the media price: nothing is priced (ratio_range)", media, c.usage.Input+c.usage.CacheRead+c.usage.CacheWrite)})
+ }
var got [3]int
seen := map[string]bool{}
for _, it := range c.items {
diff --git a/internal/engine/doc.go b/internal/engine/doc.go
index ebeee3e..a038217 100644
--- a/internal/engine/doc.go
+++ b/internal/engine/doc.go
@@ -12,10 +12,12 @@
// The design leaves some rules open. This implementation decides them as follows. Each one is
// the simplest rule that the design allows.
//
-// 1. Tokens from bytes. Size is bytes only (§4.2 rule 7). The ratio is the billed prompt tokens
+// 1. Tokens from bytes. A Size is bytes (§4.2 rule 7), or tokens when the host sets Size.Tokens:
+// those are used as they are and are not Estimated. The ratio is the billed prompt tokens
// (input + cache read + cache write) divided by the bytes of the forwarded body, because the
-// billed tokens bill the forwarded body (§5.3). Every priced rewrite and every effect that uses
-// the ratio is Estimated.
+// billed tokens bill the forwarded body (§5.3). Base64 media (PosHash.Media, Size.MediaBytes)
+// is left out of the ratio and priced at a fixed 250 bytes a token. Every priced rewrite that
+// converts bytes and every effect that uses the ratio is Estimated.
// 2. No usage. A response with no usage prices nothing (§8): every operation goes to Unpriced
// with ReasonNoUsage. The fixed fallback ratio of §4.2 rule 7 is not used.
// 3. Extents from usage. The provider reports token counts, not positions. The read extent is the
diff --git a/internal/engine/fingerprint.go b/internal/engine/fingerprint.go
index 521e4e1..997a41f 100644
--- a/internal/engine/fingerprint.go
+++ b/internal/engine/fingerprint.go
@@ -1,6 +1,11 @@
package engine
-import "time"
+import (
+ "bytes"
+ "encoding/json"
+ "strings"
+ "time"
+)
// Fingerprint is the most recent request of one thread (§5.1). The library builds it inside
// Compute, because Cached comes from the usage, which arrives after the response.
@@ -33,6 +38,9 @@ type Body struct {
type PosHash struct {
Hash uint64 `json:"hash"` // cumulative: equal hashes mean an equal prefix up to this position
Bytes int32 `json:"bytes"`
+ // Media is the part of Bytes that is a base64 image or document block. NewBody finds it. The
+ // token ratio leaves it out (§4.2 rule 7). It is not in the hash.
+ Media int32 `json:"media,omitempty"`
}
// Breakpoint is a cache breakpoint in a body.
@@ -53,7 +61,7 @@ func NewBody(tools, system, messages [][]byte) Body {
h = (h ^ uint64(c)) * 1099511628211
}
h = (h ^ 0xff) * 1099511628211 // position separator
- b.Pos = append(b.Pos, PosHash{Hash: h, Bytes: int32(len(p))})
+ b.Pos = append(b.Pos, PosHash{Hash: h, Bytes: int32(len(p)), Media: int32(mediaBytes(p))})
}
}
b.cum = make([]int, len(b.Pos)+1)
@@ -62,3 +70,109 @@ func NewBody(tools, system, messages [][]byte) Body {
}
return b
}
+
+// mediaBytes counts the base64 media payload in one position (R2: it keeps only the count). Two
+// forms count:
+// - an image or document block whose "source" has "type":"base64": the length of the source's
+// "data" string (Anthropic form);
+// - a "url", "image_url" or "file_data" string that starts with "data:" and holds ";base64,": the bytes after
+// the comma (OpenAI data-URL form).
+//
+// Text that only quotes "base64" or "data" is not media. A position that is not valid JSON counts
+// what was read before the error.
+func mediaBytes(p []byte) int {
+ dec := json.NewDecoder(bytes.NewReader(p))
+ n := 0
+ for {
+ if _, ok := walkValue(dec, &n); !ok {
+ return n
+ }
+ }
+}
+
+// valueInfo is what a parent needs to know about one parsed value.
+type valueInfo struct {
+ str string // the value, when it is a string
+ isStr bool
+ base64 int // the "data" length, when the value is a base64 source object; else 0
+}
+
+const dataURLMarker = ";base64,"
+
+// dataURLPayload returns the payload length of a base64 image or PDF data URL, or 0. Another MIME
+// type (data:text/html;base64,...) is text.
+func dataURLPayload(s string) int {
+ if !strings.HasPrefix(s, "data:image/") && !strings.HasPrefix(s, "data:application/pdf") {
+ return 0
+ }
+ i := strings.Index(s, dataURLMarker)
+ if i < 0 {
+ return 0
+ }
+ return len(s) - i - len(dataURLMarker)
+}
+
+// walkValue reads one JSON value and adds the media it holds to *n. false means the input ended
+// or is not valid JSON.
+func walkValue(dec *json.Decoder, n *int) (valueInfo, bool) {
+ tok, err := dec.Token()
+ if err != nil {
+ return valueInfo{}, false
+ }
+ switch v := tok.(type) {
+ case string:
+ return valueInfo{str: v, isStr: true}, true
+ case json.Delim:
+ if v == '[' {
+ for dec.More() {
+ if _, ok := walkValue(dec, n); !ok {
+ return valueInfo{}, false
+ }
+ }
+ _, err := dec.Token()
+ return valueInfo{}, err == nil
+ }
+ return walkObject(dec, n)
+ }
+ return valueInfo{}, true
+}
+
+// walkObject reads the members of an object whose "{" is read. It counts the payload of an image
+// or document block's base64 source, and of a data URL under "url", "image_url" or "file_data". It returns the
+// payload length when the object itself is a base64 source, so its parent block can count it.
+func walkObject(dec *json.Decoder, n *int) (valueInfo, bool) {
+ var typ string
+ var data valueInfo
+ var srcLen int
+ for dec.More() {
+ kt, err := dec.Token()
+ if err != nil {
+ return valueInfo{}, false
+ }
+ key, _ := kt.(string)
+ child, ok := walkValue(dec, n)
+ if !ok {
+ return valueInfo{}, false
+ }
+ switch key {
+ case "type":
+ typ = child.str
+ case "data":
+ data = child
+ case "source":
+ srcLen = child.base64
+ case "url", "image_url", "file_data":
+ *n += dataURLPayload(child.str)
+ }
+ }
+ if _, err := dec.Token(); err != nil {
+ return valueInfo{}, false
+ }
+ if typ == "image" || typ == "document" {
+ *n += srcLen
+ }
+ if typ == "base64" && data.isStr {
+ return valueInfo{base64: len(data.str)}, true
+ }
+ return valueInfo{}, true
+}
diff --git a/internal/engine/mincacheable_test.go b/internal/engine/mincacheable_test.go
index 8dd117a..c1ddfbc 100644
--- a/internal/engine/mincacheable_test.go
+++ b/internal/engine/mincacheable_test.go
@@ -165,3 +165,22 @@ func TestARestoreInsideBStarWithNoCacheTokensIsInput(t *testing.T) {
t.Fatalf("Caused = %+v, want no read", c)
}
}
+
+// Media in the prefix prices at the media rate, not the text ratio (§5.1, rule 7). The image is
+// 250,000 bytes, 1,000 tokens; the client prefix up to the breakpoint at m0 is 1000 + wrapper + 1000 + 600
+// = 2,667 tokens, below 4096. At the text ratio the image alone would count 250,000 and pass.
+func TestAMediaPrefixBelowMinCacheableIsInput(t *testing.T) {
+ sysB := content([]blk{b("sys", 1000)})
+ img := imageBlock(250000)
+ client := NewBody(nil, sysB, [][]byte{img, content([]blk{b("m0", 600)})[0]})
+ fwd := NewBody(nil, sysB, [][]byte{img, content([]blk{b("m0", 300)})[0]})
+ cbps := bps5(sys(0), msg(1))
+ res := req{at: t0, client: client, clientBPs: cbps,
+ ops: []Operation{rewrite("a", "op", "c:1", msg(1), 600, 300)},
+ fwd: fwd, fwdBPs: cbps, facts: minFacts(4096),
+ usage: Usage{CacheRead: 1000, Input: wrapper + 1000 + 300}}.run(NewTrackers())
+ noChecks(t, res)
+ if s := findOp(t, res, "a").Saved; s.Input != 300 || s.CacheWrite5m != 0 {
+ t.Fatalf("Saved = %+v, want Input=300", s)
+ }
+}
diff --git a/internal/engine/persist.go b/internal/engine/persist.go
index 2f6b6c3..f3c52bf 100644
--- a/internal/engine/persist.go
+++ b/internal/engine/persist.go
@@ -39,12 +39,13 @@ type wireBody struct {
type wirePos struct {
Hash string `json:"hash"`
Bytes int32 `json:"bytes"`
+ Media int32 `json:"media,omitempty"`
}
func toWireBody(b Body) wireBody {
w := wireBody{Start: b.Start, Pos: make([]wirePos, len(b.Pos))}
for i, p := range b.Pos {
- w.Pos[i] = wirePos{Hash: strconv.FormatUint(p.Hash, 10), Bytes: p.Bytes}
+ w.Pos[i] = wirePos{Hash: strconv.FormatUint(p.Hash, 10), Bytes: p.Bytes, Media: p.Media}
}
return w
}
@@ -59,7 +60,7 @@ func fromWireBody(w wireBody) (Body, error) {
if err != nil {
return Body{}, err
}
- b.Pos[i] = PosHash{Hash: h, Bytes: p.Bytes}
+ b.Pos[i] = PosHash{Hash: h, Bytes: p.Bytes, Media: p.Media}
}
return b, nil
}
diff --git a/internal/engine/result.go b/internal/engine/result.go
index 7b91045..bdb62a9 100644
--- a/internal/engine/result.go
+++ b/internal/engine/result.go
@@ -27,7 +27,7 @@ type Unpriced struct {
// CheckFailure is a failed check (§6.5). A failure does not stop pricing.
type CheckFailure struct {
- Check string // "sum" | "order" | "unique_change" | "spans" | "attribution" | "conservation" | "shared_warm"
+ Check string // "sum" | "order" | "unique_change" | "spans" | "attribution" | "conservation" | "shared_warm" | "bad_usage" | "ratio_range" | "breakpoints" | "media_tokens" | "size_tokens"
Operator string // "" when the check is not about one operator
Detail string // numbers only, no content
}
diff --git a/internal/engine/size_tokens_test.go b/internal/engine/size_tokens_test.go
new file mode 100644
index 0000000..331ea7a
--- /dev/null
+++ b/internal/engine/size_tokens_test.go
@@ -0,0 +1,368 @@
+package engine
+
+import (
+ "strings"
+ "testing"
+)
+
+// Issue 11: one bytes-per-token ratio misprices images and dense JSON.
+
+// imageBlock is a message that holds a base64 image of n payload bytes.
+func imageBlock(n int) []byte {
+ return []byte(`{"type":"image","source":{"type":"base64","media_type":"image/png","data":"` +
+ strings.Repeat("A", n) + `"}}`)
+}
+
+// wrapper is the JSON text around the payload of imageBlock: text, 4 bytes a token in these tests.
+var wrapper = len(imageBlock(0))
+
+func text(id string, n int) []byte { return []byte(id + strings.Repeat(".", n-len(id))) }
+
+// priceOne runs one request. The client body has the operation's range at its "before" size, and
+// the forwarded body has it at its "after" size, so the sum and conservation checks hold.
+func priceOne(t *testing.T, client, fwd [][]byte, op Operation, usage Usage) (Result, PricedOp) {
+ t.Helper()
+ ts := NewTrackers()
+ r := ts.BeginRequest(RequestMeta{RequestID: "r", Session: "s", At: t0}, NewBody(nil, nil, client), nil)
+ r.Add(op)
+ r.EndRequest(Thread{ID: "main"}, NewBody(nil, nil, fwd), nil, exact5m)
+ res := r.Compute(usage)
+ return res, findOp(t, res, "o1")
+}
+
+func TestNewBodyCountsBase64MediaApart(t *testing.T) {
+ body := NewBody(nil, nil, [][]byte{imageBlock(400000), []byte("plain text")})
+ if got := body.Pos[0].Media; got != 400000 {
+ t.Fatalf("Media = %d, want 400000", got)
+ }
+ if body.Pos[1].Media != 0 {
+ t.Fatalf("text position Media = %d, want 0", body.Pos[1].Media)
+ }
+}
+
+func TestMediaFormats(t *testing.T) {
+ cases := []struct {
+ name string
+ in string
+ want int
+ }{
+ {"compact source block", `{"type":"image","source":{"type":"base64","media_type":"image/png","data":"AAAA"}}`, 4},
+ {"pretty printed", "{\n\t\"type\": \"document\",\n\t\"source\": {\n\t\t\"type\": \"base64\",\n\t\t\"data\":\n\t\t\t\"AAAAAA\"\n\t}\n}", 6},
+ {"source keys in other order", `{"source":{"data":"AAA","type":"base64"},"type":"image"}`, 3},
+ {"block in a list", `[{"type":"text","text":"x"},{"type":"image","source":{"type":"base64","data":"AA"}}]`, 2},
+ {"openai data url", `{"type":"image_url","image_url":{"url":"data:image/png;base64,AAAAA"}}`, 5},
+ {"openai input_file", `{"type":"input_file","file_data":"data:application/pdf;base64,AAAAAA"}`, 6},
+ {"html data url is text", `{"type":"tool_result","url":"data:text/html;base64,AAAAAAAA"}`, 0},
+ {"plain url", `{"type":"image_url","image_url":{"url":"https://x/y.png"}}`, 0},
+ {"tool result that quotes base64 and data", `{"encoding":"base64","data":"AAAAAAAAAA"}`, 0},
+ {"schema enum", `{"enum":["base64","hex"],"properties":{"data":{"type":"string"}}}`, 0},
+ {"text block that mentions both", `{"type":"text","text":"use \"base64\" then \"data\": \"AAAA\""}`, 0},
+ {"url source", `{"type":"image","source":{"type":"url","data":"AAAA"}}`, 0},
+ {"base64 source in a text block", `{"type":"text","source":{"type":"base64","data":"AAAA"}}`, 0},
+ {"data outside an image block", `{"type":"tool_result","content":{"type":"base64","data":"AAAA"}}`, 0},
+ }
+ for _, c := range cases {
+ if got := mediaBytes([]byte(c.in)); got != c.want {
+ t.Errorf("%s: mediaBytes = %d, want %d", c.name, got, c.want)
+ }
+ }
+}
+
+// A 400,000-byte image (1600 tokens) and 3 text messages of 4000 bytes (1000 tokens each).
+// A cut of 4000 text bytes saves 1000 tokens. Before the fix the library booked 54.
+func TestImageDoesNotSkewTheTextRatio(t *testing.T) {
+ img := imageBlock(400000)
+ client := [][]byte{img, text("a", 4000), text("b", 4000), text("c", 4000)}
+ fwd := [][]byte{img, text("a", 4000), text("b", 4000), nil}
+ op := Operation{ID: "o1", Kind: KindRewrite, Operator: "mask", ChangeID: "mk", At: msg(3),
+ SizeBefore: Size{Bytes: 4000}}
+ billed := 1600 + (2*4000+wrapper)/4
+ res, p := priceOne(t, client, fwd, op, Usage{Input: billed})
+ if got := tierSum(p.Saved); got < 998 || got > 1000 { // the wrapper text rounds
+ t.Fatalf("saved = %d tokens, want about 1000", got)
+ }
+ if !p.Estimated {
+ t.Fatal("a bytes conversion must stay Estimated")
+ }
+ noChecks(t, res)
+}
+
+// 10,000 bytes of dense JSON are 5000 tokens. The request ratio would book 3500.
+func TestHostTokensAreUsedAsTheyAre(t *testing.T) {
+ client := [][]byte{text("j", 10000), text("p", 40000)}
+ fwd := [][]byte{nil, text("p", 40000)}
+ op := Operation{ID: "o1", Kind: KindRewrite, Operator: "mask", ChangeID: "mk", At: msg(0),
+ SizeBefore: Size{Bytes: 10000, Tokens: 5000}}
+ res, p := priceOne(t, client, fwd, op, Usage{Input: 14000})
+ noChecks(t, res)
+ if got := tierSum(p.Saved); got != 5000 {
+ t.Fatalf("saved = %d, want the host's 5000", got)
+ }
+ if p.Estimated {
+ t.Fatal("host-counted tokens must not be Estimated")
+ }
+}
+
+// Check 6 must pass on a correct report that uses Size.Tokens. Request 1 has an unknown phase and
+// skips the check, so the gap shows only on request 2.
+func TestConservationHoldsWithHostTokensOverTwoRequests(t *testing.T) {
+ dense, prose, extra := text("j", 10000), text("p", 40000), text("m", 400)
+ ts := NewTrackers()
+ r1 := ts.BeginRequest(RequestMeta{RequestID: "r1", Session: "s", At: t0}, NewBody(nil, nil, [][]byte{dense, prose}), nil)
+ r1.EndRequest(Thread{ID: "main"}, NewBody(nil, nil, [][]byte{dense, prose}), nil, exact5m)
+ noChecks(t, r1.Compute(Usage{Input: 14000}))
+
+ small := text("j", 100)
+ r2 := ts.BeginRequest(RequestMeta{RequestID: "r2", Session: "s", At: t0.Add(1e9)},
+ NewBody(nil, nil, [][]byte{dense, prose, extra}), nil)
+ r2.Add(Operation{ID: "o1", Kind: KindRewrite, Operator: "mask", ChangeID: "mk", At: msg(0),
+ SizeBefore: Size{Bytes: 10000, Tokens: 5000}, SizeAfter: Size{Bytes: 100, Tokens: 50}})
+ r2.EndRequest(Thread{ID: "main"}, NewBody(nil, nil, [][]byte{small, prose, extra}), nil, exact5m)
+ res := r2.Compute(Usage{Input: 9500})
+ noChecks(t, res)
+ if p := findOp(t, res, "o1"); tierSum(p.Saved) != 4950 || p.Estimated {
+ t.Fatalf("saved = %d estimated = %v, want the host's 4950, exact", tierSum(p.Saved), p.Estimated)
+ }
+}
+
+func TestMediaBytesOnASizeUseTheFixedMediaPrice(t *testing.T) {
+ img := imageBlock(400000)
+ client := [][]byte{img, text("t", 4000)}
+ fwd := [][]byte{nil, text("t", 4000)}
+ op := Operation{ID: "o1", Kind: KindRewrite, Operator: "mask", ChangeID: "mk", At: msg(0),
+ SizeBefore: Size{Bytes: len(img), MediaBytes: 400000}}
+ res, p := priceOne(t, client, fwd, op, Usage{Input: 1000})
+ want := 1600 + wrapper/4 // the media at the fixed price, the JSON wrapper at the text ratio
+ if got := tierSum(p.Saved); got < want-2 || got > want+2 {
+ t.Fatalf("saved = %d, want about %d", got, want)
+ }
+ noChecks(t, res)
+}
+
+func TestSizeWithMoreMediaThanBytesIsRefused(t *testing.T) {
+ // The reason name is negative_size; it covers any malformed Size. Left as is.
+ body := [][]byte{[]byte("abc")}
+ op := Operation{ID: "o1", Kind: KindRewrite, Operator: "mask", ChangeID: "mk", At: msg(0),
+ SizeBefore: Size{Bytes: 10, MediaBytes: 11}}
+ res, _ := priceOne(t, body, body, op, Usage{Input: 3})
+ if !hasUnpriced(res, ReasonNegativeSize) {
+ t.Fatal("want negative_size")
+ }
+}
+
+func TestHostBuiltMediaAboveBytesIsCapped(t *testing.T) {
+ b := Body{Start: [3]int{0, 0, 0}, Pos: []PosHash{{Hash: 1, Bytes: 100, Media: 5000}}}
+ if got := newRatio(50, b); got.bytes < 0 {
+ t.Fatalf("ratio bytes = %d, must not be negative", got.bytes)
+ }
+}
+
+func TestBilledBelowMediaPriceIsAFailedCheck(t *testing.T) {
+ img := imageBlock(400000) // 1600 media tokens
+ body := [][]byte{img, text("t", 40)}
+ op := Operation{ID: "o1", Kind: KindRewrite, Operator: "mask", ChangeID: "mk", At: msg(1),
+ SizeBefore: Size{Bytes: 40}, SizeAfter: Size{Bytes: 40}}
+ res, _ := priceOne(t, body, body, op, Usage{Input: 100})
+ for _, c := range res.Checks {
+ if c.Check == "media_tokens" {
+ return
+ }
+ }
+ t.Fatalf("Checks = %+v, want media_tokens", res.Checks)
+}
+
+// An unmarked cut of an image message: the body holds the media, so the cut is priced at the media
+// price. Before the fix the whole range was priced at the text ratio (100,016 tokens).
+func TestUnmarkedCutOfAnImageMessageUsesTheBodysMedia(t *testing.T) {
+ img := imageBlock(400000)
+ client := [][]byte{img, text("t", 4000)}
+ fwd := [][]byte{nil, text("t", 4000)}
+ op := Operation{ID: "o1", Kind: KindRewrite, Operator: "mask", ChangeID: "mk", At: msg(0),
+ SizeBefore: Size{Bytes: len(img)}}
+ res, p := priceOne(t, client, fwd, op, Usage{Input: 1000})
+ want := 1600 + wrapper/4
+ if got := tierSum(p.Saved); got < want-2 || got > want+2 {
+ t.Fatalf("saved = %d, want about %d", got, want)
+ }
+ noChecks(t, res)
+}
+
+// A host count far outside the bytes is a host bug: it is not used, and a check names it.
+func TestImplausibleHostTokensFailACheckAndUseBytes(t *testing.T) {
+ client := [][]byte{text("j", 10000), text("p", 40000)}
+ fwd := [][]byte{nil, text("p", 40000)}
+ op := Operation{ID: "o1", Kind: KindRewrite, Operator: "mask", ChangeID: "mk", At: msg(0),
+ SizeBefore: Size{Bytes: 10000, Tokens: 500000}}
+ res, p := priceOne(t, client, fwd, op, Usage{Input: 12500})
+ found := false
+ for _, c := range res.Checks {
+ found = found || c.Check == "size_tokens"
+ }
+ if !found {
+ t.Fatalf("Checks = %+v, want size_tokens", res.Checks)
+ }
+ if got := tierSum(p.Saved); got != 3125 || !p.Estimated {
+ t.Fatalf("saved = %d estimated = %v, want 3125 from the bytes, estimated", got, p.Estimated)
+ }
+}
+
+func TestNegativeHostMediaIsClamped(t *testing.T) {
+ b := Body{Pos: []PosHash{{Hash: 1, Bytes: 100, Media: -5000}}}
+ if got := b.cumMedia(0); got != 0 {
+ t.Fatalf("cumMedia = %d, want 0", got)
+ }
+}
+
+func TestPersistKeepsMedia(t *testing.T) {
+ in := NewBody(nil, nil, [][]byte{imageBlock(1000), []byte("x")})
+ out, err := fromWireBody(toWireBody(in))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if out.Pos[0].Media != 1000 || out.Pos[1].Media != 0 {
+ t.Fatalf("Media after a round trip = %d, %d, want 1000, 0", out.Pos[0].Media, out.Pos[1].Media)
+ }
+}
+
+// mixed is one message with a text part of n bytes and a 400,000-byte image.
+func mixed(n int) []byte {
+ return []byte(`[{"type":"text","text":"` + strings.Repeat("t", n) + `"},` + string(imageBlock(400000)) + `]`)
+}
+
+// An unmarked after-size that adds an image: the forwarded body holds the media. Before the fix
+// the 400,067 bytes priced at the text ratio (99,942 tokens).
+func TestUnmarkedAfterSizeThatAddsAnImageUsesTheForwardedMedia(t *testing.T) {
+ img := imageBlock(400000)
+ client := [][]byte{text("t", 4000)}
+ fwd := [][]byte{img, text("t", 4000)}
+ op := Operation{ID: "o1", Kind: KindRewrite, Operator: "inject", ChangeID: "in", At: msg(0),
+ SizeAfter: Size{Bytes: len(img)},
+ MessageSplice: &MessageSplice{At: 0, Removed: 0, Inserted: 1}}
+ res, p := priceOne(t, client, fwd, op, Usage{Input: 2616})
+ if got := tierSum(p.Caused); got < 1614 || got > 1620 {
+ t.Fatalf("caused = %d, want about 1618", got)
+ }
+ noChecks(t, res)
+}
+
+// A rewrite that keeps the image and trims the text around it saves the text only.
+func TestUnmarkedAfterSizeThatKeepsTheImageSavesTheTextOnly(t *testing.T) {
+ before, after := mixed(1200), mixed(200)
+ op := Operation{ID: "o1", Kind: KindRewrite, Operator: "trim", ChangeID: "tr", At: msg(0),
+ SizeBefore: Size{Bytes: len(before)}, SizeAfter: Size{Bytes: len(after)}}
+ billed := 1600 + (len(after)-400000)/4
+ res, p := priceOne(t, [][]byte{before}, [][]byte{after}, op, Usage{Input: billed})
+ if got := tierSum(p.Saved); got < 248 || got > 252 || tierSum(p.Caused) != 0 {
+ t.Fatalf("saved = %d caused = %d, want about 250 and 0", got, tierSum(p.Caused))
+ }
+ noChecks(t, res)
+}
+
+// A correct host count on an after-size over media must pass the size_tokens check.
+func TestHostTokensOnAnAfterSizeOverMediaAreNotRejected(t *testing.T) {
+ before, after := mixed(1200), mixed(200)
+ op := Operation{ID: "o1", Kind: KindRewrite, Operator: "trim", ChangeID: "tr", At: msg(0),
+ SizeBefore: Size{Bytes: len(before), Tokens: 1600 + 300},
+ SizeAfter: Size{Bytes: len(after), Tokens: 1600 + 50}}
+ billed := 1600 + (len(after)-400000)/4
+ res, p := priceOne(t, [][]byte{before}, [][]byte{after}, op, Usage{Input: billed})
+ for _, c := range res.Checks {
+ if c.Check == "size_tokens" {
+ t.Fatalf("size_tokens failed: %+v", c)
+ }
+ }
+ if got := tierSum(p.Saved); got != 250 || p.Estimated {
+ t.Fatalf("saved = %d estimated = %v, want the host's 250, exact", got, p.Estimated)
+ }
+}
+
+// run1 runs one request and returns the result, without looking for a priced operation.
+func run1(client, fwd [][]byte, op Operation, usage Usage) Result {
+ ts := NewTrackers()
+ r := ts.BeginRequest(RequestMeta{RequestID: "r", Session: "s", At: t0}, NewBody(nil, nil, client), nil)
+ r.Add(op)
+ r.EndRequest(Thread{ID: "main"}, NewBody(nil, nil, fwd), nil, exact5m)
+ return r.Compute(usage)
+}
+
+func checkNames(res Result) []string {
+ var out []string
+ for _, c := range res.Checks {
+ out = append(out, c.Check)
+ }
+ return out
+}
+
+func noopOp() Operation {
+ return Operation{ID: "o1", Kind: KindRewrite, Operator: "mask", ChangeID: "mk", At: msg(1),
+ SizeBefore: Size{Bytes: 40}, SizeAfter: Size{Bytes: 40}}
+}
+
+// A bill below the media price fails media_tokens and ratio_range, and prices nothing.
+func TestBillBelowTheMediaPricePricesNothing(t *testing.T) {
+ body := [][]byte{imageBlock(400000), text("t", 40)} // 1600 media tokens
+ res := run1(body, body, noopOp(), Usage{Input: 100})
+ if !hasUnpriced(res, ReasonRatioRange) {
+ t.Fatalf("Unpriced = %+v, want ratio_range", res.Unpriced)
+ }
+ got := checkNames(res)
+ has := func(n string) bool {
+ for _, g := range got {
+ if g == n {
+ return true
+ }
+ }
+ return false
+ }
+ if !has("media_tokens") || !has("ratio_range") {
+ t.Fatalf("Checks = %v, want media_tokens and ratio_range", got)
+ }
+}
+
+// A bill equal to the media price leaves no text token: the text ratio is out of range (a body with
+// text bytes and no text token), but media_tokens does not fail.
+func TestBillEqualToTheMediaPriceDoesNotFailMediaTokens(t *testing.T) {
+ body := [][]byte{imageBlock(400000), text("t", 40)}
+ media := 400000 / mediaBytesPerToken
+ res := run1(body, body, noopOp(), Usage{Input: media})
+ for _, n := range checkNames(res) {
+ if n == "media_tokens" {
+ t.Fatalf("media_tokens failed at a bill equal to the media price: %v", checkNames(res))
+ }
+ }
+ // With 4 text tokens for 40 text bytes + wrapper the request prices.
+ res = run1(body, body, noopOp(), Usage{Input: media + (40+wrapper)/4})
+ noChecks(t, res)
+}
+
+// A media-heavy request is priced at the text ratio (0.25), not refused by the raw formula.
+func TestMediaHeavyRequestIsPriced(t *testing.T) {
+ img := imageBlock(400000)
+ client := [][]byte{text("a", 40000), img}
+ fwd := [][]byte{text("a", 30000), img}
+ op := Operation{ID: "o1", Kind: KindRewrite, Operator: "mask", ChangeID: "mk", At: msg(0),
+ SizeBefore: Size{Bytes: 40000}, SizeAfter: Size{Bytes: 30000}}
+ billed := 1600 + (30000+wrapper)/4
+ res := run1(client, fwd, op, Usage{Input: billed})
+ noChecks(t, res)
+ if hasUnpriced(res, ReasonRatioRange) {
+ t.Fatalf("Unpriced = %+v, the request must be priced", res.Unpriced)
+ }
+ if p := findOp(t, res, "o1"); tierSum(p.Saved) != 2500 {
+ t.Fatalf("saved = %d, want 2500", tierSum(p.Saved))
+ }
+}
+
+// A body whose positions are all media is not an empty body: it prices when the bill equals the
+// media.
+func TestAllMediaBodyIsNotRefused(t *testing.T) {
+ b := Body{Pos: []PosHash{{Hash: 1, Bytes: 2500, Media: 2500}}}
+ c := &calcState{fwd: b, usage: Usage{Input: 10}, ratio: newRatio(10, b)}
+ if c.badRatio() {
+ t.Fatal("badRatio refused a body of media only that the bill covers")
+ }
+ c = &calcState{fwd: b, usage: Usage{Input: 9}, ratio: newRatio(9, b)}
+ if !c.badRatio() {
+ t.Fatal("badRatio accepted a bill below the media price")
+ }
+}
diff --git a/internal/engine/types.go b/internal/engine/types.go
index 7eefee5..68932fc 100644
--- a/internal/engine/types.go
+++ b/internal/engine/types.go
@@ -120,10 +120,23 @@ type SettingChange struct {
At *Position `json:"at,omitempty"`
}
-// Size is bytes only. A context operator does not count tokens: the library converts bytes to
-// tokens (§4.2 rule 7).
+// Size is the bytes of a range, and optionally its tokens. Bytes are required. When the host knows
+// the tokens (a tokenizer, or a count_tokens call), it sets Tokens, and the library uses them as
+// they are. Without Tokens the library converts the bytes (§4.2 rule 7).
type Size struct {
Bytes int `json:"bytes,omitempty"` // required on rewrite; the sum check compares it with the Body sizes (§6.5)
+ // Tokens is the token count the host measured for the range. 0 means not measured: the library
+ // then converts Bytes and marks the result Estimated.
+ Tokens int `json:"tokens,omitempty"`
+ // MediaBytes is the part of Bytes that is base64 media (an image or a document block). The
+ // library prices it apart from the text ratio. 0 means the range has no media, or the host did
+ // not mark it.
+ MediaBytes int `json:"mediaBytes,omitempty"`
+}
+
+// invalid says whether a size cannot be real: a negative figure, or more media than bytes.
+func (s Size) invalid() bool {
+ return s.Bytes < 0 || s.Tokens < 0 || s.MediaBytes < 0 || s.MediaBytes > s.Bytes
}
// MessageSplice describes a change to the message list.