Cost basis labels: published-rate usage value, subscription value and metered usage kept apart - #5975
Conversation
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
Visual diffComparing 44 of 70 comparison(s) flagged (>1% pixel diff).
Folder: c5c84c81a8e0. Full PNGs also attached as a workflow artefact. Generated by visual-diff bot. Pixel diffs >1% flagged; eyeball the table before merging. This check is non-blocking — fail = bot bug, not a code problem. |
Coordinator reviewVerdict: fix before merge. One blocking item. Most of the PR holds up: the vocabulary, the evidence rule, the removed zero-bill copy, the focus tip, and the tightened ratchets. Reviewed head Blocking1. AC-OBS-CEA-025.1 is marked met, but two of its named surfaces still show cost figures with no visible basis. The criterion says "Every cost figure on ... the per-session cost breakdown, the Flow brain panel ... shown as visible text beside the figure." Its evidence is API-level only (
AC-025.1 also says every cost figure on the Usage tab is labelled. The PR itself notes that 62 renders in Fix, either way:
A criterion marked met while its text is false on a live screen is a hidden gap. Non-blocking (list under Remaining or follow up)
Gates checked
🤖 Generated with Claude Code |
… metered usage kept apart Every cost figure now carries a financial basis beside its provenance basis (usage value at published rates / expected contract spend / allocated actual spend / not available). Contract and actual labels are refused without the rate version or ledger reference they rest on. The Usage coverage banner, cards and info text, the Overview hero and the inventory chip no longer describe subscription-covered usage as a $0 bill: it is shown as value, apart from metered usage, with an undetected route labelled undetected and the plan fee reported as not visible. The Sessions cost chips and the Flow brain panel cost render through the shared badge, which is keyboard-focusable and shows its explanation on focus. Refs #5937 (REQ-OBS-CEA-025). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jm9d7s4fN55hN3YzQo75o9
…hip edits A CSS ::after tip on the badge was clipped to one line by the Overview tile's overflow:hidden, and hiding it on scroll hid it the moment Tab navigation scrolled the badge into view. provenance.js now shows one fixed tip on <body> on :focus-visible and follows the badge on scroll. loadSessions() writes to #sessions-list, which no template renders, so its chip edits could not be verified and are reverted. The live Sessions-tab transcript chips are listed as not in this change. Refs #5937 (REQ-OBS-CEA-025). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jm9d7s4fN55hN3YzQo75o9
Review of #5975 found AC-OBS-CEA-025.1 claimed on screens that still printed unlabelled costs: - loadUsage kept only cbd.top10 and dropped the cost breakdown's provenance, so the session cost chart and its table printed bare dollar figures. The entry now travels with the rows: a caption above the canvas bars and a badge on the Cost column, with an unknown cost reading "not available" instead of $0.0000. - The Flow brain panel's per-call list printed a pre-formatted string. Each call now carries cost_usd (null when it had tokens but no price) under one calls[].cost_usd entry, rendered as "Cost per call: published rates". The colour thresholds are checked largest first, so red is reachable. AC-OBS-CEA-025.1 is narrowed, in the Factory requirement and the mirror, to the figures that are actually labelled. The other Usage cost cards and the Overview hero chip are named as a non-goal instead of being claimed. Three new tests run the shipped renderers under node and the payload builder in Python; all three fail on the previous head. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jm9d7s4fN55hN3YzQo75o9
5fa9794 to
79c7121
Compare
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
|
Review follow-up (head 79c7121). This addresses the blocking item on AC-OBS-CEA-025.1. Option (a), for the two screens you named:
Option (b), for the rest. AC-025.1 is narrowed in the Factory requirement (tracked edit) and in Guards. There are 3 new tests in Browser check. Scratch HOME seeded with today's OpenClaw transcript, random port, Chrome:
Non-blocking notes:
|
Regenerate the generated inventory so the gen_module_map --check lint guard passes on CI. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KUtV6jyUVMhBRWXjSSef9S
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
|
Ready to merge. Every blocking review item is addressed (see the review follow-up comment above). CI is green, drift-bot and the product-record gate pass, and GitHub reports the PR as mergeable and CLEAN. I have not merged it. Merge-after dependencies: none. The PR is branched off Companion PRs: none. No new HTTP route, so there is no Factory: the AC-OBS-CEA-025.1 narrowing and the new non-goal are pending tracked suggestions on https://factory.8090.ai/project/b415065f-ab2f-4f53-8864-0c009fd098cb/requirements/950d4687-45cc-45a8-9d58-0c0d82fd5d9b. Accept them so the requirement matches the mirror in Post-merge / post-release verification (after the
Remaining scope stays on #5937; see the Remaining section of the PR body. 🤖 Generated with Claude Code |
…s, effective dates A local price book (~/.clawmetry/pricing.json, schema clawmetry.price_book/1) with exact/prefix/pattern entries, effective_from/effective_to, rates or a discount off the published rate, and Azure deployment aliases scoped to their resource. POST /api/pricing/resolve says which entry applies to each usage record and why; GET /api/pricing/book shows the validated book, rejected entries and recorded content-addressed versions; POST /api/pricing/valuations defines the contract a future engine answers (501 until one exists). The interceptor now captures Azure OpenAI calls with their deployment and host. Integrated with the cost basis labels (#5975): every figure this surface returns carries the shared cost_basis vocabulary. A list figure is published_rate, no amount is unknown, and a contract valuation goes through cost_basis.label, so it cannot claim "contract rate" without its rate version. Rebased onto current main as one commit. acceptance_criteria.json carries the AC-OBS-CEA-024 block once (earlier merges had duplicated it and the AC-GOV-FWM block). The CHANGELOG entry is left to the release. Refs #5936. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jm9d7s4fN55hN3YzQo75o9
Attributes each session's spend to a project derived when read (operator assignment, else the recorded git repository containing its cwd, else the directory, else a visible Unassigned row), with per-project budgets on a declared period, timezone, currency and basis, 50/80/100% alerts latched once per budget period, and GET /api/usage/export?by=project. A budget on a repository keeps counting its sessions after that repository is assigned to a named project, and reports the named project under reported_under. The usage-bucket cache key snaps to a 15-minute edge so daemon ticks reuse it. Rebased on main as one commit. Integrated with the cost basis labels (#5975): /api/projects, /api/projects/budgets and /api/projects/budgets/alerts now stamp every dollar figure as usage value at published rates through clawmetry.cost_basis, the CSV carries a cost_basis column, and the alert text and notice say "at published rates, not an invoice" instead of "estimated spend". The CHANGELOG entry is left to the release. Refs #5941. REQ-OBS-PRJ-001. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jm9d7s4fN55hN3YzQo75o9
… CHANGELOG entry Builds on c1b7fed (which routed gatewayMoney through cmFigure with a hardcoded client-side entry). The label now comes from the server, in the one cost vocabulary (clawmetry/cost_basis.py): gateway_litellm stamps provenance on the gateway object with basis measured, cost_basis published_rate, and a rate_source naming LiteLLM (ClawMetry does not re-price it). A null cost renders the unknown state "not reported" via a cost_basis unknown entry, not a plain string. The column header carries the badge; the exact reported amount stays in the tooltip, so a fraction of a cent is not rounded away. The gateway object's top-level `cost_basis: "gateway_reported"` collided with that vocabulary and is renamed `cost_source` (the ledger row's name). CI script, test and docs/LITELLM.md updated. CHANGELOG.md restored to main's version (release notes are written at release time). Revert-proof: the new test assertions and the unbadged-render ratchet both fail against the previous gateway_litellm.py / app.js and pass with this. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jm9d7s4fN55hN3YzQo75o9
Publishes the six merged changes listed in CHANGELOG.md: Cost Optimizer honesty (#5951), cost basis labels (#5975), framework IDs on Guard findings (#5952), ATLAS replay scorecard (#5961), PR provenance SARIF (#5974), and the Copilot VS Code control refusal (#5954). Claude-Session: https://claude.ai/code/session_01Jm9d7s4fN55hN3YzQo75o9 Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…s, effective dates (#5959) * Price book contract: negotiated rates, Azure OpenAI deployment aliases, effective dates A local price book (~/.clawmetry/pricing.json, schema clawmetry.price_book/1) with exact/prefix/pattern entries, effective_from/effective_to, rates or a discount off the published rate, and Azure deployment aliases scoped to their resource. POST /api/pricing/resolve says which entry applies to each usage record and why; GET /api/pricing/book shows the validated book, rejected entries and recorded content-addressed versions; POST /api/pricing/valuations defines the contract a future engine answers (501 until one exists). The interceptor now captures Azure OpenAI calls with their deployment and host. Integrated with the cost basis labels (#5975): every figure this surface returns carries the shared cost_basis vocabulary. A list figure is published_rate, no amount is unknown, and a contract valuation goes through cost_basis.label, so it cannot claim "contract rate" without its rate version. Rebased onto current main as one commit. acceptance_criteria.json carries the AC-OBS-CEA-024 block once (earlier merges had duplicated it and the AC-GOV-FWM block). The CHANGELOG entry is left to the release. Refs #5936. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jm9d7s4fN55hN3YzQo75o9 * chore: regenerate MODULE_MAP.md (259 modules after price-book additions) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cF1pNfkt6jLiKc8qF9Zro --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Attributes each session's spend to a project derived when read (operator assignment, else the recorded git repository containing its cwd, else the directory, else a visible Unassigned row), with per-project budgets on a declared period, timezone, currency and basis, 50/80/100% alerts latched once per budget period, and GET /api/usage/export?by=project. A budget on a repository keeps counting its sessions after that repository is assigned to a named project, and reports the named project under reported_under. The usage-bucket cache key snaps to a 15-minute edge so daemon ticks reuse it. Rebased on main as one commit. Integrated with the cost basis labels (#5975): /api/projects, /api/projects/budgets and /api/projects/budgets/alerts now stamp every dollar figure as usage value at published rates through clawmetry.cost_basis, the CSV carries a cost_basis column, and the alert text and notice say "at published rates, not an invoice" instead of "estimated spend". The CHANGELOG entry is left to the release. Refs #5941. REQ-OBS-PRJ-001. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jm9d7s4fN55hN3YzQo75o9
… CHANGELOG entry Builds on c1b7fed (which routed gatewayMoney through cmFigure with a hardcoded client-side entry). The label now comes from the server, in the one cost vocabulary (clawmetry/cost_basis.py): gateway_litellm stamps provenance on the gateway object with basis measured, cost_basis published_rate, and a rate_source naming LiteLLM (ClawMetry does not re-price it). A null cost renders the unknown state "not reported" via a cost_basis unknown entry, not a plain string. The column header carries the badge; the exact reported amount stays in the tooltip, so a fraction of a cent is not rounded away. The gateway object's top-level `cost_basis: "gateway_reported"` collided with that vocabulary and is renamed `cost_source` (the ledger row's name). CI script, test and docs/LITELLM.md updated. CHANGELOG.md restored to main's version (release notes are written at release time). Revert-proof: the new test assertions and the unbadged-render ratchet both fail against the previous gateway_litellm.py / app.js and pass with this. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jm9d7s4fN55hN3YzQo75o9
Attributes each session's spend to a project derived when read (operator assignment, else the recorded git repository containing its cwd, else the directory, else a visible Unassigned row), with per-project budgets on a declared period, timezone, currency and basis, 50/80/100% alerts latched once per budget period, and GET /api/usage/export?by=project. A budget on a repository keeps counting its sessions after that repository is assigned to a named project, and reports the named project under reported_under. The usage-bucket cache key snaps to a 15-minute edge so daemon ticks reuse it. Rebased on main as one commit. Integrated with the cost basis labels (#5975): /api/projects, /api/projects/budgets and /api/projects/budgets/alerts now stamp every dollar figure as usage value at published rates through clawmetry.cost_basis, the CSV carries a cost_basis column, and the alert text and notice say "at published rates, not an invoice" instead of "estimated spend". The CHANGELOG entry is left to the release. Refs #5941. REQ-OBS-PRJ-001. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jm9d7s4fN55hN3YzQo75o9
…t) (#5968) * Project attribution and per-project budgets: burn against budget (#5941) Attributes each session's spend to a project derived when read (operator assignment, else the recorded git repository containing its cwd, else the directory, else a visible Unassigned row), with per-project budgets on a declared period, timezone, currency and basis, 50/80/100% alerts latched once per budget period, and GET /api/usage/export?by=project. A budget on a repository keeps counting its sessions after that repository is assigned to a named project, and reports the named project under reported_under. The usage-bucket cache key snaps to a 15-minute edge so daemon ticks reuse it. Rebased on main as one commit. Integrated with the cost basis labels (#5975): /api/projects, /api/projects/budgets and /api/projects/budgets/alerts now stamp every dollar figure as usage value at published rates through clawmetry.cost_basis, the CSV carries a cost_basis column, and the alert text and notice say "at published rates, not an invoice" instead of "estimated spend". The CHANGELOG entry is left to the release. Refs #5941. REQ-OBS-PRJ-001. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jm9d7s4fN55hN3YzQo75o9 * chore: regenerate MODULE_MAP.md (259→260 modules) for project attribution budgets branch Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HkHeWgNjveVVQbtiEqkENY --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Refs #5937
Factory requirement (REQ-OBS-CEA-025, written before the code): https://factory.8090.ai/project/b415065f-ab2f-4f53-8864-0c009fd098cb/requirements/950d4687-45cc-45a8-9d58-0c0d82fd5d9b
Child of Cost and Efficiency Analytics. Seven criteria are mirrored into
docs/acceptance_criteria.json, and each is cited by a test.Merge order: branched off
main, not stacked. #5959 (price book, #5936) is a conceptual dependency only. This PR defines the "contract rate" label and the evidence it needs (a rate version), and nothing produces that label yet. The two PRs share no files exceptCHANGELOG.md,docs/acceptance_criteria.jsonanddocs/ac_coverage_baseline.json, which are additive; on a conflict, keep both sides and regenerate the baseline. No new HTTP route, so nocloud_route_policyentry or cloud PR is needed.Review fixes (79c7121)
The review found AC-OBS-CEA-025.1 marked met while two live screens still printed unlabelled costs.
Usage tab session cost chart and table.
loadUsage()keptcbd.top10and dropped the provenance. Thetop10[].cost_usdentry now travels with the rows. The canvas bars get a caption ("Bar values and the Cost column: published rates"), and the Cost column heading gets the badge. An unknown cost reads "not available", not$0.0000.Flow brain panel per-call list. Each call now carries a numeric
cost_usd:nullwhen the call had tokens but no price, so it reads "not available". The list is labelled once, undercalls[].cost_usd, with a "Cost per call: published rates" heading. The colour thresholds are now checked largest first, so red is reachable.AC-OBS-CEA-025.1 narrowed in the Factory requirement (tracked edit) and in the mirror, to exactly the figures that carry a label:
The Usage tab's other cost cards and the Overview hero chip are now an explicit non-goal and are listed under Remaining. They are no longer claimed.
Why
Users ask whether a cost figure is an estimate or a bill, and the dashboard could not say.
What
Vocabulary.
clawmetry/cost_basis.py(new, short module) adds a financial basis that rides beside the existing provenance basis inside the same entry. Theprovenancewire shape is unchanged: consumers that ignore the new fields keep working.published_rate: usage value at published rates. This covers a runtime-reported cost too, whose source is named inrate_source.contract: expected contract spend. It needs a recordedrate_version.allocated_actual: allocated actual spend. It needs a recordedledger_ref.unknown.unknownentry with the reason.stamp()then nulls the figure, so an unevidenced "actual spend" cannot reach a screen as a number. Nothing producescontractorallocated_actualtoday.subscription,meteredorunknown.meteredonly when a metered runtime was actually detected, otherwise "not detected".plan_fee_usdisNone, labelled unknown, never0.0.Surfaces.
/api/usage(routes/usage.py)clawmetry/sync.py)dailyUsageand on both branches of the spending triple (live, and the stale state fallback)./api/sessions/cost-breakdown(routes/sessions.py)estimated, with a count.app.jsloadUsage/renderSessionCostChart)routes/components.py)today_cost_usdwith provenance, next to the legacy string. Calls that carried tokens but no price make the total a labelled floor. If nothing could be priced it is "not available", not $0.00. The per-call list carriescost_usdunder acalls[].cost_usdentry, rendered as "Cost per call: published rates".clawmetry/static/js/provenance.js)aria-labelcarries the explanation. On:focus-visibleone fixed tip on<body>shows the same text a mouse user gets from the title tooltip.clawmetry/static/js/app.js)dashboard.cssVerification
Regression guard, red before green.
tests/test_cost_basis_labels.pyhas 18 tests.renderBillingCoverageBanner,_planLabel,renderSessionCostChart,loadBrainDataandcmProv.badgeout ofapp.jsandprovenance.jsunder node.origin/mainat658342f238with only the new vocabulary module added, 12 of the 15 fail. The 3 that pass test the vocabulary alone.b2b438f5e6, checked in a separate detached worktree:test_the_usage_session_cost_chart_and_table_show_their_basistest_the_flow_brain_call_list_cost_is_labelledtest_the_flow_brain_call_list_renders_its_basistest_provenance.pyandtest_provenance_render_coverage.py.A guard that never ran now runs.
tests/test_provenance.pyandtests/test_provenance_render_coverage.pywere named in no workflow, andmainis at 66 unbadged renders against their ceiling of 63. This PR:UNBADGED_CEILINGto 62, which is downward only.docs/ci_test_coverage_baseline.jsontightened (unlisted 920 to 917).Other gates:
check_ac_coverage.py --checkOK at 124/194 on the rebased branch.gen_module_map.py --checkOK.check_py39_annotations.pyOK.node --checkpasses on both JS files.Real dashboard in a browser, first pass. I ran this branch's
dashboard.pywith a scratch HOME (a Claude Max OAuth marker, so a subscription is detected), a scratch seeded DuckDB and a random port, and drove it with Chrome DevTools.$0.59with apublished ratesbadge.Real dashboard in a browser, review-fix pass. This time the scratch HOME was seeded with today's OpenClaw transcript: one priced call ($0.0042) and one call on a model the runtime did not price. I ran
dashboard.pyon a random port and checked it in Chrome.sess-live-1 2K $0.0042.$0.0042and$0.0006(the second is ClawMetry's price-table fallback). The badge hastabindex=0, and its tip gives the formula, the window ("one assistant call"), the rate source and the source./api/component/brainreturnscalls[].cost_usdwithcost_basis=published_rate, and/api/sessions/cost-breakdownreturnstop10[].cost_usdlabelled "published rates".Remaining (not in this PR; #5937 stays open)
Other Usage tab cost cards. Each is served by its own endpoint, none of which carries a financial basis yet, so none shows a basis:
These are a non-goal in REQ-OBS-CEA-025, and the render ratchet keeps their count from growing.
Overview hero chip. It shows a cost with no basis and an unclear window, and it now carries the "included in your plan, not an extra bill" suffix. Its figure also differed from the tile's today figure in the first browser pass ($12.58 vs $0.59). That predates this PR and was not investigated.
Sessions tab transcript chips. The per-turn and per-tool cost chips come from
_buildReplayEventover several local and hosted transcript sources, so labelling them needs a transcript payload change on both sides. The Sessions-list chips inloadSessions()write to#sessions-list, which no template renders, so they were left untouched rather than claimed.Contract spend and actual spend. Needs the price book (Price book contract: negotiated rates, Azure OpenAI deployment aliases, effective dates #5959 / Pricing: custom price book for negotiated rates + Azure OpenAI deployment aliases #5936) plus a valuation engine, and an invoice or ledger ingest. The labels and their evidence rule exist; no figure uses them.
Hand-check against every source, and a cache and window-boundary audit. The refinement asks for cache tokens not counted twice, and for late usage kept consistent across local, snapshot and hosted views. Not done here.
Budgets (Attribution: project + user tags and per-project budgets (burn vs budget) #5941) and the Cost Optimizer (Cost Optimizer: 33s spinner on cold load, debug label and unlabelled figures #5934) choosing a basis. They render through the same formatter and inherit the labels; their thresholds are unchanged.
Hosted dashboard. It renders the OSS
app.js, so the corrected copy and the snapshot's labels reach it with the next pin. The hosted Usage interceptor'sbillingCoveragecarries no provenance, so the split figures render without a badge there. The same applies to hosted brain and cost-breakdown payloads that lack the new entries: they keep their legacy strings and invent no label. A cloud-side pass-through is a follow-up.Badge tab stops (review note, not addressed). Every provenance badge, compact ones included, now takes
tabindex=0, which can add many tab stops on dense tables.Other unbadged money renders. 62 remain in
app.js, held by the ratchet.🤖 Generated with Claude Code
https://claude.ai/code/session_01Jm9d7s4fN55hN3YzQo75o9