test(conformance): cross-port drift check (price-tag, negotiable-floor, discover #41) - #10
Merged
Merged
Conversation
…floor, discover #41) Introduces a CI-enforced port-drift conformance suite shared across the three Agent SDK ports. Golden vectors (input -> expected output) live in language- neutral JSON under conformance/vectors/; tests/test_conformance.py runs them through this port's own AgentCapability.from_nostr_event and AgentManager.discover. Covers three security-critical shared behaviors: - price-tag amount parsing (valid / malformed-reject / no-price / negative) - negotiable-floor parsing (valid / malformed-reject) - discover() batch resilience to one malformed relay payload (ledger #41): bad-price, missing-committed-field, non-dict-payload A shared CHECKSUMS file (identical in all three repos) pins the vectors so the copies cannot silently drift; the test verifies it over LF-normalized bytes. This port already conforms to every golden value, so no production code changed. See conformance/README.md for the design and sync mechanism. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CYXrdWWjKo5AyGqFAfrdgK
A capability price-tag amount and negotiable-floor amount must be a non-negative integer. A negative amount was previously accepted by all three ports and flagged as an open designQuestion in the conformance vectors; that question is now decided (#69): a negative price/floor is never meaningful and accepting it is a fail-open smell, so it is rejected on the same path as any other malformed amount (throw -> the event is skipped by discover()). Zero stays valid (a free service). - Add shared _parse_sats_amount() helper used by BOTH the price parse (AgentPricing.from_tag) and the floor parse (from_nostr_event) so they cannot drift; it raises the same ValueError as int() on malformed input. - Conformance vectors: flip negative-amount-accepted -> reject (negative-amount-rejected), add negative-floor-rejected, and pin zero-accepted / zero-floor-accepted; remove the designQuestion flag; regenerate CHECKSUMS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Port-drift conformance check (first version)
Turns the audit's meta-finding — where the three Agent SDK ports of a shared behavior disagree, at least two are wrong — into a CI-enforced check. Shared golden vectors (input → expected output) live in language-neutral JSON under
conformance/vectors/; each port ships a conformance test that runs the same vectors through its own implementation, so any port that drifts fails its own CI.This is the Python copy. Identical vectors +
CHECKSUMSship in the .NET and TypeScript repos (separate PRs).What's covered
["price","100"]→100 (defaults), unit/model, malformed (abc/10.5/100abc)→reject,["price"]→no-price,-5→accepted.[valid, malformed, valid]→ 2 survivors for bad-price, missing-committed-field, non-dict-payload; skip must be loud.Discovery result for this port
Python already conforms to every golden value (price and floor both reject malformed amounts via
int(); discover already handles all three malformed payloads). No production code changed here — this PR is tests + vectors only.Cross-port findings surfaced by the suite
NaN/truncated viaparseInt. Golden = reject; the TypeScript port is fixed in its own PR.-5). Encoded as the current agreement + flaggeddesignQuestioninprice-tag.jsonrather than silently changed on one side.Sync mechanism
conformance/CHECKSUMSis byte-identical across all three repos; each port's test recomputes SHA-256 (LF-normalized) of its local vectors and asserts they match, so the copies cannot drift silently. Seeconformance/README.md.Tests
Full suite: 191 → 208 (+17), no regressions.
Do not merge until the .NET and TypeScript PRs are reviewed together.