Fix: derive the spec semantic header for synthetic dispute envelopes - #26
Conversation
apply_consensus enforces the OPP MessageHeader spec and the per-outpost message chain since SEC-102, so the slashing scenario's hand-built envelopes need a real header: encodeTaggedEnvelope now derives payload_size/payload_checksum over the canonical payload bytes, header_checksum over the blanked canonical header, and message_id as the sequence-spliced checksum, using a field-complete canonical encoder that mirrors the depot's opp_canonical_codec.hpp (keccak via ethers). The delivered wire bytes stay protobuf-ts output — the depot re-canonicalizes on receipt. Each envelope chains from the outpost's current inbound message tip, read from sysio.msgch::outpcons via runDeliver. The tip is cached per (chain_code, epoch) so all deliveries an outpost receives for one contested epoch share the same pre-delivery tip: once the non-contested outpost's 2/3 majority advances the tip inline, a re-read would make the third delivery diverge and be wrongly slashed. The winning contested envelope and the consensus envelope both continue the tip and pass validation; message_tip is read off the runtime row since it is new in the deployed ABI but absent from the pinned SystemContractTypes.
| epochIndex, | ||
| epochEnvelopeIndex: Constants.EnvelopeEpochEnvelopeIndex, | ||
| epochTimestamp: Constants.EnvelopeEpochTimestampMs, | ||
| epochTimestamp: timestamp, |
There was a problem hiding this comment.
[P1] Envelope.create still leaves previousEnvelopeHash empty. Current sysio.msgch::apply_consensus compares this field with outpcons.envelope_digest once bootstrap has established a tip, so these synthetic envelopes are dropped before the new semantic-header validation. Please read/cache envelope_digest alongside message_tip and pass it as previousEnvelopeHash.
There was a problem hiding this comment.
Fixed in 0e74f1a. Confirmed against apply_consensus (Wire-Network/wire-sysio#508): it checks previous_envelope_hash against outpcons.envelope_digest before semantic_headers_ok, so an empty prev-hash is dropped once bootstrap sets a tip. readInboundTips now reads envelope_digest alongside message_tip, and runDeliver passes it as previousEnvelopeHash (previous_message_id still chains from message_tip). The codec test decodes encodeTaggedEnvelope's output and asserts both prev fields are wired.
| const cached = inboundMessageTipCache.get(cacheKey) | ||
| if (cached !== undefined) return cached | ||
|
|
||
| const { rows } = await ctx.wire |
There was a problem hiding this comment.
[P2] This cache is populated only after the awaited RPC read. The three delivery phases execute concurrently, so all callers can miss and start reads; a late read may observe the post-consensus tip and produce a distinct third envelope, causing incorrect slash classification. Cache an in-flight Promise immediately (or preload tips before parallel deliveries) so each (chainCode, epochIndex) is truly single-flight.
There was a problem hiding this comment.
Fixed in 0e74f1a. The Map (populated after the awaited read) is replaced with a SingleFlightCache that registers the in-flight promise synchronously, so the parallel deliveries for one (chain_code, epoch) share a single read that observes the pre-delivery tips; it also evicts on rejection so a transient read failure isn't cached. Covered by the concurrency test (N concurrent misses → 1 fetch).
| * @returns The serialized envelope bytes. | ||
| */ | ||
| export function encodeTaggedEnvelope(epochIndex: number, tag: string): Uint8Array { | ||
| export function encodeTaggedEnvelope( |
There was a problem hiding this comment.
[P2] This changes exported encodeTaggedEnvelope and adds an exported tip reader, but the PR adds no unit tests, contrary to this repository’s mandatory test rule. Please add a pinned canonical golden-vector test plus genesis/non-genesis and concurrent-cache coverage; those tests would also catch the envelope-tip and single-flight regressions.
There was a problem hiding this comment.
Added in 0e74f1a. I extracted the canonical codec and the single-flight cache into modules that don't import the cluster-tool harness, so they're unit-testable, and registered the package as a jest project. Coverage: the C++ golden vectors from sysio.msgch_chain_tests.cpp pinned byte-for-byte (vector A genesis + vector B chained from A), an encodeTaggedEnvelope decode-roundtrip asserting both previous_envelope_hash and previous_message_id are set (the envelope-tip regression), chain-tip parsing, and SingleFlightCache concurrency + error-eviction (the single-flight regression).
apply_consensus (wire-sysio #508) compares previous_envelope_hash against outpcons.envelope_digest before the semantic-header validation, so the synthetic dispute envelopes — which left it empty — are dropped once bootstrap has established a tip, and the epoch never advances. readInboundTips now reads envelope_digest alongside message_tip and runDeliver chains both: previous_envelope_hash from the envelope tip, previous_message_id from the message tip. Make the per-(chain_code, epoch) tip read single-flight. The three per-operator deliveries run in parallel, so the previous populate-after-await cache let each caller miss and issue its own read; a late read could observe the post-consensus tip once the non-contested outpost's 2/3 majority advances it inline, chaining a divergent envelope that is mis-classified and wrongly slashed. SingleFlightCache registers the in-flight read synchronously so all parallel misses share the one read that observes the pre-delivery tips, and evicts on rejection so a transient failure is not cached. Extract the canonical codec and the single-flight cache into cluster-tool-free modules so they can be unit-tested (the step module pulls the cluster harness), and add coverage: the C++ golden vectors from sysio.msgch_chain_tests.cpp (genesis and chained), encodeTaggedEnvelope chaining both streams, chain-tip parsing, and single-flight concurrency.
| */ | ||
| export function bytesToHex(bytes: Uint8Array): string { | ||
| return Buffer.from(bytes).toString("hex") | ||
| export function readInboundTips( |
There was a problem hiding this comment.
[P2] Add coverage for the exported tip reader. The codec and SingleFlightCache tests do not exercise this exported function composition: querying outpcons, selecting the requested chain, parsing both tips, and caching the result. The repository requires happy-path and edge-case tests for every edited/exported TypeScript symbol. Please add mocked-context tests covering a matching row, missing/genesis fields, concurrent calls, and query rejection/retry.
There was a problem hiding this comment.
Added in b22e7cd. The read is extracted into InboundTipReader with the outpcons query injected, so the exported composition is testable without the step module's cluster-tool import: readInboundTips delegates unchanged, and tests/InboundTipReader.test.ts covers a matching row (numeric and string chain_code), a missing row and genesis/zero fields, concurrent reads collapsing onto a single query, the per-epoch cache scope, and a rejected query being evicted and retried.
| encodeTaggedEnvelope, | ||
| parseChainTip, | ||
| type TaggedEnvelopeInput | ||
| } from "../src/EnvelopeCanonicalCodec.js" |
There was a problem hiding this comment.
[P2] Remove forbidden src traversal. Both new test files import through ../src/..., which CLAUDE.md explicitly forbids for all import/export specifiers, including tests. Please expose these utilities through a logic-free barrel/package alias, configure the Jest/TypeScript mapping, and import through that alias.
There was a problem hiding this comment.
Fixed in b22e7cd. The tests import through the @wireio/test-flow-batch-operator-slashing/... subpath alias: etc/tsconfig/tsconfig.base.json gains the package's paths entries (same shape as the other test-flow-* packages) and the package jest config maps the alias to source, so it resolves identically under tsc and jest. No specifier contains src/. The package entrypoint is an executable, so the tests use the directory-subpath form rather than a barrel.
| } | ||
|
|
||
| /** A protobuf field tag: `(field << 3) | wireType`, varint-encoded. */ | ||
| const canonicalTag = (field: number, wireType: number) => canonicalVarint(BigInt((field << 3) | wireType)) |
There was a problem hiding this comment.
[P3] Format the changed files. Prettier 3.8.3 reports formatting differences in this file plus SlashingScenarioConstants.ts, SlashingScenarioDisputeSteps.ts, and both new test files. Please run the repository formatter and commit its output.
There was a problem hiding this comment.
Done in b22e7cd — ran the repository prettier (3.9.5, what the lockfile resolves for the ^3.8.1 devDependency) over the PR's files, including both new ones; --check is clean on all of them.
Extract the outpcons tip read into InboundTipReader with the table query injected, so the exported composition (query rows, select the requested chain, parse both tips, single-flight cache with rejection eviction) is unit-testable without the step module's cluster-tool dependency; readInboundTips delegates unchanged. Cover it with mocked-query tests: matching row (numeric and string chain_code), missing row and genesis fields, concurrent reads collapsing onto one query, the per-epoch cache scope, and a rejected query retried instead of cached. Route the test imports through the @wireio/test-flow-batch-operator-slashing alias (tsconfig paths entries plus jest moduleNameMapper, matching the other flow packages) instead of ../src traversal, and format the package's changed files with the repository prettier.
2 participants
The batch-operator-slashing scenario hand-builds OPP envelopes and delivers them via
sysio.msgch::deliver. Since SEC-102 (Wire-Network/wire-sysio#508),apply_consensusvalidates the OPP MessageHeader spec and the per-outpost message chain, so the winning dispute envelope (and the non-contested consensus envelope) need a real, chained header — the previous headerless envelopes are dropped and the epoch never advances.encodeTaggedEnvelopederives the semantic header per spec:payload_size/payload_checksumover the canonical payload bytes,header_checksumover the blanked canonical header, andmessage_idas the sequence-spliced checksum. It uses a small field-complete canonical encoder that mirrors the depot'sopp_canonical_codec.hpp(keccak via ethers); the delivered wire bytes stayEnvelope.toBinaryoutput since the depot re-canonicalizes on receipt.runDeliverreads the outpost's current inbound message tip fromsysio.msgch::outpconsand chains from it. The tip is cached per(chain_code, epoch)so all deliveries an outpost receives for one contested epoch share the same pre-delivery tip — once the non-contested outpost's 2/3 majority advances the tip inline, a re-read would make the third delivery diverge and be wrongly slashed.message_tipis read off the runtime row: it is new in the deployed ABI but absent from the pinned SystemContractTypes.Validated: the canonical encoder reproduces the wire-sysio C++ golden vector A bytes and digest exactly. Part of the SEC-102 PR set (Wire-Network/wire-sysio#506, Wire-Network/wire-sysio#508, Wire-Network/wire-solana#370, Wire-Network/wire-ethereum#137); requires those deployed together in the e2e.