Skip to content

feat(forge): add fuzz guidance file - #17186

Closed
decofe wants to merge 12 commits into
masterfrom
centaur/fuzz-guidance-1790727287
Closed

decofe wants to merge 12 commits into
masterfrom
centaur/fuzz-guidance-1790727287

Conversation

@decofe

@decofe decofe commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

External tools can steer Forge without rewriting a test harness through a versioned JSON file accepted by forge test --fuzz-guidance, forge fuzz run --guidance, [fuzz] guidance, and FOUNDRY_FUZZ_GUIDANCE.

Dictionary values bias fuzz and invariant inputs. Selector weights bias newly generated invariant calls, including dynamic targets, merged targetInterfaces, and reentrant call_override calls. Exact artifact identifiers override bare contract names and unqualified selectors; persisted calls replay unchanged, and all-zero weights fall back to uniform selection. Inline test configuration may select a different guidance file.

This is an explicit bridge for source-aware tools or humans, not a claim of autonomous bug discovery. The format and sampling behavior are documented in foundry-rs/book#2124.

Prepared with Codex assistance. Prompted by @mattsse.

Add `--fuzz-guidance <PATH>` (`[fuzz] guidance`, `forge fuzz run --guidance`)
to inject a versioned JSON guidance file into fuzz and invariant campaigns.
Dictionary entries become a persistent sampling pool that is drawn for half
of dictionary-sourced parameters, and selector weights bias invariant target
function selection, with weight 0 excluding a function. Without guidance the
random streams are unchanged.

Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>

@mattsse mattsse left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for the qualified-target identity bug below. The other three comments are nonblocking suggestions to make rule resolution explicit and keep parsing and call-plan construction easier to maintain.

All 80 existing fuzz-library tests pass. Local probes reproduce the dynamic-target mismatch and the overlapping-rule behavior. Please also link the corresponding book update for the new configuration and file format, including its all-zero fallback and replay limitations.

Comment thread crates/evm/fuzz/src/strategies/tx.rs Resolved
Comment thread crates/evm/fuzz/src/guidance.rs Resolved
Comment thread crates/evm/fuzz/src/guidance.rs Resolved
Comment thread crates/evm/fuzz/src/strategies/tx.rs Resolved
mattsse
mattsse previously approved these changes Sep 30, 2026

@mattsse mattsse left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mattsse mattsse left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Second pass over 1dc053c. Verdict: not mergeable yet, for the two reasons already on the PR: the dynamic-target identity mismatch (setUp targets carry path:Name from ContractsByAddress, while target_contract_for_code stores artifact.name), and the missing book update for [fuzz] guidance, --fuzz-guidance, FOUNDRY_FUZZ_GUIDANCE and the file format.

The core holds up on re-read: the unguided draw path is the old one unchanged, the cumulative-weight partition_point selection can never land on a weight-0 entry and falls back to uniform when the total is 0, and revert() cannot touch the guidance pool. Two small notes inline.

Comment thread crates/evm/fuzz/src/strategies/param.rs Resolved
Comment thread crates/evm/fuzz/src/guidance.rs Resolved
Comment thread crates/evm/fuzz/src/guidance.rs Resolved
Normalize selector weights with explicit artifact/name/global precedence, retain full identities for dynamic targets, and keep hex dictionary values usable for fixed-size bytes. Emit skipped selector diagnostics at the load boundary and rebuild call strategies with their weights in one pass.
mattsse
mattsse previously approved these changes Oct 2, 2026
Comment thread crates/evm/fuzz/src/strategies/param.rs Resolved
Comment thread crates/evm/fuzz/src/strategies/tx.rs Resolved
Comment thread crates/evm/fuzz/src/strategies/tx.rs Resolved
Comment thread crates/forge/src/runner.rs Resolved
@figtracer
figtracer marked this pull request as draft October 2, 2026 13:28
Apply dictionary and selector guidance consistently to direct state mutation, merged target interfaces, reentrant call overrides, and inline test configuration.
@figtracer
figtracer marked this pull request as ready for review October 4, 2026 20:22
@figtracer
figtracer requested review from mablr and mattsse October 4, 2026 20:43
@figtracer
figtracer marked this pull request as draft October 4, 2026 20:44
Discard targets whose functions all have zero guidance weight when another target has an eligible function. Preserve uniform fallback only when every target is excluded, and cover both cases with a deterministic strategy regression.
@mattsse
mattsse marked this pull request as ready for review October 23, 2026 08:05
mattsse
mattsse previously approved these changes Oct 23, 2026
@mattsse
mattsse enabled auto-merge (squash) October 23, 2026 08:06
mattsse
mattsse previously approved these changes Oct 23, 2026
Use the patched constant-time mask generation and u128 comparisons required by RUSTSEC-2026-0385.
@figtracer figtracer closed this Oct 23, 2026
auto-merge was automatically disabled October 23, 2026 09:31

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

4 participants