fix(cargo-gamma): generate reference return value mutants

[chinayangxiaowei]

Problem

Functions returning &T or &mut T receive no whole-body fn_value replacement for most referent types, even when Gamma already has useful replacement values for T. A generated Box::leak replacement can also accumulate allocations if the mutated function is called repeatedly.

Change

• Reuse existing fn_value candidates for reference referents. Shared references to literals and name-independent tuple or array compositions borrow them directly. Empty mutable slices and zero-length arrays use &mut []. Other constructible referents use Box::leak.
• Resolve local reference and referent aliases, including type arguments and defaults, while withholding guesses for const-parameter aliases and direct references to impl Trait.
• Keep no-op detection conservative around leaked defaults and macro names, and serialize generated reference-leak mutants across workers.
• Count every generated reference-leak replacement before construction. After 8192 calls in one test process, the runtime stops it with a distinct leaklimit outcome, separate from outofmem and a test kill. On macOS, the first completed construction establishes an RSS baseline; later growth of at least 64 MiB also produces leaklimit. This is a soft process-local check: it cannot stop a single huge construction or a hang inside it. The outcome remains eligible for cargo gamma suppress.
• On other hosts, leave generated leaks pending when a reachable test binary has no enforced memory ceiling. A complete census can still report NoCoverage without running that mutant.
• Update the affected design and user docs, and add collection, instrumentation, runtime, sweep, and compiler-acceptance coverage.

Validation

• cargo test -p cargo-gamma-engine --lib -q: 827 passed.
• cargo test -p cargo-gamma-rt --lib -q: 72 passed.
• cargo test -p cargo-gamma-lib --features internals --test instrumented_compiles -q: 17 passed.
• Targeted cargo-gamma-lib tests for reference-leak policy, resource verdicts, scoring, reporting, CI output, dashboard, and suppression passed.
• cargo check -p cargo-gamma-lib --target x86_64-unknown-linux-gnu -q passed.
• cargo clippy -p cargo-gamma-engine -p cargo-gamma-rt --all-targets -- -D warnings and cargo +nightly fmt --all -- --check passed. The cargo-gamma-lib Clippy check passed when unrelated existing macOS/test lints were allowed.
• The isolated branch does not contain the separate fix(cargo-gamma): handle macOS process group cleanup and CLI builds #240 macOS process and CLI fixes, so the full CLI build and process-dependent library suite cannot serve as macOS end-to-end validation here.

This PR contains only reference return-value mutation. It does not include #240, #241, or #242.

[Copilot]

🟡 Changes recommended

Direct references to opaque iterator returns generate systematically unviable candidates, and acceptance assertions are too permissive.

3 open findings

• Direct impl Trait references leak incompatible iterator concrete types · New
• Lower-bound candidate count misses lost reference-return cases · New
• Aggregate candidate threshold misses missing diverse reference mutants · New

What changed in this PR

Extends fn_value mutation to reference-returning functions using static or leaked replacement values.

Changes:

• Generates candidates for shared/mutable references, strings, slices, arrays, and aliases.
• Allows stated values for reference returns.
• Adds compiler coverage and updates design documentation.

File	Description
crates/​cargo-gamma/​docs/​MUTATORS.md	Documents reference-return mutations.
crates/​cargo-gamma/​docs/​DESIGN.md	Records user-visible semantics.
crates/​cargo-gamma-lib/​tests/​instrumented_compiles.rs	Adds compiler acceptance cases.
crates/​cargo-gamma-engine/​src/​ops/​collect/​tests.rs	Tests candidate collection.
crates/​cargo-gamma-engine/​src/​ops/​collect/​stated.rs	Accepts stated reference values.
crates/​cargo-gamma-engine/​src/​ops/​collect/​collector/​values.rs	Constructs reference replacements.
crates/​cargo-gamma-engine/​src/​ops/​collect/​collector/​tests.rs	Tests generated reference values.
crates/​cargo-gamma-engine/​docs/​IMPLEMENTATION.md	Documents construction details.
crates/​cargo-gamma-engine/​docs/​DESIGN.md	Updates engine design contract.
crates/​cargo-gamma-attrs/​src/​lib.rs	Adds a reference-value example.
crates/​cargo-gamma-attrs/​docs/​DESIGN.md	Documents attribute behavior.
crates/​cargo-gamma-attrs-impl/​src/​implementation.rs	Tests macro validation.

🧠 Review effort: Balanced

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Referent aliases bypass special string, slice, unit, and array handling.

1 open finding

• Resolve local aliases before special shape dispatch · New

3 resolved since last review

• Aggregate candidate threshold misses missing diverse reference mutants
• Lower-bound candidate count misses lost reference-return cases
• Direct impl Trait references leak incompatible iterator concrete types

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Parameterized aliases are not instantiated before referent-shape dispatch, causing promised specialized candidates to be missed.

1 open finding

• Instantiate generic aliases before referent shape dispatch · New

1 resolved since last review

• Resolve local aliases before special shape dispatch

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

The new Vec::leak candidates can emit equivalent mutants because no-op normalization only handles boxed leaks.

1 open finding

• No-op collector misses Vec::leak normalization · New

1 resolved since last review

• Instantiate generic aliases before referent shape dispatch

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Nested leaked-reference replacements can still produce semantically identical surviving mutants.

1 open finding

• Normalize leaks inside composed return expressions · New

1 resolved since last review

• No-op collector misses Vec::leak normalization

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Standard-qualified vec! spellings can still produce equivalent, unkillable mutants.

2 open findings

• Recognize qualified vec! macros during no-op normalization · New
• Normalize leaks inside composed return expressions

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Unmetered leak mutants can exhaust host memory, and one no-op regression test is vacuous.

10 open findings

• Unbounded leak mutants can exhaust host memory · New
• Fixture selects wrong fallback and leaves no-op assertion vacuous · New
• Apply complete-census NoCoverage before the leak gate
• Recognize standard vec imports as unshadowed
• Normalize equivalent bare and standard-qualified paths
• Track macro_use modules for bare macro shadowing
• Respect extern crate aliases shadowing the std root
• Substitute const alias parameters in array lengths
• Clarify literal constructions apply only to shared references · New
• Clarify zero-length array behavior applies to shared references · New

1 resolved since last review

• Bound aggregate memory for concurrent leak mutants

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

A locally shadowed Box can bypass the memory-ceiling gate despite having arbitrary allocation behavior.

9 open findings

• Shadowed Box calls are misclassified as bounded allocations · New
• Fixture selects wrong fallback and leaves no-op assertion vacuous
• Recognize standard vec imports as unshadowed
• Normalize equivalent bare and standard-qualified paths
• Track macro_use modules for bare macro shadowing
• Respect extern crate aliases shadowing the std root
• Unit test improperly spawns a subprocess · New
• Clarify zero-length array behavior applies to shared references
• Clarify literal constructions apply only to shared references

3 resolved since last review

• Unbounded leak mutants can exhaust host memory
• Apply complete-census NoCoverage before the leak gate
• Substitute const alias parameters in array lengths

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Unbounded macOS construction and inaccurate resource classification can still produce unsafe or misleading outcomes.

9 open findings

• macOS lacks an enforced limit for unbounded construction · New
• Allocation-free Box::leak candidates are incorrectly resource-charged · New
• Call-limit failures are incorrectly reported as out of memory · New
• Shadowed Box calls are misclassified as bounded allocations
• Fixture selects wrong fallback and leaves no-op assertion vacuous
• Recognize standard vec imports as unshadowed
• Normalize equivalent bare and standard-qualified paths
• Track macro_use modules for bare macro shadowing
• Respect extern crate aliases shadowing the std root

3 resolved since last review

• Unit test improperly spawns a subprocess
• Clarify zero-length array behavior applies to shared references
• Clarify literal constructions apply only to shared references

🧠 Review effort: Balanced

[chinayangxiaowei]

This is a deliberate coverage tradeoff. The macOS monitor is documented as a soft, process-local signal: it admits the first construction, samples only after it returns, and cannot prevent a one-call OOM or descendant allocation. A normal test can make one construction, so withholding every candidate without a hard subtree ceiling would discard useful reference mutants on macOS. We classify a one-call OOM separately and do not claim a hard memory bound. I am leaving this design concern visible for maintainer judgment.

[chinayangxiaowei]

Correction to my wording: a one-call OOM is outside this guard and may be handled by the OS; Gamma does not guarantee an outofmem verdict for it on macOS without an enforced ceiling. The documented choice is to keep the candidate despite that residual risk.

[Copilot]

🟡 Changes recommended

The diagnostics bundle drops the new outcome, and concurrent RSS baseline establishment can miss the configured growth limit.

3 open findings

• macOS lacks an enforced limit for unbounded construction
• Propagate leak_limit into the diagnostics bundle · New
• Normalize equivalent bare and standard-qualified paths

7 resolved since last review

• Call-limit failures are incorrectly reported as out of memory
• Allocation-free Box::leak candidates are incorrectly resource-charged
• Shadowed Box calls are misclassified as bounded allocations
• Fixture selects wrong fallback and leaves no-op assertion vacuous
• Recognize standard vec imports as unshadowed
• Track macro_use modules for bare macro shadowing
• Respect extern crate aliases shadowing the std root

Previously missed (1)

In code that hasn't changed since last review

Retry baseline decision after a failed CAS

crates/​cargo-gamma-rt/​src/​runtime.rs:348

Two concurrent calls can both load a zero baseline and choose EstablishBaseline; after one CAS wins, the loser ignores the established value and never checks whether its RSS sample is already 64 MiB above it. If no later call occurs, the documented growth guard is missed. Retry the decision with the value returned by a failed CAS.

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

Alias handling and no-op normalization can omit valid mutants or discard behavior-changing ones.

3 open findings

• macOS lacks an enforced limit for unbounded construction
• Propagate leak_limit into the diagnostics bundle
• Normalize equivalent bare and standard-qualified paths

Previously missed (2)

In code that hasn't changed since last review

Recurse through nested leaks before retaining mutants

crates/​cargo-gamma-engine/​src/​ops/​collect/​collector/​noop.rs:116

This only compares the payload of the outer leak and does not recurse when that payload contains another generated leak. The new &Option<&T> case in ops/collect/tests.rs:4112-4135 therefore retains ...Default::default()... against an otherwise identical ...T::default()...; both use the same proven Default implementation, so the reported mutant is unchanged and cannot be killed. Recurse through matching leak structure (while checking surrounding tokens) before retaining it.

Dispatch generic values using the instantiated referent

crates/​cargo-gamma-engine/​src/​ops/​collect/​collector/​values.rs:661

The parameterized referent is instantiated into resolved_elem above, but the generic branch dispatches on the original elem and loses those substitutions again. With type Id<T> = T; fn f() -> &'static Id<bool>, values_for(Id<bool>) raw-resolves to T, so this misses the existing &true/&false candidates and falls back to an unknown/default value. Dispatch using the instantiated referent while preserving the intended alias-constructor safeguards.

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Unresolved alias candidate loss, absolute-path no-op handling, and spoofable marker detection can produce incorrect mutation outcomes.

5 open findings

• Unauthenticated marker output misclassifies test results · New
• macOS lacks an enforced limit for unbounded construction
• Fallback discards resolved referent candidates for scalar aliases · New
• Propagate leak_limit into the diagnostics bundle
• Normalize equivalent bare and standard-qualified paths

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Receipt failures can be scored as kills, and module shadows can incorrectly remove real mutants.

6 open findings

• Track module declarations in the shadow index · New
• Propagate receipt I/O failures as infrastructure results · New
• macOS lacks an enforced limit for unbounded construction
• Propagate leak_limit into the diagnostics bundle
• Normalize equivalent bare and standard-qualified paths
• Inject receipt termination effects instead of spawning processes · New

2 resolved since last review

• Unauthenticated marker output misclassifies test results
• Fallback discards resolved referent candidates for scalar aliases

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Reference instrumentation and receipt classification contain correctness gaps for valid inputs and ordinary test output.

5 open findings

• Public marker text can spoof infrastructure failure classification · New
• macOS lacks an enforced limit for unbounded construction
• Zero-length array detection misses equivalent expression forms · New
• Async stated expressions are moved into a synchronous closure · New
• Propagate leak_limit into the diagnostics bundle

4 resolved since last review

• Propagate receipt I/O failures as infrastructure results
• Track module declarations in the shadow index
• Normalize equivalent bare and standard-qualified paths
• Inject receipt termination effects instead of spawning processes

🧠 Review effort: Balanced

[Martin Taillefer (geeknoid)]

One new correctness finding: a successful leak-limit receipt can be hidden by a later normal-exit record from another nextest process, converting resource exhaustion into mutation-kill credit.

The new reference construction preserves explicit shared reborrows, instantiates supported alias arguments/defaults, and avoids allocations for empty mutable slices/literal zero-length arrays; compiler-acceptance fixtures cover the intended type matrix. The distinct leaklimit outcome is consistently undetected across scoring, schema reasons, diagnostics, and suppression. Existing discussions about the macOS soft bound, marker authentication, stated async expressions, and zero-length syntax are not duplicated.

All five passes covered collection, instrumentation, runtime/FFI, scheduler admission, verdict parsing, and outcome consumers. No builds, tests, or benchmarks were executed. macOS RSS sampling remains a documented soft bound rather than host-memory containment; native macOS assurance is not established by this static review.

[Martin Taillefer (geeknoid)]

A later nextest exit can hide a valid leak-limit receipt and earn kill credit — Correctness · High · High confidence

One attempt exports the same reach-file path to the nextest runner and its test children (observe_with:1427–1430, configure:891–893). Consider two already-running tests of this binary: process A exhausts the allowance, appends and closes REFERENCE_LEAK_LIMIT_RECORD in runtime.rs:443–452; process B then exits normally and appends its bitmap plus SEAL in runtime.rs:1946–1974, before or after A's immediate exit. The receipt is valid but no longer last, so this predicate becomes false. Nextest reports a test failure and verdict_with_reference_leak_receipt leaves it as Failed, giving the suite detection credit for a resource guard rather than an assertion.

The new scheduler resource serializes mutants, not nextest test children: launcher:692–695 requests serial nextest only for configured binary/test resources. Ordinary tests have none; the process-local census lock also cannot order different children's append streams. Fail-fast cannot prevent B's exit from interleaving with A's receipt/termination.

The new test at line 2220 explicitly asserts that [REFERENCE_LEAK_LIMIT_RECORD, SEAL] is not a limit receipt, so it locks in this reachable misclassification.

Direction: Preserve successfully recorded limit evidence across other processes' census appends, e.g. with a separately owned receipt channel or a validated shared-record protocol rather than a last-record requirement.

Done when: An in-process fixture models a valid limit receipt followed by another child's reach/seal records and still produces ReferenceLeakLimit/undetected, while malformed or failed receipt writes remain unjudged and ordinary test output cannot forge the verdict.

[Martin Taillefer (geeknoid)]

Including leaklimit in the default suppression set is the right choice: it records a resource limit reached by the generated replacement, not an assertion detecting the mutation. Users should be able to suppress these outcomes alongside timeout and outofmem, while retaining the existing preview/--apply approval step. Thanks for wiring this into the default and covering it with a regression test.

[chinayangxiaowei]

Closing this PR rather than reworking it. Thanks to everyone who reviewed it, Martin Taillefer (@geeknoid) and martinhavelka (@wukchung) in particular. The review rounds surfaced real problems, and they led to the conclusion below.

Why close. Nearly all of this PR's added coverage comes from leaked-allocation candidates (&*Box::leak(Box::new(v)), Vec::leak(vec![..])). #208 deliberately removed exactly that approach. A fabricated leaked allocation can change ownership and lifetime behaviour, and referents often cannot be constructed. I should have started from that decision instead of reintroducing the approach. Most of this PR's growth during review, including the leak-limit counter, the macOS RSS check, the receipt protocol, and the leaklimit outcome, existed only to contain the consequences of those leaks.

Usage data. We checked the draft build against a downstream project that ran it for several days (247 reports, no reruns):

• 17 unique leak-based mutants were generated. 14 failed to compile (E0277: the referent has no Default). The other 3 were killed, but they cover only two single-field accessors, and existing tests caught both.
• No reference-return mutant ever survived: 210 were killed and 63 failed to compile. None of them exposed a test gap.
• Of 266 reference-returning functions in that codebase, &str (138) and slices (16) are already covered on main. None returns a reference to a scalar or literal composition. The remaining 112 return references to user structs, Path, Arc, Vec, Option and similar types, which no allocation-free candidate can cover.

So an allocation-free subset compatible with #208, such as borrowing promoted literals like &0, &true or &(0, false), would add almost nothing in practice. Copy types are rarely returned by reference. For such sites, #[gamma::value(...)] remains the right tool, as #208 documents.

Memory. The underlying concern was that a mutant can allocate without bound on a host with no memory ceiling. That is not specific to leaks. A mutated loop condition does the same, much faster. #249 addresses it for every mutant on macOS. Linux and Windows already have a ceiling through cgroups and job objects. All open review threads here concern the leak machinery or its candidates, so they lapse with this PR.