Skip to content

fix(arty): harden retained waker lifecycle - #805

Draft
martintmk wants to merge 12 commits into
mainfrom
martintmk-arty-post-merge-hardening-ef5
Draft

martintmk wants to merge 12 commits into
mainfrom
martintmk-arty-post-merge-hardening-ef5

Conversation

@martintmk

@martintmk martintmk commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

Summary

Harden Arty after #785 by restoring a constrained independent-waker contract, making consuming wake ownership panic-safe, and applying the packaging/documentation follow-ups from the merged review.

  • Retained task wakers route to the owning worker while the task is pending, may be used cross-thread, and become inert no-ops after completion, cancellation, or runtime shutdown.
  • Wake metadata is allocated lazily from plurality::Pool<WakerState> and does not retain pinned task storage or delay shutdown.
  • Consuming wake uses an RAII owner so a panicking configured owner waker still releases counted raw-waker ownership.
  • All four previously ignored lifecycle integration cases are enabled; direct public executor integration covers retained post-shutdown wakes and a panicking owner waker.
  • arty_macros_impl resolves Cargo-renamed arty dependencies, with a real consumer compile fixture.
  • The Arty ohno dev-dependency is path-only, and public package/crate descriptions now distinguish multi-worker runtime size from per-task thread affinity.
  • docs/arty-runtime-panic-audit-disposition.md revalidates the docs(arty): preserve runtime panic audit #800 audit against the exact merged baseline.

Lifecycle contract and rationale

  1. While pending, cloned wakers may cross threads. A wake routes back to the task's original worker and never relocates the task.
  2. Completion and cancellation retire wake state before task storage is released.
  3. Wake-by-reference and consuming wake remain valid after retirement and shutdown, but are inert.
  4. Retained wakers own pooled wake metadata only; they own neither task storage, worker lifetime, nor runtime lifetime.

Alternatives considered:

  • Inline task-owned state: rejected because an escaped clone retains pinned task storage and can drive shutdown to the executor timeout.
  • Fresh std::sync::Arc<WakerState>: simpler and faster for one isolated replacement, but adds one 88-byte allocation per initialized independent state.
  • Pooled state (chosen): zero warmed allocations, stable addresses, cross-thread final release, and 15% fewer final-head allocation-path instructions. The isolated pool replacement is slower wall-clock than the allocator's optimized Arc::new, but state is created lazily only when the polling waker is cloned; representative spawn, wake, and burst instruction counts remain stable or improve.

#800 audit disposition

The disposition document preserves the report's panic/deadlock/invariant/debug-only/non-production distinctions.

  • Implemented: independent-waker retirement, retained-waker shutdown behavior, panic-safe consuming ownership, and public integration coverage.
  • Kept intentionally: one-shot repoll and direct same-pool misuse panics; task-scoped JoinError; fail-closed raw executor, accounting, poisoned-lock, pointer, and unsafe teardown invariants; terminal aborts where live referenced storage cannot be released safely.
  • Still a deadlock, not a panic: transitive saturated blocking-pool dependencies. This does not resurrect an unbounded strong-parent ownership chain.
  • Deferred with explicit prerequisites: typed partial-start/thread-spawn rollback, transactional Tick virtual-time overflow handling, and moving custom timer callbacks outside Tick's timer lock.
  • Not applicable/withdrawn: removed LocalScheduler findings, ancestry-chain I12, and N01's deleted stack-overflow fixture.

Azure DevOps

The existing child story is referenced rather than duplicated.

Validation

Package-scoped validation, plus focused reproductions for concrete CI failures:

cargo check -p arty -p arty_executor -p arty_macros -p arty_macros_impl
cargo clippy -p arty_executor --all-targets --features test-util -- -D warnings
cargo clippy -p arty_macros --all-targets -- -D warnings
cargo test -p arty_executor --release --no-run
cargo test -p arty_executor --lib
cargo test -p arty_executor --test waker_lifecycle
cargo test -p arty --test panic_waker_lifetimes --test panic_shutdown_timer
cargo test -p arty_macros --test renamed_dependency
cargo sort -c crates\arty_macros_impl
cargo doc2readme --lib --check --template ..\README.j2  # arty, arty_executor
cargo fmt -p arty -p arty_executor -p arty_macros -p arty_macros_impl --check

All passed. The Anvil wrapper could not launch its PowerShell script host locally, so equivalent package-scoped tools were invoked directly; CI remains authoritative for the broader Anvil tier.

Performance methodology

  • Baseline revision: cb3ef36dc0942dc93017df934a3fcacfc7f451ad
  • Comparison revision: 74df66f04f19cd8d35731fa0e2eb265a89f5b46b
  • Environment: same WSL2 host, x86_64-unknown-linux-gnu, Rust 1.96.1, -C target-cpu=x86-64-v3
  • Isolation: detached baseline worktree; implementation worktree remained independent.
  • Commands: equivalent arty_executor/ae_basic_operations Gungraun, allocation, and Criterion runs; equivalent arty/arty_scheduling runs. The final production-pointer revision was remeasured before this update.
cargo bench -p arty_executor --features test-util --bench ae_basic_operations -- --gungraun
cargo bench -p arty_executor --features test-util --bench ae_basic_operations -- --allocations --test
cargo bench -p arty_executor --features test-util --bench ae_basic_operations -- --criterion --criterion-arg --quick
cargo bench -p arty --bench arty_scheduling -- --gungraun
cargo bench -p arty --bench arty_scheduling -- --criterion --criterion-arg --quick
cargo bench -p arty --bench arty_scheduling -- --criterion --criterion-arg wake_w1_n20

Deterministic Callgrind instruction counts and allocation probes are the acceptance evidence. Criterion quick samples were retained as exploratory data but are not used to judge small regressions on the shared VM. The full cross-thread wake sample is included because it directly exercises routing through the public runtime.

Path Baseline Final head Delta
Gungraun warm task add 307 instr 297 instr -3.3%
Gungraun spawn/complete 1,478 instr 1,499 instr +1.4%
Gungraun yield 2,670 instr 2,715 instr +1.7%
Gungraun isolated wake producer 162 instr 192 instr +18.5%
Gungraun wake consumption, all 32 15,925 instr 9,243 instr -42.0%
Gungraun overflow wake consumption 511,509 instr 91,437 instr -82.1%
Gungraun sequential spawn/complete x1,000 1,390,191 instr 1,411,181 instr +1.5%
Gungraun spawn/complete burst x10,000 4,747,728 instr 4,796,286 instr +1.0%
Gungraun yield burst x10,000 7,848,523 instr 8,296,247 instr +5.7%
Arty cross-thread wake w1/n20, full sample 196,488.98 ns 199,900.21 ns +1.7%
Warm spawn/complete allocations 0 / 0 B 0 / 0 B unchanged
Cold task-add allocations 10 / 23,120 B 10 / 23,064 B unchanged count, -56 B
Retired-waker no-op unsupported lifecycle 121 instr; 0 / 0 B new contract
Pending-task shutdown no focused baseline case 8,787 instr; 12 / 952 B new focused case
Pooled state replacement n/a 0 / 0 B; 159 instr; 49.14 ns new
Fresh Arc state replacement n/a 1 / 88 B; 187 instr; 33.69 ns comparison

The isolated producer increase is the queue-retirement safety bookkeeping needed to remove stale task references deterministically. It does not produce a meaningful end-to-end cross-thread wake regression, while wake consumption, overflow handling, burst throughput by instruction count, and warmed allocation behavior remain stable or improve. The panicking-owner path is intentionally not timed: unwind and panic-hook cost dominates an exceptional correctness path, which is instead covered by a public regression test plus the private counted-ownership invariant test.

Add pooled independent wake state, panic-safe consuming ownership, public lifecycle coverage, audit dispositions, renamed-dependency macro resolution, and focused performance probes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9d20a52e-ad06-4f05-bf1c-1b0be788393a
Keep safety comments adjacent to unsafe operations and split benchmark orchestration into focused helpers.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9d20a52e-ad06-4f05-bf1c-1b0be788393a
Use the minimal raw-string delimiter and rely on the repository's test lint policy.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9d20a52e-ad06-4f05-bf1c-1b0be788393a
Order the new proc-macro dependency according to the workspace cargo-sort contract.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9d20a52e-ad06-4f05-bf1c-1b0be788393a
Use the repository-standard spelling in executor lifecycle documentation and regenerate the crate README.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9d20a52e-ad06-4f05-bf1c-1b0be788393a
Patch the in-workspace plurality crate so the offline Cargo-renamed consumer fixture is reproducible in clean CI caches.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9d20a52e-ad06-4f05-bf1c-1b0be788393a
@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

⚠️ SemVer check advisory

Inconclusive comparisons

cargo semver-checks could not complete the following comparisons. These failures are informational because an unbuildable baseline is not evidence of a breaking API change.

metabench (exit 101)

     Cloning cb3ef36dc0942dc93017df934a3fcacfc7f451ad
    Building metabench v0.1.1 (current)
error: running cargo-doc on crate 'metabench' failed with output:
-----
   Compiling proc-macro2 v1.0.107
   Compiling unicode-ident v1.0.26
   Compiling quote v1.0.47
   Compiling serde_core v1.0.229
   Compiling zmij v1.0.23
   Compiling serde v1.0.229
   Compiling serde_json v1.0.151
    Checking cfg-if v1.0.5
   Compiling libc v0.2.190
    Checking memchr v2.8.3
   Compiling syn v3.0.6
   Compiling syn v2.0.119
    Checking itoa v1.0.18
   Compiling zerocopy v0.8.62
    Checking bitflags v2.13.2
   Compiling rustversion v1.0.23
   Compiling shlex v2.0.1
   Compiling equivalent v1.0.2
   Compiling winnow v1.0.4
   Compiling find-msvc-tools v0.1.14
   Compiling hashbrown v0.17.1
   Compiling semver v1.0.28
   Compiling rustc_version v0.4.1
   Compiling toml_parser v1.1.5+spec-1.1.0
   Compiling indexmap v2.14.2
   Compiling serde_derive v1.0.229
   Compiling cc v1.6.0
   Compiling zerocopy-derive v0.8.62
   Compiling derive_more-impl v2.1.1
   Compiling rustix v1.1.5
   Compiling toml_datetime v1.1.2+spec-1.1.0
   Compiling autocfg v1.5.1
   Compiling toml_edit v0.25.17+spec-1.1.0
   Compiling num-traits v0.2.19
    Checking either-or-both v0.3.1
   Compiling alloca v0.4.0
   Compiling gungraun-macros v0.9.1
   Compiling proc-macro-error-attr3 v3.1.1
    Checking ciborium-io v0.2.2
    Checking either v1.19.0
   Compiling thiserror v2.0.21
    Checking regex-syntax v0.8.11
   Compiling cfg_aliases v0.2.2
    Checking clap_lex v1.1.1
   Compiling getrandom v0.4.3
    Checking anstyle v1.0.14
    Checking linux-raw-sys v0.12.1
   Compiling bincode-next v3.1.1
    Checking regex-automata v0.4.18
    Checking clap_builder v4.6.7
   Compiling nix v0.31.3
    Checking half v2.7.1
    Checking itertools v0.13.0
    Checking ciborium-ll v0.2.2
   Compiling proc-macro-error3 v3.1.1
    Checking rapidhash v4.5.1
   Compiling proc-macro-crate v3.5.0
   Compiling derive_more v2.1.1
   Compiling thiserror-impl v2.0.21
    Checking unty-next v0.1.2
    Checking cast v0.3.0
    Checking same-file v1.0.6
   Compiling pastey v0.2.3
   Compiling gungraun v0.19.4
    Checking criterion-plot v0.8.2
    Checking walkdir v2.5.0
   Compiling metabench_macros_impl v0.1.1 (/home/runner/work/oxidizer/oxidizer/crates/metabench_macros_impl)
    Checking clap v4.6.7
    Checking ciborium v0.2.2
    Checking regex v1.13.1
    Checking gungraun-runner v0.20.0
    Checking gungraun-runner v0.19.4
    Checking tinytemplate v1.2.1
    Checking folo_utils v0.1.14
    Checking nix v0.27.1
    Checking page_size v0.6.0
    Checking once_cell v1.21.4
    Checking anes v0.1.6
    Checking oorandom v11.1.5
    Checking jiff-core v0.1.1
   Compiling metabench v0.1.1 (/home/runner/work/oxidizer/oxidizer/crates/metabench)
    Checking fastrand v2.5.0
    Checking tempfile v3.27.0
    Checking jiff v0.2.38
    Checking command-group v5.0.1
    Checking criterion v0.8.2
    Checking gungraun-summary v6.0.0
    Checking alloc_tracker v0.7.12
error[E0432]: unresolved import `gungraun_runner::api::ValgrindTool`
  --> /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/gungraun-summary-6.0.0/src/v6.rs:10:59
   |
10 |     CachegrindMetric, DhatMetric, ErrorMetric, EventKind, ValgrindTool,
   |                                                           ^^^^^^^^^^^^ no `ValgrindTool` in `api`

error[E0432]: unresolved imports `gungraun_runner::metrics::model::MetricsDiff`, `gungraun_runner::metrics::model::MetricsSummary`
  --> /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/gungraun-summary-6.0.0/src/v6.rs:12:63
   |
12 | pub use gungraun_runner::metrics::model::{Metric, MetricKind, MetricsDiff, MetricsSummary};
   |                                                               ^^^^^^^^^^^  ^^^^^^^^^^^^^^ no `MetricsSummary` in `metrics::model`
   |                                                               |
   |                                                               no `MetricsDiff` in `metrics::model`

error[E0432]: unresolved imports `gungraun_runner::summary::model::Diffs`, `gungraun_runner::summary::model::FlamegraphSummary`, `gungraun_runner::summary::model::ProfileInfo`, `gungraun_runner::summary::model::SummaryFormat`, `gungraun_runner::summary::model::SummaryOutput`, `gungraun_runner::summary::model::ToolMetricSummary`
  --> /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/gungraun-summary-6.0.0/src/v6.rs:14:38
   |
14 |     BenchmarkKind, BenchmarkSummary, Diffs, FlamegraphSummary, Profile, ProfileData, ProfileInfo,
   |                                      ^^^^^  ^^^^^^^^^^^^^^^^^                        ^^^^^^^^^^^ no `ProfileInfo` in `summary::model`
   |                                      |      |
   |                                      |      no `FlamegraphSummary` in `summary::model`
   |                                      no `Diffs` in `summary::model`
15 |     ProfilePart, ProfileTotal, Profiles, SCHEMA_VERSION, SummaryFormat, SummaryOutput,
   |                                                          ^^^^^^^^^^^^^  ^^^^^^^^^^^^^ no `SummaryOutput` in `summary::model`
   |                                                          |
   |                                                          no `SummaryFormat` in `summary::model`
16 |     ToolMetricSummary, ToolRegression,
   |     ^^^^^^^^^^^^^^^^^ no `ToolMetricSummary` in `summary::model`

For more information about this error, try `rustc --explain E0432`.
error: could not compile `gungraun-summary` (lib) due to 3 previous errors
warning: build failed, waiting for other jobs to finish...

-----

error: failed to build rustdoc for crate metabench v0.1.1
note: this is usually due to a compilation error in the crate,
      and is unlikely to be a bug in cargo-semver-checks
note: the following command can be used to reproduce the error:
      cargo new --lib example &&
          cd example &&
          echo '[workspace]' >> Cargo.toml &&
          cargo add --path /home/runner/work/oxidizer/oxidizer/crates/metabench &&
          cargo check &&
          cargo doc

error: aborting due to failure to build rustdoc for crate metabench v0.1.1

Replace the obsolete shutdown-timeout expectation with the supported inert retained-waker behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9d20a52e-ad06-4f05-bf1c-1b0be788393a
Exercise concurrent state installation, queue retirement, pooled ownership release, forwarding, overflow, and drop cleanup while excluding only unreachable RawWaker and benchmark-only mutations.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9d20a52e-ad06-4f05-bf1c-1b0be788393a
Verify active-signal drop returns pooled ownership and that untracked queue entries are preserved, eliminating the remaining missed lifecycle mutants.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9d20a52e-ad06-4f05-bf1c-1b0be788393a
Use a single compare-exchange for queued notification ownership and cover both claimed and untracked paths without mutation-test deadlocks.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9d20a52e-ad06-4f05-bf1c-1b0be788393a
@codecov

codecov Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (cb3ef36) to head (74df66f).

Additional details and impacted files
@@           Coverage Diff            @@
##             main     #805    +/-   ##
========================================
  Coverage   100.0%   100.0%            
========================================
  Files         798      796     -2     
  Lines      119402   119727   +325     
========================================
+ Hits       119402   119727   +325     
Flag Coverage Δ
linux 100.0% <100.0%> (ø)
linux-arm 100.0% <100.0%> (ø)
scheduled ?
windows 100.0% <100.0%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

martintmk and others added 2 commits October 9, 2026 18:28
Build owned RawWakers from the original atomic pool pointer instead of re-deriving them from a shared reference, preserving unique provenance for final destruction under Miri.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9d20a52e-ad06-4f05-bf1c-1b0be788393a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9d20a52e-ad06-4f05-bf1c-1b0be788393a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant