Skip to content

starknet_transaction_prover: global panic hook + graceful SIGTERM shutdown#14166

Open
avi-starkware wants to merge 2 commits into
avi/prover-v3/content-idfrom
avi/prover-v3/panic-shutdown
Open

starknet_transaction_prover: global panic hook + graceful SIGTERM shutdown#14166
avi-starkware wants to merge 2 commits into
avi/prover-v3/content-idfrom
avi/prover-v3/panic-shutdown

Conversation

@avi-starkware

@avi-starkware avi-starkware commented May 24, 2026

Copy link
Copy Markdown
Collaborator

Installs a process-wide panic hook that emits one structured event=panic
tracing event with location and backtrace, and bridges SIGTERM/SIGINT into
ServerHandle::stop (extracted to server/shutdown.rs, keeping main.rs to
a single call) so container teardown becomes visible in logs and in-flight
proofs get to finish. A second signal while the drain is still in progress
forces exit(1) so an operator can always reclaim a stuck process
(tokio-rs/tokio#7905); once the server has stopped cleanly, a late signal no
longer flips the exit code.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

@cursor

cursor Bot commented May 24, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Changes are limited to process lifecycle, logging, and signal handling; no proving, RPC, or auth logic is modified.

Overview
Adds operational hardening for the transaction prover service: panics and container teardown are now visible in structured logs, and shutdown can drain in-flight work.

A process-wide panic hook (install_panic_hook, after tracing init) replaces default stderr output with a single tracing error! event (event=panic) carrying location, message, and backtrace—without changing abort-on-panic behavior. Payload extraction is covered by a unit test.

SIGTERM/SIGINT are bridged via spawn_signal_bridge to ServerHandle::stop() so the first signal starts a graceful drain; structured shutdown_started / shutdown_complete events are logged, and a second signal during drain forces exit(1) to avoid Tokio swallowing follow-up signals. Dependencies gain tokio signal and tracing-subscriber json for JSON log format support.

Reviewed by Cursor Bugbot for commit cffb6a6. Bugbot is set up for automated code reviews on this repo. Configure here.

avi-starkware commented May 24, 2026

Copy link
Copy Markdown
Collaborator Author

@reviewable-StarkWare

Copy link
Copy Markdown

This change is Reviewable

Comment thread crates/starknet_transaction_prover/src/server/panic.rs
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from c31ce65 to 72a08b0 Compare May 26, 2026 12:16
@avi-starkware avi-starkware force-pushed the avi/prover-v3/request-logs branch 2 times, most recently from 2dcb716 to f83f30c Compare May 26, 2026 12:17
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from 72a08b0 to 17dbfd9 Compare May 26, 2026 12:17
@avi-starkware avi-starkware force-pushed the avi/prover-v3/request-logs branch from f83f30c to 53b6a1e Compare May 26, 2026 12:58
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from 17dbfd9 to 9845d29 Compare May 26, 2026 12:58
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from 9845d29 to 4d23115 Compare May 26, 2026 16:14
@avi-starkware avi-starkware force-pushed the avi/prover-v3/request-logs branch from 53b6a1e to 4645e9b Compare May 26, 2026 16:14
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from 4d23115 to c3950e4 Compare May 26, 2026 16:59
@avi-starkware avi-starkware force-pushed the avi/prover-v3/request-logs branch from 4645e9b to 278ffaa Compare May 26, 2026 16:59
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from c3950e4 to fe2dccc Compare May 27, 2026 10:01
@avi-starkware avi-starkware force-pushed the avi/prover-v3/request-logs branch 2 times, most recently from 6a64324 to e28dee6 Compare May 27, 2026 10:35
@avi-starkware avi-starkware force-pushed the avi/prover-v3/request-logs branch 2 times, most recently from 91c2bfc to b232a8e Compare May 27, 2026 13:11
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from d99ffe1 to 264bf5b Compare May 27, 2026 13:11
@avi-starkware avi-starkware changed the base branch from avi/prover-v3/request-logs to graphite-base/14166 May 27, 2026 14:03
@avi-starkware avi-starkware force-pushed the graphite-base/14166 branch from b232a8e to fb6037b Compare May 27, 2026 14:03
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from 264bf5b to 282284b Compare May 27, 2026 14:04
@avi-starkware avi-starkware changed the base branch from graphite-base/14166 to avi/prover-v3/content-id May 27, 2026 14:04
@avi-starkware avi-starkware force-pushed the avi/prover-v3/content-id branch from fb6037b to 4055121 Compare May 27, 2026 14:20
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from 282284b to 073a1ff Compare May 27, 2026 14:20
@avi-starkware avi-starkware force-pushed the avi/prover-v3/content-id branch from 4055121 to bfa94e0 Compare May 31, 2026 10:23
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch 2 times, most recently from 8d4c874 to bdf34cf Compare June 1, 2026 08:17
@avi-starkware avi-starkware force-pushed the avi/prover-v3/content-id branch 2 times, most recently from 54145f3 to adf3407 Compare June 1, 2026 11:18
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from bdf34cf to 15340a8 Compare June 1, 2026 11:18
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from 15340a8 to d7e132f Compare June 7, 2026 10:11
@avi-starkware avi-starkware force-pushed the avi/prover-v3/content-id branch from adf3407 to a6853d8 Compare June 7, 2026 10:11
@avi-starkware avi-starkware force-pushed the avi/prover-v3/content-id branch from a6853d8 to ad1b9cc Compare July 1, 2026 17:32
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from d7e132f to 2c600ed Compare July 1, 2026 17:32
@avi-starkware avi-starkware force-pushed the avi/prover-v3/content-id branch from ad1b9cc to 589df8a Compare July 2, 2026 09:53
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from 2c600ed to 0abb13e Compare July 2, 2026 09:53

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 0abb13e. Configure here.

Comment thread crates/starknet_transaction_prover/src/main.rs Outdated
avi-starkware and others added 2 commits July 2, 2026 14:51
…tdown

Installs a process-wide panic hook that emits one structured `event=panic`
tracing event with location and backtrace, and bridges SIGTERM/SIGINT into
`ServerHandle::stop` so container teardown becomes visible in logs and
in-flight proofs get to finish. A second signal forces `exit(1)` to work
around tokio-rs/tokio#7905.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Force-exit on a second termination signal now only applies while the drain is
still in progress; once the server has stopped cleanly a late signal no longer
flips the exit code to 1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@avi-starkware avi-starkware force-pushed the avi/prover-v3/panic-shutdown branch from 0abb13e to cffb6a6 Compare July 2, 2026 11:53
@avi-starkware avi-starkware force-pushed the avi/prover-v3/content-id branch from 589df8a to 79191bf Compare July 2, 2026 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants