starknet_transaction_prover: proving-job duration + outcome metrics

[avi-starkware]

Adds Prometheus counters / histograms recorded by VirtualSnosProver for
each proving job: total count by outcome (success, validation_error,
internal_error, l1_provider_error) and end-to-end duration.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

[cursor]

PR Summary

Low Risk
Observability-only changes around the prover path; no change to proving logic or RPC behavior beyond recording metrics and logging outcome.

Overview
Adds Prometheus observability around prove_transaction: every call records end-to-end wall time and a bounded outcome label (success, failure_validation, failure_runner, etc.), including failures so SLO queries can separate success from error paths.

VirtualSnosProver::prove_transaction is refactored so metrics run in an outer wrapper after prove_transaction_inner returns; run_and_prove additionally emits histograms for OS run and Stwo prove sub-steps. VirtualSnosProverError::metric_outcome() centralizes error→label mapping, with a unit test locking cardinality.

New metric names live under server::metrics::names and outcomes (queue-rejection labels are declared for future admission-layer use). Integration tests assert metric deltas on successful prove and on validation failure without stwo_proving.

^{Reviewed by Cursor Bugbot for commit cddf9b5. Bugbot is set up for automated code reviews on this repo. Configure here.}

[avi-starkware]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• starknet_transaction_prover: log resolved config at startup (secrets redacted) #14684
• starknet_transaction_prover: document observability surface in README #14203
• starknet_transaction_prover: origin-level breadcrumbs for validation errors #14172
• starknet_transaction_prover: /health returns 503 when service is saturated #14171
• starknet_transaction_prover: bump prover_panics_total on panic hook fire #14170
• starknet_transaction_prover: HTTP request count + latency + in-flight metrics #14169
• starknet_transaction_prover: proving-job duration + outcome metrics #14168 👈 (View in Graphite)
• starknet_transaction_prover: Prometheus /metrics endpoint with build_info #14167
• starknet_transaction_prover: global panic hook + graceful SIGTERM shutdown #14166
• starknet_transaction_prover,tower_ohttp: OHTTP-unlinkable request-id for decapsulated content #14222
• starknet_transaction_prover: per-request structured log with request-id #14165
• starknet_transaction_prover: unify HTTP middleware stack via macro #14286
• main-v0.14.3

This stack of pull requests is managed by Graphite. Learn more about stacking.

[reviewable-StarkWare]

This change is 

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit cddf9b5. Configure here.}

[cursor]

Blocked requests skew sub-step metrics

Medium Severity

When a blocking check returns blocked, prove_with_blocking_check aborts the spawned run_and_prove task but still returns failure_blocked on the top-level outcome counter. That background task can finish OS or proving first and increment prover_os_run_duration_seconds and prover_stwo_prove_duration_seconds even though the request is not counted as success.

Additional Locations (1)

• crates/starknet_transaction_prover/src/proving/virtual_snos_prover.rs#L253-L256

 

^{Reviewed by Cursor Bugbot for commit cddf9b5. Configure here.}