ref(consumer): remove the BLQ (backlog-queue) mechanism#8166
Open
onewland wants to merge 1 commit into
Open
Conversation
The backlog-queue router (BLQRouter) redirected stale messages, by Kafka timestamp, to the DLQ topic and panicked the consumer to flush in-memory state on the fresh->stale transition. It was never fully rolled out, the FSM (Idle/RoutingStale/Flushing/Forwarding + panic-to-reset) was hard to reason about, and no storage enabled it in prod. Remove the strategy, its wiring in factory_v2/consumer, the bench fields, and the consumer.blq_* sentry-options. The DLQ policy/topic are untouched. This clears the way for a simpler DLQ-by-age lever built on the standard InvalidMessage mechanism. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
phacops
approved these changes
Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Removes the backlog-queue (BLQ) mechanism from the Rust consumer.
BLQRouterredirected stale messages (by Kafka broker timestamp) to the DLQ topic and panicked the consumer to flush in-memory state on the fresh→stale transition. It was never fully rolled out, the FSM (Idle/RoutingStale/Flushing/Forwarding+ panic-to-reset) was hard to reason about, and no storage enabled it in prod.This is a pure removal (−481 lines, no additions). The DLQ policy/topic are untouched.
Removed
rust_snuba/src/strategies/blq_router.rs(the strategy + its unit tests)pub mod blq_router;instrategies/mod.rsblq_producer_config/blq_topicfields, and the pipeline-wrapping block infactory_v2.rsblq_producer_configbuilder + factory args inconsumer.rsblq_*: Nonebench fieldsconsumer.blq_*options from the sentry-options schemaWhy
Clears the way for a simpler DLQ-by-age lever (follow-up PR) built on the standard
InvalidMessagemechanism, so we can shed a configurable proportion of too-old messages to the DLQ during a backlog and backfill later.Test plan
cargo check --all-targets— clean, no warningscargo test --lib strategies— 36 pass🤖 Generated with Claude Code