Skip to content

QUEUE-143: restack maintenance foundations after QUEUE-146 - #1745

Draft
peter-lawrey wants to merge 28 commits into
feature/QUEUE-143-exact-index-eof-recoveryfrom
codex/QUEUE-143-post-146-v2
Draft

peter-lawrey wants to merge 28 commits into
feature/QUEUE-143-exact-index-eof-recoveryfrom
codex/QUEUE-143-post-146-v2

Conversation

@peter-lawrey

@peter-lawrey peter-lawrey commented Aug 28, 2026

Copy link
Copy Markdown
Member

This is the single maintenance-foundation delivery owner after #1741. Older #1724 is retained as source/reference and must not become a competing delivery stack. #1746 owns the complete listener lifecycle above this PR.

Review table access/lock policy, passive introspection and mutating parking/naming policy as separate decisions within this stack. Preserve the inseparable read-only pairing: mapped metadata tables report read-only truthfully and retain persisted directory bounds; only the no-metadata fallback selects a filesystem listing. Keep current-capacity scans and restoration of all four cursors/limits together.

Snapshots preserve raw case variants and distinguishable legacy names. Parking requires consistent validation, replicated-name refusal, quiescence and explicit backlog/resume guidance. Reconcile #1731's older all-deleted scenarios with authoritative-current-roll retention before importing tests.

This restructuring leaves head 2238ed5af92863d820d96753bd07ed9f89f13b0a unchanged; it selects the delivery owner and records the old/new semantic boundary.

Existing implementation and validation receipt (restructuring changes only the review map)

Purpose

QUEUE-143 maintenance foundations as a literal descendant of the QUEUE-146 stack. Keep this feature separately tracked and draft for independent review.

Scope owned by this PR

  • Table lookup without creation, while retaining legacy get-or-create behavior and restoring all scan cursor/limit state.
  • Read-only access to persisted directory bounds and named-tailer metadata, with the existing no-metadata fallback.
  • Detached named-tailer snapshots and quiesced parking APIs, including raw-ID/case handling, reserved names, replicated-tailer refusal and restart behavior.
  • Persisted roll-geometry parsing for downstream maintenance through cycleForFile().
  • Reacquisition of closed cached metadata bindings and recovery of an append lock whose owning process is dead.

Q1 destination selection, Q2 ordinary EOF retry, Q3 exact recovery, fresh physical enumeration and normalization are inherited from #1739/#1740/#1741, not introduced again here. Maintenance must retain the current/latest roll and satisfy the documented quiescence and recovery-exclusion requirements. Complete Queue deletion is a separate offline operation.

Review corrections and evidence

  • Removed the redundant import-level //!; the read-only constructor decision retains its adjacent behavioral justification.
  • cycleForFileUsesPersistedNonZeroEpochAfterRestart writes cycles 0 and 2 with non-zero daily, hourly and named-weekly epochs, closes the Queue, and reopens without supplying roll/epoch settings. It verifies the persisted geometry and both parsed cycle numbers. The existing format/rejection test remains.
  • Scan-state tests check all four positions/limits and deterministic visibility after another table instance appends. Independent concurrency tests remain integration evidence, not proof of an atomic snapshot of independently updated values.
  • Parking and snapshot notes state concrete failure policies and cite the corresponding tests. Operational exclusion is a caller obligation, not a guarantee against non-cooperating external writers.

Exact ancestry and local validation

  • Base (QUEUE-146: recover exact indexed entries across a sealed cycle #1741): d1d639b7b190506a5175e078ead8b29af24c2108.
  • Head: 2238ed5af92863d820d96753bd07ed9f89f13b0a.
  • Base equals merge base; 28 commits in the comparison, 14 changed files and no POM delta. The update preserves branch history through a normal forward merge.
  • Inherited the refreshed Q1 boundary and acquisition regressions and Q3 duplicate-mapping correction.
  • Full mvn verify on Linux/OpenJDK 21.0.12 passed: 1,240 reported tests, zero failures/errors, 52 skipped, retries disabled. Checkstyle, Javadocs, packaging and binary compatibility passed.
  • Wire 2026.10-SNAPSHOT was rebuilt from merged Remove automatic roll shrinking code #1280 commit 3dd7fe5983c0133703ee72f57bced9a25270eb6f; runtime JAR SHA-256 92803fb011f5071900f5c3b1b637cf39f6254e86b4e29f77f1256ecc96dde755.

Merge gates

Keep draft. Require the reviewed/stable lower stack, fresh exact-head source/evidence review, green supported-platform TeamCity results or explicit human disposition, the merged Wire #1280 source and immutable release dependency coordinates. The staged coordinate delta remains isolated in #1750. Local Linux validation does not dispose of these gates or explain generic TeamCity failures.

@peter-lawrey

Copy link
Copy Markdown
Member Author

Reviewer follow-up for the corrected, separately tracked QUEUE-143 Queue unit:

Implemented:

  • rebuilt as a literal descendant of exact QUEUE-146 QUEUE-146: recover exact indexed entries across a sealed cycle #1741;
  • sole cached-roll deletion now refreshes and retries rather than fabricating a missing singleton or dereferencing a null store;
  • read-only queues use filesystem bounds rather than the persisted ordinary-write floor;
  • format-change compatibility, latest-roll deletion, normalisation and fresh-appender regressions pass;
  • parkNamedTailer() now documents that the owning consumer must already be stopped and closed.

Deliberately not implemented:

  • no retention/archive/EOF eligibility policy in OSS Queue; that remains CQE QUEUE-143 work;
  • no unconditional scan on ordinary appender startup; stale-bound refresh is confined to the recovery/enumeration seam;
  • no global tailer-ID compatibility tightening; validation remains scoped to the maintenance snapshot/parking use case;
  • no merge of QUEUE-143 into QUEUE-146, even where code is small, because the tracking and approval boundary is intentional.

The literal descendant #1746 passes full mvn -q clean verify, which exercises this exact tree and the unchanged QUEUE-146 suite. #1745 remains draft pending exact-head CI and fresh approval.

@peter-lawrey
peter-lawrey force-pushed the codex/QUEUE-143-post-146-v2 branch from c5507e9 to 2897c11 Compare August 28, 2026 13:36
@peter-lawrey

Copy link
Copy Markdown
Member Author

Restack/review update for exact head 2897c114ad36d98dcb946b045adb26bc54575ec9.

Tracking/ancestry is intentional: this remains a separate QUEUE-143 PR and is a literal descendant of the current QUEUE-146 stack ending at #1741 5157a60c.

Implemented here:

  • stale-bound recovery now verifies the singleton/boundary generation and refreshes once when it is missing;
  • read-only Queue bounds remain filesystem-derived rather than conflated with the persistent writer floor;
  • maintenance-specific named-tailer snapshot/parking and table-store primitives remain isolated to QUEUE-143;
  • parking documentation requires the owning consumer to be stopped and closed.

Not implemented here:

The affected Queue suite and exact Q143/Q144 focused composition pass locally. The entire Queue suite was not rerun after the final metadata fixture updates; TeamCity is pending, so please review this as a tracking/restack head rather than a ready-to-merge claim.

@peter-lawrey

Copy link
Copy Markdown
Member Author

Restack review update

Exact head: d08d933. It is a literal descendant of QUEUE-146 #1741 at 20ee4ec.

Implemented:

  • Repeated stale-bound enumeration now fails closed and cannot advance the normalization cursor.
  • Added persisted roll-filename to cycle parsing for CQE to validate destructive ordering against Queue roll geometry.
  • Focused NormaliseEOFsTest and DeletedRollFileRecoveryTest pass on the final restack.

Deliberately not implemented:

Fresh exact-head CI and review are still required.

@peter-lawrey
peter-lawrey force-pushed the codex/QUEUE-143-post-146-v2 branch from d08d933 to 8bbdc3a Compare August 28, 2026 18:12
@peter-lawrey

Copy link
Copy Markdown
Member Author

Reviewer handoff for exact head 8bbdc3a07:

Implemented:

  • a repeated inconsistent non-empty enumeration now throws instead of returning the genuine-empty sentinel;
  • a missing generation during normalization refreshes then throws;
  • the DateTime fallback is limited to a true singleton, so an interior cycle cannot be skipped;
  • the normalisation cursor advances only after the authoritative enumeration and sealing loop complete.

Not done:

So the semantic fail-closed fix is present and NormaliseEOFsTest is green (4 tests), but the exact interleaving regression should remain a reviewer/release gate.

@peter-lawrey
peter-lawrey force-pushed the codex/QUEUE-143-post-146-v2 branch 3 times, most recently from 5728a9c to b123fcd Compare August 28, 2026 21:15
@peter-lawrey
peter-lawrey force-pushed the codex/QUEUE-143-post-146-v2 branch 2 times, most recently from 0a0ca1d to e6595c1 Compare August 28, 2026 22:43
@peter-lawrey
peter-lawrey force-pushed the codex/QUEUE-143-post-146-v2 branch from e6595c1 to 2657f47 Compare August 30, 2026 08:34
@peter-lawrey
peter-lawrey force-pushed the codex/QUEUE-143-post-146-v2 branch from 2657f47 to a162423 Compare August 31, 2026 06:39
@peter-lawrey
peter-lawrey force-pushed the codex/QUEUE-143-post-146-v2 branch from ee2ec3c to 2b653a2 Compare August 31, 2026 13:07
@peter-lawrey
peter-lawrey force-pushed the codex/QUEUE-143-post-146-v2 branch from b2f43a4 to 9d2832d Compare September 4, 2026 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant