Skip to content

SUPERSEDED: QUEUE-144 appender EOF prerequisite - #1719

Closed
peter-lawrey wants to merge 1 commit into
feature/QUEUE-143-RollFileCleanupMainfrom
feature/QUEUE-144-queue-appender-eof-prereq
Closed

peter-lawrey wants to merge 1 commit into
feature/QUEUE-143-RollFileCleanupMainfrom
feature/QUEUE-144-queue-appender-eof-prereq

Conversation

@peter-lawrey

@peter-lawrey peter-lawrey commented Aug 7, 2026 •

Copy link
Copy Markdown
Member

Important

Superseded: do not merge this PR.

The underlying fix was merged into develop by #1714. This PR only reapplied it to the earlier QUEUE-143 branch. On the corrected minimal QUEUE-143 base the change cherry-picks as empty. Continue with #1724, then the recreated QUEUE-144 stack starting at #1725.

Purpose of the underlying fix

Constructing a StoreAppender performs an EOF back-scan over existing roll cycles. Before this fix, that scan left the appender holding the selected store, wires, mapped-file reservation and file descriptor even when the appender never wrote.

If another appender subsequently rolled forward, the unused appender could keep an old .cq4 file open. FileUtil.removableRollFileCandidates(...) scans oldest-first and stops at the first open file, so the parked file also prevented all later files from being considered removable.

releaseParkedStore() releases those resources after the scan and returns the appender to a never-written state. Its first write reacquires the current cycle. The related normaliseEOFs0(cycle()) change is required because the raw cycle field is reset when the parked store is released.

Regression evidence

The tests do demonstrate the failure, rather than only execute the changed code:

  • AppenderReleasesParkedStoreTest.parkedAppenderDoesNotPinAnOldCycle fails if only the releaseParkedStore() call is removed. It expects cycles 0 and 1 to be removable but receives only cycle 0, showing that cycle 1 remains pinned.
  • NormaliseEOFsTest.freshlyConstructedAppenderNormalisesWithoutWriting fails if normaliseEOFs0(cycle()) is changed back to the raw cycle field. The normalisedEOFsTo watermark remains at 0.
  • CreateAtIndexTest and NotCompleteTest account for the expected normalisedEOFsTo metadata written on the first append after reacquisition; they are compatibility adjustments, not the primary regression proof.

All four touched test classes pass at this PR head.

Remaining test limitation

The file-pinning regression test relies on /proc/self/fd and is skipped on unsupported operating systems. If this behaviour is changed again, add a portable lifecycle assertion that a fresh appender on a non-empty queue has currentFile() == null until its first write, then has the current roll file afterward. Also assert exactly one normalisedEOFsTo record rather than allowing dump cleanup to remove duplicates.

Copy link
Copy Markdown
Member Author

Superseded by the QUEUE-143 split plan. On the new minimal Queue base, this appender EOF prereq cherry-picks as empty, so there is no replacement PR for this one. Continue with #1724 first, then the recreated QUEUE-144 stack starting at #1725.

@peter-lawrey peter-lawrey self-assigned this Aug 7, 2026
@peter-lawrey peter-lawrey changed the title QUEUE-144: Add appender EOF prerequisite SUPERSEDED: QUEUE-144 appender EOF prerequisite Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants