SUPERSEDED: QUEUE-144 appender EOF prerequisite - #1719
Closed
peter-lawrey wants to merge 1 commit into
Closed
peter-lawrey wants to merge 1 commit into
peter-lawrey wants to merge 1 commit into
Conversation
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Superseded: do not merge this PR.
The underlying fix was merged into
developby #1714. This PR only reapplied it to the earlier QUEUE-143 branch. On the corrected minimal QUEUE-143 base the change cherry-picks as empty. Continue with #1724, then the recreated QUEUE-144 stack starting at #1725.Purpose of the underlying fix
Constructing a
StoreAppenderperforms an EOF back-scan over existing roll cycles. Before this fix, that scan left the appender holding the selected store, wires, mapped-file reservation and file descriptor even when the appender never wrote.If another appender subsequently rolled forward, the unused appender could keep an old
.cq4file open.FileUtil.removableRollFileCandidates(...)scans oldest-first and stops at the first open file, so the parked file also prevented all later files from being considered removable.releaseParkedStore()releases those resources after the scan and returns the appender to a never-written state. Its first write reacquires the current cycle. The relatednormaliseEOFs0(cycle())change is required because the rawcyclefield is reset when the parked store is released.Regression evidence
The tests do demonstrate the failure, rather than only execute the changed code:
AppenderReleasesParkedStoreTest.parkedAppenderDoesNotPinAnOldCyclefails if only thereleaseParkedStore()call is removed. It expects cycles 0 and 1 to be removable but receives only cycle 0, showing that cycle 1 remains pinned.NormaliseEOFsTest.freshlyConstructedAppenderNormalisesWithoutWritingfails ifnormaliseEOFs0(cycle())is changed back to the rawcyclefield. ThenormalisedEOFsTowatermark remains at 0.CreateAtIndexTestandNotCompleteTestaccount for the expectednormalisedEOFsTometadata written on the first append after reacquisition; they are compatibility adjustments, not the primary regression proof.All four touched test classes pass at this PR head.
Remaining test limitation
The file-pinning regression test relies on
/proc/self/fdand is skipped on unsupported operating systems. If this behaviour is changed again, add a portable lifecycle assertion that a fresh appender on a non-empty queue hascurrentFile() == nulluntil its first write, then has the current roll file afterward. Also assert exactly onenormalisedEOFsTorecord rather than allowing dump cleanup to remove duplicates.