Skip to content

Add end-to-end test for meshing above 4 GiB; tidy finalizeMesh - #131

Merged
bpowers merged 2 commits into
masterfrom
high-offset-mesh-test
Oct 21, 2026
Merged

bpowers merged 2 commits into
masterfrom
high-offset-mesh-test

Conversation

@emeryberger

@emeryberger emeryberger commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Follow-up to #129 (now merged), rebased onto master.

Summary

  • End-to-end regression test, //src:high-offset-mesh-test. Reserves an untouched 4 GiB placeholder so the next two spans start more than 4 GiB into the backing file, meshes them, then scavenges. It checks that:

    • the meshed object keeps its contents (on current master it reads back as an empty string instead of its ZZZ… contents);
    • after resetSpanMapping, writes to the old span don't show up in the placeholder. This check also fails on its own when only resetSpanMapping has the old 32-bit shift.

    It's a separate target because of the 4 GiB address-space reservation, which uses almost no real memory. make test and both Bazel CI jobs run it alongside unit-tests. The CMake job doesn't run it.

  • Cleanup. Merges the __APPLE__ and Linux branches of finalizeMesh, which were identical apart from hard_assert(_fd >= 0); that assert now runs on all platforms. Also widens span.length before the shift in the MADV_DONTDUMP call, which is dead code today since kAdviseDump = false.

Testing

  • CI (run 39354360163): all jobs pass. The new test passes on Linux (4K pages) and macOS (16K pages), in both debug and optimized builds.
  • Locally (macOS arm64): passes with the fix; fails at the s2 content check with master's meshable_arena.{h,cc}; fails at the alias check when only resetSpanMapping has the old shift.

Questions for review

  • Is a 4 GiB arena reservation acceptable on all CI and developer machines? It needs kArenaSize > 4 GiB plus the spans after it. That holds for 64 GB on Linux and 32 GB on macOS.
  • Is it OK for the test to write to freed arena pages after scavenge (it zeroes them again afterward)?

🤖 Generated with Claude Code

emeryberger and others added 2 commits October 21, 2026 10:33
The Apple and non-Apple branches of finalizeMesh were identical apart
from an fd assertion, which holds on all platforms. Also widen
span.length before shifting in the (currently disabled) kAdviseDump
path so spans >= 4 GiB are not truncated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reserves an untouched 4 GiB placeholder so the next two spans start past
the 4 GiB mark of the backing file, meshes them, then scavenges. Checks
that meshed objects keep their contents and that resetting the meshed
span's mapping does not alias the placeholder. Without the off_t fix in
finalizeMesh, the meshed object reads back as zeros; without the fix in
resetSpanMapping, writes to the reset span appear in the placeholder.

It is a separate target (high-offset-mesh-test) because it reserves 4 GiB
of arena address space; make test and CI run it alongside unit-tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@emeryberger
emeryberger force-pushed the high-offset-mesh-test branch from da49765 to 8024ae0 Compare October 21, 2026 14:33
@emeryberger
emeryberger marked this pull request as ready for review October 21, 2026 14:54
@bpowers
bpowers merged commit fa1542b into master Oct 21, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants