fix(#1441): formalize relation generation publication

[justinjoy]

Summary

• define relation identity, view-generation, and storage-generation publication semantics
• instrument direct mutation, COW, rollback, sorting, TDD, and shared-view paths with exact-once generation updates
• make shared-view metadata and allocation-failure paths transactional, including graph/compound metadata
• preserve the legacy col_rel_t prefix layout and document the identity allocator atomic sites

Rebase onto main (6fac60f)

The two stacked commits this branch used to carry (#1384 pin primary arrangements during joins, #1435 protect materialization cache lifetimes) are on main as identical patches and were dropped; the branch is now the two commits for #1441 and #1438.

The #1441 commit conflicted with #1433 (feat: admit retained relation growth, merged as ddce4fd/d3ef048b) in relation.c, eval.c and eval_serial.c. #1433 admits retained-EDB growth through the memory governor on the same paths that this branch rewrote around private-storage preparation and generation publication. Resolution:

• col_rel_append_row growth keeps this branch's prepare → fill → col_rel_publish_resize → touch_storage flow and folds Admit retained EDB relation growth under the memory budget #1433's admission around it: the governor reservation is taken before any storage is allocated, committed (col_rel_publish_retained_reservation) immediately before the storage swap, and rolled back together with the private buffers on any failure, so the relation and its previous token stay untouched on the error path. Admit retained EDB relation growth under the memory budget #1433's col_rel_grow_heap_admitted is superseded by this and removed.
• col_rel_set_schema publishes the retained reservation (Admit retained EDB relation growth under the memory budget #1433) and then bumps the view generation (this branch).
• The evaluator's timestamp-enable sites keep Admit retained EDB relation growth under the memory budget #1433's col_rel_enable_timestamps(); the touch_storage this branch added at those call sites now lives inside that helper on both allocation paths.
• tests/test_simd_row_cmp.c's col_rel_t mirror gained the fields main added between mem_ledger and row_scratch (Admit retained EDB relation growth under the memory budget #1433's memory_governor / retained_reservation / retained_reserved_bytes, Instrument TDD memory usage and establish baseline #1380's ledger_ts_bytes); without them the generation suffix this branch relies on sat at the wrong offsets and test 9 failed.

The #1438 commit applied cleanly.

CI on the rebased head (7a6c5fa) and fixes

• Sanitizers / macos-latest / clang → relation_generations FAIL. The four allocation-failure cases need ld --wrap, which tests/meson.build only enables on Linux; on other hosts the test counted them as failures ("allocator-failure wrappers unavailable"). They now report as skipped and the generation contract cases still run, so the test passes on macOS and Windows. Folded into the Define relation generation contract and mutation instrumentation #1441 commit.
• Build / ubuntu-latest / gcc → Check binary size: +11,635 B over the 262,305 B baseline. main itself has grown to 268,199 B since ci: ratchet text baseline for memory admission #1437 set the baseline (Define arrangement and cache lifetime/rebuild policy #1384, Admit retained EDB relation growth under the memory budget #1433, Add pin-aware lifetime and generation validation for cached relation views #1435), and this branch adds 5,741 B on top (273,940 B measured by CI). Added ci: ratchet text baseline for relation generations re-anchoring at 273,940 B, following e88a70a and ci: ratchet text baseline for memory admission #1437.

Validation (after rebase)

• build of both PRs' test sets; arrangement_cache_reuse, arrangement_lru_eviction, delta_retraction, diff_join, generation_cache, mat_cache_lifetime, relation_generations, simd_row_cmp, session, relation_append, compaction, delta_timestamp, memory_governor, mem_instrumentation_{default,nofusion}, tdd_decision_stats, tdd_multiworker, io_adapter, k_fusion_memory, arrangement, col_rel_deep_copy, recursive_scc_default: all pass
• threading_doc: 94/94 atomic sites; clang_tidy_ratchet: pass
• ASan/UBSan: 15 of the above (all relation/cache/generation tests plus tdd_multiworker) clean
• uncrustify --check and git diff --check: clean

Scope

Cache consumers and borrowed-storage lifetime integration remain in #1384/#1435 (both merged). No merge is requested by this PR creation.

Binary size gate

CI measured this branch at 272,028 B of .text (ubuntu-latest / gcc) against the committed 262,305 B baseline (+9,723 B). Of that, +5,894 B is main itself (268,199 B at 6fac60f: #1384, #1433, #1435), which is over the 5,120 B budget on its own; this branch adds +3,829 B, inside the budget. #1461 re-anchors the baseline on main at 268,199 B; the last commit here sets the same value so the gate passes before that lands, and the two merge cleanly either way.

CI fixes on the rebased head

• TSan / ubuntu-latest / gcc reported a data race in wl_columnar_relation_generation_advance from diff_join and tdd_decision_stats: the keyed, semijoin and cross-join parallel fill workers write disjoint row ranges of one fresh output through col_rel_set(), which now bumps view_generation per cell, so W workers raced on the same counter. dcf77187 adds col_rel_set_raw() (cell write without the shared-view detach and without generation publication; heap-owned, non-shared relations only) and uses it in col_join_write_pair_at(), col_semijoin_fill_worker_fn() and col_join_cross_fill_worker_fn(). The coordinator already publishes the view generation exactly once around each parallel fill. Verified with -Db_sanitize=thread -Dthreads=posix (the CI configuration): diff_join, tdd_decision_stats, tdd_decision_stats_nofusion, tdd_multiworker run with zero TSan warnings.
• Build / windows-latest / msvc failed twice over: relation.c included <stdatomic.h> and used _Atomic with the non-_explicit calls (MSVC has no C11 atomics in its default C mode), and internal.h placed the col_rel_t layout _Static_assert(offsetof(...)) checks at file scope unconditionally (C2059/C2143 on every assert). 65d597d2 moves the identity allocator onto wl_atomic_u64 and the atomic_*_explicit shims mem_ledger.h already provides for MSVC and C11 (relaxed ordering; the counter only has to hand out distinct values), and guards the asserts with __STDC_VERSION__ >= 201112L exactly as diff_trace.c does. docs/THREADING.md rows updated; the audited site count stays at 94.

Rebase onto main (#1434 / #1444 / #1462)

main gained #1434 (admit retained relation ownership transitions, reconcile COW ledger exactly once) and #1444 on the same growth paths this branch rewrote, so the #1441 commit conflicted in relation.c and session.c. The merged result keeps both contracts:

• Heap-owned growth (col_rel_append_row, col_rel_append_all) is this branch's transaction -- prepare private buffers, then col_rel_publish_resize -- with Admit retained relation ownership transitions for COW and arena promotion #1434's admission folded in for retained relations: the governor reservation is taken before any buffer is allocated, committed immediately before publication, and rolled back together with the private buffers on any failure. Admit retained relation ownership transitions for COW and arena promotion #1434's col_rel_grow_heap_admitted is superseded by this and removed.
• Ownership transitions (shared view or arena-backed relation that must grow) use Admit retained relation ownership transitions for COW and arena promotion #1434's transactional col_rel_grow_owned_transition, which now publishes the storage generation on success; col_rel_promote_arena_admitted (the session.c side of the conflict) inherits that.
• COW goes through Admit retained relation ownership transitions for COW and arena promotion #1434's admitted col_rel_cow_unshare everywhere (it publishes one storage epoch when it privatizes). It stays exported because col_rel_set() and the tests use it. The admission-free _raw variant is gone: col_rel_radix_sort now detaches through the admitted path and, if the sort cannot allocate, rolls the detach back completely (columns, ownership flags, ledger and both generations); the sort wrappers no longer pre-detach, which is what made that rollback impossible.
• col_rel_compact shrinks (and, for a shared view, privatizes) in one prepare/publish transaction; Admit retained relation ownership transitions for COW and arena promotion #1434's retained-reservation handling around it is unchanged.
• col_rel_append_all keeps Admit retained relation ownership transitions for COW and arena promotion #1434's arena-preserving growth for unmanaged arena relations and this branch's transactional type metadata (prepared before growth, installed after growth and COW; every failure releases it).
• col_rel_install_shared_view: a destination that already tracks timestamps keeps a zeroed array sized like the borrowed columns when the source has none; Admit retained relation ownership transitions for COW and arena promotion #1434's memory_admission_relation test relies on that tracking surviving installation and the later growth.
• Binary size gate: CI measures this head at 273,644 B of .text against the 268,199 B baseline (+5,445 B, budget 5,120 B). main itself is at 269,324 B (+1,125 B since the baseline: Admit retained relation ownership transitions for COW and arena promotion #1434, fix(#1434): admit retained relation ownership transitions #1444, CI: abi shell gates run against meson's 30s default and threading_doc now times out on Windows #1462) and this branch adds 4,320 B, inside the budget on its own; the last commit re-anchors the baseline at the measured value, following e88a70a, ci: ratchet text baseline for memory admission #1437 and 86d6314.

The relation_generations_oom target (Linux, -Db_lto=false + --wrap) is the one where allocation-failure injection actually fires; under the default LTO build the wrappers are inert. Verified on the rebased tree: relation_generations_oom, #1434's memory_admission_relation, both PRs' relation/cache/session tests (23 tests), threading_doc (94 sites), clang_tidy_ratchet, ASan/UBSan on the relation, admission and OOM tests, TSan (-Dthreads=posix) on diff_join, tdd_decision_stats, tdd_multiworker with zero warnings; uncrustify --check and git diff --check clean.

[justinjoy]

CI repair applied and validated.

• Rebased the PR scope to issue Define relation generation contract and mutation instrumentation #1441 only; the unrelated Add relation view generations for cache invalidation #1438 commit was removed from this branch.
• Kept relation_generations portable without Linux-only allocator wrapping; added Linux-only relation_generations_oom for the allocator-failure coverage.
• Normal and ASan/UBSan focused tests pass. Full normal suite: 346 passed, 14 expected skips.
• .text: 263856 bytes, baseline 262305, delta +1551 (within +5120); baseline unchanged.
• The PR remains open; no merge was performed.