Repository navigation
Conversation
…ogram The file loader runs on the worker pool, and what it allocates there (resolutions, include reasons, metadata) went to the workers' own arenas in the language server and the API, which never set ProgramOptions::regions (watch mode does): every full build left about 25 KiB of a 40-module project's loader data in thread arenas for the life of the process, which notes/lifecycle-leaks.md read as part of the config load. Project::create_program now gives every full build loader regions, held by the program owner and dropped last; a version reused in place inherits them through its configuration, as it does the base region, and the fallback of an update that cannot reuse the program is built by the project with regions of its own (Program::rebuild_options, which update_program uses too). ProgramRegions::new takes the first chunk size (256 KiB here, 1 MiB for the watcher). Measured with tools/lsp-mem/lifecycle_leaks.py (PR 670), 400 cycles: thread arena bytes used per cycle 46.4 -> 18.8 KiB for api-snapshot and 46.4 -> 15.7 KiB for lsp-reload; api-update unchanged at 0. What remains is the config load (PR 677) and the package.json cache entries that enter_table_owner routes to the thread arena.
The 41 KiB per full build is mostly the file loader's worker-side resolutions: the language server and the API never set ProgramOptions::regions, so every full build left them in thread arenas.
Performance change vs mainWorkflow run · main Clef gate: Clef estimated 49.0% wall-time, 74.8% peak-RSS, and 50.8% instruction-count impact probability. Release builds, default checker count;
Negative means the PR used less; positive means it used more. Deltas are medians of paired PR/main ratios for wall time and instructions. Shared CI machines and parallel scheduling add noise to small changes. |
Merging this PR will improve performance by 12.1%1 warning
Performance Changes
Tip Curious why performance improved? Comment Comparing Footnotes
|
the bigger of the two leaks #670 found. the ~41 KiB per full build that #677 couldnt explain is the file loader's worker-side data, and the lsp never gave it a region
why
new_programruns the file loader on the rayon pool. watch mode gives each programProgramOptions::regions(one per thread) so what the loader allocates on workers is freed with the program. the lsp and the api never set it, so every full build (createSnapshot, a tsconfig change, an edit that cant be reused in place) left that data in the workers' own arenas foreverwhat changed
Project::create_programgives every full build loader regions (256 KiB first chunk per thread) andprogramOwnerholds them, dropped last. a version reused in place inherits the old build's, like the base regionreuse_programand builds the fallback itself with fresh regions through a newProgram::rebuild_options, whichupdate_programnow uses too. nothing changes for watch or build modeProgramRegions::new(first_chunk), default stays 1 MiB for the watcher,allocated_bytesfor the region loghow i found it
a temporary per-allocation backtrace into thread arenas, 50 vs 200 api-snapshot cycles, not committed. per cycle on main: 23.5 KiB from
filesparser/prefetch_resolutions/create_resolved_moduleon workers, 7.8 KiB of package.json cache entries that go to the thread arena on purpose, 3.2 KiB the config load (#677)numbers
400 cycles, 50 warmup,
--splitpoison 200 cycles: 0 wrong answers, regions balanced. whats left per full build is the config load (#677) and ~8 KiB of package.json cache entries, which is an arena routing change and not this PR
tradeoff
one region per worker thread that touched a full build, 256 KiB reserved each (18 threads here, 4.5 MiB reserved per program chain, resident only as used). versions reused in place keep the loader regions alive like they keep the base region. touches the same two files as #677 and the return types #668 changes, so whichever lands second rebases with a trivial conflict