Skip to content

fix(arborist): skip normalization for canonical ASCII map keys - #10132

Open
jeffwidman wants to merge 1 commit into
npm:latestfrom
jeffwidman:perf/skip-ascii-key-normalization
Open

jeffwidman wants to merge 1 commit into
npm:latestfrom
jeffwidman:perf/skip-ascii-key-normalization

Conversation

@jeffwidman

@jeffwidman jeffwidman commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

Skip Unicode normalization and case folding for case-insensitive map keys that are already canonical ASCII.

Why

Loading a large workspace lockfile repeatedly refreshes named dependency edges. Those traversals perform millions of case-insensitive map lookups, mostly misses. Normalizing every lookup key is expensive even when the name contains only lowercase ASCII, digits, and punctuation.

This adds a cache-free fast path: ASCII strings without uppercase letters are unchanged by both NFKD normalization and toLowerCase(), so return them directly. Uppercase or non-ASCII strings retain the existing normalization path, including surrogate pairs and compatibility characters. Non-string keys retain their existing behavior.

No changes to map storage, key replacement, iteration order, or dependency resolution are intended. Tests cover case/Unicode-equivalent keys, compatibility characters, supplementary-plane characters, replacement/deletion, empty strings, undefined values, and non-string keys.

Performance

In an offline large-workspace native npm benchmark:

Flow Median CPU, before -> after Median wall, before -> after
Lockfile writer 6.389 -> 5.286 s (17.3% lower) 5.325 -> 4.124 s (22.6% lower)
Dry-run checker 5.712 -> 4.527 s (20.8% lower) 4.724 -> 3.621 s (23.3% lower)

The baseline already contains runtime adaptations of #10127, #10128, npm/hosted-git-info#347, and #10130; the candidate adds only this fast path. These are incremental native-command measurements, not savings versus unmodified npm or end-to-end Dependabot job savings. This branch itself is independently based on latest, not stacked on those PRs.

Method: Node 24.22.0, npm 11.17.0 / Arborist 9.8.0, networking disabled, 4 CPUs / 12 GB, one excluded warmup per arm and three alternating measured pairs. Writer uses --package-lock-only --force --ignore-scripts; checker uses --package-lock-only --dry-run=true --ignore-scripts; both additionally disable audit/funding and use --offline --timing.

All compared manifests, lockfiles (including the writer's hidden lockfile), configuration files, and normalized logs were identical. Separate instrumentation reduced NFKD calls during the writer from 9,650,439 to 1,054. On a public synthetic tree of 128 dependency-free packages, NFKD calls fell from 8,640 to zero.

A tiny local-dependency control over 21 pairs found no conclusive difference: paired-median CPU change +0.05%, wall -0.58%; bootstrap 95% intervals crossed zero. This is not a universal regression-free claim.

Validation

On this standalone branch, based on npm 12.2.0 / Arborist 10.0.3, using existing development dependencies:

  • node bin/npm-cli.js test --workspace=@npmcli/arborist --ignore-scripts -- --jobs=4 — passed all 59 test files; 100% statements, branches, functions, and lines.
  • eslint workspaces/arborist/lib/case-insensitive-map.js workspaces/arborist/test/case-insensitive-map.js — passed.
  • git diff --check — passed.

A separate differential harness rechecked the final publication source against the original map: all 1,114,112 code-point values, 10,000 mixed strings, and 20,000 mutation operations matched, including key collisions, iteration order, non-string keys, and clear/delete operations. No snapshots were changed.

The full npm CLI suite and Windows/macOS runs were not performed.

References

Related independent native-work reductions: #10127, #10128, npm/hosted-git-info#347, #10130.

The independent empty-leaf refresh optimization is proposed in #10131. Its savings overlap this fast path; that PR includes a separate incremental comparison with this optimization already applied. Neither PR depends on the other.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An active performance nit remains: each string-key operation creates a fresh RegExp in the hot path instead of safely reusing a shared instance.

1 open issue
What changed in this PR

Adds a cache-free fast path to Arborist’s case-insensitive map, reducing normalization work for canonical ASCII keys while preserving existing map behavior.

Changes:

  • Bypasses Unicode normalization and lowercasing for ASCII strings without uppercase letters.
  • Adds regression tests for Unicode equivalence, key replacement/deletion, empty strings, undefined values, and non-string keys.
File Description
workspaces/​arborist/​test/​case-insensitive-map.js Adds coverage for normalization and key-handling behavior.
workspaces/​arborist/​lib/​case-insensitive-map.js Adds the canonical ASCII fast path while retaining the existing normalization fallback.

Review effort: Max

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The tests would still pass if the fast path were removed; a regression test must verify that canonical ASCII hits and misses skip normalization and lowercasing while uppercase and non-ASCII inputs use the fallback.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Low severity Test that canonical ASCII lookups skip normalization and lowercasing

workspaces/​arborist/​lib/​case-insensitive-map.js:19

The new tests verify map behavior but would still pass if the fast path were removed. Add a regression test that counts normalize() and toLowerCase() calls, confirming that canonical ASCII hits and misses skip both while uppercase and non-ASCII inputs use the fallback.

1 resolved since last review

Review effort: Max

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are identified for the case-insensitive map optimization.

No open issues 🎉

Review effort: Max

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@jeffwidman
jeffwidman force-pushed the perf/skip-ascii-key-normalization branch from 82d0f9c to 58aec61 Compare November 11, 2026 03:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants