perf: lazy memory architecture — reduce per-pattern overhead 5-7x (#158)

[kolkov]

Summary

Adopts Rust regex Cache separation model to dramatically reduce memory per compiled pattern. Addresses #158 where Coraza WAF (900 OWASP CRS patterns) saw 16x memory overhead vs stdlib.

Changes:

• PikeVM lazy init — NewPikeVMLazy() defers thread queues/sparse set allocation to first search (~10 KB saved per unused PikeVM)
• Shared DFA PikeVM — SetPikeVM() eliminates duplicate PikeVM per DFA (~15-20 KB per DFA pattern)
• Deferred SearchState — removed eager allocation at compile time (~15-50 KB per pattern)
• Strategy-aware caches — newSearchState() only allocates caches needed by active strategy (30-70% fewer allocations)
• DFA initCap 64→16 — smaller initial maps/slices (~3 MB across 900 patterns)
• CI — skip benchmark workflow for docs-only PRs

Architecture (Rust model):

• Compiled regex = immutable, shareable
• Search state = mutable, per-thread, lazy
• Strategy drives what gets allocated

Test plan

• go test ./... — all packages pass
• golangci-lint run — no new issues
• gofmt — all modified files clean
• Quick benchmarks — no regression on BenchmarkFind
• CI: tests (Linux/macOS/Windows) + benchmark comparison
• CI: race detector (Linux)
• regex-bench: Go + Rust comparison on AMD EPYC

Fixes #158

[codecov]

Codecov Report

❌ Patch coverage is 83.33333% with 8 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
nfa/pikevm.go	66.66%	5 Missing and 1 partial ⚠️
dfa/lazy/builder.go	0.00%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[github-actions]

Benchmark Comparison

Comparing main → PR #160

Summary: geomean 81.49n 76.71n -5.87%

⚠️ Potential regressions detected:

geomean                               ³                +0.00%               ³
geomean                               ³                +0.00%               ³
geomean                         ³                +0.00%               ³
geomean                         ³                +0.00%               ³
MatchAnchoredLiteral/no_match_prefix-4                  2.581n ± ∞ ¹     2.604n ± ∞ ¹     +0.89% (p=0.008 n=5)
ASCIIOptimization_Issue79/short_WithoutASCII-4          302.4n ± ∞ ¹     326.6n ± ∞ ¹     +8.00% (p=0.008 n=5)
DNA_VsStdlib/stdlib/dna_4-4                             60.14m ± ∞ ¹     63.03m ± ∞ ¹     +4.80% (p=0.016 n=5)
LangArenaLogParser/ips-4                                48.13µ ± ∞ ¹     48.36µ ± ∞ ¹     +0.48% (p=0.032 n=5)
BranchDispatch_Stdlib/Digits-4                          132.1n ± ∞ ¹     132.5n ± ∞ ¹     +0.30% (p=0.048 n=5)
BranchDispatch_Stdlib/NoMatch-4                         78.14n ± ∞ ¹     78.97n ± ∞ ¹     +1.06% (p=0.008 n=5)

Full results available in workflow artifacts. CI runners have ~10-20% variance.
For accurate benchmarks, run locally: ./scripts/bench.sh --compare