Repository navigation
Unify exist-core-jmh and exist-indexes-jmh maven wiring; fix broken Lucene/conf.xml benchmarks - #6646
Merged
Merged
Conversation
…chmarks
exist-core-jmh's conf.xml was a hand-maintained "verbatim copy of
exist-core/src/test/resources-filtered/conf.xml" with no Lucene
module/index registered. LucenePhraseQueryBenchmark and
UtilExpandHighlightingBenchmark worked around this at runtime with an
ensureExistHome() that tried to borrow
extensions/indexes/lucene/src/test/resources-filtered/conf.xml —
except that path is never a real file, it's the *input* to that
module's own Maven-time XSLT codegen (conf-fixture.xsl ->
target/generated-test-resources/conf.xml), only materialized when the
lucene extension module itself is built. Since ci-benchmarks.yml only
builds `-pl exist-core-jmh -am`, that directory never exists, so both
benchmarks silently fell back to exist-core-jmh's Lucene-less conf.xml
-> NullPointerException ("index" is null) in one, undefined `ft:`
prefix (XPST0081) in the other, on every single iteration.
Give exist-core-jmh its own generated conf.xml instead, using the same
schema/generate-conf-fixture.xsl mechanism the lucene extension (and
~35 other test fixtures) already use, with keep-indexes=lucene-index
and keep-modules including the lucene module (see the new
src/main/resources-filtered/conf-fixture.xsl). Since this module's
benchmarks are main-scope (packaged into the shaded benchmarks jar,
not test-scoped), the codegen runs via its own xml-maven-plugin
execution bound to generate-resources rather than the standard
auto-activating test-resources profile — same opt-out pattern
extensions/modules/expathrepo already uses for its own non-standard
output directory.
This also surfaced a pre-existing, previously-unreachable correctness
bug in LucenePhraseQueryBenchmark: `matches = (i % matchEvery) == 1`
is always false when matchEvery=1 (i % 1 is always 0), so the
"all docs match" parameterization never actually matched any document
by the intended construction - it happened to "pass" only because
doc #1's non-matching placeholder text ("datum" + 1) coincidentally
spelled the literal search phrase. Fixed to
`((i - 1) % matchEvery) == 0`.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
exist-core-jmh ran its benchmarks via a bare `java -jar`, while exist-indexes-jmh wraps the same invocation in `mvn exec:exec`. The module also carried a vestigial uberjar.name=benchmarks property that nothing referenced (maven-shade-plugin's finalName duplicated the same expression separately, hardcoded). uberjar.name is now a real property, shared between the shade plugin's finalName and the new exec-maven-plugin execution - same pattern exist-indexes-jmh already uses. Adds a benchmark.args property (default: ArrowOperatorBenchmark -prof gc) overridable via -Dbenchmark.args, matching exist-indexes-jmh's convention. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
exist-indexes-jmh generated its conf.xml by transforming extensions/indexes/indexes-integration-tests/src/test/resources-filtered/conf.xml via its own src/main/xslt/conf-jmh.xslt. That source file was never a real static file, only the *input* to indexes-integration-tests' own Maven-time fixture codegen - and once that module switched to the canonical-fixture mechanism (PR eXist-db#6531), its resources-filtered directory holds only a conf-fixture.xsl, no static conf.xml. Result: xml-maven-plugin's transform matched zero input files ("[WARNING] No files found for transformation"), succeeded anyway (empty transformation sets aren't a build failure), and produced no conf.xml at all - the shaded benchmarks jar shipped with none. Every benchmark method across all 5 classes failed instantly on setUp(): `DatabaseConfigurationException: Unable to read configuration file at .../exist-indexes-jmh/etc/conf.xml`. No CI run had exercised this module's benchmarks since eXist-db#6531 merged (the ci-benchmarks.yml run that would have hit it was independently cancelled by the 90-minute job timeout during exist-core-jmh's step first). Fixed by sourcing directly from canonical (exist-distribution/src/main/config/conf.xml) via a new src/main/resources-filtered/conf-fixture.xsl, using the same schema/generate-conf-fixture.xsl mechanism as ~35 other fixtures in the repo (and the one exist-core-jmh now also uses), instead of depending on another module's private fixture input a second time. Removes the now-obsolete conf-jmh.xslt. Verified all 5 benchmark classes (Ngram/RangeEq/RangeFieldEq/Lucene/ GeneralComparisonWhereClauseBenchmark) run clean via `mvn exec:exec`. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ification - exist-core-jmh/README.md now documents `mvn exec:exec` as the primary invocation (matching exist-indexes-jmh), with `java -jar` kept as a secondary option; notes conf.xml is now generated from canonical. - exist-indexes-jmh/README.md's layout section and conf.xml paragraph updated for the new canonical-sourced fixture (replacing the removed conf-jmh.xslt / indexes-integration-tests dependency). - ci-benchmarks.yml: exist-core-jmh's run step switches to exec:exec (matching exist-indexes-jmh); job timeout bumped 90 -> 180 minutes, since benchmarks that previously failed instantly (the bugs fixed in prior commits) now run to their full configured duration - this is a report-only weekly/manual job that blocks no PR, so a longer timeout only costs CI minutes on a schedule. - Removed a stale "shaded benchmark jar trips a log4j2 caller-class assertion when booting a BrokerPool" javadoc caveat from AxisBenchmark and ArrowOperatorBenchmark, along with the unshaded classpath workaround recipe it prescribed. Verified stale two ways: a real prior CI run completed AxisBenchmark successfully via the shaded `java -jar` invocation, and all 4 ArrowOperatorBenchmark methods ran clean locally via the shaded jar with no errors. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…es not need conf.xml It uses raw Lucene IndexWriter/IndexSearcher APIs directly, never boots a BrokerPool or reads conf.xml, so it doesn't benefit from lucene-index registration the way LucenePhraseQueryBenchmark and UtilExpandHighlightingBenchmark do. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
duncdrum
force-pushed
the
dp-jmh-conf-codegen
branch
from
August 18, 2026 09:07
0d4072f to
a68d4d6
Compare
…est allowlist Its hand-maintained conf.xml (missing schemaVersion) was deleted and replaced by one generated from canonical, which carries schemaVersion like every other canonical-derived fixture. The audit test's hardcoded REMAINING_WITHOUT_VERSION set needs to track that. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dizzzz
approved these changes
Aug 18, 2026
reinhapa
approved these changes
Aug 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to my review comment on #6608: unifies the Maven wiring between
exist-core-jmh(java -jar) andexist-indexes-jmh(mvn exec:exec), now that #6531's conf.xml codegen mechanism has merged and unblocks it.Investigating why
ci-benchmarks.yml's last run produced a 4.3 GB log and hit its timeout also turned up two real bugs:exist-core-jmh's two Lucene benchmarks were silently falling back to a Lucene-lessconf.xmlat runtime, since they reached into another module's private, non-existent build artifact.exist-indexes-jmh's conf.xml codegen was silently producing no conf.xml at all (broken by Generate test conf.xml/controller-config.xml fixtures from canonical at build time #6531 changing its upstream fixture source) — every benchmark in that module has been failing instantly since Generate test conf.xml/controller-config.xml fixtures from canonical at build time #6531 merged, undetected because CI never reached that step.(PR #6645, separate, fixes an unrelated log-bloat bug found in the same investigation.)
What changed
conf.xmlfrom canonical via the standardschema/generate-conf-fixture.xslcodegen, instead of reaching into another module's fixtures.ensureExistHome()workarounds; fixed aLucenePhraseQueryBenchmarkcorrectness bug the NPE had been masking (matchEvery=1modulo formula).exist-core-jmhnow runs viamvn exec:exec, matchingexist-indexes-jmh.AxisBenchmark/ArrowOperatorBenchmark— verified stale by a real prior CI run plus local testing.Test plan
-Pperf-testsconf.xmlregisters lucene/ngram/range indexes and ships in both shaded jarsmvn exec:exec🤖 Generated with Claude Code