Manifest merge local public

[surendarchandra]

What does this change do?

Fixes #827

Stacked PR: this branch is built on #823 ("MTR: preserve a pre-existing component manifest"), and its first commit is that fix. Please review #823 first and this PR after it; only the second commit, "Components: support merge_local_manifest in the global manifest", is new here. Once #823 merges, this branch will be rebased so that only that commit remains.

This change adds a boolean manifest key, merge_local_manifest. When it is true in the global manifest, the global components are loaded first and the local manifest's components are appended after them.

Behaviour:

• merge_local_manifest defaults to false; manifests that do not use it, including those using read_local_manifest, behave exactly as before.
• Merged lists are de-duplicated (a URN in both manifests is loaded once, at its global position) and empty tokens such as ,, are ignored. A local manifest that resolves to the global file is not merged into itself.
• A present local manifest whose components cannot be read is a startup error, as for the global manifest; it is never silently dropped.
• The component separator is now a single constexpr shared by the splitter and the merge helper, and manifest_version_1_0 is made inline to remove a header-defined-variable ODR violation.
• MTR: when a preserved deployment manifest declares components, MTR writes a merge stub so those components stay loaded during the run and across restarts; otherwise it writes the plain redirect as before. The saved manifest is decoded as JSON (core JSON::PP) and only its top-level components string is used, as the server's reader does; a non-string value, or malformed JSON that mentions components, stops the run with the original kept.

Why is it needed?

read_local_manifest discards the global manifest reader before its components key is parsed, so the global and local component lists are mutually exclusive. A deployment cannot say "always load these components and let each instance add its own"; the global list must be duplicated in every local manifest, which is error-prone and prevents central management of components such as a keyring or audit log.

How was it tested?

• Added/updated MTR tests under mysql-test/ (Perl unit test mysql-test/lib/t/manifest.t)
• scripts/ci/mtr.sh passes locally
• Ran the relevant full suite (name it): full mysql-test suite, this branch vs the unpatched base; component_keyring_file

Details:

• gunit manifest-t (merged into merge_small_tests-t): 12 cases, 12/12 pass. Global + local composition, empty inputs, no trailing separator, cross-side and same-side duplicates, local contained in global, empty tokens, merge_local_manifest absent/present, and read_local_manifest unchanged.
• mysql-test/lib/t/manifest.t: 213 Test::More assertions (185 from MTR: preserve a pre-existing component manifest #823), all pass. Adds merge-stub creation, fallback to the plain stub when there is no components key, an unparseable components value failing loudly with the original intact, crash recovery with the merge stub on disk, and JSON handling: a nested components key is ignored, an escaped key (\u0063omponents) is recognised, the value is re-encoded, and non-string values and malformed JSON are rejected.
• component_keyring_file suite: all 38 tests successful (37 tests + shutdown report; 22 skipped: 21 need --big-test, 1 is Windows-only).
• End-to-end: with bin/mysqld.my = {"components":"file://component_keyring_file"}, main.1st and component_keyring_file.encrypt_explicit --big-test pass with component_keyring_file loaded through the merge stub, and the manifest is unchanged afterwards (same inode, mode, size, mtime and checksum; no .mtr_saved/.mtr_stub left, only the persistent .mtr_lock). (--nowarnings was used because the globally loaded keyring has no keyring configuration in this setup and logs an initialisation error.)
• Full mysql-test suite (Debug, --parallel=16 --retry=0, NDB skipped) on this branch vs the unpatched base: 6/7124 tests failing vs 8/7123 on the base. Five failures are common to both runs; the only failure seen just with the patch, perfschema.system_events_component (it loads its component with INSTALL COMPONENT, not through a manifest), passes 3/3 in isolation on both builds, and the three base-only failures passed with the patch. Zero patch-attributable regressions.

Contributor checklist

• Code is formatted (scripts/ci/format.sh)
• Commits are focused with descriptive messages

AI assistance

• I did not use AI assistance for this contribution
• I used AI assistance for this contribution

If AI assistance was used, describe the tool(s) and extent of use:

An AI coding assistant was used to draft parts of the implementation, tests and this description. All changes were reviewed, built and tested by the author, who takes responsibility for them.

Areas touched

components / manifest (include/manifest.h, sql/sql_component.cc), mysql-test harness (mysql-test/lib/My/Manifest.pm), tests (unittest/gunit/manifest-t.cc, unittest/gunit/CMakeLists.txt, mysql-test/lib/t/manifest.t).

---

This contribution is submitted under the corporate Oracle Contributor Agreement that covers the author's submissions to the MySQL project.

[github-actions]

Found one correctness issue: the saved-manifest regex can select the wrong JSON field or miss an escaped key, silently changing the components loaded during MTR. See the inline comment for examples and a suggested fix.

Reviewed all 10 changed files statically. PR-provided code and tests were not executed, as required by this workflow; CI check data was unavailable.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• ab.chatgpt.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"

See Network Configuration for more information.

Generated by Codex PR Review for #826 · codex · gpt60 · 187 AIC · ⌖ 23.8 AIC · ⊞ 13.9K
_{Comment /codex to run again}

[github-actions]

Found two correctness issues: unreadable local manifests are silently omitted during merging, and MTR's generated stub overrides existing read_local_manifest precedence. Both are detailed inline with suggested regression coverage.

Static review of the PR diff and affected callers only; PR code and tests were not executed. Existing review threads were unavailable under the access policy.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• ab.chatgpt.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"

See Network Configuration for more information.

Generated by Codex PR Review for #826 · codex · gpt60 · 196.5 AIC · ⌖ 24 AIC · ⊞ 13.9K
_{Comment /codex to run again}

[github-actions]

Found two correctness issues: merge-enabled startup skips deduplication when the local manifest is absent or zero bytes, and MTR's JSON conversion can silently change the selected components when keys repeat. Both findings include inline examples and suggested regression coverage.

Reviewed the changes and relevant callers without executing PR code or tests. Existing review threads were unavailable under the GitHub integrity policy, so duplicate-comment checking was limited.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• ab.chatgpt.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"

See Network Configuration for more information.

Generated by Codex PR Review for #826 · codex · gpt60 · 239.5 AIC · ⌖ 24.2 AIC · ⊞ 13.9K
_{Comment /codex to run again}

[github-actions]

Reviewed the changes at 2198f6d3f7d59cfbe7057766fd2e9989de350cea. Found one correctness issue: merge mode silently drops a local manifest that cannot be opened, allowing startup with an incomplete component set. See the inline finding for the suggested fix and regression coverage.

Static review only; PR code and tests were not executed. Existing review threads were unavailable because the read tool's integrity policy blocked access.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• ab.chatgpt.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"

See Network Configuration for more information.

Generated by Codex PR Review for #826 · codex · gpt60 · 170.2 AIC · ⌖ 24 AIC · ⊞ 13.9K
_{Comment /codex to run again}

[github-actions]

No high-confidence defects found in the reviewed changes. Reviewed global/local component merging, MTR manifest preservation and cleanup, and the added test coverage.

Validation was static only; PR code and tests were not executed. Existing review threads were unavailable under the GitHub read integrity policy.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• ab.chatgpt.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"

See Network Configuration for more information.

Generated by Codex PR Review for #826 · codex · gpt60 · 210 AIC · ⌖ 23.6 AIC · ⊞ 13.9K
_{Comment /codex to run again}