xenopsd/xc: sync domain.ml with Xen 4.21's Xenctrl ABI additions - #7197
xenopsd/xc: sync domain.ml with Xen 4.21's Xenctrl ABI additions#7197d3athjest3r wants to merge 3 commits into
Conversation
Yes, which is why the CI is failing. Would you mind submitting PRs to xenctrl? xs-opam should pick up changes to xenctrl's master branch automatically: https://github.com/xapi-project/xs-opam/blob/master/packages/xenctrl/xenctrl.master/opam |
domain_create_flag here is a mirror of Xenctrl.domain_create_flag, which
requires listing every constructor of the original. Xen added
CDF_TRAP_UNMAPPED_ACCESSES (currently Arm-only) in:
980aff4e8fcd xen/arm: Add way to disable traps on accesses to
unmapped addresses (Edgar E. Iglesias, 2025-06-16)
ab02a120c0a5 tools/ocaml: Update bindings for
CDF_TRAP_UNMAPPED_ACCESSES (same day)
Signed-off-by: Julian Vetter <julian.vetter@vates.tech>
domctl_create_config here is a mirror of Xenctrl.domctl_create_config.
Xen's OCaml bindings gained the altp2m_count field in:
5699554de9a9 tools/ocaml: Add altp2m_count parameter
(Petr Beneš, 2025-08-25)
exposing the pre-existing altp2m.nr count (xen_domctl_createdomain),
alongside the existing altp2m_opts.
Signed-off-by: Julian Vetter <julian.vetter@vates.tech>
Thank you for the feedback. I have opened a PR in xenctrl: xapi-project/xenctrl#17. |
| ; (iommu, CDF_IOMMU) | ||
| ; (nested_virt, CDF_NESTED_VIRT) | ||
| ; (vpmu, CDF_VPMU) | ||
| ; (is_arm, CDF_TRAP_UNMAPPED_ACCESSES) |
There was a problem hiding this comment.
This is more than just syncing the ABI. It's also changing the flags passed to Xen, and ought to be in a commit of its own saying "Make ARM guests work" or so.
Unlike the others, this has no external configuration. Is it intentional?
There was a problem hiding this comment.
Thank you @andyhhp. You're right! I added now a platform key trap-unmapped-accesses to allow enabling or disabling it.
|
@minglumlu @BengangY Be aware that this change ups the Xapi baseline to Xen 4.21, and will need adjustments in the spec file. |
domain_create_flag's Xenctrl.domain_create_flag mirror already lists CDF_TRAP_UNMAPPED_ACCESSES, but the flag was never actually added to the flags list `make` builds, so it was never set for any domain. Without it, Xen traps guest accesses to unmapped addresses instead of letting the guest handle them directly. On Arm this can fire very early in the kernel's own boot code, before the guest has installed its own exception vector table, in which case the resulting injected exception is delivered at a fixed vector offset off a zero base, which itself is also unmapped, causing an infinite faulting loop. libxl already sets this flag for Arm domains (libxl__arch_domain_prepare_config() in tools/libs/light/libxl_arm.c always includes it in config->flags), which is why domains built via libxl are unaffected while domains built through xenopsd's direct Xenctrl.domain_create binding never got it. Confirmed by comparing the actual xen_domctl_createdomain contents both toolstacks send to xc_domain_create() for the same kernel/ramdisk/memory Arm domain config: xenopsd sent flags=0x3 (CDF_HVM|CDF_HAP), libxl sent flags=0x103 (adding CDF_TRAP_UNMAPPED_ACCESSES, bit 8). domain_create_flag is shared between the Arm and x86 xenopsd builds, and Xen's x86 arch_sanitise_domain_config() rejects domain creation (-EINVAL) if this flag is set on x86 (xen/arch/x86/domain.c). libxl_x86.c enforces the same restriction at the toolstack level, returning ERROR_FAIL if trap_unmapped_accesses is requested on x86. So, mirror what libxl is doing and add a trap-unmapped-accesses platform-data key, defaulting to on for Arm domains and off on x86. This gives a user who wants Arm guest accesses to unmapped addresses to not trap a way to opt out via platform:trap-unmapped-accesses=false. Signed-off-by: Julian Vetter <julian.vetter@vates.tech>
3 participants
ocaml/xenopsd/xc/domain.mlmirrorsXenctrl.domain_create_flagandXenctrl.domctl_create_config, both of which require listing every constructor/field of the real type. Xen 4.21 added items to these types that were missing fromdomain.ml, and in one case the mirror being incomplete masked a second issue: the corresponding flag was never actually being set anywhere.CDF_TRAP_UNMAPPED_ACCESSESondomain_create_flag(currently Arm-only):- xen.git 980aff4e8fcd "xen/arm: Add way to disable traps on
accesses to unmapped addresses" (Edgar E. Iglesias, 2025-06-16)
- xen.git ab02a120c0a5 "tools/ocaml: Update bindings for
CDF_TRAP_UNMAPPED_ACCESSES" (same day)
- xen.git 5699554de9a9 "tools/ocaml: Add altp2m_count parameter"
(Petr Beneš, 2025-08-25)
Both have been part of the public Xen ABI since before the 4.21.0-rc1 tag, so any build compiling xapi against Xen 4.21's actual OCaml Xenctrl bindings fails at this type check.
Once the mirror type-checked, I could actually run an Arm domain through xenopsd against a real Xen 4.21, which showed a second bug:
domain.mlnever setsCDF_TRAP_UNMAPPED_ACCESSESin the flags list thatmakepasses toXenctrl.domain_create, even though the type now lists it. Without it, Xen traps guest accesses to unmapped addresses instead of letting the guest handle them directly. On Arm this fires very early in the kernel's own boot code, before it has installed its own exception vector table, so the resulting injected exception lands at a fixed vector offset off a zero base, which itself is unmapped, and the guest immediately double-faults into an infinite loop.libxl already sets this flag unconditionally for Arm domains (
libxl__arch_domain_prepare_config()intools/libs/light/libxl_arm.c), which is why a domain built withxl createfor the exact same kernel/ramdisk/memory boots fine while the identical domain built through xenopsd'sXenctrl.domain_createnever did. Confirmed by comparing the actualxen_domctl_createdomaincontents both toolstacks send toxc_domain_create()for the same config: xenopsd sent flags=0x3 (CDF_HVM|CDF_HAP), libxl sent flags=0x103 (addingCDF_TRAP_UNMAPPED_ACCESSES, bit 8), which was the only difference between the two, and setting it here reproducibly fixes the guest boot hang.domain_create_flagis shared between the Arm and x86 xenopsd builds, and Xen's x86arch_sanitise_domain_config()rejects domain creation with -EINVAL if this flag is set on x86 (xen/arch/x86/domain.c), so it's gated on the domain'sarch_domainconfigrather than set unconditionally.Testing
Found and verified while building xapi/xenopsd for aarch64 against a freshly built Xen 4.21.1 (linking directly against Xen's own
tools/ocaml/libs/xcoutput). Confirmed the build fails without the first two commits and succeeds with them applied, with no other call sites needing changes (onlydomain.mlreferences either type).Confirmed the third commit by booting a plain Arm PVH domain end-to-end through xapi/xenopsd (same kernel+initramfs xapi's own dom0 uses): it hung every time with the flag unset (same crash signature) and boots cleanly and consistently once set, matching
xl create's own behaviour for an identical domain config.Side question: the xenctrl opam package CI pulls in (xapi-project/xenctrl, via xs-opam) also doesn't have these Xen additions. Would that need updating as well?