Sync master to feature/sxm-v3 - #7236
Merged
LunfanZhang merged 204 commits intoAug 25, 2026
Merged
Conversation
Signed-off-by: Lin Liu <lin.liu01@citrix.com>
- Update all extauth configuration into domain_info - config_winbind_daemon take domain_info as argument - persist_extauth_config take domain_info as argument Signed-off-by: Lin Liu <lin.liu01@citrix.com>
This API can switch between ldap and ldaps so existing users can switch without rejoin domain This API would - Perform proper checks - Call this API to all hosts - Restore to previous states if any host set failed - Set winbind configuration with new ldaps and restart daemon Besides, this commit also do following refines - Move kdcs_of_domain etc to head of file to share with others - Setup DomainInfo module for encapsulation - Drop exec_with_new_task with current context Signed-off-by: Lin Liu <lin.liu01@citrix.com>
- add host-external-auth-set-ldaps for debug purpose - use find_opt to reduce backtrace - use Listext.List.try_map_collect to reduce indent and improve perf - some wording refine Co-authored-by: Pau Ruiz Safont <psafont@users.noreply.github.com> Signed-off-by: Lin Liu <lin.liu01@citrix.com>
This API can switch between ldap and ldaps so existing users can switch without rejoin domain This API would - Perform proper checks - Call this API to all hosts - Restore to previous states if any host set failed - Set winbind configuration with new ldaps and restart daemon Besides, this commit also do following refines - Move kdcs_of_domain etc to head of file to share with others - Setup DomainInfo module for encapsulation - Drop exec_with_new_task with current context
`try_map_any f l` apply f on l and return - Ok if any `f e` succeed - Error list if all failed Signed-off-by: Lin Liu <lin.liu01@citrix.com>
- Define new errors for invalid certificate, pool and host level - Remove AD joined check as it was already done in extauth.ml - Move certificate to extauth_plugin_ADwinbind.ml so it can shared between set-ldaps and enable-ad - ping_domain use try_map_any to propagate the error details Signed-off-by: Lin Liu <lin.liu01@citrix.com>
External auth configure/write ops(enable, disable, set-ldaps) should be synced with read ops(auth-username-and-password, etc). Xapi/ocaml does not prefer read/write locks, so we just use classic mutex, presuming no much concurrent ops. At the same time, serialize_auth_service is introduced to skip the lock if performance is preferred Signed-off-by: Lin Liu <lin.liu01@citrix.com>
…ct#6966) External auth configure/write ops(enable, disable, set-ldaps) should be synced with read ops(auth-username-and-password, etc). Xapi/ocaml does not prefer read/write locks, so we just use classic mutex, presuming no much concurrent ops. At the same time, serialize_auth_service is introduced to skip the lock if performance is preferred
Select CA bundles to setup TLS connection to DC in following preference - /etc/trusted-certs/ca-bundle-ldaps.pem - /etc/trusted-certs/ca-bundle-general.pem - /etc/stunnel/certs (Legacy, will be removed with trusted certs) Raise proper error code when no certs exists Signed-off-by: Lin Liu <lin.liu01@citrix.com>
Select CA bundles to setup TLS connection to DC in following preference - /etc/trusted-certs/ca-bundle-ldaps.pem - /etc/trusted-certs/ca-bundle-general.pem - /etc/stunnel/certs (Legacy, will be removed with trusted certs) Raise proper error code when no certs exists
Signed-off-by: Lin Liu <lin.liu01@citrix.com>
During pool join, if both joining host and the pool joined the same domain, pre-check is passed and the joining host is permitted to join the pool. However, the ldaps status needs to be synced during pool join to keep pool consistent. Note: certificates will be synced during the pool join, so no need to worry about the ldaps certificates. Signed-off-by: Lin Liu <lin.liu01@citrix.com>
Signed-off-by: Lin Liu <lin.liu01@citrix.com>
…roject#6986) During pool join, if both joining host and the pool joined the same domain, pre-check is passed and the joining host is permitted to join the pool. However, the ldaps status needs to be synced during pool join to keep pool consistent. Note: certificates will be synced during the pool join, so no need to worry about the ldaps certificates.
Keep winbind configuration on failure by default is good for debug - Get configuration details that cause the fail - Can perform a manual retry Signed-off-by: Lin Liu <lin.liu01@citrix.com>
…oject#6990) Keep winbind configuration on failure by default is good for debug - Get configuration details that cause the fail - Can perform a manual retry
- Add `ldaps` as one of the trusted certs purpose - Bump up `external_auth_set_ldaps` API version according to master - Get trusted certs dir by consts instead of hardcode Signed-off-by: Lin Liu <lin.liu01@citrix.com>
- Add `ldaps` as one of the trusted certs purpose - Bump up `external_auth_set_ldaps` API version according to master - Get trusted certs dir by consts instead of hardcode
…ect#7211) Reinstate the VDI.resize_online API that was removed in 31b22cc (CA-262059), so that "xe vdi-resize online=true" can resize a VDI attached to a running VM. resize_online shares the offline resize storage path (SM vdi_resize); the online/offline distinction lives in xapi's allowed-operations checks, which permit resize_online on an attached VDI only when the SM backend advertises the VDI_RESIZE_ONLINE capability. It is then up to the backend to decide whether it can satisfy the request and to fail with an appropriate error if it cannot. - datamodel: mark resize_online as published again - message_forwarding: restore resize_online forwarding - xapi_vdi: resize_online implementation; revert resize live-permission change - tests: cover resize_online allowed/blocked by VDI_RESIZE_ONLINE feature and offline resize still blocked while attached
…i-project#7101) The description of how to install a VM through the Xen API didmention storage repositories but only in passing. This PR proposes to fully introduce them before moving on to the installation of VMs.
Signed-off-by: Ming Lu <ming.lu@cloud.com>
…dir (xapi-project#7204) If `STATE_DIR` exists but is not empty, then globs for it in the form of `STATE_DIR/*` and `STATE_DIR/*/vdi-uuid` will be passed in literal form, causing errors like: cat: /etc/xensource/static-vdis-test/*/vdi-uuid: No such file or directory Toggle nullglob instead, which turns the whole glob string empty and does not enter the body of the loop iterating over the matches.
Clean up the XAPI DB when the sysprep SR directory doesn't exist. This may happen in an upgrade scenario. The sysprep SR and the directory will be re-created when the first VM.sysprep is called. Signed-off-by: Ming Lu <ming.lu@cloud.com>
No matter the feature is on or off, cleaning up the unused VDIs is reasonable. Signed-off-by: Ming Lu <ming.lu@cloud.com>
…#7179) The patch fixes an issue that can happens if a VDI operation fails on a SR and keeps the lock. In such case the XAPI can be blocked during the startup. Here is the scenario: - Xenopsd and XAPI are restarted - Xenopsd is looking for existing VBDs and check with hypervisor the status. If it needs to be destroyed it pushes a message DP.destroy2 on org.xen.xapi.storage queue. - XAPI starts, nothing listens on the queue - At some point a thread is started and handle the message posted by Xenopsd. Plugins are not yet loaded so an error `No_storage_plugin_for_sr` is returned to Xenopsd. Xenopsd will retry in 5s. - XAPI continues to boot and sync its database: - sync_pbds: it is where SMAPI plugins are setup and now we can handle the message to destroy the VBD - And here there are more syncs: sync_pci_devices, sync_pif_params, ... - until sync_local_vdi_activations. The deadlock can happen here. If it takes more than 5s to reach the sync of VDIs, Xenopsd has time to post its DP.destroy2. The message is handled by XAPI, the lock for the VDI is taken and the SM plugin is called. We see in production that the operation never returns and so the lock is never released. When XAPI call sync on VDIs, it will sync the VDI that has the problem. And it will wait for the lock preventing the XAPI to start correctly. So this patch moves the registration of the queue (Storage_access.start) after the synchronisation of the XAPI database. Pending messages remains in the queue and the sync of VDIs asked by XAPI database update process during the start can be done. Of course the problem with the VDI is still there but XAPI can now start. Tracking-reference: XCPNG-3515
When bond is created, user can configure MTU on the bond network. The underlying physical NIC and PIF object MTU is handled properly. However, the original per-NIC bond slave network MTU is not changed. A corner case is seen that the unused bond slave network MTU may cause MTU mismatch when pool join: The pool has bond and vlan on the bond with MTU 9000, while the joining host has single-NIC and same vlan with MTU 1500. After pool join, during restart, the joining host is configured vlan MTU 9000 and physical NIC MTU 1500. In normal case, the joining host should sync vlan and bond with pool. But here the MTU dismatching leads to drop packets with pool master, then the joining host can't leave maintain mode. This PR prevent this case by sync the bond slave network MTU with bond network MTU.
Signed-off-by: Christian Pardillo Laursen <christian.pardillolaursen@citrix.com>
…api-project#7218) Clean up the XAPI DB when the sysprep SR directory doesn't exist. This may happen in an upgrade scenario. The sysprep SR and the directory will be re-created when the first VM.sysprep is called. Signed-off-by: Ming Lu <ming.lu@cloud.com>
…t#7209) When make_tar ova.xml, the tar file length is topically ``` tar header (512) + xml + zero_pad to 512 + end-of-archive marker (2 × 512) + padding to RECORDSIZE ``` In metadata_handler, the with_open_archive reads to zero_pad, then it skipps end-of-archive marker. The remaining padding is not consumed. In some corner case, the server (XAPI) might close the connection when the client was still sending the padding part. Then the client would get a EPIPE error. In this PR, content-length is used to drain the remaining bytes in metadata_handler.
XCP-ng has a need for persistent VDI metadata (metadata that's preserved across snapshots/migrations). Currently none of the VDI fields (`other_config`, `sm_config`, `tags`, etc.) persist across a migration. This is because the destination VDI is created by storage backends and then db-introduce'd into XAPI. Storage backends in turn consult only some of the `vdi_info` fields of the original VDI, preserving `name_label`, `name_description` and some other fields, but not the fields that allow flexible metadata to be stored by clients. This PR allows storage backends to 1) consult `vdi.tags` 2) modify `vdi.tags`. It then modifies the SMAPIv1 migrate code to copy over the tags from the source VDI to the destination after a migration. --- I've tested that this code preserves vdi tags after a migration, and this satisfies our need for persistent metadata. Opening as a draft, however, because I'm not sure if `tags` are the right place to store such metadata, and if the approach is the right one - thoughts?
domain_create_flag here is a mirror of Xenctrl.domain_create_flag, which
requires listing every constructor of the original. Xen added
CDF_TRAP_UNMAPPED_ACCESSES (currently Arm-only) in:
980aff4e8fcd xen/arm: Add way to disable traps on accesses to
unmapped addresses (Edgar E. Iglesias, 2025-06-16)
ab02a120c0a5 tools/ocaml: Update bindings for
CDF_TRAP_UNMAPPED_ACCESSES (same day)
Signed-off-by: Julian Vetter <julian.vetter@vates.tech>
domctl_create_config here is a mirror of Xenctrl.domctl_create_config.
Xen's OCaml bindings gained the altp2m_count field in:
5699554de9a9 tools/ocaml: Add altp2m_count parameter
(Petr Beneš, 2025-08-25)
exposing the pre-existing altp2m.nr count (xen_domctl_createdomain),
alongside the existing altp2m_opts.
Signed-off-by: Julian Vetter <julian.vetter@vates.tech>
Signed-off-by: Seb Hinderer <sebastien.hinderer@vates.tech>
Most of these calls pass only integers or pointers allocated in C, so it is safe to release the OCaml runtime lock. Replace direct passing of Xfm_val/Caml_ba_data_val with storing in a temporary local variable (the result of Xfm_val or Caml_ba_data_val won't move since they are C pointers, but the OCaml value passed in as params to these macros might!) Signed-off-by: Edwin Török <edvin.torok@citrix.com> Signed-off-by: Qin Zhang （张琴） <qin.zhang@citrix.com>
…ntime lock where possible unshadow is not thread safe, but the other ones should be. TODO: double check all C API calls in unixpwd.c with the MT-safe portion of the manpage Use the caml_release_runtime_system()/caml_acquire_runtime_system() spelling rather than the historical caml_enter_blocking_section()/caml_leave_blocking_section() aliases (identical macros in caml/threads.h), to match the rest of xen-api. Signed-off-by: Edwin Török <edvin.torok@citrix.com> Signed-off-by: Qin Zhang （张琴） <qin.zhang@citrix.com>
This PR adds a section to `README.md` explaining how to work with several branches simultaneously. It gives two recommendations: using one Opam switch per branch, and using Git work trees to avoid path conflicts which may cause pinning to fail. See also PR xapi-project#7221, based on 26.1-lcm, which makes sure the build instructions for that branch are accurate. Signed-off-by: Seb Hinderer <sebastien.hinderer@vates.tech>
domain_create_flag's Xenctrl.domain_create_flag mirror already lists CDF_TRAP_UNMAPPED_ACCESSES, but the flag was never actually added to the flags list `make` builds, so it was never set for any domain. Without it, Xen traps guest accesses to unmapped addresses instead of letting the guest handle them directly. On Arm this can fire very early in the kernel's own boot code, before the guest has installed its own exception vector table, in which case the resulting injected exception is delivered at a fixed vector offset off a zero base, which itself is also unmapped, causing an infinite faulting loop. libxl already sets this flag for Arm domains (libxl__arch_domain_prepare_config() in tools/libs/light/libxl_arm.c always includes it in config->flags), which is why domains built via libxl are unaffected while domains built through xenopsd's direct Xenctrl.domain_create binding never got it. Confirmed by comparing the actual xen_domctl_createdomain contents both toolstacks send to xc_domain_create() for the same kernel/ramdisk/memory Arm domain config: xenopsd sent flags=0x3 (CDF_HVM|CDF_HAP), libxl sent flags=0x103 (adding CDF_TRAP_UNMAPPED_ACCESSES, bit 8). domain_create_flag is shared between the Arm and x86 xenopsd builds, and Xen's x86 arch_sanitise_domain_config() rejects domain creation (-EINVAL) if this flag is set on x86 (xen/arch/x86/domain.c). libxl_x86.c enforces the same restriction at the toolstack level, returning ERROR_FAIL if trap_unmapped_accesses is requested on x86. So, mirror what libxl is doing and add a trap-unmapped-accesses platform-data key, defaulting to on for Arm domains and off on x86. This gives a user who wants Arm guest accesses to unmapped addresses to not trap a way to opt out via platform:trap-unmapped-accesses=false. Signed-off-by: Julian Vetter <julian.vetter@vates.tech>
…lls (rebase of xapi-project#4916) (xapi-project#7226) ## Where this PR comes from This is a rebase of the three C-stub commits from **[xapi-project#4916](xapi-project#4916 ("CA-391656: OCaml C stubs: release runtime lock around all xenctrl calls", by @edwintorok), which was opened as a draft and has since been closed without merging.
Signed-off-by: Ming Lu <ming.lu@cloud.com>
…i-project#7197) `ocaml/xenopsd/xc/domain.ml` mirrors `Xenctrl.domain_create_flag` and `Xenctrl.domctl_create_config`, both of which require listing every constructor/field of the real type. Xen 4.21 added items to these types that were missing from `domain.ml`, and in one case the mirror being incomplete masked a second issue: the corresponding flag was never actually being set anywhere. - `CDF_TRAP_UNMAPPED_ACCESSES` on `domain_create_flag` (currently Arm-only): - xen.git 980aff4e8fcd "xen/arm: Add way to disable traps on accesses to unmapped addresses" (Edgar E. Iglesias, 2025-06-16) - xen.git ab02a120c0a5 "tools/ocaml: Update bindings for CDF_TRAP_UNMAPPED_ACCESSES" (same day) - pre-existing altp2m.nr field alongside altp2m_opts: - xen.git 5699554de9a9 "tools/ocaml: Add altp2m_count parameter" (Petr Beneš, 2025-08-25) Both have been part of the public Xen ABI since before the 4.21.0-rc1 tag, so any build compiling xapi against Xen 4.21's actual OCaml Xenctrl bindings fails at this type check. Once the mirror type-checked, I could actually run an Arm domain through xenopsd against a real Xen 4.21, which showed a second bug: `domain.ml` never sets `CDF_TRAP_UNMAPPED_ACCESSES` in the flags list that `make` passes to `Xenctrl.domain_create`, even though the type now lists it. Without it, Xen traps guest accesses to unmapped addresses instead of letting the guest handle them directly. On Arm this fires very early in the kernel's own boot code, before it has installed its own exception vector table, so the resulting injected exception lands at a fixed vector offset off a zero base, which itself is unmapped, and the guest immediately double-faults into an infinite loop. libxl already sets this flag unconditionally for Arm domains (`libxl__arch_domain_prepare_config()` in `tools/libs/light/libxl_arm.c`), which is why a domain built with `xl create` for the exact same kernel/ramdisk/memory boots fine while the identical domain built through xenopsd's `Xenctrl.domain_create` never did. Confirmed by comparing the actual `xen_domctl_createdomain` contents both toolstacks send to `xc_domain_create()` for the same config: xenopsd sent flags=0x3 (`CDF_HVM|CDF_HAP`), libxl sent flags=0x103 (adding `CDF_TRAP_UNMAPPED_ACCESSES`, bit 8), which was the only difference between the two, and setting it here reproducibly fixes the guest boot hang. `domain_create_flag` is shared between the Arm and x86 xenopsd builds, and Xen's x86 `arch_sanitise_domain_config()` rejects domain creation with -EINVAL if this flag is set on x86 (`xen/arch/x86/domain.c`), so it's gated on the domain's `arch_domainconfig` rather than set unconditionally. ## Testing Found and verified while building xapi/xenopsd for aarch64 against a freshly built Xen 4.21.1 (linking directly against Xen's own `tools/ocaml/libs/xc` output). Confirmed the build fails without the first two commits and succeeds with them applied, with no other call sites needing changes (only `domain.ml` references either type). Confirmed the third commit by booting a plain Arm PVH domain end-to-end through xapi/xenopsd (same kernel+initramfs xapi's own dom0 uses): it hung every time with the flag unset (same crash signature) and boots cleanly and consistently once set, matching `xl create`'s own behaviour for an identical domain config. Side question: the xenctrl opam package CI pulls in (xapi-project/xenctrl, via xs-opam) also doesn't have these Xen additions. Would that need updating as well?
Collaborator
Author
|
It hits the following DCO check : |
Collaborator
Author
|
changlei-li
approved these changes
Aug 25, 2026
BengangY
approved these changes
Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sync master to feature/sxm-v3