bake-images: on-hardware fixes — manifest ordering, CRI-O drop-in, preflight - #18
Merged
raghavendra-talur merged 5 commits intoAug 25, 2026
Conversation
On Linux the per-cluster store copy is a real pool upload, so the create-master-vms disk preflight adds the store size. On macOS ImportDisk APFS-clones the cached image — the copy is free and the guest mounts it read-only — so counting it double-books ~44 GB and wrongly blocks creates on tight disks. Assisted-by: Claude Code/claude-fable-5 Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>
On-hardware validation of --bake-images exposed that openshift-install's create single-node-ignition-config silently ignores manifests pre-dropped into openshift/ unless 'create manifests' ran first: neither the baked-store nor the Rosetta MachineConfig reached the installed node (no mount, no storage.conf drop-in, no binfmt), while the YAMLs sat unconsumed in the install dir. Confirmed both ways with a scratch install dir: without the manifests step the extra MC is dropped; with it, the MC renders into bootstrap-in-place-for-live-iso.ign. Add Installer.CreateManifests and call it from generate-ignition before the manifest writes (darwin or --bake-images; the plain Linux path is unchanged). The fake Installer now records call order so the stage test pins CreateManifests -> Write*Manifest -> CreateSingleNodeIgnition. Assisted-by: Claude Code/claude-fable-5 Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>
On-hardware validation round two: the store disk mounted and the storage.conf drop-in was in place, yet CRI-O saw none of the 173 store images and pulled everything from quay.io (7 GiB RX in the first five minutes). Root cause: RHCOS 9.8's containers-common (5.8) does not read /etc/containers/storage.conf.d/ — podman info showed no imagestore graph option either, so the drop-in was silently ignored. Deliver the additional image store through CRI-O's own config instead: /etc/crio/crio.conf.d/10-baked-images.conf with storage_option = ["overlay.imagestore=<mount>"]. Verified live on the node: after writing that drop-in and restarting crio, crictl went from 53 to 224 images — the store serves. RHCOS ships no active storage_option, so the list replaces nothing. Assisted-by: Claude Code/claude-fable-5 Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>
… gap Assisted-by: Claude Code/claude-fable-5 Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>
Assisted-by: Claude Code/claude-fable-5 Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>
raghavendra-talur
added a commit
that referenced
this pull request
Aug 25, 2026
Tick completed items (Rosetta-from-first-boot, bake follow-ups merged in paper #18), drop stale branch references, add the two open ODF items: full-feature validation on a big host and the --odf-profile knob with the measured request math. Assisted-by: Claude Code/claude-fable-5 Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes from the first real on-hardware run of
--bake-imageson macOS (all three bugs were invisible to unit tests,--simulate, and the scratch pack test — only a real install surfaced them), plus docs recording the validation.create manifestsmust precede extra-manifest drops (ae2c0f8):openshift-install create single-node-ignition-configsilently ignores manifests pre-dropped intoopenshift/without the state fromcreate manifests— neither the baked-store nor the Rosetta MachineConfig reached the installed node. Proven both ways with a scratch install dir;generate-ignitionnow calls the newInstaller.CreateManifestsfirst (darwin or--bake-images), with a call-order-pinned stage test. This also fixes Rosetta-from-first-boot, which shipped broken in macOS (Apple Silicon) backend: SNO on vfkit + vmnet-helper #16.storage.conf.d(0229478): RHCOS 9.8's containers-common (5.8) does not read/etc/containers/storage.conf.d/— the store sat mounted but unread and the node pulled 7 GiB from quay.io in five minutes. Replaced with/etc/crio/crio.conf.d/10-baked-images.conf(storage_option = ["overlay.imagestore=…"]); verified live: crictl went from 53 to 224 visible images.0cb9ffa):ImportDiskAPFS-clones the cached image, so counting its full size wrongly blocked creates on tight disks.0b73625,7bd4e5c): hardware-validation results inimage-baking.mdand ROADMAP (clean baked install: ~25.5 min vs 29 min storeless, 8 network pulls / 1.0 GiB RX vs ~40 GB payload; Rosetta amd64 verified from first boot), the macOS store-builder gap (skopeo cannot author overlay stores on macOS; podman-machine builder validated manually), and the offline-install gap analysis (OLM catalogs,imagePullPolicy: Alwaysinsights images, and magic-DNS as the real blocker).Test plan
make checkgreen (new/updated tests: call-order pinning ingenerate-ignition,RenderCRIODropin, MachineConfig/ignition-merge assertions).create -n baked --bake-imageswith zero hand-edits converged in ~25.5 min; store mounted ro from first boot; CRI-O served 179 images; 8 network pulls total (3 OLM catalogs + 2 pullPolicy=Always insights images + test pod); Rosetta binfmt enabled from first boot and a confined amd64 container printedx86_64.🤖 Generated with Claude Code