Skip to content

Cache VM disks in an OCI registry and add --os targeting - #46

Merged
DimmKirr merged 11 commits into
mainfrom
feature/wip
Sep 6, 2026
Merged

Cache VM disks in an OCI registry and add --os targeting#46
DimmKirr merged 11 commits into
mainfrom
feature/wip

Conversation

@DimmKirr

@DimmKirr DimmKirr commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

QEMU Windows builds now probe an OCI registry for a cached base-profile disk and pull it on a hit, skipping the full winkit provisioning pipeline (seconds instead of tens of minutes), and push their results back so later builds benefit. Users can also target a platform with --os linux/macos/windows instead of knowing engine names, enable MCP servers per repo via [mcp].enabled (CELL-512), and skip all published ports with --no-ports. Project-level flake.nix install (CELL-447) flips from opt-out to opt-in — see Breaking changes.

Breaking changes

  • Project-level flake.nix install is now opt-in, off by default: enable with --use-flake, [cell].flake = true, or DEVCELL_FLAKE=1. Cells that relied on automatic flake install must opt in after upgrading.
  • --skip-flake and --no-flake are accepted but ignored (legacy no-ops); the DEVCELL_SKIP_FLAKE container env var is no longer set.

Changes

  • feat(cmd): add cell disk-store push/pull/resolve wrapping go-diskoci — operators can seed or inspect cached disk images outside a build
  • feat(cmd): probe/pull a cached base-profile disk in QEMU builds and push ssh-able/base-profile disks after success — cache hits skip provisioning entirely; --no-cache bypasses the pull only, and DEVCELL_DISK_CACHE_REGISTRY/REF + DISKOCI_USERNAME/PASSWORD override registry and auth per environment
  • feat(engine): add --os linux/macos/windows mapped to its default engine plus a [cell].os TOML fallback, rejecting incompatible --os/--engine pairs and keeping --macos as a working alias — targets a platform without naming an engine
  • feat(cli): gate project flake.nix install behind --use-flake / [cell].flake / DEVCELL_FLAKE (CELL-447) — untrusted project flakes no longer install without an explicit opt-in
  • feat(runner): add --no-ports to suppress all docker -p mappings (user [ports] and GUI VNC/RDP) — a cell can run fully unpublished when another transport handles access
  • feat(cfg): add [mcp].enabled merged union-dedup across global/project config and resolved at container start via DEVCELL_MCP_ENABLED (CELL-512) — a repo can narrow which MCP servers start for that project
  • fix(init): cell init now resolves the engine like build/run (flag > [cell].engine/[cell].os > default) and honors the [nix].nixhome override instead of silently ignoring both
  • fix(runner): enable nix-command/flakes for flake lock/update and stack discovery — nix flake lock/update and stack list no longer fail in the bootstrap container
  • fix(scaffold): reject a nixhome source nested in its own build dir and make CopyDir skip a destination nested inside its source — a self-referential DEVCELL_NIXHOME can no longer corrupt .devcell/ mid-copy
  • test(stealth,helpers,cfg,runner): harden stealth checks against BrowserScan/CreepJS tells, resolve nixhome via nixhomeDir() instead of hardcoded ../nixhome (CELL-426), and cover flake precedence, MCP merge, and no-ports behavior
  • refactor(build): delegate WinPE ISO creation to go-winkit winpe.Build(), delete WSL transplant stubs, extract shared resolveEngine(), and require published go-diskoci v0.1.0/go-winkit instead of local replace directives — CI builds from a clean checkout

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

- refactor(qemu): remove transplant_wsl*.go — WSLEngineFiles, WSLInboxShim and TransplantWSLIntoBootWim now live in go-winkit/winpe, committed there in 3b7f219
- refactor(cmd): extract resolveEngine() from three call sites — init, build, and run now share one priority chain (CLI flag > TOML [cell].engine > "docker" default) instead of each reimplementing it
- test(cmd): add resolveEngine unit tests covering flag, TOML fallback, macOS alias, and default
- refactor(build): replace ~90 lines of inline stage/inject/master with one winpe.Build() call — WinPE ISO creation is now the library's responsibility
- chore(deps): point go-winkit at local working copy (replace directive) — drop before merging
…oding ../nixhome

- test(helpers): add nixhomeDir() honoring DEVCELL_NIXHOME > DEVCELL_NIXHOME_PATH > ../nixhome — tests keep finding nixhome once it moves to devcell-sh/community-home
- test(image,mise,modules): replace hardcoded ../nixhome with nixhomeDir() at every call site — tests survive the external repo split without editing each file again
…S runs slipped through

- test(stealth): require navigator.webdriver as native boolean false, not deleted — matches BrowserScan's expectation that the property exists
- test(stealth): flag any override of Navigator.prototype.webdriver — forces reliance on AutomationControlled flag, cutting lie surface
- test(stealth): fail when window.chrome.runtime is fabricated on ordinary pages — closes CreepJS hasBadChromeRuntime signal
- test(stealth): require window.chrome.app present alongside the runtime check — keeps real Chrome's baseline shape intact
- test(stealth): reject a SwiftShader string in the spoofed WebGL renderer — closes the top headless/datacenter GPU tell
- test(stealth): require a named Function.prototype.toString wrapper with stack scrubbing — closes CreepJS hasToStringProxy signal
- test(stealth): resolve nixhome path via nixhomeDir() helper instead of a hardcoded relative path — keeps test working after nixhome relocation
- fix(scaffold): reject nixhome source nested in its own build dir — stops DEVCELL_NIXHOME self-reference from corrupting .devcell/
- fix(scaffold): CopyDir skips a destination nested inside its source — prevents silent partial-copy corruption if the upfront guard is bypassed
- test(scaffold): add regression tests for self-referential nixhome source and nested-destination CopyDir calls
… engine

- feat(engine): add --os linux/macos/windows, mapped to its default engine — targets a platform without knowing engine names
- feat(engine): reject incompatible --os/--engine pairs (e.g. windows+tart) — catches misconfigured builds before they run
- feat(cli): keep --macos as a working alias for --os=macos — existing scripts and muscle memory keep working
- feat(cfg): add [cell].os TOML fallback — lets teams pin an OS default in project config instead of passing --os every time
- refactor(engine): resolveEngine now returns an error and takes os args — no user-facing impact
- test(engine): cover os precedence, macos alias, and invalid/incompatible combinations — locks in the new resolution order
- docs(nix): expose [cell].os in home-manager module options — keeps generated config docs in sync with the new setting
…ntainer

- fix(runner): enable nix-command/flakes for flake lock/update — `nix flake lock`/`update` no longer fails
- fix(runner): enable nix-command/flakes for stack discovery — `stack list` no longer fails in bootstrap container
…U build fast path

- feat(cmd): add `cell disk-store push/pull/resolve` wrapping go-diskoci — lets operators seed or inspect cached disk images outside a build
- feat(cmd): probe/pull a cached base-profile disk in runBuildQemu — skips the full winkit provisioning pipeline entirely on a cache hit, cutting build time
- feat(cmd): push ssh-able and base-profile disks after a successful build — populates the cache so later builds can hit it
- feat(cmd): support `--no-cache` to bypass the pull probe only — pushes still happen, so cache stays warm even when forcing a fresh build
- feat(cmd): support DEVCELL_DISK_CACHE_REGISTRY/REF and DISKOCI_USERNAME/PASSWORD env vars — lets registry and auth be overridden per environment
- test(cmd): cover disk-cache ref/fingerprint derivation and push/pull round-trip against an in-process registry
- build(go.mod): drop the local go-winkit replace directive — build now resolves the published module instead of a developer-machine path
…clean-module-cache builds

- build(deps): declare github.com/devcell-sh/go-diskoci v0.1.0 in go.mod/go.sum — lets CI's Deploy Site build resolve the import from a clean checkout instead of failing go mod tidy
- build(deps): bump go-containerregistry, docker/cli, klauspost/compress, x/mod, x/net, x/tools as incidental go mod tidy fallout — picks up upstream point-release fixes
- build(nix): refresh flake.nix vendorHash to match the updated go.sum — keeps `nix build` from failing on a hash mismatch
…avior

- test(cfg): cover FlakeEnabled() env>toml>false precedence — no user-facing impact
- test(cfg): cover [mcp].enabled merge/dedup and TOML load — no user-facing impact
- test(runner): rename SkipFlake tests to TrustFlake, add NoPorts coverage — no user-facing impact
@cloudflare-workers-and-pages

Copy link
Copy Markdown
Contributor

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
devcell 9b7b899 Commit Preview URL

Branch Preview URL
Sep 06 2026, 05:59 AM

@DimmKirr
DimmKirr merged commit 81cf0bd into main Sep 6, 2026
7 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant