From 272f18bfb4e916527f1cf715037fbb49f3d5bc92 Mon Sep 17 00:00:00 2001 From: Mykhailo Chalyi Date: Wed, 5 Aug 2026 17:58:08 +0000 Subject: [PATCH 1/3] feat(wasm): host-backed filesystem for the wasm bindings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `new Bash({ fs })` runs scripts directly against storage the embedder owns — a Durable Object, OPFS, IndexedDB — instead of the in-memory VFS. Nothing is copied in or diffed back out: every read and write during a run is a call into the host object, so there is no workspace-size ceiling beyond the host's own and no lost-update window between runs. The bridge implements `FsBackend` and wraps it in `PosixFs`, so hosts supply raw storage (seven required methods, validated at construction) and inherit POSIX semantics. `append`, `copy`, and `rename` are synthesized when omitted, `chmod` is accepted and ignored so `chmod +x` works against hosts with no permission model, and host errors carrying a `code` map onto the matching `io::ErrorKind` so builtins that branch on kind behave as they do over the built-in VFS. Host calls may return promises, so a host filesystem implies `execute()`; `executeSync` reports the suspension rather than blocking, and `files` is rejected alongside `fs` because seeding cannot complete synchronously. --- .github/workflows/ci.yml | 2 +- .github/workflows/publish-wasm.yml | 2 +- Cargo.lock | 1 + crates/bashkit-wasm/Cargo.toml | 3 + crates/bashkit-wasm/README.md | 50 ++- crates/bashkit-wasm/__test__/host-fs.test.mjs | 336 ++++++++++++++++ crates/bashkit-wasm/js/index.d.ts | 73 ++++ crates/bashkit-wasm/src/hostfs.rs | 363 ++++++++++++++++++ crates/bashkit-wasm/src/lib.rs | 34 +- justfile | 2 +- knowledge/runtimes/browser-package.md | 47 ++- 11 files changed, 906 insertions(+), 7 deletions(-) create mode 100644 crates/bashkit-wasm/__test__/host-fs.test.mjs create mode 100644 crates/bashkit-wasm/src/hostfs.rs diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8dbbf5fb8..161fc8ed3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -118,7 +118,7 @@ jobs: run: bash crates/bashkit-wasm/scripts/build.sh release - name: Integration tests (headless Node) - run: node --test crates/bashkit-wasm/__test__/bashkit-wasm.test.mjs + run: node --test "crates/bashkit-wasm/__test__/*.test.mjs" audit: name: Audit diff --git a/.github/workflows/publish-wasm.yml b/.github/workflows/publish-wasm.yml index 1f9c3f458..9a3243491 100644 --- a/.github/workflows/publish-wasm.yml +++ b/.github/workflows/publish-wasm.yml @@ -65,7 +65,7 @@ jobs: run: bash crates/bashkit-wasm/scripts/build.sh release - name: Integration tests (headless Node) - run: node --test crates/bashkit-wasm/__test__/bashkit-wasm.test.mjs + run: node --test "crates/bashkit-wasm/__test__/*.test.mjs" - name: Upload pkg uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 diff --git a/Cargo.lock b/Cargo.lock index fd2e3b426..be72417a9 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -553,6 +553,7 @@ dependencies = [ "serde_json", "wasm-bindgen", "wasm-bindgen-futures", + "web-time", ] [[package]] diff --git a/crates/bashkit-wasm/Cargo.toml b/crates/bashkit-wasm/Cargo.toml index f6fe53f33..c0bebdac3 100644 --- a/crates/bashkit-wasm/Cargo.toml +++ b/crates/bashkit-wasm/Cargo.toml @@ -42,6 +42,9 @@ serde_json = { workspace = true } serde-wasm-bindgen = "0.6" futures-util = { workspace = true } send_wrapper = { version = "0.6", features = ["futures"] } +# Same clock shim bashkit uses internally, so `Metadata` timestamps built here +# unify with `bashkit::time_compat::SystemTime` on wasm32. +web-time = { workspace = true } console_error_panic_hook = "0.1" # getrandom's browser backend (matches the workspace pin). Enables `wasm_js` # so the credential-placeholder RNG links on wasm32-unknown-unknown. diff --git a/crates/bashkit-wasm/README.md b/crates/bashkit-wasm/README.md index 37bf25848..578f49d98 100644 --- a/crates/bashkit-wasm/README.md +++ b/crates/bashkit-wasm/README.md @@ -122,6 +122,7 @@ new Bash({ maxCommands, maxLoopIterations, maxMemory, files: { "/config.json": '{"debug":true}' }, customBuiltins: { name: (ctx) => "..." }, + fs: hostFileSystem, // host-backed filesystem; see below }); ``` @@ -167,6 +168,53 @@ const resumed = new Bash(); resumed.checkout(saved.id, saved.objects); // policy defaults to "superset" ``` +## Host-backed filesystem + +Pass `fs` to run scripts directly against storage you own — a Durable Object, an +OPFS handle, IndexedDB — instead of the in-memory VFS. Nothing is copied in or +diffed back out: every read and write during the run is a call into your object. + +```js +const bash = new Bash({ cwd: "/workspace", fs: myHost }); +const r = await bash.execute("grep -rl TODO . | head -5"); +``` + +Implement seven required methods; each may return its value directly or as a +`Promise`: + +```ts +read(path) // -> Uint8Array | string (throw ENOENT when absent) +write(path, bytes) // -> void +mkdir(path, recursive) // -> void +remove(path, recursive) // -> void +stat(path) // -> { type: "file" | "dir" | "symlink", size?, mode?, mtimeMs? } +readDir(path) // -> [{ name, type, size?, mode?, mtimeMs? }] +exists(path) // -> boolean +``` + +`append`, `copy`, `rename`, and `chmod` are optional: omit them and they are +synthesized from the required primitives (`chmod` is accepted and ignored, so +`chmod +x` still works). `symlink` and `readLink` are optional too, but scripts +that reach for them fail with `ENOSYS` when the host omits them. + +Your host implements raw storage only — POSIX semantics (parent-directory +checks, "is a directory", symlink resolution) are enforced above it. Throw an +`Error` carrying a `code` (`ENOENT`, `EEXIST`, `EACCES`, `EPERM`, `EISDIR`, +`ENOTDIR`, `ENOTEMPTY`, `EXDEV`, `ENOSYS`) so bash reports the failure the way a +real shell does. + +Two contract notes: + +- **`execute()` only.** A host call can suspend the interpreter, and + `executeSync` cannot await — it reports the suspension instead of blocking. + The synchronous `bash.readFile(...)` helpers behave the same way. +- **`files` is rejected alongside `fs`.** Seeding writes through the VFS + synchronously, which a promise-returning host can never satisfy. Write seed + data through the host directly. + +Provide `/dev/null` in the host if your scripts redirect to it; with a host +filesystem there is no built-in VFS underneath to supply it. + ## What's included Plain bash plus the built-in text tooling (`grep`, `sed`, `awk`, `jq`, `find`, @@ -198,7 +246,7 @@ custom builtin (see above) so requests go through your app's own `fetch`. ```bash # Build the bundle and run the headless integration tests: bash scripts/build.sh -node --test __test__/bashkit-wasm.test.mjs +node --test "__test__/*.test.mjs" # or, from the repo root: just build-wasm ``` diff --git a/crates/bashkit-wasm/__test__/host-fs.test.mjs b/crates/bashkit-wasm/__test__/host-fs.test.mjs new file mode 100644 index 000000000..850012b42 --- /dev/null +++ b/crates/bashkit-wasm/__test__/host-fs.test.mjs @@ -0,0 +1,336 @@ +// Integration tests for the host-backed filesystem (`new Bash({ fs })`). +// +// The fake host below is deliberately async on every method: it is the shape a +// real embedder has (a Durable Object, IndexedDB, OPFS), and it proves the +// interpreter suspends and resumes correctly across host calls rather than +// relying on an accidentally-synchronous store. +// +// Run against the built package like the rest of the suite: +// +// node --test crates/bashkit-wasm/__test__/ + +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { readFileSync } from "node:fs"; +import { fileURLToPath } from "node:url"; + +import { initBashkit, Bash } from "../pkg/index.js"; + +const wasmBytes = readFileSync( + fileURLToPath(new URL("../pkg/bashkit_wasm_bg.wasm", import.meta.url)), +); +await initBashkit(wasmBytes); + +// --- Fake host ------------------------------------------------------------- + +const encoder = new TextEncoder(); +const decoder = new TextDecoder(); + +function fsError(code, message) { + const error = new Error(message); + error.code = code; + return error; +} + +// Minimal async host filesystem over a Map. Implements only the required +// methods, so `append`, `copy`, `rename`, and `chmod` exercise the synthesized +// fallbacks in the bridge. +class FakeHost { + // path -> {type, content?, mode, mtimeMs} + entries = new Map(); + calls = []; + + constructor(seed = {}) { + for (const path of ["/", "/tmp", "/dev", "/home", "/home/user", "/workspace"]) { + this.entries.set(path, { type: "dir", mode: 0o755, mtimeMs: 0 }); + } + // Scripts redirect to /dev/null often enough that a host which omits it + // looks broken. Hosts are expected to provide it; this one does. + this.entries.set("/dev/null", { type: "file", content: new Uint8Array(), mode: 0o666 }); + for (const [path, text] of Object.entries(seed)) { + this.entries.set(path, { type: "file", content: encoder.encode(text), mode: 0o644 }); + } + } + + text(path) { + const entry = this.entries.get(path); + return entry?.content === undefined ? undefined : decoder.decode(entry.content); + } + + // Every method resolves on a later microtask so nothing can complete in the + // first poll — the suspension path is the one under test. + async #tick(name) { + this.calls.push(name); + await Promise.resolve(); + } + + async read(path) { + await this.#tick("read"); + const entry = this.entries.get(path); + if (entry === undefined) throw fsError("ENOENT", `no such file: ${path}`); + if (entry.type === "dir") throw fsError("EISDIR", `is a directory: ${path}`); + return entry.content; + } + + async write(path, content) { + await this.#tick("write"); + if (path === "/dev/null") return; + this.entries.set(path, { type: "file", content, mode: 0o644, mtimeMs: 0 }); + } + + async mkdir(path, recursive) { + await this.#tick("mkdir"); + if (this.entries.has(path)) { + if (recursive) return; + throw fsError("EEXIST", `exists: ${path}`); + } + if (recursive) { + const parts = path.split("/").filter(Boolean); + let prefix = ""; + for (const part of parts) { + prefix += `/${part}`; + if (!this.entries.has(prefix)) { + this.entries.set(prefix, { type: "dir", mode: 0o755, mtimeMs: 0 }); + } + } + return; + } + this.entries.set(path, { type: "dir", mode: 0o755, mtimeMs: 0 }); + } + + async remove(path, recursive) { + await this.#tick("remove"); + if (!this.entries.has(path)) throw fsError("ENOENT", `no such file: ${path}`); + this.entries.delete(path); + if (!recursive) return; + for (const key of [...this.entries.keys()]) { + if (key.startsWith(`${path}/`)) this.entries.delete(key); + } + } + + async stat(path) { + await this.#tick("stat"); + const entry = this.entries.get(path); + if (entry === undefined) throw fsError("ENOENT", `no such file: ${path}`); + return { + type: entry.type, + size: entry.content?.length ?? 0, + mode: entry.mode, + mtimeMs: entry.mtimeMs ?? 0, + }; + } + + async readDir(path) { + await this.#tick("readDir"); + if (!this.entries.has(path)) throw fsError("ENOENT", `no such file: ${path}`); + const prefix = path === "/" ? "/" : `${path}/`; + const out = []; + for (const [key, entry] of this.entries) { + if (key === path || !key.startsWith(prefix)) continue; + const rest = key.slice(prefix.length); + if (rest.length === 0 || rest.includes("/")) continue; + out.push({ + name: rest, + type: entry.type, + size: entry.content?.length ?? 0, + mode: entry.mode, + mtimeMs: entry.mtimeMs ?? 0, + }); + } + return out; + } + + async exists(path) { + await this.#tick("exists"); + return this.entries.has(path); + } +} + +const hostBash = (host, options = {}) => + new Bash({ fs: host, cwd: "/workspace", ...options }); + +// --- Reads and writes ------------------------------------------------------ + +test("host fs: a redirect lands in the host store", async () => { + const host = new FakeHost(); + const bash = hostBash(host); + const r = await bash.execute('echo "hello" > /workspace/greeting.txt'); + assert.equal(r.stderr, ""); + assert.equal(r.exitCode, 0); + assert.equal(host.text("/workspace/greeting.txt"), "hello\n"); +}); + +test("host fs: reads come from the host store, not a copy", async () => { + const host = new FakeHost({ "/workspace/seed.txt": "from the host\n" }); + const r = await hostBash(host).execute("cat /workspace/seed.txt"); + assert.equal(r.stdout, "from the host\n"); +}); + +test("host fs: writes made between runs are visible to the next run", async () => { + const host = new FakeHost(); + const bash = hostBash(host); + await bash.execute("echo first > /workspace/log.txt"); + host.entries.set("/workspace/side.txt", { + type: "file", + content: encoder.encode("written behind bash's back\n"), + mode: 0o644, + }); + const r = await bash.execute("cat /workspace/side.txt"); + assert.equal(r.stdout, "written behind bash's back\n"); +}); + +test("host fs: append redirect uses the synthesized append", async () => { + const host = new FakeHost({ "/workspace/log.txt": "one\n" }); + const r = await hostBash(host).execute('echo two >> /workspace/log.txt'); + assert.equal(r.exitCode, 0); + assert.equal(host.text("/workspace/log.txt"), "one\ntwo\n"); +}); + +test("host fs: text tools run over host bytes", async () => { + const host = new FakeHost({ + "/workspace/data.txt": "alpha 1\nbeta 2\ngamma 3\n", + }); + const r = await hostBash(host).execute( + "grep -c . /workspace/data.txt && sed -n 2p /workspace/data.txt", + ); + assert.equal(r.stdout, "3\nbeta 2\n"); +}); + +test("host fs: jq reads a host file", async () => { + const host = new FakeHost({ "/workspace/x.json": '{"a":{"b":41}}' }); + const r = await hostBash(host).execute("jq -c '.a.b + 1' /workspace/x.json"); + assert.equal(r.stdout.trim(), "42"); +}); + +// --- Directories ----------------------------------------------------------- + +test("host fs: mkdir -p, ls, and rm -r", async () => { + const host = new FakeHost(); + const bash = hostBash(host); + const made = await bash.execute("mkdir -p /workspace/a/b && echo x > /workspace/a/b/f.txt"); + assert.equal(made.exitCode, 0, made.stderr); + assert.equal(host.entries.has("/workspace/a/b"), true); + + const listed = await bash.execute("ls /workspace/a/b"); + assert.equal(listed.stdout, "f.txt\n"); + + const removed = await bash.execute("rm -r /workspace/a"); + assert.equal(removed.exitCode, 0, removed.stderr); + assert.equal(host.entries.has("/workspace/a/b/f.txt"), false); +}); + +test("host fs: cp and mv work through the synthesized fallbacks", async () => { + const host = new FakeHost({ "/workspace/src.txt": "payload\n" }); + const bash = hostBash(host); + const copied = await bash.execute("cp /workspace/src.txt /workspace/copy.txt"); + assert.equal(copied.exitCode, 0, copied.stderr); + assert.equal(host.text("/workspace/copy.txt"), "payload\n"); + + const moved = await bash.execute("mv /workspace/copy.txt /workspace/moved.txt"); + assert.equal(moved.exitCode, 0, moved.stderr); + assert.equal(host.text("/workspace/moved.txt"), "payload\n"); + assert.equal(host.entries.has("/workspace/copy.txt"), false); +}); + +test("host fs: chmod is accepted when the host has no permission model", async () => { + const host = new FakeHost({ "/workspace/build.sh": "echo hi\n" }); + const r = await hostBash(host).execute("chmod +x /workspace/build.sh"); + assert.equal(r.exitCode, 0, r.stderr); +}); + +// --- Error mapping --------------------------------------------------------- + +test("host fs: an ENOENT host error is a miss, not a failure", async () => { + const host = new FakeHost(); + const bash = hostBash(host); + // Builtins that branch on the error kind (rather than surfacing the raw + // message) must see NotFound and print bash's own wording. + const listed = await bash.execute("ls /workspace/missing.txt"); + assert.notEqual(listed.exitCode, 0); + assert.match(listed.stderr, /No such file or directory/); + + // `test -f` is pure kind/existence logic — a mapped ENOENT must not read as + // an I/O failure. + const tested = await bash.execute( + "if [ -f /workspace/missing.txt ]; then echo yes; else echo no; fi", + ); + assert.equal(tested.stdout, "no\n"); + assert.equal(tested.stderr, ""); +}); + +test("host fs: an ENOENT host message reaches stderr for builtins that report it", async () => { + const host = new FakeHost(); + const r = await hostBash(host).execute("cat /workspace/missing.txt"); + assert.notEqual(r.exitCode, 0); + assert.match(r.stderr, /cat: \/workspace\/missing\.txt:/); +}); + +test("host fs: a host error without a code still surfaces its message", async () => { + const host = new FakeHost(); + host.read = async () => { + throw new Error("host store unreachable"); + }; + const r = await hostBash(host).execute("cat /workspace/anything.txt"); + assert.notEqual(r.exitCode, 0); + assert.match(r.stderr, /host store unreachable/); +}); + +test("host fs: a rejected write fails the command, not the run", async () => { + const host = new FakeHost(); + host.write = async () => { + throw fsError("EACCES", "read-only workspace"); + }; + const r = await hostBash(host).execute( + 'echo nope > /workspace/f.txt; echo "still running"', + ); + assert.match(r.stdout, /still running/); + assert.match(r.stderr, /read-only workspace|Permission denied/); +}); + +// --- Construction contract ------------------------------------------------- + +test("host fs: executeSync reports the suspension instead of blocking", async () => { + const host = new FakeHost({ "/workspace/a.txt": "x\n" }); + assert.throws( + () => hostBash(host).executeSync("cat /workspace/a.txt"), + /did not complete synchronously/, + ); +}); + +test("host fs: files and fs cannot be combined", () => { + assert.throws( + () => new Bash({ fs: new FakeHost(), files: { "/workspace/a.txt": "x" } }), + /cannot be combined/, + ); +}); + +test("host fs: a missing required method is rejected at construction", () => { + const complete = new FakeHost(); + // Plain object rather than a FakeHost subclass: methods on a prototype chain + // resolve fine, so the only way to be genuinely missing one is to omit it. + const partial = { + read: (p) => complete.read(p), + write: (p, c) => complete.write(p, c), + mkdir: (p, r) => complete.mkdir(p, r), + remove: (p, r) => complete.remove(p, r), + stat: (p) => complete.stat(p), + exists: (p) => complete.exists(p), + // readDir deliberately absent + }; + assert.throws(() => new Bash({ fs: partial }), /missing the required method 'readDir'/); +}); + +test("host fs: a class instance satisfies the method check via its prototype", () => { + assert.doesNotThrow(() => new Bash({ fs: new FakeHost() })); +}); + +test("host fs: a non-object fs is rejected", () => { + assert.throws(() => new Bash({ fs: 42 }), /must be an object/); +}); + +test("host fs: every call actually reached the host", async () => { + const host = new FakeHost({ "/workspace/a.txt": "x\n" }); + await hostBash(host).execute("cat /workspace/a.txt > /workspace/b.txt"); + assert.ok(host.calls.includes("read"), "expected a host read"); + assert.ok(host.calls.includes("write"), "expected a host write"); +}); diff --git a/crates/bashkit-wasm/js/index.d.ts b/crates/bashkit-wasm/js/index.d.ts index fa2e9768f..63595ca8c 100644 --- a/crates/bashkit-wasm/js/index.d.ts +++ b/crates/bashkit-wasm/js/index.d.ts @@ -48,6 +48,73 @@ export declare const ExecutionProfile: Readonly<{ Interactive: "interactive"; }>; +/** Entry kind reported by a {@link HostFileSystem}. */ +export type HostFileType = "file" | "dir" | "directory" | "symlink" | "fifo"; + +/** Metadata a {@link HostFileSystem} reports for one entry. */ +export interface HostStat { + readonly type: HostFileType; + /** Size in bytes. Defaults to 0. */ + readonly size?: number; + /** Unix permission bits. Defaults to 0o644 for files, 0o755 for directories. */ + readonly mode?: number; + /** Last modification time, ms since the epoch. Defaults to now. */ + readonly mtimeMs?: number; +} + +/** One directory entry: a {@link HostStat} plus its name. */ +export interface HostDirent extends HostStat { + readonly name: string; +} + +/** + * A filesystem the embedder owns, used in place of the built-in in-memory VFS. + * + * Every method may return its value directly or as a `Promise`. Because a host + * call can suspend the interpreter, a `Bash` constructed with `fs` must be + * driven with {@link Bash.execute}; {@link Bash.executeSync} and the + * synchronous `bash.readFile(...)` helpers report the suspension instead of + * blocking. + * + * The host implements raw storage only. POSIX semantics — parent-directory + * checks, "is a directory" errors, symlink resolution — are enforced above it, + * so hosts stay small. + * + * To surface a bash-accurate error, throw (or reject with) an `Error` carrying + * a `code` property: `ENOENT`, `EEXIST`, `EACCES`, `EPERM`, `EISDIR`, + * `ENOTDIR`, `ENOTEMPTY`, `EXDEV`, or `ENOSYS`. Anything else becomes a generic + * I/O error carrying the thrown message. + */ +export interface HostFileSystem { + /** Read raw bytes. Throw `ENOENT` when absent. */ + read(path: string): Uint8Array | string | Promise; + /** Write raw bytes, replacing any existing content. */ + write(path: string, content: Uint8Array): void | Promise; + /** Create a directory, with parents when `recursive`. */ + mkdir(path: string, recursive: boolean): void | Promise; + /** Remove an entry, with contents when `recursive`. */ + remove(path: string, recursive: boolean): void | Promise; + /** Metadata for one entry. Throw `ENOENT` when absent. */ + stat(path: string): HostStat | Promise; + /** Directory listing. Throw `ENOENT` when absent. */ + readDir(path: string): HostDirent[] | Promise; + /** Whether a path exists. Must not throw for a plain miss. */ + exists(path: string): boolean | Promise; + + /** Optional. Synthesized as read-modify-write when omitted. */ + append?(path: string, content: Uint8Array): void | Promise; + /** Optional. Synthesized as read-then-write when omitted. */ + copy?(from: string, to: string): void | Promise; + /** Optional. Synthesized as copy-then-remove when omitted. */ + rename?(from: string, to: string): void | Promise; + /** Optional. Scripts that create symlinks fail with `ENOSYS` when omitted. */ + symlink?(target: string, link: string): void | Promise; + /** Optional. Reading a symlink fails with `ENOSYS` when omitted. */ + readLink?(path: string): string | Promise; + /** Optional. Accepted and ignored when omitted, so `chmod +x` still works. */ + chmod?(path: string, mode: number): void | Promise; +} + /** Options for constructing a {@link Bash} instance. */ export interface BashOptions { /** Resource-policy baseline; individual limit options override it. */ @@ -68,6 +135,12 @@ export interface BashOptions { maxMemory?: number; /** Pre-created files seeded into the virtual filesystem (string contents). */ files?: Record; + /** + * Filesystem the embedder owns, used instead of the built-in in-memory VFS. + * Cannot be combined with {@link BashOptions.files} — seed through the host + * instead. Requires {@link Bash.execute} (not `executeSync`). + */ + fs?: HostFileSystem; /** JS callbacks registered as bash builtins. */ customBuiltins?: Record; } diff --git a/crates/bashkit-wasm/src/hostfs.rs b/crates/bashkit-wasm/src/hostfs.rs new file mode 100644 index 000000000..9ddbeefe4 --- /dev/null +++ b/crates/bashkit-wasm/src/hostfs.rs @@ -0,0 +1,363 @@ +//! Host-backed filesystem bridge for the wasm bindings. +//! +//! Important decisions (see `knowledge/runtimes/browser-package.md`): +//! +//! - **The host owns the bytes.** `new Bash({ fs })` replaces the in-memory VFS +//! with a JS object the embedder supplies, so the interpreter reads and writes +//! through the embedder's store (a Durable Object, IndexedDB, an OPFS handle) +//! instead of a copy. No seeding pass, no write-back diff, no size ceiling +//! beyond the host's own. +//! - **Async only.** Every host call may return a `Promise`, so a script that +//! touches the filesystem suspends. `executeSync` cannot await and reports the +//! suspension; embedders with a host fs use `execute()`. +//! - **`Send` bridging.** `js_sys` values and `JsFuture` are `!Send` while +//! `FsBackend` is `Send + Sync`. Same treatment as `JsBuiltin`: build and call +//! inside a synchronous scope, then cross the await point through a +//! `SendWrapper`, which is sound on single-threaded wasm. +//! - **`FsBackend`, not `FileSystem`.** The host implements raw storage; POSIX +//! semantics (parent checks, type checks, symlink resolution) stay in +//! `PosixFs` so every host gets them for free and hosts stay small. +//! - **Optional methods degrade, they do not fail silently.** `append`, `copy`, +//! `rename`, and `chmod` are synthesized from the required primitives when the +//! host omits them; `symlink` / `readLink` report `Unsupported` because they +//! cannot be faked. + +use std::io::ErrorKind; +use std::path::{Path, PathBuf}; + +use bashkit::{DirEntry, FileType, FsBackend, Metadata, Result as BashkitResult, async_trait}; +use send_wrapper::SendWrapper; +use wasm_bindgen::prelude::*; +// `Metadata` timestamps are `web_time::SystemTime` on wasm32 (bashkit's +// `time_compat` alias, which is crate-private). Depend on the same crate at the +// same workspace pin so the types unify. +use web_time::{SystemTime, UNIX_EPOCH}; + +/// Methods a host filesystem must provide. Missing ones make construction fail +/// loudly instead of surfacing as mid-script errors. +const REQUIRED: &[&str] = &[ + "read", "write", "mkdir", "remove", "stat", "readDir", "exists", +]; + +/// A `FsBackend` that forwards every operation to a JS object. +pub struct HostFs { + obj: SendWrapper, +} + +impl HostFs { + /// Validate and adopt a JS host filesystem. + pub fn new(value: JsValue) -> Result { + if !value.is_object() { + return Err(JsError::new("fs must be an object")); + } + let obj = js_sys::Object::from(value); + for name in REQUIRED { + if method(&obj, name).is_none() { + return Err(JsError::new(&format!( + "fs is missing the required method '{name}'" + ))); + } + } + Ok(Self { + obj: SendWrapper::new(obj), + }) + } + + fn has(&self, name: &str) -> bool { + method(&self.obj, name).is_some() + } + + /// Invoke a host method with already-`Send` arguments. + /// + /// Arguments are converted to `JsValue` inside this synchronous scope so no + /// `!Send` value is ever live across the await in [`HostFs::call`]. + fn invoke(&self, name: &str, args: &[Arg]) -> Result { + let func = method(&self.obj, name) + .ok_or_else(|| JsValue::from(JsError::new(&format!("fs.{name} is not a function"))))?; + let js_args = js_sys::Array::new(); + for arg in args { + js_args.push(&arg.to_js()); + } + let ret = js_sys::Reflect::apply(&func, &self.obj, &js_args)?; + match ret.dyn_into::() { + Ok(promise) => Ok(Invoked::Pending(SendWrapper::new( + wasm_bindgen_futures::JsFuture::from(promise), + ))), + Err(value) => Ok(Invoked::Done(SendWrapper::new(value))), + } + } + + /// Call a host method, awaiting the result when it returns a `Promise`. + async fn call(&self, name: &str, args: &[Arg]) -> BashkitResult> { + let invoked = self.invoke(name, args).map_err(|e| host_error(name, &e))?; + match invoked { + Invoked::Done(value) => Ok(value), + Invoked::Pending(future) => match future.await { + Ok(value) => Ok(SendWrapper::new(value)), + Err(e) => Err(host_error(name, &e)), + }, + } + } + + async fn call_unit(&self, name: &str, args: &[Arg]) -> BashkitResult<()> { + self.call(name, args).await.map(|_| ()) + } +} + +enum Invoked { + Done(SendWrapper), + Pending(SendWrapper), +} + +/// A host-call argument. `Send` by construction so it can live in the async +/// method bodies; converted to `JsValue` only inside [`HostFs::invoke`]. +enum Arg { + Str(String), + Bytes(Vec), + Bool(bool), + Num(f64), +} + +impl Arg { + fn path(path: &Path) -> Self { + Arg::Str(path.to_string_lossy().into_owned()) + } + + fn to_js(&self) -> JsValue { + match self { + Arg::Str(s) => JsValue::from_str(s), + Arg::Bytes(b) => js_sys::Uint8Array::from(b.as_slice()).into(), + Arg::Bool(b) => JsValue::from_bool(*b), + Arg::Num(n) => JsValue::from_f64(*n), + } + } +} + +#[async_trait] +impl FsBackend for HostFs { + async fn read(&self, path: &Path) -> BashkitResult> { + let value = self.call("read", &[Arg::path(path)]).await?; + to_bytes(&value) + .ok_or_else(|| invalid("fs.read must resolve to a Uint8Array, ArrayBuffer, or string")) + } + + async fn write(&self, path: &Path, content: &[u8]) -> BashkitResult<()> { + self.call_unit("write", &[Arg::path(path), Arg::Bytes(content.to_vec())]) + .await + } + + async fn append(&self, path: &Path, content: &[u8]) -> BashkitResult<()> { + if self.has("append") { + return self + .call_unit("append", &[Arg::path(path), Arg::Bytes(content.to_vec())]) + .await; + } + // Synthesized: read-modify-write. A host that cares about concurrent + // appends implements `append` itself. + let mut merged = match self.read(path).await { + Ok(existing) => existing, + Err(e) if is_not_found(&e) => Vec::new(), + Err(e) => return Err(e), + }; + merged.extend_from_slice(content); + self.write(path, &merged).await + } + + async fn mkdir(&self, path: &Path, recursive: bool) -> BashkitResult<()> { + self.call_unit("mkdir", &[Arg::path(path), Arg::Bool(recursive)]) + .await + } + + async fn remove(&self, path: &Path, recursive: bool) -> BashkitResult<()> { + self.call_unit("remove", &[Arg::path(path), Arg::Bool(recursive)]) + .await + } + + async fn stat(&self, path: &Path) -> BashkitResult { + let value = self.call("stat", &[Arg::path(path)]).await?; + parse_metadata(&value).ok_or_else(|| invalid("fs.stat must resolve to a stat object")) + } + + async fn read_dir(&self, path: &Path) -> BashkitResult> { + let value = self.call("readDir", &[Arg::path(path)]).await?; + parse_dir_entries(&value) + .ok_or_else(|| invalid("fs.readDir must resolve to an array of dirent objects")) + } + + async fn exists(&self, path: &Path) -> BashkitResult { + let value = self.call("exists", &[Arg::path(path)]).await?; + Ok(value.as_bool().unwrap_or(false)) + } + + async fn rename(&self, from: &Path, to: &Path) -> BashkitResult<()> { + if self.has("rename") { + return self + .call_unit("rename", &[Arg::path(from), Arg::path(to)]) + .await; + } + self.copy(from, to).await?; + self.remove(from, false).await + } + + async fn copy(&self, from: &Path, to: &Path) -> BashkitResult<()> { + if self.has("copy") { + return self + .call_unit("copy", &[Arg::path(from), Arg::path(to)]) + .await; + } + let content = self.read(from).await?; + self.write(to, &content).await + } + + async fn symlink(&self, target: &Path, link: &Path) -> BashkitResult<()> { + if self.has("symlink") { + return self + .call_unit("symlink", &[Arg::path(target), Arg::path(link)]) + .await; + } + Err(unsupported("symlink")) + } + + async fn read_link(&self, path: &Path) -> BashkitResult { + if !self.has("readLink") { + return Err(unsupported("readLink")); + } + let value = self.call("readLink", &[Arg::path(path)]).await?; + value + .as_string() + .map(PathBuf::from) + .ok_or_else(|| invalid("fs.readLink must resolve to a string")) + } + + async fn chmod(&self, path: &Path, mode: u32) -> BashkitResult<()> { + if !self.has("chmod") { + // Hosts without a permission model accept the call and keep the + // mode they already report from `stat`. Failing here would break + // ordinary scripts (`chmod +x build.sh`) for no benefit. + return Ok(()); + } + self.call_unit("chmod", &[Arg::path(path), Arg::Num(f64::from(mode))]) + .await + } +} + +// --------------------------------------------------------------------------- +// JS <-> Rust conversions +// --------------------------------------------------------------------------- + +fn method(obj: &js_sys::Object, name: &str) -> Option { + js_sys::Reflect::get(obj, &JsValue::from_str(name)) + .ok() + .and_then(|v| v.dyn_into::().ok()) +} + +fn to_bytes(value: &JsValue) -> Option> { + if let Some(text) = value.as_string() { + return Some(text.into_bytes()); + } + if let Some(array) = value.dyn_ref::() { + return Some(array.to_vec()); + } + if let Some(buffer) = value.dyn_ref::() { + return Some(js_sys::Uint8Array::new(buffer).to_vec()); + } + None +} + +fn get(value: &JsValue, key: &str) -> Option { + js_sys::Reflect::get(value, &JsValue::from_str(key)).ok() +} + +fn file_type(value: &JsValue) -> Option { + match get(value, "type")?.as_string()?.as_str() { + "file" => Some(FileType::File), + "dir" | "directory" => Some(FileType::Directory), + "symlink" => Some(FileType::Symlink), + "fifo" => Some(FileType::Fifo), + _ => None, + } +} + +fn parse_metadata(value: &JsValue) -> Option { + if !value.is_object() { + return None; + } + let file_type = file_type(value)?; + let default_mode = if file_type.is_dir() { 0o755 } else { 0o644 }; + Some(Metadata { + file_type, + size: get(value, "size").and_then(|v| v.as_f64()).unwrap_or(0.0) as u64, + mode: get(value, "mode") + .and_then(|v| v.as_f64()) + .map_or(default_mode, |m| m as u32), + modified: get(value, "mtimeMs") + .and_then(|v| v.as_f64()) + .map_or_else(SystemTime::now, from_epoch_ms), + created: SystemTime::now(), + }) +} + +fn from_epoch_ms(ms: f64) -> SystemTime { + let ms = if ms.is_finite() && ms > 0.0 { ms } else { 0.0 }; + UNIX_EPOCH + std::time::Duration::from_millis(ms as u64) +} + +fn parse_dir_entries(value: &JsValue) -> Option> { + let array: &js_sys::Array = value.dyn_ref::()?; + let mut entries = Vec::with_capacity(array.length() as usize); + for item in array.iter() { + let name = get(&item, "name")?.as_string()?; + entries.push(DirEntry { + name, + metadata: parse_metadata(&item)?, + }); + } + Some(entries) +} + +// --------------------------------------------------------------------------- +// Errors +// --------------------------------------------------------------------------- + +/// Map a thrown/rejected host error onto an `io::ErrorKind` so builtins print +/// the message bash users expect ("No such file or directory", not "host +/// error"). Hosts opt in by setting `error.code`; anything else is `Other`. +fn host_error(method: &str, value: &JsValue) -> bashkit::Error { + let code = get(value, "code").and_then(|v| v.as_string()); + let message = error_message(value).unwrap_or_else(|| format!("fs.{method} failed")); + let kind = match code.as_deref() { + Some("ENOENT") => ErrorKind::NotFound, + Some("EEXIST") => ErrorKind::AlreadyExists, + Some("EACCES") | Some("EPERM") => ErrorKind::PermissionDenied, + Some("EISDIR") => ErrorKind::IsADirectory, + Some("ENOTDIR") => ErrorKind::NotADirectory, + Some("ENOTEMPTY") => ErrorKind::DirectoryNotEmpty, + Some("EXDEV") => ErrorKind::CrossesDevices, + Some("ENOSYS") => ErrorKind::Unsupported, + _ => ErrorKind::Other, + }; + std::io::Error::new(kind, message).into() +} + +fn error_message(value: &JsValue) -> Option { + if let Some(text) = value.as_string() { + return Some(text); + } + get(value, "message")?.as_string() +} + +fn invalid(message: &str) -> bashkit::Error { + std::io::Error::new(ErrorKind::InvalidData, message.to_string()).into() +} + +fn unsupported(method: &str) -> bashkit::Error { + std::io::Error::new( + ErrorKind::Unsupported, + format!("the host filesystem does not implement {method}"), + ) + .into() +} + +fn is_not_found(error: &bashkit::Error) -> bool { + matches!(error, bashkit::Error::Io(e) if e.kind() == ErrorKind::NotFound) +} diff --git a/crates/bashkit-wasm/src/lib.rs b/crates/bashkit-wasm/src/lib.rs index 6cc1ca88a..8e5669ffc 100644 --- a/crates/bashkit-wasm/src/lib.rs +++ b/crates/bashkit-wasm/src/lib.rs @@ -26,7 +26,7 @@ use std::sync::{Arc, Mutex}; use bashkit::{ Bash as CoreBash, Builtin, BuiltinContext, CheckoutPolicy, CommitOptions, - ExecResult as CoreExecResult, FileSystem as FileSystemTrait, ObjectId, OutputCallback, + ExecResult as CoreExecResult, FileSystem as FileSystemTrait, ObjectId, OutputCallback, PosixFs, async_trait, }; use futures_util::future::FutureExt; @@ -35,6 +35,9 @@ use serde::Serialize; use wasm_bindgen::prelude::*; use wasm_bindgen_futures::{JsFuture, future_to_promise}; +mod hostfs; +use hostfs::HostFs; + /// Install a panic hook that forwards Rust panics to `console.error` with a /// readable message and stack, instead of the default unhelpful /// `RuntimeError: unreachable`. @@ -367,6 +370,11 @@ struct Config { max_memory: Option, files: Vec<(String, String)>, builtins: Vec, + /// Host-backed filesystem from `options.fs`, already wrapped in `PosixFs`. + /// Built once and shared across `reset()` because the bytes live on the + /// host, not in the interpreter — resetting the interpreter must not + /// disturb the embedder's store. + host_fs: Option>, } struct CustomBuiltinConfig { @@ -811,6 +819,10 @@ fn build_core(config: &Config, sync_flag: &Arc) -> Result Result { max_memory: None, files: Vec::new(), builtins: Vec::new(), + host_fs: None, }); } if !options.is_object() { @@ -917,6 +930,24 @@ fn parse_options(options: &JsValue) -> Result { let env = read_string_map(options, "env")?; let files = read_string_map(options, "files")?; + // A host filesystem replaces the in-memory VFS wholesale. `files` seeds the + // VFS through a synchronous `now_or_never` write, which a host that answers + // with a Promise can never satisfy — so reject the combination instead of + // silently dropping the seed. Hosts write their own seed data directly. + let host_fs = match js_sys::Reflect::get(options, &JsValue::from_str("fs")) { + Ok(value) if !value.is_undefined() && !value.is_null() => { + if !files.is_empty() { + return Err(JsError::new( + "options.files cannot be combined with options.fs — write seed \ + files through the host filesystem instead", + )); + } + let backend = HostFs::new(value)?; + Some(Arc::new(PosixFs::new(backend)) as Arc) + } + _ => None, + }; + // customBuiltins: { [name]: (ctx) => string | Promise } let mut builtins = Vec::new(); if let Ok(cb) = js_sys::Reflect::get(options, &JsValue::from_str("customBuiltins")) @@ -951,6 +982,7 @@ fn parse_options(options: &JsValue) -> Result { max_memory: get_usize("maxMemory"), files, builtins, + host_fs, }) } diff --git a/justfile b/justfile index eab494bb8..14df25254 100644 --- a/justfile +++ b/justfile @@ -22,7 +22,7 @@ build: # Requires: rustup target add wasm32-unknown-unknown; cargo install wasm-bindgen-cli build-wasm: bash crates/bashkit-wasm/scripts/build.sh release - node --test crates/bashkit-wasm/__test__/bashkit-wasm.test.mjs + node --test "crates/bashkit-wasm/__test__/*.test.mjs" # Run all tests (including fail-point tests) test: diff --git a/knowledge/runtimes/browser-package.md b/knowledge/runtimes/browser-package.md index 50e4ff0d7..748a001f9 100644 --- a/knowledge/runtimes/browser-package.md +++ b/knowledge/runtimes/browser-package.md @@ -66,6 +66,46 @@ Absent (need sockets, threads, or a host FS the browser sandbox lacks): mounts, and native `interop`. Reach the network from a custom builtin that calls the app's own `fetch` instead. +## Host-backed filesystem (`new Bash({ fs })`) + +Embedders can replace the in-memory VFS with their own store by passing `fs`. +The bridge (`crates/bashkit-wasm/src/hostfs.rs`) implements `FsBackend` over a JS +object and wraps it in `PosixFs`, so hosts supply raw storage and inherit POSIX +semantics (parent checks, type checks, symlink resolution) for free. + +Decisions: + +- **Live, not copied.** Every read/write during a run is a call into the host + object. No seeding pass and no write-back diff, so there is no workspace-size + ceiling beyond the host's own and no lost-update window between runs. +- **`FsBackend`, not `FileSystem`.** Fourteen raw operations instead of the full + POSIX surface; seven are required (`read`, `write`, `mkdir`, `remove`, `stat`, + `readDir`, `exists`) and validated at construction so a missing method fails + loudly rather than mid-script. `append`, `copy`, `rename` are synthesized from + the required set when omitted; `chmod` is accepted and ignored (so `chmod +x` + works against hosts with no permission model); `symlink` / `readLink` report + `Unsupported` because they cannot be faked. +- **Async only.** Host methods may return a `Promise`, so filesystem access + suspends the interpreter. `executeSync` and the synchronous `bash.readFile(...)` + helpers report the suspension instead of blocking — a host filesystem implies + `execute()`. +- **`files` + `fs` is rejected.** Seeding writes through `now_or_never`, which a + promise-returning host can never satisfy. Rejecting at construction beats + silently dropping the seed. +- **Errors map by `code`.** A thrown/rejected `Error` with `code` (`ENOENT`, + `EEXIST`, `EACCES`, `EPERM`, `EISDIR`, `ENOTDIR`, `ENOTEMPTY`, `EXDEV`, + `ENOSYS`) becomes the matching `io::ErrorKind`, so builtins that branch on kind + (`ls`, `test -f`) behave as they do over the built-in VFS. Uncoded errors carry + the host's message through as a generic I/O error. +- **No implicit `/dev/null`.** With a host filesystem there is no in-memory VFS + underneath; hosts that expect redirects to `/dev/null` provide the entry. +- **`Send` bridging.** Same `SendWrapper` treatment as `JsBuiltin`: `!Send` + `js_sys` values live only inside a synchronous scope, and the await crosses a + `SendWrapper`. + +Covered by `__test__/host-fs.test.mjs`, whose fake host resolves every method on +a later microtask so the suspend/resume path is the one under test. + ## Execution model `wasm32-unknown-unknown` is single-threaded; the whole future chain runs on the @@ -111,6 +151,8 @@ core already gates off under `cfg(target_family = "wasm")` (see `pkg/` at build time). - `scripts/build.sh` — `cargo build` → `wasm-bindgen --target web` → optional `wasm-opt -Oz`, emitting `pkg/`. +- `src/hostfs.rs` — the `FsBackend` bridge behind the `fs` option. +- `__test__/host-fs.test.mjs` — host-filesystem suite (async fake host). - `__test__/bashkit-wasm.test.mjs` — headless Node integration suite (`node --test`) that feeds the `.wasm` bytes to init (no fetch, no headers), proving the no-configuration contract and covering sync/async execution, the @@ -123,7 +165,7 @@ core already gates off under `cfg(target_family = "wasm")` (see rustup target add wasm32-unknown-unknown cargo install wasm-bindgen-cli bash crates/bashkit-wasm/scripts/build.sh # -> pkg/ -node --test crates/bashkit-wasm/__test__/bashkit-wasm.test.mjs # verify +node --test "crates/bashkit-wasm/__test__/*.test.mjs" # verify ``` `--target web` output is a bundler-agnostic ES module; the consumer calls @@ -153,6 +195,7 @@ pattern as `publish-js.yml`. Browser example smoke testing writes a file under redirects (`print > f`, `getline < f`) drive the VFS future to completion with `now_or_never` rather than a writer thread. Correct because the browser build only ever runs over the in-memory VFS, which never suspends. -- `executeSync` cannot await JS callbacks; use `execute()` for async builtins. +- `executeSync` cannot await JS callbacks; use `execute()` for async builtins or + any host filesystem. - Custom-builtin `ctx` exposes `{ name, argv, stdin, env, cwd, fs }`, where `fs` is a live handle to the same VFS the script sees (mirrors the napi bindings). From 96ff8ce83cfaf397dd7f2ef6485384f3876e439e Mon Sep 17 00:00:00 2001 From: Mykhailo Chalyi Date: Thu, 6 Aug 2026 00:40:46 +0000 Subject: [PATCH 2/3] fix(wasm): treat a non-boolean fs.exists answer as an error MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A host that answers `undefined` is broken, and reading that as "missing" turns every existence probe into a silent miss — a redirect reports the workspace as gone rather than reporting the host. Fail with a message that names the contract instead. Also documents the host filesystem as TM-FS-017: it widens the sandbox to whatever the embedder's object exposes, and its bytes live outside the VFS quotas, so scoping and storage limits are the embedder's. --- crates/bashkit-wasm/__test__/host-fs.test.mjs | 11 +++++++++++ crates/bashkit-wasm/src/hostfs.rs | 8 +++++++- crates/bashkit/docs/threat-model.md | 9 ++++++--- knowledge/security/threat-model.md | 5 ++++- 4 files changed, 28 insertions(+), 5 deletions(-) diff --git a/crates/bashkit-wasm/__test__/host-fs.test.mjs b/crates/bashkit-wasm/__test__/host-fs.test.mjs index 850012b42..72468d050 100644 --- a/crates/bashkit-wasm/__test__/host-fs.test.mjs +++ b/crates/bashkit-wasm/__test__/host-fs.test.mjs @@ -275,6 +275,17 @@ test("host fs: a host error without a code still surfaces its message", async () assert.match(r.stderr, /host store unreachable/); }); +test("host fs: a malformed host answer is an error, not a silent miss", async () => { + const host = new FakeHost({ "/workspace/there.txt": "x\n" }); + host.exists = async () => undefined; + // A redirect probes the parent directory through `exists`. Reading + // `undefined` as "missing" would report the workspace as gone; + // the host is what's broken, and the message has to say so. + const r = await hostBash(host).execute("echo x > /workspace/new.txt"); + assert.notEqual(r.exitCode, 0); + assert.match(r.stderr, /must resolve to a boolean/); +}); + test("host fs: a rejected write fails the command, not the run", async () => { const host = new FakeHost(); host.write = async () => { diff --git a/crates/bashkit-wasm/src/hostfs.rs b/crates/bashkit-wasm/src/hostfs.rs index 9ddbeefe4..16f73c103 100644 --- a/crates/bashkit-wasm/src/hostfs.rs +++ b/crates/bashkit-wasm/src/hostfs.rs @@ -186,7 +186,13 @@ impl FsBackend for HostFs { async fn exists(&self, path: &Path) -> BashkitResult { let value = self.call("exists", &[Arg::path(path)]).await?; - Ok(value.as_bool().unwrap_or(false)) + // Not `unwrap_or(false)`: a host that answers with `undefined` + // is broken, and reading that as "missing" turns every probe + // into a silent miss — `test -f`, PATH lookups, and `mkdir -p` + // would all quietly do the wrong thing. + value + .as_bool() + .ok_or_else(|| invalid("fs.exists must resolve to a boolean")) } async fn rename(&self, from: &Path, to: &Path) -> BashkitResult<()> { diff --git a/crates/bashkit/docs/threat-model.md b/crates/bashkit/docs/threat-model.md index 0df401630..25e0463f1 100644 --- a/crates/bashkit/docs/threat-model.md +++ b/crates/bashkit/docs/threat-model.md @@ -840,10 +840,12 @@ object identity covers *decoded* content, not its compressed framing: a host may recompress its store freely, while bytes appended after a compressed stream are rejected. -### RealFs Mount Security (TM-FS-*) +### Host Filesystem Security (TM-FS-*) -The `realfs` feature can mount real host directories into the VFS. Mounts are -read-only by default and gated by an allowlist. +Two features let a script reach storage outside the in-memory VFS: the `realfs` +feature, which mounts real host directories (read-only by default, gated by an +allowlist), and the wasm bindings' `new Bash({ fs })`, which routes the VFS into +an embedder-supplied JavaScript object. | Threat | Attack Example | Mitigation | Status | |--------|---------------|------------|--------| @@ -851,6 +853,7 @@ read-only by default and gated by an allowlist. | Partial filesystem mutation (TM-FS-014) | Failed write/copy or cross-mount move leaves corruption, duplication, or retained quota | Failure-atomic `FileSystem` contract; RealFs sibling staging; MountableFs destination rollback; NamespaceFs cross-device rejection; shared conformance + failpoint tests | MITIGATED | | Partial tar extraction (TM-FS-015) | A late unsafe or malformed entry leaves earlier files behind | Validate the complete archive and file limits before the first VFS mutation | MITIGATED | | yq in-place partial update (TM-FS-016) | A failed transform or write truncates the source file | Evaluate and serialize before writing; random sibling temporary file, mode preservation, and rename-on-success | MITIGATED | +| JS host filesystem widens the sandbox (TM-FS-017) | `new Bash({ fs })` gives a script whatever the embedder's object exposes | Paths are normalized by `PosixFs` before any host call, so traversal cannot select a path the embedder did not scope. The host object *is* the boundary and is the embedder's to scope; its bytes also live outside the VFS quotas, so the embedder owns the storage limit | ACCEPTED (opt-in, embedder-scoped) | ### Unicode Security (TM-UNI-*) diff --git a/knowledge/security/threat-model.md b/knowledge/security/threat-model.md index e3595cef4..a210272be 100644 --- a/knowledge/security/threat-model.md +++ b/knowledge/security/threat-model.md @@ -153,7 +153,9 @@ at subsystem or descendant boundaries; exhaustion poisons that request. Separate > writes go straight to the host with no byte/count quota. This is by design > (`--mount-rw` is sandbox-breaking, see TM-ESC-030), but it means a script > with a writable real-FS mount can exhaust host disk/inodes. Use -> `--mount-ro` for untrusted scripts. +> `--mount-ro` for untrusted scripts. The same applies to a JS host +> filesystem in the wasm bindings (`new Bash({ fs })`, TM-FS-017): bytes live +> in the embedder's store, so the embedder owns the quota. **TM-DOS-034**: Fixed. `InMemoryFs::append_file()` now uses a single write lock for the entire read-check-write operation, preventing TOCTOU races. See `fs/memory.rs:940-942`. @@ -319,6 +321,7 @@ panicked. Resolved with `wrapping_*` ops, masked shift amounts, clamped exponent | TM-FS-014 | Partial filesystem mutation | Failed write/copy or copy-delete move corrupts/replaces a destination, duplicates a source, or consumes retained quota | `FileSystem` failure-atomicity contract; locked in-memory rename; MountableFs restores cross-mount destinations while NamespaceFs rejects cross-mount rename; RealFs stages and flushes sibling files before rename; failpoint and conformance regressions | **MITIGATED** | | TM-FS-015 | Partial archive extraction | A late traversal, malformed header, or size failure leaves earlier attacker-controlled files behind | Tar validates the complete archive and per-file limits before its first VFS mutation; conformance regression uses a valid entry followed by traversal | **MITIGATED** | | TM-FS-016 | yq in-place partial or destructive update | Parse, evaluation, serialization, or write failure truncates the source; predictable temporary names permit collisions | Complete evaluation and bounded serialization first; write a random sibling temporary file, preserve mode, and rename only after success; failpoint regressions cover allocation, all backend-write classes, chmod, rename, original-byte retention, and temporary cleanup | **MITIGATED** | +| TM-FS-017 | JS host filesystem widens the sandbox to embedder storage | `new Bash({ fs })` in the wasm bindings routes every VFS operation into embedder-supplied JS, so a script reaches whatever that object exposes | Paths are normalized by `PosixFs` before any host call, so traversal cannot select a path the embedder did not scope; the host object is the security boundary and is the embedder's to scope (mount root, allowlist, read-only). Reads and writes bypass the in-memory quotas — see the FS-quota scope note in §1 | **ACCEPTED** (embedder-scoped, opt-in) | **Current Risk**: MEDIUM - Two open escape vectors (TM-ESC-012, TM-ESC-013) need remediation From 23e47627a45556506693d4ad16135110396df328 Mon Sep 17 00:00:00 2001 From: Mykhailo Chalyi Date: Thu, 6 Aug 2026 03:49:58 +0000 Subject: [PATCH 3/3] docs(fs): document the JS host-backed filesystem --- docs/filesystem.md | 38 ++++++++++++++++++++++++++++++++++++++ docs/start-node.md | 4 ++++ 2 files changed, 42 insertions(+) diff --git a/docs/filesystem.md b/docs/filesystem.md index c6ea5eb1f..85ecbf681 100644 --- a/docs/filesystem.md +++ b/docs/filesystem.md @@ -142,6 +142,40 @@ are visible as directories. Files and symlinks can be copied across mounts; cross-mount rename reports a typed cross-device error because copy-delete is not atomic. +## Host-backed filesystem (JS) + +The wasm bindings accept an `fs` object, so scripts run directly against storage +you own — a Durable Object, an OPFS handle, IndexedDB — instead of the in-memory +VFS. Nothing is copied in or diffed back out: every read and write during the run +is a call into your object. + +```js +const bash = new Bash({ cwd: "/workspace", fs: myHost }); +const r = await bash.execute("grep -rl TODO . | head -5"); +``` + +Seven methods are required — `read`, `write`, `mkdir`, `remove`, `stat`, +`readDir`, `exists` — and each may return its value directly or as a `Promise`. +`append`, `copy`, `rename`, and `chmod` are optional and synthesized from the +required primitives when omitted. Your host implements raw storage only; POSIX +semantics (parent-directory checks, "is a directory", symlink resolution) are +enforced above it. Throw an `Error` carrying a `code` (`ENOENT`, `EEXIST`, +`EACCES`, …) so bash reports the failure the way a real shell does. + +Two contract notes: + +- **`execute()` only.** A host call can suspend the interpreter, and + `executeSync` cannot await — it reports the suspension instead of blocking. +- **`files` is rejected alongside `fs`.** Seeding writes through the VFS + synchronously, which a promise-returning host can never satisfy. + +The host object is the security boundary and is yours to scope (mount root, +allowlist, read-only): paths are normalized before any host call, so traversal +cannot select a path you did not expose, but the sandbox reaches whatever the +object exposes. Reads and writes bypass the in-memory quotas. See +[`@everruns/bashkit-wasm`](https://github.com/everruns/bashkit/blob/main/crates/bashkit-wasm/README.md#host-backed-filesystem) +for the full method table and error-code list. + ## Special files and symlinks - **`/dev/null`** is handled at the interpreter level (not the filesystem), so a @@ -161,6 +195,10 @@ mounts: [{ host_path, vfs_path?, writable? }] # real FS (read-only by default) readonly_filesystem: bool # deny all VFS mutations after setup ``` +The wasm bindings additionally accept `fs` — an embedder-supplied filesystem +that replaces the in-memory VFS entirely. See +[Host-backed filesystem (JS)](#host-backed-filesystem-js) above. + ## See also - [Security](security.md) — the boundaries built on top of the VFS. diff --git a/docs/start-node.md b/docs/start-node.md index 6f413e4d0..64bbb392a 100644 --- a/docs/start-node.md +++ b/docs/start-node.md @@ -47,6 +47,10 @@ const bash = new Bash({ See [Sandbox configuration & limits](configuration.md) for the full set. +To run against storage you own rather than the in-memory filesystem, pass an +`fs` object — see +[Host-backed filesystem](filesystem.md#host-backed-filesystem-js). + ## Examples Runnable Node examples in the repo: