Codex embed browser harness - #517
Conversation
…e embedded agent)
…LABEL, wrapper sources /browser's browser-env
…pt (tool calls, command output, agent messages) streams to stdout instead of only the final response
…tdout transcript, not the run-root banner
…ult model (not hardcoded gpt-5.4), strip opinionated AGENTS.md preamble so the skill drives the workflow
…browser-harness-tui); resolve it only from that path — remove all BROWSER_HARNESS_CODEX_* env vars and sibling-dir lookup
✅ Skill review passedReviewed 1 file(s) — no findings. |
| print(json.dumps({"status": "stored", "domain": _norm_domain(args.domain), "name": args.name.strip(), "kind": kind})) | ||
| return 0 | ||
| if args.command == "list": | ||
| print(json.dumps(list_secrets(), indent=2)) |
|
|
||
| captured = capsys.readouterr() | ||
| assert captured.out == "" | ||
| assert "https://live.example" in captured.err |
There was a problem hiding this comment.
12 issues found across 16 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/browser_harness/daemon.py">
<violation number="1" location="src/browser_harness/daemon.py:98">
P3: `_cloud_only()` treats `BH_CLOUD_ONLY=FALSE` (all caps) as truthy rather than falsy. While the current tuple covers the most common falsy values, `FALSE` all-caps is a natural user input and would unexpectedly enable cloud-only mode. Consider normalizing the value to lowercase before checking, or using `os.environ.get("BH_CLOUD_ONLY", "").lower() not in ("", "0", "false")` to make the check case-insensitive.</violation>
<violation number="2" location="src/browser_harness/daemon.py:103">
P2: `BH_CLOUD_ONLY` can be bypassed with equivalent IPv6 loopback spellings, so a local CDP endpoint is accepted despite the cloud-only guard. Normalize parsed IP hosts and use `ipaddress.ip_address(host).is_loopback` (while retaining the hostname check for `localhost`) before accepting either endpoint.</violation>
</file>
<file name="src/browser_harness/agent.py">
<violation number="1" location="src/browser_harness/agent.py:73">
P2: Concurrent launches in the same second share one run workspace and can overwrite each other's browser environment, outputs, and final message. Add a collision-resistant component to the default directory name.</violation>
<violation number="2" location="src/browser_harness/agent.py:109">
P1: Generated wrapper executes shell syntax embedded in `--run-root`; metacharacters run arbitrary commands, while ordinary spaces prevent `browser-env` from loading. Shell-quote every rendered path (including this source path and the final executable/PYTHONPATH paths).</violation>
<violation number="3" location="src/browser_harness/agent.py:184">
P2: `--approval-mode deny-all` is mapped inconsistently between `run_task()` and `launch_tui()`. In `run_task()`, `deny-all` becomes `"on-failure"` (approve on failure — semantically the opposite of "deny all"), while in `launch_tui()` it becomes `"never"` (never approve). The three-way choice `"never"`, `"auto-review"`, `"deny-all"` is identical in both parsers, so a user gets different behavior from the same flag depending on subcommand. Recommend aligning both functions so `deny-all` maps to `"never"` (matching its name) or removing `deny-all` from the choices if it isn't intentionally supported.</violation>
</file>
<file name="src/browser_harness/run.py">
<violation number="1" location="src/browser_harness/run.py:370">
P1: A configured remote CDP endpoint is ignored in cloud-only mode, so a task can provision a new Browser Use browser instead of attaching to the caller-selected endpoint. Let explicit endpoints reach `ensure_daemon()`; daemon validation already rejects loopback endpoints under `BH_CLOUD_ONLY`.</violation>
<violation number="2" location="src/browser_harness/run.py:371">
P2: Cloud-only startup rejects valid stored Browser Use authentication and requires an environment key, unlike the cloud provisioning API it calls. Reuse `_cloud_auth_configured()` so `browser-harness auth login` remains usable.</violation>
</file>
<file name="src/browser_harness/helpers.py">
<violation number="1" location="src/browser_harness/helpers.py:497">
P2: A page with a native dialog open can make `available_secrets()` reveal credential names/kinds for every domain, although a page is still attached. Resolve the attached tab URL when `page_info()` reports a dialog so domain filtering remains active.</violation>
<violation number="2" location="src/browser_harness/helpers.py:525">
P3: `_secret_lookup` compares the caller's `name` argument directly against stored names without stripping whitespace, unlike `set_secret` and `get_secret_value` which both normalize with `.strip()`. If a caller passes `secret(" login-password ")` with accidental spaces, the lookup silently fails even though the value exists.
**Recommendation:** Apply `str(name or "").strip()` before the comparison for consistency with the rest of the API.</violation>
<violation number="3" location="src/browser_harness/helpers.py:542">
P3: `secret()` and `totp()` decrypt the entire secrets store twice — once in `_secret_lookup()` (via `list_secrets()` → `_load()`) and once in `get_secret_value()` (→ `_load()`). While the store is small, this doubles the AES-GCM decryption overhead on every credential lookup.
**Recommendation:** Have `_secret_lookup` accept an optional pre-decrypted data dict, or restructure so that `_secret_lookup` returns the actual value alongside the metadata, avoiding the second `_load()` entirely.</violation>
</file>
<file name="src/browser_harness/secrets.py">
<violation number="1" location="src/browser_harness/secrets.py:53">
P2: Concurrent secrets commands can fail or commit the wrong temporary contents because every writer uses the same `*.tmp` pathname. Use a uniquely created temporary file in the destination directory before `os.replace`.</violation>
<violation number="2" location="src/browser_harness/secrets.py:127">
P2: Overlapping `set`/`remove` commands can silently lose or resurrect credentials because each saves an unlocked stale snapshot. Serialize the complete load–mutate–save transaction with a process-safe lock.</violation>
</file>
Tip: instead of fixing issues one by one fix them all with cubic
Re-trigger cubic
| # the model invokes. | ||
| wrapper.write_text( | ||
| "#!/bin/sh\n" | ||
| f"[ -f {run_root!s}/browser-env ] && . {run_root!s}/browser-env\n" |
There was a problem hiding this comment.
P1: Generated wrapper executes shell syntax embedded in --run-root; metacharacters run arbitrary commands, while ordinary spaces prevent browser-env from loading. Shell-quote every rendered path (including this source path and the final executable/PYTHONPATH paths).
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/agent.py, line 109:
<comment>Generated wrapper executes shell syntax embedded in `--run-root`; metacharacters run arbitrary commands, while ordinary spaces prevent `browser-env` from loading. Shell-quote every rendered path (including this source path and the final executable/PYTHONPATH paths).</comment>
<file context>
@@ -0,0 +1,306 @@
+ # the model invokes.
+ wrapper.write_text(
+ "#!/bin/sh\n"
+ f"[ -f {run_root!s}/browser-env ] && . {run_root!s}/browser-env\n"
+ 'if [ -n "$BENCH_TASK_DIR" ] && [ -x "$BENCH_TASK_DIR/bin/browser-harness" ] '
+ f'&& [ "$BENCH_TASK_DIR/bin/browser-harness" != {str(wrapper)!r} ]; then\n'
</file context>
| if ( | ||
| # In benchmark cloud-only mode (BH_CLOUD_ONLY), local Chrome must never | ||
| # suppress cloud provisioning. | ||
| if _cloud_only(): |
There was a problem hiding this comment.
P1: A configured remote CDP endpoint is ignored in cloud-only mode, so a task can provision a new Browser Use browser instead of attaching to the caller-selected endpoint. Let explicit endpoints reach ensure_daemon(); daemon validation already rejects loopback endpoints under BH_CLOUD_ONLY.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/run.py, line 370:
<comment>A configured remote CDP endpoint is ignored in cloud-only mode, so a task can provision a new Browser Use browser instead of attaching to the caller-selected endpoint. Let explicit endpoints reach `ensure_daemon()`; daemon validation already rejects loopback endpoints under `BH_CLOUD_ONLY`.</comment>
<file context>
@@ -333,7 +365,16 @@ def _run(args):
- if (
+ # In benchmark cloud-only mode (BH_CLOUD_ONLY), local Chrome must never
+ # suppress cloud provisioning.
+ if _cloud_only():
+ if not os.environ.get("BROWSER_USE_API_KEY"):
+ raise RuntimeError("BH_CLOUD_ONLY requires BROWSER_USE_API_KEY")
</file context>
| if _cloud_only(): | |
| if _cloud_only() and not _explicit_cdp_configured(): |
|
|
||
| def _is_loopback_url(url): | ||
| host = (urlparse(url).hostname or "").lower() | ||
| return host in {"127.0.0.1", "localhost", "::1", "0.0.0.0"} or host.startswith("127.") |
There was a problem hiding this comment.
P2: BH_CLOUD_ONLY can be bypassed with equivalent IPv6 loopback spellings, so a local CDP endpoint is accepted despite the cloud-only guard. Normalize parsed IP hosts and use ipaddress.ip_address(host).is_loopback (while retaining the hostname check for localhost) before accepting either endpoint.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/daemon.py, line 103:
<comment>`BH_CLOUD_ONLY` can be bypassed with equivalent IPv6 loopback spellings, so a local CDP endpoint is accepted despite the cloud-only guard. Normalize parsed IP hosts and use `ipaddress.ip_address(host).is_loopback` (while retaining the hostname check for `localhost`) before accepting either endpoint.</comment>
<file context>
@@ -94,6 +94,15 @@ def log(msg):
+
+def _is_loopback_url(url):
+ host = (urlparse(url).hostname or "").lower()
+ return host in {"127.0.0.1", "localhost", "::1", "0.0.0.0"} or host.startswith("127.")
+
+
</file context>
|
|
||
|
|
||
| def default_run_root() -> Path: | ||
| stamp = time.strftime("%Y%m%d-%H%M%S") |
There was a problem hiding this comment.
P2: Concurrent launches in the same second share one run workspace and can overwrite each other's browser environment, outputs, and final message. Add a collision-resistant component to the default directory name.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/agent.py, line 73:
<comment>Concurrent launches in the same second share one run workspace and can overwrite each other's browser environment, outputs, and final message. Add a collision-resistant component to the default directory name.</comment>
<file context>
@@ -0,0 +1,306 @@
+
+
+def default_run_root() -> Path:
+ stamp = time.strftime("%Y%m%d-%H%M%S")
+ return Path.home() / ".browser-harness" / "agent-runs" / stamp
+
</file context>
| stamp = time.strftime("%Y%m%d-%H%M%S") | |
| stamp = f"{time.strftime('%Y%m%d-%H%M%S')}-{time.time_ns()}" |
| if not os.environ.get("BROWSER_USE_API_KEY"): | ||
| raise RuntimeError("BH_CLOUD_ONLY requires BROWSER_USE_API_KEY") |
There was a problem hiding this comment.
P2: Cloud-only startup rejects valid stored Browser Use authentication and requires an environment key, unlike the cloud provisioning API it calls. Reuse _cloud_auth_configured() so browser-harness auth login remains usable.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/run.py, line 371:
<comment>Cloud-only startup rejects valid stored Browser Use authentication and requires an environment key, unlike the cloud provisioning API it calls. Reuse `_cloud_auth_configured()` so `browser-harness auth login` remains usable.</comment>
<file context>
@@ -333,7 +365,16 @@ def _run(args):
+ # In benchmark cloud-only mode (BH_CLOUD_ONLY), local Chrome must never
+ # suppress cloud provisioning.
+ if _cloud_only():
+ if not os.environ.get("BROWSER_USE_API_KEY"):
+ raise RuntimeError("BH_CLOUD_ONLY requires BROWSER_USE_API_KEY")
+ if daemon_alive() and not _daemon_is_cloud():
</file context>
| if not os.environ.get("BROWSER_USE_API_KEY"): | |
| raise RuntimeError("BH_CLOUD_ONLY requires BROWSER_USE_API_KEY") | |
| if not _cloud_auth_configured(): | |
| raise RuntimeError("BH_CLOUD_ONLY requires Browser Use Cloud authentication; run `browser-harness auth login`") |
|
|
||
| def _write_private(path: Path, data: bytes) -> None: | ||
| # Temp file created 0600 (no world-readable window), then atomic rename. | ||
| tmp = path.with_suffix(path.suffix + ".tmp") |
There was a problem hiding this comment.
P2: Concurrent secrets commands can fail or commit the wrong temporary contents because every writer uses the same *.tmp pathname. Use a uniquely created temporary file in the destination directory before os.replace.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/secrets.py, line 53:
<comment>Concurrent secrets commands can fail or commit the wrong temporary contents because every writer uses the same `*.tmp` pathname. Use a uniquely created temporary file in the destination directory before `os.replace`.</comment>
<file context>
@@ -0,0 +1,216 @@
+
+def _write_private(path: Path, data: bytes) -> None:
+ # Temp file created 0600 (no world-readable window), then atomic rename.
+ tmp = path.with_suffix(path.suffix + ".tmp")
+ fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
+ try:
</file context>
| approval = "never" if args.approval_mode == "never" else ( | ||
| "on-request" if args.approval_mode == "auto-review" else "on-failure" | ||
| ) |
There was a problem hiding this comment.
P2: --approval-mode deny-all is mapped inconsistently between run_task() and launch_tui(). In run_task(), deny-all becomes "on-failure" (approve on failure — semantically the opposite of "deny all"), while in launch_tui() it becomes "never" (never approve). The three-way choice "never", "auto-review", "deny-all" is identical in both parsers, so a user gets different behavior from the same flag depending on subcommand. Recommend aligning both functions so deny-all maps to "never" (matching its name) or removing deny-all from the choices if it isn't intentionally supported.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/agent.py, line 184:
<comment>`--approval-mode deny-all` is mapped inconsistently between `run_task()` and `launch_tui()`. In `run_task()`, `deny-all` becomes `"on-failure"` (approve on failure — semantically the opposite of "deny all"), while in `launch_tui()` it becomes `"never"` (never approve). The three-way choice `"never"`, `"auto-review"`, `"deny-all"` is identical in both parsers, so a user gets different behavior from the same flag depending on subcommand. Recommend aligning both functions so `deny-all` maps to `"never"` (matching its name) or removing `deny-all` from the choices if it isn't intentionally supported.</comment>
<file context>
@@ -0,0 +1,306 @@
+ # output, agent messages) to stdout, exactly like `codex exec`. Downstream
+ # tooling — including the agent benchmark's step/evidence extractor — relies
+ # on that transcript; the SDK's final-response-only path starves it.
+ approval = "never" if args.approval_mode == "never" else (
+ "on-request" if args.approval_mode == "auto-review" else "on-failure"
+ )
</file context>
| approval = "never" if args.approval_mode == "never" else ( | |
| "on-request" if args.approval_mode == "auto-review" else "on-failure" | |
| ) | |
| approval = "never" if args.approval_mode == "never" else ( | |
| "on-request" if args.approval_mode == "auto-review" else "never" | |
| ) |
| for domain, entries in _secrets.list_secrets().items() | ||
| if _secrets.domain_matches(host, domain) | ||
| for e in entries | ||
| if e["name"] == name |
There was a problem hiding this comment.
P3: _secret_lookup compares the caller's name argument directly against stored names without stripping whitespace, unlike set_secret and get_secret_value which both normalize with .strip(). If a caller passes secret(" login-password ") with accidental spaces, the lookup silently fails even though the value exists.
Recommendation: Apply str(name or "").strip() before the comparison for consistency with the rest of the API.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/helpers.py, line 525:
<comment>`_secret_lookup` compares the caller's `name` argument directly against stored names without stripping whitespace, unlike `set_secret` and `get_secret_value` which both normalize with `.strip()`. If a caller passes `secret(" login-password ")` with accidental spaces, the lookup silently fails even though the value exists.
**Recommendation:** Apply `str(name or "").strip()` before the comparison for consistency with the rest of the API.</comment>
<file context>
@@ -490,6 +491,69 @@ def http_get(url, headers=None, timeout=20.0):
+ for domain, entries in _secrets.list_secrets().items()
+ if _secrets.domain_matches(host, domain)
+ for e in entries
+ if e["name"] == name
+ ]
+ if not matches:
</file context>
| if e["name"] == name | |
| if e["name"] == str(name or "").strip() |
| suffix match: accounts.github.com matches a secret stored for github.com). | ||
| Never print or log the returned value. | ||
| Example: fill_input("#password", secret("login-password")).""" | ||
| domain, entry = _secret_lookup(name) |
There was a problem hiding this comment.
P3: secret() and totp() decrypt the entire secrets store twice — once in _secret_lookup() (via list_secrets() → _load()) and once in get_secret_value() (→ _load()). While the store is small, this doubles the AES-GCM decryption overhead on every credential lookup.
Recommendation: Have _secret_lookup accept an optional pre-decrypted data dict, or restructure so that _secret_lookup returns the actual value alongside the metadata, avoiding the second _load() entirely.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/helpers.py, line 542:
<comment>`secret()` and `totp()` decrypt the entire secrets store twice — once in `_secret_lookup()` (via `list_secrets()` → `_load()`) and once in `get_secret_value()` (→ `_load()`). While the store is small, this doubles the AES-GCM decryption overhead on every credential lookup.
**Recommendation:** Have `_secret_lookup` accept an optional pre-decrypted data dict, or restructure so that `_secret_lookup` returns the actual value alongside the metadata, avoiding the second `_load()` entirely.</comment>
<file context>
@@ -490,6 +491,69 @@ def http_get(url, headers=None, timeout=20.0):
+ suffix match: accounts.github.com matches a secret stored for github.com).
+ Never print or log the returned value.
+ Example: fill_input("#password", secret("login-password"))."""
+ domain, entry = _secret_lookup(name)
+ value = _secrets.get_secret_value(domain, name)
+ # A TOTP entry never exposes its seed — hand back a live code instead.
</file context>
|
|
||
|
|
||
| def _cloud_only(): | ||
| return os.environ.get("BH_CLOUD_ONLY") not in (None, "", "0", "false", "False") |
There was a problem hiding this comment.
P3: _cloud_only() treats BH_CLOUD_ONLY=FALSE (all caps) as truthy rather than falsy. While the current tuple covers the most common falsy values, FALSE all-caps is a natural user input and would unexpectedly enable cloud-only mode. Consider normalizing the value to lowercase before checking, or using os.environ.get("BH_CLOUD_ONLY", "").lower() not in ("", "0", "false") to make the check case-insensitive.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/daemon.py, line 98:
<comment>`_cloud_only()` treats `BH_CLOUD_ONLY=FALSE` (all caps) as truthy rather than falsy. While the current tuple covers the most common falsy values, `FALSE` all-caps is a natural user input and would unexpectedly enable cloud-only mode. Consider normalizing the value to lowercase before checking, or using `os.environ.get("BH_CLOUD_ONLY", "").lower() not in ("", "0", "false")` to make the check case-insensitive.</comment>
<file context>
@@ -94,6 +94,15 @@ def log(msg):
+def _cloud_only():
+ return os.environ.get("BH_CLOUD_ONLY") not in (None, "", "0", "false", "False")
+
+
</file context>
| return os.environ.get("BH_CLOUD_ONLY") not in (None, "", "0", "false", "False") | |
| def _cloud_only(): | |
| return os.environ.get("BH_CLOUD_ONLY", "").lower() not in ("", "0", "false") |
… when there's no local build, so the TUI runs instantly with no cargo build (submodule build still takes precedence for devs)
There was a problem hiding this comment.
1 issue found across 4 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/browser_harness/agent.py">
<violation number="1" location="src/browser_harness/agent.py:71">
P1: A compromised or replaced release asset becomes an executable running with the agent's default full access because the download is never integrity-checked. Pin and verify a per-platform SHA-256 or signed artifact before making the cached file executable.</violation>
</file>
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
| if dst.exists() and os.access(dst, os.X_OK): | ||
| return dst | ||
| asset = f"codex-{_target_triple()}.gz" | ||
| url = f"https://github.com/{CODEX_AGENT_REPO}/releases/download/{CODEX_AGENT_RELEASE}/{asset}" |
There was a problem hiding this comment.
P1: A compromised or replaced release asset becomes an executable running with the agent's default full access because the download is never integrity-checked. Pin and verify a per-platform SHA-256 or signed artifact before making the cached file executable.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/agent.py, line 71:
<comment>A compromised or replaced release asset becomes an executable running with the agent's default full access because the download is never integrity-checked. Pin and verify a per-platform SHA-256 or signed artifact before making the cached file executable.</comment>
<file context>
@@ -35,28 +43,79 @@ def default_codex_repo() -> Path | None:
+ if dst.exists() and os.access(dst, os.X_OK):
+ return dst
+ asset = f"codex-{_target_triple()}.gz"
+ url = f"https://github.com/{CODEX_AGENT_REPO}/releases/download/{CODEX_AGENT_RELEASE}/{asset}"
+ dst.parent.mkdir(parents=True, exist_ok=True)
+ tmp_gz = dst.with_suffix(".gz.part")
</file context>
…ubmodule (CI workflow fix)
Explain running the Codex-fork agent RIGHT NOW from the codex-embed-browser-harness branch: clone/uv sync, first-run auto-download of the prebuilt binary (all 4 platforms), one-time Codex-style auth (login or OPENAI_API_KEY), and browser connection.
Summary by cubic
Adds a built‑in Codex‑backed agent with a TUI, domain‑scoped encrypted secrets (passwords + TOTP), and a cloud‑only mode. Prebuilt agent binaries for macOS (arm64/x86_64) and Linux (x86_64/arm64, glibc) auto‑download for instant startup; releases are published via the
codex-agentsubmodule’s CI for reliable auto‑downloads.New Features
browser-harness agentandbrowser-harness tuirun the fork from thecodex-agentsubmodule; drivesexec --jsonto stream the full transcript; uses Codex’s default model unless pinned; workspace wrapper sources/browserbrowser-env, delegates toBENCH_TASK_DIRwhen set, exportsBROWSER_HARNESS_CLI/BH_BROWSER_LABEL, and defaults to full access with no approval prompts. Binary resolution: explicit--codex-bin→ locally built submodule → cached prebuilt release (auto‑downloaded per‑platform: macOS arm64/x86_64; Linux x86_64/arm64 glibc; no env‑var or sibling‑dir guessing).browser-harness secrets set|list|remove; helpersavailable_secrets(),secret(name),totp(name)gate values by current page domain; AES‑256‑GCM at rest; never logs values.BH_CLOUD_ONLYrejects local DevTools and requires Browser Use Cloud; CLI restarts non‑cloud daemons; daemon ping exposesremote_id,cdp_ws, andcloud_only.BH_NO_OPEN_LIVE_URLdisables auto‑open of liveUrl.Migration
codex-embed-browser-harnessanduv sync; first run downloads a prebuilt agent (~86 MB, cached under~/.browser-harness/agent-bin/).~/.browser-harness/agent-bin/agent-v0.1.0/codex loginor setOPENAI_API_KEY; connect a local CDP Chrome or a Browser Use Cloud browser.git submodule update --init && cd codex-agent/codex-rs && cargo build --release -p codex-cli.browser-harness agent "open example.com and tell me the page title"orbrowser-harness tui.Written for commit 3926eb4. Summary will update on new commits.