feat(ingest): open the front door — ingest key, generated contract, setup prompt, live status

[vivekchand]

Closes #5679, #5682, #5681, #5680.

Requirement: https://factory.8090.ai/project/b415065f-ab2f-4f53-8864-0c009fd098cb/requirements/7fb88c5c-228e-4c48-9f16-da570740fc36

Stacked on #5676 (feat/build-your-own-ui) — this is the write half of that key system, so the base is that branch, not main. Merge #5676 first.

The gap

An agent is observable by ClawMetry only if the daemon runs on the same machine as the agent. /v1/* trusts loopback and otherwise wants the gateway token; the write API trusts loopback or one static secret shared by the whole install; ingest.clawmetry.com takes E2E-sealed daemon snapshots and nothing else. That excludes CI runners, containers, serverless, hosted agent products and teammates' laptops — where a lot of agent spend actually happens.

Three commits, in dependency order.

1. The ingest key (#5679)

clawmetry key create --name ci --scope write:ingest

then x-clawmetry-key: cmk_… on the OTLP endpoints. Same cmk_ shape, same store, same clawmetry key list|revoke as the read keys — not a second key system. Two routing headers come with it, written into the resource attributes the mappers already read (service.name, deployment.environment), so a header is exactly as powerful as the equivalent exporter setting and no mapper learns a second way to answer the question.

The posture: an ingest key can only push (write:ingest grants no q/1 shape); it is never given a CORS header and cannot carry a browser origin; read and write cannot be mixed on one key, refused at creation with a sentence rather than at request time with a code; and there is one gate, not two — _check_auth steps aside for a keyed /v1/ request exactly as it does for /api/q/.

Verified against a real dashboard: all three doors return 200 (loopback zero-config unchanged, gateway token unchanged, ingest key), every refusal returns its own status with its own sentence, and a span pushed with x-clawmetry-runtime: my-engine lands as agent_type=my_engine while the same span pushed bare still lands as openclaw. That live run found the one real bug here: GET /api/q/1 was answering 200 to a write-only key — verify() accepts it since it is a valid key, and nothing asked whether it held any read scope. Now 403, pinned by a test.

2. The ingest contract, generated (#5682)

Four things describe this surface — the server, docs/INGEST.md, the setup prompts, and the landing reference — and none shared a source. clawmetry/ingest_contract.py declares it as data; ingest_auth.py imports its constants from there; scripts/gen_ingest_doc.py renders the doc with --check.

The guard earns the file. It checks both directions: every attribute declared read must be named in the mapper, and every attribute declared unread must genuinely be absent. Writing that check is what surfaced #5685 — we were advertising a GenAI convention we did not implement, and cached tokens were priced as free.

The doc also states what ClawMetry does not accept — no syslog, CEF, GELF or raw text — pinned by a test, because a reference that only says what works is not one anyone can plan against.

Two cache attributes are listed as not read here, which is the honest state of this branch: the fix is #5686, open against main while this is stacked elsewhere. The unread-side guard makes whichever merges second update the list — which is why both directions are checked.

3. The setup prompt (#5681)

clawmetry setup-prompt <runtime> and GET /api/setup-prompt, rendered from the same contract. Half of it is negative space, because that is the useful half: the key goes in one header and nowhere else; the placeholder is a placeholder and the real key must be asked for, never invented; both encodings are already accepted and need no "fixing"; a key not in the prompt does not exist; and this is observability — it watches, it does not change what runs. It ends by making the agent verify and report a real event count.

Two guards worth keeping. The first reads backwards: every x-clawmetry-* token in the prompt must be a declared header. Checking only that the right headers appear was too weak — proven by mutation, where swapping the config block's header for x-clawmetry-apikey left every other assertion green because the correct name still appeared in the prose. The second is general: every registered subcommand must be in cli.py's _subcmds allowlist, because a parser with no entry there falls through to the dashboard's argparse and dies with "invalid choice", reading as though the command was never written. setup-prompt did exactly that when first added — the same two-list trap CLAUDE.md documents for runtimes — and asserting only that setup-prompt is present would not have prevented the next one.

The prompt module names no runtime and hardcodes no vendor value, pinned by a test: runtime-specific OTel knowledge stays in profiles, which paid runtimes register from clawmetry-pro.

4. "Did it work?" (#5680)

GET /api/onboarding/ingest-status answers the question that kills setup funnels, from real data, and the first-run gate renders it as a live strip. When nothing has arrived it says so and what to do, including clawmetry setup-prompt for an off-box agent. It never blocks — a confirmation, not a step, in a flow whose selling point is having no steps.

Three judgement calls worth naming:

• Two clocks, kept apart. The durable count is DuckDB and survives restart; the OTLP receiver's counters are in-process and empty on restart, so they are reported separately as has_data_this_process. Folding them together would tell a working install it was broken after every restart.
• A runtime we merely know about is not one that is sending. Idle rows are dropped rather than listed — listing them answers "is anything arriving?" with a yes they haven't earned.
• Read through _dispatch, never raw files, so it answers identically on a laptop and in a container with no ~/.openclaw. Memoised 2s because it's polled: ~200 ms first call, ~0.2 ms after.

Verified in both states against a running dashboard: strip renders inside the card in the real page; empty store returns connected:false with an actionable next_step; the populated store here returns 25,349 events across 9 runtimes. The strip first landed outside .obg-card — on the overlay backdrop rather than in the dialog — so a guard now asserts its position.

Tests

77 guards across test_ingest_key.py (25), test_ingest_contract_drift.py (26), test_setup_prompt.py (17) and test_ingest_status.py (9), all registered in ci.yml since CI runs explicit file lists. Every non-obvious guard was mutation-tested — and two of them were strengthened after the mutation passed, which is the only reason they are worth having.

Not in scope

Regional endpoint split, a base "any format" endpoint, per-key rate limiting (belongs on the hosted side where plan budgets live). The hosted endpoint is clawmetry-cloud#2343, gated on the trust-class work in clawmetry-cloud#2344 — pushed telemetry is plaintext by construction, and that needs naming before it ships rather than after it becomes a P0.

🤖 Generated with Claude Code

https://claude.ai/code/session_01YMHxYeQR1QLunY3PzURqRd

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[github-actions]

Visual diff

Bot run failed before producing screenshots. Check the workflow logs.

This check is non-blocking.

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[vivekchand]

✨ auto-fixed: merged updated base branch (feat/build-your-own-ui) into branch — base was at old SHA 90f8eb5, now at e270a12

---

Generated by Claude Code

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[8090-software-factory]

⚠️ Drift Bot (ClawMetry): 1 potential drift finding(s)

1. Requirement: Ingest key: observe an agent that is not on this machine

File: docs/MODULE_MAP.md

The MODULE_MAP.md documents /api/setup-prompt as a route served by routes/meta.py, but this endpoint is not implemented in the codebase. The PR description states both a CLI command and an HTTP endpoint should be provided, but only the CLI command exists.

[vivekchand]

Checked this rather than acting on it — the finding is a false positive, but it was pointing at a real defect.

/api/setup-prompt is implemented and live. routes/meta.py:1355, and against a running app:

HTTP 200 | prompt bytes: 2660

routes/meta.py is 2,027 lines, so the endpoint sits well past the head of the file.

Why the bot couldn't see it, and the actual defect. The module opens with a per-blueprint route index, and that index had drifted:

bp_otel           (3)  — /v1/metrics, /v1/traces, /api/otel-status

bp_otel serves six routes. /v1/logs and /api/setup-prompt had both been added without updating it. Since the index is the first ~20 lines of the module, it is what a reader sees and what a tool sampling the head of a file sees — so it reported the endpoint as missing, correctly reasoning from stale documentation.

Fixed in a798db0: every count corrected, the missing routes named, and a guard added asserting the documented count matches the decorators below it (mutation-proven by understating bp_otel back to three). The index can't silently drift again.

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[vivekchand]

awaiting confirmation — non-trivial rebase, needs human review (conflicts in: CHANGELOG.md, CLAUDE.md, clawmetry/apikeys.py, clawmetry/cli.py, dashboard.py, docs/MODULE_MAP.md, examples/custom-ui/index.html, routes/public_api.py, tests/test_public_api_keys.py)

This PR is stacked on feat/build-your-own-ui (#5676), which was just rebased onto main. The ingest-key branch carries old copies of the build-your-own-ui commits, so rebasing it now produces add/add conflicts in 9 files (exceeds the 50-LOC semantic threshold). The recommended fix: git rebase origin/feat/build-your-own-ui from a clean checkout of this branch, resolve the conflicts, and force-push.

---

Generated by Claude Code

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[vivekchand]

blocked on author decision — skipping (auto-mergeability sweep)

This PR is stacked on feat/build-your-own-ui (#5676), which has been updated since this branch was cut, creating a merge conflict that cannot be resolved without author judgment. Attempted update_pull_request_branch but it failed with a merge conflict. The author needs to rebase feat/ingest-key onto the current tip of feat/build-your-own-ui and resolve any conflicts manually.

---

Generated by Claude Code

[8090-software-factory]

⚠️ Drift Bot (ClawMetry): 8 potential drift finding(s)

1. Blueprint: Ingest key: observe an agent that is not on this machine

File: clawmetry/ingest_contract.py

The blueprint specifies (ADR-005) that ingest_contract.py must declare HEADER_KEY, HEADER_RUNTIME, HEADER_ENV, MAX_BODY_BYTES, SURFACES, HEADERS, RESPONSES, GENAI_READ, and GENAI_NOT_READ as the single source of truth. However, the file defines GEN_AI_ATTRS_READ/NOT_READ instead, and is missing HEADER_KEY, HEADER_RUNTIME, HEADER_ENV, MAX_BODY_BYTES, SURFACES, HEADERS, and RESPONSES entirely. Code imports these constants from ingest_contract (ingest_auth.py:59-65, setup_prompt.py:46-51) and will fail with ImportError at runtime.

2. Blueprint: Ingest key: observe an agent that is not on this machine

File: clawmetry/ingest_contract.py:320-324

The constants GEN_AI_ATTRS_READ and GEN_AI_ATTRS_NOT_READ are named incorrectly. The blueprint specifies them should be GENAI_READ and GENAI_NOT_READ. Tests reference the expected names (test_ingest_contract_drift.py:109+, 119+) and will fail with AttributeError when accessing the mismatched names.

3. Blueprint: Ingest key: observe an agent that is not on this machine

File: clawmetry/apikeys.py:605

The allows_ingest() function is correctly implemented and checks the SCOPE_INGEST scope. However, SCOPE_INGEST is defined correctly in this file, so this part appears correct, but it depends on the missing header constants in ingest_contract which will cause import failures upstream.

4. Blueprint: Ingest key: observe an agent that is not on this machine

File: clawmetry/ingest_auth.py:59-65

ingest_auth.py imports HEADER_KEY, HEADER_RUNTIME, HEADER_ENV, and MAX_BODY_BYTES from clawmetry.ingest_contract (lines 59-65), but these constants do not exist in the contract module. This will cause ImportError when ingest_auth.py is imported at runtime.

5. Blueprint: Ingest key: observe an agent that is not on this machine

File: clawmetry/setup_prompt.py:46-51

setup_prompt.py imports HEADER_KEY, HEADER_RUNTIME, HEADER_ENV, MAX_BODY_BYTES, and CONTRACT_VERSION from clawmetry.ingest_contract (lines 46-51), but these constants do not exist in the contract module. This will cause ImportError when setup_prompt.py is imported at runtime.

6. Blueprint: Ingest key: observe an agent that is not on this machine

File: tests/test_ingest_contract_drift.py:75

The test references ingest_contract.SURFACES, ingest_contract.HEADERS, and ingest_contract.RESPONSES, but these constants are not defined in the contract module. Tests expect these to be data structures describing surfaces, headers, and HTTP response codes but will fail with AttributeError.

7. Blueprint: Ingest key: observe an agent that is not on this machine

File: tests/test_setup_prompt.py:64

The test expects ingest_contract.HEADERS to exist and be a list of header definitions, but this constant is not defined in the contract module. Test will fail with AttributeError when trying to access the expected headers list.

8. Blueprint: Ingest key: observe an agent that is not on this machine

File: routes/meta.py:1325

The code passes runtime and env as keyword arguments to processor functions (_process_otlp_traces, _process_otlp_metrics, _process_otlp_logs) at lines 1325-1326. While the processor functions now accept these parameters and apply them via _apply_ingest_routing, the blueprint specifies routing headers should be written into resource attributes BEFORE processing, which is correctly implemented, but depends on the missing header constants that cause import failures upstream.

[8090-software-factory]

⚠️ Drift Bot (ClawMetry) — Blueprint: Ingest key: observe an agent that is not on this machine

ingest_auth.py imports HEADER_KEY, HEADER_RUNTIME, HEADER_ENV, and MAX_BODY_BYTES from clawmetry.ingest_contract (lines 59-65), but these constants do not exist in the contract module. This will cause ImportError when ingest_auth.py is imported at runtime.

[8090-software-factory]

⚠️ Drift Bot (ClawMetry): 8 potential drift finding(s)

1. Blueprint: Ingest key: observe an agent that is not on this machine

File: clawmetry/ingest_contract.py:1

Blueprint specifies ADR-005 requires ingest_contract.py to declare HEADER_KEY, HEADER_RUNTIME, HEADER_ENV, and MAX_BODY_BYTES as constants for the ingest gate to import. These constants are missing from the file. Code in ingest_auth.py (lines 59-65) and setup_prompt.py (lines 46-51) will fail with ImportError at runtime.

2. Blueprint: Ingest key: observe an agent that is not on this machine

File: clawmetry/ingest_contract.py:320

Blueprint specifies ADR-005 that GENAI_READ and GENAI_NOT_READ must be declared. The file defines these as GEN_AI_ATTRS_READ and GEN_AI_ATTRS_NOT_READ instead. Tests import by the expected names (test_ingest_contract_drift.py lines 109+, 119+) and will fail with AttributeError at runtime.

3. Blueprint: Ingest key: observe an agent that is not on this machine

File: clawmetry/ingest_contract.py:1

Blueprint ADR-005 specifies that ingest_contract.py must export SURFACES, HEADERS, and RESPONSES as named constants for the server, doc generator, and tests to read. These are not defined in the file. Tests reference them (test_ingest_contract_drift.py lines 75, 91-96; test_setup_prompt.py line 64) and will fail with AttributeError.

4. Requirement: Ingest key: observe an agent that is not on this machine

File: clawmetry/ingest_auth.py:59

Requirement R2 specifies x-clawmetry-key header on ingest surfaces. The file imports HEADER_KEY from ingest_contract (line 59), but that constant is not defined there, causing ImportError at runtime when this module is imported.

5. Requirement: Ingest key: observe an agent that is not on this machine

File: clawmetry/setup_prompt.py:46

Requirement R5 specifies routing headers x-clawmetry-runtime and x-clawmetry-env. The file imports HEADER_KEY, HEADER_RUNTIME, HEADER_ENV, and MAX_BODY_BYTES from ingest_contract (lines 46-51), but these constants are not defined there, causing ImportError at runtime.

6. Blueprint: Ingest key: observe an agent that is not on this machine

File: clawmetry/apikeys.py:65

Blueprint specifies the component adds SCOPE_INGEST to the key system and functions to check if a key allows ingest. The file correctly adds SCOPE_INGEST (line 109) and allows_ingest() function (line 608), but SCOPES is redefined (line 112) to include it. However, code imports SCOPES from query_contract as READ_SCOPES (lines 65-67), creating inconsistency between what query_contract exports and what apikeys declares as the full SCOPES list.

7. Blueprint: Ingest key: observe an agent that is not on this machine

File: routes/public_api.py:151

Requirement R3 specifies that a write-only key cannot read. The file correctly adds a check (lines 151-165) to verify that a key holds read scopes before allowing access to /api/q/. This implementation is correct and matches the requirement.

8. Blueprint: Ingest key: observe an agent that is not on this machine

File: tests/test_ingest_contract_drift.py:75

Blueprint specifies ADR-005 requires constant exports for contract validation. The test references ingest_contract.SURFACES (lines 75-79), but this constant is not defined in ingest_contract.py. The test will fail with AttributeError.

[8090-software-factory]

⚠️ Drift Bot (ClawMetry): 7 potential drift finding(s)

1. Blueprint: Ingest key: observe an agent that is not on this machine

File: clawmetry/ingest_contract.py:1

ADR-005 specifies that ingest_contract.py must declare HEADER_KEY, HEADER_RUNTIME, HEADER_ENV, MAX_BODY_BYTES, SURFACES, HEADERS, RESPONSES, GENAI_READ, and GENAI_NOT_READ as the single source of truth. However, the file defines OTLP_* constants instead, uses GEN_AI_ATTRS_READ/NOT_READ rather than GENAI_READ/NOT_READ, and omits HEADER_*, SURFACES, and HEADERS entirely. Code imports these non-existent constants and will fail with ImportError at runtime.

2. Blueprint: Ingest key: observe an agent that is not on this machine

File: clawmetry/ingest_auth.py:59

ingest_auth.py imports HEADER_KEY, HEADER_RUNTIME, HEADER_ENV, and MAX_BODY_BYTES from clawmetry.ingest_contract (lines 59-65), but these constants are not defined in that module. This will cause ImportError when ingest_auth.py is imported at runtime.

3. Blueprint: Ingest key: observe an agent that is not on this machine

File: clawmetry/setup_prompt.py:46

setup_prompt.py imports HEADER_KEY, HEADER_RUNTIME, HEADER_ENV, MAX_BODY_BYTES, and CONTRACT_VERSION from clawmetry.ingest_contract (lines 46-51), but these constants (except CONTRACT_VERSION) are not defined in that module. This will cause ImportError when setup_prompt.py is imported at runtime.

4. Blueprint: Ingest key: observe an agent that is not on this machine

File: tests/test_ingest_contract_drift.py:75

Tests reference ingest_contract.SURFACES, ingest_contract.HEADERS, and ingest_contract.RESPONSES (lines 75-96), but these constants are not defined in the contract module. The drift tests will fail with AttributeError when accessing these expected names.

5. Blueprint: Ingest key: observe an agent that is not on this machine

File: tests/test_ingest_contract_drift.py:99

Tests parametrize over ingest_contract.GENAI_READ and ingest_contract.GENAI_NOT_READ (lines 99-122), but the contract module defines them as GEN_AI_ATTRS_READ and GEN_AI_ATTRS_NOT_READ. Test will fail with AttributeError when trying to access the expected names.

6. Blueprint: Ingest key: observe an agent that is not on this machine

File: tests/test_setup_prompt.py:52

Test test_prompt_invents_no_header_of_its_own references ingest_contract.HEADERS (line 52) expecting a list of dicts with "name" key, but this constant is not defined in the contract module. Test will fail with AttributeError.

7. Blueprint: Ingest key: observe an agent that is not on this machine

File: tests/test_setup_prompt.py:60

Test test_endpoints_in_the_prompt_are_declared_surfaces references ingest_contract.SURFACES (line 60) expecting a list of dicts with "path" key, but this constant is not defined in the contract module. Test will fail with AttributeError.

[8090-software-factory]

⚠️ Drift Bot (ClawMetry) — Blueprint: Ingest key: observe an agent that is not on this machine

ingest_auth.py imports HEADER_KEY, HEADER_RUNTIME, HEADER_ENV, and MAX_BODY_BYTES from clawmetry.ingest_contract (lines 59-65), but these constants are not defined in that module. This will cause ImportError when ingest_auth.py is imported at runtime.

[8090-software-factory]

⚠️ Drift Bot (ClawMetry) — Blueprint: Ingest key: observe an agent that is not on this machine

setup_prompt.py imports HEADER_KEY, HEADER_RUNTIME, HEADER_ENV, MAX_BODY_BYTES, and CONTRACT_VERSION from clawmetry.ingest_contract (lines 46-51), but these constants (except CONTRACT_VERSION) are not defined in that module. This will cause ImportError when setup_prompt.py is imported at runtime.

[vivekchand]

✨ auto-fixed: added missing HEADER_KEY, HEADER_RUNTIME, HEADER_ENV, MAX_BODY_BYTES, HEADERS, SURFACES, RESPONSES, GENAI_READ, GENAI_NOT_READ constants to clawmetry/ingest_contract.py — ingest_auth.py and setup_prompt.py were importing them but they were never defined, causing ImportError on every POST to /v1/traces; also regenerated docs/MODULE_MAP.md after the feat/build-your-own-ui merge.

---

Generated by Claude Code

[vivekchand]

blocked on author decision — skipping (auto-mergeability sweep)

mergeable_state=dirty: merge conflict between feat/ingest-key and its base feat/build-your-own-ui (base has moved since this PR was opened). Resolving requires author context about both stacked branches.

---

Generated by Claude Code

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[vivekchand]

awaiting confirmation — non-trivial rebase, needs human review (conflicts in: stacked base feat/build-your-own-ui has diverged from the sha this branch was cut from; 21 files changed / 2 036 additions across both branches make this a conflict that crosses semantic feature code, not just boilerplate)

Automated sweeper identified this as mergeable_state=dirty. Base branch feat/build-your-own-ui (PR #5676, 104 commits) has moved ahead of the sha cb77f96 this branch forked from. Rebasing feat/ingest-key onto the current feat/build-your-own-ui head involves merge decisions in overlapping feature code — flagging for human review rather than resolving automatically.

---

Generated by Claude Code

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[vivekchand]

✨ auto-fixed: merged main into this branch (picking up the #5676 base that merged as 82e171f2) and resolved conflicts in 8 files (.github/workflows/ci.yml, clawmetry/apikeys.py, clawmetry/apikeys_public.py, clawmetry/cli.py, clawmetry/static/js/app.js, dashboard.py, docs/MODULE_MAP.md, routes/public_api.py). All PR additions (write:ingest scope, setup-prompt CLI command, ingest auth gate) were preserved. MODULE_MAP regenerated.

---

Generated by Claude Code

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

[8090-software-factory]

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.