Skip to content

feat(mt#5154): Give config doctor project-scope checks and split the health summary - #3765

Merged
edobry merged 3 commits into
mainfrom
task/mt-5154
Sep 15, 2026
Merged

edobry merged 3 commits into
mainfrom
task/mt-5154

Conversation

@minsky-ai

@minsky-ai minsky-ai Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Closes mt#5154. Every config doctor check was user-scope — config loads, validation, the reviewer token, the App's own permission set, model ids, the config dir, embeddings, index coverage. None looked at the project the doctor was pointed at. So a project with the wrong harness recorded, no compiled rules its harness could read, repository.backend: local beside a GitHub origin, and neither GitHub App installation covering it reported 9/9 healthy: true (flowtato, 2026-09-14T20:44Z; peezombie.me still did, until this PR).

Adds six project-scope checks, computed for the workspace the doctor is pointed at (its config provider is per-workspace since mt#5155), each a read:

  • (a) Project Harness — recorded workspace.harness vs the caller's harness (resolveAgentHarness, mt#5153). config.doctor joined CALLER_ACTOR_ID_TOOL_NAMES, so over MCP it compares against the caller, not the daemon's spawner. A mismatch is an error (softened to a warning when harnessSource: flag); an unset harness is a warning, not an error — that is the normal pre-mt#5153 state, the Minsky repo included.
  • (b) Compiled Outputs — minsky compile --check for the recorded harness; also reports when the caller's harness has zero reachable outputs ("nothing it loads is reachable").
  • (c) Repository Identity — recorded repository.{backend,url,github} / project.slug vs git remote get-url origin (new pure compareRepositoryIdentity; mt#5159's identity helper absorbs it).
  • (d) GitHub App Coverage — every configured App role covers the repo, the same checkAppRoleCoverage minsky setup runs (describeConfiguredAppRoles moved into the domain module so both share it). The doctor files no grant request.
  • (e) Observability Hooks — the baseline is registered and installed (new findMissingBaselineRegistration, the read-only inverse of the merge).
  • (f) Token Repository Access — github.token reaches the repo with push rights (GET /repos/{owner}/{repo}, new checkTokenRepoAccess). This is the 22:35Z addendum: session_pr_merge failed with "Resource not accessible by personal access token" because the fine-grained PAT's repository access did not include the newly onboarded repo, and nothing checked it before the loop reached merge.

Every diagnostic now carries scope: "user" | "project"; summary gains user/project sub-summaries; healthy requires zero errors in both; the CLI renders two groups (ADR-039's declared doctor renderer). The doctor never writes — it diagnoses; setup remedies.

Changes

  • packages/domain/src/doctor/ (new): project-checks.ts (entry) + harness-checks.ts / repository-checks.ts / types.ts (split to stay under the 400-line warn ceiling), deps injected so every branch is unit-testable with no network and no real fs.
  • packages/domain/src/project/repository-drift.ts (new): pure compareRepositoryIdentity.
  • packages/domain/src/setup/token-repo-access.ts (new): checkTokenRepoAccess.
  • packages/domain/src/setup/hook-provisioning.ts: findMissingBaselineRegistration.
  • packages/domain/src/setup/app-coverage.ts: describeConfiguredAppRoles moved here from the setup adapter.
  • src/adapters/shared/commands/config/validate-doctor-commands.ts: stamp existing diagnostics scope: "user", run the project pass, split the summary, thread callerActorId.
  • src/adapters/cli/customizations/config-customizations.ts: render User scope / Project scope groups (a result whose diagnostics carry no scope renders the flat pre-mt#5154 form).
  • src/mcp/server.ts: config.doctor joins CALLER_ACTOR_ID_TOOL_NAMES.

Contract changes (additive)

config.doctor's RESULT gains diagnostics[].scope, summary.user, summary.project, and workspace; DoctorDiagnostic gains an optional scope; healthy tightens to "zero errors in both scopes". config.doctor declares the hidden, server-injected callerActorId param (cli/mcp-hidden, like init/setup). No renamed or newly-required param, no env var, no config key, no deployed-environment artifact. The hosted server runs the doctor against its own cwd as before.

Test evidence

Live, this branch (verified-1b), fresh bundle, all read-only — nothing in flotato / peezombie.me / the Minsky checkout was modified:

  • AT2 config doctor --workspace /Users/edobry/Projects/flotato (correctly onboarded): healthy: true, project 7/7 pass — harness, compiled, identity, both App roles, hooks, token.
  • AT3 --workspace /Users/edobry/Projects/minsky (regression): healthy: true, project 6 pass + 1 warning (unset harness) 0 errors; user 9/9 unchanged.
  • AT4 --workspace /Users/edobry/Projects/peezombie.me (still harness: cursor): healthy: false, 2 project errors — Project Harness ("cursor" vs caller "claude-code", no harnessSource) and Compiled Outputs (cursor-rules-ts, agents.md stale/missing; .claude/rules, .claude/skills absent — nothing the caller's harness loads is reachable). The same command was 9/9 healthy: true before this PR — the defect and its fix in one comparison.

AT1 (the throwaway-repo reproduction of the 20:44Z flowtato state) is covered by the unit fixture rather than a fresh on-disk repo: project-checks.test.ts "the 2026-09-14T20:44Z flowtato state" asserts four project errors — harness, compiled (zero reachable caller outputs), identity (backend: local), App coverage — with hooks passing (cursor has no baseline).

Unit (verified-1a): project-checks.test.ts 14, repository-drift.test.ts 8, token-repo-access.test.ts 5, plus new hook-provisioning / adapter / renderer cases; the related-test selector's 22 files (46 tests in the changed partition) pass. Typecheck clean across all 8 projects; lint clean.

A correction made mid-implementation

SC1(a) originally specified "nothing recorded → error". The live AT3 run falsified it: the Minsky repo records no workspace.harness, so an error there reported its own doctor run healthy: false. An unset harness is the normal pre-mt#5153 state, so it is now a warning (only a recorded value disagreeing with the caller is an error). The principal caught the flag on the first AT3 run; the spec's SC1(a) and AT1 are corrected to match, with the reasoning recorded in the spec's Implementation record.

Deploy verification

Touches src/mcp/server.ts and shared adapters → minsky-mcp (hosted) redeploys. The doctor's project pass runs against the caller's workspace; on the hosted server no tool call carries a cwd-shaped workspace, so it runs against the server's own cwd as the whole command did before, and the change is inert for a caller passing no workspace.

After merge I will run mcp__minsky__deployment_wait-for-latest for minsky-mcp (notBefore = merge time, expectCommitSha = the merge SHA), require SUCCESS, correlate the Deploy MCP workflow run's head_sha with the merge SHA (image-source service → buildIdentity: indeterminate), confirm the runtime started via GET /health (service: minsky-mcp, status: ok), and assert a change-produced signal — the hosted daemon's config_doctor result carrying summary.project. The outcome will be recorded on the PR and the task spec.

Related

mt#5152 (umbrella) · mt#5153 (DONE — resolveAgentHarness/harnessSource, consumed by (a)) · mt#5155 (DONE — the per-workspace provider the doctor reads) · mt#5159 (owns repository re-derivation; its identity helper absorbs compareRepositoryIdentity — coordinated, not duplicated) · mt#5157 (compile gating; the doctor calls runMinskyCompile, so a change there flows into (b)) · mt#5158 (installs the skills/guards; (e) checks only the observability baseline setup provisions) · mt#982 (the merge-time PAT failure itself; (f) detects the precondition) · mt#1427 (config staleness — separate) · mt#4680 / mt#4693 (the App coverage check (d) reuses).

Had Claude implement and verify this.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AVYcp6iaRLEV6gyrfxVVAr

Execution evidence

Execution evidence:

  • SC1(a) harness: project-checks.test.ts covers pass / mismatch-error / flag-warning / unset-warning / undeterminable-warning; live AT4 peezombie.me → Project Harness error, live AT3 Minsky → warning (unset).
  • SC1(b) compiled: unit covers in-sync pass, stale error, compile-failure error, and the "zero reachable caller outputs" note; live AT4 → Compiled Outputs error naming cursor-rules-ts, agents.md + .claude/rules/.claude/skills absent.
  • SC1(c) identity: repository-drift.test.ts (8) covers match / backend-local drift / slug drift / url+github drift / no-origin / unparseable / ssh; live AT2 flotato → identity pass, AT4 peezombie.me → identity pass (origin matches; the harness/compile are its faults).
  • SC1(d) coverage: unit covers covered→pass, not-covered→error (settings URL), unknown/no-app→warning, one row per configured role; live AT2 flotato → both implementer and reviewer roles pass.
  • SC1(e) hooks: unit covers fully-registered pass, missing-registration error (events named), missing install files named, unparseable settings error, non-claude-code pass; live AT2 flotato → hooks pass, AT4 peezombie.me (cursor) → hooks pass with the no-baseline note.
  • SC1(f) token: token-repo-access.test.ts (5) covers push/read-only/404/403/non-2xx/transport; project-checks.test.ts covers push-pass, read-only-error, no-token-warning, unknown-warning; live AT2 flotato → token push pass.
  • SC2 scope split: validate-doctor-commands.test.ts asserts every diagnostic carries a scope, summary.user/summary.project totals match the partition, and healthy === (user.errors === 0 && project.errors === 0); config-customizations.test.ts asserts the two rendered groups (User scope before Project scope) and that a scope-less result renders the flat form. Live: all three AT runs report split summaries.
  • SC3 caller identity: config.doctor is in CALLER_ACTOR_ID_TOOL_NAMES (src/mcp/server.ts) and declares CALLER_ACTOR_ID_PARAM; server-declared-identity-v2-ctx.test.ts (3) passes. AT4's harness error naming the caller "claude-code" shows the injected id reaching the check over MCP.
  • SC4 read-only: the doctor files no grant request (coverage check calls checkAppRoleCoverage, not fileAppGrantRequests), writes no config, runs no compile write (runMinskyCompile({ check: true })); all live AT runs left flotato / peezombie.me / the Minsky checkout unmodified.

edobry and others added 3 commits September 14, 2026 20:53
…health summary

Every doctor check was user-scope — config loads, validation, the reviewer
token, the App's own permission set, models, the config dir, embeddings, index
coverage — so a project with the wrong harness recorded, no rules its harness
could read, `repository.backend: local` beside a GitHub origin, and neither App
covering it reported 9/9 healthy (flowtato 2026-09-14; peezombie.me still does).

Adds six project-scope checks, computed for the workspace the doctor is pointed
at (its provider is per-workspace since mt#5155), each a read:

- (a) harness: recorded `workspace.harness` vs the CALLER's harness
  (resolveAgentHarness, mt#5153); `harnessSource: flag` softens a mismatch to
  a warning. config.doctor joins CALLER_ACTOR_ID_TOOL_NAMES so an MCP call
  compares against the caller, not the daemon's spawner.
- (b) compiled: `minsky compile --check` for the recorded harness; also reports
  when the caller's harness has zero reachable outputs.
- (c) identity: recorded repository.{backend,url,github}/project.slug vs
  `git remote get-url origin` (new pure compareRepositoryIdentity; mt#5159's
  helper absorbs it).
- (d) coverage: every configured App role covers the repo — the same
  checkAppRoleCoverage setup runs; describeConfiguredAppRoles moved to domain
  so both share it. Doctor files no grant request.
- (e) hooks: the observability baseline is registered and installed
  (new findMissingBaselineRegistration, the read-only inverse of the merge).
- (f) token: `github.token` reaches the repo with push rights (GET /repos,
  new checkTokenRepoAccess) — the 22:35Z merge-time PAT failure, caught early.

Every diagnostic now carries scope: "user" | "project"; summary gains user/
project sub-summaries; healthy requires zero errors in both; the CLI renders
two groups. Checks live in packages/domain/src/doctor/ with injected deps, no
network or real fs in the unit tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AVYcp6iaRLEV6gyrfxVVAr
Live AT3 falsified the "nothing recorded → error" branch: the Minsky repo
records no workspace.harness (unset is the normal pre-mt#5153 state), so an
error there made its own `config doctor` run healthy:false. An unset harness
is now a warning with the setup remedy; only a recorded value that DISAGREES
with the caller is an error. Verified read-only: Minsky healthy:true (1
warning), peezombie.me healthy:false (cursor-vs-claude-code mismatch),
flotato 7/7 pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AVYcp6iaRLEV6gyrfxVVAr
@minsky-ai minsky-ai Bot added the authorship/co-authored Co-authored by human and AI agent label Sep 15, 2026
@minsky-reviewer

minsky-reviewer Bot commented Sep 15, 2026 •

Copy link
Copy Markdown

Minsky Reviewer Status

Verdict: APPROVED — no blocking findings
Review: View review
Model: openai/gpt-5 | Tokens: 803K prompt, 14K completion | Duration: 173s
Mode: normal

Commands

  • /review — request a fresh review

@minsky-reviewer minsky-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent adversarial review (Chinese-wall)
Reviewer: minsky-reviewer[bot] via openai:gpt-5
Tier: 2


Adds project-scope checks to config.doctor and splits output into user/project scopes with solid tests and CLI/MCP wiring. I verified domain check implementations, per-workspace integration, summary semantics, and renderer grouping; they align with the spec. Non-blocking nits: coverage default uses global config rather than the per-workspace provider; coverage check encodes role in the check label, which may surprise exact-name consumers; baseline registration check assumes path-equality; a coarse synthetic error aggregates project-check import failures; minor caching/unknown-harness note in compiled output hinting. No blocking defects found — approve.

Documentation impact

  • no-update-needed — I scanned only docs/configuration-guide.md for references to config.doctor’s output or health semantics and found none to invalidate. I did not sweep other docs for command-output screenshots/snippets. This PR preserves top-level summary fields and healthy semantics for existing consumers while adding per-scope groupings rendered by the CLI; no explicit doc text promises a flat single-scope view. Given limited check scope, no definitive invalidation detected; if we maintain a CLI examples page elsewhere with doctor output, that may warrant a follow-up refresh but is not evidenced here.

@edobry
edobry merged commit 5fbee5e into main Sep 15, 2026
21 checks passed
@edobry
edobry deleted the task/mt-5154 branch September 15, 2026 01:09
@edobry

edobry commented Sep 15, 2026

Copy link
Copy Markdown
Owner

Deploy verification — outcome (post-merge, 2026-09-15T01:1xZ): deployed, verified-1b. Posted as a comment because merge cleanup removed the session record session_pr_edit needs (mt#5177).

  • Merge 5fbee5ef944f59caa47dc130992dec8b48688f94 at 01:09:23Z.
  • deployment_wait-for-latest minsky-mcp (notBefore 01:09:23Z, expectCommitSha 5fbee5e…) → deployment fefd7e3f SUCCESS, createdAt 01:10:41Z, buildIdentity: indeterminate (image-source service — expected).
  • Correlated: Deploy MCP workflow run 34916131577, head_sha 5fbee5ef…, success, 01:09:25Z–01:10:49Z.
  • Runtime started: GET /health → {"service":"minsky-mcp","status":"ok"}.
  • Change-produced assertion: the hosted daemon's config_doctor result now carries summary.project and summary.user, with 6 project-scoped diagnostics (Project Harness, Compiled Outputs, Repository Identity, GitHub App Coverage, …) — none of which existed before this merge.

Had Claude run the verification; the same record is in the mt#5154 spec.

@minsky-reviewer minsky-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent adversarial review (Chinese-wall)
Reviewer: minsky-reviewer[bot] via openai:gpt-5
Tier: 2


R2 verification pass. The prior round approved with only non-blocking nits. This iteration introduces the project-scope checks and split rendering per spec. I verified: (1) domain-layer injectable project checks exist and cover harness/compiled/identity/coverage/hooks/token with good tests; (2) adapter stamps user-scope, runs project checks, computes split summaries, and healthy requires zero errors in both; (3) MCP server injects caller id for config.doctor; (4) all checks are read-only. CLI renderer groups by scope and degrades to flat output for older producers. I found one minor micro-optimization opportunity (non-blocking) in compiledCheck re-computing the caller output note. No regressions or blocking issues detected. Docs do not need updates for this additive result-shape change. Approve.

Findings

  • [NON-BLOCKING] packages/domain/src/doctor/harness-checks.ts:107 — Minor inefficiency: callerOutputsNote() computed multiple times
    In compiledCheck, callerOutputsNote() is invoked up to three times (for status decision, message construction, and suggestion). Since it does a few filesystem existence checks via deps.pathExists, it could be computed once and cached in a local const note = callerOutputsNote() to avoid redundant work. This is non-blocking and purely a micro-optimization; functionally correct as-is.

Spec verification

Criterion Status Evidence
1. The doctor gains PROJECT-scope diagnostics, computed for the project named by workspace ... implemented as injectable functions in packages/domain/src/doctor/project-checks.ts ... Met packages/domain/src/doctor/project-checks.ts:83-140 exports runProjectDoctorChecks that assembles six checks via injected deps; supporting checks in harness-checks.ts and repository-checks.ts implement (a)-(f). Tests in packages/domain/src/doctor/project-checks.test.ts cover pass/warn/error branches.
2. The result separates the scopes: every diagnostic carries scope, summary gains user and project sub-summaries, healthy requires zero errors in BOTH, CLI formatter prints two groups ... Met src/adapters/shared/commands/config/validate-doctor-commands.ts:480-515 stamps user diagnostics with scope and computes project diagnostics; lines 509-515 build combined array. Lines 216-235 add hidden callerActorId param. Lines 509-515 and 517-524 compute user/project summaries and healthy = user.errors === 0 && project.errors === 0. Renderer grouping at src/adapters/cli/customizations/config-customizations.ts:156-214 (renderScopedGroups). Tests at src/adapters/cli/customizations/config-customizations.test.ts:161-235 and validate-doctor-commands.test.ts:226-271 assert scopes and split summary.
3. config.doctor over MCP receives the caller's identity: tool name added to CALLER_ACTOR_ID_TOOL_NAMES and hidden CALLER_ACTOR_ID_PARAM declared. Met src/mcp/server.ts:326-333 adds CONFIG_DOCTOR_TOOL_NAME to CALLER_ACTOR_ID_TOOL_NAMES. src/adapters/shared/commands/config/validate-doctor-commands.ts:216-235 wires callerActorId: CALLER_ACTOR_ID_PARAM. Test at validate-doctor-commands.test.ts:211-224 sets callerActorId param and downstream assertions pass.
4. The doctor never writes: no grant request, no config edit, no compile output — every check is a read. Met project-scope functions in packages/domain/src/doctor/*.ts only call read/probe functions: defaultCompileCheck uses runMinskyCompile({check:true}) and probe (project-checks.ts:35-58), coverageCheck uses checkAppRoleCoverage (read-only) and formats a message (repository-checks.ts:71-119), token check performs GET (packages/domain/src/setup/token-repo-access.ts). No write operations present; adapter execute path (validate-doctor-commands.ts:480-524) only aggregates and returns diagnostics.

Adoption sweep

Symbol Kind Consumers found Classification Notes
@minsky/domain/doctor/runProjectDoctorChecks function src/adapters/shared/commands/config/validate-doctor-commands.ts:496-515 — invoked to gather project diagnostics Adopted

Documentation impact

  • no-update-needed — This PR changes internal behavior of config doctor output by adding project-scope diagnostics and grouped rendering, but I checked docs/configuration-guide.md for explicit assertions about doctor output/healthy semantics and found none. No CLI flags or public params changed beyond a hidden MCP param. Renderer behavior is additive and remains backward-compatible (flat form when no scope). Therefore no docs updates are required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

authorship/co-authored Co-authored by human and AI agent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant