fix(skills): align skill catalog names with read_skill resolution#250
Conversation
The system-prompt skill catalog advertised each skill by its internal '## Tool:' heading name (e.g. 'k8s_triage'), but read_skill resolves a skill by its loadable name — the frontmatter 'name' / directory (e.g. 'k8s-incident-triage', which is also what the agent card advertises). When the two differed, the LLM called read_skill with the advertised tool name and got 'skill not found', then abandoned the skill. Changes: - buildSkillCatalog now emits one line per skill keyed by the loadable name (frontmatter name, falling back to directory/file) — the exact string read_skill resolves and the agent card advertises. Each line also carries a 'provides: <tool names>' list so the LLM still sees the skill's capabilities for tool selection, without confusing them for the read_skill argument. Script/binary-backed '## Tool:' entries are excluded from 'provides' because they are registered as first-class callable tools (registerSkillTools) and invoked directly by name. - read_skill resolves via a frontmatter-name index (keyed by both the frontmatter name and the directory/file name, normalized for case and underscore/hyphen drift) in addition to the direct path lookup; a miss now returns the available skill names instead of a bare error. Regression tests: read_skill resolves by frontmatter name when the directory differs, normalized variants, flat layout, not-found lists available skills; catalog advertises the loadable name (not the tool heading) while surfacing tool names under 'provides'; and a catalog<->read_skill contract test asserting every advertised name resolves (the gap that let this ship).
…ference) Skills ship more than SKILL.md — helper scripts (shell, python, javascript), reference folders, and additional markdown all live in the skill's directory and reach the running agent via COPY . . at build time, but were invisible to the LLM because read_skill returned only SKILL.md. read_skill now appends a '## Skill files' listing of every other file in a subdirectory skill's folder, relative to the agent root and annotated with the script language, so the model can read or execute them as the skill's steps describe. Flat single-file skills are unaffected. Also documents why skillEntryHasScript (the catalog 'provides' exclusion) stays aligned to registerSkillTools' .sh-only registration: a py/js tool is not yet a callable tool, so it correctly stays in 'provides' and its script is surfaced by the new file listing. Broadening the extension set must wait for registerSkillTools to register those languages, or the tools would vanish from both the tool list and the catalog.
|
Folded in a follow-up (46b91a3): read_skill now surfaces the skill's directory files. Skills ship more than SKILL.md — helper scripts (shell / python / javascript), reference folders, and extra markdown all live in the skill's directory and reach the running agent ( Flat single-file skills ( On the catalog side, the Tests added: file listing includes py/js/sh scripts + reference files and excludes SKILL.md; flat skills append nothing. |
|
Follow-up tracked in #251 — skill-relative file resolution (read referenced markdown and execute referenced |
Problem
An agent with a skill whose loadable name differs from its internal
## Tool:heading could never load that skill. Observed live: the LLM was shownk8s_triagein the skill catalog and calledread_skill{"name":"k8s_triage"}, but the skill's loadable name (frontmattername/ directory, also what the agent card advertises) wask8s-incident-triage.read_skillresolves by filesystem path, so it returned{"error":"skill \"k8s_triage\" not found"}, and the model abandoned the skill and improvised rawkubectl.Root cause: three divergent identifiers for one skill.
name:k8s-incident-triage## Tool:heading (entry.Name)k8s_triageread_skillresolutionskills/<arg>/SKILL.md(+_→-)k8s-incident-triageThe catalog advertised the internal tool heading while
read_skillresolved by the skill's loadable name, and the two never had to agree. Unit tests never crossed the catalog→read_skillboundary, so nothing caught it.Changes
buildSkillCatalog(forge-cli/runtime/runner.go) — now emits one line per skill keyed by the loadable name (frontmattername, falling back to directory/file) — the exact stringread_skillresolves and the agent card advertises. Each line also carries aprovides: <tool names>list so the LLM still sees the skill's capabilities for tool selection, without mistaking them for theread_skillargument. Script/binary-backed## Tool:entries are excluded fromprovidesbecause they are registered as first-class callable tools (registerSkillTools) and invoked directly by name.read_skill(forge-core/tools/builtins/read_skill.go) — resolves via a frontmatter-name index (keyed by both the frontmatternameand the directory/file name, normalized for case and_/-drift) in addition to the direct path lookup. A miss now returns the available skill names (available_skills) so the model can retry instead of giving up. Directory-traversal guard preserved.Why both halves
read_skillwith the resolvable name in the common case (directory == frontmatter name).read_skillindex makes resolution robust when the directory differs from the frontmatter name, and turns a dead-end "not found" into an actionable list.Tests
forge-core/tools/builtins/read_skill_test.go— resolves by frontmatter name when the directory differs; normalized variants (k8s_incident_triage,K8S-Incident-Triage); flatskills/<name>.mdlayout; not-found lists available skills; traversal rejected.forge-cli/runtime/runner_skill_catalog_test.go— catalog keys the line on the loadable name (not the tool heading) while surfacing tool names underprovides; and a catalog↔read_skill contract test asserting every advertised name resolves throughread_skill(the integration gap that let this ship).Full
forge-core/tools/...andforge-cli/runtimesuites green;gofmt -w+golangci-lint runclean on both changed packages.