Skip to content

feat(spec-loop): add Grok headless harness profile - #1593

Open
KatalKavya96 wants to merge 2 commits into
apache:mainfrom
KatalKavya96:feat-grok-spec-loop
Open

KatalKavya96 wants to merge 2 commits into
apache:mainfrom
KatalKavya96:feat-grok-spec-loop

Conversation

@KatalKavya96

Copy link
Copy Markdown
Contributor

Summary

  • Add xAI Grok as a first-class headless harness for tools/spec-loop.
  • Map the existing spec-loop contract onto Grok Build 1.0.50 using --prompt-file, --cwd, --always-approve, output/model/effort flags, and explicit deny rules for remote mutation.
  • Add fixture coverage and update the spec-loop, Grok adapter, registry, and vendor-neutrality documentation so the implementation and declared harness support stay in sync.

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other: spec-loop shell runner / headless harness support

Test plan

  • prek run --all-files passes
  • For Python packages touched: uv run pytest / ruff check / mypy passes
  • For Groovy bridges touched: command-line invocation tested end-to-end
  • For skill changes: eval suite passes for the affected skill
    (PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/)
  • For skill behaviour changes: a new or updated eval fixture is included in this PR
    (a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)
  • Other:
    • bash tools/spec-loop/tests/test_runner_fixtures.sh
    • bash -n tools/spec-loop/loop.sh
    • bash -n tools/spec-loop/lib.sh
    • bash -n tools/spec-loop/tests/test_runner_fixtures.sh
    • live-tested with Grok Build 1.0.50 in a disposable repository
    • verified the headless profile can create a local file while leaving unrelated tracked files unchanged
    • live-verified the exact Bash(git push:*) and Bash(gh:*) deny rules block matching commands even with --always-approve

RFC-AI-0004 compliance

  • HITL — any new mutation is gated on explicit user confirmation
  • Sandbox — no new unrestricted host access; Grok spec-loop continues to rely on the external OS sandbox, with per-invocation deny rules as defense in depth
  • Vendor neutrality — Grok is added through the existing harness-neutral spec-loop contract and vendor-neutrality metadata is regenerated
  • Conversational + correctable — agentic-override path documented if behaviour is adopter-tunable
  • Write-access discipline — the runner explicitly blocks git push and gh from the Grok invocation; the spec-loop remains local-only and does not autonomously open or push PRs
  • Privacy LLM — private content does not reach a non-approved LLM; redactor invoked where needed

Linked issues

Refs #1416

Notes for reviewers (optional)

The Grok profile was validated against Grok Build 1.0.50 rather than inferred from another harness.

The runner maps:

  • assembled prompt -> --prompt-file
  • repository root -> --cwd
  • unattended local execution -> --always-approve
  • SPEC_LOOP_OUTPUT_FORMAT=text -> plain
  • SPEC_LOOP_OUTPUT_FORMAT=stream-json -> streaming-json
  • SPEC_LOOP_MODEL -> --model
  • SPEC_LOOP_EFFORT=low|medium|high -> --reasoning-effort low|medium|high

The exact deny rules used in the runner were also exercised against the installed CLI:

  • Bash(git push:*)
  • Bash(gh:*)

Both were denied before execution under --always-approve.

A disposable-repository probe was used instead of running a destructive full spec-loop build against the Magpie checkout. The profile successfully made the requested local edit and did not modify unrelated tracked files.

@github-actions github-actions Bot added family:tools tools/* family:docs Docs, MISSION.md, READMEs substrate:framework-dev Tool substrate: build / validate / eval the framework itself labels Oct 10, 2026
@KatalKavya96

Copy link
Copy Markdown
Contributor Author

@potiuk You can post this as the maintainer-facing PR comment:
This adds the Grok spec-loop harness slice from #1416.
I verified the profile live against Grok Build 1.0.50, including headless local edits, output/model/effort mapping, and the exact git push / gh deny rules under --always-approve.
Full prek run --all-files passes locally. The change stays within the existing spec-loop harness contract and does not introduce a Grok-specific sandbox profile; the external OS sandbox remains the primary isolation layer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

family:docs Docs, MISSION.md, READMEs family:tools tools/* substrate:framework-dev Tool substrate: build / validate / eval the framework itself

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant