Skip to content

Add native Harbor agents and managed harness runtimes for 0.3.0 - #51

Merged
hanhainebula merged 1 commit into
mainfrom
feat/harness-runtime-0.3.0
Nov 7, 2026
Merged

hanhainebula merged 1 commit into
mainfrom
feat/harness-runtime-0.3.0

Conversation

@hanhainebula

Copy link
Copy Markdown
Collaborator

Expand evaluation beyond the three integrated harnesses and manage coding-agent CLI versions centrally. Users can select Harbor's registered agents through --provider native, inspect capabilities with search-swe agents, and override harness versions without editing task Dockerfiles.

Changes

  • Add native agent discovery and configuration for model IDs, provider credentials, agent kwargs, authentication files and API hosts. Scope credentials to the selected agent, restore task network policy after trusted setup, and check ATIF compatibility before preparing assets. Preserve the integrated Codex, Claude Code and Pi routes.
  • Add src/search_swe/data/harnesses.toml for default versions and installation recipes. Manage Codex, Claude Code, Pi, Gemini CLI and Antigravity CLI with actual-version verification and a private, integrity-checked cache keyed by task image, version and installer recipe. Support --agent-version overrides; other native agents use Harbor's installers.
  • Remove repeated coding-harness installations from all 47 task Dockerfiles and update the task scaffold. Keep task dependencies in their images and verifier judge pins independent of coding-agent versions.
  • Match Grok Build and Junie BYOK credentials to the selected provider, and map Vibe's generic backend to its provider key and endpoint. Preserve service defaults and explicit native configuration.
  • Make large-scale-formula-retrieval logs work across harness layouts while retaining bounded storage and private resource evidence. Freeze solver processes before archival, check expanded archives against the existing 512 MiB diagnostic reserve, and stop oversized archives as infrastructure errors before verification while retaining source logs.
  • Share Pi's event-stream-to-ATIF export across integrated and native routes. Collect Antigravity's new or updated sessions using file identities and content fingerprints, including same-timestamp updates and SQLite WALs; Harbor continues to perform its ATIF conversion.
  • Prepare package version 0.3.0 and align CLI documentation, English/Chinese READMEs, .env.example, contributor scaffolding and evaluation skills.

CLI and checkout use the same 0.3.x release series. Task instructions, resource budgets, scoring formulas, input assets and verifier judge versions are unchanged. Existing benchmark results and trajectories are preserved.

Validation

  • Repository unit suite: 417 passed. Task-authoring scaffold: 10 passed. Maintainer skill: 12 passed.
  • All 47 task packages passed merge-ready static checks. Release checks reported 0 errors and 0 warnings; git diff --check passed.
  • Real CLI setup checks covered version verification and cold/warm installation caches for the five managed harnesses.
  • Real Harbor/Docker/FUSE lifecycle fixtures passed Codex, Claude Code, Pi integrated/native, Antigravity and Gemini, including ATIF transfer to an independent verifier. These lifecycle tests used synthetic CLI protocol fixtures and native converters.
  • A 1 GiB sparse-log fixture was rejected before expansion, retained its source and confirmed the verifier never started. Antigravity's fixture forced identical transcript/marker timestamps and completed without a sleep workaround. Regressions also cover hardlink expansion, archive metadata overhead, old-session exclusion and provider authentication mappings.
  • Documentation checks covered 39 files, 176 local links/anchors, 24 CLI examples and all three skill formats.

No coding-model or judge API requests, full benchmark reruns or historical-result rewrites were performed for this change. Native model/provider availability still follows the selected Harbor integration and its credentials; the fixtures validate infrastructure rather than live inference compatibility.

Release 0.3.0 with centralized harness versions, verified installation and private caches. Adapt task images, bounded logs and ATIF transfer for multiple harnesses, and align CLI documentation and skills.
@hanhainebula
hanhainebula merged commit 6e41a94 into main Nov 7, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant