Add the roboflow-workflow-evals skill

[joaomarcoscrs]

Adds roboflow-workflow-evals, the skill that teaches agents when and how to evaluate a Roboflow Workflow against ground truth. Workflow Evals goes GA this week, and until now agents only learned it from MCP tool descriptions.

The skill

File	Written by	Covers
SKILL.md	people	When to use Workflow Evals and when to use model evals, inference, or batch processing instead. The mental model, the engine version check, the end-to-end loop, and the rules that prevent wasted Runs.
authoring.md	people	Specs, evaluator and pass-rule choice, Eval Datasets, Cases and image uploads, ground-truth state, bindings and their path syntax, and Eval creation.
lifecycle.md	people	Case preparation, Runs and bounded polling, retry, replay and rescore, results, comparison, exports, embeddings, and two-step cleanup.
troubleshooting.md	people	The error envelope, status codes, common failures, dated known issues, and REST usage without MCP.
reference/	generated	Authoring schemas, evaluator catalog, conceptual model, the engine's create-eval procedure, and examples for one engine release, listed in reference/manifest.json.

The guidance comes from an end-to-end run of every Workflow Evals MCP tool against production, plus the platform and engine code. Behavior that differs by engine version (evaluator types, schemas) is read from reference/ when versions match and from workflow_evals_reference_get when they do not. The skill tells agents to check workflow_evals_capabilities first.

roboflow-inference and roboflow-training-and-evaluation now point here for Workflow-level evaluation. The training skill stays under the 20,000-character limit (19,933).

Keeping it current

reference/ is not edited by hand. On each engine release, roboflow/workflow-evals regenerates it with npm run agent:export-public, lets an agent update the guides, and opens a draft pull request here labeled engine-sync. The private release notes go to that agent only, never into the public pull request.

New CI in this repository:

• Validate skills (every PR): checks frontmatter names, the Cowork limits (20,000 characters in SKILL.md, 20 companion files), relative links, nested SKILL.md files, and that a generated reference/ declares exactly the files it ships. A second job fails a pull request that changes skills/*/reference/* without the engine-sync label. This PR carries the label because it adds the first snapshot.
• Promote engine sync (every 6 hours): marks the newest engine-sync draft ready once production's workflow_evals_capabilities reports that engine version, so the skill never documents an engine the API does not run yet. A draft whose engine is not newer than the reference on main would roll the reference back, so it is closed with the reason. A draft that only an unmerged newer sync outranks waits. If any open sync pull request's version cannot be read, the run promotes and closes nothing.

The snapshot here is engine 0.5.0. Production runs an older engine today, and the version check covers that gap.

Setup needed after merge

• Repository secret ROBOFLOW_API_KEY and variable ROBOFLOW_WORKSPACE for Promote engine sync. Any workspace works; the job only reads capabilities. Without them the job logs a warning and does nothing.
• The workflow-evals side needs a GitHub App installed here with contents, pull requests, and issues write access. See roboflow/workflow-evals for its variables.

Cowork

The skill joins the Cowork package (11 of 20 skills) and the package version goes to 1.1.0. python3 cowork/build.py builds and validates it locally.

Related

• roboflow/roboflow-mcp: consolidates discovery into workflow_evals_reference_get and adds the fixes this skill relies on. Dependabot then bumps the MCP's skills submodule, which serves this skill as roboflow://skills/roboflow-workflow-evals/SKILL.
• roboflow/roboflow: binding suggestion and validation fixes, plus draft Case creation.
• roboflow/workflow-evals: the export and the sync workflow.

Checks run

• python3 .github/scripts/validate_skills.py: 11 skills valid.
• python3 -m unittest discover -s .github/scripts: 17 tests.
• python3 -m unittest discover -s cowork -p "test_*.py" and python3 cowork/build.py.

[joaomarcoscrs]

[Jarbas Local João] — APPROVED at 86988b96e156

Gist: https://gist.github.com/joaomarcoscrs/176b4b8922f06dbada159a59d42e0ed2

[joaomarcoscrs]

Closing: we are keeping this change to the Workflow Evals bug fixes (roboflow/roboflow#16897 and roboflow/roboflow-mcp#206) and not changing how skills and reference material are organized for now. The branch stays available if we pick this up later.