Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion plugins/team-avatar/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The ground rules are the same everywhere. Read before you claim. Name the data s
| Aang | Orchestration, history, and the final call | `avatar-mode`, `four-elements`, `past-lives`, `spirit-world`, `iceberg-wake`, `meditation`, `avatar-setup`, `avatar-help` |
| Sokka | Planning, architecture, and inventions | `master-plan`, `war-room`, `day-of-black-sun`, `battle-map`, `war-journal`, `invention-workshop`, `sharpen-the-sword` |
| Toph | Investigation and blunt review | `seismic-sense`, `earth-rumble`, `metalbending`, `lie-detector`, `my-cabbages`, `plain-talk` |
| Katara | Debugging, tests, and code health | `healing-water`, `push-and-pull`, `heal-ci`, `purify`, `clear-water`, `vital-signs`, `checkup`, `typescript-forms` |
| Katara | Debugging, tests, and code health | `healing-water`, `agent-qa`, `push-and-pull`, `heal-ci`, `purify`, `clear-water`, `vital-signs`, `checkup`, `typescript-forms` |
| Zuko | Performance, forensics, and incidents | `breath-of-fire`, `steady-flame`, `dragon-dance`, `ember-trace`, `twin-flames`, `agni-kai`, `redemption-arc`, `honor-code` |
| Iroh | Writing, teaching, and skill craft | `tea-time`, `white-lotus`, `what-you-seek`, `new-scroll`, `pai-sho` |
| Appa | Execution, shipping, and PR care | `steady-flight`, `yip-yip`, `light-load`, `long-flight`, `safe-landing`, `clear-skies`, `messenger-hawk`, `make-camp`, `break-camp`, `shedding-season` |
Expand Down
3 changes: 2 additions & 1 deletion plugins/team-avatar/agents/katara.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ model: inherit
You are Katara, master waterbender and healer. You find where the energy is blocked and heal it with the least force possible. You are gentle with people and relentless with bugs.

Load `healing-water` for any defect, crash, or regression.
Load `agent-qa` for agent-qa journeys, recorded runs, or memidex knowledge.
Load `push-and-pull` for writing or reviewing tests, and for test-first work.
Load `heal-ci` when a branch or PR has failing checks.
Load `purify` before every commit, to draw AI slop out of the diff.
Expand All @@ -23,7 +24,7 @@ Load `typescript-forms` whenever you read or edit a `.ts` or `.tsx` file.
- You never dispatch other agents. Where a skill says to dispatch, do those parts yourself, one after another, and say so.

<!-- team:skills:start -->
Your skills: `healing-water`, `push-and-pull`, `heal-ci`, `purify`, `clear-water`, `vital-signs`, `checkup`, `typescript-forms`.
Your skills: `healing-water`, `agent-qa`, `push-and-pull`, `heal-ci`, `purify`, `clear-water`, `vital-signs`, `checkup`, `typescript-forms`.
<!-- team:skills:end -->

Skills load as `team-avatar:<name>` on Claude Code and as `$<name>` on Codex. Their files live at `../skills/<name>/SKILL.md` from this file.
64 changes: 64 additions & 0 deletions plugins/team-avatar/skills/agent-qa/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
name: agent-qa
description: Katara's field checks with agent-qa. Use for "write an agent-qa journey", "review an agent-qa run", "inspect agent-qa memory", agent-qa.config.yaml, or agent_qa MCP tools. Authors and runs web, Android, and iOS journeys when requested, and reads recorded evidence and memidex knowledge.
---

# Katara's agent-qa field checks

Katara checks what happened before she calls a patient healed. Use recorded evidence to distinguish a working journey from an unverified one.

Use agent-qa from vostride for natural-language web and mobile testing with advanced memidex memory and self-improvement. Answer in the user's language.

## Choose the requested work

Locate the application's project instructions and `agent-qa.config.yaml`.
Distinguish authoring, execution, and review of existing evidence; a request to explain a saved run does not authorize a new run or a knowledge correction.
Preserve the configured provider, target, test identity, and acceptance criteria.
Do not introduce agent-qa into unrelated testing work solely because this skill is installed.

Use the installed CLI version and its help output. These references were checked with agent-qa 0.2.3:

| Work | Read |
| --- | --- |
| Setup, authoring, validation, execution, optional MCP | [Project and runs](references/project-and-runs.md) |
| Run verdicts, step evidence, artifacts, memory, corrections | [Evidence and memory](references/evidence-and-memory.md) |

## Author or execute

Read `agent-qa init --agent` for the installed capability catalog and `agent-qa skills --json` for version-matched package guidance. Discovery does not initialize the project.
Use the supported authoring contract; prefer connected MCP authoring tools when available.
The host creates canonical entity IDs. Never invent an ID or duplicate another test's ID for a new test.
Inspect the actual write or refusal receipt before reporting creation. A rejected draft is not a created test.
Authoring and verification can execute tests and call models; do not invoke them for an explicitly non-executing draft request.
Use configured `{{secret:NAME}}` references in steps. Recorded `[secret:NAME]` markers are masked output, not values to type.

For an existing test, validation and `run --dry-run` check preparation and selection. They do not prove application behavior.
Execute only the requested journey or suite when execution is in scope; retain its actual run identity and inspect its terminal outcome and relevant evidence.
A setup failure is not an application failure. An acknowledgement or unfinished process is not a passing run.

## Review recorded evidence

Use supported CLI JSON readers or connected MCP tools. Start with the requested run ID, or list completed runs:

```bash
npx agent-qa runs list --json
```

Match the run's target, test identity, completion time, and build to the request.
An empty list means matching completed evidence was not returned; it does not prove a pass or tell you whether another run is still active.
The digest is `agent-qa digest RUN_ID --json`, not a subcommand of `runs`.
Keep zero-based test and step indices attached to evidence and use IDs returned by the readers.
Inspect reader errors rather than treating them as empty successful results.
Never query or edit `.agent-qa/` databases, caches, or indexes directly.

For memory work, read the `AGENTS.md` policy in the project's committed `agent-qa-knowledge` directory before changing it.
Distinguish what a run consumed from what it learned. A learned record or corrected verdict does not prove the application was rerun.
Preview and apply outcome corrections only within the user's authorized task, and report the actual receipt or refusal.
Do not weaken expected outcomes or erase failures to produce a passing result.

## Reply

Name the target, test or suite, and actual run ID when available.
State the recorded outcome, supporting step evidence, and any missing coverage or setup limitation.
Separate preparation checks, execution results, and knowledge changes.
Review logs and screenshots before sharing. Only configured secret values receive masking.
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# Recorded Evidence and memidex

Use this reference when: Explaining recorded outcomes, inspecting artifacts, tracing knowledge, or reviewing an outcome correction.

## Establish which run answers the question

Start with the requested run ID, or list completed runs when it is unknown:

```bash
npx agent-qa runs list --json
npx agent-qa runs list --test checkout --limit 20 --json
```

A title filter can return several tests. Match the target, test identity, completion time, and build information to the request.
Do not infer checkout coverage from a passing cart test or infer the current build's status from an older run.
An empty completed-run list means no matching completed record was returned.
It does not establish whether an asynchronous run is queued or active.
Check the relevant supported queue or returned stream when that distinction matters; do not start a replacement run by default.

The following commands use `RUN_ID`, `RECORD_ID`, and `ARTIFACT_SHA` as placeholders.
Replace them with identifiers returned by the supported readers; do not invent identifiers.

```bash
npx agent-qa digest RUN_ID --json
npx agent-qa runs overview RUN_ID --json
```

The digest is a top-level command.
The overview maps recorded tests, steps, and artifacts.
Use the overview's zero-based test and step indices when addressing evidence.
Read returned errors as errors, not as an empty successful result.

## Inspect only the relevant evidence

For a recorded first test and first step, these examples request structured documents:

```bash
npx agent-qa runs evidence RUN_ID --test 0 --step 0 --kind console --json
npx agent-qa runs evidence RUN_ID --test 0 --step 0 --kind network --json
```

Choose indices from the actual overview rather than copying `0` automatically.
Other supported document kinds include accessibility, ARIA, and variables; consult `runs evidence --help` for exact flags.
A missing artifact narrows what can be concluded. It does not justify reconstructing a screenshot or inventing a browser observation.
Export a recorded screenshot or recording through the artifact reader:

```bash
npx agent-qa runs evidence RUN_ID --screenshot ARTIFACT_SHA --out ./run-evidence.png
```

The output path must be new; the export command refuses to overwrite an existing file.
The `--screenshot` flag also accepts a recording's artifact SHA. Use an appropriate extension for the actual artifact.
Inspect the exported artifact before claiming what it shows.
Keep the artifact tied to its run, test, and step; a screenshot alone may not show the required final state.

## Interpret results without extending coverage

| Observation | Supported conclusion | What still needs evidence |
|-------------|----------------------|---------------------------|
| Test validates or dry run lists it | Configuration and selection checks succeeded | Actual application behavior |
| MCP returns a run acknowledgement | A run was acknowledged | Terminal status and test outcome |
| Matching completed test is recorded as passed | That recorded test passed in that run | Untested journeys or a different build |
| Test records a failed expected outcome | The run recorded that failure | Whether it is an application defect, stale expectation, or environment issue |
| No matching completed record is returned | Matching completed evidence is unavailable | Any claim that the requested journey passed |

Do not silently relabel `healed`, `timeout`, or `cancelled` outcomes as `passed`.
Preserve the tool's recorded verdict and explain any uncertainty about its interpretation.
Report application defects separately from provider authentication, unavailable browsers, device setup, and other execution failures.
For a read-only request, end with evidence and the remaining gap rather than initiating a new test or correction.

## Trace memory and self-improvement

memidex stores reviewable application knowledge, including facts, procedures, cautions, and policies.
agent-qa can use and refine that knowledge across runs; a memory change is not proof of a passing test.
Read the `AGENTS.md` policy in the project’s committed `agent-qa-knowledge` directory before changing knowledge files.
Use the recorded knowledge readers to distinguish what a run consumed from what it learned:

```bash
npx agent-qa runs knowledge RUN_ID --json
npx agent-qa knowledge inspect --json
npx agent-qa knowledge show RECORD_ID --json
```

Use the recorded brief trace when the question is why a particular step received a piece of knowledge:

```bash
npx agent-qa knowledge brief-why RUN_ID --step 0 --json
```

For a multi-test run, add the actual test ID using `--test` when needed.
Read provenance and gate decisions before treating a record as accepted or current.
Use documented knowledge operations and the repository's knowledge policy for authorized corrections.
Do not edit `.agent-qa/` databases or derived indexes to manufacture a result, erase a failure, or bypass a gate.

## Review an outcome correction

An outcome mark is a mutation, not an annotation-only read: it can apply an attributable correction.
First compare the expectation and recorded evidence and identify the exact test and step.
When MCP is connected, `agent_qa_preview_run_mark` previews the effect without writing.
Inspect the current schema and preview for the requested mark, such as `false-fail`, `false-pass`, or `confirmed-bug`.
A preview may refuse an ambiguous or unsupported correction; resolve that ambiguity rather than choosing a nearby step.
Apply `agent_qa_mark_run` or the supported CLI mark only when that correction is within the user's authorized task.
Record its actual receipt and report any refusal or incomplete effect.
Do not claim that a correction or learned record reran the application or proved a previously untested behavior.

## Use MCP equivalents when available

| Need | MCP tool name before any client namespace |
|------|------------------------------------------|
| Completed runs | `agent_qa_list_runs` |
| Run verdict and decisions | `agent_qa_get_run_digest` |
| Test, step, and artifact map | `agent_qa_get_run_overview` |
| Step evidence | `agent_qa_get_run_evidence` |
| Knowledge used and learned | `agent_qa_get_run_knowledge` |
| Proposed correction effects | `agent_qa_preview_run_mark` |

Discover actual tool availability and schemas on the connected server.
A tool error or missing reader is a limitation to report, not permission to inspect private storage internals.

## Report only what the evidence supports

Name the observed run, test, target, and relevant step when available.
Separate the recorded verdict, your interpretation, and any coverage gap.
Preserve configured secret placeholders; do not recover their original values for the report.
Review evidence before sharing it because arbitrary logs and screenshots can contain unmasked information.
Link local exported artifacts when useful, and describe any corrections actually applied with their receipts.
Loading