Skip to content

feat(ambient-quality-agent): release v0.3.0 - #2865

Merged
ale-grassi merged 1 commit into
mainfrom
ambient-quality-agent/release-v0.3.0
Nov 4, 2026
Merged

ale-grassi merged 1 commit into
mainfrom
ambient-quality-agent/release-v0.3.0

Conversation

@ale-grassi

Copy link
Copy Markdown
Member

AQuA 0.3.0 reviews each session with an agent that can query the whole session, lets you store an investigation selector for an attached agent, and caps what one telemetry query can bill. The CLI, the API and the dashboard now say investigation, trajectory and finding where they said run, trace and rubric. BigQuery tables and columns keep their names. Read the breaking changes before you upgrade.

Breaking changes

  • The CLI and AQuA's API use the investigation vocabulary. BigQuery tables, columns and the JSON stored in them keep their names, so queries over AQuA's tables keep working.
    • list-insights and get-insight take --investigation-id in place of --run-id.
    • list-investigations returns {"investigations": [...]} in place of {"runs": [...]}.
    • Renamed JSON fields include run_id (now investigation_id), an insight's agent_name (now observed_agent_key), trace_count (now trajectory_count), item_count (now finding_count) and the traces_* counters, such as counters.traces_scanned (now counters.trajectories_scanned).
    • In get-insight, an occurrence's rubrics[] is now examples[], in which rubric is now finding and trace is now turns. The insight's trajectories[] is now investigation_trajectories[].
    • Upgrade. Update scripts that pass --run-id or read the old field names. The agents-cli-aqua skill of this release reads the new names.
  • The investigation-finished log entry uses the new names. AQuA logs jsonPayload.investigation_id in place of jsonPayload.run_id, and trajectories_evaluated, trajectories_eval_failed and trajectories_eval_errored in place of the matching traces_* fields. The alert policy in terraform/modules/aqa reads the new fields.
    • Upgrade. Apply AQuA's Terraform when you upgrade, for example with agents-cli infra single-project --apply-aqua --apply, so that the alert reads the new fields. Update your own log queries, log-based metrics and alerts that read the old fields.
  • Session review runs an agent with its own model. For each session, a review agent reads a shortened copy of the session and queries the full session with a jq tool. In 0.2.0, session review made one call per session to AQA_INSIGHTS_MODEL. The reviewer uses AQA_REVIEW_MODEL and AQA_REVIEW_LOCATION, by default gemini-4-argon at global. A Gemini review model must support thinking_level (Gemini 3 or later). A model whose name starts with claude-, for example claude-opus-5-5 at global, runs Claude on Vertex AI. AQA_REVIEW_PRUNE_WATERMARK_TOKENS (default 400000) sets the prompt size at which the reviewer shortens its oldest tool results.
    • Upgrade. If your project cannot call gemini-4-argon at global, set AQA_REVIEW_MODEL and AQA_REVIEW_LOCATION to a model it can call. If you set AQA_INSIGHTS_MODEL to change the session review model, set AQA_REVIEW_MODEL instead.
  • Cluster verification is off by default. The verification pass runs only for an agent attached with --verification, and --enforce-verification takes effect only with --verification. Without the pass, an insight keeps its clustering label, and its occurrences have no analyses.verification. The pass puts up to three example trajectories per cluster into one prompt, so long sessions could push it past the model's context window. When every call failed, AQuA created no insights.
    • Upgrade. An agent attached without --verification stops verifying. To keep the pass, run agents-cli aqua attach --verification for that agent.
  • A telemetry query can bill at most 1 TiB. Every telemetry query job sets a 1 TiB bytes-billed limit and, by default, a 30-minute timeout. BigQuery stops a query above the limit before it runs and bills nothing, and the investigation fails with an error that names the limit.
    • Upgrade. If an investigation fails with this error, shorten the lookback window with agents-cli aqua attach --lookback-days, or store a selector that reads fewer conversations (see Changes).
  • agents-cli deploy samples every trace of the observed agent. It passes OTEL_TRACES_SAMPLER=always_on to your agent, merged into your --update-env-vars. Agent Runtime forwards some requests, such as A2A messages, unsampled, and the default sampler then drops every span they produce, so AQuA could not review those turns.
    • Upgrade. To keep your own sampler, set OTEL_TRACES_SAMPLER in --update-env-vars or in the project's .env. agents-cli deploy then leaves it as it is.
  • Dashboard trajectory pages moved. A trajectory page is at /investigations/<investigation id>/trajectories/<trajectory id>. Links to /investigations/<id>/cases/<id> from 0.2.0 do not open.
    • Upgrade. In saved links, replace /cases/ with /trajectories/.

Changes

  • Stored investigation selector. An attached agent can store an SQL selector. Scheduled, update-triggered and manual investigations then review only the conversations it selects, in place of a random sample. A custom investigation keeps its own selector.
    • Set it with agents-cli aqua attach --selector-file PATH. - reads standard input, and an empty file clears the selector.
    • The selector returns target_id, uses @window_start, @window_end and @agent_name, and reads only the telemetry table. It needs the session_review quality analysis mode. Each investigation checks it before its queries.
    • To change the telemetry table of an agent that has a stored selector, set or clear the selector in the same attach.
  • Dashboard: Investigation selector card. The Configuration page shows the stored selector, or says that investigations review a random sample. You can check, save and clear the selector there. A save needs a passing check, which is a dry run that reads no rows. "Add selector" opens the editor on the query that the random sample uses, to narrow with more conditions.
  • Custom investigations over views. A custom investigation's selector can read an attached logical view, as long as it names the view and not the tables behind it. A selector that calls a routine other than a built-in function or a routine of the view is refused as unauthorized_routine. A selector that reads a table AQuA cannot read is refused as unauthorized_table, and one whose selector table AQuA cannot read as selector_table_unreadable.
  • Progress of a running investigation. While an investigation is pending or running, get-investigation and list-investigations show its progress: the stage it is in and its counts so far.
  • Dashboard: insight detail page. The page starts with a summary: what's wrong, the root cause, a suggested fix and one example finding. The latest occurrence with its trajectories and the proposed code changes follow. Earlier occurrences are folded under History. History and "Show N more findings" stay open or closed as you last left them.
  • Coding-agent skill. The agents-cli-aqua skill asks how to proceed before it edits anything: one insight at a time (recommended), or one group of insights that share a root cause. It asks again before the next insight or group. It adds a unit test for a code defect and an eval case for a prompt defect. When agents-cli aqua info --ui-url returns a URL, it links insights and investigations to the dashboard.
  • Telemetry name checks. attach refuses a telemetry dataset or table name that BigQuery's naming rules do not allow.
  • make demo prints a link that opens the guided tour, and the dashboard shows a "Demo data" badge.
  • Dependencies. AQuA needs google-adk 2.13.0 or later, and depends on anthropic[vertex], jq and aiohttp. Dependencies with known vulnerabilities moved to fixed versions, among them aiohttp, certifi, filelock, OpenTelemetry, vite, happy-dom and zod.

Fixes

  • Custom investigations refused every selector over the Cloud Trace _AllSpans view, the cloud_ops default that attach picks. They accept them now.
  • Ingestion works on telemetry tables that require a partition filter. It reads a selected session's rows from 7 days before to 7 days after the window, so a longer session is read only within that range.
  • With the big_query source, session review read the BigQuery Agent Analytics plugin's text: '...' wrapper as part of the agent's reply, and raised a false insight on every such agent. AQuA now unwraps a reply made only of text parts.
  • attach refuses a telemetry table that it finds missing or in the wrong format after the grants, as it does before them. It revokes the grants that the run made, removes a first attachment and restores the settings of a re-pointed one. Before, it exited 0, and every later investigation failed.
  • attach and detach no longer print each resource's whole IAM policy after a grant.
  • The engine and dashboard images build on Python 3.13, inside the supported range. They were on Python 3.15.
  • The guided tour brings its step back into view after a trackpad scroll.

Known issues

  • agents-cli aqua run always talks about the default agent. Use the dashboard chat, which follows the selected agent.
  • Some progress messages can be lost when investigations overlap. The investigation's status, insights and trajectories are not affected. AQuA logs the warning could not record event row.

AQuA 0.3.0 reviews each session with an agent that can query the whole session, lets you store an investigation selector for an attached agent, and caps what one telemetry query can bill. The CLI, the API and the dashboard now say investigation, trajectory and finding where they said run, trace and rubric. BigQuery tables and columns keep their names. Read the breaking changes before you upgrade.

### Breaking changes

- **The CLI and AQuA's API use the investigation vocabulary.** BigQuery tables, columns and the JSON stored in them keep their names, so queries over AQuA's tables keep working.
  - `list-insights` and `get-insight` take `--investigation-id` in place of `--run-id`.
  - `list-investigations` returns `{"investigations": [...]}` in place of `{"runs": [...]}`.
  - Renamed JSON fields include `run_id` (now `investigation_id`), an insight's `agent_name` (now `observed_agent_key`), `trace_count` (now `trajectory_count`), `item_count` (now `finding_count`) and the `traces_*` counters, such as `counters.traces_scanned` (now `counters.trajectories_scanned`).
  - In `get-insight`, an occurrence's `rubrics[]` is now `examples[]`, in which `rubric` is now `finding` and `trace` is now `turns`. The insight's `trajectories[]` is now `investigation_trajectories[]`.
  - **Upgrade.** Update scripts that pass `--run-id` or read the old field names. The `agents-cli-aqua` skill of this release reads the new names.
- **The investigation-finished log entry uses the new names.** AQuA logs `jsonPayload.investigation_id` in place of `jsonPayload.run_id`, and `trajectories_evaluated`, `trajectories_eval_failed` and `trajectories_eval_errored` in place of the matching `traces_*` fields. The alert policy in `terraform/modules/aqa` reads the new fields.
  - **Upgrade.** Apply AQuA's Terraform when you upgrade, for example with `agents-cli infra single-project --apply-aqua --apply`, so that the alert reads the new fields. Update your own log queries, log-based metrics and alerts that read the old fields.
- **Session review runs an agent with its own model.** For each session, a review agent reads a shortened copy of the session and queries the full session with a `jq` tool. In 0.2.0, session review made one call per session to `AQA_INSIGHTS_MODEL`. The reviewer uses `AQA_REVIEW_MODEL` and `AQA_REVIEW_LOCATION`, by default `gemini-4-argon` at `global`. A Gemini review model must support `thinking_level` (Gemini 3 or later). A model whose name starts with `claude-`, for example `claude-opus-5-5` at `global`, runs Claude on Vertex AI. `AQA_REVIEW_PRUNE_WATERMARK_TOKENS` (default 400000) sets the prompt size at which the reviewer shortens its oldest tool results.
  - **Upgrade.** If your project cannot call `gemini-4-argon` at `global`, set `AQA_REVIEW_MODEL` and `AQA_REVIEW_LOCATION` to a model it can call. If you set `AQA_INSIGHTS_MODEL` to change the session review model, set `AQA_REVIEW_MODEL` instead.
- **Cluster verification is off by default.** The verification pass runs only for an agent attached with `--verification`, and `--enforce-verification` takes effect only with `--verification`. Without the pass, an insight keeps its clustering label, and its occurrences have no `analyses.verification`. The pass puts up to three example trajectories per cluster into one prompt, so long sessions could push it past the model's context window. When every call failed, AQuA created no insights.
  - **Upgrade.** An agent attached without `--verification` stops verifying. To keep the pass, run `agents-cli aqua attach --verification` for that agent.
- **A telemetry query can bill at most 1 TiB.** Every telemetry query job sets a 1 TiB bytes-billed limit and, by default, a 30-minute timeout. BigQuery stops a query above the limit before it runs and bills nothing, and the investigation fails with an error that names the limit.
  - **Upgrade.** If an investigation fails with this error, shorten the lookback window with `agents-cli aqua attach --lookback-days`, or store a selector that reads fewer conversations (see Changes).
- **`agents-cli deploy` samples every trace of the observed agent.** It passes `OTEL_TRACES_SAMPLER=always_on` to your agent, merged into your `--update-env-vars`. Agent Runtime forwards some requests, such as A2A messages, unsampled, and the default sampler then drops every span they produce, so AQuA could not review those turns.
  - **Upgrade.** To keep your own sampler, set `OTEL_TRACES_SAMPLER` in `--update-env-vars` or in the project's `.env`. `agents-cli deploy` then leaves it as it is.
- **Dashboard trajectory pages moved.** A trajectory page is at `/investigations/<investigation id>/trajectories/<trajectory id>`. Links to `/investigations/<id>/cases/<id>` from 0.2.0 do not open.
  - **Upgrade.** In saved links, replace `/cases/` with `/trajectories/`.

### Changes

- **Stored investigation selector.** An attached agent can store an SQL selector. Scheduled, update-triggered and manual investigations then review only the conversations it selects, in place of a random sample. A custom investigation keeps its own selector.
  - Set it with `agents-cli aqua attach --selector-file PATH`. `-` reads standard input, and an empty file clears the selector.
  - The selector returns `target_id`, uses `@window_start`, `@window_end` and `@agent_name`, and reads only the telemetry table. It needs the `session_review` quality analysis mode. Each investigation checks it before its queries.
  - To change the telemetry table of an agent that has a stored selector, set or clear the selector in the same `attach`.
- **Dashboard: Investigation selector card.** The Configuration page shows the stored selector, or says that investigations review a random sample. You can check, save and clear the selector there. A save needs a passing check, which is a dry run that reads no rows. "Add selector" opens the editor on the query that the random sample uses, to narrow with more conditions.
- **Custom investigations over views.** A custom investigation's selector can read an attached logical view, as long as it names the view and not the tables behind it. A selector that calls a routine other than a built-in function or a routine of the view is refused as `unauthorized_routine`. A selector that reads a table AQuA cannot read is refused as `unauthorized_table`, and one whose selector table AQuA cannot read as `selector_table_unreadable`.
- **Progress of a running investigation.** While an investigation is pending or running, `get-investigation` and `list-investigations` show its `progress`: the stage it is in and its counts so far.
- **Dashboard: insight detail page.** The page starts with a summary: what's wrong, the root cause, a suggested fix and one example finding. The latest occurrence with its trajectories and the proposed code changes follow. Earlier occurrences are folded under History. History and "Show N more findings" stay open or closed as you last left them.
- **Coding-agent skill.** The `agents-cli-aqua` skill asks how to proceed before it edits anything: one insight at a time (recommended), or one group of insights that share a root cause. It asks again before the next insight or group. It adds a unit test for a code defect and an eval case for a prompt defect. When `agents-cli aqua info --ui-url` returns a URL, it links insights and investigations to the dashboard.
- **Telemetry name checks.** `attach` refuses a telemetry dataset or table name that BigQuery's naming rules do not allow.
- **`make demo`** prints a link that opens the guided tour, and the dashboard shows a "Demo data" badge.
- **Dependencies.** AQuA needs google-adk 2.13.0 or later, and depends on `anthropic[vertex]`, `jq` and `aiohttp`. Dependencies with known vulnerabilities moved to fixed versions, among them aiohttp, certifi, filelock, OpenTelemetry, vite, happy-dom and zod.

### Fixes

- Custom investigations refused every selector over the Cloud Trace `_AllSpans` view, the `cloud_ops` default that `attach` picks. They accept them now.
- Ingestion works on telemetry tables that require a partition filter. It reads a selected session's rows from 7 days before to 7 days after the window, so a longer session is read only within that range.
- With the `big_query` source, session review read the BigQuery Agent Analytics plugin's `text: '...'` wrapper as part of the agent's reply, and raised a false insight on every such agent. AQuA now unwraps a reply made only of text parts.
- `attach` refuses a telemetry table that it finds missing or in the wrong format after the grants, as it does before them. It revokes the grants that the run made, removes a first attachment and restores the settings of a re-pointed one. Before, it exited 0, and every later investigation failed.
- `attach` and `detach` no longer print each resource's whole IAM policy after a grant.
- The engine and dashboard images build on Python 3.13, inside the supported range. They were on Python 3.15.
- The guided tour brings its step back into view after a trackpad scroll.

### Known issues

- `agents-cli aqua run` always talks about the default agent. Use the dashboard chat, which follows the selected agent.
- Some progress messages can be lost when investigations overlap. The investigation's status, insights and trajectories are not affected. AQuA logs the warning `could not record event row`.

Source-Revision: fbd66349271b15582cf787dc78d5cd59aec465a0
@ale-grassi
ale-grassi requested review from a team and maxgasztych as code owners November 4, 2026 10:09
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

This PR changes 355 files, above the 300 GitHub's diff API will serve, so automated AI review was skipped. Splitting it up would get each part reviewed.

@ale-grassi
ale-grassi merged commit dc11493 into main Nov 4, 2026
50 checks passed
@ale-grassi
ale-grassi deleted the ambient-quality-agent/release-v0.3.0 branch November 4, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants