Skip to content

docs+config: surface the two env-variable expansion syntaxes (#2615)#2851

Merged
dgageot merged 4 commits into
docker:mainfrom
dgageot:docs/2615-variable-expansion
May 21, 2026
Merged

docs+config: surface the two env-variable expansion syntaxes (#2615)#2851
dgageot merged 4 commits into
docker:mainfrom
dgageot:docs/2615-variable-expansion

Conversation

@dgageot
Copy link
Copy Markdown
Member

@dgageot dgageot commented May 21, 2026

Stop-gap for #2615 — surfaces the silently-failing variable-expansion mismatch in agent YAML without changing runtime behavior.

Background

Agent YAML uses two incompatible expansion syntaxes:

  • JS template literals${env.X} (with || defaults, ternaries, tool calls). Used in description, welcome_message, instruction, commands, headers, and api_config. Backed by pkg/js.
  • Shell-style$VAR, ${VAR}, ~. Used in path-like fields (working_dir, toolset path) and toolset/hook env values. Backed by os.Expand / pkg/path.

Mixing them up (e.g. ${USER} in a description, or ${env.HOME} in a working_dir) is a silent no-op: the literal string is passed through. This bites users every few weeks.

PR #2710 attempts a runtime unification but is blocked on design (reflection-based walker, breaking Source interface). This PR takes the smaller steps the issue explicitly invited:

  1. Document the rules.
  2. Warn at config-load time so the silent failure becomes visible.

Runtime behavior is unchanged.

What's in this PR

Documentation

docs/configuration/overview/index.md gets a new "Variable Expansion in Config Fields" section with a quick-reference table:

Field ${env.X} $X / ${X} ~
description, welcome_message
instruction (agent + toolset)
commands.*
headers, remote.headers
api_config.endpoint/headers
working_dir, path
env values (toolset/hook)

Cross-links added from docs/configuration/agents/index.md and docs/configuration/tools/index.md.

Warnings at config-load

pkg/config/expansion_warnings.go adds warnExpansionMismatches, called at the end of config.Load. It walks every agent + toolset + hook and emits a single slog.Warn per offending occurrence with location, field, variable, and a link to #2615.

Two cases:

  • Shell-style in JS field: ${IDENT} in description, welcome_message, instruction, commands, headers (headers, remote.headers, api_config.headers), api_config.endpoint, api_config.instruction. Suppressed when the same value also contains a real ${env.X} so legitimate JS templates with non-env interpolations don't false-positive.
  • JS-style in shell field: ${env.X} in working_dir, toolset path, toolset env, script shell working_dir/env, all hook working_dir/env.

The warning logs the variable name and location only — never the full field value, since path:/env: values often contain credentials.

Tests

10 table tests in pkg/config/expansion_warnings_test.go cover both directions, false-positive avoidance, secret non-leakage, lowercase identifiers, mixed JS+shell headers, APIConfig, ScriptShellToolConfig, and hooks. All run with t.Parallel() because the logger is dependency-injected (no global slog.Default() mutation).

Validation

  • task build
  • task test ✓ (all packages pass with -race)
  • task lint ✓ (0 issues, custom lint clean, go.mod tidy)

Commits

docs: clarify which env-variable expansion syntax applies per field
feat(config): warn on mismatched env-variable expansion syntax
docs: correct env-value expansion syntax
fix(config): harden expansion mismatch warnings

The two fix:/docs: follow-ups came out of self-review; happy to squash on merge.

Out of scope

Closes part of #2615 (documentation + visibility); leaves it open for the runtime unification.

docker-agent and others added 4 commits May 21, 2026 11:52
Two incompatible expansion syntaxes coexist in agent YAML today and the
mismatch fails silently at runtime:

  - JS template literals (`${env.X}`) for prompts, instructions,
    commands, headers and toolset `env` values.
  - Shell-style (`$VAR`, `${VAR}`, `~`) for path fields
    (`working_dir`, toolset `path`).

Add a 'Variable Expansion in Config Fields' section to the configuration
overview with a quick-reference table, and cross-link from the agents
and tools pages so users can find the supported syntax for each field.

This is the documentation stop-gap requested in docker#2615; runtime
unification is tracked separately.
Two incompatible expansion syntaxes coexist in agent YAML
(see docker#2615): JS template literals (`${env.X}`) for prompt-like
fields, and shell-style (`$VAR`/`${VAR}`/`~`) for path
fields. When the wrong syntax is used in a field, expansion
silently no-ops and the literal string is passed through.

Emit a slog warning at config-load time when:

  - a JS-expanded field (description, welcome_message, instruction,
    commands, headers, env values) contains a shell-style
    `${UPPER_CASE}` reference and no `${env.X}` is present;
  - a path field (toolset working_dir, path) contains `${env.X}`.

Warnings only — runtime behavior is unchanged. This makes the
silent no-op visible while a proper unification is designed
(docker#2710).
The 'Variable Expansion in Config Fields' section listed toolset env
values under the JS-template syntax, but the runtime expands them with
os.Expand (shell-style $VAR / ${VAR}). Move env to the shell-style
section, add the api_config.endpoint / api_config.headers entry to the
JS-template list, and adjust the quick-reference table accordingly.

Refs: docker#2615
Address several issues in the warnExpansionMismatches helper:

* Avoid leaking field values: warnPathField previously logged the full
  field contents, which can include credentials embedded in path or env
  strings. Log only the offending variable name.
* Match lower-case shell identifiers: os.Expand accepts ${user},
  ${db_path}, etc., but the regex was upper-case-only and let those
  silent failures slip through.
* Cover env values in shell-expanded toolsets, ScriptShellToolConfig
  (per-script working_dir and env), api_config.endpoint /
  api_config.headers / api_config.instruction on api toolsets, and hook
  working_dir / env on every HooksConfig list (including matcher-based
  PreToolUse, PostToolUse, PermissionRequest, ToolResponseTransform).
* Stop racing with slog.Default(): the helper now takes a *slog.Logger
  argument, which Load passes from slog.Default() and tests instantiate
  per-test. The test suite no longer mutates the global default logger,
  so it is safe to mark warning tests t.Parallel() alongside the rest of
  the package.

Refs: docker#2615
@dgageot dgageot requested a review from a team as a code owner May 21, 2026 11:06
Copy link
Copy Markdown

@docker-agent docker-agent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟢 APPROVE

The expansion-warning subsystem is well-structured with good test coverage. All medium-severity hypotheses were dismissed: types.Command fields are fully covered (Instruction is set for both simple-string and struct-format commands), and the broad suppression in warnJSField is an intentional documented design tradeoff. One low-severity inconsistency was found between the fast-path guard and the regex in warnPathField.

// does not recognize. The variable name is captured but the surrounding
// value is not, since path/env values frequently contain credentials.
func warnPathField(ctx context.Context, logger *slog.Logger, loc, field, value string) {
if value == "" || !strings.Contains(value, "${env.") {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[LOW] warnPathField fast-path guard misses whitespace variants like ${ env.X}

The fast-path early return uses strings.Contains(value, "${env.") which does not match ${ env.X } (whitespace after ${). However, the jsEnvRefStrict regex (\$\{\s*env\.) does allow leading whitespace inside ${}, and JavaScript template literals also permit this (${ env.HOME } is valid JS that the evaluator would expand). A user who writes ${ env.HOME}/work in a working_dir field would get no warning despite the JS evaluator being able to process it.

Suggested fix: widen the fast-path guard to check for "${" instead of the full "${env.", or remove it and rely solely on the regex:

// Before
if value == "" || !strings.Contains(value, "${env.") {

// After
if value == "" || !strings.Contains(value, "${") {

This is a minor gap — spaces in ${env.X} are rare in practice — but the inconsistency between the guard and the regex is a latent correctness issue.

@dgageot dgageot merged commit e00854a into docker:main May 21, 2026
9 checks passed
@aheritier aheritier added area/config For configuration parsing, YAML, environment variables kind/docs Documentation-only changes labels May 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config For configuration parsing, YAML, environment variables kind/docs Documentation-only changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants