reference cli

CLI reference

Active contributors: Zachary Bensalem

The fleet executable is defined by main() in src/fleet_rlm/cli.py. It has five modes: the interactive client, the API server, evaluation dataset and run management, GEPA optimization, and trace verification. The interactive client also accepts slash commands handled locally by the terminal client.

Key source files

Path	Responsibility
src/fleet_rlm/cli.py	Argument parser, command dispatch, --mlflow resolution, and validation errors.
src/fleet_rlm/optimize/gepa.py	fleet optimize arguments and execution.
src/fleet_rlm/tui/commands.py	Slash-command table, handlers, completion options, and /help text.
src/fleet_rlm/launcher.py	Service planning, health checks, and startup/shutdown for fleet cli.
docs/reference/cli.md	Maintained command reference.

Top-level modes

fleet cli [databricks] [--mlflow local|databricks] [--session SESSION_ID] [--url URL]
          [--mlflow-port PORT] [--mlflow-experiment NAME] [--startup-timeout SECONDS]
fleet serve [--host HOST] [--port PORT] [--mlflow local|databricks]
fleet eval dataset [--name NAME] [--seed N] [--long-context] [--oolong N] [--longcot N]
fleet eval run [--url URL] [--seed N] [--limit N] [--oolong N] [--collect-only]
fleet eval label TRACE_ID [--status STATUS] [--group GROUP] [--answer TEXT]
                [--expectations-file PATH] [--value V] [--route ROUTE]
                [--critical|--no-critical] [--reviewer NAME]
fleet optimize --session SESSION_ID [--gepa-budget light|medium|heavy | --max-metric-calls N]
               [--threads N] [--validation-ratio F] [--approve VERSION]
fleet trace verify TRACE_ID [--scope runtime|evaluation|optimization]

Every subcommand accepts --mlflow local|databricks; the default is local.

Mode	Behavior
cli	Starts the API and, in local mode, a managed MLflow server; waits for readiness, verifies the API tracking destination, then runs the terminal client. Services it owns are stopped when the client exits.
serve	Starts only the FastAPI server with uvicorn and no reload. Refuses a non-loopback host unless FLEET_ALLOW_NON_LOOPBACK_BIND=true.
eval dataset	Builds the synthetic evaluation dataset in the selected evaluation experiment.
eval run	Runs synthetic cases against a live server and scores them with the configured judge.
eval label	Writes gold labels onto a saved trace.
optimize	Scores approved examples and compiles a GEPA candidate, or installs a previously validated candidate with --approve.
trace verify	Flushes pending exports and reads one trace back from the configured destination.

fleet cli

• databricks is a positional shorthand for --mlflow databricks. Only databricks is accepted as a positional backend.
• --url connects to an existing server and starts nothing. The launcher still checks that its MLflow backend, tracking URI, experiments, and trace location match the local settings.
• --session sets the session ID; when omitted, a short sess_<8 hex> identifier is generated.
• --mlflow-port overrides the local MLflow port and is rejected unless the backend is local.
• --mlflow-experiment overrides the runtime experiment of the selected backend.
• --startup-timeout defaults to 180 seconds per service.
• A healthy API already on the port is reused only when its destination metadata matches. Unknown metadata, unready tracing, or a mismatch aborts the connection.
• Startup prints the selected backend, tracking URI, and experiment name.

fleet serve

--host and --port override host and port for this process. Binding a non-loopback address requires FLEET_ALLOW_NON_LOOPBACK_BIND=true; the API has no caller authentication.

fleet eval

eval run defaults --url to http://<settings.host>:<settings.port> and preflights the API destination with launcher.verify_api_destination before creating work. It requires the selected evaluation experiment; a missing name is an error. --collect-only runs and labels Turns without invoking the judge suite. --limit 0 means all cases. --oolong N replaces the synthetic case set with N Oolong-synth cases.

eval dataset names the dataset from --name, or derives <catalog>.<schema>.fleet_rlm_eval_cases in Databricks mode and fleet_rlm_eval_cases locally. --long-context adds long log cases; --oolong and --longcot fetch benchmark cases over HTTP.

eval label behavior:

• --status accepts the review statuses from src/fleet_rlm/replay.py; a new label defaults to approved, and updates preserve existing status and source group unless supplied.
• --group sets the source group used for train/dev/holdout splits; a new group defaults to the trace's session ID.
• --value is repeatable and adds expected values that must appear in the answer.
• --route accepts direct, flex, or rlm.
• --critical pins the deterministic example to the GEPA holdout and requires a perfect candidate score; --no-critical clears it.
• --expectations-file replaces structured artifact and delegation gold with a JSON object.
• --reviewer defaults to human and must be non-blank.
• The command retries trace reads for up to 15 seconds, validates labels before writing, and requires a matching replay snapshot when the trace was selected for replay. MLflow labels are written first; a later Lakebase curation failure reports a retry instruction rather than claiming success.

fleet trace verify

TRACE_ID is the ID from a completed Turn or /trace; in Databricks mode use the fully qualified trace:/... reference. --scope selects the runtime, evaluation, or optimization experiment and its trace table prefix. In Databricks mode the command also confirms that the experiment is bound to the configured Unity Catalog location and that the stored trace carries the same location. A trace with zero spans is an error. This is an explicit operator read, not part of serving.

fleet optimize

--session is required. --gepa-budget accepts light, medium, or heavy and defaults to light; it is mutually exclusive with --max-metric-calls N. --threads defaults to 4 and must be between 1 and 32. --validation-ratio defaults to 0.35. --approve VERSION installs a candidate that was evaluated against the current incumbent and configuration; search flags are not used for an approval. The search budget does not cover reflection calls or the three paired validation passes.

optimize is a live operation. It needs a curated dataset of at least 8 approved examples from 3 source groups, immutable replay snapshots, and the configured model, Daytona, database, and tracking services. Candidate creation and installation are separate: a normal run never installs a candidate.

--mlflow rules

All commands default to local MLflow. The backend is an explicit command choice, never an environment default.

• fleet cli databricks and --mlflow databricks are equivalent. Combining databricks with --mlflow local is an argument error.
• --mlflow env is retired and is not an accepted choice.
• Environment variables override values inside the selected backend but cannot select it.
• --mlflow-port is valid only for local.
• [mlflow.local] and [mlflow.databricks] are independent; the resolver never borrows a value from the other profile.
• Standalone serve, eval, optimize, and trace commands connect to an existing tracking server. In local mode eval, optimize, and trace require a healthy MLflow at the configured URI; the exception is optimize --approve, which skips that health check. serve skips the CLI health check, but a tracking initialization failure still prevents API readiness.
• Conflicting or invalid configuration is reported as invalid configuration: <details> before dispatch.

Local MLflow can be started by fleet cli or separately:

mkdir -p .fleet/mlflow
uv run mlflow server --host 127.0.0.1 --port 5001 \
  --backend-store-uri sqlite:///$PWD/.fleet/mlflow/mlflow.db \
  --default-artifact-root "$PWD/.fleet/mlflow/artifacts"

Slash commands

Slash commands are parsed by handle_command in src/fleet_rlm/tui/commands.py and executed against the HTTP API for the current Session. A line without a leading slash is submitted as a task. Unknown commands print a hint; /help lists every command by category; /exit and /quit end the client.

Workspace and artifacts

Command	Purpose
/attach <path>	Upload a local file to the Session workspace.
/files	List workspace files.
/artifacts	List generated artifacts.
/tree	Show the workspace tree.
/cat <path>	Read a workspace file.
/stat <path>	Show size and SHA-256 for a workspace file.
/put <local-path> [workspace-path]	Upload a file to a chosen workspace path.
/get <workspace-path> [local-path]	Download a workspace file.
/rm <path>	Delete a workspace file.

Context and execution

Command	Purpose
/context <workspace-path>|clear	Select an input dataset file or clear the selection.
/mode [rlm|flex|hybrid]	Inspect or change the execution mode for this Session.

Models and configuration

Command	Purpose
/config [mode <mode>]	Inspect non-secret Session settings, limits, digest, and active overrides. /config json prints the raw payload.
/settings	Alias for /config.
/models [catalog.schema]	Discover accessible chat-capable Unity Gateway services; defaults to system.ai.
/model [root|sub] [model-id|catalog.schema]	Choose a model interactively or set it explicitly.

Root and sub model choices are validated by the server, isolated to the Session, applied at Turn boundaries, and persisted. A busy Session rejects model changes.

Sessions and memory

Command	Purpose
/sessions	List saved Sessions with turn counts and update times.
/memory [query]	Show saved memories, optionally filtered by query.

Skills

Command	Purpose
/skills [pull|push]	Show the Session Skill catalog; pull exports to .fleet/skills/, push synchronizes local Skills.

Traces and optimization

Command	Purpose
/history	Show saved Turns.
/trace	Show the most recent persisted Turn trace ID, if one exists.
/optimize [--max-metric-calls N | --approve VERSION]	Score past Turns, compile a GEPA candidate, and gate it; --approve serves it from the next Turn.

Replay and Turn inspection

Command	Purpose
/replay [on|off]	Enable or disable bounded replay snapshot capture for subsequent Turns in this client process. Off by default.
/turn [turn-id]	Inspect durable status and outcome for the latest or named Turn.

/optimize inheritance and --approve

/optimize runs a subprocess equivalent to:

python -m fleet_rlm.cli optimize --session <current-session-id> <extra arguments> --mlflow <session-backend>

It inherits the Session's MLflow backend and effective profile settings. Any --mlflow value in the extra arguments must equal the Session backend; a conflicting value is rejected before the subprocess starts. MLFLOW_TRACKING_URI, MLFLOW_EXPERIMENT_NAME, and MLFLOW_EXPERIMENT_ID are removed from the child environment, and the selected profile's FLEET_MLFLOW_* values are added.

In Textual fullscreen mode the command captures output into an "Optimization run" panel; in the Rich fallback it streams to the terminal.

/optimize --approve <version> installs one candidate. Approval rechecks the candidate artifact against the current incumbent, configuration, code, and dependency identities. Old manifests require reevaluation. A candidate installed this way is loaded by the engine at the start of the next Turn. Rollback is not an unchecked path. For the candidate lifecycle, evidence, and holdout gates, see Optimization.

Argument and runtime errors

The parser reports configuration problems before dispatch, including pydantic validation details with field paths. Runtime failures use SystemExit with a message rather than a traceback for known conditions:

• Local MLflow is unreachable for a standalone command.
• The API destination does not match the selected evaluation destination.
• A trace cannot be read back or has no spans.
• The Databricks experiment trace location does not match configuration.
• The selected evaluation experiment name is empty.
• A non-loopback bind is requested without the opt-in.
• Service startup times out or a service exits.

Ctrl-C during fleet cli startup or operation exits with status 130 after cleanup. A lost client connection is not proof that a Turn was rolled back; check /history before retrying.

Related

• Configuration for the settings behind every flag and default.
• Getting started for the first local session.
• Deployment for server and service operation.
• Security for bind and credential boundaries.
• Optimization for GEPA internals and approval rules.
• Terminal client for rendering and interaction behavior.