console — design

What this is

The ModelSphere community portal. A full-stack BFF that owns identity and federates everything else. It exists because the open-source inference stack (swiss and friends) has no user management, login, or unified entry point.

Why a separate service (not part of swiss)

swiss is a deploy control plane: near-stateless, "the cluster is the only source of truth." Identity is the opposite — users, roles and sessions are data the portal owns. Keeping them apart lets swiss stay a backend and lets the portal be the front door for many backends, mirroring Rise Global's apiserver/console split.

Wire-compatibility with Rise Global (the upgrade path)

The identity layer is deliberately a subset of Global's, kept compatible so a community install can be upgraded to the commercial Global without a migration:

Surface	Kept identical to Global
User/role data	`iam.theriseunion.io/v1alpha1` User, IAMRole, IAMRoleBinding CRDs
Tokens	OAuth2 password grant, HS256 JWT, same claim set
Authorization	K8s-RBAC-shaped, `system:masters` short-circuit, platform scope only

Upgrade = share the JWT secret and point the identity endpoints at Global's apiserver. The console frontend contract does not change.

Decision (2026-09-26): the API group stays iam.theriseunion.io. A separate group (iam.modelsphere.dev was considered) would isolate the two products but turn the upgrade into a data migration. Consequences of sharing it:

On a cluster where Global runs (e.g. a member cluster), console and Global read and write the same users, roles and bindings. That is the upgraded state, not a conflict; the helm chart leaves existing CRDs untouched.
Those CRDs hold roles for every Global scope (platform, workspace, cluster, namespace, nodegroup) side by side. Console guards only platform-level things, so its authorizer honours only bindings labelled scope=platform, scope-value=global -- Global's own selector at that level. Anything else would let a namespace admin act as a console admin.
The seeded admin matches Global's (group system:masters, same default hash).

Running without a login (`server.auth.disabled`)

A laptop, a demo, or a single-tenant cluster that is already behind something else does not want a login page. Set it and console stops issuing and verifying tokens; every request carries a synthetic admin in system:masters, which is the group the authorizer already short-circuits on.

Decision	Why
A synthetic identity, not a User CRD	nothing is read, so the mode works on a cluster where the iam CRDs were never installed
Reuse `system:masters` rather than bypass the authorizer	the RBAC path stays wired up and simply always answers yes; there is no second code path to keep correct
`jwtSecret` becomes optional	nothing is signed. It is still generated by the chart, so turning the login back on is one value
The password-reset guard is skipped	the seeded admin still carries the annotation; with nobody logging in there is no password to change, and the guard would lock the API behind a screen nothing can satisfy
`/api/me` reports `authDisabled`	the shell skips the login page and hides logout and change-password, which it cannot honour
The admin User is still seeded	dropping it from the manifest would have helm delete it, so flipping the flag twice would lose an operator-chosen password

Nothing in the stack authorises anything in this mode: swissd behind console already runs with its own auth off and trusts console's RBAC. It is logged as a warning at startup and called out in the chart's NOTES.

Delegating login to a backend's own auth (swissd's file-based credentials) is deliberately not built. It would be a third authentication path for the one case that wants a password without the CRDs, and the two modes here cover the installs that exist.

Architecture

browser ─▶ web (Vite / React 19 / Tailwind 4 / @modelsphere/ui)
             shell: login, layout, sidebar, route guards
             modules: iam, swiss, playground, router, … (src/modules/*)
                     │  same-origin /oauth, /api/iam, /api/deploy, /api/llm…
                     ▼
             console (Go BFF)
               internal/iam      identity kernel: CRD types + OAuth2 (HS256) + RBAC
               internal/gateway  resolves the inference entrypoint from the cluster
               internal/server   mux + auth middleware + static SPA
                                 + backend proxy (proxy.go)
                     │
                     ▼  HTTP: X-Remote-User / X-Remote-Group + Bearer JWT,
                        or the backend's own key (apiKeyEnv, or the gateway's
                        Secret read at runtime)
             swissd, llm-openresty, and other backends

Users and roles are read/written as CRDs via the dynamic client — no scheme, no generated clients, one less thing to keep in sync with Global.

The web stack pins the same versions as swiss/web, so swiss pages compile here unchanged. Moving either one moves both.

Modules

The console is a shell plus compile-time modules. A module is a feature area (iam, inferences, playground, router, docs, later container management); the shell owns everything around it.

Owned by	What
shell (`web/src/shell/`)	login, password reset, layout, sidebar, route guards, query client, session on every request
module (`web/src/modules/<id>/`)	its pages, its API client, its overview cards
`web/src/modules/index.ts`	which modules are installed, in sidebar order — plug in/out = one line

A module exports one declaration (ConsoleModule, web/src/shell/module.tsx):

export const inferencesModule: ConsoleModule = {
  id: "inferences",
  title: "模型服务",            // sidebar group
  basePath: "/inferences",     // every page mounts below it
  overview: InferencesOverview, // optional cards on the home page
  pages: [
    { path: "", element: <InferenceList />, permission: "swiss.view", menu: { label: "推理服务", icon: Bot } },
    { path: "catalog/:name", element: <ModelPage />, permission: "swiss.view" },
  ],
};

Rules:

A module imports from @/shell only, never from a file under it or from another module. One exception: inferences builds on swiss (see "The inferences module").
permission is a UI permission (a Role's uiPermissions); it guards the route and hides the menu entry.
topbar: true adds the page's menu entry to the header; topbar: { label: "模型广场" } gives it a separate header label without changing the sidebar or route.
The first home overview receives shell-owned session content through ModuleOverviewProps.session; it places this below cluster information, or renders it alone when inference viewing is forbidden. Without overview modules, Home renders the session itself.
Links go through useModulePath(): p("catalog") → /inferences/catalog. A leading / is still relative to the module.
API calls go through apiFetch (drop-in fetch with the session) or request<T> (JSON helper).
The sidebar opens the group of the module the current path is mounted under (activeGroupTitle), so a page with no menu entry -- a detail page -- still keeps its module's group open.
validateModules rejects duplicate ids/basePaths, shell-reserved paths and ids (common, shell: they are i18n namespaces) and pages declared twice at startup.
Text goes through useT() (see "i18n"); the declaration's title and menu labels stay Chinese.

Deliberately not built: runtime loading, micro-frontends, cross-module slots. The earlier Cordis design (branch docs/console-plugin-design) is parked; revisit it when one of those is actually needed.

i18n

Rise Global's engine and model (console/src/i18n/engine.ts, ADR-0011), cut to this console: zh-CN and en-US, switchable at runtime.

Concern	Decision
Engine	one i18next instance with ICU messages (`{name}`, `{n, plural, …}`), `web/src/shell/i18n.ts`, exported through `@/shell`
Choice	`localStorage["rise-locale"]` -- Global's key, so it carries over on upgrade; else the browser's language; else zh-CN. Mirrored to `<html lang>`, followed across tabs
Switch	top-bar language menu, the preferences panel, the login page; one state behind all three
Namespaces	`common` (shared words), `shell`, `ui` (`@modelsphere/ui`'s component strings, see "UI library"), and one per module = its `id`
A module's strings	`modules/<id>/locales/{zh-CN,en-US}.json`, registered by `modules/<id>/i18n.ts`, which also exports the module's `useT()`
Component text	key-based: `t("users.title")`; another namespace with a prefix, `t("common:actions.cancel")`; a node inside a message with `tNodes`
Non-React code	`getT(ns)(key)` at call time, never at import time: the locale changes under it
Dates, numbers	`formatDateTime` / `formatNumber` follow the locale
Missing key	falls back to zh-CN

Sidebar labels are the contract layer, as in Global: the declaration keeps its Chinese literal, which is the zh-CN text, and other languages look it up by a derived key, falling back to the literal:

Label	Key
module `title`	`{id}:group.{title}`
page `menu.label`	`{id}:menu.{pageKey(path)}`: `""` → `index`, `login-history` → `loginHistory`
page `topbar.label`	`{id}:menu.{pageKey(path)}Topbar`; Chinese uses the declared label

So group/menu appear only in the non-Chinese files. t() returns one function per (locale, namespace): a fresh one per render would re-run every effect keyed on it (Global's request loop).

modules/swiss is a copy of swiss/web and is not edited, so the swiss panels Model Serving embeds stay Chinese until swiss itself takes i18n.

web/src/shell/i18n.test.tsx checks, for every locales/*.json: the same keys in each language, valid ICU, no {{…}}, no group/menu in zh-CN; and an English label for every installed module's group and menu entry.

Adding a language: append it to LOCALES, add *.<locale>.json beside each zh-CN file (the parity test lists what is missing).

UI library (`@modelsphere/ui`)

web/packages/ui: the open UI kit, a fork of the parts of Rise Global's design system (rise-global/design@684df9fc) the console uses. Open-sourcing approved; Rise Global keeps its own, closed library -- two libraries, no syncing.

Concern	Decision
Contents	shadcn (base-nova) on Base UI; `ResourceTable`, `PageBanner`, `ConfirmDialog`, `DataSelect`, `Spinner`, …; tokens as a Tailwind preset; `base.css`
Delivery	source, not built: `"@modelsphere/ui": "file:./packages/ui"`, compiled by Vite and checked by the console's `tsc`; Tailwind scans `packages/ui/src`
Where	inside console while it has one consumer. It imports nothing from the console, so moving it to its own repository (`git subtree split`) is mechanical; do that when swiss or another ModelSphere UI adopts it
Component strings	the kit's own `ui` namespace (`src/locales/ui.*.ts`); the shell registers it and plugs its i18next instance in through `configureUiI18n` (`shell/i18n.ts`). Without that the kit renders Chinese
Look	Rise Global's current tokens, including its neutral (near-black) default primary
Brand	the Spinner's logo variants draw a neutral mark, not the Rise logo
Usage rules	the kit's `README.md`; Global's `PATTERNS.md` is where they came from

The swiss module's two UI kits

The swiss pages are a copy of swiss/web/src (see "Bringing swiss in"), so they must stay editable only upstream. That rules out rewriting 56 Badge and 33 Field call sites onto the console's kit: the next sync would undo it.

So the seam is the import path, not the call site. The pages import @swiss/components/ui/*; an alias resolves it to one of two kits with the same export names and prop shapes.

Kit	Comes from	Picked by
`components/ui/shadcn/`	swiss, verbatim	`UI_KIT=shadcn` / `tsconfig.shadcn.json`
`components/ui/rise/`	console, adapters over `@modelsphere/ui`	the default

The directory keeps its name: it is the kit in Rise Global's look, now drawn by @modelsphere/ui. Adapting rather than re-exporting is not uniform:

Adapter	Over the kit
`button`, `card`, `table`, `Input`	same shadcn lineage: re-exported
`Dialog`	`{open,onClose,title,subtitle,footer,size}` over the Base UI compound, re-laid-out as fixed header, scrolling body, pinned footer; `sm:max-w-*` as well, or the kit's `sm:max-w-sm` wins
`Tabs`, `Switch`, `HoverHint`	swiss's props (`onSelect`, `onChange`/`label`, `text`) onto Base UI's (`onValueChange`, `onCheckedChange` + `aria-label`, `render`)
`Badge`	the kit's, which renders a `<span>` and has `success`/`warning`; only swiss's `muted` is local, on `secondary`
`code`, `toast`	no kit equivalent: the shadcn ones, re-exported

The shell is Rise in both builds. The flag swaps the module, not console's chrome, and is there for the case where a Rise component reads wrong on a deploy page -- the tables are the likely one, since Rise's TableHead/TableCell are h-12 px-4 against swiss's h-10 px-3.

The inferences module

/inferences (Model Serving) is every swiss page redone in Rise Global's model service layout. It replaced the swiss module's own pages, Model Deployment at /swiss, which the console no longer mounts: modules/swiss is a library here (API client, form model, panels) and the source of the standalone swiss UI (build:swiss).

Page	Path	Layout
Home dashboard	`/`	six capability/resource totals; primary 24-hour deployment curve beside capacity gauges, followed by the live service table and a two-column resources/activity row; cluster information and collapsed personal session in the right rail
Inference services	`/inferences`	resource table, server-side paging, row menu
Service detail	`/inferences/:release/details?namespace=…&tab=…`	header card over tabs: overview, instances, logs, events, cluster resources, health check, versions, activity, plan; SLO edited in a sheet from the header
Model library	`/inferences/catalog?catalog=…&q=…`	card grid, facet filters in the URL (swiss's parameter names)
Model detail	`/inferences/catalog/:name?catalog=…&version=…`	header card over variants and information tabs; add, edit and delete variants on a writable catalog (see "Catalog writes")
Nodes	`/inferences/nodes`	GPU KPIs, a card per GPU product, node table with pods, taints and conditions on expand
Activity	`/inferences/runs?action=…&namespace=…&release=…`	every run, output on expand
Site profile	`/inferences/site-profile`	grouped read view; edit in a sheet, form or YAML
Setup	`/inferences/setup`	first run; swiss's gate sends an uninitialised site here

The home overview is gated by swiss.view and does not run SwissScope, so an uninitialised backend cannot redirect the whole console home to setup.

Dashboard data	Contract
Service total	server summary, not the six-row preview count
Capability totals	all configured catalogs; non-deprecated models deduplicated by model/source/revision identity, recipes from all listed versions deduplicated by that identity/version/digest/variant id; any unreadable catalog keeps totals unknown
Deployment curve	last 24 hours of completed audit operations by `startedAt`; changed successful installs are starts, changed successful apply/rollback are updates, failed install/apply/rollback are failures; not a history of Pods serving
Timeline completeness	current Console reads only the published `/api/runs?limit=500` endpoint and aggregates on the client; reaching that cap is incomplete and never shown as a complete zero curve; no database remains unknown. It does not call the unmerged `/api/runs/timeline` prototype. Complete server-side aggregation and serving metrics require separately reviewed backend work before Console integration
Cluster sidebar	name, namespace, deployment mode and swissd version from cluster info; distinct Kubernetes node versions from `Node.Kubelet`, never from the swissd version; ready/schedulable counts from node inventory
Service state and instances	live release status for each preview row; a deployed Helm release alone is not serving
Allocated GPUs	pod resource allocation, not GPU utilisation; `usageError` means unknown
Resource charts	node allocation bars and GPU allocation/node online gauges from the existing nodes snapshot; unavailable readings stay unknown, with no synthetic GPU-utilisation or token/request history
Refresh	services/status every 15 s, activity every 20 s, nodes every 30 s; no overview-wide refresh control, with local retries on read failures
Read failures	inline error and retry, never a successful empty state
Service preview	standard `ResourceTable` surface and title header; `pagination={false}` renders all supplied rows and `emptySize="compact"` bounds the empty state
Activity preview	five recent operations in a compact `SectionCard` feed, with timestamps, result and detail links

Writes are right-side sheets, as Rise Global's are:

deploy / upgrade sheet:  form (anchored, collapsible sections) -> compose -> dry run -> review (what moves, diff, plan, note, force) -> install | apply
rollback sheet:          diff of the revision -> note -> rollback(expectRevision)
SLO sheet:               swiss's SLO form (save, reset)
uninstall:               ConfirmDialog, typed to confirm
variant sheet:           file YAML (add: a copy of the default variant) -> check (dryRun) -> commit (If-Match | If-None-Match: *)
delete variant:          ConfirmDialog, typed to confirm; If-Match read at confirm time

Part	Where it comes from
pages, sheets, forms	`web/src/modules/inferences/`, on `@modelsphere/ui`
state, tabs, install track, revision/run join, catalog filters and fit, node aggregation, pipeline state, chart version choice, service-id rule	`lib.ts`, `catalog-lib.ts`, `nodes-lib.ts`, `deploy-lib.ts`, `profile-lib.ts`; pure and tested
API client, types, form model (`Form`, `planRequest`, `formFromPlan`), catalog and GPU helpers, route directives	swiss: `@swiss/lib/*`, `@swiss/components/DeploySettings` (its pure exports)
cluster resources, health check, SLO, plan, diff panels	swiss's components (`ReleaseObjects`, `Endpoint`, `SLOCard`, `Provenance`, `DiffView`)

It is the one module that imports another. The API client, the form model and the panels are swiss's domain, and a copy would be a second implementation of the same thing drifting from the first; the layout is what this module owns. Helpers swiss keeps private are ported into the module's *-lib.ts with a note. The cost: inferences cannot be built without modules/swiss.

Contexts: swiss's Gate runs in this module's context (SwissScope), so its redirect to /setup lands on /inferences/setup; swiss links in the embedded components (/site-profile) resolve the same way. SwissScope also shows swissd's cluster warnings (site profile or catalog failing to load) above every page, as swiss's layout does.

The service id becomes the helm release and the chart's Service names (<id>-cart), so it is checked as a DNS-1035 label and defaults to the model name made into one (mimo-v2.5 -> mimo-v2-5).

The SLO is shown read-only under cluster resources; editing it is a header action, "Edit SLO", opening swiss's SLO form in a sheet. The action is always there: greyed out, with the reason on hover, until the plan is applied with SLO on (when swiss's page shows its SLO card).

The logs and events tabs, and the overview's latest events, read console's own workload API rather than swissd's (see "Workload reads").

Not there yet:

Search on the services list. swissd pages its list (at most 100 a page) and takes no query; filtering one page on the client would miss the others. It needs ?q= on swissd's /api/deployments.
swiss's panels are English. They are swiss's, and swiss's pages have no Chinese yet. Route option labels are translated by inferences; untranslated help and new options use the engine's text.
Cluster resources needs a recent swissd. An older one (0.5.6, for one) has no /objects: the tab says so, and the install track reports the route and scaler as unreadable rather than absent.
Upgrade picks chart versions through /api/catalog/:name/chart-versions, which swissd has from 0.6.2 (swiss #13).

Deploy route options

The deploy and upgrade form reads the running openresty's catalog instead of treating bundled defaults as deployment defaults (console #52; agreed design).

DeploySheet -> GET /api/llm/route-options[?route=<existing>]
            -> configured gateway -> /<route>/_route_opts
            -> common grouped rows + advanced picker + runtime state

Concern	Decision
Authorization	Console login and `get backends` for `llm`, the same grant as the inference proxy
New deployment	Probe existing ConfigMap routes for a catalog; never show their per-route values as the new deployment's
Upgrade	Query the recorded route, display `value` and `set` separately from saved overrides
Defaults	Live `fallback` supplies placeholders; no copied values are submitted automatically
Legacy or empty gateway	404 uses bundled metadata with all fallback values stripped
Other failures	Surface a catalog warning; do not claim a deployment default
Text	Per-key locale translations where present; new options retain engine text
Ownership	Inferences adapts the Swiss metadata without editing the copied upstream module

In-flight generation recovery

The deploy and upgrade forms expose the charts' existing continuationGateway values (console #63).

form -> recovery values -> adaptive compose -> optional timeout reset -> validate -> diff -> apply
nginx -> continuation gateway -> CART -> engine

Concern	Decision
Controls	Enable gateway, actual model identity, idle timeout, continuation execution and tool-call buffering
Upgrade state	Seed from the stored plan once; query refreshes preserve edits
Validation	Check all recorded plan layers, chart compatibility, CART, ModelRoute, actual model, timeout and engine
Advanced YAML	Preserve precedence; stop when its effective settings differ from the recovery controls
Timeout reset	Clearing removes a retained form override by recomposing the validated form without `fromRelease`, pinned to the resolved source and chart
Integrity	Recovery and adaptive cleanup share source pinning and normalized plan comparison; reject unrelated changes
Ownership	Existing plan API and chart values; copied Swiss sources remain unchanged

Unpublished Monitor integration

modules/inferences/features.ts holds the single compile-time ENABLE_MONITOR flag, false until the Monitor backend is public (console #76).

Concern	Decision
Deploy / upgrade	Hide Monitor and its model/GPU inputs; public defaults and form requests explicitly disable it
Profile	Hide the Monitor ConfigMap form field and summary; retain stored profile data
ServiceMonitor	Independent Prometheus Operator integration; visible and configurable as before
Re-enable	Change the one flag and rebuild; the existing controls, translations and request handling remain
Ownership	Console adapter applies public defaults; copied Swiss UI is unchanged

Adaptive concurrency form

The master switch and every needsAdaptive field share one section (console #75); the floor fraction has its own label.

Concern	Decision
Groups	Gateway `needsAdaptive` metadata decides membership; known `adaptive_cc_*` keys are recognized without a live catalog
Off	Hide dependent common/advanced inputs, clear their entered values, and prevent adding adaptive custom keys while off
Capacity	`default_max` remains in Capacity & failover; it governs limits even with adaptive control off
Upgrade removal	Swiss `fromRelease` recursively retains omitted map keys. Compose normally first for its model-move/chart validation; if stale adaptive form keys survive, recompose the full validated form with those keys removed, pinned to the just-resolved model/version/chart
Integrity	Preserve unrelated form values and explicit advanced YAML; reject source/chart identity or other layer changes between the two compositions
Application	Only the cleaned plan enters the existing diff/revision/apply flow; composition performs no workload write

Workload reads (`/api/k8s/...`)

Events and logs are Kubernetes data, not swiss's, so console serves them itself with its own login and authorization instead of a swissd endpoint.

Endpoint	Returns	Authorized as	ServiceAccount needs
`GET /api/k8s/namespaces/{ns}/events?prefix=<release>`	events of objects named `<release>` or `<release>-*`, newest first, at most 500	`list events`	`events` list, watch
`GET /api/k8s/namespaces/{ns}/pods?prefix=<release>`	those pods with their containers' state	`list pods`	`pods` list
`GET /api/k8s/namespaces/{ns}/pods/{pod}`	one pod's containers	`get pods`	`pods` get
`GET /api/k8s/namespaces/{ns}/pods/{pod}/log?container=&tailLines=&sinceSeconds=&previous=`	`text/plain`, at most 5000 lines and 2 MiB, with timestamps	`get pods/log`	`pods/log` get

The resources are console's (iam.theriseunion.io), not the backend's: a role grants reading workloads on its own. The seeded admin passes everything.
Authorization is platform-scope like the rest of console: a user who may read events reads them in every namespace.
A 403 from the cluster means console's ServiceAccount lacks the grant and is returned as 502 with that said, so it is not read as the caller's.
The prefix match is the server's coarse cut; the page narrows it to the names a release's chart gives (ofRelease).
Logs need a typed clientset; cluster.Kube builds one beside the dynamic client.

The docs module (user guide)

The user guide (/help/docs) is part of the web bundle, not a separate site.

Content is Markdown, one file per chapter per locale: web/src/modules/docs/content/{locale}/{slug}.md. The first # line is the chapter title; CHAPTERS in chapters.ts is the order.
Shipped with the binary, so it describes the pages of this build and needs no network access — the same reason the UI is embedded.
Links starting with / are console routes and navigate in place; chapters.test.ts fails when one points at a page that does not exist, and when a locale lacks a chapter.
Linked from the top bar as well as the sidebar: the page declares topbar: true, and the shell renders every such page the user may open next to the language switch. The shell knows no module paths.
No permission: every signed-in user may read it. Chapters still say which permission each page needs.
Changing a page's behaviour means updating its chapter in both locales in the same PR.

Backends

Each module's backend is a backends entry; console proxies it after login.

browser  /api/deploy/catalog  ──▶ console
   authenticate (token) ─▶ password-reset guard ─▶ RBAC: verb on backends/<name>
   strip client X-Remote-*, Cookie; set X-Remote-User, X-Remote-Group, Bearer
                             ──▶ swissd  /api/catalog

Aspect	Behaviour
Path	prefix replaced by the url's path (`/api/deploy` + `http://swissd/api`)
Prefix	must be under `/api/` (that is what puts it behind login), not `/api/iam` or `/api/me`
Authorization	resource `backends`, resourceName = backend name; GET/HEAD/OPTIONS `get`, POST `create`, PUT/PATCH `update`, DELETE `delete`
Identity	`X-Remote-User` / `X-Remote-Group`, as Rise Global's apiserver sets for a ReverseProxy — a backend runs unchanged behind either
Credential	`apiKeyEnv` names an environment variable console sends as `Authorization: Bearer <value>`, replacing the caller's token. The browser never sees it. Empty means the token is forwarded unchanged. swissd does not read console's JWT: behind console it runs with `server.auth.disabled` and trusts console's RBAC
Streaming	flushed as it arrives (SSE, chunked progress)
Backend down	JSON `502`

A backend must trust X-Remote-* only from console (network policy / mTLS), never from browsers.

In-cluster, a static backend's credential comes from a Secret rather than the chart's values: helm/console renders an env entry per backendSecrets key. A variable that is named but unset is logged at startup and the request goes without it — console starts, and the backend answers 401, rather than the pod refusing to boot.

A backend that resolves itself (`gateway`)

Naming an inference gateway in a values file means copying three things that all move: the Service address, which route serves every deployed model, and the key. The first goes stale when the gateway is reinstalled, the second lies about what is deployed the moment a model is added, and the third ends up in release history or in a second copy that nobody rotates.

So the llm backend names where the truth is instead:

backends:
  - name: llm
    prefix: /api/llm
    gateway:
      profile: llm/site-profile          # swiss's site profile, "namespace/name"
      # ...or, with no swissd:
      # configMap: llm/openresty-conf    # route keys: session_route_<route>.conf
      # service: llm/openresty           # the entrypoint
      # route: llm-gateway               # optional: pin one route
      # secretRef: llm/openresty-keys    # optional: override where the key lives

Step	Where it comes from
Entrypoint	profile `route.nginxService` + `route.nginxPort` (default 8080) → `http://<name>.<ns>.svc:<port>`
Routes	every `session_route_<route>.conf` in the ConfigMap, aggregate ones (`peers_by_model`, serving several models) first; `route:` pins one. Which route serves which model is not in the ConfigMap -- see "Models across routes"
Key	profile `route.auth.secretRef`/`secretKey` (default entry `keys`, format `key1:owner1,…`; the first key is used), sent as `Authorization: Bearer <key>` — llm-openresty accepts nothing else. A bare Secret name means the entrypoint's namespace
Overrides	`apiKeyEnv` (a copied key in the console's namespace), and `route`

Resolution is cached for 30s and re-read after that, so a model deployed later appears without redeploying the console; a refresh that fails keeps the last good answer and logs it, because a transient API error should not take inference down. An entrypoint that has never resolved answers 502 with the reason (backend llm: site profile llm/absent: not found) instead of a confusing 404.

RBAC follows the references: helm/console renders one read-only Role and binding per namespace they name, on configmaps and secrets. No resourceNames, because the profile is what names the route ConfigMap and the key Secret, and a chart cannot know at render time what a ConfigMap will say later. Nothing reads the Service: the address is assembled from the reference.

Catalog writes

The model library edits a writable catalog through swissd, under /api/deploy like the rest of swiss: swissd stands in for the catalog's address and passes its catalog-server API through as it is, adding the token it keeps in its own Secret (swiss's docs/swiss-design.md). So the page calls catalog-server's own paths (model-catalog's docs/openapi.yaml) below /api/deploy/catalogs/<name>/api/v2/. Console adds nothing to the path.

Concern	Decision
Which catalogs	those the site profile marks `writable` and whose token is set (`GET /api/deploy/secrets` lists keys, never values); the model page links to the site profile when the token is missing
Token	a field of the profile editor, shown once a catalog is ticked writable; optional, so a missing token never blocks a profile save (the field, the profile page and the model page say it is missing); after the profile saves, typed tokens are written to `<profile>-secrets` and the tokens of catalogs no longer writable removed
Authorization	the `swiss` backend's verbs: `update` adds or replaces a variant and sets a token, `delete` deletes either
Author	catalog-server names the commit's author by the token, so every console edit is the token owner's; the console user is not recorded yet
Setup	an empty catalog (`GET init` says so) offers Set up this catalog instead of editing: `site` and `repo`, starting from the catalog's URL in the profile, then `POST init`
Settings, model	Catalog settings (`catalog.yaml`) in the library toolbar and Edit model (`metadata.yaml`) on a model's page: the file at its digest, checked then committed
New model	New model in the library toolbar, with fields for its name and first variant id that write into both files: `metadata.yaml` and a first variant copied from one the catalog has, in one commit (catalog-server's `POST models`); the page then opens the new model
Freshness	swissd drops its cached index on a write, so the page refetches and shows the commit

Console owns no catalog storage and publishes no index of its own. A writable catalog is a catalog-server the site profile names; an install whose own catalog is a static index gets one from the swiss chart, which vendors catalog-server since 0.7.0 (swiss.catalogServer; the chart README covers enabling it, its token, and the profile entry). A console-hosted catalog was rejected: it would duplicate catalog-server's repository and write API, and its publication would be deprecated schema v1, which the LLMSvc backend cannot deploy.

Playground

The Playground is chat against the inference gateway, for the question a deployment page cannot answer: does this model actually generate? swissd's own chat probe is one prompt, 32 tokens, non-streaming, and it lives on a release; the Playground is a conversation, streaming, for anyone with the permission.

Concern	Decision
Where it runs	`web/src/modules/playground/`, mounted at `/playground`; the page is lazy-loaded (Markdown and highlighting are most of the weight)
Pages	`/playground` — one conversation. A compare page (2–4 columns, one prompt to each) was removed to get the one conversation right first
Which backend	`llm` → the gateway, resolved from the cluster; one picker lists every model on every route (see "Models across routes")
Gateway key	read by console from the Secret the site profile names (or `apiKeyEnv`). Never in a values file, never in a chart value, never in the browser
Conversation identity	each conversation sends its own `X-Session-Id`; the gateway pins it to one engine, so its prefix cache stays warm across turns
Parameters	system prompt, temperature, top_p, max_tokens, seed, stop (one per line), frequency/presence penalty, reasoning_effort; an empty field is left out of the request, so the engine's default applies
Token counts	the page sends `stream_options.include_usage`: openresty injects it only on aggregate routes, and autoconfig's per-model routes are not
Stats per answer	TTFT, total time, input/output tokens, tok/s over the decoding window (after TTFT), and cache hit rate = `prompt_tokens_details.cached_tokens / prompt_tokens`, shown only when the engine reports it
Reasoning models	`delta.reasoning_content`, or a leading `<think>…</think>` in content, is shown in a collapsible block above the answer
Reasoning in history	never sent back: the next turn's history carries only the answer, as the OpenAI-style APIs expect
View code	cURL / Python / Node.js reproducing the current request against `<origin>/v1`, key read from `$MODELSPHERE_API_KEY` — an API key an admin issued (see Router)
Access	UI permission `playground.use` guards the pages; the backend needs `get` and `create` on `backends/llm` (models are a GET, completions a POST)

A user therefore needs a role carrying both:

{"uiPermissions": ["playground.use"],
 "rules": [{"apiGroups": ["iam.theriseunion.io"], "resources": ["backends"],
            "resourceNames": ["llm"], "verbs": ["get", "create"]}]}

Models across routes

The modelsphere stack's gateway is fed by autoconfig: one plain route per model (session_route_<model>.conf, peers only), and no route that lists them all. Console makes one model list and one endpoint out of it:

resolve (30 s): ConfigMap keys -> routes [aggregate..., plain...]
index (per backend): GET <gateway>/<route>/v1/models for every route, in parallel
                     model -> first route listing it
request: GET /v1/models          -> answered from the index
         anything with "model"   -> <gateway>/<route of that model>/...

Concern	Decision
Freshness	the index is kept 30 s and refreshed behind a stale answer; an unknown model re-asks at once (at most every 2 s), so a model deployed a moment ago works on its first request
A route that does not answer	left out, not fatal. openresty loads a route up to a minute after autoconfig writes it and answers 502 until then, so a list missing a route is re-asked after 2 s instead of 30 s
Same model on two routes	the first wins; aggregate routes come first, so a hand-written aggregate route beside autoconfig's is preferred
No model in the request	fine with one route, 400 with several
Unknown model	`/api/llm`: 400 listing what is served; `/v1`: 404 `model_not_found`, as OpenAI answers
Scoped keys	`/v1/models` and `/v1/models/<id>` are answered by the router from the index, filtered by the key; nothing is rewritten on the way back
Replicas	each keeps its own index; a route added or removed is seen by all within one resolve (≤ 30 s)

Pointing an install at the stack's gateway:

playground:
  gateway:
    configMap: llm-route/openresty-conf
    service: llm-route/openresty
    secretRef: llm-route/openresty      # the openresty chart's key Secret, if auth is on

Verified on daocloud-ce against the stack's own charts (helm-charts openresty 0.1.20 + autoconfig 0.4.0, ModelRoute routing.modelsphere.dev): two llama.cpp engines, two ModelRoutes, two plain routes. The Playground lists and chats with both, with token stats; /v1 routes each model to its engine through the OpenAI SDK; a key scoped to one model sees and uses only it; deleting and recreating a ModelRoute is followed by both replicas.

Router

The router is how programs reach the models: /v1, opened by API keys admins issue. It is its own domain -- internal/router, the router web module, /api/router -- not part of iam: a key authorizes inference, not a console user. console owns the keys; the gateway keeps its single key and is not changed.

client --Bearer ms_…--> console /v1 --gateway key--> llm-openresty
                         verify key (hash, expiry)
                         read body (capped) -> check `model` against the key

Concern	Decision
Endpoint	`/v1/` → the backend `router.backend` names (`llm`), resolved like it; `Authorization`, `Cookie`, `X-Remote-` stripped, the backend's credential added; `X-Session-Id` passes unchanged; streams as it arrives
Errors	OpenAI envelope (`{"error":{"message","type","code"}}`): 401 `invalid_api_key` / `expired_api_key`, 403 `model_not_allowed`, 413 `request_too_large`, 502/503 when the gateway or the key store is unavailable
Format	`ms_<16 hex id>_<32 hex secret>`; the list shows `ms_<first 4 of id>***`
Hashing	salted SHA-256 of the secret. The secret is 128 random bits, so a slow hash buys nothing and would cost CPU on every inference call
Issuing	`/api/router/apikeys` list/create/delete, RBAC resource `apikeys` — system:masters unless a role grants it; the page (路由 → API 密钥) needs UI permission `apikeys.view`. The plaintext is in the create response only
Expiry	7, 30 or 180 days, or never; nothing else is accepted
Model scope	none = every model. A scoped key: JSON bodies must name an allowed `model`; a request naming none (multipart, files) is refused; `GET /v1/models` is filtered to the allowed ones
Body cap	`router.maxBodyBytes`, default 16 MiB — the body is read whole to find `model`
Storage	one Secret `<release>-api-keys` in console's namespace, one entry per key id holding its JSON. console creates it; the chart never owns it, so upgrade/rollback/uninstall leave issued keys alone. No CRD in the shared `iam.theriseunion.io` group
Replicas	each replica caches the keys: re-read every 10 s, and at most once a second for an unknown key. A key deleted elsewhere keeps working here for up to 10 s; a failed re-read keeps the last good set
Writes	create and delete only, each a read-modify-write retried on resourceVersion conflicts. Usage is never written back: see Router metrics
Availability	console is now on the inference path: 2 replicas by default, spread across nodes, PDB `maxUnavailable: 1`; on SIGTERM `/readyz` fails, 5 s for endpoints to catch up, then up to 60 s for in-flight generations (`terminationGracePeriodSeconds: 75`)

Router metrics

Usage is observed, not stored: writing "last used" to the keys' Secret would put an apiserver write on the inference path. Served on server.metricsAddr (:9090), a listener of its own so metrics are not reachable wherever the UI is, behind a ClusterIP Service of its own (<release>-metrics) whatever type the UI's Service is; the chart adds prometheus.io/* pod annotations and an optional ServiceMonitor.

Metric	Labels	Meaning
`router_requests_total`	`key_id`, `key_name`, `model`, `code`	requests forwarded, by the status the gateway answered. `model` is empty unless that status was 2xx, so a caller cannot mint series by naming models that do not exist
`router_key_last_request_timestamp_seconds`	`key_id`, `key_name`	last request that passed the key check -- "last used", per replica: take `max` across pods
`router_rejected_total`	`reason`	answered by the router itself: `invalid_key`, `expired`, `model_not_allowed`, `too_large`, `unreadable`, `keys_unavailable`, `no_backend`

A deleted key's series are dropped on the replica that deleted it; elsewhere they stay until that pod restarts. Go runtime and process metrics are exported beside them.

One-command install

helm install modelsphere ./helm/console installs console, its CRDs and the seeded admin. The chart carries no inference path of its own: models come from swiss, and the Playground and /v1 reach them through swiss's gateway.

browser ─► console ─/api/llm─┐
program ─► console ─/v1──────┴─► swiss's gateway (llm-openresty) ─► models deployed in Model Serving

Concern	Decision
Models	from swiss: `swiss.enabled=true` (the subchart) or `externalSwiss` (an existing swissd). The `llm` backend reads the gateway from swissd's site profile; `playground.gateway` names another gateway instead
No swiss	a valid install: login, users and roles. There is no `llm` backend, so the Playground lists nothing and the router is off
Access	Service type NodePort by default, so the address works when the install returns; `NOTES` prints it
Not bundled	a gateway (removed in #9) and a CPU demo model (`demo.enabled`, removed): the Playground lists Model Serving's deployments, so a model running outside swiss never showed there

Still missing for a public one-command install

CI (.github/workflows/publish.yml, the same scheme as swiss) publishes the image to ghcr.io/modelsphere/console and the chart to oci://ghcr.io/modelsphere/charts/console at one version: <appVersion>-git<sha7> for every commit on main, X.Y.Z for a release tag. The packaged chart points at the image published beside it.

Gap	Why it matters	Where it is fixed
swiss is off by default	the chart pulls the published swiss chart, but its engines need GPUs and hostPath weights, and routing needs autoconfig with the ModelRoute and LLMSLORequirement CRDs	`swiss.enabled=true`; see "Bringing swiss in"
swiss and console read the gateway key differently	swiss sends the Secret value verbatim (`apiKey`), console parses `key:owner` (`keys`)	one format, in swiss

Bringing swiss in

swiss's frontend is copied into web/src/modules/swiss/ without changes on the swiss side yet. The copy is two commits: a verbatim one naming the swiss commit, then the edits that mount it, so a resync is "copy again, replay the edits".

Concern	In the copy
Left out	`main.tsx`, `index.css`, `vite-env.d.ts`, `components/Layout.tsx`, `routes/Login.tsx`, `routes/PreviewDeploySettings.tsx`
Imports	`@/…` → `@swiss/…` (alias to `src/modules/swiss`)
Links	`Link`, `NavLink`, `Navigate`, `useNavigate` come from `lib/host.ts`, which resolves absolute paths against the mounting module (`/inferences`)
API	`lib/api.ts` fetches through `hostFetch(apiPath(…))`: `/api/x` → `/api/deploy/x`, with the console session
Login	console's. `components/Session.tsx`'s gate shows an error instead of swiss's login, and still sends an uninitialised site to setup
Layout	console's. Model Serving's `SwissScope` keeps swiss's cluster warnings above each page; the site switcher is not carried over
Theme	Rise tokens; `warning` and `success` added in `src/index.css`
Access	every page needs UI permission `swiss.view`; calls need the verb on `backends/swiss`

What swiss could still take on so the copy needs configuration, not page edits:

#	Today in `swiss/web`	Change in swiss	Why
1	imports use `@/…`	use `@swiss/…`	`@/` is the console's root; console adds one alias `@swiss` → `src/modules/swiss`
2	absolute links `to="/catalog"`, `navigate("/")`	one `swissPath()` helper (console binds it to `useModulePath`)	panels render under `/inferences`
3	`fetch("/api/...")`	one client with a configurable base and fetch	console sets base `/api/deploy` and `apiFetch`
4	`main.tsx` holds routes, router, query client, layout	`module.tsx` exports pages + menu; `main.tsx` keeps the standalone shell	console mounts the pages, not the app
5	`index.css` declares the theme variables	move them to a standalone-only file	the Rise tokens provide the theme here

API types: swissd exports openapi.json (committed, CI-checked), and the console generates the TypeScript client from it, replacing the hand-written types in swiss/web/src/lib/api.ts.

Rise Global target

Not in scope now. The module contract keeps the door open: pages take no props from the shell, links and fetches go through shell functions, and backend requests carry the same headers Global's apiserver sets.

Known debt

The token preset is Tailwind 3-style, loaded through @config. A Tailwind 4 CSS build of the tokens would drop tailwind.config.ts.

Phases

Phase	Scope
P0	Scaffold: Go BFF skeleton, config, dynamic client, Docker, helm chart. (done)
P1	Login loop: iam CRD types, dynamic CRUD, `/oauth/token`, auth middleware, helm-seeded admin, frontend login + guard.
P2	User management: user CRUD API + UI, i18n (zh-CN/en-US). (i18n done: shell, iam, playground, router; swiss pages pending swiss.) Open UI kit `@modelsphere/ui` replacing `@riseaicloud/ui` everywhere, swiss module included; `web/vendor/` deleted. (done)
P3	Full roles: authorizer + role/binding CRUD + role/permission UI.
P4	Federation: module shell, stack aligned with swiss, backend proxy with identity headers and per-backend RBAC. (done)
P5	swiss module: copy into `modules/swiss`, mounted through `lib/host.ts` (see "Bringing swiss in"). (done; unmounted again once Model Serving covered every page) Then: swiss prepares its frontend (table above) and exports `openapi.json`; CODEOWNERS.
P6	Playground: chat module, `llm` backend resolved from the cluster (site profile or route ConfigMap) with the gateway key read from its Secret, streaming SSE end to end. (done) Then: Markdown, compare page (since removed), full parameters, stats, view code. (done) The router: `/v1`, API keys, usage metrics. (done) One-command install. (done; its built-in gateway and CPU demo model since removed: models come from swiss)
P7	Container management modules, moved over from Rise Global.

Each phase is independently committable and verifiable.

Navigation Menu

Repository navigation

console-design.md

console-design.md

console — design

What this is

Why a separate service (not part of swiss)

Wire-compatibility with Rise Global (the upgrade path)

Running without a login (`server.auth.disabled`)

Architecture

Modules

i18n

UI library (`@modelsphere/ui`)

The swiss module's two UI kits

The inferences module

Deploy route options

In-flight generation recovery

Unpublished Monitor integration

Adaptive concurrency form

Workload reads (`/api/k8s/...`)

The docs module (user guide)

Backends

A backend that resolves itself (`gateway`)

Catalog writes

Playground

Models across routes

Router

Router metrics

One-command install

Still missing for a public one-command install

Bringing swiss in

Rise Global target

Known debt

Phases

FilesExpand file tree

console-design.md

Latest commit

History

console-design.md

File metadata and controls

console — design

What this is

Why a separate service (not part of swiss)

Wire-compatibility with Rise Global (the upgrade path)

Running without a login (server.auth.disabled)

Architecture

Modules

i18n

UI library (@modelsphere/ui)

The swiss module's two UI kits

The inferences module

Deploy route options

In-flight generation recovery

Unpublished Monitor integration

Adaptive concurrency form

Workload reads (/api/k8s/...)

The docs module (user guide)

Backends

A backend that resolves itself (gateway)

Catalog writes

Playground

Models across routes

Router

Router metrics

One-command install

Still missing for a public one-command install

Bringing swiss in

Rise Global target

Known debt

Phases

Running without a login (`server.auth.disabled`)

UI library (`@modelsphere/ui`)

Workload reads (`/api/k8s/...`)

A backend that resolves itself (`gateway`)