feat: Codex OAuth (ChatGPT subscription) as LLM provider#453
feat: Codex OAuth (ChatGPT subscription) as LLM provider#453TatsuKo-Tsukimi wants to merge 5 commits intodataelement:mainfrom
Conversation
Adds a new provider 'codex-oauth' so users can authenticate with their ChatGPT Plus/Pro subscription instead of a static OpenAI API key. Follows the community OAuth 2.1 + PKCE flow used by OpenClaw / Hermes / numman-ali/opencode-openai-codex-auth. - Pure primitives in services/llm/codex_oauth.py (PKCE, authorize URL, exchange/refresh, JWT decode) - New CodexOAuthClient subclassing OpenAIResponsesClient - PROVIDER_REGISTRY entry + create_llm_client dispatch - llm_models schema: auth_type + oauth_* columns, api_key_encrypted nullable (+ Alembic migration) - get_llm_client_for_model() dispatcher; call sites switched - Loopback listener on :1455 + 4 REST endpoints (/start /poll /complete /paste-creds) - docker-compose: 1455 port + CODEX_OAUTH_LOOPBACK_HOST env - Unit tests for OAuth primitives Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the end-to-end UX surface for the backend codex-oauth provider
introduced in the previous commit.
- ConnectChatGPTModal component: two tabs (browser OAuth + paste tokens).
OAuth tab drives /start -> opens authorize URL in a new tab -> polls
/poll every 1.5s up to 5 min; falls back to manual redirect-URL paste
when loopback is unavailable. Paste tab lets users import tokens
directly from a local codex login (~/.codex/auth.json).
- EnterpriseSettings LLM tab: secondary 'Connect ChatGPT' button next to
Add Model; invalidates the llm-models query on success so the new row
appears immediately.
- services/api.ts: typed codexOauthApi client (start / poll / complete /
pasteCreds) plus CODEX_OAUTH_MODELS whitelist mirroring the backend.
- i18n: enterprise.llm.codex.* keys in en.json and zh.json.
- test_codex_oauth.sh: helper smoke test that drives the paste-creds
flow end-to-end from a local ~/.codex/auth.json.
The backend PUT /llm-models/{id} handler already guards api_key against
empty/masked overwrites, so editing an auth_type='codex_oauth' model
through the existing form is safe without a dedicated edit surface.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Spins up an independent Clawith instance on port 3009 with its own postgres/redis volumes and a dedicated bridge network (clawith_codex_network), so the Codex OAuth feature can be exercised end-to-end without touching a user's primary Clawith deployment. Usage: docker compose -p clawith-codex -f docker-compose.test.yml up -d --build Binds host:1455 for the loopback OAuth callback and sets CODEX_OAUTH_LOOPBACK_HOST=0.0.0.0 inside the container so the browser's http://localhost:1455/auth/callback reaches the backend listener. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
TatsuKo-Tsukimi
force-pushed
the
feat/codex-oauth-provider
branch
from
96136ed to
f2c1fa3
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 96136ed4b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ) -> LLMModel: | ||
| settings = get_settings() | ||
| row = LLMModel( | ||
| tenant_id=getattr(current_user, "tenant_id", None), |
There was a problem hiding this comment.
This path always writes tenant_id from current_user and never accepts a tenant override, unlike the existing /enterprise/llm-models flow. In a platform-admin session managing a different selected tenant, the new Codex OAuth model is inserted under the admin's own tenant (often null) instead of the target tenant, so it won't show up in that tenant's model list or be usable by its agents.
Useful? React with 👍 / 👎.
| ) | ||
|
|
||
| return create_llm_client( | ||
| provider=model.provider, |
There was a problem hiding this comment.
The static-auth branch forwards model.provider directly to create_llm_client; once codex-oauth is exposed via provider discovery, admins can create/edit models through the normal static form with provider='codex-oauth' but auth_type='static'. Those models hit this branch and fail at runtime because create_llm_client requires OAuth context (model_id/session_factory) for codex_oauth, causing chat calls to error instead of executing.
Useful? React with 👍 / 👎.
…outing Two issues raised on PR dataelement#453 by the Codex auto-reviewer, both boundary gaps that would silently route callers to the wrong place. 1. Tenant scoping on model creation /complete and /paste-creds hardcoded tenant_id=current_user.tenant_id, unlike POST /enterprise/llm-models which accepts an optional tenant_id query override. A platform admin managing another tenant would write the new Codex OAuth row to their own tenant (often NULL), where the target tenant's agents can't see it. Adds tenant_id query param to /complete and /paste-creds with the same 'override or fall back to current_user.tenant_id' semantics as the existing endpoint. Refactored to _resolve_tenant_id() so both flows share validation (rejects malformed UUIDs with 400). 2. Static form routing static-auth rows through the OAuth client Once codex-oauth is registered in PROVIDER_REGISTRY it automatically shows up in /enterprise/llm-providers, so admins can create it via the normal Add Model form. The row lands as provider='codex-oauth' + auth_type='static' (DB default), and chat calls then hit create_llm_client's codex_oauth dispatch branch without the required model_id / session_factory context, raising ValueError instead of serving inference. Three-layer defense: - get_provider_manifest() skips providers whose protocol is marked hidden (codex_oauth for now), so the UI can't offer it. - POST /enterprise/llm-models rejects provider='codex-oauth' with a 400 that points at the dedicated endpoints. - PUT /enterprise/llm-models/{id}, for rows already auth_type= codex_oauth, locks down the identity fields (provider / base_url / api_key) while still allowing label / enabled / timeouts / etc. to be edited. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 30b58232cf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| from app.services.llm import create_llm_client, get_max_tokens, LLMMessage, LLMError, get_model_api_key | ||
| from app.services.llm import ( | ||
| create_llm_client, | ||
| get_llm_client_for_model, |
There was a problem hiding this comment.
This import pulls get_llm_client_for_model from app.services.llm, but backend/app/services/llm/__init__.py does not export that symbol. When heartbeat/A2A/supervision paths hit this block, Python raises ImportError before any LLM call, so those workflows fail even for non-OAuth models. Please either re-export get_llm_client_for_model from __init__.py or import it from app.services.llm.utils at call sites.
Useful? React with 👍 / 👎.
| ), | ||
|
|
||
| complete: (body: { state: string; code: string; label: string; model: string }) => | ||
| request<CodexOauthModelResponse>('/llm-models/codex-oauth/complete', { |
There was a problem hiding this comment.
Fresh evidence relative to the earlier tenant-scope concern: although the backend now accepts tenant_id on Codex OAuth endpoints, this new client call never appends the selected tenant, so platform-admin sessions provisioning for another tenant still default to the caller's own tenant context. As a result, created OAuth models can land under the wrong tenant and not appear in the target tenant’s model list.
Useful? React with 👍 / 👎.
| raise HTTPException(status_code=404, detail="Model not found") | ||
|
|
||
| # Block turning a static row INTO an OAuth-only provider via this form. | ||
| _reject_oauth_provider_via_static_form(data.provider) |
There was a problem hiding this comment.
This guard runs before checking model.auth_type, so updates that include an unchanged provider='codex-oauth' are rejected outright. The edit form submits provider with every save, which means normal edits (label/model settings) on already-provisioned Codex OAuth models fail with 400, despite the subsequent logic intending to only block identity-field mutation.
Useful? React with 👍 / 👎.
Three follow-ups on PR dataelement#453 after the first fix round. 1. Re-export get_llm_client_for_model from app.services.llm (P1) The utility was defined in services/llm/utils.py but the package's __init__.py only re-exported the older create_llm_client / get_* set. agent_tools.py, heartbeat.py, and supervision_reminder.py all import via 'from app.services.llm import ... get_llm_client_for_model', so any A2A / heartbeat / supervision path — with ANY model, OAuth or not — raised ImportError at the deferred-import line before the LLM call could start. WebSocket chat worked because caller.py imports directly from .utils, masking the gap in earlier testing. 2. Thread tenant_id through the frontend Codex OAuth calls (P1) Previous fix added a tenant_id query parameter to /complete and /paste-creds, but api.ts and ConnectChatGPTModal never forwarded one. Platform admins provisioning for another tenant still defaulted to the caller's own tenant. ConnectChatGPTModal now takes a tenantId prop; EnterpriseSettings passes selectedTenantId; the API client appends ?tenant_id=... when present. 3. Reorder the update_llm_model OAuth guards (P2) _reject_oauth_provider_via_static_form ran before the model.auth_type check, so editing an already-OAuth row with its own unchanged provider='codex-oauth' in the request body hit the reject path. Edit forms re-submit the current provider on every save, so routine label/enabled updates on Codex OAuth rows returned 400. Moved the reject into the else branch of the auth_type check; the OAuth-row path still locks identity fields (provider / base_url / api_key) but lets passive fields through. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ea8348e304
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| def _consume_session(state: str) -> dict[str, Any] | None: | ||
| """Read-and-remove a session on successful code exchange.""" | ||
| with _sessions_lock: | ||
| return _sessions.pop(state, None) |
There was a problem hiding this comment.
_consume_session removes and returns any cached state without checking expires_at, so /llm-models/codex-oauth/complete can still accept sessions well past the documented 10-minute TTL until another /start happens to garbage-collect them. This extends the validity window of stale authorization states and makes the “expired session” guard unreliable for manual-complete flows.
Useful? React with 👍 / 👎.
1 participant
Summary
Adds a new LLM provider
codex-oauthso users can authenticate Clawith's LLM calls with their ChatGPT Plus/Pro subscription instead of a static OpenAI API key. Follows the same OAuth 2.1 + PKCE flow used by OpenClaw, Hermes, andnumman-ali/opencode-openai-codex-auth:app_EMoamEEZ73f0CkXaXp7hrann(shared across OSS Codex integrations)auth.openai.com/oauth/{authorize,token}chatgpt.com/backend-api/responses— same Responses API protocol Clawith already speaks viaOpenAIResponsesClient, so the new client reuses all payload / parsing / streaming logic from the existing class and only overrides base URL + required headers (OpenAI-Beta: responses=experimental,originator: codex_cli_rs,chatgpt-account-iddecoded from the access-token JWT)Why
Codex subscribers currently can't reuse their flat-rate ChatGPT Plus/Pro coverage inside Clawith — they have to provision a separate OpenAI API key with per-token billing. This PR adds parity with other open-source agent platforms that integrated Codex OAuth.
How it works
Backend
CodexOAuthClientextendsOpenAIResponsesClient. Per-call,_ensure_fresh_token()opens a session,SELECT ... FOR UPDATEs thellm_modelsrow, refreshes ifexpires_at < now + 60s, and persists the new token pair (AES-256-CBC via the existingencrypt_data/decrypt_datautilities). Concurrency-safe against multiple workers / sessions touching the same row.ProviderSpec.protocolLiteral extended with"codex_oauth"; registry entry added;create_llm_clientdispatches to the new client.get_llm_client_for_model(model, ...)factory inservices/llm/utils.pyreads the newauth_typecolumn and routes static vs OAuth. All call sites incaller.py,agent_tools.py,heartbeat.py,supervision_reminder.pyswitched to the factory.Schema
New Alembic migration (
add_codex_oauth_to_llm_models, chained onincrease_api_key_length) extendsllm_models:auth_type VARCHAR(20) NOT NULL DEFAULT 'static'—'static'keeps the existing API-key path,'codex_oauth'marks OAuth rows.oauth_access_token_encrypted VARCHAR(4096),oauth_refresh_token_encrypted VARCHAR(1024),oauth_expires_at TIMESTAMPTZ,oauth_account_id VARCHAR(255)— all nullable.api_key_encryptedrelaxed to nullable (OAuth rows don't carry a static key). The existingPUT /enterprise/llm-models/{id}handler already guards against empty / masked-value overwrites, so editing an OAuth model through the existing form won't clobber its tokens.Frontend
ConnectChatGPTModalcomponent with two tabs:/start→ opens the authorize URL in a new tab → polls/pollevery 1.5s up to 5 minutes → captures the code from the loopback listener; manual redirect-URL paste as fallback if the browser's redirect can't reachlocalhost:1455.access_token/refresh_token/expires_in_secondsdirectly (e.g. from a localcodex login's~/.codex/auth.json). Clawith takes over refresh from there.llm-modelsquery on success so the new row appears immediately.enterprise.llm.codex.*.Loopback listener
http.serverbound toCODEX_OAUTH_LOOPBACK_HOST:1455(defaults127.0.0.1; Docker Compose sets0.0.0.0with a1455:1455host mapping so the browser'shttp://localhost:1455/auth/callbackreaches the backend)./startreturnsloopback_ready=falseand the UI surfaces the manual-paste fallback.state → {verifier, code}cache with 10-minute TTL and thread-safe access.Multi-tenant
llm_models.tenant_idalready scopes LLM configs per tenant. Each OAuth grant is a separate row keyed by tenant, so one Clawith instance can hold multiple ChatGPT accounts.Files
backend/app/services/llm/codex_oauth.py— PKCE / authorize URL / token exchange / refresh / JWT decode (pure, unit-testable)backend/app/services/llm/client.py—CodexOAuthClient, registry entry, dispatchbackend/app/services/llm/utils.py—get_llm_client_for_modelfactorybackend/app/services/llm/caller.py,agent_tools.py,heartbeat.py,supervision_reminder.py— swap in factorybackend/app/models/llm.py— new columnsbackend/alembic/versions/add_codex_oauth_to_llm_models.py— idempotent migrationbackend/app/api/codex_oauth.py— 4 REST endpoints + loopback listenerbackend/app/main.py— register routerbackend/tests/test_codex_oauth.py— PKCE, URL params, exchange / refresh (httpx mocked), JWT decodefrontend/src/components/ConnectChatGPTModal.tsxfrontend/src/pages/EnterpriseSettings.tsx— wire the buttonfrontend/src/services/api.ts— typedcodexOauthApiclient +CODEX_OAUTH_MODELSwhitelistfrontend/src/i18n/{en,zh}.json—enterprise.llm.codex.*keysdocker-compose.yml—1455:1455port +CODEX_OAUTH_LOOPBACK_HOST=0.0.0.0docker-compose.test.yml— isolated test harness (port 3009, separate volumes)test_codex_oauth.sh— paste-creds smoke test driverTest plan
pytest backend/tests/test_codex_oauth.py— PKCE S256 consistency, authorize URL params, exchange / refresh over mocked httpx, JWT claim decodeupgrade headapplies cleanly fromincrease_api_key_length;downgradedrops the added columns and restores NOT NULL onapi_key_encryptedPOST /start→ authorize URL renders with all Codex-CLI params (codex_cli_simplified_flow=true,id_token_add_organizations=true,originator=codex_cli_rs);/pollon unknown state →{expired:true};/completeon expired session → 400~/.codex/auth.jsoncontents → model row created withauth_type='codex_oauth',oauth_account_iddecoded from JWT, tokens encrypted withSECRET_KEY/start→ ChatGPT login → browser redirect tolocalhost:1455/auth/callback→ loopback listener captures code →/pollreturns it →/completepersists modeloauth_expires_atin DB → next inference call triggersrefresh_token, updates the row underSELECT FOR UPDATE, retries the requestPOST https://chatgpt.com/backend-api/responseswithAuthorization: Bearer <jwt>,OpenAI-Beta: responses=experimental,originator: codex_cli_rs, andchatgpt-account-idheadersKnown limitations
http://localhost:1455/auth/callback. For Clawith deployments where the user's browser can't reach the backend's port 1455 (e.g. browser on laptop, backend on remote cloud host), the paste-tokens mode covers the gap./startand/pollmight hit different workers.chatgpt.com/backend-api/responsesis subject to OpenAI's own geo policies.References
🤖 Generated with Claude Code