feat(realtime): add Azure OpenAI and Z.ai/Zhipu GLM realtime providers#396
Merged
Conversation
Both use OpenAI's realtime event schema, so they slot into the existing transparent proxy with just a provider RealtimeTarget. - Z.ai/Zhipu GLM-Realtime: endpoint and core events mirror OpenAI (…/api/paas/v4/realtime); derives via the shared providers.OpenAIRealtimeURL, honors ZAI_BASE_URL region (api.z.ai vs open.bigmodel.cn), Bearer auth. Adds ChatCompatible.GetBaseURL so the base URL is read live (no staleable copy). - Azure OpenAI GPT Realtime: same schema, different dial shape — builds wss://<resource>/openai/realtime?api-version=…&deployment=… from the resource root and authenticates with the api-key header (not Bearer). The model selects the deployment. - Compile-time core.RealtimeProvider assertions and unit tests (URL/auth/region/ no-key/missing-model) for both. Skipped MiniMax: its only documented realtime websocket (TTS) uses a non-OpenAI schema (task_start/continue/finish) and no OpenAI-compatible conversational realtime endpoint is published, so a transparent relay can't be confirmed. Gemini Live and AWS Nova Sonic remain out of scope (separate event schemas, need a translation adapter). Note: these were unit-tested only — no Azure/Z.ai credentials available locally to live-verify, unlike OpenAI/Bailian/xAI in the prior PR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughAdds realtime WebSocket ( ChangesRealtime Provider Implementations
Sequence Diagram(s)sequenceDiagram
participant Caller
participant ZAI as zai.Provider
participant Azure as azure.Provider
participant ChatCompat as ChatCompatible
participant ZAIHelper as zai.realtimeURL
participant AzureHelper as azure.realtimeURL
Caller->>ZAI: RealtimeTarget(ctx, req)
Caller->>Azure: RealtimeTarget(ctx, req)
ZAI->>ZAIHelper: build URL
Azure->>AzureHelper: build URL
ZAIHelper->>ChatCompat: GetBaseURL()
ChatCompat-->>ZAIHelper: baseURL
ZAIHelper->>ZAIHelper: normalize scheme, fix path, add model param
AzureHelper->>AzureHelper: parse, normalize scheme, rewrite path, add query params
ZAIHelper-->>ZAI: wss URL
AzureHelper-->>Azure: wss URL
ZAI-->>Caller: RealtimeTarget{URL, Authorization header}
Azure-->>Caller: RealtimeTarget{URL, api-key header}
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR adds realtime websocket support for Azure OpenAI and Z.ai/Zhipu GLM. It changes:
Confidence Score: 5/5This looks safe to merge.
Reviews (2): Last reviewed commit: "fix(realtime): pin Z.ai realtime path; s..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| } | ||
| u.Path = strings.TrimRight(u.Path, "/") + "/openai/realtime" | ||
| q := url.Values{} | ||
| q.Set("api-version", p.apiVersion) |
There was a problem hiding this comment.
When AZURE_API_VERSION is omitted, this sends the provider default 2024-10-21 on realtime websocket dials. The PR documentation says Azure realtime requires a realtime-capable API version and that the default may be too old, so existing Azure configs can now fail with an opaque upstream websocket error instead of a clear local configuration error.
|
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/providers/zai/realtime.go`:
- Around line 17-33: The RealtimeTarget method validates req.Model by trimming
it but then passes the untrimmed req.Model to the OpenAIRealtimeURL function.
Store the trimmed model value in a variable once at the beginning of the method
(before the validation check), then use this trimmed variable consistently in
both the empty check and when calling OpenAIRealtimeURL to avoid validating one
value while using another.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 4fe69372-98b7-4eee-9a21-761cc5f43047
📒 Files selected for processing (8)
CLAUDE.mdinternal/providers/azure/azure.gointernal/providers/azure/realtime.gointernal/providers/azure/realtime_test.gointernal/providers/openai/chat_compatible.gointernal/providers/zai/realtime.gointernal/providers/zai/realtime_test.gointernal/providers/zai/zai.go
Address PR #396 review: - Z.ai (Greptile P1): GLM-Realtime lives at a fixed /api/paas/v4/realtime path, but appending /realtime to the configured chat base broke the Coding Plan base (/api/coding/paas/v4 -> .../coding/paas/v4/realtime). Derive the realtime URL from the host with the pinned path instead (mirrors Bailian). Also trims the model once (CodeRabbit nitpick). - Azure (Greptile P2): strip a trailing /openai or /openai/v1 from the resource root before appending /openai/realtime, so a base already rooted at /openai no longer yields /openai/openai/realtime. Regression tests added for both (Coding Plan base; /openai[/v1] base). Skipped Greptile P2 "reject old Azure API version": hardcoding which api-versions support realtime is brittle (would break at GA); it is documented, and a failed dial surfaces as a clear 502 rather than a silent error. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
internal/providers/azure/realtime_test.go (1)
12-84: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick winConsider adding test coverage for error paths in
realtimeURL.The Codecov report shows 71.42% coverage for
azure/realtime.gowith 6 missing lines, likely in the error paths ofrealtimeURL(invalid URL parse, unsupported scheme, empty host). Adding 2-3 test cases for these scenarios would improve coverage and ensure error handling behaves correctly.Example scenarios to cover:
- Invalid base URL that fails
url.Parse- Base URL with an unsupported scheme (e.g.,
ftp://...)- Base URL with missing host
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/providers/azure/realtime_test.go` around lines 12 - 84, Add test coverage for the error paths in the realtimeURL function that are currently not covered. Create 3 new test functions (similar in structure to TestRealtimeTargetMissingModel) that test: (1) an invalid base URL that fails url.Parse, (2) a base URL with an unsupported scheme like ftp://, and (3) a base URL with a missing host. Each test should call RealtimeTarget with the problematic configuration and verify that it returns a non-nil error. These tests will exercise the error handling paths in realtimeURL that are currently uncovered.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/providers/zai/realtime_test.go`:
- Line 58: The url.Parse call on line 58 ignores the error return value by using
the blank identifier, which creates an inconsistency with the error-checking
pattern shown earlier in TestRealtimeTarget (lines 24-27) and leaves the code
vulnerable to a panic when accessing u.Path if parsing fails. Capture and check
the error returned from url.Parse(target.URL), and handle it appropriately in
the same manner as the earlier error checks, ensuring that execution only
proceeds if the parse operation succeeds.
---
Outside diff comments:
In `@internal/providers/azure/realtime_test.go`:
- Around line 12-84: Add test coverage for the error paths in the realtimeURL
function that are currently not covered. Create 3 new test functions (similar in
structure to TestRealtimeTargetMissingModel) that test: (1) an invalid base URL
that fails url.Parse, (2) a base URL with an unsupported scheme like ftp://, and
(3) a base URL with a missing host. Each test should call RealtimeTarget with
the problematic configuration and verify that it returns a non-nil error. These
tests will exercise the error handling paths in realtimeURL that are currently
uncovered.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 87ec477c-bacf-4fe7-9226-b8e219854b33
📒 Files selected for processing (4)
internal/providers/azure/realtime.gointernal/providers/azure/realtime_test.gointernal/providers/zai/realtime.gointernal/providers/zai/realtime_test.go
Address CodeRabbit: the new Coding-Plan and /openai-strip tests ignored the url.Parse error and would panic on a nil *url.URL if parsing failed. Check the error consistently with the other realtime tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Extends realtime (speech-to-speech) support — shipped for OpenAI, xAI, and Bailian in #394 — to two more providers that speak OpenAI's realtime event schema, so they drop into the existing transparent websocket proxy with just a provider
RealtimeTarget:…/api/paas/v4/realtime) and core events mirror OpenAI's. Derives via the sharedproviders.OpenAIRealtimeURL, honors theZAI_BASE_URLregion (api.z.aivsopen.bigmodel.cn), Bearer auth. AddsChatCompatible.GetBaseURLso the base URL is read live (no staleable copy — same root-cause approach as feat(realtime): add OpenAI Realtime voice via websocket reverse proxy #394's late fix).wss://<resource>/openai/realtime?api-version=…&deployment=…from the resource root and authenticates with theapi-keyheader (not Bearer). The model selects the Azure deployment.Both get compile-time
core.RealtimeProviderassertions and unit tests (URL / auth / region / no-key / missing-model).Deliberately skipped
task_start/task_continue/task_finish) and there's no published OpenAI-compatible conversational realtime endpoint. A transparent relay can't be confirmed, so shipping it would be a guess.Provider notes (documented in CLAUDE.md)
/modelsdiscovery → setXAI_MODELS; xAI bills per-minute (no token usage).AZURE_API_VERSION(default may be too old).Testing
make test-race+make lintgreen.🤖 Generated with Claude Code
Summary by CodeRabbit