Repository navigation
Fix black/frozen desktop video stream and make render failures visible - #2929
Merged
Merged
Conversation
The scheduler could latch a preroll hold it could never release, stopping all presentation while frames kept arriving and being dropped. The canvas froze (or stayed black) while socket, decoder and encoder stayed healthy. Two causes, both fixed at source: - Buffer depth was derived from median socket ARRIVAL spacing. After any stall the catch-up burst collapses that to well under 1ms, so floor(MAX_DELAY_MS/median) inflated the depth target from ~7 frames to ~150 - a 2.5s buffer against a 120ms cap. Cadence now comes from PTS deltas, which bursts cannot distort. - Preroll ended only when queue length EXCEEDED the target, unreachable once the target hit the 30-frame queue cap. It now ends at >= target, and the target is capped below queue capacity so the condition is structurally always satisfiable. Observed before: targetFrames=152, prerolling=true, queueLen=30 pinned, 0 frames presented in 6s, framesDropped climbing ~44/s. Observed after: 56.4fps presented, targetFrames=1, depth 50ms, queueLen=4. Spec-Ref: helix-specs@917c35d06:002546_debug-and-fix-the
…ck video
A lost GPU context made every GL call a silent no-op: decode kept
succeeding, videoStarted had already fired, stats stayed green, and the
canvas was black indefinitely with no error. Safari hits this routinely.
- WebGLVideoRenderer now handles webglcontextlost/webglcontextrestored,
reallocating GL objects on restore. draw() returns false while lost so a
non-paint is never counted as a paint. The WEBGL_lose_context handle is
captured while the context is alive (getExtension returns null once lost,
which silently defeated restore requests).
- dispose() no longer calls loseContext(): a canvas hands out one context
per type for its lifetime, so force-losing it bricked the element. A
second setCanvas() on the same canvas then got the dead context back,
failed, and fell through to getContext('2d') which returns null - leaving
both render targets null and every frame silently dropped.
- setCanvas() reuses a live render target instead of tearing it down, and
close() no longer destroys it (close() also runs for bitrate/mode
switches; renderer lifetime belongs to the canvas, not the socket).
- Deleted the broken 2D fallback on the WebKit branch.
- Render watchdog: socket open + decoding but nothing painted for 5s emits
renderStalled, and the viewer shows a real message with Retry. Nothing
covered this window before - videoStarted fires from the decode path.
- An unrestorable context is escaped by remounting the canvas element,
which is the only way to obtain a working context again.
- Retry budget is refunded on first PAINT rather than first decode, and a
deferred-while-hidden reconnect self-heals if visibilitychange is missed.
Spec-Ref: helix-specs@ccaa1126f:002546_debug-and-fix-the
Spec-Ref: helix-specs@ccaa1126f:002546_debug-and-fix-the
Spec-Ref: helix-specs@ccaa1126f:002546_debug-and-fix-the
The deferral was only lifted by the visibilitychange listener, but the heartbeat that would otherwise re-check is stopped in onClose - so if that one event was missed the stream stayed dead with no way back. Verified: hidden + disconnect defers, then becoming visible WITHOUT dispatching visibilitychange now reconnects (it did not before). Spec-Ref: helix-specs@7633544ba:002546_debug-and-fix-the
Spec-Ref: helix-specs@7633544ba:002546_debug-and-fix-the
Spec-Ref: helix-specs@a11df3244:002546_debug-and-fix-the
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Luke hit a black video stream on Safari with a provably healthy server pipeline
(60 fps NVENC,
pipeline_dropped=0, no backpressure, one subscribed client, proxysocket never closed) and no error message for 7 minutes.
The reported incident was not caused by the two Jul 29 streaming commits, and not
by WebGL context loss. It was a deadlock in
PlayoutScheduler, reproduced live inthe inner Helix. A second, independent silent-black mode (no WebGL context-loss
handling) was also found and fixed, along with the reason either one was invisible.
Full write-up with observed values:
design/2026-08-04-black-video-stream-regression.md.Root cause
Presentation was gated on a preroll flag cleared only by
q.length > target, whilepush()caps the queue atMAX_QUEUE = 30. OncetargetFrames >= 30that test isunsatisfiable, so nothing was ever presented again and every frame was dropped at
the cap — canvas frozen, socket and decoder perfectly healthy.
targetFramesreached 152 because the depth cap wasfloor(MAX_DELAY_MS / median)where
medianis median socket inter-arrival spacing, not frame cadence. Anystall produces a catch-up burst that collapses arrival spacing below 1 ms, turning a
"120 ms max buffer" rule into a 152-frame (2.5 s) buffer. Peak-hold latched it
instantly; decay was 1 frame per 2 s, i.e. >4 minutes to recover.
Captured live while frozen:
{"targetFrames":152,"prerolling":true,"queueLen":30,"MAX_QUEUE":30, "depthMs":2508,"canEverClearPreroll":false}Introduced in
bcf7a34f9(2026-06-17), not the Jul 29 commits. Engine-independent —reproduced in Chromium.
Changes
playout-scheduler.ts— cadence now measured from PTS deltas (immune to bursts);depth target capped below
MAX_QUEUEso the preroll test is structurally reachable;preroll ends at
>= target(was>, also costing a needless frame of latency).webgl-video-renderer.ts— handlewebglcontextlost/webglcontextrestoredandreallocate GL state on restore;
draw()returnsfalsewhile lost so a non-paint isnever counted as a paint; capture the
WEBGL_lose_contexthandle while the context isalive (
getExtensionreturns null once lost, which silently defeated restore);dispose()no longer callsloseContext()— it permanently bricked the canvas.websocket-stream.ts—setCanvas()reuses a live render target instead oftearing it down and re-acquiring a dead context;
close()no longer destroys rendertargets (it also runs for bitrate/mode switches — renderer lifetime belongs to the
canvas, not the socket); deleted the broken 2D fallback on the WebKit branch
(
getContext("2d")returns null on a WebGL-bound canvas, so it silently dropped everyframe); added a render watchdog; retry budget refunded on first paint, not first
decode; a deferred-while-hidden reconnect now polls
document.hidden.DesktopStreamViewer.tsx— surfacerenderStalledas a real message with Retry,reusing the existing
gaveUperror surface; remount the<canvas>when its context isunrecoverable, which is the only way to obtain a working context again.
Testing
End-to-end in the inner Helix against a real spec-task desktop with a live 60 fps
stream, measured in actual paints (
framesDecodedcounts decode attempts and keptclimbing at 60/s while nothing was on screen — it is not a health signal).
setCanvas(), same elementgaveUplatch, video livevisibilitychangeNOT tested: real Safari — none available in this Linux sandbox. The WebKit branch
was exercised under Chromium by stubbing
isAppleWebKit()totrue(reverted beforecommit) and driving loss with
WEBGL_lose_context. That verifies the logic andrecovery, not WebKit's own context-loss timing. The primary defect is
engine-independent and was reproduced directly.
A synchronous console
close()+reconnect()still yields 0 fps because the serverdedups the reused
client_id(Close received after close). That is not a viewer codepath; the real Retry path is verified above.
Screenshots
🔗 Open in Helix
📋 Spec:
🚀 Built with Helix