Skip to content

Add support capturing QtQuick windows backed by Qt Rhi - #1111

Merged
Waqar144 merged 19 commits into
masterfrom
work/josh/quick-rhi-support
Nov 16, 2026
Merged

Waqar144 merged 19 commits into
masterfrom
work/josh/quick-rhi-support

Conversation

@redstrate

@redstrate redstrate commented Feb 10, 2026 •

Copy link
Copy Markdown
Contributor

This is an interesting QtQuick backend to support because of some of the inherent design limitations we have to work around. But we are now able to capture and draw target decorations for these windows backed by Rhi now.

The first problem we have to tackle is actually capturing the window contents, but fortunately Qt already has public API to do this. They don't provide a mapping from their bespoke Rhi texture format to QImage unfortunately, but I added a user-visible error message so it's easier for people to submit bugreports for any missing cases.

I tried to optimize the readback as best I could, and the performance is about on-par to the OpenGL screen grabber. (I didn't actually benchmark, but it looks good enough to my own eyes.)

Another tricky problem to tackle is drawing the remote target decorations, because we previously assumed we always had a QPainter to draw in. (For context, the client's viewport is drawn in QPainter and in the OpenGL there's QOpenGLPaintDevice available.) Not only that, but we can't exactly tell when to let Qt readback the swapchain image. The pre-existing screengrabbers grabbed the backbuffer before we drew target decorations so we could draw them client-side.

For the drawing part I don't think there's anything we could do save for rewriting it in QtRHI, so we upload it into a texture and draw it onto the backbuffer. Yes it is "slow" but it's more than good enough than it not working at all.

For the target decoration part that's not going to be easily possible, so I decided to not do it! Instead, we let ourselves capture the full window contents including the decorations we drew. We then skip drawing decorations client-side, and it's practically the same experience save for anything drawn outside the window. That limitation is going to be planned to be addressed in a future patch.

TODO:

  • Separate out the expose part of this patch - it's unrelated. (Don't show an image in the remote view for unexposed QtQuick windows #1113)
  • Address the TODO/FIXME still in the code
  • Separate out the documentation improvements, that's just making the diff bigger for no reason (Small documentation improvements #1112)
  • Check to see how hard it would be to address the clipping issue for remote target decorations (I was struggling with the QPainter ordering here) but I don't see that as a blocker
  • Fix the issue where clicking in the viewport is delayed, despite other controls like the element selector list being instantaneous when using remote target decorations (EDIT: Nevermind it isn't completely foolproof for those controls, it looks like it drifts in timing)
  • Test on QQuickWidget applications

NOTE:
Vulkan capturing may be broken on your system without https://codereview.qt-project.org/c/qt/qtbase/+/711621

@redstrate redstrate self-assigned this Feb 10, 2026
@redstrate
redstrate force-pushed the work/josh/quick-rhi-support branch 6 times, most recently from 63d5887 to 37b93cf Compare February 13, 2026 19:27
@redstrate
redstrate force-pushed the work/josh/quick-rhi-support branch from 37b93cf to a00f3fc Compare February 23, 2026 12:36
@redstrate

Copy link
Copy Markdown
Contributor Author

Note that for the QRhi device loss case, it's super underspecified (what does it mean to "release resources" Qt?!) but I think I have it nailed down without testing. I made sure to check if the QRhi is available before using it - as it could be null - and sceneGraphInvalidated should be a good indicator for device loss.

I did spend a few days trying to figure out how to simulate device loss locally to no avail, so :-(

@redstrate
redstrate force-pushed the work/josh/quick-rhi-support branch from a00f3fc to ea314a0 Compare February 23, 2026 15:13
@redstrate
redstrate force-pushed the work/josh/quick-rhi-support branch 5 times, most recently from f146e23 to ab61ec8 Compare March 20, 2026 18:40
@redstrate

redstrate commented Mar 30, 2026 •

Copy link
Copy Markdown
Contributor Author

After another day of hacking I'm still being humbled by the viewport delay issue, but its otherwise ready for review.

@redstrate
redstrate marked this pull request as ready for review March 30, 2026 19:54
@redstrate
redstrate force-pushed the work/josh/quick-rhi-support branch from ab61ec8 to 0c8e04e Compare March 30, 2026 19:55
This is an interesting QtQuick backend to support because of some of the
inherent design limitations we have to work around. But we are now able
to capture and draw target decorations for these windows backed by Rhi
now.

The first problem we have to tackle is actually capturing the window
contents, but fortunately Qt already has public API to do this. They
don't provide a mapping from their bespoke Rhi texture format to QImage
unfortunately, but I added a user-visible error message so it's easier
for people to submit bugreports for any missing cases.

I tried to optimize the readback as best I could, and the performance is
about on-par to the OpenGL screen grabber. (I didn't actually benchmark,
but it looks good enough to my own eyes.)

Another tricky problem to tackle is drawing the remote target
decorations, because we previously assumed we always had a QPainter to
draw in. (For context, the client's viewport is drawn in QPainter and in
the OpenGL there's QOpenGLPaintDevice available.) Not only that, but we
can't exactly tell *when* to let Qt readback the swapchain image. The
pre-existing screengrabbers grabbed the backbuffer before we drew target
decorations so we could draw them client-side.

For the drawing part I don't think there's anything we could do save for
rewriting it in QtRHI, so we upload it into a texture and draw it onto
the backbuffer. Yes it is "slow" but it's more than good enough than it
not working at all.

For the target decoration part that's not going to be easily possible,
so I decided to not do it! Instead, we let ourselves capture the full
window contents *including* the decorations we drew. We then skip
drawing decorations client-side, and it's practically the same
experience save for anything drawn outside the window. That limitation
is going to be planned to be addressed in a future patch.
@Waqar144
Waqar144 force-pushed the work/josh/quick-rhi-support branch from 0c8e04e to eb0681f Compare November 13, 2026 21:24
QRhi reuses the readback result's buffer for the next readback, while
the grabbed image is queued to the GUI thread and kept by the remote
view. Keep a reference to the buffer alive as long as the image.
The readback result and the decoration pipeline, SRB, texture and
sampler were never freed. QRhi may still hold the readback result when
the grabber goes away, so it now lives in a separately owned state that
the completion callback deletes in that case. The RHI resources are
released on the render thread, when the scene graph is invalidated or
the grabber is destroyed.
The decoration texture and SRB were recreated from updateOverlay(), on
the GUI thread, while the render thread may be using them, and the old
ones were leaked. Recreate them before rendering instead, and release
the old ones with deleteLater() as frames in flight may still use them.
Don't dereference a null command buffer, and release a resource update
batch that can't be submitted instead of leaking it from QRhi's pool.
QQuickWidget was found through the _q_parentWidget property, which only
existed in Qt 6.10 and 6.11, so it was never grabbed on other versions.
Take the render pass descriptor and the readback texture from the
window's redirected render target instead when there is no swapchain.
This also covers custom QQuickRenderControl users and drops the
QQuickWidgetPrivate dependency. Wait with creating the pipeline until
the render target is set.
@Waqar144
Waqar144 force-pushed the work/josh/quick-rhi-support branch 3 times, most recently from 9ae40ba to db587ea Compare November 14, 2026 19:12
Warn once, skip the decorations and retry only after the scene graph
was invalidated. Grabbing keeps working.
Waqar144 and others added 8 commits November 15, 2026 00:52
sendRenderedScene() also runs for the empty frame sent while ~QWindow
hides the window, when the QQuickWindow part is already destroyed, and
crashed calling rendererInterface(). Carry the graphics API in the
grabbed frame instead.
The frame's wire format changed when the graphics API was added to it.
Use a proper CMake version check, put the forward declarations inside
the Qt namespace, drop a duplicate include and connect the signals in
the order QQuickWindow emits them instead of leaving a TODO.
The decoration texture had the window's logical size and was stretched
over the physical pixels with nearest filtering, so the decorations
looked blocky with a device pixel ratio above 1. Paint them at device
pixels, as the OpenGL grabber does.
The vertex shader generated the quad from gl_VertexIndex, which GLSL 120
and GLSL ES 100 don't have. On OpenGL 2.1 and OpenGL ES 2 contexts, e.g.
the software OpenGL on the Windows CI, creating the decoration pipeline
failed and no decorations were drawn.
Grabs a QQuickView, a resized view, a QQuickWidget and a window driven
by a custom QQuickRenderControl, checks that server side decorations
end up in the frame and go away again, and destroys a window right
after a grab. Every test fails when QRhi reports unreleased resources.

The tests need a GPU and skip themselves when Qt Quick doesn't render
through QRhi, e.g. with the software backend used on the Linux CI. On
Linux they also run with Vulkan.
@Waqar144
Waqar144 force-pushed the work/josh/quick-rhi-support branch from e50e09c to 973f46e Compare November 14, 2026 19:54
The decoration quad mapped NDC y = -1 to the top row of the decoration
image. That is only the top on backends where Y points down in NDC
(Vulkan); on OpenGL, Direct3D and Metal the decorations were mirrored
vertically. Flip the texture coordinates when QRhi::isYUpInNDC().

The test now checks that the decorations cover the selected item and
nothing else.
The pipeline always used one sample, also for multisampled swapchains
and render targets. That is invalid on Vulkan and Metal; on Vulkan the
decorations came out at the wrong size.
Vulkan (and other backends) complete a readback only when its frame
slot comes round again, a few frames later. In a static scene those
frames never come, so a single change was never delivered to the
client.

The grabber also signalled that it was ready again before the
readback completed, so the next grab could queue a second readback
into the same QRhiReadbackResult while the first was still pending.

Call QRhi::finish() after queuing the readback, as
QQuickWindow::grabWindow() does, so the frame is delivered before the
grabber reports ready again.
Format_BGR30 ignores the two alpha bits; Format_A2BGR30_Premultiplied
has the same layout and keeps them.
@Waqar144
Waqar144 merged commit 235a337 into master Nov 16, 2026
14 checks passed
@Waqar144
Waqar144 deleted the work/josh/quick-rhi-support branch November 16, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants