WSI layer: FIFO presents don't block when SteamVR's vrclient created a VkInstance first in the process #2539

Is there an existing issue for this?

• I have searched the existing issues

Are you using any gamescope patches or a forked version of gamescope?

• The issue occurs on upstream gamescope without any modifications

Current Behavior

A Vulkan app's FIFO swapchain (X11 window, one present per frame) stops blocking under the WSI layer when SteamVR's vrclient has created its own VkInstance (steamvr_vrclient_interop) in the app's process before the app's own. Here that happens because the app starts OpenXR through SteamVR. The headset isn't worn and audio is off, so the window's present is the app's only pacing, and the app runs several times too fast. It happens in the normal SteamOS session (OpenVR backend, 120 Hz) and under --backend headless (60 Hz).

App play time counts the app's 60 fps content frames, so in F it reads 2x because presents block at the session's 120 Hz; the present rate is the measure that matters.

The layer gives Mesa MAILBOX and does FIFO itself (gamescope_swapchain_set_present_mode), so it looks like the layer's own FIFO stops pacing; forcing FIFO in Mesa works around it.

A second instance alone is not enough. A minimal program that first creates an instance named steamvr_vrclient_interop (with or without a device and a thread of its own), then presents FIFO, blocks at 60 Hz under the same 3.16.30 headless gamescope on the Frame, directly and inside Steam Linux Runtime 4, with the layer inserted for both instances. Whatever else SteamVR's client does in the process seems to be needed.

Steps To Reproduce

1. Steam Frame, SteamVR running, headset not worn. Run RetroArch with video_driver = "vulkan", video_openxr_enable = "true", audio_enable = "false" and vsync on (FIFO), with a Vulkan core (Flycast here), either:
   • from a non-Steam shortcut in the normal SteamOS session (OpenVR backend), launching SteamLinuxRuntime_4-arm64/run -- retroarch --config retroarch.cfg -L flycast_libretro.so game.chd, or
   • PRESSURE_VESSEL_FILESYSTEMS_RO=/opt/steamvr gamescope --backend headless -W 1280 -H 720 -- SteamLinuxRuntime_4-arm64/run -- retroarch --config retroarch.cfg -L flycast_libretro.so game.chd
2. After ~75 s, compare the wall time with RetroArch's "Content ran for" line, or read its present rate from the log.
3. Repeat with the env changes from the table.

Hardware information

- Distro: SteamOS 0.6.0 (Steam Frame), build 20261103.6153902
- CPU: 8 cores, Cortex-X4 + Cortex-A720 + Cortex-A520
- GPU: Adreno 750 (Turnip)
- Driver Version: Mesa 26.3.0-devel (git-6a15f899c8); Vulkan loader 1.4.309

Software information

- Desktop environment: SteamOS session (and none, for headless)
- Session type: gamescope OpenVR backend (SteamOS session) and `--backend headless`, app on Xwayland
- Gamescope version: 3.16.30 (SteamOS package 3.16.30-4), layer `VK_LAYER_FROG_gamescope_wsi_aarch64`
- Gamescope launch command(s): the SteamOS session's own; and see Steps To Reproduce for headless. SteamVR 2.18.4, Steam Linux Runtime 4.0 arm64

Which gamescope backends have the issue you are reporting?

• Wayland (default for nested gamescope)
• DRM (default for embedded gamescope, i.e. gamescope-session)
• SDL
• OpenVR

Logging, screenshots, or anything else

Also --backend headless, which isn't in the backend list.

Layer output in run A (VK_LOADER_DEBUG=layer); the layer is inserted for both instances, and run E shows the same pair:

[Gamescope WSI] Application info:
  pApplicationName: steamvr_vrclient_interop
[Gamescope WSI] Executable name: retroarch
ATTENTION: default value of option vk_wsi_force_swapchain_to_current_extent overridden by environment.
[Gamescope WSI] Application info:
  pApplicationName: Flycast
[Gamescope WSI] Creating swapchain for xid: 0x400002 - oldSwapchain: (nil) - provided minImageCount: 3 - minImageCount: 3 - format: VK_FORMAT_B8G8R8A8_UNORM - colorspace: VK_COLOR_SPACE_SRGB_NONLINEAR_KHR - flip: true
[Gamescope WSI] Swapchain received new refresh cycle: 16.67ms

Run D shows only the Flycast instance and no ATTENTION line; its swapchain lines are the same.

Not tested: gamescope newer than 3.16.30 (a minimal two-instance program doesn't reproduce on 3.17.0 on desktop RADV either, but that doesn't involve SteamVR).

Full stderr (loader + layer) and RetroArch logs for each run are attached; the letters match the table.
A-headless-openxr-on.txt
B-headless-wsi-layer-off.txt
C-headless-mesa-fifo.txt
D-headless-openxr-off.txt
E-openvr-session-openxr-on.txt
F-openvr-session-mesa-fifo.txt