SurfaceFlinger renderer: RGBA_8888 scan-out slots under the daemon's layout rule

Merged
UT
8d ago
+92-12

Description

The SurfaceFlinger display renderer hands each X window's CPU swapchain slot to surfaceflinger as a BGRA_8888 AHardwareBuffer. Android 16 on a Skia GLES render engine cannot wrap one and aborts surfaceflinger. The gamescope daemon already answers this with an RGBA_8888 pool chosen by a device rule (SDK 36 and a GLES render engine, GNDRM_SCANOUT_LAYOUT override); the X11 path had no equivalent.

Report: RedMagic Astra (nubia NP05J, Android 16), GameNative 1.4.0 (24). Play works since the daemon's RGBA layout; "Open container" (an X11 session with the SurfaceFlinger renderer) restarts the whole UI: Unable to generate SkImage ... GrGLTextureInfo fFormat 32856 colorType 6 on a 1280x720 buffer, 30-40 ms after the wfm.exe window maps.

The renderer's slots now follow the daemon's rule, read through GameNativeDrm.nativeScanoutLayout() (valid without a session) before the first drawable, with the container's GNDRM_SCANOUT_LAYOUT override honoured the same way:

  • AHBImage: the three scan-out slots are RGBA_8888 under the rule; the X server's B,G,R,A bytes are swapped on the CPU copy into the slot. The front buffer the X server writes into stays BGRA (never handed to surfaceflinger).
  • ASurfaceRenderer: an RGBA slot goes to surfaceflinger as is, skipping the BGRA-to-RGBA blit, which swaps an RGBA-declared source and would undo the copy.
  • XServerScreen: resolves the layout for X11 sessions and logs it (X11 scan-out layout rgba (device rule)).
  • libahbimage.so rebuilt from the changed ahbimage.c with NDK 27.3.13750724 (the asurfacerenderer libs ship prebuilt).

BGRA devices are unchanged: same formats, same copy, same blit.

Recording

Not applicable: the reporter's device is the only one that reproduces it (Android 16 + Skia GLES). Not yet confirmed on it.

Type of Change

  • Bug fix
  • Performance / stability improvement
  • Compatibility improvements
  • Other (requires prior approval)

Checklist

  • If I have access to #code-changes, I have discussed this change there and it has been green-lighted. If I do not have access, I have still provided clear context in this PR. If I skip both, I accept that this change may face delays in review, may not be reviewed at all, or may be closed.
  • This change aligns with the current project scope (core functionality, stability, or performance). If not, it has been explicitly approved beforehand.
  • I have attached a recording of the change.
  • I have read and agree to the contribution guidelines in CONTRIBUTING.md.

Summary by cubic

Fixes a SurfaceFlinger abort when the X11 path hands BGRA_8888 buffers to surfaceflinger on Android 16 devices with a Skia GLES render engine. The renderer's CPU swapchain slots now follow the same RGBA_8888 scan-out layout rule the gamescope daemon applies, resolved before the first drawable from the container's GNDRM_SCANOUT_LAYOUT override or the device rule.

  • RGBA slots get the X server's B,G,R,A bytes swapped on the CPU copy into the slot; the front buffer stays BGRA.
  • An RGBA slot goes to surfaceflinger as is, skipping the BGRA-to-RGBA blit that would swap it back.
  • BGRA devices are unchanged: same formats, copy, and blit.
  • libahbimage.so rebuilt from the modified C source with NDK 27.3.

Summary by CodeRabbit

  • Bug Fixes
    • Improved color accuracy when displaying X11 content by handling RGBA and BGRA scanout formats correctly.
    • Fixed red and blue channel conversion in SurfaceFlinger compatibility mode to avoid unnecessary or missing color adjustments.
    • Added fallback behavior when scanout format settings are unavailable, helping display continue with a default format.