Zero-copy EGL bridge for Unity-surface video fallback

[oddgames-david]

Context

1.7.31 added a Unity-surface video fallback for when the user denies Android MediaProjection consent. 1.7.32 moved the RGBA→NV12 conversion onto the GPU via a compute shader (Resources/BugpunchRgbaToNv12.compute). That brings the capture path down to ~0.5% CPU + a cheap GPU blit + compute dispatch + AsyncGPUReadback of the packed NV12 bytes.

The remaining overhead is the readback itself — a CPU-side memcpy of the whole NV12 buffer plus JNI byte-array marshalling per frame. It works, but it isn't ideal.

What to build

Skip the readback entirely by rendering directly onto MediaCodec's input Surface via an EGL bridge. Same pattern Android's own screenrecord binary uses, same pattern Grafika's EncodeAndMuxTest demonstrates.

• Java side already exposes BugpunchRecorder.getInputSurface(). Add a native variant that operates on a MediaCodec.createInputSurface() we own, dedicated to buffer-mode capture (so we don't collide with the projection path).
• New native plugin (package/Plugins/Android/BugpunchGlBridge.cpp or NDK module in android-src/cpp/):
  • ANativeWindow_fromSurface() to wrap the Java Surface.
  • Create an EGL window surface over that ANW, sharing Unity's render-thread EGL context so we can sample Unity's RenderTexture as a GL_TEXTURE_2D.
  • Expose a C function registered through IUnityGraphicsV2 that fires on Unity's render thread: eglMakeCurrent(encSurf) → glBlitFramebuffer (or a textured-quad shader) from Unity's RT → eglPresentationTimeANDROID(ptsNs) → eglSwapBuffers.
• Unity side (BugpunchSurfaceRecorder.cs): replace the readback path with a GL.IssuePluginEvent(nativeCallback, frameId) every capture tick. Drop the ComputeBuffer + async readback when this path is available.
• Keep the compute-shader + readback path as a fallback for devices / graphics backends where the bridge can't attach.

Expected benefit

• 0% CPU overhead on the capture path (vs ~0.5% today).
• Lower end-to-end latency: the readback introduces ~2 frames of delay.
• Can comfortably go to 30 fps without cost — today 15 fps is the sweet spot for readback stall avoidance.

Blockers / caveats

• Vulkan: Unity 6 defaults to Vulkan on new Android builds. EGL interop is GL-only. For Vulkan we'd need VkImage ↔ AHardwareBuffer interop, which is a different API surface. Simplest first cut: detect backend and gate this path behind GL; fall back to compute+readback on Vulkan.
• EGL config match: Unity's context vs MediaCodec's surface must agree on pixel format. Usually RGBA8888 on both but some vendors expose odd configs.
• Lifecycle: handling app pause/resume, surface invalidation, MediaCodec restart, recovery from eglSwapBuffers failures. None insurmountable but adds code paths that want device testing.
• Size: ~300–500 lines of C++ plus the Unity plumbing. Needs real-device validation on a range of GPUs (Adreno, Mali, Xclipse, PowerVR) before shipping.

Priority

Not urgent. The compute-shader path covers the real-world cost today. Worth picking up if we start targeting 30+ fps recording, or if profiling on a low-end device shows the readback as a bottleneck.

Related

• 1.7.31 — Unity-surface fallback introduced
• 1.7.32 — GPU RGBA→NV12 (current state)