In-game Runtime API

[chyyran]

Similar to how the Steam overlay works, I want to be able to overlay a browser instance over the emulators Snowflake launches. However, I'm not familiar with hooking code, let alone attempting to render it over an arbitrary surface hooked into a game. If anyone is familiar with this, let me know, it'd be a big help.

Resources

Core hook entry: https://github.com/SnowflakePowered/sabinokaku
Hook impl: https://github.com/Reloaded-Project/Reloaded.Hooks
Example impl of hooking FFXIV: https://github.com/goatcorp/Dalamud
CEF hosting: https://github.com/Styr1x/Browsingway
Example impl: https://github.com/Reloaded-Project/Reloaded-II
In-process hosting: https://gitlab.com/kaminariss/nextui-plugin/
hooking input on linux: https://tronche.com/gui/x/xlib/event-handling/XSelectInput.html
sharing resources between dx11/dx12: https://stackoverflow.com/questions/52869111/sharing-id3d11buffer-and-id3d12resource
GL_EXT_memory_object_win32/GL_EXT_memory_object for OpenGL buffer sharing. May be possible to offprocess chrome, render CEF to texture then paint.
KMT: https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_win32_keyed_mutex.txt
https://www.intel.com/content/www/us/en/developer/articles/technical/sharing-surfaces-between-opencl-and-directx-11-on-intel-processor-graphics.html
https://github.com/flightlessmango/MangoHud/blob/master/src/vulkan.cpp#L1581
https://blog.techlab-xe.net/post-5439/
vk ex: https://github.com/scenerygraphics/scenery/blob/master/src/main/kotlin/graphics/scenery/controls/Hololens.kt#L232

Todo

• DirectX 11
• OpenGL
• DirectX 12 (xenia)
• Vulkan

DX9 support probably not worth maintaining.

[andres-asm]

https://github.com/spazzarama/Direct3DHook

[chyyran]

I've looked into that, and actually got a Bitmap rendering on top of an arbitrary RetroArch instance, but I couldn't get CEF to initialize properly and render to it.

[andres-asm]

what do you mean with CEF?

[chyyran]

Chromium Embedded Framework, like the Steam Overlay

…

On Tue, Jul 4, 2017, 12:27 AM Andrés ***@***.***> wrote: what do you mean with CEF? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#235 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA9EN7-rg1GAQ3NXMWxwGNdlU3JD4sCHks5sKb8egaJpZM4JEC2z> .

[chyyran]

To clarify: I need to be able to render the webpage through CEF on to the injected surface, and be able to pass input to the instance of CEF so that users can interact with the webpage. This will be used to build an ingame-UI framework of sorts, and be able to control launched applications through a socket interface/message passing/some other hacky method abstracted away.

The Steam Overlay is not built in CEF, but does allow rendering of a CEF instance when using the ingame web browser.

It doesn't have to be done in managed code (C#), but should be able to be invoked through some standard calling convention (so probably C).

[iongion]

Did you guys find a solution to this ?

[chyyran]

Haven't yet no, it hasn't been a priority for a while either. Although if I could get this working it would be really nice to have.

…

On Wed, Jan 10, 2018, 3:51 AM ionut stoica, ***@***.***> wrote: Did you guys find a solution to this ? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#235 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA9EN8xAVPnfoBIdNdbV_vPuPHGF3heOks5tJHocgaJpZM4JEC2z> .

[chyyran]

So.. after playing some multi-disc games I have come to the conclusion that an ergonomic API for this is rather necessary UX wise. There is no other way to properly swap games without requiring the user memorize some hotkeys and this is of course not acceptable. Interfacing with the emulator process can be done easily via window messaging but we need an overlay UX.

[chyyran]

Work in #836

[chyyran]

dropping this here

[chyyran]

Seems like while DX11 shared resources work fine, there's not much information in sharing ID3D11Texture2D with DirectX12 or OpenGL resources, which I need for interprocess shared resource. Ideal here would be to use either Vulkan or DX12 which can interop with DX11 natively, and OGL with Ext_mem_obj_w32.

Hosting CEF in-process is another option but I'd like to avoid that

[chyyran]

DX12 is trivial-ish since it can be done with the same loader. Vulkan is just weird so I'm probably going to have to build a mini-layer in https://github.com/SnowflakePowered/sabinokaku and inject CLR that way.