Cannot view local media files in video tag

[petegleeson]

Description
Hello 👋 I'm building an app that takes screen recordings and presents them to watch back. It's important that these recordings be high quality (e.g. 4k 30fps). I've picked WebView2 so I can use web technology for the UI with low latency access to recording files.

I cannot find a way to achieve this with a video element and WebView2. I've tried various different ways but I keep running into crashes, videos not playing or other deal breaking UX issues. I'll detail what I've tried and what I observe. Here is a repo that demonstrates the issues https://github.com/petegleeson/WebView2VideoExample

SetVirtualHostNameToFolderMapping

I was able to load videos successfully with this approach. A larger video (7 GB) took 9 seconds to load before I could start playing. Once loaded I could not seek the video (using the video controls or with JS). These were both deal breaking UX problems for my application so I parked the approach.

WebResourceRequested

I created a response with the local recording file as a stream in a web resource request handler. After I got the correct content response headers, the video loaded successfully and I was able to seek with the controls and JS. Unfortunately, I quickly ran into problems caused by streaming the whole file in the response. The whole file had to be downloaded before the video could be played. When I tried with the larger video file my app consistently crashed with Unhandled exception at 0x00007FFD59EE474C (KernelBase.dll) in WebView2VideoExample.exe: 0xE0000008 (parameters: 0x00000000C0000000). during the request.

The last thing I tried was to use 206 Partial Content response code and the Content-Range header to stream chunks of the file. To me this sounds like the right approach, but I've had the least success with this. The network tab shows only the chunk being downloaded but the video never loads.

My expectation is to be able to use a video tag and WebView2 to view high quality local video files. I've raised this as a bug because I think that this is a use-case that WebView2 caters for. Please take a look at the repo I prepared for this issue. Happy to provide any more information if needed, I'm very excited about the possibilities of C++ and WebView2 desktop apps.

Version
SDK: 10.0.20348.0
Runtime: Canary 1.0.1293.44
Framework: Win32, WinRT
OS: Win11

Repro Steps

• Clone https://github.com/petegleeson/WebView2VideoExample and run the solution

Screenshots
Screen shot of the demo:

Hi @petegleeson , first of thanks for the explanatory sample to help debug this! I think returning partial content response to WebResourceRequested is the right approach here as the response stream cannot be too large. The reason why it wasn't working for you is due to the way WebView2 reads the response stream that the app provides. In your sample, every time your sample handles the WebResourceRequested event, it gives the WebView2 a stream of the whole file. WebView2 does not look at the range headers that's set in the response and merely attempts to read the whole stream that the app gives it (after rewinding the stream) and tries to use that as the response data. As a result, it ends up reading the full file, which is again too large for it to handle (and data read is wrong because WebView2 also rewound stream to begin). To fix this, instead of creating a stream on the whole file, you can pass a smaller stream that contains the part of the file that is requested by the WebView2. Video seek also works in this case as the WebView2 will only request and buffer the part of video that the user seeks to. The following code fixes it for me:

#include "pch.h"

#include <wil/com.h>
#include <wchar.h>
#include <sstream>
#include <regex>
#include "WebView2.h"
#include "WebView2EnvironmentOptions.h"


namespace winrt {
	using namespace Windows::Foundation;
}

static wil::com_ptr<ICoreWebView2Controller> webviewController;
static winrt::com_ptr<ICoreWebView2> webview;


class MediaFileStream : public winrt::implements<MediaFileStream, IStream> {
public:
	MediaFileStream(LPCWSTR path, ULONG begin, ULONG end) {
		_hFile = CreateFile(path, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
		_begin.QuadPart = begin;
		_size.QuadPart = end - begin;
	}

	~MediaFileStream() {
		CloseHandle(_hFile);
	}

	virtual HRESULT __stdcall Read(void* pv, ULONG cb, ULONG* pcbRead) override
	{
		auto bytesRemaining = _size.QuadPart - _bytesRead.QuadPart;
		auto bytesToRead = cb < bytesRemaining ? cb : bytesRemaining;
		BOOL rc = ReadFile(_hFile, pv, bytesToRead, pcbRead, NULL);
		_bytesRead.QuadPart += *pcbRead;
		return (rc) ? S_OK : HRESULT_FROM_WIN32(GetLastError());
	}
	virtual HRESULT __stdcall Write(const void* pv, ULONG cb, ULONG* pcbWritten) override
	{
		return E_NOTIMPL;
	}
	virtual HRESULT __stdcall Seek(LARGE_INTEGER dlibMove, DWORD dwOrigin, ULARGE_INTEGER* plibNewPosition) override
	{
		DWORD dwMoveMethod;

		switch (dwOrigin)
		{
		case STREAM_SEEK_SET:
			dwMoveMethod = FILE_BEGIN;
			break;
		case STREAM_SEEK_CUR:
			dwMoveMethod = FILE_CURRENT;
			break;
		case STREAM_SEEK_END:
			dwMoveMethod = FILE_END;
			break;
		default:
			return STG_E_INVALIDFUNCTION;
			break;
		}
		LARGE_INTEGER setLoc = { 0 };
		setLoc.QuadPart += _begin.QuadPart;
		setLoc.QuadPart += dlibMove.QuadPart;
		if (SetFilePointerEx(_hFile, setLoc, (PLARGE_INTEGER)plibNewPosition, dwMoveMethod) == 0)
			return HRESULT_FROM_WIN32(GetLastError());
		_bytesRead.QuadPart = 0;
		return S_OK;
	}
	virtual HRESULT __stdcall SetSize(ULARGE_INTEGER libNewSize) override
	{
		_size = libNewSize;
		return S_OK;
	}
	virtual HRESULT __stdcall CopyTo(IStream* pstm, ULARGE_INTEGER cb, ULARGE_INTEGER* pcbRead, ULARGE_INTEGER* pcbWritten) override
	{
		return E_NOTIMPL;
	}
	virtual HRESULT __stdcall Commit(DWORD grfCommitFlags) override
	{
		return E_NOTIMPL;
	}
	virtual HRESULT __stdcall Revert(void) override
	{
		return E_NOTIMPL;
	}
	virtual HRESULT __stdcall LockRegion(ULARGE_INTEGER libOffset, ULARGE_INTEGER cb, DWORD dwLockType) override
	{
		return E_NOTIMPL;
	}
	virtual HRESULT __stdcall UnlockRegion(ULARGE_INTEGER libOffset, ULARGE_INTEGER cb, DWORD dwLockType) override
	{
		return E_NOTIMPL;
	}
	virtual HRESULT __stdcall Stat(STATSTG* pstatstg, DWORD grfStatFlag) override
	{
		pstatstg->cbSize = _size;
		return S_OK;
	}
	virtual HRESULT __stdcall Clone(IStream** ppstm) override
	{
		return E_NOTIMPL;
	}

private:
	HANDLE _hFile = NULL;
	ULARGE_INTEGER _begin{};
	ULARGE_INTEGER _size{};
	ULARGE_INTEGER _bytesRead{};
	
};

LRESULT CALLBACK WndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam)
{
	switch (message)
	{
	case WM_SIZE:
		if (webviewController != nullptr) {
			RECT bounds;
			GetClientRect(hWnd, &bounds);
			webviewController->put_Bounds(bounds);
		};
		break;
	case WM_DESTROY:
		PostQuitMessage(0);
		break;
	default:
		return DefWindowProc(hWnd, message, wParam, lParam);
		break;
	}

	return 0;
}


int __stdcall wWinMain(HINSTANCE hInstance, HINSTANCE, LPWSTR, int nCmdShow)
{
    winrt::init_apartment(winrt::apartment_type::single_threaded);
	WNDCLASSEX wcex{};

	wcex.cbSize = sizeof(WNDCLASSEX);
	wcex.style = CS_HREDRAW | CS_VREDRAW;
	wcex.lpfnWndProc = WndProc;
	wcex.cbClsExtra = 0;
	wcex.cbWndExtra = 0;
	wcex.hInstance = hInstance;
	wcex.hIcon = LoadIcon(hInstance, IDI_APPLICATION);
	wcex.hCursor = LoadCursor(NULL, IDC_ARROW);
	wcex.hbrBackground = (HBRUSH)(COLOR_WINDOW + 1);
	wcex.lpszMenuName = NULL;
	wcex.lpszClassName = L"DesktopApp";
	wcex.hIconSm = LoadIcon(wcex.hInstance, IDI_APPLICATION);

	RegisterClassEx(&wcex);

	HWND hWnd = CreateWindow(
		L"DesktopApp",
		L"WebView2Video",
		WS_OVERLAPPEDWINDOW,
		CW_USEDEFAULT, CW_USEDEFAULT,
		1200, 900,
		NULL,
		NULL,
		hInstance,
		NULL
	);

	ShowWindow(hWnd, nCmdShow);
	UpdateWindow(hWnd);

	auto hFile = CreateFile(L".\\local-video.mp4", GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
	LARGE_INTEGER fileSize = {};
	GetFileSizeEx(hFile, (PLARGE_INTEGER)&fileSize);

	auto options = Microsoft::WRL::Make<CoreWebView2EnvironmentOptions>();
	options->put_AdditionalBrowserArguments(L"--ignore-certificate-errors");
	CreateCoreWebView2EnvironmentWithOptions(
		nullptr,
		nullptr,
		options.Get(),
		Microsoft::WRL::Callback<ICoreWebView2CreateCoreWebView2EnvironmentCompletedHandler>(
			[hWnd,fileSize](HRESULT result, ICoreWebView2Environment* env) -> HRESULT {
				env->CreateCoreWebView2Controller(hWnd, Microsoft::WRL::Callback<ICoreWebView2CreateCoreWebView2ControllerCompletedHandler>(
					[hWnd, env, fileSize](HRESULT result, ICoreWebView2Controller* controller) -> HRESULT {
						if (controller != nullptr) {
							webviewController = controller;
							webviewController->get_CoreWebView2(webview.put());
						}

						ICoreWebView2Settings* Settings;
						webview->get_Settings(&Settings);
						Settings->put_IsScriptEnabled(TRUE);
						Settings->put_AreDefaultScriptDialogsEnabled(TRUE);
						Settings->put_IsWebMessageEnabled(TRUE);

						RECT bounds;
						GetClientRect(hWnd, &bounds);
						webviewController->put_Bounds(bounds);

						std::wstringstream ss;
						ss << L"<html><body>";
						ss << L"<div>";
						ss << L"<p>SetVirtualHostNameToFolderMapping 200 response</p>";
						ss << L"<p>Entire video loaded, cannot seek</p>";
						ss << L"<video height=\"240\" controls muted=\"muted\" src=\"http://video.example/local-video.mp4\"></video>";
						ss << L"</div>";
						ss << L"<div>";
						ss << L"<p>WebResourceRequested 200 response with content headers</p>";
						ss << L"<p>Entire video loaded, can seek</p>";
						ss << L"<video height=\"240\" controls muted=\"muted\" src=\"http://localhost:8888/local-video.mp4\"></video>";
						ss << L"</div>";
						ss << L"<div>";
						ss << L"<p>WebResourceRequested 206 response with content headers</p>";
						ss << L"<p>1MB of video loaded, cannot play video</p>";
						ss << L"<video height=\"240\" controls muted=\"muted\" src = \"http://localhost:8888/local-video-partial.mp4\"></video>";
						ss << L"</div>";
						ss << L"</body></html>";

						webview->NavigateToString(ss.str().c_str());
						webview->OpenDevToolsWindow();

						auto webView3 = webview.try_as<ICoreWebView2_3>();
						webView3->SetVirtualHostNameToFolderMapping(
							L"video.example",
							L".",
							COREWEBVIEW2_HOST_RESOURCE_ACCESS_KIND_DENY_CORS);

						webview->AddWebResourceRequestedFilter(
							L"http://localhost:8888/local-video*.mp4", COREWEBVIEW2_WEB_RESOURCE_CONTEXT_ALL);
						webview->add_WebResourceRequested(
							Microsoft::WRL::Callback<ICoreWebView2WebResourceRequestedEventHandler>(
								[fileSize](
									ICoreWebView2* sender,
									ICoreWebView2WebResourceRequestedEventArgs* args) {
										wil::com_ptr<ICoreWebView2WebResourceRequest> request;
										args->get_Request(&request);
										wil::unique_cotaskmem_string uri;
										request->get_Uri(&uri);
										auto partial = std::wstring{ uri.get() }.find(L"partial") != std::string::npos;
										wil::com_ptr<ICoreWebView2HttpRequestHeaders> requestHeaders;
										request->get_Headers(&requestHeaders);
										wil::unique_cotaskmem_string range;
										requestHeaders->GetHeader(L"Range", &range);
										std::wcmatch matches{};
										auto results = std::regex_match(range.get(), matches, std::wregex(L"bytes=([0-9]*)-"));
										auto streamStart = std::stoul(matches[1].str());

										auto stream = winrt::make<MediaFileStream>(L".\\local-video.mp4", streamStart, streamStart + 1000000);
										
										auto fullStreamSize = fileSize.QuadPart;
										LARGE_INTEGER seek{ 0 };
										ULARGE_INTEGER newPosition{ 0 };
										stream->Seek(seek, STREAM_SEEK_SET, &newPosition);
										auto remainingSize = fullStreamSize - newPosition.QuadPart;
										auto partialSize = remainingSize < 1000000 ? remainingSize : 1000000;
										ULARGE_INTEGER chunkSize{ partial ? partialSize : remainingSize };
										if (partial) {
											stream->SetSize(chunkSize);
										}

										wil::com_ptr<ICoreWebView2Environment> environment;
										wil::com_ptr<ICoreWebView2_2> webview2;
										webview->QueryInterface(IID_PPV_ARGS(&webview2));
										webview2->get_Environment(&environment);
										wil::com_ptr<ICoreWebView2WebResourceResponse> response;
										environment->CreateWebResourceResponse(
											stream.get(),
											partial ? 206 : 200,
											L"OK",
											L"Content-Type: video/mp4",
											&response);
										wil::com_ptr<ICoreWebView2HttpResponseHeaders> responseHeaders;
										response->get_Headers(&responseHeaders);

										std::wstringstream ss;
										ss << L"bytes " << newPosition.QuadPart << "-" << newPosition.QuadPart + chunkSize.QuadPart - 1 << L"/" << fullStreamSize;
										responseHeaders->AppendHeader(L"Content-Range", ss.str().c_str());
										responseHeaders->AppendHeader(L"Content-Length", std::to_wstring(chunkSize.QuadPart).c_str());
										responseHeaders->AppendHeader(L"Accept-Ranges", L"bytes");

										args->put_Response(response.get());
										return S_OK;
								}).Get(), nullptr);

						return S_OK;
					}).Get());
				return S_OK;
			}).Get());

	MSG msg;
	while (GetMessage(&msg, NULL, 0, 0))
	{
		TranslateMessage(&msg);
		DispatchMessage(&msg);
	}
	CloseHandle(hFile);

	return (int)msg.wParam;

}

Let me know if this works.

Closing this for now, feel free to reopen if you have more questions about this.

[john-preston]

@yildirimcagri-msft Hi.

I try to do exactly that, video streaming (from memory) using such request interception and it strangely not works for me. I receive the first request from the

I use a custom scheme, but everything else works - HTML itself, the images, and even video in case I reply with the whole file - everything works. But if I reply with shown headers and partial data, I never receive another request for that resource and the video doesn't play.

Do you have any hints? Could this be a bug in WebView2 or am I doing something wrong?

Thank you.

[john-preston]

@yildirimcagri-msft nevermind, I've read in #3230 that this won't work with custom schemes. This wasn't what I expected :( But I guess I can change to some hand-crafted http:// urls instead and with them everything works.

[mahdirastegari]

I tried to use your C++ idea over my c# problem (thanks to the university for teaching us C++). Even with Partial Content, I can see there is a memory pick when you compare it with opening a file exactly from file storage on the web (using an HTML input tag).

Also, set SetVirtualHostNameToFolderMapping which resolves the memory issue but cannot seek the video.

[BenjaminGopp]

This C# code works perfectly for me

//register event handler
            webview.CoreWebView2InitializationCompleted += (s, e) =>
            {
                webview.CoreWebView2.AddWebResourceRequestedFilter("*", Microsoft.Web.WebView2.Core.CoreWebView2WebResourceContext.Media);
                webview.CoreWebView2.WebResourceRequested += CoreWebView2_WebResourceRequested;
            };

 private async void CoreWebView2_WebResourceRequested(object? sender, Microsoft.Web.WebView2.Core.CoreWebView2WebResourceRequestedEventArgs e)
 {
     if (e.Request.Uri.StartsWith("data")) return;


     var fileName = Path.GetFileName(e.Request.Uri);
     string videoPath = Path.Combine(LOCAL_VIDEO_DIR, fileName);
     var isLocalVideo = File.Exists(videoPath);

     if (isLocalVideo == false)
     {
         Debug.WriteLine($"remote VIDEO {e.Request.Uri} loading");
         return;
     }
     else 
     {
         Debug.WriteLine($"local VIDEO {videoPath} loading");
     }

     
     var httpRequest = WebView2Extensions.ToHttpRequest(e.Request);

     var acceptRange = httpRequest.Headers.Range is not null;

     if (!acceptRange) return;

     var fi = new FileInfo(videoPath);


     int chunkSize = 10 * 1024 * 1024; //10 Mb

     long chunkLength = fi.Length;

     var from = httpRequest.Headers.Range?.Ranges.First().From ?? 0;

     var contentLength = fi.Length;

     int bytes = (int)Math.Min(chunkSize, fi.Length);
     var buffer = new byte[bytes];
     var content = new MemoryStream(buffer);

     if (from == 0)
     {
         var fs = File.OpenRead(fi.FullName);
         fs.Read(buffer, 0, buffer.Length);
     }
     else
     {
         contentLength = chunkSize;
         var fs = File.OpenRead(fi.FullName);
         fs.Seek(from, SeekOrigin.Begin);
         fs.Read(buffer, 0, buffer.Length);
     }

     content.Position = 0;

     var response = webview.CoreWebView2.Environment.CreateWebResourceResponse(content, 206, "Partial Content", null);
     response.Headers.AppendHeader("accept-ranges", "bytes");
     response.Headers.AppendHeader("access-control-allow-origin", "*");
     response.Headers.AppendHeader("content-disposition", $"attachment; filename=\"{fileName}\"");
     response.Headers.AppendHeader("content-length", contentLength.ToString());
     response.Headers.AppendHeader("content-range", $"bytes {from}-{chunkLength - 1}/{chunkLength}");
     response.Headers.AppendHeader("content-type", "video/mp4");
     e.Response = response;

 }

[sameerdube9]

I am not able to find WebView2Extensions.ToHttpRequest(e.Request);

[bwedding]

@petegleeson Just going to comment that the virtual host issue seems to be fixed now. I used your example solution, added a 4Gb video file. It loads as fast as I can move the mouse over and click play. It seeks to any point in the file without buffering. The only changes I made were to update your Nuget packages to the latest versions:

• WebView2 1.0.3595.46
• CppWinRT 2.0.250303.1
• Windows Implementation Library 1.0.250325.1