CoreWebView2_WebResourceRequested not providing binary multi-part POST Data in Request.Content

[RickStrahl]

I have an application that captures Web Request data for later playback (WebSurge). This works great for standard text based content be it API requests or captured form data - the data gets captured into the Request stream and I can grab it out of there. Works great.

However, if the data is multi-part form data with binary data, the Request.Content is always null:

In contrast, here is a POST request with standard form data which does work and returns a Request.Content stream:

So for some reason it looks if the data is multi-part or contains binary data the request content is not provided.

AB#38186390

[champnic]

Thanks for the bug report @RickStrahl! I've added this to our backlog.

[RickStrahl]

Is there any word on whether this will be addressed?

Tested today with latest version and found this still doesn't work. Confirmed that:

• as soon as binary content is included in form data, the .Content property is null.
• multi-part forms with text only fields work fine

[RickStrahl]

@champnic Ping! Any progress or at least confirmation that this is a bug and will get addressed in the future? Or a workaround?

[champnic]

Thanks for the ping Rick. @yildirimcagri do you have any info on this issue?

[SharpEnvelope]

I also ran into exactly this issue. Ran into it when using WebView2 from C++ but also reproduced it in C# WinForms.

This can be reproduced with the following JavaScript code:

const obj = { hello: "world" };
const blob = new Blob([JSON.stringify(obj, null, 2)], {
    type: "application/json",
});

const data = new FormData();
data.append("val1", "val1");
data.append("blob", blob);

const req = new XMLHttpRequest();
req.addEventListener("load", () => {
    alert("Received: " + req.responseText);
});
req.open("POST", "https://post.asset/");
req.send(data);

And the following C# code:

this.webView2Control.CoreWebView2.WebResourceRequested += CoreWebView2_WebResourceRequested;
this.webView2Control.CoreWebView2.AddWebResourceRequestedFilter("https://post.asset/", CoreWebView2WebResourceContext.All);

private void CoreWebView2_WebResourceRequested(object sender, CoreWebView2WebResourceRequestedEventArgs eventArgs)
{
    System.IO.Stream content = eventArgs.Request.Content;

    if(content != null)
    {
        byte[] bytes = new byte[content.Length + 10];
        content.Read(bytes, 0, (int)content.Length);

        var stringData = System.Text.Encoding.Default.GetString(bytes);
        MessageBox.Show("Received: " + stringData);
    }
    else
    {
        MessageBox.Show("No content stream :(");
    }

    eventArgs.Response = this.webView2Control.CoreWebView2.Environment.CreateWebResourceResponse(null, 200, "OK", "");
}

The attached solution contains a running sample:
WebView2BinaryResourceIssue.zip

[yildirimcagri-msft]

Thanks for the sample code for this. I missed the earlier ping about this, but I will prioritize this issue now.

[stffabi]

Seeing the same behaviour in a Wails application.

[RickStrahl]

Any update on this @yildirimcagri? @champnic?

[SharpEnvelope]

Are there any plans to address this issue @yildirimcagri-msft?

[visuall]

Same for me. The WebResourceRequested handler is the only way to intercept/filter requests and this bug makes this impossible. @yildirimcagri-msft , you say you will prioritize this issue for the last year or so and still nothing. Will it be possible finally this issue to be solved ?

[bogerj]

I am also the same. This issue has caused my solution to be unable to proceed due to this bug. Will this issue be resolved? Looking forward to your reply

[steno916]

@champnic @yildirimcagri-msft Is there any status update on this? Is there a workaround? Thanks.

[MarcoB89]

We are in the same situation, any updates? Is there a workaround to avoid this problem?

[Yuxiza]

Is there any updates? Or some alternatives?

[Yuxiza]

Here is a temporary solution:

const formData= new FormData();
formData.append("val1", "val1");
formData.append("blob", blob);
const req = new Request(uploadUrl, {
      method: "POST",
      body: formData,
});
const buffer = await req.arrayBuffer();
const headers = req.headers;
const response = await fetch(uploadUrl,{ 
    method:"POST",
    body:buffer,
    headers
});

By transmut multi-part data into arrayBuffer, we can access the request body though Request.Content.

[MarcoB89]

Here is a temporary solution:

const formData= new FormData();
formData.append("val1", "val1");
formData.append("blob", blob);
const req = new Request(uploadUrl, {
      method: "POST",
      body: formData,
});
const buffer = await req.arrayBuffer();
const headers = req.headers;
const response = await fetch(uploadUrl,{ 
    method:"POST",
    body:buffer,
    headers
});

By transmut multi-part data into arrayBuffer, we can access the request body though Request.Content.

Seems this workaround works, but not in my scenario, I'm appending to the body not a blob but a file from an input type file for upload it. Using arrayBuffer() the file is loaded in memory, and for upload of large files that's not good...

[MarcoB89]

Any updates? This bug caused a complete reengineering of our solution. It's not possible do a multi-part POST request for uploading file!

[victorhuangwq]

From what I can see, this is being worked on. We will post any updates here.

[leonidukg]

Been watching this bug for a long time too. But this bug is not in Webview2, it is Chromium itself that has this problem. It can be seen in Devtools.

Even if you connect via DevTools Protocol FetchRequestPaused, the Post request will be empty.

BUT!

I found a workaround. The full Post request can be seen through:

DevTools Protocol Network.requestWillBeSent

It's a miracle! Next is a simple code:

1. protocol initialization:

private Dictionary<string, string> HelpPostNetworkData = new();

        private async void WebView_CoreWebView2InitializationCompleted(object sender, CoreWebView2InitializationCompletedEventArgs e)
        {
            await WebView.CoreWebView2.CallDevToolsProtocolMethodAsync("Network.enable", "{}");
            WebView.CoreWebView2.GetDevToolsProtocolEventReceiver("Network.requestWillBeSent").DevToolsProtocolEventReceived += WebView2_NetworkData;
}

2. Processing our data and saving the Postdata to the backup dictionary

        private async void WebView2_NetworkData(object sender, CoreWebView2DevToolsProtocolEventReceivedEventArgs e)
        {

            if (e == null || e.ParameterObjectAsJson == null)
            {
                return;
            }


            var doc = JsonDocument.Parse(e.ParameterObjectAsJson);

            string payload = "{\"requestId\":\"" + doc.RootElement.GetProperty("requestId") + "\"}";
            string bodyResponse;
            string nowurl = doc.RootElement.GetProperty("request").GetProperty("url").ToString().ToLower();

            if (doc.RootElement.GetProperty("request").GetProperty("method").ToString().Equals("post", StringComparison.OrdinalIgnoreCase))
            {
                try
                {
                    bodyResponse = await WebView.CoreWebView2.CallDevToolsProtocolMethodAsync("Network.getRequestPostData", payload);
                }
                catch 
                {
                    return;
                }

                if (!string.IsNullOrEmpty(bodyResponse))
                {
                    HelpPostNetworkData.Remove(nowurl);
                    HelpPostNetworkData.Add(nowurl, bodyResponse);
                }

            }
        }

And now you can use this dictionary to work and check if Content == null in CoreWebView2WebResourceResponseReceived, then take data from the dictionary at the URL

[bob-dawson]

I tried leonidukg's comment. It works, but when the post data include file, the file values will omit. So, it still need to fix.

[leonidukg]

I tried leonidukg's comment. It works, but when the post data include file, the file values will omit. So, it still need to fix.

You are right because Network.requestWillBeSent is executed BEFORE it sends all the data. And most likely the file download is not included in this check.
I only need the field names to work, the values are not important.

p.s. I have corrected my code to prevent requests from hanging by accident.

[bob-dawson]

I tried leonidukg's comment. It works, but when the post data include file, the file values will omit. So, it still need to fix.

You are right because Network.requestWillBeSent is executed BEFORE it sends all the data. And most likely the file download is not included in this check. I only need the field names to work, the values are not important.

p.s. I have corrected my code to prevent requests from hanging by accident.

I mean file upload. you can't get upload file content.

[leonidukg]

Once again encountered this problem and even my workaround won't help. So I had to look for another one, but I already had a function to modify HTML code and I was surprised to see PostData perfectly through this function.
So who needs not just POST variables, but their content, use it:

private Dictionary<string, string> HelpPostNetworkData = new();

        private async void WebView_CoreWebView2InitializationCompleted(object sender, CoreWebView2InitializationCompletedEventArgs e)
        {
			WebView.CoreWebView2.GetDevToolsProtocolEventReceiver("Fetch.requestPaused").DevToolsProtocolEventReceived += WebView2_FetchRequestPaused;
			await WebView.CoreWebView2.CallDevToolsProtocolMethodAsync("Fetch.enable", "{\"patterns\":[{\"urlPattern\":\"*\", \"requestStage\":\"Response\"}]}");
}

        private async void WebView2_NetworkData(object sender, CoreWebView2DevToolsProtocolEventReceivedEventArgs e)
        {

            if (e == null || e.ParameterObjectAsJson == null)
            {
                return;
            }


            var doc = JsonDocument.Parse(e.ParameterObjectAsJson);

            string payload = "{\"requestId\":\"" + doc.RootElement.GetProperty("requestId") + "\"}";
            string nowurl = doc.RootElement.GetProperty("request").GetProperty("url").ToString().ToLower();

            if (doc.RootElement.TryGetProperty("responseHeaders", out JsonElement JSHeadersPost))
            {
                if (!JSHeadersPost.ToString().Contains("application/json", StringComparison.OrdinalIgnoreCase))
                {
                    if (doc.RootElement.GetProperty("request").GetProperty("method").ToString().Equals("post", StringComparison.OrdinalIgnoreCase))
                    {
                        if (doc.RootElement.GetProperty("request").TryGetProperty("postData", out JsonElement JSbodyResponse))
                        {
                                HelpPostNetworkData.Remove(nowurl);
                                HelpPostNetworkData.Add(nowurl, JSbodyResponse.ToString());
                        }
                    }
                }
            }
await Application.Current.Dispatcher.InvokeAsync(() => { try { WebView.CoreWebView2.CallDevToolsProtocolMethodAsync("Fetch.continueRequest", payload); } catch { } });
        }

[yildirimcagri-msft]

Hello, this should be working now. Please reactivate if it is not.

[RickStrahl]

As of which release?

[RickStrahl]

So I'm looking at this in the latest release build and I'm not sure what I'm looking at or how I'm supposed to get at this data.

It looks like Request content returns some sort of stream, but the stream is way too small - it doesn't contain the upload data:

and indeed when I try to read the actual file content it's always empty. I do get the headers now.

Here's what I do:

• I capture the original stream in byte[]
• Then run it into MultipartFormParser.Parse()

That works to give me the variables - content type, length of content, filename. But the data is always 0 length which makes sense since the incoming data does not reflect the full body that includes the full binary file size.

From the looks of it the payload captured is only the non-binary header data.

---

Using latest release SDK 1.0.2849.39

[RickStrahl]

Your handling of this issue seriously is terrible people at MSFT.

Nearly 3 years since this was opened and since then no real discussion from the team contributors here, only some vague reference to things happening behind closed doors.

Then a note that it's fixed without any context of exactly what was fixed or even what build it was fixed in - and it certainly DID NOT FIX the issue in the original bug report, which is easy enough to verify.

You offer to reopen if not fixed. but there's no actual option to do just that.

Clearly this issue touches on a use case that a lot of people have run into, so this isn't an edge case scenario.

Bottom line the bug is NOT FIXED. Maybe @yildirimcagri-msft is talking about something else, but who knows... it doesn't get more minimal than the response given which would be fine if it had actually worked!

Maybe there's a workaround? The people on this team presumably are somewhat expert at Chromium and DevTools development and should at least have something to contribute on workarounds or other solutions or at least some knowledge on what's possible and what's not? Instead we get nothing...

If this is an issue in Chromium itself - this might be a good feature suggestion or perhaps something to help fix in Chromium? Wasn't there all this talk of the Edge/WebView folks contributing to Chromium specifically for Windows features for example? So there's some involvement. But even beyond that you're making it hard for any of use complaining providing more useful feedback or possibly help in getting this to work by your deafening non-involvement.

Please do better.

If you're not going to take a serious stab at this because there are no resources or it can't be done to some limitation in Chromium or the Runtime - that's fair enough, but at least come out and say so!

Thanks for listening!

[yildirimcagri-msft]

Hello @RickStrahl , First of, I appreciate your feedback and apologies for leaving you in the dark for a while, unfortunately, I just saw your reply when you mentioned me as the issue had remained closed. I just reopened the issue.

We have been actively looking at fixing this issue by directly obtaining this information from network stack, which has proven to be challenging. However, we also discovered that Chromium had recently made a change in the CDP event, which is where we originally obtain the information to fire the WebResourceRequested event, that allows multi-part POST data to be observed now. That's why I thought this would have been fixed now. However, it does only work for blob or byte types and potentially not for files. For files, unless Chromium also makes a change to get it working in CDP, we would need to fix this on our side. One thing I want to ask is, can you tell from the filename, that should be available in the multi part post data, which file to read? If you could do that, at least you could read the same file from your app via Windows APIs?

[RickStrahl]

@yildirimcagri-msft Thanks for getting back so quickly - that is much appreciated.

This response is what I was hoping we could get from this 'forum' in general. It's a good example as it provides some insight into what the issues are and also what the obstacles are on your end, so we as consumers of the API can set our own expectations without speculation on what's possible and what to expect.

So, again thank you for that!

... and it would be nice to see more of this for many issues posted here.

[RickStrahl]

The data that we receive from the event API provides all the data except the file attachment data. Variable names come in just fine it's just the Data stream is always null for multi-part data. It works perfectly fine for none multi-part content. I'd have to check if there's a requestId in the args passed, but that should also be available.

The data from the stream on a single file file upload field comes in like this:

------WebKitFormBoundaryessVIBntQ0CPdAm8
Content-Disposition: form-data; name="upload"; filename="YouTubePlay.png"
Content-Type: image/png


------WebKitFormBoundaryessVIBntQ0CPdAm8--

As you can see the actual file content is not there but everything else is.

Note: It was also suggested that file size matters here, but just to be clear the file attach here is a very small 2k icon file so size shouldn't be an issue in this particular example.

I believe this is an issue in Chromium, but it's seems like this is an arbitrary limitation - ie. content had to be explicitly stripped from the outgoing HTTP data. Perhaps there's some other mechanism that allows intercepting the files that are being sent as attachments, but I have not been able to dig up any information to that effect. That would have to come from someone more knowledgeable of Chromium APIs.

[fabiorocha]

Thanks, Cagri for following up and reactivating the issue. Assigning it to myself while I find a new owner for this.