Proxy requests through WKURLSchemeHandler to bypass Cookies and ITP restrictions

[NiklasMerz]

Platforms affected

iOS

Motivation and Context

iOS became more restrictive for CORS and tracking prevention which makes it hard for some apps to communicate with their server and use cookies.

See #922 #605 and possibly many more

Description

This PR implements a kind of "proxy" that takes requests like the normal webview and runs them through native code to make the request work without the CORS restrictions. Cookies are synced with the webview.

Originally taken from a modification in Ionics webview: ionic-team/cordova-plugin-ionic-webview#448

TODO

⚠️ The whitelist settings need to be part of this

• Should this be part of the ios platform or better a (third) party plugin?
• Tests?
• Docs

Testing

In a sample app for now. Use this code snippet to run code through the proxy:

fetch(window.WkWebView.convertProxyUrl("https://cordova.apache.org"));

Checklist

• I've run the tests to see all new and existing tests pass
• I added automated test coverage as appropriate for this change
• Commit is prefixed with (platform) if this change only applies to one platform (e.g. (android))
• If this Pull Request resolves an issue, I linked to the issue in the text above (and used the correct keyword to close issues using keywords)
• I've updated the documentation if necessary

[codecov-io]

Codecov Report

Merging #1004 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1004   +/-   ##
=======================================
  Coverage   74.91%   74.91%           
=======================================
  Files          13       13           
  Lines        1718     1718           
=======================================
  Hits         1287     1287           
  Misses        431      431           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7b785d9...829cc31. Read the comment docs.

[adamdport]

Do you have any concerns with Apple rejecting apps that essentially "bypass" the WkWebView restrictions they've added?

[NiklasMerz]

I don't have any concerns but I am not entirely sure. We are using this in an app for over a year now without any problems. I think an Apple engineer even suggested this as an possible solution while discussing options how to solve it. Native apps or the xhr plugin just do it like that, too and don't have the restrictions.

[PieterVanPoyer]

Hey Niklas

I think this is a great addition.
Not having to deal with CORS issue, will make building a Cordova app a lot nicer.
I cannot review the code as I am no ios developer, but I think the addition of code looks good structured and is rather small for such a great feature.

Will it also work with sending multipartform data to the backend? [ multipart/form-data ]
Will it also work for loading local assets like local images or translations or config jsons?

In my opinion this must be an addition to the ios platform.
This should be the default way for Http requests.

Maybe there should be a flag or setting to Opt out of the proxy. (If this is not yet foreseen)

Another remark is the ability to inspect the HTTP traffic. We are used to inspect it with the network inspector of the browser, but this will not be possible anymore.
Maybe there must be an addition in some documentation about how to inspect the network traffic with this setup.

Kind regards
Pieter

[NiklasMerz]

Hey Niklas

I think this is a great addition.
Not having to deal with CORS issue, will make building a Cordova app a lot nicer.
I cannot review the code as I am no ios developer, but I think the addition of code looks good structured and is rather small for such a great feature.

Thank you for the feedback

Will it also work with sending multipartform data to the backend? [ multipart/form-data ]
Will it also work for loading local assets like local images or translations or config jsons?

This should work for form data too if you use the special URL. For local stuff you could use window.WkWebView.convertFileSrc. This has been part of the platform for a while now.

In my opinion this must be an addition to the ios platform.
This should be the default way for Http requests.

Maybe there should be a flag or setting to Opt out of the proxy. (If this is not yet foreseen)

They way it is built now the "proxy" will only work if you convert you URL like https://mysite.com with convertProxyUrl to app://myscheme/_https_proxy_mysite.com. If you use normal URLs the webview will work just like it used too.

Another remark is the ability to inspect the HTTP traffic. We are used to inspect it with the network inspector of the browser, but this will not be possible anymore.
Maybe there must be an addition in some documentation about how to inspect the network traffic with this setup.

This is no problem somehow. All requests done with the special URLs appear in the inspector as usual.

Kind regards
Pieter

[razam217]

@NiklasMerz Can you please suggest me the steps I should follow to use this PR in my app?

[NiklasMerz]

@razam217 Sure. But you need to be aware that this is work in progress and not part of any release yet. It would be nice if you could test it and give feedback. This PR may have bugs or change in the future.

At first you need to remove your iOS platform cordova platform rm ios. Then you can add the platform from the branch this PR is based of cordova platform add https://github.com/GEDYSIntraWare/cordova-ios#proxy. If you want to use the proxy now for an URL you need to call this helper function to convert the URL window.WkWebView.convertProxyUrl(myURL). This converts the URL to a special URL (using a custom scheme) and the proxy will handle it if you use the URL in an XHR, fetch tag etc. You can still inspect the network tab in the Safari console to check your requests.

Good luck!

[vvsou862]

Hi @NiklasMerz , I have tried the above mentioned approach in our app and i am able to load the application using cookies.
Thank you for the update. I have found below observation.

1. When i cancel the ongoing request and re issue again it is throwing below error. (using ajax,abort)
   Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'This task has already been stopped'

It would be great if you can help me what i am missing here.

[adamdport]

@NiklasMerz I tested this PR out to see if it would resolve my cookie issues. In safari's network tab, you can click a request. In the headers tab of that request, there are several sections: Summary, Request, Response, and Request Data. When using this branch, the Request Data section seems to be missing, and my endpoint is not receiving the data I've tried to attach to the request. If it helps, I'm using Angular 10.1.3's HttpClient to generate the request.

If you have any ideas or need more information from me please let me know. Thanks!

[NiklasMerz]

From my experience request data is always there for post requests done with fetch. Any ideas what is different?

[adamdport]

It looks like the issue is only present in iOS 12 or iOS11 (I hadn't realized I hadn't updated yet). I don't see it in iOS 13 or iOS 14. Not sure if you intend to support <13, but it looks like [email protected] supports 11 or later.

A couple other things that should be considered:

• Looks like it uses a default request timeout of 60s. I was able to get past this by setting my own setTimeoutInterval on the request, but this probably needs to be either disabled or made configurable from cordova
• Cookies still didn't appear in the network tab, but they did appear to be sent. Not sure if this is a safari issue (or "feature")
• When I first build, I get an error Value for SWIFT_VERSION cannot be empty. I can set it manually in XCode and rebuild without issue, but I haven't had to do that before (with [email protected])

Thanks!

Edit: Your comment above mentions convertFileSrc, but I believe you meant convertFilePath. Probably worth an edit for those that arrive here via google.

[adamdport]

I found one more thing: it doesn't look like this works for iframes. An iframe that's hosted by the same remote that I'm authenticated with does not seem to have access to authentication cookies.

Thanks again.

[NiklasMerz]

New PR #1030 is the new path forward. cordova-ios will just allow plugins to plug into the WKURLSchemeHandler. The proxy is done in a plugin.

[girish2711]

@NiklasMerz Thanks for the WKURLSchemeHandler implementation. I took #1030 PR and integrated with your plugin https://github.com/GEDYSIntraWare/cordova-plugin-webview-proxy. Also changed the URL per your suggestion. All my XHR calls are now directed to webiewproxy.m and completes with proper response.

However, in my case my API responds with a few session cookies and I don't see them getting synced to my Cordova main web view. I see cookies in native cookie store but not in Cordova main web view. Is this expected ? Any solution to this?

I have a Single Sign On implementation which needs these session cookies to be sent along with http GET URL request (window.location = url to a subdomain) to a my own server. But the request is sent without cookies and hence it fails.

I tried by configuring CDVWebViewEngine's webdatastore to a nonPersistant one and added cookies to the same store as in below, but that didn't sync cookie to Cordova Main webview

first added app bound domain in the info.plist

and then in CDWebViewEngine:77.

if (@available(iOS 14.0, *)) {

 configuration.limitsNavigationsToAppBoundDomains = YES;    
 WKWebsiteDataStore *dStore =[[CDVWebViewProcessPoolFactory sharedFactory] sharedDataStore]; // by adding new method in sharedFactory

    configuration.websiteDataStore = dStore; 
} else {
    // Fallback on earlier versions
}

and in webviewproxy.m:40

WKWebsiteDataStore* dataStore = [[CDVWebViewProcessPoolFactory sharedFactory] sharedDataStore];
    WKHTTPCookieStore* cookieStore = dataStore.httpCookieStore;

Any idea if cookies are not meant for syncing or am I missing something ? I am testing in iOS14 with XCode 12.3.