WPAD not working for us!?

[ccbur]

I'm currently trying to use WPAD/PAC support on Win10 clients and I get the following log:

MainProcess: MainThread: 1523945168: attach_console: No parent console to attach to
MainProcess: MainThread: 1523945168: parse_config: Turning allow off
MainProcess: MainThread: 1523945168: load_proxy: AutoConfigURL = http://info-pac.rwe.com/innogy.pac
MainProcess: MainThread: 1523945168: load_proxy: Proxy mode = 3
Serving at :3128 proc MainProcess
MainProcess: MainThread: 1523945168: parse_config: proxy:server = 
MainProcess: MainThread: 1523945168: parse_config: proxy:listen = 
MainProcess: MainThread: 1523945168: parse_config: proxy:port = 3128
MainProcess: MainThread: 1523945168: parse_config: proxy:gateway = 0
MainProcess: MainThread: 1523945168: parse_config: proxy:hostonly = 1
MainProcess: MainThread: 1523945168: parse_config: proxy:allow = 
MainProcess: MainThread: 1523945168: parse_config: proxy:noproxy = 
MainProcess: MainThread: 1523945168: parse_config: proxy:useragent = 
MainProcess: MainThread: 1523945168: parse_config: settings:workers = 2
MainProcess: MainThread: 1523945168: parse_config: settings:threads = 5
MainProcess: MainThread: 1523945168: parse_config: settings:idle = 30
MainProcess: MainThread: 1523945168: parse_config: settings:socktimeout = 5.0
MainProcess: MainThread: 1523945168: parse_config: settings:proxyreload = 60
MainProcess: MainThread: 1523945168: parse_config: settings:foreground = 0
MainProcess: MainThread: 1523945168: parse_config: settings:log = 0
MainProcess: MainThread: 1523945204: verify_request: Client address: 127.0.0.1
MainProcess: MainThread: 1523945204: verify_request: Host-only IP allowed
MainProcess: Thread_0: 1523945204: do_CONNECT: Entering
MainProcess: Thread_0: 1523945204: do_transaction: Entering
MainProcess: Thread_0: 1523945204: get_destination: chocolatey.org:443
MainProcess: Thread_0: 1523945204: load_proxy: Skip proxy refresh
MainProcess: Thread_0: 1523945204: winhttp_find_proxy_for_url: WinHttpGetProxyForUrl error 6
MainProcess: Thread_0: 1523945204: find_proxy_for_url: Proxy found: 
MainProcess: Thread_0: 1523945204: do_transaction: No proxy server specified and not in noproxy list
MainProcess: Thread_0: 1523945204: do_CONNECT: Error 501
MainProcess: Thread_0: 1523945204: log_message: code 501, message Not Implemented
MainProcess: Thread_0: 1523945204: log_message: "CONNECT chocolatey.org:443 HTTP/1.1" 501 -
MainProcess: Thread_0: 1523945204: do_CONNECT: Transferred 0 bytes
MainProcess: Thread_0: 1523945204: do_CONNECT: Done

I tried to debug myself, but was not able to figure out why WinHttpGetProxyForUrl() fails with a ERROR_INVALID_HANDLE (6). The function WinHttpOpen() returns a valid handle! What other handle could be invalid?

I googled for other winhttp+ctypes examples on the web and they fail also! Same error at the same location. Any idea what could be different at our setup!? I can reproduce this on multiple Win10 clients.

I'm a little bit lost now... :-(

[genotrance]

What does the actual PAC file look like? Does it actually return a proxy string for that host?

It might be worth trying with pypac and seeing what it thinks the proxy string is.

[genotrance]

So just to clarify, WPAD = discovery using DNS/DHCP. It looks like you have set AutoConfigURL = PAC mode and not WPAD.

That being said, can you also verify that IE is able to use this PAC file and resolve correctly?

[awbrackm]

I have the same error on Windows 7. Pinned it to the usage of WINHTTP_ACCESS_TYPE_AUTOMATIC_PROXY which is only supported on Windows 8.1 and newer.

[ccbur]

Too be honest, I'm not sure where PAC starts and WPAD ends (cp. https://en.wikipedia.org/wiki/Web_Proxy_Auto-Discovery_Protocol), so maybe my wording was a little bit inaccurate.

But IE works without problems and the PAC file is valid. It should return "PROXY proxy.rwe.com:8080".

As we are using Win10, WINHTTP_ACCESS_TYPE_AUTOMATIC_PROXY should work for us. I changed to the deprecated version WINHTTP_ACCESS_TYPE_DEFAULT_PROXY, but WinHttpGetProxyForUrl() still gives me the same ERROR_INVALID_HANDLE (6).

I've still no idea, what handle this error code is referring to. :-(

[ccbur]

I tried an older version of px (2018-03-03) and pypac seems to work correctly:

MainProcess: Thread_0: 1525689668: do_CONNECT: Entering
MainProcess: Thread_0: 1525689668: do_transaction: Entering
MainProcess: Thread_0: 1525689668: get_destination: chocolatey.org:443
MainProcess: Thread_0: 1525689668: load_proxy: Skip proxy refresh
MainProcess: Thread_0: 1525689668: find_proxy_for_url: pypac = [('proxy.rwe.com', 8080)] -1
MainProcess: Thread_0: 1525689668: get_destination: Proxy from PAC = [('proxy.rwe.com', 8080)]
MainProcess: Thread_0: 1525689668: do_socket_connect: New connection: ('proxy.rwe.com', 8080)
MainProcess: Thread_0: 1525689668: do_proxy_type: Searching proxy type
MainProcess: Thread_0: 1525689668: do_socket: Entering

[ccbur]

Maybe I found something related:

https://stackoverflow.com/questions/45783063/obtaining-windows-service-handle-in-python

asweigart/pyperclip#25

Summary: Python 3.6 has some problems with 32/64 Bit Pointers which might lead to problems with ctypes and Win32-APIs on 64 Bit machines.

No idea if this can be fixed, maybe this is not related at all!?

[ccbur]

I think I was right. If I explicitly set the hinternet to c_void_p it works for us, too:

diff --git a/px.py b/px.py
index 25a4e56..62587af 100644
--- a/px.py
+++ b/px.py
@@ -922,6 +922,7 @@ WINHTTP_ACCESS_TYPE_NAMED_PROXY = 3
 WINHTTP_ACCESS_TYPE_AUTOMATIC_PROXY = 4

 def winhttp_find_proxy_for_url(url, autodetect=False, pac_url=None, autologon=True):
+    ctypes.windll.winhttp.WinHttpOpen.restype = ctypes.c_void_p
     hInternet = ctypes.windll.winhttp.WinHttpOpen(
         ctypes.wintypes.LPCWSTR("Px"),
         WINHTTP_ACCESS_TYPE_AUTOMATIC_PROXY, WINHTTP_NO_PROXY_NAME,
@@ -945,6 +946,7 @@ def winhttp_find_proxy_for_url(url, autodetect=False, pac_url=None, autologon=Tr

     proxy_info = WINHTTP_PROXY_INFO()

+    ctypes.windll.winhttp.WinHttpGetProxyForUrl.argtypes = [ctypes.c_void_p, ctypes.wintypes.LPCWSTR, ctypes.POINTER(WINHTTP_AUTOPROXY_OPTIONS), ctypes.POINTER(WINHTTP_PROXY_INFO)]
     ok = ctypes.windll.winhttp.WinHttpGetProxyForUrl(hInternet, ctypes.wintypes.LPCWSTR(url),
             ctypes.byref(autoproxy_options), ctypes.byref(proxy_info))
     if not ok:

I testet it on 64 Bit Windows 7 and Windows 10 with Python 3.6.3 and 3.6.5. Anyone with a 32 Bit machine and/or an older Python version around to test this?

[genotrance]

I've implemented your fix in the latest release, please verify and close. Thank you!

[ccbur]

The new version works for us. Tested on W7 and W10. Thanks!!