win32 system service

[totaam]

Split from #389, similar to #1105 for Linux, we want to have a system wide service running even before the user logs in.
Users can then trigger a login.

Some links already added in #389 comment 23.

[totaam]

Blocked by #1528.

[totaam]

With the cx_freeze 5 workarounds added in #1528#comment:2 and the service stub added in r15933 +
r15934 (based on cx_Freeze samples: service), a "Xpra-Service.exe" is created - it just doesn't do anything when you run it...
The documentation is non-existent, and the executable doesn't give you any help at all. But I eventually found that we're supposed to install it by running:

Xpra-Service.exe --install NAME [configfile]

Problem is that this fails with:

Service not installed. See log file for details.

What log file you ask? Well "./.log" obviously, NOT. This file contains:

[04380] 2017/05/23 22:46:14.473 starting logging at level ERROR
[04380] 2017/05/23 22:46:14.520 Win32 error 0x5 encountered.
[04380] 2017/05/23 22:46:14.520     Context: cannot open service manager
[04380] 2017/05/23 22:46:14.520     Message: Access is denied.

[04380] 2017/05/23 22:46:14.520 ending logging

Fine, let's run it as administrator:

[02964] 2017/05/23 22:49:55.695 starting logging at level ERROR
[02964] 2017/05/23 22:49:55.757 Win32 error 0x57 encountered.
[02964] 2017/05/23 22:49:55.757     Context: cannot start service
[02964] 2017/05/23 22:49:55.757     Message: The parameter is incorrect.

[02964] 2017/05/23 22:49:55.773 ending logging

At least the service is registered and visible in the "Services" control tool, and the same error message is available in the "eventvwr".
We'll need to get rid of "cx_Logging", it's unmaintained since 2014, undocumented (which filename is used from the service?) and generally not helpful.

[totaam]

OK, the reason why this fails with the cryptic error 0x57 is because the path to the service binary is empty (for whatever reason).
Assuming that the service has been installed using:

Xpra-Service.exe --install XPRA-TEST

The path can then be changed using regedit, or using Sc config.
First query the service:

sc qc XpraXPRA-TEST

Change the path:

sc config XpraXPRA-TEST binPath= "C:\Program Files\Xpra\Xpra-Service.exe"

We can now also set it to auto-start:

sc config XpraXPRA-TEST start= auto

With these changes, starting the service takes longer to fail.. but fail it does with: Error 1053: The service did not respond to the start or control request in a timely fashion
We can find the service PID with:

sc queryex XpraXPRA-TEST

Then kill it with:

taskkill /f /pid $PID

We could workaround the broken path installation by running the "sc config" commands after the service registration, still as part of the EXE / MSI
installation process.

But the fact that the process does not respond is more problematic...
Ideas:

• maybe the paths workarounds need to be applied to the service class (in C)
• maybe use a simple shim? a simple C service that (re)starts the real xpra proxy server?
  The shim would allow us to continue to use cx_freeze 4.x (no need for cx_freeze 5 support on MS Windows #1528). Some links for that:
• MSDN: Service Programs
• The Complete Service Sample
• The Windows Services Win32 Programming
• extra difficulty with mingw? service and mingw

[totaam]

Difficulties in building the Complete Service Example with mingw:

• What is the difference between _tmain() and main() in C++? - just change it to "main", was missing int return value
• GetModuleFileName function signature was used wrong, pass NULL
• StringCbPrintf function: just replace StringCchPrintf with snprintf, or better: add #undef __CRT__NO_INLINE before #include <strsafe.h> (maybe we should link against some lib instead?)
• to avoid string warnings, build with:

g++ -o Svc.exe Svc.cpp -Wno-write-strings

• __try and __except: msdn: try-except Statement, workaround is to not handle this at all...

#define __try
#define __finally

• after compiling, the binaries need these libs to run: libgcc_s_dw2-1.dll and libwinpthread-1.dll

[totaam]

2017-05-24 17:31:26: antoine uploaded file service-mingw.patch (2.7 KiB)

patch for building the service example with mingw

[totaam]

Stub cx_freeze5 service removed in r15960 (#1528 re-scheduled), replaced in r15962 by a shim implemented in C.

Still TODO:

• fix authentication problems
• remove hard-coded command line path: load settings from a config file?
• innosetup script changes: upgrading windows service using inno setup - maybe just ship the "SvcConfig" and "SvcControl" command line tools instead?
• start new sessions / locate existing ones and run the shadow under that user account
• fix event logging: we have to use event codes in the resource file (yuk)
• re-enable SSL: substitute commonappdata to locate the cert - or just generated the path in the config file

---

Notes:

• the log files for the proxy process when running from the "local system account" end up here: Where can I find data stored by a Windows Service running as “Local System Account”?, ie: C:\Windows\System32\config\systemprofile\AppData\Roaming\Xpra
• msdn: Interactive Services
• How can a Windows service execute a GUI application?
• Named Pipe Security and Access Rights
• Launching an interactive process from Windows Service in Windows Vista and later
• is there a good example of using pywin32 createprocessasuser and getting the output? covers CreateProcessWithTokenW in this answer

[totaam]

2017-05-25 15:21:27: antoine uploaded file start-shadow-from-proxy.patch (8.1 KiB)

ugly work in progress patch to use on top of r15971

[totaam]

2017-05-26 12:30:43: antoine uploaded file logon.patch (22.6 KiB)

work in progress logon patch

[totaam]

The ugly and incomplete patch above does:

• GetCurrentProcess
• OpenProcessToken
• [GetTokenInformation](https:#### 2017-05-26 12:34:49: antoine commented

---

The ugly and incomplete patch above does:

• GetCurrentProcess
• OpenProcessToken
• GetTokenInformation
• LookupPrivilegeValueW
• AdjustTokenPrivileges
• LsaRegisterLogonProcess

And adds a small Login-Test.exe utility. Problem is that this works when running from the service context, but not when running directly from the utility - even when running from an administrator shell. This is going to make development and testing tedious.

Some useful links:

• Determine if Windows process has privilege to create symbolic link
• Windows Logon as a user without a password with Python
• What is the api to impersonate a Windows local user account without providing the password?//msdn.microsoft.com/en-us/library/windows/desktop/aa446671(v=vs.85).aspx)

• LookupPrivilegeValueW
• AdjustTokenPrivileges
• LsaRegisterLogonProcess

And adds a small Login-Test.exe utility. Problem is that this works when running from the service context, but not when running directly from the utility - even when running from an administrator shell. This is going to make development and testing tedious.

Some useful links:

• Determine if Windows process has privilege to create symbolic link
• Windows Logon as a user without a password with Python
• What is the api to impersonate a Windows local user account without providing the password?

[totaam]

2017-05-27 10:12:19: antoine uploaded file logon-v2.patch (50.2 KiB)

logon succeeds but launching the new process does not..

[totaam]

With the patch above, I could logon but when trying to start the server, or even a test application like whoami.exe with all of its dlls installed, the event log would show:

Application popup: Xpra_cmd.exe - Application Error : \
The application was unable to start correctly (0xc0000142). Click OK to close the application.

The environment looked suspicious, but since we use the LOGON_WITH_PROFILE flag, it should be OK. (will need to re-check that)
See What is up with "The application failed to initialize properly (0xc0000142)" error?, which has lots of relevant information. (and some dead links too.. sigh)
Also some pointers here: The Perils and Pitfalls of Launching a Process Under New Credentials.
It was just missing the desktop name in the STARTUPINFO.... (not obvious)
Now maybe we also need permission to access that desktop? As per CreateProcessAsUser() windowstations and desktops, because the resulting screen is empty.

[totaam]

r15980 adds all the hooks for starting the shadow process from the service.
Still TODO:

• remove hard-coded command line path: load settings from a config file?
• innosetup integration
• add firewall rules: How to use the "netsh advfirewall firewall", ie:

C:\Windows\system32>netsh advfirewall firewall add rule name="Open Port 14500" d
ir=in action=allow protocol=TCP localport=14500

• re-enable SSL: substitute commonappdata to locate the cert - or just generated the path in the config file
• innosetup script changes: ​upgrading windows service using inno setup - maybe just ship the "SvcConfig" and "SvcControl" command line tools instead?
• don't use a fixed TCP port, use named pipes via the system proxy - or enhance the client to support a redirect and use OTP (which we can supply to the shadow process using env auth)
• html5 client connection fails, re-tries: we end up creating new shadow server instances... don't retry on fatal error (html5), don't create a new shadow if one exists!
• fix event logging: we have to use event codes in the resource file (yuk)
• the desktop we get is empty! why?
• start new sessions / locate existing ones and run the shadow under that user account
• integrate with "remote desktop services" win32: remote desktop services #1532

[totaam]

2017-05-28 07:15:54: antoine uploaded file logon.py (17.8 KiB)

this implementation does not work - but has some useful functions

[totaam]

See #389 comment 23 for LogonUser links.

[totaam]

See Another Windows 10 SKU is on its way, this time for remote desktops: With the new SKU, the multi-session capability is now a part of desktop Windows

[totaam]

See also: python-win32: runas analog: Starting a subprocess in Windows is somewhat of a black art, I fear.

[totaam]

Fixes and updates:

• r23582 support more SDK paths
• r23583 cosmetic
• r23584 prefer 64-bit tools
• r23585 make it easier to override the log file name (+r23586 tidy up)
• r23588 adds an "uninstall" command to the service utility

The service somehow fails to find the glib typelib.
Debugging this is tedious, will add yet another shim to make it easier to debug.

[totaam]

The gi bindings error is an unrelated packaging blocker bug: #2393.

[totaam]

aUpdates:

• r23593 fix named-pipe handling of spurious timeouts (should backport)
• r23594 don't set CKCON_X11_DISPLAY
• r23595 typo
• r23596 don't use roaming profile location for log files
• r23597 don't create app dir in system profile
• r23598 skip loading "pwd" module on non-posix systems
• r23599 fix win32 authenticator with python3 (should backport)
• r23600 better auth debug logging

Some related pointers:

• Starting an Interactive Client Process in C++ found here: process started from service with CreateProcessWithLogonW terminates right away
• Why is this process crashing as soon as it is launched?: CreateProcessWithLogonW cannot be called from a Windows Service Environment
• Call binary without elevated privilege

[totaam]

aUpdates:

• r23601 build the service by default
• r23604 we can't use blocking sockets when wrapping tcp sockets as ssl with python3?
• r23605 use ssl cert if we find it (generated by innosetup installer)
• r23606 + r23608 refactoring
• r23607 workaround python3 bug: ctypes string pointer fields should accept embedded null characters
• r23611 + r23613 wait a little bit for the shadow subprocess - don't block on read
• r23612 make env whitelist configurable, allow all on win32
• r23614 fix detection of invalid handles
• r23615 + r23618 make is easier to use named pipes: no need to specify the Xpra\ prefix, assume named-pipe protocol on win32
• r23616 + r23617 + r23619: debug logging, constify, cleanup, etc

The shadow we start creates a named-pipe, but the proxy server running as "system" cannot connect to it.
Some pointers:

• Change Named Pipe Access Permissions
• Named Pipe Security and Access Rights
• Access Control Lists
• Windows Named Pipe Access control
• NT AUTHORITY\SYSTEM through Named Pipe Impersonation using Python - good ctypes examples
• How to turn off Named Pipe security?
• How do I create a NULL/empty DACL?
• Check if an arbitrary user is in Administrator Group with Python - good example of ctypes SID code
• How to create an anonymous pipe that gives access to everyone

[totaam]

Updates:

• r23620 better logging for connection problems
• r23621 create a specific named pipe, generate the path from the username we shadow as
• r23622 add more security API definitions
• r23623 remove all security on the named pipes when XPRA_NAMED_PIPE_UNRESTRICTED=1
• r23624 by default, don't log the backtrace for socket timeouts (as we get lots of them on win32..)
• r23625 remove all security on the proxy named pipe, add sys authentication

With these changes, the proxy manages to start a shadow subprocess, but it doesn't seem to reach a usable state.
Redirecting its output to a log file shows that it goes through server initialization, it stops after loading a few icons.

[totaam]

• r23626 + r23627 + r23631 + r23633 debug logging tweaks
• r23629 fix pylint warnings
• r23634 shadow server now shows as "Xpra-Shadow" in the task manager
• r23635 better named pipe listener code (large)
• r23636 catch proxy start errors, ie: bug with ssl sockets: stateful sockets (ssl, ssh) cannot be serialized #2399
• r23638 use a shim for starting the shadow server (+r23641 freeze support)faa7973608c2da4de13765cfd3a9cc5) + r23633 debug logging tweaks

[totaam]

2019-09-01 12:32:06: @totaam uploaded file service-test.patch (3.8 KiB)

tweaks to make it easier to test and debug the service

[totaam]

2019-09-01 12:33:09: @totaam uploaded file logon.c (18.9 KiB)

logon.c example from "Starting an Interactive Client Process in C++"

[totaam]

We need to either shadow the console (so remote users can login from the login screen) or find a way to create a session programmatically.

More pointers:

• Understanding Windows at a deeper level - Sessions, Window Stations, and Desktops
• Window Station and Desktop Creation
• Launching an interactive process from Windows Service in Windows Vista and later
• Window Stations and Desktops 5

[totaam]

Progress made using the openssh server (#2711): we can shadow the Winlogon secure desktop using https://docs.microsoft.com/en-us/sysinternals/downloads/psexec

More pointers:

• Opening native graphical application from remote shell: On Windows there is instead an entire zoo of kernel objects called terminal sessions, login sessions, windows stations, desktops, and finally windows, some of which have their own access-control lists and associations with processes. In addition, there are security tokens
  Also: can bitvise ssh server overcome the session 0 isolation problem?
  No, nothing can get around it. What Bitvise does is properly associate a Windows Station (something which exists within a Desktop, which exists within a Session) with programs so that they will manage to run when invoked headlessly when the server is running within an interactive session.
• How can I run a win32 gui program from cygwin ssh? : Something like PsExec.exe -i -s "full path to program" args should work. -i runs the program in interactive mode on the desktop, and -s runs it under the "system" user so you don't have to punch in user credentials.
• PSExec to open interactive program with no user logged in (answer is not satisfactory..)
• psexec tools

[totaam]

r25977 starts the shadow server on the Winlogon secure desktop, it's more useful than the proxy it replaces since we can actually see the desktop.

To make more progress, it would be useful to:

• make it easier to run commands on the server (emulate xdg menu data - use code from winswitch?)
• run in debug mode and send to a log file so it can be tailed and grepped via ssh in real time
• distinguish the Winlogon secure desktop from real sessions

As for the change of desktop, that's going to be harder.
GTK probably can't be taught to re-initialize against the new display, so we would need to spawn a new process and either give it the existing socket connection, or proxy it.

[totaam]

r25980: we can now easily start new commands when connected to a shadow server

[totaam]

Maybe the blank display problems (comment:17 IIRC), can be fixed by using a SYSTEM privilege, just like -s with psexec, see #2711#comment:1.

Also: CreateProcessAsUser creates blank/black window: The call to LogonUser generates a new session (and associated logon SID) rather than reusing the existing one, so your process does not have access to the desktop, and only has minimal access to the window station

[totaam]

r26188 uses paexec (as per #2711) to start the shadow server.

Tested by reverting r25977 and then connecting to the proxy server using:

xpra shadow "ssl://user:password@server:14500/" --ssl-server-verify=none

Progress!
This now allows the shadow server to access the GUI session, even though it is started from a SYSTEM account.
But only if the user is already logged in.

---

If the user is not already logged in, the shadow command will just fail:

New ssl connection received
 from 'CLIENTIP:44224'
 on '0.0.0.0:14500'
Authentication required by win32 authenticator module 1
sending challenge for username 'windows 10 test' using xor digest
proxy_auth win32.get_sessions()=(0, 0, [], {}, {})
proxy_auth(Protocol(..), {..}, None) found sessions: (0, 0, [], {}, {})
proxy_session: displays=[], start_sessions=True, start-new-session={b'mode': b'shadow', b'display': b''}
start_new_session('windows 10 test', 0, 0, {b'mode': b'shadow', b'display': b''}, [])
start_win32_shadow('windows 10 test', {b'mode': b'shadow', b'display': b''})
exec_command('windows 10 test', ['paexec.exe', '-i', '1', '-s', 'C:\\Program Files\\Xpra\\Xpra-Shadow.exe', '--bind=windows_10_test', '-d', 'proxy,win32'], {..})
lsa_logon_user(..)
logon_msv1_s4u(windows 10 test)=..
creation_info=<xpra.platform.win32.create_process_lib.CREATIONINFO object at 0x0000000000c9ca40>
Popen(['paexec.exe', '-i', '1', '-s', 'C:\\Program Files\\Xpra\\Xpra-Shadow.exe', '--bind=windows_10_test', '-d', 'proxy,win32'])=<xpra.platform.win32.create_process_lib.Popen object at 0x0000000000ceb070>
poll()=4294967287
stdout=b''
stderr=b''
start_server_subprocess failed
Traceback (most recent call last):
  File "E:\Xpra\trunk\src/xpra/server/proxy/proxy_server.py", line 349, in proxy_session
    proc, socket_path, display = self.start_new_session(username, uid, gid, sns, displays)
  File "C:\Program Files\Xpra\lib\xpra\platform\win32\proxy_server.py", line 49, in start_new_session
    return self.start_win32_shadow(username, new_session_dict)
  File "C:\Program Files\Xpra\lib\xpra\platform\win32\proxy_server.py", line 94, in start_win32_shadow
    raise Exception("shadow subprocess failed with exit code %s" % r)
Exception: shadow subprocess failed with exit code 4294967287
Error: failed to start server subprocess:
 shadow subprocess failed with exit code 4294967287
disconnect(server error, ('failed to start a new session',))

So we need some other call to create the GUI session. No idea which one.

Pointers:

• Create Windows session programmatically
• Remote Desktop Protocol Provider API
• Win32-OpenSSH : CMDKEY fails to add credential in ssh session 0
• Win32-OpenSSH : Opening native graphical application from remote shell

[totaam]

creating window station and windows desktop using c#

Programmatically create and launch and RDP session (without gui)

All of those solutions use windows forms (GUI based), Creating a Remote Desktop Client Application without using Windows Forms (C#), How to use ActiveX component in ClassLibrary without Winforms.

To compile any of those C# based solutions, we need the AxMSTSCLib.dll.
Where is the AxMSTSCLib library located? The answer (not found in that link) is actually here: SharpRDP: If you do not want to use the provided DLLs you will need to .NET SDK to create the AxMSTSCLib.dll DLL. To create it you'll need to run aximp from the SDK on mstscax.dll. %%\aximp.exe %windir%\system32\mstscax.dll.
(aximp.exe)

To logoff once we're done: WTSLogoffSession function

Eventually, I would like to compile this DLL from the command line (Command-line build with csc.exe), as part of the build process.

[totaam]

Next difficulty: calling this dotnet code from the python server without using pythonnet since that's not supported under mingw.

Options:

• Exposing .NET components to COM - don't like COM, python-comtypes is not reliable enough
• Calling a C# library from python
• How to load a C# dll in python? - the wrapper approach?
• RGiesecke.DllExport / UnmanagedExports

[totaam]

Updates in r26196 + r26197 including the VS project to create the DesktopLogon.dll.
Reverting r25977 should work, but paexec is now failing (was working fine before..)

The server log files can be located using xpra info | grep log-file.
On my win10 test system, they end up in: C:\Windows\System32\config\systemprofile\AppData\Local\Xpra.
(both the Xpra-Proxy.log and Xpra-Shadow.log)

[totaam]

2020-05-01 15:49:42: @totaam uploaded file win32-system-proxy.patch (3.8 KiB)

switch back to launching the proxy

[totaam]

With the patch above, the proxy tries to run:

paexec.exe -i 1 -s "C:\Program Files\Xpra\Xpra-Shadow.exe" --bind=Windows_10_Test -d win32,proxy

When trying this command from an ssh session, paexec now fails (used to work?!):

Remote app failed to start.  Returned error:
  Failed to start "C:\Program Files\Xpra\Xpra-Shadow.exe" --bind=Windows_10_Test -d win32,proxy. Access is denied. [Err=0x5, 5]

PAExec returning exit code -9

Then I also got File not found - that one was completely misleading, nothing to do with the path, just the -i argument needed a different session id!

[totaam]

2020-05-02 06:53:59: @totaam uploaded file sessions.py (7.5 KiB)

various ill-fated attempts to get the list of sessions

[totaam]

Sort of working after:

• r26210 : xpra service uses the installation path we place in the registry
• r26211 : we can't use the systray on the login screen (avoid errors)
• r26212 : default to using the paexec we bundle (don't rely on psexec being installed)
• r26213 : don't log the password, also show stdout and stdin when process exited without an error
• r26214 : python half of the desktoplogon dll
• r26215 + r26216 : wts ctypes definitions
• r26219 + r26222 : locate the correct session id to use for the username given (fixes issue from comment:30)
• r26220 : disable debug
• r26221 : to be able to access the password and use it again, win32_auth keeps hold of it - needs a better solution
• r26225 : exit-with-children by default, poll the named-pipe, etc..
• r26226 : we can't use multiprocessing with the proxy server on win32, so use a threaded proxy instead
• r26227 : switch back to running the proxy server as the service

---

TODO:

• proxy shows up a cmd window
• local named pipe shadow connection chokes on memoryview:

Error: write connection \\.\pipe\Xpra\test reset
 bytes or integer address expected instead of memoryview instance: 0

• proxy becomes unresponsive
• activex Logon doesn't work? (doesn't run on the correct desktop?)
• nvenc slows down server startup? (causes timeouts?)
• shadow server errors when the login screen is shown (cursor, capture, etc) - rate limit them?
• fix multiprocessing:

• TypeError: can't pickle _thread.lock objects
• multiprocessing.Pool - PicklingError: Can't pickle <type 'thread.lock'>: attribute lookup thread.lock failed

• generic way to keep hold of the password (so we can use this with other authentication modules)
• stop proxy using xpra stop to prevent zombies? (needs peercred, or a private socket)
• fix event messages using message files
• add a proxy service config file so we don't need to modify win32_proxy_service.py to change settings (ie: debug), and rename to just "xpra service" since we can then configure "shadow" or "proxy" on demand
• shadow server exit does not remove tray (not until the pointer hover over it)
• add icon to proxy service as per How do I add an icon to a mingw-gcc compiled executable?

---

Other APIs and projects which may still be useful:

• NetWkstaGetInfo: In addition to level 101 information, return the number of users who are logged on to the local computer
• WTSLogoffSession on exit when we created the session ourselves?
• SetSuspendState ie: ovirt hibernate
• LockWorkStation ie: ovirt LockWorkstation
• SetThreadDesktop, CreateDesktopA, OpenDesktopA
• SetProcessWindowStation, OpenWindowStationA
• rdpy

[totaam]

Updates:

• r26228 also update default command (oops)
• r26231 fix named pipe connections handling of memoryview
• r26232 + r26233: more ctypes API definitions
• r26234 (wrong commit message): make the proxy server's named-pipe private
• r26235 try to force redirection of shadow instances output
• r26237 + r26238 + r26241 (fixup) hide the cmd window when lauching the proxy / paexec
• r26239 rate limit cursor warnings
• r26240 handle workarea calculation errors
• r26242 rename, generalize and stop the proxy using an xpra stop command

New weird bug: the service responds on boot (if auto-start is on), responds if started using the service manager when logged in, but it seems to die when a user logs out? (only if it had started a shadow server? even if that process is now gone.. maybe a SIGPIPE?)

So the DesktopSession.dll hack still doesn't work, but using rdesktop -i $USER -p $PASSWORD $IP does what we need ... Just need to emulate it? Without actually showing anything via RDP, and without terminating the session... because disconnecting RDP stops our server receiving anything - the station / desktop are suspended? (logging in resumes the session!)

New links:

• Terminal Server Walkthrough: Startup, Connection, and Application
• How to start a new Windows logon session (RDP or console) programmatically
• How to programmatically interact with winlogon?
• Window Station and Desktop Creation
• Understanding Windows at a deeper level - Sessions, Window Stations, and Desktops
• Remote Desktop ActiveX control reference
• How to show AxMsRdpClient9NotSafeForScripting object programmatically?
• Creating a Remote Desktop Client Application without using Windows Forms (C#)
• How to use ActiveX component in ClassLibrary without Winforms
• Programmatically create and launch and RDP session (without gui)

[totaam]

This will have to do for this release, follow up in #2756.

The proxy server does work and can start a shadow server for any user, but it is unable to create the station / desktop if one does not exist already..