-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add SSHManager support for invoking Windows workers via cmd.exe #38353
Conversation
LGTM pending #38352 |
Is it possible to make this work for when ssh conects to powershell instead of cmd? Maybe by first starting cmd.exe? |
Adding PowerShell command-invocation support is on my todo list, but will be a future, separate PR. (Proper meta-character escaping in PowerShell sadly can be even more complicated than in |
So it wouldn't be possible to just start cmd.exe from powershell and then use the work from this PR? |
Looks like this needs a rebase to remove the cmd.exe escaping functions since we've merged #38352 |
Distributed.addprocs() now supports four new keyword arguments `shell`, `ssh`, `env` and `cmdline_cookie`. Specifying `shell=:wincmd` now makes it possible to start workers on a Windows machine with an sshd server that invokes `cmd.exe` as the shell (e.g. Microsoft's OpenSSH port does that by default). Previously SSHManager only supported ssh connections to a POSIX shell. Specifying `ssh="/usr/bin/ssh"` makes it possible to specify the ssh client that SSHManager will use (useful for debugging and where a custom-version of ssh is required). The new `env` parameter now allows to pass arbitrary environment variables to workers. Specifying `cmdline_cookie=true` is a workaround for an ssh problem with Windows workers that run older (pre-ConPTY) version of Windows, Julia, or OpenSSH.
Rebased after merge of #38352 |
Failures look unrelated. Will merge tomorrow sans objections. |
I made this change to allow the set to be expanded for my own package, but I noticed this also helps unify #38353 and existing ssh-only options.
I made this change to allow the set to be expanded for my own package, but I noticed this also helps unify #38353 and existing ssh-only options.
Distributed.addprocs() now supports four new keyword arguments `shell`, `ssh`, `env` and `cmdline_cookie`. Specifying `shell=:wincmd` now makes it possible to start workers on a Windows machine with an sshd server that invokes `cmd.exe` as the shell (e.g. Microsoft's OpenSSH port does that by default). Previously SSHManager only supported ssh connections to a POSIX shell. Specifying `ssh="/usr/bin/ssh"` makes it possible to specify the ssh client that SSHManager will use (useful for debugging and where a custom-version of ssh is required). The new `env` parameter now allows to pass arbitrary environment variables to workers. Specifying `cmdline_cookie=true` is a workaround for an ssh problem with Windows workers that run older (pre-ConPTY) version of Windows, Julia, or OpenSSH.
Distributed.addprocs() now supports four new keyword arguments
shell
,ssh
,env
andcmdline_cookie
.Specifying
shell=:wincmd
now makes it possible to start workers on aWindows machine with an sshd server that invokes
cmd.exe
as the shell(e.g. Microsoft's OpenSSH port does that by default). Previously
SSHManager only supported ssh connections to a POSIX shell.
Specifying
ssh="/usr/bin/ssh"
makes it possible to specify the sshclient that SSHManager will use (useful for debugging and where a
custom-version of ssh is required).
The new
env
parameter now allows to pass arbitrary environmentvariables to workers.
Specifying
cmdline_cookie=true
is a workaround for an ssh problemwith Windows workers that run older (pre-ConPTY) version of Windows,
Julia, or OpenSSH.
(This is a revised version of PR #30614)