Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/crypto/ssh: client requires first hostkey to match, knownhosts doesn't expose available key types #29286

Open
beiriannydd opened this issue Dec 15, 2018 · 13 comments
Labels
NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@beiriannydd
Copy link

beiriannydd commented Dec 15, 2018

What version of Go are you using (go version)?

$ go version
go version go1.11.3 linux/amd64

Does this issue reproduce with the latest release?

This is the latest docker release (yes)

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/fsalwin/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/Users/fsalwin/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build745071247=/tmp/go-build -gno-record-gcc-switches"

What did you do?

	config := &ssh.ClientConfig{
		User:            GetRemoteUsername(host),
		HostKeyCallback: knownHostsCallback,
		Timeout:         30 * time.Second,
		Auth:            []ssh.AuthMethod{},
	}

knownHostsCallback is an ssh.HostKeysCallback.

2 machines:
machine A has ecdsa-sha2-nistp256 key for remote host in known hosts file
machine B has ssh-ed25519 key for remote host in known hosts file

If machine A connects to the remote host, the remote host sends the ecdsa-sha2-nistp256 key and is allowed to continue the handshake
If machine B connects to the remote host, the remote host sends the ecdsa-sha2-nistp256 key and is rejected because the callback returns an error.

If I set the host key list to prefer ["ssh-ed25519","ecdsa-sha2-nistp256"...]
Machine A fails
Machine B succeeds

What did you expect to see?

Since there is no mechanism for discovering the available hostkeys for a host published in ssh/knownhosts , I expected it to try each hostkey in the preference list in turn, failing only if all the keys had been tried.

What did you see instead?

SSH Client ceases handshake after receiving the first error response from the HostKeysCallback.

Since only the method on the database is returned by knownhosts.New() it is not easy to add another method accessing the same hostKeyDB instance.

I propose adding an initializing function for the database from knownhosts which returns the database - NewDB to complement New which would return a hostkeys interface.

There would be published methods on the receiver:

type KnownHostDB interface {
	// HostKeyCallback is knownhosts.New without the DB initialization.
	HostKeyCallback() ssh.HostKeyCallback
	// HostKeyAlgorithms takes an address and returns a list of matching key types.
	HostKeyAlgorithms(address string) ([]string, error)
}
// NewDB is knownhosts.New without the callback code
func NewDB(filename string) (KnownHostDB, error)

That way you can just use:

	knownHosts := knownhosts.NewDB(knownhostfile)
	// just to cut down on example code, the error is ignored.
	algos, _ := knownHosts.HostKeyAlgorithms(host)
	config := &ssh.ClientConfig{
		User:            GetRemoteUsername(host),
		HostKeyCallback: knownHosts.HostKeysCallback(),
		Timeout:         30 * time.Second,
		Auth:            []ssh.AuthMethod{},
		HostKeyAlgorithms: algos,
	}

Alternatively if the protocol allows for it, the host key algorithms could be tried in turn until you run out of algorithms (fail) or you have a match (success)

@beiriannydd
Copy link
Author

golang/crypto#68

@agnivade
Copy link
Contributor

If you are proposing to add new API, then this should be a proposal, rather than a bug report.

/cc @hanwen

@hanwen
Copy link
Contributor

hanwen commented Dec 17, 2018

you can have only one hostkey algorithm per connection attempt, so it doesn't make sense to automate this, I think.

We could look at exposing the DB in knownhosts. I think that should just make the existing types public, rather than introducing an interface.

@beiriannydd
Copy link
Author

beiriannydd commented Dec 17, 2018

db is a local inside the New() method currently, are you suggesting making it create a global, or having a different method which returns a db, which you call instead of .New()?
If you return the db then you need a different method to construct the CertChecker for use as the callback function.

@hanwen
Copy link
Contributor

hanwen commented Dec 18, 2018

different method.

@beiriannydd
Copy link
Author

So it sounds like the only thing you don't like about my solution is that it provides the interface. I guess the wonderful thing about Go is that you don't have to know that you respond to an interface, so as long as we call the methods something sane, people who need to be able to access file based knownhosts and eg. registry based knownhosts can create their own interface.

@hanwen
Copy link
Contributor

hanwen commented Jan 23, 2019

interfaces should be defined where they are consumed, and functions should return concrete types if possible. See https://github.com/golang/go/wiki/CodeReviewComments#interfaces

I know that the SSH package violates this guidelines in some places, but unfortunately, we can't fix it without breaking callers. However, I don't want to introduce more instances of this pattern.

@TheoBrigitte
Copy link

I've run into this issue recently, where x/crypto/ssh fails to match host key for github.com with the one in my known_hosts file. But my regular openssh client does work correctly.

known_hosts contains a host key for ssh-ed25519 algorithm.

$ cat /home/theo/.ssh/known_hosts|grep github.com
github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl

openssh client do use the correct ssh-ed25519 host key algorithm
openssh-client.log

golang/x/crypto fail and use the ecdsa-sha2-nistp256 host key algorithm
debug.log (running a slightly modified version with more debug log)


Current workaround: manually get the host key with ecdsa-sha2-nistp256 algorithm

$ sed -i '/^github.com/d' ~/.ssh/known_hosts
$ ssh -o HostKeyAlgorithms=ecdsa-sha2-nistp256 [email protected]

@rolandshoemaker
Copy link
Member

This seems like intended behavior? If ClientConfig.HostKeyAlgorithms is not set, a reasonable default is set for acceptable host key type, which may be one for which you do not have a matching host key. In general if you are providing a ClientConfig.HostKeyCallback, it seems like you should be setting ClientConfig.HostKeyAlgorithms to a list of algorithms that your callback will accept keys for.

For @TheoBrigitte, is this fixed by setting ClientConfig.HostKeyAlgorithms = []string{KeyAlgoED25519}?

@TheoBrigitte
Copy link

For @TheoBrigitte, is this fixed by setting ClientConfig.HostKeyAlgorithms = []string{KeyAlgoED25519}?

Yes this fixes it, but ideally I would not have to set this manually, and it should be auto-discovered as in openssh client.

@FiloSottile
Copy link
Contributor

It does look like there should be an easy way to set or better yet sort ClientConfig.HostKeyAlgorithms based on the available known keys (which is how OpenSSH handles this), but it's not immediately clear what it would be, especially when using FixedHostKey. At the very least, we should document the need to match them.

(Relatedly, OpenSSH has a discovery mechanism for new host key algorithms, [email protected]. Might be interesting adding a callback for it.)

@FiloSottile
Copy link
Contributor

Actually, we can make this almost automagic. We can type-assert the HostKeyCallback returned by knownhosts.New and by FixedHostKey in the ssh client and prioritize those key types automatically. Then document you need to do it yourself for custom callbacks.

(We can't actually type-assert across packages, but an interface upgrade might work even if the named type is a function. If not, we can add knownhosts.KeyFormatForCallback.)

@seankhliao seankhliao added the NeedsFix The path to resolution is known, but the work has not been done. label Jun 5, 2022
evanelias added a commit to evanelias/go-git that referenced this issue Jun 20, 2022
…Fixes go-git#411

This commit adjusts the transport/ssh logic in command.connect(), so that it
now auto-populates ssh.ClientConfig.HostKeyAlgorithms. The algorithms are
chosen based on the known host keys for the target host, as obtained from the
known_hosts file.

In order to look-up the algorithms from the known_hosts file, external module
github.com/skeema/knownhosts is used. This package is just a thin wrapper
around golang.org/x/crypto/ssh/knownhosts, adding an extra mechanism to query
the known_hosts keys, implemented in a way which avoids duplication of any
golang.org/x/crypto/ssh/knownhosts logic.

Because HostKeyAlgorithms vary by target host, some related logic for setting
HostKeyCallback has been moved out of the various AuthMethod implementations.
This was necessary because the old HostKeyCallbackHelper is not host-specific.
Since known_hosts handling isn't really tied to AuthMethod anyway, it seems
reasonable to separate these. Previously-exported types/methods remain in
place for backwards compat, but some of them are now unused.

For testing approach, see pull request. Issue go-git#411 can only be reproduced
via end-to-end / integration testing, since it requires actually launching
an SSH connection, in order to see the key mismatch error triggered from
golang/go#29286 as the root cause.
@evanelias
Copy link

For anyone else stuck on this: it's actually possible to populate ClientConfig.HostKeyAlgorithms based on the contents of known_hosts, even without any code changes to x/crypto/ssh/knownhosts... it's just extremely non-obvious. Essentially, you can ask knownhosts for all keys for a specific host by using a subtle hack: invoke the host key callback with a valid host but a bogus key. The resulting KeyError.Want contains the list of the valid known public keys for that host; from there you can extract out the types and then populate HostKeyAlgorithms correctly and automatically 🥂

I've just open-sourced a new package github.com/skeema/knownhosts which is a thin wrapper around x/crypto/ssh/knownhosts making it easier to use this technique, as well as adding a few other niceties. In my testing it consistently solves this problem, at least when using known_hosts for host key callbacks. (My package does not handle FixedHostKey, as I suspect that would require x/crypto/ssh changes to be possible.)

mcuadros pushed a commit to go-git/go-git that referenced this issue Sep 22, 2022
…Fixes #411

This commit adjusts the transport/ssh logic in command.connect(), so that it
now auto-populates ssh.ClientConfig.HostKeyAlgorithms. The algorithms are
chosen based on the known host keys for the target host, as obtained from the
known_hosts file.

In order to look-up the algorithms from the known_hosts file, external module
github.com/skeema/knownhosts is used. This package is just a thin wrapper
around golang.org/x/crypto/ssh/knownhosts, adding an extra mechanism to query
the known_hosts keys, implemented in a way which avoids duplication of any
golang.org/x/crypto/ssh/knownhosts logic.

Because HostKeyAlgorithms vary by target host, some related logic for setting
HostKeyCallback has been moved out of the various AuthMethod implementations.
This was necessary because the old HostKeyCallbackHelper is not host-specific.
Since known_hosts handling isn't really tied to AuthMethod anyway, it seems
reasonable to separate these. Previously-exported types/methods remain in
place for backwards compat, but some of them are now unused.

For testing approach, see pull request. Issue #411 can only be reproduced
via end-to-end / integration testing, since it requires actually launching
an SSH connection, in order to see the key mismatch error triggered from
golang/go#29286 as the root cause.
rohankmr414 pushed a commit to rohankmr414/go-git that referenced this issue Oct 12, 2022
…Fixes go-git#411

This commit adjusts the transport/ssh logic in command.connect(), so that it
now auto-populates ssh.ClientConfig.HostKeyAlgorithms. The algorithms are
chosen based on the known host keys for the target host, as obtained from the
known_hosts file.

In order to look-up the algorithms from the known_hosts file, external module
github.com/skeema/knownhosts is used. This package is just a thin wrapper
around golang.org/x/crypto/ssh/knownhosts, adding an extra mechanism to query
the known_hosts keys, implemented in a way which avoids duplication of any
golang.org/x/crypto/ssh/knownhosts logic.

Because HostKeyAlgorithms vary by target host, some related logic for setting
HostKeyCallback has been moved out of the various AuthMethod implementations.
This was necessary because the old HostKeyCallbackHelper is not host-specific.
Since known_hosts handling isn't really tied to AuthMethod anyway, it seems
reasonable to separate these. Previously-exported types/methods remain in
place for backwards compat, but some of them are now unused.

For testing approach, see pull request. Issue go-git#411 can only be reproduced
via end-to-end / integration testing, since it requires actually launching
an SSH connection, in order to see the key mismatch error triggered from
golang/go#29286 as the root cause.
gibchikafa pushed a commit to gibchikafa/go-git that referenced this issue Nov 23, 2022
…Fixes go-git#411

This commit adjusts the transport/ssh logic in command.connect(), so that it
now auto-populates ssh.ClientConfig.HostKeyAlgorithms. The algorithms are
chosen based on the known host keys for the target host, as obtained from the
known_hosts file.

In order to look-up the algorithms from the known_hosts file, external module
github.com/skeema/knownhosts is used. This package is just a thin wrapper
around golang.org/x/crypto/ssh/knownhosts, adding an extra mechanism to query
the known_hosts keys, implemented in a way which avoids duplication of any
golang.org/x/crypto/ssh/knownhosts logic.

Because HostKeyAlgorithms vary by target host, some related logic for setting
HostKeyCallback has been moved out of the various AuthMethod implementations.
This was necessary because the old HostKeyCallbackHelper is not host-specific.
Since known_hosts handling isn't really tied to AuthMethod anyway, it seems
reasonable to separate these. Previously-exported types/methods remain in
place for backwards compat, but some of them are now unused.

For testing approach, see pull request. Issue go-git#411 can only be reproduced
via end-to-end / integration testing, since it requires actually launching
an SSH connection, in order to see the key mismatch error triggered from
golang/go#29286 as the root cause.
durandj pushed a commit to durandj/go-git that referenced this issue Jul 1, 2023
…Fixes go-git#411

This commit adjusts the transport/ssh logic in command.connect(), so that it
now auto-populates ssh.ClientConfig.HostKeyAlgorithms. The algorithms are
chosen based on the known host keys for the target host, as obtained from the
known_hosts file.

In order to look-up the algorithms from the known_hosts file, external module
github.com/skeema/knownhosts is used. This package is just a thin wrapper
around golang.org/x/crypto/ssh/knownhosts, adding an extra mechanism to query
the known_hosts keys, implemented in a way which avoids duplication of any
golang.org/x/crypto/ssh/knownhosts logic.

Because HostKeyAlgorithms vary by target host, some related logic for setting
HostKeyCallback has been moved out of the various AuthMethod implementations.
This was necessary because the old HostKeyCallbackHelper is not host-specific.
Since known_hosts handling isn't really tied to AuthMethod anyway, it seems
reasonable to separate these. Previously-exported types/methods remain in
place for backwards compat, but some of them are now unused.

For testing approach, see pull request. Issue go-git#411 can only be reproduced
via end-to-end / integration testing, since it requires actually launching
an SSH connection, in order to see the key mismatch error triggered from
golang/go#29286 as the root cause.
l0rd added a commit to l0rd/containers-common that referenced this issue Oct 22, 2024
golang.org/x/crypto/ssh/knownhosts has an issue when
the SSH server key in the `known_hosts` file doesn't
match the first key returned by the SSH server (but
should be still considered valid because it matches
one of the other keys of the SSH server).
See golang/go#29286 and
containers/podman#23575.

Signed-off-by: Mario Loriedo <[email protected]>
l0rd added a commit to l0rd/containers-common that referenced this issue Oct 22, 2024
Package [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts)
has an issue when an SSH server has many public keys (i.e. supports multiple crypto algorithms).

For instance, if the local `known_hosts` file entries don't match the first SSH server key but match
other keys of the SSH server, the handshake fails with a key mismatch error.

See golang/go#29286 and containers/podman#23575.

Package [github.com/skeema/knownhosts](https://github.com/skeema/knownhosts) is a wrapper of
`x/crypto/ssh/knownhosts` that addresses this issue.

This commit replaces the usage of `x/crypto/ssh/knownhosts` in containers/common with
`github.com/skeema/knownhosts`.

Signed-off-by: Mario Loriedo <[email protected]>
l0rd added a commit to l0rd/containers-common that referenced this issue Oct 23, 2024
Package [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts)
has an issue when an SSH server has many public keys (i.e. supports multiple crypto algorithms).

For instance, if the local `known_hosts` file entries don't match the first SSH server key but match
other keys of the SSH server, the handshake fails with a key mismatch error.

See golang/go#29286 and containers/podman#23575.

Package [github.com/skeema/knownhosts](https://github.com/skeema/knownhosts) is a wrapper of
`x/crypto/ssh/knownhosts` that addresses this issue.

This commit replaces the usage of `x/crypto/ssh/knownhosts` in containers/common with
`github.com/skeema/knownhosts`.

Signed-off-by: Mario Loriedo <[email protected]>
l0rd added a commit to l0rd/containers-common that referenced this issue Oct 23, 2024
Package [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts)
has an issue when an SSH server has many public keys (i.e. supports multiple crypto algorithms).

For instance, if the local `known_hosts` file entries don't match the first SSH server key but match
other keys of the SSH server, the handshake fails with a key mismatch error.

See golang/go#29286 and containers/podman#23575.

Package [github.com/skeema/knownhosts](https://github.com/skeema/knownhosts) is a wrapper of
`x/crypto/ssh/knownhosts` that addresses this issue.

This commit replaces the usage of `x/crypto/ssh/knownhosts` in containers/common with
`github.com/skeema/knownhosts`.

Signed-off-by: Mario Loriedo <[email protected]>
l0rd added a commit to l0rd/containers-common that referenced this issue Oct 25, 2024
Package [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts)
has an issue when an SSH server has many public keys (i.e. supports multiple crypto algorithms).

For instance, if the local `known_hosts` file entries don't match the first SSH server key but match
other keys of the SSH server, the handshake fails with a key mismatch error.

See golang/go#29286 and containers/podman#23575.

Package [github.com/skeema/knownhosts](https://github.com/skeema/knownhosts) is a wrapper of
`x/crypto/ssh/knownhosts` that addresses this issue.

This commit replaces the usage of `x/crypto/ssh/knownhosts` in containers/common with
`github.com/skeema/knownhosts`.

Signed-off-by: Mario Loriedo <[email protected]>
traidare pushed a commit to traidare/go-git that referenced this issue Oct 26, 2024
…Fixes go-git#411

This commit adjusts the transport/ssh logic in command.connect(), so that it
now auto-populates ssh.ClientConfig.HostKeyAlgorithms. The algorithms are
chosen based on the known host keys for the target host, as obtained from the
known_hosts file.

In order to look-up the algorithms from the known_hosts file, external module
github.com/skeema/knownhosts is used. This package is just a thin wrapper
around golang.org/x/crypto/ssh/knownhosts, adding an extra mechanism to query
the known_hosts keys, implemented in a way which avoids duplication of any
golang.org/x/crypto/ssh/knownhosts logic.

Because HostKeyAlgorithms vary by target host, some related logic for setting
HostKeyCallback has been moved out of the various AuthMethod implementations.
This was necessary because the old HostKeyCallbackHelper is not host-specific.
Since known_hosts handling isn't really tied to AuthMethod anyway, it seems
reasonable to separate these. Previously-exported types/methods remain in
place for backwards compat, but some of them are now unused.

For testing approach, see pull request. Issue go-git#411 can only be reproduced
via end-to-end / integration testing, since it requires actually launching
an SSH connection, in order to see the key mismatch error triggered from
golang/go#29286 as the root cause.
l0rd added a commit to l0rd/containers-common that referenced this issue Oct 28, 2024
Package [golang.org/x/crypto/ssh/knownhosts](https://pkg.go.dev/golang.org/x/crypto/ssh/knownhosts)
has an issue when an SSH server has many public keys (i.e. supports multiple crypto algorithms).

For instance, if the local `known_hosts` file entries don't match the first SSH server key but match
other keys of the SSH server, the handshake fails with a key mismatch error.

See golang/go#29286 and containers/podman#23575.

Package [github.com/skeema/knownhosts](https://github.com/skeema/knownhosts) is a wrapper of
`x/crypto/ssh/knownhosts` that addresses this issue.

This commit replaces the usage of `x/crypto/ssh/knownhosts` in containers/common with
`github.com/skeema/knownhosts`.

Signed-off-by: Mario Loriedo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

9 participants