Paramiko SSH key issue

[jason-weirather]

Running wes automator on a new environment yields an error

Traceback (most recent call last):
  File "./wes_automator.py", line 366, in <module>
    main()
  File "./wes_automator.py", line 287, in main
    _zone)
  File "./wes_automator.py", line 108, in createInstanceDisk
    connection = ssh(ip_addr, ssh_config['user'], ssh_config['key'])
  File "./wes_automator.py", line 31, in __init__
    self.client.connect(address, username=username, key_filename=key_filename, look_for_keys=False)
  File "/Users/jasonw/miniconda3/envs/wes_automator/lib/python2.7/site-packages/paramiko/client.py", line 446, in connect
    passphrase,
  File "/Users/jasonw/miniconda3/envs/wes_automator/lib/python2.7/site-packages/paramiko/client.py", line 764, in _auth
    raise saved_exception
paramiko.ssh_exception.SSHException: Invalid key(base)

Issue may be exclusive to mac os as I think it partially relates to how keys are labeled and paramiko's assumptions.

@lentaing if users encounter, my workaround was to generate a key in PEM format then log into a VM to propagate that key to the project.

    $ ssh-keygen -t rsa -m PEM -f ~/.ssh/google_compute_engine
    $ gcloud compute config-ssh
    $ gcloud compute ssh taing@wes-auto-jason-wes-local-test4

[lentaing]

Hi Jason,

I've never encountered this error. I run wes_automator fine on my own mac air, so I'm not sure that is the issue. Can you share the steps you took so I can try to recreate the bug? And where exactly you implemented your work-around to get it to work?

Also, I looked at the paramiko client.py code and this looks to be the relevant code snippet: (~/miniconda3/envs/wes_automator/lib/python2.7/site-packages/paramiko/client.py lines 755-760):
elif two_factor:
try:
self._transport.auth_interactive_dumb(username)
return
except SSHException as e:
saved_exception = e

This is raising a no-too-informative SSHException; it could be the key or it could be the username or it could be something else.

[jason-weirather]

Hi @lentaing

As I best I could understand from reading other similar issues, current os x when you run ssh-keygen creates an RSA key with head / tail that is like --BEGIN OPENSSH KEY-- or something like that whereas the old keys are of the format --BEGIN RSA KEY--. If you can check your private key and see if the header line contains "OPENSSH", if it does, then my characterization of this error is probably not correct, and it may have more to do with me not properly propagating keys to the project in my new environment. If you see "RSA" in your header then you may be able to reproduce this error either by running on a fresh environment or updating components in yours and trying the following

Steps to recreate

1. Generate a non-PEM formatted private key for the google cloud key. This should have "OPENSSH" in the preamble of the private key rather than the word "RSA"
2. Configure that key as the google cloud key
3. Verify you can log into a VM using your new key
4. Try to run wes automator with your new key

[lentaing]

Hi Jason,

Which Mac OSX version are you on? I'm still on MacOSX Sierra (10.12.6).

My computer generates private keys with this header:
-----BEGIN RSA PRIVATE KEY-----

I googled around and found this paramiko issue-
paramiko/paramiko#340

It mentions that at least in an older versions of paramiko, the start of the header mattered (search for "Normally, keys start with this line, and this is what paramiko is checking for:")

We're on paramiko version 2.6.0, so I'm not sure if paramiko is still hard-coding this header requirement. But could another work-around be as simple as changing the header and footer to:
-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----
?