unsupported cipher algorithm (rc4) breaks winrm and anything using it on Windows (test-kitchen, kitchen-azurerm, kitchen-vagrant, ...)

[mirogta]

Hi,

I appreciate removal of the unsecure cipher RC4 as part of #50 - Remove RC4.

Unfortunately it has dire consequences, because it broke heaps of workflows since usage of RC4 cipher is hard-coded in
https://github.com/WinRb/rubyntlm/blob/master/lib/net/ntlm/client/session.rb

...and even if we change it to something "sane" like "aes-128-gcm" it just doesn't work.

It took us tremendous effort to get to the bottom of this, because of red herring errors leading nowhere and if you google "unsecure cipher algorithm (RC4)" you don't really get useful answers. Finally I've found the message is triggered here:
https://github.com/ruby/openssl/blob/b3825945d11dd2f18f9f3b07a7a2b61b7b752591/ext/openssl/ossl_cipher.c

Please, can you help? Since the rubyntlm project looks dead, I was wondering if you could add a backward compatible support to re-enable RC4 cipher when specified, so that it stays disabled by default but we can enable it when we actually need to use it? Otherwise we can throw away years of development on Test-Kitchen with WinRM.

Since this would be used only to connect from a local development box to another local development box running in e.g. Vagrant VM, we don't really care if we use unsecure RC4 algorithm, but we do care that our code and workflows we've developed over the years keeps working.

Unfortunately our team is in no way ready to start hacking openssl, rubyntlm or winrm gems to make it work by ourselves.

Thank you

[emre141]

Hi

I am also my testing environment on my local virtualbox that is running Openshift platform and a defined my github account a webhook and to communicate both them configure ultrahook but there is a redirect traffic from payload url to private unsecure (https.//private_network/8443/oapi/v1/names paces/third/buildconfigs/php/webhooks/4f940a9e-6281-4f08-9386-a1ccfeb12bca/githu b) but i have get a ruby openssl issue as below

D:/Ruby24-x64/lib/ruby/2.4.0/openssl/buffering.rb:182:in sysread_nonblock': An existing connection was forcibly closed by the remote host. (Errno::ECONNRESET) from D:/Ruby24-x64/lib/ruby/2.4.0/openssl/buffering.rb:182:in read_nonb lock'
from D:/Ruby24-x64/lib/ruby/2.4.0/net/protocol.rb:172:in rbuf_fill' from D:/Ruby24-x64/lib/ruby/2.4.0/net/protocol.rb:154:in readuntil'
from D:/Ruby24-x64/lib/ruby/2.4.0/net/protocol.rb:164:in readline' from D:/Ruby24-x64/lib/ruby/2.4.0/net/http/response.rb:40:in read_statu s_line'
from D:/Ruby24-x64/lib/ruby/2.4.0/net/http/response.rb:29:in read_new' from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1446:in block in transpor t_request'
from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1443:in catch' from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1443:in transport_request '
from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1416:in request' from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1409:in block in request'
from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:877:in start' from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1407:in request'
from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1430:in send_entity' from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1218:in post'
from D:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/ultrahook-0.1.4/lib/ultrahoo k/client.rb:127:in http_post' from D:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/ultrahook-0.1.4/lib/ultrahoo k/client.rb:116:in process_request'
from D:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/ultrahook-0.1.4/lib/ultrahoo k/client.rb:90:in process' from D:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/ultrahook-0.1.4/lib/ultrahoo k/client.rb:160:in block (3 levels) in init_stream'
from D:/Ruby24-x64/lib/ruby/2.4.0/net/protocol.rb:429:in call_block' from D:/Ruby24-x64/lib/ruby/2.4.0/net/protocol.rb:420:in <<'
from D:/Ruby24-x64/lib/ruby/2.4.0/net/protocol.rb:126:in read' from D:/Ruby24-x64/lib/ruby/2.4.0/net/http/response.rb:321:in read_chun ked'
from D:/Ruby24-x64/lib/ruby/2.4.0/net/http/response.rb:285:in block in read_body_0' from D:/Ruby24-x64/lib/ruby/2.4.0/net/http/response.rb:278:in inflater'
from D:/Ruby24-x64/lib/ruby/2.4.0/net/http/response.rb:283:in read_body _0' from D:/Ruby24-x64/lib/ruby/2.4.0/net/http/response.rb:204:in read_body '
from D:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/ultrahook-0.1.4/lib/ultrahoo k/client.rb:155:in block (2 levels) in init_stream' from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1455:in block in transpor t_request'
from D:/Ruby24-x64/lib/ruby/2.4.0/net/http/response.rb:165:in reading_b ody' from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1454:in transport_request '
from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1416:in request' from D:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/ultrahook-0.1.4/lib/ultrahoo k/client.rb:153:in block in init_stream'
from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:877:in start' from D:/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:608:in start'
from D:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/ultrahook-0.1.4/lib/ultrahoo k/client.rb:148:in init_stream' from D:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/ultrahook-0.1.4/lib/ultrahoo k/client.rb:65:in start'
from D:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/ultrahook-0.1.4/bin/ultrahoo k:6:in <top (required)>' from D:/Ruby24-x64/bin/ultrahook:22:in load'
from D:/Ruby24-x64/bin/ultrahook:22:in `

'

There is any workaroud solution on ruby ?

Thank you

[rhenium]

#50 is about the default configuration used by OpenSSL::SSL::SSLContext#set_params and has nothing to do with OpenSSL::Cipher.

To my best knowledge, the error "unsupported cipher algorithm" indicates the OpenSSL library linked in has no RC4 support. What does running openssl enc -ciphers on terminal show?

@emre141 Please open a new issue for separate problems.

[mirogta]

>openssl enc -ciphers

unknown option '-ciphers'
options are
-in input file
-out output file
-pass pass phrase source
-e encrypt
-d decrypt
-a/-base64 base64 encode/decode, depending on encryption flag
-k passphrase is the next argument
-kfile passphrase is the first line of the file argument
-md the next argument is the md to use to create a key
from a passphrase. One of md2, md5, sha or sha1
-S salt in hex is the next argument
-K/-iv key/iv in hex is the next argument
-[pP] print the iv/key (then exit if -P)
-bufsize buffer size
-nopad disable standard block padding
-engine e use engine e, possibly a hardware device.
Cipher Types
-aes-128-cbc -aes-128-ccm -aes-128-cfb
-aes-128-cfb1 -aes-128-cfb8 -aes-128-ctr
-aes-128-ecb -aes-128-gcm -aes-128-ofb
-aes-128-xts -aes-192-cbc -aes-192-ccm
-aes-192-cfb -aes-192-cfb1 -aes-192-cfb8
-aes-192-ctr -aes-192-ecb -aes-192-gcm
-aes-192-ofb -aes-256-cbc -aes-256-ccm
-aes-256-cfb -aes-256-cfb1 -aes-256-cfb8
-aes-256-ctr -aes-256-ecb -aes-256-gcm
-aes-256-ofb -aes-256-xts -aes128
-aes192 -aes256 -bf
-bf-cbc -bf-cfb -bf-ecb
-bf-ofb -blowfish -camellia-128-cbc
-camellia-128-cfb -camellia-128-cfb1 -camellia-128-cfb8
-camellia-128-ecb -camellia-128-ofb -camellia-192-cbc
-camellia-192-cfb -camellia-192-cfb1 -camellia-192-cfb8
-camellia-192-ecb -camellia-192-ofb -camellia-256-cbc
-camellia-256-cfb -camellia-256-cfb1 -camellia-256-cfb8
-camellia-256-ecb -camellia-256-ofb -camellia128
-camellia192 -camellia256 -cast
-cast-cbc -cast5-cbc -cast5-cfb
-cast5-ecb -cast5-ofb -des
-des-cbc -des-cfb -des-cfb1
-des-cfb8 -des-ecb -des-ede
-des-ede-cbc -des-ede-cfb -des-ede-ofb
-des-ede3 -des-ede3-cbc -des-ede3-cfb
-des-ede3-cfb1 -des-ede3-cfb8 -des-ede3-ofb
-des-ofb -des3 -desx
-desx-cbc -id-aes128-CCM -id-aes128-GCM
-id-aes128-wrap -id-aes192-CCM -id-aes192-GCM
-id-aes192-wrap -id-aes256-CCM -id-aes256-GCM
-id-aes256-wrap -id-smime-alg-CMS3DESwrap -rc2
-rc2-40-cbc -rc2-64-cbc -rc2-cbc
-rc2-cfb -rc2-ecb -rc2-ofb
-rc4 -rc4-40 -rc4-hmac-md5
-seed -seed-cbc -seed-cfb
-seed-ecb -seed-ofb

[mirogta]

It seems that running openssl on a terminal has no effect on what is used by rubyntlm (via Ruby), since it is using a different version of openssl.so which has the RC4 cipher disabled in it.

[emre141]

Hi mirogta

Thanks for your answer.What is your suggestion there is any workaround to using rubyntlm? to using RC4 There is anything to doing on openshift server side or rubyntlm side ?

Thanks

[rhenium]

OpenSSL::Cipher.ciphers can also be used to get the list of supported algorithm names. Actually, Ruby/OpenSSL (openssl.so) doesn't have a hard-coded list, but just queries the OpenSSL library (libcrypto.so) at runtime.

[mirogta]

Currently we have no workaround, because we've moved to Puppet 5/Ruby 2.4. We have only little knowledge of NTLM to try to re-implement rubyntlm with another cipher - we've just tried to change the hard-coded RC4 to something else and it didn't work.

If a cipher is removed from openssl, it would be great to give backwards compatible support by some kind of configuration override.

Puppet 1.10 / Ruby 2.3.3
OpenSSL::Cipher.ciphers

=> ["AES-128-CBC", "AES-128-CBC-HMAC-SHA1", "AES-128-CBC-HMAC-SHA256", "AES-128-CFB", "AES-128-CFB1", "AES-128-CFB8", "AES-128-CTR", "AES-128-ECB", "AES-128-OFB", "AES-128-XTS", "AES-192-CBC",
"AES-192-CFB", "AES-192-CFB1", "AES-192-CFB8", "AES-192-CTR", "AES-192-ECB", "AES-192-OFB", "AES-256-CBC", "AES-256-CBC-HMAC-SHA1", "AES-256-CBC-HMAC-SHA256", "AES-256-CFB", "AES-256-CFB1", "
AES-256-CFB8", "AES-256-CTR", "AES-256-ECB", "AES-256-OFB", "AES-256-XTS", "AES128", "AES192", "AES256", "BF", "BF-CBC", "BF-CFB", "BF-ECB", "BF-OFB", "CAMELLIA-128-CBC", "CAMELLIA-128-CFB", "
CAMELLIA-128-CFB1", "CAMELLIA-128-CFB8", "CAMELLIA-128-ECB", "CAMELLIA-128-OFB", "CAMELLIA-192-CBC", "CAMELLIA-192-CFB", "CAMELLIA-192-CFB1", "CAMELLIA-192-CFB8", "CAMELLIA-192-ECB", "CAMELLIA
-192-OFB", "CAMELLIA-256-CBC", "CAMELLIA-256-CFB", "CAMELLIA-256-CFB1", "CAMELLIA-256-CFB8", "CAMELLIA-256-ECB", "CAMELLIA-256-OFB", "CAMELLIA128", "CAMELLIA192", "CAMELLIA256", "CAST", "CAST-
cbc", "CAST5-CBC", "CAST5-CFB", "CAST5-ECB", "CAST5-OFB", "DES", "DES-CBC", "DES-CFB", "DES-CFB1", "DES-CFB8", "DES-ECB", "DES-EDE", "DES-EDE-CBC", "DES-EDE-CFB", "DES-EDE-OFB", "DES-EDE3", "D
ES-EDE3-CBC", "DES-EDE3-CFB", "DES-EDE3-CFB1", "DES-EDE3-CFB8", "DES-EDE3-OFB", "DES-OFB", "DES3", "DESX", "DESX-CBC", "IDEA", "IDEA-CBC", "IDEA-CFB", "IDEA-ECB", "IDEA-OFB", "RC2", "RC2-40-CB
C", "RC2-64-CBC", "RC2-CBC", "RC2-CFB", "RC2-ECB", "RC2-OFB", "RC4", "RC4-40", "RC4-HMAC-MD5", "SEED", "SEED-CBC", "SEED-CFB", "SEED-ECB", "SEED-OFB", "aes-128-cbc", "aes-128-cbc-hmac-sha1", "
aes-128-cbc-hmac-sha256", "aes-128-ccm", "aes-128-cfb", "aes-128-cfb1", "aes-128-cfb8", "aes-128-ctr", "aes-128-ecb", "aes-128-gcm", "aes-128-ofb", "aes-128-xts", "aes-192-cbc", "aes-192-ccm",
"aes-192-cfb", "aes-192-cfb1", "aes-192-cfb8", "aes-192-ctr", "aes-192-ecb", "aes-192-gcm", "aes-192-ofb", "aes-256-cbc", "aes-256-cbc-hmac-sha1", "aes-256-cbc-hmac-sha256", "aes-256-ccm", "a
es-256-cfb", "aes-256-cfb1", "aes-256-cfb8", "aes-256-ctr", "aes-256-ecb", "aes-256-gcm", "aes-256-ofb", "aes-256-xts", "aes128", "aes192", "aes256", "bf", "bf-cbc", "bf-cfb", "bf-ecb", "bf-of
b", "blowfish", "camellia-128-cbc", "camellia-128-cfb", "camellia-128-cfb1", "camellia-128-cfb8", "camellia-128-ecb", "camellia-128-ofb", "camellia-192-cbc", "camellia-192-cfb", "camellia-192-
cfb1", "camellia-192-cfb8", "camellia-192-ecb", "camellia-192-ofb", "camellia-256-cbc", "camellia-256-cfb", "camellia-256-cfb1", "camellia-256-cfb8", "camellia-256-ecb", "camellia-256-ofb", "c
amellia128", "camellia192", "camellia256", "cast", "cast-cbc", "cast5-cbc", "cast5-cfb", "cast5-ecb", "cast5-ofb", "des", "des-cbc", "des-cfb", "des-cfb1", "des-cfb8", "des-ecb", "des-ede", "d
es-ede-cbc", "des-ede-cfb", "des-ede-ofb", "des-ede3", "des-ede3-cbc", "des-ede3-cfb", "des-ede3-cfb1", "des-ede3-cfb8", "des-ede3-ofb", "des-ofb", "des3", "desx", "desx-cbc", "id-aes128-CCM",
"id-aes128-GCM", "id-aes128-wrap", "id-aes192-CCM", "id-aes192-GCM", "id-aes192-wrap", "id-aes256-CCM", "id-aes256-GCM", "id-aes256-wrap", "id-smime-alg-CMS3DESwrap", "idea", "idea-cbc", "ide
a-cfb", "idea-ecb", "idea-ofb", "rc2", "rc2-40-cbc", "rc2-64-cbc", "rc2-cbc", "rc2-cfb", "rc2-ecb", "rc2-ofb", "rc4", "rc4-40", "rc4-hmac-md5", "seed", "seed-cbc", "seed-cfb", "seed-ecb", "see
d-ofb"]

Puppet 5.1 / Ruby 2.4.1
OpenSSL::Cipher.ciphers

=> ["AES-128-CBC", "AES-128-CFB", "AES-128-CFB1", "AES-128-CFB8", "AES-128-CTR", "AES-128-ECB", "AES-128-OFB", "AES-128-XTS", "AES-192-CBC", "AES-192-CFB", "AES-192-CFB1", "AES-192-CFB8", "AES-192-CTR", "AES-192-ECB", "AES-192-OFB", "AES-256-CBC", "AES-256-CFB", "AES-256-CFB1", "AES-256-CFB8", "AES-256-CTR", "AES-256-ECB", "AES-256-OFB", "AES-256-XTS", "AES128", "AES192", "AES256", "BF", "BF-CBC", "BF-CFB", "BF-ECB", "BF-OFB", "CAST", "CAST-cbc", "CAST5-CBC", "CAST5-CFB", "CAST5-ECB", "CAST5-OFB", "DES", "DES-CBC", "DES-CFB", "DES-CFB1", "DES-CFB8", "DES-ECB", "DES-EDE", "DES-EDE-CBC", "DES-EDE-CFB", "DES-EDE-OFB", "DES-EDE3", "DES-EDE3-CBC", "DES-EDE3-CFB", "DES-EDE3-CFB1", "DES-EDE3-CFB8", "DES-EDE3-OFB", "DES-OFB", "DES3", "DESX", "DESX-CBC", "RC2", "RC2-40-CBC", "RC2-64-CBC", "RC2-CBC", "RC2-CFB", "RC2-ECB", "RC2-OFB", "aes-128-cbc", "aes-128-ccm", "aes-128-cfb", "aes-128-cfb1", "aes-128-cfb8", "aes-128-ctr", "aes-128-ecb", "aes-128-gcm", "aes-128-ofb", "aes-128-xts", "aes-192-cbc", "aes-192-ccm", "aes-192-cfb", "aes-192-cfb1", "aes-192-cfb8", "aes-192-ctr", "aes-192-ecb", "aes-192-gcm", "aes-192-ofb", "aes-256-cbc", "aes-256-ccm", "aes-256-cfb", "aes-256-cfb1", "aes-256-cfb8", "aes-256-ctr", "aes-256-ecb", "aes-256-gcm", "aes-256-ofb", "aes-256-xts", "aes128", "aes192", "aes256", "bf", "bf-cbc", "bf-cfb", "bf-ecb", "bf-ofb", "blowfish", "cast", "cast-cbc", "cast5-cbc", "cast5-cfb", "cast5-ecb", "cast5-ofb", "des", "des-cbc", "des-cfb", "des-cfb1", "des-cfb8", "des-ecb", "des-ede", "des-ede-cbc", "des-ede-cfb", "des-ede-ofb", "des-ede3", "des-ede3-cbc", "des-ede3-cfb", "des-ede3-cfb1", "des-ede3-cfb8", "des-ede3-ofb", "des-ofb", "des3", "desx", "desx-cbc", "id-aes128-CCM", "id-aes128-GCM", "id-aes128-wrap", "id-aes192-CCM", "id-aes192-GCM", "id-aes192-wrap", "id-aes256-CCM", "id-aes256-GCM", "id-aes256-wrap", "id-smime-alg-CMS3DESwrap", "rc2", "rc2-40-cbc", "rc2-64-cbc", "rc2-cbc", "rc2-cfb", "rc2-ecb", "rc2-ofb"]

[emre141]

rhenium so could you clarify how can i use any workaround or it is not any workaround? I have not enough knowledge about ruby/openssl could you show me step by step that i would be applied if it is a solution?

Thanks

[mirogta]

@rhenium - Thank you for the update. I have found openssl.so in Ruby folder, but can't find libcrypto.so.

Any clue on how to "inject" RC4 support back to it?

We've found out that there is a list of ciphers in C:\Puppet\sys\ruby\lib\ruby\2.4.0\openssl\ssl.rb, which updates OpenSSL::SSL.SSLContext DEFAULT_PARAMS, so we can hack the list returned by OpenSSL::Cipher.ciphers to anything we want.

We've even tried to add this piece of code to it:

 module OpenSSL
   class Cipher
     def self.ciphers()
       %w{
         AES-128-CBC
         RC4
       }
     end
   end
 end

So that OpenSSL::Cipher.ciphers returns just these two.

However that has no effect during runtime. When we run OpenSSL::Cipher.new("RC4") with this change in place, it throws the same error -

unsupported cipher algorithm (rc4)

[emre141]

Hi

After i have changed it below mentioned do i recompile ruby or if it is require any recompile? if it is ok how can i recomple it? And also if it not just when i restart my shell is enough?

[emre141]

Yes i changed it and i have got the same error.

[rhenium]

Puppet 5.1 / Ruby 2.4.1
OpenSSL::Cipher.ciphers

It looks like the OpenSSL library used by Ruby/OpenSSL has no RC4 support -- it is likely disabled at compile-time. I guess you have to recompile OpenSSL with RC4 support (i.e., remove no-rc4 option from the arguments passed to OpenSSL's Configure script).

We've found out that there is a list of ciphers in C:\Puppet\sys\ruby\lib\ruby\2.4.0\openssl\ssl.rb

As I said in my first comment, it is only effective for OpenSSL::SSL::SSLContext#set_params which Ruby/NTLM does not use.

[rhenium]

@emre141 I can't help you without more information. The Errno::ECONNRESET error usually indicates that the HTTP server forcibly closed the TCP connection without sending a close_notify alert. Anyway, it's an irrelevant topic. Please open a new issue at https://github.com/ruby/openssl/issues/new, preferably with a script so I can reproduce myself.

[glennsarti]

@mirogta @rhenium
The compiled OpenSSL library that puppet-agent uses, does not contain the RC4 ciphers as of Puppet 5
puppetlabs/puppet-agent@0da0492

[mirogta]

@glennsarti - thank you, this is very useful. I've ask Puppet to look into that on that changeset.

[glennsarti]

@mirogta FYI - https://tickets.puppetlabs.com/browse/PA-1712 . That's to re-enable RC4

[Iristyle]

@mirogta A little more context. At the time one of our (Puppets) security engineers removed RC4 from our custom OpenSSL build in Puppet 5, it was during a security audit for insecure cipher support. Since RC4 is considered weak and is not recommended any longer, it was removed.

It wasn't realized that RC4 was in use for NTLM connectivity over WinRM / HTTP as there isn't anything natively in Puppet or its tools that builds NTLM messages directly. We recently discovered this ourselves given our new task running tool, Bolt, consumes the same rubyntlm library that Test Kitchen uses, and does build NTLM messages directly using the RC4 cipher. RC4 is baked into the NTLM protocol, and as such, cannot be reconfigured to use something more secure, which is why Microsoft recommends against using it as well. That said, it's still in wide use.

As @glennsarti mentioned, we're re-adding to our custom OpenSSL / Ruby stack and should have it addressed in upcoming Z releases of Puppet.

Sorry we didn't catch this sooner!

[mirogta]

@Iristyle Thank you for the update

[mirogta]

Since this will be fixed in one of the upcoming Puppet releases and it's in a custom OpenSSL build only, this issue can be closed.

[mirogta]

Looks like the issue has been fixed in puppet-agent 5.3.4 (released on 2018/02/05)