Merge pull request #156 from jborean93/ntlm-encryption

NTLM/CredSSP Message Encryption

CHANGELOG.md

@@ -1,5 +1,9 @@
 # Changelog
 
+### Developing
+- Added support for message encryption over HTTP when using NTLM and CredSSP
+- Added parameter to disable TLSv1.2 when using CredSSP for Server 2008 support
+
 ### Version 0.2.2
 - Added support for CredSSP authenication (via requests-credssp)
 - Improved README, see 'Valid transport options' section

README.md

@@ -134,21 +134,58 @@ pywinrm supports various transport methods in order to authenticate with the Win
 * `certificate`: Authentication is done through a certificate that is mapped to a local Windows account on the server.
 * `ssl`: When used in conjunction with `cert_pem` and `cert_key_pem` it will use a certificate as above. If not will revert to basic auth over HTTPS.
 * `kerberos`: Will use Kerberos authentication for domain accounts which only works when the client is in the same domain as the server and the required dependencies are installed. Currently a Kerberos ticket needs to be initiliased outside of pywinrm using the kinit command.
-* `ntlm`: Will use NTLM authentication for both domain and local accounts. Currently no support for NTLMv2 auth and other features included in that version (WIP).
+* `ntlm`: Will use NTLM authentication for both domain and local accounts.
 * `credssp`: Will use CredSSP authentication for both domain and local accounts. Allows double hop authentication. This only works over a HTTPS endpoint and not HTTP.
 
-### HTTP or HTTPS endpoint
+### Encryption
 
-While either a HTTP or HTTPS endpoint can be used as the transport method, using HTTPS is prefered as the messages are encrypted using SSL. To use HTTPS either a self signed certificate or one from a CA can be used. You can use this [guide](http://www.joseph-streeter.com/?p=1086) to set up a HTTPS endpoint with a self signed certificate.
+By default WinRM will not accept unencrypted messages from a client and Pywinrm
+currently has 2 ways to do this.
+
+1. Using a HTTPS endpoint instead of HTTP (Recommended)
+2. Use NTLM or CredSSP as the transport auth and setting `message_encryption` to `auto` or `always`
+
+Using a HTTPS endpoint is recommended as it will encrypt all the data sent
+through to the server including the credentials and works with all transport
+auth types. You can use [this script](https://github.com/ansible/ansible/blob/devel/examples/scripts/ConfigureRemotingForAnsible.ps1)
+to easily set up a HTTPS endpoint on WinRM with a self signed certificate but
+in a production environment this should be hardened with your own process.
+
+The second option is to use NTLM or CredSSP and set the `message_encryption`
+arg to protocol to `auto` or `always`. This will use the authentication GSS-API
+Wrap and Unwrap methods if available to encrypt the message contents sent to
+the server. This form of encryption is independent from the transport layer
+like TLS and is currently only supported by the NTLM and CredSSP transport
+auth. Kerberos currently does not have the methods available to achieve this.
+
+To configure message encryption you can use the `message_encryption` argument
+when initialising protocol. This option has 3 values that can be set as shown
+below.
+
+* `auto`: Default, Will only use message encryption if it is available for the auth method and HTTPS isn't used.
+* `never`: Will never use message encryption even when not over HTTPS.
+* `always`: Will always use message encryption even when running over HTTPS.
+
+If you set the value to `always` and the transport opt doesn't support message
+encryption i.e. Basic auth, Pywinrm will throw an exception.
+
+If you do not use a HTTPS endpoint or message encryption then the Windows
+server will automatically reject Pywinrm. You can change the settings on the
+Windows server to allow unencrypted messages and credentials but this highly
+insecure and shouldn't be used unless necessary. To allow unencrypted message
+run the following command either from cmd or powershell
 
-If you still wish to use a HTTP endpoint and loose confidentiality in your messages you will need to enable unencrypted messages in the server by running the following command
 ```
-# from cmd:
+# from cmd
 winrm set winrm/config/service @{AllowUnencrypted="true"}
+
+# or from powershell
+Set-Item -Path "WSMan:\localhost\Service\AllowUnencrypted" -Value $true
 ```
-As a repeat this should definitely not be used as your credentials and messages will allow anybody to see what is sent over the wire.
 
-There are plans in place to allow message encryption for messages sent with Kerberos or NTLM messages in the future.
+As a repeat this should definitely not be used as your credentials and messages
+will allow anybody to see what is sent over the wire.
+
 
 ### Enabling WinRM on remote host
 Enable WinRM over HTTP and HTTPS with self-signed certificate (includes firewall rules):

appveyor.yml

@@ -49,5 +49,10 @@ test_script:
 
       py.test -v --cov-report=term-missing --cov=.
 
+      # Run integration tests with NTLM to check message encryption
+      $env:WINRM_TRANSPORT="ntlm"
+      Set-Item WSMan:\localhost\Service\AllowUnencrypted $false
+      py.test -v winrm/tests/test_integration_protocol.py winrm/tests/test_integration_session.py
+
 after_test:
   - echo after_test

setup.cfg

@@ -4,6 +4,6 @@ requires = python-xmltodict
 [bdist_wheel]
 universal = 1
 
-[pytest]
+[tool:pytest]
 norecursedirs = .git .idea env
 pep8ignore = tests/*.py E501

winrm/encryption.py

@@ -0,0 +1,208 @@
+import requests
+import re
+import struct
+
+from winrm.exceptions import WinRMError
+
+class Encryption(object):
+
+    SIXTEN_KB = 16384
+    MIME_BOUNDARY = b'--Encrypted Boundary'
+
+    def __init__(self, session, protocol):
+        """
+        [MS-WSMV] v30.0 2016-07-14
+
+        2.2.9.1 Encrypted Message Types
+        When using Encryption, there are three options available
+            1. Negotiate/SPNEGO
+            2. Kerberos
+            3. CredSSP
+        Details for each implementation can be found in this document under this section
+
+        This init sets the following values to use to encrypt and decrypt. This is to help generify
+        the methods used in the body of the class.
+            wrap: A method that will return the encrypted message and a signature
+            unwrap: A method that will return an unencrypted message and verify the signature
+            protocol_string: The protocol string used for the particular auth protocol
+
+        :param session: The handle of the session to get GSS-API wrap and unwrap methods
+        :param protocol: The auth protocol used, will determine the wrapping and unwrapping method plus
+                         the protocol string to use. Currently only NTLM and CredSSP is supported
+        """
+        self.protocol = protocol
+        self.session = session
+
+        if protocol == 'ntlm': # Details under Negotiate [2.2.9.1.1] in MS-WSMV
+            self.protocol_string = b"application/HTTP-SPNEGO-session-encrypted"
+            self._build_message = self._build_ntlm_message
+            self._decrypt_message = self._decrypt_ntlm_message
+        elif protocol == 'credssp': # Details under CredSSP [2.2.9.1.3] in MS-WSMV
+            self.protocol_string = b"application/HTTP-CredSSP-session-encrypted"
+            self._build_message = self._build_credssp_message
+            self._decrypt_message = self._decrypt_credssp_message
+        # TODO: Add support for Kerberos encryption
+        else:
+            raise WinRMError("Encryption for protocol '%s' not yet supported in pywinrm" % protocol)
+
+    def prepare_encrypted_request(self, session, endpoint, message):
+        """
+        Creates a prepared request to send to the server with an encrypted message
+        and correct headers
+
+        :param session: The handle of the session to prepare requests with
+        :param endpoint: The endpoint/server to prepare requests to
+        :param message: The unencrypted message to send to the server
+        :return: A prepared request that has an encrypted message
+        """
+        if self.protocol == 'credssp' and len(message) > self.SIXTEN_KB:
+            content_type = 'multipart/x-multi-encrypted'
+            encrypted_message = b''
+            message_chunks = [message[i:i+self.SIXTEN_KB] for i in range(0, len(message), self.SIXTEN_KB)]
+            for message_chunk in message_chunks:
+                encrypted_chunk = self._encrypt_message(message_chunk)
+                encrypted_message += encrypted_chunk
+        else:
+            content_type = 'multipart/encrypted'
+            encrypted_message = self._encrypt_message(message)
+        encrypted_message += self.MIME_BOUNDARY + b"--\r\n"
+
+        request = requests.Request('POST', endpoint, data=encrypted_message)
+        prepared_request = session.prepare_request(request)
+        prepared_request.headers['Content-Length'] = str(len(prepared_request.body))
+        prepared_request.headers['Content-Type'] = '{0};protocol="{1}";boundary="Encrypted Boundary"'\
+            .format(content_type, self.protocol_string.decode())
+
+        return prepared_request
+
+    def parse_encrypted_response(self, response):
+        """
+        Takes in the encrypted response from the server and decrypts it
+
+        :param response: The response that needs to be decrytped
+        :return: The unencrypted message from the server
+        """
+        content_type = response.headers['Content-Type']
+        if 'protocol="{0}"'.format(self.protocol_string.decode()) in content_type:
+            msg = self._decrypt_response(response)
+        else:
+            msg = response.text
+
+        return msg
+
+    def _encrypt_message(self, message):
+        message_length = str(len(message)).encode()
+        encrypted_stream = self._build_message(message)
+
+        message_payload = self.MIME_BOUNDARY + b"\r\n" \
+                          b"\tContent-Type: " + self.protocol_string + b"\r\n" \
+                          b"\tOriginalContent: type=application/soap+xml;charset=UTF-8;Length=" + message_length + b"\r\n" + \
+                          self.MIME_BOUNDARY + b"\r\n" \
+                          b"\tContent-Type: application/octet-stream\r\n" + \
+                          encrypted_stream
+
+        return message_payload
+
+    def _decrypt_response(self, response):
+        parts = response.content.split(self.MIME_BOUNDARY + b'\r\n')
+        parts = list(filter(None, parts)) # filter out empty parts of the split
+        message = b''
+
+        for i in range(0, len(parts)):
+            if i % 2 == 1:
+                continue
+
+            header = parts[i].strip()
+            payload = parts[i + 1]
+
+            expected_length = int(header.split(b'Length=')[1])
+
+            # remove the end MIME block if it exists
+            if payload.endswith(self.MIME_BOUNDARY + b'--\r\n'):
+                payload = payload[:len(payload) - 24]
+
+            encrypted_data = payload.replace(b'\tContent-Type: application/octet-stream\r\n', b'')
+            decrypted_message = self._decrypt_message(encrypted_data)
+            actual_length = len(decrypted_message)
+
+            if actual_length != expected_length:
+                raise WinRMError('Encrypted length from server does not match the '
+                                 'expected size, message has been tampered with')
+            message += decrypted_message
+
+        return message
+
+    def _decrypt_ntlm_message(self, encrypted_data):
+        signature_length = struct.unpack("<i", encrypted_data[:4])[0]
+        signature = encrypted_data[4:signature_length + 4]
+        encrypted_message = encrypted_data[signature_length + 4:]
+
+        message = self.session.auth.session_security.unwrap(encrypted_message, signature)
+
+        return message
+
+    def _decrypt_credssp_message(self, encrypted_data):
+        # trailer_length = struct.unpack("<i", encrypted_data[:4])[0]
+        encrypted_message = encrypted_data[4:]
+
+        message = self.session.auth.unwrap(encrypted_message)
+
+        return message
+
+    def _build_ntlm_message(self, message):
+        sealed_message, signature = self.session.auth.session_security.wrap(message)
+        signature_length = struct.pack("<i", len(signature))
+
+        return signature_length + signature + sealed_message
+
+    def _build_credssp_message(self, message):
+        sealed_message = self.session.auth.wrap(message)
+
+        trailer_length = self._get_credssp_trailer_length(len(message), self.session.auth.cipher_negotiated)
+
+        return struct.pack("<i", trailer_length) + sealed_message
+
+    def _get_credssp_trailer_length(self, message_length, cipher_suite):
+        # I really don't like the way this works but can't find a better way, MS
+        # allows you to get this info through the struct SecPkgContext_StreamSizes
+        # but there is no GSSAPI/OpenSSL equivalent so we need to calculate it
+        # ourselves
+
+        if re.match('^.*-GCM-[\w\d]*$', cipher_suite):
+            # We are using GCM for the cipher suite, GCM has a fixed length of 16
+            # bytes for the TLS trailer making it easy for us
+            trailer_length = 16
+        else:
+            # We are not using GCM so need to calculate the trailer size. The
+            # trailer length is equal to the length of the hmac + the length of the
+            # padding required by the block cipher
+            hash_algorithm = cipher_suite.split('-')[-1]
+
+            # while there are other algorithms, SChannel doesn't support them
+            # as of yet https://msdn.microsoft.com/en-us/library/windows/desktop/aa374757(v=vs.85).aspx
+            if hash_algorithm == 'MD5':
+                hash_length = 16
+            elif hash_algorithm == 'SHA':
+                hash_length = 20
+            elif hash_algorithm == 'SHA256':
+                hash_length = 32
+            elif hash_algorithm == 'SHA384':
+                hash_length = 48
+            else:
+                hash_length = 0
+
+            pre_pad_length = message_length + hash_length
+
+            if "RC4" in cipher_suite:
+                # RC4 is a stream cipher so no padding would be added
+                padding_length = 0
+            elif "DES" in cipher_suite or "3DES" in cipher_suite:
+                # 3DES is a 64 bit block cipher
+                padding_length = 8 - (pre_pad_length % 8)
+            else:
+                # AES is a 128 bit block cipher
+                padding_length = 16 - (pre_pad_length % 16)
+
+            trailer_length = (pre_pad_length + padding_length) - message_length
+
+        return trailer_length

winrm/protocol.py

@@ -30,6 +30,8 @@ def __init__(
             read_timeout_sec=DEFAULT_READ_TIMEOUT_SEC,
             operation_timeout_sec=DEFAULT_OPERATION_TIMEOUT_SEC,
             kerberos_hostname_override=None,
+            message_encryption='auto',
+            credssp_disable_tlsv1_2=False
         ):
         """
         @param string endpoint: the WinRM webservice endpoint
@@ -47,6 +49,7 @@ def __init__(
         @param int read_timeout_sec: maximum seconds to wait before an HTTP connect/read times out (default 30). This value should be slightly higher than operation_timeout_sec, as the server can block *at least* that long. # NOQA
         @param int operation_timeout_sec: maximum allowed time in seconds for any single wsman HTTP operation (default 20). Note that operation timeouts while receiving output (the only wsman operation that should take any significant time, and where these timeouts are expected) will be silently retried indefinitely. # NOQA
         @param string kerberos_hostname_override: the hostname to use for the kerberos exchange (defaults to the hostname in the endpoint URL)
+        @param bool message_encryption_enabled: Will encrypt the WinRM messages if set to True and the transport auth supports message encryption (Default True).
         """
 
         if operation_timeout_sec >= read_timeout_sec or operation_timeout_sec < 1:
@@ -65,7 +68,10 @@ def __init__(
             server_cert_validation=server_cert_validation,
             kerberos_delegation=kerberos_delegation,
             kerberos_hostname_override=kerberos_hostname_override,
-            auth_method=transport)
+            auth_method=transport,
+            message_encryption=message_encryption,
+            credssp_disable_tlsv1_2=credssp_disable_tlsv1_2
+        )
 
         self.username = username
         self.password = password
@@ -75,6 +81,7 @@ def __init__(
         self.server_cert_validation = server_cert_validation
         self.kerberos_delegation = kerberos_delegation
         self.kerberos_hostname_override = kerberos_hostname_override
+        self.credssp_disable_tlsv1_2 = credssp_disable_tlsv1_2
 
     def open_shell(self, i_stream='stdin', o_stream='stdout stderr',
                    working_directory=None, env_vars=None, noprofile=False,