Feature tcp custom response (#135)

* Custom responses added for TCP and UDP
* Removing unnecessary `fcntl` that bars Windows testing

Co-authored-by: Matthew Haigh <[email protected]>

CHANGELOG.txt

@@ -1,3 +1,9 @@
+Version 1.4.11
+--------------
+* TCP Custom Response and UDP Custom Response features
+* Removed stray `fcntl` import from `test.py` that prevented Windows testing in
+  some cases
+
 Version 1.4.10
 --------------
 * Fix format string errors due to line length limit

docs/CustomResponse.md

@@ -1,47 +1,79 @@
-# HTTP Listener Custom Responses
+# Custom Response Configuration
 
-The `HTTPListener` can accept a setting named `Custom` that enables
-customizable responses beyond what can be achieved by emplacing files in the
-web root (e.g. under `defaultFiles/`).
+The Custom Response feature enables customization of responses either
+statically or as user-supplied Python script code.
 
-The `HTTPListener` `Custom` setting indicates the name of an HTTP custom
-response configuration file in the same location and format as the active
-FakeNet-NG configuration file. An example HTTP custom response configuration
-file is supplied in `fakenet/configs/sample_http_custom.ini`.
+This enables convenient C2 server development for malware analysis purposes
+beyond what can be achieved by emplacing files in the service root (e.g. under
+`defaultFiles/` in the default configuration) and without having to modify
+default listener source code that is currently only available in the source
+release of FakeNet-NG.
 
-The sections of the HTTP custom response configuration file can define a series
-of named rules regarding how to match requests and what to return.
+Currently the `RawListener` (both TCP and UDP) and the `HTTPListener` accept a
+setting named `Custom` that allows the user to specify Custom Response
+behavior.
 
-Valid matching specifications are:
-* `MatchHosts`: a comma-separated list of hostnames that will match against
+The `Custom` setting indicates the name of a custom response configuration file
+in the same location and format as the active FakeNet-NG configuration file.
+Example custom response configuration files are supplied under
+`fakenet/configs/`.
+
+Each section of the custom response configuration file must specify which
+listener(s) to configure. Valid listener configuration specifications are:
+* `ListenerType`: The type of listener to affect:
+    * `HTTP` for any `HTTPListener`
+	* `TCP` for any TCP `RawListener`
+	* `UDP` for any UDP `RawListener`
+* `InstanceName`: The name of the listener instance to be configured,
+  as found in the section name within the FakeNet-NG configuration file.
+
+If both the `ListenerType` and `InstanceName` listener specifications are
+present in a single section, they will be evaluated disjunctively (logical or).
+
+Further details are documented below for HTTP and raw Listeners subsequently.
+
+## HTTP Listener Custom Responses
+
+Aside from which listener instances to configure, the HTTP custom response
+configuration section must specify:
+* Which requests to match
+* What response to return.
+
+Valid HTTP request matching specifications are:
+* `HttpHosts`: a comma-separated list of hostnames that will match against
   host headers.
-* `MatchURIs`: a comma-separated list of URIs that will match against request
+* `HttpURIs`: a comma-separated list of URIs that will match against request
   URIs.
 
-If both matching specifications are present in a single section, they will be
-evaluated conjunctively (logical and).
+If the `HttpHosts` specification includes a colon-delimited port number, it
+will only match if the host header includes the same colon-delimited port
+number.
+
+If both the `HttpHosts` and `HttpURIs` matching specifications are present in
+a single section, they will be evaluated conjunctively (logical and).
 
-Valid response specifications are:
-* `RawFile`: Returns the raw contents of the specified file located under
+Valid HTTP custom response specifications are:
+* `HttpRawFile`: Returns the raw contents of the specified file located under
   the web root, with the exception of date replacement.
-* `StaticString`: Wraps the specified string with server headers and a 200 OK
-  response code, replacing `\r\n` tokens with actual CRLFs and performing date
-  replacement as necessary.
-    * `ContentType`: Optionally, you accompany the `StaticString` setting with
-      an  HTTP `Content-Type` header value to send. It is an error to specify
-      this setting with any other kind of response specification.
-* `Dynamic`: Loads the specified Python file located under the web root
-  and invokes its `HandleRequest` function as described below.
+* `HttpStaticString`: Wraps the specified string with server headers and a 200
+  OK response code, replacing `\r\n` tokens with actual CRLFs and performing
+  date replacement as necessary.
+	* `ContentType`: Optionally, you accompany the `HttpStaticString` setting
+	  with an  HTTP `Content-Type` header value to send. It is an error to
+	  specify this setting with any other kind of response specification.
+* `HttpDynamic`: Loads the specified Python file located under the web root
+  and invokes its `HandleHttp` function as described below.
 
-Date replacement applies to both `RawFile` and `StaticString`, and replaces any
-occurrences of `<RAW-DATE>` with a server-formatted date.
+Date replacement applies to both `HttpRawFile` and `HttpStaticString`, and
+replaces any occurrences of `<RAW-DATE>` in the specified text with a
+server-formatted date.
 
-## Implementing a Dynamic Response Handler
+### Implementing the HttpDynamic Response Handler
 
-The `HandleRequest` method must conform to the following prototype:
+The `HandleHttp` function must conform to the following prototype:
 
 ```
-def HandleRequest(req, method, post_data=None):
+def HandleHttp(req, method, post_data=None):
 	"""Handle an HTTP request.
 
 	Parameters
@@ -57,5 +89,77 @@ def HandleRequest(req, method, post_data=None):
 	pass
 ```
 
-An example dynamic response handler is supplied in
-`defaultFiles/HTTPCustomProviderExample.py`
+An example HTTP dynamic response handler is supplied in
+`configs/CustomProviderExample.py`
+
+## Raw Listener Custom Responses
+
+Raw Listener (TCP and UDP) Custom Responses implement no request filtering and
+implement only response specifications.
+
+### TCP Listener Custom Responses
+Valid TCP custom response specifications are:
+* `TcpRawFile`: Returns the raw contents of the specified file located under
+  the configuration root.
+* `TcpStaticString`: Sends the specified string as-is.
+* `TcpStaticBase64`: Base64 decodes the specified Base64-encoded data and sends
+  the result as-is.
+* `TcpDynamic`: Loads the specified Python file located under the web root
+  and invokes its `HandleTcp` function as described below.
+
+#### Implementing the TcpDynamic Response Handler
+
+The `HandleTcp` function must conform to the following prototype:
+
+```
+def HandleTcp(sock):
+	"""Handle a TCP buffer.
+
+	Parameters
+	----------
+	sock : socket
+		The connected socket with which to recv and send data
+	"""
+	pass
+```
+
+The socket object's `recv` function is hooked so that any time it is called, a
+hex dump will be emitted in the FakeNet-NG log output on behalf of the user.
+
+### UDP Listener Custom Responses
+Valid UDP custom response specifications have the same meanings as their TCP
+analogs documented above:
+* `UdpRawFile`
+* `UdpStaticString`
+* `UdpStaticBase64`
+* `UdpDynamic`: invokes `HandleUdp`
+
+#### Implementing the UdpDynamic Response Handler
+
+The `HandleUdp` function's prototype differs significantly from its TCP analog
+due to the differences in protocols. With UDP, the data has already been
+received before it is time to call any user-implemented callback. Furthermore,
+the remote address of the peer is needed to be able to send data via UDP.
+
+```
+def HandleUdp(sock, data, addr):
+    """Handle a UDP buffer.
+
+    Parameters
+    ----------
+    sock : socket
+		The connected socket with which to recv and send data
+    data : str
+        The data received
+    addr : tuple
+        The host and port of the remote peer
+    """
+	pass
+```
+
+To send data, the programmer must supply not only the buffer to send, but also
+the remote address to transmit to, as provided in the `addr` argument:
+
+```
+	sock.sendto(buf, addr)
+```

docs/developing.md

@@ -233,7 +233,10 @@ fakenet1.4.3\
 	+-- configs\
 	|   +-- CustomProviderExample.py
 	|   +-- default.ini
-	|   +-- sample_http_custom.ini
+	|   +-- CustomProviderExample.py
+	|   +-- sample_custom_response.ini
+	|   +-- sample_raw_response.txt
+	|   +-- sample_raw_tcp_response.txt
 	|
 	+-- defaultFiles\
 	|   +-- FakeNet.gif
@@ -244,9 +247,7 @@ fakenet1.4.3\
 	|   +-- FakeNet.pdf
 	|   +-- FakeNet.png
 	|   +-- FakeNet.txt
-	|   +-- HTTPCustomProviderExample.py
 	|   +-- ncsi.txt
-	|   +-- sample_raw_response.txt
 	|
 	+-- listeners\
 	    +-- ssl_utils

docs/srs.md

@@ -458,9 +458,9 @@ string, the HTTP Custom Response feature must replace occurrences of
 `<RAW-DATE>` in the 
 
 ### TCP and UDP Listener Custom Response Configuration
-The TCP and UDP listeners should accommodate Custom Response configuration
-constrained to an operator-defined regular expression of octets and permit
-responses to be returned according to three configuration specifications:
+The TCP and UDP listeners must accommodate Custom Response configuration
+and permit responses to be returned according to three configuration
+specifications:
 * The raw contents of a configured binary file
 * The contents of a statically configured string
 * The delegation of control to a Python script file

fakenet/configs/CustomProviderExample.py

@@ -0,0 +1,72 @@
+import socket
+
+# To read about customizing HTTP responses, see docs/CustomResponse.md
+def HandleRequest(req, method, post_data=None):
+    """Sample dynamic HTTP response handler.
+
+    Parameters
+    ----------
+    req : BaseHTTPServer.BaseHTTPRequestHandler
+        The BaseHTTPRequestHandler that recevied the request
+    method: str
+        The HTTP method, either 'HEAD', 'GET', 'POST' as of this writing
+    post_data: str
+        The HTTP post data received by calling `rfile.read()` against the
+        BaseHTTPRequestHandler that received the request.
+    """
+    response = 'Ahoy\r\n'
+
+    if method == 'GET':
+        req.send_response(200)
+        req.send_header('Content-Length', len(response))
+        req.end_headers()
+        req.wfile.write(response)
+
+    elif method == 'POST':
+        req.send_response(200)
+        req.send_header('Content-Length', len(response))
+        req.end_headers()
+        req.wfile.write(response)
+
+    elif method == 'HEAD':
+        req.send_response(200)
+        req.end_headers()
+
+
+def HandleTcp(sock):
+    """Handle a TCP buffer.
+
+    Parameters
+    ----------
+    sock : socket
+        The connected socket with which to recv and send data
+    """
+    while True:
+        try:
+            data = None
+            data = sock.recv(1024)
+        except socket.timeout:
+            pass
+
+        if not data:
+            break
+
+        resp = raw_input('\nEnter a response for the TCP client: ')
+        sock.sendall(resp)
+
+
+def HandleUdp(sock, data, addr):
+    """Handle a UDP buffer.
+
+    Parameters
+    ----------
+    sock : socket
+        The connected socket with which to recv and send data
+    data : str
+        The data received
+    addr : tuple
+        The host and port of the remote peer
+    """
+    if data:
+        resp = raw_input('\nEnter a response for the UDP client: ')
+        sock.sendto(resp, addr)

fakenet/configs/default.ini

@@ -232,6 +232,8 @@ Listener:    RawListener
 UseSSL:      No
 Timeout:     10
 Hidden:      False
+# To read about customizing responses, see docs/CustomResponse.md
+# Custom:    sample_custom_response.ini
 
 [RawUDPListener]
 Enabled:     True
@@ -241,6 +243,8 @@ Listener:    RawListener
 UseSSL:      No
 Timeout:     10
 Hidden:      False
+# To read about customizing responses, see docs/CustomResponse.md
+# Custom:    sample_custom_response.ini
 
 [FilteredListener]
 Enabled:     False
@@ -276,8 +280,8 @@ Timeout:     10
 DumpHTTPPosts: Yes
 DumpHTTPPostsFilePrefix: http
 Hidden:      False
-# To read about customizing HTTP responses, see docs/CustomResponse.md
-# Custom:    sample_http_custom.ini
+# To read about customizing responses, see docs/CustomResponse.md
+# Custom:    sample_custom_response.ini
 
 [HTTPListener443]
 Enabled:     True

fakenet/configs/sample_custom_response.ini

@@ -0,0 +1,25 @@
+# To read about customizing HTTP responses, see docs/CustomResponse.md
+[Example0]
+InstanceName:     HTTPListener80
+HttpURIs:         /test.txt
+HttpStaticString: Wraps this with normal FakeNet HTTP headers (<RAW-DATE>)\r\n
+
+[Example1]
+InstanceName:     HTTPListener80
+ListenerType:     HTTP
+HttpHosts:        some.random.c2.com, other.c2.com
+HttpRawFile:      sample_raw_response.txt
+
+[Example2]
+ListenerType:     HTTP
+HttpHosts:        both_host.com
+HttpURIs:         and_uri.txt
+HttpDynamic:      CustomProviderExample.py
+
+[ExampleTCP]
+InstanceName:     RawTCPListener
+TcpDynamic:       CustomProviderExample.py
+
+[ExampleUDP]
+InstanceName:     RawUDPListener
+UdpDynamic:       CustomProviderExample.py