diff --git a/CHANGELOG.md b/CHANGELOG.md index ef3b5526c3..b37c7a9dd1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,6 +27,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ([#1323](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1323)) - `opentelemetry-instrumentation-wsgi` Add support for regular expression matching and sanitization of HTTP headers. ([#1402](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1402)) +- `opentelemetry-instrumentation-falcon` Add support for regular expression matching and sanitization of HTTP headers. + ([#1412](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1412)) - `opentelemetry-instrumentation-flask` Add support for regular expression matching and sanitization of HTTP headers. ([#1413](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1413)) - `opentelemetry-instrumentation-pyramid` Add support for regular expression matching and sanitization of HTTP headers. @@ -42,6 +44,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ([#1333](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1333)) - `opentelemetry-instrumentation-asgi` Make ASGIGetter.get() compare all keys in a case insensitive manner. ([#1333](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1333)) +- Use resp.text instead of resp.body for Falcon 3 to avoid a deprecation warning. + ([#1412](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1412)) ## [1.13.0-0.34b0](https://github.com/open-telemetry/opentelemetry-python/releases/tag/v1.13.0-0.34b0) - 2022-09-26 diff --git a/instrumentation/opentelemetry-instrumentation-falcon/src/opentelemetry/instrumentation/falcon/__init__.py b/instrumentation/opentelemetry-instrumentation-falcon/src/opentelemetry/instrumentation/falcon/__init__.py index e94c0659f0..33b9a13360 100644 --- a/instrumentation/opentelemetry-instrumentation-falcon/src/opentelemetry/instrumentation/falcon/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-falcon/src/opentelemetry/instrumentation/falcon/__init__.py @@ -19,15 +19,16 @@ * The Falcon resource and method name is used as the Span name. * The ``falcon.resource`` Span attribute is set so the matched resource. -* Error from Falcon resources are properly caught and recorded. +* Errors from Falcon resources are properly caught and recorded. Configuration ------------- Exclude lists ************* -To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_FALCON_EXCLUDED_URLS`` -(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude. +To exclude certain URLs from tracking, set the environment variable ``OTEL_PYTHON_FALCON_EXCLUDED_URLS`` +(or ``OTEL_PYTHON_EXCLUDED_URLS`` to cover all instrumentations) to a string of comma delimited regexes that match the +URLs. For example, @@ -39,8 +40,8 @@ Request attributes ******************** -To extract certain attributes from Falcon's request object and use them as span attributes, set the environment variable ``OTEL_PYTHON_FALCON_TRACED_REQUEST_ATTRS`` to a comma -delimited list of request attribute names. +To extract attributes from Falcon's request object and use them as span attributes, set the environment variable +``OTEL_PYTHON_FALCON_TRACED_REQUEST_ATTRS`` to a comma delimited list of request attribute names. For example, @@ -48,7 +49,7 @@ export OTEL_PYTHON_FALCON_TRACED_REQUEST_ATTRS='query_string,uri_template' -will extract query_string and uri_template attributes from every traced request and add them as span attritbues. +will extract the ``query_string`` and ``uri_template`` attributes from every traced request and add them as span attributes. Falcon Request object reference: https://falcon.readthedocs.io/en/stable/api/request_and_response.html#id1 @@ -73,8 +74,9 @@ def on_get(self, req, resp): Request and Response hooks *************************** -The instrumentation supports specifying request and response hooks. These are functions that get called back by the instrumentation right after a Span is created for a request -and right before the span is finished while processing a response. The hooks can be configured as follows: +This instrumentation supports request and response hooks. These are functions that get called +right after a span is created for a request and right before the span is finished for the response. +The hooks can be configured as follows: :: @@ -88,54 +90,93 @@ def response_hook(span, req, resp): Capture HTTP request and response headers ***************************************** -You can configure the agent to capture predefined HTTP headers as span attributes, according to the `semantic convention `_. +You can configure the agent to capture specified HTTP headers as span attributes, according to the +`semantic convention `_. Request headers *************** -To capture predefined HTTP request headers as span attributes, set the environment variable ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST`` -to a comma-separated list of HTTP header names. +To capture HTTP request headers as span attributes, set the environment variable +``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST`` to a comma delimited list of HTTP header names. For example, - :: export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST="content-type,custom_request_header" -will extract ``content-type`` and ``custom_request_header`` from request headers and add them as span attributes. +will extract ``content-type`` and ``custom_request_header`` from the request headers and add them as span attributes. + +Request header names in Falcon are case-insensitive and ``-`` characters are replaced by ``_``. So, giving the header +name as ``CUStom_Header`` in the environment variable will capture the header named ``custom-header``. + +Regular expressions may also be used to match multiple headers that correspond to the given pattern. For example: +:: + + export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST="Accept.*,X-.*" -It is recommended that you should give the correct names of the headers to be captured in the environment variable. -Request header names in falcon are case insensitive and - characters are replaced by _. So, giving header name as ``CUStom_Header`` in environment variable will be able capture header with name ``custom-header``. +Would match all request headers that start with ``Accept`` and ``X-``. -The name of the added span attribute will follow the format ``http.request.header.`` where ```` being the normalized HTTP header name (lowercase, with - characters replaced by _ ). -The value of the attribute will be single item list containing all the header values. +To capture all request headers, set ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST`` to ``".*"``. +:: -Example of the added span attribute, + export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST=".*" + +The name of the added span attribute will follow the format ``http.request.header.`` where ```` +is the normalized HTTP header name (lowercase, with ``-`` replaced by ``_``). The value of the attribute will be a +single item list containing all the header values. + +For example: ``http.request.header.custom_request_header = [","]`` Response headers **************** -To capture predefined HTTP response headers as span attributes, set the environment variable ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE`` -to a comma-separated list of HTTP header names. +To capture HTTP response headers as span attributes, set the environment variable +``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE`` to a comma delimited list of HTTP header names. For example, - :: export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE="content-type,custom_response_header" -will extract ``content-type`` and ``custom_response_header`` from response headers and add them as span attributes. +will extract ``content-type`` and ``custom_response_header`` from the response headers and add them as span attributes. + +Response header names in Falcon are case-insensitive. So, giving the header name as ``CUStom-Header`` in the environment +variable will capture the header named ``custom-header``. + +Regular expressions may also be used to match multiple headers that correspond to the given pattern. For example: +:: + + export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE="Content.*,X-.*" + +Would match all response headers that start with ``Content`` and ``X-``. -It is recommended that you should give the correct names of the headers to be captured in the environment variable. -Response header names captured in falcon are case insensitive. So, giving header name as ``CUStomHeader`` in environment variable will be able capture header with name ``customheader``. +To capture all response headers, set ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE`` to ``".*"``. +:: + + export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE=".*" -The name of the added span attribute will follow the format ``http.response.header.`` where ```` being the normalized HTTP header name (lowercase, with - characters replaced by _ ). -The value of the attribute will be single item list containing all the header values. +The name of the added span attribute will follow the format ``http.response.header.`` where ```` +is the normalized HTTP header name (lowercase, with ``-`` replaced by ``_``). The value of the attribute will be a +single item list containing all the header values. -Example of the added span attribute, +For example: ``http.response.header.custom_response_header = [","]`` +Sanitizing headers +****************** +In order to prevent storing sensitive data such as personally identifiable information (PII), session keys, passwords, +etc, set the environment variable ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS`` +to a comma delimited list of HTTP header names to be sanitized. Regexes may be used, and all header names will be +matched in a case-insensitive manner. + +For example, +:: + + export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS=".*session.*,set-cookie" + +will replace the value of headers such as ``session-id`` and ``set-cookie`` with ``[REDACTED]`` in the span. + Note: - Environment variable names to capture http headers are still experimental, and thus are subject to change. + The environment variable names used to capture HTTP headers are still experimental, and thus are subject to change. API --- diff --git a/instrumentation/opentelemetry-instrumentation-falcon/tests/app.py b/instrumentation/opentelemetry-instrumentation-falcon/tests/app.py index 6cc60faee6..3e4c62ec3e 100644 --- a/instrumentation/opentelemetry-instrumentation-falcon/tests/app.py +++ b/instrumentation/opentelemetry-instrumentation-falcon/tests/app.py @@ -8,7 +8,13 @@ class HelloWorldResource: def _handle_request(self, _, resp): # pylint: disable=no-member resp.status = falcon.HTTP_201 - resp.body = "Hello World" + + _parsed_falcon_version = package_version.parse(falcon.__version__) + if _parsed_falcon_version < package_version.parse("3.0.0"): + # Falcon 1 and Falcon 2 + resp.body = "Hello World" + else: + resp.text = "Hello World" def on_get(self, req, resp): self._handle_request(req, resp) @@ -44,6 +50,15 @@ def on_get(self, _, resp): "my-custom-header", "my-custom-value-1,my-custom-header-2" ) resp.set_header("dont-capture-me", "test-value") + resp.set_header( + "my-custom-regex-header-1", + "my-custom-regex-value-1,my-custom-regex-value-2", + ) + resp.set_header( + "My-Custom-Regex-Header-2", + "my-custom-regex-value-3,my-custom-regex-value-4", + ) + resp.set_header("my-secret-header", "my-secret-value") def make_app(): diff --git a/instrumentation/opentelemetry-instrumentation-falcon/tests/test_falcon.py b/instrumentation/opentelemetry-instrumentation-falcon/tests/test_falcon.py index 7e714342a7..aeba57a9b5 100644 --- a/instrumentation/opentelemetry-instrumentation-falcon/tests/test_falcon.py +++ b/instrumentation/opentelemetry-instrumentation-falcon/tests/test_falcon.py @@ -41,6 +41,7 @@ from opentelemetry.test.wsgitestutil import WsgiTestBase from opentelemetry.trace import StatusCode from opentelemetry.util.http import ( + OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE, ) @@ -421,8 +422,9 @@ def test_mark_span_internal_in_presence_of_span_from_other_framework(self): @patch.dict( "os.environ", { - OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,invalid-header", - OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "content-type,content-length,my-custom-header,invalid-header", + OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS: ".*my-secret.*", + OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,invalid-header,Regex-Test-Header-.*,Regex-Invalid-Test-Header-.*,.*my-secret.*", + OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "content-type,content-length,my-custom-header,invalid-header,my-custom-regex-header-.*,invalid-regex-header-.*,.*my-secret.*", }, ) class TestCustomRequestResponseHeaders(TestFalconBase): @@ -431,6 +433,9 @@ def test_custom_request_header_added_in_server_span(self): "Custom-Test-Header-1": "Test Value 1", "Custom-Test-Header-2": "TestValue2,TestValue3", "Custom-Test-Header-3": "TestValue4", + "Regex-Test-Header-1": "Regex Test Value 1", + "regex-test-header-2": "RegexTestValue2,RegexTestValue3", + "My-Secret-Header": "My Secret Value", } self.client().simulate_request( method="GET", path="/hello", headers=headers @@ -443,6 +448,11 @@ def test_custom_request_header_added_in_server_span(self): "http.request.header.custom_test_header_2": ( "TestValue2,TestValue3", ), + "http.request.header.regex_test_header_1": ("Regex Test Value 1",), + "http.request.header.regex_test_header_2": ( + "RegexTestValue2,RegexTestValue3", + ), + "http.request.header.my_secret_header": ("[REDACTED]",), } not_expected = { "http.request.header.custom_test_header_3": ("TestValue4",), @@ -459,6 +469,9 @@ def test_custom_request_header_not_added_in_internal_span(self): headers = { "Custom-Test-Header-1": "Test Value 1", "Custom-Test-Header-2": "TestValue2,TestValue3", + "Regex-Test-Header-1": "Regex Test Value 1", + "regex-test-header-2": "RegexTestValue2,RegexTestValue3", + "My-Secret-Header": "My Secret Value", } self.client().simulate_request( method="GET", path="/hello", headers=headers @@ -470,6 +483,13 @@ def test_custom_request_header_not_added_in_internal_span(self): "http.request.header.custom_test_header_2": ( "TestValue2,TestValue3", ), + "http.request.header.regex_test_header_1": ( + "Regex Test Value 1", + ), + "http.request.header.regex_test_header_2": ( + "RegexTestValue2,RegexTestValue3", + ), + "http.request.header.my_secret_header": ("[REDACTED]",), } self.assertEqual(span.kind, trace.SpanKind.INTERNAL) for key, _ in not_expected.items(): @@ -494,6 +514,13 @@ def test_custom_response_header_added_in_server_span(self): "http.response.header.my_custom_header": ( "my-custom-value-1,my-custom-header-2", ), + "http.response.header.my_custom_regex_header_1": ( + "my-custom-regex-value-1,my-custom-regex-value-2", + ), + "http.response.header.my_custom_regex_header_2": ( + "my-custom-regex-value-3,my-custom-regex-value-4", + ), + "http.response.header.my_secret_header": ("[REDACTED]",), } not_expected = { "http.response.header.dont_capture_me": ("test-value",) @@ -524,6 +551,13 @@ def test_custom_response_header_not_added_in_internal_span(self): "http.response.header.my_custom_header": ( "my-custom-value-1,my-custom-header-2", ), + "http.response.header.my_custom_regex_header_1": ( + "my-custom-regex-value-1,my-custom-regex-value-2", + ), + "http.response.header.my_custom_regex_header_2": ( + "my-custom-regex-value-3,my-custom-regex-value-4", + ), + "http.response.header.my_secret_header": ("[REDACTED]",), } self.assertEqual(span.kind, trace.SpanKind.INTERNAL) for key, _ in not_expected.items():