Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for generic OTEL_PYTHON_EXCLUDED_URLS variable #790

Merged
merged 5 commits into from
Nov 10, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
([#793](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/793))
- `opentelemetry-instrumentation-pymysql` Add support for PyMySQL 1.x series
([#792](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/792))
- Add support for generic OTEL_PYTHON_EXCLUDED_URLS variable
([#790](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/790))

### Fixed

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@

Exclude lists
*************
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_DJANGO_EXCLUDED_URLS`` with comma delimited regexes representing which URLs to exclude.
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_DJANGO_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude.

For example,

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ Configuration

Exclude lists
*************
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_FALCON_EXCLUDED_URLS`` with comma delimited regexes representing which URLs to exclude.
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_FALCON_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude.

For example,

Expand All @@ -34,7 +35,7 @@ will exclude requests such as ``https://site/client/123/info`` and ``https://sit
Request attributes
********************
To extract certain attributes from Falcon's request object and use them as span attributes, set the environment variable ``OTEL_PYTHON_FALCON_TRACED_REQUEST_ATTRS`` to a comma
delimited list of request attribute names.
delimited list of request attribute names.

For example,

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@

Exclude lists
*************
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_FALCON_EXCLUDED_URLS`` with comma delimited regexes representing which URLs to exclude.
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_FALCON_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude.

For example,

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@ async def foobar():

Exclude lists
*************
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_FASTAPI_EXCLUDED_URLS`` with comma delimited regexes representing which URLs to exclude.
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_FASTAPI_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude.

For example,

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ Configuration

Exclude lists
*************
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_FLASK_EXCLUDED_URLS`` with comma delimited regexes representing which URLs to exclude.
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_FLASK_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude.

For example,

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@ def hello():

Exclude lists
*************
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_FLASK_EXCLUDED_URLS`` with comma delimited regexes representing which URLs to exclude.
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_FLASK_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude.

For example,

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,8 @@

Exclude lists
*************
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_PYRAMID_EXCLUDED_URLS`` with comma delimited regexes representing which URLs to exclude.
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_PYRAMID_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude.

For example,

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,14 @@ Installation
::

pip install opentelemetry-instrumentation-requests

Configuration
-------------

Exclude lists
*************
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_REQUESTS_EXCLUDED_URLS`` with comma delimited regexes representing which URLs to exclude.
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_REQUESTS_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude.

For example,

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@

Exclude lists
*************
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_REQUESTS_EXCLUDED_URLS`` with comma delimited regexes representing which URLs to exclude.
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_REQUESTS_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude.

For example,

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@ def home(request):

Exclude lists
*************
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_STARLETTE_EXCLUDED_URLS`` with comma delimited regexes representing which URLs to exclude.
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_STARLETTE_EXCLUDED_URLS``
(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude.

For example,

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,14 @@
from os import environ
from re import compile as re_compile
from re import search
from typing import Iterable
from urllib.parse import urlparse, urlunparse


class ExcludeList:
"""Class to exclude certain paths (given as a list of regexes) from tracing requests"""

def __init__(self, excluded_urls):
def __init__(self, excluded_urls: Iterable[str]):
self._excluded_urls = excluded_urls
if self._excluded_urls:
self._regex = re_compile("|".join(excluded_urls))
Expand All @@ -47,24 +48,29 @@ def get_traced_request_attrs(instrumentation):
return traced_request_attrs


def get_excluded_urls(instrumentation):
def get_excluded_urls(instrumentation: str) -> ExcludeList:
# Get instrumentation-specific excluded URLs. If not set, retrieve them
# from generic variable.
excluded_urls = environ.get(
_root.format(f"{instrumentation}_EXCLUDED_URLS"), []
_root.format(f"{instrumentation}_EXCLUDED_URLS"),
environ.get(_root.format("EXCLUDED_URLS"), ""),
)

return parse_excluded_urls(excluded_urls)


def parse_excluded_urls(excluded_urls):
def parse_excluded_urls(excluded_urls: str) -> ExcludeList:
"""
Small helper to put an arbitrary url list inside of ExcludeList
"""
if excluded_urls:
excluded_urls = [
excluded_url_list = [
excluded_url.strip() for excluded_url in excluded_urls.split(",")
]
else:
excluded_url_list = []

return ExcludeList(excluded_urls)
return ExcludeList(excluded_url_list)


def remove_url_credentials(url: str) -> str:
Expand Down
72 changes: 72 additions & 0 deletions util/opentelemetry-util-http/tests/test_http_excluded_urls.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Copyright The OpenTelemetry Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from unittest.mock import patch

from opentelemetry.test.test_base import TestBase
from opentelemetry.util.http import get_excluded_urls


class TestGetExcludedUrls(TestBase):
@patch.dict(
"os.environ",
{
"OTEL_PYTHON_DJANGO_EXCLUDED_URLS": "excluded_arg/123,excluded_noarg"
},
)
def test_config_from_instrumentation_env(self):
exclude_list = get_excluded_urls("DJANGO")

self.assertTrue(exclude_list.url_disabled("/excluded_arg/123"))
self.assertTrue(exclude_list.url_disabled("/excluded_noarg"))
self.assertFalse(exclude_list.url_disabled("/excluded_arg/125"))

@patch.dict(
"os.environ",
{"OTEL_PYTHON_EXCLUDED_URLS": "excluded_arg/123,excluded_noarg"},
)
def test_config_from_generic_env(self):
exclude_list = get_excluded_urls("DJANGO")

self.assertTrue(exclude_list.url_disabled("/excluded_arg/123"))
self.assertTrue(exclude_list.url_disabled("/excluded_noarg"))
self.assertFalse(exclude_list.url_disabled("/excluded_arg/125"))

@patch.dict(
"os.environ",
{
"OTEL_PYTHON_DJANGO_EXCLUDED_URLS": "excluded_arg/123,excluded_noarg",
"OTEL_PYTHON_EXCLUDED_URLS": "excluded_arg/125",
},
)
def test_config_from_instrumentation_env_takes_precedence(self):
exclude_list = get_excluded_urls("DJANGO")

self.assertTrue(exclude_list.url_disabled("/excluded_arg/123"))
self.assertTrue(exclude_list.url_disabled("/excluded_noarg"))
self.assertFalse(exclude_list.url_disabled("/excluded_arg/125"))

@patch.dict(
"os.environ",
{
"OTEL_PYTHON_DJANGO_EXCLUDED_URLS": "",
"OTEL_PYTHON_EXCLUDED_URLS": "excluded_arg/125",
},
)
def test_config_from_instrumentation_env_empty(self):
exclude_list = get_excluded_urls("DJANGO")

self.assertFalse(exclude_list.url_disabled("/excluded_arg/123"))
self.assertFalse(exclude_list.url_disabled("/excluded_noarg"))
self.assertFalse(exclude_list.url_disabled("/excluded_arg/125"))