Implement PEP3134 to discover underlying problems with python3

[lehmat]

Issue #, if available:
#354

Description of changes:
Implement PEP3134 through six.raise_from to support python2 and python3.

Also move one import to top of the module to better comply with pylint (C0413:wrong-import-position)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[lehmat]

Just moved upwards to remove pylint:C0413

[lehmat]

This ticket is tied to the following troubleshooting and discovering this issue: Pynamodb 1073

[carolabadeer]

LGTM!

[michael-k]

I've never seen raise e from e where both the exception and the cause are the same. When playing with that in IPython, I don't see any benefit either. Instead it adds a lot of noise:

Python 3.11.0 (main, Nov 12 2022, 07:10:51) [GCC 11.2.1 20220219]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.6.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: try:
   ...:     raise ValueError("foo")
   ...: except ValueError as exc:
   ...:     print("bar")
   ...:     raise exc from exc
   ...:
bar
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
    [... skipping hidden 1 frame]

Cell In [1], line 5
      4 print("bar")
----> 5 raise exc from exc

Cell In [1], line 2
      1 try:
----> 2     raise ValueError("foo")
      3 except ValueError as exc:

ValueError: foo

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
    [... skipping hidden 1 frame]

Cell In [1], line 5
      4 print("bar")
----> 5 raise exc from exc

Cell In [1], line 2
      1 try:
----> 2     raise ValueError("foo")
      3 except ValueError as exc:

ValueError: foo

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
Cell In [1], line 5
      3 except ValueError as exc:
      4     print("bar")
----> 5     raise exc from exc

Cell In [1], line 2
      1 try:
----> 2     raise ValueError("foo")
      3 except ValueError as exc:
      4     print("bar")

ValueError: foo

PEP 3134 doesn't mention such a use case either.
@lehmat Can you elaborate?

raise

In [2]: raise ValueError("foo")
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In [2], line 1
----> 1 raise ValueError("foo")

ValueError: foo

try-catch-raise without from

In [3]: try:
   ...:     raise ValueError("foo")
   ...: except ValueError as exc:
   ...:     print("bar")
   ...:     raise exc
   ...:
bar
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In [3], line 5
      3 except ValueError as exc:
      4     print("bar")
----> 5     raise exc

Cell In [3], line 2
      1 try:
----> 2     raise ValueError("foo")
      3 except ValueError as exc:
      4     print("bar")

ValueError: foo

[lehmat]

This PR was made because there was underlying issue hidden by the missing raise from syntax in aws-xray-sdk core. If this would have been implemented previously, the underlying issue would have been seen without disabling xray in the application code.

What was seen in logs prior to this PR

• Botocore raises an issue about quote_from_bytes() expected bytes pynamodb/PynamoDB#1073

What this PR improves

• Before: The root cause is found at File "aws_xray_sdk/core/recorder.py", line 436, in record_subsegment, with invalid quote_from_bytes arguments
• After: The root cause if found at File sentry_sdk/tracing_utils.py", line 541 with invalid quote_from_bytes argument

Essentially it will give that The above exception was the direct cause of the following exception: to be pointing at underlying issue at sentry-sdk instead of stopping the traceback at aws_xray_sdk

Reference of the underlying issue that was actually the issue:

• quote_from_bytes() expected bytes getsentry/sentry-python#1626

[michael-k]

I saw both issues and read through them, but that doesn't explain why this is an issue in the first place. A simple re-raise should not omit parts of the stacktrace. In pynamodb/PynamoDB#1073 you said that you “disable[d] AWS X-ray to get the proper root cause to popup.” Did you also try it with this patch and X-Ray enabled? I'm not convinced that this actually solves your problem.

Maybe X-Ray accessing the stacktrace has something to do with it? I played around with that, but found no hint that would support this theory. But my toy stacktraces were small.

From raise's documentation:

Changed in version 3.11: If the traceback of the active exception is modified in an except clause, a subsequent raise statement re-raises the exception with the modified traceback. Previously, the exception was re-raised with the traceback it had when it was caught.

getsentry/sentry-python#1626 mentions Python 3.9, therefore the re-raise shouldn't have changed the traceback.

[lehmat]

After reviewing your comments I agree with you that this does not solve the problem.

Did you also try it with this patch and X-Ray enabled?

No, manipulating dependencies would be troublesome with our way of deploying lambda

Essentially this would behave as it did (with sentry-sdk[flask]==1.9.8)

import os
import serverless_wsgi

import sentry_sdk
from flask import Flask
import boto3
from aws_xray_sdk.core import xray_recorder
from aws_xray_sdk.core import patch_all

xray_recorder.configure(service='Test')
patch_all()


app = Flask(__name__)


sentry_sdk.init(
    dsn=os.getenv("SENTRY_DSN"),  # need to have some sane here to actually make sentry work
    release=1,  # This was the breaking on sentry-sdk, integer release causing it
    traces_sample_rate=1,
)

@app.route("/hello")
def hello_world():
    segment = xray_recorder.begin_segment('segment_name')
    boto3.client('sts').get_caller_identity().get('Account')
    xray_recorder.end_segment()
    return "<p>Hello, World!</p>"

def handler(event, context):
    return serverless_wsgi.handle_request(app, event, context)