Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Python 3.13 #3185

Merged
merged 1 commit into from
Oct 7, 2024
Merged

Add support for Python 3.13 #3185

merged 1 commit into from
Oct 7, 2024

Conversation

nateprewitt
Copy link
Contributor

This PR will start provisional testing on the recently released beta for Python 3.13. This will be used to catch any issues early before the official release in October.

@nateprewitt nateprewitt requested a review from jonathan343 May 23, 2024 18:40
@nateprewitt
Copy link
Contributor Author

nateprewitt commented May 23, 2024

It looks like all 3.13 builds are failing on these two tests due to increased memory usage with paginators:

FAILED functional/leak/test_resource_leaks.py::TestDoesNotLeakMemory::test_create_single_paginator_memory_constant
FAILED functional/leak/test_resource_leaks.py::TestDoesNotLeakMemory::test_create_single_waiter_memory_constant

We'll need to figure out what the delta is between 3.12 and 3.13, then see if we have an issue or there's some form of new memory leak happening in the 3.13 interpreter.

Example Stacktrace
Memory increases ~10.8MB which exceeds our arbitrary 10MB threshold. I'd like to see what our change looks like in 3.12 to determine if we barely crossed a threshold we were already approaching or if this is a significant increase.


self = <tests.functional.leak.test_resource_leaks.TestDoesNotLeakMemory testMethod=test_create_single_paginator_memory_constant>

    def test_create_single_paginator_memory_constant(self):
        self.cmd('create_paginator', 's3', 'list_objects')
        self.cmd('free_paginators')
        self.record_memory()
        for _ in range(100):
            self.cmd('create_paginator', 's3', 'list_objects')
            self.cmd('free_paginators')
        self.record_memory()
        start, end = self.memory_samples
>       self.assertTrue((end - start) < self.MAX_GROWTH_BYTES, (end - start))
E       AssertionError: False is not true : 10878976

@jonathan343
Copy link
Contributor

We'll need to figure out what the delta is between 3.12 and 3.13, then see if we have an issue or there's some form of new memory leak happening in the 3.13 interpreter.

I had a few minutes to compare the delta values between 3.12 a 3.13. The results are shown below:

Test Python 3.12 Memory Delta (B) Python 3.13 Memory Delta (B) % Increase
test_create_memory_clients_in_loop -9,052,160 176,128 -101.95 ??
test_create_memory_paginators_in_loop 1,503,232 1,937,408 28.88
test_create_memory_waiters_in_loop 1,675,264 1,794,048 7.09
test_create_single_client_memory_constant 1,785,856 10,338,304 478.90
test_create_single_paginator_memory_constant 2,551,808 11,886,592 365.81
test_create_single_waiter_memory_constant 2,490,368 11,870,208 376.64

There is definitely weird something going on with the 3.13 interpreter. I don't have to time to do a deeper dive now, but wanted to add this information here for future reference.

Notes:

  1. What is going on with test_create_memory_clients_in_loop??

Copy link
Contributor

@jonathan343 jonathan343 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Python 3.13.0 release candidate 3 was released on 10/01/2024 and made available in GitHub CI with actions/python-versions#308. I tested locally and this should resolved the current failures we're seeing. I requested some minor changes.

.github/workflows/run-crt-test.yml Outdated Show resolved Hide resolved
tests/functional/leak/test_resource_leaks.py Outdated Show resolved Hide resolved
tests/functional/leak/test_resource_leaks.py Outdated Show resolved Hide resolved
@nateprewitt nateprewitt changed the title Start testing on 3.13 beta build Add support for Python 3.13 Oct 7, 2024
@nateprewitt nateprewitt marked this pull request as ready for review October 7, 2024 18:57
Copy link
Contributor

@jonathan343 jonathan343 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@nateprewitt nateprewitt merged commit 66dc1f8 into develop Oct 7, 2024
78 checks passed
@nateprewitt nateprewitt deleted the 3.13 branch October 7, 2024 19:43
aws-sdk-python-automation added a commit that referenced this pull request Oct 8, 2024
* release-1.35.36:
  Bumping version to 1.35.36
  Update endpoints model
  Update to latest models
  Add support for Python 3.13 (#3185)
alexgromero pushed a commit to alexgromero/botocore that referenced this pull request Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants