_PyMem_DebugRawFree under Python 3.13 starting with version 1.0.0 in PYTHONDEVMODE

[sbrandtb]

Description

Starting with Python 3.13 and watchfiles version 1.0.0, I get memory allocation issues when Python dev mode enabled.

The error goes away when either:

• Disabling Python Dev Mode
• Downgrading Python to <3.13
• Downgrading watchfiles to <1.0.0

Example Code

docker run --rm -it --entrypoint bash python:3.13-slim
pip install watchfiles
env PYTHONDEVMODE=1 python -c 'import watchfiles'

Watchfiles Output

Debug memory block at address p=0x740802034840: API 'm'
    103 bytes originally requested                                                                                          
    The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
    The 8 pad bytes at tail=0x7408020348a7 are FORBIDDENBYTE, as expected.
    Data at p: 52 75 73 74 4e 6f 74 69 ... 64 29 0a 2d 2d 0a 0a 00
                                                                                                                            
Enable tracemalloc to get the memory block allocation traceback                                 
                                                              
Fatal Python error: _PyMem_DebugRawFree: bad ID: Allocated using API 'm', verified using API 'o'
Python runtime state: initialized                 
                                                                                                                            
Current thread 0x0000740802bcbb80 (most recent call first):                                                                 
  File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed                                                                                                                                                                             File "<frozen importlib._bootstrap_external>", line 1316 in create_module                                       
  File "<frozen importlib._bootstrap>", line 813 in module_from_spec
  File "<frozen importlib._bootstrap>", line 921 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1331 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1360 in _find_and_load
  File "/usr/local/lib/python3.13/site-packages/watchfiles/main.py", line 11 in <module>
  File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1022 in exec_module
  File "<frozen importlib._bootstrap>", line 935 in _load_unlocked    
  File "<frozen importlib._bootstrap>", line 1331 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1360 in _find_and_load
  File "/usr/local/lib/python3.13/site-packages/watchfiles/__init__.py", line 2 in <module>
  File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1022 in exec_module
  File "<frozen importlib._bootstrap>", line 935 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1331 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1360 in _find_and_load
  File "<python-input-0>", line 1 in <module>
  File "/usr/local/lib/python3.13/code.py", line 92 in runcode 
  File "/usr/local/lib/python3.13/_pyrepl/console.py", line 205 in runsource
  File "/usr/local/lib/python3.13/code.py", line 313 in push
  File "/usr/local/lib/python3.13/_pyrepl/simple_interact.py", line 160 in run_multiline_interactive_console
  File "/usr/local/lib/python3.13/_pyrepl/main.py", line 59 in interactive_console
  File "/usr/local/lib/python3.13/_pyrepl/__main__.py", line 6 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main
Aborted (core dumped)

Operating System & Architecture

Linux-6.8.0-48-generic-x86_64-with-glibc2.36
#48-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 27 14:04:52 UTC 2024

Environment

docker

Python & Watchfiles Version

python: 3.13.0 (main, Nov 12 2024, 02:27:08) [GCC 12.2.0], watchfiles: 1.0.0

Rust & Cargo Version

No response

[samuelcolvin]

Are you using 3.13t?

@davidhewitt, any chance this is a bug with PyO3?

[sbrandtb]

Are you using 3.13t?

To my understanding not.

To check if the current interpreter supports free-threading, python -VV and sys.version contain “experimental free-threading build”.

My version string is Python 3.13.0 (main, Nov 12 2024, 02:27:08) [GCC 12.2.0]

I would be surprised if that would be the default python executable in their official Docker image, anyway :)

[davidhewitt]

Thanks for the ping, sounds potentially serious, will try to investigate once the kids are in bed.

[davidhewitt]

Pretty sure the root cause will be PyO3/pyo3#4757 and re-releasing a build with PyO3 0.23.3 will fix.