Some qiskit unit tests fail when using symengine with tcmalloc

[iyanmv]

Some tests from qiskit work fine when using symengine from PyPI but failed when using symengine and python-symengine from the Arch Linux repos. After using [the same PKGBUILD] to rebuild the packages but just disabling tcmalloc the tests don't crash anymore.

I've reported this in qiskit repo (Qiskit/qiskit#12359) but they think it's a symengine bug.

Here is a minimal example that can produce the crash affecting the unit tests:

from qiskit.circuit.parameter import Parameter
from qiskit.circuit.parameterexpression import ParameterExpression


theta = Parameter("theta")
phi = Parameter("phi")
sum_param = theta + phi

sum_param._symbol_expr.__reduce__()
# src/tcmalloc.cc:304] Attempt to free invalid pointer 0x645c8e3eaa30 
# [1]    987079 IOT instruction (core dumped)  python

This same example works fine when symegine is compiled without tcmalloc. Curiously, if one imports symengine in the same script, it also works (even with symengine with tcmalloc):

import symengine
from qiskit.circuit.parameter import Parameter
from qiskit.circuit.parameterexpression import ParameterExpression


theta = Parameter("theta")
phi = Parameter("phi")
sum_param = theta + phi

sum_param._symbol_expr.__reduce__()
# (<cyfunction load_basic at 0x784de9289970>, (b'\x01\x00\x00\x0b\x00\x01\x00\x00\x80\x10\x00\x00\x00\x02\x00\x00\x80\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x000\x02\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x80\r\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00phi\x04\x00\x00\x80\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x001\x05\x00\x00\x80\r\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00theta\x06\x00\x00\x80\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x001',))

Perhaps some extra 👀 from here can help understand this issue better :) Also, can you clarify if the recommended way to create packages for distros is to use tcmalloc or not? Arch uses tcmalloc but also -DWITH_SYMENGINE_THREAD_SAFE=ON. I guess it's a mixture of the flags recommended for distros and the optimized build. Is this properly tested? I noticed that the official wheel packages for PyPI, for example, do not use tcmalloc.

[iyanmv]

Here is a backtrace after running above example:

(gdb) bt
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007ffff76a8eb3 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:78
#2  0x00007ffff7650a30 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007ffff76384c3 in __GI_abort () at abort.c:79
#4  0x00007ffff361317e in tcmalloc::Log (mode=mode@entry=tcmalloc::kCrash, filename=filename@entry=0x7ffff36390c6 "src/tcmalloc.cc", line=line@entry=304, a=..., b=..., c=..., d=...) at src/internal_logging.cc:118
#5  0x00007ffff3617a54 in (anonymous namespace)::InvalidFree (ptr=<optimized out>) at src/tcmalloc.cc:304
#6  0x00007ffff40602d9 in std::__new_allocator<SymEngine::RCP<SymEngine::Basic const> >::deallocate (this=<optimized out>, __n=<optimized out>, __p=<optimized out>) at /usr/include/c++/13.2.1/bits/new_allocator.h:152
#7  std::allocator_traits<std::allocator<char> >::deallocate (__n=<optimized out>, __p=<optimized out>, __a=...) at /usr/include/c++/13.2.1/bits/alloc_traits.h:516
#8  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_destroy (__size=<optimized out>, this=0x7fffffffd420) at /usr/include/c++/13.2.1/bits/basic_string.h:289
#9  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose (this=0x7fffffffd420) at /usr/include/c++/13.2.1/bits/basic_string.h:283
#10 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string (this=<optimized out>, this=<optimized out>) at /usr/include/c++/13.2.1/bits/basic_string.h:792
#11 std::__cxx11::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >::~basic_stringbuf (this=<optimized out>, this=<optimized out>) at /usr/include/c++/13.2.1/sstream:79
#12 std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >::~basic_ostringstream (this=<optimized out>, this=<optimized out>) at /usr/include/c++/13.2.1/sstream:851
#13 SymEngine::wrapper_dumps[abi:cxx11](SymEngine::Basic const&) (x=...) at /usr/src/debug/python-symengine/symengine.py-0.11.0/symengine/lib/pywrapper.cpp:350
#14 0x00007ffff3ecd96a in __pyx_pf_9symengine_3lib_17symengine_wrapper_5Basic_4__reduce__ (__pyx_v_self=0x7ffff6ea2d90) at /usr/include/symengine/symengine_rcp.h:150
#15 __pyx_pw_9symengine_3lib_17symengine_wrapper_5Basic_5__reduce__ (__pyx_v_self=0x7ffff6ea2d90, __pyx_args=<optimized out>, __pyx_nargs=<optimized out>, __pyx_kwds=<optimized out>) at /usr/src/debug/python-symengine/build/symengine/lib/symengine_wrapper.cpp:53890
#16 0x00007ffff799a844 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x7ffff7f97070, callable=0x7fffe81357d0, tstate=0x7ffff7e22ae8 <_PyRuntime+459656>) at ./Include/internal/pycore_call.h:92
#17 PyObject_Vectorcall (callable=0x7fffe81357d0, args=0x7ffff7f97070, nargsf=9223372036854775809, kwnames=0x0) at Objects/call.c:325
#18 0x00007ffff788bdfa in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=0x7ffff7f97020, throwflag=<optimized out>) at Python/bytecodes.c:2706
#19 0x00007ffff7a3d767 in PyEval_EvalCode (co=0x7ffff6e18530, globals=<optimized out>, locals=0x7ffff6ff9dc0) at Python/ceval.c:578
#20 0x00007ffff7a608b7 in run_eval_code_obj (tstate=tstate@entry=0x7ffff7e22ae8 <_PyRuntime+459656>, co=co@entry=0x7ffff6e18530, globals=globals@entry=0x7ffff6ff9dc0, locals=locals@entry=0x7ffff6ff9dc0) at Python/pythonrun.c:1722
#21 0x00007ffff7a5b9dc in run_mod (mod=mod@entry=0x55555563b298, filename=filename@entry=0x7ffff6ff9f70, globals=globals@entry=0x7ffff6ff9dc0, locals=locals@entry=0x7ffff6ff9dc0, flags=flags@entry=0x7fffffffda40, arena=arena@entry=0x7ffff6f1be30) at Python/pythonrun.c:1743
#22 0x00007ffff7a74f33 in pyrun_file (fp=fp@entry=0x5555555a4f30, filename=filename@entry=0x7ffff6ff9f70, start=start@entry=257, globals=globals@entry=0x7ffff6ff9dc0, locals=locals@entry=0x7ffff6ff9dc0, closeit=closeit@entry=1, flags=0x7fffffffda40) at Python/pythonrun.c:1643
#23 0x00007ffff7a74346 in _PyRun_SimpleFileObject (fp=0x5555555a4f30, filename=0x7ffff6ff9f70, closeit=1, flags=0x7fffffffda40) at Python/pythonrun.c:433
#24 0x00007ffff7a73f88 in _PyRun_AnyFileObject (fp=0x5555555a4f30, filename=0x7ffff6ff9f70, closeit=1, flags=0x7fffffffda40) at Python/pythonrun.c:78
#25 0x00007ffff7a6cc67 in pymain_run_file_obj (skip_source_first_line=0, filename=0x7ffff6ff9f70, program_name=0x7ffff6ff9f30) at Modules/main.c:360
#26 pymain_run_file (config=0x7ffff7dc56c8 <_PyRuntime+77672>) at Modules/main.c:379
#27 pymain_run_python (exitcode=0x7fffffffda14) at Modules/main.c:629
#28 Py_RunMain () at Modules/main.c:709
#29 0x00007ffff7a28fab in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:763
#30 0x00007ffff7639c88 in __libc_start_call_main (main=main@entry=0x555555555120 <main>, argc=argc@entry=2, argv=argv@entry=0x7fffffffdca8) at ../sysdeps/nptl/libc_start_call_main.h:58
#31 0x00007ffff7639d4c in __libc_start_main_impl (main=0x555555555120 <main>, argc=2, argv=0x7fffffffdca8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdc98) at ../csu/libc-start.c:360
#32 0x0000555555555045 in _start ()

(gdb) thread apply all backtrace full

Thread 1 (Thread 0x7ffff7e95740 (LWP 412129) "python"):
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
        tid = <optimized out>
        ret = 0
        pd = <optimized out>
        old_mask = {__val = {140737488342832}}
        ret = <optimized out>
#1  0x00007ffff76a8eb3 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:78
No locals.
#2  0x00007ffff7650a30 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
        ret = <optimized out>
#3  0x00007ffff76384c3 in __GI_abort () at abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0x20, sa_sigaction = 0x20}, sa_mask = {__val = {5028771909574692352, 140737488342864, 1, 93825007878193, 140737488343088, 140737488343536, 5028771909574692352, 140737488343536, 93825007906816, 93825007906752, 140737276632937, 140737488342982, 140737488343072, 140737488343008, 140737276633073, 140737488343056}}, sa_flags = 892719408, sa_restorer = 0x303365323363}
#4  0x00007ffff361317e in tcmalloc::Log (mode=mode@entry=tcmalloc::kCrash, filename=filename@entry=0x7ffff36390c6 "src/tcmalloc.cc", line=line@entry=304, a=..., b=..., c=..., d=...) at src/internal_logging.cc:118
        state = {static kBufSize = 200, p_ = 0x7fffffffd065 "", end_ = 0x7fffffffd0e8 "", buf_ = "src/tcmalloc.cc:304] Attempt to free invalid pointer 0x555556c32e30 \n\000\000\000 \343|\363\377\177\000\000\200rDVUU\000\000\f\000\000\000\000\000\000\000\240\320\377\377\377\177\000\000\017`b\363\377\177\000\000 \343|\363\377\177\000\000\240pDVUU\000\000\340\320\377\377\377\177\000\000\272bb\363\377\177\000\000\030r\371\367\377\177\000\000\200\n\273\364\377\177\000\000\360\000\343\366\377\177\000\000 \000\000\000\000\000\000\000 \343|\363\377\177\000\000@\300\304UUU\000\0000\321\377\377\377\177\000"}
        msglen = <optimized out>
        first_crash = <optimized out>
#5  0x00007ffff3617a54 in (anonymous namespace)::InvalidFree (ptr=<optimized out>) at src/tcmalloc.cc:304
No locals.
#6  0x00007ffff40602d9 in std::__new_allocator<SymEngine::RCP<SymEngine::Basic const> >::deallocate (this=<optimized out>, __n=<optimized out>, __p=<optimized out>) at /usr/include/c++/13.2.1/bits/new_allocator.h:152
No locals.
#7  std::allocator_traits<std::allocator<char> >::deallocate (__n=<optimized out>, __p=<optimized out>, __a=...) at /usr/include/c++/13.2.1/bits/alloc_traits.h:516
No locals.
#8  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_destroy (__size=<optimized out>, this=0x7fffffffd420) at /usr/include/c++/13.2.1/bits/basic_string.h:289
No locals.
#9  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose (this=0x7fffffffd420) at /usr/include/c++/13.2.1/bits/basic_string.h:283
No locals.
#10 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string (this=<optimized out>, this=<optimized out>) at /usr/include/c++/13.2.1/bits/basic_string.h:792
No locals.
#11 std::__cxx11::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >::~basic_stringbuf (this=<optimized out>, this=<optimized out>) at /usr/include/c++/13.2.1/sstream:79
No locals.
#12 std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >::~basic_ostringstream (this=<optimized out>, this=<optimized out>) at /usr/include/c++/13.2.1/sstream:851
No locals.
#13 SymEngine::wrapper_dumps[abi:cxx11](SymEngine::Basic const&) (x=...) at /usr/src/debug/python-symengine/symengine.py-0.11.0/symengine/lib/pywrapper.cpp:350
        oss = Python Exception <class 'UnicodeDecodeError'>: 'utf-8' codec can't decode byte 0x80 in position 8: invalid start byte

        major = 0
        minor = 11
#14 0x00007ffff3ecd96a in __pyx_pf_9symengine_3lib_17symengine_wrapper_5Basic_4__reduce__ (__pyx_v_self=0x7ffff6ea2d90) at /usr/include/symengine/symengine_rcp.h:150
        __pyx_v_s = 0x0
        __pyx_t_3 = 0x0
        __pyx_clineno = 0
        __pyx_r = 0x0
        __pyx_t_1 = ""
        __pyx_t_4 = 0x0
        __pyx_filename = 0x0
        __pyx_t_2 = 0x0
        __pyx_lineno = 0
        __pyx_L0 = <optimized out>
        __pyx_L1_error = <optimized out>
        __pyx_v_s = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--c
        __pyx_r = <optimized out>
        __pyx_t_1 = <optimized out>
        __pyx_t_2 = <optimized out>
        __pyx_t_3 = <optimized out>
        __pyx_t_4 = <optimized out>
        __pyx_lineno = <optimized out>
        __pyx_filename = <optimized out>
        __pyx_clineno = <optimized out>
#15 __pyx_pw_9symengine_3lib_17symengine_wrapper_5Basic_5__reduce__ (__pyx_v_self=0x7ffff6ea2d90, __pyx_args=<optimized out>, __pyx_nargs=<optimized out>, __pyx_kwds=<optimized out>) at /usr/src/debug/python-symengine/build/symengine/lib/symengine_wrapper.cpp:53890
        __pyx_kwvalues = <optimized out>
        __pyx_r = 0x0
#16 0x00007ffff799a844 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x7ffff7f97070, callable=0x7fffe81357d0, tstate=0x7ffff7e22ae8 <_PyRuntime+459656>) at ./Include/internal/pycore_call.h:92
        func = <optimized out>
        res = <optimized out>
        func = <optimized out>
        res = <optimized out>
        nargs = <optimized out>
#17 PyObject_Vectorcall (callable=0x7fffe81357d0, args=0x7ffff7f97070, nargsf=9223372036854775809, kwnames=0x0) at Objects/call.c:325
        tstate = 0x7ffff7e22ae8 <_PyRuntime+459656>
#18 0x00007ffff788bdfa in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=0x7ffff7f97020, throwflag=<optimized out>) at Python/bytecodes.c:2706
        callable = 0x7fffe81357d0
        res = <optimized out>
        method = <optimized out>
        self = <optimized out>
        op = <optimized out>
        dealloc = <optimized out>
        word = <optimized out>
        args = 0x7ffff7f97070
        is_meth = <optimized out>
        total_args = 1
        cache = 0x7ffff6e18660
        positional_args = <optimized out>
        opcode = 171 '\253'
        oparg = <optimized out>
        cframe = {current_frame = 0x7ffff7f97020, previous = 0x7ffff7e22bf8 <_PyRuntime+459928>}
        entry_frame = {f_code = 0x7ffff6f12730, previous = 0x0, f_funcobj = 0x0, f_globals = 0x7ffff798ad2e <_PyEvalFramePushAndInit+206>, f_builtins = 0x7fffffffd7f0, f_locals = 0x7ffff796c043 <_PyObject_GC_New+259>, frame_obj = 0x7ffff7e22ae8 <_PyRuntime+459656>, prev_instr = 0x7ffff6f127f0, stacktop = 0, return_offset = 0, owner = 3 '\003', localsplus = {0x7ffff7e22ae8 <_PyRuntime+459656>}}
        kwnames = <optimized out>
        prev_cframe = <optimized out>
        next_instr = 0x7ffff6e18660
        stack_pointer = <optimized out>
        exception_unwind = <optimized out>
        dying = <optimized out>
        __func__ = "_PyEval_EvalFrameDefault"
        opcode_targets = {0x7ffff788b87a <_PyEval_EvalFrameDefault-1053862>, 0x7ffff7896f38 <_PyEval_EvalFrameDefault-1007080>, 0x7ffff7896f00 <_PyEval_EvalFrameDefault-1007136>, 0x7ffff78967e7 <_PyEval_EvalFrameDefault-1008953>, 0x7ffff78940bb <_PyEval_EvalFrameDefault-1018981>, 0x7ffff7898c30 <_PyEval_EvalFrameDefault-999664>, 0x7ffff7891a02 <_PyEval_EvalFrameDefault-1028894>, 0x7ffff78970b6 <_PyEval_EvalFrameDefault-1006698>, 0x7ffff7895036 <_PyEval_EvalFrameDefault-1015018>, 0x7ffff7894513 <_PyEval_EvalFrameDefault-1017869>, 0x7ffff7891aa0 <_PyEval_EvalFrameDefault-1028736>, 0x7ffff78926af <_PyEval_EvalFrameDefault-1025649>, 0x7ffff7892643 <_PyEval_EvalFrameDefault-1025757>, 0x7ffff789366b <_PyEval_EvalFrameDefault-1021621>, 0x7ffff789608f <_PyEval_EvalFrameDefault-1010833>, 0x7ffff789618b <_PyEval_EvalFrameDefault-1010581>, 0x7ffff78950de <_PyEval_EvalFrameDefault-1014850>, 0x7ffff788b86d <_PyEval_EvalFrameDefault-1053875>, 0x7ffff78935d1 <_PyEval_EvalFrameDefault-1021775>, 0x7ffff7892a8e <_PyEval_EvalFrameDefault-1024658>, 0x7ffff7892999 <_PyEval_EvalFrameDefault-1024903>, 0x7ffff7895c21 <_PyEval_EvalFrameDefault-1011967>, 0x7ffff7895b68 <_PyEval_EvalFrameDefault-1012152>, 0x7ffff788bac5 <_PyEval_EvalFrameDefault-1053275>, 0x7ffff788f9e8 <_PyEval_EvalFrameDefault-1037112>, 0x7ffff7892b63 <_PyEval_EvalFrameDefault-1024445>, 0x7ffff78992a8 <_PyEval_EvalFrameDefault-998008>, 0x7ffff789428d <_PyEval_EvalFrameDefault-1018515>, 0x7ffff788bc9f <_PyEval_EvalFrameDefault-1052801>, 0x7ffff7891127 <_PyEval_EvalFrameDefault-1031161>, 0x7ffff788d983 <_PyEval_EvalFrameDefault-1045405>, 0x7ffff788f354 <_PyEval_EvalFrameDefault-1038796>, 0x7ffff7890820 <_PyEval_EvalFrameDefault-1033472>, 0x7ffff788de58 <_PyEval_EvalFrameDefault-1044168>, 0x7ffff788c584 <_PyEval_EvalFrameDefault-1050524>, 0x7ffff788d54f <_PyEval_EvalFrameDefault-1046481>, 0x7ffff7894569 <_PyEval_EvalFrameDefault-1017783>, 0x7ffff78945e8 <_PyEval_EvalFrameDefault-1017656>, 0x7ffff788ce15 <_PyEval_EvalFrameDefault-1048331>, 0x7ffff788d5b7 <_PyEval_EvalFrameDefault-1046377>, 0x7ffff788ee0f <_PyEval_EvalFrameDefault-1040145>, 0x7ffff788f213 <_PyEval_EvalFrameDefault-1039117>, 0x7ffff788c3c2 <_PyEval_EvalFrameDefault-1050974>, 0x7ffff788dc82 <_PyEval_EvalFrameDefault-1044638>, 0x7ffff788cc34 <_PyEval_EvalFrameDefault-1048812>, 0x7ffff788b1fe <_PyEval_EvalFrameDefault-1055522>, 0x7ffff788d77e <_PyEval_EvalFrameDefault-1045922>, 0x7ffff788e1df <_PyEval_EvalFrameDefault-1043265>, 0x7ffff788e102 <_PyEval_EvalFrameDefault-1043486>, 0x7ffff788f678 <_PyEval_EvalFrameDefault-1037992>, 0x7ffff7892491 <_PyEval_EvalFrameDefault-1026191>, 0x7ffff78923e6 <_PyEval_EvalFrameDefault-1026362>, 0x7ffff788e8b3 <_PyEval_EvalFrameDefault-1041517>, 0x7ffff788e76b <_PyEval_EvalFrameDefault-1041845>, 0x7ffff78917f3 <_PyEval_EvalFrameDefault-1029421>, 0x7ffff789743f <_PyEval_EvalFrameDefault-1005793>, 0x7ffff788e581 <_PyEval_EvalFrameDefault-1042335>, 0x7ffff789215a <_PyEval_EvalFrameDefault-1027014>, 0x7ffff78987e7 <_PyEval_EvalFrameDefault-1000761>, 0x7ffff789231a <_PyEval_EvalFrameDefault-1026566>, 0x7ffff7893872 <_PyEval_EvalFrameDefault-1021102>, 0x7ffff789530e <_PyEval_EvalFrameDefault-1014290>, 0x7ffff7890d0f <_PyEval_EvalFrameDefault-1032209>, 0x7ffff788ed8b <_PyEval_EvalFrameDefault-1040277>, 0x7ffff788e019 <_PyEval_EvalFrameDefault-1043719>, 0x7ffff788df66 <_PyEval_EvalFrameDefault-1043898>, 0x7ffff78961fa <_PyEval_EvalFrameDefault-1010470>, 0x7ffff7898299 <_PyEval_EvalFrameDefault-1002119>, 0x7ffff7890889 <_PyEval_EvalFrameDefault-1033367>, 0x7ffff788eb63 <_PyEval_EvalFrameDefault-1040829>, 0x7ffff7896737 <_PyEval_EvalFrameDefault-1009129>, 0x7ffff78943ed <_PyEval_EvalFrameDefault-1018163>, 0x7ffff78906fc <_PyEval_EvalFrameDefault-1033764>, 0x7ffff7895260 <_PyEval_EvalFrameDefault-1014464>, 0x7ffff7897161 <_PyEval_EvalFrameDefault-1006527>, 0x7ffff7890081 <_PyEval_EvalFrameDefault-1035423>, 0x7ffff789519e <_PyEval_EvalFrameDefault-1014658>, 0x7ffff7896618 <_PyEval_EvalFrameDefault-1009416>, 0x7ffff7891b52 <_PyEval_EvalFrameDefault-1028558>, 0x7ffff7891bef <_PyEval_EvalFrameDefault-1028401>, 0x7ffff788dbf7 <_PyEval_EvalFrameDefault-1044777>, 0x7ffff788e624 <_PyEval_EvalFrameDefault-1042172>, 0x7ffff788e6a0 <_PyEval_EvalFrameDefault-1042048>, 0x7ffff7893e1b <_PyEval_EvalFrameDefault-1019653>, 0x7ffff789203d <_PyEval_EvalFrameDefault-1027299>, 0x7ffff78941c5 <_PyEval_EvalFrameDefault-1018715>, 0x7ffff7895649 <_PyEval_EvalFrameDefault-1013463>, 0x7ffff7891ef7 <_PyEval_EvalFrameDefault-1027625>, 0x7ffff7893b63 <_PyEval_EvalFrameDefault-1020349>, 0x7ffff7894924 <_PyEval_EvalFrameDefault-1016828>, 0x7ffff7894358 <_PyEval_EvalFrameDefault-1018312>, 0x7ffff78979b7 <_PyEval_EvalFrameDefault-1004393>, 0x7ffff7895f13 <_PyEval_EvalFrameDefault-1011213>, 0x7ffff788d0ad <_PyEval_EvalFrameDefault-1047667>, 0x7ffff7896409 <_PyEval_EvalFrameDefault-1009943>, 0x7ffff7893aaf <_PyEval_EvalFrameDefault-1020529>, 0x7ffff7893a36 <_PyEval_EvalFrameDefault-1020650>, 0x7ffff7895515 <_PyEval_EvalFrameDefault-1013771>, 0x7ffff78954bd <_PyEval_EvalFrameDefault-1013859>, 0x7ffff788d453 <_PyEval_EvalFrameDefault-1046733>, 0x7ffff78965c7 <_PyEval_EvalFrameDefault-1009497>, 0x7ffff78969b9 <_PyEval_EvalFrameDefault-1008487>, 0x7ffff7894e3f <_PyEval_EvalFrameDefault-1015521>, 0x7ffff7894dc1 <_PyEval_EvalFrameDefault-1015647>, 0x7ffff78986d5 <_PyEval_EvalFrameDefault-1001035>, 0x7ffff7897014 <_PyEval_EvalFrameDefault-1006860>, 0x7ffff789378b <_PyEval_EvalFrameDefault-1021333>, 0x7ffff7892268 <_PyEval_EvalFrameDefault-1026744>, 0x7ffff78971ab <_PyEval_EvalFrameDefault-1006453>, 0x7ffff7891792 <_PyEval_EvalFrameDefault-1029518>, 0x7ffff7890ad7 <_PyEval_EvalFrameDefault-1032777>, 0x7ffff7893e78 <_PyEval_EvalFrameDefault-1019560>, 0x7ffff7893f48 <_PyEval_EvalFrameDefault-1019352>, 0x7ffff7893bd3 <_PyEval_EvalFrameDefault-1020237>, 0x7ffff7890a45 <_PyEval_EvalFrameDefault-1032923>, 0x7ffff788e2ed <_PyEval_EvalFrameDefault-1042995>, 0x7ffff7896877 <_PyEval_EvalFrameDefault-1008809>, 0x7ffff7892ee0 <_PyEval_EvalFrameDefault-1023552>, 0x7ffff7892e24 <_PyEval_EvalFrameDefault-1023740>, 0x7ffff7894a24 <_PyEval_EvalFrameDefault-1016572>, 0x7ffff788ebc0 <_PyEval_EvalFrameDefault-1040736>, 0x7ffff7897d0e <_PyEval_EvalFrameDefault-1003538>, 0x7ffff788d49b <_PyEval_EvalFrameDefault-1046661>, 0x7ffff7893048 <_PyEval_EvalFrameDefault-1023192>, 0x7ffff7892d05 <_PyEval_EvalFrameDefault-1024027>, 0x7ffff7898df8 <_PyEval_EvalFrameDefault-999208>, 0x7ffff78957ea <_PyEval_EvalFrameDefault-1013046>, 0x7ffff7892d4f <_PyEval_EvalFrameDefault-1023953>, 0x7ffff788da63 <_PyEval_EvalFrameDefault-1045181>, 0x7ffff788da18 <_PyEval_EvalFrameDefault-1045256>, 0x7ffff789680e <_PyEval_EvalFrameDefault-1008914>, 0x7ffff7894b15 <_PyEval_EvalFrameDefault-1016331>, 0x7ffff788f493 <_PyEval_EvalFrameDefault-1038477>, 0x7ffff788deaf <_PyEval_EvalFrameDefault-1044081>, 0x7ffff788d9e3 <_PyEval_EvalFrameDefault-1045309>, 0x7ffff788b408 <_PyEval_EvalFrameDefault-1055000>, 0x7ffff7892dc9 <_PyEval_EvalFrameDefault-1023831>, 0x7ffff7896003 <_PyEval_EvalFrameDefault-1010973>, 0x7ffff7895fb5 <_PyEval_EvalFrameDefault-1011051>, 0x7ffff7897c94 <_PyEval_EvalFrameDefault-1003660>, 0x7ffff788fcf3 <_PyEval_EvalFrameDefault-1036333>, 0x7ffff78983db <_PyEval_EvalFrameDefault-1001797>, 0x7ffff788caf4 <_PyEval_EvalFrameDefault-1049132>, 0x7ffff7898fe5 <_PyEval_EvalFrameDefault-998715>, 0x7ffff788ba8c <_PyEval_EvalFrameDefault-1053332>, 0x7ffff789636c <_PyEval_EvalFrameDefault-1010100>, 0x7ffff78962f9 <_PyEval_EvalFrameDefault-1010215>, 0x7ffff78928ce <_PyEval_EvalFrameDefault-1025106>, 0x7ffff7895700 <_PyEval_EvalFrameDefault-1013280>, 0x7ffff78934a5 <_PyEval_EvalFrameDefault-1022075>, 0x7ffff7894983 <_PyEval_EvalFrameDefault-1016733>, 0x7ffff7890575 <_PyEval_EvalFrameDefault-1034155>, 0x7ffff788f3a8 <_PyEval_EvalFrameDefault-1038712>, 0x7ffff7893c7b <_PyEval_EvalFrameDefault-1020069>, 0x7ffff78955d3 <_PyEval_EvalFrameDefault-1013581>, 0x7ffff788ec0d <_PyEval_EvalFrameDefault-1040659>, 0x7ffff789412e <_PyEval_EvalFrameDefault-1018866>, 0x7ffff7893352 <_PyEval_EvalFrameDefault-1022414>, 0x7ffff78920b5 <_PyEval_EvalFrameDefault-1027179>, 0x7ffff7895387 <_PyEval_EvalFrameDefault-1014169>, 0x7ffff78937ad <_PyEval_EvalFrameDefault-1021299>, 0x7ffff789648e <_PyEval_EvalFrameDefault-1009810>, 0x7ffff78918f9 <_PyEval_EvalFrameDefault-1029159>, 0x7ffff7891886 <_PyEval_EvalFrameDefault-1029274>, 0x7ffff788ea63 <_PyEval_EvalFrameDefault-1041085>, 0x7ffff7895994 <_PyEval_EvalFrameDefault-1012620>, 0x7ffff7892bfc <_PyEval_EvalFrameDefault-1024292>, 0x7ffff789772d <_PyEval_EvalFrameDefault-1005043>, 0x7ffff788b66e <_PyEval_EvalFrameDefault-1054386>, 0x7ffff788b768 <_PyEval_EvalFrameDefault-1054136>, 0x7ffff788b768 <_PyEval_EvalFrameDefault-1054136>, 0x7ffff788bf38 <_PyEval_EvalFrameDefault-1052136>, 0x7ffff788dbb6 <_PyEval_EvalFrameDefault-1044842>, 0x7ffff7891dcb <_PyEval_EvalFrameDefault-1027925>, 0x7ffff7891d37 <_PyEval_EvalFrameDefault-1028073>, 0x7ffff7891e65 <_PyEval_EvalFrameDefault-1027771>, 0x7ffff7892573 <_PyEval_EvalFrameDefault-1025965>, 0x7ffff788b768 <_PyEval_EvalFrameDefault-1054136> <repeats 60 times>, 0x7ffff7892749 <_PyEval_EvalFrameDefault-1025495>, 0x7ffff788e345 <_PyEval_EvalFrameDefault-1042907>, 0x7ffff788fd3a <_PyEval_EvalFrameDefault-1036262>, 0x7ffff7898f02 <_PyEval_EvalFrameDefault-998942>, 0x7ffff788e4ff <_PyEval_EvalFrameDefault-1042465>, 0x7ffff7893d8d <_PyEval_EvalFrameDefault-1019795>, 0x7ffff7892fa6 <_PyEval_EvalFrameDefault-1023354>, 0x7ffff788caf0 <_PyEval_EvalFrameDefault-1049136>, 0x7ffff788efaf <_PyEval_EvalFrameDefault-1039729>, 0x7ffff788fe18 <_PyEval_EvalFrameDefault-1036040>, 0x7ffff7895782 <_PyEval_EvalFrameDefault-1013150>, 0x7ffff788d000 <_PyEval_EvalFrameDefault-1047840>, 0x7ffff788e3dc <_PyEval_EvalFrameDefault-1042756>...}
#19 0x00007ffff7a3d767 in PyEval_EvalCode (co=0x7ffff6e18530, globals=<optimized out>, locals=0x7ffff6ff9dc0) at Python/ceval.c:578
        tstate = 0x7ffff7e22ae8 <_PyRuntime+459656>
        builtins = <optimized out>
        desc = {fc_globals = 0x7ffff6ff9dc0, fc_builtins = 0x7ffff6f9cd00, fc_name = 0x7ffff7db8550 <_PyRuntime+24048>, fc_qualname = 0x7ffff7db8550 <_PyRuntime+24048>, fc_code = 0x7ffff6e18530, fc_defaults = 0x0, fc_kwdefaults = 0x0, fc_closure = 0x0}
        func = 0x7ffff6e45440
        res = <optimized out>
#20 0x00007ffff7a608b7 in run_eval_code_obj (tstate=tstate@entry=0x7ffff7e22ae8 <_PyRuntime+459656>, co=co@entry=0x7ffff6e18530, globals=globals@entry=0x7ffff6ff9dc0, locals=locals@entry=0x7ffff6ff9dc0) at Python/pythonrun.c:1722
        v = <optimized out>
#21 0x00007ffff7a5b9dc in run_mod (mod=mod@entry=0x55555563b298, filename=filename@entry=0x7ffff6ff9f70, globals=globals@entry=0x7ffff6ff9dc0, locals=locals@entry=0x7ffff6ff9dc0, flags=flags@entry=0x7fffffffda40, arena=arena@entry=0x7ffff6f1be30) at Python/pythonrun.c:1743
        tstate = 0x7ffff7e22ae8 <_PyRuntime+459656>
        co = 0x7ffff6e18530
        v = <optimized out>
#22 0x00007ffff7a74f33 in pyrun_file (fp=fp@entry=0x5555555a4f30, filename=filename@entry=0x7ffff6ff9f70, start=start@entry=257, globals=globals@entry=0x7ffff6ff9dc0, locals=locals@entry=0x7ffff6ff9dc0, closeit=closeit@entry=1, flags=0x7fffffffda40) at Python/pythonrun.c:1643
        arena = 0x7ffff6f1be30
        mod = 0x55555563b298
        ret = <optimized out>
#23 0x00007ffff7a74346 in _PyRun_SimpleFileObject (fp=0x5555555a4f30, filename=0x7ffff6ff9f70, closeit=1, flags=0x7fffffffda40) at Python/pythonrun.c:433
        m = 0x7ffff6fcb2e0
        d = 0x7ffff6ff9dc0
        v = <optimized out>
        set_file_name = 1
        ret = -1
        pyc = <optimized out>
#24 0x00007ffff7a73f88 in _PyRun_AnyFileObject (fp=0x5555555a4f30, filename=0x7ffff6ff9f70, closeit=1, flags=0x7fffffffda40) at Python/pythonrun.c:78
        decref_filename = 0
        res = <optimized out>
#25 0x00007ffff7a6cc67 in pymain_run_file_obj (skip_source_first_line=0, filename=0x7ffff6ff9f70, program_name=0x7ffff6ff9f30) at Modules/main.c:360
        fp = 0x5555555a4f30
        sb = {st_dev = 65024, st_ino = 786451, st_nlink = 1, st_mode = 33188, st_uid = 1000, st_gid = 1000, __pad0 = 0, st_rdev = 0, st_size = 227, st_blksize = 4096, st_blocks = 8, st_atim = {tv_sec = 1715415552, tv_nsec = 782346367}, st_mtim = {tv_sec = 1715415543, tv_nsec = 712323243}, st_ctim = {tv_sec = 1715415543, tv_nsec = 712323243}, __glibc_reserved = {0, 0, 0}}
        cf = {cf_flags = 0, cf_feature_version = 12}
        run = <optimized out>
        fp = <optimized out>
        sb = <optimized out>
        cf = <optimized out>
        run = <optimized out>
        ch = <optimized out>
#26 pymain_run_file (config=0x7ffff7dc56c8 <_PyRuntime+77672>) at Modules/main.c:379
        filename = 0x7ffff6ff9f70
        program_name = 0x7ffff6ff9f30
        res = <optimized out>
        filename = <optimized out>
        program_name = <optimized out>
        res = <optimized out>
#27 pymain_run_python (exitcode=0x7fffffffda14) at Modules/main.c:629
        main_importer_path = <optimized out>
        interp = 0x7ffff7dc51c8 <_PyRuntime+76392>
        config = 0x7ffff7dc56c8 <_PyRuntime+77672>
        path0 = 0x7ffff6e209f0
        main_importer_path = <optimized out>
        interp = <optimized out>
        config = <optimized out>
        error = <optimized out>
        path0 = <optimized out>
        done = <optimized out>
        res = <optimized out>
        _tmp_op_ptr = <optimized out>
        _tmp_old_op = <optimized out>
        wstr = <optimized out>
        old_alloc = <optimized out>
        res = <optimized out>
#28 Py_RunMain () at Modules/main.c:709
        exitcode = 0
#29 0x00007ffff7a28fab in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:763
        args = {argc = 2, use_bytes_argv = 1, bytes_argv = 0x7fffffffdca8, wchar_argv = 0x0}
#30 0x00007ffff7639c88 in __libc_start_call_main (main=main@entry=0x555555555120 <main>, argc=argc@entry=2, argv=argv@entry=0x7fffffffdca8) at ../sysdeps/nptl/libc_start_call_main.h:58
        self = <optimized out>
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737488346280, 1086504349247724366, 2, 0, 140737354125312, 93824992247232, 1086504349262404430, 1086520667977669454}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x2, 0x7fffffffdca0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 2}}}
        not_first_call = <optimized out>
#31 0x00007ffff7639d4c in __libc_start_main_impl (main=0x555555555120 <main>, argc=2, argv=0x7fffffffdca8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdc98) at ../csu/libc-start.c:360
No locals.
#32 0x0000555555555045 in _start ()
No symbol table info available.