Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_mpm99 crash: Cannot lookup function substep_c2328_0_kernel_3_range_for #6593

Closed
feisuzhu opened this issue Nov 14, 2022 · 2 comments
Closed

test_mpm99 crash: Cannot lookup function substep_c2328_0_kernel_3_range_for #6593

feisuzhu opened this issue Nov 14, 2022 · 2 comments
Assignees
@PGZXB
Labels
flaky-test

Comments

@feisuzhu
Copy link
Contributor

https://github.com/taichi-dev/taichi/actions/runs/3459744305/attempts/1

=================================== FAILURES ===================================
_______________________________ test_mpm99[none] _______________________________
[gw2] linux -- Python 3.8.13 /home/dev/miniconda/envs/py38/bin/python3

    @pytest.mark.skipif(os.environ.get('TI_LITE_TEST') or '0', reason='Lite test')
    def test_mpm99():
        from taichi.examples.simulation.mpm99 import dt, initialize, substep
    
        initialize()
        for i in range(FRAMES):
            for s in range(int(2e-3 // dt)):
>               substep()

tests/python/examples/simulation/test_mpm99.py:18: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../miniconda/envs/py38/lib/python3.8/site-packages/taichi/lang/kernel_impl.py:947: in wrapped
    return primal(*args, **kwargs)
../miniconda/envs/py38/lib/python3.8/site-packages/taichi/lang/shell.py:37: in new_call
    ret = old_call(*args, **kwargs)
../miniconda/envs/py38/lib/python3.8/site-packages/taichi/lang/kernel_impl.py:874: in __call__
    return self.runtime.compiled_functions[key](*args)
../miniconda/envs/py38/lib/python3.8/site-packages/taichi/lang/kernel_impl.py:799: in func__
    raise e from None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (), tmps = [], callbacks = [], actual_argument_slot = 0

    def func__(*args):
        assert len(args) == len(
            self.arguments
        ), f'{len(self.arguments)} arguments needed but {len(args)} provided'
    
        tmps = []
        callbacks = []
    
        actual_argument_slot = 0
        launch_ctx = t_kernel.make_launch_context()
        for i, v in enumerate(args):
            needed = self.arguments[i].annotation
            if isinstance(needed, template):
                continue
            provided = type(v)
            # Note: do not use sth like "needed == f32". That would be slow.
            if id(needed) in primitive_types.real_type_ids:
                if not isinstance(v, (float, int)):
                    raise TaichiRuntimeTypeError.get(
                        i, needed.to_string(), provided)
                launch_ctx.set_arg_float(actual_argument_slot, float(v))
            elif id(needed) in primitive_types.integer_type_ids:
                if not isinstance(v, int):
                    raise TaichiRuntimeTypeError.get(
                        i, needed.to_string(), provided)
                if is_signed(cook_dtype(needed)):
                    launch_ctx.set_arg_int(actual_argument_slot, int(v))
                else:
                    launch_ctx.set_arg_uint(actual_argument_slot, int(v))
            elif isinstance(needed, sparse_matrix_builder):
                # Pass only the base pointer of the ti.types.sparse_matrix_builder() argument
                launch_ctx.set_arg_uint(actual_argument_slot,
                                        v._get_addr())
            elif isinstance(needed,
                            ndarray_type.NdarrayType) and isinstance(
                                v, taichi.lang._ndarray.Ndarray):
                launch_ctx.set_arg_ndarray(actual_argument_slot, v.arr)
            elif isinstance(needed,
                            texture_type.TextureType) and isinstance(
                                v, taichi.lang._texture.Texture):
                launch_ctx.set_arg_texture(actual_argument_slot, v.tex)
            elif isinstance(needed,
                            texture_type.RWTextureType) and isinstance(
                                v, taichi.lang._texture.Texture):
                launch_ctx.set_arg_rw_texture(actual_argument_slot, v.tex)
            elif isinstance(needed, ndarray_type.NdarrayType):
                # Element shapes are already specialized in Taichi codegen.
                # The shape information for element dims are no longer needed.
                # Therefore we strip the element shapes from the shape vector,
                # so that it only holds "real" array shapes.
                is_soa = needed.layout == Layout.SOA
                array_shape = v.shape
                element_dim = needed.element_dim
                if element_dim:
                    array_shape = v.shape[
                        element_dim:] if is_soa else v.shape[:-element_dim]
                if isinstance(v, np.ndarray):
                    if v.flags.c_contiguous:
                        launch_ctx.set_arg_external_array_with_shape(
                            actual_argument_slot, int(v.ctypes.data),
                            v.nbytes, array_shape)
                    elif v.flags.f_contiguous:
                        # TODO: A better way that avoids copying is saving strides info.
                        tmp = np.ascontiguousarray(v)
                        # Purpose: DO NOT GC |tmp|!
                        tmps.append(tmp)
    
                        def callback(original, updated):
                            np.copyto(original, np.asfortranarray(updated))
    
                        callbacks.append(
                            functools.partial(callback, v, tmp))
                        launch_ctx.set_arg_external_array_with_shape(
                            actual_argument_slot, int(tmp.ctypes.data),
                            tmp.nbytes, array_shape)
                    else:
                        raise ValueError(
                            "Non contiguous numpy arrays are not supported, please call np.ascontiguousarray(arr) before passing it into taichi kernel."
                        )
                elif has_pytorch() and isinstance(v, torch.Tensor):
                    tmp, torch_callbacks = self.get_torch_callbacks(v)
                    callbacks += torch_callbacks
                    launch_ctx.set_arg_external_array_with_shape(
                        actual_argument_slot, int(tmp.data_ptr()),
                        tmp.element_size() * tmp.nelement(), array_shape)
                elif has_paddle() and isinstance(v, paddle.Tensor):
                    # For now, paddle.fluid.core.Tensor._ptr() is only available on develop branch
                    tmp, paddle_callbacks = self.get_paddle_callbacks(v)
                    callbacks += paddle_callbacks
                    launch_ctx.set_arg_external_array_with_shape(
                        actual_argument_slot, int(tmp._ptr()),
                        v.element_size() * v.size, array_shape)
                else:
                    raise TaichiRuntimeTypeError.get(
                        i, needed.to_string(), v)
    
            elif isinstance(needed, MatrixType):
                if needed.dtype in primitive_types.real_types:
                    for a in range(needed.n):
                        for b in range(needed.m):
                            val = v[a, b] if needed.ndim == 2 else v[a]
                            if not isinstance(val, (int, float)):
                                raise TaichiRuntimeTypeError.get(
                                    i, needed.dtype.to_string(), type(val))
                            launch_ctx.set_arg_float(
                                actual_argument_slot, float(val))
                            actual_argument_slot += 1
                elif needed.dtype in primitive_types.integer_types:
                    for a in range(needed.n):
                        for b in range(needed.m):
                            val = v[a, b] if needed.ndim == 2 else v[a]
                            if not isinstance(val, int):
                                raise TaichiRuntimeTypeError.get(
                                    i, needed.dtype.to_string(), type(val))
                            if is_signed(needed.dtype):
                                launch_ctx.set_arg_int(
                                    actual_argument_slot, int(val))
                            else:
                                launch_ctx.set_arg_uint(
                                    actual_argument_slot, int(val))
                            actual_argument_slot += 1
                else:
                    raise ValueError(
                        f'Matrix dtype {needed.dtype} is not integer type or real type.'
                    )
                continue
            else:
                raise ValueError(
                    f'Argument type mismatch. Expecting {needed}, got {type(v)}.'
                )
            actual_argument_slot += 1
    
        if actual_argument_slot > 8 and impl.current_cfg(
        ).arch == _ti_core.cc:
            raise TaichiRuntimeError(
                f"The number of elements in kernel arguments is too big! Do not exceed 8 on {_ti_core.arch_name(impl.current_cfg().arch)} backend."
            )
    
        if actual_argument_slot > 64 and impl.current_cfg(
        ).arch != _ti_core.cc:
            raise TaichiRuntimeError(
                f"The number of elements in kernel arguments is too big! Do not exceed 64 on {_ti_core.arch_name(impl.current_cfg().arch)} backend."
            )
    
        try:
>           t_kernel(launch_ctx)
E           RuntimeError: [jit_cuda.h:lookup_function@58] Cannot look up function substep_c2328_0_kernel_3_range_for

../miniconda/envs/py38/lib/python3.8/site-packages/taichi/lang/kernel_impl.py:796: RuntimeError
----------------------------- Captured stdout call -----------------------------
[Taichi] Starting on arch=cuda
[W 11/14/22 09:11:52.338 5049] [cuda_driver.h:call_with_warning@81] CUDA Error CUDA_ERROR_NOT_FOUND: named symbol not found while calling module_get_function (cuModuleGetFunction)
[E 11/14/22 09:11:52.338 5049] [jit_cuda.h:lookup_function@58] Cannot look up function substep_c2328_0_kernel_3_range_for


***********************************
* Taichi Compiler Stack Traceback *
***********************************
/home/dev/miniconda/envs/py38/lib/python3.8/site-packages/taichi/_lib/core/taichi_python.cpython-38-x86_64-linux-gnu.so(+0x325c3a7) [0x7fc1188933a7]
/home/dev/miniconda/envs/py38/lib/python3.8/site-packages/taichi/_lib/core/taichi_python.cpython-38-x86_64-linux-gnu.so(+0x900a77) [0x7fc115f37a77]
/home/dev/miniconda/envs/py38/lib/python3.8/site-packages/taichi/_lib/core/taichi_python.cpython-38-x86_64-linux-gnu.so(+0x900c52) [0x7fc115f37c52]
/home/dev/miniconda/envs/py38/lib/python3.8/site-packages/taichi/_lib/core/taichi_python.cpython-38-x86_64-linux-gnu.so(+0x8dc85a) [0x7fc115f1385a]
/home/dev/miniconda/envs/py38/lib/python3.8/site-packages/taichi/_lib/core/taichi_python.cpython-38-x86_64-linux-gnu.so: taichi::lang::Kernel::operator()(taichi::lang::Kernel::LaunchContextBuilder&)
/home/dev/miniconda/envs/py38/lib/python3.8/site-packages/taichi/_lib/core/taichi_python.cpython-38-x86_64-linux-gnu.so(+0x78577a) [0x7fc115dbc77a]
/home/dev/miniconda/envs/py38/lib/python3.8/site-packages/taichi/_lib/core/taichi_python.cpython-38-x86_64-linux-gnu.so(+0x732fd0) [0x7fc115d69fd0]
/home/dev/miniconda/envs/py38/bin/python3(+0x13c00e) [0x557304eba00e]
/home/dev/miniconda/envs/py38/bin/python3: _PyObject_MakeTpCall
/home/dev/miniconda/envs/py38/bin/python3(+0x166cba) [0x557304ee4cba]
/home/dev/miniconda/envs/py38/bin/python3: PyObject_Call
/home/dev/miniconda/envs/py38/bin/python3(+0x194de6) [0x557304f12de6]
/home/dev/miniconda/envs/py38/bin/python3: _PyObject_MakeTpCall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: PyObject_Call
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: PyObject_Call
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyObject_FastCallDict
/home/dev/miniconda/envs/py38/bin/python3(+0x194d2b) [0x557304f12d2b]
/home/dev/miniconda/envs/py38/bin/python3: PyObject_Call
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: PyObject_Call
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: PyObject_Call
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3(+0x166b2e) [0x557304ee4b2e]
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyObject_FastCallDict
/home/dev/miniconda/envs/py38/bin/python3(+0x194d2b) [0x557304f12d2b]
/home/dev/miniconda/envs/py38/bin/python3: _PyObject_MakeTpCall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: PyObject_Call
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3(+0x166b2e) [0x557304ee4b2e]
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyObject_FastCallDict
/home/dev/miniconda/envs/py38/bin/python3(+0x194d2b) [0x557304f12d2b]
/home/dev/miniconda/envs/py38/bin/python3: PyObject_Call
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3(+0x166b2e) [0x557304ee4b2e]
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: PyObject_Call
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: PyObject_Call
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3(+0x166b2e) [0x557304ee4b2e]
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyObject_FastCallDict
/home/dev/miniconda/envs/py38/bin/python3(+0x194d2b) [0x557304f12d2b]
/home/dev/miniconda/envs/py38/bin/python3: _PyObject_MakeTpCall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3(+0x166bf8) [0x557304ee4bf8]
/home/dev/miniconda/envs/py38/bin/python3: PyObject_Call
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3(+0x166b2e) [0x557304ee4b2e]
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyObject_FastCallDict
/home/dev/miniconda/envs/py38/bin/python3(+0x194d2b) [0x557304f12d2b]
/home/dev/miniconda/envs/py38/bin/python3: _PyObject_MakeTpCall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: PyObject_Call
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3(+0x166b2e) [0x557304ee4b2e]
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyObject_FastCallDict
/home/dev/miniconda/envs/py38/bin/python3(+0x194d2b) [0x557304f12d2b]
/home/dev/miniconda/envs/py38/bin/python3: _PyObject_MakeTpCall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3(+0x1f97d9) [0x557304f777d9]
/home/dev/miniconda/envs/py38/bin/python3(+0x13b23d) [0x557304eb923d]
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3(+0x166bf8) [0x557304ee4bf8]
/home/dev/miniconda/envs/py38/bin/python3: PyObject_Call
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: _PyFunction_Vectorcall
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: PyEval_EvalCode
/home/dev/miniconda/envs/py38/bin/python3(+0x242622) [0x557304fc0622]
/home/dev/miniconda/envs/py38/bin/python3(+0x2531d2) [0x557304fd11d2]
/home/dev/miniconda/envs/py38/bin/python3: PyRun_StringFlags
/home/dev/miniconda/envs/py38/bin/python3(+0x1f9a05) [0x557304f77a05]
/home/dev/miniconda/envs/py38/bin/python3(+0x13b23d) [0x557304eb923d]
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalFrameDefault
/home/dev/miniconda/envs/py38/bin/python3: _PyEval_EvalCodeWithName
/home/dev/miniconda/envs/py38/bin/python3: PyEval_EvalCode
/home/dev/miniconda/envs/py38/bin/python3(+0x242622) [0x557304fc0622]
/home/dev/miniconda/envs/py38/bin/python3(+0x2531d2) [0x557304fd11d2]
/home/dev/miniconda/envs/py38/bin/python3: PyRun_StringFlags
/home/dev/miniconda/envs/py38/bin/python3: PyRun_SimpleStringFlags
/home/dev/miniconda/envs/py38/bin/python3: Py_RunMain
/home/dev/miniconda/envs/py38/bin/python3: Py_BytesMain
/lib/x86_64-linux-gnu/libc.so.6: __libc_start_main
/home/dev/miniconda/envs/py38/bin/python3(+0x1f9ad7) [0x557304f77ad7]

Internal error occurred. Check out this page for possible solutions:
https://docs.taichi-lang.org/docs/install
@feisuzhu feisuzhu added the potential bug Something that looks like a bug but not yet confirmed label Nov 14, 2022
@taichi-gardener taichi-gardener moved this to Untriaged in Taichi Lang Nov 14, 2022
@ailzhang ailzhang added flaky-test and removed potential bug Something that looks like a bug but not yet confirmed labels Nov 16, 2022
@ailzhang
Copy link
Contributor

cc: @PGZXB
Let's track the progress of fixing this flaky test here.
Here's another example: https://github.com/taichi-dev/taichi/actions/runs/3472951989/jobs/5804424771

@PGZXB PGZXB mentioned this issue Nov 16, 2022
Closed
@PENGUINLIONG PENGUINLIONG moved this from Untriaged to Todo in Taichi Lang Nov 18, 2022
@PENGUINLIONG PENGUINLIONG moved this from Todo to In Progress in Taichi Lang Nov 18, 2022
@PGZXB PGZXB mentioned this issue Nov 22, 2022
Closed
@lin-hitonami lin-hitonami mentioned this issue Nov 24, 2022
Merged
@PGZXB
Copy link
Contributor

PGZXB commented Nov 25, 2022

@lin-hitonami's #6723 has fixed the bug. Thanks!

@PGZXB PGZXB closed this as completed Nov 25, 2022
Repository owner moved this from In Progress to Done in Taichi Lang Nov 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@PGZXB PGZXB

Labels
flaky-test
Projects
Taichi Lang
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[bug] Fix lookup symbol from cuda module failed PGZXB/taichi
3 participants
@feisuzhu @ailzhang @PGZXB