Ops unit tests fail on ROCm #920
Comments
|
cc AMD folks @fsx950223 @asroy |
khabinov
pushed a commit
to khabinov/AITemplate-1
that referenced
this issue
Oct 3, 2023
Summary: Pull Request resolved: facebookincubator#662 A few issues in the CUTLASS codebase blocking the integration of the CUTLASS 3.x SM90 kernels in AITemplate have been fixed upstream (see, e.g., the merged PRs [facebookincubator#920](NVIDIA/cutlass#920) and [facebookincubator#927](NVIDIA/cutlass#927)). The CUTLASS version is synced with the upstream to proceed with the SM90 integration. Reviewed By: chenyang78 Differential Revision: D45603657 fbshipit-source-id: 6b64f6ee0b9f87c2f379144d0fa568487aef8076
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello,
I installed AIT on AMD MI-250 with ROCm 5.4 from source.
when I run:
./tests/unittest/ops# python test_groupnorm.py
I got the following errors:
ERROR: test_groupnorm_float16 (main.GroupnormTestCase)
Traceback (most recent call last):
File "test_groupnorm.py", line 122, in test_groupnorm_float16
self._test_groupnorm()
File "test_groupnorm.py", line 84, in _test_groupnorm
module = compile_model(X4, target, "./tmp", op_name, dll_name=dll_name)
File "/usr/local/lib/python3.8/dist-packages/aitemplate/utils/misc.py", line 93, in inner_function
return f(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/aitemplate/compiler/compiler.py", line 276, in compile_model
compiler.transform.profile(
File "/usr/local/lib/python3.8/dist-packages/aitemplate/compiler/transform/profile.py", line 106, in profile
f.profile(
File "/usr/local/lib/python3.8/dist-packages/aitemplate/compiler/ops/groupnorm/groupnorm.py", line 389, in profile
best_algo, workspace = self._profile_single_workload(
File "/usr/local/lib/python3.8/dist-packages/aitemplate/compiler/ops/groupnorm/groupnorm.py", line 320, in _profile_single_workload
command = self._gen_profile_cmd(profiler_prefix, cfg, x_shape_dict)
File "/usr/local/lib/python3.8/dist-packages/aitemplate/compiler/ops/groupnorm/groupnorm.py", line 278, in _gen_profile_cmd
raise RuntimeError("Profiler %s is not executable" % exe_path)
RuntimeError: Profiler ./tmp/profiler/groupnorm/groupnorm_rank5_256_8_32_1_8_1_1_1_1_1_1_1 is not executable
======================================================================
ERROR: test_groupnorm_swish (main.GroupnormTestCase)
Traceback (most recent call last):
File "test_groupnorm.py", line 132, in test_groupnorm_swish
self._test_groupnorm(use_swish=True)
File "test_groupnorm.py", line 84, in _test_groupnorm
module = compile_model(X4, target, "./tmp", op_name, dll_name=dll_name)
File "/usr/local/lib/python3.8/dist-packages/aitemplate/utils/misc.py", line 93, in inner_function
return f(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/aitemplate/compiler/compiler.py", line 276, in compile_model
compiler.transform.profile(
File "/usr/local/lib/python3.8/dist-packages/aitemplate/compiler/transform/profile.py", line 106, in profile
f.profile(
File "/usr/local/lib/python3.8/dist-packages/aitemplate/compiler/ops/groupnorm/groupnorm.py", line 389, in profile
best_algo, workspace = self._profile_single_workload(
File "/usr/local/lib/python3.8/dist-packages/aitemplate/compiler/ops/groupnorm/groupnorm.py", line 320, in _profile_single_workload
command = self._gen_profile_cmd(profiler_prefix, cfg, x_shape_dict)
File "/usr/local/lib/python3.8/dist-packages/aitemplate/compiler/ops/groupnorm/groupnorm.py", line 278, in _gen_profile_cmd
raise RuntimeError("Profiler %s is not executable" % exe_path)
RuntimeError: Profiler ./tmp/profiler/groupnorm_swish/groupnorm_rank5_256_8_32_1_8_1_1_1_1_1_1_1 is not executable
Does anyone know how to fix it?
Thanks.
Du
The text was updated successfully, but these errors were encountered: