Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove using splitkv kernel from fmha fwd training path #49

Merged
merged 1 commit into from
Jan 13, 2025

Conversation

qianfengz
Copy link
Collaborator

This PR removes using the fmha fwd splitkv kernel for fmha-fwd training, since we found that there were several test_backward cases with which the backward result dKey failed the checking. which probably was caused by the inaccuracy or error in the lse output of the splitkv kernel.

The following files will not be used after the updating, but they are kept at present:

 xformers/csrc/attention/hip_fmha/ck_tiled_fmha_batched_forward_splitkv_dispatch.h
 xformers/csrc/attention/hip_fmha/ck_tiled_fmha_batched_forward_splitkv_smallq_dispatch.h
 xformers/csrc/attention/hip_fmha/ck_tiled_fmha_grouped_forward_splitkv_dispatch.h
 xformers/csrc/attention/hip_fmha/ck_tiled_fmha_grouped_forward_splitkv_smallq_dispatch.h

@qianfengz qianfengz merged commit 2f66b19 into develop Jan 13, 2025
1 of 5 checks passed
@qianfengz qianfengz deleted the hack_test_backward branch January 13, 2025 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant