Would support other headdim

[v4if]

Would support other headdim? Like 512.

[jason-huang03]

In the upcoming release, we will include kernels for head dimension of 256. However, models with a head dimension of 256 are already quite rare (only gemma-2 2b/9b as far as I know), and those with 512 are even more uncommon. Could you provide examples of models that use a head dimension of 512? This would give us a stronger incentive to optimize this type of kernel.

[v4if]

In the upcoming release, we will include kernels for head dimension of 256. However, models with a head dimension of 256 are already quite rare (only gemma-2 2b/9b as far as I know), and those with 512 are even more uncommon. Could you provide examples of models that use a head dimension of 512? This would give us a stronger incentive to optimize this type of kernel.

tks. internal model.