About the Kronecker Decomposition

[BerenChou]

I did not understand this green-underlined equation. With the shapes you noted, it does not even satisfy the matrix multiplication.

[JeremieMelo]

A and B are sqrt(C) x sqrt(C) matrices. the multiplication is on the last two dimensions of \tilde{x}. The "HW" is the batch dimension, where the matmul will be broadcasted. Details can be found in class DES at line 85 of https://github.com/facebookresearch/HRViT/blob/main/models/hrvit.py.