Minor bug in ALTI's code

[gegallego]

Hi,

We've recently found a minor bug in ALTI's code. It's in this line:

stopes/stopes/eval/alti/wrappers/transformer_wrapper.py

Line 474 in 0c6535a

ln_std_coef = 1 / (out_q_pre_ln_th + eps_ln).std(-1).view(

The parameter unbiased from torch.std should be set to False, since the default is True. The expected behavior of this is without the Bessel's correction.

Relevant discussion related with this: pytorch/pytorch#50010

[avidale]

Hi @gegallego,
thank you for this remark!
Yes, indeed, the LayerNorm implementation that is used here computes the variance without the Bessel's correction (https://pytorch.org/docs/stable/generated/torch.nn.LayerNorm.html#torch.nn.LayerNorm), so should we.

I'll update the code.