Transformer Softmax Bottleneck An analysis of the effects of Mixture of Softmaxes on the Transformer architecture. Results
