Data leakage with pl-bolt implementation of v2

[JSGrondin]

First, I want to say thanks for this paper and the code contribution, which I found quite useful and interesting.

While going through the code for the DR part, I noticed some flaws that I thought you may be interested in knowing:

1. Most importantly, the implementation of Moco that you have been using (from pl-bolts 0.3.0) has a data leakage that was subsequently fixed in Remove momentum updating from val step and add separate val queue Lightning-Universe/lightning-bolts#631. The leakage comes from the fact that during your validation step, you are enqueuing validation samples that will contribute to biasing your loss calculation and update, as well as your validation metrics.
2. Less importantly but for thoroughness, I noticed that for your baseline (mocov2 only), you did not use the same optimizer schedule.