Why didn't we use random scaffold split?

[lvkd84]

Hi,
Thank you for publishing such an amazing paper!
I have a question regarding the splitting of the datasets used in the experiments. As shown in the paper and in the code, you used scaffold splitting. However, it is also reasonable to use random scaffold splitting on the small datasets reported. Is there a intuitions for preferring scaffold splitting over random scaffold splitting?

[linfeng-du]

Also having doubt on this. I would also like to know if the results in paper are averaged among determinsitc scaffold splits with different model seeds or randomized scaffold splits with different splitting seeds.

I think a possible explationation is that scaffold splitting generates out-of-distribution train/val/test sets since molecules in them contains completely different backbond structures (scaffolds), so it might be diffcult to tune a model that generalizes well to a number of OOD scenarios (i.e., different splitting seeds using randomized_scaffold_split)