Benchmark scripts for training time performance

[HanYuanyuaner]

❓ Questions & Help

Hi,
I run the gcn.py in /example folder, and change dataset name to "PubMed". In the website the training time about this dataset with gcn.py is 2.0s. But, in my server it only need about 0.7+s. The training time of gat.py is about 3s not 12s. My GPU is GTX 1080 Ti, 200 epochs. So do you know the reason?

[rusty1s]

Yes, you are right! Training speed performance increased once more with this PR. However, training speed may still vary for same GPUs for different PyTorch or CUDA versions. I will try to keep the performance table as up to date as possible and provide training time evaluation scripts for verification.

[HanYuanyuaner]

Yes, you are right! Training speed performance increased once more with this PR. However, training speed may still vary for same GPUs for different PyTorch or CUDA versions. I will try to keep the performance table as up to date as possible and provide training time evaluation scripts for verification.

Thank you for your respond. I checked my code in message_passing.py. The code is still old and torch embedding is not used. And my environment is ubuntu 16.04, torch 1.0, cuda 9.0 and cudnn7.0. For dataset Cora and Citeseer the experiment results are similar with yours. Only PubMed is different. I only modify dataset name in gcn.py/gat.py. Do I need to modify any other code?

[rusty1s]

I added a small script in benchmark/runtime to check current running times of model-dataset pairs. Actually, PubMed is now similar in speed to Cora and CiteSeer. I wonder what caused the delay back then. I will update running times ASAP.

[HanYuanyuaner]

I added a small script in benchmark/runtime to check current running times of model-dataset pairs. Actually, PubMed is now similar in speed to Cora and CiteSeer. I wonder what caused the delay back then. I will update running times ASAP.

Thank you for your respond. Another question is about distributed computation. In example the dataset is small, for real case the dataset may be large for one worker, so how to separate graph to sub-graph may be a problem, do you have any suggestion? Or dose PyG have the potential to support distributed computation?

[rusty1s]

Hi, you can always use more workers if this proves to be beneficial. We support distributed training via torch.distributed or nn.DataParallel. What do you mean with "so how to separate graph to sub-graph may be a problem"?

[HanYuanyuaner]

Hi, you can always use more workers if this proves to be beneficial. We support distributed training via torch.distributed or nn.DataParallel. What do you mean with "so how to separate graph to sub-graph may be a problem"?

I mean if the graph is too large for one GPU to store, how to store it?

[rusty1s]

Sadly, we currently do not support giant graph processing. Giant graphs are usually processed via sampling techniques. This is a rather difficult but important feature for PyG and it is definitively on my ToDo list. I will close this request in favour of #64.