How to run the network in parallel on multiple cards

[activate-an]

When I try to increase the size of the output image, it will prompt that there is not enough memory, so how can I run the network on multiple cards in parallel to achieve a larger output?

[fyw1999]

I have run the code in multiple GPUS successfully! You need to wrap models by DataParallel firstly. There are still some changes to be made in the code, beacuse when you use multiple cards, a batch will be separated and distributed on each GPU, after each part of a batch through models, they will be merged to a batch again, which will cause a lot of bugs in code. For example in RNN model, when different part of a batch through the model, the max value in argument cap_lens will be different.

[AudityGhosh]

@fyw1999 , could you please enlighten us with the changes specifically ?