Why not use a bigger meta training batch?

[rshaojimmy]

Hi @liyiying. Thanks for your implementation!

I have a question that:
This implementation feeds batches of each meta train dataset into feature_extractor_network and sums up the losses (meta_train_loss_dg += loss_dg). Then it conducts the backpropagation (meta_train_loss_dg.backward(create_graph=True)). It involves many forward propagations in feature_extractor_network.

May I ask why not firstly combine the batches in all meta train datasets together and just do one forward propagation in feature_extractor_network? What are the differences between these two ways? Thanks.

[liyiying]

Sorry for the delayed notice of this message. I don't know if I understand your question in a correct way. You mean you want to compute the loss-main of all batches data in just one time? But in our setting, in each episode, the feature network, classifier network, and critic network all update. Then in the next episode, the loss-main computed based on the updated parameters of the networks, so you need to compute the loss for the mini-batch in the current episode. Moreover, I think maybe your understanding for the 'create_graph=True' is not very exact from your question.