2nd Order or 1st Order Approximation?

[Vampire-Vx]

Is this implementation a 1st order Approximation version of Maml ?
In meta.py, when you do autograd.grad, you do not specify create_graph = True, which means that the gradient operation would not be included in the computation graph.

Thus, although the design here is trying to calculate the 2nd order derivatives, the grad is not included, so only 1st order approximation.

[yinxiaojian]

I think this implementation is only for first order version of MAML.
If for second version, need to set retain_graph = True, create_graph=True when calculating "torch.autograd.grad"

[MrDavidG]

@Vampire-Vx @yinxiaojian
I have also tried to set retain_graph = True, create_graph=True, but for mini-Imagenet, the performance is weaker than before.
Besides, the hidden dimensions I used for mini-imagent is [32,32,32,32] rather than 64, which is the same with the setting in the original MAML paper