maml_omniglot model with functorch #1179
Conversation
zou3519
force-pushed
the
functorch_bench
branch
from
ce02542 to
ec13aeb
Compare
Very similar to the maml_omniglot model in torchbench (which is what we originally based this model off of). We also load some sample inputs there because the model takes the same inputs. Test Plan: - python test.py -k TestBenchmark.test_functorch_maml_omniglot_train_cuda
zou3519
force-pushed
the
functorch_bench
branch
from
ec13aeb to
c7c06fb
Compare
|
|
||
| class Model(BenchmarkModel): | ||
| task = OTHER.OTHER_TASKS | ||
| DEFAULT_TRAIN_BSIZE = 1 |
There was a problem hiding this comment.
Is this the batch size used in the original training code? I am working on another version of maml/protonet originated from fewshot(https://github.com/pytorch/benchmark/pull/1184/files#diff-9c209c92cea7ccad363e7d477e210a3120028678829b11086cd9772af97181eaR9) and it looks like the batch size should be larger than this.
There was a problem hiding this comment.
I copied this from
benchmark/torchbenchmark/models/maml_omniglot/__init__.py
Lines 33 to 34 in 8045bb7
"batch_size" is a term that doesn't apply to the maml model in the traditional sense. The original maml omniglot (https://arxiv.org/pdf/1703.03400.pdf) was trained under 32 tasks (also referred to as meta batch-size) and done using both 1-shot and 5-shot.
The code provided in this PR does 32 tasks and 5-shot. I can change DEFAULT_TRAIN_BSIZE to 32 instead and add a comment if that sounds reasonable to you? (The first dimension of the inputs is the task dimension and has size 32)
There was a problem hiding this comment.
I see, maml_omniglot and maml models were marked as "incorrect" in #328, so we created #1184 trying to use an actual correct implementation, but I haven't go time to get it work. I am okay with landing this PR for now, but I think we should investigate if #1184 is a better implementation, then rebase all maml models to fewshot.
There was a problem hiding this comment.
I will merge this for now. Happy to rebase this model if we do want to replace it with the one in the fewshot repo.
|
@zou3519 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Very similar to the maml_omniglot model in torchbench (which is what we originally based this model off of). We also load some sample inputs from there because the model takes the same inputs.
Test Plan: