ValueError: Optimizer attributes must have a 'type' key specifying the type of optimizer. (Custom or PyTorch)

[AjinkyaP1998]

❓ Questions and Help

After running the following code to train an existing model :

mmf_run config=projects/hateful_memes/configs/mmbt/defaults.yaml model=mmbt dataset=hateful_memes training.log_interval=50 training.max_updates=3000 training.batch_size=16 training.evaluation_interval=500

I get the following error :

ValueError: Optimizer attributes must have a 'type' key specifying the type of optimizer. (Custom or PyTorch)

What's going wrong? I am referring to this colab example and trying to execute the code on my machine's cmd.

[vedanuj]

Can you provide your environment details?

[AjinkyaP1998]

Environment details: @vedanuj

PyTorch version: 1.5.0+cu101
Is debug build: No
CUDA used to build PyTorch: 10.1

OS: Microsoft Windows 10 Home Single Language
GCC version: Could not collect
CMake version: version 3.18.0

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: GeForce GTX 1050
Nvidia driver version: 419.72
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.19.1
[pip3] torch==1.5.0+cu101
[pip3] torchtext==0.5.0
[pip3] torchvision==0.6.0+cu101
[conda] Could not collect

[vedanuj]

We observed this problem earlier with Windows as well. Can you install mmf from source and try again?

[AjinkyaP1998]

I did ... I still get the same error: Here is the complete output: @vedanuj

`c:\users\acer\appdata\local\programs\python\python36\lib\site-packages\omegaconf\dictconfig.py:252: UserWarning: Keys with dot (../../../mmbt/configs/hateful_memes/defaults.yaml) are deprecated and will have different semantic meaning the next major version of OmegaConf (2.1)
See the compact keys issue for more details: omry/omegaconf#152
You can disable this warning by setting the environment variable OC_DISABLE_DOT_ACCESS_WARNING=1
warnings.warn(message=msg, category=UserWarning)
2020-08-25T00:18:01 | mmf.utils.configuration: Overriding option config to projects/hateful_memes/configs/mmbt/defaults.yaml
2020-08-25T00:18:01 | mmf.utils.configuration: Overriding option model to mmbt
2020-08-25T00:18:01 | mmf.utils.configuration: Overriding option datasets to hateful_memes
2020-08-25T00:18:01 | mmf.utils.configuration: Overriding option training.log_interval to 50
2020-08-25T00:18:01 | mmf.utils.configuration: Overriding option training.max_updates to 3000
2020-08-25T00:18:01 | mmf.utils.configuration: Overriding option training.batch_size to 16
2020-08-25T00:18:01 | mmf.utils.configuration: Overriding option training.evaluation_interval to 500
2020-08-25T00:18:01 | mmf: Logging to: ./save\train.log
2020-08-25T00:18:01 | mmf_cli.run: Namespace(config_override=None, local_rank=None, opts=['config=projects/hateful_memes/configs/mmbt/defaults.yaml', 'model=mmbt', 'dataset=hateful_memes', 'training.log_interval=50', 'training.max_updates=3000', 'training.batch_size=16', 'training.evaluation_interval=500'])
2020-08-25T00:18:01 | mmf_cli.run: Torch version: 1.5.0+cu101
2020-08-25T00:18:01 | mmf.utils.general: CUDA Device 0 is: GeForce GTX 1050
2020-08-25T00:18:01 | mmf_cli.run: Using seed 1325186
2020-08-25T00:18:01 | mmf.trainers.mmf_trainer: Loading datasets
2020-08-25T00:18:03 | torchtext.vocab: Loading vectors from C:\Users\Acer/.cache\torch\mmf\glove.6B.300d.txt.pt
2020-08-25T00:18:05 | torchtext.vocab: Loading vectors from C:\Users\Acer/.cache\torch\mmf\glove.6B.300d.txt.pt
2020-08-25T00:18:09 | mmf.trainers.mmf_trainer: Loading model
2020-08-25T00:18:12 | transformers.configuration_utils: loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-config.json from cache at C:\Users\Acer.cache\torch\transformers\4dad0251492946e18ac39290fcfe91b89d370fee250efe9521476438fe8ca185.7156163d5fdc189c3016baca0775ffce230789d7fa2a42ef516483e4ca884517
2020-08-25T00:18:12 | transformers.configuration_utils: Model config {
"architectures": [
"BertForMaskedLM"
],
"attention_probs_dropout_prob": 0.1,
"finetuning_task": null,
"hidden_act": "gelu",
"hidden_dropout_prob": 0.1,
"hidden_size": 768,
"id2label": {
"0": "LABEL_0",
"1": "LABEL_1"
},
"initializer_range": 0.02,
"intermediate_size": 3072,
"is_decoder": false,
"label2id": {
"LABEL_0": 0,
"LABEL_1": 1
},
"layer_norm_eps": 1e-12,
"max_position_embeddings": 512,
"model_type": "bert",
"num_attention_heads": 12,
"num_hidden_layers": 12,
"num_labels": 2,
"output_attentions": false,
"output_hidden_states": false,
"output_past": true,
"pad_token_id": 0,
"pruned_heads": {},
"torchscript": false,
"type_vocab_size": 2,
"use_bfloat16": false,
"vocab_size": 30522
}

2020-08-25T00:18:13 | transformers.modeling_utils: loading weights file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-pytorch_model.bin from cache at C:\Users\Acer/.cache\torch\mmf\distributed_-1\aa1ef1aede4482d0dbcd4d52baad8ae300e60902e88fcb0bebdec09afd232066.36ca03ab34a1a5d5fa7bc3d03d55c4fa650fed07220e2eeebc06ce58d0e9a157
2020-08-25T00:18:21 | transformers.modeling_utils: loading weights file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-pytorch_model.bin from cache at C:\Users\Acer/.cache\torch\mmf\distributed_-1\aa1ef1aede4482d0dbcd4d52baad8ae300e60902e88fcb0bebdec09afd232066.36ca03ab34a1a5d5fa7bc3d03d55c4fa650fed07220e2eeebc06ce58d0e9a157
WARNING 2020-08-25T00:18:25 | py.warnings: c:\users\acer\mmf\mmf\models\base_model.py:96: UserWarning: No losses are defined in model configuration. You are expected to return loss in your return dict from forward.
"No losses are defined in model configuration. You are expected "

WARNING 2020-08-25T00:18:25 | py.warnings: c:\users\acer\mmf\mmf\models\base_model.py:96: UserWarning: No losses are defined in model configuration. You are expected to return loss in your return dict from forward.
"No losses are defined in model configuration. You are expected "

2020-08-25T00:18:30 | mmf.trainers.mmf_trainer: Loading optimizer
Traceback (most recent call last):
File "C:\Users\Acer\AppData\Local\Programs\Python\Python36\Scripts\mmf_run-script.py", line 33, in
sys.exit(load_entry_point('mmf', 'console_scripts', 'mmf_run')())
File "c:\users\acer\mmf\mmf_cli\run.py", line 122, in run
main(configuration, predict=predict)
File "c:\users\acer\mmf\mmf_cli\run.py", line 52, in main
trainer.load()
File "c:\users\acer\mmf\mmf\trainers\mmf_trainer.py", line 41, in load
super().load()
File "c:\users\acer\mmf\mmf\trainers\base_trainer.py", line 31, in load
self.load_optimizer()
File "c:\users\acer\mmf\mmf\trainers\mmf_trainer.py", line 90, in load_optimizer
self.optimizer = build_optimizer(self.model, self.config)
File "c:\users\acer\mmf\mmf\utils\build.py", line 198, in build_optimizer
"Optimizer attributes must have a 'type' key "
ValueError: Optimizer attributes must have a 'type' key specifying the type of optimizer. (Custom or PyTorch)`

[vedanuj]

Thanks. This might be similar to #295 . Can you try the solution provided there?

git clone --config core.symlinks=true https://github.com/facebookresearch/mmf.git

[AjinkyaP1998]

Hey @vedanuj , I followed the steps given in #295 and also executed the command :

git clone --config core.symlinks=true https://github.com/facebookresearch/mmf.git

and got the same error as mentioned in #295, which is :

Cloning into 'mmf'... remote: Enumerating objects: 112, done. remote: Counting objects: 100% (112/112), done. remote: Compressing objects: 100% (86/86), done. remote: Total 14450 (delta 35), reused 65 (delta 25), pack-reused 14338 Receiving objects: 100% (14450/14450), 11.88 MiB | 1.75 MiB/s, done. Resolving deltas: 100% (9146/9146), done. error: unable to create symlink mmf/projects: Permission denied error: unable to create symlink projects/hateful_memes/configs/concat_bert/defaults.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/concat_bow/defaults.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/late_fusion/defaults.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/mmbt/defaults.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/mmbt/with_features.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/unimodal/bert.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/unimodal/image.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/unimodal/text.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/unimodal/with_features.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/vilbert/defaults.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/vilbert/direct.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/vilbert/from_cc.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/visual_bert/defaults.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/visual_bert/direct.yaml: Permission denied error: unable to create symlink projects/hateful_memes/configs/visual_bert/from_coco.yaml: Permission denied error: unable to create symlink projects/m4c_captioner/configs/butd/textcaps: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/vilbert/masked_coco/defaults.yaml: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/vilbert/masked_coco/full.yaml: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/vilbert/masked_conceptual_captions/defaults.yaml: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/vilbert/masked_conceptual_captions/full.yaml: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/vilbert/masked_vqa2/defaults.yaml: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/vilbert/masked_vqa2/full.yaml: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/visual_bert/masked_coco/defaults.yaml: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/visual_bert/masked_coco/full.yaml: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/visual_bert/masked_coco/full_train_val.yaml: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/visual_bert/masked_conceptual_captions/defaults.yaml: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/visual_bert/masked_conceptual_captions/full.yaml: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/visual_bert/masked_vqa2/defaults.yaml: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/visual_bert/masked_vqa2/full.yaml: Permission denied error: unable to create symlink projects/pretrain_vl_right/configs/visual_bert/masked_vqa2/full_train_val.yaml: Permission denied fatal: unable to checkout working tree warning: Clone succeeded, but checkout failed. You can inspect what was checked out with 'git status' and retry with 'git restore --source=HEAD :/'

The solution to this error given by @apsdehal in #295 is :

`As per alicevision/Meshroom#735, this happens when your google drive is not properly mounted. Can you follow the conversation and steps in that issue to see if that fixes your problem? As with meshroom, this problem is more related to colab rather than mmf.

We will also open up a PR on our end to fix this properly by removing all symlinks.

Also, if possible try using colab's internal storage instead of your Google Drive. For e.g. clone mmf directly in /content/.`

---

But, as I said I am trying to train the model using my local machine's cmd and not on colab...
Is there something else to resolve this issue ?

[apsdehal]

Can you check why you are getting permission denied on local. That seems odd.

[graydove]

I meet a same problem, which is:
mmf_run config=projects/pythia/configs/visual_genome/defaults.yaml datasets=visual_genome model=pythia run_type=train_val
2020-11-30T10:41:27 | mmf.utils.configuration: Overriding option config to projects/pythia/configs/visual_genome/defaults.yaml
2020-11-30T10:41:27 | mmf.utils.configuration: Overriding option datasets to visual_genome
2020-11-30T10:41:27 | mmf.utils.configuration: Overriding option model to pythia
2020-11-30T10:41:27 | mmf.utils.configuration: Overriding option run_type to train_val
2020-11-30T10:41:27 | mmf: Logging to: ./save/train.log
2020-11-30T10:41:27 | mmf_cli.run: Namespace(config_override=None, local_rank=None, opts=['config=projects/pythia/configs/visual_genome/defaults.yaml', 'datasets=visual_genome', 'model=pythia', 'run_type=train_val'])
2020-11-30T10:41:27 | mmf_cli.run: Torch version: 1.6.0
2020-11-30T10:41:27 | mmf.utils.general: CUDA Device 0 is: GeForce GTX 1070 Ti
2020-11-30T10:41:27 | mmf_cli.run: Using seed 27462622
2020-11-30T10:41:27 | mmf.trainers.mmf_trainer: Loading datasets
2020-11-30T10:41:27 | mmf.datasets.builders.visual_genome.builder: Downloading the Visual Genome vocabs now.
[ Downloading: https://dl.fbaipublicfiles.com/pythia/data/vocab.tar.gz to /home/graydove/.cache/torch/mmf/data/vocab.tar.gz ]
Downloading vocab.tar.gz: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 596k/596k [00:01<00:00, 359kB/s]
2020-11-30T10:41:30 | mmf.datasets.builders.visual_genome.builder: Extracting the Visual Genome vocabs now. This may take time
Unpacking vocab.tar.gz
2020-11-30T10:41:30 | mmf.datasets.builders.visual_genome.builder: Visual Genome imdb already present. Skipping download.
2020-11-30T10:41:30 | mmf.datasets.builders.visual_genome.builder: Visual Genome features already present. Skipping download.
2020-11-30T10:41:37 | torchtext.vocab: Loading vectors from /home/graydove/.cache/torch/mmf/glove.6B.300d.txt.pt
2020-11-30T10:41:37 | torchtext.vocab: Loading vectors from /home/graydove/.cache/torch/mmf/glove.6B.300d.txt.pt
2020-11-30T10:41:38 | mmf.trainers.mmf_trainer: Loading model
2020-11-30T10:41:41 | mmf.trainers.mmf_trainer: Loading optimizer
Traceback (most recent call last):
File "/usr/local/bin/mmf_run", line 33, in
sys.exit(load_entry_point('mmf', 'console_scripts', 'mmf_run')())
File "/home/graydove/Project/mmf/mmf_cli/run.py", line 122, in run
main(configuration, predict=predict)
File "/home/graydove/Project/mmf/mmf_cli/run.py", line 52, in main
trainer.load()
File "/home/graydove/Project/mmf/mmf/trainers/mmf_trainer.py", line 42, in load
super().load()
File "/home/graydove/Project/mmf/mmf/trainers/base_trainer.py", line 31, in load
self.load_optimizer()
File "/home/graydove/Project/mmf/mmf/trainers/mmf_trainer.py", line 96, in load_optimizer
self.optimizer = build_optimizer(self.model, self.config)
File "/home/graydove/Project/mmf/mmf/utils/build.py", line 207, in build_optimizer
"Optimizer attributes must have a 'type' key "
ValueError: Optimizer attributes must have a 'type' key specifying the type of optimizer. (Custom or PyTorch)

The framework is installed in my PC, and I'm sure with "core.symlinks=true". emmm...,, so, could you tell me what wrong with it, please？