How to load the torch models from internet if there is bad network connection?

[elenacliu]

Thank you for your great work!

When trying to extract dino features, we need to load model from github through the code:

lerf/lerf/data/utils/dino_extractor.py

Lines 67 to 82 in 3b2cb90

	if 'dino' in model_type:
	model = torch.hub.load('facebookresearch/dino:main', model_type)
	else: # model from timm -- load weights from timm to dino model (enables working on arbitrary size images).
	temp_model = timm.create_model(model_type, pretrained=True)
	model_type_dict = {
	'vit_small_patch16_224': 'dino_vits16',
	'vit_small_patch8_224': 'dino_vits8',
	'vit_base_patch16_224': 'dino_vitb16',
	'vit_base_patch8_224': 'dino_vitb8'
	}
	model = torch.hub.load('facebookresearch/dino:main', model_type_dict[model_type])
	temp_state_dict = temp_model.state_dict()
	del temp_state_dict['head.weight']
	del temp_state_dict['head.bias']
	model.load_state_dict(temp_state_dict)
	return model

but I cannot find a way to do that without available network. Do you have any alternative methods?

[elenacliu]

I have tried to download the corresponding weights from github to my local machine, and scp it to the server and modify your code to:

from torchvision.models import vit_b_16

if 'dino' in model_type:
            model_path = '/path/to/ckpt/dino_vitbase16_pretrain_full_checkpoint.pth'
            if os.path.exists(model_path):
                 model = vit_b_16(pretrained=True)
                 state_dict = torch.load(model_path)
                 model.load_state_dict(state_dict)
                 model.eval()
            else: 
                 model = torch.hub.load('facebookresearch/dino:main', model_type)

But it seems that the backbone cannot match with the weights, with some keys missing.

[elenacliu]

I also tried to use torch.hub.load(source='local'):

model_dir = '~/.cache/torch/checkpoints'  # I have a hubconf.py and the weights dino_vitbase16_pretrain_full_checkpoint.pth under the directory
model_name = 'vit_b_16'
model = torch.hub.load(model_dir, model=model_name, source='local')

which also encountered into error in dino_extractor.py:

lerf/lerf/data/utils/dino_extractor.py

Line 128 in 3b2cb90

patch_size = model.patch_embed.patch_size

AttributeError: 'VisionTransformer' object has no attribute 'patch_embed'

[kxwangzju]

Hi, I am having the same problem. Have you fixed that?

[elenacliu]

Sorry about that, but I haven't fixed that.