Query Regarding SUN397 Dataset

[SHIBOYA]

I hope this email finds you well. My name is [Your Name], and I am currently working on [Your Project/Research Topic]. I am very interested in using [Your Model/Software Name] for my work.

While attempting to fine-tune the model on the SUN397 dataset, I encountered the following error: AssertionError: Unsupported dataset: SUN397. Supported datasets: ['CIFAR10', 'CIFAR100', 'Cars', 'DTD', 'EuroSAT', 'EuroSATVal', 'FashionMNIST', 'GTSRB', 'ImageNet', 'KITTI', 'KITTIVal', 'MNIST', 'MTSD', 'RESISC45', 'STL10', 'SVHN']

I was wondering if there is a way to resolve this issue or if you have plans to include SUN397 in the list of supported datasets in the future.

Any guidance or advice on this matter would be greatly appreciated.

Thank you for your time and consideration. I am looking forward to your response.

[SHIBOYA]

Additionally, could you please guide me on where to change the dataset path in the code? Knowing this would greatly assist me in setting up the project environment.

[SHIBOYA]

DTD Dataset Split
When working with the DTD dataset, I noticed that the original dataset does not contain pre-defined train and test splits. Could you please elaborate on how you managed the splitting? Additionally, would it be possible to provide a dataset with the splits you used or share the trained model parameters?

Any guidance or advice on these matters would be greatly appreciated.

Thank you for your time and consideration. I am looking forward to your response.

[gabrielilharco]

Hi @SHIBOYA. Regarding SUN397, I just pushed an update that should fix your issue. Please let me know if it works.

could you please guide me on where to change the dataset path in the code?

you can use the --data-location flag.

DTD Dataset Split

see mlfoundations/task_vectors#1

[SHIBOYA]

when I try to patch on SUN397:

--train-dataset=SUN397  \
--epochs=5  \
--lr=0.00001  \
--batch-size=128  \
--model=ViT-B/16  \
--eval-datasets=ImageNet  \
--results-db=results.jsonl  \
--save=models/patch/ViTL14  \
--data-location=~/data \
--alpha 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

an error message appears: This dataset is not supported.

How can this be resolved?

[gabrielilharco]

Did you pull from the latest version?

[SHIBOYA]

I have just noticed that the updated codebase you provided has successfully addressed my concerns, and I am truly thankful for your support. I am writing to express my sincere gratitude for your assistance and prompt response to my queries.

Thank you once again for your invaluable support！

[gabrielilharco]

Our pleasure! Closing for now, but feel free to re-open if you have any other questions or concerns.

[SHIBOYA]

Hi, I am looking to understand the correct commands to run the model on different datasets and whether the learning rates or other hyperparameters need to be adjusted for different base models.

Commands for Different Datasets:
The example provided in the GitHub repository is as follows:
python src/patch.py
--train-dataset=MNIST
--epochs=5
--lr=0.00001
--batch-size=128
--model=ViT-L/14
--eval-datasets=ImageNet,MNIST
--results-db=results.jsonl
--save=models/patch/ViTL14
--data-location=~/data
--alpha 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Could you please provide guidance or examples on how to modify this command for other supported datasets? Additionally, I am curious to know whether the learning rate or other hyperparameters should be adjusted when using different base models, such as ViT-L/14 and ViT-B/16, and if so, what would be the recommended learning rates for these models?

Any additional details on specific parameters or configurations for different datasets and models would be greatly appreciated.

Thank you very much for your time and assistance. I am looking forward to your response.

[gabrielilharco]

We learning rate 1e-5 for all models and fine-tune for roughly 2000 iterations for all datasets. To change the training dataset, you need to change the --train-dataset flag.