Paper: Compositional Image Retrieval via Instruction-Aware Contrastive Learning
You will need to create a conda environment (Python 3.10) and install the following dependencies to run the InstructCIR codebase:
pip install torch==2.1.2, transformers==4.42.0, datasets==2.20.0, peft==0.10.0, bitsandbytes==0.43.1, accelerate==0.30.1, torchvision==0.16.2, numpy==1.26.4, scikit-learn==1.2.2, requests==2.32.3, timm==0.9.16, einops==0.6.1, einops-exts==0.0.4, deepspeed==0.14.4, ninja==1.11.1.1, wandb==0.16, tqdm==4.66.5, scipy==1.13.0, fire==0.6.0, flash-attn==2.5.7, ipykernel==6.29.3, ipython==8.22.2, Jinja2==3.1.3, matplotlib==3.8.4, openai==1.30.3, opencv-python==4.9.0.80, pandas==2.2.2, tensorboard==2.17.1, tensorboardX==2.6.2.2
If you have trouble installing these environments, feel free to open an issue. I will address it ASAP.
We provide trained checkpoints that can be directly used for evaluation.
Please follow this site to prepare datasets CIRR, CIRCO, and FashionIQ. Follow this site to download GeneCIS.
python generate_test_submission.py \
--submission-name cirr_results \
--dataset cirr \
--dataset-path path_to_downloaded_cirr \
--model_name_or_path uta-smile/instructcir_llava_phi35_clip224_lp
python generate_test_submission.py \
--submission-name circo_results \
--dataset circo \
--dataset-path path_to_downloaded_circo \
--model_name_or_path uta-smile/instructcir_llava_phi35_clip224_lp
Upon running commands above, results will be saved under ./submission/ as json files. Use this website for CIRR and this website for CIRCO to submit and obtain evaluation results. This work is involved in another reserach and results may be slightly different from reproted results in the paper due to hyperparameter changes.
Ongoing...
Ongoing...
Ongoing...
Ongoing...