README.md

Zero-shot Composed Text-Image Retrieval

This repository contains the official Pytorch implementation of TransAgg: https://arxiv.org/abs/2306.07272

Environment

Create the environment for running our code as follow:

conda create --name transagg python=3.9.16
conda activate transagg
pip install -r requirements.txt

Datasets

Laion-CIR-Template、Laion-CIR-LLM and Laion-CIR-Combined: please refer to this link

FashionIQ: Please refer to the FashionIQ repo to get the datasets.

CIRR: Please refer to the CIRR repo for instructions.

Model Zoo

Pretrained Model

clip-Vit-B/32: please refer to this link

clip-Vit-L/14: please refer to this link

blip: please refer to this link

Checkpoints

https://drive.google.com/drive/folders/1EGpylkOMj9tduUjAhTLtaX5UqjPMyN3X?usp=sharing

Train

note that, you can modify the relevant parameters in the config.py file

CUDA_VISIBLE_DEVICES=0 python main.py

Test CIRR Dataset

note that, you can modify the relevant parameters in the config.py file

python cirr_test_submission.py

Citation

if you use this code for your research or project, please cite:

@article{liu2023zeroshot,
  title={Zero-shot Composed Text-Image Retrieval}, 
  author={Yikun Liu and Jiangchao Yao and Ya Zhang and Yanfeng Wang and Weidi Xie},
  year={2023},
  journal={arXiv preprint arXiv:2306.07272},
}

Star History

Acknowledgements

Many thanks to the code bases from CLIP4CIR, CLIP, BLIP