Text-Conditioned Outfit Recommendation

This respository contains source code for the paper Text-Conditioned Outfit Recommendation With Hybrid Attention Layer.

The model in this paper aim to recommend a whole set of fashion items with internal compatibility and text description coherence.

Data Preparation

The dataset used in this paper is from UIUC Polyvore. You can download the dataset from this line

First use pretrained model to extract features from item images, item text, outfit text. You can download extracted features or extract by yourself. If download extracted features, put these file into project directory:

• imgs_feats_fashionClip.pkl: item image features
• encoded_itemTitleDescription_distiluse-base-multilingual-cased-v2.pkl: item text features
• encoded_category_distiluse-base-multilingual-cased-v2.pkl: item category features
• encoded_outfitUrlTitle_en_fashionClip.pkl: outfit text features

If you extract features by yourself, The following scripts may help you:

# Prepare extracted image, text, category features of items
python scripts/extract_item_image_with_fashionClip.py
python scripts/extract_item_txt_with_sentenceBERT.py
python scripts/extract_item_category_with_sentenceBERT.py

# Prepare text features of outfit text
python scripts/translate_outfit_txt.py # translate to english
python scripts/extract_outfit_txt_fashionClip.py

Train

Train with compatibility prediction task:

python train_cp.py

Train with CIR task:

python train_cir.py

Evaluate

To evaluate compatibility prediction task, use the following script:

python evaluate_cp.py

To evaluate CIR task, use the following script:

python evaluate_cir.py