Intrinsic Subgraph Generation for Interpretable Graph based Visual Question Answering

Paper Links

Discrete Subgraph Sampling

Intrinsic Subgraph Generation

Approach

Documentation

Documentation

Installation

Python Environment

Create a virtual python environment with e.g. conda:

conda create --name isubgvqa python=3.11

Activate the environment

conda activate isubgvqa

PyTorch

Please install PyTorch

pip install torch torchvision torchaudio

PyG (PyTorch-Geometric)

Please install PyG

pip install torch_geometric

Install optional packages:

pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.2.0+cu121.html

Spacy

Install spacy and download en_core_web_sm

pip install -U pip setuptools wheel
pip install -U spacy
python -m spacy download en_core_web_sm

Citations

@article{tilli2024discretesubgraphsamplinginterpretable,
      title={Discrete Subgraph Sampling for Interpretable Graph based Visual Question Answering}, 
      author={Pascal Tilli and Ngoc Thang Vu},
      year={2024},
      eprint={2412.08263},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2412.08263}, 
}

@inproceedings{tilli-vu-2024-intrinsic-subgraph,
    title = "Intrinsic Subgraph Generation for Interpretable Graph Based Visual Question Answering",
    author = "Tilli, Pascal  and Vu, Ngoc Thang",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italy",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.806",
    pages = "9204--9223",
}

GitHub References

IMLE
https://github.com/uclnlp/torch-imle

AIMLE
https://github.com/EdinburghNLP/torch-adaptive-imle

Gumbel / Simple
https://github.com/chendiqian/PR-MPNN/