InTraGen: Trajectory-controlled Video Generation for Object Interactions
_{PyTorch Implementation}

InTraGen: Trajectory-controlled Video Generation for Object Interactions
Zuhao Liu*, Aleksandar Yanev*, Ahmad Mahmood, Ivan Nikolov, Saman Motamed, Wei-Shi Zheng, Xi Wang, Luc Van Gool, Danda Pani Paudel (*Equal Contribution)

How to Run

Dowload dataset

MoVi-Extended | Domino | Football | Pool

1. Save and unzip the datasets into datasets folder

2. Install the Environment

bash install.sh

3. Download the Pre-trained Checkpoints from Open-Sora-Plan, and save the Open-Sora-Plan-v1.1.0 into models folder

4. Run the Training

bash train.sh

Citation

If you find this work useful for your research, please consider citing it.

@article{liu2024intragen,
  title={InTraGen: Trajectory-controlled Video Generation for Object Interactions},
  author={Liu, Zuhao and Yanev, Aleksandar and Mahmood, Ahmad and Nikolov, Ivan and Motamed, Saman and Zheng, Wei-Shi and Wang, Xi and Van Gool, Luc and Paudel, Danda Pani},
  journal={arXiv preprint arXiv:2411.16804},
  year={2024}
}

Acknowledgement

Open-Sora-Plan     ShareGPT4Video