Slow Flow: Generating Optical Flow Reference Data

This code is based on the paper Slow Flow: Exploiting High-Speed Cameras for Accurate and Diverse Optical Flow Reference Data.

The high speed flow code (slow_flow) is based on EpicFlow v1.00. We extended it to reason over multiple frames and occlusions. The extended code is included and for details on the original code we refer to the paper by Revaud et al. EpicFlow: Edge-Preserving Interpolation of Correspondences for Optical Flow and the project webpage https://thoth.inrialpes.fr/src/epicflow/.

We provide two teaser sequences to run our code. We are working on publishing the complete high speed datasets used in the project.

Compiling

The following libraries are necessary to compile and use our code:

Eigen3, Boost, Atlas, Blas, Lapack, Flann, GSL, PNG and JPEG
sudo apt-get install libeigen3-dev libboost-all-dev libatlas-base-dev libblas-dev liblapack-dev libflann-dev libgsl-dev libpng-dev libjpeg-dev
 
Download and install opencv2.4
http://docs.opencv.org/2.4/doc/tutorials/introduction/linux_install/linux_install.html

Download SED for edges
https://www.microsoft.com/en-us/download/details.aspx?id=52370&from=http%3A%2F%2Fresearch.microsoft.com%2Fen-us%2Fdownloads%2F389109f6-b4e8-404c-84bf-239f7cbf4e3d%2F

Download Piotr Dollar's toolbox
https://pdollar.github.io/toolbox/

Download Deep Matching 
(Optional: using coarse-to-fine by setting ‘deep_matching’ to 0 and ‘slow_flow_layers’  larger than 1)
http://lear.inrialpes.fr/src/deepmatching/

Download flow-code from Middlebury OF dataset and compile ImageLib
http://vision.middlebury.edu/flow/code/flow-code.zip

Download gco-v3.0 library (Multi-label optimization)
http://vision.csd.uwo.ca/code/

Download the tree-reweighted message passing algorithm (v1.3)
(You might need to modify instances.inc and change "template ..." to "template class ...")
https://www.microsoft.com/en-us/download/details.aspx?id=52499&from=http%3A%2F%2Fresearch.microsoft.com%2Fen-us%2Fdownloads%2Fdad6c31e-2c04-471f-b724-ded18bf70fe3%2F

(Optional) Download Gunturk-Altunbasak-Mersereau Alternating Projections Image Demosaicking 
By setting ‘raw_demosaicing’ to 1 in the configuration file, uncommenting line 25 and 26 in configuration.h and uncommenting line 13 in CMakeLists.txt
http://www.ipol.im/pub/art/2011/g_gapd/

The paths to the libraries need to be specified in the following files:

CMakeLists.txt 			(lines 10-13)
configuration.h 			(lines 20-25)
matlab/detect_edges.m    (lines 3-5)

Run cmake and make to compile the code

mkdir build
cd build
cmake ../
make

Run Pipeline

1. Run epic flow on low resolution to use adaptive frame rates for slow flow

./adaptiveFR -path [path] -folder [folder]

Examples for our teaser_sequences:
./adaptiveFR -path '[path to teaser]/sequence/' -folder 'sheeps' -raw
./adaptiveFR -path '[path to teaser]/sequence/' -folder 'ambush_2' -format 'out_%i_%03i.png' -start 491 -sintel

2. Run slow_flow for flow estimations of all consecutive high speed frames

./slow_flow [slow flow cfg file]

An example configuration file is provided in "cfgs".

Optional: 
	-jet		compute the flow for one specified high speed pair
	-fr		using adaptive frame rate compute flow for 0: high frame rate, 1: low frame rate 
	-resume		resume processing configuration file
	-deep_settings	specify settings for deep matching

3. Run dense_tracking using the output of slow flow

./dense_tracking [dense tracking cfg file]

Optional: 
	-select		compute the flow for one specific final image pair
	-resume		resume processing configuration file

An example configuration file is provided in "cfgs".

Citation

If you use our code, please cite our paper:

@INPROCEEDINGS{Janai2017CVPR,
  author = {Joel Janai and Fatma Güney and Jonas Wulff and Michael Black and Andreas Geiger},
  title = {Slow Flow: Exploiting High-Speed Cameras for Accurate and Diverse Optical Flow Reference Data},
  booktitle = {Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2017}
}