code

.gitignore

@@ -1 +1,3 @@
 .vscode
+.idea/*
+ada_fuse_pretrained/*.tar

README.md

@@ -0,0 +1,26 @@
+# AdaFuse
+
+## Dependencies
+Red Hat 7.6, Tesla-V100x4
+
+(Here shows how to install the environment using `conda`)
+```bash
+conda create --name newenv python=3.7.6
+conda activate newenv
+conda install pytorch=1.2.0 torchvision
+pip install thop==0.0.31-2001170342
+```
+
+## Preparations
+Here taking SomethingV1 dataset for an example:
+1. Download all the tar-filed video frames from [https://20bn.com/datasets/something-something/v1](https://20bn.com/datasets/something-something/v1) and  untar them by `cat 20bn-something-something-v1-?? | tar zx`
+2. In `common.py`,  modify `ROOT_DIR`, `DATA_PATH` and `EXPS_PATH` to setup the dataset path and logs path (where to save checkpoints and logs)
+
+## Evaluation on SomethingV1
+Here we are trying to reproduce the results for `AdaFuse-TSN50`, `AdaFuse-TSM50` and `AdaFuse-TSM50Last` as shown in Table 3 in the main paper for SomethingV1 dataset.
+1. Download the pretrained models from [https://drive.google.com/drive/folders/1riSu-E0EzvpLHha5tcbC9OoAAX5i3cBL?usp=sharing](https://drive.google.com/drive/folders/1riSu-E0EzvpLHha5tcbC9OoAAX5i3cBL?usp=sharing) and put the folder under your experiment directory `EXPS_PATH`
+2. Run `sh zz_eval.sh` and you will see the expected/real results (4 GPUS are needed to guarantee the exact results)
+
+## Training on SomethingV1
+To train `AdaFuse-TSM50Last` on SomethingV1 dataset, simply run: `sh zz_train.sh`. It takes less than 14 hours on 4 TeslaV100 GPUs and should get similar accuracy (~ 46.8%) as shown in Table 3 in the main paper.
+

bn_archs/__init__.py

@@ -0,0 +1 @@
+from .bn_inception import *

common.py

@@ -0,0 +1,32 @@
+from os.path import join as ospj
+
+from os.path import expanduser
+ROOT_DIR = ospj(expanduser("~"), "scratch")  # TODO change this to your root path // root dir
+DATA_PATH = ospj(ROOT_DIR, "datasets")       # TODO change this to your data path // dataset path
+EXPS_PATH = ospj(ROOT_DIR, "logs_tsm")       # TODO change this to your logs path // saving logs
+
+
+def inner_set_manual_data_path(data_path, exps_path):
+    if data_path is not None:
+        global DATA_PATH
+        DATA_PATH = data_path
+
+    if exps_path is not None:
+        global EXPS_PATH
+        EXPS_PATH = exps_path
+
+
+def set_manual_data_path(data_path, exps_path):
+    inner_set_manual_data_path(data_path, exps_path)
+
+    global STHV1_FRAMES
+    STHV1_FRAMES = ospj(DATA_PATH, "sthv1", "20bn-something-something-v1")
+
+    global STHV2_FRAMES
+    STHV2_FRAMES = ospj(DATA_PATH, "something2something-v2", "frames")
+
+    global MINIK_FRAMES
+    MINIK_FRAMES = ospj(DATA_PATH, "kinetics-qf")
+
+    global JESTER_FRAMES
+    JESTER_FRAMES = ospj(DATA_PATH, "jester", "20bn-jester-v1")

data/jester/classInd.txt

@@ -0,0 +1,27 @@
+0,Swiping Left
+1,Swiping Right
+2,Swiping Down
+3,Swiping Up
+4,Pushing Hand Away
+5,Pulling Hand In
+6,Sliding Two Fingers Left
+7,Sliding Two Fingers Right
+8,Sliding Two Fingers Down
+9,Sliding Two Fingers Up
+10,Pushing Two Fingers Away
+11,Pulling Two Fingers In
+12,Rolling Hand Forward
+13,Rolling Hand Backward
+14,Turning Hand Clockwise
+15,Turning Hand Counterclockwise
+16,Zooming In With Full Hand
+17,Zooming Out With Full Hand
+18,Zooming In With Two Fingers
+19,Zooming Out With Two Fingers
+20,Thumb Up
+21,Thumb Down
+22,Shaking Hand
+23,Stop Sign
+24,Drumming Fingers
+25,No gesture
+26,Doing other things