How to prepare input data when settings.method=user?

[BroJackfeely]

I have paired dataset {x,y}, where x is the low-resolution images and y the corresponding high-resolution ones. I want to train my own model to achieve a super-resolution task. Therefore, I set the method as USER. How can I prepare my .t7 input data file?

[nagadomi]

-method user supports 1x and 2x mapping x -> y.
First, prepare the directory where each image is stored, for example, ./x and ./y.
tools/make_pairwise_list.lua is a helper script for generating the paired image list.

mkdir user_data
th tools/make_pairwise_list.lua -x_dir ./x -y_dir ./y > ./user_data/image_list.txt

make images.t7

th convert_data.lua -data_dir ./user_data

train (if your dataset x,y is the same size, specify -scale 1)

th train.lua -data_dir ./user_data -scale 2 -method user -test test_input_image.png  -name your_model_name -model_dir ./models/user

And, I recommend you to use dev branch.

EDIT:
Sorry, I did not mention to -model option.
When use -scale 2, specify -model upconv_7 or -model upcunet.
When use -scale 1, specify -model vgg_7 or -model cunet.

[Archit01]

i created two folders one with noisy and low resoultion images 512512 in x folder and there corresponding 2x high resoultion image 10241024 noise free in y folder and run the same commands as above but got this error `th train.lua -data_dir ./user_data -scale 2 -method user -test test.jpg -name photo2 -model_dir ./models/user -downsampling_filters "Box,Sinc,Catrom" -style photo -color rgb -random_unsharp_mask_rate 0.5 -thread 3 -nr_rate 1 -resize_blur_min 1 -resize_blur_max 1
{
grayscale : false
thread : 3
name : "photo2"
loss : "huber"
use_transparent_png : false
model : "vgg_7"
patches : 64
random_pairwise_negate_x_rate : 0
downsampling_filters :
{
1 : "Box"
2 : "Sinc"
3 : "Catrom"
}
resume : ""
crop_size : 48
random_pairwise_rotate_min : -6
random_blur_size : "3,5"
random_pairwise_scale_max : 1.176
random_blur_rate : 0
random_pairwise_negate_rate : 0
random_pairwise_rotate_max : 6
oracle_drop_rate : 0.5
inner_epoch : 4
invert_x : false
epoch : 50
update_criterion : "mse"
random_overlay_rate : 0
jpeg_chroma_subsampling_rate : 0.5
image_list : "./user_data/image_list.txt"
oracle_rate : 0.1
model_dir : "./models/user"
backend : "cunn"
random_pairwise_scale_min : 0.85
active_cropping_rate : 0.5
batch_size : 16
method : "user"
max_size : 256
padding_y_zero : false
plot : false
save_history : false
resize_blur_max : 1
gpu :
{
1 : 1
}
random_pairwise_scale_rate : 0
random_pairwise_rotate_rate : 0
random_color_noise_rate : 0
validation_filename_split : false
resize_blur_min : 1
learning_rate_decay : 3e-07
images : "./user_data/images.t7"
random_unsharp_mask_rate : 0.5
random_blur_sigma_min : 0.5
color : "rgb"
pairwise_y_binary : false
active_cropping_tries : 10
style : "photo"
data_dir : "./user_data"
random_half_rate : 0
model_file : "./models/user/photo2_model.t7"
validation_rate : 0.05
pairwise_flip : true
max_training_image_size : -1
noise_level : 1
random_blur_sigma_max : 1
validation_crops : 200
nr_rate : 1
learning_rate : 0.00025
scale : 2
seed : 11
test : "test.jpg"
padding : 0
}

make validation-set

load .. 1

1

resampling

/home/archit/torch/install/bin/luajit: ...e/archit/torch/install/share/lua/5.1/threads/threads.lua:179: [thread 3 endcallback] train.lua:377: inconsistent tensor size, expected tensor [3 x 48 x 48] and src [3 x 24 x 24] to have the same number of elements, but got 6912 and 1728 elements respectively at /home/archit/torch/pkg/torch/lib/TH/generic/THTensorCopy.c:86
stack traceback:
[C]: in function 'copy'
train.lua:377: in function train.lua:374
[C]: in function 'xpcall'
...e/archit/torch/install/share/lua/5.1/threads/threads.lua:174: in function 'dojob'
...e/archit/torch/install/share/lua/5.1/threads/threads.lua:264: in function 'synchronize'
train.lua:395: in function 'resampling'
train.lua:557: in function 'train'
train.lua:642: in main chunk
[C]: in function 'dofile'
...chit/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x55720ffac560
stack traceback:
[C]: in function 'error'
...e/archit/torch/install/share/lua/5.1/threads/threads.lua:179: in function 'dojob'
...e/archit/torch/install/share/lua/5.1/threads/threads.lua:264: in function 'synchronize'
train.lua:395: in function 'resampling'
train.lua:557: in function 'train'
train.lua:642: in main chunk
[C]: in function 'dofile'
...chit/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x55720ffac560
i tried changeing the orders of file high resolution images in x folder and low ones in y and got this errorth train.lua -data_dir ./user_data -scale 2 -method user -test test.jpg -name photo2 -model_dir ./models/user -downsampling_filters "Box,Sinc,Catrom" -style photo -color rgb -random_unsharp_mask_rate 0.5 -thread 3 -nr_rate 1 -resize_blur_min 1 -resize_blur_max 1
{
grayscale : false
thread : 3
name : "photo2"
loss : "huber"
use_transparent_png : false
model : "vgg_7"
patches : 64
random_pairwise_negate_x_rate : 0
downsampling_filters :
{
1 : "Box"
2 : "Sinc"
3 : "Catrom"
}
resume : ""
crop_size : 48
random_pairwise_rotate_min : -6
random_blur_size : "3,5"
random_pairwise_scale_max : 1.176
random_blur_rate : 0
random_pairwise_negate_rate : 0
random_pairwise_rotate_max : 6
oracle_drop_rate : 0.5
inner_epoch : 4
invert_x : false
epoch : 50
update_criterion : "mse"
random_overlay_rate : 0
jpeg_chroma_subsampling_rate : 0.5
image_list : "./user_data/image_list.txt"
oracle_rate : 0.1
model_dir : "./models/user"
backend : "cunn"
random_pairwise_scale_min : 0.85
active_cropping_rate : 0.5
batch_size : 16
method : "user"
max_size : 256
padding_y_zero : false
plot : false
save_history : false
resize_blur_max : 1
gpu :
{
1 : 1
}
random_pairwise_scale_rate : 0
random_pairwise_rotate_rate : 0
random_color_noise_rate : 0
validation_filename_split : false
resize_blur_min : 1
learning_rate_decay : 3e-07
images : "./user_data/images.t7"
random_unsharp_mask_rate : 0.5
random_blur_sigma_min : 0.5
color : "rgb"
pairwise_y_binary : false
active_cropping_tries : 10
style : "photo"
data_dir : "./user_data"
random_half_rate : 0
model_file : "./models/user/photo2_model.t7"
validation_rate : 0.05
pairwise_flip : true
max_training_image_size : -1
noise_level : 1
random_blur_sigma_max : 1
validation_crops : 200
nr_rate : 1
learning_rate : 0.00025
scale : 2
seed : 11
test : "test.jpg"
padding : 0
}

make validation-set

/home/archit/torch/install/bin/luajit: ...e/archit/torch/install/share/lua/5.1/threads/threads.lua:183: [thread 1 callback] bad argument #2 to '?' (end index out of bound at /home/archit/torch/pkg/torch/generic/Tensor.c:992)
stack traceback:
[C]: at 0x7f3d7b75b5b0
[C]: in function '__index'
lib/iproc.lua:16: in function 'crop'
lib/pairwise_transform_utils.lua:130: in function 'active_cropping'
lib/pairwise_transform_user.lua:34: in function 'transformer'
train.lua:261: in function train.lua:260
[C]: in function 'xpcall'
...e/archit/torch/install/share/lua/5.1/threads/threads.lua:234: in function 'callback'
/home/archit/torch/install/share/lua/5.1/threads/queue.lua:65: in function </home/archit/torch/install/share/lua/5.1/threads/queue.lua:41>
[C]: in function 'pcall'
/home/archit/torch/install/share/lua/5.1/threads/queue.lua:40: in function 'dojob'
[string " local Queue = require 'threads.queue'..."]:15: in main chunk
stack traceback:
[C]: in function 'error'
...e/archit/torch/install/share/lua/5.1/threads/threads.lua:183: in function 'dojob'
...e/archit/torch/install/share/lua/5.1/threads/threads.lua:223: in function 'addjob'
train.lua:259: in function 'make_validation_set'
train.lua:503: in function 'train'
train.lua:642: in main chunk
[C]: in function 'dofile'
...chit/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x55cdb9b7e560
`

[nagadomi]

Sorry, in the above command, i did not mention to -model option.
When use -scale 2, specify -model upconv_7 or -model resnet_14l(renset_14l is only available dev branch).
When use -scale 1, specify -model vgg_7(it is default).

[78Alpha]

Just tried this out and it did not work.

/root/torch/install/bin/luajit: bad argument #1 to '?' (must be strictly positive at /root/torch/pkg/torch/lib/TH/generic/THTensorMath.c:2033)
stack traceback:
[C]: at 0x7f5482c32c90
[C]: in function 'randperm'
tools/make_pairwise_list.lua:54: in function 'output'
tools/make_pairwise_list.lua:108: in main chunk
[C]: in function 'dofile'
/root/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x56477add4570

Will be tinkering with it, but it should have just been copy and paste commands...

[nagadomi]

No paired image is found.
./x and ./y must contain paired images with the same name.
like

x/1.png
x/abcd.png
...
y/1.png
y/abcd.png

[78Alpha]

They are indeed pairs. However, i am using the docker image, so it must be different. The docker image has a REALLY hard time finding and understanding what files are available...

Also, managed to reach the convert lua step, but the script is not able to read the directories it made earlier. It tries to access /some/directory/other/ instead of ./some/directory/other/

Hopefully I can fix it without editing 100's of lines of the image list

EDIT:

Decided to just sh into the thing and build everything from there, it all works now, but for some reason it couldn't find the very first image without altering the file list. A very random thing.

[ErrorRaffyline0]

I am a complete noob at C++ and cmd stuff, so please don't cringe. For me "th" is not recognised as a command when "cd'ded" into the waifu2x-caffe-master or waifu2x-caffe folders. Or can you only do this on the non-caffe version (bc no linux (and i'm having issues with the windows subsystem for linux))

[nagadomi]

waifu2x-caffe does not support training the model.