Refactor imresize using Interpolations

[Evizero]

On my plane ride back from a work-related trip, I experimented with writing an imresize function using interpolations. The best I could come up with was this approach. It seems to work quite well for upscaling an Image, but when downscaling it, it is more of a sub-sampling than anything, which may not be the best way to do it.

Also, while playing around with Interpolations and Extrapolation (which are both an AbstractArray which trigger ImageInTerminal rendering), I noticed that restrict triggered some exceptions. For example one cant to A[1] if A is an interpolation

[timholy]

Looking good! What's the performance like?

[timholy]

Interesting. I haven't coded with Interpolations recently, but if it supports what we need then

R = CartesianRange(indices(A))
f = A[first(R)]

should work. Though perhaps the better strategy would to add support for first to Interpolations.

[Evizero]

first(A) works for interpolations. A[1] does not. For extrapolations neither work

[timholy]

Might want to dispatch to a version of imresize! that accepts original::AbstractInterpolation. That way if people want higher-order interpolation they can get it easily.

[Evizero]

Good idea

[Evizero]

What's the performance like?

Have not benchmarked yet. To be honest I was unsure if this is even close to what you had in mind

[timholy]

It's basically spot on. Nice work. My only concern is the inner loop, it seems possible we'd need to optimize it to get rid of the splat. But it's possible that the compiler will do all the hard work for us (which would be much better than doing it by hand).

[Evizero]

Understood. I will look into performance

[Evizero]

To get a little glimpse of where I am at I tried the following little micro benchmark as reference for the status quo

current master before my refactor:

julia> using Colors, ImageTransformations, BenchmarkTools

julia> org = rand(RGB, 20, 20);

julia> dst = similar(org, (1000,1000));

julia> @benchmark ImageTransformations.imresize!($dst, $org)
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     9.838 ms (0.00% GC)
  median time:      9.869 ms (0.00% GC)
  mean time:        9.873 ms (0.00% GC)
  maximum time:     10.129 ms (0.00% GC)
  --------------
  samples:          506
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

This PR currently without additional optimization:

julia> @benchmark ImageTransformations.imresize!($dst, $org)
BenchmarkTools.Trial: 
  memory estimate:  9.59 KiB
  allocs estimate:  5
  --------------
  minimum time:     17.118 ms (0.00% GC)
  median time:      17.126 ms (0.00% GC)
  mean time:        17.153 ms (0.00% GC)
  maximum time:     18.202 ms (0.00% GC)
  --------------
  samples:          292
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

(results are pretty consistent for both 0.5 and today's master)

[Evizero]

The 0.6 test issue on travis is related to JuliaArrays/StaticArrays.jl#106

[Evizero]

Concerning benchmark. The following implementation using a generated function yields the same regression as the current state of the PR

@generated function imresize!{T,S,N}(resized::AbstractArray{T,N}, original::AbstractInterpolation{S,N})
    quote
        org_size = size(original)
        dst_size = size(resized)
        @inbounds @nexprs $N i->(scale_i = org_size[i] / dst_size[i])
        @inbounds for I in CartesianRange(size(resized))
            I_d = I.I
            @nexprs $N i->(O_i = I_d[i] * scale_i)
            resized[I] = @nref $N original i->O_i
        end
        resized
    end
end

I don't think the tuple splatting is the issue (I also tried hardcoding the tuple splat and it made no difference). I am guessing the overhead is from interpolations maybe? The memory certainly is from interpolations as just creating the interpolation allocates the same amount.

[timholy]

OK, let's go with the simple version and then figure out how to improve Interpolations performance.

With regards to sub-sampling etc: I think when making a smaller image we're going to first have to do some filtering. We could either use ImageFiltering, or perhaps one could just restrict until it's smaller and then interpolate up? I'm sure there's literature on this, but I confess I haven't read it.

This seems fine to me. I'd say merge whenever you feel ready. To me it looks like the tests bork before we get to the imresize tests, so just make sure those work properly and we can fix other issues separately if you prefer.

[timholy]

Yay!

[Evizero]

Thanks for your comments and advice so far. I am looking forward to further contribute to this package

[timholy]

I think this is an important area for future growth, so I'm very glad to hear of your interest in pitching in! Right now my main focus is on getting the whole Images suite passing on 0.6, and in preparing the way for GSoC, so it's great to have people driving things forward in more "fundamental" respects.