Wrapping a single function

This page is outdated! Consult with the code in master.

Say we're implementing imgproc module bindings for Torch. So let's see what we have in it by inspecting opencv2/imgproc.hpp as it contains function signatures and their doc descriptions.

It looks like there's cv::GaussianBlur function:

CV_EXPORTS_W void GaussianBlur(InputArray src, OutputArray dst, Size ksize,
                               double sigmaX, double sigmaY = 0,
                               int borderType = BORDER_DEFAULT);

Note the CV_EXPORTS macro -- from it, we infer the need to export this function into foreign language bindings.

We start off by defining a Lua function in ./cv/imgproc.lua:

function cv.GaussianBlur(t)
    -- function body
end

Note that this function belongs to cv table.

As can be seen from the signature and the docs, this filter is intended for application in 3 ways:

(1) output to an empty cv::Mat:

cv::Mat image = imread('img.png');
cv::Mat blurred;
cv::GaussianBlur(image, blurred, cv::Size(3, 3), 1.0, 1.0);

(2) output to a cv::Mat of the same size and type as src:

cv::Mat image = imread('img.png');
cv::Mat blurred = image.clone() * 0;
cv::GaussianBlur(image, blurred, cv::Size(3, 3), 1.0, 1.0);

(3) filter in-place:

cv::Mat image = imread('img.png');
cv::GaussianBlur(image, image, cv::Size(3, 3), 1.0, 1.0);

Let's make these use cases possible for Torch users!

We want cv.GaussianBlur to accept a table of arguments t, so the calls for those use cases should look like this:

-- output to retval, as dst is not provided!
local image_B = cv.GaussianBlur{src=image, ksize={7, 7}, sigmaX=3.5, sigmaY=3.5}

-- output to another Tensor of same size & type
local image_A = image * 0
cv.GaussianBlur{src=image, dst=image_A, ksize={7, 7}, sigmaX=3.5, sigmaY=3.5}

-- or filter in-place
cv.GaussianBlur{src=image, dst=image, ksize={width=7, height=7}, sigmaX=3.5, sigmaY=3.5}

So let's convert t's fields to local variables representing arguments, with regard to the original function signature. For this, we have cv.argcheck function:

./cv/imgproc.lua

function cv.GaussianBlur(t)
    local argRules = {
        {"src", required = true},
        {"dst", default = nil},
        {"ksize", required = true}, -- problem! how to convert this?
        {"sigmaX", required = true},
        {"sigmaY", default = 0},
        {"borderType", default = cv.BORDER_DEFAULT}
    }
    local src, dst, ksize, sigmaX, sigmaY, borderType = cv.argcheck(t, argRules)

    -- to be continued
end

Some things to note here:

• Every argRules item either has default = <...> or required = true
• src and dst are assumed to be torch.Tensors
• dst is optional because of the first use case
• Almost all (more than 1000) OpenCV constants and enums like cv.BORDER_DEFAULT are defined in ./cv/constants.lua

And another thing: how to cope with that argument of type cv::Size? A great deal of OpenCV functions require argument(s) of some OpenCV class type, like cv::Size, cv::Point, cv::TermCriteria, cv::RotatedRect etc. We have to somehow create their instances in Lua, that's why for each such class cv::<class-name> we create a C wrapper struct <class-name>Wrapper in common code (cv.lua and Common.[hpp|cpp]). So let's wrap our cv::Size:

./include/Common.hpp

struct SizeWrapper {
    int width, height;

    // an operator for implicit automatic conversion to cv::Size
    inline operator cv::Size() { return cv::Size(width, height); }
};

./cv.lua

function cv.Size(data)
    return ffi.new('struct SizeWrapper', data)
end

Here, we exploit the intelligence of this ffi.new function: if you want to make a struct SizeWrapper from cv::Size(150, 280), you may pass either of the following to cv.Size:

• 150, 280
• {150, 280}
• {width=150, height=280}

Cool. Now we can directly convert t.ksize to struct SizeWrapper. operator = func in argRules tells cv.argcheck to apply func to the incoming argument:

./cv/imgproc.lua

function cv.GaussianBlur(t)
    local argRules = {
        {"src", required = true},
        {"dst", default = nil},
        {"ksize", required = true, operator = cv.Size},
        {"sigmaX", required = true},
        {"sigmaY", default = 0},
        {"borderType", default = cv.BORDER_DEFAULT}
    }
    local src, dst, ksize, sigmaX, sigmaY, borderType = cv.argcheck(t, argRules)

    -- a call to a C function that invokes cv::GaussianBlur goes here
end

Now let's turn to our C function that will be called through FFI. According to how we've prepared our args, the signature has to be as follows:

./include/imgproc.hpp

extern "C" struct TensorWrapper GaussianBlur(
    struct TensorWrapper src, struct TensorWrapper dst,
    struct SizeWrapper ksize, double sigmaX,
    double sigmaY, int borderType
);

Remember the 3 use cases of this filter? That's what we have to implement in ./src/imgproc.cpp. The code should be self-explanatory:

./src/imgproc.cpp

extern "C"
struct TensorWrapper GaussianBlur(struct TensorWrapper src, struct TensorWrapper dst,
                                  struct SizeWrapper ksize, double sigmaX,
                                  double sigmaY, int borderType)
{
    // first, check if dst is provided
    if (dst.isNull()) {
        // [use case 1]: output to return value
        cv::Mat retval;
        cv::GaussianBlur(
                src.toMat(), retval, ksize, sigmaX, sigmaY, borderType);
        // create a NEW Tensor in C and pass it to Lua
        return TensorWrapper(retval);
    } else if (dst.tensorPtr == src.tensorPtr) {
        // [use case 3]: filter in-place
        cv::Mat source = src.toMat();
        cv::GaussianBlur(
                source, source, ksize, sigmaX, sigmaY, borderType);
    } else {
        // [use case 2]: try to output to a Tensor dst of same size & type as src
        cv::GaussianBlur(
                src.toMat(), dst.toMat(), ksize, sigmaX, sigmaY, borderType);
    }
    return dst;
}

Note that:

• ksize is passed to cv::GaussianBlur directly as SizeWrapper::operator cv::Size() is defined. It would be also correct to write cv::Size(ksize).
• In contrast, struct TensorWrapper can't be converted to cv::Mat implicitly. Instead, you have to call .toMat().
• There's also a TensorWrapper::operator cv::Mat, so a return retval could be fine, but I construct TensorWrapper explicitly for code readability.
• Some OpenCV functions provide a cv::noArray() default value for InputArray/OutputArray arguments. Unfortunately, this can't be constructed (in an elegant way) in Lua, so you should use TO_MAT_OR_NOARRAY(tensor) macro.

Next, compile the libs:

mkdir build
cd build
cmake ..
# or, optionally: `cmake .. -DOpenCV_DIR=<path-to-your-opencv-3.x.x>`
make

Our function now resides in ./lib/libimgproc.so (for Linux and the like) so we can call it through FFI:

./cv/imgproc.lua

function cv.GaussianBlur(t)
    local argRules = {
        {"src", required = true},
        {"dst", default = nil},
        {"ksize", required = true, operator = cv.Size},
        {"sigmaX", required = true},
        {"sigmaY", default = 0},
        {"borderType", default = cv.BORDER_DEFAULT}
    }
    local src, dst, ksize, sigmaX, sigmaY, borderType = cv.argcheck(t, argRules)

    return cv.unwrap_tensors(
        C.GaussianBlur(
            cv.wrap_tensor(src), cv.wrap_tensor(dst), ksize, sigmaX, sigmaY, borderType))
end

cv.wrap_tensor(tensor) is a special function that takes one Tensor and returns a TensorWrapper over it. It should be applied to every torch.Tensor that is going to be passed to C++ via FFI.

cv.wrap_tensors(tensors) is a similar one, but it takes multiple Tensors (either in a table or not) and returns a TensorArray containing them.

cv.unwrap_tensors(wrapper, toTable) converts a TensorWrapper to a torch.Tensor. If a TensorArray is passed, then all the tensors are returned unpacked; if you need these tensors in a table, call .cv.unwrap_tensors(wrapper, true).

That's it! See the usage of cv.GaussianBlur in ./demo/filtering.lua.