Skip to content

Latest commit

 

History

History
651 lines (526 loc) · 30.5 KB

README.md

File metadata and controls

651 lines (526 loc) · 30.5 KB
Raw

opencv-mobile

License build download

Android iOS ARM Linux Windows Ubuntu MacOS HarmonyOS Firefox Chrome

✔️ This project provides the minimal build of opencv library for the Android, iOS and ARM Linux platforms.

✔️ Packages for Windows, Linux, MacOS, HarmonyOS and WebAssembly are available now.

✔️ We provide prebuild binary packages for opencv 2.4.13.7, 3.4.20 and 4.10.0.

✔️ We also provide prebuild package for Mac-Catalyst, watchOS, tvOS, visionOS and Apple xcframework.

✔️ All the binaries are compiled from source on github action, no virus, no backdoor, no secret code.

✔️ NEW FEATURE cv::putText supports full-width CJK characters

✔️ NEW FEATURE cv::imshow supports Linux framebuffer and Windows

opencv 4.10.0 package size The official opencv opencv-mobile
source zip 95.2 MB 8.25 MB
android 292 MB 17.7 MB
ios 207 MB 3.97 MB
Technical Exchange QQ Group
623504742（opencv-mobile夸夸群）

Download

https://github.com/nihui/opencv-mobile/releases/latest

Source

Android

HarmonyOS

iOS

iOS-Simulator

macOS

Mac-Catalyst

watchOS

watchOS-Simulator

tvOS

tvOS-Simulator

visionOS

visionOS-Simulator

Apple xcframework

Ubuntu-20.04

Ubuntu-22.04

VS2015

VS2017

VS2019

VS2022

WebAssembly

ARM-Linux

  • Android package build with ndk r26c and android api 21
  • iOS / MacOS / Mac-Catalyst / watchOS / tvOS / visionOS package build with Xcode 15.2
  • ARM Linux package build with cross-compiler on Ubuntu-22.04
  • WebAssembly package build with Emscripten 3.1.28

opencv-mobile package for development boards

milkv-duo
milkv-duo
riscv64-linux-musl
✅ HW JPG decoder
✅ MIPI CSI camera
opencv4-milkv-duo 		licheerv-nano
licheerv-nano
riscv64-linux-musl
✅ HW JPG decoder
✅ MIPI CSI camera
opencv4-licheerv-nano 		luckfox-pico
luckfox-pico
arm-linux-uclibcgnueabihf
✅ HW JPG encoder
✅ MIPI CSI camera
✅ DPI LCD screen
opencv4-luckfox-pico
yuzuki-lizard
yuzuki-lizard
arm-linux-uclibcgnueabihf
opencv4-yuzuki-lizard 		tinyvision
tinyvision
arm-linux-uclibcgnueabihf
✅ HW JPG decoder
✅ HW JPG encoder
✅ MIPI CSI camera
✅ SPI LCD screen
opencv4-tinyvision 		yuzuki-chameleon
yuzuki-chameleon
arm-openwrt-linux-gnueabi
✅ HW JPG decoder
✅ HW JPG encoder
opencv4-yuzuki-chameleon
purple-pi
purple-pi
arm-linux-uclibcgnueabihf
opencv4-purple-pi 		myir-t113i
myir-t113i
arm-linux-gnueabi
✅ HW JPG decoder
✅ HW JPG encoder
opencv4-myir-t113i 		2k0300-fengniao
2k0300-fengniao
loongarch64-linux-gnu
opencv4-2k0300-fengniao
lockzhiner-vision-module
lockzhiner-vision-module
arm-linux-uclibcgnueabihf
✅ HW JPG encoder
✅ MIPI CSI camera
✅ DPI LCD screen
opencv4-lockzhiner-vision-module

Usage Android

  1. Extract archive to <project dir>/app/src/main/jni/
  2. Modify <project dir>/app/src/main/jni/CMakeListst.txt to find and link opencv
set(OpenCV_DIR ${CMAKE_SOURCE_DIR}/opencv-mobile-4.10.0-android/sdk/native/jni)
find_package(OpenCV REQUIRED)

target_link_libraries(your_jni_target ${OpenCV_LIBS})

Usage iOS, macOS, watchOS, tvOS, visionOS

  1. Extract archive, and drag opencv2.framework or opencv2.xcframework into your project

Usage ARM Linux, Windows, Linux, WebAssembly

  1. Extract archive to <project dir>/
  2. Modify <project dir>/CMakeListst.txt to find and link opencv
  3. Pass -DOpenCV_STATIC=ON to cmake option for windows build
set(OpenCV_DIR ${CMAKE_SOURCE_DIR}/opencv-mobile-4.10.0-armlinux/arm-linux-gnueabihf/lib/cmake/opencv4)
find_package(OpenCV REQUIRED)

target_link_libraries(your_target ${OpenCV_LIBS})

How-to-build your custom package

We reduce the binary size of opencv-mobile in 3 ways

  1. Reimplement some modules (such as highgui) and functions (such as putText)
  2. Apply patches to disable rtti/exceptions and do not install non-essential files
  3. Carefully select cmake options to retain only the modules and functions you want

Steps 1 and 2 are relatively cumbersome and difficult, and require intrusive changes to the opencv source code. If you want to know the details, please refer to the steps in .github/workflows/release.yml

The opencv-mobile source code package is the result of steps 1 and 2. Based on it, we can adjust the cmake option to compile our own package and further delete and add modules and other functions.

step 1. download opencv-mobile source

wget -q https://github.com/nihui/opencv-mobile/releases/latest/download/opencv-mobile-4.10.0.zip
unzip -q opencv-mobile-4.10.0.zip
cd opencv-mobile-4.10.0

step 2. apply your opencv option changes to options.txt

vim options.txt

step 3. build your opencv package with cmake

mkdir -p build
cd build
cmake -DCMAKE_INSTALL_PREFIX=install \
  -DCMAKE_BUILD_TYPE=Release \
  `cat ../options.txt` \
  -DBUILD_opencv_world=OFF ..
make -j4
make install

step 4. make a package

zip -r -9 opencv-mobile-4.10.0-mypackage.zip install

Some notes

  • The minimal opencv build contains most basic opencv operators and common image processing functions, with some handy additions like keypoint feature extraction and matching, image inpainting and opticalflow estimation.

  • Many computer vision algorithms that reside in dedicated modules are discarded, such as face detection etc. You could try deep-learning based algorithms with neural network inference library optimized for mobile.

  • Image IO functions in highgui module, like cv::imread and cv::imwrite, are re-implemented using stb for smaller code size. GUI functions, like cv::imshow, are discarded.

  • cuda and opencl are disabled because there is no cuda on mobile, no opencl on ios, and opencl on android is slow. opencv on gpu is not suitable for real productions. Write metal on ios and opengles/vulkan on android if you need good gpu acceleration.

  • C++ RTTI and exceptions are disabled for minimal build on mobile platforms and webassembly build. Be careful when you write cv::Mat roi = image(roirect); :P

cv::putText supports full-width CJK characters

  1. Open https://nihui.github.io/opencv-mobile/patches/fontface.html or opencv-mobile-X.Y.Z/fontface.html in your browser.
  2. In the opened page, enter all the text to be drawn, select the TTF font file (optional), click the Convert to Font Header button to download the fontface header. This step is completely local operation, without connecting to a remote server, your data is private and safe.
  3. Include the generated fontface header, initialize a fontface instance, and pass it as the argument to cv::putText. The source file must be encoded in UTF-8.

Since all characters have been converted to embedded bitmap, the drawing routine does not depend on freetype library or any font files at runtime.

#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include "myfontface.h"

int main()
{
    cv::Mat bgr = cv::imread("atari.jpg", 1);

    // use this font
    MyFontFace myfont;

    // draw full-width text with myfont
    const char* zhtext = "称呼机器人为破铜烂铁，\n违反了禁止歧视机器人法！";
    cv::putText(bgr, zhtext, cv::Point(30, 250), cv::Scalar(127, 0, 127), myfont, 20);

    // get bounding rect
    cv::Rect rr = cv::getTextSize(bgr.size(), zhtext, cv::Point(30, 250), myfont, 20);

    cv::imwrite("out.jpg", bgr);

    return 0;
}

cv::imshow supports Linux framebuffer and Windows

In Linux, cv::imshow can display images on the screen (/dev/fb0) via the Linux Framebuffer API. cv::imshow can work without desktop environment (gnome, KDE Plasma, xfce, etc.) or window manager (X or wayland), making it suitable for embedded scenarios. The first argument to cv::imshow must be fb.

In Windows, cv::imshow will use the Windows API to create a simple window for displaying.

display image

#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>

int main()
{
    cv::Mat bgr = cv::imread("im.jpg", 1);

    cv::imshow("fb", bgr);

    return 0;
}

realtime camera preview

#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>

int main()
{
    cv::VideoCapture cap;
    cap.set(cv::CAP_PROP_FRAME_WIDTH, 320);
    cap.set(cv::CAP_PROP_FRAME_HEIGHT, 240);
    cap.open(0);

    cv::Mat bgr;
    while (1)
    {
        cap >> bgr;
        cv::imshow("fb", bgr);
    }

    return 0;
}

opencv modules included

module comment
opencv_core Mat, matrix operations, etc
opencv_imgproc resize, cvtColor, warpAffine, etc
opencv_highgui imread, imwrite
opencv_features2d keypoint feature and matcher, etc (not included in opencv 2.x package)
opencv_photo inpaint, etc
opencv_video opticalflow, etc

opencv modules discarded

module comment
opencv_androidcamera use android Camera api instead
opencv_calib3d camera calibration, rare uses on mobile
opencv_contrib experimental functions, build part of the source externally if you need
opencv_dnn very slow on mobile, try ncnn for neural network inference on mobile
opencv_dynamicuda no cuda on mobile
opencv_flann feature matching, rare uses on mobile, build the source externally if you need
opencv_gapi graph based image processing, little gain on mobile
opencv_gpu no cuda/opencl on mobile
opencv_imgcodecs link with opencv_highgui instead
opencv_java wrap your c++ code with jni
opencv_js write native code on mobile
opencv_legacy various good-old cv routines, build part of the source externally if you need
opencv_ml train your ML algorithm on powerful pc or server
opencv_nonfree the SIFT and SURF, use ORB which is faster and better
opencv_objdetect HOG, cascade detector, use deep learning detector which is faster and better
opencv_ocl no opencl on mobile
opencv_python no python on mobile
opencv_shape shape matching, rare uses on mobile, build the source externally if you need
opencv_stitching image stitching, rare uses on mobile, build the source externally if you need
opencv_superres do video super-resolution on powerful pc or server
opencv_ts test modules, useless in production anyway
opencv_videoio use android MediaCodec or ios AVFoundation api instead
opencv_videostab do video stablization on powerful pc or server
opencv_viz vtk is not available on mobile, write your own data visualization routines