Custom Vision + Azure IoT Edge on a Raspberry Pi 3

This is a sample showing how to deploy a Custom Vision model to a Raspberry Pi 3 device running Azure IoT Edge. Custom Vision is an image classifier and object detector that is trained in the cloud with your own images. IoT Edge gives you the possibility to run this model next to your cameras, where the video data is being generated. You can thus add meaning to your video streams to detect road traffic conditions, estimate wait lines, find parking spots, etc. while keeping your video footage private, lowering your bandwidth costs and even running offline.

The project consists of the custom vision model deployed on Raspberry PI and detecting objects visible through the camera.

Toolbox.

• RPi 3 or 4 with power adapter
• External Camera or RPi camera.
• LAN or WiFi access.
• USB Keyboard.
• HDMI cable.
• HDMI Monitor.
• Objects for detection (eg fruits).
• Laptop with VS code and docker.
• Azure Subscription (Free trial works).

Prepare development environment.

1. You should have the following prerequisites in place:

   • A free or standard-tier IoT Hub in Azure.
   • A container registry, like Azure Container Registry.
   • Visual Studio Code configured with the Azure IoT Tools.
   • Docker CE is configured to run Linux containers.

2. To develop an IoT Edge module with the Custom Vision service, install the following additional prerequisites on your development machine:

   • Python (https://www.python.org/downloads/)
   • Git (https://git-scm.com/downloads)
   • Python extension for Visual Studio Code (https://marketplace.visualstudio.com/items?itemName=ms-python.python)

Create a new Custom Vision model.

1. In your web browser, navigate to the Custom Vision web page. Sign in and sign in with the same account that you use to access Azure resources then create a new project for object detection.

2. The project set up and train described in the tutorial.

3. Be aware you need to use a compact domain to be able to export models for RPi.

4. Images for training can be found in the "Images" folder of this repository.

5. Train the model and test the model on any custom images. Details described in the tutorial.

6. Export model as Docker file in zip format as explained in the tutorial.

7. Extract files from archive to the folder modules\ObjDetector but do not override app.py and Dockerfile. Those files has some customization.

Build project

1. Make sure in VS Code you select arm32v7 as target platform.

2. Update .evn file in root folder with blob account details where analyzed files should be uploaded and credential for Azure Container Registry.

CONTAINER_REGISTRY_USERNAME=<your acr short name>
CONTAINER_REGISTRY_PASSWORD=4nKYmt8m=<your acr user password>
BLOB_ACC=<blob account short name>
BLOB_KEY=<blob account key>

1. In the deployment.template.json update registryCredentials with full name of your ACR.

2. From terminal window in VS Code run command to sign in az acr login -n <short name of your ACR>.

3. From terminal run command py version.py to generate new versions of the module.json files.

4. Select file deployment.template.json in project view and from context menu choice Build and Push IoT Edge Solution. Then docker builds images and pushes them in ACR.

First time building takes up to 20 min. You can proceed with RPi preparation while waiting for the build.

1. In case of the pillow error follow the proposed fix.

2. Make sure the build ends up successfully and your Azure Container Registry has required container images.

   

Prepare RPi to Host Iot Edge

1. Download and flash the latest Raspbian image from the official web site. Lite version without desktop will be a vise choice. For flashing SD cards you can use Etcher.

2. Start RPi and run configuration raspi-config to allow SSH, camera access and network access. You need to connect the monitor and keyboard to complete configuration.

3. Connect by SSH to RPi (PuTTY) and change the default password.

4. For testing USB cameras you need to install fswebcam as explained in the following tutorial. Then you can use fswebcam to take a snapshot. You can use WinSCP to connect to the RPi and download images to observe.

5. In case of the Pi Camera you can follow the tutorial and use Python code to test.

6. Before install IoT Edge you first need to install the container engine by executing the following commands in the tutorial.

7. You also need to install IoT Edge on your RPi device. You also need to update security demon.

1.To properly configure /etc/aziot/config.toml you need to register your IoT device in Iot Hub. The following tutorial explained the registration process with a symmetric key. From the Azure portal create an IoT Hub and add a new Iot Edge device. Provide a unique name and keep all default settings and generated keys. Copy primary key into config.toml.

1. Restart your device and wait for status update of device on the IoT Hub. You also can execute command "iotedge list" to monitor process. Please pay attention for the version of azureiotedge-hub and azureiotedge-agentyou use. Version 1.0.0 having an issue and you can update that to version 1.2.3. You can use runtime settings as explained in following tutorial

2. Finlay you should have your IoTEdge modules running on RPi without errors.

   

Deploy modules on RPi

1. Make sure your build is successful and ACR contains the version you set up in configuration.

2. From the VS code context menu select command Build and Publish Iot Edge Solution. Then you can use command palette Azure IoT Edge: Create deployment for a single device and choose your device and file config\deployment.arm32v7.json.

   If you lost with deployment follow the steps in tutorial

3. As result of deployment your RPi should contain 4 modules (azureiotedge-agent, azureiotedge-hub, camera-capture, ObjDetector). You can monitor modules from IoT Hub or from the SSH console by command iotedge list. Remember that some images are quite heavy and require time to be downloaded from ACR to PI. You also can retrieve the logs of the container run by command iotedge log <your container name>.

   

Setting up the scene

1. Left the objects for identification on the solid background with about 3 fit distance from the camera.The blurry images will be unrecognizable.

   

2. Make sure that there are enough lights in the picture area.

3. You can also use the camera's tool on the Pi to take a few snapshots of objects and deploy images to the Custom vision and tag them to train better models.

   

Monitor and diagnose applications.

1. From the SSH console you can use commands like iotedge logs camera-capture or iotedge logs ObjDetector to monitor errors and issues. Correct output of the detector should looks as following:

   

   troubleshot iotedge service

2. The camera-capture module is hosting a local web site on port 5052. You can access it to monitor current camera setup. The image is updated once in a minute..

   

3. Finlay if the detection completed successfully the boundaries will be created on the image and analyzed images will be uploaded to the storage account you set up above.

   

References

1. How to run object detection with Tensorflow 2 on the Raspberry PI using Docker

2. Build RPi solution from Ch9

3. How To Access the Raspberry Pi Camera Inside Docker and OpenCV

4. Fixing Docker build issue

5. Uncompleted MS tutorial: Perform image classification at the edge with Custom Vision Service

6. Connection PI camera to RPi