Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add meta-hailo support #569

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open
11 changes: 11 additions & 0 deletions Kconfig
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,17 @@ config KAS_INCLUDE_EIO
default "kas/opt/eio.yml"
depends on EIO

config META_HAILO
bool "Meta-Hailo AI card support"
help
Enable building of meta-hailo, which includes the hailo driver,
firmware and userspace runtime libraries.

config KAS_INCLUDE_META_HAILO
string
default "kas/opt/meta-hailo.yml"
depends on META_HAILO

endif

# Provide two entries so that the help text can be adjusted to the image.
Expand Down
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,22 @@ Then build with
```
(or select `EIO subsystem support` in `kas menu`)

A tool to upgrade the customized signal module (SENS SM) firmware is also
avaiable, to build it with:

```shell
./kas-container build kas-iot2050-example.yml:kas/opt/eio.yml:kas/opt/module.yml
```

### Build with Hailo8 AI card support

To build the example image with the Hailo8 AI card support:

```shell
./kas-container build kas-iot2050-example.yml:kas/opt/meta-hailo.yml
```

Please visit [meta-hailo](./meta-hailo/README.md) for the details.

## Build user SDK
>>>
Expand Down
2 changes: 2 additions & 0 deletions kas/iot2050.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ defaults:

repos:
meta-iot2050:
layers:
.:

isar:
url: https://github.com/ilbers/isar
Expand Down
2 changes: 1 addition & 1 deletion kas/opt/eio.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ header:
version: 14

local_conf_header:
no-eio: |
eio: |
IOT2050_EIO_SUPPORT = "1"
23 changes: 23 additions & 0 deletions kas/opt/meta-hailo.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#
# Copyright (c) Siemens AG, 2024
#
# Authors:
# Li Hua Qian <[email protected]>
#
# This file is subject to the terms and conditions of the MIT License. See
# COPYING.MIT file in the top-level directory.
#

header:
version: 15

local_conf_header:
hailo: |
IOT2050_META_HAILO = "1"
hailo_override: |
OVERRIDES .= ":meta-hailo"

repos:
meta-iot2050:
layers:
meta-hailo:
huaqianli marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 1 addition & 1 deletion kas/opt/module.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ header:
version: 14

local_conf_header:
no-module: |
module: |
IOT2050_MODULE_FWU = "1"
1 change: 1 addition & 0 deletions kas/opt/secure-boot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ local_conf_header:
secureboot_override: |
OVERRIDES .= ":uefi-stmm"
OVERRIDES .= ":secureboot"
OVERRIDES .= ":pvu-dma"
secure-boot-signer: |
IMAGER_BUILD_DEPS += "ebg-secure-boot-signer"
IMAGER_INSTALL += "ebg-secure-boot-signer"
Expand Down
5 changes: 5 additions & 0 deletions meta-hailo/MAINTAINERS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Maintainers

- Diogo Ivo <[email protected]>
- Felix Moessbauer <[email protected]>
- Li Hua Qian <[email protected]>
25 changes: 25 additions & 0 deletions meta-hailo/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# meta-hailo - Support for HAILO NPU chips

This repo provides recipes to build the kernel and userspace drivers and utilities
(V4.18) for the Hailo-8 NPU chip.
This includes the following main components:

- hailo-pci (OSS kernel module)
- hailo-firmware (proprietary firmware for hailo8 chip)
- hailort (Userspace API)
- hailortcli (tool)
- hailort (service)
- libhailort (c library)
- libhailort-dev (dev package)
- python3-hailort (python library)

> Note: `numpy==1.23.3` is preinstalled. Please do not upgrade it, as
> `hailort v4.18.0` requires this specific version. Upgrading `numpy` may cause
> `pyhailort` to fail to execute.
Comment on lines +16 to +18
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will apt-upgrade updates it? If do, I bet you need a package pin config.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You would need to pin it via apt preferences then. But that is security-wise a bad idea as well, thus this issue of hailo stack needs fixing.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, we are using Hailo V4.18, which officially requires numpy 1.23.3. Even if Hailo fixes this issue, they may not backport the fix to V4.18, making the situation quite challenging. There are significant differences between numpy 1.23 and 1.24, meaning that backporting the patches to V4.18 would require a substantial effort.

So far, IMHO, it would be a good choice for us.

Copy link
Collaborator

@jan-kiszka jan-kiszka Dec 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are already patching hailo for other reasons. Key question is what is breaking when using a (distro-)maintained numpy version in their stack? Your version will not receive any updates (EOL in upstream, not under Debian maintenance), even if there should be bugs or even CVEs.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the issue is caused by the changes mentioned in the numpy 1.24.0 release notes:
https://numpy.org/doc/2.0/release/1.24.0-notes.html

There are also a large number of new and expired deprecations due to changes in promotion and cleanups. This might be called a deprecation release. Highlights are

    Many new deprecations, check them out.

    Many expired deprecations,

    New F2PY features and fixes.

    New “dtype” and “casting” keywords for stacking functions.

Copy link
Collaborator Author

@huaqianli huaqianli Dec 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose Hailort will require a bunch of modifications to eliminate the deprecations. Anyway I'll suggest Hailo to follow the numpy version of the Debian LTS.


## Versioning

This layer is versioned according to the major hailo driver version.
Note, that the kernel ABI is not stable and by that the version of the
userspace components need to perfectly match the version of the firmware
and the kernel module.
15 changes: 15 additions & 0 deletions meta-hailo/conf/layer.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# SPDX-FileCopyrightText: Copyright 2023-2024 Siemens AG
# SPDX-License-Identifier: MIT
BBPATH .= ":${LAYERDIR}"

BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
${LAYERDIR}/recipes-*/*/*.bbappend"

BBFILE_COLLECTIONS += "hailo"
BBFILE_PATTERN_hailo = "^${LAYERDIR}/recipes-"
BBFILE_PRIORITY_hailo = "6"
LAYERVERSION_hailo = "3"

LAYERSERIES_COMPAT_hailo = "next"

LAYERDIR_hailo = "${LAYERDIR}"
94 changes: 94 additions & 0 deletions meta-hailo/recipes-app/hailort/files/debian/control
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
Source: hailort
Maintainer: Felix Moessbauer <[email protected]>, Li Hua Qian <[email protected]>
Section: devel
Priority: optional
Standards-Version: 4.6.1
Build-Depends:
debhelper-compat (=13),
cmake,
bash-completion,
libspdlog-dev,
libprotobuf-dev,
nlohmann-json3-dev,
libgrpc-dev,
libgrpc++-dev,
libreaderwriterqueue-dev:native,
protobuf-compiler,
protobuf-compiler-grpc,
libbenchmark-dev,
dh-python,
libpython3-all-dev,
python3-all-dev:any,
pybind11-dev,
python3-setuptools,
python3-wheel:native,
python3-verboselogs (>= 1.7~),
libeigen3-dev,
pkg-config,
libglib2.0-dev,
libgstreamer1.0-dev,
libgstreamer-plugins-base1.0-dev

Package: hailortcli
Architecture: any
Multi-Arch: foreign
Description: userspace API for running inference on the hailo8 chip (tool)
Depends:
libhailort,
${misc:Depends},
${shlibs:Depends}
Suggests:
bash-completion

Package: hailort
Architecture: any
Multi-Arch: foreign
Description: userspace API for running inference on the hailo8 chip (service)
Depends:
libhailort,
${misc:Depends},
${shlibs:Depends}

Package: libhailort
Architecture: any
Multi-Arch: same
Description: userspace API for running inference on the hailo8 chip (lib)
Depends:
${misc:Depends},
${shlibs:Depends}

Package: python3-hailort
Architecture: any
Multi-Arch: same
Description: userspace API for running inference on the hailo8 chip (python package)
Depends:
${python3:Depends},
${misc:Depends},
${shlibs:Depends},
python3-numpy (= 1.23.3)

Package: libhailort-dev
Architecture: any
Multi-Arch: foreign
Description: userspace API for running inference on the hailo8 chip (dev)
Depends:
libhailort (=${binary:Version}),
${misc:Depends},
${shlibs:Depends}

Package: libgsthailo
Architecture: any
Multi-Arch: same
Description: hailort GStreamer element (HailoNet) (lib)
Depends:
${misc:Depends},
${shlibs:Depends}

Package: libgsthailo-dev
Architecture: any
Multi-Arch: foreign
Description: hailort GStreamer element (HailoNet) (dev)
Depends:
libgsthailo (=${binary:Version}),
${misc:Depends},
${shlibs:Depends}
4 changes: 4 additions & 0 deletions meta-hailo/recipes-app/hailort/files/debian/hailort.install
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
etc/default/hailort_service
usr/bin/hailort_service
# erronous upstream install path, fix here
usr/usr/share/bash-completion/completions/hailortcli usr/share/bash-completion/completions/
16 changes: 16 additions & 0 deletions meta-hailo/recipes-app/hailort/files/debian/hailort.service
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[Unit]
Description=HailoRT service
Documentation=https://github.com/hailo-ai/hailort

[Service]
Type=forking
EnvironmentFile=/etc/default/hailort_service
ExecStart=/usr/bin/hailort_service
Restart=on-failure
RemainAfterExit=yes
PIDFile=/run/hailo/hailort_service.pid
ExecReload=/bin/kill -HUP $MAINPID
ExecStartPost=/bin/sleep 0.1

[Install]
WantedBy=multi-user.target
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
usr/bin/hailortcli
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
usr/include/gstreamer-1.0/gst/hailo/tensor_meta.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
usr/lib/aarch64-linux-gnu/libgsthailo.so
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
usr/include/hailo
usr/lib/*/libhailort.so
usr/lib/*/cmake/HailoRT
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
usr/lib/*/libhailort.so.*
3 changes: 3 additions & 0 deletions meta-hailo/recipes-app/hailort/files/debian/not-installed
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
usr/bin/hailo
usr/lib/systemd/system/hailort.service
usr/lib/python3.*/dist-packages/hailo_tutorials/*
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
usr/lib/python3*/dist-packages/hailort-*.egg-info
usr/lib/python3*/dist-packages/hailo_platform
27 changes: 27 additions & 0 deletions meta-hailo/recipes-app/hailort/files/debian/rules
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#!/usr/bin/make -f

export DEB_BUILD_MAINT_OPTIONS=hardening=+all

%:
dh $@ --with python3

ifeq ($(DEB_HOST_ARCH),arm64)
CMAKE_TOOLCHAIN_FILE := $(CURDIR)/hailort/libhailort/cmake/toolchains/linux.aarch64.cmake
endif

override_dh_auto_configure:
dh_auto_configure --buildsystem=pybuild --sourcedir=hailort/libhailort/bindings/python/platform
dh_auto_configure --buildsystem=cmake -- \
-DHAILO_BUILD_SERVICE=ON \
-DHAILO_BUILD_GSTREAMER=ON \
-DCMAKE_BUILD_TYPE=Release

override_dh_auto_build:
CMAKE_TOOLCHAIN_FILE=$(CMAKE_TOOLCHAIN_FILE) \
dh_auto_build --buildsystem=pybuild --sourcedir=hailort/libhailort/bindings/python/platform
dh_auto_build --buildsystem=cmake

override_dh_auto_install:
CMAKE_TOOLCHAIN_FILE=$(CMAKE_TOOLCHAIN_FILE) \
dh_auto_install --buildsystem=pybuild --sourcedirectory=hailort/libhailort/bindings/python/platform
dh_auto_install --buildsystem=cmake
Loading
Loading