armv7 QNAP Docker: Restarting loop - Core finish process exit code 256 - Core finish process received signal 11 (exit code 139 and exit code 135)

[magicse]

Version 2022.7+ currently boot loop on QNAP NAS ts-231p

INFO: Home Assistant Core finish process exit code 256
INFO: Home Assistant Core finish process received signal 11
…

What version of Home Assistant Core has the issue?
2022.7+

What was the last working version of Home Assistant Core?
2022.6.7

What type of installation are you running?
Home Assistant Container

Additional information
Running on QNAP NAS (TS-231P).

Processor: Annapurna Labs Alpine AL314 Quad-core ARM Cortex-A15 CPU @ 1.70GH

[magicse]

#74707 (comment)

[magicse]

Every version from 2022.7+ is failed to start on Docker (QNAP).

The same error message is occurred.

[14:44:09] INFO: Home Assistant Core finish process exit code 256
[14:44:09] INFO: Home Assistant Core finish process received signal 11
[14:44:10] INFO: Home Assistant Core finish process exit code 256
[14:44:10] INFO: Home Assistant Core finish process received signal 11
[14:44:11] INFO: Home Assistant Core finish process exit code 256
[14:44:11] INFO: Home Assistant Core finish process received signal 11

It is looping infinitely.

There is no resolution in any discussion only reverting to a lower version 2022.6.

[Wetzel402]

#75142

[Wetzel402]

This issue might be resolved finally. If the latest official container still isn't working you can try linuxserver.io's container.

[Gerigot]

@magicse can you try with a pre-release version?
like the new one from today 2023.2.0b9

So that we can ensure that it's related with the problem #75142.

Because the fix was for armv6 and I'm not sure the problem with your device is the same, although it seems similar.

[magicse]

@Gerigot Due to the constant power outage, it's a little difficult for me to do this, but I'll try.

[boyarale]

I have the same problem with Qnap TS-431X.
Tested versions 2023.3.0.dev20230202, 2023.2.0.dev20230120, 2023.2.0b9 and current "stable".

2022.6.7 works well.

[Wetzel402]

@boyarale, have you tried linuxserver.io's image? Please test and report back.

[magicse]

Because the fix was for armv6 and I'm not sure the problem with your device is the same, although it seems similar.

QNAP TS-231p3 is arm32v7

[Gerigot]

I've seen other talking about that problem with QNAP and they solved it using the Linuxserver.io image, unfortunately I don't have a QNAP so I can't investing the problem

[magicse]

It may be possible to somehow analyze what has changed during the transition from version 2022.6.7. There should be some explanation for this phenomenon.

[Gerigot]

I think the problem relays on some libraries that HA uses not on the core itself so it's very difficult investigate without a device.
I fixed the problem for "pi zero" because I'm using it on that so I could test if the solution was good.

[magicse]

like the new one from today 2023.2.0b9
My test config

version: '3.8'
services:
  homeassistant:
    restart: always
    image: homeassistant/home-assistant:2023.2.0b9
    network_mode: host
    privileged: true
    environment:
      - DISABLE_JEMALLOC=true
      - TZ=Europe/Berlin
    volumes:
      - /share/Container/hass-config:/config

Error log

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun home-assistant (no readiness notification)
s6-rc: info: service legacy-services successfully started
[17:41:21] INFO: Home Assistant Core finish process exit code 256
[17:41:21] INFO: Home Assistant Core finish process received signal 11
[17:41:22] INFO: Home Assistant Core finish process exit code 256
[17:41:22] INFO: Home Assistant Core finish process received signal 11
[17:41:23] INFO: Home Assistant Core finish process exit code 256

[magicse]

bash-5.1# python3 
Python 3.10.7 (main, Nov 24 2022, 13:02:43) [GCC 11.2.1 20220219] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import jwt
Segmentation fault
bash-5.1# 

bash-5.1# python3 
Python 3.10.7 (main, Nov 24 2022, 13:02:43) [GCC 11.2.1 20220219] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from cryptography.hazmat.primitives.asymmetric import ec, padding
Segmentation fault
bash-5.1# 

bash-5.1# gdb --args python -c "import sys, numpy; print(numpy.__version__, sys.version)"
GNU gdb (GDB) 11.2
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "armv7-alpine-linux-musleabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python...
(No debugging symbols found in python)
(gdb) r
Starting program: /usr/local/bin/python -c import\ sys,\ numpy\;\ print\(numpy.__version__,\ sys.version\)

Program received signal SIGSEGV, Segmentation fault.
0x75fb792c in ?? () from /lib/ld-musl-armhf.so.1

apk --print-arch' gives armv7, but /lib/libc.musl-armv7.so.1 is link to /lib/ld-musl-armhf.so.1
it is somewhat strange

[magicse]

#75142 (comment)
Just for completeness, on 2022.6.7 both imports, numpy and jwt, work without issues:

[magicse]

#75142 (comment)

[magicse]

@Wetzel402

This issue might be resolved finally. If the latest official container still isn't working you can try linuxserver.io's container.

linuxserver.io's container Home Assistant 2023.2.1 work well.

[Wetzel402]

I was hoping that @Gerigot's fix would also correct the issue with QNAP but it appears that isn't the case. More investigation and research is needed...

[magicse]

@Wetzel402 No problem any needed logs and tests for analytics we could make.

[Wetzel402]

@magicse, right now I think comparing the repositories to find differences would be a good start. Why does linuxserver.io's image run when the official does not?

[Wetzel402]

I'm already noticing that linuxserver.io has a dedicated dockerfile for armhf.

Edit: They also use their own wheels. This makes me suspect it is a wheels issue as some have previously suspected...

One more edit... Linuxserver is using their own Alpine 3.16 base image.

[Wetzel402]

It could be worth trying to build the docker image using the Linuxserver base image with offical HA wheels as well as vice versa.

[Gerigot]

It is really a strange behavior and unfortunately I don't have an ARMv7 to make some test myself it will be really difficult to find out what's going on

[magicse]

@Wetzel402

Edit: They also use their own wheels. This makes me suspect it is a wheels issue as some have previously suspected...

May be you are right and they use different wheels for example for cryptography.
I see by names that they didn't use musllinux for arch armv7l .
musllinux used for aarch64 and x86_64 arch.
Also aarch64 and x86_64 arch builded for Python 3.6, and armv7l armv8l builded for Python 3.10

[cryptography-39.0.0-cp310-cp310-linux_armv7l.whl](https://wheels.linuxserver.io/alpine-3.16/cryptography-39.0.0-cp310-cp310-linux_armv7l.whl)
[cryptography-39.0.0-cp310-cp310-linux_armv8l.whl](https://wheels.linuxserver.io/alpine-3.16/cryptography-39.0.0-cp310-cp310-linux_armv8l.whl)
[cryptography-39.0.0-cp36-abi3-musllinux_1_1_aarch64.whl](https://wheels.linuxserver.io/alpine-3.16/cryptography-39.0.0-cp36-abi3-musllinux_1_1_aarch64.whl)
[cryptography-39.0.0-cp36-abi3-musllinux_1_1_x86_64.whl](https://wheels.linuxserver.io/alpine-3.16/cryptography-39.0.0-cp36-abi3-musllinux_1_1_x86_64.whl)

Debugging of official image give me error in ld-musl-armhf.so.1 while importing (for exmaple) numpy or cryptography

(gdb) r
bash-5.1# gdb --args python -c "import sys, numpy; print(numpy.__version__, sys.version)"
Starting program: /usr/local/bin/python -c import\ sys,\ numpy\;\ print\(numpy.__version__,\ sys.version\)

Program received signal SIGSEGV, Segmentation fault.
0x75fb792c in ?? () from /lib/ld-musl-armhf.so.1

I think we could start from this point.... For example when simple importing of numpy inside of container will be without segmentation fault.

**bash-5.1# python3 
Python 3.10.7 (main, Nov 24 2022, 13:02:43) [GCC 11.2.1 20220219] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Segmentation fault
bash-5.1#** 

[magicse]

QNAP armv7l
May be problem with armv7l <-- > armv7hf and package architecture (armv7l) does not match system (armhf)
also it not armv7hf-musl armv7hf armv7l-musl

bash-5.1# cat /proc/cpuinfo
processor : 0
model name : Annapurna Labs Alpine AL314 Quad-core ARM Cortex-A15 CPU @ 1.70GHz
Speed : 1.7GHz
Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x2
CPU part : 0xc0f
CPU revision : 4

[magicse]

@Gerigot @Wetzel402
I have tried build wheels for armv7 on Ubuntu 64 with next command

export ARCH=armv7
sudo docker build  --build-arg CPYTHON_ABI=cp310 --build-arg BUILD_FROM=ghcr.io/home-assistant/wheels/${ARCH}/musllinux_1_2/cp310:dev --build-arg BUILD_ARCH=${ARCH} --tag wheel-builder:${ARCH} .

And all time got error about import cmake.

After that I checked Dockerfile and requirements_cp310.txt and files at this link "https://wheels.home-assistant.io/musllinux/" and there is different versions of cmake.
In requirements_cp310.txt version of cmake 3.25.2 and at link "https://wheels.home-assistant.io/musllinux/" cmake for armv7 is 3.22.2.
After changing version of cmake in file requirements_cp310.txt to 3.22.2 - all builds well.

Could you check this problem with versions?

[Wetzel402]

@magicse,

The cmake version was only changed a couple of weeks ago so I'm not sure that's our problem. Theoretically something changed between the June and July builds of HA and we need to find it.

[magicse]

I think need dig in musllinux becouse i get seg fault during debug from ld-musl-armhf.so.1

gdb --args python -c "import sys, numpy; print(numpy.__version__, sys.version)"
Starting program: /usr/local/bin/python -c import\ sys,\ numpy\;\ print\(numpy.__version__,\ sys.version\)
Program received signal SIGSEGV, Segmentation fault.
0x75fb792c in ?? () from /lib/ld-musl-armhf.so.1

Also additional information abut cpu from here ARMonQEMUforDebianUbuntu.md
ARMv7 CPU which Debian calls as armhf (ARM hard float) and Cortex-A8, A9, A15 are all ARMv7.
ARMv6 CPU which Debian calls as armel and ARMv5, v6. Raspberry Pi uses ARMv6. In this case, the cpu is arm1176

And addidtiona information ARM v6 and v7 target musl
Building a cross compiler targeting musl libc MUSL LIBC for ARMv6 and ARMv7 Building
Unable run grpcio wheel on alpine arvm7 run grpcio wheel on alpine arvm7

[magicse]

some interesting article
https://lwn.net/Articles/822868/

[magicse]

Hi @Wetzel402
I tried installing the latest original home assistant on a QNAP TS-131K (same CPU as TS-231p3) but with 1 Gb of memory and it still works with a 4K page size. And home assistant last version started and works without problems. So the problem is with the page size. Now I'm trying to compile the Alpine Linux 3.18 kernel inside the Alpine Linux 3.17 container, let's see what happens.

I also wrote to QNAP support, but they are stubborn and say that 32K is good and they are not interested in third-party containers.

Commands to build Alpine inside container

adduser -D builder
addgroup builder abuild
su builder
cd /home/builder

git clone git://git.alpinelinux.org/aports

cd aports
git checkout -b custom-alpine
cd main/linux-lts
abuild-keygen -a
echo "PACKAGER_PRIVKEY=\"/home/builder/.abuild/your-key.rsa\"" >> /home/builder/.abuild/abuild.conf
cd /home/builder/aports/main/linux-lts
abuild -r
cd  /home/build/aports/scripts
./mkimage.sh --tag custom-alpine --repository /home/build/aports/main/linux-lts

Once the build process is complete, the customized ISO image in the output/ directory.

As I understand it, PAGE SIZE is set through the definition of PAGE_SHIFT in the configuration files.. Where PAGE_SHIFЕ = 12 by default 2^12 = 4K (4096). For 32K must be 15, 2^15=32768

[Wetzel402]

I also wrote to QNAP support, but they are stubborn and say that 32K is good and they are not interested in third-party containers.

I wish for those of us who don't need the larger page size they would offer an alternate kernel.

[magicse]

I also wrote to QNAP support, but they are stubborn and say that 32K is good and they are not interested in third-party containers.

I wish for those of us who don't need the larger page size they would offer an alternate kernel.

totally agree with you!

[magicse]

hi @Wetzel402 interesting moment why 3.17 Alpine Linux work under 32k and 4K Page size but 3.18 with 4 K Page Size only.
I think may be need compare built config files from Alpine 3.17 and Alpine 3.18
With command
zcat /proc/config.gz
But I could execute this zcat /proc/config.gz only in 3.17 because 3.18 didn't work yet ))

[Wetzel402]

I recently picked up an x86 based 8 bay QNAP NAS and will be migrating this weekend. I can keep my old unit for now to continue troubleshooting this problem. If I run zcat /proc/config.gz using my new machine on Alpine 3.18 will that provide the data we need or does it need to be the armv7 based NAS with 32k page size?

[magicse]

If I run zcat /proc/config.gz using my new machine on Alpine 3.18 will that provide the data we need or does it need to be the armv7 based NAS with 32k page size?

Will be better if this will be armv7 based NAS with 4K page size. I will try ask about this my friends because they have TS-131K with page size 4K and worked Home Assistant with Alpine 3.18

[magicse]

Hi @Wetzel402
Here strings that have differences from zcat /proc/config.gz

Alpine Linux 3.18 for TS-131K with 4K page size

# CONFIG_ARM_PAGE_SIZE_32KB is not set
CONFIG_ARM_PAGE_SIZE_4KB=y
CONFIG_ARM_PAGE_SIZE_LARGE_SHIFT=12

# CONFIG_HIGHMEM is not set
CONFIG_CMDLINE="pci=pcie_bus_perf console=ttyS0,115200 root=/dev/ram mtdparts=mx_nand:32M(boot1_kernel),216M(boot1_rootfs2),32M(boot2_kernel),216M(boot2_rootfs2),15M(config);spi0.0:1088K(loader)ro,384K(env) zswap.enabled=0 zswap.compressor=lz4 memmap=2M$0x7000000 ramoops.mem_address=0x7000000 ramoops.mem_size=0x200000 ramoops.console_size=0x100000"

Alpine Linux 3.17 from TS-231P with 32K page size

CONFIG_ARM_PAGE_SIZE_32KB=y
# CONFIG_ARM_PAGE_SIZE_4KB is not set
CONFIG_ARM_PAGE_SIZE_LARGE_SHIFT=15

CONFIG_HIGHMEM=y
# CONFIG_HIGHPTE is not set
CONFIG_CMDLINE="pci=pcie_bus_perf console=ttyS0,115200 root=/dev/ram mtdparts=mx_nand:32M(boot1_kernel),216M(boot1_rootfs2),32M(boot2_kernel),216M(boot2_rootfs2),15M(config);spi0.0:1088K(loader)ro,384K(env) vmalloc=560M zswap.enabled=0 zswap.compressor=lz4 memmap=2M$0x7000000 ramoops.mem_address=0x7000000 ramoops.mem_size=0x200000 ramoops.console_size=0x100000"

[Wetzel402]

Additional info from the musl team...

This doesn't help immediately, but a major ingredient to fix this
situation would be getting the kernel to stop doing the wrong thing.
Right now, it's ignoring the fact that the ELF program header
constraints are incompatible with mmap given the oversized system
pagesize, and just incorrectly mapping the executable and trying to
run it anyway, whereby it blows up.
The right thing to do would be either to fail with ENOEXEC in this
case, or when mmap with the required offset constraint fails, falling
back to making an anonymous map and copying the whole content of the
loadable segment into that (no COW sharing). The latter is really not
all that bad for got/data/etc. mappings which you expect will be dirty
(modified) anyway.
BTW the former choice (ENOEXEC) would allow doing the latter in
userspace with a binfmt_misc loader.

[redhell]

Hello together,
I tried to build the image and got one (more or less) working image.

Building from the docker file by @Wetzel402 leads to starting image but it will throw some errors and the mobile app isn't working.

024-01-31 00:16:23.772 ERROR (MainThread) [homeassistant.setup] Setup failed for cloud: Unable to import component: No module named 'webrtc_noise_gain_cpp'
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/homeassistant/setup.py", line 215, in _async_setup_component
    component = integration.get_component()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/homeassistant/loader.py", line 818, in get_component
    ComponentProtocol, importlib.import_module(self.pkg_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/lib/python3.11/site-packages/homeassistant/components/cloud/__init__.py", line 35, in <module>
    from . import account_link, http_api
  File "/usr/local/lib/python3.11/site-packages/homeassistant/components/cloud/http_api.py", line 19, in <module>
    from homeassistant.components import assist_pipeline, conversation, websocket_api
  File "/usr/local/lib/python3.11/site-packages/homeassistant/components/assist_pipeline/__init__.py", line 14, in <module>
    from .pipeline import (
  File "/usr/local/lib/python3.11/site-packages/homeassistant/components/assist_pipeline/pipeline.py", line 19, in <module>
    from webrtc_noise_gain import AudioProcessor
  File "/usr/local/lib/python3.11/site-packages/webrtc_noise_gain/__init__.py", line 1, in <module>
    from webrtc_noise_gain_cpp import AudioProcessor
ModuleNotFoundError: No module named 'webrtc_noise_gain_cpp'
2024-01-31 00:16:27.394 ERROR (MainThread) [homeassistant.setup] Setup failed for assist_pipeline: Unable to import component: No module named 'webrtc_noise_gain_cpp'
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/homeassistant/setup.py", line 215, in _async_setup_component
    component = integration.get_component()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/homeassistant/loader.py", line 818, in get_component
    ComponentProtocol, importlib.import_module(self.pkg_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/lib/python3.11/site-packages/homeassistant/components/assist_pipeline/__init__.py", line 14, in <module>
    from .pipeline import (
  File "/usr/local/lib/python3.11/site-packages/homeassistant/components/assist_pipeline/pipeline.py", line 19, in <module>
    from webrtc_noise_gain import AudioProcessor
  File "/usr/local/lib/python3.11/site-packages/webrtc_noise_gain/__init__.py", line 1, in <module>
    from webrtc_noise_gain_cpp import AudioProcessor
ModuleNotFoundError: No module named 'webrtc_noise_gain_cpp'
2024-01-31 00:16:32.036 ERROR (MainThread) [homeassistant.setup] Setup failed for mobile_app: Unable to import component: No module named 'webrtc_noise_gain_cpp'
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/homeassistant/setup.py", line 215, in _async_setup_component
    component = integration.get_component()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/homeassistant/loader.py", line 818, in get_component
    ComponentProtocol, importlib.import_module(self.pkg_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/lib/python3.11/site-packages/homeassistant/components/mobile_app/__init__.py", line 5, in <module>
    from homeassistant.components import cloud, notify as hass_notify
  File "/usr/local/lib/python3.11/site-packages/homeassistant/components/cloud/__init__.py", line 35, in <module>
    from . import account_link, http_api
  File "/usr/local/lib/python3.11/site-packages/homeassistant/components/cloud/http_api.py", line 19, in <module>
    from homeassistant.components import assist_pipeline, conversation, websocket_api
  File "/usr/local/lib/python3.11/site-packages/homeassistant/components/assist_pipeline/__init__.py", line 14, in <module>
    from .pipeline import (
  File "/usr/local/lib/python3.11/site-packages/homeassistant/components/assist_pipeline/pipeline.py", line 19, in <module>
    from webrtc_noise_gain import AudioProcessor
  File "/usr/local/lib/python3.11/site-packages/webrtc_noise_gain/__init__.py", line 1, in <module>
    from webrtc_noise_gain_cpp import AudioProcessor
ModuleNotFoundError: No module named 'webrtc_noise_gain_cpp'
2024-01-31 00:16:32.160 ERROR (MainThread) [homeassistant.setup] Unable to set up dependencies of default_config. Setup failed for dependencies: assist_pipeline, cloud, mobile_app
2024-01-31 00:16:32.160 ERROR (MainThread) [homeassistant.setup] Setup failed for default_config: (DependencyError(...), 'Could not setup dependencies: assist_pipeline, cloud, mobile_app')
ERROR: Cannot set filter: libpcap is not available. Cannot compile filter !
2024-01-31 00:16:33.927 ERROR (SyncWorker_4) [scapy.runtime] Cannot set filter: libpcap is not available. Cannot compile filter !
2024-01-31 00:16:34.118 ERROR (SyncWorker_4) [homeassistant.components.dhcp] Cannot watch for dhcp packets without a functional packet filter: libpcap is not available. Cannot compile filter !

After this I tried to build it against the last Version of 2023 and figured out that it can't get build because of the outdated rustc needed for orison. I tried an orjson wheel from piwheels and it was build successfully but now when starting the container I got following error and it won't start:

homeassistant2  | ModuleNotFoundError: No module named 'orjson.orjson'
homeassistant2  | Traceback (most recent call last):
homeassistant2  |   File "/usr/local/bin/hass", line 8, in <module>
homeassistant2  |     sys.exit(main())
homeassistant2  |              ^^^^^^
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/__main__.py", line 173, in main
homeassistant2  |     args = get_arguments()
homeassistant2  |            ^^^^^^^^^^^^^^^
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/__main__.py", line 81, in get_arguments
homeassistant2  |     from . import config as config_util
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/config.py", line 25, in <module>
homeassistant2  |     from . import auth
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/auth/__init__.py", line 13, in <module>
homeassistant2  |     from homeassistant import data_entry_flow
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/data_entry_flow.py", line 15, in <module>
homeassistant2  |     from .core import HomeAssistant, callback
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/core.py", line 83, in <module>
homeassistant2  |     from .helpers.json import json_dumps
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/helpers/json.py", line 11, in <module>
homeassistant2  |     import orjson
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/orjson/__init__.py", line 3, in <module>
homeassistant2  |     from .orjson import *
homeassistant2  | ModuleNotFoundError: No module named 'orjson.orjson'
homeassistant2  | Traceback (most recent call last):
homeassistant2  |   File "/usr/local/bin/hass", line 8, in <module>
homeassistant2  |     sys.exit(main())
homeassistant2  |              ^^^^^^
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/__main__.py", line 173, in main
homeassistant2  |     args = get_arguments()
homeassistant2  |            ^^^^^^^^^^^^^^^
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/__main__.py", line 81, in get_arguments
homeassistant2  |     from . import config as config_util
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/config.py", line 25, in <module>
homeassistant2  |     from . import auth
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/auth/__init__.py", line 13, in <module>
homeassistant2  |     from homeassistant import data_entry_flow
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/data_entry_flow.py", line 15, in <module>
homeassistant2  |     from .core import HomeAssistant, callback
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/core.py", line 83, in <module>
homeassistant2  |     from .helpers.json import json_dumps
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/helpers/json.py", line 11, in <module>
homeassistant2  |     import orjson
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/orjson/__init__.py", line 3, in <module>
homeassistant2  |     from .orjson import *
homeassistant2  | ModuleNotFoundError: No module named 'orjson.orjson'
homeassistant2  | Traceback (most recent call last):
homeassistant2  |   File "/usr/local/bin/hass", line 8, in <module>
homeassistant2  |     sys.exit(main())
homeassistant2  |              ^^^^^^
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/__main__.py", line 173, in main
homeassistant2  |     args = get_arguments()
homeassistant2  |            ^^^^^^^^^^^^^^^
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/__main__.py", line 81, in get_arguments
homeassistant2  |     from . import config as config_util
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/config.py", line 25, in <module>
homeassistant2  |     from . import auth
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/auth/__init__.py", line 13, in <module>
homeassistant2  |     from homeassistant import data_entry_flow
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/data_entry_flow.py", line 15, in <module>
homeassistant2  |     from .core import HomeAssistant, callback
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/core.py", line 83, in <module>
homeassistant2  |     from .helpers.json import json_dumps
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/homeassistant/helpers/json.py", line 11, in <module>
homeassistant2  |     import orjson
homeassistant2  |   File "/usr/local/lib/python3.11/site-packages/orjson/__init__.py", line 3, in <module>
homeassistant2  |     from .orjson import *
homeassistant2  | ModuleNotFoundError: No module named 'orjson.orjson'

Do you have any clue how to get those dependencies satisfied?

I tried to install those dependencies at the end of the docker file:

RUN pip install --extra-index-url=https://www.piwheels.org/simple orjson==3.9.9 libpcap

Edit:
I run a TS-431p3 with 32k page size.

[magicse]

Edit:
I run a TS-431p3 with 32k page size.

What version of Alpine ?

[redhell]

Docker image:
FROM --platform=linux/arm/v7 python:3.11.7-alpine3.17

Zcat: zcat /proc/config.gz

CONFIG_ARM_PAGE_SIZE_LARGE=y
CONFIG_ARM_PAGE_SIZE_32KB=y
# CONFIG_ARM_PAGE_SIZE_4KB is not set
CONFIG_ARM_PAGE_SIZE_LARGE_SHIFT=15

CONFIG_HIGHMEM=y
CONFIG_CMDLINE="pci=pcie_bus_perf console=ttyS0,115200 root=/dev/ram mtdparts=mx_nand:32M(boot1_kernel),216M(boot1_rootfs2),32M(boot2_kernel),216M(boot2_rootfs2),15M(config);spi0.0:1088K(loader)ro,384K(env) vmalloc=560M zswap.enabled=0 zswap.compressor=lz4 memmap=2M$0x7000000 ramoops.mem_address=0x7000000 ramoops.mem_size=0x200000 ramoops.console_size=0x100000"

[magicse]

Docker image:
FROM --platform=linux/arm/v7 python:3.11.7-alpine3.17

Try create simple minimal container of Alpine Linux 3.17 5Mb size from Docker hub. dont forget mount /config to config volume.
Login in to container and try simple make this commands with sh and bash

apk add bash
bash
apk add g++ gcc make
apk add libcap libpcap-dev
apk add python3
python3 -m ensurepip --upgrade
apk add git
apk add python3-dev libffi-dev libftdi1-dev bzip2-dev openssl-dev cargo jpeg-dev zlib-dev
pip3 install aiohttp
pip3 install ffmpeg
pip3 install libpcap
pip3 install tzdata
pip3 install PyNaCl
pip3 install Pillow
pip3 install numpy
pip3 install git+https://github.com/boto/botocore
pip3 install homeassistant
hass --config /config

Wait until home assistant build needed dependencies for first start (it could take some time)

Home Assistant 2023.7.3

[Wetzel402]

After this I tried to build it against the last Version of 2023 and figured out that it can't get build because of the outdated rustc needed for orison. I tried an orjson wheel from piwheels and it was build successfully but now when starting the container I got following error and it won't start:

In my experience any version >2023.10.3 will fail to build on Alpine 3.17 due to dependency issues. Until we can get Alpine 3.18+ running on QNAP with 32k page size we are dead in the water. Feel free to help over on the Gitlab issue to resolve the problem.

[redhell]

So I managed to bring up 2023.10.5 (lsat version with orjson 3.9.7)
Additionalls:
apk add ffmpeg ffmpeg-dev openblas openblas-dev cmake
pip install ha-av==10.1.1 numpy=1.2.24(?) PyTurboJPEG
pip install --no-binary=:all: webrtc-noise-gain

[Mactomik]

Is running Alpine 3.18+ on QNAP with 32k page size reached to dead end or is there still some hope left?

[magicse]

Is running Alpine 3.18+ on QNAP with 32k page size reached to dead end or is there still some hope left?

I no longer have the strength. I'm tired of fighting. Of course you need to write to the Alpine developers about supporting containers for ARMs with a page size of 32k

[magicse]

Hi @Wetzel402 @Mactomik

(2/3) Upgrading busybox-binsh (1.35.0-r29 -> 1.36.1-r5)
(3/3) Upgrading ssl_client (1.35.0-r29 -> 1.36.1-r5)
Executing busybox-1.36.1-r5.trigger
OK: 5 MiB in 15 packages
/# apk upgrade musl --repository=https://dl-cdn.alpinelinux.org/alpine/v3.18/main --repository=https://dl-cdn.alpinelinux.org/alpine/v3.18/community
(1/1) Upgrading musl (1.2.3-r5 -> 1.2.4-r2)
OK: 5 MiB in 15 packages
/# apk upgrade rust --repository=https://dl-cdn.alpinelinux.org/alpine/v3.18/main --repository=https://dl-cdn.alpinelinux.org/alpine/v3.18/community
Segmentation fault

Building 3.18-stable packages under Apline 3.17

apk update
apk upgrade
apk add bash
apk add mc
apk add doas
apk add alpine-sdk

adduser -D magic
addgroup magic abuild
passwd magic

# <editor> /etc/doas.d/doas.conf
#Add the following line and save the file:
permit persist :abuild

# <editor> /etc/apk/repositories
#Add the following line at begin and save the file:
/home/magic/packages/main

su - magic
cd /home/magic

#Generate keys
abuild-keygen -a -i

git clone https://gitlab.alpinelinux.org/alpine/aports.git
cd aports
git checkout 3.18-stable

#Build custom packages
cd /home/magic/aports/main/musl
abuild -r
cd /home/magic/aports/main/glib
abuild -r
cd /home/magic/aports/main/busybox
abuild -r

exit

apk update
apk upgrade --no-self-upgrade  --simulate
apk upgrade --no-self-upgrade

And i get fully working musl, glib and busybox QNAP with 32k page size

c35d59e2d2cb:/# apk info -vv --quiet | grep musl
musl-1.2.4-r2
musl-utils-1.2.4-r2
musl-dev-1.2.4-r2
musl-fts-1.2.7-r5
musl-obstack-1.2.3-r2
c35d59e2d2cb:/# apk info -vv --quiet | grep busybox
busybox-1.36.1-r5
busybox-binsh-1.36.1-r5
c35d59e2d2cb:/# 
c35d59e2d2cb:/# getconf PAGESIZE
32768
c35d59e2d2cb:/# 

In same way you could build and install other packages from 3.18-stable source localy.
Or may be need build all packages and create custom repository, to allow make full update from Alpine 3.17 to 3.18 without rebuilding every time.
To bouild all packages you could use buildrepo
buildrepo main -a /home/magic/aports -k
Some helpful links about this
Building / consuming alpine Linux packages inside containers and images
CREATING AN ALPINE LINUX REPOSITORY
BUILD ALPINE PACKAGES IN A TEMPORARY CONTAINER

Also for experiments i propose use static apk and with key --no-self-upgrade
apk add busybox-static apk-tools-static

And if something wrong for example after command
apk.static upgrade musl --no-self-upgrade --repository=https://dl-cdn.alpinelinux.org/alpine/v3.18/main --repository=https://dl-cdn.alpinelinux.org/alpine/v3.18/community

You always could rollback with command
apk.static upgrade --no-self-upgrade --available

[Mactomik]

Hi, thanks.

Had an hour to kill, gived it os hot building 3.18-stable packages under Apline 3.17
On step #Build custom packages runing abuild -r I got unfotunately stuck with:
abuild-sign: votmed: File not found
ERROR: musl: all failed
("votmed" was random name I put in when I was promted to input my keyfile name by abuild-keygen -a -i command)
checked I had keyfile present in user home directory and /etc/apk/keys so according to https://wiki.alpinelinux.org/wiki/Include:Abuild-keygen I should be fine. Any idea what may I be doing wrong?

[magicse]

Hi @Mactomik
check again that you create user for abuild and give all needed access to user. And abuild run not from root

For example

adduser -D magic
addgroup magic abuild
passwd magic

# <editor> /etc/doas.d/doas.conf
#Add the following line and save the file:
permit persist :abuild

# <editor> /etc/apk/repositories
#Add the following line at begin and save the file:
/home/magic/packages/main 

su - magic
cd /home/magic

#Generate keys
abuild-keygen -a -i

If You want we could try to work together with anydesk

[Mactomik]

Hi @magicse
Thanks for reaching out once again. I created a new user, generated keys to the new user and now building 3.18 packages works like ccharm so that problem is solved.

I am kind of monkey repeating steps described here with not that clear undeststanding linux packet building and dependency issues with the aim of getting more up to date HA (I have currently 2023.10.3) working on my 431-xeu 32k system page size. Can you comment now (with the new ability to build 3.18 packages) is my goal of getting more up to date HA running any closer than before and if yes what would be the next steps? I thoght of taking Wetzel402 docker file packages installed on alpine 3.17, buld them from 3.18 and see if this get me anywhere. Does this make any sense? Noted that HA requires python 3.12 and alpine 3.18 has python 3.11 package so the latest HA would be out of reach for me even with alpine 3.18 built packages?

[magicse]

I am kind of monkey repeating steps described here with not that clear undeststanding linux packet building and dependency issues with the aim of getting more up to date HA (I have currently 2023.10.3) working on my 431-xeu 32k system page size. Can you comment now (with the new ability to build 3.18 packages) is my goal of getting more up to date HA running any closer than before and if yes what would be the next steps? I thoght of taking Wetzel402 docker file packages installed on alpine 3.17, buld them from 3.18 and see if this get me anywhere. Does this make any sense? Noted that HA requires python 3.12 and alpine 3.18 has python 3.11 package so the latest HA would be out of reach for me even with alpine 3.18 built packages?

hi @Mactomik
I'm now trying to figure out this problem and trying to find a way to get a new Alpina on QNAP. Since the Alpina developers do not give a clear answer about supporting 32K page for 32bit Arm like armv7.
I can’t understand why Alpina 3.17 works without problems on 32K pages, but Apline 3.18 and higher does not.

If you want to compile the entire packages repository (build all packages) then you can use the following command.... taking into account that the user is magic.

su -
apk add aports-build
su - magic
buildrepo main -a /home/magic/aports  -k

also python 3.12 you could build by yourself. Somewhere above in this post I posted commands for compiling Python.

Also i try experiment in emulation on x86 like this

sudo apt-get install qemu binfmt-support qemu-user-static
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
for 32K PAGE SIZE
docker run -e "QEMU_CPU=cortex-a15" -e "QEMU_PAGESIZE=32768" --rm --platform linux/arm/v7 -it alpine:3.17.0 /bin/sh
docker run -e "QEMU_CPU=cortex-a15" -e "QEMU_PAGESIZE=32768" --rm --platform linux/arm/v7 -it alpine:3.18.0 /bin/sh

for 4K PAGE SIZE
docker run -e "QEMU_CPU=cortex-a15" -e "QEMU_PAGESIZE=4096" --rm --platform linux/arm/v7 -it alpine:3.17.0 /bin/sh
docker run -e "QEMU_CPU=cortex-a15" -e "QEMU_PAGESIZE=4096" --rm --platform linux/arm/v7 -it alpine:3.18.0 /bin/sh

alpinelinux/docker-alpine#342 (comment)

[Mactomik]

Just to give update of my results for the pursue of installing newer than 2023.10.5 HA on qnap ts-431Xeu containerstation using 32K page file till we have working Alpine 3.18 or HA container. I have managed to get HA 2024.4.3 Core version running (with some minor python packet erros to tweak further). I ended up using Ubuntu 22.04 instead of Alpine as with Python I had difficulties to update to python 3.12.x. eather using instructions above to buld 3.18 packets for Alpine 3.17 or building python from source code. Can be that the issues were more related to my limited skills than Alpine. I installed Python 3.12.3 from DeadSnakes PPA on Ubuntu 22.04 (used Qnap default app template) and followed HA core instructions and installed additional packages whenever I was missing something. If anybody is intrested I can share more detailed steps.

[issue-triage-workflows]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.