[FATAL]: <jemalloc>: Unsupported system page size

[GittyGitGitGitty]

Describe the bug

Introduced in 0.40.0, as noted here:

<jemalloc>: Unsupported system page size

This is a known issue with jemalloc on Raspberry Pi 5 (aarch64).

Workaround

Raspberry Pi 5 users can load the rpi-v8 kernel (4k page size kernel) by applying the following to /boot/firmware/config.txt and rebooting:

kernel=kernel8.img

How to reproduce it

docker run --rm -it \
  --privileged \
  -v /root/.falco:/root/.falco \
  -v /proc:/host/proc:ro \
  -v /boot:/host/boot:ro \
  -v /lib/modules:/host/lib/modules:ro \
  -v /usr:/host/usr:ro \
  -v /etc:/host/etc:ro \
  falcosecurity/falco-driver-loader:latest ebpf

docker run -it -d \
  --name falco \
  --cap-drop all \
  --cap-add sys_admin \
  --cap-add sys_resource \
  --cap-add sys_ptrace \
  -v /var/run/docker.sock:/host/var/run/docker.sock \
  -v /root/.falco:/root/.falco:ro \
  -v /proc:/host/proc:ro \
  -v /etc:/host/etc:ro \
  -v /home/falco/config/config.yaml:/etc/falco/config.d/config.yaml:ro \
  -v /home/falco/rules/falco_rules.local.yaml:/etc/falco/falco_rules.local.yaml:ro \
  --restart unless-stopped \
  falcosecurity/falco:latest-debian falco -o engine.kind=ebpf

Expected behaviour

Normal operation.

Environment

• Raspberry Pi 5
• Raspberry Pi OS (Debian GNU/Linux 12 (bookworm))
• Linux version 6.6.62+rpt-rpi-2712 ([email protected]) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT Debian 1:6.6.62-1+rpt1 (2024-11-25)

Installation method

docker run (see above)

Comments

Perhaps falco security can revert their jemalloc usage? This is currently a performance hit for users who have 16k page size kernels available.

[FedeDP]

Hi! Thanks for opening this bug report!
Actually, we decided to go with jemalloc to try to fix #2495, that is hitting us since 2023 :/ the situation should improve with jemalloc.

Perhaps falco security can revert their jemalloc usage? This is currently a performance hit for users who have 16k page size kernels available.

If jemalloc helped us in reducing/fixing #2495, i don't think we should revert it. At the same time, i agree that we should find a solution; i see 2 solutions:

• either we build also non-jemalloc packages and images
• or we runtime load jemalloc through LD_PRELOAD instead of building it as a library inside Falco, and we add an env variable to our docker images to disable the LD_PRELOAD.

The latter is obviously better, BUT then the non-docker installation (ie: host packages) would not automatically use jemalloc.

Let's hear more points from other maintainers: @falcosecurity/falco-maintainers

[FedeDP]

/milestone 0.41.0

This is currently a performance hit for users who have 16k page size kernels available.

Also, can you share how big of an impact is loading the 4k page size kernel?

[GittyGitGitGitty]

/milestone 0.41.0

This is currently a performance hit for users who have 16k page size kernels available.

Also, can you share how big of an impact is loading the 4k page size kernel?

My performance loss for falco appears negligible but I've not done any formal profiling (sorry, I don't have the time). It's the other server processes I'm running that I'm concerned about since 16k runs smoother and with no complaints about kernel memory allocation as seen with the 4k kernel (a raspberry pi concern with another app, not falco).

I've read on the internet that the delta can be around ~7% which feels about right. Maybe others will disagree though, I don't know.

Hi! Thanks for opening this bug report! Actually, we decided to go with jemalloc to try to fix #2495, that is hitting us since 2023 :/ the situation should improve with jemalloc.

Perhaps falco security can revert their jemalloc usage? This is currently a performance hit for users who have 16k page size kernels available.

If jemalloc helped us in reducing/fixing #2495, i don't think we should revert it. At the same time, i agree that we should find a solution; i see 2 solutions:

• either we build also non-jemalloc packages and images
• or we runtime load jemalloc through LD_PRELOAD instead of building it as a library inside Falco, and we add an env variable to our docker images to disable the LD_PRELOAD.

The latter is obviously better, BUT then the non-docker installation (ie: host packages) would not automatically use jemalloc.

Let's hear more points from other maintainers: @falcosecurity/falco-maintainers

I'd personally avoid LD_PRELOAD for security reasons, especially anything related to memory allocation, but maybe it would be managed within a context I don't know about.

[Darkness4]

Is it possible to simply add support for a larger kernel page?

--- a/cmake/modules/jemalloc.cmake
+++ b/cmake/modules/jemalloc.cmake
@@ -40,12 +40,17 @@ else()
 	set(JEMALLOC_SRC "${PROJECT_BINARY_DIR}/jemalloc-prefix/src")
 	set(JEMALLOC_LIB "${JEMALLOC_SRC}/jemalloc/lib/libjemalloc${JEMALLOC_LIB_SUFFIX}")
 	set(JEMALLOC_INCLUDE "${JEMALLOC_SRC}/jemalloc/include/jemalloc")
+	if(CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
+		set(JEMALLOC_ARCH_SPECIFIC_CONFIGURE_ARGS --with-lg-page=14)
+	else()
+		set(JEMALLOC_ARCH_SPECIFIC_CONFIGURE_ARGS "")
+	endif()
 	ExternalProject_Add(
 		jemalloc
 		PREFIX "${PROJECT_BINARY_DIR}/jemalloc-prefix"
 		URL "https://github.com/jemalloc/jemalloc/archive/refs/tags/5.3.0.tar.gz"
 		URL_HASH "SHA256=ef6f74fd45e95ee4ef7f9e19ebe5b075ca6b7fbe0140612b2a161abafb7ee179"
-		CONFIGURE_COMMAND ./autogen.sh --enable-prof --disable-libdl
+		CONFIGURE_COMMAND ./autogen.sh --enable-prof --disable-libdl ${JEMALLOC_ARCH_SPECIFIC_CONFIGURE_ARGS}
 		BUILD_IN_SOURCE 1
 		BUILD_COMMAND make build_lib_static
 		INSTALL_COMMAND ""

--with-lg-page=14 would allow up to 16k-sized kernel page (Raspberry pi 5 for example).

There's a discussion on jemalloc/jemalloc#2639 to push --with-lg-page=16 (64k) as default, which would allow most kernels, but it seems that compiling with --with-lg-page=16 has a cost... But I'm no expert on the subject, so I just patched on my fork blindly.

[FedeDP]

Uh it makes sense! Would you mind opening a PR with the patch? I am ok with setting lg-page=14 for aarch64 only.