Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build of picolibc module with LTO fails #81674

Closed
benediktibk opened this issue Nov 20, 2024 · 15 comments · Fixed by #83132
Closed

Build of picolibc module with LTO fails #81674

benediktibk opened this issue Nov 20, 2024 · 15 comments · Fixed by #83132
Assignees
Labels
area: picolibc Picolibc C Standard Library bug The issue is a bug, or the PR is fixing a bug priority: low Low impact/importance bug

Comments

@benediktibk
Copy link
Collaborator

Describe the bug
Building a minimal example with the picolibc module and LTO fails with a bunch of undefined references.

To Reproduce
CMakeLists.txt:

cmake_minimum_required(VERSION 3.20.0)

find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(lto)

target_sources(app PRIVATE 
        src/main.c)

prj.conf:

CONFIG_COMPILER_OPT="-Wall -Werror"
CONFIG_DEBUG_OPTIMIZATIONS=y
CONFIG_LOG=y
CONFIG_ISR_TABLES_LOCAL_DECLARATION=y
CONFIG_LTO=y
CONFIG_PICOLIBC_USE_MODULE=y

main.c:

#include <zephyr/kernel.h>
#include <zephyr/logging/log.h>

LOG_MODULE_REGISTER(main, CONFIG_LOG_DEFAULT_LEVEL);

int main(void)
{
	LOG_INF("Hello!");

	return 0;
}

Expected behavior
It should be possible to build the picolibc module with LTO.

Impact
It is not possible to use the picolibc module in combination with LTO.

Logs and console output

west build -p auto -b nucleo_f746zg -d build/lto lto
[0/1] Re-running CMake...
Loading Zephyr default modules (Zephyr base (cached)).
-- Application: .../zephyr_playground/lto
-- CMake version: 3.28.3
-- Cache files will be written to: /home/bschmidt/.cache/zephyr
-- Zephyr version: 4.0.99 (.../zephyr_playground/zephyr)
-- Found west (found suitable version "1.2.0", minimum required is "0.14.0")
-- Board: nucleo_f746zg, qualifiers: stm32f746xx
-- Found host-tools: zephyr 0.16.1 (/home/bschmidt/zephyr-sdk-0.16.1)
-- Found toolchain: zephyr 0.16.1 (/home/bschmidt/zephyr-sdk-0.16.1)
-- Found BOARD.dts: .../zephyr_playground/zephyr/boards/st/nucleo_f746zg/nucleo_f746zg.dts
-- Generated zephyr.dts: .../zephyr_playground/build/lto/zephyr/zephyr.dts
-- Generated pickled edt: .../zephyr_playground/build/lto/zephyr/edt.pickle
-- Generated zephyr.dts: .../zephyr_playground/build/lto/zephyr/zephyr.dts
-- Generated devicetree_generated.h: .../zephyr_playground/build/lto/zephyr/include/generated/zephyr/devicetree_generated.h
-- Including generated dts.cmake file: .../zephyr_playground/build/lto/zephyr/dts.cmake
Parsing .../zephyr_playground/zephyr/Kconfig
Loaded configuration '.../zephyr_playground/zephyr/boards/st/nucleo_f746zg/nucleo_f746zg_defconfig'
Merged configuration '.../zephyr_playground/lto/prj.conf'
Configuration saved to '.../zephyr_playground/build/lto/zephyr/.config'
Kconfig header saved to '.../zephyr_playground/build/lto/zephyr/include/generated/zephyr/autoconf.h'
-- Using ccache: /usr/bin/ccache
-- Looking for xtensa/config/core-isa.h
-- Looking for xtensa/config/core-isa.h - not found
-- Configuring done (6.1s)
-- Generating done (0.1s)
-- Build files have been written to: .../zephyr_playground/build/lto
[1092/1097] Linking C executable zephyr/zephyr_pre0.elf
FAILED: zephyr/zephyr_pre0.elf zephyr/zephyr_pre0.map .../zephyr_playground/build/lto/zephyr/zephyr_pre0.map 
: && ccache /home/bschmidt/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc  -gdwarf-4 -flto=auto -fno-ipa-sra -ffunction-sections -fdata-sections -gdwarf-4 -flto=auto -fno-ipa-sra -ffunction-sections -fdata-sections zephyr/CMakeFiles/zephyr_pre0.dir/misc/empty_file.c.obj -o zephyr/zephyr_pre0.elf  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -T  zephyr/linker_zephyr_pre0.cmd  -Wl,-Map=.../zephyr_playground/build/lto/zephyr/zephyr_pre0.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/libarch__arm__core.a  zephyr/arch/arch/arm/core/cortex_m/libarch__arm__core__cortex_m.a  zephyr/arch/arch/arm/core/mpu/libarch__arm__core__mpu.a  zephyr/lib/libc/picolibc/liblib__libc__picolibc.a  zephyr/lib/libc/common/liblib__libc__common.a  zephyr/drivers/interrupt_controller/libdrivers__interrupt_controller.a  zephyr/drivers/clock_control/libdrivers__clock_control.a  zephyr/drivers/console/libdrivers__console.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/pinctrl/libdrivers__pinctrl.a  zephyr/drivers/reset/libdrivers__reset.a  zephyr/drivers/serial/libdrivers__serial.a  zephyr/drivers/timer/libdrivers__timer.a  modules/hal_stm32/stm32cube/lib..__modules__hal__stm32__stm32cube.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  -L.../zephyr_playground/build/lto/zephyr  zephyr/arch/common/libisr_tables.a  -mcpu=cortex-m7  -mthumb  -mabi=aapcs  -mfp16-format=ieee  -mtp=soft  -fuse-ld=bfd  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn  -Wl,-no-pie -L"/home/bschmidt/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/thumb/v7e-m/nofp" .../zephyr_playground/build/lto/modules/picolibc/libc.a -lgcc && cd .../zephyr_playground/build/lto/zephyr && /usr/bin/cmake -E true
/home/bschmidt/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: /tmp/ccXfOQ2f.ltrans0.ltrans.o: in function `z_tls_copy':
.../zephyr_playground/zephyr/kernel/include/kernel_tls.h:50: undefined reference to `memset'
/home/bschmidt/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: /tmp/ccXfOQ2f.ltrans0.ltrans.o: in function `cbvprintf':
.../zephyr_playground/zephyr/lib/libc/picolibc/cbprintf.c:30: undefined reference to `vfprintf'
/home/bschmidt/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: /tmp/ccXfOQ2f.ltrans0.ltrans.o: in function `log_output_dropped_process':
.../zephyr_playground/zephyr/subsys/logging/log_output.c:764: undefined reference to `snprintf'
/home/bschmidt/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: /tmp/ccXfOQ2f.ltrans0.ltrans.o: in function `z_early_memset':
.../zephyr_playground/zephyr/kernel/init.c:197: undefined reference to `memset'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: /usr/bin/cmake --build .../zephyr_playground/build/lto
make: *** [Makefile:345: build/lto/zephyr/zephyr.elf] Error 1

Environment
Zephyr Version 6843240

@benediktibk benediktibk added the bug The issue is a bug, or the PR is fixing a bug label Nov 20, 2024
@benediktibk
Copy link
Collaborator Author

Tested the same minimal project with the zephyr SDK 0.17.0, same result:

west build -p auto -b nucleo_f746zg -d build/lto lto
-- west build: generating a build system
Loading Zephyr default modules (Zephyr base).
-- Application: /home/benedikt/Development/zephyr_playground/lto
-- CMake version: 3.28.3
-- Found Python3: /home/benedikt/Development/zephyr_playground/.venv/bin/python (found suitable version "3.12.3", minimum required is "3.10") found components: Interpreter 
-- Cache files will be written to: /home/benedikt/.cache/zephyr
-- Zephyr version: 4.0.99 (/home/benedikt/Development/zephyr_playground/zephyr)
-- Found west (found suitable version "1.2.0", minimum required is "0.14.0")
-- Board: nucleo_f746zg, qualifiers: stm32f746xx
-- ZEPHYR_TOOLCHAIN_VARIANT not set, trying to locate Zephyr SDK
-- Found host-tools: zephyr 0.17.0 (/home/benedikt/zephyr-sdk-0.17.0)
-- Found toolchain: zephyr 0.17.0 (/home/benedikt/zephyr-sdk-0.17.0)
-- Found Dtc: /home/benedikt/zephyr-sdk-0.17.0/sysroots/x86_64-pokysdk-linux/usr/bin/dtc (found suitable version "1.6.0", minimum required is "1.4.6") 
-- Found BOARD.dts: /home/benedikt/Development/zephyr_playground/zephyr/boards/st/nucleo_f746zg/nucleo_f746zg.dts
-- Generated zephyr.dts: /home/benedikt/Development/zephyr_playground/build/lto/zephyr/zephyr.dts
-- Generated pickled edt: /home/benedikt/Development/zephyr_playground/build/lto/zephyr/edt.pickle
-- Generated zephyr.dts: /home/benedikt/Development/zephyr_playground/build/lto/zephyr/zephyr.dts
-- Generated devicetree_generated.h: /home/benedikt/Development/zephyr_playground/build/lto/zephyr/include/generated/zephyr/devicetree_generated.h
-- Including generated dts.cmake file: /home/benedikt/Development/zephyr_playground/build/lto/zephyr/dts.cmake
Parsing /home/benedikt/Development/zephyr_playground/zephyr/Kconfig
Loaded configuration '/home/benedikt/Development/zephyr_playground/zephyr/boards/st/nucleo_f746zg/nucleo_f746zg_defconfig'
Merged configuration '/home/benedikt/Development/zephyr_playground/lto/prj.conf'
Configuration saved to '/home/benedikt/Development/zephyr_playground/build/lto/zephyr/.config'
Kconfig header saved to '/home/benedikt/Development/zephyr_playground/build/lto/zephyr/include/generated/zephyr/autoconf.h'
-- Found GnuLd: /home/benedikt/zephyr-sdk-0.17.0/arm-zephyr-eabi/arm-zephyr-eabi/bin/ld.bfd (found version "2.38") 
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /home/benedikt/zephyr-sdk-0.17.0/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc
-- Using ccache: /usr/bin/ccache
-- Looking for xtensa/config/core-isa.h
-- Looking for xtensa/config/core-isa.h - not found
-- Configuring done (9.3s)
-- Generating done (0.1s)
-- Build files have been written to: /home/benedikt/Development/zephyr_playground/build/lto
-- west build: building application
[1/1103] Preparing syscall dependency handling

[41/1103] Generating include/generated/zephyr/version.h
-- Zephyr version: 4.0.99 (/home/benedikt/Development/zephyr_playground/zephyr), build: v4.0.0-515-g197f8179b977
[1098/1103] Linking C executable zephyr/zephyr_pre0.elf
FAILED: zephyr/zephyr_pre0.elf zephyr/zephyr_pre0.map /home/benedikt/Development/zephyr_playground/build/lto/zephyr/zephyr_pre0.map 
: && ccache /home/benedikt/zephyr-sdk-0.17.0/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc  -gdwarf-4 -flto=auto -fno-ipa-sra -ffunction-sections -fdata-sections -gdwarf-4 -flto=auto -fno-ipa-sra -ffunction-sections -fdata-sections zephyr/CMakeFiles/zephyr_pre0.dir/misc/empty_file.c.obj -o zephyr/zephyr_pre0.elf  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -T  zephyr/linker_zephyr_pre0.cmd  -Wl,-Map=/home/benedikt/Development/zephyr_playground/build/lto/zephyr/zephyr_pre0.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/libarch__arm__core.a  zephyr/arch/arch/arm/core/cortex_m/libarch__arm__core__cortex_m.a  zephyr/arch/arch/arm/core/mpu/libarch__arm__core__mpu.a  zephyr/lib/libc/picolibc/liblib__libc__picolibc.a  zephyr/lib/libc/common/liblib__libc__common.a  zephyr/drivers/interrupt_controller/libdrivers__interrupt_controller.a  zephyr/drivers/clock_control/libdrivers__clock_control.a  zephyr/drivers/console/libdrivers__console.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/pinctrl/libdrivers__pinctrl.a  zephyr/drivers/reset/libdrivers__reset.a  zephyr/drivers/serial/libdrivers__serial.a  zephyr/drivers/timer/libdrivers__timer.a  modules/hal_stm32/stm32cube/lib..__modules__hal__stm32__stm32cube.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  -L/home/benedikt/Development/zephyr_playground/build/lto/zephyr  zephyr/arch/common/libisr_tables.a  -mcpu=cortex-m7  -mthumb  -mabi=aapcs  -mfp16-format=ieee  -mtp=soft  -fuse-ld=bfd  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn  -Wl,-no-pie -L"/home/benedikt/zephyr-sdk-0.17.0/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/thumb/v7e-m/nofp" /home/benedikt/Development/zephyr_playground/build/lto/modules/picolibc/libc.a -lgcc && cd /home/benedikt/Development/zephyr_playground/build/lto/zephyr && /usr/bin/cmake -E true
/home/benedikt/zephyr-sdk-0.17.0/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: /tmp/ccuEj3Lf.ltrans0.ltrans.o: in function `z_tls_copy':
/home/benedikt/Development/zephyr_playground/zephyr/kernel/include/kernel_tls.h:50: undefined reference to `memset'
/home/benedikt/zephyr-sdk-0.17.0/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: /tmp/ccuEj3Lf.ltrans0.ltrans.o: in function `cbvprintf':
/home/benedikt/Development/zephyr_playground/zephyr/lib/libc/picolibc/cbprintf.c:30: undefined reference to `vfprintf'
/home/benedikt/zephyr-sdk-0.17.0/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: /tmp/ccuEj3Lf.ltrans0.ltrans.o: in function `log_output_dropped_process':
/home/benedikt/Development/zephyr_playground/zephyr/subsys/logging/log_output.c:764: undefined reference to `snprintf'
/home/benedikt/zephyr-sdk-0.17.0/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: /tmp/ccuEj3Lf.ltrans0.ltrans.o: in function `z_early_memset':
/home/benedikt/Development/zephyr_playground/zephyr/kernel/init.c:197: undefined reference to `memset'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: /usr/bin/cmake --build /home/benedikt/Development/zephyr_playground/build/lto
make: *** [Makefile:345: build/lto/zephyr/zephyr.elf] Error 1

@benediktibk
Copy link
Collaborator Author

benediktibk commented Nov 20, 2024

Changing the prj.conf to CONFIG_PICOLIBC_USE_MODULE=n resolves the issue, so the trigger is definitely the combination of LTO with the module version of picolibc. The testcases where LTO is used (samples/bluetooth/hci_ipc, tests/kernel/common, tests/kernel/interrupt) do not use the module version of picolibc.

@benediktibk
Copy link
Collaborator Author

benediktibk commented Nov 21, 2024

Running arm-none-eabi-nm on the resulting zephyr.elf for CONFIG_LTO=n reveals the following:

 .text.memset   0x0000000008006eba       0x10 .../zephyr-sdk-0.17.0/arm-zephyr-eabi/bin/../lib/gcc/../../picolibc/arm-zephyr-eabi/lib/thumb/v7e-m/nofp/libc.a(memset.c.o)
                0x0000000008006eba                memset

Do I understand this correctly that I'm pulling in memset from the SDK, although CONFIG_PICOLIBC_USE_MODULE=y is selected?

@keith-packard
Copy link
Collaborator

I'm happy to help out, but this sure looks like a compiler issue and not something the library has any control over.

@keith-packard
Copy link
Collaborator

Ok, I did some experiments using the above project to see what's up. The compiler flags for picolibc and the rest of Zephyr do differ a bit (picolibc enables some floating point consistency flags). I hacked things up to make the flags match as closely as I could manage; picolibc still requires -fno-builtin when building things like memset though. Even with those changes, the linker still cannot manage to find the C library symbols. Checking on the .obj files with nm shows that they are present. I'm stumped here.

@benediktibk
Copy link
Collaborator Author

Do you mind sharing your patch with the differences in the flags? I suppose getting them as close as possible would be a good idea anyway, although it might not solve this problem.

And for the actual problem: I still think something with the module version of picolibc is off, because why would it pull in for this case memset from the SDK? Or am I missing something?

@keith-packard
Copy link
Collaborator

keith-packard commented Nov 22, 2024

picolibc_supported_compile_options(
+  "-std=c99"
   "-fno-common"
-  "-fno-stack-protector"
   "-ffunction-sections"
   "-fdata-sections"
   "-Wall"
@@ -567,8 +567,6 @@ picolibc_supported_compile_options(
   "-Werror=vla"
   "-Warray-bounds"
   "-Wold-style-definition"
-  "-frounding-math"
-  "-fsignaling-nans"
   )

The two math options are necessary so that picolibc gets built correctly, but we should probably let Zephyr control the -std and -fstack-protector values.

I assume it's pulling in the function from the SDK because it couldn't find it in the module? Why is it even referencing the SDK library at all though?

@fabiobaltieri fabiobaltieri added the priority: low Low impact/importance bug label Nov 26, 2024
@benediktibk
Copy link
Collaborator Author

Triggering the failure is the following command:

/home/bschmidt/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc
-gdwarf-4
-flto=auto
-fno-ipa-sra
-ffunction-sections
-fdata-sections
-gdwarf-4
-flto=auto
-fno-ipa-sra
-ffunction-sections
-fdata-sections
zephyr/CMakeFiles/zephyr_pre0.dir/misc/empty_file.c.obj
-o
zephyr/zephyr_pre0.elf
zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj
-T
zephyr/linker_zephyr_pre0.cmd
-Wl,-Map=/home/bschmidt/Development/zephyr_playground/build/lto/zephyr/zephyr_pre0.map
-Wl,--whole-archive
app/libapp.a
zephyr/libzephyr.a
zephyr/arch/common/libarch__common.a
zephyr/arch/arch/arm/core/libarch__arm__core.a
zephyr/arch/arch/arm/core/cortex_m/libarch__arm__core__cortex_m.a
zephyr/arch/arch/arm/core/mpu/libarch__arm__core__mpu.a
zephyr/lib/libc/picolibc/liblib__libc__picolibc.a
zephyr/lib/libc/common/liblib__libc__common.a
zephyr/drivers/interrupt_controller/libdrivers__interrupt_controller.a
zephyr/drivers/clock_control/libdrivers__clock_control.a
zephyr/drivers/console/libdrivers__console.a
zephyr/drivers/gpio/libdrivers__gpio.a
zephyr/drivers/pinctrl/libdrivers__pinctrl.a
zephyr/drivers/reset/libdrivers__reset.a
zephyr/drivers/serial/libdrivers__serial.a
zephyr/drivers/timer/libdrivers__timer.a
modules/hal_stm32/stm32cube/lib..__modules__hal__stm32__stm32cube.a
-Wl,--no-whole-archive
zephyr/kernel/libkernel.a
-L/home/bschmidt/Development/zephyr_playground/build/lto/zephyr
zephyr/arch/common/libisr_tables.a
-mcpu=cortex-m7
-mthumb
-mabi=aapcs
-mfp16-format=ieee
-mtp=soft
-fuse-ld=bfd
-Wl,--gc-sections
-Wl,--build-id=none
-Wl,--sort-common=descending
-Wl,--sort-section=alignment
-Wl,-u,_OffsetAbsSyms
-Wl,-u,_ConfigAbsSyms
-nostdlib
-static
-Wl,-X
-Wl,-N
-Wl,--orphan-handling=warn
-Wl,-no-pie
-L"/home/bschmidt/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/thumb/v7e-m/nofp"
-lgcc
/home/bschmidt/Development/zephyr_playground/build/lto/modules/picolibc/libc.a

memset is defined in the last file libc.a:

bschmidt@bschmidt-desktop:~/Development/zephyr_playground/build/lto$ arm-none-eabi-nm /home/bschmidt/Development/zephyr_playground/build/lto/modules/picolibc/libc.a | grep memset
arm-none-eabi-nm: memchr.c.obj: no symbols
aeabi_memset.c.obj:
00000000 W __aeabi_memset
00000000 W __aeabi_memset4
00000000 W __aeabi_memset8
         U memset
         U memset
arm-none-eabi-nm: memcpy.c.obj: no symbols
memset.c.obj:
00000000 T memset
...

But still bfd fails to find it. If I add the object file, which contains the memset definition, manually to the link command bfd is able to find memset. Linking still fails due to other undefined symbols, but memset can be found:

/home/bschmidt/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc
-gdwarf-4
-flto=auto
-fno-ipa-sra
-ffunction-sections
-fdata-sections
-gdwarf-4
-flto=auto
-fno-ipa-sra
-ffunction-sections
-fdata-sections
zephyr/CMakeFiles/zephyr_pre0.dir/misc/empty_file.c.obj
-o
zephyr/zephyr_pre0.elf
zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj
-T
zephyr/linker_zephyr_pre0.cmd
-Wl,-Map=/home/bschmidt/Development/zephyr_playground/build/lto/zephyr/zephyr_pre0.map
-Wl,--whole-archive
app/libapp.a
zephyr/libzephyr.a
zephyr/arch/common/libarch__common.a
zephyr/arch/arch/arm/core/libarch__arm__core.a
zephyr/arch/arch/arm/core/cortex_m/libarch__arm__core__cortex_m.a
zephyr/arch/arch/arm/core/mpu/libarch__arm__core__mpu.a
zephyr/lib/libc/picolibc/liblib__libc__picolibc.a
zephyr/lib/libc/common/liblib__libc__common.a
zephyr/drivers/interrupt_controller/libdrivers__interrupt_controller.a
zephyr/drivers/clock_control/libdrivers__clock_control.a
zephyr/drivers/console/libdrivers__console.a
zephyr/drivers/gpio/libdrivers__gpio.a
zephyr/drivers/pinctrl/libdrivers__pinctrl.a
zephyr/drivers/reset/libdrivers__reset.a
zephyr/drivers/serial/libdrivers__serial.a
zephyr/drivers/timer/libdrivers__timer.a
modules/hal_stm32/stm32cube/lib..__modules__hal__stm32__stm32cube.a
-Wl,--no-whole-archive
zephyr/kernel/libkernel.a
-L/home/bschmidt/Development/zephyr_playground/build/lto/zephyr
zephyr/arch/common/libisr_tables.a
-mcpu=cortex-m7
-mthumb
-mabi=aapcs
-mfp16-format=ieee
-mtp=soft
-fuse-ld=bfd
-Wl,--gc-sections
-Wl,--build-id=none
-Wl,--sort-common=descending
-Wl,--sort-section=alignment
-Wl,-u,_OffsetAbsSyms
-Wl,-u,_ConfigAbsSyms
-nostdlib
-static
-Wl,-X
-Wl,-N
-Wl,--orphan-handling=warn
-Wl,-no-pie
-L"/home/bschmidt/zephyr-sdk-0.16.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/thumb/v7e-m/nofp"
-lgcc
/home/bschmidt/Development/zephyr_playground/build/lto/modules/picolibc/libc.a
modules/picolibc/CMakeFiles/c.dir/newlib/libc/machine/arm/memset.c.obj

So there seems to be a difference in the handling of archive files and object files. Anybody an idea what is going on here?

@benediktibk
Copy link
Collaborator Author

I think I was able to narrow it down even further. Running nm on memset.c.obj shows that memset is defined, but running objdump on it does not. So there is obviously a difference between nm and objdump in how they find symbols in a binary. When I create memset.c.obj without the command line flag -flto=auto also objdump is able to find the symbol. If I then recreate libc.at with the exact same arguments the build of the final elf-file does not complain anymore about an undefined reference to memset.

Hence, I'm slowly starting to suspect a gcc bug to be the cause of this issue.

@keith-packard
Copy link
Collaborator

I wonder if something in -lgcc is referencing memset and that's causing problems? libgcc is probably not built with -lto enabled.

@benediktibk
Copy link
Collaborator Author

benediktibk commented Dec 16, 2024

It rather looks like some functions should be generally excluded from LTO:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=118063

Edit: fixed link

@benediktibk
Copy link
Collaborator Author

Result of the discussion with the gcc-guys:

Is there some documentation available which functions should never be LTOed?
Because I'm running during the build in undefined references for these
functions:

all of C library. all of libgcc.

@keith-packard
Copy link
Collaborator

Their answer makes sense -- you can't LTO a function used in the late stages of code generation. This shouldn't include all of libc, but that's certainly an easy answer. Have we tried -ffat-lto-objects when building the C library? That should provide both LTO and regular binary bits for every function.

@benediktibk
Copy link
Collaborator Author

Unfortunately -ffat-lto-objects did not help, probably the linker is then still incorrectly throwing memset out of the window for some reason.

I've been trying to adapt cmake to get rid of LTO just for picolibc, but I'm failing to achieve something useful. My approach would have been to add a suppress_lto target property:

--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -572,6 +572,7 @@ picolibc_supported_compile_options(
   )
 
 add_library(c STATIC)
+set_property(TARGET c PROPERTY suppress_lto)
 
 target_compile_options(c PRIVATE ${PICOLIBC_COMPILE_OPTIONS})

And then using this in a cmake generator expression to get rid of LTO if this has been set:

 --- a/CMakeLists.txt
 +++ b/CMakeLists.txt
 @@ -225,6 +225,7 @@ zephyr_compile_options(${OPTIMIZATION_FLAG})
  if(CONFIG_LTO)
    zephyr_compile_options($<TARGET_PROPERTY:compiler,optimization_lto>)
 -  add_link_options($<TARGET_PROPERTY:linker,lto_arguments>)
 +  add_link_options($<TARGET_PROPERTY:linker,suppress_lto:$<TARGET_PROPERTY:linker,lto_arguments>>)
  endif()
  
  if(CONFIG_STD_C23)

But clearly my cmake-foo is too weak. @keith-packard Could you maybe help me out a little bit? Or do you have a different suggestion?

@benediktibk
Copy link
Collaborator Author

Consider the cmake modifications in the previous comment just as a basic idea, they are definitely wrong with respect to cmake syntax.

keith-packard added a commit to keith-packard/zephyr that referenced this issue Dec 17, 2024
The compiler requires that much of the C library be built without using LTO
so that various symbols are available for use by generated code, including
things like memset and memcpy.

Add -fno-lto when building both picolibc itself as well as the Zephyr
interface bits.

Closes: zephyrproject-rtos#81674

Signed-off-by: Keith Packard <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: picolibc Picolibc C Standard Library bug The issue is a bug, or the PR is fixing a bug priority: low Low impact/importance bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants