Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rust: need a way to pass ldflags to rustc? #24375

Open
vortexilation opened this issue Jun 13, 2024 · 20 comments
Open

rust: need a way to pass ldflags to rustc? #24375

vortexilation opened this issue Jun 13, 2024 · 20 comments

Comments

@vortexilation
Copy link
Contributor

Maintainer: @lu-zero @1715173329
Environment: x86_64, glibc

Description:

I wanted to compile einat , but somehow rustc cannot found libintl at the end.

I have tried everything

TARGET_CFLAGS += -Xlinker -rpath=$(STAGING_DIR)/usr/lib/libintl-full/lib
or
TARGET_LDFLAGS += -L$(STAGING_DIR)/usr/lib/libintl-full/lib -lintl
or
export LD_LIBRARY_PATH=$(STAGING_DIR)/usr/lib/libintl-full/lib

but rustc still didn't picked it up.

   Compiling einat v0.1.2 (/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3)
     Running `rustc --crate-name build_script_build --edition=2021 build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C overflow-checks=on --cfg 'feature="default"' --cfg 'feature="ipv6"' -C metadata=13037c25982734ec -C extra-filename=-13037c25982734ec --out-dir /home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec -C linker=x86_64-openwrt-linux-gnu-gcc -C strip=symbols -L dependency=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps --extern libbpf_cargo=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-fc7173ecf7072a22.rlib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release -L native=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out`
error: linking with `x86_64-openwrt-linux-gnu-gcc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/bin:/home/user/works/openwrt/staging_dir/hostpkg/bin:/home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/bin:/home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/bin:/home/user/works/openwrt/staging_dir/host/bin:/home/user/works/openwrt/staging_dir/host/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "x86_64-openwrt-linux-gnu-gcc" "-m64" "/home/user/works/openwrt/tmp/rustca9rJSt/symbols.o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec/build_script_build-13037c25982734ec.build_script_build.900026c213c08482-cgu.0.rcgu.o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec/build_script_build-13037c25982734ec.2kuwii56gzittnew.rcgu.o" "-Wl,--as-needed" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-fc7173ecf7072a22.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcargo_metadata-77b698b469ccad23.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libserde_json-d18e1952940c9d6c.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libitoa-c99bf860a1243733.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libryu-42733ab1749368f9.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libthiserror-aa231274481bae73.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcargo_platform-c20f7cad56b30019.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcamino-a1fd712e042bd596.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libmemmap2-73fca818d05ca4c9.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_rs-12af648727bedc4e.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libvsprintf-52f5d1afc74d566c.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibc-a50be116079b6680.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_sys-6ce4fe8ed806fc57.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libsemver-d9af4a6bbaa40b09.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libserde-73abfa3b10e52ca3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex-662c0e78adcebbf3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex_automata-e8e6e085a55c8ab3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex_syntax-f1d457fe75bcbfd6.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libtempfile-e4833a2ad6c4f429.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libfastrand-906c084e5ef6039e.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/librustix-4d3b20d2716f8231.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libbitflags-3a495cf5bbfede0b.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblinux_raw_sys-ba02cfb0a81adfba.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcfg_if-1c8851ce38e8a7c1.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libanyhow-0deb8dd3c263e7fc.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-18d1c84127d04dc5.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-bc53c6e5d5f4111f.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-a79bc22e6ae9ee83.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-9c256084533d6dad.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-e53b44d64d10bb35.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-9d0b3274a5b15e82.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-b0fb619b58a6601a.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-1319f451087d6793.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-e7ba5787d3fb181b.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-e23167ed65317fed.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-52af0786d05d61a5.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-28f9d348427e8d43.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-689892118ddcc60e.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-a1f86f0be6e775e6.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-cbc93ea126f55c98.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-390f90785cbfdde3.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-aded7817e4ea9283.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-6c16e1294e732b84.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-a8a94ae78107d089.rlib" "-Wl,-Bdynamic" "-lelf" "-lz" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec/build_script_build-13037c25982734ec" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,--strip-all" "-nodefaultlibs"
  = note: /home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/lib/gcc/x86_64-openwrt-linux-gnu/14.1.0/../../../../x86_64-openwrt-linux-gnu/bin/ld: warning: libintl.so.8, needed by /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so, not found (try using -rpath or -rpath-link)
          /home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/lib/gcc/x86_64-openwrt-linux-gnu/14.1.0/../../../../x86_64-openwrt-linux-gnu/bin/ld: /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so: undefined reference to `libintl_dgettext'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `einat` (build script) due to 1 previous error

Caused by:
  process didn't exit successfully: `rustc --crate-name build_script_build --edition=2021 build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C overflow-checks=on --cfg 'feature="default"' --cfg 'feature="ipv6"' -C metadata=13037c25982734ec -C extra-filename=-13037c25982734ec --out-dir /home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec -C linker=x86_64-openwrt-linux-gnu-gcc -C strip=symbols -L dependency=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps --extern libbpf_cargo=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-fc7173ecf7072a22.rlib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release -L native=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out` (exit status: 1)
warning: build failed, waiting for other jobs to finish...
error: failed to compile `einat v0.1.2 (/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3)`, intermediate artifacts can be found at `/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.
@lu-zero
Copy link
Contributor

lu-zero commented Jun 13, 2024

Doesn't look right. Where is libintl.so.8 and why it is needed?

@lu-zero
Copy link
Contributor

lu-zero commented Jun 13, 2024

The build.rs for libbpf uses LD_LIBRARY_PATH https://github.com/libbpf/libbpf-sys/blob/master/build.rs#L188C36-L188C51

@vortexilation
Copy link
Contributor Author

vortexilation commented Jun 13, 2024

Doesn't look right. Where is libintl.so.8 and why it is needed?

Because my OpenWrt build is NLS enabled (CONFIG_BUILD_NLS), libintl.so.8 is located in here :

/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib
or
$(STAGING_DIR)/usr/lib/libintl-full/lib

The build.rs for libbpf uses LD_LIBRARY_PATH https://github.com/libbpf/libbpf-sys/blob/master/build.rs#L188C36-L188C51

The compilation is failing at einat not libbpf

root@buildsystem:/home/user/works/openwrt# ldd /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so
        linux-vdso.so.1 (0x00007ffd965f1000)
        libintl.so.8 => not found
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f146c828000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f146c7fb000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f146c616000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f146c872000)

These below are only workaround that i have tried in other to make rustc able to pickup libintl.so.8 but still not being picked up with rustc

TARGET_CFLAGS += -Xlinker -rpath=$(STAGING_DIR)/usr/lib/libintl-full/lib
or
TARGET_LDFLAGS += -L$(STAGING_DIR)/usr/lib/libintl-full/lib -lintl
or
export LD_LIBRARY_PATH=$(STAGING_DIR)/usr/lib/libintl-full/lib

@lu-zero
Copy link
Contributor

lu-zero commented Jun 13, 2024

you may try your luck using RUSTFLAGS for that purpose. I guess we could add those search paths in a central config.toml if it solves your problem.

@vortexilation
Copy link
Contributor Author

vortexilation commented Jun 13, 2024

I did have tried using RUSTFLAGS but still not picked up with rustc, that's why it's weird.

Based on here :

RUSTFLAGS  += "-C linker-flag=-Wl,-rpath-link,$(STAGING_DIR)/usr/lib/libintl-full/lib"

in dl/cargo/config.toml I have tried the following :

[target.x86_64-unknown-linux-gnu]
linker = "x86_64-openwrt-linux-gnu-gcc"
rustflags = ["-L", "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib"]
rustc-link-search = { all = "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib" }

but still not picked up with rustc

If you have time, you could try to reproduce this issue by enabling CONFIG_BUILD_NLS and glibc.

@vortexilation
Copy link
Contributor Author

@lu-zero
I thought that it was libelf issue, after i have tried to patchelf the libelf with the location of libintl.so.8 but still errror out, this might be OpenWrt's rust build process issue ?.

I have tried to include the following to einat build.rs and "rpath = true" to einat's Cargo.toml :

println!("cargo:rustc-link-lib=dylib=libintl.so.8");
println!("cargo:rustc-link-search=native=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib");
println!("cargo:rustc-link-arg=-Wl,-rpath,/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib");

But still rustc seems to ignore it

@lu-zero
Copy link
Contributor

lu-zero commented Jun 14, 2024

The order should be the opposite, I think.

Check what's the rustc final invocation passing -v to cargo

@vortexilation
Copy link
Contributor Author

@lu-zero

Here is cargo very verbose log :

   Compiling einat v0.1.2 (/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3)
     Running `CARGO=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3 CARGO_PKG_AUTHORS='Huang-Huang Bao <[email protected]>' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE=GPL-2.0 CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=einat CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='https://github.com/EHfive/einat-ebpf' CARGO_PKG_RUST_VERSION=1.74 CARGO_PKG_VERSION=0.1.2 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=2 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps rustc --crate-name build_script_build --edition=2021 build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C overflow-checks=on --cfg 'feature="default"' --cfg 'feature="ipv6"' -C metadata=13037c25982734ec -C extra-filename=-13037c25982734ec --out-dir /home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec -C linker=x86_64-openwrt-linux-gnu-gcc -C strip=symbols -L dependency=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps --extern libbpf_cargo=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-fc7173ecf7072a22.rlib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release -L native=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out`
warning: `toml` (lib) generated 1 warning
error: linking with `x86_64-openwrt-linux-gnu-gcc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/bin:/home/user/works/openwrt/staging_dir/hostpkg/bin:/home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/bin:/home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/bin:/home/user/works/openwrt/staging_dir/host/bin:/home/user/works/openwrt/staging_dir/host/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "x86_64-openwrt-linux-gnu-gcc" "-m64" "/home/user/works/openwrt/tmp/rustck84dLP/symbols.o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec/build_script_build-13037c25982734ec.build_script_build.900026c213c08482-cgu.0.rcgu.o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec/build_script_build-13037c25982734ec.2kuwii56gzittnew.rcgu.o" "-Wl,--as-needed" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-fc7173ecf7072a22.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcargo_metadata-77b698b469ccad23.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libserde_json-d18e1952940c9d6c.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libitoa-c99bf860a1243733.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libryu-42733ab1749368f9.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libthiserror-aa231274481bae73.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcargo_platform-c20f7cad56b30019.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcamino-a1fd712e042bd596.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libmemmap2-73fca818d05ca4c9.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_rs-12af648727bedc4e.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libvsprintf-52f5d1afc74d566c.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibc-a50be116079b6680.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_sys-6ce4fe8ed806fc57.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libsemver-d9af4a6bbaa40b09.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libserde-73abfa3b10e52ca3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex-662c0e78adcebbf3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex_automata-e8e6e085a55c8ab3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex_syntax-f1d457fe75bcbfd6.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libtempfile-e4833a2ad6c4f429.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libfastrand-906c084e5ef6039e.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/librustix-4d3b20d2716f8231.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libbitflags-3a495cf5bbfede0b.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblinux_raw_sys-ba02cfb0a81adfba.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcfg_if-1c8851ce38e8a7c1.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libanyhow-0deb8dd3c263e7fc.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-18d1c84127d04dc5.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-bc53c6e5d5f4111f.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-a79bc22e6ae9ee83.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-9c256084533d6dad.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-e53b44d64d10bb35.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-9d0b3274a5b15e82.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-b0fb619b58a6601a.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-1319f451087d6793.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-e7ba5787d3fb181b.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-e23167ed65317fed.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-52af0786d05d61a5.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-28f9d348427e8d43.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-689892118ddcc60e.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-a1f86f0be6e775e6.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-cbc93ea126f55c98.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-390f90785cbfdde3.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-aded7817e4ea9283.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-6c16e1294e732b84.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-a8a94ae78107d089.rlib" "-Wl,-Bdynamic" "-lelf" "-lz" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec/build_script_build-13037c25982734ec" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,--strip-all" "-nodefaultlibs"
  = note: /home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/lib/gcc/x86_64-openwrt-linux-gnu/14.1.0/../../../../x86_64-openwrt-linux-gnu/bin/ld: warning: libintl.so.8, needed by /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so, not found (try using -rpath or -rpath-link)
          /home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/lib/gcc/x86_64-openwrt-linux-gnu/14.1.0/../../../../x86_64-openwrt-linux-gnu/bin/ld: /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so: undefined reference to `libintl_dgettext'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `einat` (build script) due to 1 previous error

Caused by:
  process didn't exit successfully: `CARGO=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3 CARGO_PKG_AUTHORS='Huang-Huang Bao <[email protected]>' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE=GPL-2.0 CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=einat CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='https://github.com/EHfive/einat-ebpf' CARGO_PKG_RUST_VERSION=1.74 CARGO_PKG_VERSION=0.1.2 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=2 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps rustc --crate-name build_script_build --edition=2021 build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C overflow-checks=on --cfg 'feature="default"' --cfg 'feature="ipv6"' -C metadata=13037c25982734ec -C extra-filename=-13037c25982734ec --out-dir /home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec -C linker=x86_64-openwrt-linux-gnu-gcc -C strip=symbols -L dependency=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps --extern libbpf_cargo=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-fc7173ecf7072a22.rlib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release -L native=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out` (exit status: 1)
warning: build failed, waiting for other jobs to finish...
warning: `toml_edit` (lib) generated 12 warnings
error: failed to compile `einat v0.1.2 (/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3)`, intermediate artifacts can be found at `/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.
make[2]: *** [Makefile:92: /home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/.built] Error 101
make[2]: Leaving directory '/home/user/works/openwrt/feeds/packages/net/einat'
time: package/feeds/packages/einat/compile#137.74#13.65#30.42
    ERROR: package/feeds/packages/einat failed to build.
make[1]: *** [package/Makefile:173: package/feeds/packages/einat/compile] Error 1
make[1]: Leaving directory '/home/user/works/openwrt'
make: *** [/home/user/works/openwrt/include/toplevel.mk:233: package/einat/compile] Error 2

@lu-zero
Copy link
Contributor

lu-zero commented Jun 14, 2024

that I can see they are not present.

@vortexilation
Copy link
Contributor Author

vortexilation commented Jun 14, 2024

That's why i said rustc seems ignoring everything that we are sets to cargo or Cargo.toml or config.toml or env variables.

Please try it yourself, it's einat , and with CONFIG_BUILD_NLS enabled.

It's also happening on musl & CONFIG_BUILD_NLS enabled.

@EHfive
Copy link

EHfive commented Jun 14, 2024

Hi, I am the author of https://github.com/EHfive/einat-ebpf.

I think there is some misunderstanding going on here.

So both einat and its build script link against libelf, the dependency path for the former is einat -> libbpf(-sys) -> libelf.so, and it's "build script of einat" -> libbpf-cargo -> libbpf(-sys) -> libelf.so for the latter.

As you can see, einat does not use libintl, it's just dynamically linked by libelf, so the issue is not really on einat or cargo/rustc.

The main problem here is both ld and ld-linux.so cannot find libintl.so used by libelf.so. In the case of ld, it has caused ld to report the unresolved symbol error due to ld 's default --unresolved-symbols=report policy. And in the case of ld-linux.so, it would cause binary linked with libelf thus indirectly using libintl fail to execute.

What need to addressed is why libintl has not been setup properly so both ld or ld-linux.so can locate it? Could it be rpath to libintl was not set properly in libelf? Or is there some misconfiguration in Rust build process? Note I am just guessing here as I know little about OpenWrt build system.

Another minor issue here is einat's build script is linking with target build of libelf instead of host build of libelf(in tools or build machine?).

@lu-zero
Copy link
Contributor

lu-zero commented Jun 14, 2024

I guess part of the problem is that host and target are the same x86_64 to add additional confusion to the issue ...

@rx78gp01
Copy link

It seems like depend $(INTL_DEPENDS) is missing in Makefile.

@EHfive
Copy link

EHfive commented Jun 14, 2024

It seems like depend $(INTL_DEPENDS) is missing in Makefile.

You mean in openwrt-einat-ebpf's Makefile? But einat does not link to libintl or use its symbols.

cc @muink

@muink
Copy link
Contributor

muink commented Jun 15, 2024

It seems like depend $(INTL_DEPENDS) is missing in Makefile.Makefile 中似乎缺少 dependent $(INTL_DEPENDS) 。

You mean in openwrt-einat-ebpf's Makefile? But einat does not link to libintl or use its symbols.

I tryied, still failed.
muink/openwrt-einat-ebpf@master...dev
And in fact $(INTL_DEPENDS) has been depended on by libelf, and there is no need to depend on it again in einat.
https://github.com/openwrt/openwrt/blob/22f92cce22401ef7df44357a7e6921d2cf45c840/package/libs/elfutils/Makefile#L51

@EHfive
Copy link

EHfive commented Jun 21, 2024

Noticed that there is a commented

# RUSTC_LDFLAGS+= -C link-args=$(INTL_LDFLAGS)

line in @muink 's openwrt-einat-ebpf dev branch. RUSTC_LDFLAGS would eventually be included into RUSTFLAGS, so it seems right at first glance.

However INTL_LDFLAGS has more than one link flags separated with space, so only the first flag in INTL_LDFLAGS would be passed as link-args.

The following change should make openwrt-einat-ebpf built again with CONFIG_BUILD_NLS for musl target.

RUSTC_LDFLAGS+= $(foreach arg,$(INTL_LDFLAGS),-C link-arg=$(arg))
# or
RUSTC_LDFLAGS+= -C link-arg=-L$(INTL_PREFIX)/lib -C link-arg=-Wl,-rpath-link=$(INTL_PREFIX)/lib

The following also works as rustc would pass proper link flags for libintl it found to linker(via cc -Wl).

RUSTC_LDFLAGS+= -L$(INTL_PREFIX)/lib -lintl

einat has build script depends on host libelf, and that build script would build target binary that depends on target libelf.

@vortexilation build openwrt-einat-ebpf for x86_64-unknown-linux-gnu target on x86_64-unknown-linux-gnu host. So Cargo would link to target libelf due to share of x86_64-unknown-linux-gnu triple on host and target. And due to Cargo's lack of host and target separation for build script, rust-lang/cargo#4423 , there is no graceful way to handle this case.

This means the libintl searching error would also happens on compilation of build script if host triple and target triple are same.

I guess passing the flags above can also make the build script built. However RUSTFLAGS only apply to compilation of target object as CARGO_BUILD_TARGET is specified, this requires patching rust-values.mk to use CARGO_TARGET_<TARGET>_RUSTFLAGS instead so these flags can also apply to compilation of build script. See also https://doc.rust-lang.org/cargo/reference/config.html#buildrustflags .

diff --git a/lang/rust/rust-values.mk b/lang/rust/rust-values.mk
index 534aaa0..aca2c24 100644
--- a/lang/rust/rust-values.mk
+++ b/lang/rust/rust-values.mk
@@ -97,7 +97,7 @@ CARGO_PKG_CONFIG_VARS= \
        CARGO_PROFILE_RELEASE_PANIC=unwind \
        CARGO_PROFILE_RELEASE_RPATH=false \
        CARGO_TARGET_$(subst -,_,$(call toupper,$(RUSTC_TARGET_ARCH)))_LINKER=$(TARGET_CC_NOCACHE) \
-       RUSTFLAGS="$(CARGO_RUSTFLAGS)" \
+       CARGO_TARGET_$(subst -,_,$(call toupper,$(RUSTC_TARGET_ARCH)))_RUSTFLAGS="$(CARGO_RUSTFLAGS)" \
        TARGET_CC=$(TARGET_CC_NOCACHE) \
        TARGET_CFLAGS="$(TARGET_CFLAGS) $(RUSTC_CFLAGS)"

Though on execution the build script of einat would probably instead link to host libelf by ld-linux.so. So this is just an ugly hack. I did not test this part as I am using musl SDK and I don't want to spend times for compiling of glibc toolchain builds.

edit: fix typo and reword

@muink
Copy link
Contributor

muink commented Jun 21, 2024

@vortexilation
Copy link
Contributor Author

vortexilation commented Jun 21, 2024

RUSTC_LDFLAGS+= $(foreach arg,$(INTL_LDFLAGS),-C link-arg=$(arg))
# or
RUSTC_LDFLAGS+= -C link-arg=-L$(INTL_PREFIX)/lib -C link-arg=-Wl,-rpath-link=$(INTL_PREFIX)/lib

RUSTC_LDFLAGS+= -L$(INTL_PREFIX)/lib -lintl

@EHfive
Thanks a lot for spending time debugging these, weird It seems still doesn't get passed to ld in glibc, haven't tried with musl :

   Compiling einat v0.1.2 (/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3)
     Running `rustc --crate-name build_script_build --edition=2021 build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C overflow-checks=on --cfg 'feature="default"' --cfg 'feature="ipv6"' -C metadata=c676d821f7a18d82 -C extra-filename=-c676d821f7a18d82 --out-dir /home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-c676d821f7a18d82 -C linker=x86_64-openwrt-linux-gnu-gcc -C strip=symbols -L dependency=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps --extern libbpf_cargo=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-0e08b88b74bdd281.rlib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release -L native=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out`
error: linking with `x86_64-openwrt-linux-gnu-gcc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/bin:/home/user/works/openwrt/staging_dir/hostpkg/bin:/home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/bin:/home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/bin:/home/user/works/openwrt/staging_dir/host/bin:/home/user/works/openwrt/staging_dir/host/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "x86_64-openwrt-linux-gnu-gcc" "-m64" "/home/user/works/openwrt/tmp/rustcF0JBOI/symbols.o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-c676d821f7a18d82/build_script_build-c676d821f7a18d82.build_script_build.47baaadd226efc23-cgu.0.rcgu.o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-c676d821f7a18d82/build_script_build-c676d821f7a18d82.xlv39qanpodgvnm.rcgu.o" "-Wl,--as-needed" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-0e08b88b74bdd281.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcargo_metadata-e8a4001291ac79c8.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libserde_json-a5b38a65cb4ef017.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libitoa-c99bf860a1243733.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libryu-42733ab1749368f9.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libthiserror-86094f3a3cb5876d.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcargo_platform-f5c5cc58210f0695.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcamino-2bffcd7edd9fbed6.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libmemmap2-73fca818d05ca4c9.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_rs-63fc6c477f434b84.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libvsprintf-52f5d1afc74d566c.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibc-a50be116079b6680.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_sys-6ce4fe8ed806fc57.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libsemver-fdf0c5b9390e1872.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libserde-f69114cfae5034f0.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex-662c0e78adcebbf3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex_automata-e8e6e085a55c8ab3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex_syntax-f1d457fe75bcbfd6.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libtempfile-e4833a2ad6c4f429.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libfastrand-906c084e5ef6039e.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/librustix-4d3b20d2716f8231.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libbitflags-3a495cf5bbfede0b.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblinux_raw_sys-ba02cfb0a81adfba.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcfg_if-1c8851ce38e8a7c1.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libanyhow-0deb8dd3c263e7fc.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-18d1c84127d04dc5.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-bc53c6e5d5f4111f.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-a79bc22e6ae9ee83.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-9c256084533d6dad.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-e53b44d64d10bb35.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-9d0b3274a5b15e82.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-b0fb619b58a6601a.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-1319f451087d6793.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-e7ba5787d3fb181b.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-e23167ed65317fed.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-52af0786d05d61a5.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-28f9d348427e8d43.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-689892118ddcc60e.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-a1f86f0be6e775e6.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-cbc93ea126f55c98.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-390f90785cbfdde3.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-aded7817e4ea9283.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-6c16e1294e732b84.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-a8a94ae78107d089.rlib" "-Wl,-Bdynamic" "-lelf" "-lz" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-c676d821f7a18d82/build_script_build-c676d821f7a18d82" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,--strip-all" "-nodefaultlibs"
  = note: /home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/lib/gcc/x86_64-openwrt-linux-gnu/14.1.0/../../../../x86_64-openwrt-linux-gnu/bin/ld: warning: libintl.so.8, needed by /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so, not found (try using -rpath or -rpath-link)
          /home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/lib/gcc/x86_64-openwrt-linux-gnu/14.1.0/../../../../x86_64-openwrt-linux-gnu/bin/ld: /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so: undefined reference to `libintl_dgettext'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `einat` (build script) due to 1 previous error

Caused by:
  process didn't exit successfully: `rustc --crate-name build_script_build --edition=2021 build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C overflow-checks=on --cfg 'feature="default"' --cfg 'feature="ipv6"' -C metadata=c676d821f7a18d82 -C extra-filename=-c676d821f7a18d82 --out-dir /home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-c676d821f7a18d82 -C linker=x86_64-openwrt-linux-gnu-gcc -C strip=symbols -L dependency=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps --extern libbpf_cargo=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-0e08b88b74bdd281.rlib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release -L native=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out` (exit status: 1)
warning: build failed, waiting for other jobs to finish...
error: failed to compile `einat v0.1.2 (/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3)`, intermediate artifacts can be found at `/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

@muink

Are you sure haven't forgot to re-enable "CONFIG_BUILD_NLS" ?. Haven't tried with musl but glibc still error on me.

@EHfive
Copy link

EHfive commented Jun 21, 2024

@vortexilation You didn't read my previous comment carefully, right? Especially the second section.

And you are not even trying to read logs from yourself or changes in @muink 's commit. As if you read, you would know that error is for build script instead if einat target binary, and @muink 's commit only covers the first change mentioned in my comment that only fixes building for musl target.

... built again with CONFIG_BUILD_NLS for musl target.

... share of x86_64-unknown-linux-gnu triple on host and target.
This means the libintl searing error would also happens on build script if host triple and target triple are same.

Question yourself before question someone else next time.

@vortexilation
Copy link
Contributor Author

vortexilation commented Jun 21, 2024

@EHfive
I have had already read my logs or @muink changes, but maybe because typos and gotten the hypes up of @muink's succeed.

diff --git a/lang/rust/rust-values.mk b/lang/rust/rust-values.mk
index 534aaa0..aca2c24 100644
--- a/lang/rust/rust-values.mk
+++ b/lang/rust/rust-values.mk
@@ -97,7 +97,7 @@ CARGO_PKG_CONFIG_VARS= \
        CARGO_PROFILE_RELEASE_PANIC=unwind \
        CARGO_PROFILE_RELEASE_RPATH=false \
        CARGO_TARGET_$(subst -,_,$(call toupper,$(RUSTC_TARGET_ARCH)))_LINKER=$(TARGET_CC_NOCACHE) \
-       RUSTFLAGS="$(CARGO_RUSTFLAGS)" \
+       CARGO_TARGET_$(subst -,_,$(call toupper,$(RUSTC_TARGET_ARCH)))_RUSTFLAGS="$(CARGO_RUSTFLAGS)" \
        TARGET_CC=$(TARGET_CC_NOCACHE) \
        TARGET_CFLAGS="$(TARGET_CFLAGS) $(RUSTC_CFLAGS)"

I have already previously tried adding CARGO_TARGET_x86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS="-C link-args=-Wl,-rpath,$(STAGING_DIR)/usr/lib/libintl-full/lib" into rust-values.mk but it was seems getting no differences.

Let me retry it now with your exact changes as snippet above. Recompiling rust now for OpenWrt.

In Makefile, i just need to do the following ? :

CARGO_RUSTFLAGS += -C link-args=-Wl,-rpath,$(STAGING_DIR)/usr/lib/libintl-full/lib

[EDIT]
Rust recompiled, with snippet above being applied and added these to Makefile :

RUSTC_LDFLAGS += $(foreach arg,$(INTL_LDFLAGS),-C link-arg=$(arg))
CARGO_RUSTFLAGS += -C link-args=-Wl,-rpath,$(STAGING_DIR)/usr/lib/libintl-full/lib

But still the same errror.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants