[LLD] symbol not found with PROVIDE script #111478

DianQK · 2024-10-08T04:31:56Z

I tried the following code. I expect lld to link successful while also removing unused symbols, but it reports symbol not found: bar.

a.s:

.global _start
_start:
 nop
.section .text.foo,"ax",@progbits
.global foo
foo:
 nop
.section .text.bar,"ax",@progbits
.global bar
bar:
 nop

script.t:

PROVIDE(foo = bar);

commands:

llvm-mc -filetype=obj -triple=x86_64 a.s -o a.o
ld.lld -o a_gc a.o --gc-sections -T script.t

output:

ld.lld: error: script.t:1: symbol not found: bar
ld.lld: error: script.t:1: symbol not found: bar
ld.lld: error: script.t:1: symbol not found: bar

cc @MaskRay ebb326a

The text was updated successfully, but these errors were encountered:

MaskRay · 2024-10-09T04:55:28Z

I cannot reproduce ld.lld: error: script.t:1: symbol not found: bar. Are you testing the latest lld?

foo is not in the symbol table. lld considers it unneeded.

DianQK · 2024-10-09T05:12:08Z

I cannot reproduce ld.lld: error: script.t:1: symbol not found: bar. Are you testing the latest lld?

foo is not in the symbol table. lld considers it unneeded.

Sorry, I mistakenly deleted the definition of foo. I may have forgotten to run the llvm-mc command to verify after the deletion. I have now fixed this description.

MaskRay · 2024-10-11T03:59:05Z

Does #111945 address your use case?

llvmbot · 2024-10-11T04:01:19Z

@llvm/issue-subscribers-lld-elf

Author: DianQK (DianQK)

I tried the following code. I expect lld to link successful while also removing unused symbols, but it reports `symbol not found: bar`.

a.s:

.global _start
_start:
 nop
.section .text.foo,"ax",@<!-- -->progbits
.global foo
foo:
 nop
.section .text.bar,"ax",@<!-- -->progbits
.global bar
bar:
 nop

script.t:

PROVIDE(foo = bar);

commands:

llvm-mc -filetype=obj -triple=x86_64 a.s -o a.o
ld.lld -o a_gc a.o --gc-sections -T script.t

output:

ld.lld: error: script.t:1: symbol not found: bar
ld.lld: error: script.t:1: symbol not found: bar
ld.lld: error: script.t:1: symbol not found: bar

cc @MaskRay ebb326a

tru · 2024-10-11T05:49:51Z

Does this need a backport?

DianQK · 2024-10-11T06:06:38Z

Does #111945 address your use case?

I have verified that #111945 has fixed the build error in rust-lang/rust#131164.

I’d like to ask for your advice on the backport. I reverted the related changes in rust-lang#178. What do you think is better, revert or cherry-pick the new patch?

DianQK · 2024-10-11T06:07:54Z

Does this need a backport?

Yes. I think so. :)

…to Undefined Case: `PROVIDE(f1 = bar);` when both `f1` and `bar` are in separate sections that would be discarded by GC. Due to `demoteDefined`, `shouldAddProvideSym(f1)` may initially return false (when Defined) and then return true (been demoted to Undefined). ``` addScriptReferencedSymbolsToSymTable shouldAddProvideSym(f1): false // the RHS (bar) is not added to `referencedSymbols` and may be GCed declareSymbols shouldAddProvideSym(f1): false markLive demoteSymbolsAndComputeIsPreemptible // demoted f1 to Undefined processSymbolAssignments addSymbol shouldAddProvideSym(f1): true ``` The inconsistency can cause `cmd->expression()` in `addSymbol` to be evaluated, leading to `symbol not found: bar` errors (since `bar` in the RHS is not in `referencedSymbols` and is GCed) (#111478). Fix this by adding a `sym->isUsedInRegularObj` condition, making `shouldAddProvideSym(f1)` values consistent. In addition, we need a `sym->exportDynamic` condition to keep provide-shared.s working. Fixes: ebb326a Pull Request: #111945

DianQK · 2024-10-12T02:08:29Z

/cherry-pick 1c6688a

llvmbot · 2024-10-12T02:13:34Z

Failed to cherry-pick: 1c6688a

https://github.com/llvm/llvm-project/actions/runs/11301798093

Please manually backport the fix and push it to your github fork. Once this is done, please create a pull request

…to Undefined Case: `PROVIDE(f1 = bar);` when both `f1` and `bar` are in separate sections that would be discarded by GC. Due to `demoteDefined`, `shouldAddProvideSym(f1)` may initially return false (when Defined) and then return true (been demoted to Undefined). ``` addScriptReferencedSymbolsToSymTable shouldAddProvideSym(f1): false // the RHS (bar) is not added to `referencedSymbols` and may be GCed declareSymbols shouldAddProvideSym(f1): false markLive demoteSymbolsAndComputeIsPreemptible // demoted f1 to Undefined processSymbolAssignments addSymbol shouldAddProvideSym(f1): true ``` The inconsistency can cause `cmd->expression()` in `addSymbol` to be evaluated, leading to `symbol not found: bar` errors (since `bar` in the RHS is not in `referencedSymbols` and is GCed) (llvm#111478). Fix this by adding a `sym->isUsedInRegularObj` condition, making `shouldAddProvideSym(f1)` values consistent. In addition, we need a `sym->exportDynamic` condition to keep provide-shared.s working. Fixes: ebb326a Pull Request: llvm#111945 (cherry picked from commit 1c6688a)

DianQK · 2024-10-13T13:37:49Z

Backport PR: #112136

The inaccurate #111945 condition fixes a PROVIDE regression (#111478) but introduces another regression: in a DSO link, if a symbol referenced only by bitcode files is defined as PROVIDE_HIDDEN, lld would not set the visibility correctly, leading to an assertion failure in DynamicReloc::getSymIndex (https://reviews.llvm.org/D123985). This is because `(sym->isUsedInRegularObj || sym->exportDynamic)` is initially false (bitcode undef does not set `isUsedInRegularObj`) then true (in `addSymbol`, after LTO compilation). Fix this by making the condition accurate: use a map to track defined symbols. Reviewers: smithp35 Reviewed By: smithp35 Pull Request: #112386

…to Undefined Case: `PROVIDE(f1 = bar);` when both `f1` and `bar` are in separate sections that would be discarded by GC. Due to `demoteDefined`, `shouldAddProvideSym(f1)` may initially return false (when Defined) and then return true (been demoted to Undefined). ``` addScriptReferencedSymbolsToSymTable shouldAddProvideSym(f1): false // the RHS (bar) is not added to `referencedSymbols` and may be GCed declareSymbols shouldAddProvideSym(f1): false markLive demoteSymbolsAndComputeIsPreemptible // demoted f1 to Undefined processSymbolAssignments addSymbol shouldAddProvideSym(f1): true ``` The inconsistency can cause `cmd->expression()` in `addSymbol` to be evaluated, leading to `symbol not found: bar` errors (since `bar` in the RHS is not in `referencedSymbols` and is GCed) (llvm#111478). Fix this by adding a `sym->isUsedInRegularObj` condition, making `shouldAddProvideSym(f1)` values consistent. In addition, we need a `sym->exportDynamic` condition to keep provide-shared.s working. Fixes: ebb326a Pull Request: llvm#111945

The inaccurate llvm#111945 condition fixes a PROVIDE regression (llvm#111478) but introduces another regression: in a DSO link, if a symbol referenced only by bitcode files is defined as PROVIDE_HIDDEN, lld would not set the visibility correctly, leading to an assertion failure in DynamicReloc::getSymIndex (https://reviews.llvm.org/D123985). This is because `(sym->isUsedInRegularObj || sym->exportDynamic)` is initially false (bitcode undef does not set `isUsedInRegularObj`) then true (in `addSymbol`, after LTO compilation). Fix this by making the condition accurate: use a map to track defined symbols. Reviewers: smithp35 Reviewed By: smithp35 Pull Request: llvm#112386

DianQK added lld regression:19 Regression in 19 release labels Oct 8, 2024

DianQK added this to the LLVM 19.X Release milestone Oct 8, 2024

github-project-automation bot added this to LLVM Release Status Oct 8, 2024

github-project-automation bot moved this to Needs Triage in LLVM Release Status Oct 8, 2024

This was referenced Oct 8, 2024

non-pub function no longer compiled in debug profile, causing link errors on thumbv7em-none-eabihf with defmt rust-lang/rust#131164

Closed

[ELF] PROVIDE: fix spurious "symbol not found" #87530

Closed

MaskRay mentioned this issue Oct 11, 2024

[ELF] Make shouldAddProvideSym return values consistent when demoted to Undefined #111945

Merged

EugeneZelenko added lld:ELF and removed lld labels Oct 11, 2024

tru moved this from Needs Triage to Needs Pull Request in LLVM Release Status Oct 11, 2024

DianQK closed this as completed Oct 12, 2024

github-project-automation bot moved this from Needs Pull Request to Done in LLVM Release Status Oct 12, 2024

EugeneZelenko added the release:backport label Oct 12, 2024

llvmbot added the release:cherry-pick-failed label Oct 12, 2024

DianQK reopened this Oct 12, 2024

github-project-automation bot moved this from Done to Needs Triage in LLVM Release Status Oct 12, 2024

DianQK closed this as completed Oct 13, 2024

github-project-automation bot moved this from Needs Triage to Done in LLVM Release Status Oct 13, 2024

MaskRay mentioned this issue Oct 15, 2024

[ELF] Fix PROVIDE_HIDDEN -shared regression with bitcode file references #112386

Merged

MaskRay mentioned this issue Oct 16, 2024

Backport "[ELF] Make shouldAddProvideSym return values consistent when demoted to Undefined" #112136

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[LLD] symbol not found with PROVIDE script #111478

[LLD] symbol not found with PROVIDE script #111478

DianQK commented Oct 8, 2024 •

edited

Loading

MaskRay commented Oct 9, 2024 •

edited

Loading

DianQK commented Oct 9, 2024

MaskRay commented Oct 11, 2024

llvmbot commented Oct 11, 2024

tru commented Oct 11, 2024

DianQK commented Oct 11, 2024

DianQK commented Oct 11, 2024

DianQK commented Oct 12, 2024

llvmbot commented Oct 12, 2024

DianQK commented Oct 13, 2024

[LLD] symbol not found with PROVIDE script #111478

[LLD] symbol not found with PROVIDE script #111478

Comments

DianQK commented Oct 8, 2024 • edited Loading

MaskRay commented Oct 9, 2024 • edited Loading

DianQK commented Oct 9, 2024

MaskRay commented Oct 11, 2024

llvmbot commented Oct 11, 2024

tru commented Oct 11, 2024

DianQK commented Oct 11, 2024

DianQK commented Oct 11, 2024

DianQK commented Oct 12, 2024

llvmbot commented Oct 12, 2024

DianQK commented Oct 13, 2024

DianQK commented Oct 8, 2024 •

edited

Loading

MaskRay commented Oct 9, 2024 •

edited

Loading