Doesn't compile on OSX Lion (10.7)

[ramosian-glider]

Originally reported on Google Code with ID 32

What steps will reproduce the problem?
1. Get the sources
2. Run the specified command (using Makefile.old)

What is the expected output? What do you see instead?

A series of tests and a successful build.

instead I get : 

----
/private/tmp/wwwi/llvm/projects/compiler-rt/lib/asan/../../../../build/Release+Asserts/bin/clang++
 -faddress-sanitizer -mllvm -asan-blacklist=/private/tmp/wwwi/llvm/projects/compiler-rt/lib/asan/tests/asan_test.ignore
-mllvm -asan-stack=1 -mllvm -asan-globals=1 -mllvm -asan-mapping-scale=0   -mllvm -asan-mapping-offset-log=-1
  -mllvm -asan-use-after-return=0 -DASAN_UAR=0 -DASAN_HAS_EXCEPTIONS=1 -DASAN_NEEDS_SEGV=1
-DASAN_HAS_BLACKLIST=1 -I.. -I../include -Wall -fvisibility=hidden -m64 -g -w sample1.o
sample1_unittest.o gtest_main.a -o sample1_unittest
clang: warning: argument unused during compilation: '-mllvm -asan-blacklist=/private/tmp/wwwi/llvm/projects/compiler-rt/lib/asan/tests/asan_test.ignore'
clang: warning: argument unused during compilation: '-mllvm -asan-stack=1'
clang: warning: argument unused during compilation: '-mllvm -asan-globals=1'
clang: warning: argument unused during compilation: '-mllvm -asan-mapping-scale=0'
clang: warning: argument unused during compilation: '-mllvm -asan-mapping-offset-log=-1'
clang: warning: argument unused during compilation: '-mllvm -asan-use-after-return=0'
/private/tmp/wwwi/llvm/projects/compiler-rt/lib/asan/../../../../build/Release+Asserts/bin/clang++
 -faddress-sanitizer  -Wall -fvisibility=hidden -m64 -g -O3 bin_darwin/asan_test64.o
bin_darwin/asan_globals_test64.o bin_darwin/asan_break_optimization64.o bin_darwin/asan_noinst_test64.o
bin_darwin/asan_interface_test64.o bin_darwin/asan_mac_test64.o \
        -framework Foundation -o bin_darwin/asan_test64  third_party/googletest/make-darwin64/gtest-all.o

Assertion failed: (0), function targetContent, file /SourceCache/ld64/ld64-127.2/src/ld/parsers/macho_relocatable_file.cpp,
line 4898.
0  0x10920c041  __assert_rtn + 81
1  0x10924565e  mach_o::relocatable::CFStringSection<x86_64>::targetContent(mach_o::relocatable::Atom<x86_64>
const*, ld::IndirectBindingTable const&, mach_o::relocatable::CFStringSection<x86_64>::ContentType*,
unsigned int*) + 398
2  0x10924529e  mach_o::relocatable::CFStringSection<x86_64>::contentHash(mach_o::relocatable::Atom<x86_64>
const*, ld::IndirectBindingTable const&) const + 30
3  0x10924304a  mach_o::relocatable::Atom<x86_64>::contentHash(ld::IndirectBindingTable
const&) const + 42
4  0x109273d97  ld::tool::SymbolTable::findSlotForReferences(ld::Atom const*, ld::Atom
const**) + 439
5  0x109273b3f  ld::tool::SymbolTable::addByReferences(ld::Atom const&) + 31
6  0x109273f3a  ld::tool::SymbolTable::add(ld::Atom const&, bool) + 58
7  0x10927771a  ld::tool::Resolver::doAtom(ld::Atom const&) + 698
8  0x10924714c  mach_o::relocatable::File<x86_64>::forEachAtom(ld::File::AtomHandler&)
const + 60
9  0x10927022c  ld::tool::InputFiles::forEachInitialAtom(ld::File::AtomHandler&) const
+ 44
10  0x109279eda  ld::tool::Resolver::resolve() + 202
11  0x10920c345  main + 549
12  0x1091fb9a4  start + 52
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [bin_darwin/asan_test64] Error 1
make: *** [b64] Error 2
-----

What version of the product are you using? On what operating system?

$ xcodebuild -version
4.2.1
Build version 4D502

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.7.2
BuildVersion:   11C74

Please provide any additional information below.

Svn revision : 149195

Reported by mkvtoolnix.build.jonthn on 2012-01-29 16:22:19

[ramosian-glider]

Reported by ramosian.glider on 2012-01-30 06:48:59

• Status changed: Accepted

[ramosian-glider]

This is reproducible for me. I consider this a bug in ld, but it is triggered only with
-faddress-sanitizer (I'm reducing the testcase now), so we can probably make some workaround.

Reported by ramosian.glider on 2012-01-30 08:10:10

[ramosian-glider]

$ cat a_t.ii
typedef const struct __CFString * CFStringRef;
void TestBody() {
  CFStringRef str = ((CFStringRef) __builtin___CFStringMakeConstantString ("" "Hello
world!\n" ""));
}


$ /Users/glider/src/asan/llvm/projects/compiler-rt/lib/asan/../../../../build/Release+Asserts/bin/clang++
 -faddress-sanitizer  a_t.ii -o a_t.o 2>&1 -v
clang version 3.1 (trunk 149199)
Target: x86_64-apple-darwin11.2.0
Thread model: posix
 "/Users/glider/src/asan/llvm/build/Release+Asserts/bin/clang" -cc1 -triple x86_64-apple-macosx10.7.2
-emit-obj -mrelax-all -disable-free -main-file-name a_t.ii -pic-level 1 -mdisable-fp-elim
-masm-verbose -munwind-tables -target-cpu core2 -target-linker-version 127.2 -v -resource-dir
/Users/glider/src/asan/llvm/build/Release+Asserts/bin/../lib/clang/3.1 -fdeprecated-macro
-fdebug-compilation-dir /Users/glider/src/asan/llvm/projects/compiler-rt/lib/asan -ferror-limit
19 -fmessage-length 145 -faddress-sanitizer -stack-protector 1 -mstackrealign -fblocks
-fobjc-runtime-has-arc -fobjc-runtime-has-weak -fobjc-dispatch-method=mixed -fcxx-exceptions
-fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o /var/folders/00/0k0r8000h01000cxqpysvccm002831/T/a_t-hJP3Ut.o
-x c++-cpp-output a_t.ii
clang -cc1 version 3.1 based upon LLVM 3.1svn default target x86_64-apple-darwin11.2.0
ignoring nonexistent directory "/usr/include/c++/4.2.1/i686-apple-darwin10/x86_64"
ignoring nonexistent directory "/usr/include/c++/4.0.0"
ignoring nonexistent directory "/usr/include/c++/4.0.0/i686-apple-darwin8/"
ignoring nonexistent directory "/usr/include/c++/4.0.0/backward"
ignoring nonexistent directory "/usr/local/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/include/c++/4.2.1
 /usr/include/c++/4.2.1/backward
 /Users/glider/src/asan/llvm/build/Release+Asserts/bin/../lib/clang/3.1/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
 "/usr/llvm-gcc-4.2/libexec/gcc/i686-apple-darwin11/4.2.1/ld" -demangle -dynamic -arch
x86_64 -macosx_version_min 10.7.2 -o a_t.o -lcrt1.10.6.o /var/folders/00/0k0r8000h01000cxqpysvccm002831/T/a_t-hJP3Ut.o
-lstdc++ /Users/glider/src/asan/llvm/build/Release+Asserts/bin/../lib/clang/3.1/lib/darwin/libclang_rt.asan_osx.a
-lstdc++ -framework CoreFoundation -lSystem /Users/glider/src/asan/llvm/build/Release+Asserts/bin/../lib/clang/3.1/lib/darwin/libclang_rt.osx.a
Assertion failed: (0), function targetContent, file /SourceCache/ld64/ld64-127.2/src/ld/parsers/macho_relocatable_file.cpp,
line 4898.
0  0x10fe20041  __assert_rtn + 81
1  0x10fe5965e  mach_o::relocatable::CFStringSection<x86_64>::targetContent(mach_o::relocatable::Atom<x86_64>
const*, ld::IndirectBindingTable const&, mach_o::relocatable::CFStringSection<x86_64>::ContentType*,
unsigned int*) + 398
2  0x10fe5929e  mach_o::relocatable::CFStringSection<x86_64>::contentHash(mach_o::relocatable::Atom<x86_64>
const*, ld::IndirectBindingTable const&) const + 30
3  0x10fe5704a  mach_o::relocatable::Atom<x86_64>::contentHash(ld::IndirectBindingTable
const&) const + 42
4  0x10fe87d97  ld::tool::SymbolTable::findSlotForReferences(ld::Atom const*, ld::Atom
const**) + 439
5  0x10fe87b3f  ld::tool::SymbolTable::addByReferences(ld::Atom const&) + 31
6  0x10fe87f3a  ld::tool::SymbolTable::add(ld::Atom const&, bool) + 58
7  0x10fe8b71a  ld::tool::Resolver::doAtom(ld::Atom const&) + 698
8  0x10fe5b14c  mach_o::relocatable::File<x86_64>::forEachAtom(ld::File::AtomHandler&)
const + 60
9  0x10fe8422c  ld::tool::InputFiles::forEachInitialAtom(ld::File::AtomHandler&) const
+ 44
10  0x10fe8deda  ld::tool::Resolver::resolve() + 202
11  0x10fe20345  main + 549
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Reported by ramosian.glider on 2012-01-30 09:08:50

[ramosian-glider]

$ /Users/glider/src/asan/llvm/projects/compiler-rt/lib/asan/../../../../build/Release+Asserts/bin/clang++
a_t.ii -S -emit-llvm -o a_t.ll
$ /Users/glider/src/asan/llvm/projects/compiler-rt/lib/asan/../../../../build/Release+Asserts/bin/clang++
a_t.ii -S -emit-llvm -o a_t_asan.ll -faddress-sanitizer
=============================
$ cat a_t.ll; ModuleID = 'a_t.ii'
target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
target triple = "x86_64-apple-macosx10.7.2"

%struct.NSConstantString = type { i32*, i32, i8*, i64 }
%struct.__CFString = type opaque

@__CFConstantStringClassReference = external global [0 x i32]
@.str = linker_private unnamed_addr constant [14 x i8] c"Hello world!\0A\00", align
1
@_unnamed_cfstring_ = private constant %struct.NSConstantString { i32* getelementptr
inbounds ([0 x i32]* @__CFConstantStringClassReference, i32 0, i32 0), i32 1992, i8*
getelementptr inbounds ([14 x i8]* @.str, i32 0, i32 0), i64 13 }, section "__DATA,__cfstring"

define void @_Z8TestBodyv() nounwind uwtable ssp {
entry:
  %str = alloca %struct.__CFString*, align 8
  store %struct.__CFString* bitcast (%struct.NSConstantString* @_unnamed_cfstring_
to %struct.__CFString*), %struct.__CFString** %str, align 8
  ret void
}
=============================
$ cat a_t_asan.ll
; ModuleID = 'a_t.ii'
target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
target triple = "x86_64-apple-macosx10.7.2"

%struct.NSConstantString = type { i32*, i32, i8*, i64 }
%struct.__CFString = type opaque

@__CFConstantStringClassReference = external global [0 x i32]
@.str = linker_private unnamed_addr constant [14 x i8] c"Hello world!\0A\00", align
1
@_unnamed_cfstring_ = private constant { %struct.NSConstantString, [64 x i8] } { %struct.NSConstantString
{ i32* getelementptr inbounds ([0 x i32]* @__CFConstantStringClassReference, i32 0,
i32 0), i32 1992, i8* getelementptr inbounds ([14 x i8]* @.str, i32 0, i32 0), i64
13 }, [64 x i8] zeroinitializer }, section "__DATA,__cfstring", align 32
@0 = private constant [28 x i8] c"_unnamed_cfstring_ (a_t.ii)\00"
@1 = private global [1 x { i64, i64, i64, i64 }] [{ i64, i64, i64, i64 } { i64 ptrtoint
({ %struct.NSConstantString, [64 x i8] }* @_unnamed_cfstring_ to i64), i64 32, i64
96, i64 ptrtoint ([28 x i8]* @0 to i64) }]
@llvm.global_dtors = appending global [1 x { i32, void ()* }] [{ i32, void ()* } {
i32 1, void ()* @asan.module_dtor }]
@__asan_mapping_offset = linkonce_odr constant i64 17592186044416
@__asan_mapping_scale = linkonce_odr constant i64 3
@2 = private constant [27 x i8] c"_Z8TestBodyv 1 32 8 3 str \00"
@llvm.global_ctors = appending global [1 x { i32, void ()* }] [{ i32, void ()* } {
i32 1, void ()* @asan.module_ctor }]

define void @_Z8TestBodyv() nounwind uwtable ssp address_safety {
entry:
  %MyAlloca = alloca [96 x i8], align 32
  %0 = ptrtoint [96 x i8]* %MyAlloca to i64
  %1 = add i64 %0, 32
  %2 = inttoptr i64 %1 to %struct.__CFString**
  %3 = inttoptr i64 %0 to i64*
  store i64 1102416563, i64* %3
  %4 = add i64 %0, 8
  %5 = inttoptr i64 %4 to i64*
  store i64 ptrtoint ([27 x i8]* @2 to i64), i64* %5
  %6 = lshr i64 %0, 3
  %7 = or i64 %6, 17592186044416
  %8 = inttoptr i64 %7 to i32*
  store i32 -235802127, i32* %8
  %9 = add i64 %7, 4
  %10 = inttoptr i64 %9 to i32*
  store i32 -185273344, i32* %10
  %11 = add i64 %7, 8
  %12 = inttoptr i64 %11 to i32*
  store i32 -202116109, i32* %12
  %str = alloca %struct.__CFString*, align 8
  %13 = ptrtoint %struct.__CFString** %2 to i64
  %14 = lshr i64 %13, 3
  %15 = or i64 %14, 17592186044416
  %16 = inttoptr i64 %15 to i8*
  %17 = load i8* %16
  %18 = icmp ne i8 %17, 0
  br i1 %18, label %25, label %19

; <label>:19                                      ; preds = %entry
  store %struct.__CFString* bitcast ({ %struct.NSConstantString, [64 x i8] }* @_unnamed_cfstring_
to %struct.__CFString*), %struct.__CFString** %2, align 8
  store i64 1172321806, i64* %3
  %20 = inttoptr i64 %7 to i32*
  store i32 0, i32* %20
  %21 = add i64 %7, 4
  %22 = inttoptr i64 %21 to i32*
  store i32 0, i32* %22
  %23 = add i64 %7, 8
  %24 = inttoptr i64 %23 to i32*
  store i32 0, i32* %24
  ret void

; <label>:25                                      ; preds = %entry
  call void @__asan_report_store8(i64 %13) noreturn
  unreachable
}

define internal void @asan.module_ctor() {
  call void @__asan_init()
  call void @__asan_register_globals(i64 ptrtoint ([1 x { i64, i64, i64, i64 }]* @1
to i64), i64 1)
  %1 = load volatile i64* @__asan_mapping_scale
  %2 = load volatile i64* @__asan_mapping_offset
  ret void
}

declare void @__asan_init()

declare void @__asan_register_globals(i64, i64)

define internal void @asan.module_dtor() {
  call void @__asan_unregister_globals(i64 ptrtoint ([1 x { i64, i64, i64, i64 }]*
@1 to i64), i64 1)
  ret void
}

declare void @__asan_unregister_globals(i64, i64)

declare void @__asan_report_store8(i64)

Reported by ramosian.glider on 2012-01-30 09:35:09

[ramosian-glider]

Here's the layout of an instrumented NSString global that causes ld to crash:

========================
$ cat a_t.S
    .section    __TEXT,__text,regular,pure_instructions
    .section    __TEXT,__cstring,cstring_literals
l_.str:                                 ## @.str
    .asciz   "Hello world!\n"

    .section    __DATA,__cfstring
    .align  4                       ## @_unnamed_cfstring_
L__unnamed_cfstring_:
    .quad   ___CFConstantStringClassReference
    .long   1992                    ## 0x7c8
    .space  4
    .quad   l_.str
    .quad   13                      ## 0xd
    .space  64

.subsections_via_symbols
========================

(See http://jens.ayton.se/blag/objc-constant-objects/ for the description of NSString
fields)

Removing the redzone (the ".space 64" line) fixes the problem.

Assuming that nobody writes the code that accesses the string in "__DATA,__cfstring"
manually, we actually do not want to add redzones to such objects.

We may probably want to add a redzone at the end of l_.str string instead -- this will
require ASan to instrument linker_private strings.
Kostya, WDYT? I understand that linker_private symbols are removed from the executables,
but we may still want to instrument them in order to prevent incorrect accesses.

Reported by ramosian.glider on 2012-01-30 10:13:01

[ramosian-glider]

r149243 should fix the problem.
However, the tests crash on Lion due to some memcpy/memmove interception problem. I'm
going to take a look.

Reported by ramosian.glider on 2012-01-30 10:45:19

[ramosian-glider]

See issue 34 for the memcpy discussion.

Reported by ramosian.glider on 2012-01-30 13:03:42

[ramosian-glider]

As per discussion with Kostya, we'll not instrument the linker_private symbols for now,
but we may want to get back to it someday.
Closing the bug.

Reported by ramosian.glider on 2012-02-01 09:32:27

• Status changed: Fixed

[ramosian-glider]

Reported by ramosian.glider on 2012-09-13 13:40:24

• Labels added: OpSys-OSX

[ramosian-glider]

Adding Project:AddressSanitizer as part of GitHub migration.

Reported by ramosian.glider on 2015-07-30 09:12:58

• Labels added: ProjectAddressSanitizer