code_api|tool.histogram.offline hangs on ARM

[fhahn]

Splitting this off #2416 as it is not a flaky test.

bin32/drrun -s 90 -loglevel 3 -t drcachesim -- /home/pi/projects/dynamorio-space/build-debug/suite/tests/bin/pthreads.ptsig

.....
instrument_basic_block ******************

before instrumentation:
TAG  0x00011390
 +0    L3              b086b580   add.lt %r6 %r0 lsl $0x0b -> %r11
 +4    L3              60f8af00   rscs.vs %r8 %r0 lsl $0x1e -> %r10
 +8    L3              607a60b9   ldrht.vs (%r10)[2byte] $0xfffffff7 %r10 -> %r6 %r10
 +12   L3              2b0a68fb   bl.cs  $0x002ab790 -> %lr
END 0x00011390

drreg_event_bb_analysis @0.0x0001139c: r0=1 r1=1 r2=1 r3=1 r4=1 r5=1 r6=1 r7=1 r8=1 r9=1 r10=1 r11=1 r12=1 sp=1 lr=1 pc=1 flags=63
drreg_event_bb_analysis @1.0x00011398: r0=1 r1=1 r2=1 r3=1 r4=1 r5=1 r6=1 r7=1 r8=1 r9=1 r10=1 r11=1 r12=1 sp=1 lr=1 pc=1 flags=63
drreg_event_bb_analysis @2.0x00011394: r0=1 r1=1 r2=1 r3=1 r4=1 r5=1 r6=1 r7=1 r8=1 r9=1 r10=1 r11=1 r12=1 sp=1 lr=1 pc=1 flags=63
drreg_event_bb_analysis @3.0x00011390: r0=1 r1=1 r2=1 r3=1 r4=1 r5=1 r6=1 r7=1 r8=1 r9=1 r10=1 r11=1 r12=1 sp=1 lr=1 pc=1 flags=63
drreg_reserve_reg_internal @1.0x00011398: spilling r1 to slot 1
spill_reg @1.0x00011398 r1 1
mutex_delete 0 lock 0x4b0f21f0: name=dr_client_mutex(mutex)@/home/pi/projects/dynamorio-space/dynamorio/core/lib/instrument.c:3462
rank=41 owner=0 owning_dc=0x00000000 contended_event=0xffffffff prev=0x00000000
lock                             0                              0                               0                              0                               0+2 dr_client_mutex(mutex)@/home/pi/projects/dynamorio-space/dynamorio/core/lib/instrument.c:3462
remove_process_lock 0 lock 0x4b0f21f0: name=dr_client_mutex(mutex)@/home/pi/projects/dynamorio-space/dynamorio/core/lib/instrument.c:3462
rank=41 owner=0 owning_dc=0x00000000 contended_event=0xffffffff prev=0x00000000
lock                             0                              0                               0                              0                               0+2 dr_client_mutex(mutex)@/home/pi/projects/dynamorio-space/dynamorio/core/lib/instrument.c:3462
	never acquired
drreg_reserve_reg_internal @1.0x00011398: spilling r2 to slot 2
spill_reg @1.0x00011398 r2 2
drreg_unreserve_register @1.0x00011398 r2
drreg_reserve_reg_internal @1.0x00011398: using un-restored r2 slot 2
drreg_reserve_reg_internal @1.0x00011398: r2 already spilled to slot 2
drreg_unreserve_register @1.0x00011398 r2
drreg_reserve_reg_internal @1.0x00011398: using un-restored r2 slot 2
drreg_reserve_reg_internal @1.0x00011398: r2 already spilled to slot 2
drreg_unreserve_register @1.0x00011398 r2
drreg_reserve_reg_internal @1.0x00011398: using un-restored r2 slot 2
drreg_reserve_reg_internal @1.0x00011398: r2 already spilled to slot 2
drreg_unreserve_register @1.0x00011398 r2
drreg_unreserve_register @1.0x00011398 r1
drreg_reserve_reg_internal @0.0x0001139c: using un-restored r1 slot 1
drreg_reserve_reg_internal @0.0x0001139c: r1 already spilled to slot 1
mutex_delete 0 lock 0x4b0f21f0: name=dr_client_mutex(mutex)@/home/pi/projects/dynamorio-space/dynamorio/core/lib/instrument.c:3462
rank=41 owner=0 owning_dc=0x00000000 contended_event=0xffffffff prev=0x00000000
lock                             0                              0                               0                              0                               0+2 dr_client_mutex(mutex)@/home/pi/projects/dynamorio-space/dynamorio/core/lib/instrument.c:3462
remove_process_lock 0 lock 0x4b0f21f0: name=dr_client_mutex(mutex)@/home/pi/projects/dynamorio-space/dynamorio/core/lib/instrument.c:3462
rank=41 owner=0 owning_dc=0x00000000 contended_event=0xffffffff prev=0x00000000
lock                             0                              0                               0                              0                               0+2 dr_client_mutex(mutex)@/home/pi/projects/dynamorio-space/dynamorio/core/lib/instrument.c:3462
	never acquired
drreg_reserve_reg_internal @0.0x0001139c: using un-restored r2 slot 2
drreg_reserve_reg_internal @0.0x0001139c: r2 already spilled to slot 2
drreg_unreserve_register @0.0x0001139c r2
drreg_reserve_reg_internal @0.0x0001139c: using un-restored r2 slot 2
drreg_reserve_reg_internal @0.0x0001139c: r2 already spilled to slot 2
CLEANCALL: insert clean call to 0x76f2d199
drreg_unreserve_register @0.0x0001139c r2
drreg_unreserve_register @0.0x0001139c r1
drreg_event_bb_insert_late @0.0x0001139c: lazily restoring r1
drreg_restore_reg_now @0.0x0001139c: restoring r1
restore_reg @0.0x0001139c r1 slot=1 release=1
drreg_event_bb_insert_late @0.0x0001139c: lazily restoring r2
drreg_restore_reg_now @0.0x0001139c: restoring r2
restore_reg @0.0x0001139c r2 slot=2 release=1

after instrumentation:
TAG  0x00011390
 +0    L3              b086b580   add.lt %r6 %r0 lsl $0x0b -> %r11
 +4    L3              60f8af00   rscs.vs %r8 %r0 lsl $0x1e -> %r10
 +8    m4 @0x4b18a64c  e58a1084   str    %r1 -> +0x84(%r10)[4byte]
 +12   m4 @0x4b18bae4  e59a108c   ldr    +0x8c(%r10)[4byte] -> %r1
 +16   m4 @0x4b18f27c  e58a2088   str    %r2 -> +0x88(%r10)[4byte]
 +20   m4 @0x4b1922a4  e300200a   movw   $0x0000000a -> %r2
 +24   m4 @0x4b14e238  e3402004   movt   $0x00000004 -> %r2
 +28   m4 @0x4b1906a0  e5812000   str    %r2 -> (%r1)[4byte]
 +32   m4 @0x4b18c030  e3012390   movw   $0x00001390 -> %r2
 +36   m4 @0x4b18a4d0  e3402001   movt   $0x00000001 -> %r2
 +40   m4 @0x4b18e724  e5812004   str    %r2 -> +0x04(%r1)[4byte]
 +44   m4 @0x4b191f6c  e3002011   movw   $0x00000011 -> %r2
 +48   m4 @0x4b194c1c  e3402001   movt   $0x00000001 -> %r2
 +52   m4 @0x4b14ea40  e5812008   str    %r2 -> +0x08(%r1)[4byte]
 +56   m4 @0x4b18aedc  e3002004   movw   $0x00000004 -> %r2
 +60   m4 @0x4b18dcc0  e581200c   str    %r2 -> +0x0c(%r1)[4byte]
 +64   m4 @0x4b1927a4  e300200a   movw   $0x0000000a -> %r2
 +68   m4 @0x4b14f318  e3402004   movt   $0x00000004 -> %r2
 +72   m4 @0x4b18b5f0  e5812010   str    %r2 -> +0x10(%r1)[4byte]
 +76   m4 @0x4b197070  e3012398   movw   $0x00001398 -> %r2
 +80   m4 @0x4b18b0b0  e3402001   movt   $0x00000001 -> %r2
 +84   m4 @0x4b18b310  e5812014   str    %r2 -> +0x14(%r1)[4byte]
 +88   m4 @0x4b190a30  e5812014   <label>
 +88   m4 @0x4b192628  e2811018   add    %r1 $0x00000018 -> %r1
 +92   m4 @0x4b192a9c  e58a108c   str    %r1 -> +0x8c(%r10)[4byte]
 +96   m4 @0x4b19259c  659a2010   ldr.vs +0x10(%r10)[4byte] -> %r2
 +100  m4 @0x4b189b90  65812004   str.vs %r2 -> +0x04(%r1)[4byte]
 +104  m4 @0x4b18b9f4  63002000   movw.vs $0x00000000 -> %r2
 +108  m4 @0x4b18c40c  63402002   movt.vs $0x00000002 -> %r2
 +112  m4 @0x4b18dd58  65812000   str.vs %r2 -> (%r1)[4byte]
 +116  m4 @0x4b18b85c  65812000   <label>
 +116  m4 @0x4b19388c  62811008   add.vs %r1 $0x00000008 -> %r1
 +120  m4 @0x4b15081c  658a108c   str.vs %r1 -> +0x8c(%r10)[4byte]
 +124  m4 @0x4b18fbfc  658a108c   <label>
 +124  L3              607a60b9   ldrht.vs (%r10)[2byte] $0xfffffff7 %r10 -> %r6 %r10
 +128  m4 @0x4b1934a0  e59a108c   ldr    +0x8c(%r10)[4byte] -> %r1
 +132  m4 @0x4b18c1ac  e300200e   movw   $0x0000000e -> %r2
 +136  m4 @0x4b189d18  e3402004   movt   $0x00000004 -> %r2
 +140  m4 @0x4b18af28  e5812000   str    %r2 -> (%r1)[4byte]
 +144  m4 @0x4b18f0d8  e301239c   movw   $0x0000139c -> %r2
 +148  m4 @0x4b19361c  e3402001   movt   $0x00000001 -> %r2
 +152  m4 @0x4b18e7fc  e5812004   str    %r2 -> +0x04(%r1)[4byte]
 +156  m4 @0x4b18f77c  e5812004   <label>
 +156  m4 @0x4b191f14  e2811008   add    %r1 $0x00000008 -> %r1
 +160  m4 @0x4b150738  e58a108c   str    %r1 -> +0x8c(%r10)[4byte]
 +164  m4 @0x4b18c458  e5911000   ldr    (%r1)[4byte] -> %r1
 +168  m4 @0x4b18b26c  e10f2000   mrs    %cpsr -> %r2
 +172  m4 @0x4b192be8  e3510000   cmp    %r1 $0x00000000
 +176  m4 @0x4b192ccc  0afffffe   b.eq   @0x4b18cd40[4byte]
 +180  m4 @0x4b18fee8  e58a0000   str    %r0 -> (%r10)[4byte]
 +184  m4 @0x4b18d02c  e59a0014   ldr    +0x14(%r10)[4byte] -> %r0
 +188  m4 @0x4b18cad4  e580d034   str    %sp -> +0x34(%r0)[4byte]
 +192  m4 @0x4b192914  e590d16c   ldr    +0x016c(%r0)[4byte] -> %sp
 +196  m4 @0x4b18dc74  e59a0000   ldr    (%r10)[4byte] -> %r0
 +200  m4 @0x4b189a60  ed6d0b20   vstmdb %d16 %d17 %d18 %d19 %d20 %d21 %d22 %d23 %d24 %d25 %d26 %d27 %d28 %d29 %d30 %d31 %sp -> (%sp) %sp
 +204  m4 @0x4b1903f4  ed2d0b20   vstmdb %d0 %d1 %d2 %d3 %d4 %d5 %d6 %d7 %d8 %d9 %d10 %d11 %d12 %d13 %d14 %d15 %sp -> (%sp) %sp
 +208  m4 @0x4b19333c  e58a0000   str    %r0 -> (%r10)[4byte]
 +212  m4 @0x4b190ff0  e10f0000   mrs    %cpsr -> %r0
 +216  m4 @0x4b190e4c  e52d0004   str    %r0 $0xfffffffc %sp -> -0x04(%sp)[4byte] %sp
 +220  m4 @0x4b1914a4  e3000000   movw   $0x00000000 -> %r0
 +224  m4 @0x4b1903b4  e52d0004   str    %r0 $0xfffffffc %sp -> -0x04(%sp)[4byte] %sp
 +228  m4 @0x4b14e094  e59a0000   ldr    (%r10)[4byte] -> %r0
 +232  m4 @0x4b18db38  e92d7fff   stmdb  %r0 %r1 %r2 %r3 %r4 %r5 %r6 %r7 %r8 %r9 %r10 %r11 %r12 %sp %lr %sp -> (%sp) %sp
 +236  m4 @0x4b18cb6c  e30db199   movw   $0x0000d199 -> %r11
 +240  m4 @0x4b18f950  e347b6f2   movt   $0x000076f2 -> %r11
 +244  m4 @0x4b14e794  e120003b   blx    %r11 -> %lr
 +248  m4 @0x4b190868  e8bd1fff   ldm    (%sp) %sp -> %r0 %r1 %r2 %r3 %r4 %r5 %r6 %r7 %r8 %r9 %r10 %r11 %r12 %sp
 +252  m4 @0x4b18d1f4  e28dd004   add    %sp $0x00000004 -> %sp
 +256  m4 @0x4b18ad78  e49de004   ldr    (%sp)[4byte] $0x00000004 %sp -> %lr %sp
 +260  m4 @0x4b1916c4  e28dd004   add    %sp $0x00000004 -> %sp
 +264  m4 @0x4b191678  e58a0000   str    %r0 -> (%r10)[4byte]
 +268  m4 @0x4b192200  e49d0004   ldr    (%sp)[4byte] $0x00000004 %sp -> %r0 %sp
 +272  m4 @0x4b192d70  e12cf000   msr    $0x0c %r0 -> %cpsr
 +276  m4 @0x4b14ff8c  e59a0000   ldr    (%r10)[4byte] -> %r0
 +280  m4 @0x4b1970bc  ecbd0b20   vldm   (%sp) %sp -> %d0 %d1 %d2 %d3 %d4 %d5 %d6 %d7 %d8 %d9 %d10 %d11 %d12 %d13 %d14 %d15 %sp
 +284  m4 @0x4b1932bc  ecfd0b20   vldm   (%sp) %sp -> %d16 %d17 %d18 %d19 %d20 %d21 %d22 %d23 %d24 %d25 %d26 %d27 %d28 %d29 %d30 %d31 %sp
 +288  m4 @0x4b18f3ac  e58a0000   str    %r0 -> (%r10)[4byte]
 +292  m4 @0x4b19211c  e59a0014   ldr    +0x14(%r10)[4byte] -> %r0
 +296  m4 @0x4b18adb8  e590d034   ldr    +0x34(%r0)[4byte] -> %sp
 +300  m4 @0x4b190d68  e59a0000   ldr    (%r10)[4byte] -> %r0
 +304  m4 @0x4b188cf8  e59a0000   <label>
 +304  m4 @0x4b18cd40  e59a0000   <label>
 +304  m4 @0x4b190e0c  e12cf002   msr    $0x0c %r2 -> %cpsr
 +308  m4 @0x4b193c74  e12cf002   <label>
 +308  m4 @0x4b18ef5c  e12cf002   <label>
 +308  m4 @0x4b18d318  e59a1084   ldr    +0x84(%r10)[4byte] -> %r1
 +312  m4 @0x4b14ead8  e59a2088   ldr    +0x88(%r10)[4byte] -> %r2
 +316  L3              2b0a68fb   bl.cs  $0x002ab790 -> %lr
END 0x00011390

setting cur_pc (for fall-through) to 0x000113a0
exit_branch_type=0x0 bb->exit_target=0x000113a0
exit_branch_type=0x0 target=0x002ab790 l->flags=0x1
exit_branch_type=0x0 target=0x000113a0 l->flags=0x1001
Exit cti 0x4b2fe480 is targeting 0x4b2fe488 + 0x0 => 0x4b2fe488
Exit cti 0x4b2fe484 is targeting 0x4b2fe49c + 0x0 => 0x4b2fe49c
Fragment 1910, tag 0x00011390, flags 0x9000030, shared, size 424:

Entry into F1910(0x00011390).0x4b2fe30c (A32)(shared)

Exit from F1910(0x00011390).0x4b2fe480 (shared)
 (target 0x002ab790 not in cache)
receive_pending_signal
	clearing signals_pending flag

dispatch: target = 0x002ab790
application tried to execute from unreadable 0x002ab790 is_allocated_mem=0 prot=0x0
Call stack:
	0x002ab790
	frame ptr 0x76a3d410 => parent 0x6c6c616d, 0x2928636f
SYSLOG_WARNING: Application tried to execute from unreadable memory 0x002ab790.
This may be a result of an unsuccessful attack or a potential application vulnerability.
record_pending_signal(11 at pc 0x002ab790): signal is currently blocked
	action is not SIG_IGN
copy_frame_to_pending from 0x4b149974
sigcontext:
	r0  =0x0000000a
	r1  =0x7ec97ef0
	r2  =0x7ec97f70
	r3  =0x00020f70
	r4  =0x76a597c4
	r5  =0x76a59794
	r6  =0x00020f70
	r7  =0x76a597c4
	r8  =0x00000090
	r9  =0x00002710
	r10 =0x000001ff
	r11 =0x76a3d410
	r12 =0x0000010c
	sp  =0x7ec97ef0
	r14 =0x000113a0
	pc  =0x002ab790
	cpsr=0x20000010
transfer_to_dispatch: pc=0x002ab790, xsp=0x7ec97ef0, initstack=0
Exit from asynch event

Call stack:
	0x002ab790
	frame ptr 0x76a3d410 => parent 0x6c6c616d, 0x2928636f
record_pending_signal(11 at pc 0x002ab790): signal is currently blocked
	action is not SIG_IGN
	non-rt signal already in queue, ignoring this one!
transfer_to_dispatch: pc=0x002ab790, xsp=0x7ec97ef0, initstack=0
Exit from asynch event

[fhahn]

Looks like we are branching to unreadable memory at 0x002ab790. Maybe something messed up the status register.

[derekbruening]

I don't think this one is reproducible? I can't get histogram.offline to fail in debug or release build on the Jenkins machine (so having trouble reproducing #3980 too). Let's re-open if we see this again; there have been numerous a64 fixes since this was filed.