Problem decoding instruction in the libc : endbr32 / endbr64

[laluka]

OS / Environment

LSB Version: n/a
Distributor ID: ManjaroLinux
Description: Manjaro Linux
Release: 18.0.0-rc
Codename: Illyria

Manticore version

Version: 0.2.1.1

Python version

Python 2.7.15 - But same results with the 3.7.0

Dependencies

appdirs==1.4.3
asn1crypto==0.24.0
autopep8==1.4
backcall==0.1.0
bcrypt==3.1.4
binwalk==2.1.1
btrfsutil==1.0.0
capstone==3.0.5
cffi==1.11.5
chardet==3.0.4
cryptography==2.3.1
cupshelpers==1.0
cycler==0.10.0
decorator==4.3.0
docopt==0.6.2
future==0.16.0
gufw==18.10.0
httplib2==0.11.3
idna==2.7
intervaltree==2.1.0
ipython==6.5.0
ipython-genutils==0.1.0
isort==4.3.4
jedi==0.13.0
keystone-engine==0.9.1.post3
kiwisolver==1.0.1
lensfun==0.3.2
louis==3.7.0
Mako==1.0.7
manticore==0.2.1.1
MarkupSafe==1.0
matplotlib==2.2.3
npyscreen==4.10.5
numpy==1.15.2
openshot-qt==2.4.3
packaging==18.0
pacman-mirrors==4.12.5
paramiko==2.4.2
parso==0.3.1
pexpect==4.6.0
pickleshare==0.7.4
Pillow==5.3.0
pluggy==0.7.1
ply==3.11
prompt-toolkit==1.0.15
psutil==5.4.7
ptyprocess==0.6.0
pwn==1.0
pwntools==3.12.1
py==1.6.0
pyasn1==0.4.4
pycairo==1.17.0
pycodestyle==2.4.0
pycparser==2.19
pycups==1.9.73
pycurl==7.43.0.2
pyelftools==0.25
pyevmasm==0.1.1
Pygments==2.2.0
PyGObject==3.30.1
PyNaCl==1.2.1
pypandoc==1.4
pyparsing==2.2.2
pypng==0.0.18
python-dateutil==2.7.3
python-ptrace==0.9.3
python-xapp==1.2.0
pytz==2018.5
PyYAML==3.13
pyzmq==17.1.0
requests==2.19.1
ROPGadget==5.3
setproctitle==1.1.10
simplegeneric==0.8.1
sip==4.19.13
six==1.11.0
sortedcontainers==1.5.10
ufw==0.35
unicorn==1.0.1
urllib3==1.23
virtualenv==16.0.0
z3-solver==4.8.0.0.post1

Summary of the problem

Manticore seems unable to decode endbr64 or endbr32, present in the libc, thus making it impossible to use dynamically linked binaries

Step to reproduce the behavior

$ manticore vuln
[laluka@laluka-pc manticore]$ file vuln
vuln: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=b52cd255bf1c93aee7d90da797ec1c9e7c2e5497, not stripped
[laluka@laluka-pc manticore]$ ldd vuln
linux-vdso.so.1 (0x00007fff0e5e7000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007ff698434000)
/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007ff69863e000)
[laluka@laluka-pc manticore]$ readelf -h vuln
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
Class: ELF64
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: Advanced Micro Devices X86-64
Version: 0x1
Entry point address: 0x4004a0
Start of program headers: 64 (bytes into file)
Start of section headers: 5184 (bytes into file)
Flags: 0x0
Size of this header: 64 (bytes)
Size of program headers: 56 (bytes)
Number of program headers: 8
Size of section headers: 64 (bytes)
Number of section headers: 30
Section header string table index: 27

Expected behavior

Decode endbr64 the right way, or 'skip' it, or stub the libc ?

Actual behavior

Any not static binary, x86 or x86-64 crashes while analyzing such instruction in the libc

Any relevant logs

manticore.core.cpu.abstractcpu.DecodeException: ('Error decoding instruction @%08x', 140737488051712)
At : 0x7ffffffb5e00 - endbr64 - OPCODES : 0xfa1e0ff3

Any help would me much appreciated, thank you for the good work, keep it up ! ;)

[laluka]

Related

capstone-engine/capstone#1129

It seems that it's a new instruction from intel, used for Indirect Branch Tracking (IBT), imo to catch up with the meltdown / spectre things, it seems that it has been fixed in new versions of capstone, wtill working on it, will keep you informed.

But even if it workd, adding stubs for the libc, or a way to disable libc's parsing would be a good idea, if not necessary... :)

... Aaaaaand my capstone is broken. Yay ! \o/
Updating with pip isn't enough, neither is building from the git -> I stop my research here as I have others projects going on. Good luck ! :)

[disconnect3d]

Hey, @TheLaluka can you provide an example source code/binary that triggers this so we can reproduce?

[laluka]

Hi, sure !
So, the problem is, as said, with the endbr{32,64} instructions, they are not 'in' the binary I uploaded, but they are in the libc attached. So once the linker linnks the libc and the program's libc is analysed, a crash will happend at the first endbr64 occurence. So either implement it, or add stubs for the libc.
But in the end... Both should be done, one for fixing the project, the other to fastening the analysis process. But The stubs may be long to implement, maybe a crossover with angr is to consider ?

Here are the sources : https://mega.nz/#!8NJlWaBL!gXIqecylWPX9qv7KK2KxJtwHxX5IyDQBOWO0eKiHB3M

Have a great day ! ^~^

[offlinemark]

thanks @TheLaluka !

in this case, maybe we can add a hack to manticore to look for instructions that look like these new ones and just skip over the instruction if we see it, since it does not have any semantics (i think? i didn't actually look into them much). we'd have a little mini decoder implemented in our decode_instruction function. this check on every insn might affect performance a little bit, but it's probably negligible compared to other sources of slowness.

if someone would like to implement a PR for this, that would be great, we probably won't have time to get to this very soon.

implementing libc stubs could also solve this, but i view that as a totally different feature, and it would not necessarily even totally solve this root issue anyway (e.g. if the endbr instructions appear in the binary)

[disconnect3d]

In the future we can test whether capstone supports those instruction with this code:

from capstone import *


CODE = b'\x90'
CODE += '\xf3\x0f\x1e\xfa'  # endbr64
CODE += '\x90'
CODE += '\xf3\x0f\x1e\xfb'  # endbr32
CODE += '\x90'

md = Cs(CS_ARCH_X86, CS_MODE_64)
md.detail = True
for i in md.disasm(CODE, 0x1000):
    print("0x%x:\t%s\t%s" % (i.address, i.mnemonic, i.op_str))```

[laluka]

@Mossberg yes, implementing stubs is a new feature and takes time, so not the priority here.
Mini decoder, or skipper is the way, at least until capstone has merged the fix.
And I won't have time to PR this issue either, sadly...

[catenacyber]

The fix is already in capstone next branch.
So is this worth a workaround until next release of capstone ?

[disconnect3d]

Capstone got a release https://github.com/aquynh/capstone/releases/tag/4.0 !

TODO: check if this issue can be fixed by updating capstone.

[catenacyber]

Yes this issues is fixed (the release includes what was in next branch in October)

[disconnect3d]

This has been fixed with #1312