"elftools.construct.core.FieldError: expected 1, found 0 " and "elftools.construct.core.ArrayError: ('missing terminator', FieldError('expected 1, found 0'))" #591
Comments
|
我的系统版本是ubuntu22.04，pyelftools版本是： |
|
当我换成ubuntu20.04的docker就不会报错，ubuntu20.04的虚拟机：e = ELF('./nsctf_online_2019_pwn1') 也会报错 |
|
The file It's a 64-bit, little endian ELF file. Its segment header (aka "program headers") table contains entries of size 0x38, starting at offset 0x40 (just past the file header), a total of 11 entries (0xb), occupying the file offsets from 0x40 to 0x2A8. The segment entry at index 3 is of type PT_NOTE (4) and has If interpreted as a note header, the bytes go: 00 00 00 00 This reads as a note of type 0 with no name (allowed by the ABI, not handled well by pyelftools) and 0x30 bytes worth of data. Together it's 0x3C bytes of data - short of 0x44. The next note header is cut short (8 bytes where at least 12 are expected) if the segment boundary is a hard one - but pyelftools happily reads from the source file past the segment boundaries. The second note is also funny looking but technically legit. Then the "past the end" condition kicks in and parsing stops. Whether or not reading past the end of the segment is legitimate is a tricky question. The file is loaded as a whole; all segments are mapped to the process' address space. In this binary, in particular, there is a segment 0 of type PHDR that covers offsets from 0x40 to 0x2A8 - so in a loaded binary, reading past the end of the NOTE segment in question won't crash, seeing that the same memory is mapped as another segment. I'm not sure what's the logic with overlapping segments. All in all, there are two problems with pyelftools that this binary uncovers:
If the former is fixed but the latter is not, the file will parse. 😄 If the latter is fixed literally (i. e. no reading past the end of the segment), there will be a stream underflow error. On a side note, the ABI document at https://refspecs.linuxbase.org/elf/gabi4+/ch5.pheader.html#note_section claims that in 64 bit files, the fields of the note section are 64 bit. That is wrong, in the sense that all the tools out there disagree - in both 64 bit and 32 bit ELF binaries, the note header fields are 32 bit, and the name/data alignment is to 4 bytes. An alternative format document at https://docs.oracle.com/cd/E23824_01/html/819-0690/chapter6-18048.html states that header fields are 32 bit always. I wonder who should I bother about that discrepancy... @eliben - ideas? |
|
I tried pyelftools-0.30 on my docker-ubuntu20.04 and it works fine. root@b66:/ctf/work/nsctf_online_2019_pwn1# pip3 list | grep pyelf
|
|
Check the versions of pwntools too. It is possible that some versions of pwntools don't try to parse the note segment(s), or catch exceptions there. |
|
I encountered the problem again from another binary. |
|
This one has a PT_NOTE segment at the same offset (0x254) with the same size (0x44). Do they come straight from a compiler/linker, or were they modified manually? This bogus note could mean something beyond the obvious, but I fail to see what it is. |
|
It should be compiled directly, not manually modified, because these are two different binary files from two different CTF competitions, but both binary files contain the same issue. |
|
PT_NOTE is usually used to describe some additional information or comments in the Program Header Table, which typically includes some debugging information, symbol table information, etc. The appearance of PT_NOTE is to allow the program to obtain some additional information at runtime. For example, a debugger can use this information to help debug the program. It seems that the probability of appearing in a binary file is still quite high. |
|
I know the definition and the stated purpose. This particular note looks as if it's not supposed to be treated by the loader like any other segment is treated - as an instruction to create a range of bytes at a certain virtual address in the process space. Chances are, the Linux loader ignores If that's indeed the case, the whole logic of |
|
@0xsyj : are you building those binaries yourself? One of possible explanations is that the generation of a bogus PT_NOTE is/was a linker bug. If you are building them yourself, what is the build environment? One way to confirm or debunk this theory would be reproducing your environment, and following the linker logic in that particular version of the toolchain. |
|
It's a pity that these two binaries were not compiled by me. |
|
|
|
I've performed that analysis already. The next question is, was it a linker bug that pyelftools should work around, or a piece of undocumented linker magic that pyelftools should recognize and parse? For now, the bug option seems more likely, but I would prefer a confirmation from someone with knowledge of the GNU linker's bug tracker. |
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "", line 1, in
File "/home/syj/.local/lib/python3.10/site-packages/pwnlib/elf/elf.py", line 362, in init
self._describe()
File "/home/syj/.local/lib/python3.10/site-packages/pwnlib/elf/elf.py", line 457, in _describe
self.checksec(*a, **kw)
File "/home/syj/.local/lib/python3.10/site-packages/pwnlib/elf/elf.py", line 2117, in checksec
if self.shadowstack:
File "/home/syj/.local/lib/python3.10/site-packages/pwnlib/elf/elf.py", line 2192, in shadowstack
for prop in self.iter_properties():
File "/home/syj/.local/lib/python3.10/site-packages/pwnlib/elf/elf.py", line 532, in iter_properties
for note in self.iter_notes():
File "/home/syj/.local/lib/python3.10/site-packages/pwnlib/elf/elf.py", line 523, in iter_notes
for note in seg.iter_notes():
File "/home/syj/.local/lib/python3.10/site-packages/elftools/elf/notes.py", line 31, in iter_notes
CString('').parse(elffile.stream.read(disk_namesz)))
File "/home/syj/.local/lib/python3.10/site-packages/elftools/construct/core.py", line 180, in parse
return self.parse_stream(BytesIO(data))
File "/home/syj/.local/lib/python3.10/site-packages/elftools/construct/core.py", line 190, in parse_stream
return self._parse(stream, Container())
File "/home/syj/.local/lib/python3.10/site-packages/elftools/construct/core.py", line 261, in _parse
return self.subcon._parse(stream, context)
File "/home/syj/.local/lib/python3.10/site-packages/elftools/construct/core.py", line 276, in _parse
return self._decode(self.subcon._parse(stream, context), context)
File "/home/syj/.local/lib/python3.10/site-packages/elftools/construct/core.py", line 578, in _parse
raise ArrayError("missing terminator", ex)
elftools.construct.core.ArrayError: ('missing terminator', FieldError('expected 1, found 0'))
nsctf_online_2019_pwn1.zip
The text was updated successfully, but these errors were encountered: