Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-prof_pcs fails on win7: NtCreateProfile returns 0xc000000d #413

Open
derekbruening opened this issue Nov 28, 2014 · 2 comments
Open

-prof_pcs fails on win7: NtCreateProfile returns 0xc000000d #413

derekbruening opened this issue Nov 28, 2014 · 2 comments

Comments

@derekbruening
Copy link
Contributor

From [email protected] on March 11, 2011 16:35:49

xref issue #140 on win7 I see:

ChildEBP RetAddr

00 0030f06c 66bcc98c ntdll!NtRaiseHardError+0x12
01 0030f0bc 66b9d024 dynamorio!nt_messagebox+0xec [c:\src\dr\git\src\core\win32\ntdll.c @ 3251]
02 0030f0d0 66a60c72 dynamorio!debugbox+0x54 [c:\src\dr\git\src\core\win32\os.c @ 3421]
03 0030f3e8 66a6101b dynamorio!notify+0x1a2 [c:\src\dr\git\src\core\utils.c @ 1854]
04 0030f45c 66a5cbe6 dynamorio!report_dynamorio_problem+0x34b [c:\src\dr\git\src\core\utils.c @ 2073]
05 0030f594 66bcd049 dynamorio!internal_error+0x116 [c:\src\dr\git\src\core\utils.c @ 191]
06 0030f5b4 66b9bed6 dynamorio!nt_create_profile+0x49 [c:\src\dr\git\src\core\win32\ntdll.c @ 3629]
07 0030f5e4 66b927d1 dynamorio!create_profile+0x1c6 [c:\src\dr\git\src\core\win32\os.c @ 2995]
08 0030f718 66b92489 dynamorio!init_global_profiles+0x2d1 [c:\src\dr\git\src\core\win32\os.c @ 417]
09 0030f750 669d7f6b dynamorio!os_init+0x669 [c:\src\dr\git\src\core\win32\os.c @ 836]
0a 0030f9b8 66b6f247 dynamorio!dynamorio_app_init+0x1bb [c:\src\dr\git\src\core\dynamo.c @ 498]
0b 0030fa04 66981008 dynamorio!auto_setup+0x27 [c:\src\dr\git\src\core\x86\x86_code.c @ 139]
0c 00000000 00000000 dynamorio!dynamo_auto_start+0x8
0:000> .frame 6
06 0030f5b4 66b9bed6 dynamorio!nt_create_profile+0x49 [c:\src\dr\git\src\core\win32\ntdll.c @ 3629]
0:000> dv
process_handle = 0xffffffff
start = 0x00000000
size = 0xffffffff
buffer = 0x1b9b1024
buffer_size = 0x10
shift = 0x1e
prof_handle = 0x00000000
res = -1073741811
0:000> !error @@(res)
Error code: (NTSTATUS) 0xc000000d (3221225485) - An invalid parameter was passed to a service or function.

Original issue: http://code.google.com/p/dynamorio/issues/detail?id=413

@derekbruening
Copy link
Contributor Author

From [email protected] on October 27, 2011 11:06:52

they added NtCreateProfileEx in win7: it takes 10 args so 1 extra

requires SeSystemProfilePrivilege but running as admin makes no difference,
and you'd think that would return access denied or similar privilege error code

% cd ~/dr/git/build_x86_dbg
% cmake --build . --target dynamorio && bin32/drrun.exe -debug -ops "-stderr_mask 15 -prof_pcs -msgbox_mask 0" e:/src/dr/test/hello.exe

editrights -a SeSystemProfilePrivilege -u bruening
editrights -a SeProfileSingleProcessPrivilege -u bruening
=> still 0xc000000d
log out and back in => no difference

here we go, got something different.
don't add 1 to size => STATUS_BUFFER_TOO_SMALL for whole-addr-space (not
for DR dll though):
start=0x00000000, size=0xffffffff, bufsz=0x0000000c, shift=30
res = 0xc0000023
start=0x712f0000, size=0x00390000, bufsz=0x00390000, shift=2
res = 0xc000000d

so buffer is big enough.
passing NULL for buffer itself => 0xc0000005 STATUS_ACCESS_VIOLATION
ditto passing NULL for the output handle

bigger shift makes no difference:
start=0x71de0000, size=0x00390000, bufsz=0x0000e404, shift=8
res = 0xc000000d
start=0x71a50000, size=0x00390000, bufsz=0x00000004, shift=28
res = 0xc000000d

pass 2 or 19 (ProfileTotalCycles) instead of ProfileTime => 0xc000000d
though if also make buffer small, returns that.
pass 0 for process, like Nebbett 11.1 example does (instead of
NT_CURRENT_PROCESS) => no difference: still 0xc000000d

align buffer to 1024:
start=0x00000000, size=0xffffffff, buf=0x24621400, bufsz=0x00000010, shift=30
res = 0xc000000d
start=0x71a50000, size=0x00390000, buf=0x24641400, bufsz=0x00390004, shift=2
res = 0xc000000d
start=0x77e30000, size=0x00180000, buf=0x24621800, bufsz=0x00000004, shift=30
res = 0xc000000d

align to 4096:
start=0x00000000, size=0xffffffff, buf=0x22712000, bufsz=0x00000010, shift=30
res = 0xc000000d
start=0x71de0000, size=0x00390000, buf=0x22732000, bufsz=0x00390004, shift=2
res = 0xc000000d
start=0x77e30000, size=0x00180000, buf=0x22713000, bufsz=0x00000004, shift=30
res = 0xc000000d

kernel address range:
start=0x88000000, size=0x00010000, buf=0x1de53000, bufsz=0x00000044, shift=12
res = 0xc000000d

pass 1 for mask => STATUS_PRIVILEGE_NOT_HELD (even if run as admin or w/
additional 2 rights above)
start=0x00000000, size=0xffffffff, buf=0x18c52000, bufsz=0x00000010, shift=30
res = 0xc0000061

so still stumped as to what's wrong

@derekbruening
Copy link
Contributor Author

This is causing the prof_pcs tests to fail in the long suite:

http://dynamorio.org/CDash/testDetails.php?test=68307&build=7764

abhinav92003 added a commit that referenced this issue Feb 1, 2024
Fixes missing instruction encodings for some kernel code execution captured
using Intel-PT.

The root-cause seemed to be that JIT code executed by the kernel, eBPF code in
this case, does not have entries in /proc/kallsyms, so our kcore dump logic
did not include them. This fix looks for BPF related symbols in /proc/kallsyms
and includes them in the copied regions from /proc/kcore.

Note that BPF JIT symbols are not included in /proc/kallsyms by default. One
needs to set /proc/sys/net/core/bpf_jit_harden and
/proc/sys/net/core/bpf_jit_kallsyms appropriately (see
https://docs.kernel.org/admin-guide/sysctl/net.html#proc-sys-net-core-network-core-options
for more details). Added this suggestion to documentation.

Tested PT tracing related tests locally on a machine that supports Intel-PT:

$ ctest -R 'drpttracer|drcacheoff.kernel'
...
    Start 213: code_api|client.drpttracer_SUDO-test
[sudo] password for sharmaabhinav:
1/5 Test #213: code_api|client.drpttracer_SUDO-test .....................   Passed    4.29 sec
    Start 412: code_api|tool.drcacheoff.kernel.simple_SUDO
2/5 Test #412: code_api|tool.drcacheoff.kernel.simple_SUDO ..............   Passed    4.66 sec
    Start 413: code_api|tool.drcacheoff.kernel.opcode-mix_SUDO
3/5 Test #413: code_api|tool.drcacheoff.kernel.opcode-mix_SUDO ..........   Passed    4.71 sec
    Start 414: code_api|tool.drcacheoff.kernel.syscall-mix_SUDO
4/5 Test #414: code_api|tool.drcacheoff.kernel.syscall-mix_SUDO .........   Passed    4.59 sec
    Start 415: code_api|tool.drcacheoff.kernel.invariant-checker_SUDO
5/5 Test #415: code_api|tool.drcacheoff.kernel.invariant-checker_SUDO ...   Passed    5.75 sec

100% tests passed, 0 tests failed out of 5

Issue: #6486
abhinav92003 added a commit that referenced this issue Feb 5, 2024
Fixes drmemtrace kernel trace libipt post-processing failures caused by
missing instruction encodings for some kernel code execution captured
using Intel-PT.

The root-cause seems to be that JIT code executed by the kernel, BPF
code in this case, does not have entries in `/proc/modules`. So, our
kcore dump logic did not include them. This fix looks for BPF related
symbols in `/proc/kallsyms` and includes them in the copied regions from
`/proc/kcore`.

Note that BPF JIT symbols are not included in `/proc/kallsyms` by
default. One needs to set `/proc/sys/net/core/bpf_jit_harden` and
`/proc/sys/net/core/bpf_jit_kallsyms` appropriately (see
https://docs.kernel.org/admin-guide/sysctl/net.html#proc-sys-net-core-network-core-options
for more details). Added this suggestion to documentation. It may be
better to not automatically make this possibly-too-intrusive change to
the user's machine in cmake. This is probably fine because the issue
is not widespread (not reproduced on public Linux distributions).

Tested PT tracing related tests locally on a machine that supports
Intel-PT:

```
$ ctest -R 'drpttracer|drcacheoff.kernel'
...
    Start 213: code_api|client.drpttracer_SUDO-test
[sudo] password for sharmaabhinav: 
1/5 Test #213: code_api|client.drpttracer_SUDO-test .....................   Passed    4.29 sec
    Start 412: code_api|tool.drcacheoff.kernel.simple_SUDO
2/5 Test #412: code_api|tool.drcacheoff.kernel.simple_SUDO ..............   Passed    4.66 sec
    Start 413: code_api|tool.drcacheoff.kernel.opcode-mix_SUDO
3/5 Test #413: code_api|tool.drcacheoff.kernel.opcode-mix_SUDO ..........   Passed    4.71 sec
    Start 414: code_api|tool.drcacheoff.kernel.syscall-mix_SUDO
4/5 Test #414: code_api|tool.drcacheoff.kernel.syscall-mix_SUDO .........   Passed    4.59 sec
    Start 415: code_api|tool.drcacheoff.kernel.invariant-checker_SUDO
5/5 Test #415: code_api|tool.drcacheoff.kernel.invariant-checker_SUDO ...   Passed    5.75 sec

100% tests passed, 0 tests failed out of 5
```

Unfortunately the decode errors do not go away completely even after
this fix, but they have become very less frequent now (tool.kernel.simple
in release build failed after 40 successful runs with this fix, which failed
every run before).

Issue: #6486
xdje42 pushed a commit that referenced this issue Feb 6, 2024
Fixes drmemtrace kernel trace libipt post-processing failures caused by
missing instruction encodings for some kernel code execution captured
using Intel-PT.

The root-cause seems to be that JIT code executed by the kernel, BPF
code in this case, does not have entries in `/proc/modules`. So, our
kcore dump logic did not include them. This fix looks for BPF related
symbols in `/proc/kallsyms` and includes them in the copied regions from
`/proc/kcore`.

Note that BPF JIT symbols are not included in `/proc/kallsyms` by
default. One needs to set `/proc/sys/net/core/bpf_jit_harden` and
`/proc/sys/net/core/bpf_jit_kallsyms` appropriately (see
https://docs.kernel.org/admin-guide/sysctl/net.html#proc-sys-net-core-network-core-options
for more details). Added this suggestion to documentation. It may be
better to not automatically make this possibly-too-intrusive change to
the user's machine in cmake. This is probably fine because the issue
is not widespread (not reproduced on public Linux distributions).

Tested PT tracing related tests locally on a machine that supports
Intel-PT:

```
$ ctest -R 'drpttracer|drcacheoff.kernel'
...
    Start 213: code_api|client.drpttracer_SUDO-test
[sudo] password for sharmaabhinav: 
1/5 Test #213: code_api|client.drpttracer_SUDO-test .....................   Passed    4.29 sec
    Start 412: code_api|tool.drcacheoff.kernel.simple_SUDO
2/5 Test #412: code_api|tool.drcacheoff.kernel.simple_SUDO ..............   Passed    4.66 sec
    Start 413: code_api|tool.drcacheoff.kernel.opcode-mix_SUDO
3/5 Test #413: code_api|tool.drcacheoff.kernel.opcode-mix_SUDO ..........   Passed    4.71 sec
    Start 414: code_api|tool.drcacheoff.kernel.syscall-mix_SUDO
4/5 Test #414: code_api|tool.drcacheoff.kernel.syscall-mix_SUDO .........   Passed    4.59 sec
    Start 415: code_api|tool.drcacheoff.kernel.invariant-checker_SUDO
5/5 Test #415: code_api|tool.drcacheoff.kernel.invariant-checker_SUDO ...   Passed    5.75 sec

100% tests passed, 0 tests failed out of 5
```

Unfortunately the decode errors do not go away completely even after
this fix, but they have become very less frequent now (tool.kernel.simple
in release build failed after 40 successful runs with this fix, which failed
every run before).

Issue: #6486
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant