BugId fails with error: "No module for cdb id 'ntdll'"

[maxcoderrrr]

Hi,
I was running the latest release of BugId and encountered the following error:

┌───[ Software license warning ]───────────────────────────────────────────────────────────────────────────────────────
│ ▲ You have no license for BugId and your trial period will expire on May 9th, 2023
│ ▲ You have no license for mBugId and your trial period will expire on May 9th, 2023
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
→ Command line: C:\test\test.exe C:\test\test.input
+ Main process 7940/0x1F04 (test.exe, x86, IL:3): Attached ("C:\test\test.exe" C:\test\test.input).
┌───[ Fatal builtins.AssertionError Exception in thread 6536/0x1988 (cThread#1ED06B6CF10{main = __fRun, #6536, running}) ]
│ No module for cdb id 'ntdll' (address = 0x7734`0000):
│ Module <cdb id not cached> (C:\test\test.exe) at 0xA8`0000, Module <cdb id not cached> (C:\Windows\SYSTEM32\ntdll.dll) at 0x7FFA`9662`0000, Module <cdb id not cached> (C:\Windows\System32\verifier.dll) at 0x400`0000, Module <cdb id not cached> (C:\Windows\System32\wow64.dll) at 0x7FFA`94C4`0000, Module <cdb id not cached> (C:\Windows\System32\wow64win.dll) at 0x7FFA`95DA`0000, Module <cdb id not cached> (C:\Windows\System32\wow64cpu.dll) at 0x7733`0000
│ 
│ Local variables:
│   o0Module = None
│   oModule = <instance mBugId.cModule.cModule:cModule>#1ED06B6F650
│   oSelf = <mBugId.cProcess.cProcess.cProcess object at 0x000001ED06BB1D10>#1ED06BB1D10
│   sbCdbId = <instance builtins:bytes "b'ntdll'">#1ED06AF2EE0
│   u0StartAddress = 1999896576
│ 
│ Stack for thread 6536/0x1988 (cThread#1ED06B6CF10{main = __fRun, #6536, running}):
│ ─┐ __fRun @ C:\bugid\modules\mBugId\cCdbWrapper\cCdbWrapper_cHelperThread.py:66
│  │ 65:      try:
│  │ 66:        oSelf.__fActivity(*oSelf.__axActivityArguments);
│  ├─┐ cCdbWrapper_fCdbStdInOutHelperThread @ C:\bugid\modules\mBugId\cCdbWrapper\cCdbWrapper_fCdbStdInOutHelperThread.py:54
│  ╷ │ 53:    oCdbWrapper.fbFireCallbacks("Log message", "Main loop #%d" % uMainLoopCounter);
│  ╷ │ 54:    (bEventIsFatal, bEventHasBeenHandled) = oCdbWrapper.ftbHandleLastCdbEvent(asbOutputWhileRunningApplication);
│  ╷ ├─┐ cCdbWrapper_ftbHandleLastCdbEvent @ C:\bugid\modules\mBugId\cCdbWrapper\cCdbWrapper_ftbHandleLastCdbEvent.py:214
│  ╷ ╷ │ 213:    oCdbWrapper.o0ReservedMemoryVirtualAllocation = None;
│  ╷ ╷ │ 214:  o0BugReport = cBugReport.fo0CreateForException(
│  ╷ ╷ ├─┐ cBugReport?.fo0CreateForException @ C:\bugid\modules\mBugId\cBugReport\cBugReport.py:134
│  ╷ ╷ ╷ │ 133:    # Create a preliminary error report.
│  ╷ ╷ ╷ │ 134:    o0BugReport = cBugReport.foCreate(
│  ╷ ╷ ╷ ├─┐ cBugReport?.foCreate @ C:\bugid\modules\mBugId\cBugReport\cBugReport.py:181
│  ╷ ╷ ╷ ╷ │ 180:    );
│  ╷ ╷ ╷ ╷ │ 181:    fApplyBugTranslationsToBugReport(oCdbWrapper, oBugReport);
│  ╷ ╷ ╷ ╷ ├─┐ fApplyBugTranslationsToBugReport @ C:\bugid\modules\mBugId\mBugTranslations\fApplyBugTranslationsToBugReport.py:65
│  ╷ ╷ ╷ ╷ ╷ │ 64:    for oBugTranslation in aoBugTranslations:
│  ╷ ╷ ╷ ╷ ╷ │ 65:      if oBugTranslation.fbApplyToBugReport(oCdbWrapper, oBugReport):
│  ╷ ╷ ╷ ╷ ╷ ├─┐ cBugTranslation?.fbApplyToBugReport @ C:\bugid\modules\mBugId\mBugTranslations\cBugTranslation.py:137
│  ╷ ╷ ╷ ╷ ╷ ╷ │ 136:    # If more top frames may be irrelevant, see if they are there and hide them:
│  ╷ ╷ ╷ ╷ ╷ ╷ │ 137:    if (oSelf.bAdditionalIrrelevantNoneTopStackFrames or oSelf.rb0AdditionalIrrelevantStackFrameSymbols) and oBugReport.o0Stack:
│  ╷ ╷ ╷ ╷ ╷ ╷ ├─┐ o0Stack @ C:\bugid\modules\mBugId\cBugReport\cBugReport.py:101
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ │ 100:    if oSelf.__o0Stack is None:
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ │ 101:      oSelf.__o0Stack = cStack.foCreate(oSelf.__oProcess, oSelf.__oWindowsAPIThread, oSelf.uStackFramesCount);
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ ├─┐ cStack?.foCreate @ C:\bugid\modules\mBugId\cStack.py:269
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ │ 268:        o0Function, i0OffsetFromStartOfFunction
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ │ 269:      ) = oProcess.ftxSplitSymbolOrAddress(sbCdbSymbolOrAddress);
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ├─┐ cProcess_ftxSplitSymbolOrAddress @ C:\bugid\modules\mBugId\cProcess\cProcess_ftxSplitSymbolOrAddress.py:40
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ │ 39:    sbModuleCdbId = sb0ModuleCdbIdOrAddress;
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ │ 40:    o0Module = oProcess.fo0GetModuleForCdbId(sbModuleCdbId);
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ├─┐ cProcess?.fo0GetModuleForCdbId @ C:\bugid\modules\mBugId\cProcess\cProcess.py:114
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ │ 113:    o0Module = oSelf.fo0GetModuleForStartAddress(u0StartAddress);
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ │ 114:    assert o0Module, \
│  ╒═══════════════════╛ ▲ Assertion failed: "No module for cdb id 'ntdll' (address = 0x7734`0000):\nModule <cdb id not cached> (C:\\test\\test.exe) at 0xA8`0000, Module <cdb id not cached> (C:\\Windows\\SYSTEM32\\ntdll.dll) at 0x7FFA`9662`0000, Module <cdb id not cached> (C:\\Windows\\System32\\verifier.dll) at 0x400`0000, Module <cdb id not cached> (C:\\Windows\\System32\\wow64.dll) at 0x7FFA`94C4`0000, Module <cdb id not cached> (C:\\Windows\\System32\\wow64win.dll) at 0x7FFA`95DA`0000, Module <cdb id not cached> (C:\\Windows\\System32\\wow64cpu.dll) at 0x7733`0000"
│  │ __fRun @ C:\bugid\modules\mBugId\cCdbWrapper\cCdbWrapper_cHelperThread.py:74
│  │ 73:        cException, oException, oTraceBack = sys.exc_info();
│  │ 74:        if not oSelf.__oCdbWrapper.fbFireCallbacks("Internal exception", oSelf.__oThread, oException, oTraceBack):
│ ═╛ ▲ Application terminated because exception was not handled.
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Please report the above details at the below web-page so it can be addressed:
    https://github.com/SkyLined/BugId/issues/new
If you do not have a github account, or you want to report this issue
privately, you can also send an email to:
    [email protected]

In your report, please copy ALL the information about the exception reported
above, as well as the stack trace and BugId version information. This makes
it easier to determine the cause of this issue and makes for faster fixes.

If you can reproduce the issue, it would help a lot if you can run BugId in
verbose mode by adding the --verbose command-line argument.
as in: BugId -v --isa=x86 --n0ApplicationMaxRunTimeInSeconds=15 C:\test\test.exe -- C:\test\test.input


  ____________________________________________________________________________
                              __
   ││▌║█▐▐║▌▌█│║║│      _,siSP**YSis,_       ╒╦╦══╦╗             ╒╦╦╕    ╔╦╕
   ││▌║█▐▐║▌▌█│║║│    ,SP*'`    . `'*YS,      ║╠══╬╣ ╔╗ ╔╗ ╔╦═╦╗  ║║  ╔╦═╬╣
   ╵2808197631337╵   dS'  _    |    _ 'Sb    ╘╩╩══╩╝ ╚╩═╩╝ ╚╩═╬╣ ╘╩╩╛ ╚╩═╩╝
                    dP     \,-` `-<` `  Y;                 ╚╩═╩╝    ╮╷╭
      ╮╷╭          ,S`  \+' \      \    `Sissssssssssssssssssss,   :O()    ╲ö╱
     :O()          (S   (   | --====)   :SSSSSSSSSSSSSSSSSSSSSSD    ╯╵╰    ─O─
      ╯╵╰  ╮╷╭     'S,  /+, /      /    ,S?********************'           ╱O╲
           ()O:     Yb    _/'-_ _-<._.  dP
           ╯╵╰       YS,       |      ,SP         https://bugid.skylined.nl
  ____________________`Sbs,_    ' _,sdS`______________________________________
                        `'*YSissiSY*'`
                              ``
┌───[ Version information ]────────────────────────────────────────────────────────────────────────────────────────────
│ ▲ BugId version: 2022-12-12 12:05 (in trial period).
│ ▲ mBugId version: 2022-12-12 12:05 (in trial period).
│ √ mConsole version: 2022-12-12 12:05.
│ √ mDateTime version: 2022-12-12 12:04.
│ √ mDebugOutput version: 2022-12-12 12:05.
│ √ mFileSystemItem version: 2022-12-12 12:05.
│ √ mHumanReadable version: 2022-12-12 12:04.
│ √ mMultiThreading version: 2022-12-12 12:05.
│ √ mNotProvided version: 2022-12-12 12:04.
│ √ mProductDetails version: 2022-12-12 12:05.
│ √ mRegistry version: 2022-12-12 12:05.
│ √ mWindowsAPI version: 2022-12-12 12:05.
│ √ mWindowsSDK version: 2022-12-12 12:04.
│ • Windows version: Windows 10 Pro release 1909, build 18363 x64.
│ • Python version: 3.11.1 x64.
│ • cdb.exe (x86) version: 10.0.22621.755.
│ • cdb.exe (x64) version: 10.0.22621.755.
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Thank you in advance for helping to improve BugId!
√ A copy of the error report can be found in C:\bugid\Internal error reports\2023-04-09 12։27։26.419274 BugId error report #1.txt.

Any help to fix this issue would be great. Please let me know in case any further information is required.

[SkyLined]

Wow, time flies when you are busy. I finally have some time to look at this.

This is a weird bug. ntdll should never get unloaded during normal application run. Even if cdb forgets about it, I should be able to get information by using the address at which it is loaded. I have no idea how this can happen or how to solve it. A few questions:

1. Can you reproduce this issue?
2. Can you reproduce with the --verbose flag and send me the output?
3. Can you share the test case that triggers it so I can reproduce locally?

I see the trial period has expired by now. If you need a license for BugId to test this, just request one at https://license.skylined.nl. You do not need to pay for it, just let me know when you've requested one and I will give it to you for free.

[maxcoderrrr]

Thank you for the update (also regarding the other two reported bugs)!

I have already deleted my test VM. However, when I was evaluating BugId, I remember that I came across this error a couple of times with different applications. I think it was also only on x86 Windows, but not on x64 - not completely sure anymore, it's been a while.

As previously mentioned, I am happy to test the updated version if it helps. Thanks also for the license, I will likely send you an email tomorrow or on Monday.

If I come across this error again with the updated version, I will make sure to provide an updated log file with the verbose flag added to this thread for your reference. I triggered it many times, so I don't think it will take long before running into it again.

[maxcoderrrr]

@SkyLined I have sent you a request, thanks!

[SkyLined]

License granted. Thanks for helping me improve BugId by filing bugs!

[maxcoderrrr]

I have sent you an email with more details, including a log file that had the verbose flag enabled, when I triggered again this bug. Hope it helps to identify the root cause.

[maxcoderrrr]

@SkyLined Did the log help to identify the root cause of the bug?

[maxcoderrrr]

@SkyLined I wanted to see if you had a chance to further investigate this and the other pending bugs?

[SkyLined]

I did make quite a few changes to BugId that may solve this. However, it appears my changes have uncovered a hard-to-debug issue triggered by one of the unit-tests. I'm not happy to ship code that doesn't even pass unit-tests, so I'll have to fix that issue before I can commit the new code.

Unfortunately, I'm working on another project until the end of the month. I'm hoping I'll have some time then to analyze and fix this issue, so I can release a new version.