[Bug] Exceptions on blender database

[Bazmundi]

Describe the bug
I download blender database which is a mix of c++ (cc) and python. Joern broke on the import of cc code, so I create a python code only copy.

Running against python only I am getting:

2025-01-26 12:14:38.429 WARN CodeToCpg$: Failed to convert file examples/ocd_addon_1_8_2/__MACOSX/ocd_addon/.___init__.py
io.joern.pythonparser.TokenMgrError: Lexical error at line 1, column 1.  Encountered: '0' (0),
	at io.joern.pythonparser.PythonParserTokenManager.getNextToken(PythonParserTokenManager.java:3692) ~[io.joern.pysrc2cpg-4.0.226.jar:4.0.226]
	at io.joern.pythonparser.PythonParser.jj_ntk_f(PythonParser.java:7599) ~[io.joern.pysrc2cpg-4.0.226.jar:4.0.226]
	at io.joern.pythonparser.PythonParser.module(PythonParser.java:179) ~[io.joern.pysrc2cpg-4.0.226.jar:4.0.226]
	at io.joern.pythonparser.PyParser.parse(PyParser.scala:23) ~[io.joern.pysrc2cpg-4.0.226.jar:4.0.226]
	at io.joern.pythonparser.PyParser.parse(PyParser.scala:15) ~[io.joern.pysrc2cpg-4.0.226.jar:4.0.226]
	at io.joern.pysrc2cpg.CodeToCpg.runOnPart(CodeToCpg.scala:25) [io.joern.pysrc2cpg-4.0.226.jar:4.0.226]
	at io.joern.pysrc2cpg.CodeToCpg.runOnPart(CodeToCpg.scala:20) [io.joern.pysrc2cpg-4.0.226.jar:4.0.226]
	at io.shiftleft.passes.ForkJoinParallelCpgPass$$anon$2.accept(CpgPass.scala:124) [io.shiftleft.codepropertygraph_3-1.7.23.jar:1.7.23]
	at io.shiftleft.passes.ForkJoinParallelCpgPass$$anon$2.accept(CpgPass.scala:123) [io.shiftleft.codepropertygraph_3-1.7.23.jar:1.7.23]
	at java.util.stream.ReduceOps$4ReducingSink.accept(ReduceOps.java:220) [?:?]
	at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992) [?:?]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) [?:?]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) [?:?]
	at java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:960) [?:?]
	at java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:934) [?:?]
	at java.util.stream.AbstractTask.compute(AbstractTask.java:327) [?:?]
	at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754) [?:?]
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) [?:?]
	at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) [?:?]
	at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) [?:?]
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) [?:?]
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) [?:?]
2025-01-26 12:14:39.299 WARN PythonAstVisitor$: Could not parse file blender/extern/audaspace/src/respec/filter_design.py at line 54 column 4. Invalid code: print("\n", end='')
Parser exception message: Encountered " <NAME> "print "" at line 54, column 4.
Was expecting:
    <INDENT> ...
    
2025-01-26 12:14:39.300 WARN PythonAstVisitor$: Could not parse file blender/extern/audaspace/src/respec/filter_design.py at line 55 column 1. Invalid code: print(f'}};')
Parser exception message: Encountered " <DEDENT> " "" at line 55, column 1.
Was expecting one of:
    "if" ...
    "not" ...
    "def" ...
    "async" ...
    "lambda" ...
    "for" ...
    "with" ...
    "return" ...
    "try" ...
    "class" ...
    "while" ...
    "import" ...
    "from" ...
    "raise" ...
    "pass" ...
    "del" ...
    "yield" ...
    "assert" ...
    "break" ...
    "continue" ...
    "global" ...
    "nonlocal" ...
    "await" ...
    "None" ...
    "True" ...
    "False" ...
    "..." ...
    "+" ...
    "-" ...
    "@" ...
    "~" ...
    "(" ...
    "[" ...
    "{" ...
    "*" ...
    <DEC_INTEGER> ...
    <BIN_INTEGER> ...
    <OCT_INTEGER> ...
    <HEX_INTEGER> ...
    <FLOAT> ...
    <IMAGINARY> ...
    <NAME> ...
    <STRING_PREFIX> ...
    <FORMAT_STRING_PREFIX> ...
    "STRING_QUOTE_OPEN" ...

To Reproduce
Steps to reproduce the behavior:

1. Download latest blender from https://github.com/blender/blender
2. Download and install latest joern
3. run joern against blender folder
4. errors raised against cc
5. move python files into separate folder structure
6. run joern against abbreviated blender files (python only)
7. see results above

Otherwise seems to produce too many warnings to give the sense it doesn't handle production code.

Expected behavior
joern parses python.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS Ubuntu 22.04

• Joern Version
  joern> version
  val res2: String = "4.0.226"

• Java version
  java --version
  openjdk 17.0.13 2024-10-15
  OpenJDK Runtime Environment (build 17.0.13+11-Ubuntu-2ubuntu122.04)
  OpenJDK 64-Bit Server VM (build 17.0.13+11-Ubuntu-2ubuntu122.04, mixed mode, sharing)

Additional context
There is otherwise a database created, presumeably bereft of information:

val res1: io.shiftleft.codepropertygraph.generated.Cpg = Cpg[Graph[2016749 nodes]]

But joern-export dunna like it.

[max-leuthaeuser]

Multiple fixes for C/C++ for crashes happening during a scan on blender: #5259.
Once its in Master you may want to try it again (with enough memory e.g., -Xmx48G if your machine can do that).

Can't say anything about the Python part.