Improve detection of machO backdoors & stealers

[tstromberg]

Based on studying some recent submissions to the Malware Bazaar.

[egibs]

It's odd that make integration doesn't seem to be completing with the changes in this branch.

Edit, it finished:

$ go test -timeout 0 ./tests/...
ok  	github.com/chainguard-dev/malcontent/tests	1009.494s

That's strange.

Tests are slow locally, too:

--- PASS: TestSimple/javascript/clean/faker.js (138.85s)
--- PASS: TestSimple/javascript/clean/napi_rs_runtime.js (140.04s)
--- PASS: TestSimple/linux/clean/clickhouse (351.76s)
--- PASS: TestSimple/linux/clean/libgcj.so.17 (70.17s)
--- PASS: TestSimple/linux/clean/libgcj.so.17.0.0 (68.54s)
--- PASS: TestSimple/linux/clean/trivy (109.63s)
--- PASS: TestSimple/linux/clean/wolfictl (67.87s)
--- PASS: TestSimple/macOS/2024.AMOS/Cosmical_setup (96.59s)
--- PASS: TestSimple/macOS/2024.AMOS/FlaUI_Driver (145.44s)
--- PASS: TestSimple/macOS/2024.AMOS/Installer (106.86s)
--- PASS: TestSimple/macOS/2024.AMOS/InstallerCrack (106.53s)
--- PASS: TestSimple/macOS/2024.AMOS/WeChat (106.71s)
--- PASS: TestSimple/macOS/2024.FakeZoom/Zoom (235.95s)
...
--- PASS: TestMarkdown/linux/clean/code-oss (33.84s)
--- PASS: TestMarkdown/linux/clean/pandoc (186.62s)
--- PASS: TestMarkdown/linux/clean/slack (42.93s)
--- PASS: TestMarkdown/linux/clean/trufflehog (33.13s)

PASS
ok  	github.com/chainguard-dev/malcontent/tests	548.563s

From main (same samples as above for a direct comparison except for FakeZoom):

--- PASS: TestSimple/javascript/clean/faker.js (0.57s)
--- PASS: TestSimple/javascript/clean/napi_rs_runtime.js (0.18s)
--- PASS: TestSimple/linux/clean/clickhouse (52.61s)
--- PASS: TestSimple/linux/clean/libgcj.so.17 (22.74s)
--- PASS: TestSimple/linux/clean/libgcj.so.17.0.0 (22.37s)
--- PASS: TestSimple/linux/clean/trivy (59.61s)
--- PASS: TestSimple/linux/clean/wolfictl (25.42s)
--- PASS: TestSimple/macOS/2024.AMOS/Cosmical_setup (0.02s)
--- PASS: TestSimple/macOS/2024.AMOS/FlaUI_Driver (0.02s)
--- PASS: TestSimple/macOS/2024.AMOS/Installer (0.02s)
--- PASS: TestSimple/macOS/2024.AMOS/InstallerCrack (0.02s)
--- PASS: TestSimple/macOS/2024.AMOS/WeChat (0.02s)
...
--- PASS: TestMarkdown/linux/clean/code-oss (14.45s)
--- PASS: TestMarkdown/linux/clean/pandoc (29.06s)
--- PASS: TestMarkdown/linux/clean/slack (18.45s)
--- PASS: TestMarkdown/linux/clean/trufflehog (15.71s)

PASS
ok  	github.com/chainguard-dev/malcontent/tests	352.669s

Running make integration takes ~145 seconds for me; the verbose logging slowed things down a bit.

Seems like a new rule is slowing down scans?

[tstromberg]

Yeah, it looks like there may be no fast way to match a long string of alphanumeric text. It turns out that compiled regexps are run across all files - not just the ones matching conditions:

https://github.com/Neo23x0/YARA-Performance-Guidelines

[tstromberg]

commit incoming to improve performance ...

[tstromberg]

updated rules are still slower than I like, but they are faster than they were.

there doesn't seem to be a fast way to detect a very large string with YARA :(

[tstromberg]

finished in 6 minutes - not bad! I had to remove a query, but the best improvement was adding a nul character up front to find the start character.