develop utility script to detect feature overlap between new and existing capa rules

[mike-hunhoff]

It would be useful to have a utility script that takes a new rule, under development, and compares its feature(s) against the existing rules to determine what existing rules, if any, contain the same feature(s). I see such a script being leveraged during rule development to prevent duplicate detection e.g. https://github.com/mandiant/capa-rules/pull/744/files#r1168985328.

The script in my mind prints a list of existing rules containing feature overlap and leaves it up to the rule developer to determine whether the overlapping features are being used to detect the same/similar capability. "Overlap" could be as simple as feature A exists in both the new rule and an existing rule, ignoring other logic like scopes, structural expressions, etc.

I'm currently handling this by manually searching the existing rules 😢

[Aayush-Goel-04]

Hi @mike-hunhoff,
I would like to work on this. Can you explain a bit more about overlap?
In case of example feature given in readme -

1  features:
2    - or:
3      - and:
4        - mnemonic: shr
5        - or:
6          - number: 0xEDB88320
7          - bytes: 00 00 00 00 96 30 07 77 2C 61 0E EE BA 51 09 99 19 C4 6D 07 8F F4 6A 70 35 A5 63 E9 A3 95 64 9E = crc32_tab
8        - number: 8
9        - characteristic: nzxor
10     - and:
11       - number: 0x8320
12       - number: 0xEDB8
13       - characteristic: nzxor
14     - api: RtlComputeCrc32

Here its like features in multiple levels - F2 represents feature at line 1
[F4 && ( F6 || F7 ) && F8 && F9 ] || [F11 && F12 & F13] || [F14].

Under which cases will be considered a duplicate/ overlap.

• When any of the features match like individual feature F7 is found in both rules.
• Combination of features are are found at first level F11 && F12 & F13 is found.