Yarn Audit Scan takes a lot of resources

[nishils]

When scanning a few different repos like https://github.com/coinbase/rest-hooks, yarn audit --json will produce 19 GB which then get entirely placed into memory and is handled inefficiently. This particularly affects CI scans that may expect some sort of response from salus within a time period. Currently, Salus only outputs on completion.

There's a couple of options to fix this issue.

1. Wait for upstream fix yarn audit --json produces large amounts of data yarnpkg/yarn#7404
2. (Stop gap) Implement a heartbeat to prevent CI from timing out
3. (Longer term if no upstream fix) Implement a streaming parser that de-dupes without having to load the entire file into memory.

[nishils]

2.6.0 addresses this by adding a heartbeat that emits every minute. This can turned off using the heartbeat flag (--heartbeat false). This flag also respects quiet mode.

2.6.1 (still in dev phase) will add in stream parsing. We will watch how well this does when yarn seems to run into issues. Sometimes yarn outputs without this enormous dataset and sometimes it does.

[nishils]

Planning to get a fix in 2.6.2 that allows to configure yarn audit scans to not include devDependencies configurable via a salus config file.