Scanning NPM packages that list SPDX IDs results in false positives

[sschuberth]

Scanning the source code of NPM packages like

• https://github.com/shinnn/spdx-license-ids
• https://github.com/jslicense/spdx-correct.js

that contain lists of SPDX license IDs trigger a bunch of false positive license findings. I wonder whether there's something smart we could do about that, like not reporting licenses if you have many short identifiers very close together.

[pombredanne]

@sschuberth this is a thorny issue indeed. It has been discussed a bit in the past in #270 with @spf13 and @yahalom5776 .... the problem is that if we scan for licensing a package that itself is a licensing-related tool and contains a lot of license references and texts (such as golang's cobra or the npms you mentioned here, or if you were to scan scancode itself), we end up getting a lot of licenses detected... in some cases some false positive'ish looking detections and these licenses are typically NOT the licenses of the package being scanned. Imagine you scan scancode... you will be getting literally about 10K+ license matches all of which would likely be correct and at the same time completely useful and meaningless.

IMHO the way out is this: as part of the general theme of classification/summarization #426 we should eventually store a set of patterns and names of these packages (there is a finite number of these such as scancode proper, the SPDX and licensing-related npms, and possibly less than a 100 overall) such that these packages are clearly identified and that either

1. specific filtering patterns are applied in these cases
2. OR a clear message is returned when these are found to indicate that the detected licenses are not meaningful and explain why.

[pombredanne]

Note that we could also --and right away-- create a bunch of license detection rules (typically "negative" rule)s that would avoid the detection there. This would be a lot of rules, though they are fairly easy to generate. Let me push a quickie branch to show you this approach. These rules would end up being very specific, though in the case of a long list of SPDX ids, they may be a tad more universal.

[pombredanne]

@sschuberth the branch https://github.com/nexB/scancode-toolkit/tree/1032-license-tools-false-positive shows you the negative rules approach. This is a quick shot with mostly generated rules using ngrams, probably a few too many ;).... The tests show that you tow npm above would be detected fullyu accurately with these rules.... Though I am not 100% sure this is the right way to go as this creates a lot of rules for only a few not so common cases IMHO.

[pombredanne]

@sschuberth any feedback?

[sschuberth]

Though I am not 100% sure this is the right way to go as this creates a lot of rules for only a few not so common cases IMHO.

That's also what I'm asking myself. Not sure how maintainable this approach is. But probably the only way to tell is to merge this (now that you have it) and see how many more of these we would need over time?

[pombredanne]

How many common packages do we see with these issues beyond the common SPDX-related npms?

[sschuberth]

Hard to say, these are the first I saw in about a thousand scans.

[pombredanne]

@sschuberth OK, I will wait a bit, because adding 100+ new "negative" detection rules may not be the most scalable way to handle this.

[pombredanne]

And I merged it after all!

[pombredanne]

I am doing some cleanup and review as a prep for the 3.0 release... and since this is fixed in develop, I am closing this!