Prevent symlinks causing duplicate package-file relationships

[jedevc]

As symlinks are traversed as part of file resolution, a scenario in which a package owns a file and its respective symlinks, causes multiple relationships to be created between the package and the file (as the symlinks do not appear in the list of files in the output). This seems to have been introduced in #782.

For example, see the following scan of an alpine image that contains the libz package:

    ...
    {
      "SPDXID": "SPDXRef-5baca92b12b31c62",
      "name": "zlib",
      "licenseConcluded": "Zlib",
      "description": "A compression/decompression Library",
      "downloadLocation": "https://zlib.net/",
      "externalRefs": [
        {
          "referenceCategory": "SECURITY",
          "referenceLocator": "cpe:2.3:a:zlib:zlib:1.2.12-r3:*:*:*:*:*:*:*",
          "referenceType": "cpe23Type"
        },
        {
          "referenceCategory": "PACKAGE_MANAGER",
          "referenceLocator": "pkg:alpine/[email protected]?arch=x86_64&upstream=zlib&distro=alpine-3.16.2",
          "referenceType": "purl"
        }
      ],
      "filesAnalyzed": false,
      "hasFiles": [
        "SPDXRef-6761db52f4cfb40f",
        "SPDXRef-6761db52f4cfb40f"
      ],
      "licenseDeclared": "Zlib",
      "originator": "Person: Natanael Copa <[email protected]>",
      "sourceInfo": "acquired package info from APK DB: lib/apk/db/installed",
      "versionInfo": "1.2.12-r3"
    }
    ...
    {
      "SPDXID": "SPDXRef-6761db52f4cfb40f",
      "licenseConcluded": "NOASSERTION",
      "fileName": "lib/libz.so.1.2.12"
    },
    ...
    {
      "spdxElementId": "SPDXRef-5baca92b12b31c62",
      "relationshipType": "CONTAINS",
      "relatedSpdxElement": "SPDXRef-6761db52f4cfb40f"
    },
    {
      "spdxElementId": "SPDXRef-5baca92b12b31c62",
      "relationshipType": "CONTAINS",
      "relatedSpdxElement": "SPDXRef-6761db52f4cfb40f"
    },
    ...

As you can see, we have two copies of the exact same relationship, one for the symlink /lib/libz.so.1 and one for the regular file libz.so.1.2.12 - which are both owned by the libz package. With #1156 merged, this also reflects in incorrect output for the hasFiles field. As of v0.3.0 the golang SPDX parser produces an incorrect result, and produces a nil value in the Package.Files field: see here.

We prevent these files from being confused with each other by de-duplicating the files at the point of creating ownerships, and
removing duplicate coordinates. This ensures we only get a single copy of each relationship.

[spiffcs]

Changes look good - I can add a test for this new behavior so we explicitly show future developers that duplicate ID are supposed to be filtered out.

[jedevc]

Changes look good - I can add a test for this new behavior so we explicitly show future developers that duplicate ID are supposed to be filtered out.

That would be awesome :) I'm definitely not as familiar with how syft does testing, so any help is massively appreciated 🎉