Questions about branding and hash pinning

[vsoch]

Hey scorecard maintainers! So I just watched this video and haven't tried out the action yet, but I wanted to ask some questions about / discuss some of the things mentioned there.

• branding: hearing the things that are checked, the tool feels a bit less security oriented and a little more "open source best practices" oriented? This might have just been the way the video was presented, but I wasn't convinced all these checks are security oriented. E.g., having your list of contributor orgs reflects that scope of your contributors but in and of itself isn't going to tell you about a security issue. How are people using this in practice for actual security checks? I was expecting to see some kind of sbom generation and then comparison against a vulnerbility database.

• container hashes: I'm not sure I agree with the "if you have this debian image you should pin the hash) - if the base images are commonly rebuilt (even nightly) with things such as security updates and you are also doing your own regular builds, pinning a hash means you miss that security fix. You'd want the latest. So either you'd want to not specify a hash, or (maybe better) have a tool that updates the hash regularly before rebuild (I have one called uptodate). I do agree that setting hashes for GitHub actions that you've checked from a security standpoint makes sense, but I don't think the same is for base containers (maybe less trusted containers which arguably we shouldn't be using).

Excited to try out this tool - waiting for #78 to be addressed then I will try it out. Thanks y'all!

[laurentsimon]

Thanks for the feedback!

re: branding: you're right that not all checks are about security, and you're right many are about best practices and "healthy practices". The tool is used both by repo maintainer and dependency consumers. As a consumer, knowing who contributes to a project gives you a sense (or a signal) you can use to know more about the project. For example, if it's developed by Amazon, Google and Facebook, it may tell you that it's well maintained, and may have a certain level of coding guidelines. Heres an example of Envoyproxy's dependency policy and how they use the output of scorecard to vet their dependencies.
For maintainers, it's more about following a set of good practices in general

re: hashes: we typically ask users to use a tool for updating hashes (dependabot, renovatebot) on GitHub. The check Dependency-Update-Tool is the check that looks for the use of a such a tool.

This is the current list of checks we have https://github.com/ossf/scorecard/blob/main/docs/checks.md

Thanks again for the feedback, and please feel free to tell us what we can do better. Thanks!

[vsoch]

@laurentsimon makes total sense! I think my suggestion would be to tweak the intro paragraph here:

Right now it mentions security twice, and albeit it's for something different, I quickly saw it and (along with the youtube talk I watched) it gave me the impression that everything was about security (which as I understand now is not correct!). Maybe one line at the top like "This is a tool for.."

Thank you for making the tool!