EFI: Scorecard

[sheplu]

Motivation

Scorecard are a universal way to "grade" a project, using multiple metrics like security, quality, frequency of updates... Providing this score and being able to to monitor its evolution can display the current life in the organization but also help us priorise some work on specific repository. We should try to target to have an improving score

Expectation

Implement scorecard (automatic?) part of all the ecosystem and try to improve the value over time

Implementation

Add a scorecard for all projects
Share the data publicly
Simplify access to said data

Status

Part: Technical

Draft

Security is key and leveraging the scorecard initiative (from the OpenSSF?) would help us standardize the quality of all libraries and allow the project to display the current status and help us find how we can improve.
Should we target a specific score ?

[UlisesGascon]

For reference: expressjs/security-wg#2 and #165

Should we target a specific score ?

I think rather than targeting an score, we can start to implement the low hanging fruits and monitor regularly, as a key topic in the Security WG Agenda to review any potential changes. This is what we do in the Node.js Security WG.

Then we can go with more complex things such as branch protection and so on. Many of the low hanging fruits are great first time contributions for our community, so we can onboard more collaborators in the process 🙂

[jonchurch]

PRs have been opened for this on some repos, I posted a comment in the first PR I saw but really I should ask it here:

Not worth blocking on, but is there a reason to use this ad hoc workflow vs the official ossf github action? Does this workflow do different things than the action or is otherwise preferred over the versioned and maintained action?

https://github.com/ossf/scorecard-action

jshttp/statuses#24
jshttp/mime-types#121
jshttp/http-errors#107

[UlisesGascon]

Not worth blocking on, but is there a reason to use this ad hoc workflow vs the official ossf github action? Does this workflow do different things than the action or is otherwise preferred over the versioned and maintained action?

Great point @jonchurch!

Actually we use the official one:

uses: ossf/scorecard-action@99c53751e09b9529366343771cc321ec74e9bd3d 

There is one small difference compared to the official example, as we also trigger it when there is a push in the main branch aside of the typical cron (ref) so we get all the commits scanned and not the last one from that day:

on:
  schedule:
    - cron: '16 21 * * 1'
  push:
    branches: [ "master" ]

In the official documentation is suggested to use the Workflow setup as the main way to include this action, but unfortunately this require for the contributor to have at least admin access to the repo (security tab) that will limit a lot who can do this great contributions. We achieve the same result with the manual addition or with the GUI setup, but this way we can allow new contributors to participate in the project and cultivate ownership.

IMO the scorecard setup is a great way to bring new people to the project, as it is large domain with many small tasks that can be achieve without a big understanding of the code base. Also offers a lot of improvements later on when we will monitor the scoring evolution from the following weeks.