Scorecard doesn't penalize unpinned reusable workflows in "Pinned-Dependencies" check

[sethmlarson]

Describe the bug
A clear and concise description of what the bug is.

Reproduction steps

• Create a workflow which uses a reusable workflow (e.g. slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml)
• Use a tag instead of a commit sha (@v1.2.0)
• Run Scorecard for Pinned-Dependencies ($ scorecard-linux-amd64 --repo https://github.com/sethmlarson/python-slsa-release-test --checks Pinned-Dependencies
• Observe a 10/10

Expected behavior

To not receive a 10/10, instead would be penalized for not pinning the workflow to SHA.

Additional context

Related and unfortunately in direct contention with: slsa-framework/slsa-github-generator#722

[stefreak]

It also does not find issues in local actions (e.g. uses: ./.github/actions/some-local-action)

[azeemshaikh38]

It also does not find issues in local actions (e.g. uses: ./.github/actions/some-local-action)

Should probably be a separate issue - but local actions not being penalized is ok since running your own action can be considered trustworthy?

[stefreak]

@azeemshaikh38 Local composite actions or local Docker actions could refer to third party Docker containers and third party GitHub actions without pinning. This is why the code scanner should discover all of the GitHub action code (incl. reusable workflows and composite actions / docker actions), not just the workflows themselves.

But I think you're right, this is a separate issue and I will open it separately

[spencerschrock]

So currently we assume every reference to an action will be as a step

scorecard/checks/raw/pinned_dependencies.go

Lines 524 to 531 in 0b9dfb6

	for jobName, job := range workflow.Jobs {
	jobName := jobName
	job := job
	if len(fileparser.GetJobName(job)) > 0 {
	jobName = fileparser.GetJobName(job)
	}
	for _, step := range job.Steps {
	if !fileparser.IsStepExecKind(step, actionlint.ExecKindAction) {

We miss the syntax used in your example file which is covered by:
https://docs.github.com/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_iduses

Luckily our workflow parser seems to support this via the WorkflowCall field, so should be a straighforward fix.
https://pkg.go.dev/github.com/rhysd/[email protected]#Job
https://pkg.go.dev/github.com/rhysd/[email protected]#WorkflowCall