[BUG]: Code Coverage comment fails with disallowed conclusion: action_required

[Rd4dev]

Describe the bug

The PR #5560, #5559 fails to comment the generated code coverage report.

The stack trace mentions:

Error: Run#11587680371 is completed with disallowed conclusion: action_required

Suspect:

Since a review was requested and the trigger relies on the pull request status, it may have marked the action as "required" rather than as a success or failure, which are currently the only two permitted conclusions for the coverage report. (Earlier reviews were submitted as comments, so they didn't lead to this issue.)

Updated Findings:

I initially thought the action_required status might have been triggered by a review request, but during testing, I was able to replicate it by setting rules to manually require workflow approvals. This makes me think the issue could indeed be related to the approval requirements (notably, both PRs where this occurred required workflow approval).

Given this, simply adding an action_required conclusion or adjusting the check run handling might not fully resolve the issue. None of the reproductions were conclusive, so a more thorough investigation is needed to identify and resolve the underlying cause.

Steps To Reproduce

• Create a PR and request a review
• Ensure this review is of the type that requires an action ("Request Changes") instead of just adding comments.
• Once the review is submitted, it should be marked as "action_required" in the comment coverage stack trace

Expected Behavior

It should get a success conclusion and post a coverage comment with the generated coverage data.

Screenshots/Videos

No response

What device/emulator are you using?

No response

Which Android version is your device/emulator running?

No response

Which version of the Oppia Android app are you using?

No response

Additional Context

Adding "action_required" to the desired conclusions could be a solution if that's the only cause of this issue, yet it need to be validated.

[manas-yu]

@adhiamboperes can i work on this issue? We can fix this issue by explicitly handling the action_required status in the check_coverage_results section of the code_coverage.yml file.

- name: Handle action required status
        if: ${{ needs.code_coverage_run.result == 'action_required' || needs.evaluate_code_coverage_reports.result == 'action_required' }}
  run: |
    echo "Action required on code coverage. Review manually before proceeding."
 exit 0

[Rd4dev]

@manas-yu, thank you for looking into this! Assigning this issue to you to let you take over.

I’d suggest prioritizing a straightforward approach by adding 'action_required' to allowed_conclusion and testing if that resolves it. Also, please verify separately that code coverage still runs and posts comments, as this is needed even when a PR review request is made.

[Rd4dev]

Updated Findings:

I initially thought the action_required status might have been triggered by a review request, but during testing, I was able to replicate it by setting rules to manually require workflow approvals. This makes me think the issue could indeed be related to the approval requirements (notably, both PRs where this occurred required workflow approval).

Given this, simply adding an action_required conclusion or adjusting the check run handling might not fully resolve the issue. None of the reproductions were conclusive, so a more thorough investigation is needed to identify and resolve the underlying cause.

[BenHenning]

From discussing this in the CLaM meeting, I'm starting to think that it many not be possible to fully fix this at this time. Some thoughts:

• It may be worth filing a bug against https://github.com/kamilchodola/wait-for-workflow-action since it's handled a bit weirdly.
• Could we maybe have a better error message that indicates the workflow will need to be re-run?

I think if we can handle the failure case a bit better, this might not be a bad outcome since it should only affect new contributors who need their workflows approved.

[Rd4dev]

Just for clarity, the utilized external action is: https://github.com/ArcticLampyrid/action-wait-for-workflow

Filed the issue: ArcticLampyrid/action-wait-for-workflow#269