Consolidate azure-rest-api-specs PR review workflow

[konrad-jamrozik]

There is a workflow that is being followed by several actors when a PR is submitted against https://github.com/Azure/azure-rest-api-specs

This work item is about mapping out this entire workflow and consolidating the guidance. This will allow us to make informed improvements to it and provide support to involved actors when necessary, including the PR submitter and the assigned PR reviewer. One person that can help with that is Mike Kistler, who is a member of the API Stewardship Board.

Notes on the workflow

The guidance to submit a PR is in the following:

• https://aka.ms/specprreview
• and in the other URLs linked by Mariana below
• In the list given in the doc map article.
• this OneNote
• This FAQ
• WIP Guide to design and creation of Data Plane REST API and Client Libraries
• List from Crystal Barragan here (issue Draft content structure in Engineering Hub docs site #5989)

There are also templates:

• https://github.com/Azure/azure-rest-api-specs/tree/main/.github/PULL_REQUEST_TEMPLATE
• https://github.com/Azure/azure-rest-api-specs/blob/main/.github/PULL_REQUEST_TEMPLATE/control_plane_template.md
• https://github.com/Azure/azure-rest-api-specs/blob/main/.github/PULL_REQUEST_TEMPLATE/data_plane_template.md
• The templates have unclear ownership, per: Update azure-rest-api-specs PR template to allow for dev branches #5290 (comment)

Once PR is submitted happens, a person is assigned at random to it, per this list:
https://github.com/Azure/azure-rest-api-specs/blob/main/.github/pull_request_assignment.yml

After this there is a flow based on labels depending on what happens. See the labelling workflow diagram.

For example, if a breaking change is detected by the OAD tool from Unified Pipeline, then the PR gets auto-labelled with a label denoting there is a breaking change requiring review. Then a reviewer from a "breaking change board" is adding another label, e.g. one denoting that the breaking change got approved.

There is also suppression process, described here:
https://dev.azure.com/azure-sdk/internal/_wiki/wikis/internal.wiki/85/Swagger-Suppression-Process

Detailed tasks

Below is a non-exhaustive list of subtasks:

We need to:

• understand what are the responsibilities of the auto-assigned PR assignee.
• come up with a process for PR assignees to escalate to us, e.g. file issues against azure-sdk-tools repo
• understand the mechanics of the labelling process
  • this issue has good pointers: [PR Workflow] Swagger unified pipeline avoid adding ARM review label for those PR target branch to dev-* #4675
  • https://dev.azure.com/azure-sdk/internal/_wiki/wikis/internal.wiki/909/API-spec-PR-review
    • and specifically the labelling workflow diagram
• understand the breaking changes process
  • https://aka.ms/azbreakingchangespolicy from https://github.com/microsoft/api-guidelines/tree/vNext/azure, mentioned by Jeffrey Richter
  • info here https://aka.ms/specprreview
    • links to https://dev.azure.com/msazure/AzureWiki/_wiki/wikis/AzureWiki.wiki/37684/Breaking-Changes?anchor=best-practice/guidelines-on-common-scenarios
      • links to https://eng.ms/docs/cloud-ai-platform/azure-core/azure-core-pm-and-design/trusted-platform-pm-karimb/service-lifecycle-actions-team/service-lifecycle-actions-team/testfolder/media/overview_breakingchanges

Related work

• Consolidate support guidance for API spec PR tools #5867
• [Breaking Change][Documentation] Consolidate "breaking changes" workflow guidance #5983
• https://dev.azure.com/azure-sdk/internal/_wiki/wikis/internal.wiki/942/Doc-map

[maririos]

The guidance to submit a PR is here:
https://aka.ms/specprreview

I wish this was the only place where there was guidance. Here are the other links I have seen people use and share around. I have been collecting them but haven't had the time to look at each and see the accuracy of the information, etc. Sharing them here as FYI for you:

• https://dev.azure.com/azure-sdk/internal/_wiki/wikis/internal.wiki/630/Walkthrough-Creating-a-new-service-specification-in-a-team
• https://armwiki.azurewebsites.net/rp_onboarding/process/api_review.html
• https://github.com/Azure/typespec-azure/blob/main/docs/howtos/rest-api-publish/checking-in-api-specs-guide.md

[weshaggard]

@mikekistler @josefree are a couple folks that I think can help with this.

[konrad-jamrozik]

Closing as completed. Big chunk of this work was taken care in:

• [Documentation] Migrate spec PR review workflow support docs to EngHub #6618

Other parts are obsolete / subsumed / no longer applicable. Yet other elements are scheduled to be done in other work items, like:

• [Breaking Change][PR Workflow] Use more granular labels for Breaking Changes approvals #6374