Management plane: Naming Review - Azure DevEx Arch Board

[ladonnaq]

Let's use this GitHub issue to track the outstanding requirements for automation of the naming process for initial management plane libraries.

Status on support of naming approval for management plane via APIView per @praveenkuttappan

API reviews for management packages are automatically generated now for C#, Java and JS but not for Python. We should consider the following before we enable release check to enforce "Approval for first preview" for management packages.

Even though API reviews are already getting generated for few languages, we do not enforce approval of these API reviews and allow management packages to be released without API approval. Enabling approval status check for management package in APIView (approval for first preview release) by release pipeline will require architects to approve any new package that goes out as preview. Approval is not required if package was already released as GA before.

More API reviews will get created and visible in APIView for Python once we enable api review gen for management for Python.

Changes required for APIView:

• Update Python CI pipelines to enable automatic reviews for management packages.

• Update all CI and release pipelines for management package supported languages to verify "Approved for first preview" status, which means package name is approved for preview release, and fail the release pipeline if not approved.

I assume API approval status check, which is done by architects after reviewing all APIs, is not in the scope of this change and feature request here is to check only for preview version basic approval like package name is valid etc.

Arch Board management plane naming process per @ronniegeraghty

• Service team requests a review of their MGMT Namespaces and link their API Spec. 
• Automate the generation of the namespaces based on the submitted API Spec. 
• Eventually have the automatically generated namespaces added to API View were someone from the MGMT side (most likely Michael Nash) approves the generated names.
• Today (02/23/2023) the names are just put into a GitHub Issue and they sign off on the names there with a comment. Once the names are approved, send an FYI email to all the architects with the approved names. (Could automate this as well)
• Architects have 1 week to object to the names. If week passes with no objections the names are approved. If architect object the approval is held until their comments are addressed.

Release Planner requirements

• Remove the "review ticket" from API readiness. This was added so that management plane teams could include the GitHub issue related to naming as a item to be tracked for API readiness. This does not belong in API readiness. This should be moved to SDK release.
• Instead have a UI that will allow user's to suggest the package names and automate the rest of the process. If we still need a GitHub issue, that would be created. Status of the naming approval should be shown on the SDK release dashboard. Naming approval has to be completed before work related to SDK release. For example, the testing and manual steps for .NET release.

Dashboard/Reporting

• The APIView link(s) for approval stored and surfaced in the Release Planner work item (so that we can surface links to API Views in Release Plan dashboard)
• Track interval between request of SDK namespace review and the approval of the SDK namespaces for all languages. Store this data and surface to PowerBI Dashboard via Azure Devops or dataverse. This data can be used to determine increased productivity.
• Track the number of management SDK namespace requests where the requested namespace differs from the Azure Resource Manager (ARM)/Resource Provider (RP) namespace.
• Track number of objections by architect to the service partner proposed namespaces.

###Interim TODOs (until we can execute plan above)

• Create an GitHub issue template for management plane naming approval  #5680
• Update Arch Board Review wiki page and relevant email/calendar body templates to include info on namespace approval  #5682

[ladonnaq]

@kyle-patterson please review the scenarios that require naming review and edit accordingly. Also, do you think a new meeting type is needed? We could also have a form in the release planner app that takes as input the suggested namespaces, then it could create a GitHub issue which is assigned to the language architects for asynchronous review. If a review was needed the architects could add sync label (similar to what we discussed yesterday for the SDK reviews).

[ladonnaq]

@kyle-patterson and @ronniegeraghty I would like to get this feature in the release planner app. Would you please review and provide more detail so I can hand it over to engineering. The release planner can automatically create the GitHub issue, put it in a GitHub project for you if GitHub action is enabled, and add the appropriate labels.

[ladonnaq]

There is work in progress that is a dependency for the release planner for the naming feature. The release planner needs to add a step to the SDK release plan and a blade so that users can create a GitHub issue for naming.

Work in progress or recently complete in APIView

• Implementing a button for naming approval for each language. I am not sure if this is enabled yet for all languages. I will check with Praveen on status. I did not GitHub issue in the APIView project.
• APIView is implementing a feature to enforce that naming has been approved before preview packages are released. The build will fail if naming approval is not complete.
• Related GitHub issues below. I will ask Praveen about the other languages, did not see the GitHub issues in the APIView project.
  Enable check to ensure preview release has package name approval azure-sdk-for-js#23892
  Enable check for preview release approval in APIView for Python azure-sdk-for-python#27572
• While not directly related to naming but important to the initial beta SDK releases - There is additional work in progress to support approval for first release (Beta) in APIView as well - UI improvements/approve buttons #4812
• Until we can complete implementation, I have documented and Ronnie has reviewed the manual process to do this task - https://dev.azure.com/azure-sdk/internal/_wiki/wikis/internal.wiki/821/Naming-for-new-initial-management-or-client-libraries-(new-SDKs). I have also added to the launch criteria and enforcing manually at the moment. I have also been connecting with service teams that have upcoming initial SDK releases so that they are aware and complete the naming approval process. This includes data and management plane.

[RickWinter]

@ladonnaq Can this work be expanded to have APIView require C/C++/GO to approve the name on first beta release. I'd like to avoid shipping unapproved package names in the those languages as well.

[maririos]

@ladonnaq as I mentioned in our Teams channel, according to the requirements asked from us, all implementation from APIView has been done. @RickWinter this works for all languages that support APIView, same as API aprovals

[maririos]

Correction on my statement: This is only true for data plane libraries.

There is no support for naming approval or API approval for management plane libraries. The later was a request for architects as the libraries are mostly autogenerated.
If this request changes, please involve @mikekistler as the APIView PM

[RickWinter]

@maririos , @mikekistler , @ladonnaq - We need this naming validation for control plane as well. That is where the two most recent mistakes have occurred.

[ladonnaq]

@maririos , @mikekistler , @ladonnaq - We need this naming validation for control plane as well. That is where the two most recent mistakes have occurred.

I just found out management plane is not supported. I opened a GitHub issue and tagged @RickWinter
and assigned to @mikekistler #4922

[ladonnaq]

@maririos @mikekistler Have there been any discussions around how to support management plane naming request in APIView as well? If management plane will not be supported for approving naming, then we should have the release planner gather all of the necessary information to create the GitHub issue and use labels to track approval of the each package. Here is the manual process today for naming - https://dev.azure.com/azure-sdk/internal/_wiki/wikis/internal.wiki/821/Naming-for-new-initial-management-or-client-libraries-(new-SDKs)

[maririos]

Meeting scheduled to discuss this topic.

[maririos]

@praveenkuttappan could you provide the requirements and implications for this feature to be done?

[ladonnaq]

@maririos @ronniegeraghty @praveenkuttappan I updated the description with the current summary of requirements. Let's use comments for discussions and then when we have agreement, hide the comment and update the description.

[ladonnaq]

@maririos @ronniegeraghty @praveenkuttappan

Discussion the arch board agreed upon process for naming approval for management plane

Let's review and identify what is needed to automate and include this in the release planner.

• Service team requests a review of their MGMT Namespaces and link their API Spec. 

• Automate the generation of the namespaces based on the submitted API Spec.

• Eventually have the automatically generated namespaces added to API View were someone from the MGMT side (Michael Nash has been nominated) approves the generated names.

• Right now the names are just put into a GitHub Issue and they sign off on the names there with a comment. 

• Once the names are approved, send an FYI email to all the architects with the approved names. (Could automate this as well) Architects have 1 week to object to the names. If week passes with no objections the names are approved. If architect object the approval is held until their comments are addressed.

• [La Donna] I suggest that we automate this via the release planner.

• The Release Planner should be able to figure out the link to the API Spec based on the PR.

• The release planner can generate the namespaces.

• The release planne can create the GitHub issue and include any relevant data (APIView links, API Spec link, PR link, ...)

• The release planner should be able to automatically add the generated namespaces to API View. Michael Nash can be notified as the approval that new management plane namespaces are ready for review.

• Once the generated namespaces are added to API View there should be a label added to the GitHub issue. Perhaps Mgmt-namespace-inReview

• Once Michael approves the names in API View there should be a label added to the GitHub issue. Perhaps Mgmt-namespace-approvalPending

• Automatically send the email to the architects.

• After 1 week has passed, the arch board PM adds the label Mgmt-namespace-approved.

• If there is a language that requires additional time to review, we should have a label or some way to communicate to the service partner that we may need their input and what the outstanding issues are blocking approval.

[maririos]

@ladonnaq

The release planner can generate the namespaces.

Could you provide the logic to make this happen?

[ladonnaq]

@ladonnaq

The release planner can generate the namespaces.

Could you provide the logic to make this happen?

@ronniegeraghty How would the generated namespace be determined from the PR?

• Here a PR for a new service for management plane - https://github.com/Azure/azure-rest-api-specs-pr/pull/11212.

• From the naming wiki page this is format and bolded is the "namespace" guidance we provide now.

Example: management (ARM) libraries

bolded and italicized text is namespace

• .NET
  Resource Management - Confidential Ledger
  Azure.ResourceManager.ConfidentialLedger

• Java
  Resource Management - Confidential Ledger
  azure-resourcemanager-confidentialledger

• JavaScript
  Resource Management - Confidentialledger
  @azure/arm-confidentialledger

• Go/Golang
  Resource Management - Confidentialledger
  sdk/resourcemanager/confidentialledger/armconfidentialledger

• Python

• Resource Management - Confidential Ledger
  azure-mgmt-confidentialledger

• So with this example - what would be the generated namespace? devtunnels? I think that we will want the service team to confirm that the generated namespace is acceptable before it is routed to APIView for approval.

• We could show the generated namespace in the release planner, allow user to modify, and then create the GitHub issue for namespace/naming approval for the management libraries. This would be done in the SDK release app as part of the Public Preview release plan for new services and new ARM namespaces that result in new initial libraries.

[maririos]

Issue moved to the SDK Release epic so we can look into it there

[maririos]

@ronniegeraghty the current process is the service team needs to go to GH and create an issue following a template to provide the name.
do you consider this could be something we should automate in the Release Planner? is there a template for the names to be proposed by automation too?

[ronniegeraghty]

YES!
The Mgmt Plane Namespace Reviews are a great opportunity to automate a manual process.
The names use the following patterns for each language:

• .NET: Azure.ResourceManager.[ResourceProviderName]
• Java: azure-resourcemanager-[resourceprovidername] (com.azure.resourcemanager.[resourceprovidername])
• Go/Golang: sdk/resourcemanager/[resourceprovidername]/arm[resourceprovidername]
• JavaScript: @azure/arm-[resourcecprovidername]
• Python: azure-mgmt-[resourceprovidername]

The [ResourceProviderName] section should be replaced with the services branding name, usually the RP name. The casing of the section that replaces [ResourceProviderName] should follow the same casing it replaces.

Normally the service partner team submits a GH issues with their proposed names following the patterns. Then Arthur from the MGMT Plane team review the names with the API Spec and signs off on them. Then the names are shown to the whole architect team and the architects have a week to make any objections to the names. If there are no objections in the week then they are considered approved. If there are objections, the service partner team and architects work together to come up with a name that both sides agree on.

An automated version of this could look like the following:
Service partner team comes to section in the release planner where they are asked to proposed namespaces for their MGMT Plane libraries. The names are auto-filled using the patterns and the services RP name from their spec, but the service team can edit them if that doesn't match their branding name. Once they confirm their proposed namespaces the architects are sent approvals to review in API View for the namespaces.

[maririos]

Awesome.

Once they confirm their proposed namespaces the architects are sent approvals to review in API View for the namespaces.
Could you elaborate more on how from the proposed namespace confirmation on the Release Planner we go to APIViews with the namespace to approve?

An intermediate step could be: once they confirm their proposed namespaces the tool creates the GH issue with the proposal.
And after that the process you said. Arthur reviews, and then the info is sent to architects.

[ronniegeraghty]

An intermediate step could be: once they confirm their proposed namespaces the tool creates the GH issue with the proposal. And after that the process you said. Arthur reviews, and then the info is sent to architects.

This could work. If the Release planner opens the GitHub Issue and Assigns it to Arthur. Then Arthur works with the partner team in the issue for the 1st phase of the review. Then once Arthur and the service team have come to an agreement, Arthur could add a specific label that triggers an email to be sent out to the architects and starts the 1 week count down. The architects we'll have to start putting their comments and objections in the issue comments and not in the email thread.

[ladonnaq]

@ronniegeraghty @maririos - Since Crystal will be on medical leave for 1H 2024, should I go ahead and schedule a meeting with you, Ronnie, and Jonathon so we can work out UI and back-end needed to implement this feature. I think it would be very helpful for our service teams.

[maririos]

Closing in favor of #7607