Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KeyVault: Feature/update security domain spec #12863

Merged
merged 11 commits into from
Feb 9, 2021
Merged

KeyVault: Feature/update security domain spec #12863

merged 11 commits into from
Feb 9, 2021

Conversation

docschmidt
Copy link
Contributor

MSFT employees can try out our new experience at OpenAPI Hub - one location for using our validation tools and finding your workflow.

Changelog

Please ensure to add changelog with this PR by answering the following questions.

  1. What's the purpose of the update?

    • new service onboarding
    • new API version
    • update existing version for new feature
    • update existing version to fix swagger quality issue in s360
    • Other, please clarify
  2. When you are targeting to deploy new service/feature to public regions? Please provide date, or month to public if date is not available yet.

    • April 2021
  3. When you expect to publish swagger? Please provide date, or month to public if date is not available yet.

    • April 2021
  4. If it's an update to existing version, please select SDKs of specific language and CLIs that require refresh after swagger is published.

    • SDK of .NET (need service team to ensure code readiness)
    • SDK of Python
    • SDK of Java
    • SDK of Js
    • SDK of Go
    • [] PowerShell
    • [] CLI
    • Terraform
    • No, no need to refresh for updates in this PR

Contribution checklist:

If any further question about AME onboarding or validation tools, please view the FAQ.

ARM API Review Checklist

  • Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

    • Adding new API(s)
    • Adding a new API version
    • Adding a new service
  • Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

  • If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist

If there are following updates in the PR, ensure to request an approval from API Review Board as defined in the Breaking Change Policy.

  • Removing API(s) in stable version
  • Removing properties in stable version
  • Removing API version(s) in stable version
  • Updating API in stable version with Breaking Change Validation errors
  • Updating API(s) in preview over 1 year

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.

Please follow the link to find more details on PR review process.

@openapi-workflow-bot
Copy link

Hi, @docschmidt Thanks for your PR. I am workflow bot for review process. Here are some small tips.

  • Please ensure to do self-check against checklists in first PR comment.
  • PR assignee is the person auto-assigned and responsible for your current PR reviewing and merging.
  • For specs comparison cross API versions, Use API Specs Comparison Report Generator
  • If there is CI failure(s), to fix CI error(s) is mandatory for PR merging; or you need to provide justification in PR comment for explanation. How to fix?

  • Any feedback about review process or workflow bot, pls contact swagger and tools team. [email protected]

    Copy link
    Member

    @heaths heaths left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    All comments made to the 7.2-preview version apply to 7.3-preview.

    "type": "string"
    }
    }
    },
    "200": {
    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Why return both a 202 or a 200? When should the SDK - and customers - expect 1 vs. the other? This isn't consistent with other LROs in Key Vault. Looking at others, they return a 202 only that represents the operation. Polling the pending method is how status and the eventual object (security domain) is retrieved.

    Copy link
    Contributor Author

    @docschmidt docschmidt Feb 5, 2021

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    When a managed HSM resource is created, the resource is in what we call the Provisioned state. At this point, customers have to call either the /securitydomain/upload endpoint to restore a Security Domain or the /securitydomain/download endpoint to confirm the Security Domain of the created pool. Both restoring and confirming are long running operations. Once complete the managed HSM goes into Active state at which point the managed HSM will accept other service calls.

    Once the managed HSM is active, we want to allow customers to redownload the Security Domain, for example to rotate encryption keys. Merely downloading the Security Domain can be satisfied right away, and so the service would return 200 instead of 202. If this is an anti-pattern, please let me know. We may need to implement an additional endpoint then (or change to "get" for download only).

    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Yes, this can be a problem. In one case you're asking to return a long-running operation object (LRO) but in another the desired model. Even if we used a union of sorts for strongly-typed languages (.NET doesn't support this, but we could fake it) how do customers know to intuitively use one over the other?

    Instead, always returning an LRO allows you to return one in an already-resolved state, which we support today.

    /cc @johanste @JeffreyRichter for guidance.

    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Another possibility - though not ideal since we can't really rely on the generated code - is to have the SDKs always return an LRO anyway (e.g. in .NET: something deriving from Operation<T>) that could be either in a fully resolved state for HTTP 200, or act like a normal LRO for HTTP 202. We actually do something like this for delete and recover operations, which aren't declared as LROs but may take a while, so we "override" those methods to work like LROs even if they are already completed. (Also, the REST APIs themselves return the model right away regardless of whether the operation is done, so we already have the model.)

    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    The service should not flip-flop between 200 and 202 responses. It should pick one pattern and stick with it. It becomes challenging for clients to have to fork their code paths depending on if the request could be satisfied "synchronously" or not.

    "type": "string",
    "description": "SHA 256 hash of certificate"
    "format": "base64url",
    "description": "A blob containing the Security Domain."
    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Like the security policy for key export/release, should this have some sort of contentType to inform the customer what it is or how to store it or, more importantly, that needs to be sent back as-is for compatibility? Or will this opaque blob always work (or maybe even contain an embedded version)?

    /cc @herveyw-msft

    Copy link
    Contributor Author

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    The opaque blob contains a version number and it is our intent that the opaque blob will always work.

    ….2-preview/securitydomain.json
    
    Co-authored-by: Heath Stewart <[email protected]>
    @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented Feb 5, 2021

    Swagger Validation Report

    ️❌BreakingChange: 26 Errors, 0 Warnings failed [Detail] Only 10 items are listed, please refer to log for more details.

    Rule Message
    1006 - RemovedDefinition The new version is missing a definition that was found in the old version. Was 'SecurityDomainUploadObject' removed or renamed?
    New: Microsoft.KeyVault/preview/7.2-preview/securitydomain.json#L238:3
    Old: Microsoft.KeyVault/preview/7.2-preview/securitydomain.json#L187:3
    1006 - RemovedDefinition The new version is missing a definition that was found in the old version. Was 'SecurityDomainUploadObject' removed or renamed?
    New: Microsoft.KeyVault/preview/7.3-preview/securitydomain.json#L238:3
    Old: Microsoft.KeyVault/preview/7.3-preview/securitydomain.json#L187:3
    1011 - AddingResponseCode The new version adds a response code '202'.
    New: Microsoft.KeyVault/preview/7.2-preview/securitydomain.json#L84:11
    1011 - AddingResponseCode The new version adds a response code '204'.
    New: Microsoft.KeyVault/preview/7.2-preview/securitydomain.json#L192:11
    1011 - AddingResponseCode The new version adds a response code '202'.
    New: Microsoft.KeyVault/preview/7.3-preview/securitydomain.json#L84:11
    1011 - AddingResponseCode The new version adds a response code '204'.
    New: Microsoft.KeyVault/preview/7.3-preview/securitydomain.json#L192:11
    1023 - TypeFormatChanged The new version has a different format than the previous one.
    New: Microsoft.KeyVault/preview/7.2-preview/securitydomain.json#L292:9
    Old: Microsoft.KeyVault/preview/7.2-preview/securitydomain.json#L431:9
    1023 - TypeFormatChanged The new version has a different format than the previous one.
    New: Microsoft.KeyVault/preview/7.3-preview/securitydomain.json#L292:9
    Old: Microsoft.KeyVault/preview/7.3-preview/securitydomain.json#L431:9
    1026 - TypeChanged The new version has a different type 'string' than the previous one 'object'.
    New: Microsoft.KeyVault/preview/7.2-preview/securitydomain.json#L292:9
    Old: Microsoft.KeyVault/preview/7.2-preview/securitydomain.json#L431:9
    1026 - TypeChanged The new version has a different type 'string' than the previous one 'object'.
    New: Microsoft.KeyVault/preview/7.3-preview/securitydomain.json#L292:9
    Old: Microsoft.KeyVault/preview/7.3-preview/securitydomain.json#L431:9
    ️❌LintDiff: 2 Errors, 0 Warnings failed [Detail]

    Rule Message
    AutoRest exception "readme":"keyvault/data-plane/readme.md",
    "tag":"package-preview-7.3-preview",
    "details":"Error: Lexical error on line 1. Unrecognized text."
    AutoRest exception "readme":"keyvault/data-plane/readme.md",
    "tag":"package-7.2-preview",
    "details":"Error: Lexical error on line 1. Unrecognized text."
    ️️✔️Avocado succeeded [Detail] [Expand]
    Validation passes for Avocado.
    ️️✔️ModelValidation succeeded [Detail] [Expand]
    Validation passes for ModelValidation.
    ️️✔️SemanticValidation succeeded [Detail] [Expand]
    Validation passes for SemanticValidation.
    ️️✔️[Staging] Cross Version BreakingChange (Base on preview version) succeeded [Detail] [Expand]
    There are no breaking changes.
    ️️✔️[Staging] Cross Version BreakingChange (Base on stable version) succeeded [Detail] [Expand]
    There are no breaking changes.
    ️️✔️CredScan succeeded [Detail] [Expand]
    There is no credential detected.
    ️⚠️[Staging] SDK Track2 Validation: 728 Warnings warning [Detail] Only 10 items are listed, please refer to log for more details.

    Rule Message
    ⚠️ PreCheck/SchemaMissingType "readme":"keyvault/data-plane/readme.md",
    "tag":"package-preview-7.3-preview",
    "details":"The schema 'RestoreOperationParameters' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'"
    ⚠️ PreCheck/SchemaMissingType "readme":"keyvault/data-plane/readme.md",
    "tag":"package-preview-7.3-preview",
    "details":"The schema 'SelectiveKeyRestoreOperationParameters' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'"
    ⚠️ PreCheck/SchemaMissingType "readme":"keyvault/data-plane/readme.md",
    "tag":"package-preview-7.3-preview",
    "details":"The schema 'SelectiveKeyRestoreOperation' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'"
    ⚠️ PreCheck/SchemaMissingType "readme":"keyvault/data-plane/readme.md",
    "tag":"package-preview-7.3-preview",
    "details":"The schema 'SASTokenParameter' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'"
    ⚠️ PreCheck/SchemaMissingType "readme":"keyvault/data-plane/readme.md",
    "tag":"package-preview-7.3-preview",
    "details":"The schema 'FullBackupOperation' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'"
    ⚠️ PreCheck/SchemaMissingType "readme":"keyvault/data-plane/readme.md",
    "tag":"package-preview-7.3-preview",
    "details":"The schema 'RestoreOperation' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'"
    ⚠️ PreCheck/SchemaMissingType "readme":"keyvault/data-plane/readme.md",
    "tag":"package-preview-7.3-preview",
    "details":"The schema 'CertificateAttributes' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'"
    ⚠️ PreCheck/SchemaMissingType "readme":"keyvault/data-plane/readme.md",
    "tag":"package-preview-7.3-preview",
    "details":"The schema 'CertificateItem' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'"
    ⚠️ PreCheck/SchemaMissingType "readme":"keyvault/data-plane/readme.md",
    "tag":"package-preview-7.3-preview",
    "details":"The schema 'CertificateIssuerItem' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'"
    ⚠️ PreCheck/SchemaMissingType "readme":"keyvault/data-plane/readme.md",
    "tag":"package-preview-7.3-preview",
    "details":"The schema 'CertificateBundle' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'"
    ️️✔️[Staging] PrettierCheck succeeded [Detail] [Expand]
    Validation passes for PrettierCheck.
    ️️✔️[Staging] SpellCheck succeeded [Detail] [Expand]
    Validation passes for SpellCheck.
    Posted by Swagger Pipeline | How to fix these errors?

    @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented Feb 5, 2021

    Swagger Generation Artifacts

    ️️✔️ azure-sdk-for-go succeeded [Detail] [Expand]
    • ️✔️Succeeded [Logs]Release - Generate from 0d32cdf. SDK Automation 14.0.0
      command	sh ./initScript.sh ../../../../../azure-sdk-for-go_tmp/initInput.json ../../../../../azure-sdk-for-go_tmp/initOutput.json
      command	go run ./tools/generator/main.go ../../../../../azure-sdk-for-go_tmp/generateInput.json ../../../../../azure-sdk-for-go_tmp/generateOutput.json
    • ️✔️keyvault/v7.1/keyvault [View full logs]  [Release SDK Changes]
      info	[Changelog] No exported changes
    • ️✔️keyvault/v7.0/keyvault [View full logs]  [Release SDK Changes]
      info	[Changelog] No exported changes
    • ️✔️keyvault/2016-10-01/keyvault [View full logs]  [Release SDK Changes]
      info	[Changelog] No exported changes
    • ️✔️keyvault/2015-06-01/keyvault [View full logs]  [Release SDK Changes]
      info	[Changelog] No exported changes
    • ️✔️preview/keyvault/v7.2-preview/keyvault [View full logs]  [Release SDK Changes] Breaking Change Detected
      Only show 24 items here, please refer to log for details.
      info	[Changelog] - New function `RoleDefinitionsClient.CreateOrUpdateResponder(*http.Response) (RoleDefinition, error)`
      info	[Changelog] - New function `HSMSecurityDomainClient.DownloadPendingSender(*http.Request) (*http.Response, error)`
      info	[Changelog] - New function `RoleDefinitionsClient.CreateOrUpdateSender(*http.Request) (*http.Response, error)`
      info	[Changelog] - New function `RoleDefinitionsClient.Get(context.Context, string, string, string) (RoleDefinition, error)`
      info	[Changelog] - New function `PossibleRoleScopeValues() []RoleScope`
      info	[Changelog] - New function `RoleDefinitionsClient.DeleteResponder(*http.Response) (RoleDefinition, error)`
      info	[Changelog] - New function `RoleDefinitionsClient.Delete(context.Context, string, string, string) (RoleDefinition, error)`
      info	[Changelog] - New function `HSMSecurityDomainClient.DownloadPendingResponder(*http.Response) (SecurityDomainOperationStatus, error)`
      info	[Changelog] - New function `PossibleDataActionValues() []DataAction`
      info	[Changelog] - New function `HSMSecurityDomainClient.DownloadPending(context.Context, string) (SecurityDomainOperationStatus, error)`
      info	[Changelog] - New function `PossibleRoleDefinitionTypeValues() []RoleDefinitionType`
      info	[Changelog] - New function `PossibleRoleTypeValues() []RoleType`
      info	[Changelog] - New function `HSMSecurityDomainClient.DownloadPendingPreparer(context.Context, string) (*http.Request, error)`
      info	[Changelog] - New function `RoleDefinitionsClient.CreateOrUpdate(context.Context, string, string, string, RoleDefinitionCreateParameters) (RoleDefinition, error)`
      info	[Changelog] - New struct `HSMSecurityDomainDownloadFuture`
      info	[Changelog] - New struct `RoleDefinitionCreateParameters`
      info	[Changelog] - New field `Value` in struct `SecurityDomainObject`
      info	[Changelog] - New anonymous field `autorest.Response` in struct `RoleDefinition`
      info	[Changelog] - New field `Iv` in struct `KeyOperationResult`
      info	[Changelog] - New field `AuthenticationTag` in struct `KeyOperationResult`
      info	[Changelog] - New field `AdditionalAuthenticatedData` in struct `KeyOperationResult`
      info	[Changelog]
      info	[Changelog] Total 41 breaking change(s), 61 additive change(s).
      info	[Changelog]
    ️🔄[Staging] ApiDocPreview inProgress [Detail]
    Posted by Swagger Pipeline | How to fix these errors?

    Co-authored-by: Heath Stewart <[email protected]>
    @openapi-workflow-bot
    Copy link

    Hi @docschmidt, Your PR has some issues. Please fix the CI sequentially by following the order of Avocado, semantic validation, model validation, breaking change, lintDiff.

    TaskHow to fixPrioritySupport (Microsoft alias)
    AvocadoFix-AvocadoHighruowan
    Semantic validationFix-SemanticValidation-ErrorHighraychen, jianyxi
    Model validationFix-ModelValidation-ErrorHighraychen,jianyxi
    LintDiffFix-LintDiffhighjianyxi, ruoxuan
    If you need further help, please feedback via swagger feedback."

    @openapi-workflow-bot
    Copy link

    NewApiVersionRequired reason:

    A service’s API is a contract with customers and is represented by using the api-version query parameter. Changes such as adding an optional property to a request/response or introducing a new operation is a change to the service’s contract and therefore requires a new api-version value. This is critically important for documentation, client libraries, and customer support.

    EXAMPLE: if a customer calls a service in the public cloud using api-version=2020-07-27, the new property or operation may exist but if they call the service in a government cloud, air-gapped cloud, or Azure Stack Hub cloud using the same api-version, the property or operation may not exist. Because there is no clear relationship between the service api-version and the new property/operation, customers can’t trust the documentation and Azure customer have difficulty helping customers diagnose issues. In addition, each client library version documents the service version it supports. When an optional property or new operation is added to a service and its Swagger, new client libraries must be produced to expose this functionality to customers. Without updating the api-version, it is unclear to customers which version of a client library supports these new features.

    @lmazuel lmazuel merged commit 0d32cdf into Azure:master Feb 9, 2021
    iscai-msft added a commit to iscai-msft/azure-rest-api-specs that referenced this pull request Feb 10, 2021
    …into fix_resource_multiapi_submodule
    
    * 'master' of https://github.com/Azure/azure-rest-api-specs: (452 commits)
      Dev cost management microsoft.cost management 2020 12 01 preview new (Azure#12259)
      Edit securityContact swagger for 2020-01-01 (Azure#12265)
      [Hub Generated] Review request for Microsoft.Insights to add version stable/2020-10-01 (Azure#11579)
      [Hub Generated] Review request for Microsoft.Media to add version stable/2020-05-01 (Azure#12681)
      Fix TimeSeriesInsights swagger issues (Azure#12204)
      Fixing error in lastModifiedAt description (Azure#12854)
      Update comment.yml (Azure#12910)
      Fixing RI s360 Flagged corrections in 2019-10-01 version (Azure#12777)
      [Hub Generated] Review request for Microsoft.Web to add version stable/2020-10-01 (Azure#11636)
      [Ready For Review] New version 20210101 (Azure#12111)
      Update scheduledQueryRule_API.json (Azure#12895)
      Remove UserAssigned MSI (Azure#12900)
      Update library.json (Azure#12922)
      KeyVault: Feature/update security domain spec (Azure#12863)
      Fixes ExampleId type and incorrect ArmTokenParameter name (Azure#12896)
      Update contract for anomaly detector (Azure#12487)
      [NetAppFiles] Urgent bug fix, backup response, remove systemData(not in response yet) (Azure#12852)
      [AML] Add PipelineEndpoint with version and DataPathAssignments fields in AzureMLExecutePipeline (Azure#12744)
      remove duplicated schema (Azure#12779)
      [Web] Add Swagger for Service Principal (Azure#12780)
      ...
    @docschmidt docschmidt deleted the feature/update-security-domain-spec branch February 11, 2021 05:30
    mkarmark pushed a commit to mkarmark/azure-rest-api-specs that referenced this pull request Jul 21, 2021
    * Convert SD download operation to async operation
    
    * Remove Security Domain structure definition
    
    * Minor update
    
    * Update 7.3-preview spec
    
    * Fix validation errors
    
    * Update specification/keyvault/data-plane/Microsoft.KeyVault/preview/7.2-preview/securitydomain.json
    
    Co-authored-by: Heath Stewart <[email protected]>
    
    * Address review feedback
    
    Co-authored-by: Heath Stewart <[email protected]>
    
    * Fix prettier check failures
    
    * Fix suppression statement in readme.md
    
    * Fix typo
    
    * Another prettier fix
    
    Co-authored-by: Heath Stewart <[email protected]>
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    5 participants