Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix schema generation failure for securityinsights #10038

Merged
merged 1 commit into from
Jul 8, 2020
Merged

fix schema generation failure for securityinsights #10038

merged 1 commit into from
Jul 8, 2020

Conversation

leni-msft
Copy link
Contributor

MSFT employees can try out our new experience at OpenAPI Hub - one location for using our validation tools and finding your workflow.

Contribution checklist:

If any further question about AME onboarding or validation tools, please view the FAQ.

ARM API Review Checklist

  • Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
    • adding/removing APIs.
    • adding/removing properties.
    • adding/removing API-version.
    • adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

  • If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
    Please follow the link to find more details on API review process.

@openapi-pipeline-app
Copy link

openapi-pipeline-app bot commented Jul 6, 2020 •
edited
Loading

[Staging] Swagger Validation Report

️❌BreakingChange: 2 Errors, 0 Warnings [Detail] [Expand]
Rule Message
❌ 1032 - DifferentAllOf The new version has a different 'allOf' property than the previous one.
New: Microsoft.SecurityInsights/stable/2020-01-01/SecurityInsights.json#L1704:9
Old: Microsoft.SecurityInsights/stable/2020-01-01/SecurityInsights.json#L1704:9
❌ 1034 - AddedRequiredProperty The new version has new required property 'state' that was not found in the old version.
New: Microsoft.SecurityInsights/stable/2020-01-01/SecurityInsights.json#L1928:7
️️✔️LintDiff [Detail] 
 Validation passes for LintDiff.
️️✔️Avocado [Detail] 
 Validation passes for Avocado.
Posted by Swagger Pipeline | How to fix these errors?

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@openapi-sdkautomation
Copy link

openapi-sdkautomation bot commented Jul 6, 2020 •
edited
Loading

azure-sdk-for-js - Release

⚠️ warning [Logs] [Expand Details]
  • ⚠️ Generate from 4d16c35 with merge commit e60f1b4. SDK Automation 13.0.17.20200619.4 
    Failed to find any diff after autorest so no changed packages was found.

@openapi-sdkautomation
Copy link

openapi-sdkautomation bot commented Jul 6, 2020 •
edited
Loading

Azure CLI Extension Generation - Release

  • Installation instruction for service securityinsight [Preview Changes]
    az extension add --source=https://sdkautomation.blob.core.windows.net/sdkautomation-pipeline/Azure/azure-rest-api-specs/10038/azure-cli-extensions/Azure/azure-cli-extensions/securityinsight/sentinel-0.1.0-py3-none-any.whl
️✔️ succeeded [Logs] [Expand Details]
  • ️✔️ Generate from 4d16c35 with merge commit e60f1b4. Azure CLI Automation 13.0.17.20200619.4 
    [AutoRest] WARNING (PreCheck/AllOfWhenYouMeantRef): Schema 'AwsCloudTrailDataConnectorDataTypes-logs' is using an 'allOf' instead of a $ref. This creates a wasteful anonymous type when generating code.
[AutoRest] WARNING (PreCheck/AllOfWhenYouMeantRef): Schema 'OfficeDataConnectorDataTypes-exchange' is using an 'allOf' instead of a $ref. This creates a wasteful anonymous type when generating code.
[AutoRest] WARNING (PreCheck/AllOfWhenYouMeantRef): Schema 'OfficeDataConnectorDataTypes-sharePoint' is using an 'allOf' instead of a $ref. This creates a wasteful anonymous type when generating code.
[AutoRest] WARNING (PreCheck/AllOfWhenYouMeantRef): Schema 'TIDataConnectorDataTypes-indicators' is using an 'allOf' instead of a $ref. This creates a wasteful anonymous type when generating code.
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'ActionsList' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'AlertRuleTemplatesList' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'AlertRulesList' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'BookmarkList' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'CloudError' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'DataConnectorList' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'IncidentCommentList' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'IncidentList' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'OfficeConsentList' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'Operation' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'OperationsList' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'Resource' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'ResourceWithEtag' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'ErrorResponse' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/SchemaMissingType): The schema 'ErrorAdditionalInfo' with an undefined type and decalared properties is a bit ambigious. This has been auto-corrected to 'type:object'
[AutoRest] WARNING (PreCheck/PropertyRedeclarationWarning): Schema 'MCASDataConnectorDataTypes' has a property 'alerts' that is already declared the parent schema 'AlertsDataTypeOfDataConnector' but isn't significantly different. The property has been removed from MCASDataConnectorDataTypes
[AutoRest] WARNING (): cli.auto-parameter-hidden is not true, skip visibility cleaner
[AutoRest] 
[AutoRest] NO TEST SCENARIO PROVIDED - DEFAULT WILL BE USED
[AutoRest] ADD FOLLOWING SECTION TO readme.cli.md FILE TO MODIFY IT
[AutoRest] --------------------------------------------------------
[AutoRest]   test-scenario:
[AutoRest]     - name: /Actions/get/Get all actions of alert rule.
[AutoRest]     - name: /AlertRuleTemplates/get/Get alert rule template by Id.
[AutoRest]     - name: /AlertRuleTemplates/get/Get all alert rule templates.
[AutoRest]     - name: /AlertRules/put/Creates or updates a Fusion alert rule.
[AutoRest]     - name: /AlertRules/put/Creates or updates a MicrosoftSecurityIncidentCreation rule.
[AutoRest]     - name: /AlertRules/put/Creates or updates a Scheduled alert rule.
[AutoRest]     - name: /AlertRules/put/Creates or updates an action of alert rule.
[AutoRest]     - name: /AlertRules/get/Get a Fusion alert rule.
[AutoRest]     - name: /AlertRules/get/Get a MicrosoftSecurityIncidentCreation rule.
[AutoRest]     - name: /AlertRules/get/Get a Scheduled alert rule.
[AutoRest]     - name: /AlertRules/get/Get all alert rules.
[AutoRest]     - name: /AlertRules/get/Get an action of alert rule.
[AutoRest]     - name: /Bookmarks/put/Creates or updates a bookmark.
[AutoRest]     - name: /Bookmarks/get/Get a bookmark.
[AutoRest]     - name: /Bookmarks/get/Get all bookmarks.
[AutoRest]     - name: /DataConnectors/put/Creates or updates an Office365 data connector.
[AutoRest]     - name: /DataConnectors/get/Get a ASC data connector.
[AutoRest]     - name: /DataConnectors/get/Get a MCAS data connector.
[AutoRest]     - name: /DataConnectors/get/Get a MDATP data connector
[AutoRest]     - name: /DataConnectors/get/Get a TI data connector.
[AutoRest]     - name: /DataConnectors/get/Get all data connectors.
[AutoRest]     - name: /DataConnectors/get/Get an AAD data connector.
[AutoRest]     - name: /DataConnectors/get/Get an AATP data connector.
[AutoRest]     - name: /DataConnectors/get/Get an AwsCloudTrail data connector.
[AutoRest]     - name: /DataConnectors/get/Get an Office365 data connector.
[AutoRest]     - name: /IncidentComments/put/Creates an incident comment.
[AutoRest]     - name: /IncidentComments/get/Get all incident comments.
[AutoRest]     - name: /IncidentComments/get/Get an incident comment.
[AutoRest]     - name: /Incidents/put/Creates or updates an incident.
[AutoRest]     - name: /Incidents/get/Get all incidents.
[AutoRest]     - name: /Incidents/get/Get an incident.
[AutoRest]     - name: /AlertRules/delete/Delete an alert rule.
[AutoRest]     - name: /AlertRules/delete/Delete an action of alert rule.
[AutoRest]     - name: /Bookmarks/delete/Delete a bookmark.
[AutoRest]     - name: /DataConnectors/delete/Delete an Office365 data connector.
[AutoRest]     - name: /Incidents/delete/Delete an incident.
[AutoRest] --------------------------------------------------------
  • ️✔️securityinsight [View full logs]  [Release Azure CLI Changes]

@openapi-sdkautomation
Copy link

openapi-sdkautomation bot commented Jul 6, 2020 •
edited
Loading

azure-sdk-for-java - Release

No readme.md specification configuration files were found that are associated with the files modified in this pull request, or swagger_to_sdk section in readme.md is not configured

@openapi-sdkautomation
Copy link

openapi-sdkautomation bot commented Jul 6, 2020 •
edited
Loading

azure-sdk-for-python - Release

⚠️ warning [Logs] [Expand Details]
  • ⚠️ Generate from 4d16c35 with merge commit e60f1b4. SDK Automation 13.0.17.20200619.4 
    [azure-mgmt-securityinsight][ChangeLog] WARNING: Failed to generate ChangeLog.
  • ⚠️azure-mgmt-securityinsight [View full logs]  [Release SDK Changes] 
    [build_conf] INFO:packaging_tools:Building template azure-mgmt-securityinsight
[build_conf] INFO:packaging_tools.conf:Build default conf for azure-mgmt-securityinsight
[build_conf] INFO:packaging_tools:Template done azure-mgmt-securityinsight
[build_package] /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'
[build_package]   warnings.warn(msg)
[build_package] warning: no files found matching '*.py' under directory 'tests'
[build_package] warning: no files found matching '*.yaml' under directory 'tests'
[build_package] /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'
[build_package]   warnings.warn(msg)
[build_package] warning: no files found matching '*.py' under directory 'tests'
[build_package] warning: no files found matching '*.yaml' under directory 'tests'
[breaking_change_setup] Ignoring mock: markers 'python_version <= "2.7"' don't match your environment
Not exact 2 reports found:
Not generating changelog.
[ChangeLog] WARNING: Failed to generate ChangeLog.

@openapi-sdkautomation
Copy link

openapi-sdkautomation bot commented Jul 6, 2020 •
edited
Loading

Trenton Generation - Release

No readme.md specification configuration files were found that are associated with the files modified in this pull request, or swagger_to_sdk section in readme.md is not configured

@openapi-sdkautomation
Copy link

openapi-sdkautomation bot commented Jul 6, 2020 •
edited
Loading

azure-sdk-for-go - Release

️✔️ succeeded [Logs] [Expand Details]

@openapi-sdkautomation
Copy link

openapi-sdkautomation bot commented Jul 6, 2020 •
edited
Loading

azure-sdk-for-net - Release

️✔️ succeeded [Logs] [Expand Details]
  • ️✔️ Generate from 4d16c35 with merge commit e60f1b4. SDK Automation 13.0.17.20200619.4 
    [AutoRest] realpath(): Permission denied
[AutoRest] realpath(): Permission denied
[AutoRest] realpath(): Permission denied
[AutoRest] realpath(): Permission denied
[AutoRest] realpath(): Permission denied
[AutoRest] realpath(): Permission denied
  • ️✔️Microsoft.Azure.Management.SecurityInsights [View full logs]  [Release SDK Changes]
      No Artifact Generated.

    This was referenced Jul 6, 2020
    Closed
    Closed
    Closed
    @leni-msft
    Copy link
    Contributor Author

    Fixes schema generation failure:

    2020-07-05T11:38:44.1174839Z [/home/vsts/work/1/s/generator] executing: autorest-beta --version=3.0.6274 --use=@autorest/[email protected] --azureresourceschema --output-folder=/tmp/8ciywltrstk --tag=all-api-versions --api-version=2020-01-01 --title=none --pass-thru:subset-reducer /tmp/schm_azspc/specification/securityinsights/resource-manager/readme.md
2020-07-05T11:38:44.4805090Z AutoRest code generation utility [cli version: 3.0.6187; node: v12.18.1, max-memory: 8192 gb]
2020-07-05T11:38:44.4815930Z (C) 2018 Microsoft Corporation.
2020-07-05T11:38:44.4816835Z https://aka.ms/autorest
2020-07-05T11:38:45.2966932Z    Loading AutoRest core      '/home/vsts/.autorest/@[email protected]/node_modules/@autorest/core/dist' (3.0.6274)
2020-07-05T11:38:46.3235067Z    Loading AutoRest extension '@autorest/azureresourceschema' (3.0.79->3.0.79)
2020-07-05T11:38:53.4391585Z FATAL: System.InvalidOperationException: Found incompatible property types ,  for property 'alerts' in schema inheritance chain 
2020-07-05T11:38:53.4393943Z    at AutoRest.Modeler.SchemaResolver.ExpandAllOf(Schema schema) in /home/vsts/work/1/s/autorest.common/src/SchemaResolver.cs:line 158
2020-07-05T11:38:53.4395005Z    at AutoRest.Modeler.SchemaResolver.Unwrap(Schema schema) in /home/vsts/work/1/s/autorest.common/src/SchemaResolver.cs:line 76
2020-07-05T11:38:53.4396072Z    at AutoRest.Modeler.SchemaBuilder.BuildServiceType(String serviceTypeName, Boolean required) in /home/vsts/work/1/s/autorest.common/src/SchemaBuilder.cs:line 122
2020-07-05T11:38:53.4397306Z    at AutoRest.Modeler.SchemaBuilder.BuildServiceType(String serviceTypeName, Boolean required) in /home/vsts/work/1/s/autorest.common/src/SchemaBuilder.cs:line 138
2020-07-05T11:38:53.4398419Z    at AutoRest.Modeler.SwaggerModeler.BuildCompositeTypes() in /home/vsts/work/1/s/autorest.common/src/SwaggerModeler.cs:line 349
2020-07-05T11:38:53.4399475Z    at AutoRest.Modeler.SwaggerModeler.Build(ServiceDefinition serviceDefinition) in /home/vsts/work/1/s/autorest.common/src/SwaggerModeler.cs:line 66
2020-07-05T11:38:53.4400522Z    at AutoRest.Modeler.ModelerPlugin.<ProcessInternal>d__1.MoveNext() in /home/vsts/work/1/s/autorest.common/src/Program.cs:line 43
2020-07-05T11:38:53.4401788Z --- End of stack trace from previous location where exception was thrown ---
2020-07-05T11:38:53.4402558Z    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2020-07-05T11:38:53.4403354Z    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2020-07-05T11:38:53.4404139Z    at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
2020-07-05T11:38:53.4405303Z    at NewPlugin.<Process>d__20.MoveNext() in /home/vsts/work/1/s/autorest.common/src/common/Plugins/NewPlugin.cs:line 164
2020-07-05T11:38:53.4433931Z   Error: Plugin imodeler2 reported failure.
2020-07-05T11:38:53.4804191Z Caught exception processing autogenlist entry securityinsights/resource-manager.
2020-07-05T11:38:53.4805516Z Exited with code 1

    @weidongxu-microsoft
    Copy link
    Member

    Please add code owner to reviewer.

    @azuresdkci
    Copy link
    Contributor

    Can one of the admins verify this patch?

    @jorgecotillo
    Copy link
    Contributor

    @alshulma can you please review this PR? the schema generation failed due to "alerts" property having different schemas: https://dev.azure.com/azure/azure-resource-manager-schemas/_build/results?buildId=11699&view=logs&j=ae501f3d-63ca-563e-a3d9-7c46870865a7&t=5b372c7a-4bf1-530e-6011-652a8f555b0d Line 1122.

    @alshulma
    Copy link
    Contributor

    alshulma commented Jul 6, 2020 via email

    @jorgecotillo jorgecotillo mentioned this pull request Jul 7, 2020
    Merged
    @weidongxu-microsoft
    Copy link
    Member

    weidongxu-microsoft commented Jul 7, 2020 •
    edited
    Loading

    Follow the guideline... (however this is a fix on previously incorrect swagger)

    As identified in the automated checks there are breaking changes, Please follow Azure Breaking changes approval request form as defined @ Breaking Change Process

    @leni-msft
    Copy link
    Contributor Author

    @weidongxu-microsoft I confirmed with breakingchange tool owner. The tool doesn't support this case yet. It's not breaking change, since the schema doesn't change.

    @weidongxu-microsoft
    Copy link
    Member

    @leni-msft Sure, you can let him do the merge after code owner is OK with it.

    The problem with that BreakingChange CI is that if it fails, I won't be able to press the merge button at all...

    @ofshomro
    Copy link
    Contributor

    ofshomro commented Jul 7, 2020

    code owner approves changes

    @leni-msft leni-msft added Approved-BreakingChange DO NOT USE! OBSOLETE label. See https://github.com/Azure/azure-sdk-tools/issues/6374 and removed BreakingChangeReviewRequired labels Jul 8, 2020
    @akning-ms akning-ms merged commit e60f1b4 into Azure:master Jul 8, 2020
    @openapi-sdkautomation
    Copy link

    openapi-sdkautomation bot commented Jul 8, 2020 •
    edited
    Loading

    azure-sdk-for-python-track2 - Release

    No readme.md specification configuration files were found that are associated with the files modified in this pull request, or swagger_to_sdk section in readme.md is not configured

    00Kai0 pushed a commit to 00Kai0/azure-rest-api-specs that referenced this pull request Oct 12, 2020
    @leni-msft @00Kai0
    fix schema generation failure for securityinsights (Azure#10038)
    1015426
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Reviewers

    @weidongxu-microsoft weidongxu-microsoft weidongxu-microsoft approved these changes

    Assignees

    @weidongxu-microsoft weidongxu-microsoft

    Labels
    Approved-BreakingChange DO NOT USE! OBSOLETE label. See https://github.com/Azure/azure-sdk-tools/issues/6374
    Projects
    None yet
    Milestone
    No milestone
    Development

    Successfully merging this pull request may close these issues.
    7 participants
    @leni-msft @weidongxu-microsoft @azuresdkci @jorgecotillo @alshulma @ofshomro @akning-ms