Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
…into regenerate_keys

* 'master' of https://github.com/Azure/azure-sdk-for-python:
  Set http_logging_policy in Configuration (Azure#12218)
  Sync eng/common directory with azure-sdk-tools repository (Azure#11990)
  AzureCliCredential instructs CLI not to color output (Azure#11362)
  Sdk automation/track2 azure mgmt storage (Azure#12238)
  Fix changelog of CustomVision (Azure#12225)
  Doc update for conf file name (Azure#12224)
  update doc for content_type (Azure#12220)
  update API version to use 2020-06-30 (Azure#12208)
  Use MSAL's custom transport API (Azure#11892)
  add breadcumbs for training filter (Azure#12196)
  [formrecognizer] arch board feedback renames (Azure#12207)
  Dataplane autogeneration (Azure#12210)
  update version number and API_version support (Azure#12154)
  Update SECURITY.md (Azure#12209)
  adding a pip freeze to ensure we fully understand what our environment has (Azure#12173)
  FaceAPI 0.4.1 (Azure#12199)
  [ServiceBus] Enable Async Unittest (Azure#12001)
  add search owver (Azure#12190)
  update training docstring (Azure#12168)
  • Loading branch information
iscai-msft committed Jun 30, 2020
2 parents c3a2d16 + de7168a commit 6308a6a
Show file tree
Hide file tree
Showing 399 changed files with 5,299 additions and 5,160 deletions.
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
/sdk/hdinsight/ @idear1203
/sdk/machinelearningcompute/ @shutchings
/sdk/recoveryservices/ @DheerendraRathor
/sdk/search/ @xiangyan99
/sdk/servicefabric/ @QingChenmsft @samedder
/sql/sql/ @jaredmoo
/sdk/servicebus/ @annatisch @yunhaoling @YijunXieMS @KieranBrantnerMagee
Expand Down
41 changes: 37 additions & 4 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,41 @@
# Security Policy
<!-- BEGIN MICROSOFT SECURITY.MD V0.0.5 BLOCK -->

## Reporting a Vulnerability
## Security

Security issues and bugs should be reported privately, via email, to the Microsoft Security Response Center (MSRC) <[email protected]>. You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Further information, including the MSRC PGP key, can be found in the [Security TechCenter](https://www.microsoft.com/msrc/faqs-report-an-issue).
Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/).

Please do not open issues for anything you think might have a security implication.
If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://docs.microsoft.com/en-us/previous-versions/tn-archive/cc751383(v=technet.10)), please report it to us as described below.

## Reporting Security Issues

**Please do not report security vulnerabilities through public GitHub issues.**

Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://msrc.microsoft.com/create-report).

If you prefer to submit without logging in, send email to [[email protected]](mailto:[email protected]). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://www.microsoft.com/en-us/msrc/pgp-key-msrc).

You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://www.microsoft.com/msrc).

Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue:

* Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.)
* Full paths of source file(s) related to the manifestation of the issue
* The location of the affected source code (tag/branch/commit or direct URL)
* Any special configuration required to reproduce the issue
* Step-by-step instructions to reproduce the issue
* Proof-of-concept or exploit code (if possible)
* Impact of the issue, including how an attacker might exploit the issue

This information will help us triage your report more quickly.

If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://microsoft.com/msrc/bounty) page for more details about our active programs.

## Preferred Languages

We prefer all communications to be in English.

## Policy

Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://www.microsoft.com/en-us/msrc/cvd).

<!-- END MICROSOFT SECURITY.MD BLOCK -->
16 changes: 16 additions & 0 deletions doc/dev/mgmt/mgmt_release.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,22 @@ Once you have a PR that contains accurate with correct tests (or no tests at all

IMPORTANT NOTE: All the commands in this page assumes you have loaded the [dev_setup](../dev_setup.md) in your currently loaded virtual environment.

## Manual generation

If the automation is not doing its job to create an auto PR, Python has a SwaggerToSdk CLI that can be used to generate a specific Readme. You need
a virtual environment loaded with at least `tools/azure-sdk-tools` installed.

```shell
# Using default configuration (this can be a Github raw link)
generate_sdk -v -m ..\azure-rest-api-specs\specification\compute\resource-manager\readme.md

# Forcing Track1 generation
generate_sdk -v -c eng\swagger_to_sdk_config_v4.json -m ..\azure-rest-api-specs\specification\cognitiveservices\data-plane\Face\readme.md

# For more details about the available options
generate_sdk --help
```

## Building the packaging information

If the automation is doing its job correctly, there is a pipeline called "update PR" that is supposed to update the package on the branch.
Expand Down
17 changes: 17 additions & 0 deletions eng/common/TestResources/New-TestResources.ps1.md
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,23 @@ Accept pipeline input: False
Accept wildcard characters: False
```
### -OutFile
save test environment settings into a test-resources.json.env file next to test-resources.json.
The file is protected via DPAPI. The environment file would be scoped to the current repository directory.
Note: Supported only on Windows.
```yaml
Type: SwitchParameter
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False
```
### CommonParameters
This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable, -InformationAction, -InformationVariable, -OutVariable, -OutBuffer, -PipelineVariable, -Verbose, -WarningAction, and -WarningVariable. For more information, see [about_CommonParameters](http://go.microsoft.com/fwlink/?LinkID=113216).
Expand Down
6 changes: 6 additions & 0 deletions eng/common/TestResources/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,12 @@ eng\common\TestResources\New-TestResources.ps1 `
-TestApplicationSecret (ConvertFrom-SecureString $sp.Secret -AsPlainText)
```

If you are running this for a .NET project on Windows, the recommended method is to
add the `-OutFile` switch to the above command. This will save test environment settings
into a test-resources.json.env file next to test-resources.json. The file is protected via DPAPI.
The environment file would be scoped to the current repository directory and avoids the need to
set environment variables or restart your IDE to recognize them.

Along with some log messages, this will output environment variables based on
your current shell like in the following example:

Expand Down
44 changes: 24 additions & 20 deletions eng/common/pipelines/templates/steps/create-pull-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,27 +35,31 @@ steps:
workingDirectory: ${{ parameters.WorkingDirectory }}
ignoreLASTEXITCODE: true

- pwsh: |
${{ parameters.ScriptDirectory }}/git-branch-push.ps1 `
-PRBranchName "${{ parameters.PRBranchName }}" `
-CommitMsg "${{ parameters.CommitMsg }}" `
-GitUrl "https://$(azuresdk-github-pat)@github.com/${{ parameters.PROwner }}/${{ parameters.RepoName }}.git" `
-PushArgs "${{ parameters.PushArgs }}"
- task: PowerShell@2
displayName: Push changes
workingDirectory: ${{ parameters.WorkingDirectory }}
condition: and(succeeded(), eq(variables['HasChanges'], 'true'))
inputs:
pwsh: true
workingDirectory: ${{ parameters.WorkingDirectory }}
filePath: ${{ parameters.ScriptDirectory }}/git-branch-push.ps1
arguments: >
-PRBranchName "${{ parameters.PRBranchName }}"
-CommitMsg "${{ parameters.CommitMsg }}"
-GitUrl "https://$(azuresdk-github-pat)@github.com/${{ parameters.PROwner }}/${{ parameters.RepoName }}.git"
-PushArgs "${{ parameters.PushArgs }}"
- pwsh: |
${{ parameters.ScriptDirectory }}/Submit-PullRequest.ps1 `
-RepoOwner "${{ parameters.RepoOwner }}" `
-RepoName "${{ parameters.RepoName }}" `
-BaseBranch "${{ parameters.BaseBranchName }}" `
-PROwner "${{ parameters.PROwner }}" `
-PRBranch "${{ parameters.PRBranchName }}" `
-AuthToken "$(azuresdk-github-pat)" `
-PRTitle "${{ parameters.PRTitle }}"
- task: PowerShell@2
displayName: Create pull request
workingDirectory: ${{ parameters.WorkingDirectory }}
condition: and(succeeded(), eq(variables['HasChanges'], 'true'))
condition: and(succeeded(), eq(variables['HasChanges'], 'true'))
inputs:
pwsh: true
workingDirectory: ${{ parameters.WorkingDirectory }}
filePath: ${{ parameters.ScriptDirectory }}/Submit-PullRequest.ps1
arguments: >
-RepoOwner "${{ parameters.RepoOwner }}"
-RepoName "${{ parameters.RepoName }}"
-BaseBranch "${{ parameters.BaseBranchName }}"
-PROwner "${{ parameters.PROwner }}"
-PRBranch "${{ parameters.PRBranchName }}"
-AuthToken "$(azuresdk-github-pat)"
-PRTitle "${{ parameters.PRTitle }}"
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,13 @@ steps:
- task: PowerShell@2
displayName: 'Verify Package Tags and Create Git Releases'
inputs:
targetType: filePath
filePath: ${{ parameters.ScriptDirectory }}/create-tags-and-git-release.ps1
arguments: -artifactLocation ${{parameters.ArtifactLocation}} -packageRepository ${{parameters.PackageRepository}} -releaseSha ${{parameters.ReleaseSha}} -repoId ${{parameters.RepoId}} -workingDirectory '${{parameters.WorkingDirectory}}'
arguments: >
-artifactLocation ${{ parameters.ArtifactLocation }}
-packageRepository ${{ parameters.PackageRepository }}
-releaseSha ${{ parameters.ReleaseSha }}
-repoId ${{ parameters.RepoId }}
-workingDirectory '${{ parameters.WorkingDirectory }}'
pwsh: true
timeoutInMinutes: 5
env:
Expand Down
24 changes: 12 additions & 12 deletions eng/common/pipelines/templates/steps/docs-metadata-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ parameters:
steps:
- pwsh: |
git clone https://github.com/${{ parameters.TargetDocRepoOwner }}/${{ parameters.TargetDocRepoName }} ${{ parameters.WorkingDirectory }}/repo
try {
Push-Location ${{ parameters.WorkingDirectory }}/repo
Write-Host "git checkout smoke-test"
git checkout smoke-test
} finally {
Expand All @@ -33,14 +33,14 @@ steps:
inputs:
targetType: filePath
filePath: ${{ parameters.ScriptDirectory }}/update-docs-metadata.ps1
arguments: >
-ArtifactLocation ${{parameters.ArtifactLocation}}
-Repository ${{parameters.PackageRepository}}
-ReleaseSHA ${{parameters.ReleaseSha}}
-RepoId ${{parameters.RepoId}}
-WorkDirectory '${{parameters.WorkingDirectory}}'
-DocRepoLocation "${{parameters.WorkingDirectory}}/repo"
-Language "${{parameters.Language}}"
arguments: >
-ArtifactLocation ${{ parameters.ArtifactLocation }}
-Repository ${{ parameters.PackageRepository }}
-ReleaseSHA ${{ parameters.ReleaseSha }}
-RepoId ${{ parameters.RepoId }}
-WorkDirectory "${{ parameters.WorkingDirectory }}"
-DocRepoLocation "${{ parameters.WorkingDirectory }}/repo"
-Language "${{parameters.Language}}"
-DocRepoContentLocation ${{ parameters.DocRepoDestinationPath }}
pwsh: true
env:
Expand All @@ -54,5 +54,5 @@ steps:
CommitMsg: "Update readme content for ${{ parameters.ArtifactName }}"
PRTitle: "Docs.MS Readme Update."
BaseBranchName: smoke-test
WorkingDirectory: ${{parameters.WorkingDirectory}}/repo
ScriptDirectory: ${{parameters.WorkingDirectory}}/${{parameters.ScriptDirectory}}
WorkingDirectory: ${{ parameters.WorkingDirectory }}/repo
ScriptDirectory: ${{ parameters.WorkingDirectory }}/${{ parameters.ScriptDirectory }}
12 changes: 8 additions & 4 deletions eng/common/pipelines/templates/steps/publish-blobs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,20 @@ parameters:

steps:
- pwsh: |
Invoke-WebRequest -MaximumRetryCount 10 -Uri "https://aka.ms/downloadazcopy-v10-windows" `
-OutFile "azcopy.zip" | Wait-Process; Expand-Archive -Path "azcopy.zip" -DestinationPath "$(Build.BinariesDirectory)/azcopy/"
Invoke-WebRequest -MaximumRetryCount 10 -Uri "https://aka.ms/downloadazcopy-v10-windows" -OutFile "azcopy.zip" | Wait-Process;
Expand-Archive -Path "azcopy.zip" -DestinationPath "$(Build.BinariesDirectory)/azcopy/"
workingDirectory: $(Build.BinariesDirectory)
displayName: Download and Extract azcopy Zip

- task: Powershell@2
inputs:
targetType: 'filePath'
filePath: ${{ parameters.ScriptPath }}
arguments: -AzCopy $(Resolve-Path "$(Build.BinariesDirectory)/azcopy/azcopy_windows_amd64_*/azcopy.exe")[0] -DocLocation "${{ parameters.FolderForUpload }}" -SASKey "${{ parameters.BlobSASKey }}" -Language "${{ parameters.TargetLanguage }}" -BlobName "${{ parameters.BlobName }}"
arguments: >
-AzCopy $(Resolve-Path "$(Build.BinariesDirectory)/azcopy/azcopy_windows_amd64_*/azcopy.exe")[0]
-DocLocation "${{ parameters.FolderForUpload }}"
-SASKey "${{ parameters.BlobSASKey }}"
-Language "${{ parameters.TargetLanguage }}"
-BlobName "${{ parameters.BlobName }}"
pwsh: true
workingDirectory: $(Pipeline.Workspace)
displayName: Copy Docs to Blob
Expand Down
1 change: 0 additions & 1 deletion eng/common/pipelines/templates/steps/verify-agent-os.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
# Template for all Python Scripts in this repository
parameters:
OSVmImage: $(OSVmImage)

Expand Down
25 changes: 25 additions & 0 deletions eng/common/pipelines/templates/steps/verify-changelog.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
parameters:
- name: PackageName
type: string
default: 'not-specified'
- name: ServiceName
type: string
default: 'not-specified'
- name: ForRelease
type: boolean
default: false

steps:
- task: Powershell@2
inputs:
filePath: /eng/common/scripts/Verify-ChangeLog.ps1
arguments: >
-PackageName ${{ parameters.PackageName }}
-ServiceName ${{ parameters.ServiceName }}
-RepoRoot $(Build.SourcesDirectory)
-RepoName $(Build.Repository.Name)
-ForRelease ${{ parameters.ForRelease }}
pwsh: true
workingDirectory: $(Pipeline.Workspace)
displayName: Verify ChangeLog / Release Notes
continueOnError: false
12 changes: 12 additions & 0 deletions eng/common/pipelines/templates/steps/verify-links.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
parameters:
Directory: 'not-specified'

steps:
- task: PowerShell@2
displayName: Link verification check
inputs:
pwsh: true
workingDirectory: $(Build.SourcesDirectory)/${{ parameters.Directory }}
filePath: eng/common/scripts/Verify-Links.ps1
arguments: >
-urls $(dir -r -i *.md) -rootUrl "file://$(Build.SourcesDirectory)/${{ parameters.Directory }}"
29 changes: 20 additions & 9 deletions eng/common/scripts/Submit-PullRequest.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -41,21 +41,25 @@ param(
$PRBody = $PRTitle
)

Write-Host "> $PSCommandPath $args"
$headers = @{
Authorization = "bearer $AuthToken"
}

$query = "state=open&head=${PROwner}:${PRBranch}&base=${BaseBranch}"

$resp = Invoke-RestMethod "https://api.github.com/repos/$RepoOwner/$RepoName/pulls?$query"
try {
$resp = Invoke-RestMethod -Headers $headers "https://api.github.com/repos/$RepoOwner/$RepoName/pulls?$query"
}
catch {
Write-Error "Invoke-RestMethod [https://api.github.com/repos/$RepoOwner/$RepoName/pulls?$query] failed with exception:`n$_"
exit 1
}
$resp | Write-Verbose

if ($resp.Count -gt 0) {
Write-Host -f green "Pull request already exists $($resp[0].html_url)"
}
else {
$headers = @{
Authorization = "bearer $AuthToken"
}

$data = @{
title = $PRTitle
head = "${PROwner}:${PRBranch}"
Expand All @@ -64,9 +68,16 @@ else {
maintainer_can_modify = $true
}

$resp = Invoke-RestMethod -Method POST -Headers $headers `
https://api.github.com/repos/$RepoOwner/$RepoName/pulls `
-Body ($data | ConvertTo-Json)
try {
$resp = Invoke-RestMethod -Method POST -Headers $headers `
"https://api.github.com/repos/$RepoOwner/$RepoName/pulls" `
-Body ($data | ConvertTo-Json)
}
catch {
Write-Error "Invoke-RestMethod [https://api.github.com/repos/$RepoOwner/$RepoName/pulls] failed with exception:`n$_"
exit 1
}

$resp | Write-Verbose
Write-Host -f green "Pull request created https://github.com/$RepoOwner/$RepoName/pull/$($resp.number)"
}
22 changes: 22 additions & 0 deletions eng/common/scripts/Verify-ChangeLog.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Wrapper Script for ChangeLog Verification
param (
[Parameter(Mandatory=$true)]
[string]$PackageName,
[Parameter(Mandatory=$true)]
[string]$ServiceName,
[string]$RepoRoot,
[ValidateSet("net","java","js","python")]
[string]$Language,
[string]$RepoName,
[boolean]$ForRelease=$False
)

Import-Module "${PSScriptRoot}/modules/common-manifest.psd1"

if ([System.String]::IsNullOrEmpty($Language))
{
$Language = $RepoName.Substring($RepoName.LastIndexOf('-') + 1)
}

$PackageProp = Get-PkgProperties -PackageName $PackageName -ServiceName $ServiceName -Language $Language -RepoRoot $RepoRoot
Confirm-ChangeLog -ChangeLogLocation $PackageProp.pkgChangeLogPath -VersionString $PackageProp.pkgReadMePath -ForRelease $ForRelease
Loading

0 comments on commit 6308a6a

Please sign in to comment.