Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI Example Analyzer] Support analysis for any PowerShell script #19191

Merged
merged 4 commits into from
Aug 12, 2022
Merged

[CI Example Analyzer] Support analysis for any PowerShell script #19191

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
066ee9e
support analyze any powershell script
MoChilia Aug 10, 2022
5c46cd9
fix a bug
MoChilia Aug 11, 2022
c2280ce
update
MoChilia Aug 11, 2022
e56d778
change clean script to not clean
MoChilia Aug 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 27 additions & 25 deletions tools/StaticAnalysis/ExampleAnalyzer/Measure-MarkdownOrScript.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@
.PARAMETER MarkdownPaths
Markdown searching paths. Empty for current path. Supports wildcard.
.PARAMETER ScriptPath
PowerShell script searching path.
PowerShell script searching paths. Empty for current path. Supports wildcard.
.PARAMETER RulePaths
PSScriptAnalyzer custom rules paths. Empty for current path. Supports wildcard.
.PARAMETER CodeMapPath
Code map path bound with the PowerShell script.
.PARAMETER Recurse
.PARAMETER MarkdownRecurse
To search markdowns recursively in the folders.
.PARAMETER ScriptRecurse
To search scripts recursively in the folders.
.PARAMETER IncludeDefaultRules
To analyze default rules provided by PSScriptAnalyzer.
.PARAMETER OutputFolder
Expand All @@ -32,12 +32,12 @@ param (
[string[]]$MarkdownPaths,
[Parameter(Mandatory, ParameterSetName = "Script")]
[AllowEmptyString()]
[string[]]$ScriptPath,
[string[]]$ScriptPaths,
[string[]]$RulePaths,
[Parameter(Mandatory, ParameterSetName = "Script")]
[string]$CodeMapPath,
[Parameter(ParameterSetName = "Markdown")]
[switch]$Recurse,
[switch]$MarkdownRecurse,
[Parameter(ParameterSetName = "Script")]
[switch]$ScriptRecurse,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use one single -Recurse parameter for both parameter sets?

[switch]$IncludeDefaultRules,
[string]$OutputFolder = "$PSScriptRoot\..\..\..\artifacts\StaticAnalysisResults\ExampleAnalysis",
[Parameter(ParameterSetName = "Markdown")]
Expand All @@ -51,24 +51,26 @@ $analysisResultsTable = @()
$codeMap = @()
$totalLine = 1

# Clean caches, remove files in "output" folder
Remove-Item $OutputFolder\TempScript.ps1 -ErrorAction SilentlyContinue
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please introduce a constant for "TempScript.ps1" and "TempCodeMap.csv"
It's quite possible that we may need to rename them

Remove-Item $OutputFolder\TempCodeMap.csv -ErrorAction SilentlyContinue
Remove-Item $PSScriptRoot\..\..\..\artifacts\StaticAnalysisResults\ExampleIssues.csv -ErrorAction SilentlyContinue
Remove-Item $OutputFolder -ErrorAction SilentlyContinue
# Create output folder and temp script
$null = New-Item -ItemType Directory -Path $OutputFolder -ErrorAction SilentlyContinue
$null = New-Item -ItemType File $OutputFolder\TempScript.ps1

# Find examples in "help\*.md", output ".ps1"
if ($PSCmdlet.ParameterSetName -eq "Markdown") {
# Clean caches, remove files in "output" folder
Remove-Item $OutputFolder\TempScript.ps1 -ErrorAction SilentlyContinue
Remove-Item $OutputFolder\TempCodeMap.csv -ErrorAction SilentlyContinue
Remove-Item $PSScriptRoot\..\..\..\artifacts\StaticAnalysisResults\ExampleIssues.csv -ErrorAction SilentlyContinue
Remove-Item $OutputFolder -ErrorAction SilentlyContinue
$null = New-Item -ItemType Directory -Path $OutputFolder -ErrorAction SilentlyContinue
$null = New-Item -ItemType File $OutputFolder\TempScript.ps1
# When the input $MarkdownPaths is the path of txt file
# When the input $MarkdownPaths is the path of txt file contained markdown paths
if ($MarkdownPaths -cmatch ".*\.txt") {
$MarkdownPath = Get-Content $MarkdownPaths
}
# When the input $MarkdownPaths is the path of a folder
else{
$MarkdownPath = $MarkdownPaths
}
foreach($_ in Get-ChildItem $MarkdownPath -Recurse:$Recurse.IsPresent){
foreach($_ in Get-ChildItem $MarkdownPath -Recurse:$MarkdownRecurse.IsPresent){
# Filter the .md of overview in "\help\"
if ((Get-Item -Path $_.FullName).Directory.Name -eq "help" -and $_.FullName -cmatch ".*\.md" -and $_.BaseName -cmatch "^[A-Z][a-z]+-([A-Z][a-z0-9]*)+$") {
if((Get-Item -Path $_.FullName).Directory.Parent.Name -eq "netcoreapp3.1"){
Expand All @@ -91,19 +93,18 @@ if ($PSCmdlet.ParameterSetName -eq "Markdown") {
}
}
$codeMap| Export-Csv "$OutputFolder\TempCodeMap.csv" -NoTypeInformation
if (!$SkipAnalyzing.IsPresent) {
$ScriptPath = "$OutputFolder\TempScript.ps1"
$CodeMapPath = "$OutputFolder\TempCodeMap.csv"
}
}

# Analyze scripts
if ($PSCmdlet.ParameterSetName -eq "Script" -or !$SkipAnalyzing.IsPresent) {
# Read code map from file
$codeMap = Import-Csv $CodeMapPath
$TempScriptPath = "$OutputFolder\TempScript.ps1"
if ($PSCmdlet.ParameterSetName -eq "Script"){
$codeMap = Set-ScriptsIntoSingleScript -ScriptPaths $ScriptPaths -Recurse:$ScriptRecurse.IsPresent -OutputFolder $OutputFolder
$codeMap| Export-Csv "$OutputFolder\TempCodeMap.csv" -NoTypeInformation
}
# Read and analyze ".ps1" in \ScriptsByExample
Write-Output "Analyzing file ..."
$analysisResultsTable += Get-ScriptAnalyzerResult (Get-Item -Path $ScriptPath) $RulePaths -IncludeDefaultRules:$IncludeDefaultRules.IsPresent $codeMap -ErrorAction Continue
$analysisResultsTable += Get-ScriptAnalyzerResult -ScriptPath $TempScriptPath -RulePaths $RulePaths -IncludeDefaultRules:$IncludeDefaultRules.IsPresent -CodeMap $codeMap -ErrorAction Continue

# Summarize analysis results, output in Result.csv
if($analysisResultsTable){
Expand All @@ -113,5 +114,6 @@ if ($PSCmdlet.ParameterSetName -eq "Script" -or !$SkipAnalyzing.IsPresent) {

# Clean caches
if ($CleanScripts.IsPresent) {
Remove-Item $ScriptPath -Exclude *.csv -Recurse -ErrorAction Continue
Remove-Item $OutputFolder\TempScript.ps1 -ErrorAction Continue
Remove-Item $OutputFolder\TempCodeMap.csv -ErrorAction Continue
}
95 changes: 73 additions & 22 deletions tools/StaticAnalysis/ExampleAnalyzer/utils.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@
Get-NonExceptionRecord
Get-RecordsNotInAllowList
Measure-SectionMissingAndOutputScript
Set-SingleScriptAndCodeMap
Set-ScriptsIntoSingleScript
Get-ScriptAnalyzerResult
Set-AnalysisOutput
Set-ExampleProperties
Expand Down Expand Up @@ -357,27 +359,11 @@ function Measure-SectionMissingAndOutputScript {
}
}


# Output example codes to "TempScript.ps1"
if ($missingExampleCode -eq 0) {
$cmdletExamplesScriptPath = "$OutputFolder\TempScript.ps1"
$line = $exampleCodes.Count
if($line -ne 0){
$functionHead = "function $Module-$Cmdlet-$exampleNumber{"
Add-Content -Path (Get-Item $cmdletExamplesScriptPath).FullName -Value $functionHead
Add-Content -Path (Get-Item $cmdletExamplesScriptPath).FullName -Value $exampleCodes
$functionTail = "}"
Add-Content -Path (Get-Item $cmdletExamplesScriptPath).FullName -Value $functionTail
for($i = 0; $i -le $line + 1; $i++){
$codeMap += @{
TotalLine = $TotalLine + $i
Module = $Module
Cmdlet = $Cmdlet
Example = $exampleNumber
Line = $i
}
}
$TotalLine = $TotalLine + $line + 2
($tempCodeMap, $TotalLine) = Set-SingleScriptAndCodeMap -Content $exampleCodes -Module $Module -Cmdlet $Cmdlet -Example $exampleNumber -TotalLine $TotalLine -OutputFolder $OutputFolder
$codeMap += $tempCodeMap
}
}
}
Expand All @@ -394,6 +380,69 @@ function Measure-SectionMissingAndOutputScript {
}
}

<#
.SYNOPSIS
Merge and set PowerShell scripts into one.
#>
function Set-SingleScriptAndCodeMap {
param(
[string[]]$Content,
[string]$Module,
[string]$Cmdlet,
[int]$Example,
[int]$TotalLine,
[string]$OutputFolder
)
$codeMap =@()
$line = $Content.Count
$tempScriptPath = "$OutputFolder\TempScript.ps1"
$functionHead = "function $Module"
if($null -ne $Cmdlet){
$functionHead += "-$Cmdlet"
}
if($null -ne $exampleNumber){
$functionHead += "-$exampleNumber"
}
$functionHead += "{"
Add-Content -Path (Get-Item $tempScriptPath).FullName -Value $functionHead
Add-Content -Path (Get-Item $tempScriptPath).FullName -Value $Content
Add-Content -Path (Get-Item $tempScriptPath).FullName -Value "}"
for($i = 0; $i -le $line + 1; $i++){
$codeMap += @{
TotalLine = $TotalLine + $i
Module = $Module
Cmdlet = $Cmdlet
Example = $Example
Line = $i
}
}
$TotalLine = $TotalLine + $line + 2
return ($codeMap, $TotalLine)
}

<#
.SYNOPSIS
Set PowerShell scripts into one and generate the code map.
#>
function Set-ScriptsIntoSingleScript {
param(
[string]$ScriptPaths,
[switch]$Recurse,
[string]$OutputFolder
)
$TotalLine = 1
$codeMap = @()
foreach($_ in Get-ChildItem $ScriptPaths -Recurse:$Recurse.IsPresent){
if((Test-Path $_ -PathType Leaf) -and $_.FullName.EndsWith(".ps1")){
$fileName = (Get-Item -Path $_.FullName).Name
$scriptContent = Get-Content $_
($tempCodeMap, $TotalLine) = Set-SingleScriptAndCodeMap -Content $scriptContent -Module $fileName -TotalLine $TotalLine -OutputFolder $OutputFolder
$codeMap += $tempCodeMap
}
}
return $codeMap
}

<#
.SYNOPSIS
Set properties for an AnalysisOutput object.
Expand Down Expand Up @@ -429,12 +478,14 @@ function Set-AnalysisOutput {
<#
.SYNOPSIS
Invoke PSScriptAnalyzer with custom rules, return the error set.
.PARAMETER RulePath
PSScriptAnalyzer custom rules path. Supports wildcard.
#>
function Get-ScriptAnalyzerResult {
param (
[string]$ScriptPath,
[Parameter(Mandatory, HelpMessage = "PSScriptAnalyzer custom rules path. Supports wildcard.")]
[string[]]$RulePath,
[Parameter(Mandatory)]
[string[]]$RulePaths,
[switch]$IncludeDefaultRules,
[Object[]]$CodeMap
)
Expand All @@ -444,11 +495,11 @@ function Get-ScriptAnalyzerResult {
}

# Invoke PSScriptAnalyzer : input scriptblock, output error set in $result with property: RuleName, Message, Extent
if ($null -eq $RulePath) {
if ($null -eq $RulePaths) {
$analysisResults = Invoke-ScriptAnalyzer -Path $ScriptPath -IncludeDefaultRules:$IncludeDefaultRules.IsPresent
}
else {
$analysisResults = Invoke-ScriptAnalyzer -Path $ScriptPath -CustomRulePath $RulePath -IncludeDefaultRules:$IncludeDefaultRules.IsPresent
$analysisResults = Invoke-ScriptAnalyzer -Path $ScriptPath -CustomRulePath $RulePaths -IncludeDefaultRules:$IncludeDefaultRules.IsPresent
}
$errors = @()
foreach($analysisResult in $analysisResults){
Expand Down