Skip to content

Commit

Permalink
Added PackageAction Set for Update-AzSynapseSparkPool to support …
Browse files Browse the repository at this point in the history
…removing and adding packages in one action (#21579)

* Added AzureSynapseSparkPool PackageAction Set to allow removing and update of a package in one action

* updated Synapse/ChangeLog.md

* updated help file Update-AzSynapseSparkPool.md

* added PackageAction Set example to help file for Update-AzSynapseSparkPool command

---------

Co-authored-by: Morris Emming <[email protected]>
  • Loading branch information
morrisjdev and memmingbag authored Apr 20, 2023
1 parent 3966a39 commit a745e49
Show file tree
Hide file tree
Showing 4 changed files with 23 additions and 7 deletions.
1 change: 1 addition & 0 deletions src/Synapse/Synapse/ChangeLog.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@

## Upcoming Release
* Updated Azure.Core to 1.31.0.
* Added PackageAction `Set` for `Update-AzSynapseSparkPool` to support removing and adding packages in one action

## Version 2.3.0
* Upgraded Azure.Analytics.Synapse.Artifacts to 1.0.0-preview.17
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -267,9 +267,9 @@ public override void ExecuteCmdlet()

if (this.IsParameterBound(c => c.PackageAction) && this.IsParameterBound(c => c.Package))
{
if (this.PackageAction == SynapseConstants.PackageActionType.Add)
if (this.PackageAction == SynapseConstants.PackageActionType.Add || this.PackageAction == SynapseConstants.PackageActionType.Set)
{
if (existingSparkPool.CustomLibraries == null)
if (existingSparkPool.CustomLibraries == null || this.PackageAction == SynapseConstants.PackageActionType.Set)
{
existingSparkPool.CustomLibraries = new List<LibraryInfo>();
}
Expand Down
3 changes: 2 additions & 1 deletion src/Synapse/Synapse/Models/SynapseConstants.cs
Original file line number Diff line number Diff line change
Expand Up @@ -332,7 +332,8 @@ public class RepositoryType
public enum PackageActionType
{
Add,
Remove
Remove,
Set
}
public enum UserAssignedManagedIdentityActionType
{
Expand Down
22 changes: 18 additions & 4 deletions src/Synapse/Synapse/help/Update-AzSynapseSparkPool.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,28 +169,42 @@ The first command retrieves an Apache Spark pool in Azure Synapse Analytics. The

### Example 14
```powershell
$workspace_packages = Get-AzSynapseWorkspacePackage -WorkspaceName ContosoWorkspace
$pool = Get-AzSynapseSparkPool -ResourceGroupName ContosoResourceGroup -WorkspaceName ContosoWorkspace -Name ContosoSparkPool
$library_names = $pool.WorkspacePackages | Where-Object {$_.name -notlike "new_package-*"} | ForEach-Object {$_.name}
$library_names += "new_package-2.0-py3-none-any.whl"
$new_pool_packages = @($workspace_packages | Where-Object {$_.name -in $library_names})
Update-AzSynapseSparkPool -ResourceGroupName ContosoResourceGroup -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -PackageAction Set -Package $new_pool_packages
```

The first command retrieves the packages available in the workspace. The second command group retrieves the spark pool to get the packages currently linked to this pool and removes all versions of the package starting with `new_package-` from the retrieved list. The new version of the package is then added to this list. In the third group of commands the package list, containing only package names, is tranformed into a list of workspace packages by filtering the list of available workspace_packages accordingly and is then linked to the spark pool.

### Example 15
```powershell
$config = Get-AzSynapseSparkConfiguration -WorkspaceName ContosoWorkspace -Name ContosoSparkConfig1
Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -Tag @{"key" = "value"} -NodeCount 5 -NodeSize Medium -SparkConfiguration $configs
```

This command updates an Apache Spark pool in Azure Synapse Analytics and specify a Spark configuration for the Spark pool.

### Example 15
### Example 16
```powershell
Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -NodeSize small -ForceApplySetting
```

This command updates an Apache Spark pool in Azure Synapse Analytics, set NodeSize to small for the spark pool and force stop any running jobs in the Spark pool to apply this new setting.

### Example 16
### Example 17
```powershell
$pool = Get-AzSynapseSparkPool -ResourceGroupName ContosoResourceGroup -WorkspaceName ContosoWorkspace -Name ContosoSparkPool
$pool | Update-AzSynapseSparkPool -PackageAction Remove -Package $pool.WorkspacePackages -ForceApplySetting
```

The first command retrieves an Apache Spark pool in Azure Synapse Analytics. The second command removes all workspace packages that are linked to that Apache Spark pool and force stop any running jobs in the Spark pool to apply this new setting.

### Example 17
### Example 18
```powershell
Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -EnableIsolatedCompute $true -NodeSize XXXLarge
```
Expand Down Expand Up @@ -480,7 +494,7 @@ Package action must be specified when you add or remove a workspace package from
Type: Microsoft.Azure.Commands.Synapse.Models.SynapseConstants+PackageActionType
Parameter Sets: (All)
Aliases:
Accepted values: Add, Remove
Accepted values: Add, Remove, Set

Required: False
Position: Named
Expand Down

0 comments on commit a745e49

Please sign in to comment.