merging changes with master branch

documentation/development-docs/sdkbased-vs-autogen.md

@@ -0,0 +1,101 @@
+# Description
+
+The document is for those, who are familiar with SDK-based modules and want to give a try with auto-gen modules. The content will contain the comparation between these two kinds of modules from architecture, cmdlets differences, and onboard process, etc.
+
+# Architecture
+
+## SDK-based Module
+![image](images/sdkbased.PNG)
+
+## Auto-gen Module
+![image](images/autogen.PNG)
+
+# Cmdlet Parameters
+
+Highlight parameters differences in auto-gen modules as below.
+
+- No Set-* cmdlets, Set-* is replaced by New-* and Update-*
+- Add *[-SubscriptionId <String[]>]* parameter for get-* cmdlets and add for *[-SubscriptionId <String>]* parameter for the others
+- Do not support breaking change warning yet
+- Pipeline
+    - Do not support to pipe in parent resource
+    - Unify two types (through *-InputObject and -ResourceId*) of pipeline into one *-InputObject,* Users could either pipe in the resourceId or resource object
+# Implementation
+
+## Process
+
+### SDK-based Module
+
+- Design review
+- Get a released management SDK for Dotnet
+- Implement cmdlets in C# based on management SDK for Dotnet
+- Implement testcases
+- Generate docs, and populate them with examples
+- Create a PR in master branch for code review
+
+### Auto-gen Module
+
+- Design review
+- Write a readme.md as the autorest.powershell configuration for the module, and normally, it should contains
+    - links to swaggers in [azure-rest-api-specs](https://github.com/Azure/azure-rest-api-specs) (should pin to specific version through commit id to avoid potential breaking changes)
+    - A reference to readme.azure.noprofile.md to inline common configurations
+    - Module specific configurations, e.g. *module-version, title, subject-prefix...*
+    - directive
+- customization
+- Implement testcases
+- Generate docs and populate them with examples
+- Create a PR in **generation** branch for code review
+- Azure team members will help merge code from generation branch to master branch
+
+## Customization
+
+### SDK-based Module
+
+Since the cmdlets are handcrafted by developers, no special customization mechanism is required.
+
+### Auto-gen Module
+
+Customization could be done through three ways.
+
+- Through directive in readme.md, please see [directives.md](https://github.com/Azure/autorest.powershell/blob/master/docs/directives.md) for details
+- Through PowerShell script, please see [customization.md](https://github.com/Azure/autorest.powershell/blob/master/docs/customization.md) for details
+- Through C# code
+
+## Hero Scenario
+
+### SDK-based Module
+
+If your module relies on some other modules, you will need to rely on dotnet SDK of these modules, which could put in your module if only your module depends on them or in Az.Accouts they are some common modules required by several modules. 
+
+### Auto-gen Module
+
+Generally speaking, hero scenarios are implemented through customization through PowerShell script above. Regarding to third party dependencies, there are two ways to handle it.
+
+- Add the swaggers of the dependencies to readme.md either directly or by adding related helpers defined src/helpers if they are common shared modules
+- Rely on related Azure PowerShell modules, in this case, it is better you could add the logic to check whether the modules you rely on have installed or not
+
+## Docs and Examples
+
+### SDK-based Module
+
+Docs are generated through playPS, please see [platyPS](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/help-generation.md#Installing-platyPS) for details. And developers should populate generated docs with examples.
+
+### Auto-gen Module
+
+Autorest.powershell will generate example stubs in the examples folder, which should be populated by the developers manually. When building, examples in the examples folder will be automatically merged into docs. 
+
+## Test
+
+### SDK-based Module
+
+Tests are written in C# code.
+
+### Auto-gen Module
+
+Tests are written in PowerShell script. And we will leverage PowerShell test framework [Pester](https://github.com/pester/Pester).
+
+# Others
+
+## Data plane support
+
+We are still working on bringing data plane to auto-gen.

src/Accounts/Accounts.Test/AutosaveTests.cs

@@ -53,7 +53,6 @@ void ResetState()
             Environment.SetEnvironmentVariable("Azure_PS_Data_Collection", "false");
             PowerShellTokenCacheProvider tokenProvider = new InMemoryTokenCacheProvider();
             AzureSession.Instance.RegisterComponent(PowerShellTokenCacheProvider.PowerShellTokenCacheProviderKey, () => tokenProvider);
-            AzureSession.Instance.RegisterComponent(nameof(PowerShellTokenCache), () => tokenProvider.GetTokenCache());
         }
 
         [Fact]

src/Accounts/Accounts.Test/AzureSessionTestInitializer.cs

@@ -29,7 +29,6 @@ public static void Initialize()
 
             PowerShellTokenCacheProvider tokenCacheProvider = new InMemoryTokenCacheProvider();
             AzureSession.Instance.RegisterComponent(PowerShellTokenCacheProvider.PowerShellTokenCacheProviderKey, () => tokenCacheProvider);
-            AzureSession.Instance.RegisterComponent(nameof(PowerShellTokenCache), () => tokenCacheProvider.GetTokenCache());
             IAuthenticatorBuilder builder = new DefaultAuthenticatorBuilder();
             AzureSession.Instance.RegisterComponent(AuthenticatorBuilder.AuthenticatorBuilderKey, () => builder);
             AzureSession.Instance.RegisterComponent(nameof(AzureCredentialFactory), () => new AzureCredentialFactory());

src/Accounts/Accounts/Account/ConnectAzureRmAccount.cs

@@ -14,7 +14,6 @@
 
 using System;
 using System.Collections.Concurrent;
-using System.Collections.Generic;
 using System.Management.Automation;
 using System.Security;
 using System.Threading;
@@ -31,6 +30,7 @@
 using Microsoft.Azure.Commands.Profile.Models.Core;
 using Microsoft.Azure.Commands.Profile.Properties;
 using Microsoft.Azure.Commands.ResourceManager.Common;
+using Microsoft.Azure.Commands.ResourceManager.Common.ArgumentCompleters;
 using Microsoft.Azure.PowerShell.Authenticators;
 using Microsoft.Azure.PowerShell.Authenticators.Factories;
 using Microsoft.Identity.Client;
@@ -57,6 +57,7 @@ public class ConnectAzureRmAccountCommand : AzureContextModificationCmdlet, IMod
         public const string MSISecretVariable = "MSI_SECRET";
         public const int DefaultMaxContextPopulation = 25;
         public const string DefaultMaxContextPopulationString = "25";
+        private const int DefaultManagedServicePort = 50342;
 
         private IAzureEnvironment _environment;
 
@@ -127,19 +128,6 @@ public class ConnectAzureRmAccountCommand : AzureContextModificationCmdlet, IMod
         [Alias("MSI", "ManagedService")]
         public SwitchParameter Identity { get; set; }
 
-        [Parameter(ParameterSetName = ManagedServiceParameterSet, Mandatory = false, HelpMessage = "Obsolete. To use customized MSI endpoint, please set environment variable MSI_ENDPOINT, e.g. \"http://localhost:50342/oauth2/token\". Port number for managed service login.")]
-        [PSDefaultValue(Help = "50342", Value = 50342)]
-        public int ManagedServicePort { get; set; } = 50342;
-
-        [Parameter(ParameterSetName = ManagedServiceParameterSet, Mandatory = false, HelpMessage = "Obsolete. To use customized MSI endpoint, please set environment variable MSI_ENDPOINT, e.g. \"http://localhost:50342/oauth2/token\". Host name for managed service login.")]
-        [PSDefaultValue(Help = "localhost", Value = "localhost")]
-        public string ManagedServiceHostName { get; set; } = "localhost";
-
-        [Parameter(ParameterSetName = ManagedServiceParameterSet, Mandatory = false, HelpMessage = "Obsolete. To use customized MSI secret, please set environment variable MSI_SECRET. Secret, used for some kinds of managed service login.")]
-        [ValidateNotNullOrEmpty]
-        public SecureString ManagedServiceSecret { get; set; }
-
-
         [Alias("SubscriptionName", "SubscriptionId")]
         [Parameter(ParameterSetName = UserParameterSet,
                     Mandatory = false, HelpMessage = "Subscription Name or ID", ValueFromPipeline = true)]
@@ -156,6 +144,33 @@ public class ConnectAzureRmAccountCommand : AzureContextModificationCmdlet, IMod
         [ValidateNotNullOrEmpty]
         public string Subscription { get; set; }
 
+        private const string AuthScopeHelpMessage = "Optional OAuth scope for login, supported pre-defined values: AadGraph, AnalysisServices, Attestation, Batch, DataLake, KeyVault, OperationalInsights, Storage, Synapse. It also supports resource id like 'https://storage.azure.com/'.";
+
+        [Alias("AuthScopeTypeName")]
+        [Parameter(ParameterSetName = UserParameterSet,
+            Mandatory = false, HelpMessage = AuthScopeHelpMessage)]
+        [Parameter(ParameterSetName = UserWithCredentialParameterSet,
+            Mandatory = false, HelpMessage = AuthScopeHelpMessage)]
+        [Parameter(ParameterSetName = ServicePrincipalParameterSet,
+            Mandatory = false, HelpMessage = AuthScopeHelpMessage)]
+        [Parameter(ParameterSetName = ServicePrincipalCertificateParameterSet,
+            Mandatory = false, HelpMessage = AuthScopeHelpMessage)]
+        [Parameter(ParameterSetName = ManagedServiceParameterSet,
+            Mandatory = false, HelpMessage = AuthScopeHelpMessage)]
+        [PSArgumentCompleter(
+            SupportedResourceNames.AadGraph,
+            SupportedResourceNames.AnalysisServices,
+            SupportedResourceNames.Attestation,
+            SupportedResourceNames.Batch,
+            SupportedResourceNames.DataLake,
+            SupportedResourceNames.KeyVault,
+            SupportedResourceNames.ManagedHsm,
+            SupportedResourceNames.OperationalInsights,
+            SupportedResourceNames.Storage,
+            SupportedResourceNames.Synapse
+            )]
+        public string AuthScope { get; set; }
+
         [Parameter(Mandatory = false, HelpMessage = "Name of the default context from this login")]
         [ValidateNotNullOrEmpty]
         public string ContextName { get; set; }
@@ -291,55 +306,7 @@ public override void ExecuteCmdlet()
                     break;
                 case ManagedServiceParameterSet:
                     azureAccount.Type = AzureAccount.AccountType.ManagedService;
-                    var builder = new UriBuilder
-                    {
-                        Scheme = "http",
-                        Host = ManagedServiceHostName,
-                        Port = ManagedServicePort,
-                        Path = "/oauth2/token"
-                    };
-
-                    //ManagedServiceHostName/ManagedServicePort/ManagedServiceSecret are obsolete, should be removed in next major release
-                    if (this.IsBound(nameof(ManagedServiceHostName)) || this.IsBound(nameof(ManagedServicePort)) || this.IsBound(nameof(ManagedServiceSecret)))
-                    {
-                        WriteWarning(Resources.ObsoleteManagedServiceParameters);
-                    }
-
-                    var envSecret = System.Environment.GetEnvironmentVariable(MSISecretVariable);
-
-                    var msiSecret = this.IsBound(nameof(ManagedServiceSecret))
-                        ? ManagedServiceSecret.ConvertToString()
-                        : envSecret;
-
-                    var envUri = System.Environment.GetEnvironmentVariable(MSIEndpointVariable);
-
-                    var suppliedUri = this.IsBound(nameof(ManagedServiceHostName))
-                        ? builder.Uri.ToString()
-                        : envUri;
-
-                    if (!this.IsBound(nameof(ManagedServiceHostName)) && !string.IsNullOrWhiteSpace(envUri) 
-                        && !this.IsBound(nameof(ManagedServiceSecret)) && !string.IsNullOrWhiteSpace(envSecret))
-                    {
-                        // set flag indicating this is AppService Managed Identity ad hoc mode
-                        azureAccount.SetProperty(AuthenticationFactory.AppServiceManagedIdentityFlag, "the value not used");
-                    }
-
-                    if (!string.IsNullOrWhiteSpace(msiSecret))
-                    {
-                        azureAccount.SetProperty(AzureAccount.Property.MSILoginSecret, msiSecret);
-                    }
-
-                    if (!string.IsNullOrWhiteSpace(suppliedUri))
-                    {
-                        azureAccount.SetProperty(AzureAccount.Property.MSILoginUri, suppliedUri);
-                    }
-                    else
-                    {
-                        azureAccount.SetProperty(AzureAccount.Property.MSILoginUriBackup, builder.Uri.ToString());
-                        azureAccount.SetProperty(AzureAccount.Property.MSILoginUri, AuthenticationFactory.DefaultMSILoginUri);
-                    }
-
-                    azureAccount.Id = this.IsBound(nameof(AccountId)) ? AccountId : string.Format(Constants.DefaultMsiAccountIdPrefix + "{0}", ManagedServicePort);
+                    azureAccount.Id = this.IsBound(nameof(AccountId)) ? AccountId : $"{Constants.DefaultMsiAccountIdPrefix}{DefaultManagedServicePort}";
                     break;
                 default:
                     //Support username + password for both Windows PowerShell and PowerShell 6+
@@ -390,6 +357,8 @@ public override void ExecuteCmdlet()
                 }
             }
 
+            var resourceId = PreProcessAuthScope();
+
             if (ShouldProcess(string.Format(Resources.LoginTarget, azureAccount.Type, _environment.Name), "log in"))
             {
                 if (AzureRmProfileProvider.Instance.Profile == null)
@@ -429,7 +398,8 @@ public override void ExecuteCmdlet()
                         WriteWarningEvent, //Could not use WriteWarning directly because it may be in worker thread
                         name,
                         shouldPopulateContextList,
-                        MaxContextPopulation));
+                        MaxContextPopulation,
+                        resourceId));
                    task.Start();
                    while (!task.IsCompleted)
                    {
@@ -467,6 +437,18 @@ public override void ExecuteCmdlet()
             }
         }
 
+        private string PreProcessAuthScope()
+        {
+            string mappedScope = AuthScope;
+            if (!string.IsNullOrEmpty(AuthScope) &&
+                SupportedResourceNames.DataPlaneResourceNameMap.ContainsKey(AuthScope))
+            {
+                mappedScope = SupportedResourceNames.DataPlaneResourceNameMap[AuthScope];
+                WriteDebug($"Map AuthScope from {AuthScope} to {mappedScope}.");
+            }
+            return mappedScope;
+        }
+
         private bool IsUsingInteractiveAuthentication()
         {
             return ParameterSetName == UserParameterSet && UseDeviceAuthentication == false;
@@ -616,12 +598,10 @@ public void OnImport()
                 {
                     provider = new InMemoryTokenCacheProvider();
                 }
-                var tokenCache = provider.GetTokenCache();
                 IAzureEventListenerFactory azureEventListenerFactory = new AzureEventListenerFactory();
                 AzureSession.Instance.RegisterComponent(nameof(CommonUtilities), () => new CommonUtilities());
                 AzureSession.Instance.RegisterComponent(PowerShellTokenCacheProvider.PowerShellTokenCacheProviderKey, () => provider);
                 AzureSession.Instance.RegisterComponent(nameof(IAzureEventListenerFactory), () => azureEventListenerFactory);
-                AzureSession.Instance.RegisterComponent(nameof(PowerShellTokenCache), () => tokenCache);
                 AzureSession.Instance.RegisterComponent(nameof(AzureCredentialFactory), () => new AzureCredentialFactory());
                 AzureSession.Instance.RegisterComponent(nameof(MsalAccessTokenAcquirerFactory), () => new MsalAccessTokenAcquirerFactory());
 #if DEBUG

src/Accounts/Accounts/Accounts.format.ps1xml

@@ -240,5 +240,42 @@
         </TableRowEntries>
       </TableControl>
     </View>
+    <View>
+      <Name>Microsoft.Azure.Commands.Profile.Models.PSAzureRmAccount</Name>
+      <ViewSelectedBy>
+        <TypeName>Microsoft.Azure.Commands.Profile.Models.PSAzureRmAccount</TypeName>
+      </ViewSelectedBy>
+      <ListControl>
+        <ListEntries>
+          <ListEntry>
+            <ListItems>
+              <ListItem>
+                <PropertyName>Id</PropertyName>
+              </ListItem>
+              <ListItem>
+                <PropertyName>Type</PropertyName>
+              </ListItem>
+              <ListItem>
+                <PropertyName>Tenants</PropertyName>
+              </ListItem>
+              <ListItem>
+                <PropertyName>Credential</PropertyName>
+              </ListItem>
+              <ListItem>
+                <PropertyName>TenantMap</PropertyName>
+              </ListItem>
+              <ListItem>
+                <PropertyName>CertificateThumbprint</PropertyName>
+              </ListItem>
+              <ListItem>
+                <Label>ExtendedProperties</Label>
+                <ScriptBlock>$_.ExtendedProperties.GetEnumerator() | Where-Object { $_.Key -ne "ServicePrincipalSecret" }</ScriptBlock>
+              </ListItem>
+            </ListItems>
+          </ListEntry>
+        </ListEntries>
+      </ListControl>
+    </View>
+
   </ViewDefinitions>
 </Configuration>