Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature]: Disable instance discovery #22535

Closed
TheOnlyWei opened this issue Aug 9, 2023 · 8 comments
Closed

[Feature]: Disable instance discovery #22535

TheOnlyWei opened this issue Aug 9, 2023 · 8 comments
Assignees
@msJinLei
Labels
feature-request This issue requires a new behavior in the product in order be resolved.
Milestone
Az 11.5.0 (2024-0...

Comments

@TheOnlyWei
Copy link

TheOnlyWei commented Aug 9, 2023 •
edited
Loading

Description of the new feature

Add ability to disable instance discovery during connect-azaccount.

MSAL library already has a way to disable instance discovery (link).

Azure.Identity also supports disabling instance discovery (link).

Currently, no way to disable instance discovery for latest Az.Accounts:
https://learn.microsoft.com/en-us/powershell/module/az.accounts/connect-azaccount?view=azps-10.2.0

This is required for on-premise environments or private clouds.

Proposed implementation details (optional)

Example ServicePrincipalAuthenticator.cs fix:
Add DisableInstanceDiscovery property (link) to ClientCertificateCredentialOptions and ClientSecretCredentialOptions object if requested:
https://github.com/Azure/azure-powershell/blob/15fa28fea98d8bee400ae851f0c5c92099a96621/src/Accounts/Authenticators/ServicePrincipalAuthenticator.cs#L49C34-L49C34
https://github.com/Azure/azure-powershell/blob/15fa28fea98d8bee400ae851f0c5c92099a96621/src/Accounts/Authenticators/ServicePrincipalAuthenticator.cs#L70C48-L70C48
The above would have to be done for all other authenticators.
@TheOnlyWei TheOnlyWei added feature-request This issue requires a new behavior in the product in order be resolved. needs-triage This is a new issue that needs to be triaged to the appropriate team. labels Aug 9, 2023
@isra-fel
Copy link
Member

@TheOnlyWei thanks for reporting :) I hope to clarify a few questions:

  1. instance discovery is different from endpoint discovery or they are the same thing?
  2. We never enabled instance discovery. Are you suggesting that it's enabled by default and you need to turn it off?
  3. What's the impact of instance discovery to on-premise enviornments? Do customers get an error when Connect-AzAccount or it's slow?

@isra-fel isra-fel added needs-author-feedback More information is needed from author to address the issue. and removed needs-triage This is a new issue that needs to be triaged to the appropriate team. labels Aug 10, 2023
@TheOnlyWei
Copy link
Author

TheOnlyWei commented Aug 10, 2023 •
edited
Loading

@isra-fel

  1. Instance discovery is different from endpoint discovery, assuming endpoint discovery means you are parsing the ARM metadata endpoint. From the documentation:

The process of instance discovery entails retrieving authority metadata from https://login.microsoft.com/ to validate the authority.

https://learn.microsoft.com/en-us/dotnet/api/azure.identity.clientsecretcredentialoptions.disableinstancediscovery?view=azure-dotnet#azure-identity-clientsecretcredentialoptions-disableinstancediscovery

  1. Instance discovery is enabled unless disabled.
  2. Yes, customers get below error if instance discovery is on because it automatically uses the Azure Cloud login or active directory authority endpoint (login.microsoftonline.com):
DEBUG: 2:12:40 PM - ConnectAzureRmAccountCommand begin processing with ParameterSet 'ServicePrincipalWithSubscriptionId'.
DEBUG: 2:12:41 PM - [ConfigManager] Got nothing from [DisplayBreakingChangeWarning], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: 2:12:41 PM - Autosave setting from startup session: 'CurrentUser'
DEBUG: 2:12:41 PM - No autosave setting detected in environment variable 'AzContextAutoSave'. 
DEBUG: 2:12:41 PM - Using Autosave scope 'CurrentUser'
DEBUG: 2:12:41 PM - Autosave setting from startup session: 'CurrentUser'
DEBUG: 2:12:41 PM - No autosave setting detected in environment variable 'AzContextAutoSave'. 
DEBUG: 2:12:41 PM - Using Autosave scope 'CurrentUser'
DEBUG: 2:12:42 PM - Autosave setting from startup session: 'CurrentUser'
DEBUG: 2:12:42 PM - No autosave setting detected in environment variable 'AzContextAutoSave'. 
DEBUG: 2:12:42 PM - Using Autosave scope 'CurrentUser'
DEBUG: 2:12:42 PM - [ServicePrincipalAuthenticator] Calling ClientSecretCredential.GetTokenAsync - ApplicationId:'<customer service principal client or application id>', TenantId:'<customer tenant id>', Scopes:'<customer resource manager url>/.default', AuthorityHost:'<customr login or active directory host endpoint>'
DEBUG: ClientSecretCredential.GetToken invoked. Scopes: [<customr resource manager url>/.default ] ParentRequestId: 
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] MSAL MSAL.Desktop with as
sembly version '4.49.1.0'. CorrelationId(a3b5f9aa-f27d-4d8b-8454-743633adad68)
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] === AcquireTokenForClient
Parameters ===
SendX5C: False
ForceRefresh: False

DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] 
=== Request Data ===
Authority Provided? - True
Scopes - <customer resource manager url>/.default
Extra Query Params Keys (space separated) - 
ApiId - AcquireTokenForClient
IsConfidentialClient - True
SendX5C - False
LoginHint ? False
IsBrokerConfigured - False
HomeAccountId - False
CorrelationId - a3b5f9aa-f27d-4d8b-8454-743633adad68
UserAssertion set: False
LongRunningOboCacheKey set: False
Region configured: 

DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] === Token Acquisition (Cl
ientCredentialRequest) started:
	 Scopes: <resource manager url>/.default
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] [Instance Discovery] Instance discovery is enabled and will be performed
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] [Region discovery] Not using a regional authority. 
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] Fetching instance discovery from the network from host login.microsoftonline.com. 
DEBUG: Request [ebcac509-60c2-441e-823b-b4dc3e0096b8] GET https://login.microsoftonline.com/common/discovery/instance?api-version=1.1&authorization_endpoint=REDACTED
x-client-SKU:REDACTED
x-client-Ver:REDACTED
x-client-CPU:REDACTED
x-client-OS:REDACTED
client-request-id:REDACTED
return-client-request-id:REDACTED
x-app-name:REDACTED
x-app-ver:REDACTED
x-ms-client-request-id:ebcac509-60c2-441e-823b-b4dc3e0096b8
x-ms-return-client-request-id:true
User-Agent:azsdk-net-Identity/1.6.1 (.NET Framework 4.8.4644.0; Microsoft Windows 10.0.20348 )
client assembly: Azure.Identity
DEBUG: Error response [ebcac509-60c2-441e-823b-b4dc3e0096b8] 400 Bad Request (00.2s)
Strict-Transport-Security:REDACTED
X-Content-Type-Options:REDACTED
Access-Control-Allow-Origin:REDACTED
Access-Control-Allow-Methods:REDACTED
client-request-id:REDACTED
x-ms-request-id:936676f7-990e-488d-9bbc-97371b3cbd00
x-ms-ests-server:REDACTED
X-XSS-Protection:REDACTED
Cache-Control:private
Content-Type:application/json; charset=utf-8
P3P:REDACTED
Set-Cookie:REDACTED
Date:Wed, 09 Aug 2023 21:12:42 GMT
Content-Length:461
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] Response status code does not indicate success: 400 (BadRequest). 
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] Request retry failed.
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] HttpStatusCode: 400: BadRequest
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] === Token Acquisition (1004) failed.
	Host: <this prints customer's actual login or active directory endpoint for some reason even though it used Azure Cloud's  login.microsoftonline.com in above output>.
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] Exception type: Microsoft.Identity.Client.MsalServiceException
, ErrorCode: invalid_instance
HTTP StatusCode 400
CorrelationId a3b5f9aa-f27d-4d8b-8454-743633adad68

DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] [Instance Discovery] Instance discovery failed - invalid instance!
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] Fetching a new AT failed. Is exception retry-able? False. Is there an AT in the cache that is usable? False
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] Either the exception does not indicate a problem with AAD or the token cache does not have an AT that is usable. 
DEBUG: False MSAL 4.49.1.0 MSAL.Desktop 4.8 or later Windows Server 2022 Datacenter [2023-08-09 21:12:42Z - a3b5f9aa-f27d-4d8b-8454-743633adad68] Exception type: Microsoft.Identity.Client.MsalServiceException
, ErrorCode: invalid_instance
HTTP StatusCode 400
CorrelationId a3b5f9aa-f27d-4d8b-8454-743633adad68
   at Microsoft.Identity.Client.Internal.Requests.RequestBase.<HandleTokenRefreshErrorAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.<ExecuteAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Identity.Client.Internal.Requests.RequestBase.<RunAsync>d__12.MoveNext()
DEBUG: ClientSecretCredential.GetToken was unable to retrieve an access token. Scopes: [ <customer's resource manager URL>/.default ] ParentRequestId:  Except
ion: Azure.Identity.AuthenticationFailedException (0x80131500): ClientSecretCredential authentication failed: AADSTS50049: Unknown or invalid instance.
Trace ID: 936676f7-990e-488d-9bbc-97371b3cbd00
Correlation ID: a3b5f9aa-f27d-4d8b-8454-743633adad68
Timestamp: 2023-08-09 21:12:42Z
 ---> Microsoft.Identity.Client.MsalServiceException (0x80131500): AADSTS50049: Unknown or invalid instance.
Trace ID: 936676f7-990e-488d-9bbc-97371b3cbd00
Correlation ID: a3b5f9aa-f27d-4d8b-8454-743633adad68
Timestamp: 2023-08-09 21:12:42Z
DEBUG: 2:12:42 PM - [ConfigManager] Got nothing from [EnableDataCollection], Module = [], Cmdlet = []. Returning default value [True].

Connect-AzAccount : ClientSecretCredential authentication failed: AADSTS50049: Unknown or invalid instance.
Trace ID: 936676f7-990e-488d-9bbc-97371b3cbd00
Correlation ID: a3b5f9aa-f27d-4d8b-8454-743633adad68
Timestamp: 2023-08-09 21:12:42Z
Could not find tenant id for provided tenant domain '<customer tenant ID>'. Please ensure that the provided service principal 
'<service principal client id or application id>' is found in the provided tenant domain.
At line:1 char:1
+ Connect-AzAccount -EnvironmentName "<customer environment name>" `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [Connect-AzAccount], ArgumentNullException
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.Profile.ConnectAzureRmAccountCommand

DEBUG: 2:12:43 PM - [ConfigManager] Got nothing from [DisplayBreakingChangeWarning], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: 2:12:43 PM - [ConfigManager] Got nothing from [DisplayRegionIdentified], Module = [], Cmdlet = []. Returning default value [True].
DEBUG: AzureQoSEvent:  Module: Az.Accounts:2.12.5; CommandName: Connect-AzAccount; PSVersion: 5.1.20348.1366; IsSuccess: False; Duration: 00:00:02.6731210; Exception: Clie
ntSecretCredential authentication failed: AADSTS50049: Unknown or invalid instance.
Trace ID: 936676f7-990e-488d-9bbc-97371b3cbd00
Correlation ID: a3b5f9aa-f27d-4d8b-8454-743633adad68
Timestamp: 2023-08-09 21:12:42Z
Could not find tenant id for provided tenant domain '<customer tenant id>'. Please ensure that the provided service principal '<customer service principal client or application id>' is found in the provided tenant domain.;
DEBUG: 2:12:43 PM - ConnectAzureRmAccountCommand end processing.

@TheOnlyWei TheOnlyWei removed the needs-author-feedback More information is needed from author to address the issue. label Aug 11, 2023
@msJinLei msJinLei added the Tracking We will track status and follow internally label Oct 9, 2023
@msJinLei msJinLei self-assigned this Oct 9, 2023
@msJinLei msJinLei removed the Tracking We will track status and follow internally label Oct 9, 2023
@domggarrity
Copy link
Member

Hi @isra-fel and @msJinLei, my name is Dominic and I'm the Azure Stack developer experience PM. Do you have any update on this? This is required for our customers' on-prem/private environments.

@domggarrity
Copy link
Member

Hi @isra-fel and @msJinLei, just following up on this.

msJinLei added a commit to msJinLei/azure-powershell that referenced this issue Feb 5, 2024
@msJinLei
Enabled globally disabling instance discovery before token acquisition
3464802 
Fix Azure#22535
msJinLei added a commit to msJinLei/azure-powershell that referenced this issue Feb 5, 2024
@msJinLei
Enabled globally disabling instance discovery before token acquisition
827aa98 
Fix Azure#22535
msJinLei added a commit to msJinLei/azure-powershell that referenced this issue Mar 11, 2024
@msJinLei
Enabled globally disabling instance discovery before token acquisition
a397e89 
Fix Azure#22535
msJinLei added a commit to msJinLei/azure-powershell that referenced this issue Mar 11, 2024
@msJinLei
Enabled globally disabling instance discovery before token acquisition
d628886 
Fix Azure#22535
msJinLei added a commit to msJinLei/azure-powershell that referenced this issue Mar 11, 2024
@msJinLei
Enabled globally disabling instance discovery before token acquisition
c4d051e 
Fix Azure#22535
@isra-fel isra-fel added this to the Az 11.5.0 (2024-04-02) milestone Mar 13, 2024
@isra-fel
Copy link
Member

@domggarrity @TheOnlyWei this is being worked on. ETA is April 2nd.

@msJinLei
Copy link
Contributor

msJinLei commented Mar 13, 2024 •
edited
Loading

@domggarrity @TheOnlyWei this is being worked on. ETA is April 2nd.

The way we are going to support disable instance discovery is use AzConfig.

Update-AzConfig -DisableInstanceDiscovery $true
Connect-AzAccount

Please let us know if you have any questions.

msJinLei added a commit to msJinLei/azure-powershell that referenced this issue Mar 18, 2024
@msJinLei
Enabled globally disabling instance discovery before token acquisition
e67b2df 
Fix Azure#22535
@domggarrity
Copy link
Member

Thanks for the update, @isra-fel and @msJinLei! Just to confirm, this change only applied to Azure Stack/hybrid cloud, right?

@msJinLei
Copy link
Contributor

msJinLei commented May 6, 2024

Thanks for the update, @isra-fel and @msJinLei! Just to confirm, this change only applied to Azure Stack/hybrid cloud, right?

No. Anyone can enable the change by running Update-AzConfig -DisableInstanceDiscovery $true

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@msJinLei msJinLei

Labels
feature-request This issue requires a new behavior in the product in order be resolved.
Projects
None yet
Milestone
Az 11.5.0 (2024-04-02)
Development

No branches or pull requests
4 participants
@isra-fel @TheOnlyWei @msJinLei @domggarrity