Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] SecretClientBuilder : java.lang.NoClassDefFoundError: reactor/netty/tcp/ProxyProvider$TypeSpec #19957

Closed
Jeevankumar555 opened this issue Mar 18, 2021 · 15 comments
Assignees
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported Issues that are reported by GitHub users external to the Azure organization. dependency-issue Issue that is caused by dependency conflicts HttpClient KeyVault needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team
Milestone

Comments

@Jeevankumar555
Copy link

Jeevankumar555 commented Mar 18, 2021

Describe the bug
The above issue is when we tried to use azure sdk for keyvaults and instantiate SecretClient using SecretClientBuilder.

Issue in com.azure: azure-security-keyvault-secrets 4.2.x
in combination with io.projectreactor.netty:reactor-netty:jar:1.0.3

Exception or Stack Trace
2021-03-18T18:58:53.6444488Z Caused by: java.lang.NoClassDefFoundError: reactor/netty/tcp/ProxyProvider$TypeSpec 2021-03-18T18:58:53.6445215Z at com.azure.core.http.netty.NettyAsyncHttpClientBuilder.lambda$build$7(NettyAsyncHttpClientBuilder.java:142) 2021-03-18T18:58:53.6446059Z at reactor.netty.http.client.HttpClient.tcpConfiguration(HttpClient.java:1465) 2021-03-18T18:58:53.6446779Z at com.azure.core.http.netty.NettyAsyncHttpClientBuilder.build(NettyAsyncHttpClientBuilder.java:122) 2021-03-18T18:58:53.6447555Z at com.azure.core.http.netty.NettyAsyncHttpClientProvider.createInstance(NettyAsyncHttpClientProvider.java:17) 2021-03-18T18:58:53.6448328Z at com.azure.core.implementation.http.HttpClientProviders.createInstance(HttpClientProviders.java:51) 2021-03-18T18:58:53.6449000Z at com.azure.core.http.HttpClient.createDefault(HttpClient.java:50) 2021-03-18T18:58:53.6449617Z at com.azure.core.http.HttpClient.createDefault(HttpClient.java:40) 2021-03-18T18:58:53.6450243Z at com.azure.core.http.HttpPipelineBuilder.build(HttpPipelineBuilder.java:62) 2021-03-18T18:58:53.6450942Z at com.azure.security.keyvault.secrets.SecretClientBuilder.buildAsyncClient(SecretClientBuilder.java:161)
**2021-03-18T18:58:53.6451836Z at com.azure.security.keyvault.secrets.SecretClientBuilder.buildClient(SecretClientBuilder.java:104)

To Reproduce
Using below dependencies
spring boot version : 2.4.2 in combination with

 <dependency>
      <groupId>com.azure</groupId>
      <artifactId>azure-security-keyvault-secrets</artifactId>
      <version>4.2.x</version>
      <exclusions>
      </exclusions>
  </dependency>

    <dependency>
      <groupId>com.azure</groupId>
      <artifactId>azure-identity</artifactId>
      <version>1.2.0</version>
    </dependency>

and try to build SecretClientBuilder object

Code Snippet

 new SecretClientBuilder()
        .vaultUrl(keyVaultUri)
        .credential(new DefaultAzureCredentialBuilder().build())
        .buildClient();
Maven dependency tree:
+- com.azure:azure-security-keyvault-secrets:jar:4.2.0:compile
[INFO] |  +- com.azure:azure-core:jar:1.7.0:compile
[INFO] |  |  +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.11.4:compile
[INFO] |  |  +- com.fasterxml.jackson.dataformat:jackson-dataformat-xml:jar:2.11.4:compile
[INFO] |  |  |  +- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.11.4:compile
[INFO] |  |  |  +- org.codehaus.woodstox:stax2-api:jar:4.2.1:compile
[INFO] |  |  |  \- com.fasterxml.woodstox:woodstox-core:jar:6.2.3:compile
[INFO] |  |  +- io.projectreactor:reactor-core:jar:3.4.2:compile
[INFO] |  |  |  \- org.reactivestreams:reactive-streams:jar:1.0.3:compile
[INFO] |  |  \- io.netty:netty-tcnative-boringssl-static:jar:2.0.36.Final:compile
[INFO] |  \- com.azure:azure-core-http-netty:jar:1.5.4:compile
[INFO] |     +- io.netty:netty-handler:jar:4.1.58.Final:compile
[INFO] |     |  +- io.netty:netty-common:jar:4.1.58.Final:compile
[INFO] |     |  +- io.netty:netty-resolver:jar:4.1.58.Final:compile
[INFO] |     |  +- io.netty:netty-transport:jar:4.1.58.Final:compile
[INFO] |     |  \- io.netty:netty-codec:jar:4.1.58.Final:compile
[INFO] |     +- io.netty:netty-handler-proxy:jar:4.1.58.Final:compile
[INFO] |     |  \- io.netty:netty-codec-socks:jar:4.1.58.Final:compile
[INFO] |     +- io.netty:netty-buffer:jar:4.1.58.Final:compile
[INFO] |     +- io.netty:netty-codec-http:jar:4.1.58.Final:compile
[INFO] |     +- io.netty:netty-codec-http2:jar:4.1.58.Final:compile
[INFO] |     +- io.netty:netty-transport-native-unix-common:jar:4.1.58.Final:compile
[INFO] |     +- io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.58.Final:compile
[INFO] |     +- io.netty:netty-transport-native-kqueue:jar:osx-x86_64:4.1.58.Final:compile
[INFO] |     **\- io.projectreactor.netty:reactor-netty:jar:1.0.3:compile**
[INFO] |        +- io.projectreactor.netty:reactor-netty-core:jar:1.0.3:compile
[INFO] |        |  +- io.netty:netty-resolver-dns:jar:4.1.58.Final:compile
[INFO] |        |  |  \- io.netty:netty-codec-dns:jar:4.1.58.Final:compile
[INFO] |        |  \- io.netty:netty-resolver-dns-native-macos:jar:osx-x86_64:4.1.58.Final:compile
[INFO] |        +- io.projectreactor.netty:reactor-netty-http:jar:1.0.3:compile
[INFO] |        \- io.projectreactor.netty:reactor-netty-http-brave:jar:1.0.3:runtime
[INFO] |           \- io.zipkin.brave:brave-instrumentation-http:jar:5.13.3:runtime
[INFO] |              \- io.zipkin.brave:brave:jar:5.13.3:runtime
[INFO] |                 \- io.zipkin.reporter2:zipkin-reporter-brave:jar:2.16.3:runtime
[INFO] |                    \- io.zipkin.reporter2:zipkin-reporter:jar:2.16.3:runtime
[INFO] |                       \- io.zipkin.zipkin2:zipkin:jar:2.23.2:runtime

```

**Expected behavior**
The SecretClientBuilder has instatiated the SecretClient sucessfully.

**Screenshots**
NA


**Setup (please complete the following information):**
 - OS: Windows
 - IDE : Intellij
 -   azure-security-keyvault-secrets 4.2.0

**Additional context**
Add any other context about the problem here.

**Fix for now**
  ```
  <dependency>
      <groupId>com.azure</groupId>
      <artifactId>azure-security-keyvault-secrets</artifactId>
      <version>4.2.x</version>
      <exclusions>
      </exclusions>
    </dependency>
    <dependency>
    <groupId>io.projectreactor.netty</groupId>
    <artifactId>reactor-netty</artifactId>
    <version>0.9.12.RELEASE</version>
    </dependency>
```

**Information Checklist**
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report
- [x ] Bug Description Added
- [x ] Repro Steps Added
- [x ] Setup information Added
@ghost ghost added needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Mar 18, 2021
@alzimmermsft
Copy link
Member

Hi @Jeevankumar555, could you try upgrading the azure-security-keyvault-secrets 4.2.6 which was released this month. This version should support working with Reactor Netty 1.0.x.

@Jeevankumar555
Copy link
Author

Jeevankumar555 commented Mar 19, 2021

using 4.2.6 I run into other issue

Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.azure.identity.implementation.IdentityClient
	at com.azure.identity.implementation.IdentityClientBuilder.build(IdentityClientBuilder.java:113)
	at com.azure.identity.ManagedIdentityCredential.<init>(ManagedIdentityCredential.java:39)
	at com.azure.identity.DefaultAzureCredentialBuilder.getCredentialsChain(DefaultAzureCredentialBuilder.java:129)
	at com.azure.identity.DefaultAzureCredentialBuilder.build(DefaultAzureCredentialBuilder.java:123)

An attempt was made to call a method that does not exist. The attempt was made from the following location:

com.azure.core.util.serializer.JacksonAdapter.<init>(JacksonAdapter.java:106)

The following method did not exist:

`com.fasterxml.jackson.databind.ObjectMapper.coercionConfigDefaults()Lcom/fasterxml/jackson/databind/cfg/MutableCoercionConfig;`

The method's class, com.fasterxml.jackson.databind.ObjectMapper, is available from the following locations:

jar:file:/C:/Users/<userId>/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.11.4/jackson-databind-2.11.4.jar!/com/fasterxml/jackson/databind/ObjectMapper.class

@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Mar 19, 2021
@conniey conniey added bug This issue requires a change to an existing behavior in the product in order to be resolved. and removed question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Mar 19, 2021
@alzimmermsft
Copy link
Member

Hi @Jeevankumar555, that issue should be resolved once the fix for #19897 is released. I will let you know once that has happened.

@alzimmermsft
Copy link
Member

Hi @Jeevankumar555, could you also include azure-core 1.14.1 in your project. This should resolve the new exception you are seeing.

@alzimmermsft alzimmermsft self-assigned this Mar 22, 2021
@alzimmermsft alzimmermsft added the needs-author-feedback Workflow: More information is needed from author to address the issue. label Mar 22, 2021
@Jeevankumar555
Copy link
Author

Jeevankumar555 commented Mar 23, 2021

azure-core 1.14.1 + azure-security-keyvault-secrets 4.2.6 : is the way to go ?

version 4.2.6 of azure keyvault secrets already has azure-core 1.14.0 , so I excluded it and included 1.14.1 and that combination does work.

    <dependency>
      <groupId>com.azure</groupId>
      <artifactId>azure-security-keyvault-secrets</artifactId>
      <version>4.2.6</version>
      <exclusions>
        <exclusion>
          <groupId>com.azure</groupId>
          <artifactId>azure-core</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.azure</groupId>
      <artifactId>azure-core</artifactId>
      <version>1.14.1</version>
    </dependency>
    

Is there a plan to release azure-security-keyvault-secrets new version to have the azure-core 1.14.1 or above inclusive?

@ghost ghost added needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team and removed needs-author-feedback Workflow: More information is needed from author to address the issue. labels Mar 23, 2021
@alzimmermsft
Copy link
Member

alzimmermsft commented Mar 23, 2021

For right now, azure-security-keyvault-secrets + overriding azure-core to 1.14.1 is the way to go until a newer version of azure-security-keyvault-secrets is released. New versions are generally shipped around the first few weeks of the month, so a new release should happen in early April, this new version will include all code in 1.14.1 azure-core plus any additional enhancements added in March.

cc: @vcolin7

@Jeevankumar555
Copy link
Author

Jeevankumar555 commented Mar 24, 2021

@vcolin7 , @alzimmermsft : I face a new run time exception with the combination of spring boot 2.4.2 and azure-security-keyvault-secrets 4.2.6 + overriding azure-core to 1.14.1 dependencies that it is not able to connect to my keyvault uri , as per suggestion of my colleague he used spring boot 2.3.9-Release that seems to work for him. I am going to try that. I have created an MSI for my app service and assigned all the vault permissions to it and try to use SecretClient in retrieving the scerets.

java.lang.RuntimeException: Max retries 3 times exceeded. Error Details: failed to resolve 'myKevault.vault.azure.net' after 2 queries at com.azure.core.http.policy.RetryPolicy.lambda$attemptAsync$1(RetryPolicy.java:127) at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(MonoFlatMap.java:172) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onError(FluxHide.java:141) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(MonoFlatMap.java:172) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onError(FluxHide.java:141) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(MonoFlatMap.java:172) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onError(FluxHide.java:141) at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onError(FluxMapFuseable.java:140) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onError(FluxHide.java:141) at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onError(FluxMapFuseable.java:140) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onError(FluxHide.java:141) at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onError(FluxPeekFuseable.java:234) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onError(FluxHide.java:141) at reactor.core.publisher.MonoSingle$SingleSubscriber.onError(MonoSingle.java:155) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onError(MonoFlatMapMany.java:204) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.SerializedSubscriber.onError(SerializedSubscriber.java:124) at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.whenError(FluxRetryWhen.java:224) at reactor.core.publisher.FluxRetryWhen$RetryWhenOtherSubscriber.onError(FluxRetryWhen.java:273) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.innerError(FluxConcatMap.java:308) at reactor.core.publisher.FluxConcatMap$ConcatMapInner.onError(FluxConcatMap.java:872) at reactor.core.publisher.Operators.error(Operators.java:196) at reactor.core.publisher.MonoError.subscribe(MonoError.java:52) at reactor.core.publisher.Mono.subscribe(Mono.java:4046) at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:448) at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onNext(FluxConcatMap.java:250) at reactor.core.publisher.EmitterProcessor.drain(EmitterProcessor.java:491) at reactor.core.publisher.EmitterProcessor.tryEmitNext(EmitterProcessor.java:299) at reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:97) at reactor.core.publisher.InternalManySink.emitNext(InternalManySink.java:27) at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onError(FluxRetryWhen.java:189) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:189) at reactor.netty.http.client.HttpClientConnect$MonoHttpConnect$ClientTransportSubscriber.onError(HttpClientConnect.java:306) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:189) at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.onError(DefaultPooledConnectionProvider.java:166) at reactor.netty.internal.shaded.reactor.pool.AbstractPool$Borrower.fail(AbstractPool.java:427) at reactor.netty.internal.shaded.reactor.pool.SimpleDequePool.lambda$drainLoop$5(SimpleDequePool.java:309) at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onError(FluxDoOnEach.java:186) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:189) at reactor.netty.resources.DefaultPooledConnectionProvider$PooledConnectionAllocator$PooledConnectionInitializer.onError(DefaultPooledConnectionProvider.java:565) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.lambda$onError$1(TracingSubscriber.java:48) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.withActiveSpan(TracingSubscriber.java:64) at io.opentelemetry.javaagent.shaded.instrumentation.reactor.TracingSubscriber.onError(TracingSubscriber.java:48) at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:192) at reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:259) at reactor.netty.transport.TransportConnector$MonoChannelPromise.tryFailure(TransportConnector.java:464) at reactor.netty.transport.TransportConnector.lambda$doResolveAndConnect$6(TransportConnector.java:271) at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:552) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) at io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:109) at io.netty.resolver.InetSocketAddressResolver$1.operationComplete(InetSocketAddressResolver.java:62) at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571) at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) at io.netty.resolver.dns.DnsNameResolver.tryFailure(DnsNameResolver.java:936) at io.netty.resolver.dns.DnsNameResolver.access$500(DnsNameResolver.java:90) at io.netty.resolver.dns.DnsNameResolver$5.operationComplete(DnsNameResolver.java:956) at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:552) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) at io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1021) at io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(DnsResolveContext.java:966) at io.netty.resolver.dns.DnsResolveContext.query(DnsResolveContext.java:414) at io.netty.resolver.dns.DnsResolveContext.access$600(DnsResolveContext.java:63) at io.netty.resolver.dns.DnsResolveContext$2.operationComplete(DnsResolveContext.java:463) at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571) at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) at io.netty.resolver.dns.DnsQueryContext.tryFailure(DnsQueryContext.java:225) at io.netty.resolver.dns.DnsQueryContext$4.run(DnsQueryContext.java:177) at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:170) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:834) Suppressed: java.lang.Exception: #block terminated with an error at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99) at reactor.core.publisher.Mono.block(Mono.java:1703) at com.azure.security.keyvault.secrets.SecretClient.getSecretWithResponse(SecretClient.java:173) at com.azure.security.keyvault.secrets.SecretClient.getSecret(SecretClient.java:152) at com.myorg.mgr.employeesessionmgmt.utils.KeyVaultUtils.getSecretValue(KeyVaultUtils.java:33) at com.myorg.mgr.employeesessionmgmt.utils.KeyVaultUtils$$FastClassBySpringCGLIB$$e9d57fe1.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89) at com.myorg.mgr.employeesessionmgmt.loggerhandler.LoggerAdvice.applicationLogger(LoggerAdvice.java:49) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692) at com.myorg.mgr.employeesessionmgmt.utils.KeyVaultUtils$$EnhancerBySpringCGLIB$$9681f95b.getSecretValue(<generated>) at com.myorg.mgr.employeesessionmgmt.validator.Validator.validateInputs(Validator.java:70) at com.myorg.mgr.employeesessionmgmt.validator.Validator$$FastClassBySpringCGLIB$$dcf74cb6.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89) at com.myorg.mgr.employeesessionmgmt.loggerhandler.LoggerAdvice.applicationLogger(LoggerAdvice.java:49) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692) at com.myorg.mgr.employeesessionmgmt.validator.Validator$$EnhancerBySpringCGLIB$$65d813d8.validateInputs(<generated>) at com.myorg.mgr.employeesessionmgmt.controller.EmployeeSessionResource.getLoginDetailsForPingFederate(EmployeeSessionResource.java:69) at com.myorg.mgr.employeesessionmgmt.controller.EmployeeSessionResource$$FastClassBySpringCGLIB$$62c4c2a4.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.aspectj.AspectJAfterThrowingAdvice.invoke(AspectJAfterThrowingAdvice.java:64) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89) at com.myorg.mgr.employeesessionmgmt.loggerhandler.LoggerAdvice.applicationLogger(LoggerAdvice.java:49) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692) at com.myorg.mgr.employeesessionmgmt.controller.EmployeeSessionResource$$EnhancerBySpringCGLIB$$cf6a9a62.getLoginDetailsForPingFederate(<generated>) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:197) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:141) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:894) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1060) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:962) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) at javax.servlet.http.HttpServlet.service(HttpServlet.java:626) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at io.opentelemetry.javaagent.instrumentation.springwebmvc.HandlerMappingResourceNameFilter.doFilter(HandlerMappingResourceNameFilter.java:59) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.microsoft.applicationinsights.web.internal.WebRequestTrackingFilter.doFilter(WebRequestTrackingFilter.java) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:888) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1597) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ... 1 more Caused by: java.net.UnknownHostException: failed to resolve 'myKevault.vault.azure.net' after 2 queries at io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1013) ... 22 more Caused by: io.netty.resolver.dns.DnsNameResolverTimeoutException: [/8.8.4.4:53] query via UDP timed out after 5000 milliseconds (no stack trace available)

@alzimmermsft alzimmermsft added the dependency-issue Issue that is caused by dependency conflicts label Mar 24, 2021
@Jeevankumar555
Copy link
Author

tested with 2.3.9.RELEASE of spring boot with 4.2.3 version of azure-security-keyvault-secrets , then I am able to connect to keyvault from my app service that has manged identity, but unfortunately I am locked with this version of spring boot now

@vcolin7 vcolin7 added azure-spring-keyvault Spring keyvault related issues. and removed azure-spring-keyvault Spring keyvault related issues. labels Mar 26, 2021
@vcolin7
Copy link
Member

vcolin7 commented Mar 26, 2021

Just out of curiosity, have you tried using our Spring Boot Starter for Key Vault?

@Jeevankumar555
Copy link
Author

nope, will try with that too

@Jeevankumar555
Copy link
Author

Jeevankumar555 commented Mar 26, 2021

I tried with below combination due to the dependency bug I face with using 3.3.0 version of azure-spring-boot-starter-keyvault-secrets that carries 4.2.6 of azure-security-keyvault-secrets. But then no luck in connecting to Keyvault so I fall back again to use 2.3.9.RELEASE of spring boot with 4.2.3 version of azure-security-keyvault-secrets ( bug I see is, it reties to connect 3 times and fails which I have posted above the stack trace).

    <dependency>
      <groupId>com.azure.spring</groupId>
      <artifactId>azure-spring-boot-starter-keyvault-secrets</artifactId>
      <version>${azure-spring-boot-starter-keyvault-secrets.version}</version>
      <exclusions>
        <exclusion>
          <groupId>com.azure</groupId>
          <artifactId>azure-security-keyvault-secrets</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.azure</groupId>
      <artifactId>azure-security-keyvault-secrets</artifactId>
      <version>4.2.6</version>
      <exclusions>
        <exclusion>
          <groupId>com.azure</groupId>
          <artifactId>azure-core</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.azure</groupId>
      <artifactId>azure-core</artifactId>
      <version>1.14.1</version>
    </dependency>
    

@alzimmermsft
Copy link
Member

Adding the HttpClient tag as this investigation will need to look into DNS resolution of the KeyVault host as that is the root in the exception posted earlier.

java.net.UnknownHostException: failed to resolve 'myKevault.vault.azure.net' after 2 queries at
io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1013) ... 22 more Caused by:
io.netty.resolver.dns.DnsNameResolverTimeoutException: [/8.8.4.4:53] query via UDP timed out after 5000 milliseconds 
(no stack trace available)

@vcolin7 vcolin7 added this to the [2021] April milestone Mar 26, 2021
@vcolin7
Copy link
Member

vcolin7 commented Mar 26, 2021

As a side note, and this is me not knowing your exact use case, but the Spring Boot Starter for Key Vault has compatible dependencies for Spring Boot and Key Vault Secrets and allows you to access all your secrets on a given Key Vault as if they were properties of your Spring Boot application. So instead of manually instantiating a SecretClient to retrieve what your secrets, they are loaded in-memory when the application is started so you can retrieve them as shown here. Would this help with your use case?

@Jeevankumar555
Copy link
Author

Jeevankumar555 commented Mar 27, 2021

Hi,

  1. Initial issue was with dependencies using azure-security-keyvault-secrets 4.2.6 since it comes up with 1.14.0 of azure-core that has run time class not found exceptions , I had to use azure-core 1.14.1 to resolve that as per @alzimmermsft , hence the issue got tagged as dependency issue first.

  2. Then also I realized after resolving dependency issues I am facing compatibility issue with spring boot in pulling secrets dynamically from key vault
    failed to resolve 'myKevault.vault.azure.net' after 2 queries at using SecretClient ,
    and the fix for that is using spring boot version 2.39-RELEASE or lesser versions of it.
    As per suggestion from @vcolin7 : I tried also using azure-spring-boot-starter-keyvault-secrets 3.3.0 but still not luck in pulling secrets with SecretClient.

My use case here is not to lookup for fixed key labels from keyvault and bind it to an instance variable of class but to lookup dynamically based on some values I get from my incoming HTTP request param.

I may have reported/discussed two different issues in this issue thread as and when I faced it, but if needed I can make two issues separately to avoid confusion.

@vcolin7 vcolin7 modified the milestones: [2021] April, [2021] June May 13, 2021
@vcolin7
Copy link
Member

vcolin7 commented Jun 7, 2021

Hi @Jeevankumar555,

Would you mind closing this issue as the original problem has been resolved and then opening a new one for problem No. 2 in your list? That is, if you are still facing it. Thanks!

@vcolin7 vcolin7 closed this as completed Jul 13, 2021
azure-sdk pushed a commit to azure-sdk/azure-sdk-for-java that referenced this issue Aug 31, 2022
add x-ms-identifiers for hdinsight (Azure#19957)

* add x-ms-identifiers for hdinsight

* Update cluster.json
@github-actions github-actions bot locked and limited conversation to collaborators Apr 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported Issues that are reported by GitHub users external to the Azure organization. dependency-issue Issue that is caused by dependency conflicts HttpClient KeyVault needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team
Projects
None yet
Development

No branches or pull requests

4 participants