The onstatus method of webclient causes a memory leak. [SPR-17473]

[spring-projects-issues]

KimSeongIl opened SPR-17473 and commented

 

private WebClient webClient;

    private final static int DEFAULT_CONNECT_TIMEOUT = 900;
    private final static int DEFAULT_SOCKET_TIMEOUT = 900;

    @PostConstruct
    public void initialize() {
        ReactorClientHttpConnector connector = new ReactorClientHttpConnector(options ->
                options.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, DEFAULT_CONNECT_TIMEOUT)
                        .poolResources(PoolResources.fixed("httpPool", 1000))
                        .afterNettyContextInit(ctx -> {
                            ctx.addHandlerLast(new ReadTimeoutHandler(DEFAULT_SOCKET_TIMEOUT, TimeUnit.MILLISECONDS));
                        }));

        webClient = WebClient.builder()
                .clientConnector(connector)
                .build();

    }

@GetMapping("/test")
    public Mono<String> webclientTest() {

        return webClient
                .get()
                .uri("http://testurl.com/response500test")
                .retrieve()
                .onStatus(httpStatus -> httpStatus != HttpStatus.OK, clientResponse -> Mono.error(new RuntimeException(String.format("api status: %s", clientResponse.statusCode()))))
                .bodyToMono(String.class);
    }

The above source code called an API that responded to 500 errors with 600 tps.

 

io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 byte(s) of direct memory (used: 8573157383, max: 8589934592)

The onstatus method of webclient causes a memory leak.

 

!스크린샷 2018-11-07 오후 2.01.06.png!

 

In the photo attached above
We used the onStatus method from 13:40 to 13:48,
We removed the onStatus method from 13:55 to 14:00.
Is this a bug?
Why does the onStatus method cause a memory leak?

I used SpringBoot version 2.0.4 and upgraded to 2.1.0, but I still have a problem.

---

Affects: 5.0.8, 5.1.2

Attachments:

• 스크린샷 2018-11-07 오후 2.01.06.png (36.31 kB)

Issue Links:

• WebClient logs "Only one connection receive subscriber allowed" when response status is an error [SPR-17564] #22096 WebClient logs "Only one connection receive subscriber allowed" when response status is an error

Referenced from: commits f73a522, c187cb2

Backported to: 5.0.11

[spring-projects-issues]

Rossen Stoyanchev commented

Technically, you need to always consume the body of the response, or call bodyToMono(Void.class) if you don't expect or don't care for any content that may be in the response. This is explained in the Javadoc of ClientResponse and the exchange() method, and is also mentioned in the reference. It is not explicitly mentioned on the retrieve() method because then you have to choose a bodyToXxx method, but that's not the case if using the onStatus hook as you have found out.

In short technically this is not a bug, since the application is generally expected to consume the body. However unlike the case with exchange() where the application gets the response and the framework does not have a good place to ensure the response is consumed or released, in the case of the onStatus hook, we can do that after the error function is applied. So I'm turning this into an improvement. Meanwhile please use clientResponse.bodyToMono(Void.class).then(Mono.error(new RuntimeException(...))).

[spring-projects-issues]

KimSeongIl commented

Rossen Stoyanchev

Thank you for the reply.

there was no memory leak after the code was modified.

i expect version 5.1.3 :)

[spring-projects-issues]

Rossen Stoyanchev commented

This should be now fixed if you want to give it a try to 5.1.3 snapshots once the current build completes.