Destroying kafka triggers 100 ms sleep

[Tezd]

Description

Destroying rdkafka adds 100 ms latency due to rdkafka_broker.c line of code rd_usleep(100*1000/*100ms*/, &rk->rk_terminate); Those changes were made in 1c044abc59063fab56001d016b44d54374a67b2a commit with message about memory leak. Is there any way of avoiding these 100 ms sleep or I am hitting it due to some misconfiguration?

How to reproduce

1. Bring up environment with docker and docker-compose as follows:

version: "3.6"

services:
  zookeeper:
    image: zookeeper:3.4.11

  kafka:
    image: wurstmeister/kafka:1.0.0
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_BROKER_ID: 42
    depends_on:
      - zookeeper
    ports:
      - 9092:9092

2. Add rd_rkb_dbg(rkb, GENERIC, "SLEEP", "Sleeping for 100 ms"); before https://github.com/edenhill/librdkafka/blob/master/src/rdkafka_broker.c#L3562 to verify that we are hitting that condition.
3. modify examples/rdkafka_simple_producer.c as follows:
   rdkafka_simple_producer.c
4. cd ${LIBRDKAFKA_DIR}/examples && make rdkafka_simple_producer
5. time ./rdkafka_simple_producer 127.0.0.1 test
6. Verify that it sleeps for 100 ms

Checklist

Please provide the following information:

• librdkafka version (release number or git tag): v.0.11.5
• Apache Kafka version: 1.0.0
• librdkafka client configuration: debug=all
• Operating system: Ubuntu 16.04.1
• Provide logs (with debug=.. as necessary) from librdkafka
  librdkafka.log
• Provide broker log excerpts (they are empty for test run but will provide startup logs)
  brokers.log
• Critical issue

[rnpridgeon]

At a glance this does not look to be avoidable with configuration changes. I'll take a deeper look and get back to you though.

[edenhill]

Thank you for a great report!

I don't think that sleep is necessary any more, it was there to protect from a tight reconnect loop on protocol errors, but we have reconnect.backoff.jitter.ms to slow things down instead.

Will fix.