Skip to content
This repository has been archived by the owner on Mar 17, 2024. It is now read-only.

Lags not reported for topics with no active member #290

Open
andypp opened this issue Dec 1, 2021 · 2 comments · May be fixed by Gor027/kafka-lag-exporter#1
Open

Lags not reported for topics with no active member #290

andypp opened this issue Dec 1, 2021 · 2 comments · May be fixed by Gor027/kafka-lag-exporter#1

Comments

@andypp
Copy link

andypp commented Dec 1, 2021

Describe the bug
As per #66, consumer groups with no active members also export the lags for all the topics. However in case where a group consumes from multiple topics and it has active members in only some topic, no metrics are reported for the topics without active member.

when group has no active members

$ kafka-consumer-groups --bootstrap-server $BROKERS --describe --group my-group

Consumer group 'my-group' has no active members.

GROUP    TOPIC   PARTITION  CURRENT-OFFSET  LOG-END-OFFSET LAG CONSUMER-ID  HOST  CLIENT-ID
my-group topic-1 0          100             100            0   -            -     -
my-group topic-2 0          100             100            0   -            -     -
$ curl -s http://localhost:8000 | grep my-group | grep kafka_consumergroup_group_lag
kafka_consumergroup_group_lag{cluster_name="test", group="my-group", topic="topic-1", partition="0", member_host="unknown", consumer_id="unknown", client_id="unknown"} 0.0
kafka_consumergroup_group_lag{cluster_name="test", group="my-group", topic="topic-2", partition="0", member_host="unknown", consumer_id="unknown", client_id="unknown"} 0.0

when group has some active members

$ kafka-consumer-groups --bootstrap-server $BROKERS --describe --group my-group


GROUP    TOPIC   PARTITION  CURRENT-OFFSET  LOG-END-OFFSET LAG CONSUMER-ID     HOST        CLIENT-ID
my-group topic-1 0          100             100            0   my-consumer-id  /10.11.0.1  rdkafka
my-group topic-2 0          100             100            0                   -           -
$ curl -s http://localhost:8000 | grep my-group | grep kafka_consumergroup_group_lag
kafka_consumergroup_group_lag{cluster_name="test", group="my-group", topic="topic-1", partition="0", member_host="/10.11.0.1 ", consumer_id="my-consumer-id", client_id="rdkafka"} 0.0

To Reproduce
application.conf

kafka-lag-exporter {
  port = 8000
  reporters.prometheus.port = 8000
  clusters = [
    {
      name = "test"
      bootstrap-brokers = "localhost:29092"
    }
  ]
}

Environment

  • Version: 0.6.8
  • Version of Apache Kafka cluster: Confluent-kafka-2.12-5.3.2
  • Run with Standalone
@theanotherwise
Copy link

any idea? i see same problem, metrics are empty

@piif
Copy link

piif commented Oct 7, 2022

Hi,
I just got the same issue.
I can't help on code, but let me know if there are some tests to do to help analysis

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants