MQTT connected, But telegraf is not to write data to influxdb

[bigdatasunil]

I am also facing same issue, telegraf unable to consumer messages from MQTT
Below is the config file i am using
[agent]
debug = true
[[inputs.mqtt_consumer]]

servers = ["tcp://localhost:1883"]
topics = [
"telegraf/#",
]
qos = 1
username = "admin"
password = "2021"
insecure_skip_verify = true
data_format = "influx"

[[outputs.influxdb]]

urls = ["http://127.0.0.1:8086"]
database = "test"

[[outputs.file]]
files = ["stdout"]

##################### OUTPUT ##################################
2020-06-24T14:03:41Z I! Starting Telegraf 1.14.4
2020-06-24T14:03:41Z I! Loaded inputs: mqtt_consumer
2020-06-24T14:03:41Z I! Loaded aggregators:
2020-06-24T14:03:41Z I! Loaded processors:
2020-06-24T14:03:41Z I! Loaded outputs: influxdb file
2020-06-24T14:03:41Z I! Tags enabled: host=ubuntu
2020-06-24T14:03:41Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"ubuntu", Flush Interval:10s
2020-06-24T14:03:41Z D! [agent] Initializing plugins
2020-06-24T14:03:41Z D! [agent] Connecting outputs
2020-06-24T14:03:41Z D! [agent] Attempting connection to [outputs.influxdb]
2020-06-24T14:03:41Z D! [agent] Successfully connected to outputs.influxdb
2020-06-24T14:03:41Z D! [agent] Attempting connection to [outputs.file]
2020-06-24T14:03:41Z D! [agent] Successfully connected to outputs.file
2020-06-24T14:03:41Z D! [agent] Starting service inputs
2020-06-24T14:03:41Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
2020-06-24T14:03:41Z D! [inputs.mqtt_consumer] Session found [tcp://localhost:1883]
2020-06-24T14:04:00Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-06-24T14:04:00Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-06-24T14:04:10Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-06-24T14:04:10Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-06-24T14:04:20Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-06-24T14:04:20Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-06-24T14:04:30Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-06-24T14:04:30Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics

Could anyone please suggest me what is the issue here?

Input string to mqtt topic is:
cpu,cpu=cpu-total,dc=us-east-1,host=ubuntu usage_irq=0,usage_softirq=1.1858726475895787,usage_steal=0,usage_guest_nice=0,usage_system=9.409641660221729,usage_nice=0,usage_iowait=0.025779840164989846,usage_guest=0,usage_user=4.305233307553291,usage_idle=85.07347254446832 1592835240000000000

Originally posted by @bigdatasunil in #4580 (comment)

[ssoroka]

looks to me like the topics don't match, so you're seeing zero messages from the input. What topics are you publishing messages under?

[bigdatasunil]

Thanks for quick reply.
I am publishing messages to "telegraf/test topic.
is this anything wrong with this??

[bigdatasunil]

Could you please share mqtt_consumer config which matches all topics ??

[bigdatasunil]

I have tried like this, still not consuming any messages

Topics that will be subscribed to.

topics = [
"telegraf/#",
"#",
]

[ssoroka]

I'm not super familiar with MQTT. From a search, it's not clear to me if "#" should work. I'll see if I can replicate the problem.

[bigdatasunil]

i m using ActiveMQ (artemis) as mqtt broker. that could be problem??

[ssoroka]

I don't know. Can you try connecting directly instead of through the broker?

[danielnelson]

@bigdatasunil Can you check the ActiveMQ logs when you are connecting?

[Tautcius]

Same problem here, same configuration, was working before...

[ssoroka]

@Tautcius as in, was working on a previous version and after upgrading it stopped working? Or do you mean something else?

[thomasdannenmuller]

I faced the same issue as well, but telegraf version was 1.9 when the problem occured. It happened after an influxDB crash, whose lead telegraf to crash too. Docker restarted them, all the connections where restored but no more logging. And absolutely no clue in telegraf logs

For the record we are using QoS 1 with persistent-session, and mosquitto 1.6

I've tried to recreate the container, no go. Then updated the telegraf to 1.14, and it fixed the issue. Subscription was still alive in the borker, and telegraf have recovered all the missed messages just fine. 😅

So, if this is the same problem, it do not seems to be exclusive to 1.14.x version. However the common denominator could be the wildcard subscription.

[sjwang90]

@bigdatasunil @Tautcius Is this still an issue on the latest version of Telegraf?

[bigdatasunil]

I don't know, because we are not using telegraf anymore. We have written our own connector for this.

…

On Mon, Nov 16, 2020 at 7:47 PM Samantha Wang ***@***.***> wrote: @bigdatasunil <https://github.com/bigdatasunil> @Tautcius <https://github.com/Tautcius> Is this still an issue on the latest version of Telegraf? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7735 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA7EQGZ4CZNGRTJ6L3CZNBTSQFXUBANCNFSM4OHFPVEA> .

[domwoe]

@sjwang90 I tested it with 1.16.0-1 about a week ago and it worked.

[pepijnverburg]

I see that your telegraf MQTT consumer data_format is still set to influx. Are you sure you are sending valid influx formatted data in the MQTT message?

You can change it to json and test it with a message {"test": 24} to be sure. Note: the key value pairs in the JSON string should have numeric values to be classified as metrics.

[sjwang90]

Thanks @domwoe and @pepijnverburg.

@bigdatasunil if you'd like to try out the MQTT plugin again check out @pepijnverburg's comments.

Closing this issue, feel free to re-open if issue persists.