mongodb plugin with prometheus_client output wont show member_status #2928

freeseacher · 2017-06-15T23:02:31Z

Bug report

Relevant telegraf.conf:

[[inputs.mongodb]]
  ## An array of URI to gather stats about. Specify an ip or hostname
  ## with optional port add password. ie,
  ##   mongodb://user:[email protected]:27017,
  ##   mongodb://10.10.3.33:18832,
  ##   10.0.0.1:10000, etc.
  #
  # we use 127.0.0.1 hre because of it will be launched on each mongodb server
  #
  servers = ["mongodb://root:[email protected]:27017/admin"]
  gather_perdb_stats = true

# Configuration for the Prometheus client to spawn
[[outputs.prometheus_client]]
  ## Address to listen on
  listen = "0.0.0.0:9126"
  expiration_interval = "10s"

System info:

Telegraf v1.3.2 (git: release-1.3 0f419e9)

Steps to reproduce:

run telegraf --config mongodb.conf --test

* Plugin: inputs.mongodb, Collection 1
* Plugin: inputs.mongodb, Collection 2
> mongodb,hostname=127.0.0.1:27017,host=hostname active_writes=1i,vsize_megabytes=11588i,repl_updates_per_sec=1240i,repl_lag=0i,jumbo_chunks=0i,updates_per_sec=0i,queries_per_sec=940i,state="SECONDARY",percent_cache_dirty=0.7,repl_getmores_per_sec=0i,member_status="SEC",flushes_per_sec=0i,open_connections=1402i,ttl_passes_per_sec=0i,active_reads=0i,queued_reads=0i,repl_queries_per_sec=0i,deletes_per_sec=0i,net_in_bytes=267791i,queued_writes=0i,repl_commands_per_sec=0i,repl_inserts_per_sec=0i,percent_cache_used=6.5,resident_megabytes=8793i,inserts_per_sec=0i,ttl_deletes_per_sec=0i,repl_deletes_per_sec=0i,commands_per_sec=9i,net_out_bytes=1632198i,getmores_per_sec=2i 1497567411000000000
> mongodb_db_stats,host=hostname,db_name=admin,hostname=127.0.0.1:27017 num_extents=0i,indexes=3i,type="db_stat",collections=2i,objects=6i,ok=1i,storage_size=53248i,index_size=65536i,avg_obj_size=212,data_size=1272i 1497567411000000000
> mongodb_db_stats,hostname=127.0.0.1:27017,host=hostname,db_name=mydb indexes=501i,index_size=10194022400i,type="db_stat",num_extents=0i,ok=1i,avg_obj_size=574.4335489103034,data_size=59652610236i,storage_size=21825556480i,collections=187i,objects=103845972i 1497567411000000000

run curl http://0.0.0.0:9126/metrics

mongodb_active_reads{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_active_writes Telegraf collected metric
# TYPE mongodb_active_writes untyped
mongodb_active_writes{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_commands_per_sec Telegraf collected metric
# TYPE mongodb_commands_per_sec untyped
mongodb_commands_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 21
# HELP mongodb_db_stats_avg_obj_size Telegraf collected metric
# TYPE mongodb_db_stats_avg_obj_size untyped
mongodb_db_stats_avg_obj_size{db_name="mydb",dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 574.4350308437449
# HELP mongodb_db_stats_collections Telegraf collected metric
# TYPE mongodb_db_stats_collections untyped
mongodb_db_stats_collections{db_name="mydb",dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 187
# HELP mongodb_db_stats_data_size Telegraf collected metric
# TYPE mongodb_db_stats_data_size untyped
mongodb_db_stats_data_size{db_name="mydb",dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 5.9652881888e+10
# HELP mongodb_db_stats_index_size Telegraf collected metric
# TYPE mongodb_db_stats_index_size untyped
mongodb_db_stats_index_size{db_name="mydb",dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 1.0194034688e+10
# HELP mongodb_db_stats_indexes Telegraf collected metric
# TYPE mongodb_db_stats_indexes untyped
mongodb_db_stats_indexes{db_name="mydb",dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 501
# HELP mongodb_db_stats_num_extents Telegraf collected metric
# TYPE mongodb_db_stats_num_extents untyped
mongodb_db_stats_num_extents{db_name="mydb",dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_db_stats_objects Telegraf collected metric
# TYPE mongodb_db_stats_objects untyped
mongodb_db_stats_objects{db_name="mydb",dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 1.03846177e+08
# HELP mongodb_db_stats_ok Telegraf collected metric
# TYPE mongodb_db_stats_ok untyped
mongodb_db_stats_ok{db_name="mydb",dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 1
# HELP mongodb_db_stats_storage_size Telegraf collected metric
# TYPE mongodb_db_stats_storage_size untyped
mongodb_db_stats_storage_size{db_name="mydb",dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 2.1825572864e+10
# HELP mongodb_deletes_per_sec Telegraf collected metric
# TYPE mongodb_deletes_per_sec untyped
mongodb_deletes_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_flushes_per_sec Telegraf collected metric
# TYPE mongodb_flushes_per_sec untyped
mongodb_flushes_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_getmores_per_sec Telegraf collected metric
# TYPE mongodb_getmores_per_sec untyped
mongodb_getmores_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 2
# HELP mongodb_inserts_per_sec Telegraf collected metric
# TYPE mongodb_inserts_per_sec untyped
mongodb_inserts_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_jumbo_chunks Telegraf collected metric
# TYPE mongodb_jumbo_chunks untyped
mongodb_jumbo_chunks{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_net_in_bytes Telegraf collected metric
# TYPE mongodb_net_in_bytes untyped
mongodb_net_in_bytes{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 554257
# HELP mongodb_net_out_bytes Telegraf collected metric
# TYPE mongodb_net_out_bytes untyped
mongodb_net_out_bytes{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 3.244973e+06
# HELP mongodb_open_connections Telegraf collected metric
# TYPE mongodb_open_connections untyped
mongodb_open_connections{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 1404
# HELP mongodb_percent_cache_dirty Telegraf collected metric
# TYPE mongodb_percent_cache_dirty untyped
mongodb_percent_cache_dirty{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0.6
# HELP mongodb_percent_cache_used Telegraf collected metric
# TYPE mongodb_percent_cache_used untyped
mongodb_percent_cache_used{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 6.6
# HELP mongodb_queries_per_sec Telegraf collected metric
# TYPE mongodb_queries_per_sec untyped
mongodb_queries_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 1953
# HELP mongodb_queued_reads Telegraf collected metric
# TYPE mongodb_queued_reads untyped
mongodb_queued_reads{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_queued_writes Telegraf collected metric
# TYPE mongodb_queued_writes untyped
mongodb_queued_writes{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_repl_commands_per_sec Telegraf collected metric
# TYPE mongodb_repl_commands_per_sec untyped
mongodb_repl_commands_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_repl_deletes_per_sec Telegraf collected metric
# TYPE mongodb_repl_deletes_per_sec untyped
mongodb_repl_deletes_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 1
# HELP mongodb_repl_getmores_per_sec Telegraf collected metric
# TYPE mongodb_repl_getmores_per_sec untyped
mongodb_repl_getmores_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_repl_inserts_per_sec Telegraf collected metric
# TYPE mongodb_repl_inserts_per_sec untyped
mongodb_repl_inserts_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 2
# HELP mongodb_repl_lag Telegraf collected metric
# TYPE mongodb_repl_lag untyped
mongodb_repl_lag{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_repl_queries_per_sec Telegraf collected metric
# TYPE mongodb_repl_queries_per_sec untyped
mongodb_repl_queries_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_repl_updates_per_sec Telegraf collected metric
# TYPE mongodb_repl_updates_per_sec untyped
mongodb_repl_updates_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 2478
# HELP mongodb_resident_megabytes Telegraf collected metric
# TYPE mongodb_resident_megabytes untyped
mongodb_resident_megabytes{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 8926
# HELP mongodb_ttl_deletes_per_sec Telegraf collected metric
# TYPE mongodb_ttl_deletes_per_sec untyped
mongodb_ttl_deletes_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_ttl_passes_per_sec Telegraf collected metric
# TYPE mongodb_ttl_passes_per_sec untyped
mongodb_ttl_passes_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_updates_per_sec Telegraf collected metric
# TYPE mongodb_updates_per_sec untyped
mongodb_updates_per_sec{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 0
# HELP mongodb_vsize_megabytes Telegraf collected metric
# TYPE mongodb_vsize_megabytes untyped
mongodb_vsize_megabytes{dc="DC",env="prod",host="hostname",hostname="127.0.0.1:27017"} 11720

Expected behavior:

state="SECONDARY" and member_status="SEC" will be shown as tags

Actual behavior:

none of states is shown

The text was updated successfully, but these errors were encountered:

danielnelson · 2017-06-15T23:16:51Z

This is because these are string fields, prometheus has only float values. Are you thinking we should add string fields as labels?

freeseacher · 2017-06-16T10:19:32Z

in my opinion that info are tags for prometheus. and later in query i want to search them as a tags not as values

danielnelson · 2017-06-16T18:06:05Z

If we move string fields to labels, we need to consider it for all metrics. Would love to hear from other prometheus users too.

freeseacher · 2017-06-16T19:08:30Z

probably there we have 3 options here

ignore string values. like now.
convert string values to maps and send them to most outputs as strings and to some as ints
make strings values tags.

to make a decidion we have to understand which values are currently lost in case of luck of convertion.

danielnelson · 2017-06-16T19:13:34Z

convert string values to maps and send them to most outputs as strings and to some as ints

I'm not sure what you mean by this, can you explain in a little more detail?

danielnelson · 2017-06-16T19:15:13Z

One more idea that I think is being worked on, a processor that can promote a field to tag and vice versa.

freeseacher · 2017-06-16T19:29:03Z

telegraf/plugins/inputs/mongodb/mongostat.go

Line 559 in e7f9db2

if newStat.Repl != nil {

looks a little weird for me.
probably should be replaced with something like map[int]string

freeseacher · 2017-09-17T03:36:01Z

any news?

freeseacher · 2017-10-18T00:53:27Z

ping?

danielnelson · 2017-10-18T00:59:55Z

Hey @freeseacher, I'm actually in this code right now, I will go ahead and add the string fields as labels and we can see how it ends up looking.

freeseacher · 2017-10-18T01:07:16Z

oh! cool. :)

sbadia mentioned this issue Jun 23, 2017

elasticsearch plugin with prometheus_client output wont show cluster_health status #2957

Closed

danielnelson added the area/prometheus label Aug 24, 2017

danielnelson mentioned this issue Oct 18, 2017

Use labels in prometheus output for string fields #3350

Merged

3 tasks

danielnelson closed this as completed in #3350 Oct 19, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mongodb plugin with prometheus_client output wont show member_status #2928

mongodb plugin with prometheus_client output wont show member_status #2928

freeseacher commented Jun 15, 2017 •

edited

Loading

danielnelson commented Jun 15, 2017

freeseacher commented Jun 16, 2017

danielnelson commented Jun 16, 2017

freeseacher commented Jun 16, 2017

danielnelson commented Jun 16, 2017

danielnelson commented Jun 16, 2017

freeseacher commented Jun 16, 2017

freeseacher commented Sep 17, 2017

freeseacher commented Oct 18, 2017

danielnelson commented Oct 18, 2017

freeseacher commented Oct 18, 2017

mongodb plugin with prometheus_client output wont show member_status #2928

mongodb plugin with prometheus_client output wont show member_status #2928

Comments

freeseacher commented Jun 15, 2017 • edited Loading

Bug report

Relevant telegraf.conf:

System info:

Steps to reproduce:

Expected behavior:

Actual behavior:

danielnelson commented Jun 15, 2017

freeseacher commented Jun 16, 2017

danielnelson commented Jun 16, 2017

freeseacher commented Jun 16, 2017

danielnelson commented Jun 16, 2017

danielnelson commented Jun 16, 2017

freeseacher commented Jun 16, 2017

freeseacher commented Sep 17, 2017

freeseacher commented Oct 18, 2017

danielnelson commented Oct 18, 2017

freeseacher commented Oct 18, 2017

freeseacher commented Jun 15, 2017 •

edited

Loading