Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Gauge for total_fields setting on a index #411

Merged
merged 15 commits into from
Jun 4, 2021
Merged

Add Gauge for total_fields setting on a index #411

merged 15 commits into from
Jun 4, 2021

Conversation

repl-mike-roest
Copy link
Contributor

Includes #386 changes as well

msodi and others added 10 commits May 31, 2021 08:28
@msodi @repl-mike-roest
prometheus-community#257: Add gauge for total fields mapped in an index
9ff2e26 
Signed-off-by: Mike Roest <[email protected]>
@repl-mike-roest
Add gauge for field count per index
57d50e4 
Signed-off-by: Mike Roest <[email protected]>
@repl-mike-roest
Fix auto refactor fail
fa2fbde 
Signed-off-by: Mike Roest <[email protected]>
@repl-mike-roest
Fix index request error log
edff21a 
Signed-off-by: Mike Roest <[email protected]>
@repl-mike-roest
Fix index request error log
66eeef0 
Signed-off-by: Mike Roest <[email protected]>
@repl-mike-roest
Use json.Unmarshal to address pr feedback
5e1e4dc 
Signed-off-by: Mike Roest <[email protected]>
@repl-mike-roest
add total fields
ce38e43 
Signed-off-by: Mike Roest <[email protected]>
@repl-mike-roest
Add tests and support index settings
16af3dc 
Signed-off-by: Mike Roest <[email protected]>
@repl-mike-roest
update readme with new metric
60f8a34 
Signed-off-by: Mike Roest <[email protected]>
@repl-mike-roest
add file licenses
55aaa2e 
Signed-off-by: Mike Roest <[email protected]>
@repl-mike-roest
Copy link
Contributor Author

@SuperQ I've rebased this to remove my previous build fixes

SuperQ
SuperQ requested changes May 31, 2021
View reviewed changes
Copy link
Contributor

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of minor nits. Plus I think there are some metrics missing from the README update.

collector/indices_mappings.go Outdated Show resolved Hide resolved
collector/indices_mappings.go Outdated Show resolved Hide resolved
@repl-mike-roest
Copy link
Contributor Author

@SuperQ I've update the metric names for the 2 new metrics in this PR (and update the README with the new metric).

Running the promtool

elasticsearch_breakers_overhead counter metrics should have "_total" suffix
elasticsearch_breakers_tripped counter metrics should have "_total" suffix
elasticsearch_cluster_health_json_parse_failures counter metrics should have "_total" suffix
elasticsearch_cluster_health_total_scrapes counter metrics should have "_total" suffix
elasticsearch_clustersettings_stats_json_parse_failures counter metrics should have "_total" suffix
elasticsearch_clustersettings_stats_total_scrapes counter metrics should have "_total" suffix
elasticsearch_filesystem_io_stats_device_operations_count counter metrics should have "_total" suffix
elasticsearch_filesystem_io_stats_device_read_operations_count counter metrics should have "_total" suffix
elasticsearch_filesystem_io_stats_device_read_size_kilobytes_sum counter metrics should have "_total" suffix
elasticsearch_filesystem_io_stats_device_write_operations_count counter metrics should have "_total" suffix
elasticsearch_filesystem_io_stats_device_write_size_kilobytes_sum counter metrics should have "_total" suffix
elasticsearch_index_stats_json_parse_failures counter metrics should have "_total" suffix
elasticsearch_indices_completion_size_in_bytes counter metrics should have "_total" suffix
elasticsearch_indices_docs_total non-counter metrics should not have "_total" suffix
elasticsearch_indices_fielddata_evictions counter metrics should have "_total" suffix
elasticsearch_indices_filter_cache_evictions counter metrics should have "_total" suffix
elasticsearch_indices_flush_time_seconds counter metrics should have "_total" suffix
elasticsearch_indices_get_exists_time_seconds counter metrics should have "_total" suffix
elasticsearch_indices_get_time_seconds counter metrics should have "_total" suffix
elasticsearch_indices_query_miss_count counter metrics should have "_total" suffix
elasticsearch_node_stats_total_scrapes counter metrics should have "_total" suffix
elasticsearch_indices_mappings_stats_field_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_indices_request_miss_count counter metrics should have "_total" suffix
elasticsearch_indices_deleted_docs_total non-counter metrics should not have "_total" suffix
elasticsearch_indices_mappings_stats_json_parse_failures counter metrics should have "_total" suffix
elasticsearch_indices_query_miss_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_indices_search_query_time_seconds counter metrics should have "_total" suffix
elasticsearch_indices_request_miss_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_indices_search_suggest_time_seconds counter metrics should have "_total" suffix
elasticsearch_indices_segment_count_total non-counter metrics should not have "_total" suffix
elasticsearch_indices_segment_fields_memory_bytes_total non-counter metrics should not have "_total" suffix
elasticsearch_indices_settings_stats_total_scrapes counter metrics should have "_total" suffix
elasticsearch_indices_segment_points_memory_bytes_total non-counter metrics should not have "_total" suffix
elasticsearch_indices_translog_operations counter metrics should have "_total" suffix
elasticsearch_jvm_gc_collection_seconds_count counter metrics should have "_total" suffix
elasticsearch_snapshot_stats_json_parse_failures counter metrics should have "_total" suffix
elasticsearch_indices_segment_index_writer_memory_bytes_total non-counter metrics should not have "_total" suffix
elasticsearch_indices_segment_memory_bytes_total non-counter metrics should not have "_total" suffix
elasticsearch_indices_segment_version_map_memory_bytes_total non-counter metrics should not have "_total" suffix
elasticsearch_filesystem_io_stats_device_operations_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_filesystem_io_stats_device_read_operations_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_filesystem_io_stats_device_write_operations_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_indices_segment_terms_memory_total non-counter metrics should not have "_total" suffix
elasticsearch_indices_segments_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_jvm_gc_collection_seconds_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_filesystem_io_stats_device_write_size_kilobytes_sum non-histogram and non-summary metrics should not have "_sum" suffix
elasticsearch_filesystem_io_stats_device_write_size_kilobytes_sum use base unit "bytes" instead of "kilobytes"
elasticsearch_index_stats_total_scrapes counter metrics should have "_total" suffix
elasticsearch_indices_get_missing_time_seconds counter metrics should have "_total" suffix
elasticsearch_indices_mappings_stats_total_scrapes counter metrics should have "_total" suffix
elasticsearch_indices_query_cache_count counter metrics should have "_total" suffix
elasticsearch_indices_query_cache_evictions counter metrics should have "_total" suffix
elasticsearch_indices_request_cache_count counter metrics should have "_total" suffix
elasticsearch_indices_request_cache_evictions counter metrics should have "_total" suffix
elasticsearch_indices_search_fetch_time_seconds counter metrics should have "_total" suffix
elasticsearch_indices_search_scroll_time_seconds counter metrics should have "_total" suffix
elasticsearch_indices_translog_size_in_bytes counter metrics should have "_total" suffix
elasticsearch_jvm_gc_collection_seconds_sum counter metrics should have "_total" suffix
elasticsearch_indices_segment_norms_memory_bytes_total non-counter metrics should not have "_total" suffix
elasticsearch_indices_settings_stats_json_parse_failures counter metrics should have "_total" suffix
elasticsearch_jvm_memory_pool_max_bytes counter metrics should have "_total" suffix
elasticsearch_jvm_memory_pool_peak_max_bytes counter metrics should have "_total" suffix
elasticsearch_jvm_memory_pool_peak_used_bytes counter metrics should have "_total" suffix
elasticsearch_node_stats_json_parse_failures counter metrics should have "_total" suffix
elasticsearch_process_cpu_time_seconds_sum counter metrics should have "_total" suffix
elasticsearch_thread_pool_rejected_count counter metrics should have "_total" suffix
elasticsearch_indices_query_cache_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_snapshot_stats_total_scrapes counter metrics should have "_total" suffix
elasticsearch_indices_completion_bytes_total non-counter metrics should not have "_total" suffix
elasticsearch_indices_request_cache_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_indices_segment_doc_values_memory_bytes_total non-counter metrics should not have "_total" suffix
elasticsearch_indices_segment_fixed_bit_set_memory_bytes_total non-counter metrics should not have "_total" suffix
elasticsearch_indices_store_size_bytes_total non-counter metrics should not have "_total" suffix
elasticsearch_process_open_files_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_thread_pool_active_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_thread_pool_largest_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_thread_pool_queue_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_thread_pool_rejected_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_thread_pool_threads_count non-histogram and non-summary metrics should not have "_count" suffix
elasticsearch_filesystem_io_stats_device_read_size_kilobytes_sum non-histogram and non-summary metrics should not have "_sum" suffix
elasticsearch_jvm_gc_collection_seconds_sum non-histogram and non-summary metrics should not have "_sum" suffix
elasticsearch_process_cpu_time_seconds_sum non-histogram and non-summary metrics should not have "_sum" suffix
elasticsearch_filesystem_io_stats_device_read_size_kilobytes_sum use base unit "bytes" instead of "kilobytes"
elasticsearch_thread_pool_completed_count counter metrics should have "_total" suffix
elasticsearch_thread_pool_completed_count non-histogram and non-summary metrics should not have "_count" suffix

There's a significant number of metrics that are failing the validations. So like you said a separate PR that can be merged with the breaking changes to metric names probably makes the most sense (probably makes sense to make an issue i suppose?)

@repl-mike-roest repl-mike-roest mentioned this pull request May 31, 2021
Open
@SuperQ
Copy link
Contributor

SuperQ commented Jun 1, 2021

Yes, a separate PR for fixing the naming issues would make the most sense.

SuperQ
SuperQ requested changes Jun 1, 2021
View reviewed changes
Copy link
Contributor

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more naming issue.

collector/indices_mappings.go Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@repl-mike-roest
Copy link
Contributor Author

👍

@hansh0801
Copy link

when this change will be released?

SuperQ
SuperQ reviewed Jun 1, 2021
View reviewed changes
README.md Outdated Show resolved Hide resolved
SuperQ
SuperQ approved these changes Jun 4, 2021
View reviewed changes
Copy link
Contributor

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks for all the work.

@SuperQ SuperQ merged commit 23ad1e7 into prometheus-community:master Jun 4, 2021
@repl-mike-roest repl-mike-roest deleted the index-stats branch June 4, 2021 13:11
sysadmind added a commit to sysadmind/elasticsearch_exporter that referenced this pull request Jun 10, 2021
@sysadmind
Condense prometheus-community#411 to one line in CHANGELOG
fadbab7 
Signed-off-by: Joe Adams <[email protected]>
@mandarg mandarg mentioned this pull request Jul 16, 2021
Merged
P6rguVyrst pushed a commit to transferwise/elasticsearch_exporter that referenced this pull request Jul 21, 2021
@repl-mike-roest @msodi @P6rguVyrst
Add Gauge for total_fields setting on a index (prometheus-community#411)
fa51cbc 
* prometheus-community#257: Add gauge for total fields mapped in an index

Signed-off-by: Mike Roest <[email protected]>
Co-authored-by: Marco Sohns <[email protected]>
Co-authored-by: Jeff Nadler <[email protected]>
Signed-off-by: Toomas Ormisson <[email protected]>
P6rguVyrst pushed a commit to transferwise/elasticsearch_exporter that referenced this pull request Jul 21, 2021
@mandarg @P6rguVyrst
Document es.indices_mappings flag
49465b8 
This was introduced in prometheus-community#411, and is in the 1.2.0 release.

Signed-off-by: Mandar Gokhale <[email protected]>
Signed-off-by: Toomas Ormisson <[email protected]>
jnadler pushed a commit to jnadler/elasticsearch_exporter that referenced this pull request Oct 27, 2022
@repl-mike-roest @msodi
Add Gauge for total_fields setting on a index (prometheus-community#411)
60caa16 
* prometheus-community#257: Add gauge for total fields mapped in an index

Signed-off-by: Mike Roest <[email protected]>
Co-authored-by: Marco Sohns <[email protected]>
Co-authored-by: Jeff Nadler <[email protected]>
jnadler pushed a commit to jnadler/elasticsearch_exporter that referenced this pull request Oct 27, 2022
@mandarg
Document es.indices_mappings flag
c068cee 
This was introduced in prometheus-community#411, and is in the 1.2.0 release.

Signed-off-by: Mandar Gokhale <[email protected]>
@jnadler jnadler mentioned this pull request Nov 3, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SuperQ SuperQ SuperQ approved these changes

@caarlos0 caarlos0 caarlos0 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@repl-mike-roest @SuperQ @hansh0801 @caarlos0 @msodi