Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feat add ps memory collector #515

Merged
merged 7 commits into from
Nov 13, 2020
Merged

Feat add ps memory collector #515

merged 7 commits into from
Nov 13, 2020

Conversation

dafydd-t
Copy link
Contributor

@dafydd-t dafydd-t commented Nov 11, 2020 •
edited
Loading

This PR adds a collector for performance_schema.memory_summary_global_by_event_name, creating three metrics as described in #75 .

  • The collector is disabled by default
  • Include a flag to remove the memory/ prefix from each event label for this metric.
  • Tested locally against MySQL 5.6, 5.7 and 8.
  • 5.6 doesn't have this table, so upon scrape, the exporter logs:
level=error ts=2020-11-11T17:06:26.873Z caller=exporter.go:174 msg="Error from scraper" scraper=perf_schema.memory_events err="Error 1146: Table 'performance_schema.memory_summary_global_by_event_name' doesn't exist"

  • Running against mysql:8 and mysql:5.7 with performance_schema=OFF, the exporter logs no errors and does not export any memory_events metrics, but does expose mysql_exporter_collector_duration_seconds for the collector.

  • Add new collector flags to the README. Also included missing flag for file_instances.remove_prefix.

@dafydd-t dafydd-t force-pushed the feat-add-ps-memory-collector branch from 5ab4cb8 to 9584fb1 Compare November 11, 2020 18:03
SuperQ
SuperQ reviewed Nov 11, 2020
View reviewed changes
Copy link
Member

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking pretty good. Please also add an entry to the CHANGELOG.md.

collector/perf_schema_memory_events.go Outdated Show resolved Hide resolved
collector/perf_schema_memory_events.go Outdated Show resolved Hide resolved
@dafydd-t dafydd-t force-pushed the feat-add-ps-memory-collector branch from 3221711 to 8350d0b Compare November 12, 2020 09:15
@dafydd-t
Copy link
Contributor Author

Cheers @SuperQ , I've amended the metric names and added to the changelog.

@dafydd-t dafydd-t marked this pull request as ready for review November 12, 2020 09:17
SuperQ
SuperQ reviewed Nov 12, 2020
View reviewed changes
CHANGELOG.md Show resolved Hide resolved
SuperQ
SuperQ approved these changes Nov 12, 2020
View reviewed changes
Copy link
Member

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@SuperQ SuperQ merged commit 44a8301 into prometheus:master Nov 13, 2020
@dafydd-t dafydd-t deleted the feat-add-ps-memory-collector branch November 13, 2020 15:57
@dafydd-t dafydd-t mentioned this pull request Nov 19, 2020
Merged
SuperQ added a commit that referenced this pull request Apr 23, 2021
@SuperQ
Release 0.13.0-rc.0
b281883 
Changes related to `replication_group_member_stats` collector:
* metric "transaction_in_queue" was Counter instead of Gauge
* renamed 3 metrics starting with `mysql_perf_schema_transaction_` to start with `mysql_perf_schema_transactions_` to be consistent with column names
* exposing only server's own stats by matching MEMBER_ID with @@server_uuid resulting "member_id" label to be dropped.

* [CHANGE] Switch to go-kit for logs. #433
* [BUGFIX] Fix binlog metrics on mysql 8.x #419
* [BUGFIX] Fix output value of wsrep_cluster_status #473
* [BUGFIX] Fix collect.info_schema.innodb_metrics for new field names (mariadb 10.5+) #494
* [BUGFIX] Fix log output of collect[] params #505
* [BUGFIX] Fix collect.info_schema.innodb_tablespaces for new table names #516
* [BUGFIX] Fix innodb_metrics for mariadb 10.5+ #523
* [BUGFIX] Allow perf_schema.memory summary current_bytes to be negative #517
* [ENHANCEMENT] Support heartbeats in UTC #471
* [FEATURE] Add `tls.insecure-skip-verify` flag to ignore tls verification errors #417
* [FEATURE] Add collector for AWS Aurora information_schema.replica_host_status #435
* [FEATURE] Add collector for `replication_group_members` #459
* [FEATURE] Add new metrics to `replication_group_member_stats` collector to support MySQL 8.x. #462
* [FEATURE] Add collector for `performance_schema.memory_summary_global_by_event_name` #515
* [FEATURE] Support authenticating using mTLS client cert and no password #539

Signed-off-by: Ben Kochie <[email protected]>
@SuperQ SuperQ mentioned this pull request Apr 23, 2021
Merged
SuperQ added a commit that referenced this pull request Apr 25, 2021
@SuperQ
Release 0.13.0-rc.0
2517b84 
BREAKING CHANGES:

Changes related to `replication_group_member_stats` collector:
* metric "transaction_in_queue" was Counter instead of Gauge
* renamed 3 metrics starting with `mysql_perf_schema_transaction_` to start with `mysql_perf_schema_transactions_` to be consistent with column names
* exposing only server's own stats by matching MEMBER_ID with @@server_uuid resulting "member_id" label to be dropped.

Changes:

* [CHANGE] Switch to go-kit for logs. #433
* [FEATURE] Add `tls.insecure-skip-verify` flag to ignore tls verification errors #417
* [FEATURE] Add collector for AWS Aurora information_schema.replica_host_status #435
* [FEATURE] Add collector for `replication_group_members` #459
* [FEATURE] Add new metrics to `replication_group_member_stats` collector to support MySQL 8.x. #462
* [FEATURE] Add collector for `performance_schema.memory_summary_global_by_event_name` #515
* [FEATURE] Support authenticating using mTLS client cert and no password #539
* [ENHANCEMENT] Support heartbeats in UTC #471
* [BUGFIX] Fix binlog metrics on mysql 8.x #419
* [BUGFIX] Fix output value of wsrep_cluster_status #473
* [BUGFIX] Fix collect.info_schema.innodb_metrics for new field names (mariadb 10.5+) #494
* [BUGFIX] Fix log output of collect[] params #505
* [BUGFIX] Fix collect.info_schema.innodb_tablespaces for new table names #516
* [BUGFIX] Fix innodb_metrics for mariadb 10.5+ #523
* [BUGFIX] Allow perf_schema.memory summary current_bytes to be negative #517

Signed-off-by: Ben Kochie <[email protected]>
SuperQ added a commit that referenced this pull request Apr 25, 2021
@SuperQ
Release 0.13.0-rc.0 (#546)
0dde132 
BREAKING CHANGES:

Changes related to `replication_group_member_stats` collector:
* metric "transaction_in_queue" was Counter instead of Gauge
* renamed 3 metrics starting with `mysql_perf_schema_transaction_` to start with `mysql_perf_schema_transactions_` to be consistent with column names
* exposing only server's own stats by matching MEMBER_ID with @@server_uuid resulting "member_id" label to be dropped.

Changes:

* [CHANGE] Switch to go-kit for logs. #433
* [FEATURE] Add `tls.insecure-skip-verify` flag to ignore tls verification errors #417
* [FEATURE] Add collector for AWS Aurora information_schema.replica_host_status #435
* [FEATURE] Add collector for `replication_group_members` #459
* [FEATURE] Add new metrics to `replication_group_member_stats` collector to support MySQL 8.x. #462
* [FEATURE] Add collector for `performance_schema.memory_summary_global_by_event_name` #515
* [FEATURE] Support authenticating using mTLS client cert and no password #539
* [ENHANCEMENT] Support heartbeats in UTC #471
* [BUGFIX] Fix binlog metrics on mysql 8.x #419
* [BUGFIX] Fix output value of wsrep_cluster_status #473
* [BUGFIX] Fix collect.info_schema.innodb_metrics for new field names (mariadb 10.5+) #494
* [BUGFIX] Fix log output of collect[] params #505
* [BUGFIX] Fix collect.info_schema.innodb_tablespaces for new table names #516
* [BUGFIX] Fix innodb_metrics for mariadb 10.5+ #523
* [BUGFIX] Allow perf_schema.memory summary current_bytes to be negative #517

Signed-off-by: Ben Kochie <[email protected]>
SuperQ added a commit that referenced this pull request May 18, 2021
@SuperQ
Release 0.13.0
693f62f 
BREAKING CHANGES:

Changes related to `replication_group_member_stats` collector:
* metric "transaction_in_queue" was Counter instead of Gauge
* renamed 3 metrics starting with `mysql_perf_schema_transaction_` to start with `mysql_perf_schema_transactions_` to be consistent with column names
* exposing only server's own stats by matching MEMBER_ID with @@server_uuid resulting "member_id" label to be dropped.

Changes:

* [CHANGE] Switch to go-kit for logs. #433
* [FEATURE] Add `tls.insecure-skip-verify` flag to ignore tls verification errors #417
* [FEATURE] Add collector for AWS Aurora information_schema.replica_host_status #435
* [FEATURE] Add collector for `replication_group_members` #459
* [FEATURE] Add new metrics to `replication_group_member_stats` collector to support MySQL 8.x. #462
* [FEATURE] Add collector for `performance_schema.memory_summary_global_by_event_name` #515
* [FEATURE] Support authenticating using mTLS client cert and no password #539
* [FEATURE] Add TLS and basic authentication #522
* [ENHANCEMENT] Support heartbeats in UTC #471
* [ENHANCEMENT] Improve parsing of boolean strings #548
* [BUGFIX] Fix binlog metrics on mysql 8.x #419
* [BUGFIX] Fix output value of wsrep_cluster_status #473
* [BUGFIX] Fix collect.info_schema.innodb_metrics for new field names (mariadb 10.5+) #494
* [BUGFIX] Fix log output of collect[] params #505
* [BUGFIX] Fix collect.info_schema.innodb_tablespaces for new table names #516
* [BUGFIX] Fix innodb_metrics for mariadb 10.5+ #523
* [BUGFIX] Allow perf_schema.memory summary current_bytes to be negative #517

Signed-off-by: Ben Kochie <[email protected]>
@SuperQ SuperQ mentioned this pull request May 18, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SuperQ SuperQ SuperQ approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dafydd-t @SuperQ