Skip to content
This repository has been archived by the owner on Sep 21, 2023. It is now read-only.

Expose go-elasticsearch BulkIndexer output performance metrics #153

Open
2 tasks
Tracked by #16
cmacknz opened this issue Oct 31, 2022 · 0 comments
Open
2 tasks
Tracked by #16

Expose go-elasticsearch BulkIndexer output performance metrics #153

cmacknz opened this issue Oct 31, 2022 · 0 comments

Comments

@cmacknz
Copy link
Member

cmacknz commented Oct 31, 2022

With the switch to the go-elasticsearch client we should expose the Elasticsearch client, transport, and BulkIndexer metrics as part of the shipper metrics.

We should capture and expose at least the following metrics as part of the shipper HTTP stats endpoint:

We should also ideally expose metrics for:

  • The number of times a BulkIndexer flushed because the byte threshold was hit.
  • The number of times a BulkIndexer flushed because the timeout expired.

These two metrics may give us a good idea of if additional workers or indexers are needed. Frequently flushing because the byte threshold is hit may indicate the shipper could benefit from additional workers.

@cmacknz cmacknz changed the title Expose shipper output performance metrics Expose shipper Elasticsearch output performance metrics Oct 31, 2022
@cmacknz cmacknz changed the title Expose shipper Elasticsearch output performance metrics Expose go-elasticsearch BulkIndexer output performance metrics Mar 1, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant