Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(observability): emit RequestBuildError when HttpSink::request_builder errors #14575

Merged

Conversation

StephenWakely
Copy link
Contributor

Ref: #14526
Epic: #13995

This emits a RequestBuildError if the request_builder function returns an error.

I have looked through all the sinks that implement this trait, and none currently emit an event here, so as far as I can tell there doesn't appear to be any risk of duplicate events.

Signed-off-by: Stephen Wakely [email protected]

@netlify
Copy link

netlify bot commented Sep 26, 2022

Deploy Preview for vector-project ready!

Name Link
🔨 Latest commit 397b4e1
🔍 Latest deploy log https://app.netlify.com/sites/vector-project/deploys/6331bd1b71db00000855161d
😎 Deploy Preview https://deploy-preview-14575--vector-project.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@github-actions github-actions bot added the domain: sinks Anything related to the Vector's sinks label Sep 26, 2022
let byte_size = request.body().len();
let request = request.map(Body::from);
let (protocol, endpoint) = uri::protocol_endpoint(request.uri().clone());

// Any errors raised in `http_client.call` results in a `GotHttpWarning` event being emmited
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 unrelated to this change, but since you mentioned it - should that always be a warning or does it already depend on the error raised?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think it is badly named. Will fix that in a separate PR.

@github-actions
Copy link

Soak Test Results

Baseline: 677ef7f
Comparison: 397b4e1
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
datadog_agent_remap_blackhole_acks 865.24KiB 1.44 100.00% 58.72MiB 4.22MiB 87.98KiB 0 0.0719167 59.57MiB 2.99MiB 62.63KiB 0 0.0502133 False False
datadog_agent_remap_blackhole 698.76KiB 1.15 100.00% 59.33MiB 4.0MiB 83.24KiB 0 0.0673451 60.01MiB 2.9MiB 60.55KiB 0 0.0483427 False False
datadog_agent_remap_datadog_logs_acks 591.02KiB 0.98 100.00% 58.81MiB 3.04MiB 63.68KiB 0 0.0517273 59.38MiB 4.54MiB 94.56KiB 0 0.0764853 False False
http_text_to_http_json 362.7KiB 0.94 100.00% 37.69MiB 1.01MiB 21.03KiB 0 0.0266863 38.05MiB 966.64KiB 19.74KiB 0 0.0248054 False False
datadog_agent_remap_datadog_logs 553.12KiB 0.88 100.00% 61.1MiB 1.42MiB 29.82KiB 0 0.0232794 61.64MiB 4.54MiB 94.61KiB 0 0.0737121 False False
http_pipelines_blackhole_acks 5.5KiB 0.44 94.13% 1.22MiB 115.06KiB 2.34KiB 0 0.0921228 1.22MiB 84.67KiB 1.73KiB 0 0.0674915 False False
http_to_http_acks 57.98KiB 0.33 19.52% 17.25MiB 7.96MiB 166.42KiB 0 0.461466 17.31MiB 7.92MiB 165.27KiB 0 0.457548 True True
splunk_hec_to_splunk_hec_logs_noack 36.05KiB 0.15 99.06% 23.8MiB 592.24KiB 12.07KiB 0 0.0242936 23.84MiB 334.2KiB 6.82KiB 0 0.0136886 False False
syslog_regex_logs2metric_ddmetrics 15.44KiB 0.12 73.95% 12.22MiB 522.68KiB 10.65KiB 0 0.0417701 12.23MiB 424.3KiB 8.65KiB 0 0.0338662 False False
socket_to_socket_blackhole 19.36KiB 0.08 97.14% 23.03MiB 238.15KiB 4.86KiB 0 0.0100985 23.04MiB 361.87KiB 7.39KiB 0 0.0153321 False False
enterprise_http_to_http 529.57B 0 5.64% 23.85MiB 254.38KiB 5.19KiB 0 0.0104153 23.85MiB 251.79KiB 5.15KiB 0 0.0103091 False False
file_to_blackhole -33.25KiB -0.03 29.49% 95.34MiB 2.9MiB 60.07KiB 0 0.0303864 95.31MiB 3.08MiB 64.11KiB 0 0.03232 False False
splunk_hec_indexer_ack_blackhole -6.89KiB -0.03 20.44% 23.75MiB 907.81KiB 18.47KiB 0 0.0373224 23.74MiB 941.02KiB 19.14KiB 0 0.0386987 False False
syslog_log2metric_splunk_hec_metrics -4.3KiB -0.03 18.22% 16.23MiB 570.77KiB 11.63KiB 0 0.0343463 16.22MiB 715.96KiB 14.58KiB 0 0.0430943 False False
splunk_hec_to_splunk_hec_logs_acks -10.61KiB -0.04 32.22% 23.75MiB 868.3KiB 17.66KiB 0 0.0356945 23.74MiB 907.09KiB 18.44KiB 0 0.0373053 False False
http_to_http_json -25.16KiB -0.1 97.15% 23.85MiB 329.74KiB 6.73KiB 0 0.0134993 23.82MiB 454.91KiB 9.3KiB 0 0.018643 False False
syslog_humio_logs -19.75KiB -0.12 94.88% 15.93MiB 342.46KiB 6.99KiB 0 0.0209829 15.92MiB 357.73KiB 7.32KiB 0 0.0219452 False False
fluent_elasticsearch -115.85KiB -0.14 100.00% 79.47MiB 53.42KiB 1.08KiB 0 0.000656295 79.36MiB 1.2MiB 24.72KiB 0 0.015126 False False
syslog_splunk_hec_logs -59.77KiB -0.36 99.56% 16.09MiB 773.26KiB 15.74KiB 0 0.0469248 16.03MiB 679.97KiB 13.86KiB 0 0.0414136 False False
http_to_http_noack -101.16KiB -0.41 100.00% 23.85MiB 259.1KiB 5.3KiB 0 0.0106089 23.75MiB 1.07MiB 22.37KiB 0 0.045181 False False
splunk_hec_route_s3 -103.58KiB -0.54 88.47% 18.81MiB 2.26MiB 47.05KiB 0 0.120098 18.71MiB 2.2MiB 45.95KiB 0 0.117391 False False
syslog_loki -101.44KiB -0.68 100.00% 14.48MiB 330.31KiB 6.76KiB 0 0.0222789 14.38MiB 740.94KiB 15.06KiB 0 0.0503202 False False
http_pipelines_blackhole -16.32KiB -0.95 100.00% 1.68MiB 14.76KiB 309.16B 0 0.00858061 1.66MiB 102.94KiB 2.1KiB 0 0.0604037 False False
http_pipelines_no_grok_blackhole -154.66KiB -1.42 100.00% 10.61MiB 225.17KiB 4.6KiB 0 0.0207269 10.46MiB 1.03MiB 21.39KiB 0 0.0982147 False False
syslog_log2metric_humio_metrics -361.95KiB -2.76 100.00% 12.8MiB 457.09KiB 9.33KiB 0 0.0348562 12.45MiB 560.34KiB 11.41KiB 0 0.0439427 False False

@StephenWakely StephenWakely merged commit f38e3c3 into master Sep 27, 2022
@StephenWakely StephenWakely deleted the stephen/http_batch_service_request_builder_error branch September 27, 2022 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: sinks Anything related to the Vector's sinks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants