enhancement(http source): emit Error and ErrorDropped events per instrumentation spec

[neuronull]

Closes: #14056.
Part of: #13995.

[netlify]

✅ Deploy Preview for vector-project canceled.

Name	Link
🔨 Latest commit	70cdba8
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/630e2cd3411d490009c78321

[github-actions]

Soak Test Results

Baseline: 9bfcfbf
Comparison: 76e2ed2
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
splunk_hec_route_s3	951.0KiB	5.11	100.00%	18.16MiB	2.49MiB	51.93KiB	0	0.137225	19.09MiB	2.19MiB	45.76KiB	0	0.114671	False	False
syslog_humio_logs	656.33KiB	4.08	100.00%	15.71MiB	704.5KiB	14.38KiB	0	0.0437856	16.35MiB	517.59KiB	10.59KiB	0	0.0309081	False	False
datadog_agent_remap_blackhole	1.65MiB	2.79	100.00%	59.19MiB	6.14MiB	127.85KiB	0	0.103701	60.84MiB	4.89MiB	102.14KiB	0	0.080395	False	False
syslog_splunk_hec_logs	149.46KiB	0.93	100.00%	15.63MiB	934.57KiB	19.02KiB	0	0.0583817	15.78MiB	954.01KiB	19.44KiB	0	0.0590445	False	False
http_pipelines_blackhole_acks	11.45KiB	0.91	100.00%	1.22MiB	108.48KiB	2.21KiB	0	0.0865614	1.23MiB	67.09KiB	1.37KiB	0	0.0530488	False	False
syslog_regex_logs2metric_ddmetrics	116.25KiB	0.9	100.00%	12.55MiB	498.14KiB	10.16KiB	0	0.038769	12.66MiB	491.48KiB	10.03KiB	0	0.0379079	False	False
socket_to_socket_blackhole	155.62KiB	0.66	100.00%	23.08MiB	230.89KiB	4.71KiB	0	0.00976665	23.23MiB	183.75KiB	3.75KiB	0	0.0077217	False	False
http_text_to_http_json	200.89KiB	0.49	100.00%	39.76MiB	760.37KiB	15.52KiB	0	0.0186697	39.96MiB	768.47KiB	15.68KiB	0	0.0187758	False	False
syslog_log2metric_splunk_hec_metrics	80.96KiB	0.44	100.00%	17.77MiB	374.82KiB	7.65KiB	0	0.0205901	17.85MiB	726.29KiB	14.79KiB	0	0.0397212	False	False
splunk_hec_to_splunk_hec_logs_acks	22.3KiB	0.09	64.33%	23.75MiB	871.88KiB	17.73KiB	0	0.0358466	23.77MiB	808.26KiB	16.45KiB	0	0.0332001	False	False
splunk_hec_to_splunk_hec_logs_noack	6.67KiB	0.03	47.87%	23.83MiB	385.81KiB	7.88KiB	0	0.0158064	23.84MiB	332.32KiB	6.79KiB	0	0.0136112	False	False
http_pipelines_blackhole	-52.48B	-0	1.68%	1.62MiB	60.69KiB	1.24KiB	0	0.036471	1.62MiB	103.05KiB	2.1KiB	0	0.0619278	False	False
enterprise_http_to_http	-489.3B	-0	5.32%	23.85MiB	245.38KiB	5.01KiB	0	0.0100465	23.85MiB	250.54KiB	5.12KiB	0	0.0102581	False	False
splunk_hec_indexer_ack_blackhole	-12.51KiB	-0.05	37.89%	23.76MiB	850.68KiB	17.31KiB	0	0.0349567	23.75MiB	907.91KiB	18.47KiB	0	0.0373278	False	False
file_to_blackhole	-44.34KiB	-0.05	34.84%	95.34MiB	3.24MiB	67.17KiB	0	0.0339777	95.3MiB	3.44MiB	71.65KiB	0	0.0361395	False	False
http_to_http_json	-34.61KiB	-0.14	99.32%	23.84MiB	350.44KiB	7.15KiB	0	0.0143497	23.81MiB	518.25KiB	10.58KiB	0	0.0212511	False	False
fluent_elasticsearch	-175.33KiB	-0.22	100.00%	79.47MiB	54.23KiB	1.1KiB	0	0.000666284	79.3MiB	1.54MiB	31.75KiB	0	0.0194486	False	False
datadog_agent_remap_datadog_logs_acks	-203.76KiB	-0.32	92.26%	62.09MiB	3.28MiB	68.53KiB	0	0.052794	61.89MiB	4.46MiB	92.78KiB	0	0.0720042	False	False
http_to_http_noack	-114.07KiB	-0.47	100.00%	23.85MiB	250.24KiB	5.12KiB	0	0.0102461	23.73MiB	1.15MiB	24.01KiB	0	0.0485183	False	False
syslog_loki	-90.65KiB	-0.62	100.00%	14.34MiB	652.65KiB	13.36KiB	0	0.0444389	14.25MiB	868.6KiB	17.66KiB	0	0.0595102	False	False
datadog_agent_remap_datadog_logs	-513.56KiB	-0.8	100.00%	63.08MiB	343.07KiB	7.03KiB	0	0.00530981	62.58MiB	4.21MiB	87.64KiB	0	0.0672309	False	False
http_pipelines_no_grok_blackhole	-144.37KiB	-1.27	100.00%	11.11MiB	56.17KiB	1.15KiB	0	0.00493592	10.97MiB	1.06MiB	22.09KiB	0	0.0966667	False	False
http_to_http_acks	-289.93KiB	-1.63	78.26%	17.34MiB	7.95MiB	166.15KiB	0	0.458144	17.06MiB	7.97MiB	166.25KiB	0	0.467007	True	True
syslog_log2metric_humio_metrics	-217.05KiB	-1.66	100.00%	12.81MiB	528.18KiB	10.79KiB	0	0.0402706	12.59MiB	660.72KiB	13.46KiB	0	0.0512242	False	False
datadog_agent_remap_blackhole_acks	-1.15MiB	-1.81	100.00%	63.46MiB	4.4MiB	91.7KiB	0	0.0693747	62.31MiB	4.66MiB	97.41KiB	0	0.0748165	False	False

[github-actions]

Soak Test Results

Baseline: 2731071
Comparison: 70cdba8
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
http_text_to_http_json	2.0MiB	5.3	100.00%	37.81MiB	1.11MiB	23.12KiB	0	0.029257	39.81MiB	1.03MiB	21.52KiB	0	0.0258536	False	False
socket_to_socket_blackhole	1.12MiB	4.88	100.00%	22.87MiB	128.01KiB	2.61KiB	0	0.0054647	23.99MiB	245.96KiB	5.02KiB	0	0.0100115	False	False
datadog_agent_remap_blackhole	1.69MiB	2.93	100.00%	57.7MiB	5.57MiB	116.05KiB	0	0.0965134	59.39MiB	5.1MiB	106.48KiB	0	0.0859319	False	False
syslog_regex_logs2metric_ddmetrics	282.24KiB	2.42	100.00%	11.41MiB	1.07MiB	22.28KiB	0	0.0936571	11.68MiB	1.01MiB	21.18KiB	0	0.0868409	False	False
syslog_humio_logs	275.2KiB	1.69	100.00%	15.87MiB	442.74KiB	9.03KiB	0	0.0272412	16.14MiB	329.33KiB	6.75KiB	0	0.019926	False	False
syslog_log2metric_splunk_hec_metrics	266.7KiB	1.54	100.00%	16.96MiB	929.38KiB	18.94KiB	0	0.0535154	17.22MiB	1.03MiB	21.44KiB	0	0.059764	False	False
splunk_hec_route_s3	277.43KiB	1.5	99.99%	18.09MiB	2.38MiB	49.61KiB	0	0.13165	18.36MiB	2.32MiB	48.43KiB	0	0.126118	False	False
http_to_http_acks	201.31KiB	1.15	60.01%	17.07MiB	8.03MiB	168.0KiB	0	0.470367	17.27MiB	8.16MiB	170.18KiB	0	0.472429	True	True
syslog_splunk_hec_logs	191.76KiB	1.15	100.00%	16.34MiB	700.1KiB	14.26KiB	0	0.0418403	16.52MiB	619.2KiB	12.63KiB	0	0.0365857	False	False
datadog_agent_remap_blackhole_acks	658.19KiB	1.06	100.00%	60.39MiB	5.72MiB	119.13KiB	0	0.0947213	61.04MiB	4.92MiB	102.81KiB	0	0.080567	False	False
datadog_agent_remap_datadog_logs_acks	452.84KiB	0.72	100.00%	61.63MiB	2.93MiB	61.28KiB	0	0.0475408	62.08MiB	4.37MiB	90.97KiB	0	0.0703886	False	False
splunk_hec_to_splunk_hec_logs_noack	11.2KiB	0.05	70.73%	23.83MiB	404.0KiB	8.26KiB	0	0.0165513	23.84MiB	329.47KiB	6.73KiB	0	0.0134919	False	False
syslog_log2metric_humio_metrics	5.34KiB	0.04	31.75%	12.91MiB	345.76KiB	7.05KiB	0	0.0261572	12.91MiB	539.66KiB	10.99KiB	0	0.0408092	False	False
splunk_hec_to_splunk_hec_logs_acks	7.54KiB	0.03	23.39%	23.74MiB	896.95KiB	18.24KiB	0	0.0368849	23.75MiB	864.78KiB	17.59KiB	0	0.0355512	False	False
splunk_hec_indexer_ack_blackhole	1.4KiB	0.01	4.54%	23.76MiB	858.36KiB	17.47KiB	0	0.0352727	23.76MiB	855.07KiB	17.4KiB	0	0.0351355	False	False
enterprise_http_to_http	-1.77KiB	-0.01	18.77%	23.85MiB	258.15KiB	5.27KiB	0	0.01057	23.84MiB	258.85KiB	5.29KiB	0	0.0105992	False	False
file_to_blackhole	-62.63KiB	-0.06	47.60%	95.34MiB	3.1MiB	64.29KiB	0	0.0325197	95.28MiB	3.58MiB	74.35KiB	0	0.0375363	False	False
http_to_http_json	-47.58KiB	-0.19	99.94%	23.85MiB	338.49KiB	6.91KiB	0	0.0138598	23.8MiB	583.43KiB	11.9KiB	0	0.0239356	False	False
fluent_elasticsearch	-192.32KiB	-0.24	100.00%	79.47MiB	53.05KiB	1.07KiB	0	0.00065171	79.29MiB	1.87MiB	38.44KiB	0	0.0236067	False	False
http_pipelines_no_grok_blackhole	-34.8KiB	-0.31	90.43%	10.98MiB	75.97KiB	1.55KiB	0	0.00675615	10.94MiB	1023.26KiB	20.82KiB	0	0.091281	False	False
http_to_http_noack	-104.29KiB	-0.43	100.00%	23.85MiB	249.94KiB	5.11KiB	0	0.0102339	23.74MiB	1.11MiB	23.12KiB	0	0.0466963	False	False
http_pipelines_blackhole_acks	-8.66KiB	-0.72	99.61%	1.17MiB	106.05KiB	2.16KiB	0	0.08868	1.16MiB	101.84KiB	2.07KiB	0	0.0857822	False	False
http_pipelines_blackhole	-20.26KiB	-1.16	100.00%	1.7MiB	58.39KiB	1.19KiB	0	0.0334433	1.68MiB	114.55KiB	2.33KiB	0	0.0663728	False	False
syslog_loki	-233.14KiB	-1.54	100.00%	14.77MiB	271.41KiB	5.56KiB	0	0.0179402	14.54MiB	717.68KiB	14.59KiB	0	0.0481816	False	False
datadog_agent_remap_datadog_logs	-1.08MiB	-1.75	100.00%	62.04MiB	1.05MiB	22.12KiB	0	0.0170012	60.96MiB	4.44MiB	92.41KiB	0	0.0727827	False	False