enhancement(datadog_events sink): Adhere to instrumentation spec

[neuronull]

Closes: #14232

Epic: #13995

• Add ComponentEventsDropped emission
• Use assert_sink_compliance and assert_sink_error

Note- I believe CI will fail for this because of the double emission bug I introduced, but once #14356 is in, I will merge that in here.

[netlify]

✅ Deploy Preview for vector-project canceled.

Name	Link
🔨 Latest commit	3acdf87
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/631ba2cd0e69a0000759c8e0

[github-actions]

Soak Test Results

Baseline: 675cbf4
Comparison: 40949cd
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
syslog_log2metric_splunk_hec_metrics	350.54KiB	1.99	100.00%	17.22MiB	980.67KiB	20.01KiB	0	0.0556097	17.56MiB	1.15MiB	23.86KiB	0	0.0652436	False	False
datadog_agent_remap_datadog_logs_acks	1.15MiB	1.97	100.00%	58.19MiB	3.96MiB	82.74KiB	0	0.0679797	59.34MiB	4.7MiB	97.87KiB	0	0.0792106	False	False
syslog_humio_logs	188.54KiB	1.13	100.00%	16.32MiB	109.3KiB	2.23KiB	0	0.006539	16.5MiB	123.31KiB	2.52KiB	0	0.00729517	False	False
syslog_regex_logs2metric_ddmetrics	139.2KiB	1.09	100.00%	12.52MiB	610.88KiB	12.44KiB	0	0.0476336	12.66MiB	565.11KiB	11.52KiB	0	0.043592	False	False
datadog_agent_remap_blackhole_acks	568.31KiB	0.92	100.00%	60.24MiB	5.28MiB	109.84KiB	0	0.0875663	60.79MiB	3.94MiB	82.52KiB	0	0.0648199	False	False
http_pipelines_blackhole_acks	6.29KiB	0.51	93.88%	1.22MiB	134.05KiB	2.73KiB	0	0.107685	1.22MiB	96.24KiB	1.96KiB	0	0.0769244	False	False
splunk_hec_to_splunk_hec_logs_noack	9.43KiB	0.04	60.73%	23.83MiB	427.33KiB	8.72KiB	0	0.01751	23.84MiB	330.59KiB	6.75KiB	0	0.0135408	False	False
splunk_hec_indexer_ack_blackhole	-853.46B	-0	2.57%	23.75MiB	891.2KiB	18.13KiB	0	0.0366336	23.75MiB	905.91KiB	18.43KiB	0	0.0372392	False	False
enterprise_http_to_http	-2.81KiB	-0.01	29.37%	23.85MiB	255.79KiB	5.22KiB	0	0.010473	23.84MiB	260.54KiB	5.33KiB	0	0.0106686	False	False
syslog_splunk_hec_logs	-5.34KiB	-0.03	10.83%	15.35MiB	1.32MiB	27.55KiB	0	0.0861206	15.34MiB	1.33MiB	27.88KiB	0	0.0868614	False	False
file_to_blackhole	-39.52KiB	-0.04	27.72%	95.33MiB	3.5MiB	72.57KiB	0	0.0367223	95.29MiB	4.06MiB	84.51KiB	0	0.0426027	False	False
splunk_hec_to_splunk_hec_logs_acks	-11.82KiB	-0.05	38.53%	23.77MiB	789.28KiB	16.06KiB	0	0.0324233	23.76MiB	842.58KiB	17.14KiB	0	0.0346299	False	False
http_to_http_json	-19.34KiB	-0.08	91.69%	23.85MiB	331.05KiB	6.76KiB	0	0.0135541	23.83MiB	433.93KiB	8.88KiB	0	0.0177801	False	False
http_text_to_http_json	-96.61KiB	-0.25	99.99%	37.98MiB	850.18KiB	17.35KiB	0	0.0218559	37.89MiB	869.02KiB	17.74KiB	0	0.0223958	False	False
splunk_hec_route_s3	-65.92KiB	-0.35	68.79%	18.61MiB	2.24MiB	46.61KiB	0	0.120279	18.55MiB	2.18MiB	45.61KiB	0	0.117554	False	False
http_to_http_noack	-86.74KiB	-0.36	99.99%	23.85MiB	252.36KiB	5.16KiB	0	0.010333	23.76MiB	1.02MiB	21.23KiB	0	0.0428326	False	False
http_pipelines_blackhole	-9.73KiB	-0.55	99.98%	1.72MiB	42.0KiB	879.13B	0	0.0238919	1.71MiB	122.45KiB	2.49KiB	0	0.0700475	False	False
fluent_elasticsearch	-488.78KiB	-0.6	100.00%	79.47MiB	53.48KiB	1.08KiB	0	0.000657066	79.0MiB	4.88MiB	100.25KiB	0	0.0617688	False	False
http_pipelines_no_grok_blackhole	-72.23KiB	-0.65	99.93%	10.87MiB	58.21KiB	1.19KiB	0	0.0052273	10.8MiB	1.02MiB	21.14KiB	0	0.0939513	False	False
datadog_agent_remap_datadog_logs	-438.4KiB	-0.69	100.00%	61.86MiB	240.25KiB	4.92KiB	0	0.00379218	61.43MiB	4.12MiB	85.78KiB	0	0.0670428	False	False
syslog_log2metric_humio_metrics	-135.9KiB	-1.03	100.00%	12.9MiB	178.56KiB	3.65KiB	0	0.0135196	12.76MiB	551.5KiB	11.22KiB	0	0.042191	False	False
http_to_http_acks	-183.49KiB	-1.03	57.04%	17.36MiB	7.87MiB	164.44KiB	0	0.453098	17.18MiB	7.86MiB	164.05KiB	0	0.457585	True	True
datadog_agent_remap_blackhole	-817.72KiB	-1.41	100.00%	56.52MiB	5.62MiB	117.13KiB	0	0.0994407	55.72MiB	5.26MiB	109.73KiB	0	0.0944688	False	False
socket_to_socket_blackhole	-342.49KiB	-1.46	100.00%	22.84MiB	476.87KiB	9.73KiB	0	0.0203813	22.51MiB	600.92KiB	12.27KiB	0	0.0260649	False	False
syslog_loki	-362.51KiB	-2.39	100.00%	14.84MiB	376.65KiB	7.71KiB	0	0.0247777	14.49MiB	749.07KiB	15.23KiB	0	0.0504813	False	False

[github-actions]

Soak Test Results

Baseline: f705842
Comparison: 3acdf87
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
http_text_to_http_json	886.31KiB	2.28	100.00%	37.96MiB	819.96KiB	16.74KiB	0	0.0210911	38.82MiB	818.73KiB	16.72KiB	0	0.0205898	False	False
syslog_loki	186.57KiB	1.29	100.00%	14.08MiB	537.0KiB	10.99KiB	0	0.0372259	14.27MiB	815.1KiB	16.57KiB	0	0.055782	False	False
splunk_hec_route_s3	164.11KiB	0.86	98.71%	18.71MiB	2.3MiB	47.94KiB	0	0.123016	18.87MiB	2.17MiB	45.32KiB	0	0.114792	False	False
http_pipelines_blackhole_acks	7.72KiB	0.61	99.10%	1.23MiB	123.67KiB	2.52KiB	0	0.0984498	1.23MiB	76.07KiB	1.55KiB	0	0.0601875	False	False
datadog_agent_remap_blackhole	280.41KiB	0.45	99.20%	60.54MiB	4.12MiB	85.77KiB	0	0.0679954	60.81MiB	2.96MiB	61.82KiB	0	0.0487108	False	False
splunk_hec_to_splunk_hec_logs_noack	11.82KiB	0.05	70.49%	23.83MiB	434.58KiB	8.87KiB	0	0.0178078	23.84MiB	341.38KiB	6.97KiB	0	0.0139818	False	False
enterprise_http_to_http	1.4KiB	0.01	15.05%	23.85MiB	253.05KiB	5.17KiB	0	0.0103615	23.85MiB	257.54KiB	5.27KiB	0	0.0105445	False	False
splunk_hec_indexer_ack_blackhole	-5.77KiB	-0.02	17.81%	23.76MiB	877.61KiB	17.86KiB	0	0.0360642	23.75MiB	903.58KiB	18.38KiB	0	0.0371399	False	False
splunk_hec_to_splunk_hec_logs_acks	-6.65KiB	-0.03	22.03%	23.76MiB	807.93KiB	16.44KiB	0	0.0331961	23.76MiB	845.49KiB	17.2KiB	0	0.0347488	False	False
http_to_http_json	-33.63KiB	-0.14	99.26%	23.85MiB	332.09KiB	6.78KiB	0	0.0135971	23.81MiB	516.37KiB	10.55KiB	0	0.0211713	False	False
fluent_elasticsearch	-135.62KiB	-0.17	100.00%	79.47MiB	52.89KiB	1.07KiB	0	0.00064983	79.34MiB	1.32MiB	27.23KiB	0	0.0166659	False	False
file_to_blackhole	-206.3KiB	-0.21	88.99%	95.33MiB	3.43MiB	71.12KiB	0	0.0360091	95.13MiB	5.19MiB	107.74KiB	0	0.0545225	False	False
syslog_regex_logs2metric_ddmetrics	-37.64KiB	-0.29	98.06%	12.53MiB	600.39KiB	12.23KiB	0	0.0467762	12.5MiB	513.49KiB	10.47KiB	0	0.0401236	False	False
datadog_agent_remap_datadog_logs_acks	-214.44KiB	-0.35	94.83%	60.08MiB	2.97MiB	62.14KiB	0	0.0494312	59.87MiB	4.37MiB	91.01KiB	0	0.0730148	False	False
http_to_http_noack	-87.13KiB	-0.36	99.94%	23.83MiB	519.47KiB	10.62KiB	0	0.0212849	23.74MiB	1.11MiB	23.07KiB	0	0.0465817	False	False
http_pipelines_blackhole	-8.11KiB	-0.47	99.88%	1.69MiB	17.69KiB	370.04B	0	0.0102039	1.68MiB	120.99KiB	2.47KiB	0	0.0701252	False	False
socket_to_socket_blackhole	-118.99KiB	-0.49	100.00%	23.48MiB	261.18KiB	5.33KiB	0	0.0108589	23.37MiB	407.45KiB	8.32KiB	0	0.0170244	False	False
datadog_agent_remap_blackhole_acks	-320.02KiB	-0.52	99.44%	60.06MiB	4.43MiB	92.16KiB	0	0.0736681	59.75MiB	3.33MiB	69.58KiB	0	0.0557114	False	False
http_pipelines_no_grok_blackhole	-67.07KiB	-0.61	99.74%	10.82MiB	31.9KiB	666.7B	0	0.00287804	10.76MiB	1.07MiB	22.23KiB	0	0.0992363	False	False
http_to_http_acks	-109.63KiB	-0.62	35.13%	17.25MiB	8.15MiB	170.3KiB	0	0.472108	17.15MiB	8.14MiB	170.05KiB	0	0.474857	True	True
syslog_log2metric_splunk_hec_metrics	-128.37KiB	-0.71	100.00%	17.66MiB	526.2KiB	10.73KiB	0	0.0290972	17.53MiB	778.34KiB	15.84KiB	0	0.0433475	False	False
syslog_splunk_hec_logs	-194.07KiB	-1.19	100.00%	15.99MiB	1014.86KiB	20.65KiB	0	0.0619872	15.8MiB	915.55KiB	18.68KiB	0	0.0565922	False	False
syslog_humio_logs	-209.26KiB	-1.25	100.00%	16.41MiB	110.86KiB	2.26KiB	0	0.00659426	16.21MiB	325.28KiB	6.66KiB	0	0.0195931	False	False
datadog_agent_remap_datadog_logs	-1012.2KiB	-1.62	100.00%	61.05MiB	2.34MiB	49.05KiB	0	0.0382921	60.06MiB	4.54MiB	94.7KiB	0	0.0756352	False	False
syslog_log2metric_humio_metrics	-218.46KiB	-1.69	100.00%	12.63MiB	306.59KiB	6.25KiB	0	0.0237032	12.42MiB	449.27KiB	9.15KiB	0	0.0353311	False	False