enhancement(internal_logs, internal_metrics source): Add test helpers to assert source compliance

[neuronull]

Closes: #14129.
Closes: #14130.

Part of #13995.

• Added test helper assertions
• Added BytesReceived event emission to internal_metrics in order to comply

[netlify]

✅ Deploy Preview for vector-project ready!

Name	Link
🔨 Latest commit	b36dda7
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/630946f1fe61a500091a9468
😎 Deploy Preview	https://deploy-preview-14133--vector-project.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[jszwedko]

Sweet!

[github-actions]

Soak Test Results

Baseline: 1295eac
Comparison: b36dda7
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
http_to_http_acks	203.33KiB	1.16	62.91%	17.16MiB	7.49MiB	156.52KiB	0	0.436337	17.36MiB	7.89MiB	164.74KiB	0	0.454568	True	True
syslog_log2metric_humio_metrics	137.55KiB	1.09	100.00%	12.35MiB	337.11KiB	6.88KiB	0	0.0266522	12.48MiB	520.04KiB	10.59KiB	0	0.0406726	False	False
splunk_hec_to_splunk_hec_logs_noack	14.6KiB	0.06	79.54%	23.83MiB	455.93KiB	9.31KiB	0	0.018684	23.84MiB	331.32KiB	6.76KiB	0	0.0135696	False	False
splunk_hec_to_splunk_hec_logs_acks	-2.82KiB	-0.01	9.09%	23.75MiB	854.48KiB	17.38KiB	0	0.0351275	23.75MiB	862.87KiB	17.55KiB	0	0.0354763	False	False
enterprise_http_to_http	-2.42KiB	-0.01	26.10%	23.85MiB	248.55KiB	5.07KiB	0	0.0101762	23.84MiB	255.04KiB	5.22KiB	0	0.0104429	False	False
splunk_hec_indexer_ack_blackhole	-8.47KiB	-0.03	26.64%	23.76MiB	857.13KiB	17.45KiB	0	0.0352168	23.76MiB	871.85KiB	17.74KiB	0	0.035834	False	False
http_to_http_json	-20.61KiB	-0.08	91.56%	23.84MiB	352.18KiB	7.19KiB	0	0.0144227	23.82MiB	466.28KiB	9.53KiB	0	0.0191114	False	False
file_to_blackhole	-92.58KiB	-0.09	58.11%	95.36MiB	3.56MiB	73.61KiB	0	0.0373023	95.27MiB	4.23MiB	87.72KiB	0	0.0444065	False	False
syslog_regex_logs2metric_ddmetrics	-19.06KiB	-0.15	71.79%	12.65MiB	660.78KiB	13.45KiB	0	0.0510077	12.63MiB	566.1KiB	11.54KiB	0	0.0437632	False	False
http_pipelines_blackhole_acks	-1.83KiB	-0.15	48.63%	1.21MiB	109.83KiB	2.24KiB	0	0.088593	1.21MiB	82.88KiB	1.69KiB	0	0.0669551	False	False
syslog_splunk_hec_logs	-55.44KiB	-0.33	99.67%	16.45MiB	721.7KiB	14.69KiB	0	0.0428457	16.39MiB	579.72KiB	11.83KiB	0	0.0345299	False	False
http_to_http_noack	-93.75KiB	-0.38	99.98%	23.83MiB	509.09KiB	10.4KiB	0	0.0208589	23.74MiB	1.11MiB	23.23KiB	0	0.0469446	False	False
fluent_elasticsearch	-413.33KiB	-0.51	100.00%	79.47MiB	54.78KiB	1.11KiB	0	0.000672997	79.07MiB	3.69MiB	75.7KiB	0	0.0466455	False	False
http_pipelines_blackhole	-9.27KiB	-0.59	99.99%	1.53MiB	46.18KiB	965.82B	0	0.0294269	1.52MiB	108.52KiB	2.21KiB	0	0.0695644	False	False
syslog_humio_logs	-115.02KiB	-0.67	100.00%	16.76MiB	111.21KiB	2.27KiB	0	0.00647826	16.65MiB	133.09KiB	2.72KiB	0	0.00780497	False	False
http_pipelines_no_grok_blackhole	-80.22KiB	-0.72	99.88%	10.82MiB	353.38KiB	7.21KiB	0	0.0318906	10.74MiB	1.14MiB	23.66KiB	0	0.105802	False	False
splunk_hec_route_s3	-153.89KiB	-0.8	98.21%	18.7MiB	2.21MiB	46.02KiB	0	0.118106	18.55MiB	2.2MiB	45.9KiB	0	0.11835	False	False
syslog_log2metric_splunk_hec_metrics	-164.71KiB	-0.89	100.00%	18.1MiB	674.12KiB	13.73KiB	0	0.0363633	17.94MiB	771.41KiB	15.7KiB	0	0.041984	False	False
datadog_agent_remap_blackhole	-638.44KiB	-1.02	100.00%	61.29MiB	4.6MiB	95.72KiB	0	0.0750091	60.67MiB	3.33MiB	69.48KiB	0	0.0548809	False	False
datadog_agent_remap_datadog_logs_acks	-1.64MiB	-2.67	100.00%	61.56MiB	3.22MiB	67.34KiB	0	0.0523244	59.92MiB	4.32MiB	89.86KiB	0	0.0720335	False	False
datadog_agent_remap_datadog_logs	-2.33MiB	-3.77	100.00%	61.73MiB	2.9MiB	60.77KiB	0	0.0469435	59.4MiB	4.84MiB	100.83KiB	0	0.0814712	False	False
datadog_agent_remap_blackhole_acks	-3.17MiB	-5.1	100.00%	62.14MiB	4.86MiB	101.11KiB	0	0.0781123	58.97MiB	4.21MiB	88.32KiB	0	0.0714516	False	False
http_text_to_http_json	-1.98MiB	-5.18	100.00%	38.25MiB	1007.89KiB	20.57KiB	0	0.0257245	36.27MiB	855.69KiB	17.48KiB	0	0.0230337	False	False
syslog_loki	-983.9KiB	-6.59	100.00%	14.57MiB	425.38KiB	8.7KiB	0	0.028502	13.61MiB	710.46KiB	14.44KiB	0	0.050964	False	False
socket_to_socket_blackhole	-1.98MiB	-8.49	100.00%	23.35MiB	342.17KiB	6.98KiB	0	0.0143071	21.37MiB	896.41KiB	18.3KiB	0	0.0409582	False	False

[spencergilbert]

I'm not 100% on the protocol here, perhaps it shouldn't be required in the spec?

[neuronull]

Do you mean that you aren't 100% convinced that protocol needs to be a parameter here?
I think I was mimicking InternalLogsBytesReceived.

[spencergilbert]

Yeah, I'm not sure internal makes sense as a "protocol" but I think it's in the spec for these. May be a place to re-evaluate if it's just noise later on.