POC for demo mode

[StephenWakely]

This is just a proof of concept for demo mode. See #14947

It implements demo mode for the stdin source, configured as:

[sources.stdin]
type = "stdin"
demo_mode = true

This is most definitely NOT a final version!

[netlify]

✅ Deploy Preview for vector-project canceled.

Name	Link
🔨 Latest commit	af71829
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/635fba46935c4e0008ceab39

[netlify]

❌ Deploy Preview for vrl-playground failed.

Name	Link
🔨 Latest commit	c794045
🔍 Latest deploy log	https://app.netlify.com/sites/vrl-playground/deploys/635fb9ba83f4200008d6e558

[netlify]

❌ Deploy Preview for vrl-playground failed.

Name	Link
🔨 Latest commit	af71829
🔍 Latest deploy log	https://app.netlify.com/sites/vrl-playground/deploys/635fba465bc1cf0008bf1be0

[github-actions]

Soak Test Results

Baseline: ac69a47
Comparison: af71829
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
syslog_loki	458.24KiB	2.93	100.00%	15.28MiB	758.11KiB	15.53KiB	0	0.0484388	15.73MiB	780.17KiB	15.86KiB	0	0.0484306	False	False
syslog_log2metric_splunk_hec_metrics	377.21KiB	2.28	100.00%	16.13MiB	582.92KiB	11.87KiB	0	0.0352913	16.5MiB	621.7KiB	12.67KiB	0	0.0367984	False	False
syslog_humio_logs	219.94KiB	1.34	100.00%	16.01MiB	121.48KiB	2.48KiB	0	0.00740676	16.23MiB	110.0KiB	2.25KiB	0	0.00661779	False	False
syslog_splunk_hec_logs	182.41KiB	1.12	100.00%	15.86MiB	695.72KiB	14.17KiB	0	0.0428291	16.04MiB	639.9KiB	13.05KiB	0	0.0389553	False	False
datadog_agent_remap_blackhole	401.42KiB	0.7	99.99%	56.28MiB	3.56MiB	74.29KiB	0	0.0632844	56.67MiB	3.37MiB	70.2KiB	0	0.0594139	False	False
http_pipelines_blackhole_acks	5.93KiB	0.49	97.74%	1.18MiB	102.98KiB	2.1KiB	0	0.085103	1.19MiB	75.17KiB	1.53KiB	0	0.0618158	False	False
http_pipelines_blackhole	6.35KiB	0.39	99.95%	1.59MiB	32.08KiB	671.48B	0	0.0197147	1.6MiB	83.34KiB	1.7KiB	0	0.0510093	False	False
datadog_agent_remap_blackhole_acks	87.73KiB	0.15	58.84%	56.67MiB	3.97MiB	82.66KiB	0	0.0700137	56.76MiB	3.24MiB	67.66KiB	0	0.0570435	False	False
splunk_hec_to_splunk_hec_logs_acks	16.39KiB	0.07	50.68%	23.75MiB	862.28KiB	17.54KiB	0	0.0354472	23.77MiB	798.82KiB	16.26KiB	0	0.0328166	False	False
splunk_hec_to_splunk_hec_logs_noack	10.37KiB	0.04	64.74%	23.83MiB	430.09KiB	8.78KiB	0	0.0176235	23.84MiB	337.09KiB	6.88KiB	0	0.0138068	False	False
splunk_hec_indexer_ack_blackhole	3.46KiB	0.01	10.58%	23.74MiB	903.98KiB	18.38KiB	0	0.0371716	23.75MiB	907.51KiB	18.46KiB	0	0.0373112	False	False
enterprise_http_to_http	-4.08KiB	-0.02	40.12%	23.85MiB	273.74KiB	5.59KiB	0	0.0112073	23.84MiB	262.23KiB	5.36KiB	0	0.0107377	False	False
syslog_regex_logs2metric_ddmetrics	-2.34KiB	-0.03	6.87%	7.45MiB	942.11KiB	19.19KiB	0	0.12345	7.45MiB	943.66KiB	19.24KiB	0	0.123691	False	False
file_to_blackhole	-75.04KiB	-0.08	56.81%	95.34MiB	3.06MiB	63.49KiB	0	0.0321171	95.27MiB	3.43MiB	71.29KiB	0	0.0359859	False	False
http_to_http_json	-39.35KiB	-0.16	99.52%	23.85MiB	384.5KiB	7.85KiB	0	0.0157434	23.81MiB	564.15KiB	11.52KiB	0	0.0231361	False	False
http_pipelines_no_grok_blackhole	-24.61KiB	-0.23	73.86%	10.52MiB	373.93KiB	7.63KiB	0	0.0347131	10.49MiB	1009.63KiB	20.54KiB	0	0.0939404	False	False
http_to_http_noack	-60.48KiB	-0.25	99.21%	23.83MiB	521.42KiB	10.66KiB	0	0.0213649	23.77MiB	987.58KiB	20.12KiB	0	0.0405657	False	False
fluent_elasticsearch	-365.68KiB	-0.45	100.00%	79.47MiB	53.35KiB	1.08KiB	0	0.000655429	79.12MiB	4.01MiB	82.34KiB	0	0.0506734	False	False
http_to_http_acks	-80.94KiB	-0.46	26.72%	17.33MiB	8.04MiB	168.08KiB	0	0.463834	17.25MiB	8.01MiB	167.13KiB	0	0.464095	True	True
splunk_hec_route_s3	-170.33KiB	-0.77	99.60%	21.53MiB	2.03MiB	42.25KiB	0	0.0942476	21.37MiB	1.98MiB	41.48KiB	0	0.0928775	False	False
syslog_log2metric_humio_metrics	-78.01KiB	-0.78	100.00%	9.8MiB	215.77KiB	4.41KiB	0	0.0215021	9.72MiB	397.7KiB	8.1KiB	0	0.0399418	False	False
datadog_agent_remap_datadog_logs	-664.35KiB	-1.1	100.00%	59.22MiB	403.83KiB	8.26KiB	0	0.0066582	58.57MiB	3.55MiB	73.94KiB	0	0.0605773	False	False
datadog_agent_remap_datadog_logs_acks	-743.14KiB	-1.19	100.00%	60.97MiB	2.74MiB	57.42KiB	0	0.0450091	60.25MiB	4.26MiB	88.7KiB	0	0.0707122	False	False
socket_to_socket_blackhole	-358.78KiB	-1.51	100.00%	23.28MiB	216.02KiB	4.41KiB	0	0.00905986	22.93MiB	185.08KiB	3.78KiB	0	0.00788061	False	False
http_text_to_http_json	-1.66MiB	-4.06	100.00%	40.82MiB	701.7KiB	14.32KiB	0	0.0167854	39.16MiB	772.31KiB	15.77KiB	0	0.0192558	False	False