enhancement(vdev): Inject labels block to ComposeConfig

[jonathanpv]

Problem

Injecting labels to the temporary compose file we generate

Solution

To avoid modifying every compose config file, the solution is to load and parse the compose file in vdev, inject the labels, and write it to a temporary file before launching docker-compose. The main changes made in this PR are:

1. Added labels field to the ComposeService struct in config.rs.
2. Implemented the inject_labels function in integration.rs, which adds the required labels to the compose configuration.
3. Modified the labels method in runner.rs to collect labels from the process list provided by docker ps --all.

By implementing these changes, we can now inject labels into the configuration and store state data without having to modify every compose file.

Testing

Currently testing is done by ensuring the changes don't break any integration tests in CI

Related Issues

• Ref Detect vdev integration running state #16381

[netlify]

✅ Deploy Preview for vector-project ready!

Name	Link
🔨 Latest commit	36fa046
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/642da49c6ad503000877611a
😎 Deploy Preview	https://deploy-preview-17063--vector-project.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[netlify]

❌ Deploy Preview for vrl-playground failed.

Name	Link
🔨 Latest commit	36fa046
🔍 Latest deploy log	https://app.netlify.com/sites/vrl-playground/deploys/642da49c6d462100084be5a8

[jonathanpv]

Haven't tested this yet fully, hence this is marked as a draft. However I am curious if the general step of reading from command.check_output().lines() is what we expect.

Would also like suggestions on what the return value should be

[datadog-vectordotdev]

Datadog Report

Branch report: jonathanpv/vdev-inject-labels
Commit report: a9e0061

✅ vector: 0 Failed, 0 New Flaky, 2138 Passed, 0 Skipped, 9m 39.8s Wall Time

[github-actions]

Regression Detector Results

Run ID: 9dc71497-2210-4080-a01b-5cfc3778d2c9
Baseline: 6e6f1eb
Comparison: 36fa046
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

Changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%:

experiment	goal	Δ mean	Δ mean %	confidence
syslog_log2metric_humio_metrics	ingress throughput	-333.85KiB/CPU-s	-5.01	100.00%

Fine details of change detection per experiment.

experiment	goal	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
http_text_to_http_json	ingress throughput	720.81KiB/CPU-s	2.91	100.00%	24.22MiB/CPU-s	609.67KiB/CPU-s	7.5KiB/CPU-s	0.0	0.024577	24.93MiB/CPU-s	888.22KiB/CPU-s	10.93KiB/CPU-s	0.0	0.034795	False	False
datadog_agent_remap_datadog_logs_acks	ingress throughput	546.16KiB/CPU-s	1.67	100.00%	31.97MiB/CPU-s	1.36MiB/CPU-s	17.09KiB/CPU-s	0.0	0.042407	32.51MiB/CPU-s	1.31MiB/CPU-s	16.52KiB/CPU-s	0.0	0.040336	False	False
syslog_log2metric_splunk_hec_metrics	ingress throughput	129.6KiB/CPU-s	1.36	100.00%	9.33MiB/CPU-s	334.73KiB/CPU-s	4.12KiB/CPU-s	0.0	0.035027	9.46MiB/CPU-s	301.78KiB/CPU-s	3.71KiB/CPU-s	0.0	0.031156	False	False
syslog_loki	ingress throughput	70.65KiB/CPU-s	0.81	100.00%	8.49MiB/CPU-s	238.4KiB/CPU-s	2.93KiB/CPU-s	0.0	0.027407	8.56MiB/CPU-s	213.31KiB/CPU-s	2.62KiB/CPU-s	0.0	0.024325	False	False
file_to_blackhole	egress throughput	24.73KiB/CPU-s	0.39	11.24%	6.16MiB/CPU-s	4.69MiB/CPU-s	125.58KiB/CPU-s	0.0	0.760083	6.19MiB/CPU-s	4.5MiB/CPU-s	121.86KiB/CPU-s	0.0	0.727626	True	True
splunk_hec_route_s3	ingress throughput	28.79KiB/CPU-s	0.24	99.12%	11.5MiB/CPU-s	623.79KiB/CPU-s	7.68KiB/CPU-s	0.0	0.052954	11.53MiB/CPU-s	609.19KiB/CPU-s	7.86KiB/CPU-s	0.0	0.051589	False	False
socket_to_socket_blackhole	ingress throughput	22.77KiB/CPU-s	0.17	99.78%	13.38MiB/CPU-s	437.25KiB/CPU-s	5.38KiB/CPU-s	0.0	0.031903	13.41MiB/CPU-s	417.56KiB/CPU-s	5.14KiB/CPU-s	0.0	0.030416	False	False
syslog_humio_logs	ingress throughput	7.09KiB/CPU-s	0.08	95.31%	9.01MiB/CPU-s	156.95KiB/CPU-s	1.93KiB/CPU-s	0.0	0.017007	9.02MiB/CPU-s	244.14KiB/CPU-s	3.0KiB/CPU-s	0.0	0.026435	False	False
enterprise_http_to_http	ingress throughput	5.79KiB/CPU-s	0.04	88.88%	13.62MiB/CPU-s	253.16KiB/CPU-s	3.12KiB/CPU-s	0.0	0.018152	13.62MiB/CPU-s	152.5KiB/CPU-s	1.88KiB/CPU-s	0.0	0.01093	False	False
splunk_hec_to_splunk_hec_logs_noack	ingress throughput	1.14KiB/CPU-s	0.01	19.21%	13.61MiB/CPU-s	273.06KiB/CPU-s	3.36KiB/CPU-s	0.0	0.019586	13.62MiB/CPU-s	265.1KiB/CPU-s	3.26KiB/CPU-s	0.0	0.019013	False	False
syslog_splunk_hec_logs	ingress throughput	-232.05B/CPU-s	-0.00	3.77%	9.07MiB/CPU-s	283.24KiB/CPU-s	3.48KiB/CPU-s	0.0	0.030506	9.07MiB/CPU-s	267.32KiB/CPU-s	3.29KiB/CPU-s	0.0	0.028792	False	False
splunk_hec_to_splunk_hec_logs_acks	ingress throughput	-149.83B/CPU-s	-0.00	1.92%	13.61MiB/CPU-s	355.41KiB/CPU-s	4.37KiB/CPU-s	0.0	0.025493	13.61MiB/CPU-s	345.04KiB/CPU-s	4.24KiB/CPU-s	0.0	0.02475	False	False
http_to_http_noack	ingress throughput	-198.62B/CPU-s	-0.00	2.63%	13.61MiB/CPU-s	337.76KiB/CPU-s	4.16KiB/CPU-s	0.0	0.024231	13.61MiB/CPU-s	339.49KiB/CPU-s	4.18KiB/CPU-s	0.0	0.024355	False	False
fluent_elasticsearch	ingress throughput	28.78B/CPU-s	0.00	4.33%	45.41MiB/CPU-s	30.06KiB/CPU-s	374.55B/CPU-s	0.0	0.000646	45.41MiB/CPU-s	30.06KiB/CPU-s	374.6B/CPU-s	0.0	0.000646	False	False
splunk_hec_indexer_ack_blackhole	ingress throughput	-1.25KiB/CPU-s	-0.01	21.52%	13.62MiB/CPU-s	259.25KiB/CPU-s	3.19KiB/CPU-s	0.0	0.018594	13.61MiB/CPU-s	265.05KiB/CPU-s	3.26KiB/CPU-s	0.0	0.019011	False	False
datadog_agent_remap_datadog_logs	ingress throughput	-11.34KiB/CPU-s	-0.03	46.09%	32.55MiB/CPU-s	1.12MiB/CPU-s	14.07KiB/CPU-s	0.0	0.034307	32.54MiB/CPU-s	972.01KiB/CPU-s	11.96KiB/CPU-s	0.0	0.029168	False	False
http_to_http_json	ingress throughput	-24.6KiB/CPU-s	-0.18	100.00%	13.6MiB/CPU-s	260.91KiB/CPU-s	3.21KiB/CPU-s	0.0	0.01873	13.58MiB/CPU-s	292.53KiB/CPU-s	3.6KiB/CPU-s	0.0	0.021038	False	False
http_to_http_acks	ingress throughput	-24.49KiB/CPU-s	-0.46	39.05%	5.23MiB/CPU-s	2.7MiB/CPU-s	34.05KiB/CPU-s	0.0	0.516093	5.21MiB/CPU-s	2.68MiB/CPU-s	33.77KiB/CPU-s	0.0	0.514363	True	False
datadog_agent_remap_blackhole_acks	ingress throughput	-149.65KiB/CPU-s	-0.47	100.00%	30.92MiB/CPU-s	1.32MiB/CPU-s	16.58KiB/CPU-s	0.0	0.042574	30.77MiB/CPU-s	1.29MiB/CPU-s	16.21KiB/CPU-s	0.0	0.041797	False	False
datadog_agent_remap_blackhole	ingress throughput	-420.31KiB/CPU-s	-1.35	100.00%	30.38MiB/CPU-s	1.91MiB/CPU-s	24.01KiB/CPU-s	0.0	0.062738	29.97MiB/CPU-s	1.14MiB/CPU-s	14.4KiB/CPU-s	0.0	0.038129	False	False
otlp_grpc_to_blackhole	ingress throughput	-22.77KiB/CPU-s	-2.10	100.00%	1.06MiB/CPU-s	50.44KiB/CPU-s	635.54B/CPU-s	0.0	0.046518	1.04MiB/CPU-s	44.39KiB/CPU-s	559.34B/CPU-s	0.0	0.041809	False	False
otlp_http_to_blackhole	ingress throughput	-34.62KiB/CPU-s	-2.14	100.00%	1.58MiB/CPU-s	103.82KiB/CPU-s	1.28KiB/CPU-s	0.0	0.064125	1.55MiB/CPU-s	108.75KiB/CPU-s	1.34KiB/CPU-s	0.0	0.068642	False	False
syslog_regex_logs2metric_ddmetrics	ingress throughput	-188.62KiB/CPU-s	-4.90	100.00%	3.76MiB/CPU-s	485.68KiB/CPU-s	5.98KiB/CPU-s	0.0	0.126261	3.57MiB/CPU-s	362.12KiB/CPU-s	4.46KiB/CPU-s	0.0	0.098995	True	True
syslog_log2metric_humio_metrics	ingress throughput	-333.85KiB/CPU-s	-5.01	100.00%	6.51MiB/CPU-s	290.82KiB/CPU-s	3.58KiB/CPU-s	0.0	0.043652	6.18MiB/CPU-s	354.86KiB/CPU-s	4.37KiB/CPU-s	0.0	0.056073	False	False

[jonathanpv]

This work will be paused as it will wait on the progress of implementing the Runner block first.

• Replace custom integration runner in vdev with compose #17051