Skip to content

chore: upgrade to rust 1.66.0 #15093

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Jan 5, 2023
Merged

chore: upgrade to rust 1.66.0 #15093

merged 16 commits into from
Jan 5, 2023

Conversation

lukesteensen
Copy link
Member

I worked through a decent amount of the clippy changes, but ended up hitting an ICE. Pushing this now for visibility.

Signed-off-by: Luke Steensen <luke.steensen@gmail.com>
@netlify
Copy link

netlify bot commented Nov 3, 2022

Deploy Preview for vector-project ready!

Name Link
🔨 Latest commit 06f7b22
🔍 Latest deploy log https://app.netlify.com/sites/vector-project/deploys/63b6f649636d750009cd0d95
😎 Deploy Preview https://deploy-preview-15093--vector-project.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@netlify
Copy link

netlify bot commented Nov 3, 2022

Deploy Preview for vrl-playground ready!

Name Link
🔨 Latest commit 06f7b22
🔍 Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/63b6f649b142710008d7cffe
😎 Deploy Preview https://deploy-preview-15093--vrl-playground.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@github-actions github-actions bot added domain: core Anything related to core crates i.e. vector-core, core-common, etc domain: vrl Anything related to the Vector Remap Language labels Nov 3, 2022
@github-actions
Copy link

github-actions bot commented Nov 3, 2022

Regression Test Results

Baseline: 981f449
Comparison: c0de868
Total vector CPUs: 4

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json 690.8KiB/CPU-s 1.85 100.00% 36.42MiB/CPU-s 1.58MiB/CPU-s 20.89KiB/CPU-s 0.0 0.043370 37.09MiB/CPU-s 913.35KiB/CPU-s 11.79KiB/CPU-s 0.0 0.024045 False False
http_to_http_acks 124.12KiB/CPU-s 1.21 72.38% 10.02MiB/CPU-s 6.07MiB/CPU-s 80.22KiB/CPU-s 0.0 0.605640 10.14MiB/CPU-s 6.12MiB/CPU-s 80.96KiB/CPU-s 0.0 0.603496 True False
syslog_humio_logs 161.08KiB/CPU-s 1.11 100.00% 14.23MiB/CPU-s 431.27KiB/CPU-s 5.57KiB/CPU-s 0.0 0.029601 14.38MiB/CPU-s 220.17KiB/CPU-s 2.84KiB/CPU-s 0.0 0.014946 False False
splunk_hec_route_s3 104.89KiB/CPU-s 0.78 100.00% 13.2MiB/CPU-s 937.66KiB/CPU-s 12.1KiB/CPU-s 0.0 0.069343 13.31MiB/CPU-s 889.26KiB/CPU-s 11.48KiB/CPU-s 0.0 0.065257 False False
syslog_regex_logs2metric_ddmetrics 47.91KiB/CPU-s 0.76 100.00% 6.18MiB/CPU-s 575.65KiB/CPU-s 7.43KiB/CPU-s 0.0 0.090989 6.22MiB/CPU-s 474.96KiB/CPU-s 6.13KiB/CPU-s 0.0 0.074509 False False
http_pipelines_blackhole_acks 5.46KiB/CPU-s 0.75 100.00% 724.52KiB/CPU-s 35.47KiB/CPU-s 468.89B/CPU-s 0.0 0.048951 729.98KiB/CPU-s 30.05KiB/CPU-s 397.1B/CPU-s 0.0 0.041166 False False
socket_to_socket_blackhole 117.39KiB/CPU-s 0.49 100.00% 23.57MiB/CPU-s 515.17KiB/CPU-s 6.65KiB/CPU-s 0.0 0.021344 23.68MiB/CPU-s 439.97KiB/CPU-s 5.68KiB/CPU-s 0.0 0.018140 False False
splunk_hec_to_splunk_hec_logs_noack 71.1KiB/CPU-s 0.38 100.00% 18.33MiB/CPU-s 861.13KiB/CPU-s 11.11KiB/CPU-s 0.0 0.045882 18.4MiB/CPU-s 854.35KiB/CPU-s 11.03KiB/CPU-s 0.0 0.045349 False False
datadog_agent_remap_blackhole 114.98KiB/CPU-s 0.27 100.00% 41.19MiB/CPU-s 1.2MiB/CPU-s 15.81KiB/CPU-s 0.0 0.029040 41.31MiB/CPU-s 1.39MiB/CPU-s 18.41KiB/CPU-s 0.0 0.033727 False False
splunk_hec_to_splunk_hec_logs_acks 48.01KiB/CPU-s 0.26 99.84% 18.11MiB/CPU-s 836.53KiB/CPU-s 10.79KiB/CPU-s 0.0 0.045103 18.16MiB/CPU-s 834.8KiB/CPU-s 10.77KiB/CPU-s 0.0 0.044894 False False
file_to_blackhole 124.86KiB/CPU-s 0.13 37.26% 92.33MiB/CPU-s 13.95MiB/CPU-s 184.12KiB/CPU-s 0.0 0.151049 92.45MiB/CPU-s 13.61MiB/CPU-s 179.62KiB/CPU-s 0.0 0.147252 True False
http_to_http_noack 14.35KiB/CPU-s 0.06 91.63% 23.82MiB/CPU-s 544.57KiB/CPU-s 7.03KiB/CPU-s 0.0 0.022320 23.84MiB/CPU-s 340.63KiB/CPU-s 4.4KiB/CPU-s 0.0 0.013953 False False
http_to_http_json 7.7KiB/CPU-s 0.03 29.88% 22.23MiB/CPU-s 1.07MiB/CPU-s 14.15KiB/CPU-s 0.0 0.048155 22.23MiB/CPU-s 1.08MiB/CPU-s 14.23KiB/CPU-s 0.0 0.048401 False False
splunk_hec_indexer_ack_blackhole 2.94KiB/CPU-s 0.01 28.98% 23.83MiB/CPU-s 441.97KiB/CPU-s 5.7KiB/CPU-s 0.0 0.018112 23.83MiB/CPU-s 425.84KiB/CPU-s 5.49KiB/CPU-s 0.0 0.017449 False False
enterprise_http_to_http -219.55B/CPU-s -0.00 3.54% 23.84MiB/CPU-s 265.25KiB/CPU-s 3.43KiB/CPU-s 0.0 0.010863 23.84MiB/CPU-s 263.69KiB/CPU-s 3.41KiB/CPU-s 0.0 0.010800 False False
otlp_http_to_blackhole -650.37B/CPU-s -0.03 11.13% 2.46MiB/CPU-s 242.77KiB/CPU-s 3.13KiB/CPU-s 0.0 0.096482 2.46MiB/CPU-s 254.43KiB/CPU-s 3.28KiB/CPU-s 0.0 0.101142 True False
datadog_agent_remap_datadog_logs_acks -21.39KiB/CPU-s -0.06 71.81% 33.37MiB/CPU-s 1.23MiB/CPU-s 16.19KiB/CPU-s 0.0 0.036716 33.35MiB/CPU-s 894.27KiB/CPU-s 11.54KiB/CPU-s 0.0 0.026185 False False
fluent_elasticsearch -90.44KiB/CPU-s -0.11 99.71% 79.47MiB/CPU-s 56.24KiB/CPU-s 735.56B/CPU-s 0.0 0.000691 79.39MiB/CPU-s 2.32MiB/CPU-s 30.33KiB/CPU-s 0.0 0.029240 False False
http_pipelines_no_grok_blackhole -8.34KiB/CPU-s -0.14 100.00% 5.72MiB/CPU-s 64.45KiB/CPU-s 852.03B/CPU-s 0.0 0.011005 5.71MiB/CPU-s 48.64KiB/CPU-s 643.16B/CPU-s 0.0 0.008317 False False
syslog_splunk_hec_logs -22.03KiB/CPU-s -0.15 95.88% 14.19MiB/CPU-s 607.78KiB/CPU-s 7.85KiB/CPU-s 0.0 0.041833 14.17MiB/CPU-s 573.51KiB/CPU-s 7.41KiB/CPU-s 0.0 0.039534 False False
datadog_agent_remap_blackhole_acks -81.23KiB/CPU-s -0.19 100.00% 42.38MiB/CPU-s 741.4KiB/CPU-s 9.57KiB/CPU-s 0.0 0.017085 42.3MiB/CPU-s 1.29MiB/CPU-s 17.03KiB/CPU-s 0.0 0.030464 False False
datadog_agent_remap_datadog_logs -62.46KiB/CPU-s -0.21 99.97% 29.73MiB/CPU-s 811.53KiB/CPU-s 10.48KiB/CPU-s 0.0 0.026656 29.67MiB/CPU-s 1.02MiB/CPU-s 13.53KiB/CPU-s 0.0 0.034515 False False
otlp_grpc_to_blackhole -6.39KiB/CPU-s -0.37 100.00% 1.69MiB/CPU-s 69.48KiB/CPU-s 918.41B/CPU-s 0.0 0.040262 1.68MiB/CPU-s 81.08KiB/CPU-s 1.05KiB/CPU-s 0.0 0.047154 False False
syslog_log2metric_humio_metrics -39.85KiB/CPU-s -0.50 100.00% 7.86MiB/CPU-s 274.41KiB/CPU-s 3.54KiB/CPU-s 0.0 0.034097 7.82MiB/CPU-s 263.17KiB/CPU-s 3.4KiB/CPU-s 0.0 0.032863 False False
syslog_log2metric_splunk_hec_metrics -87.79KiB/CPU-s -0.62 100.00% 13.92MiB/CPU-s 479.21KiB/CPU-s 6.19KiB/CPU-s 0.0 0.033626 13.83MiB/CPU-s 468.67KiB/CPU-s 6.05KiB/CPU-s 0.0 0.033091 False False
syslog_loki -124.45KiB/CPU-s -0.86 100.00% 14.09MiB/CPU-s 541.79KiB/CPU-s 6.99KiB/CPU-s 0.0 0.037560 13.96MiB/CPU-s 568.69KiB/CPU-s 7.34KiB/CPU-s 0.0 0.039768 False False
http_pipelines_blackhole -8.01KiB/CPU-s -0.86 100.00% 934.08KiB/CPU-s 19.15KiB/CPU-s 253.43B/CPU-s 0.0 0.020505 926.06KiB/CPU-s 40.17KiB/CPU-s 531.07B/CPU-s 0.0 0.043373 False False

@github-actions
Copy link

github-actions bot commented Nov 3, 2022

Soak Test Results

Baseline: 981f449
Comparison: c0de868
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_humio_logs 953.07KiB 5.86 100.00% 15.88MiB 215.17KiB 4.39KiB 0 0.0132264 16.81MiB 142.46KiB 2.91KiB 0 0.00827187 False False
syslog_log2metric_splunk_hec_metrics 956.61KiB 5.81 100.00% 16.07MiB 604.46KiB 12.31KiB 0 0.0367318 17.0MiB 603.6KiB 12.3KiB 0 0.0346644 False False
syslog_splunk_hec_logs 888.7KiB 5.53 100.00% 15.7MiB 723.39KiB 14.72KiB 0 0.0449988 16.56MiB 542.4KiB 11.07KiB 0 0.0319726 False False
socket_to_socket_blackhole 925.75KiB 3.89 100.00% 23.22MiB 255.01KiB 5.21KiB 0 0.0107212 24.13MiB 410.75KiB 8.39KiB 0 0.016622 False False
syslog_loki 538.92KiB 3.44 100.00% 15.28MiB 134.64KiB 2.76KiB 0 0.00860457 15.8MiB 794.49KiB 16.15KiB 0 0.0490846 False False
syslog_regex_logs2metric_ddmetrics 186.02KiB 2.53 100.00% 7.17MiB 990.3KiB 20.18KiB 0 0.134906 7.35MiB 1.01MiB 21.17KiB 0 0.137965 False False
datadog_agent_remap_datadog_logs_acks 1.01MiB 1.85 100.00% 54.54MiB 3.06MiB 63.97KiB 0 0.0561272 55.54MiB 3.95MiB 82.23KiB 0 0.0711084 False False
syslog_log2metric_humio_metrics 131.09KiB 1.33 100.00% 9.64MiB 245.32KiB 5.01KiB 0 0.0248559 9.76MiB 364.64KiB 7.43KiB 0 0.0364609 False False
datadog_agent_remap_blackhole 613.13KiB 1.05 100.00% 56.86MiB 3.73MiB 77.71KiB 0 0.06558 57.45MiB 2.78MiB 57.93KiB 0 0.0483165 False False
splunk_hec_route_s3 164.12KiB 0.77 98.65% 20.83MiB 2.31MiB 48.15KiB 0 0.11096 20.99MiB 2.19MiB 45.76KiB 0 0.104255 False False
datadog_agent_remap_datadog_logs 427.36KiB 0.75 100.00% 55.29MiB 1.34MiB 28.02KiB 0 0.0241679 55.71MiB 3.75MiB 78.08KiB 0 0.0672959 False False
splunk_hec_to_splunk_hec_logs_noack 10.83KiB 0.04 67.55% 23.83MiB 422.47KiB 8.63KiB 0 0.0173115 23.84MiB 333.72KiB 6.81KiB 0 0.0136685 False False
enterprise_http_to_http -3.84KiB -0.02 38.75% 23.85MiB 266.13KiB 5.43KiB 0 0.0108956 23.84MiB 258.75KiB 5.29KiB 0 0.0105953 False False
splunk_hec_indexer_ack_blackhole -8.27KiB -0.03 25.49% 23.75MiB 860.27KiB 17.5KiB 0 0.0353637 23.74MiB 908.31KiB 18.47KiB 0 0.0373516 False False
file_to_blackhole -51.95KiB -0.05 38.82% 95.34MiB 3.29MiB 68.2KiB 0 0.0344977 95.29MiB 3.67MiB 76.31KiB 0 0.0384905 False False
splunk_hec_to_splunk_hec_logs_acks -16.97KiB -0.07 50.83% 23.76MiB 825.13KiB 16.79KiB 0 0.0339079 23.74MiB 889.27KiB 18.08KiB 0 0.0365691 False False
http_pipelines_no_grok_blackhole -12.72KiB -0.12 43.97% 10.43MiB 194.45KiB 3.97KiB 0 0.0182035 10.42MiB 1.03MiB 21.47KiB 0 0.0989644 False False
http_to_http_json -35.07KiB -0.14 99.04% 23.85MiB 382.29KiB 7.8KiB 0 0.0156519 23.81MiB 540.93KiB 11.05KiB 0 0.0221788 False False
http_to_http_noack -43.1KiB -0.18 89.90% 23.8MiB 759.79KiB 15.51KiB 0 0.0311659 23.76MiB 1.02MiB 21.21KiB 0 0.0427863 False False
fluent_elasticsearch -157.96KiB -0.19 100.00% 79.47MiB 52.51KiB 1.06KiB 0 0.000645156 79.32MiB 1.44MiB 29.74KiB 0 0.0182122 False False
http_pipelines_blackhole_acks -5.44KiB -0.45 94.87% 1.19MiB 103.66KiB 2.11KiB 0 0.0848719 1.19MiB 89.67KiB 1.83KiB 0 0.0737413 False False
datadog_agent_remap_blackhole_acks -369.22KiB -0.63 98.86% 56.83MiB 4.91MiB 102.18KiB 0 0.0863208 56.47MiB 4.98MiB 104.14KiB 0 0.0881659 False False
http_text_to_http_json -437.9KiB -1.09 100.00% 39.38MiB 738.39KiB 15.07KiB 0 0.0183066 38.95MiB 828.78KiB 16.92KiB 0 0.0207733 False False
http_to_http_acks -225.4KiB -1.26 64.74% 17.53MiB 8.29MiB 173.26KiB 0 0.472784 17.31MiB 8.12MiB 169.59KiB 0 0.468954 True True
http_pipelines_blackhole -32.46KiB -1.91 100.00% 1.66MiB 19.07KiB 398.95B 0 0.0111895 1.63MiB 123.55KiB 2.52KiB 0 0.073917 False False

@fuchsnj
Copy link
Member

fuchsnj commented Nov 4, 2022

I ran a full cargo check and cargo build on this without any ICE's. Did you try a cargo clean first?

Ah nevermind, I see it's in clippy in CI.

Looks like there is already a bug report matching the one I got:

rust-lang/rust-clippy#9781

@bruceg
Copy link
Member

bruceg commented Nov 30, 2022

I updated this with current master and fixed a few more lints, but it still has a repeatable ICE when running clippy.

@github-actions
Copy link

Regression Test Results

Baseline: 7e67279
Comparison: 9a20969
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
syslog_log2metric_splunk_hec_metrics 564.72KiB/CPU-s 6.08 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_splunk_hec_metrics 564.72KiB/CPU-s 6.08 100.00% 9.08MiB/CPU-s 260.36KiB/CPU-s 3.36KiB/CPU-s 0.0 0.028007 9.63MiB/CPU-s 193.22KiB/CPU-s 2.49KiB/CPU-s 0.0 0.019594 False False
syslog_loki 390.63KiB/CPU-s 4.53 100.00% 8.42MiB/CPU-s 230.76KiB/CPU-s 2.98KiB/CPU-s 0.0 0.026756 8.8MiB/CPU-s 151.17KiB/CPU-s 1.95KiB/CPU-s 0.0 0.016768 False False
syslog_splunk_hec_logs 382.53KiB/CPU-s 4.10 100.00% 9.12MiB/CPU-s 143.51KiB/CPU-s 1.85KiB/CPU-s 0.0 0.015365 9.49MiB/CPU-s 179.79KiB/CPU-s 2.32KiB/CPU-s 0.0 0.018492 False False
datadog_agent_remap_datadog_logs 1.1MiB/CPU-s 3.70 100.00% 29.83MiB/CPU-s 872.98KiB/CPU-s 11.27KiB/CPU-s 0.0 0.028581 30.93MiB/CPU-s 1.07MiB/CPU-s 14.1KiB/CPU-s 0.0 0.034499 False False
datadog_agent_remap_blackhole 1010.54KiB/CPU-s 3.22 100.00% 30.6MiB/CPU-s 1.5MiB/CPU-s 19.81KiB/CPU-s 0.0 0.048961 31.59MiB/CPU-s 1005.5KiB/CPU-s 12.97KiB/CPU-s 0.0 0.031080 False False
http_text_to_http_json 656.69KiB/CPU-s 2.80 100.00% 22.91MiB/CPU-s 520.0KiB/CPU-s 6.72KiB/CPU-s 0.0 0.022165 23.55MiB/CPU-s 995.6KiB/CPU-s 12.86KiB/CPU-s 0.0 0.041282 False False
syslog_humio_logs 241.29KiB/CPU-s 2.57 100.00% 9.15MiB/CPU-s 331.73KiB/CPU-s 4.28KiB/CPU-s 0.0 0.035395 9.39MiB/CPU-s 439.33KiB/CPU-s 5.68KiB/CPU-s 0.0 0.045700 False False
datadog_agent_remap_datadog_logs_acks 791.21KiB/CPU-s 2.57 100.00% 30.01MiB/CPU-s 979.75KiB/CPU-s 12.64KiB/CPU-s 0.0 0.031879 30.78MiB/CPU-s 868.14KiB/CPU-s 11.21KiB/CPU-s 0.0 0.027538 False False
syslog_regex_logs2metric_ddmetrics 59.07KiB/CPU-s 1.54 100.00% 3.74MiB/CPU-s 438.91KiB/CPU-s 5.67KiB/CPU-s 0.0 0.114454 3.8MiB/CPU-s 477.39KiB/CPU-s 6.16KiB/CPU-s 0.0 0.122601 True False
otlp_grpc_to_blackhole 14.57KiB/CPU-s 1.53 100.00% 954.93KiB/CPU-s 44.23KiB/CPU-s 584.64B/CPU-s 0.0 0.046315 969.51KiB/CPU-s 37.83KiB/CPU-s 500.08B/CPU-s 0.0 0.039014 False False
http_pipelines_no_grok_blackhole 86.57KiB/CPU-s 1.39 100.00% 6.1MiB/CPU-s 50.49KiB/CPU-s 667.45B/CPU-s 0.0 0.008084 6.18MiB/CPU-s 44.57KiB/CPU-s 589.18B/CPU-s 0.0 0.007038 False False
datadog_agent_remap_blackhole_acks 176.78KiB/CPU-s 0.56 100.00% 30.95MiB/CPU-s 503.89KiB/CPU-s 6.51KiB/CPU-s 0.0 0.015896 31.13MiB/CPU-s 1.13MiB/CPU-s 14.89KiB/CPU-s 0.0 0.036184 False False
splunk_hec_route_s3 29.09KiB/CPU-s 0.26 99.18% 10.91MiB/CPU-s 565.83KiB/CPU-s 7.3KiB/CPU-s 0.0 0.050647 10.94MiB/CPU-s 636.91KiB/CPU-s 8.22KiB/CPU-s 0.0 0.056862 False False
syslog_log2metric_humio_metrics 15.01KiB/CPU-s 0.26 100.00% 5.57MiB/CPU-s 154.87KiB/CPU-s 2.0KiB/CPU-s 0.0 0.027167 5.58MiB/CPU-s 181.08KiB/CPU-s 2.34KiB/CPU-s 0.0 0.031682 False False
http_to_http_noack 4.08KiB/CPU-s 0.03 67.31% 13.62MiB/CPU-s 258.32KiB/CPU-s 3.33KiB/CPU-s 0.0 0.018523 13.62MiB/CPU-s 193.35KiB/CPU-s 2.5KiB/CPU-s 0.0 0.013860 False False
splunk_hec_to_splunk_hec_logs_noack 1.77KiB/CPU-s 0.01 29.91% 13.62MiB/CPU-s 259.9KiB/CPU-s 3.35KiB/CPU-s 0.0 0.018639 13.62MiB/CPU-s 246.27KiB/CPU-s 3.18KiB/CPU-s 0.0 0.017660 False False
splunk_hec_indexer_ack_blackhole -560.91B/CPU-s -0.00 9.73% 13.62MiB/CPU-s 244.13KiB/CPU-s 3.15KiB/CPU-s 0.0 0.017506 13.62MiB/CPU-s 247.25KiB/CPU-s 3.19KiB/CPU-s 0.0 0.017731 False False
enterprise_http_to_http -322.19B/CPU-s -0.00 9.19% 13.62MiB/CPU-s 148.85KiB/CPU-s 1.92KiB/CPU-s 0.0 0.010669 13.62MiB/CPU-s 149.57KiB/CPU-s 1.93KiB/CPU-s 0.0 0.010720 False False
splunk_hec_to_splunk_hec_logs_acks 625.43B/CPU-s 0.00 7.01% 13.61MiB/CPU-s 383.66KiB/CPU-s 4.95KiB/CPU-s 0.0 0.027519 13.61MiB/CPU-s 377.71KiB/CPU-s 4.87KiB/CPU-s 0.0 0.027091 False False
file_to_blackhole -23.29KiB/CPU-s -0.04 72.17% 54.5MiB/CPU-s 1.06MiB/CPU-s 14.04KiB/CPU-s 0.0 0.019517 54.47MiB/CPU-s 1.23MiB/CPU-s 16.26KiB/CPU-s 0.0 0.022621 False False
fluent_elasticsearch -16.84KiB/CPU-s -0.04 93.76% 45.41MiB/CPU-s 29.52KiB/CPU-s 386.34B/CPU-s 0.0 0.000635 45.4MiB/CPU-s 706.9KiB/CPU-s 9.03KiB/CPU-s 0.0 0.015205 False False
socket_to_socket_blackhole -50.81KiB/CPU-s -0.39 100.00% 12.64MiB/CPU-s 313.14KiB/CPU-s 4.04KiB/CPU-s 0.0 0.024194 12.59MiB/CPU-s 427.01KiB/CPU-s 5.51KiB/CPU-s 0.0 0.033122 False False
http_to_http_json -69.58KiB/CPU-s -0.50 100.00% 13.62MiB/CPU-s 210.66KiB/CPU-s 2.72KiB/CPU-s 0.0 0.015099 13.56MiB/CPU-s 298.76KiB/CPU-s 3.86KiB/CPU-s 0.0 0.021521 False False
http_pipelines_blackhole -11.43KiB/CPU-s -1.36 100.00% 841.28KiB/CPU-s 28.6KiB/CPU-s 378.46B/CPU-s 0.0 0.033998 829.85KiB/CPU-s 34.77KiB/CPU-s 459.8B/CPU-s 0.0 0.041899 False False
http_pipelines_blackhole_acks -10.22KiB/CPU-s -1.62 100.00% 629.93KiB/CPU-s 18.54KiB/CPU-s 245.03B/CPU-s 0.0 0.029424 619.72KiB/CPU-s 26.62KiB/CPU-s 351.64B/CPU-s 0.0 0.042947 False False
http_to_http_acks -95.02KiB/CPU-s -1.75 93.08% 5.3MiB/CPU-s 2.78MiB/CPU-s 36.77KiB/CPU-s 0.0 0.524285 5.21MiB/CPU-s 2.81MiB/CPU-s 37.17KiB/CPU-s 0.0 0.539284 True False
otlp_http_to_blackhole -30.42KiB/CPU-s -2.10 100.00% 1.42MiB/CPU-s 134.64KiB/CPU-s 1.74KiB/CPU-s 0.0 0.092847 1.39MiB/CPU-s 144.38KiB/CPU-s 1.86KiB/CPU-s 0.0 0.101696 True False

@github-actions
Copy link

Soak Test Results

Baseline: 7e67279
Comparison: 9a20969
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_splunk_hec_metrics 613.15KiB 3.69 100.00% 16.21MiB 610.77KiB 12.44KiB 0 0.0367834 16.81MiB 609.15KiB 12.41KiB 0 0.0353795 False False
syslog_splunk_hec_logs 594.59KiB 3.66 100.00% 15.87MiB 652.24KiB 13.29KiB 0 0.0401311 16.45MiB 662.7KiB 13.51KiB 0 0.0393356 False False
datadog_agent_remap_blackhole 1.76MiB 3.12 100.00% 56.36MiB 3.39MiB 70.6KiB 0 0.0600509 58.13MiB 2.53MiB 52.79KiB 0 0.0435075 False False
syslog_humio_logs 478.19KiB 2.93 100.00% 15.95MiB 153.12KiB 3.13KiB 0 0.00937032 16.42MiB 195.92KiB 4.01KiB 0 0.0116486 False False
datadog_agent_remap_blackhole_acks 1.63MiB 2.89 100.00% 56.34MiB 4.15MiB 86.51KiB 0 0.0737277 57.97MiB 2.69MiB 56.42KiB 0 0.0464716 False False
http_pipelines_blackhole_acks 91.76KiB 2.69 100.00% 3.33MiB 323.09KiB 6.57KiB 0 0.0947001 3.42MiB 227.15KiB 4.63KiB 0 0.0648368 False False
syslog_log2metric_humio_metrics 260.41KiB 2.67 100.00% 9.51MiB 175.46KiB 3.58KiB 0 0.0180068 9.77MiB 386.07KiB 7.86KiB 0 0.0385892 False False
http_text_to_http_json 1017.63KiB 2.56 100.00% 38.81MiB 683.35KiB 13.95KiB 0 0.0171906 39.81MiB 1.31MiB 27.39KiB 0 0.0329151 False False
datadog_agent_remap_datadog_logs_acks 1.23MiB 2.25 100.00% 54.87MiB 2.59MiB 54.09KiB 0 0.0471339 56.1MiB 3.95MiB 82.32KiB 0 0.0704771 False False
syslog_loki 316.98KiB 2.02 100.00% 15.34MiB 398.86KiB 8.16KiB 0 0.0253797 15.65MiB 845.8KiB 17.19KiB 0 0.0527549 False False
datadog_agent_remap_datadog_logs 1.07MiB 1.97 100.00% 54.44MiB 1014.01KiB 20.75KiB 0 0.0181869 55.51MiB 3.77MiB 78.57KiB 0 0.067957 False False
http_pipelines_blackhole 83.16KiB 1.79 100.00% 4.53MiB 55.4KiB 1.13KiB 0 0.0119421 4.61MiB 329.92KiB 6.72KiB 0 0.0698608 False False
splunk_hec_route_s3 336.48KiB 1.49 100.00% 21.99MiB 1.15MiB 23.98KiB 0 0.0523031 22.32MiB 1.24MiB 25.9KiB 0 0.0557159 False False
syslog_regex_logs2metric_ddmetrics 54.95KiB 0.77 94.45% 7.0MiB 976.03KiB 19.89KiB 0 0.136138 7.05MiB 1013.76KiB 20.67KiB 0 0.140325 False False
file_to_blackhole 39.48KiB 0.04 39.84% 95.34MiB 2.5MiB 51.79KiB 0 0.0261989 95.38MiB 2.65MiB 55.1KiB 0 0.0277918 False False
splunk_hec_to_splunk_hec_logs_noack 5.81KiB 0.02 42.92% 23.83MiB 379.4KiB 7.75KiB 0 0.0155428 23.84MiB 328.57KiB 6.71KiB 0 0.0134572 False False
splunk_hec_indexer_ack_blackhole 1.79KiB 0.01 6.46% 23.77MiB 769.5KiB 15.66KiB 0 0.0316097 23.77MiB 765.98KiB 15.59KiB 0 0.0314627 False False
splunk_hec_to_splunk_hec_logs_acks -1.56KiB -0.01 5.38% 23.76MiB 798.28KiB 16.24KiB 0 0.0328057 23.76MiB 809.4KiB 16.46KiB 0 0.033265 False False
enterprise_http_to_http -3.64KiB -0.01 36.97% 23.84MiB 260.11KiB 5.31KiB 0 0.0106507 23.84MiB 262.68KiB 5.37KiB 0 0.0107577 False False
http_to_http_json -42.65KiB -0.17 99.91% 23.85MiB 383.55KiB 7.83KiB 0 0.0156998 23.81MiB 497.67KiB 10.15KiB 0 0.0204071 False False
http_to_http_noack -64.6KiB -0.26 99.48% 23.82MiB 511.8KiB 10.47KiB 0 0.0209755 23.76MiB 1009.91KiB 20.57KiB 0 0.0414996 False False
fluent_elasticsearch -372.34KiB -0.46 100.00% 79.47MiB 53.61KiB 1.04KiB 0 0.00065861 79.11MiB 3.78MiB 74.93KiB 0 0.0477316 False False
otlp_grpc_to_blackhole -8.21KiB -0.48 99.34% 1.65MiB 68.09KiB 1.4KiB 0 0.0401992 1.65MiB 131.32KiB 2.68KiB 0 0.0779089 False False
http_pipelines_no_grok_blackhole -71.47KiB -0.65 99.89% 10.67MiB 43.29KiB 904.89B 0 0.00396259 10.6MiB 1.05MiB 21.92KiB 0 0.0993214 False False
otlp_http_to_blackhole -36.64KiB -1.44 99.99% 2.49MiB 309.56KiB 6.3KiB 0 0.121325 2.46MiB 329.3KiB 6.7KiB 0 0.130942 False False
http_to_http_acks -249.67KiB -2.68 93.28% 9.1MiB 4.61MiB 96.44KiB 0 0.506771 8.86MiB 4.6MiB 96.44KiB 0 0.519078 True True
socket_to_socket_blackhole -779.97KiB -3.27 100.00% 23.29MiB 1.07MiB 22.18KiB 0 0.0457358 22.53MiB 1.39MiB 28.93KiB 0 0.0616906 False False

Signed-off-by: Luke Steensen <luke.steensen@gmail.com>
Signed-off-by: Luke Steensen <luke.steensen@gmail.com>
@github-actions github-actions bot added domain: sinks Anything related to the Vector's sinks domain: sources Anything related to the Vector's sources domain: transforms Anything related to Vector's transform components labels Dec 16, 2022
@lukesteensen lukesteensen changed the title chore: upgrade to rust 1.65.0 chore: upgrade to rust 1.66.0 Dec 16, 2022
Signed-off-by: Luke Steensen <luke.steensen@gmail.com>
Signed-off-by: Luke Steensen <luke.steensen@gmail.com>
@lukesteensen lukesteensen marked this pull request as ready for review December 16, 2022 16:16
Copy link
Member

@bruceg bruceg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM aside from one possible pair of further simplifications and a question.

let value = self.captures.at(*v as usize).unwrap_or("");
let value = self.captures.at(*v).unwrap_or("");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a lot of these conversions being dropped in this PR, but I don't see the variable types as having changed nor do I recall something in the Rust release notes that referenced it. What has enabled this change?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These were all unnecessary casts (i.e. usize as usize) that clippy now seems to detect.

@github-actions
Copy link

Regression Test Results

Run ID: ec6a6cf7-dd6a-4f25-8d71-7ee3df85004c
Baseline: a39ba04
Comparison: 0c01874
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_splunk_hec_metrics 352.66KiB/CPU-s 3.79 100.00% 9.1MiB/CPU-s 198.12KiB/CPU-s 2.56KiB/CPU-s 0.0 0.021266 9.44MiB/CPU-s 241.85KiB/CPU-s 3.12KiB/CPU-s 0.0 0.025013 False False
syslog_humio_logs 299.05KiB/CPU-s 3.33 100.00% 8.76MiB/CPU-s 216.02KiB/CPU-s 2.79KiB/CPU-s 0.0 0.024078 9.05MiB/CPU-s 294.74KiB/CPU-s 3.81KiB/CPU-s 0.0 0.031793 False False
syslog_splunk_hec_logs 263.73KiB/CPU-s 2.97 100.00% 8.66MiB/CPU-s 197.44KiB/CPU-s 2.55KiB/CPU-s 0.0 0.022264 8.92MiB/CPU-s 242.38KiB/CPU-s 3.13KiB/CPU-s 0.0 0.026542 False False
otlp_http_to_blackhole 44.31KiB/CPU-s 2.85 100.00% 1.52MiB/CPU-s 118.31KiB/CPU-s 1.53KiB/CPU-s 0.0 0.076047 1.56MiB/CPU-s 105.46KiB/CPU-s 1.36KiB/CPU-s 0.0 0.065909 False False
otlp_grpc_to_blackhole 27.23KiB/CPU-s 2.74 100.00% 993.34KiB/CPU-s 47.93KiB/CPU-s 633.34B/CPU-s 0.0 0.048246 1020.58KiB/CPU-s 56.92KiB/CPU-s 751.88B/CPU-s 0.0 0.055765 False False
syslog_regex_logs2metric_ddmetrics 93.96KiB/CPU-s 2.46 100.00% 3.74MiB/CPU-s 426.74KiB/CPU-s 5.51KiB/CPU-s 0.0 0.11154 3.83MiB/CPU-s 471.94KiB/CPU-s 6.09KiB/CPU-s 0.0 0.120398 True False
syslog_loki 191.52KiB/CPU-s 2.22 100.00% 8.44MiB/CPU-s 148.9KiB/CPU-s 1.92KiB/CPU-s 0.0 0.017226 8.63MiB/CPU-s 170.52KiB/CPU-s 2.2KiB/CPU-s 0.0 0.019301 False False
datadog_agent_remap_datadog_logs_acks 724.44KiB/CPU-s 2.13 100.00% 33.15MiB/CPU-s 1.02MiB/CPU-s 13.42KiB/CPU-s 0.0 0.030629 33.86MiB/CPU-s 1.11MiB/CPU-s 14.7KiB/CPU-s 0.0 0.032865 False False
splunk_hec_route_s3 192.2KiB/CPU-s 1.62 100.00% 11.6MiB/CPU-s 572.24KiB/CPU-s 7.38KiB/CPU-s 0.0 0.048173 11.79MiB/CPU-s 540.9KiB/CPU-s 6.98KiB/CPU-s 0.0 0.04481 False False
socket_to_socket_blackhole 201.89KiB/CPU-s 1.49 100.00% 13.27MiB/CPU-s 209.99KiB/CPU-s 2.71KiB/CPU-s 0.0 0.015454 13.47MiB/CPU-s 226.74KiB/CPU-s 2.93KiB/CPU-s 0.0 0.016443 False False
datadog_agent_remap_datadog_logs 435.49KiB/CPU-s 1.27 100.00% 33.6MiB/CPU-s 1.19MiB/CPU-s 15.66KiB/CPU-s 0.0 0.035278 34.02MiB/CPU-s 1.39MiB/CPU-s 18.36KiB/CPU-s 0.0 0.040856 False False
http_text_to_http_json 198.16KiB/CPU-s 0.8 100.00% 24.1MiB/CPU-s 574.25KiB/CPU-s 7.41KiB/CPU-s 0.0 0.023266 24.3MiB/CPU-s 731.8KiB/CPU-s 9.47KiB/CPU-s 0.0 0.029413 False False
datadog_agent_remap_blackhole 254.39KiB/CPU-s 0.79 100.00% 31.34MiB/CPU-s 1.01MiB/CPU-s 13.34KiB/CPU-s 0.0 0.032215 31.58MiB/CPU-s 922.23KiB/CPU-s 11.9KiB/CPU-s 0.0 0.028513 False False
syslog_log2metric_humio_metrics 29.83KiB/CPU-s 0.47 100.00% 6.14MiB/CPU-s 106.4KiB/CPU-s 1.37KiB/CPU-s 0.0 0.016909 6.17MiB/CPU-s 207.43KiB/CPU-s 2.68KiB/CPU-s 0.0 0.032808 False False
enterprise_http_to_http 8.29KiB/CPU-s 0.06 92.23% 13.61MiB/CPU-s 308.87KiB/CPU-s 3.99KiB/CPU-s 0.0 0.022154 13.62MiB/CPU-s 192.75KiB/CPU-s 2.49KiB/CPU-s 0.0 0.013817 False False
http_to_http_json 7.81KiB/CPU-s 0.06 94.61% 13.61MiB/CPU-s 233.01KiB/CPU-s 3.01KiB/CPU-s 0.0 0.016712 13.62MiB/CPU-s 210.65KiB/CPU-s 2.72KiB/CPU-s 0.0 0.0151 False False
http_to_http_noack 4.05KiB/CPU-s 0.03 52.53% 13.61MiB/CPU-s 334.56KiB/CPU-s 4.32KiB/CPU-s 0.0 0.024001 13.62MiB/CPU-s 283.67KiB/CPU-s 3.66KiB/CPU-s 0.0 0.020344 False False
splunk_hec_to_splunk_hec_logs_noack 3.22KiB/CPU-s 0.02 60.93% 13.62MiB/CPU-s 219.31KiB/CPU-s 2.83KiB/CPU-s 0.0 0.015721 13.63MiB/CPU-s 191.12KiB/CPU-s 2.47KiB/CPU-s 0.0 0.013697 False False
splunk_hec_indexer_ack_blackhole 1.09KiB/CPU-s 0.01 18.54% 13.62MiB/CPU-s 256.97KiB/CPU-s 3.31KiB/CPU-s 0.0 0.018429 13.62MiB/CPU-s 250.48KiB/CPU-s 3.23KiB/CPU-s 0.0 0.017962 False False
fluent_elasticsearch 59.88B/CPU-s 0.0 8.57% 45.41MiB/CPU-s 29.95KiB/CPU-s 391.62B/CPU-s 0.0 0.000644 45.41MiB/CPU-s 30.24KiB/CPU-s 395.39B/CPU-s 0.0 0.00065 False False
file_to_blackhole -7.23KiB/CPU-s -0.01 26.07% 54.5MiB/CPU-s 1.15MiB/CPU-s 15.22KiB/CPU-s 0.0 0.021148 54.49MiB/CPU-s 1.17MiB/CPU-s 15.48KiB/CPU-s 0.0 0.021523 False False
splunk_hec_to_splunk_hec_logs_acks -2.29KiB/CPU-s -0.02 27.93% 13.62MiB/CPU-s 347.54KiB/CPU-s 4.48KiB/CPU-s 0.0 0.02492 13.62MiB/CPU-s 354.34KiB/CPU-s 4.57KiB/CPU-s 0.0 0.025412 False False
http_to_http_acks -12.91KiB/CPU-s -0.24 19.65% 5.28MiB/CPU-s 2.78MiB/CPU-s 36.68KiB/CPU-s 0.0 0.525963 5.27MiB/CPU-s 2.78MiB/CPU-s 36.72KiB/CPU-s 0.0 0.527766 True False
datadog_agent_remap_blackhole_acks -529.64KiB/CPU-s -1.63 100.00% 31.83MiB/CPU-s 692.93KiB/CPU-s 8.95KiB/CPU-s 0.0 0.02126 31.31MiB/CPU-s 1.51MiB/CPU-s 19.9KiB/CPU-s 0.0 0.048077 False False

Signed-off-by: Luke Steensen <luke.steensen@gmail.com>
@lukesteensen lukesteensen enabled auto-merge (squash) December 16, 2022 17:12
Signed-off-by: Luke Steensen <luke.steensen@gmail.com>
@github-actions
Copy link

Regression Test Results

Run ID: a9e4d07e-0af3-4a54-9233-a4c1f3173ad0
Baseline: 680ba60
Comparison: b3e7cfe
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_humio_logs 361.05KiB/CPU-s 4.03 100.00% 8.76MiB/CPU-s 137.4KiB/CPU-s 1.77KiB/CPU-s 0.0 0.015324 9.11MiB/CPU-s 189.18KiB/CPU-s 2.44KiB/CPU-s 0.0 0.020282 False False
syslog_loki 303.45KiB/CPU-s 3.59 100.00% 8.26MiB/CPU-s 231.56KiB/CPU-s 2.99KiB/CPU-s 0.0 0.027366 8.56MiB/CPU-s 263.23KiB/CPU-s 3.4KiB/CPU-s 0.0 0.030031 False False
syslog_log2metric_splunk_hec_metrics 317.44KiB/CPU-s 3.4 100.00% 9.12MiB/CPU-s 174.24KiB/CPU-s 2.25KiB/CPU-s 0.0 0.018657 9.43MiB/CPU-s 327.05KiB/CPU-s 4.22KiB/CPU-s 0.0 0.033867 False False
syslog_splunk_hec_logs 255.99KiB/CPU-s 2.85 100.00% 8.78MiB/CPU-s 132.91KiB/CPU-s 1.72KiB/CPU-s 0.0 0.01478 9.03MiB/CPU-s 262.46KiB/CPU-s 3.39KiB/CPU-s 0.0 0.028377 False False
datadog_agent_remap_datadog_logs 900.05KiB/CPU-s 2.62 100.00% 33.53MiB/CPU-s 1.39MiB/CPU-s 18.39KiB/CPU-s 0.0 0.041533 34.4MiB/CPU-s 1.11MiB/CPU-s 14.61KiB/CPU-s 0.0 0.032149 False False
syslog_regex_logs2metric_ddmetrics 93.73KiB/CPU-s 2.45 100.00% 3.74MiB/CPU-s 505.14KiB/CPU-s 6.52KiB/CPU-s 0.0 0.131922 3.83MiB/CPU-s 470.62KiB/CPU-s 6.08KiB/CPU-s 0.0 0.11997 True False
otlp_http_to_blackhole 37.24KiB/CPU-s 2.39 100.00% 1.52MiB/CPU-s 114.22KiB/CPU-s 1.47KiB/CPU-s 0.0 0.073187 1.56MiB/CPU-s 111.63KiB/CPU-s 1.44KiB/CPU-s 0.0 0.069859 False False
datadog_agent_remap_datadog_logs_acks 673.34KiB/CPU-s 1.97 100.00% 33.31MiB/CPU-s 1.03MiB/CPU-s 13.63KiB/CPU-s 0.0 0.030962 33.97MiB/CPU-s 904.29KiB/CPU-s 11.67KiB/CPU-s 0.0 0.025996 False False
otlp_grpc_to_blackhole 19.53KiB/CPU-s 1.95 100.00% 999.78KiB/CPU-s 41.7KiB/CPU-s 551.23B/CPU-s 0.0 0.041707 1019.31KiB/CPU-s 45.84KiB/CPU-s 605.88B/CPU-s 0.0 0.044967 False False
splunk_hec_route_s3 174.09KiB/CPU-s 1.46 100.00% 11.67MiB/CPU-s 586.15KiB/CPU-s 7.56KiB/CPU-s 0.0 0.049031 11.84MiB/CPU-s 527.78KiB/CPU-s 6.81KiB/CPU-s 0.0 0.043514 False False
syslog_log2metric_humio_metrics 86.65KiB/CPU-s 1.43 100.00% 5.9MiB/CPU-s 298.21KiB/CPU-s 3.85KiB/CPU-s 0.0 0.049319 5.99MiB/CPU-s 272.91KiB/CPU-s 3.52KiB/CPU-s 0.0 0.044496 False False
http_text_to_http_json 314.28KiB/CPU-s 1.28 100.00% 23.95MiB/CPU-s 586.83KiB/CPU-s 7.58KiB/CPU-s 0.0 0.023928 24.26MiB/CPU-s 646.44KiB/CPU-s 8.34KiB/CPU-s 0.0 0.026025 False False
datadog_agent_remap_blackhole 399.29KiB/CPU-s 1.25 100.00% 31.27MiB/CPU-s 1.31MiB/CPU-s 17.29KiB/CPU-s 0.0 0.041842 31.66MiB/CPU-s 559.32KiB/CPU-s 7.22KiB/CPU-s 0.0 0.01725 False False
datadog_agent_remap_blackhole_acks 183.38KiB/CPU-s 0.57 100.00% 31.45MiB/CPU-s 1.04MiB/CPU-s 13.74KiB/CPU-s 0.0 0.033051 31.63MiB/CPU-s 1.15MiB/CPU-s 15.24KiB/CPU-s 0.0 0.036451 False False
http_to_http_acks 30.59KiB/CPU-s 0.57 44.63% 5.23MiB/CPU-s 2.78MiB/CPU-s 36.72KiB/CPU-s 0.0 0.530838 5.26MiB/CPU-s 2.75MiB/CPU-s 36.34KiB/CPU-s 0.0 0.522495 True False
enterprise_http_to_http 12.44KiB/CPU-s 0.09 98.35% 13.61MiB/CPU-s 353.93KiB/CPU-s 4.57KiB/CPU-s 0.0 0.025394 13.62MiB/CPU-s 190.6KiB/CPU-s 2.46KiB/CPU-s 0.0 0.013663 False False
socket_to_socket_blackhole 7.37KiB/CPU-s 0.05 86.19% 13.28MiB/CPU-s 231.76KiB/CPU-s 2.99KiB/CPU-s 0.0 0.017042 13.29MiB/CPU-s 307.22KiB/CPU-s 3.97KiB/CPU-s 0.0 0.022579 False False
splunk_hec_to_splunk_hec_logs_noack 5.92KiB/CPU-s 0.04 86.35% 13.62MiB/CPU-s 239.64KiB/CPU-s 3.09KiB/CPU-s 0.0 0.017181 13.63MiB/CPU-s 193.29KiB/CPU-s 2.5KiB/CPU-s 0.0 0.013852 False False
splunk_hec_indexer_ack_blackhole 2.62KiB/CPU-s 0.02 43.49% 13.62MiB/CPU-s 258.44KiB/CPU-s 3.33KiB/CPU-s 0.0 0.018534 13.62MiB/CPU-s 239.84KiB/CPU-s 3.09KiB/CPU-s 0.0 0.017196 False False
fluent_elasticsearch 83.52B/CPU-s 0.0 12.02% 45.41MiB/CPU-s 29.91KiB/CPU-s 391.08B/CPU-s 0.0 0.000643 45.41MiB/CPU-s 29.82KiB/CPU-s 389.83B/CPU-s 0.0 0.000641 False False
splunk_hec_to_splunk_hec_logs_acks -1.84KiB/CPU-s -0.01 22.13% 13.62MiB/CPU-s 349.6KiB/CPU-s 4.51KiB/CPU-s 0.0 0.025071 13.61MiB/CPU-s 366.89KiB/CPU-s 4.73KiB/CPU-s 0.0 0.026314 False False
http_to_http_noack -2.03KiB/CPU-s -0.01 29.06% 13.62MiB/CPU-s 285.9KiB/CPU-s 3.69KiB/CPU-s 0.0 0.020504 13.61MiB/CPU-s 309.71KiB/CPU-s 4.0KiB/CPU-s 0.0 0.022214 False False
file_to_blackhole -14.44KiB/CPU-s -0.03 49.44% 54.5MiB/CPU-s 1.16MiB/CPU-s 15.3KiB/CPU-s 0.0 0.02125 54.48MiB/CPU-s 1.17MiB/CPU-s 15.37KiB/CPU-s 0.0 0.021385 False False
http_to_http_json -13.09KiB/CPU-s -0.09 99.84% 13.62MiB/CPU-s 207.53KiB/CPU-s 2.68KiB/CPU-s 0.0 0.014877 13.61MiB/CPU-s 245.59KiB/CPU-s 3.17KiB/CPU-s 0.0 0.017622 False False

@github-actions
Copy link

Regression Test Results

Run ID: 14fc17ec-9db4-48a6-acb8-47f3f81706d9
Baseline: 05a176a
Comparison: 3bebcb9
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_splunk_hec_metrics 340.83KiB/CPU-s 3.64 100.00% 9.15MiB/CPU-s 138.89KiB/CPU-s 1.79KiB/CPU-s 0.0 0.014817 9.49MiB/CPU-s 228.7KiB/CPU-s 2.95KiB/CPU-s 0.0 0.023542 False False
syslog_humio_logs 320.93KiB/CPU-s 3.58 100.00% 8.76MiB/CPU-s 162.47KiB/CPU-s 2.1KiB/CPU-s 0.0 0.018111 9.07MiB/CPU-s 246.7KiB/CPU-s 3.18KiB/CPU-s 0.0 0.026549 False False
syslog_splunk_hec_logs 300.35KiB/CPU-s 3.34 100.00% 8.79MiB/CPU-s 166.41KiB/CPU-s 2.15KiB/CPU-s 0.0 0.018492 9.08MiB/CPU-s 203.03KiB/CPU-s 2.62KiB/CPU-s 0.0 0.021833 False False
syslog_loki 240.99KiB/CPU-s 2.79 100.00% 8.43MiB/CPU-s 192.72KiB/CPU-s 2.49KiB/CPU-s 0.0 0.022334 8.66MiB/CPU-s 182.59KiB/CPU-s 2.36KiB/CPU-s 0.0 0.020586 False False
otlp_grpc_to_blackhole 21.88KiB/CPU-s 2.18 100.00% 1002.71KiB/CPU-s 41.66KiB/CPU-s 550.6B/CPU-s 0.0 0.04154 1.0MiB/CPU-s 52.82KiB/CPU-s 697.83B/CPU-s 0.0 0.051546 False False
otlp_http_to_blackhole 32.37KiB/CPU-s 2.07 100.00% 1.53MiB/CPU-s 111.25KiB/CPU-s 1.44KiB/CPU-s 0.0 0.071163 1.56MiB/CPU-s 112.96KiB/CPU-s 1.46KiB/CPU-s 0.0 0.070794 False False
datadog_agent_remap_datadog_logs 699.65KiB/CPU-s 2.03 100.00% 33.67MiB/CPU-s 1.15MiB/CPU-s 15.13KiB/CPU-s 0.0 0.034018 34.36MiB/CPU-s 1.17MiB/CPU-s 15.5KiB/CPU-s 0.0 0.034145 False False
splunk_hec_route_s3 224.76KiB/CPU-s 1.89 100.00% 11.61MiB/CPU-s 600.85KiB/CPU-s 7.75KiB/CPU-s 0.0 0.050531 11.83MiB/CPU-s 528.4KiB/CPU-s 6.82KiB/CPU-s 0.0 0.043613 False False
syslog_regex_logs2metric_ddmetrics 65.96KiB/CPU-s 1.72 100.00% 3.75MiB/CPU-s 443.63KiB/CPU-s 5.73KiB/CPU-s 0.0 0.115668 3.81MiB/CPU-s 468.57KiB/CPU-s 6.05KiB/CPU-s 0.0 0.120105 True False
datadog_agent_remap_datadog_logs_acks 509.67KiB/CPU-s 1.48 100.00% 33.6MiB/CPU-s 1.01MiB/CPU-s 13.4KiB/CPU-s 0.0 0.030193 34.09MiB/CPU-s 869.96KiB/CPU-s 11.23KiB/CPU-s 0.0 0.024916 False False
socket_to_socket_blackhole 176.58KiB/CPU-s 1.29 100.00% 13.34MiB/CPU-s 181.27KiB/CPU-s 2.34KiB/CPU-s 0.0 0.013269 13.51MiB/CPU-s 152.6KiB/CPU-s 1.97KiB/CPU-s 0.0 0.011028 False False
syslog_log2metric_humio_metrics 75.06KiB/CPU-s 1.2 100.00% 6.1MiB/CPU-s 180.36KiB/CPU-s 2.33KiB/CPU-s 0.0 0.028877 6.17MiB/CPU-s 214.9KiB/CPU-s 2.77KiB/CPU-s 0.0 0.033998 False False
http_text_to_http_json 280.7KiB/CPU-s 1.13 100.00% 24.16MiB/CPU-s 533.85KiB/CPU-s 6.89KiB/CPU-s 0.0 0.021573 24.44MiB/CPU-s 589.83KiB/CPU-s 7.61KiB/CPU-s 0.0 0.023568 False False
http_to_http_acks 28.97KiB/CPU-s 0.54 42.49% 5.25MiB/CPU-s 2.77MiB/CPU-s 36.63KiB/CPU-s 0.0 0.528185 5.28MiB/CPU-s 2.76MiB/CPU-s 36.45KiB/CPU-s 0.0 0.522706 True False
datadog_agent_remap_blackhole 170.79KiB/CPU-s 0.52 100.00% 31.81MiB/CPU-s 748.21KiB/CPU-s 9.66KiB/CPU-s 0.0 0.022966 31.98MiB/CPU-s 709.94KiB/CPU-s 9.16KiB/CPU-s 0.0 0.021678 False False
enterprise_http_to_http 10.41KiB/CPU-s 0.07 96.42% 13.61MiB/CPU-s 332.89KiB/CPU-s 4.3KiB/CPU-s 0.0 0.023881 13.62MiB/CPU-s 192.14KiB/CPU-s 2.48KiB/CPU-s 0.0 0.013774 False False
splunk_hec_to_splunk_hec_logs_noack 3.62KiB/CPU-s 0.03 66.12% 13.62MiB/CPU-s 221.79KiB/CPU-s 2.86KiB/CPU-s 0.0 0.015899 13.63MiB/CPU-s 191.84KiB/CPU-s 2.48KiB/CPU-s 0.0 0.013748 False False
splunk_hec_to_splunk_hec_logs_acks 1.58KiB/CPU-s 0.01 19.25% 13.61MiB/CPU-s 362.14KiB/CPU-s 4.67KiB/CPU-s 0.0 0.025975 13.62MiB/CPU-s 348.68KiB/CPU-s 4.5KiB/CPU-s 0.0 0.025007 False False
http_to_http_noack 1.9KiB/CPU-s 0.01 27.46% 13.61MiB/CPU-s 307.23KiB/CPU-s 3.96KiB/CPU-s 0.0 0.022037 13.62MiB/CPU-s 285.42KiB/CPU-s 3.68KiB/CPU-s 0.0 0.02047 False False
splunk_hec_indexer_ack_blackhole 152.13B/CPU-s 0.0 2.53% 13.62MiB/CPU-s 255.89KiB/CPU-s 3.3KiB/CPU-s 0.0 0.018351 13.62MiB/CPU-s 257.78KiB/CPU-s 3.33KiB/CPU-s 0.0 0.018487 False False
fluent_elasticsearch 430.04B/CPU-s 0.0 56.17% 45.41MiB/CPU-s 29.99KiB/CPU-s 392.24B/CPU-s 0.0 0.000645 45.41MiB/CPU-s 30.01KiB/CPU-s 392.32B/CPU-s 0.0 0.000645 False False
file_to_blackhole -13.21KiB/CPU-s -0.02 46.44% 54.5MiB/CPU-s 1.12MiB/CPU-s 14.74KiB/CPU-s 0.0 0.020471 54.49MiB/CPU-s 1.17MiB/CPU-s 15.41KiB/CPU-s 0.0 0.021434 False False
http_to_http_json -33.3KiB/CPU-s -0.24 100.00% 13.62MiB/CPU-s 207.82KiB/CPU-s 2.68KiB/CPU-s 0.0 0.014898 13.59MiB/CPU-s 261.57KiB/CPU-s 3.37KiB/CPU-s 0.0 0.018795 False False
datadog_agent_remap_blackhole_acks -323.36KiB/CPU-s -0.99 100.00% 31.82MiB/CPU-s 900.33KiB/CPU-s 11.63KiB/CPU-s 0.0 0.027631 31.5MiB/CPU-s 990.6KiB/CPU-s 12.79KiB/CPU-s 0.0 0.030706 False False

Signed-off-by: Luke Steensen <luke.steensen@gmail.com>
…for macos)

Signed-off-by: Luke Steensen <luke.steensen@gmail.com>
@github-actions
Copy link

Regression Test Results

Run ID: 19cbe4d4-21f7-4221-9fba-3570153e20e3
Baseline: 05a176a
Comparison: 43773ee
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
syslog_humio_logs 559.07KiB/CPU-s 6.26 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_humio_logs 559.07KiB/CPU-s 6.26 100.00% 8.72MiB/CPU-s 169.19KiB/CPU-s 2.18KiB/CPU-s 0.0 0.018947 9.27MiB/CPU-s 188.51KiB/CPU-s 2.43KiB/CPU-s 0.0 0.019867 False False
syslog_loki 326.0KiB/CPU-s 3.98 100.00% 7.99MiB/CPU-s 277.13KiB/CPU-s 3.58KiB/CPU-s 0.0 0.033867 8.31MiB/CPU-s 275.11KiB/CPU-s 3.55KiB/CPU-s 0.0 0.032332 False False
datadog_agent_remap_datadog_logs 1.32MiB/CPU-s 3.95 100.00% 33.31MiB/CPU-s 1.36MiB/CPU-s 17.93KiB/CPU-s 0.0 0.040751 34.63MiB/CPU-s 1.16MiB/CPU-s 15.26KiB/CPU-s 0.0 0.03336 False False
syslog_splunk_hec_logs 324.52KiB/CPU-s 3.6 100.00% 8.8MiB/CPU-s 121.01KiB/CPU-s 1.56KiB/CPU-s 0.0 0.013428 9.12MiB/CPU-s 204.56KiB/CPU-s 2.64KiB/CPU-s 0.0 0.021911 False False
otlp_grpc_to_blackhole 29.64KiB/CPU-s 2.99 100.00% 992.01KiB/CPU-s 37.79KiB/CPU-s 499.59B/CPU-s 0.0 0.038092 1021.65KiB/CPU-s 52.83KiB/CPU-s 697.99B/CPU-s 0.0 0.051706 False False
syslog_log2metric_humio_metrics 181.94KiB/CPU-s 2.91 100.00% 6.11MiB/CPU-s 171.34KiB/CPU-s 2.21KiB/CPU-s 0.0 0.027365 6.29MiB/CPU-s 141.54KiB/CPU-s 1.83KiB/CPU-s 0.0 0.021967 False False
syslog_log2metric_splunk_hec_metrics 268.6KiB/CPU-s 2.85 100.00% 9.19MiB/CPU-s 114.95KiB/CPU-s 1.48KiB/CPU-s 0.0 0.012207 9.46MiB/CPU-s 243.3KiB/CPU-s 3.14KiB/CPU-s 0.0 0.025122 False False
datadog_agent_remap_blackhole_acks 831.08KiB/CPU-s 2.69 100.00% 30.17MiB/CPU-s 749.15KiB/CPU-s 9.67KiB/CPU-s 0.0 0.02425 30.98MiB/CPU-s 1.39MiB/CPU-s 18.38KiB/CPU-s 0.0 0.044892 False False
datadog_agent_remap_blackhole 717.9KiB/CPU-s 2.25 100.00% 31.13MiB/CPU-s 1.15MiB/CPU-s 15.2KiB/CPU-s 0.0 0.036907 31.83MiB/CPU-s 846.34KiB/CPU-s 10.92KiB/CPU-s 0.0 0.025964 False False
datadog_agent_remap_datadog_logs_acks 710.98KiB/CPU-s 2.06 100.00% 33.64MiB/CPU-s 938.66KiB/CPU-s 12.11KiB/CPU-s 0.0 0.02725 34.33MiB/CPU-s 757.41KiB/CPU-s 9.78KiB/CPU-s 0.0 0.021543 False False
otlp_http_to_blackhole 28.69KiB/CPU-s 1.85 100.00% 1.52MiB/CPU-s 113.65KiB/CPU-s 1.47KiB/CPU-s 0.0 0.073098 1.55MiB/CPU-s 115.52KiB/CPU-s 1.49KiB/CPU-s 0.0 0.072954 False False
http_text_to_http_json 424.02KiB/CPU-s 1.72 100.00% 24.08MiB/CPU-s 569.2KiB/CPU-s 7.35KiB/CPU-s 0.0 0.02308 24.5MiB/CPU-s 568.58KiB/CPU-s 7.34KiB/CPU-s 0.0 0.022666 False False
socket_to_socket_blackhole 226.7KiB/CPU-s 1.68 100.00% 13.19MiB/CPU-s 233.65KiB/CPU-s 3.02KiB/CPU-s 0.0 0.017298 13.41MiB/CPU-s 201.8KiB/CPU-s 2.61KiB/CPU-s 0.0 0.014693 False False
splunk_hec_route_s3 172.7KiB/CPU-s 1.45 100.00% 11.63MiB/CPU-s 643.85KiB/CPU-s 8.31KiB/CPU-s 0.0 0.054075 11.8MiB/CPU-s 534.71KiB/CPU-s 6.9KiB/CPU-s 0.0 0.044267 False False
http_to_http_acks 75.4KiB/CPU-s 1.41 85.43% 5.21MiB/CPU-s 2.81MiB/CPU-s 37.13KiB/CPU-s 0.0 0.539473 5.28MiB/CPU-s 2.74MiB/CPU-s 36.16KiB/CPU-s 0.0 0.518238 True False
http_to_http_json 18.71KiB/CPU-s 0.13 100.00% 13.6MiB/CPU-s 273.96KiB/CPU-s 3.53KiB/CPU-s 0.0 0.019665 13.62MiB/CPU-s 211.06KiB/CPU-s 2.72KiB/CPU-s 0.0 0.015129 False False
enterprise_http_to_http 8.48KiB/CPU-s 0.06 95.82% 13.62MiB/CPU-s 286.04KiB/CPU-s 3.69KiB/CPU-s 0.0 0.020514 13.62MiB/CPU-s 149.06KiB/CPU-s 1.92KiB/CPU-s 0.0 0.010684 False False
splunk_hec_to_splunk_hec_logs_noack 3.57KiB/CPU-s 0.03 65.61% 13.62MiB/CPU-s 221.11KiB/CPU-s 2.85KiB/CPU-s 0.0 0.01585 13.63MiB/CPU-s 191.41KiB/CPU-s 2.47KiB/CPU-s 0.0 0.013718 False False
http_to_http_noack 1.5KiB/CPU-s 0.01 21.05% 13.61MiB/CPU-s 305.71KiB/CPU-s 3.95KiB/CPU-s 0.0 0.02193 13.61MiB/CPU-s 309.63KiB/CPU-s 4.0KiB/CPU-s 0.0 0.022209 False False
splunk_hec_indexer_ack_blackhole -441.07B/CPU-s -0.0 7.43% 13.62MiB/CPU-s 252.1KiB/CPU-s 3.25KiB/CPU-s 0.0 0.018078 13.62MiB/CPU-s 253.91KiB/CPU-s 3.28KiB/CPU-s 0.0 0.018209 False False
fluent_elasticsearch 347.05B/CPU-s 0.0 46.73% 45.41MiB/CPU-s 30.03KiB/CPU-s 392.6B/CPU-s 0.0 0.000646 45.41MiB/CPU-s 30.13KiB/CPU-s 394.02B/CPU-s 0.0 0.000648 False False
splunk_hec_to_splunk_hec_logs_acks -1.1KiB/CPU-s -0.01 13.61% 13.62MiB/CPU-s 350.87KiB/CPU-s 4.53KiB/CPU-s 0.0 0.025163 13.61MiB/CPU-s 350.5KiB/CPU-s 4.52KiB/CPU-s 0.0 0.025139 False False
file_to_blackhole -3.0KiB/CPU-s -0.01 11.55% 54.5MiB/CPU-s 1.08MiB/CPU-s 14.28KiB/CPU-s 0.0 0.019842 54.49MiB/CPU-s 1.13MiB/CPU-s 14.89KiB/CPU-s 0.0 0.020704 False False
syslog_regex_logs2metric_ddmetrics -51.55KiB/CPU-s -1.41 100.00% 3.58MiB/CPU-s 459.75KiB/CPU-s 5.93KiB/CPU-s 0.0 0.125468 3.53MiB/CPU-s 510.98KiB/CPU-s 6.6KiB/CPU-s 0.0 0.141441 True False

Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
@github-actions
Copy link

Regression Test Results

Run ID: b97b305b-01b4-4832-8749-0ce4ab7d665a
Baseline: c122394
Comparison: 7708e02
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
syslog_log2metric_splunk_hec_metrics 473.77KiB/CPU-s 5.04 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_splunk_hec_metrics 473.77KiB/CPU-s 5.04 100.00% 9.18MiB/CPU-s 174.28KiB/CPU-s 2.25KiB/CPU-s 0.0 0.018539 9.64MiB/CPU-s 170.9KiB/CPU-s 2.21KiB/CPU-s 0.0 0.017307 False False
syslog_humio_logs 434.57KiB/CPU-s 4.79 100.00% 8.87MiB/CPU-s 145.27KiB/CPU-s 1.87KiB/CPU-s 0.0 0.015999 9.29MiB/CPU-s 132.33KiB/CPU-s 1.71KiB/CPU-s 0.0 0.013908 False False
syslog_loki 243.83KiB/CPU-s 2.82 100.00% 8.45MiB/CPU-s 163.96KiB/CPU-s 2.12KiB/CPU-s 0.0 0.018941 8.69MiB/CPU-s 114.91KiB/CPU-s 1.48KiB/CPU-s 0.0 0.012911 False False
otlp_grpc_to_blackhole 25.88KiB/CPU-s 2.49 100.00% 1.01MiB/CPU-s 45.59KiB/CPU-s 602.52B/CPU-s 0.0 0.043865 1.04MiB/CPU-s 54.32KiB/CPU-s 717.53B/CPU-s 0.0 0.050994 False False
syslog_splunk_hec_logs 220.99KiB/CPU-s 2.43 100.00% 8.89MiB/CPU-s 81.97KiB/CPU-s 1.06KiB/CPU-s 0.0 0.009008 9.1MiB/CPU-s 273.08KiB/CPU-s 3.52KiB/CPU-s 0.0 0.029299 False False
datadog_agent_remap_datadog_logs_acks 797.7KiB/CPU-s 2.32 100.00% 33.6MiB/CPU-s 1.23MiB/CPU-s 16.29KiB/CPU-s 0.0 0.03671 34.37MiB/CPU-s 880.99KiB/CPU-s 11.37KiB/CPU-s 0.0 0.025026 False False
datadog_agent_remap_datadog_logs 610.4KiB/CPU-s 1.75 100.00% 34.07MiB/CPU-s 1.1MiB/CPU-s 14.48KiB/CPU-s 0.0 0.032175 34.67MiB/CPU-s 1.08MiB/CPU-s 14.27KiB/CPU-s 0.0 0.03116 False False
datadog_agent_remap_blackhole_acks 437.3KiB/CPU-s 1.35 100.00% 31.59MiB/CPU-s 542.36KiB/CPU-s 7.0KiB/CPU-s 0.0 0.016766 32.02MiB/CPU-s 977.92KiB/CPU-s 12.62KiB/CPU-s 0.0 0.029827 False False
syslog_regex_logs2metric_ddmetrics 48.27KiB/CPU-s 1.25 100.00% 3.78MiB/CPU-s 439.38KiB/CPU-s 5.67KiB/CPU-s 0.0 0.113598 3.82MiB/CPU-s 469.75KiB/CPU-s 6.06KiB/CPU-s 0.0 0.119953 True False
datadog_agent_remap_blackhole 65.02KiB/CPU-s 0.2 100.00% 31.83MiB/CPU-s 811.07KiB/CPU-s 10.47KiB/CPU-s 0.0 0.024884 31.89MiB/CPU-s 892.75KiB/CPU-s 11.52KiB/CPU-s 0.0 0.027335 False False
syslog_log2metric_humio_metrics 6.51KiB/CPU-s 0.11 96.58% 6.02MiB/CPU-s 183.7KiB/CPU-s 2.37KiB/CPU-s 0.0 0.029774 6.03MiB/CPU-s 151.67KiB/CPU-s 1.96KiB/CPU-s 0.0 0.024557 False False
splunk_hec_route_s3 9.3KiB/CPU-s 0.08 64.66% 11.8MiB/CPU-s 568.83KiB/CPU-s 7.34KiB/CPU-s 0.0 0.047058 11.81MiB/CPU-s 528.66KiB/CPU-s 6.82KiB/CPU-s 0.0 0.043701 False False
enterprise_http_to_http 10.08KiB/CPU-s 0.07 97.96% 13.61MiB/CPU-s 302.36KiB/CPU-s 3.9KiB/CPU-s 0.0 0.021687 13.62MiB/CPU-s 148.58KiB/CPU-s 1.92KiB/CPU-s 0.0 0.010649 False False
splunk_hec_to_splunk_hec_logs_noack 2.13KiB/CPU-s 0.02 43.70% 13.62MiB/CPU-s 212.04KiB/CPU-s 2.74KiB/CPU-s 0.0 0.015199 13.63MiB/CPU-s 190.15KiB/CPU-s 2.45KiB/CPU-s 0.0 0.013627 False False
splunk_hec_indexer_ack_blackhole 1.09KiB/CPU-s 0.01 18.56% 13.62MiB/CPU-s 255.95KiB/CPU-s 3.3KiB/CPU-s 0.0 0.018356 13.62MiB/CPU-s 253.36KiB/CPU-s 3.27KiB/CPU-s 0.0 0.018168 False False
http_to_http_noack -336.02B/CPU-s -0.0 5.65% 13.62MiB/CPU-s 248.75KiB/CPU-s 3.21KiB/CPU-s 0.0 0.017836 13.62MiB/CPU-s 258.49KiB/CPU-s 3.34KiB/CPU-s 0.0 0.018535 False False
http_to_http_json -321.86B/CPU-s -0.0 6.54% 13.62MiB/CPU-s 206.49KiB/CPU-s 2.66KiB/CPU-s 0.0 0.014802 13.62MiB/CPU-s 213.14KiB/CPU-s 2.75KiB/CPU-s 0.0 0.015279 False False
splunk_hec_to_splunk_hec_logs_acks 112.9B/CPU-s 0.0 1.41% 13.62MiB/CPU-s 342.85KiB/CPU-s 4.42KiB/CPU-s 0.0 0.024585 13.62MiB/CPU-s 340.58KiB/CPU-s 4.39KiB/CPU-s 0.0 0.024422 False False
fluent_elasticsearch -553.78B/CPU-s -0.0 68.22% 45.41MiB/CPU-s 29.98KiB/CPU-s 392.05B/CPU-s 0.0 0.000645 45.41MiB/CPU-s 29.98KiB/CPU-s 391.95B/CPU-s 0.0 0.000645 False False
file_to_blackhole -1.05KiB/CPU-s -0.0 4.03% 54.5MiB/CPU-s 1.07MiB/CPU-s 14.14KiB/CPU-s 0.0 0.019648 54.5MiB/CPU-s 1.15MiB/CPU-s 15.18KiB/CPU-s 0.0 0.021096 False False
http_to_http_acks -4.07KiB/CPU-s -0.08 6.24% 5.28MiB/CPU-s 2.78MiB/CPU-s 36.79KiB/CPU-s 0.0 0.527165 5.28MiB/CPU-s 2.78MiB/CPU-s 36.73KiB/CPU-s 0.0 0.526719 True False
otlp_http_to_blackhole -3.27KiB/CPU-s -0.21 87.63% 1.54MiB/CPU-s 114.06KiB/CPU-s 1.47KiB/CPU-s 0.0 0.072455 1.53MiB/CPU-s 119.0KiB/CPU-s 1.54KiB/CPU-s 0.0 0.075753 False False
socket_to_socket_blackhole -340.57KiB/CPU-s -2.46 100.00% 13.53MiB/CPU-s 273.41KiB/CPU-s 3.53KiB/CPU-s 0.0 0.019739 13.19MiB/CPU-s 205.17KiB/CPU-s 2.65KiB/CPU-s 0.0 0.015186 False False
http_text_to_http_json -1.04MiB/CPU-s -4.11 100.00% 25.36MiB/CPU-s 626.66KiB/CPU-s 8.09KiB/CPU-s 0.0 0.024129 24.32MiB/CPU-s 929.03KiB/CPU-s 11.99KiB/CPU-s 0.0 0.037303 False False

@jszwedko
Copy link
Member

Seemingly blocked by dtolnay/inventory#58

jszwedko and others added 2 commits January 5, 2023 08:09
* chore(ci): Use codegen-units = 1 for Windows

To workaround issue in inventory crate when upgrading to Rust 1.65.0.

Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>

* Fix flags

Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>

Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
@github-actions
Copy link

github-actions bot commented Jan 5, 2023

Regression Test Results

Run ID: f8cf9e47-35aa-4e70-8672-d58e928a6862
Baseline: 95f2f3a
Comparison: 06f7b22
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
syslog_humio_logs 558.94KiB/CPU-s 6.35 100.00%
syslog_regex_logs2metric_ddmetrics 202.23KiB/CPU-s 5.59 100.00%
socket_to_socket_blackhole 707.9KiB/CPU-s 5.43 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_humio_logs 558.94KiB/CPU-s 6.35 100.00% 8.6MiB/CPU-s 289.02KiB/CPU-s 3.73KiB/CPU-s 0.0 0.032815 9.15MiB/CPU-s 362.41KiB/CPU-s 4.68KiB/CPU-s 0.0 0.038692 False False
syslog_regex_logs2metric_ddmetrics 202.23KiB/CPU-s 5.59 100.00% 3.54MiB/CPU-s 483.29KiB/CPU-s 6.24KiB/CPU-s 0.0 0.133484 3.73MiB/CPU-s 499.38KiB/CPU-s 6.45KiB/CPU-s 0.0 0.13063 True False
socket_to_socket_blackhole 707.9KiB/CPU-s 5.43 100.00% 12.73MiB/CPU-s 619.09KiB/CPU-s 7.99KiB/CPU-s 0.0 0.047499 13.42MiB/CPU-s 307.68KiB/CPU-s 3.97KiB/CPU-s 0.0 0.022391 False False
datadog_agent_remap_datadog_logs_acks 1.3MiB/CPU-s 4.03 100.00% 32.15MiB/CPU-s 1.45MiB/CPU-s 19.2KiB/CPU-s 0.0 0.04518 33.45MiB/CPU-s 1.47MiB/CPU-s 19.49KiB/CPU-s 0.0 0.044085 False False
datadog_agent_remap_blackhole 1013.37KiB/CPU-s 3.21 100.00% 30.78MiB/CPU-s 1.35MiB/CPU-s 17.78KiB/CPU-s 0.0 0.043708 31.77MiB/CPU-s 737.37KiB/CPU-s 9.52KiB/CPU-s 0.0 0.022661 False False
syslog_splunk_hec_logs 277.92KiB/CPU-s 3.11 100.00% 8.72MiB/CPU-s 192.08KiB/CPU-s 2.48KiB/CPU-s 0.0 0.021506 8.99MiB/CPU-s 390.89KiB/CPU-s 5.04KiB/CPU-s 0.0 0.042443 False False
syslog_loki 245.95KiB/CPU-s 2.85 100.00% 8.42MiB/CPU-s 190.97KiB/CPU-s 2.46KiB/CPU-s 0.0 0.022155 8.66MiB/CPU-s 249.5KiB/CPU-s 3.22KiB/CPU-s 0.0 0.028142 False False
datadog_agent_remap_blackhole_acks 837.54KiB/CPU-s 2.66 100.00% 30.72MiB/CPU-s 1.47MiB/CPU-s 19.45KiB/CPU-s 0.0 0.047886 31.54MiB/CPU-s 994.76KiB/CPU-s 12.84KiB/CPU-s 0.0 0.030801 False False
otlp_grpc_to_blackhole 23.49KiB/CPU-s 2.24 100.00% 1.02MiB/CPU-s 37.47KiB/CPU-s 495.33B/CPU-s 0.0 0.035712 1.05MiB/CPU-s 48.22KiB/CPU-s 637.16B/CPU-s 0.0 0.04495 False False
syslog_log2metric_splunk_hec_metrics 176.94KiB/CPU-s 1.94 100.00% 8.91MiB/CPU-s 443.1KiB/CPU-s 5.72KiB/CPU-s 0.0 0.048547 9.09MiB/CPU-s 475.49KiB/CPU-s 6.14KiB/CPU-s 0.0 0.051104 False False
otlp_http_to_blackhole 21.57KiB/CPU-s 1.39 100.00% 1.52MiB/CPU-s 122.39KiB/CPU-s 1.58KiB/CPU-s 0.0 0.078616 1.54MiB/CPU-s 120.04KiB/CPU-s 1.55KiB/CPU-s 0.0 0.076052 False False
datadog_agent_remap_datadog_logs 235.52KiB/CPU-s 0.68 100.00% 33.69MiB/CPU-s 1.31MiB/CPU-s 17.3KiB/CPU-s 0.0 0.038878 33.92MiB/CPU-s 1.61MiB/CPU-s 21.28KiB/CPU-s 0.0 0.047503 False False
enterprise_http_to_http 10.47KiB/CPU-s 0.08 94.89% 13.61MiB/CPU-s 351.42KiB/CPU-s 4.53KiB/CPU-s 0.0 0.025214 13.62MiB/CPU-s 222.35KiB/CPU-s 2.87KiB/CPU-s 0.0 0.015941 False False
splunk_hec_to_splunk_hec_logs_acks 4.27KiB/CPU-s 0.03 49.45% 13.61MiB/CPU-s 371.64KiB/CPU-s 4.79KiB/CPU-s 0.0 0.026657 13.62MiB/CPU-s 329.9KiB/CPU-s 4.26KiB/CPU-s 0.0 0.023656 False False
splunk_hec_to_splunk_hec_logs_noack 1.06KiB/CPU-s 0.01 23.36% 13.62MiB/CPU-s 199.52KiB/CPU-s 2.58KiB/CPU-s 0.0 0.0143 13.63MiB/CPU-s 192.29KiB/CPU-s 2.48KiB/CPU-s 0.0 0.013781 False False
fluent_elasticsearch -206.96B/CPU-s -0.0 21.48% 45.41MiB/CPU-s 30.59KiB/CPU-s 399.72B/CPU-s 0.0 0.000658 45.41MiB/CPU-s 49.46KiB/CPU-s 645.62B/CPU-s 0.0 0.001063 False False
file_to_blackhole -1.89KiB/CPU-s -0.0 6.78% 54.49MiB/CPU-s 1.13MiB/CPU-s 14.97KiB/CPU-s 0.0 0.020791 54.49MiB/CPU-s 1.24MiB/CPU-s 16.38KiB/CPU-s 0.0 0.02277 False False
splunk_hec_indexer_ack_blackhole -829.15B/CPU-s -0.01 14.12% 13.62MiB/CPU-s 246.35KiB/CPU-s 3.18KiB/CPU-s 0.0 0.017664 13.62MiB/CPU-s 252.66KiB/CPU-s 3.26KiB/CPU-s 0.0 0.018118 False False
http_to_http_noack -4.18KiB/CPU-s -0.03 48.48% 13.61MiB/CPU-s 330.23KiB/CPU-s 4.26KiB/CPU-s 0.0 0.02369 13.61MiB/CPU-s 372.8KiB/CPU-s 4.81KiB/CPU-s 0.0 0.026752 False False
http_to_http_json -9.89KiB/CPU-s -0.07 98.19% 13.62MiB/CPU-s 206.58KiB/CPU-s 2.67KiB/CPU-s 0.0 0.014808 13.61MiB/CPU-s 249.98KiB/CPU-s 3.22KiB/CPU-s 0.0 0.017933 False False
http_text_to_http_json -91.98KiB/CPU-s -0.36 100.00% 24.9MiB/CPU-s 711.25KiB/CPU-s 9.18KiB/CPU-s 0.0 0.027897 24.81MiB/CPU-s 801.71KiB/CPU-s 10.34KiB/CPU-s 0.0 0.031558 False False
syslog_log2metric_humio_metrics -53.13KiB/CPU-s -0.88 100.00% 5.88MiB/CPU-s 346.35KiB/CPU-s 4.47KiB/CPU-s 0.0 0.057486 5.83MiB/CPU-s 257.5KiB/CPU-s 3.32KiB/CPU-s 0.0 0.04312 False False
http_to_http_acks -71.18KiB/CPU-s -1.31 83.01% 5.29MiB/CPU-s 2.75MiB/CPU-s 36.38KiB/CPU-s 0.0 0.520013 5.22MiB/CPU-s 2.8MiB/CPU-s 36.96KiB/CPU-s 0.0 0.535316 True False
splunk_hec_route_s3 -422.15KiB/CPU-s -3.54 100.00% 11.64MiB/CPU-s 654.45KiB/CPU-s 8.44KiB/CPU-s 0.0 0.05488 11.23MiB/CPU-s 725.89KiB/CPU-s 9.37KiB/CPU-s 0.0 0.063105 False False

@lukesteensen lukesteensen merged commit b56193e into master Jan 5, 2023
@lukesteensen lukesteensen deleted the upgrade-rust-1.65 branch January 5, 2023 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: core Anything related to core crates i.e. vector-core, core-common, etc domain: sinks Anything related to the Vector's sinks domain: sources Anything related to the Vector's sources domain: transforms Anything related to Vector's transform components domain: vrl Anything related to the Vector Remap Language
Projects
None yet
Development

Successfully merging this pull request may close these issues.

opaque_hidden_inferred_bound warning when compiling with rust 1.66
4 participants