Skip to content

Conversation

@fuchsnj
Copy link
Member

@fuchsnj fuchsnj commented May 11, 2022

This allows LogEvent to be any value, not just an object. This has no change for traces / metrics.

This PR is mostly internal cleanup / changes to allow the "log namespace" RFC to utilize this. There should be no user-facing changes here (except for the VRL playground). Changes to sources/transforms/sinks will happen later, only if the new log namespace is selected, since that could be a breaking change otherwise.

Summary of changes:

  • Any function on LogEvent that would panic if the event root was not an object was updated to return an Option.
  • LogEvent::from_parts() was renamed to LogEvent::from_map()
  • LogEvent::from_parts() now takes a Value instead of a BTreeMap
  • All "old" Value insert / get / etc. methods were mostly unused, and removed. These assumed the root was an object.
    All of the *_by_path_v2 functions were implemented and the suffix was removed to replace the old methods.
  • Any sink that requires key/value pairs but receives a non-object event will nest the value under a hard-coded "message" key as an object. This is for backwards compatibility. In the future any case this can happen should be a compile-time error and the user must restructure the event appropriately. This feature exists but is currently unused until more of the "log namespace" RFC is implemented.
  • All _flat functions on LogEvent were removed. The new lookup code is now fast enough and flexible enough that the normal non *_flat functions can do the same thing.
  • the protobuf code was updated to allow non-object root values.
  • The VRL sandbox checked the root type on each access, and errored if it wasn't an object. This was removed.

@netlify
Copy link

netlify bot commented May 11, 2022

Deploy Preview for vector-project canceled.

Name Link
🔨 Latest commit 8cd74bb
🔍 Latest deploy log https://app.netlify.com/sites/vector-project/deploys/6286922d9695d3000867b6fa

@github-actions github-actions bot added domain: core Anything related to core crates i.e. vector-core, core-common, etc domain: sinks Anything related to the Vector's sinks domain: sources Anything related to the Vector's sources domain: transforms Anything related to Vector's transform components labels May 11, 2022
@fuchsnj fuchsnj changed the title Fuchsnj/non object root chore(core): Allow log events to have a non-object root value May 11, 2022
@github-actions

This comment was marked as outdated.

@github-actions

This comment was marked as outdated.

@github-actions
Copy link

Soak Test Results

Baseline: 86eb58f
Comparison: 2cd9ccb
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
socket_to_socket_blackhole 1.12MiB 4.74 100.00% 23.66MiB 797.95KiB 16.47KiB 0 0.0329293 24.78MiB 663.48KiB 13.84KiB 0 0.0261406 False False
http_pipelines_no_grok_blackhole 67.77KiB 0.48 99.43% 13.81MiB 841.01KiB 17.61KiB 0 0.0594482 13.88MiB 812.89KiB 17.02KiB 0 0.0571863 False False
http_datadog_filter_blackhole 44.59KiB 0.29 93.13% 15.28MiB 814.25KiB 17.04KiB 0 0.0520413 15.32MiB 839.92KiB 17.59KiB 0 0.0535295 False True
http_pipelines_blackhole_acks 8.96KiB 0.22 37.05% 3.95MiB 625.83KiB 13.09KiB 0 0.154768 3.96MiB 628.68KiB 13.17KiB 0 0.155128 False True
datadog_agent_remap_blackhole 57.61KiB 0.08 86.40% 66.75MiB 1.28MiB 27.46KiB 0 0.019177 66.81MiB 1.27MiB 27.19KiB 0 0.0189704 False False
datadog_agent_remap_blackhole_acks 46.28KiB 0.07 96.18% 67.83MiB 794.77KiB 16.33KiB 0 0.0114401 67.88MiB 729.99KiB 15.22KiB 0 0.0105006 False False
splunk_hec_to_splunk_hec_logs_acks 1.53KiB 0.01 12.66% 23.84MiB 331.45KiB 6.79KiB 0 0.0135768 23.84MiB 331.25KiB 6.77KiB 0 0.0135678 False False
splunk_hec_indexer_ack_blackhole 915.33B 0 7.47% 23.84MiB 328.69KiB 6.72KiB 0 0.0134624 23.84MiB 330.69KiB 6.76KiB 0 0.0135436 False False
fluent_elasticsearch 2.63B 0 0.13% 79.47MiB 54.36KiB 1.1KiB 0 0.00066784 79.47MiB 53.6KiB 1.09KiB 0 0.000658473 False False
http_to_http_noack -1.86KiB -0.01 19.79% 23.85MiB 253.25KiB 5.23KiB 0 0.0103684 23.85MiB 254.73KiB 5.27KiB 0 0.0104296 False False
splunk_hec_to_splunk_hec_logs_noack -3.25KiB -0.01 25.26% 23.84MiB 339.72KiB 7.03KiB 0 0.013913 23.84MiB 348.1KiB 7.21KiB 0 0.014258 False False
http_to_http_json -8.13KiB -0.03 54.63% 23.84MiB 359.08KiB 7.39KiB 0 0.0147052 23.83MiB 384.0KiB 7.94KiB 0 0.0157312 False False
splunk_hec_route_s3 -97.88KiB -0.46 87.87% 20.89MiB 2.11MiB 44.69KiB 0 0.100991 20.79MiB 2.1MiB 44.65KiB 0 0.101079 False False
http_pipelines_blackhole -35.89KiB -0.87 91.67% 4.03MiB 712.27KiB 14.69KiB 0 0.172535 4.0MiB 699.78KiB 14.61KiB 0 0.170994 False True
splunk_transforms_splunk3 -178.15KiB -1.07 99.23% 16.25MiB 2.23MiB 46.86KiB 0 0.137004 16.08MiB 2.24MiB 47.57KiB 0 0.139354 False False
http_to_http_acks -691.42KiB -3.73 99.86% 18.08MiB 6.81MiB 143.55KiB 0 0.376296 17.41MiB 7.59MiB 160.89KiB 0 0.436076 True True
syslog_log2metric_humio_metrics -612.13KiB -4.29 100.00% 13.92MiB 200.06KiB 4.1KiB 0 0.0140335 13.32MiB 205.17KiB 4.21KiB 0 0.0150377 False False
datadog_agent_remap_datadog_logs -3.5MiB -4.62 100.00% 75.83MiB 3.14MiB 66.36KiB 0 0.0414535 72.33MiB 2.52MiB 53.55KiB 0 0.0348021 False False
syslog_loki -812.63KiB -4.76 100.00% 16.66MiB 428.73KiB 8.79KiB 0 0.0251266 15.87MiB 411.27KiB 8.53KiB 0 0.0253091 False False
datadog_agent_remap_datadog_logs_acks -3.5MiB -4.77 100.00% 73.47MiB 2.19MiB 46.7KiB 0 0.0298602 69.97MiB 2.11MiB 44.24KiB 0 0.0301589 False False
syslog_splunk_hec_logs -1.04MiB -5.72 100.00% 18.27MiB 678.07KiB 14.0KiB 0 0.0362378 17.22MiB 579.36KiB 11.96KiB 0 0.0328394 False False
syslog_humio_logs -1.25MiB -6.58 100.00% 19.03MiB 180.64KiB 3.77KiB 0 0.00926983 17.78MiB 176.91KiB 3.62KiB 0 0.00971715 False False
syslog_log2metric_splunk_hec_metrics -1.3MiB -6.81 100.00% 19.15MiB 491.95KiB 10.27KiB 0 0.0250763 17.85MiB 553.86KiB 11.6KiB 0 0.0302954 False False
syslog_regex_logs2metric_ddmetrics -1.0MiB -7.33 100.00% 13.65MiB 498.85KiB 10.23KiB 0 0.0356782 12.65MiB 443.9KiB 9.25KiB 0 0.0342599 False False

@fuchsnj fuchsnj marked this pull request as ready for review May 16, 2022 14:29
@fuchsnj fuchsnj requested review from JeanMertz and pablosichert May 16, 2022 15:01
Copy link
Contributor

@blt blt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, makes a lot of sense to me. Glad to see this done.

@github-actions
Copy link

Soak Test Results

Baseline: 58dcd7f
Comparison: 23a01df
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

Changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

experiment Δ mean Δ mean % confidence
http_pipelines_no_grok_blackhole -5.29MiB -26.34 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_to_http_acks 673.45KiB 3.69 99.51% 17.82MiB 8.26MiB 174.17KiB 0 0.463583 18.48MiB 7.76MiB 164.38KiB 0 0.41964 True True
splunk_transforms_splunk3 99.07KiB 0.63 87.75% 15.47MiB 2.12MiB 44.63KiB 0 0.137314 15.56MiB 2.17MiB 46.07KiB 0 0.139337 False False
splunk_hec_to_splunk_hec_logs_noack 1.14KiB 0 9.31% 23.84MiB 335.46KiB 6.94KiB 0 0.0137367 23.84MiB 331.7KiB 6.88KiB 0 0.0135821 False False
splunk_hec_to_splunk_hec_logs_acks -1.09KiB -0 9.00% 23.84MiB 333.44KiB 6.82KiB 0 0.0136559 23.84MiB 332.06KiB 6.79KiB 0 0.0136 False False
fluent_elasticsearch -778.48B -0 37.85% 79.47MiB 52.6KiB 1.07KiB 0 0.000646198 79.47MiB 54.42KiB 1.11KiB 0 0.000668621 False False
splunk_hec_indexer_ack_blackhole -19.03B -0 0.15% 23.84MiB 334.25KiB 6.83KiB 0 0.0136906 23.84MiB 329.96KiB 6.75KiB 0 0.0135147 False False
http_to_http_noack -2.01KiB -0.01 21.79% 23.85MiB 249.31KiB 5.15KiB 0 0.0102074 23.84MiB 247.37KiB 5.12KiB 0 0.0101287 False False
http_to_http_json -3.88KiB -0.02 27.61% 23.84MiB 377.82KiB 7.78KiB 0 0.015474 23.84MiB 374.92KiB 7.76KiB 0 0.0153575 False False
datadog_agent_remap_blackhole_acks -542.83KiB -0.79 100.00% 67.08MiB 1.73MiB 36.38KiB 0 0.025761 66.55MiB 1.73MiB 36.9KiB 0 0.0259799 False False
datadog_agent_remap_blackhole -827.47KiB -1.21 100.00% 66.99MiB 2.13MiB 45.76KiB 0 0.0318424 66.18MiB 1.94MiB 41.5KiB 0 0.0292367 False False
syslog_log2metric_humio_metrics -256.71KiB -1.84 100.00% 13.65MiB 164.34KiB 3.37KiB 0 0.0117579 13.4MiB 152.11KiB 3.12KiB 0 0.0110867 False False
splunk_hec_route_s3 -392.02KiB -1.84 100.00% 20.8MiB 2.07MiB 43.83KiB 0 0.0995116 20.41MiB 2.12MiB 45.03KiB 0 0.103882 False False
socket_to_socket_blackhole -740.02KiB -2.91 100.00% 24.86MiB 620.86KiB 12.82KiB 0 0.0243828 24.14MiB 583.9KiB 12.18KiB 0 0.0236177 False False
syslog_splunk_hec_logs -779.54KiB -4.25 100.00% 17.9MiB 576.02KiB 11.89KiB 0 0.0314111 17.14MiB 580.45KiB 11.99KiB 0 0.0330583 False False
syslog_humio_logs -878.83KiB -4.64 100.00% 18.51MiB 265.23KiB 5.52KiB 0 0.0139922 17.65MiB 251.33KiB 5.14KiB 0 0.0139039 False False
syslog_regex_logs2metric_ddmetrics -639.57KiB -4.72 100.00% 13.24MiB 581.42KiB 11.92KiB 0 0.0428779 12.61MiB 507.51KiB 10.58KiB 0 0.0392796 False False
syslog_loki -834.14KiB -4.75 100.00% 17.15MiB 147.99KiB 3.03KiB 0 0.00842363 16.34MiB 173.93KiB 3.61KiB 0 0.0103932 False False
syslog_log2metric_splunk_hec_metrics -957.89KiB -5.01 100.00% 18.66MiB 603.8KiB 12.61KiB 0 0.031592 17.72MiB 582.97KiB 12.21KiB 0 0.0321118 False False
datadog_agent_remap_datadog_logs -3.85MiB -5.07 100.00% 75.92MiB 1.56MiB 33.01KiB 0 0.0205824 72.08MiB 1.46MiB 31.01KiB 0 0.020228 False False
datadog_agent_remap_datadog_logs_acks -4.11MiB -5.28 100.00% 77.84MiB 433.17KiB 9.0KiB 0 0.00543306 73.73MiB 327.06KiB 6.7KiB 0 0.00433098 False False
http_pipelines_blackhole_acks -450.69KiB -9.88 100.00% 4.45MiB 610.58KiB 12.77KiB 0 0.133842 4.01MiB 628.99KiB 13.17KiB 0 0.152995 False True
http_pipelines_blackhole -498.88KiB -11.14 100.00% 4.37MiB 618.67KiB 12.75KiB 0 0.138157 3.88MiB 705.95KiB 14.74KiB 0 0.177417 False True
http_pipelines_no_grok_blackhole -5.29MiB -26.34 100.00% 20.07MiB 658.11KiB 13.78KiB 0 0.0320092 14.79MiB 785.38KiB 16.45KiB 0 0.0518608 False False

@blt
Copy link
Contributor

blt commented May 16, 2022

Most recent soak results might be a consequence of being behind master branch.

@github-actions
Copy link

Soak Test Results

Baseline: f238e5e
Comparison: f6ef3c0
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_pipelines_no_grok_blackhole 960.73KiB 5.07 100.00% 18.51MiB 857.89KiB 17.97KiB 0 0.0452633 19.44MiB 792.51KiB 16.6KiB 0 0.0397961 False False
socket_to_socket_blackhole 371.66KiB 1.47 100.00% 24.61MiB 549.99KiB 11.37KiB 0 0.0218185 24.97MiB 336.04KiB 7.01KiB 0 0.0131371 False False
syslog_loki 213.25KiB 1.35 100.00% 15.41MiB 1.54MiB 32.37KiB 0 0.100065 15.62MiB 790.73KiB 16.42KiB 0 0.0494308 False False
http_pipelines_blackhole_acks 45.73KiB 1 99.57% 4.48MiB 540.07KiB 11.29KiB 0 0.117679 4.53MiB 542.22KiB 11.36KiB 0 0.116983 False True
http_to_http_acks 112.89KiB 0.6 36.92% 18.29MiB 7.98MiB 168.32KiB 0 0.436396 18.4MiB 7.73MiB 163.79KiB 0 0.420047 True True
http_pipelines_blackhole 24.66KiB 0.57 82.01% 4.23MiB 611.75KiB 12.61KiB 0 0.14111 4.26MiB 640.76KiB 13.38KiB 0 0.146964 False True
datadog_agent_remap_blackhole 327.0KiB 0.49 100.00% 65.06MiB 1.47MiB 31.49KiB 0 0.0225663 65.38MiB 1.8MiB 38.7KiB 0 0.0275958 False False
splunk_transforms_splunk3 73.61KiB 0.45 72.00% 16.11MiB 2.29MiB 48.16KiB 0 0.142081 16.18MiB 2.27MiB 48.19KiB 0 0.140185 False False
splunk_hec_to_splunk_hec_logs_acks 470.39B 0 3.84% 23.84MiB 329.16KiB 6.73KiB 0 0.0134819 23.84MiB 331.36KiB 6.77KiB 0 0.0135718 False False
http_to_http_json -545.6B -0 4.10% 23.84MiB 358.97KiB 7.39KiB 0 0.0147025 23.84MiB 351.73KiB 7.27KiB 0 0.0144062 False False
splunk_hec_indexer_ack_blackhole 804.66B 0 6.52% 23.84MiB 331.02KiB 6.77KiB 0 0.0135589 23.84MiB 333.18KiB 6.81KiB 0 0.013647 False False
fluent_elasticsearch -442.51B -0 22.28% 79.47MiB 52.69KiB 1.07KiB 0 0.00064727 79.47MiB 53.46KiB 1.09KiB 0 0.000656772 False False
http_to_http_noack -5.3KiB -0.02 51.14% 23.85MiB 260.02KiB 5.37KiB 0 0.0106457 23.84MiB 263.05KiB 5.46KiB 0 0.010772 False False
splunk_hec_to_splunk_hec_logs_noack -4.72KiB -0.02 36.02% 23.84MiB 336.02KiB 6.96KiB 0 0.0137609 23.84MiB 351.94KiB 7.3KiB 0 0.0144158 False False
datadog_agent_remap_blackhole_acks -48.56KiB -0.07 67.30% 66.0MiB 1.73MiB 36.33KiB 0 0.0261575 65.95MiB 1.58MiB 33.66KiB 0 0.0239271 False False
splunk_hec_route_s3 -297.31KiB -1.4 100.00% 20.67MiB 2.12MiB 45.01KiB 0 0.102767 20.38MiB 2.12MiB 45.04KiB 0 0.104191 False False
syslog_splunk_hec_logs -716.54KiB -3.87 100.00% 18.1MiB 1.24MiB 26.3KiB 0 0.0686879 17.4MiB 1.3MiB 27.5KiB 0 0.0747387 False False
datadog_agent_remap_datadog_logs_acks -3.16MiB -4.15 100.00% 76.1MiB 552.73KiB 11.48KiB 0 0.00709156 72.94MiB 490.22KiB 10.04KiB 0 0.00656187 False False
datadog_agent_remap_datadog_logs -3.21MiB -4.22 100.00% 75.93MiB 651.93KiB 13.43KiB 0 0.00838278 72.72MiB 726.27KiB 15.08KiB 0 0.00975058 False False
syslog_log2metric_humio_metrics -598.12KiB -4.23 100.00% 13.81MiB 233.77KiB 4.79KiB 0 0.0165249 13.23MiB 264.2KiB 5.42KiB 0 0.0195005 False False
syslog_humio_logs -1.08MiB -5.86 100.00% 18.42MiB 504.63KiB 10.51KiB 0 0.0267417 17.35MiB 504.29KiB 10.31KiB 0 0.0283867 False False
syslog_regex_logs2metric_ddmetrics -965.36KiB -6.87 100.00% 13.73MiB 478.98KiB 9.82KiB 0 0.0340681 12.78MiB 451.52KiB 9.42KiB 0 0.0344826 False False
syslog_log2metric_splunk_hec_metrics -1.48MiB -7.75 100.00% 19.09MiB 182.88KiB 3.82KiB 0 0.00935208 17.61MiB 814.69KiB 17.06KiB 0 0.0451615 False False

@fuchsnj fuchsnj enabled auto-merge (squash) May 18, 2022 15:55
@github-actions
Copy link

Soak Test Results

Baseline: d03a526
Comparison: 2d9d221
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_loki 256.82KiB 1.7 100.00% 14.72MiB 222.34KiB 4.56KiB 0 0.0147487 14.97MiB 253.43KiB 5.26KiB 0 0.0165294 False False
datadog_agent_remap_blackhole_acks 997.88KiB 1.5 100.00% 64.93MiB 2.06MiB 43.29KiB 0 0.0316783 65.91MiB 2.0MiB 42.68KiB 0 0.0303448 False False
splunk_hec_to_splunk_hec_logs_acks 776.06B 0 6.28% 23.84MiB 334.38KiB 6.84KiB 0 0.0136961 23.84MiB 331.73KiB 6.78KiB 0 0.0135873 False False
splunk_hec_indexer_ack_blackhole 1.15KiB 0 9.63% 23.84MiB 326.6KiB 6.68KiB 0 0.0133773 23.84MiB 332.84KiB 6.8KiB 0 0.0136323 False False
splunk_hec_to_splunk_hec_logs_noack -4.91KiB -0.02 37.55% 23.84MiB 335.06KiB 6.94KiB 0 0.013721 23.84MiB 349.0KiB 7.24KiB 0 0.0142948 False False
http_to_http_noack -3.88KiB -0.02 40.24% 23.85MiB 251.55KiB 5.2KiB 0 0.0102991 23.84MiB 250.58KiB 5.19KiB 0 0.010261 False False
http_to_http_json -7.39KiB -0.03 54.04% 23.84MiB 347.21KiB 7.14KiB 0 0.014219 23.83MiB 337.79KiB 6.98KiB 0 0.0138374 False False
datadog_agent_remap_blackhole -157.59KiB -0.23 80.41% 66.2MiB 3.67MiB 78.63KiB 0 0.0553763 66.04MiB 4.34MiB 93.05KiB 0 0.0656889 False False
fluent_elasticsearch -237.72KiB -0.29 99.99% 79.5MiB 1.39MiB 28.76KiB 0 0.0174771 79.27MiB 2.55MiB 53.03KiB 0 0.0321567 False False
http_pipelines_blackhole_acks -18.76KiB -0.4 75.82% 4.55MiB 542.75KiB 11.35KiB 0 0.116583 4.53MiB 540.07KiB 11.31KiB 0 0.116476 False False
http_pipelines_blackhole -20.87KiB -0.51 67.99% 4.02MiB 716.7KiB 14.76KiB 0 0.174071 4.0MiB 715.21KiB 14.93KiB 0 0.174592 False False
http_to_http_acks -107.09KiB -0.57 35.59% 18.51MiB 7.82MiB 164.67KiB 0 0.422341 18.4MiB 7.7MiB 163.18KiB 0 0.418407 True True
splunk_hec_route_s3 -247.36KiB -1.15 100.00% 20.99MiB 2.0MiB 42.31KiB 0 0.0951981 20.75MiB 2.05MiB 43.49KiB 0 0.0987621 False False
splunk_transforms_splunk3 -202.92KiB -1.29 99.76% 15.32MiB 2.14MiB 45.06KiB 0 0.139785 15.12MiB 2.32MiB 49.21KiB 0 0.153204 False False
socket_to_socket_blackhole -404.52KiB -1.58 100.00% 25.08MiB 544.09KiB 11.25KiB 0 0.0211802 24.69MiB 481.46KiB 10.04KiB 0 0.0190421 False False
syslog_log2metric_humio_metrics -356.48KiB -2.63 100.00% 13.26MiB 317.23KiB 6.5KiB 0 0.0233604 12.91MiB 275.59KiB 5.65KiB 0 0.0208412 False False
http_pipelines_no_grok_blackhole -1012.05KiB -4.96 100.00% 19.94MiB 746.52KiB 15.63KiB 0 0.0365439 18.96MiB 780.77KiB 16.35KiB 0 0.0402132 False False
syslog_splunk_hec_logs -959.8KiB -5.1 100.00% 18.39MiB 809.04KiB 16.7KiB 0 0.0429429 17.46MiB 790.77KiB 16.34KiB 0 0.0442269 False False
syslog_regex_logs2metric_ddmetrics -770.76KiB -5.44 100.00% 13.83MiB 452.19KiB 9.27KiB 0 0.0319276 13.08MiB 488.2KiB 10.19KiB 0 0.0364544 False False
syslog_humio_logs -1.07MiB -5.55 100.00% 19.26MiB 243.47KiB 5.07KiB 0 0.0123421 18.19MiB 184.66KiB 3.78KiB 0 0.00991133 False False
syslog_log2metric_splunk_hec_metrics -1.02MiB -5.59 100.00% 18.24MiB 243.56KiB 5.09KiB 0 0.013035 17.22MiB 204.22KiB 4.28KiB 0 0.0115774 False False
datadog_agent_remap_datadog_logs -4.47MiB -5.73 100.00% 77.99MiB 576.85KiB 11.88KiB 0 0.00722152 73.52MiB 545.29KiB 11.32KiB 0 0.00724117 False False
datadog_agent_remap_datadog_logs_acks -4.57MiB -5.86 100.00% 78.06MiB 291.65KiB 6.06KiB 0 0.00364803 73.49MiB 324.38KiB 6.64KiB 0 0.00430991 False False

@github-actions
Copy link

Soak Test Results

Baseline: 2003705
Comparison: 7b1ae9c
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_loki 338.43KiB 2.28 100.00% 14.51MiB 367.67KiB 7.54KiB 0 0.0247351 14.84MiB 386.0KiB 8.01KiB 0 0.0253904 False False
splunk_hec_indexer_ack_blackhole 111.03B 0 0.91% 23.84MiB 323.52KiB 6.61KiB 0 0.0132512 23.84MiB 330.78KiB 6.77KiB 0 0.0135484 False False
splunk_hec_to_splunk_hec_logs_acks 169.92B 0 1.37% 23.84MiB 333.97KiB 6.83KiB 0 0.0136796 23.84MiB 335.38KiB 6.86KiB 0 0.0137372 False False
fluent_elasticsearch -85.17B -0 4.27% 79.47MiB 54.07KiB 1.1KiB 0 0.000664263 79.47MiB 54.06KiB 1.1KiB 0 0.000664103 False False
http_to_http_json 886.73B 0 7.06% 23.84MiB 340.94KiB 7.02KiB 0 0.013963 23.84MiB 328.61KiB 6.79KiB 0 0.0134575 False False
splunk_hec_to_splunk_hec_logs_noack -3.1KiB -0.01 25.08% 23.85MiB 329.28KiB 6.81KiB 0 0.0134822 23.84MiB 332.04KiB 6.88KiB 0 0.013597 False False
http_to_http_noack -5.48KiB -0.02 55.15% 23.85MiB 246.64KiB 5.09KiB 0 0.0100982 23.84MiB 247.42KiB 5.13KiB 0 0.0101321 False False
datadog_agent_remap_blackhole -214.2KiB -0.31 100.00% 67.65MiB 1.1MiB 23.51KiB 0 0.0161976 67.44MiB 1.03MiB 22.07KiB 0 0.0152736 False False
splunk_hec_route_s3 -95.89KiB -0.46 88.02% 20.51MiB 2.05MiB 43.32KiB 0 0.0997295 20.42MiB 2.07MiB 43.84KiB 0 0.101174 False False
http_pipelines_blackhole -49.48KiB -1.04 99.69% 4.65MiB 574.45KiB 11.84KiB 0 0.120619 4.6MiB 563.43KiB 11.77KiB 0 0.119547 False False
http_pipelines_blackhole_acks -48.05KiB -1.05 99.68% 4.46MiB 554.72KiB 11.6KiB 0 0.121386 4.41MiB 546.03KiB 11.44KiB 0 0.120755 False False
splunk_transforms_splunk3 -305.68KiB -1.88 100.00% 15.89MiB 2.31MiB 48.63KiB 0 0.145383 15.59MiB 2.32MiB 49.15KiB 0 0.148472 False False
datadog_agent_remap_blackhole_acks -1.49MiB -2.22 100.00% 67.46MiB 2.16MiB 45.33KiB 0 0.0319554 65.97MiB 2.91MiB 62.04KiB 0 0.0440942 False False
socket_to_socket_blackhole -552.11KiB -2.26 100.00% 23.9MiB 144.31KiB 2.98KiB 0 0.00589422 23.37MiB 155.99KiB 3.25KiB 0 0.00651824 False False
http_to_http_acks -429.06KiB -2.28 93.13% 18.36MiB 7.87MiB 166.0KiB 0 0.428697 17.94MiB 7.9MiB 167.28KiB 0 0.440135 True True
syslog_log2metric_humio_metrics -334.54KiB -2.44 100.00% 13.4MiB 475.18KiB 9.74KiB 0 0.0346291 13.07MiB 438.54KiB 8.99KiB 0 0.0327581 False False
datadog_agent_remap_datadog_logs -4.67MiB -6.14 100.00% 76.0MiB 1.75MiB 36.96KiB 0 0.0230395 71.33MiB 1.54MiB 32.63KiB 0 0.021517 False False
datadog_agent_remap_datadog_logs_acks -4.49MiB -6.15 100.00% 72.89MiB 656.29KiB 13.64KiB 0 0.00879092 68.4MiB 657.2KiB 13.46KiB 0 0.00938042 False False
syslog_regex_logs2metric_ddmetrics -937.04KiB -6.67 100.00% 13.71MiB 528.06KiB 10.82KiB 0 0.037603 12.8MiB 482.57KiB 10.06KiB 0 0.0368211 False False
syslog_log2metric_splunk_hec_metrics -1.25MiB -6.67 100.00% 18.79MiB 285.6KiB 5.96KiB 0 0.0148442 17.53MiB 307.53KiB 6.44KiB 0 0.0171262 False False
http_pipelines_no_grok_blackhole -1.38MiB -6.87 100.00% 20.03MiB 887.11KiB 18.58KiB 0 0.0432413 18.65MiB 908.66KiB 19.03KiB 0 0.0475586 False False
syslog_splunk_hec_logs -1.28MiB -7.13 100.00% 17.92MiB 182.97KiB 3.78KiB 0 0.0099706 16.64MiB 191.83KiB 3.96KiB 0 0.0112559 False False
syslog_humio_logs -1.36MiB -7.3 100.00% 18.59MiB 866.49KiB 18.05KiB 0 0.0455038 17.23MiB 820.64KiB 16.78KiB 0 0.0464895 False False

@github-actions
Copy link

Soak Test Results

Baseline: cf08648
Comparison: 8cd74bb
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_to_http_acks 237.13KiB 1.27 70.67% 18.26MiB 7.68MiB 161.91KiB 0 0.420665 18.49MiB 7.42MiB 157.13KiB 0 0.401152 True True
syslog_loki 167.07KiB 1.1 100.00% 14.79MiB 641.9KiB 13.16KiB 0 0.0423807 14.95MiB 612.12KiB 12.7KiB 0 0.0399732 False False
splunk_transforms_splunk3 127.08KiB 0.77 93.80% 16.03MiB 2.34MiB 49.08KiB 0 0.145598 16.16MiB 2.22MiB 47.16KiB 0 0.137474 False False
datadog_agent_remap_blackhole 100.4KiB 0.15 74.72% 64.77MiB 2.61MiB 55.99KiB 0 0.0403001 64.87MiB 3.15MiB 67.62KiB 0 0.0485971 False False
splunk_hec_route_s3 16.22KiB 0.08 19.19% 20.32MiB 2.23MiB 47.17KiB 0 0.109573 20.33MiB 2.23MiB 47.28KiB 0 0.109554 False False
splunk_hec_to_splunk_hec_logs_acks 1.3KiB 0.01 9.64% 23.84MiB 362.56KiB 7.42KiB 0 0.0148468 23.84MiB 379.68KiB 7.76KiB 0 0.0155469 False False
datadog_agent_remap_blackhole_acks 4.8KiB 0.01 8.70% 67.3MiB 1.55MiB 32.51KiB 0 0.0229597 67.3MiB 1.39MiB 29.58KiB 0 0.0205945 False False
splunk_hec_to_splunk_hec_logs_noack -972.24B -0 7.51% 23.84MiB 343.06KiB 7.09KiB 0 0.0140494 23.84MiB 345.05KiB 7.15KiB 0 0.0141314 False False
fluent_elasticsearch 178.39B 0 8.92% 79.47MiB 53.62KiB 1.09KiB 0 0.000658733 79.47MiB 54.69KiB 1.11KiB 0 0.000671867 False False
splunk_hec_indexer_ack_blackhole -281.13B -0 2.27% 23.84MiB 334.93KiB 6.85KiB 0 0.0137185 23.84MiB 333.38KiB 6.82KiB 0 0.0136552 False False
http_to_http_json -1.83KiB -0.01 14.74% 23.84MiB 345.25KiB 7.1KiB 0 0.0141394 23.84MiB 329.09KiB 6.8KiB 0 0.0134788 False False
http_to_http_noack -3.05KiB -0.01 32.66% 23.85MiB 244.37KiB 5.05KiB 0 0.0100049 23.84MiB 249.27KiB 5.17KiB 0 0.0102067 False False
http_pipelines_blackhole_acks -40.2KiB -0.86 98.79% 4.57MiB 536.06KiB 11.22KiB 0 0.114578 4.53MiB 546.05KiB 11.44KiB 0 0.117726 False False
http_pipelines_blackhole -52.37KiB -1.14 99.72% 4.47MiB 591.61KiB 12.19KiB 0 0.129177 4.42MiB 601.76KiB 12.56KiB 0 0.132913 False False
socket_to_socket_blackhole -327.3KiB -1.31 100.00% 24.49MiB 116.56KiB 2.41KiB 0 0.00464716 24.17MiB 231.33KiB 4.83KiB 0 0.00934455 False False
syslog_log2metric_humio_metrics -237.59KiB -1.71 100.00% 13.56MiB 582.88KiB 11.95KiB 0 0.0419669 13.33MiB 453.43KiB 9.29KiB 0 0.0332153 False False
datadog_agent_remap_datadog_logs_acks -3.35MiB -4.44 100.00% 75.42MiB 3.82MiB 81.37KiB 0 0.0506677 72.07MiB 2.56MiB 53.59KiB 0 0.0354602 False False
syslog_log2metric_splunk_hec_metrics -899.35KiB -4.63 100.00% 18.98MiB 1.09MiB 23.33KiB 0 0.0574587 18.11MiB 782.77KiB 16.39KiB 0 0.0422126 False False
syslog_splunk_hec_logs -882.9KiB -4.64 100.00% 18.57MiB 815.22KiB 16.83KiB 0 0.0428677 17.71MiB 447.03KiB 9.24KiB 0 0.0246512 False False
datadog_agent_remap_datadog_logs -4.36MiB -5.61 100.00% 77.85MiB 481.93KiB 9.93KiB 0 0.006044 73.49MiB 594.96KiB 12.35KiB 0 0.00790471 False False
syslog_regex_logs2metric_ddmetrics -943.22KiB -6.59 100.00% 13.97MiB 528.23KiB 10.83KiB 0 0.0369152 13.05MiB 472.83KiB 9.87KiB 0 0.0353758 False False
http_pipelines_no_grok_blackhole -1.38MiB -6.74 100.00% 20.45MiB 681.5KiB 14.27KiB 0 0.032536 19.07MiB 710.12KiB 14.87KiB 0 0.0363534 False False
syslog_humio_logs -1.29MiB -6.9 100.00% 18.72MiB 777.78KiB 16.19KiB 0 0.0405707 17.43MiB 721.88KiB 14.77KiB 0 0.0404478 False False

@fuchsnj fuchsnj merged commit 83bd797 into master May 19, 2022
@fuchsnj fuchsnj deleted the fuchsnj/non_object_root branch May 19, 2022 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: core Anything related to core crates i.e. vector-core, core-common, etc domain: sinks Anything related to the Vector's sinks domain: sources Anything related to the Vector's sources domain: transforms Anything related to Vector's transform components domain: vrl Anything related to the Vector Remap Language

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants