[ASCII-2016] Copy the config model into a new source folder, envvar to enable it #29528
Conversation
dustmop
added
changelog/no-changelog
team/agent-shared-components
qa/done
Go Package Import DifferencesBaseline: bcd3df0
|
This comment has been minimized.
This comment has been minimized.
Copy source files from the viper version of the config to a new directory. This will make it easier to see the diff when implementing the new model.
There is no difference in the new model just yet, but this envvar will let us develop without disrupting the existing config behavior.
dustmop
force-pushed
the
dustin.long/tee-config-model
branch
from
46b5423
to
1a04867
Compare
This comment has been minimized.
This comment has been minimized.
dustmop
force-pushed
the
dustin.long/tee-config-model
branch
from
1a04867
to
da41ae3
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Test changes on VMUse this command from test-infra-definitions to manually test this PR changes on a VM: inv create-vm --pipeline-id=45330272 --os-family=ubuntuNote: This applies to commit e11169e |
Regression DetectorRegression Detector ResultsRun ID: 681fa970-8812-4c25-8a3f-8aa692756cd0 Metrics dashboard Target profiles Baseline: bcd3df0 Performance changes are noted in the perf column of each table:
No significant changes in experiment optimization goalsConfidence level: 90.00% There were no significant changes in experiment optimization goals at this confidence level and effect size tolerance.
|
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | idle | memory utilization | +1.00 | [+0.96, +1.04] | 1 | Logs |
| ➖ | idle_all_features | memory utilization | +0.95 | [+0.89, +1.02] | 1 | Logs |
| ➖ | file_tree | memory utilization | +0.13 | [+0.04, +0.22] | 1 | Logs |
| ➖ | uds_dogstatsd_to_api | ingress throughput | +0.01 | [-0.09, +0.11] | 1 | Logs |
| ➖ | tcp_dd_logs_filter_exclude | ingress throughput | -0.00 | [-0.01, +0.01] | 1 | Logs |
| ➖ | basic_py_check | % cpu utilization | -0.02 | [-2.75, +2.71] | 1 | Logs |
| ➖ | tcp_syslog_to_blackhole | ingress throughput | -0.23 | [-0.28, -0.18] | 1 | Logs |
| ➖ | uds_dogstatsd_to_api_cpu | % cpu utilization | -0.31 | [-1.04, +0.41] | 1 | Logs |
| ➖ | otel_to_otel_logs | ingress throughput | -0.35 | [-1.16, +0.45] | 1 | Logs |
| ➖ | pycheck_lots_of_tags | % cpu utilization | -1.27 | [-3.75, +1.20] | 1 | Logs |
Bounds Checks
| perf | experiment | bounds_check_name | replicates_passed |
|---|---|---|---|
| ✅ | idle | memory_usage | 10/10 |
Explanation
A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".
For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:
-
Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
-
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
-
Its configuration does not mark it "erratic".
@dustmop so close to |
pkg/config/setup/config.go
Outdated
| if val, found := os.LookupEnv("DD_CONF_NODETREEMODEL"); found && val == "enable" { | ||
| datadog = nodetreemodel.NewConfig("datadog", "DD", strings.NewReplacer(".", "_")) | ||
| } else if val, found := os.LookupEnv("DD_CONF_NODETREEMODEL"); found && val == "tee" { | ||
| var viperConfig = pkgconfigmodel.NewConfig("datadog", "DD", strings.NewReplacer(".", "_")) | ||
| var nodetreeConfig = nodetreemodel.NewConfig("datadog", "DD", strings.NewReplacer(".", "_")) | ||
| datadog = teeconfig.NewTeeConfig(viperConfig, nodetreeConfig) | ||
| } else { | ||
| datadog = pkgconfigmodel.NewConfig("datadog", "DD", strings.NewReplacer(".", "_")) | ||
| } | ||
|
|
There was a problem hiding this comment.
Nitpick: We could simplify this block to only look for the DD_CONF_NODETREEMODEL just once 😄
val, found := os.LookupEnv("DD_CONF_NODETREEMODEL")
if found {
if val == "enable" {
datadog = nodetreemodel.NewConfig("datadog", "DD", strings.NewReplacer(".", "_"))
} else if val == "tee" {
var viperConfig = pkgconfigmodel.NewConfig("datadog", "DD", strings.NewReplacer(".", "_"))
var nodetreeConfig = nodetreemodel.NewConfig("datadog", "DD", strings.NewReplacer(".", "_"))
datadog = teeconfig.NewTeeConfig(viperConfig, nodetreeConfig)
}
} else {
datadog = pkgconfigmodel.NewConfig("datadog", "DD", strings.NewReplacer(".", "_"))
}
| @@ -241,7 +243,16 @@ var serverlessConfigComponents = []func(pkgconfigmodel.Setup){ | |||
| func init() { | |||
| osinit() | |||
| // Configure Datadog global configuration | |||
| datadog = pkgconfigmodel.NewConfig("datadog", "DD", strings.NewReplacer(".", "_")) | |||
There was a problem hiding this comment.
Could we add a comment explaining the different values for DD_CONF_NODETREEMODEL and what the different config models do?
| // by a call to the 'Set' method. | ||
| // Callbacks are only called if the value is effectively changed. | ||
| func (t *teeConfig) OnUpdate(callback model.NotificationReceiver) { | ||
| t.baseline.OnUpdate(callback) |
There was a problem hiding this comment.
Should we add the OnUpdate to the compare as well?
This comment has been minimized.
This comment has been minimized.
Debug infoIf you have questions, we are happy to help, come visit us in the #serverless slack channel and provide a link to this comment. These dependencies were added to the serverless extension by this pull request: View dependency graphs for each added dependency in the artifacts section of the github action. We suggest you consider adding the |
Serverless Benchmark Results
tl;drUse these benchmarks as an insight tool during development.
What is this benchmarking?The The benchmark is run using a large variety of lambda request payloads. In the charts below, there is one row for each event payload type. How do I interpret these charts?The charts below comes from The benchstat docs explain how to interpret these charts.
I need more helpFirst off, do not worry if the benchmarks are failing. They are not tests. The intention is for them to be a tool for you to use during development. If you would like a hand interpreting the results come chat with us in Benchmark stats |
|
/merge |
|
🚂 MergeQueue: waiting for PR to be ready This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals. Use |
|
🚂 MergeQueue: pull request added to the queue The median merge time in Use |
What does this PR do?
Copies the config model into a new source folder, and adds an envvar to enable it.
Motivation
This will allow us to put a new config model into place without disrupting the expected behavior of the existing config.
Describe how to test/QA your changes
No functional changes, behavior is disabled without an envvar. Enabling the envvar is not going to do anything useful yet.
Possible Drawbacks / Trade-offs
Additional Notes