Skip to content

Runtime config endpoint now supports diff parameter #3700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

ethervoid
Copy link
Contributor

What this PR does:

This PR introduces a new parameter for the /runtime_config endpoint that will allow the user to filter the outcome and just get the non-default values of the runtime configuration.

I've tried to add tests for the runtimeConfigurationHandler but given that the handler depends on runtimeConfigManager the tests become very complex unless we're able to mock the runtimeConfigManager

Which issue(s) this PR fixes:
Fixes #3672

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@ethervoid
Copy link
Contributor Author

@pstibrany Don't know where I should ping to ask for review so given that you gently did the last review... :D

Copy link
Contributor

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your PR! Very useful, especially for overrides!

As it is now, "diff" doesn't show diff for entire "runtime config", only for overrides.

I'd also suggest defining the handler in the pkg/cortex/runtime_config.go file, and passing only fully constructed http.Handler to the API registration. This may require moving some methods (eg. yamlMarshalUnmarshal) to util package. There should be no need for api package to depend on validation.

CHANGELOG.md Outdated
Comment on lines 39 to 40
* [ENHANCEMENT] API: Add a `mode` query parameter for the runtime config endpoint: #3645
* `/runtime_config?mode=diff`: Shows the YAML runtime configuration with all values that differ from the defaults.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* [ENHANCEMENT] API: Add a `mode` query parameter for the runtime config endpoint: #3645
* `/runtime_config?mode=diff`: Shows the YAML runtime configuration with all values that differ from the defaults.
* [ENHANCEMENT] API: Add a `mode` query parameter for the runtime config endpoint. `/runtime_config?mode=diff` now shows the YAML runtime configuration with all values that differ from the defaults. #3645

I see it's copied from another entry, but I'd suggest using "Runtime Config:" prefix instead of API:.

CHANGELOG.md Outdated
@@ -36,6 +36,8 @@
* [ENHANCEMENT] Disabled in-memory shuffle-sharding subring cache in the store-gateway, ruler and compactor. This should reduce the memory utilisation in these services when shuffle-sharding is enabled, without introducing a significantly increase CPU utilisation. #3601
* [ENHANCEMENT] Shuffle sharding: optimised subring generation used by shuffle sharding. #3601
* [ENHANCEMENT] New /runtime_config endpoint that returns the defined runtime configuration in YAML format. The returned configuration includes overrides. #3639
* [ENHANCEMENT] API: Add a `mode` query parameter for the runtime config endpoint: #3645
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're used to link the PR number, not the issue number. Please change #3645 > #3700. Thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've just released 1.7.0-rc.0. Could you rebase master and move the CHANGELOG entry under the master / unreleased section, please?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On it!

@ethervoid
Copy link
Contributor Author

@pstibrany I'll tackle the changes today, thank you for the review.

@ethervoid ethervoid force-pushed the issue_3672_diff_for_runtime_config_endpoint branch from d1c304c to 555e4a3 Compare January 22, 2021 19:18
@pull-request-size pull-request-size bot added size/L and removed size/M labels Jan 22, 2021
@ethervoid ethervoid force-pushed the issue_3672_diff_for_runtime_config_endpoint branch 2 times, most recently from f3d9f70 to c786408 Compare January 22, 2021 20:51
@ethervoid
Copy link
Contributor Author

@pstibrany Changes made.

I've made it possible to do a diff in the kv.Multi object but right now I can't see clear default values for that struct, like the ones you can see in the overrides.

Copy link
Contributor

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your continuous work on this PR!

I took a liberty to modify and simplify runtimeConfigHandler, to avoid comparing individual fields of runtimeConfigValues but work with default value which is simply empty struct. (See suggestion)

The rest of the PR looks good.

@ethervoid ethervoid force-pushed the issue_3672_diff_for_runtime_config_endpoint branch from d28ae87 to cf6cc00 Compare January 26, 2021 17:45
ethervoid and others added 5 commits January 26, 2021 18:48
The runtime config endpoint now supports the diff parameter meaning that
using /runtime_config?mode=diff will show only the non-default values
for the configuration

Signed-off-by: Mario de Frutos <mario@defrutos.org>
Signed-off-by: Mario de Frutos <mario@defrutos.org>
To avoid having a dependency of the validation package in the api one we
have move the runtime configuration handle to the cortex package.

This change also led to move some function of the handle to the util
package

Signed-off-by: Mario de Frutos <mario@defrutos.org>
Now we're doing diff with the kv.Multi component for the runtime config

Signed-off-by: Mario de Frutos <mario@defrutos.org>
Co-authored-by: Peter Štibraný <pstibrany@gmail.com>
Signed-off-by: Mario de Frutos <mario@defrutos.org>
@ethervoid ethervoid force-pushed the issue_3672_diff_for_runtime_config_endpoint branch from cf6cc00 to 2d4280e Compare January 26, 2021 17:48
Copy link
Contributor

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for PR and addressing my feedback, this is going to be very useful!

Copy link
Contributor

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks!

@pracucci pracucci merged commit ac54ba9 into cortexproject:master Jan 27, 2021
@ethervoid ethervoid deleted the issue_3672_diff_for_runtime_config_endpoint branch January 27, 2021 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add the diff option for the runtime_config endpoint
3 participants