Skip to content

Add /config handler which exposes the current YAML config of cortex. #2165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Feb 24, 2020

Conversation

tomwilkie
Copy link
Contributor

@tomwilkie tomwilkie commented Feb 20, 2020

Signed-off-by: Tom Wilkie tom@grafana.com

Fixes #1886

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@tomwilkie tomwilkie force-pushed the config-endpoint branch 2 times, most recently from 575e91e to 961ffa0 Compare February 20, 2020 16:06
Copy link
Contributor

@cyriltovena cyriltovena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, We need this in Loki too and also add overrides via /config/{tenant_id}/overrides.

LGTM

Copy link
Contributor

@Wing924 Wing924 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to mask passwords.

@cyriltovena
Copy link
Contributor

I think we need to mask passwords.

Good point, although I think the usage is meant to be internal.

Copy link
Contributor

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Tom for this! I would love to have it as well.

I gave it a try and unfortunately the generated config doesn't work if you try to use it (ie. you get cannot unmarshal !!map into string) because the marshalling of some custom data types is missing or incorrect. I personally would expect to be able to copy-paste the config and get it working.

Few examples of data types with this issue:

flagext.URLValue:

  externalurl:
    url:
      scheme: ""
      opaque: ""
      user: null
      host: ""
      path: ""
      rawpath: ""
      forcequery: false
      rawquery: ""
      fragment: ""

url.URL:

  externalurl:
    url: null

In this regards, an integration test which runs a component (ie. distributor), gets the /config, save the config to file and re-run the same component with the store config could help to spot regressions over the time.

I think we need to mask passwords.

What if we add a config option to enable this API while keeping it disabled by default?

tomwilkie and others added 5 commits February 22, 2020 16:48
Signed-off-by: Tom Wilkie <tom@grafana.com>
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Signed-off-by: Tom Wilkie <tom@grafana.com>
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
(Also regen docs)

Signed-off-by: Tom Wilkie <tom@grafana.com>
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Signed-off-by: Tom Wilkie <tom@grafana.com>
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Signed-off-by: Tom Wilkie <tom@grafana.com>
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Signed-off-by: Marco Pracucci <marco@pracucci.com>
Signed-off-by: Marco Pracucci <marco@pracucci.com>
Signed-off-by: Marco Pracucci <marco@pracucci.com>
Signed-off-by: Marco Pracucci <marco@pracucci.com>
Copy link
Contributor

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tomwilkie I've pushed some commits. Please take a look.

In particular, the omitempty across the config structs causes some troubles with this API. For example, if in the config you set min_ready_duration: 0s, given 0s is the zero value, the min_ready_duration will not be included in the exported config and loading the exported config will load the default value (1m). I think it could be worth to get rid of all omitempty. What's your take?

@tomwilkie
Copy link
Contributor Author

Thanks @pracucci! Your commits LGTM. Good catch re:omitempty, yes I think these should all be removed - and there are a lot of them.

Copy link
Contributor

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re:omitempty, yes I think these should all be removed - and there are a lot of them.

Yes, but we can postpone it to a separate PR.

@pracucci pracucci merged commit 537b34c into master Feb 24, 2020
@pracucci pracucci deleted the config-endpoint branch February 24, 2020 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expose HTTP endpoint to export live config in yaml format
4 participants