Http plugin add cookie auth

[jh125486]

Required for all PRs:

• Updated associated README.md.
• Wrote appropriate unit tests.

resolves #5932

1. Added a few configuration options for inputs.http cookie auth:
   auth url
   body
   method
   refresh interval
2. HTTP Client is updated to use a CookieJar
3. During plugin Init(), a initial auth is attempted and is successful, a Go routine is fired off for cookie auth renewal

[srebhan]

Hey @jh125486, thanks for tackling this issue!

The code, with one exeception, looks good with only minor issues (see my comments in the code). However, I have a big concern reusing the http client for the cookie authentication as you are also reusing e.g. username and password with a URL potentially different to the settings in urls. This might leak sensible data.

Furthermore, I ask myself if this new feature wouldn't be a good candidate to add to plugins/common/http so it can be reused by other plugins as well. Do you think this makes sense?

[jh125486]

Hey @jh125486, thanks for tackling this issue!

The code, with one exeception, looks good with only minor issues (see my comments in the code). However, I have a big concern reusing the http client for the cookie authentication as you are also reusing e.g. username and password with a URL potentially different to the settings in urls. This might leak sensible data.

Furthermore, I ask myself if this new feature wouldn't be a good candidate to add to plugins/common/http so it can be reused by other plugins as well. Do you think this makes sense?

I can move it to plugins/common/http ... I wasn't even aware that was there really :)
Does it make more sense in the telegraf codebase to have it in plugins/common/cookie and embed it into the plugins/common/http HTTPClientConfig (mirroring the HTTPProxy and TLS config)?

Regarding the http.Client re-use, there should be no direct leaking of creds with using the same client, since that is per-request. Currently the HTTP Basic Auth and OAuth mechanisms are for all URLs in the URLs slice, so that leak is already happening.
As for using the same auth methods/headers for the Cookie Auth, I can definitely break that out into a separate request fn.

[srebhan]

Wow that's really nice. I have some more comments and questions, but I think we are very close. Thanks for working on this!

[srebhan]

Almost. I had one idea about passing the auth() errors to the caller using a non-blocking channel. Maybe you can check if this is doable.
Furthermore, I see the timing issues in your tests. While I do not have a good idea on how to do it correctly in your case, I fear flakey tests in later (unrelated) PRs (we have been there...).

[srebhan]

One more suggestion for allowing to omit the logging. There might be some plugins that do not have a logger.
Otherwise this looks good.

[srebhan]

Looks good to me. Thanks @jh125486 for working hard to get this PR to the current state!

[helenosheaa]

Thanks for the PR, looks good. I think it would be useful if you could provide more information in the readme as to the use-case of this. So new users can understand why they might want to use verses when they don't need to use this setting.

[jh125486]

provide more information in the readme as to the use-case of this. So new users can understand why they might want to use verses when they don't need to use this setting.

The READMEs currently do not reference any of the options within the toml, so what section name should I create for this?

[srebhan]

@jh125486 I guess @helenosheaa wants a small section of text in plugins/inputs/http/README.md (and plugins/outputs/http/README.md accordingly) describing under which circumstances a user should fill this. Is there e.g. a certain popular service that uses this type of authentication that you could use as an example?
So something like

Cookie authentication allows to derive an authentication cookie from a server providing your credentials as body. A popular example is XYZ which work with the following configuration...

[jh125486]

@jh125486 I guess @helenosheaa wants a small section of text in plugins/inputs/http/README.md (and plugins/outputs/http/README.md accordingly) describing under which circumstances a user should fill this. Is there e.g. a certain popular service that uses this type of authentication that you could use as an example?
So something like

Cookie authentication allows to derive an authentication cookie from a server providing your credentials as body. A popular example is XYZ which work with the following configuration...

Correct, so what should the name of that section be?

[helenosheaa]

@jh125486 perhaps - Optional Cookie Authentication Settings: with the explanation and an example config.

[telegraf-tiger]

Looks like new artifacts were built from this PR. Get them here!

Artifact URLs

• windows_amd64.zip

• windows_i386.zip

• i386.rpm

• freebsd_i386.tar.gz

• linux_i386.tar.gz

• i386.deb

• aarch64.rpm

• linux_arm64.tar.gz

• arm64.deb

• s390x.rpm

• linux_s390x.tar.gz

• s390x.deb

• linux_mipsel.tar.gz

• mipsel.deb

• x86_64.rpm

• freebsd_amd64.tar.gz

• linux_amd64.tar.gz

• amd64.deb

• ppc64le.rpm

• linux_ppc64le.tar.gz

• ppc64el.deb

• linux_mips.tar.gz

• mips.deb

• armel.rpm

• linux_armel.tar.gz

• armel.deb

• static_linux_amd64.tar.gz

• armv6hl.rpm

• freebsd_armv7.tar.gz

• linux_armhf.tar.gz

• armhf.deb

• darwin_amd64.tar.gz

[MyaLongmire]

This is looking great so far! Can you please add a sample config? Also, is there any way to make your optional cookie authentication section a little shorter? It seems a touch wordy.

[jh125486]

This is looking great so far! Can you please add a sample config? Also, is there any way to make your optional cookie authentication section a little shorter? It seems a touch wordy.

1. An example config in addition to the example config in the README? What should that file be called?
2. That section is three sentences long. What is the max number of sentences I should have?

[srebhan]

@MyaLongmire there is the usual sample-config part in the http input/output sections (options starting with cookie_auth_). Do you have a suggestion on how to shorten the text?