feat: Google API Auth

[crflanigan]

Required for all PRs:

• Updated associated README.md.
• Wrote appropriate unit tests.
• Pull request title or commits are in conventional commit format

resolves #7913

Enhanced the HTTP output plugin enabling it to authenticate against a Google API protected by the OAuth 2.0 protocol.

[reimda]

Looks like this is a continuation of #10369, right @crflanigan?

[crflanigan]

Looks like this is a continuation of #10369, right @crflanigan?

Sure is. We had some git issues that we resolved, and since the old PR description and title had drifted based on feedback from the community, so we went ahead and created an updated one. @reimda

[crflanigan]

@srebhan @Hipska
Here's the new PR 😊

[Hipska]

Looks all good to me, however I would add some extra info in the README file on how to get that credentials file or how it does look like.

[srebhan]

Wow that looks very clean @crflanigan and @zachmares! I just have some small comments regarding import ordering and one regarding choosing a fixed port. Can you please take a look?

[srebhan]

Looks good to me. Well done sons of awesome! ;-) Thanks for the great work @crflanigan and @zachmares!

[crflanigan]

Looks good to me. Well done sons of awesome! ;-) Thanks for the great work @crflanigan and @zachmares!

Thank you for the support!

[reimda]

HTTPClientConfig is also used by plugins/inputs/http and plugins/inputs/openstack. Would it make sense to move this section to a function and call it in those two plugins too?

[reimda]

@zachmares what do you think about adding it to these other plugins? I don't really like the idea of adding an undocumented variable to other plugins if it won't ever be useful.

[zachmares]

@reimda Apologies, I missed this comment yesterday. I'm not entirely sure at the moment. I'd need to defer to the experts for this one.

[crflanigan]

Hi @reimda @srebhan,
This PR was really just for getting through the Google auth using the HTTP output plugin.
I can see that it could make sense to make this available to other plugins, but being that this doesn't fall within the scope of the description or the issue it's trying to solve for I would say it should be a seperate PR.
Does that make sense?

[zachmares]

I'm not sure oauth makes sense in the context of a Telegraf input plugin. This contribution sets auth headers on the payloads written from the output plugin.

I'm happy to discuss and explore this further, but I agree with @crflanigan that those discussions are better suited for a future and separate PR. A use case would be helpful too.

I believe this contribution is self-contained and ready to go.

[Hipska]

I can imagine there are APIs you want to pull with the http input that require Oauth2. I do agree not to implement it in input plugin for now as there is no need for it (yet?)

edit: actually there is: https://github.com/influxdata/telegraf/issues?q=is%3Aissue+is%3Aopen+oauth2

[zachmares]

I moved the Google API Auth settings and code to outputs/http.go as requested.

@reimda @srebhan Please take a look when you get the chance.

[srebhan]

After a short discussion on Slack, @zachmares agrees that this might also be helpful to at least inputs.http, so I suggest to keep the code in common and add the

	// google api auth
	if h.HTTPClientConfig.OAuth2Config.CredentialsFile != "" {
		token, err := h.HTTPClientConfig.OAuth2Config.GetAccessToken(context.Background(), h.URL)
		if err != nil {
			return err
		}
		token.SetAuthHeader(req)
	}

part to both http input and output plugin at least.

[srebhan]

Sorry, seems like I got it wrong. @zachmares, can you please state if this auth method is useful for anything beyond outputs.http or not!? If it is only useful for outputs.http it should move there, otherwise, please keep it in common and add the above code to any plugin where it makes sense...

Sorry for the confusion I caused!

[zachmares]

No worries, Sven! As far as @crflanigan and I know, this contribution only makes sense in the outputs/http context. To recap, our use case is using Telegraf to gather system metrics and flushing them to an endpoint hosted in GCP where only authenticated invocations of the service are allowed.

If a use case arises wherein an inputs plugin wishes to invoke a service in GCP, we'd be happy to move the code back to common.

Until then, we agree with @reimda's principle that if it's not used by both inputs and outputs, then it doesn't belong in common.

[zachmares]

It appears tests are now failing as a result of changes from feat: [inputs/burrow] fill more http transport parameters #6948

edit: resolved.

[Hipska]

All tests seem to succeed..

[zachmares]

"plugins/outputs/graylog TestWriteTCP" passes when I run CGO_ENABLED=1 make test from my development environment. Re-run test? Requesting some intervention on this one... @srebhan

[powersj]

"plugins/outputs/graylog TestWriteTCP" passes when I run CGO_ENABLED=1 make test from my development environment. Re-run test? Requesting some intervention on this one... @srebhan

I've hit the re-run from failed button. These are flaky tests

[reimda]

Marking "request changes" to show we're not ready to merge

[telegraf-tiger]

Download PR build artifacts for linux_amd64.tar.gz, darwin_amd64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

☺️ This pull request doesn't significantly change the Telegraf binary size (less than 1%)

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB	RPM	TAR GZ	ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_i386.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz
		static_linux_amd64.tar.gz