feat: generate the plugins sample config

[sspaink]

resolve: #9733

Description

Adding a tool called generate_plugindata that reads a plugins README.md and extracts the sample config, then using the source file as a template inject the sample configuration into it. The purpose is to avoid having to maintain the sample config in two different spots, in the README.md and in the code. This will lead to removing 11,000 lines of duplicate sample configuration from our source code that we no longer have to maintain!

Each plugin would have a header like this:

//go:generate go run ../../../tools/generate_plugindata/main.go
//go:generate go run ../../../tools/generate_plugindata/main.go --clean

Then the sample config function would be updated like this:

func (p *Plugin) SampleConfig() string {
	return `{{ .SampleConfig }}`
}

and if you run make you would see this output:

$ make
go mod download -x
go generate -run="plugindata/main.go$" ./plugins/inputs/... ./plugins/outputs/... ./plugins/processors/... ./plugins/aggregators/...
go build -ldflags " -X main.commit=7d8fc6f0 -X main.branch=generatesampleconfig -X main.goos=linux -X main.goarch=amd64 -X main.version=1.23.0-7d8fc6f0" ./cmd/telegraf
go generate -run="plugindata/main.go --clean" ./plugins/inputs/... ./plugins/outputs/... ./plugins/processors/... ./plugins/aggregators/...

Plugin migration

All the plugins have been migrated to this new format in separate pull requests. Separate pull requests were made to keep this pull request smaller and easier to review originally this was 396 files changed with all plugins migrated. I wrote a simple local tool to help automate the migration of the plugins. Unfortunately, there are minor differences between the README.md and source code configs that will have to be manually reviewed and fixed to determine the correct one.

• Output plugins: feat: migrate output plugins to new sample config format #10910
• Input Plugins:
  • (A-L): feat: migrate input plugins to new sample config format (A-L) #10924
  • (M-Z): feat: migrate input plugins to new sample config format (M-Z) #10926
• Aggregator Plugins: feat: migrate aggregator plugins to new sample config format #10912
• Processor Plugins: feat: migrate processor plugins to new sample config format #10913

Testing

Resulting telegraf.conf: https://gist.github.com/sspaink/554f4843e87afbaa39bb436625a39b4d

You can generate your own local config by running the following commands:

1. git checkout -b testbranch
2. git merge origin/generatesampleconfig && git merge origin/updateinputconifg_a-l --no-edit && git merge origin/updateinputconifg_m-z --no-edit && git merge origin/updateaggregatorconfig --no-edit && git merge origin/updateprocessorconfig --no-edit && git merge origin/updateoutputconfig -s recursive -X theirs --no-edit
3. make
4. ./telegraf config > telegraf.conf
5. git reset --hard origin/generatesampleconfig

[reimda]

I love this idea

[powersj]

This is awesome! A few questions before this afternoon

[sspaink]

TODO:

• move func SampleConfig() string into separate file, keep it separate from main source file.
• Look into using https://github.com/yuin/goldmark
• Split feat: migrate input plugins to new sample config format #10911 into multiple pull requests
• Look into rewriting file opposed to renaming/removing file

[telegraf-tiger]

Download PR build artifacts for linux_amd64.tar.gz, darwin_amd64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

☺️ This pull request doesn't significantly change the Telegraf binary size (less than 1%)

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB	RPM	TAR GZ	ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_i386.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz
		static_linux_amd64.tar.gz