x509_cert: add support for SMTP certificates over STARTTLS

[tryfail]

Feature Request

Some mail servers I support, cannot be monitored through telegraf's x509_cert input plugin. This happens, because these services only expose certificates after receiving a STARTTLS SMTP message.

I believe, this will also make the plugin more complete as per its current description.

Proposal:

Add support for SMTP+STARTTLS through telegraf's x509_cert input plugin.

Current behavior:

Using a source like tcp://smtp.example.com:587, will produce the following error:

[inputs.x509_cert] Error in plugin: cannot get SSL cert 'tcp://smtp.example.com:587': tls: first record does not look like a TLS handshake

Desired behavior:

Using something like smtp+starttls://smtp.example.com:587 will produce the certificates as usual for HTTPS connections and other TLS-wrapped protocols.

Use case:

The use case is to support certificate monitoring for SMTP servers that do not have a TLS-wrapped endpoint and will only support TLS with STARTTLS in the long term.

[danielnelson]

I not sure we want this plugin to become a plugin that speaks various protocols to extract the certificate, outside of the very common tcp handshake special case. There are so many potential ways you could get a certificate. Wouldn't be better to run Telegraf on the SMTP server and report the certificate from there as a local file?

[tryfail]

I not sure we want this plugin to become a plugin that speaks various protocols to extract the certificate, outside of the very common tcp handshake special case. There are so many potential ways you could get a certificate. Wouldn't be better to run Telegraf on the SMTP server and report the certificate from there as a local file?

Hello Daniel,

Thank you very much for all you do.

Unfortunately, for my use case, the factor of STARTTLS-only enabled servers to telegraf-allowed servers has been increasing. I am familiar with the alternative of monitoring with a local instance of telegraf, but this is often not possible at least due to policy, 3rd-party hosted environments and non-familiar software stacks (e.g. proprietary servers)

From the little I read about the history of the plugin and its description, I believe supporting SMTP STARTTLS makes sense design-wise, and I eventually found a use-case for it. If this still does not fit, I would very much recommend changing the description for the plugin to include only those type of TLS connections. Alternatively, do you believe this would make more sense as a separate plugin (e.g. x509_cert_smtp)?

For example, I would also be very interested in TLS validation of UDP OpenVPN servers, but that is complex on another level. This is the kind of thing that I imagine the plugin not supporting.

Thanks

[danielnelson]

Alternatively, do you believe this would make more sense as a separate plugin (e.g. x509_cert_smtp)?

No, definitely not a standalone plugin.

For example, I would also be very interested in TLS validation of UDP OpenVPN servers, but that is complex on another level. This is the kind of thing that I imagine the plugin not supporting.

I guess this is what I'm struggling to decide on, how many protocols for obtaining the certificate to we want to support and what should be considered in scope and out of scope for the plugin.

Right now we have file system and tcp/https (also apparently udp?). However there are many ways the certificate could be fetched over the network: SMTP+STARTTLS, LDAP+STARTTLS, OpenVPN, etc.

I don't think we want to say that we will allow any protocol. I'd like to come up with a strategy for deciding what this plugin should and should not handle that we can apply consistently.

@reimda @ssoroka What are your thoughts on how much to expand this plugin?

[reimda]

x509_cert seems like the right place to add new cert sources. I don't see a problem adding lots of new protocols to it as long as we can keep the configuration clear.

Since the sources config option is a list of URIs, it's tempting to invent a URI scheme for each new protocol and then shoehorn all new protocols into URIs. I'd prefer if we stick to registered URI schemes (see https://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml). If we want to add a protocol that doesn't have a registered scheme or just doesn't fit in a URI maybe we should add new config options to the plugin.

For SMTP with STARTTLS, I don't think we would need to use smtp+starttls://. Schemes with a plus are for when one protocol wraps another. STARTTLS is part of the SMTP protocol so it makes sense to me to use smtp:// which is a registered scheme.

[ssoroka]

On the non-registered scheme: I don't have strong feelings on this, except to say that there are non-registered schemes in standard use, such as svn+ssh://.... and that cramming communication protocol layers into the scheme seems problematic past trivial examples.

There do seem to be other protocols that support StartTLS, LDAP maybe.

I'd be in favor of this with a simple or clean implementation, and re-evaluate when we get a second request to extend