Description
openedon Feb 15, 2023
Overview
Over the years, the Logstash plugins have evolved without keeping consistency in mind, both for user experience and feature supported. This is especially evident for plugins that support SSL/TLS, there is a lack of naming convention and functionality across all the plugins, making it difficult for users to understand and properly configure SSL/TLS on those plugins. The goals of this project are to to standardize SSL settings names and funcionality across supported Logstash's plugins.
SSL/TLS Settings Naming Convention
Not all the plugins will support all these options, It will depend on a few aspects such as the plugin’s network client, language/libraries's support to settings, if it’s acting as a client or server, etc.
For all cases, if the plugin decides to support a new SSL setting that is defined in this naming convention, they should do it following this specifications:
| Setting name | Type | Default | Allowed values | Description |
|---|---|---|---|---|
| ssl_enabled | boolean | true | Used to enable or disable TLS/SSL | |
| ssl_verification_mode | string | Plugins acting as a client: fullserver: certificate |
full: Validates that the provided certificate: - has an issue date that’s within the not_before and not_after dates; - chains to a trusted Certificate Authority (CA); has a hostname or IP address that matches the names within the certificate. certificate: Validates the provided certificate and verifies that it’s signed by a trusted authority (CA), but doesn’t check the certificate hostname.none: Performs no certificate validation. |
Used to define the SSL/TLS verification mode. The rules are much the same for client and server mode, but for server, this option is secondary to ssl_client_authentication |
| ssl_client_authentication | string | none | optional: Requests a client certificate but the client is not required to present one.required: Forces a client to present a certificate.none: No client authentication |
Controls the server’s behavior in regard to requesting a certificate from client connections. |
| ssl_certificate | path | SSL certificate to use. This certificate should be PEM-formatted, and may contain a chain of certificates starting with the certificate that identifies itself, followed by zero or more ordered intermediates optionally ending with the root signing authority. | ||
| ssl_certificate_authorities | array | List of paths to PEM encoded certificate files that should be trusted. | ||
| ssl_key | path | Path to a PEM encoded file containing the private key. If client authentication is required, it should use this SSL key. | ||
| ssl_key_passphrase | string | The passphrase that is used to decrypt the private key. Since the key might not be encrypted, this value is optional. | ||
| ssl_keystore_path | path | The path for the keystore file that contains a private key and certificate. It must be either a Java keystore (jks) or a PKCS#12 | ||
| ssl_keystore_password | string | The password for the keystore. | ||
| ssl_keystore_type | string | If the keystore path ends in ".p12", ".pfx", or ".pkcs12", this setting could defaults to pkcs12. Otherwise, it defaults to jks. | jks, pkcs12 | The format of the keystore file. |
| ssl_keystore_key_password | The password for the key in the keystore. The default is the keystore password. | |||
| ssl_truststore_path | path | The path for the keystore that contains the certificates to trust. It must be either a Java keystore (jks) or a PKCS#12 file. | ||
| ssl_truststore_password | string | The password for the truststore. | ||
| ssl_truststore_type | string | If the keystore path ends in ".p12", ".pfx", or ".pkcs12", this setting could defaults to pkcs12. Otherwise, it defaults to jks. | jks, pkcs12 | The format of the truststore file. |
| ssl_cipher_suites | array | Java’s default cipher suites | The list of cipher suites to use, listed by priorities. Supported cipher suites vary depending on which version of Java is used. | |
| ssl_handshake_timeout | number | 10.000 | Time in milliseconds for an incomplete ssl handshake to timeout | |
| ssl_supported_protocols | array | Depends on the JDK being used. With up-to-date Logstash, the default should be: ['TLSv1.2', 'TLSv1.3']. | TLSv1.1, TLSv1.2, TLSv1.3 | Supported protocols with versions. |
Implementation Plan
Phase 0 - Technical Specification
- Initial plugins exploration
- Logstash SSL Standardization - Naming convention and functionality
Phase 1 - Standardizing settings names and add supported missing configs
This phase consists of making plugin's SSL settings consistent with the naming convention defined in this issue. Missing settings already supported by the plugin's HTTP/TCP client, which only requires a small coding change on the plugin source, should also be added during this phase.
Taks (sorted by priority)
Elasticsearch
Phase 2 - Standardize functionality (TBD)
In terms of SSL settings, all the plugins should - as much as possible - behave consistently, using the same allowed and default values for settings and providing the same set of functionality across them.
General functionality
- Standardize allowed and default values for settings
-
ssl_enable: There are different behavior implemented for this flag, from disabling the settings to prefixing hosts. -
ssl_verification_mode: Some plugins havenoneandfull, others havetrue/false
Certificate formats
Plugins are not consistent in terms of certificate format, there are plugins accepting PEM files/PKCS8, PKCS12 files, and the actual X.509 certificate content in a base64-encoded string. To improve the user experience, plugins should accept the same set of certificate types.
Tasks (to be planned)
- List and define supported-by-default certificate formats: PEM, PKS8, PKS12, DER
- Discuss the possibility of having a shared library to handle different certificate/keys formats (eg. Elasticsearch ssl_config)
Phase 3 - Add missing/additional functionality (TBD)
Add missing/additional functionality that was too complex/big to fit into phases 1 & 2.
- ssl_cipher_suites not all plugins support it.
- Some plugins support using the truststore others don't.
- Some plugins don't support ssl_verification_mode
- Check plugins candidates and add support to ssl_peer_metadata/enrich
Discussion Items
- Should we consider a central mixin library that implements all of this? Similar to what logstash-mixin-http_client
Related issues
- plugins https/ssl options inconsistencies #2151
- [Meta] Support TLS 1.3 #10494
- Add trusted CA cert setting logstash-plugins/logstash-output-s3#237
- Provide option to disable SSL hostname verification logstash-plugins/logstash-filter-elasticsearch#49
- Logstash “Elasticsearch” output plugin does not support SSL client certificates logstash-plugins/logstash-output-elasticsearch#672
ssl_key_passphrasedoes not work logstash-plugins/logstash-input-beats#364- Specifying
ssl => trueleads to exception "URI::InvalidURIError: bad URI(is not URI?)" logstash-plugins/logstash-filter-elasticsearch#102 - Can only pass .crt as cacert, not pem logstash-plugins/logstash-output-elasticsearch#929
- Is there a way to specify multiple cacert files logstash-plugins/logstash-output-elasticsearch#964
- Disabling TLSv1, TLSv1.1 in logstash tcp input is not working in Logstash 6.8.23 logstash-plugins/logstash-input-tcp#206
- Support pkcs#12 stores logstash-plugins/logstash-input-http#97
- Add support for pkcs#1 keys logstash-plugins/logstash-input-http#96
- https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/998/files
- Documentation: ssl_verify_mode verification logstash-plugins/logstash-input-beats#317 # Related to this issue, we need to investigate why we don't have hostname verification, when we should.
