Support TLS for Kafka #1414

MichaHoffmann · 2019-03-09T12:00:33Z

Which problem is this PR solving?

Resolves TLS with Kafka Storage Backend #1401

Short description of the changes

add TLSConfig to kafka configuration and use it in the factories
add flags to enable and configure TLS CA/Cert/Key

yurishkuro · 2019-03-12T21:20:10Z

Missing sign-off, and

Lint Failures
/home/travis/gopath/src/github.com/jaegertracing/jaeger/pkg/kafka/config/config.go:9:6: exported type Configuration should have comment or be unexported
/home/travis/gopath/src/github.com/jaegertracing/jaeger/pkg/kafka/config/config.go:22:1: comment on exported method TLSConfig.GetTLSConfig should be of the form "GetTLSConfig ..."

MichaHoffmann · 2019-03-17T09:40:33Z

sorry i rebased and force pushed; i did not realize that you had a local copy for the review!

codecov · 2019-03-17T10:17:22Z

Codecov Report

Merging #1414 into master will decrease coverage by 0.04%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #1414      +/-   ##
==========================================
- Coverage   98.21%   98.17%   -0.05%     
==========================================
  Files         195      195              
  Lines        9602     9602              
==========================================
- Hits         9431     9427       -4     
- Misses        134      137       +3     
- Partials       37       38       +1

Impacted Files	Coverage Δ
cmd/agent/app/builder.go	`100% <100%> (ø)`	⬆️
cmd/query/app/static_handler.go	`83.33% <0%> (-3.51%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7d339ef...e89911b. Read the comment docs.

yurishkuro · 2019-03-17T21:50:17Z

pkg/kafka/config/.nocover

@@ -0,0 +1 @@
+requires connection to Kafka


Inaccurate. It seems all errors in config.go can be easily simulated by giving non-existent file names, so this package is easy to test.

Yeah, unfortunately i was just lazy.

yurishkuro · 2019-03-17T22:10:56Z

cmd/ingester/app/flags.go

+	// SuffixKey is a suffix for the tls key path flag
+	SuffixKey = ".tls.key"
+	// SuffixCA is a suffix for the tls ca path flag
+	SuffixCA = ".tls.ca"


We're adding the same flags in two places. I think it would be better to refactor these common settings into the new kafka/config package you created, to deal with these flags that are common for produce & consumer:

suffixBrokers = ".brokers" suffixTopic = ".topic" suffixEncoding = ".encoding" suffixTLS = ".tls" suffixCert = ".tls.cert" suffixKey = ".tls.key" suffixCA = ".tls.ca"

It means creating these functions:

func (c *Configuration) AddFlags(prefix string, flagSet *flag.FlagSet) {} func (c *Configuration) InitFromViper(v *viper.Viper) {}

and delegating to them from the respective places in ingester and kafka storage impl

You're right; i will do that

There seems to be no ZipkinThriftMarshaller. That means the encoding has different options for consumer and producer applications. Would something like that be okay for you:

// Initialize implements storage.Factory func (f *Factory) Initialize(metricsFactory metrics.Factory, logger *zap.Logger) error { f.metricsFactory, f.logger = metricsFactory, logger logger.Info("Kafka factory", zap.Any("producer builder", f.Builder), zap.Any("topic", f.options.config.Topic)) p, err := f.NewProducer() if err != nil { return err } f.producer = p switch f.options.config.Encoding { case config.EncodingProto: f.marshaller = newProtobufMarshaller() case config.EncodingJSON: f.marshaller = newJSONMarshaller() case config.EncodingZipkinThrift: return fmt.Errorf("producers do not support '%s' yet", config.EncodingZipkinThrift) default: return fmt.Errorf(`encoding '%s' not recognised, use one of ("%s")`, f.options.config.Encoding, strings.Join(config.AllEncodings, "\", \"")) } return nil }

I'm fine with that. When calling AddFlags on the shared config, you may want to pass a list of allowed encodings so that the help text doesn't say zipkin for producers

pavolloffay · 2019-05-15T12:02:22Z

@MichaHoffmann any chance to rebase this and finalize?

cc) @Demonian

MichaHoffmann · 2019-05-15T12:05:45Z

hey, sorry i was busy last month! i was just now starting to rebase and wanted to finalize it this week!

backjo · 2019-05-19T05:26:13Z

Could we add a configuration for disabling hostname verification?

backjo · 2019-06-04T20:16:31Z

@MichaHoffmann is there any active work on this? Would love to help take this across the finish if i can help!

MichaHoffmann · 2019-06-04T20:34:13Z

@backjo Hey, sorry! I was busy the last weeks; i will polish what i have done so far up on saturday. Regarding Hostname Verification: the sarama config just takes a reference to a tls.Config (https://golang.org/src/crypto/tls/common.go?s=15521:24369#L403), and this has the field InsecureSkipVerify. Maybe one can use this, but i cannot confirm.

Demonian · 2019-06-21T22:36:42Z

@MichaHoffmann Hi, as I can see this PR is almost resolved, maybe you need some help to finish it?=)

netiperher · 2019-07-13T20:45:45Z

pkg/kafka/config/config.go

+		return nil, errors.Wrapf(err, "error reading ca")
+	}
+
+	cert, err := tls.LoadX509KeyPair(filepath.Clean(tlsConfig.KeyPath), filepath.Clean(tlsConfig.CertPath))


I tested this PR and TLS support works for me expect that I had to switch the kafka.producer.tls.cert and kafka.producer.tls.key command line arguments otherwise it couldn't parse the certificate.
According to the documentation the parameter order should be (i.e. swapped):

func LoadX509KeyPair(certFile, keyFile string) (Certificate, error)

https://golang.org/pkg/crypto/tls/#LoadX509KeyPair

thanks, i did miss that!

vikrambe · 2019-08-28T05:02:51Z

black-adder, jpkrohling, objectiser, pavolloffay, tiffon, vprithvi, and/or yurishkuro
Is this PR waiting for the conflicts to be resolved or do we have any other items pending ?

MichaHoffmann · 2019-08-28T07:05:09Z

hey, sorry i did not get to work on this the last weeks, ill resolve the conflicts asap.

Signed-off-by: mhoffmann <michoffmann.potsdam@gmail.com>

yurishkuro · 2019-09-24T00:51:52Z

@MichaHoffmann would you consider some refactoring as part of this? This really bothers me:

$ grep -r LoadX509KeyPair .
./cmd/collector/app/grpcserver/grpc_server.go:48:	tlsCert, err := tls.LoadX509KeyPair(filepath.Clean(cert), filepath.Clean(key))
./cmd/agent/app/reporter/grpc/builder.go:95:			tlsCert, err := tls.LoadX509KeyPair(b.TLSCert, b.TLSKey)
./pkg/es/config/config.go:371:	privateKey, err := tls.LoadX509KeyPair(tlsConfig.CertPath, tlsConfig.KeyPath)

i.e. we already have 3 places in the code that perform essentially identical work of loading TLS keys, and you're adding the 4th place. Similarly, unit tests are repeated, whereas we could've tested loading of TLS files in just one place.

yurishkuro · 2019-09-24T00:53:21Z

And it's not just the loading of the files, all the CLI flags are also repeated. We could have a single pkg that takes a prefix and FlagSet and adds all the usual TLS flags.

MichaHoffmann · 2019-09-24T04:32:34Z

I can do that this weekend, would you be okay with adding flags and config to pkg/config/auth?

yurishkuro · 2019-09-24T14:05:41Z

Maybe pkg/config/tls, because auth is broader.

jpkrohling · 2019-09-26T14:11:27Z

Sorry for the noise, looks like I was using an older version of this PR. I'm still unable to get a working setup, but not sure it has to do with the state of this PR.

~~There's something strange with the TLS' Enabled flag. This is what I see in the collector's logs when passing the TLS-specific options:~~

{"level":"info","ts":1569506473.3507824,"caller":"kafka/factory.go:62","msg":"Kafka factory","producer builder":{"Brokers":["my-cluster-kafka-brokers.kafka:9092"],"TLS":{"Enabled":false,"CertPath":"/var/run/secrets/kafkauser/user.crt","KeyPath":"/var/run/secrets/kafkauser/user.key","CaPath":"/var/run/secrets/kafkauser/ca.crt"}},"topic":"jaeger-spans"}

~~Here are the flags I'm passing:~~

--kafka.producer.brokers=my-cluster-kafka-brokers.kafka:9092
--kafka.producer.tls.ca=/var/run/secrets/kafkauser/ca.crt
--kafka.producer.tls.cert=/var/run/secrets/kafkauser/user.crt
--kafka.producer.tls.key=/var/run/secrets/kafkauser/user.key
--kafka.producer.topic=jaeger-spans

Changing the cert/key to contain invalid chars or a missing location does cause error messages like error parsing tls configuration: error loading certificate: tls: failed to find certificate PEM data in certificate ..., indicating that it did reach the appropriate path from getTLS().

~~I can confirm that spans are being sent and consumed, so, I'm not sure if it's only a problem with the logging, or if TLS isn't being used at all.~~

EDIT:

~~I realized that I was using the wrong port (9092 vs 9093 for TLS), and I changed it accordingly. After that, I get an auth failure, logged on both sides:~~

~~Collector:~~

{"level":"info","ts":1569508180.688255,"caller":"kafka/factory.go:62","msg":"Kafka factory","producer builder":{"Brokers":["my-cluster-kafka-brokers.kafka:9093"],"TLS":{"Enabled":false,"CertPath":"/var/run/secrets/kafkauser/user.crt","KeyPath":"/var/run/secrets/kafkauser/user.key","CaPath":"/var/run/secrets/kafkauser/ca.crt"}},"topic":"jaeger-spans"}
{"level":"fatal","ts":1569508181.4714146,"caller":"collector/main.go:87","msg":"Failed to init storage factory","error":"kafka: client has run out of available brokers to talk to (Is your cluster reachable?)","stacktrace":"main.main.func1\n\t/Users/kearls/go/src/github.com/jaegertracing/jaeger/cmd/collector/main.go:87\ngithub.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra.(*Command).execute\n\t/Users/kearls/go/src/github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra/command.go:762\ngithub.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra.(*Command).ExecuteC\n\t/Users/kearls/go/src/github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra/command.go:852\ngithub.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra.(*Command).Execute\n\t/Users/kearls/go/src/github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra/command.go:800\nmain.main\n\t/Users/kearls/go/src/github.com/jaegertracing/jaeger/cmd/collector/main.go:177\nruntime.main\n\t/usr/local/Cellar/go/1.12.6/libexec/src/runtime/proc.go:200"}

~~Kafka:~~

2019-09-26 14:30:26,211 INFO [SocketServer brokerId=0] Failed authentication with /172.17.0.14 (SSL handshake failed) (org.apache.kafka.common.network.Selector) [data-plane-kafka-network-thread-0-ListenerName(CLIENTTLS)-SSL-8]

~~Either way, the TLS: Enabled flag looks wrong. The auth problem is probably on me, though :)~~

MichaHoffmann · 2019-09-26T18:15:39Z

@jpkrohling at the current HEAD you should add the --kafka.producer.authentication=tls flag i think.

jpkrohling · 2019-09-27T09:17:22Z

@MichaHoffmann, thanks! I'm still unable to get it working, though:

$ git log -1
commit e89911b67177c2b02aa2faee985960eb7d1c4754 (HEAD -> 1401_kafka_storage_tls)
Author: mhoffmann <michoffmann.potsdam@gmail.com>
Date:   Sat Sep 21 18:28:25 2019 +0200

    checkout correct commits in submodules
    
    Signed-off-by: mhoffmann <michoffmann.potsdam@gmail.com>

$ SPAN_STORAGE_TYPE=kafka go run ./cmd/collector/main.go \
    --kafka.producer.authentication=tls \
    --kafka.producer.brokers=my-cluster-kafka-brokers.kafka:9093 \
    --kafka.producer.tls.ca=/tmp/cluster-ca.crt \
    --kafka.producer.tls.cert=/tmp/user.crt \
    --kafka.producer.tls.key=/tmp/user.key \
    --kafka.producer.topic=jaeger-spans

In Kafka's logs, I see this:

2019-09-27 09:14:41,002 INFO [SocketServer brokerId=0] Failed authentication with /127.0.0.1 (SSL handshake failed) (org.apache.kafka.common.network.Selector) [data-plane-kafka-network-thread-0-ListenerName(CLIENTTLS)-SSL-6]

Running OpenSSL Client with the same certs does work, though:

$ openssl s_client \
    -connect my-cluster-kafka-brokers.kafka:9093 \
    -CAfile /tmp/cluster-ca.crt \
    -cert /tmp/user.crt \
    -key /tmp/user.key
CONNECTED(00000003)
depth=1 O = io.strimzi, CN = cluster-ca v0
verify return:1
depth=0 O = io.strimzi, CN = my-cluster-kafka
verify return:1
---
Certificate chain
 0 s:O = io.strimzi, CN = my-cluster-kafka
   i:O = io.strimzi, CN = cluster-ca v0
 1 s:O = io.strimzi, CN = cluster-ca v0
   i:O = io.strimzi, CN = cluster-ca v0
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDzzCCAregAwIBAgIJAMAX8YFJZFRrMA0GCSqGSIb3DQEBCwUAMC0xEzARBgNV
BAoMCmlvLnN0cmltemkxFjAUBgNVBAMMDWNsdXN0ZXItY2EgdjAwHhcNMTkwOTI2
MTMyNDM0WhcNMjAwOTI1MTMyNDM0WjAwMRMwEQYDVQQKDAppby5zdHJpbXppMRkw
FwYDVQQDDBBteS1jbHVzdGVyLWthZmthMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A
MIIBCgKCAQEAqGDpx9psr6ZPQMCXYRTe+F+KNKr+AXCCwhX2pT8+aGnEd4kHwwnX
hBuAc86COBN9Uo90HQPYvfHvUOXVw0usVzjNZHGeHGpngkA7QOkr0Nrbq8HOHvqd
USCE+a+ctQq+orv2zQeFBmx9E5uH2COjO5jVcrV+TGtPlx17rLv7sOZxjUXhNs4n
j3EemEKFuFZue8zOZ2AV3a1QzMwlhhomtO2D/o8nL6gb5V1kOq3yWNvEpgNmiH+D
nR6xwhDs/Xo0kolxFOVEmxJiMCDDXTP3dhRiugup6kAXhz21sp1DXiOmfOziTsYr
klHPzZy2B76WmrMfUj09bZLWCg81ORNRnQIDAQABo4HuMIHrMIHoBgNVHREEgeAw
gd2CQ215LWNsdXN0ZXIta2Fma2EtMC5teS1jbHVzdGVyLWthZmthLWJyb2tlcnMu
a2Fma2Euc3ZjLmNsdXN0ZXIubG9jYWyCJG15LWNsdXN0ZXIta2Fma2EtYm9vdHN0
cmFwLmthZmthLnN2Y4IybXktY2x1c3Rlci1rYWZrYS1ib290c3RyYXAua2Fma2Eu
c3ZjLmNsdXN0ZXIubG9jYWyCGm15LWNsdXN0ZXIta2Fma2EtYm9vdHN0cmFwgiBt
eS1jbHVzdGVyLWthZmthLWJvb3RzdHJhcC5rYWZrYTANBgkqhkiG9w0BAQsFAAOC
AQEAqHNDTKSm5HNm5ws1bWVmOFnu0kamTTkg79IQyFfZWHqUxstO0vMLUgHpZfVK
MAYLSaDGOZf9DWirf8slExaLcmwMkxskFHso7b5YD3hqsaUSVo3jzxLzf2IywqfK
vEbUBQFI2QD+OLrsJQmiOwxnmJoOa+6IbquJ1d9TCmWuiKBB1n8edj2mbnPyCl66
3lOfMp9Fl8y8Eh0PWJB7FUThR7vUUKfyUkD8XuZZorgdUamDGOh7kq+xIEEIykgM
IFhmXcbNCNxOfVuIwgaDdqeNa6aX7GHtlHEX3NDSb4h8wPUSxIOYVAWtOVcgAdWe
u0GmC4ER8XOwOd/JEU2aEuyBVw==
-----END CERTIFICATE-----
subject=O = io.strimzi, CN = my-cluster-kafka

issuer=O = io.strimzi, CN = cluster-ca v0

---
Acceptable client certificate CA names
O = io.strimzi, CN = clients-ca v0
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: ECDSA+SHA512:RSA+SHA512:ECDSA+SHA384:RSA+SHA384:ECDSA+SHA256:RSA+SHA256:DSA+SHA256:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:ECDSA+SHA1:RSA+SHA1:DSA+SHA1
Shared Requested Signature Algorithms: ECDSA+SHA512:RSA+SHA512:ECDSA+SHA384:RSA+SHA384:ECDSA+SHA256:RSA+SHA256:DSA+SHA256:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:ECDSA+SHA1:RSA+SHA1:DSA+SHA1
Peer signing digest: SHA256
Peer signature type: RSA
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2374 bytes and written 1455 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 5D8DD34C65E88EEED2EFAA0F726D4E286FB82914CC80791EE7BE78B8C1449920
    Session-ID-ctx: 
    Master-Key: E1FA47895E2F758890D0C954D7382720A16827FCF52C9F6210ABE2499D8A2148AD4CB66C0106521F5ACE0E67E9E4CA73
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1569575756
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
---

I'm debugging the Go code now to see where the problem might be.

jpkrohling · 2019-09-27T09:32:50Z

Turns out, there's a tool in the sarama repository that was very useful to figure out the problem:

$ kafka-console-consumer -brokers my-cluster-kafka-brokers.kafka:9093 -tls-client-cert /tmp/user.crt -tls-client-key /tmp/user.key -topic jaeger-spans -verbose -tls-enabled true
2019/09/27 11:23:51 Initializing new client
2019/09/27 11:23:51 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2019/09/27 11:23:51 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2019/09/27 11:23:51 client/metadata fetching metadata for all topics from broker my-cluster-kafka-brokers.kafka:9093
2019/09/27 11:23:51 Failed to connect to broker my-cluster-kafka-brokers.kafka:9093: x509: certificate is valid for my-cluster-kafka-0.my-cluster-kafka-brokers.kafka.svc.cluster.local, my-cluster-kafka-bootstrap.kafka.svc, my-cluster-kafka-bootstrap.kafka.svc.cluster.local, my-cluster-kafka-bootstrap, my-cluster-kafka-bootstrap.kafka, not my-cluster-kafka-brokers.kafka
2019/09/27 11:23:51 client/metadata got error from broker -1 while fetching metadata: x509: certificate is valid for my-cluster-kafka-0.my-cluster-kafka-brokers.kafka.svc.cluster.local, my-cluster-kafka-bootstrap.kafka.svc, my-cluster-kafka-bootstrap.kafka.svc.cluster.local, my-cluster-kafka-bootstrap, my-cluster-kafka-bootstrap.kafka, not my-cluster-kafka-brokers.kafka
2019/09/27 11:23:51 client/metadata no available broker to send metadata request to
2019/09/27 11:23:51 client/brokers resurrecting 1 dead seed brokers
2019/09/27 11:23:51 client/metadata retrying after 250ms... (3 attempts remaining)
2019/09/27 11:23:51 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2019/09/27 11:23:51 client/metadata fetching metadata for all topics from broker my-cluster-kafka-brokers.kafka:9093
2019/09/27 11:23:51 Failed to connect to broker my-cluster-kafka-brokers.kafka:9093: x509: certificate is valid for my-cluster-kafka-0.my-cluster-kafka-brokers.kafka.svc.cluster.local, my-cluster-kafka-bootstrap.kafka.svc, my-cluster-kafka-bootstrap.kafka.svc.cluster.local, my-cluster-kafka-bootstrap, my-cluster-kafka-bootstrap.kafka, not my-cluster-kafka-brokers.kafka
2019/09/27 11:23:51 client/metadata got error from broker -1 while fetching metadata: x509: certificate is valid for my-cluster-kafka-0.my-cluster-kafka-brokers.kafka.svc.cluster.local, my-cluster-kafka-bootstrap.kafka.svc, my-cluster-kafka-bootstrap.kafka.svc.cluster.local, my-cluster-kafka-bootstrap, my-cluster-kafka-bootstrap.kafka, not my-cluster-kafka-brokers.kafka
2019/09/27 11:23:51 client/metadata no available broker to send metadata request to
2019/09/27 11:23:51 client/brokers resurrecting 1 dead seed brokers
2019/09/27 11:23:51 client/metadata retrying after 250ms... (2 attempts remaining)
2019/09/27 11:23:51 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2019/09/27 11:23:51 client/metadata fetching metadata for all topics from broker my-cluster-kafka-brokers.kafka:9093
2019/09/27 11:23:51 Failed to connect to broker my-cluster-kafka-brokers.kafka:9093: x509: certificate is valid for my-cluster-kafka-0.my-cluster-kafka-brokers.kafka.svc.cluster.local, my-cluster-kafka-bootstrap.kafka.svc, my-cluster-kafka-bootstrap.kafka.svc.cluster.local, my-cluster-kafka-bootstrap, my-cluster-kafka-bootstrap.kafka, not my-cluster-kafka-brokers.kafka
2019/09/27 11:23:51 client/metadata got error from broker -1 while fetching metadata: x509: certificate is valid for my-cluster-kafka-0.my-cluster-kafka-brokers.kafka.svc.cluster.local, my-cluster-kafka-bootstrap.kafka.svc, my-cluster-kafka-bootstrap.kafka.svc.cluster.local, my-cluster-kafka-bootstrap, my-cluster-kafka-bootstrap.kafka, not my-cluster-kafka-brokers.kafka
2019/09/27 11:23:51 client/metadata no available broker to send metadata request to
2019/09/27 11:23:51 client/brokers resurrecting 1 dead seed brokers
2019/09/27 11:23:51 client/metadata retrying after 250ms... (1 attempts remaining)
2019/09/27 11:23:51 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2019/09/27 11:23:51 client/metadata fetching metadata for all topics from broker my-cluster-kafka-brokers.kafka:9093
2019/09/27 11:23:51 Failed to connect to broker my-cluster-kafka-brokers.kafka:9093: x509: certificate is valid for my-cluster-kafka-0.my-cluster-kafka-brokers.kafka.svc.cluster.local, my-cluster-kafka-bootstrap.kafka.svc, my-cluster-kafka-bootstrap.kafka.svc.cluster.local, my-cluster-kafka-bootstrap, my-cluster-kafka-bootstrap.kafka, not my-cluster-kafka-brokers.kafka
2019/09/27 11:23:51 client/metadata got error from broker -1 while fetching metadata: x509: certificate is valid for my-cluster-kafka-0.my-cluster-kafka-brokers.kafka.svc.cluster.local, my-cluster-kafka-bootstrap.kafka.svc, my-cluster-kafka-bootstrap.kafka.svc.cluster.local, my-cluster-kafka-bootstrap, my-cluster-kafka-bootstrap.kafka, not my-cluster-kafka-brokers.kafka
2019/09/27 11:23:51 client/metadata no available broker to send metadata request to
2019/09/27 11:23:51 client/brokers resurrecting 1 dead seed brokers
2019/09/27 11:23:51 Closing Client
ERROR: Failed to start consumer: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

Using the hostname my-cluster-kafka-0.my-cluster-kafka-brokers.kafka.svc.cluster.local did the trick and I could verify that this PR works as intended.

MichaHoffmann · 2019-09-27T12:32:16Z

@jpkrohling thanks for testing!

yurishkuro

I booked #1837 to refactor all TLS into a single package

MichaHoffmann requested review from black-adder, jpkrohling, objectiser, pavolloffay, tiffon, vprithvi and yurishkuro as code owners March 9, 2019 12:00

ghost assigned yurishkuro Mar 12, 2019

ghost added the review label Mar 12, 2019

MichaHoffmann force-pushed the 1401_kafka_storage_tls branch 3 times, most recently from 01def27 to 695738c Compare March 17, 2019 09:39

MichaHoffmann force-pushed the 1401_kafka_storage_tls branch from 0983dc5 to 7d79203 Compare March 17, 2019 10:08

yurishkuro reviewed Mar 18, 2019

View reviewed changes

Demonian mentioned this pull request May 15, 2019

Tracking release 1.12.0 #1528

Closed

MichaHoffmann force-pushed the 1401_kafka_storage_tls branch 2 times, most recently from 3233799 to 89402a8 Compare May 15, 2019 14:57

MichaHoffmann force-pushed the 1401_kafka_storage_tls branch from 9ffac74 to 75fff2e Compare June 9, 2019 12:09

netiperher reviewed Jul 13, 2019

View reviewed changes

MichaHoffmann force-pushed the 1401_kafka_storage_tls branch from 008da1f to d7a2e21 Compare September 21, 2019 14:44

add tls config to kafka storage

472a9f7

Signed-off-by: mhoffmann <michoffmann.potsdam@gmail.com>

MichaHoffmann force-pushed the 1401_kafka_storage_tls branch from d7a2e21 to a96cd4b Compare September 21, 2019 14:54

fmt & lint

93ae5ea

Signed-off-by: mhoffmann <michoffmann.potsdam@gmail.com>

MichaHoffmann force-pushed the 1401_kafka_storage_tls branch from a96cd4b to 93ae5ea Compare September 21, 2019 14:57

readd pkg/es/config.nocover

93a1238

Signed-off-by: mhoffmann <michoffmann.potsdam@gmail.com>

MichaHoffmann force-pushed the 1401_kafka_storage_tls branch from 3c4a832 to 93a1238 Compare September 21, 2019 15:02

checkout correct commits in submodules

e89911b

Signed-off-by: mhoffmann <michoffmann.potsdam@gmail.com>

MichaHoffmann changed the title ~~WIP: add tls config to kafka storage~~ RFC: add tls config to kafka storage Sep 23, 2019

MichaHoffmann requested a review from yurishkuro September 23, 2019 09:26

jpkrohling mentioned this pull request Sep 27, 2019

[Bug] Missing DNS name in the server certificate strimzi/strimzi-kafka-operator#2027

Closed

jpkrohling approved these changes Oct 1, 2019

View reviewed changes

yurishkuro mentioned this pull request Oct 6, 2019

Refactor TLS flags & logic into a shared module #1837

Closed

yurishkuro approved these changes Oct 6, 2019

View reviewed changes

yurishkuro changed the title ~~RFC: add tls config to kafka storage~~ Support TLS for Kafka Oct 6, 2019

yurishkuro merged commit fcc0adb into jaegertracing:master Oct 6, 2019

MichaHoffmann deleted the 1401_kafka_storage_tls branch October 8, 2019 08:12

yurishkuro added this to the Release 1.15 milestone Nov 7, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support TLS for Kafka #1414

Support TLS for Kafka #1414

MichaHoffmann commented Mar 9, 2019 •

edited

Loading

yurishkuro commented Mar 12, 2019

MichaHoffmann commented Mar 17, 2019 •

edited

Loading

codecov bot commented Mar 17, 2019 •

edited

Loading

yurishkuro Mar 17, 2019

MichaHoffmann Mar 18, 2019

yurishkuro Mar 17, 2019

MichaHoffmann Mar 18, 2019

MichaHoffmann Mar 18, 2019 •

edited

Loading

yurishkuro Mar 18, 2019

pavolloffay commented May 15, 2019

MichaHoffmann commented May 15, 2019

backjo commented May 19, 2019

backjo commented Jun 4, 2019

MichaHoffmann commented Jun 4, 2019

Demonian commented Jun 21, 2019

netiperher Jul 13, 2019

MichaHoffmann Aug 28, 2019

vikrambe commented Aug 28, 2019

MichaHoffmann commented Aug 28, 2019

yurishkuro commented Sep 24, 2019

yurishkuro commented Sep 24, 2019

MichaHoffmann commented Sep 24, 2019

yurishkuro commented Sep 24, 2019

jpkrohling commented Sep 26, 2019 •

edited

Loading

MichaHoffmann commented Sep 26, 2019

jpkrohling commented Sep 27, 2019

jpkrohling commented Sep 27, 2019 •

edited

Loading

MichaHoffmann commented Sep 27, 2019

yurishkuro left a comment

Support TLS for Kafka #1414

Support TLS for Kafka #1414

Conversation

MichaHoffmann commented Mar 9, 2019 • edited Loading

Which problem is this PR solving?

Short description of the changes

yurishkuro commented Mar 12, 2019

MichaHoffmann commented Mar 17, 2019 • edited Loading

codecov bot commented Mar 17, 2019 • edited Loading

Codecov Report

yurishkuro Mar 17, 2019

Choose a reason for hiding this comment

MichaHoffmann Mar 18, 2019

Choose a reason for hiding this comment

yurishkuro Mar 17, 2019

Choose a reason for hiding this comment

MichaHoffmann Mar 18, 2019

Choose a reason for hiding this comment

MichaHoffmann Mar 18, 2019 • edited Loading

Choose a reason for hiding this comment

yurishkuro Mar 18, 2019

Choose a reason for hiding this comment

pavolloffay commented May 15, 2019

MichaHoffmann commented May 15, 2019

backjo commented May 19, 2019

backjo commented Jun 4, 2019

MichaHoffmann commented Jun 4, 2019

Demonian commented Jun 21, 2019

netiperher Jul 13, 2019

Choose a reason for hiding this comment

MichaHoffmann Aug 28, 2019

Choose a reason for hiding this comment

vikrambe commented Aug 28, 2019

MichaHoffmann commented Aug 28, 2019

yurishkuro commented Sep 24, 2019

yurishkuro commented Sep 24, 2019

MichaHoffmann commented Sep 24, 2019

yurishkuro commented Sep 24, 2019

jpkrohling commented Sep 26, 2019 • edited Loading

MichaHoffmann commented Sep 26, 2019

jpkrohling commented Sep 27, 2019

jpkrohling commented Sep 27, 2019 • edited Loading

MichaHoffmann commented Sep 27, 2019

yurishkuro left a comment

Choose a reason for hiding this comment

MichaHoffmann commented Mar 9, 2019 •

edited

Loading

MichaHoffmann commented Mar 17, 2019 •

edited

Loading

codecov bot commented Mar 17, 2019 •

edited

Loading

MichaHoffmann Mar 18, 2019 •

edited

Loading

jpkrohling commented Sep 26, 2019 •

edited

Loading

jpkrohling commented Sep 27, 2019 •

edited

Loading