Support skipping certificate verification when establishing a Quic connection

[mum4k]

Envoys seems to skip certificate verifications by default as documented in the API.

However the Quic implementation of the createQuicNetworkConnection function currently doesn't take in any configuration and always uses the EnvoyQuicProofVerifier.

This prevents establishment of Quic connection in load testing applications where we might be running with test / bogus certificates. This might also be affecting other use cases, so hoping to start a discussion whether this is something we want to improve by allowing a configuration that skips the certificate verification.

[danzh2010]

Worth pointing out that the Envoy config trusted_ca allows skipping root cert verification. This should be supported already by EnvoyQuicProofVerifier which calls into the same defatul cert validator.
However, the difference is that EnvoyQuicProofVerifier also verifies hostname in CHLO matches SNI domains in leaf cert:

envoy/source/common/quic/envoy_quic_proof_verifier.cc

Line 47 in 110559f

for (const absl::string_view& config_san : cert_view->subject_alt_name_domains()) {

It worth discussing whether this match should be mandatory or configurable.

[mum4k]

Thank you @danzh2010 for adding the context, I confirm that this is the verification in question.

[alyssawilk]

I'm torn here - I like QUIC matching existing TLS behavior but I also like verifying by default.
I am inclined to say if we want to change the default behavior we should do it before QUIC exits alpha (and config locks down). Given QUIC and TLS share config I lean towards compatibility and sorting out TLS defaults later.
cc @yanavlasov @RyanTheOptimist @mattklein123 for thoughts

[RyanTheOptimist]

Wow! I feel very strongly that certificates should be verified by default. Having things insecure by default seems like a disaster waiting to happen. What would it take to change this behavior? (Maybe this is not practical in the short term but I really think this needs some investigation)

[alyssawilk]

generally changing high risk config defaults like this in Envoy involves a envoy-announce email, and a runtime guarded change if the config isn't present, so folks can flip the flag while they realize the envoy-announce email affected them. This is likely to break some deployments but I agree I think for security it's one of those breaking changes we really ought to do.
I'll take on comms and the deprecation dance if @mattklein123 agrees it's worth tackling.

[mattklein123]

All things being equal I agree we should verify by default, however 2 gotchas:

1. As Alyssa says we will need to warn first, then fail with runtime, etc. It will be a lengthy migration.
2. The verification settings are incredibly convoluted. Figuring out how to decide in code/config whether verification is "enabled" is going to require some debate in and of itself, and then quite a bit of testing.

I think it would be worth it to track ^ as a separate issue.

[alyssawilk]

Ok, so what I'd suggest then is we leave QUIC verifying by default, and add the knob to turn it off at Nighthawk's convenience: as it'll keep the prior default there's no need to lock this in before alpha wraps up.

What we should do before we leave alpha @danzh2010 is to document the inversion in both QUIC and TLS docs so folks don't get surprised. The explanation can link to the issue tracking TLS switching defaults, which I've just filed as #17771

[danzh2010]

I will update the document about the difference. Adding the knob to turn off verification is not a blocker for MVP, right?

[alyssawilk]

correct!