tls: Add certificate compression algorithm configuration (RFC 8879) #11
+382,182
−98,956
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Jonh Wendell <jwendell@redhat.com>
In STREAMED or FULL_DUPLEX_STREAMED mode, the body or trailers stats are sometime not logged. This is due to a bug in ext_proc logging implementation which only do the logging when the call_start_time_ is set. However, in STREAMED or FULL_DUPLEX_STREAMED body mode cases, there could be multiple data chunks are sent to the server, and the 1st response will clear the call_start_time_. This causes the rest body responses won't be logged. The solution is to not clear call_start_time_ when a response is received. --------- Signed-off-by: Yanjun Xiang <yanjunxiang@google.com>
Signed-off-by: Ryan Northey <ryan@synca.io>
…upstream filter chain (envoyproxy#41879) Fixes an issue when an HTTP filter is in the upstream filter chain, and its encodeHeaders() method is returning StopIteration() , and route retry policy is configured with retry on 5xx. In this case, when a 503 response is received, onUpstreamHeaders() will be called first, and the UpstreamRequest object will be removed from the upstream_request_ list. onUpstreamData() will be called right after, and should just return. --------- Signed-off-by: Yanjun Xiang <yanjunxiang@google.com>
- consolidate/rationalize rbe/cache/bes flags - add sections and reorder bazelrc Signed-off-by: Ryan Northey <ryan@synca.io>
Signed-off-by: Ryan Northey <ryan@synca.io>
improve alphabetic ordering, remove temp compat, fix mobile/local llvm --------- Signed-off-by: Ryan Northey <ryan@synca.io>
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
this isnt strictly required yet - but will be needed for the switch to hermetic llvm toolchains Signed-off-by: Ryan Northey <ryan@synca.io>
…nvoyproxy#42192) building the client cache is expensive/slow, and is about to become much more so this prevents rebuilds on changes to bzl files that dont generally change deps once bzlmod lands this can be improved - eg using lock file expiry Signed-off-by: Ryan Northey <ryan@synca.io>
…nvoyproxy#42018) Also, publish a resource pressure gauge like normal resource monitors. Signed-off-by: Greg Greenway <ggreenway@apple.com>
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
…nsistency (envoyproxy#42198) Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
this does some groundwork for switching to hermetic llvm toolchains - move platform registration to `bazel/platforms` - add `BAZEL_LLVM_PATH` var that can be set via `repo_env` for setting local llvm install - add `@mobile_clang_platform` Signed-off-by: Ryan Northey <ryan@synca.io>
…voyproxy#42158) Commit Message: Track downstream_end_stream_ to support stream recreation with fully read request bodies. Previously, the filter rejected all requests with bodies by checking !callbacks_->decodingBuffer(), even when the body was complete. Now properly distinguishes incomplete vs complete requests by tracking end_stream state through decoder methods, following the same pattern as the router filter's internal redirect support. Additional Description: Risk Level: Testing: updated tests Docs Changes: Release Notes: Platform Specific Features: [Optional Runtime guard:] [Optional Fixes #Issue] Fixes envoyproxy#17891 [Optional Fixes commit #PR or SHA] [Optional Deprecated:] [Optional [API Considerations](https://github.com/envoyproxy/envoy/blob/main/api/review_checklist.md):] --------- Signed-off-by: William Dauchy <william.dauchy@datadoghq.com>
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Signed-off-by: Ryan Northey <ryan@synca.io>
Signed-off-by: Ryan Northey <ryan@synca.io>
Signed-off-by: Ryan Northey <ryan@synca.io>
…42206) Signed-off-by: Ryan Northey <ryan@synca.io>
Signed-off-by: Ryan Northey <ryan@synca.io>
Signed-off-by: Ryan Northey <ryan@synca.io>
## Description Addresses envoyproxy#42132 (comment) --- **Commit Message:** docs: address some nits on the ExtAuthZ docs **Additional Description:** This PR addresses some small nits on the ExtAUthZ docs. **Risk Level:** N/A **Testing:** CI **Docs Changes:** N/A **Release Notes:** N/A Signed-off-by: Rohit Agrawal <rohit.agrawal@databricks.com>
…shment (envoyproxy#41696) ## Description This PR adds a new `upstream_connect_trigger` in TCP Proxy to control how TCP Proxy establishes connections with upstream. In some use-cases we want to delay the connection until we receive the data or let it complete the TLS handshake which is not possible today. --- **Commit Message:** tcp_proxy: add upstream_connect_trigger to control connection establishment **Additional Description:** Adds a new `upstream_connect_trigger` in TCP Proxy to control how TCP Proxy establishes connections with upstream. **Risk Level:** Low **Testing:** Added Unit + Integration Tests **Docs Changes:** Added **Release Notes:** Added --------- Signed-off-by: Rohit Agrawal <rohit.agrawal@databricks.com>
Signed-off-by: dependency-envoy[bot] <148525496+dependency-envoy[bot]@users.noreply.github.com>
…2145) Signed-off-by: Sumit Kumar <sumitkmr@google.com>
…#42191) Signed-off-by: Sumit Kumar <sumitkmr@google.com>
Signed-off-by: wbpcode <wbphub@gmail.com>
…eader (envoyproxy#42416) ## Description This PR adds a new `ip_address_header` which could be used to provide the client IP address instead of computing it using XFF hops. It's valuable in cases where we trust the first layer to correctly determine the client IP and then provide that as a header. --- **Commit Message:** geoip: allow extracting the client IP address from a custom request header **Additional Description:** Adds a new `ip_address_header` which could be used to provide the client IP address instead of computing it using XFF hops. **Risk Level:** Low **Testing:** Added Unit + Integration Tests **Docs Changes:** Added **Release Notes:** Added --------- Signed-off-by: Rohit Agrawal <rohit.agrawal@databricks.com>
…42553) ## Docs Changes: The link under “Partitioning: how to split data among multiple Redis instances” in the Envoy documentation (https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_protocols/redis) no longer points to the intended Redis documentation page and instead redirects to the main Redis docs. This PR updates that link to point to the correct Redis documentation page. ## Reference - Previous Redis Doc Repo: https://github.com/redis/redis-doc - Current Redis Doc Repo: https://github.com/redis/docs Signed-off-by: jonghoonpark <dev@jonghoonpark.com>
Signed-off-by: zirain <zirain2009@gmail.com>
envoyproxy#42314 Signed-off-by: Boteng Yao <boteng@google.com>
## Description This PR adds support for MaxMind Country DB in GeoIP filter. Sometimes corporates purchases the paid version of these datasets and they may not have the city database. --- **Commit Message:** geoip: add support for the MaxMind country db **Additional Description:** Adds support for MaxMind Country DB in GeoIP filter. **Risk Level:** Low **Testing:** Added Unit + Integration Tests **Docs Changes:** Added **Release Notes:** Added --------- Signed-off-by: Rohit Agrawal <rohit.agrawal@databricks.com>
<!-- !!!ATTENTION!!! If you are fixing *any* crash or *any* potential security issue, *do not* open a pull request in this repo. Please report the issue via emailing envoy-security@googlegroups.com where the issue will be triaged appropriately. Thank you in advance for helping to keep Envoy secure. !!!ATTENTION!!! For an explanation of how to fill out the fields, please see the relevant section in [PULL_REQUESTS.md](https://github.com/envoyproxy/envoy/blob/main/PULL_REQUESTS.md) --> Commit Message: there's such log in local ratelimit, so better to have in global. Additional Description: Risk Level: Testing: Docs Changes: Release Notes: Platform Specific Features: [Optional Runtime guard:] [Optional Fixes #Issue] [Optional Fixes commit #PR or SHA] [Optional Deprecated:] [Optional [API Considerations](https://github.com/envoyproxy/envoy/blob/main/api/review_checklist.md):] Signed-off-by: zirain <zirain2009@gmail.com>
## Description This PR adds reference to the Country DB in the GeoIP filter docs. --- **Commit Message:** docs: add reference to country DB in GeoIP docs **Additional Description:** Adds reference to the Country DB in the GeoIP filter docs. **Risk Level:** Low **Testing:** CI **Docs Changes:** N/A **Release Notes:** N/A Signed-off-by: Rohit Agrawal <rohit.agrawal@databricks.com>
## Description This PR bumps up the LCOV numbers for a few targets to reflect the current state. --- **Commit Message:** coverage: adjust the LCOV numbers **Additional Description:** Bumps up the LCOV numbers for a few targets to reflect the current state. **Risk Level:** Low **Testing:** CI **Docs Changes:** N/A **Release Notes:** N/A Signed-off-by: Rohit Agrawal <rohit.agrawal@databricks.com>
Unblock envoyproxy#42570 Signed-off-by: Ryan Northey <ryan@synca.io>
Signed-off-by: dependency-envoy[bot] <148525496+dependency-envoy[bot]@users.noreply.github.com>
currently supported: - 2.31 (default/tested) - 2.28 Signed-off-by: Ryan Northey <ryan@synca.io>
Follows up envoyproxy#41886 Also remove `--build_tests_only` from the CI test so that everything is built and errors like this are caught. Signed-off-by: Jonh Wendell <jwendell@redhat.com>
…nd Replace (envoyproxy#41602) Signed-off-by: Melissa Ginaldi <mginaldi@google.com>
…proxy#42574) Commit Message: Don't access variable used as move construction src Additional Description: Avoid a potential use-after-free or logic error caused by access of a variable that was already used as the argument to a move constructor. Risk Level: low Testing: ran existing unit tests Docs Changes: n/a Release Notes: n/a Platform Specific Features: n/a Signed-off-by: Alex Bakon <abakon@netflix.com>
… logs (envoyproxy#42420) Propagate detailed certificate validation errors (e.g., "certificate has expired", "unable to get local issuer certificate", "SAN matcher failed") from DefaultCertValidator through SslExtendedSocketInfo to the transport failure reason string in access logs. Previously, when TLS certificate validation failed, access logs only showed the generic CERTIFICATE_VERIFY_FAILED error without actionable details. This made debugging TLS issues difficult in production where debug logging is not feasible. The detailed error is now visible in %UPSTREAM_TRANSPORT_FAILURE_REASON% and %DOWNSTREAM_TRANSPORT_FAILURE_REASON% access log formatters. Fixes envoyproxy#42266 Signed-off-by: Prashanth Josyula <prashanth.chaitanya@salesforce.com>
…proxy#42462) ## Description This PR clearly documents the limitations of using `filter_enabled_metadata` in the ExtAuthZ filter and that it doesn't work if the filter itself is disabled in the main filter chain or using a per-route override. We have also provided examples using `ExtensionWithMatcher` which should be the recommended way to invoke filters conditionally. The [previous attempt](envoyproxy#41937) which broke some expectations is reverted. Fix envoyproxy#41501 --- **Commit Message:** ext_authz: document the limitations of filter_enabled_metadata **Additional Description:** Add documentation for describing the limitations around using `filter_enabled_metadata` and provide workaround examples using `ExtensionWithMatcher`. **Risk Level:** N/A **Testing:** CI **Docs Changes:** Added **Release Notes:** N/A --------- Signed-off-by: Rohit Agrawal <rohit.agrawal@databricks.com>
…#42577) Signed-off-by: Jonh Wendell <jwendell@redhat.com>
…42461) Allow envoy_cmake to accept cache_entries as either a dict or a select() statement. This enables conditional CMake cache entries based on build configuration, which is required for FIPS builds where different library paths must be used for FIPS vs non-FIPS builds. When cache_entries is a dict (the common case), the function continues to merge default_cache_entries and wrap the result in a select() for debug builds. When cache_entries is already a select(), it is passed through directly to avoid nested select() statements, which Bazel does not support. This change enables targets like ipp-crypto to use selects.with_or() to conditionally set OPENSSL_CRYPTO_LIBRARY based on whether the build is FIPS-compliant (using libcrypto.a) or non-FIPS (using libcrypto_internal.a). without this patch the build fails with: ``` -- Found OpenSSL: /build/.cache/bazel/_bazel_envoybuild/5510e63bd001cefa746eb005f1949cb5/sandbox/processwrapper-sandbox/6607/execroot/envoy/bazel-out/k8-opt/bin/contrib/cryptomb/private_key_providers/source/ipp-crypto.ext_build_deps/lib/libcrypto_internal.a (found version "") -- Configuring done (6.3s) CMake Error at src/CMakeLists.txt:235 (target_link_libraries): Target "crypto_mb_s" links to: OpenSSL::Crypto but the target was not found. Possible reasons include: * There is a typo in the target name. * A find_package call is missing for an IMPORTED target. * An ALIAS target is missing. CMake Error at src/CMakeLists.txt:273 (target_link_libraries): Target "crypto_mb" links to: OpenSSL::Crypto but the target was not found. Possible reasons include: * There is a typo in the target name. * A find_package call is missing for an IMPORTED target. * An ALIAS target is missing. -- Generating done (0.4s) ``` Signed-off-by: William Dauchy <william.dauchy@datadoghq.com>
this fixes the issue where configure detects kernel support for openat, but then make is unable to build with that support Fix envoyproxy#42562 Signed-off-by: Ryan Northey <ryan@synca.io> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
…nvoyproxy#42303) Commit Message: tracing: add new ability to customize span operation with formatter Additional Description: To close envoyproxy#42288 Risk Level: low. Testing: unit. Docs Changes: n/a. Release Notes: added. Platform Specific Features: n/a. --------- Signed-off-by: wbpcode <wbphub@gmail.com> Signed-off-by: code <wbphub@gmail.com> Co-authored-by: Adi (Suissa) Peleg <adip@google.com>
this fixes _some_ of the remaining issues when trying to build or test using a non-hermetic llvm install Signed-off-by: Ryan Northey <ryan@synca.io>
With a TODO to fix. We can't roll back the PR that introduced the flakiness (envoyproxy#42370) because it was a security patch. Relevant flakiness described in envoyproxy#39856 (comment) Risk Level: test-only Testing: it is Docs Changes: n/a Release Notes: n/a Platform Specific Features: n/a Signed-off-by: Raven Black <ravenblack@dropbox.com>
…voyproxy#42456) #### MCP Request/Response Handling | Operation | Direction | Handling | |-----------|-----------|----------| | `initialize` | Fanout | Aggregate session IDs into composite | | `tools/list` | Fanout | Aggregate tools, prefix names | | `tools/call` | Targeted | Strip prefix, route to specific backend | | `ping` | Fanout | Return 200 with empty JSON response | | `notification/initialized` | Fanout | Return 202 | #### Key Decisions - Handle the streamable HTTP for MCP POST request. - This PR only handles the `application/json` content, `event/stream` response should also be supported but not this PR. - **Body rewrite for `tools/call`**: Use string search to replace prefixed tool name. -- The mcp decoder could provide a cursor to be more efficient and more robust - **Content-Length handling**: Set explicit length after body modification - **Response aggregation**: Full JSON parsing required for `tools/list` (multiple tool names to prefix) - `mcp-session-id` is the combined one from multiple backends, and there should be a limit. - Capability is gateway default now and should be controlled by the gateway. - Should combine with the mcp_filter. And the request flow is mcp_fitler -> other_filters (attributes aware) -> mcp_router -> backends. <img width="3462" height="1252" alt="82yHuofKFAMcHi2" src="https://github.com/user-attachments/assets/25dfd31b-7f8c-4b3a-8306-650801adf283" /> envoyproxy#39174 ``` static_resources: listeners: - name: mcp_listener address: socket_address: address: 0.0.0.0 port_value: 10001 filter_chains: - filters: - name: envoy.filters.network.http_connection_manager typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager stat_prefix: ingress_http route_config: name: local_route virtual_hosts: - name: mcp_gateway domains: ["gateway.com"] routes: - match: path: "/mcp" route: cluster: mcp_service http_filters: - name: envoy.filters.http.mcp typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.mcp.v3.Mcp traffic_mode: REJECT_NO_MCP clear_route_cache: true - name: envoy.filters.http.mcp_router typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.mcp_router.v3.McpRouter servers: - name: echo_one mcp_cluster: cluster: mcp_service path: "/mcp" - name: echo_two mcp_cluster: cluster: mcp_service_2 path: "/mcp" clusters: - name: mcp_service type: STATIC lb_policy: ROUND_ROBIN load_assignment: cluster_name: mcp_service endpoints: - lb_endpoints: - endpoint: address: socket_address: address: 127.0.0.1 port_value: 9000 - name: mcp_service_2 type: STATIC lb_policy: ROUND_ROBIN load_assignment: cluster_name: mcp_service endpoints: - lb_endpoints: - endpoint: address: socket_address: address: 127.0.0.1 port_value: 9001 ``` Commit Message: Additional Description: Risk Level: Testing: Docs Changes: Release Notes: Platform Specific Features: --------- Signed-off-by: Boteng Yao <boteng@google.com>
…42436) Commit Message: [Cleanup] Move exception-throwing configs out of headers Additional Description: There's a block of validation config in `tools/code_format/config.yaml` which in a comment asserts that there should be no other throws from headers, followed by several headers disobeying that rule. This PR moves two of them that seemed like they fit the "templates" criteria up into to the headers block, removes one that already doesn't contain any throws, and for others moves the functions with throws in them out of the header files into cc files. Risk Level: Negligible, should be no behavior change. Testing: No behavior change. Docs Changes: n/a Release Notes: n/a Platform Specific Features: n/a Signed-off-by: Raven Black <ravenblack@dropbox.com>
Add histogram stats to track DER-encoded certificate chain sizes: - peer_certificate_chain_bytes: Size of peer's certificate chain - local_certificate_chain_bytes: Size of local certificate chain Add utility functions to measure certificate sizes: - getCertificateDerSize(): Get DER size of single certificate - getPeerCertificateChainDerSize(): Get total peer chain size - getLocalCertificateChainDerSize(): Get total local chain size These metrics provide visibility into TLS certificate overhead, which is useful for understanding handshake costs and optimizing certificate chains.
Relocate certificate compression implementation from source/common/quic/
to source/common/tls/ to enable compression for both TCP TLS and QUIC:
- Add source/common/tls/cert_compression.{h,cc} with zlib implementation
- Update source/common/tls/context_impl.cc to register zlib compression
- Convert source/common/quic/cert_compression.h to a thin backward-compat
wrapper that delegates to the TLS implementation
- Remove source/common/quic/cert_compression.cc (no longer needed)
- Update BUILD files to reflect new dependencies
This enables TLS certificate compression (RFC 8879) for TCP connections,
which was previously only available for QUIC. Certificate compression
reduces TLS handshake size and bandwidth overhead.
This change adds support for two additional certificate compression algorithms: - Brotli (algorithm ID 2): Better compression ratio, widely supported - Zstd (algorithm ID 3): Fast compression with good compression ratio Changes: - Add registerAll() to register all algorithms in priority order (brotli > zstd > zlib) - Implement compressBrotli/decompressBrotli using libbrotli - Implement compressZstd/decompressZstd using libzstd - Update context_impl.cc to use registerAll() instead of registerZlib() - Update QUIC wrapper to use registerAll() for full algorithm support - Add comprehensive unit tests for all three algorithms The TLS handshake will negotiate the best mutually supported algorithm. Brotli typically provides the best compression for certificates.
This change adds TLS certificate compression support with brotli, zstd, and zlib
algorithms per RFC 8879. Certificate compression reduces TLS handshake size,
especially beneficial for QUIC where the ServerHello needs to fit in the initial
response.
Features:
- Support for brotli (ID 2), zstd (ID 3), and zlib (ID 1) compression algorithms
- Individual registration functions for each algorithm
- Compression stats: certificate_compression.<algo>.{compressed,total_uncompressed_bytes,total_compressed_bytes}
- Runtime feature flag: envoy.reloadable_features.tls_support_certificate_compression (default: false)
Testing:
- Unit tests for compression/decompression round-trips
- Registration tests with real SSL_CTX
- Integration tests for TLS handshake with compression enabled/disabled
Documentation:
- Changelog entry for the new feature
- Stats documentation in ssl_stats.rst
Part of certificate compression implementation (RFC 8879).
bellatoris
force-pushed
the
doogie/tls-cert-compression-config
branch
from
5395759 to
2380347
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Add user-configurable TLS certificate compression algorithms to CommonTlsContext (RFC 8879).
This allows users to specify which algorithms (BROTLI, ZSTD, ZLIB) to use and their priority order.
Key Changes
CertificateCompressionAlgorithmproto message incommon.protocertificate_compression_algorithmsfield toCommonTlsContextregisterFromConfig()toCertCompressionclassenvoy.reloadable_features.tls_support_certificate_compressionDesign Decisions
CommonTlsContext(shared by client and server)How to Enable
envoy.reloadable_features.tls_support_certificate_compression: trueTest plan
//test/common/tls:cert_compression_test- PASSED🤖 Generated with Claude Code