Skip to content

HTTP/2 to origin #7622

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shinrich wants to merge 3 commits into from
Closed

HTTP/2 to origin #7622

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4deeb25
Revert "Remove incompatible changes for 9.2.0 (#8316)"
bneradt Sep 9, 2021
b13ddc3
Http2 to origin.
shinrich Dec 16, 2020
d40fb86
Fix documentation typo
shinrich Oct 24, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions configs/records.config.default.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ CONFIG proxy.config.http.insert_response_via_str INT 0
# https://docs.trafficserver.apache.org/en/latest/admin-guide/files/parent.config.en.html
##############################################################################
CONFIG proxy.config.http.parent_proxy.retry_time INT 300
CONFIG proxy.config.http.parent_proxy.connect_attempts_timeout INT 30
CONFIG proxy.config.http.forward.proxy_auth_to_parent INT 0
CONFIG proxy.config.http.uncacheable_requests_bypass_parent INT 1

Expand All @@ -60,7 +59,6 @@ CONFIG proxy.config.http.connect_attempts_max_retries INT 3
CONFIG proxy.config.http.connect_attempts_max_retries_dead_server INT 1
CONFIG proxy.config.http.connect_attempts_rr_retries INT 3
CONFIG proxy.config.http.connect_attempts_timeout INT 30
CONFIG proxy.config.http.post_connect_attempts_timeout INT 1800
CONFIG proxy.config.http.down_server.cache_time INT 60

##############################################################################
Expand Down
6 changes: 3 additions & 3 deletions doc/.tx/config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -697,9 +697,9 @@ file_filter = locale/<lang>/LC_MESSAGES/developer-guide/api/functions/TSContMute
source_file = _build/locale/pot/developer-guide/api/functions/TSContMutexGet.en.pot
source_lang = en

[apache-traffic-server-6x.developer-guide--api--functions--TSContSchedule_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/api/functions/TSContSchedule.en.po
source_file = _build/locale/pot/developer-guide/api/functions/TSContSchedule.en.pot
[apache-traffic-server-6x.developer-guide--api--functions--TSContScheduleOnPool_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/api/functions/TSContScheduleOnPool.en.po
source_file = _build/locale/pot/developer-guide/api/functions/TSContScheduleOnPool.en.pot
source_lang = en

[apache-traffic-server-6x.developer-guide--api--functions--TSDebug_en]
Expand Down
54 changes: 9 additions & 45 deletions doc/admin-guide/files/records.config.en.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1296,14 +1296,6 @@ Parent Proxy Configuration
The total number of connection attempts allowed per parent for a specific
transaction, if multiple parents are used.

.. ts:cv:: CONFIG proxy.config.http.parent_proxy.connect_attempts_timeout INT 30
:reloadable:
:overridable:

The timeout value (in seconds) for parent cache connection attempts.

See :ref:`admin-performance-timeouts` for more discussion on |TS| timeouts.

.. ts:cv:: CONFIG proxy.config.http.parent_proxy.mark_down_hostdb INT 0
:reloadable:
:overridable:
Expand Down Expand Up @@ -1573,15 +1565,9 @@ Origin Server Connect Attempts

Set a limit for the number of concurrent connections to an upstream server group. A value of
``0`` disables checking. If a transaction attempts to connect to a group which already has the
maximum number of concurrent connections the transaction either rechecks after a delay or a 503
maximum number of concurrent connections a 503
(``HTTP_STATUS_SERVICE_UNAVAILABLE``) error response is sent to the user agent. To configure

Number of transactions that can be delayed concurrently
See :ts:cv:`proxy.config.http.per_server.connection.queue_size`.

How long to delay before rechecking
See :ts:cv:`proxy.config.http.per_server.connection.queue_delay`.

Upstream server group definition
See :ts:cv:`proxy.config.http.per_server.connection.match`.

Expand Down Expand Up @@ -1615,26 +1601,6 @@ Origin Server Connect Attempts
This setting is independent of the :ts:cv:`setting for upstream session sharing matching
<proxy.config.http.server_session_sharing.match>`.

.. ts:cv:: CONFIG proxy.config.http.per_server.connection.queue_size INT 0
:reloadable:

Controls the number of transactions that can be waiting on an upstream server group.

``-1``
Unlimited.

``0``
Never wait. If the connection maximum has been reached immediately respond with an error.

A positive number
If there are less than this many waiting transactions, delay this transaction and try again. Otherwise respond immediately with an error.

.. ts:cv:: CONFIG proxy.config.http.per_server.connection.queue_delay INT 100
:reloadable:
:units: milliseconds

If a transaction is delayed due to too many connections in an upstream server group, delay this amount of time before checking again.

.. ts:cv:: CONFIG proxy.config.http.per_server.connection.alert_delay INT 60
:reloadable:
:units: seconds
Expand Down Expand Up @@ -1668,15 +1634,6 @@ Origin Server Connect Attempts

See :ref:`admin-performance-timeouts` for more discussion on |TS| timeouts.

.. ts:cv:: CONFIG proxy.config.http.post_connect_attempts_timeout INT 1800
:reloadable:
:overridable:

The timeout value (in seconds) for an origin server connection when the client request is a ``POST`` or ``PUT``
request.

See :ref:`admin-performance-timeouts` for more discussion on |TS| timeouts.

.. ts:cv:: CONFIG proxy.config.http.post.check.content_length.enabled INT 1

Enables (``1``) or disables (``0``) checking the Content-Length: Header for a POST request.
Expand Down Expand Up @@ -3773,7 +3730,7 @@ SSL Termination
Client-Related Configuration
----------------------------

.. ts:cv:: CONFIG proxy.config.ssl.client.verify.server.policy STRING PERMISSIVE
.. ts:cv:: CONFIG proxy.config.ssl.client.verify.server.policy STRING ENFORCED
:reloadable:
:overridable:

Expand Down Expand Up @@ -3902,6 +3859,13 @@ Client-Related Configuration

Enables (``1``) or disables (``0``) TLSv1_3 in the ATS client context. If not specified, enabled by default

.. ts:cv:: CONFIG proxy.config.ssl.client.alpn_protocols STRING ""

Set the alpn string that ATS will send to origin during new connections. By default no ALPN string will be set.
To enable HTTP/2 communication to the origin, set this to "h2,http1.1".
Comment on lines +3864 to +3865
Copy link
Member

@maskit maskit Mar 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to know why it has no ALPN string by default. Having "http1.1" (or "http1.1,http1.0") by default and adding "h2" to enable HTTP/2 sound more natural to me. Also, it may be better to mention about the order.

What will we do if we support H3 to origin? Will we internally filter the string to not offer protocols that are not supported on a transport, or have another setting for QUIC transport?

I'm not sure if users want to specify raw ALPN protocol names. For server_ports, we don't use it.

I don't think this is really going to be a problem, but ALPN protocol name may have commas and any characters. The spec says protocol names are non-empty byte strings.

Copy link
Member Author

@shinrich shinrich Mar 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good questions. I kept it empty by default so the initial integration would have less impact. Of course, there are enough other code changes that landing this PR potentially have an impact even if you don't set the ALPN strings.

We need to have a more general discussion about the preferred order, etc. Should probably look how other similar proxies handle this.

Copy link
Member

@SolidWallOfCode SolidWallOfCode Apr 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the benefit of defaulting to "http1.1". To add HTTP/2, it would still be necessary to specific "http2,http1.1" in the configuration. I agree with Susan's view that, at least initially, this should be backwards compatible in that no HTTP/2 is done outbound unless explicitly configured. That's how HTTP/2 inbound was done - it originally required an explicit "enable" configuration variable to be set.

Copy link
Member

@maskit maskit Apr 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand having "http1.1" by default doesn't make much sense in terms of the behavior, and I know no ALPN negotiation is the current behavior. But if I was not familiar with ALPN, I'd wonder why I have to have "http1.1"as well to enable HTTP/2. I might write only "h2" and say "Aha, H2 works. Seems like http/1.1 is not necessary.". Also listing numbers in descending order seem unnatural if I thought it's just a list. That's why I'm not keen on having this as ALPN setting.


:overridable:

.. ts:cv:: CONFIG proxy.config.ssl.async.handshake.enabled INT 0

Enables the use of OpenSSL async job during the TLS handshake. Traffic
Expand Down
3 changes: 0 additions & 3 deletions doc/admin-guide/files/sni.yaml.en.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,9 +121,6 @@ client_sni_policy Outbound Policy of SNI on outbound connection.
http2 Inbound Indicates whether the H2 protocol should be added to or removed from the
protocol negotiation list. The valid values are :code:`on` or :code:`off`.

disable_h2 Inbound Deprecated for the more general h2 setting. Setting disable_h2
to :code:`true` is the same as setting http2 to :code:`on`.

tunnel_route Inbound Destination as an FQDN and port, separated by a colon ``:``.
Match group number can be specified by ``$N`` where N should refer to a specified group
in the FQDN, ``tunnel_route: $1.domain``.
Expand Down
15 changes: 15 additions & 0 deletions doc/admin-guide/monitoring/statistics/core/http-connection.en.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,21 @@ HTTP/2

Represents the current number of HTTP/2 active connections from client to the |TS|.

.. ts:stat:: global proxy.process.http2.total_server_connections integer
:type: counter

Represents the total number of HTTP/2 connections from |TS| to the origin.

.. ts:stat:: global proxy.process.http2.current_server_connections integer
:type: gauge

Represents the current number of HTTP/2 connections from |TS| to the origin.

.. ts:stat:: global proxy.process.http2.current_active_server_connections integer
:type: gauge

Represents the current number of HTTP/2 active connections from |TS| to the origin.

.. ts:stat:: global proxy.process.http2.connection_errors integer
:type: counter

Expand Down
10 changes: 10 additions & 0 deletions doc/admin-guide/monitoring/statistics/core/http-transaction.en.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,16 @@ HTTP/2

Represents the current number of HTTP/2 streams from client to the |TS|.

.. ts:stat:: global proxy.process.http2.total_server_streams integer
:type: counter

Represents the total number of HTTP/2 streams from |TS| to the origin.

.. ts:stat:: global proxy.process.http2.current_server_streams integer
:type: gauge

Represents the current number of HTTP/2 streams from |TS| to the origin.

.. ts:stat:: global proxy.process.http2.total_transactions_time integer
:type: counter
:units: seconds
Expand Down
19 changes: 0 additions & 19 deletions doc/admin-guide/performance/index.en.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -331,28 +331,9 @@ connection attempt until the origin fully establishes a connection (the connecti
After the connection is established the value of
:ts:cv:`proxy.config.http.transaction_no_activity_timeout_out` is used to established timeouts on the data over the connection.

In the case where you wish to have a different (generally longer) timeout for
``POST`` and ``PUT`` connections to an origin server, you may also adjust
:ts:cv:`proxy.config.http.post_connect_attempts_timeout` which applies only to
origin connections using those HTTP verbs.

::

CONFIG proxy.config.http.connect_attempts_timeout INT 30
CONFIG proxy.config.http.post_connect_attempts_timeout INT 1800

Parent Proxy Timeout
~~~~~~~~~~~~~~~~~~~~

In hierarchical caching configurations, the :ts:cv:`proxy.config.http.parent_proxy.connect_attempts_timeout`
setting is used for all connection attempts to parent caches. It may be useful,
in cases where you wish to have |TS| fall back to an alternate parent cache
(in configurations where you have multiple parents for the same cache) more
quickly, to lower this timeout.

::

CONFIG proxy.config.http.parent_proxy.connect_attempts_timeout INT 30

Polling Timeout
~~~~~~~~~~~~~~~
Expand Down
1 change: 0 additions & 1 deletion doc/admin-guide/plugins/lua.en.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3941,7 +3941,6 @@ Http config constants
TS_LUA_CONFIG_HTTP_CONNECT_ATTEMPTS_RR_RETRIES
TS_LUA_CONFIG_HTTP_CONNECT_ATTEMPTS_TIMEOUT
TS_LUA_CONFIG_HTTP_DOWN_SERVER_CACHE_TIME
TS_LUA_CONFIG_HTTP_DOWN_SERVER_ABORT_THRESHOLD
TS_LUA_CONFIG_HTTP_DOC_IN_CACHE_SKIP_DNS
TS_LUA_CONFIG_HTTP_BACKGROUND_FILL_ACTIVE_TIMEOUT
TS_LUA_CONFIG_HTTP_RESPONSE_SERVER_STR
Expand Down
58 changes: 0 additions & 58 deletions doc/developer-guide/api/functions/TSContScheduleEvery.en.rst
View file
Open in desktop

This file was deleted.

28 changes: 14 additions & 14 deletions ...guide/api/functions/TSContSchedule.en.rst → ...unctions/TSContScheduleEveryOnPool.en.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@

.. default-domain:: c

TSContSchedule
**************
TSContScheduleEveryOnPool
*************************

Synopsis
========
Expand All @@ -28,31 +28,31 @@ Synopsis

#include <ts/ts.h>

.. function:: TSAction TSContSchedule(TSCont contp, TSHRTime timeout)
.. function:: TSAction TSContScheduleEveryOnPool(TSCont contp, TSHRTime every)

Description
===========

Schedules :arg:`contp` to run :arg:`delay` milliseconds in the future. This is approximate. The delay
will be at least :arg:`delay` but possibly more. Resolutions finer than roughly 5 milliseconds will
not be effective. :arg:`contp` is required to have a mutex, which is provided to
Schedules :arg:`contp` to run :arg:`every` milliseconds, on a random thread that belongs to
:arg:`tp`. The :arg:`every` is an approximation, meaning it will be at least :arg:`every`
milliseconds but possibly more. Resolutions finer than roughly 5 milliseconds will not be
effective. Note that :arg:`contp` is required to have a mutex, which is provided to
:func:`TSContCreate`.

The return value can be used to cancel the scheduled event via :func:`TSActionCancel`. This is
effective until the continuation :arg:`contp` is being dispatched. However, if it is scheduled on
another thread this can be problematic to be correctly timed. The return value can be checked with
:func:`TSActionDone` to see if the continuation ran before the return, which is possible if
:arg:`timeout` is `0`. Returns ``nullptr`` if thread affinity was cleared.
The return value can be used to cancel the scheduled event via :func:`TSActionCancel`. This
is effective until the continuation :arg:`contp` is being dispatched. However, if it is
scheduled on another thread this can be problematic to be correctly timed. The return value
can be checked with :func:`TSActionDone` to see if the continuation ran before the return,
which is possible if :arg:`timeout` is `0`.

TSContSchedule() or TSContScheduleEvery() will default to set the thread affinity to the calling thread
when no affinity is already set for example, using :func:`TSContThreadAffinitySet`
If :arg:`contp` has no thread affinity set, the thread it is now scheduled on will be set
as its thread affinity thread.

Note that the TSContSchedule() family of API shall only be called from an ATS EThread.
Calling it from raw non-EThreads can result in unpredictable behavior.

See Also
========

:doc:`TSContScheduleEvery.en`
:doc:`TSContScheduleOnPool.en`
:doc:`TSContScheduleOnThread.en`
25 changes: 19 additions & 6 deletions doc/developer-guide/api/functions/TSContScheduleOnPool.en.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,16 @@ Synopsis
Description
===========

Mostly the same as :func:`TSContSchedule`. Schedules :arg:`contp` on a random thread that belongs to :arg:`tp`.
If thread type of the thread specified by thread affinity is the same as :arg:`tp`, the :arg:`contp` will
be scheduled on the thread specified by thread affinity.
Schedules :arg:`contp` to run :arg:`timeout` milliseconds in the future, on a random thread that
belongs to :arg:`tp`. The :arg:`timeout` is an approximation, meaning it will be at least
:arg:`timeout` milliseconds but possibly more. Resolutions finer than roughly 5 milliseconds will
not be effective. Note that :arg:`contp` is required to have a mutex, which is provided to
:func:`TSContCreate`.

The continuation is scheduled for a particular thread selected from a group of similar threads, as indicated by :arg:`tp`.
The continuation is scheduled for a particular thread selected from a group of similar threads,
as indicated by :arg:`tp`. If :arg:`contp` already has an thread affinity set, and the thread
type of thread affinity is the same as :arg:`tp`, then :arg:`contp` will be scheduled on the
thread specified by thread affinity.

=========================== =======================================================================================
Pool Properties
Expand All @@ -54,6 +59,15 @@ called and continuations that use them have the same restrictions. ``TS_THREAD_P
are threads that exist to perform long or blocking actions, although sufficiently long operation can
impact system performance by blocking other continuations on the threads.

The return value can be used to cancel the scheduled event via :func:`TSActionCancel`. This is
effective until the continuation :arg:`contp` is being dispatched. However, if it is scheduled on
another thread this can be problematic to be correctly timed. The return value can be checked with
:func:`TSActionDone` to see if the continuation ran before the return, which is possible if
:arg:`timeout` is `0`.

If :arg:`contp` has no thread affinity set, the thread it is now scheduled on will be set
as its thread affinity thread.

Note that the TSContSchedule() family of API shall only be called from an ATS EThread.
Calling it from raw non-EThreads can result in unpredictable behavior.

Expand Down Expand Up @@ -124,6 +138,5 @@ the same continuation on two different threads of the same type.
See Also
========

:doc:`TSContSchedule.en`
:doc:`TSContScheduleEvery.en`
:doc:`TSContScheduleEveryOnPool.en`
:doc:`TSContScheduleOnThread.en`
Loading