spec: Add spec items for environment fallback hosts

content/client-lib-development-guide/features.textile

@@ -91,13 +91,23 @@ h3(#restclient). RestClient
 *** @(RSC8e2)@ In the case of a response with a @statusCode@ less than 400, an error must be propagated back to the caller with a @statusCode@ of 400, a @code@ of @40013@ and an error message containing the original @statusCode@ with an error message indicating that the content type is unsupported. In order to help to diagnose the cause of the error, the error message should also include text obtained by truncating the response body to a maximum of 512 bytes followed by base64-encoding.
 * @(RSC9)@ Uses @Auth@ to establish what authentication scheme to use, how to authenticate, and automatic issuing of tokens when necessary
 * @(RSC10)@ If a REST request responds with a token error (401 HTTP status code and an Ably error value @40140 <= code < 40150@), then the Auth class is responsible for reissuing a token and the request should be reattempted, see "RSA4a":#RSA4a and "RSA4b":#RSA4b
-* @(RSC11)@ Requests are sent to the default endpoint @rest.ably.io@. However, if the @restHost@ option is set, the client will send requests to the specified host. If @environment@ option is configured and is not "production", the environment name is prefixed to the default host endpoint and the @restHost@ is set accordingly. For example, if the @environment@ is set to sandbox, then the @restHost@ endpoint will become @sandbox-rest.ably.io@. See "TO3k2":#TO3k2 for constraints.
+* @(RSC11)@ Requests are sent to the default endpoint @rest.ably.io@. However, this endpoint can be overriden by setting either the @restHost@ or @environment@ option. See "TO3k2":#TO3k2 for constraints.
+** @(RSC11a)@ If @restHost@ is set, send requests to the specified host
+** @(RSC11b)@ If @environment@ is set to a value other than "production", send requests to @[environment]-rest.ably.io@. For example, if the @environment@ is set to @sandbox@, send requests to @sandbox-rest.ably.io@
 * @(RSC12)@ REST endpoint host is configurable in the Client constructor with the option @restHost@
 * @(RSC13)@ The client library must use the connection and request timeouts specified in the @ClientOptions@, falling back to the defaults described in @ClientOptions@ below
 * @(RSC15)@ Host Fallback
-** @(RSC15b)@ The fallback behavior described below only applies when the default @rest.ably.io@ endpoint is being used and has not been overriden (see "RSC11":#RSC11), @ClientOptions#fallbackHostsUseDefault@ is @true@, or an array of @ClientOptions#fallbackHosts@ is provided. If host fallback is not supported, failing HTTP requests that would have "qualified for a retry against a fallback host (see RSC15d)":#RSC15d, will instead result in an error immediately
+** @(RSC15b)@ The fallback behavior described below only applies when either @ClientOptions#restHost@ has not been set to an explicit value (see "RSC11a":#RSC11a), the deprecated @ClientOptions#fallbackHostsUseDefault@ option is set to @true@, or an array of @ClientOptions#fallbackHosts@ is provided. If host fallback does not apply, failing HTTP requests that would have "qualified for a retry against a fallback host (see RSC15d)":#RSC15d, will instead result in an error immediately
 ** @(RSC15e)@ The primary host is by default @rest.ably.io@ (unless overriden in @ClientOptions#environment@ or @ClientOptions#restHost@), which, through DNS, is automatically routed to the client's closest datacenter. New HTTP requests (except where @RSC15f@ applies and a cached fallback host is in effect) are first attempted against the primary host.
-** @(RSC15a)@ In the case of an error necessitating use of an alternative host (see "RSC15d":#RSC15d), try fallback hosts (with a matching Host header as this is necessary when fallbacks are proxied through a CDN) in random order, continuing to try further hosts if "qualifying errors":#RSC15d occur, failing when all have been tried or the configured @httpMaxRetryCount@ has been reached (see "TO3l@":#TO3l5). This ensures that a client library is able to work around routing or other problems for the user's closest datacenter. For example, if a @POST@ request to @rest.ably.io@ fails because the default endpoint is unreachable or unserviceable, then the @POST@ request should be retried again against the fallback hosts in attempt to find an alternate healthy datacenter to service the request. The five default fallback hosts are @[a-e].ably-realtime.com@. If an array of custom fallback hosts are provided in @ClientOptions#fallbackHosts@, then they will be used instead. If an empty array of fallback hosts is provided, then fallback host functionality is disabled
+** @(RSC15a)@ In the case of an error necessitating use of an alternative host (see "RSC15d":#RSC15d), try fallback hosts in random order, continuing to try further hosts if "qualifying errors":#RSC15d occur, failing when all have been tried or the configured @httpMaxRetryCount@ has been reached (see "TO3l@":#TO3l5). This ensures that a client library is able to work around routing or other problems for the user's closest datacenter. For example, if a @POST@ request to @rest.ably.io@ fails because the default endpoint is unreachable or unserviceable, then the @POST@ request should be retried again against the fallback hosts in attempt to find an alternate healthy datacenter to service the request
+** @(RSC15g)@ When the use of fallbacks applies, the set of fallback hosts is chosen as follows:
+*** @(RSC15g1)@ If @ClientOptions#fallbackHosts@ is set to an array, use those as the fallback hosts. See "TO3k6":#TO3k6 for constraints
+*** @(RSC15g2)@ If @ClientOptions#environment@ is set to a value other than "production" and @ClientOptions#fallbackHosts@ is not set, use the environment fallback hosts (see "RSC15i":#RSC15i)
+*** @(RSC15g3)@ If both @ClientOptions#fallbackHosts@ and @ClientOptions#environment@ are not set, use the default fallback hosts (see "RSC15h":#RSC15h)
+*** @(RSC15g4)@ For backwards-compatibility only, if @ClientOptions#fallbackHostsUseDefault@ is set to @true@ and @ClientOptions#fallbackHosts@ is not set, do not consider the value of the @ClientOptions#environment@ option and use the default fallback hosts (see "RSC15h":#RSC15h)
+** @(RSC15h)@ The default fallback hosts are @a.ably-realtime.com@, @b.ably-realtime.com@, @c.ably-realtime.com@, @d.ably-realtime.com@, and @e.ably-realtime.com@
+** @(RSC15i)@ The environment fallback hosts have the format @[environment]-[a-e]-fallback.ably-realtime.com@. For example, if @ClientOptions#environment@ is set to @sandbox@, the environment fallback hosts are @sandbox-a-fallback.ably-realtime.com@, @sandbox-b-fallback.ably-realtime.com@, @sandbox-c-fallback.ably-realtime.com@, @sandbox-d-fallback.ably-realtime.com@, and @sandbox-e-fallback.ably-realtime.com@
+** @(RSC15j)@ Requests to fallback hosts must use a matching Host header as this is necessary when fallbacks are proxied through a CDN. For example, if a request to @rest.ably.io@ fails and will be retried to @c.ably-realtime.com@, the Host header must be set to @c.ably-realtime.com@ in the retried request
 ** @(RSC15d)@ Errors that necessitate use of an alternative host include: host unresolvable or unreachable, request timeout, or a response but with an applicable HTTP status code in the range @500 <= code <= 504@. Resending requests that have failed for other failure conditions will not fix the problem and will simply increase the load on other datacenters unnecessarily
 ** @(RSC15f)@ Once/if a given fallback host succeeds, the client should store that successful fallback host for @ClientOptions.fallbackRetryTimeout@. Future HTTP requests during that period should use that host. If during this period a "qualifying errors":#RSC15d occurs on that host, or after @fallbackRetryTimeout@ has expired, it should be un-stored, and the fallback sequence begun again from scratch, starting with the default primary host (@rest.ably.io@ or @ClientOptions#restHost@) or, in the case of an existing fallback realtime connection as per (RTN17e), with the current fallback realtime host.
 * @(RSC17)@ When instancing the library, if a @clientId@ attribute is set in @ClientOptions@, then the @Auth#clientId@ attribute will contain the provided @clientId@
@@ -332,7 +342,7 @@ h3(#realtimeclient). RealtimeClient
 ** @(RTC1b)@ @autoConnect@ boolean is true by default. If true, as soon as the client library is instanced, it will connect to Ably. If false, the client library will wait for an explicit @Connection#connect@ to be called before connecting
 ** @(RTC1c)@ @recover@ string, when set, will attempt to recover the connection state of a previous connection
 ** @(RTC1d)@ @realtimeHost@ string, when set, will modify the realtime endpoint host used by this client library
-** @(RTC1e)@ @environment@ string, when set, will modify both the REST and realtime endpoint hosts by prefixing the environment to the default endpoint host with a hyphen delimiter. For example, a @RealtimeClient@ with an @environment@ of "sandbox", would use "sandbox-rest.ably.io" as the @restHost@ and @sandbox-realtime.ably.io@ as the @realtimeHost@. See "TO3k3":#TO3k3 for constraints.
+** @(RTC1e)@ @environment@ string, when set to a value other than "production", use @[environment]-rest.ably.io@ as the REST endpoint and @[environment]-realtime.ably.io@ as the realtime endpoint. For example, a @RealtimeClient@ with an @environment@ of "sandbox" would use "sandbox-rest.ably.io" as the REST endpoint and @sandbox-realtime.ably.io@ as the realtime endpoint. See "TO3k3":#TO3k3 for constraints.
 ** @(RTC1f)@ @transportParams@ map or equivalent, additional parameters to be sent in the querystring when initiating a realtime connection. Keys are @Strings@, values are @Stringifiable@ (a value that can be coerced to a string in order to be sent as a querystring parameter. Supported values should be at least strings, numbers, and booleans, with booleans stringified as @true@ and @false@. If this is unidiomatic to the language, the implementer may consider this as equivalent to @String@).
 *** @(RTC1f1)@ If a key in @transportParams@ is one the library sends by default (for example, @v@ or @heartbeats@), the value in @transportParams@ takes precedence.
 * @(RTC2)@ @RealtimeClient#connection@ attribute provides access to the underlying @Connection@ object
@@ -454,9 +464,9 @@ h3(#realtime-connection). Connection
 ** @(RTN16e)@ If the @recover@ option is missing or no longer valid when connecting to Ably, the client will connect anyway, but emit a @ConnectionStateChange@ with a @reason@, and will additionally set the @Connection#errorReason@ with an @ErrorInfo@ object describing the failure
 ** @(RTN16f)@ The @msgSerial@ component of the @recoveryKey@, unlike the other two components, is not sent to Ably, but rather is used to set the library internal @msgSerial@. (If the recover fails, the counter should be reset to 0 per "RTN15c3":#RTN15c3 )
 * @(RTN17)@ Host Fallback
-** @(RTN17b)@ The fallback behavior described below only applies when the default @realtime.ably.io@ endpoint is being used and has not been overriden (see "RTC1d":#RTC1d and "RTC1e":#RTC1e), @ClientOptions#fallbackHostsUseDefault@ is @true@, or an array of @ClientOptions#fallbackHosts@ is provided.
-** @(RTN17a)@ By default, every connection attempt is first attempted to the default primary host @realtime.ably.io@ (unless overriden in @ClientOptions#realtimeHost@), which, through DNS, is automatically routed to the client's closest datacenter. The client library must always prefer the default endpoint (closest datacenter), even if a previous connection attempt to that endpoint has failed. (That is, @RSC15f@ does not apply)
-** @(RTN17c)@ In the case of an error necessitating use of an alternative host (see "RTN17d":#RTN17d), the @Connection@ manager should first check if an internet connection is available by issuing a @GET@ request to @https://internet-up.ably-realtime.com/is-the-internet-up.txt@. If the request succeeds and the text "yes" is included in the body, then the client library can assume it has a viable internet connection and should then immediately retry the connection against all fallback hosts to find an alternative healthy datacenter. The five default fallback hosts are @[a-e].ably-realtime.com@ and should be attempted in random order. The connection requests must include a matching Host header as this is necessary when fallbacks are proxied through a CDN. See "RSC15a":#RSC15a for details on how custom fallback hosts are specified and used
+** @(RTN17b)@ The fallback behavior described below only applies when either @ClientOptions#realtimeHost@ has not been set to an explicit value (see "RTC1d":#RTC1d), the deprecated @ClientOptions#fallbackHostsUseDefault@ option is set to @true@, or an array of @ClientOptions#fallbackHosts@ is provided.
+** @(RTN17a)@ By default, every connection attempt is first attempted to the default primary host @realtime.ably.io@ (unless overriden in @ClientOptions#environment@ or @ClientOptions#realtimeHost@), which, through DNS, is automatically routed to the client's closest datacenter. The client library must always prefer the default endpoint (closest datacenter), even if a previous connection attempt to that endpoint has failed. (That is, @RSC15f@ does not apply)
+** @(RTN17c)@ In the case of an error necessitating use of an alternative host (see "RTN17d":#RTN17d), the @Connection@ manager should first check if an internet connection is available by issuing a @GET@ request to @https://internet-up.ably-realtime.com/is-the-internet-up.txt@. If the request succeeds and the text "yes" is included in the body, then the client library can assume it has a viable internet connection and should then immediately retry the connection against fallback hosts in random order to find an alternative healthy datacenter. Fallback hosts are chosen as per "RSC15g":#RSC15g, and must use a matching Host header as per "RSC15j":#RSC15j
 ** @(RTN17d)@ Errors that necessitate use of an alternative host include: host unresolvable or unreachable, connection timeout, or a @DISCONNECTED@ response with an @error.statusCode@ in the range @500 <= code <= 504@ or HTTP response status code in the range @500 <= code <= 504@. Attempting to reconnect to a fallback host for other failure conditions will not fix the problem and will simply increase the load on other data-centers unnecessarily
 ** @(RTN17e)@ If the realtime client is connected to a fallback host endpoint, then for the duration that the transport is connected to that host, all HTTP requests, such as history or token requests, should be first attempted to the same datacenter the realtime connection is established with i.e. the same fallback host must be used as the default HTTP request host. If however the HTTP request against that fallback host fails, then the normal fallback host behavior should be followed attempting the request against another fallback host as described in "RSC15":#RSC15
 * @(RTN19)@ Transport state side effects - when a transport is upgraded or disconnected for any reason:
@@ -1346,7 +1356,7 @@ h4. ClientOptions
 *** @(TO3k2)@ @restHost@ string - for development environments only; allows a non-default Ably REST host to be specified. It is never valid to provide both a @restHost@ and @environment@ value
 *** @(TO3k3)@ @realtimeHost@ string - for development environments only; allows a non-default Ably Realtime host to be specified. It is never valid to provide both a @realtimeHost@ and @environment@ value
 *** @(TO3k6)@ @fallbackHosts@ string array - optionally allows one or more fallback hosts to be used instead of the default fallback hosts. If an empty array is specified, then fallback host functionality is disabled
-*** @(TO3k7)@ @fallbackHostsUseDefault@ boolean - optionally allows the default fallback hosts @[a-e].ably-realtime.com@ to be used when @environment@ is not production or a custom realtime or REST host endpoint is being used. It is never valid to configure @fallbackHost@ and set @fallbackHostsUseDefault@ to @true@
+*** @(TO3k7)@ @fallbackHostsUseDefault@ boolean (deprecated) - optionally allows the default fallback hosts @[a-e].ably-realtime.com@ to be used when @environment@ is not production or a custom realtime or REST host endpoint is being used. It is never valid to configure @fallbackHost@ and set @fallbackHostsUseDefault@ to @true@. The @fallbackHostsUseDefault@ option is deprecated and future library releases will ignore any supplied value
 *** @(TO3k4)@ @port@ integer - for development environments only; allows a non-default Ably non-TLS port to be specified
 *** @(TO3k5)@ @tlsPort@ integer - for development environments only; allows a non-default Ably TLS port to be specified
 ** @(TO3l)@ The follow attributes, if set, are used to change the default behavior of the library: