Frequently getting socket timeouts when downloading the datafile

[MThoma202]

Hello, our production service is frequently getting socket timeouts when downloading the datafile.

Here's the full stack trace:

2020-02-10 21:51:33.759Z  ERROR 1 --- [pool-6-thread-1] c.o.ab.config.HttpProjectConfigManager   : Error fetching datafile
java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
    at java.net.SocketInputStream.read(SocketInputStream.java:171)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
    at sun.security.ssl.InputRecord.read(InputRecord.java:503)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
    at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:933)
    at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
    at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
    at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
    at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
    at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
    at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
    at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
    at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
    at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
    at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
    at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
    at com.optimizely.ab.OptimizelyHttpClient.execute(OptimizelyHttpClient.java:64)
    at com.optimizely.ab.config.HttpProjectConfigManager.poll(HttpProjectConfigManager.java:115)
    at com.optimizely.ab.config.PollingProjectConfigManager$ProjectConfigFetcher.run(PollingProjectConfigManager.java:182)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Looks like the socket timeout is hard-coded to 10 seconds in HttpClientUtils.java which should be sufficient, so it's a bit surprising that we're seeing this issue.

We're using version 3.3.1 of com.optimizely.ab:core-api and com.optimizely.ab:core-httpclient-impl.

It's not possible to customize the socket timeout used by the OptimizelyHttpClient because the build() method uses HttpClientUtils.DEFAULT_REQUEST_CONFIG.

Could OptimizelyHttpClient please be updated to allow us to specify our own socket timeout setting? Or could anyone give a recommendation of what else we could try doing?

Thanks,
Mark

[jaeopt]

@MThoma202 we released a rev (3.8.2) with this fix. Let us know if it works for you.
Note that we switched to MavenCentral (Bintray/JCenter sunset) and you can find the new version (3.8.2) in there.
We'll upload new versions to MavenCentral while serving old versions in JCenter until Feb. 01, 2022.