Skip to content

[improve][broker] Add idle timeout support for http#25224

Merged
lhotari merged 2 commits intoapache:masterfrom
nodece:add-idle-timeout-support-for-http
Feb 6, 2026
Merged

[improve][broker] Add idle timeout support for http#25224
lhotari merged 2 commits intoapache:masterfrom
nodece:add-idle-timeout-support-for-http

Conversation

@nodece
Copy link
Member

@nodece nodece commented Feb 6, 2026 •
edited
Loading

Motivation

When uploading a large package through the Pulsar proxy, the default idle timeout is too short to accommodate the duration of the upload. As a result, the HTTP client triggers a timeout, causing the upload to fail and returning a 504 Gateway Timeout to the user. This behavior is illustrated by the following logs:

2026-02-06T07:14:57,181+0000 [JettyHttpClient@22d8f4ed-scheduler-1] DEBUG org.eclipse.jetty.client.HttpReceiver - Request/Response failed: Result[HttpRequest[POST /admin/v3/packages/function/public/default/test-pkg-2/7.0 HTTP/1.1]@363b647f > HttpResponse[null 0 null]@26b34803] java.util.concurrent.TimeoutException: Idle timeout 30000 ms, notifying [org.eclipse.jetty.proxy.ProxyServlet$ProxyResponseListener@4fc68ea1]
2026-02-06T07:14:57,181+0000 [JettyHttpClient@22d8f4ed-scheduler-1] DEBUG org.apache.pulsar.proxy.server.AdminProxyHandler.15400fff - 38844857 proxying failed
java.util.concurrent.TimeoutException: Idle timeout 30000 ms
	at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onIdleExpired(HttpConnectionOverHTTP.java:160) ~[org.eclipse.jetty-jetty-client-9.4.54.v20240208.jar:9.4.54.v20240208]
	at org.eclipse.jetty.io.AbstractEndPoint.onIdleExpired(AbstractEndPoint.java:402) ~[org.eclipse.jetty-jetty-io-9.4.54.v20240208.jar:9.4.54.v20240208]
	at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:171) ~[org.eclipse.jetty-jetty-io-9.4.54.v20240208.jar:9.4.54.v20240208]
	at org.eclipse.jetty.io.IdleTimeout.idleCheck(IdleTimeout.java:113) ~[org.eclipse.jetty-jetty-io-9.4.54.v20240208.jar:9.4.54.v20240208]
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[?:?]
	at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
	at java.lang.Thread.run(Unknown Source) ~[?:?]

Modifications

  • Add httpServerIdleTimeout to the broker and proxy config, which controls the connection between the client and broker/proxy
  • Add httpProxyIdleTimeout to the proxy config, which controls the connection between the proxy and the broker

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Feb 6, 2026
lhotari
lhotari approved these changes Feb 6, 2026
View reviewed changes
Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just comments about adding the keys to config files

@nodece nodece force-pushed the add-idle-timeout-support-for-http branch 2 times, most recently from a7b249d to 84ae0b7 Compare February 6, 2026 12:32
@nodece nodece force-pushed the add-idle-timeout-support-for-http branch from 84ae0b7 to b8e2444 Compare February 6, 2026 12:33
@nodece
Copy link
Member Author

nodece commented Feb 6, 2026

This PR has been updated to unify the httpServerIdleTimeout configuration for both the proxy and broker, with the key added to each component's config file.

@lhotari lhotari merged commit 63220ea into apache:master Feb 6, 2026
54 of 55 checks passed
@lhotari lhotari added this to the 4.2.0 milestone Feb 6, 2026
lhotari pushed a commit that referenced this pull request Feb 6, 2026
@nodece @lhotari
[improve][broker] Add idle timeout support for http (#25224)
6421425 
(cherry picked from commit 63220ea)
@nodece nodece deleted the add-idle-timeout-support-for-http branch February 6, 2026 15:32
lhotari pushed a commit that referenced this pull request Feb 6, 2026
@nodece @lhotari
[improve][broker] Add idle timeout support for http (#25224)
144e064 
(cherry picked from commit 63220ea)
nodece added a commit to ascentstream/pulsar that referenced this pull request Feb 9, 2026
@nodece
[improve][broker] Add idle timeout support for http (apache#25224)
16bcfc9 
(cherry picked from commit 63220ea)
nodece added a commit to ascentstream/pulsar that referenced this pull request Feb 9, 2026
@nodece
[improve][broker] Add idle timeout support for http (apache#25224)
e1d8e00 
(cherry picked from commit 63220ea)
hshankar31 pushed a commit to datastax/pulsar that referenced this pull request Feb 11, 2026
@nodece @hshankar31
[improve][broker] Add idle timeout support for http (apache#25224)
464cedc 
(cherry picked from commit 63220ea)
(cherry picked from commit 144e064)
hshankar31 pushed a commit to datastax/pulsar that referenced this pull request Feb 16, 2026
@nodece @hshankar31
[improve][broker] Add idle timeout support for http (apache#25224)
150ab70 
(cherry picked from commit 63220ea)
(cherry picked from commit 144e064)
priyanshu-ctds pushed a commit to datastax/pulsar that referenced this pull request Feb 18, 2026
@hshankar31 @Demogorgon314
@zymap @lhotari @dependabot @nodece @thetumbled @BewareMyPower @zjxxzjwang @guptas6est @merlimat @oneby-wang @poorbarcode @sandeep-mst @0asys @omarkj @gulecroc @izumo27 @coderzc @Denovo1998 @Technoboy- @berg223 @zhanghaou @zhaizhibo
[improve][misc]LS: 1408 cherry pick commit from pulsar branch 4 0 to …
a54cf70 
…datastax 4 0 ds 16 feb (#589)

* [improve][broker] Ensure metadata session state visibility and improve Unstable observability for ServiceUnitStateChannelImpl (apache#25132)

(cherry picked from commit 2a29be0)
(cherry picked from commit 85dc758)

* [improve][broker] Upgrade bookkeeper to 4.17.3 (apache#25166)

(cherry picked from commit 45def39)
(cherry picked from commit 333110a)

* fix license and pom file

* [fix][ml] Fix NoSuchElementException in EntryCountEstimator caused by a race condition (apache#25177)

(cherry picked from commit 9b70ba3)
(cherry picked from commit 9261869)

* [fix][test] Bump org.assertj:assertj-core from 3.27.5 to 3.27.7 (apache#25186)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit ce4ebea)
(cherry picked from commit 2c3402e)

* [improve][misc] Upgrade snappy version to 1.1.10.8 (apache#25182)

(cherry picked from commit b15f53b)
(cherry picked from commit 304fea1)

* [fix][proxy] Close client connection immediately when credentials expire and forwardAuthorizationCredentials is disabled (apache#25179)

(cherry picked from commit 3348470)
(cherry picked from commit c06f8ba)

* [fix][client] ControlledClusterFailover avoid unnecessary reconnection. (apache#25178)

Co-authored-by: fengwenzhi <fengwenzhi.max@bigo.sg>
(cherry picked from commit f0ec07b)
(cherry picked from commit b41488d)

* [fix][sec] Bump org.apache.solr:solr-core from 9.8.0 to 9.10.1 in /pulsar-io/solr (apache#25175)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit a2f888a)
(cherry picked from commit b532068)

* [improve][pip] PIP-453: Improve the metadata store threading model (apache#25173)

(cherry picked from commit c51346f)
(cherry picked from commit d81d6b3)

* [improve][client]Reduce unnecessary getPartitionedTopicMetadata requests when using retry and DLQ topics. (apache#25172)

(cherry picked from commit 52a4d5e)
(cherry picked from commit 71a3994)

* [fix][misc] Allow JWT tokens in OpenID auth without nbf claim (apache#25197)

(cherry picked from commit d630394)
(cherry picked from commit 2760ee9)

* [fix][sec] Exclude org.lz4:lz4-java and standardize on at.yawk.lz4-java to remediate CVE-2025-12183 and CVE-2025-66566 (apache#25198)

(cherry picked from commit c07f2ad)
(cherry picked from commit 2ac6d03)

* fix checkstyle failure and license issues

* [fix] [test] Upgrade docker-java to 3.7.0 (apache#25209)

(cherry picked from commit 4add84c)
(cherry picked from commit 92b5d55)

* [fix][client] Fix race condition between isDuplicate() and flushAsync() method in PersistentAcknowledgmentsGroupingTracker due to incorrect use Netty Recycler (apache#25208)

(cherry picked from commit 5aab2f0)
(cherry picked from commit 2206949)

* [improve][monitor] Upgrade OpenTelemetry to 1.56.0, Otel instrumentation to 2.21.0 and Otel semconv to 1.37.0 (apache#24994)

(cherry picked from commit 53162ff)
(cherry picked from commit a1d5b6c)

* [improve][proxy] Add regression tests for package upload with 'Expect: 100-continue' (apache#25211)

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
(cherry picked from commit e8fedb1)
(cherry picked from commit 0947639)

* fix license issues

* [fix][test]Fix flaky ExtensibleLoadManagerImplTest_testGetMetrics (apache#25216)

(cherry picked from commit 257d42f)
(cherry picked from commit a8eac91)

* [fix][broker] Fix ManagedCursorImpl.asyncDelete() method may lose previous async mark delete properties in race condition (apache#25165)

(cherry picked from commit bea6f8a)
(cherry picked from commit 4332a44)

* [fix][broker]Fix ledgerHandle failed to read by using new BK API (apache#25199)

(cherry picked from commit 6d51f88)
(cherry picked from commit 1631fed)

* [fix][client] Fix producer synchronous retry handling in failPendingMessages method (apache#25207)

(cherry picked from commit 611efe4)
(cherry picked from commit 30ae8fb)

* [fix][broker] Prevent missed topic changes in topic watchers and schedule periodic refresh with patternAutoDiscoveryPeriod interval (apache#25188)

(cherry picked from commit 2e06cc0)
(cherry picked from commit ba2a230)

* fix for complilation error

* [feat][io] implement pip-297 for jdbc sinks (apache#25195)

(cherry picked from commit 6f4ac21)
(cherry picked from commit 998a4b1)

* [fix][broker] Fix httpProxyTimeout config (apache#25223)

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
(cherry picked from commit 2d6ef6f)
(cherry picked from commit 3b39c7b)

* [improve][broker] Add strictAuthMethod to require explicit authentication method (apache#25185)

Co-authored-by: Ómar K. Yasin <oyasin@apple.com>
(cherry picked from commit bae9173)
(cherry picked from commit 27e34f6)

* [feat][client] oauth2 trustcerts file and timeouts (apache#24944)

(cherry picked from commit b789d82)
(cherry picked from commit f8827bd)

* [improve][client] Make authorization server metadata path configurable in AuthenticationOAuth2 (apache#25052)

Co-authored-by: hoguni <hoguni@lycorp.co.jp>
(cherry picked from commit 3cb7a7b)
(cherry picked from commit 705a99d)

* Revert "[improve][broker] Add strictAuthMethod to require explicit authentication method (apache#25185)"

This reverts commit 531eb91.

* [improve][broker] Add idle timeout support for http (apache#25224)

(cherry picked from commit 63220ea)
(cherry picked from commit 144e064)

* [fix][broker] Fix incomplete futures in topic property update/delete methods (apache#25228)

(cherry picked from commit c2ae180)
(cherry picked from commit ab05ca2)

* [fix][test] Fix Mockito stubbing race in TopicListServiceTest (apache#25227)

(cherry picked from commit c93dd7a)
(cherry picked from commit 38a126b)

* [improve][broker] Give the detail error msg when authenticate failed with AuthenticationException (apache#25221)

(cherry picked from commit 0a0ce6d)
(cherry picked from commit 2a46c70)

* [fix][client] Send all chunkMessageIds to broker for redelivery (apache#25229)

(cherry picked from commit 0a0ce6d)
(cherry picked from commit f49c7b2)

* [fix][broker] Fix transactionMetadataFuture completeExceptionally with null value (apache#25231)

Co-authored-by: 张浩 <zhanghao60@100.me>
(cherry picked from commit 0e5d424)
(cherry picked from commit 42283f4)

* uncomment distribution management in pom

* Reapply "[improve][meta] PIP-453: Improve the metadata store threading model (apache#25187)"

This reverts commit a6aab86.

(cherry picked from commit 4f9b2ca)

* [improve] Upgrade Netty to 4.1.131.Final (apache#25232)

(cherry picked from commit db91b93)
(cherry picked from commit a6c602a)

* [fix][test] fix testBatchMetadataStoreMetrics. (apache#25241)

(cherry picked from commit 9db31cc)
(cherry picked from commit abbd478)

* [fix][test] Fix ResourceQuotaCalculatorImplTest#testNeedToReportLocalUsage (apache#25247)

(cherry picked from commit 48774de)
(cherry picked from commit 9343837)

* [fix][meta] Metadata cache refresh might not take effect (apache#25246)

(cherry picked from commit 24eba10)
(cherry picked from commit 6d81292)

* fix pulsar-proxy unit test case failure

* fix safe delete URLRegexLookupProxyHandler which is not used

* Revert "fix safe delete URLRegexLookupProxyHandler which is not used"

This reverts commit 158fc14.

* Revert "fix pulsar-proxy unit test case failure"

This reverts commit 4efcf70.

* updated hardcoded newLookupProxyHandler in ProxyService for failing URLRegexLookupProxyHandlerTest

* Revert "[improve][monitor] Upgrade OpenTelemetry to 1.56.0, Otel instrumentation to 2.21.0 and Otel semconv to 1.37.0 (apache#24994)"

This reverts commit 5e5328e

* reverted lincense for opentelemetry upgrade changes

* Revert "updated hardcoded newLookupProxyHandler in ProxyService for failing URLRegexLookupProxyHandlerTest"

This reverts commit a4f07dc.

* reverted mismatch commits changes in ProxyConnection.java

* fix code-style issue

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Kai Wang <kwang@apache.org>
Co-authored-by: Yong Zhang <zhangyong1025.zy@gmail.com>
Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zixuan Liu <nodeces@gmail.com>
Co-authored-by: Wenzhi Feng <thetumbled@apache.org>
Co-authored-by: fengwenzhi <fengwenzhi.max@bigo.sg>
Co-authored-by: Yunze Xu <xyzinfernity@163.com>
Co-authored-by: zhenJiangWang <zhenjiang427@gmail.com>
Co-authored-by: guptas6est <sanaya.gupta@est.tech>
Co-authored-by: Matteo Merli <mmerli@apache.org>
Co-authored-by: Oneby Wang <44369297+oneby-wang@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: fengyubiao <yubiao.feng@streamnative.io>
Co-authored-by: Malla Sandeep <sandeep.malla78@gmail.com>
Co-authored-by: Bäm <dev@sandchaschte.ch>
Co-authored-by: Omar Yasin <omarkj@icloud.com>
Co-authored-by: Ómar K. Yasin <oyasin@apple.com>
Co-authored-by: gulecroc <gu.lecroc@gmail.com>
Co-authored-by: Hideaki Oguni <22386882+izumo27@users.noreply.github.com>
Co-authored-by: hoguni <hoguni@lycorp.co.jp>
Co-authored-by: Cong Zhao <zhaocong@apache.org>
Co-authored-by: sinan liu <liusinan1998@gmail.com>
Co-authored-by: Jiwei Guo <technoboy@apache.org>
Co-authored-by: cai minjian <905767378@qq.com>
Co-authored-by: Hao Zhang <zhanghao1@cmss.chinamobile.com>
Co-authored-by: 张浩 <zhanghao60@100.me>
Co-authored-by: Lari Hotari <lhotari@apache.org>
Co-authored-by: zzb <48124861+zhaizhibo@users.noreply.github.com>
priyanshu-ctds pushed a commit to datastax/pulsar that referenced this pull request Feb 23, 2026
@nodece @priyanshu-ctds
[improve][broker] Add idle timeout support for http (apache#25224)
9e8d57d 
(cherry picked from commit 63220ea)
(cherry picked from commit 144e064)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lhotari lhotari lhotari approved these changes

Assignees

@nodece nodece

Labels

cherry-picked/branch-4.0 cherry-picked/branch-4.1 doc-not-needed Your PR changes do not impact docs release/4.0.9 release/4.1.3

Projects

None yet

Milestone

4.2.0

Development

Successfully merging this pull request may close these issues.

2 participants

@nodece @lhotari