Skip to content

fix(engine): classify EL HTTP 4xx as fatal via typed HTTPStatusError#3109

Merged
calbera merged 1 commit into
mainfrom
fix-engine-4xx-retry-loop
May 21, 2026
Merged

fix(engine): classify EL HTTP 4xx as fatal via typed HTTPStatusError#3109
calbera merged 1 commit into
mainfrom
fix-engine-4xx-retry-loop

Conversation

@fridrik01

@fridrik01 fridrik01 commented May 20, 2026

Copy link
Copy Markdown
Contributor

This PR Introduces a typed rpc.HTTPStatusError so the EL's HTTP status code propagates through to handleRPCError. Previously, callRaw collapsed every non-200 response into fmt.Errorf("unexpected status code %d: ..."), which didn't satisfy the jsonrpc.Error interface and so fell into the catch-all ErrBadConnection (retryable) bucket.

With the typed error in place, handleRPCError now classifies HTTP 4xx as fatal (ErrHTTPClientError) and leaves 5xx and other transport failures retryable (ErrBadConnection), so the engine no longer retries forever on rejections the EL won't accept on a second try.

Will likely made a follow-up PR which will split the engine retry policy between ProcessProposal (bounded retry where a failing proposal should be rejected so consensus can move on) and FinalizeBlock (unbounded retry where the block is already agreed, must be applied).

@fridrik01
fridrik01 requested review from bar-bera and calbera May 20, 2026 14:10
@fridrik01
fridrik01 force-pushed the fix-engine-4xx-retry-loop branch from 940581c to b8b4c63 Compare May 20, 2026 14:20
@fridrik01
fridrik01 marked this pull request as ready for review May 20, 2026 14:24
@fridrik01
fridrik01 requested a review from a team as a code owner May 20, 2026 14:24
Copilot AI review requested due to automatic review settings May 20, 2026 14:24
@codecov

codecov Bot commented May 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.34%. Comparing base (ffe5e16) to head (fd10658).

Files with missing lines Patch % Lines
execution/client/client.go 0.00% 2 Missing ⚠️
execution/client/ethclient/rpc/client.go 60.00% 1 Missing and 1 partial ⚠️
execution/client/ethclient/rpc/errors.go 0.00% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3109      +/-   ##
==========================================
+ Coverage   61.21%   61.34%   +0.13%     
==========================================
  Files         369      370       +1     
  Lines       18925    18936      +11     
==========================================
+ Hits        11584    11617      +33     
+ Misses       6382     6363      -19     
+ Partials      959      956       -3     
Files with missing lines Coverage Δ
execution/client/errors.go 53.01% <100.00%> (+26.98%) ⬆️
primitives/net/http/errors.go 100.00% <ø> (ø)
execution/client/client.go 54.47% <0.00%> (ø)
execution/client/ethclient/rpc/client.go 68.36% <60.00%> (+1.70%) ⬆️
execution/client/ethclient/rpc/errors.go 0.00% <0.00%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves execution-layer (EL) RPC error propagation by introducing a typed rpc.HTTPStatusError so handleRPCError can distinguish HTTP status failures and classify EL HTTP 4xx responses as fatal (non-retryable), while keeping 5xx/transport failures retryable—preventing infinite retries on request rejections.

Changes:

  • Add rpc.HTTPStatusError to preserve EL HTTP status codes (and body) from callRaw.
  • Update EngineClient.handleRPCError to classify HTTP 4xx as fatal via ErrHTTPClientError, while preserving unauthorized handling.
  • Add unit tests covering the new classification behavior (including HTTP timeout detection).

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
execution/client/ethclient/rpc/errors.go Introduces HTTPStatusError to carry HTTP status/body for upstream classification.
execution/client/ethclient/rpc/client.go Returns HTTPStatusError on non-200 responses instead of collapsing into a generic error / unauthorized sentinel.
execution/client/errors.go Adds ErrHTTPClientError and updates handleRPCError + IsFatalError to treat HTTP 4xx as fatal.
execution/client/errors_test.go Adds tests validating classification across HTTP status, JSON-RPC codes, and timeouts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread execution/client/ethclient/rpc/errors.go Outdated
Comment thread execution/client/ethclient/rpc/client.go
Comment thread execution/client/errors_test.go Outdated
@fridrik01
fridrik01 force-pushed the fix-engine-4xx-retry-loop branch 3 times, most recently from daa79f1 to abbd8d4 Compare May 20, 2026 17:16
Comment thread execution/client/errors.go Outdated
Comment thread execution/client/client.go
@fridrik01
fridrik01 force-pushed the fix-engine-4xx-retry-loop branch from abbd8d4 to fd10658 Compare May 20, 2026 18:18

@bar-bera bar-bera left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

@calbera
calbera merged commit 1a84691 into main May 21, 2026
22 checks passed
@calbera
calbera deleted the fix-engine-4xx-retry-loop branch May 21, 2026 22:46
@fridrik01 fridrik01 self-assigned this May 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants