Feature: AUto retry on "LLM response status: 401 body: unauthorized: token expired"

[snoopier]

Feature request:

github copilot often has error respone 401. then i have to type some like "go on".
that is annoying :-)

maybe we can have a auto retry or something like that ?

thx

Log:

[OPENAI-CHAT] 5046 Unexpected response status: %s body: %s 401 unauthorized: token expired

{:message LLM response status: 401 body: unauthorized: token expired
, :status 401, :body unauthorized: token expired
}
:db/upsert-cache 52ms
:eca/chat-prompt 27ms
{:exception #error {
:cause Received RST_STREAM: Stream cancelled
:via
[{:type java.io.IOException
:message closed
:at [jdk.internal.net.http.ResponseSubscribers$HttpResponseInputStream current ResponseSubscribers.java 464]}
{:type java.io.IOException
:message Received RST_STREAM: Stream cancelled
:at [jdk.internal.net.http.Stream handleReset Stream.java 746]}]
:trace
[[jdk.internal.net.http.Stream handleReset Stream.java 746]
[jdk.internal.net.http.Stream schedule Stream.java 202]
[jdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask run SequentialScheduler.java 205]
[jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask run SequentialScheduler.java 149]
[jdk.internal.net.http.common.SequentialScheduler$SchedulableTask run SequentialScheduler.java 230]
[jdk.internal.net.http.common.SequentialScheduler runOrSchedule SequentialScheduler.java 303]
[jdk.internal.net.http.common.SequentialScheduler runOrSchedule SequentialScheduler.java 256]
[jdk.internal.net.http.Stream receiveResetFrame Stream.java 447]
[jdk.internal.net.http.Stream incoming_reset Stream.java 710]
[jdk.internal.net.http.Stream otherFrame Stream.java 567]
[jdk.internal.net.http.Stream incoming Stream.java 560]
[jdk.internal.net.http.Http2Connection processFrame Http2Connection.java 962]
[jdk.internal.net.http.frame.FramesDecoder decode FramesDecoder.java 155]
[jdk.internal.net.http.Http2Connection$FramesController processReceivedData Http2Connection.java 240]
[jdk.internal.net.http.Http2Connection asyncReceive Http2Connection.java 770]
[jdk.internal.net.http.Http2Connection$Http2TubeSubscriber processQueue Http2Connection.java 1638]
[jdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask run SequentialScheduler.java 205]
[jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask run SequentialScheduler.java 149]
[jdk.internal.net.http.common.SequentialScheduler$SchedulableTask run SequentialScheduler.java 230]
[jdk.internal.net.http.common.SequentialScheduler runOrSchedule SequentialScheduler.java 303]
[jdk.internal.net.http.common.SequentialScheduler runOrSchedule SequentialScheduler.java 256]
[jdk.internal.net.http.Http2Connection$Http2TubeSubscriber runOrSchedule Http2Connection.java 1655]
[jdk.internal.net.http.Http2Connection$Http2TubeSubscriber onNext Http2Connection.java 1682]
[jdk.internal.net.http.Http2Connection$Http2TubeSubscriber onNext Http2Connection.java 1616]
[jdk.internal.net.http.common.SSLTube$DelegateWrapper onNext SSLTube.java 210]
[jdk.internal.net.http.common.SSLTube$SSLSubscriberWrapper onNext SSLTube.java 492]
[jdk.internal.net.http.common.SSLTube$SSLSubscriberWrapper onNext SSLTube.java 295]
[jdk.internal.net.http.common.SubscriberWrapper$DownstreamPusher run1 SubscriberWrapper.java 316]
[jdk.internal.net.http.common.SubscriberWrapper$DownstreamPusher run SubscriberWrapper.java 259]
[jdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask run SequentialScheduler.java 205]
[jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask run SequentialScheduler.java 149]
[jdk.internal.net.http.common.SequentialScheduler$SchedulableTask run SequentialScheduler.java 230]
[jdk.internal.net.http.common.SequentialScheduler runOrSchedule SequentialScheduler.java 303]
[jdk.internal.net.http.common.SequentialScheduler runOrSchedule SequentialScheduler.java 256]
[jdk.internal.net.http.common.SubscriberWrapper outgoing SubscriberWrapper.java 232]
[jdk.internal.net.http.common.SubscriberWrapper outgoing SubscriberWrapper.java 198]
[jdk.internal.net.http.common.SSLFlowDelegate$Reader processData SSLFlowDelegate.java 458]
[jdk.internal.net.http.common.SSLFlowDelegate$Reader$ReaderDownstreamPusher run SSLFlowDelegate.java 282]
[jdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask run SequentialScheduler.java 205]
[jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask run SequentialScheduler.java 149]
[jdk.internal.net.http.common.SequentialScheduler$SchedulableTask run SequentialScheduler.java 230]
[java.util.concurrent.ThreadPoolExecutor runWorker ThreadPoolExecutor.java 1136]
[java.util.concurrent.ThreadPoolExecutor$Worker run ThreadPoolExecutor.java 635]
[java.lang.Thread run Thread.java 840]]}}

[ericdallo]

We do have auto retry for specific status code and body here but it seems you receive a 401 because of expired token and in the next prompt ECA gets a new token, so there is some case where it's not getting a renewing a token, this is the problem but we do in two places already:

• when receiving a user prompt (first request)
• when requesting LLM after running a tool call (subsequent requests)

We need to understand why you are facing this and when

[snoopier]

Get a lot of 401 today and it's absolutely random :-(
with command "go on" it goes on.

So no idea to debug

[ericdallo]

I noticed anthropic and other models are throwing this randomly. I believe we can have a way to to configure in ECA a match for status-code and body to consider as retry

[ericdallo]

Folks, good news, I added support for retryRules config under provider, docs here, this makes possible to you configure whatever status code and/or body with a custom message :)

[snoopier]

Hey Eric,

today i have tested the new retry feature for github-copilot and the retry itself is working fine.
But it can't fix the status 401. Remember if i prompt "go on" it works fine. So my question what is different between
you retry function and a manual prompt ?!

BR Dennis

[ericdallo]

@snoopier why not a retryRule of 401 doesn't work? does ECA recognizes your retryRule and tries to retry and it doesn't work even after 10 retries?

[snoopier]

Yes i see the retry attemps but Never ends i see you increase time

[ericdallo]

@snoopier this 401 may be something that happens only with copilot and maybe on purpose, let me explain why I think it doesn't work with retry but works if you send a prompt.

github-copilot require that we send a header "x-initiator" which can be user or agent, eca and other tools send user only for user prompts and all the rest (tool calls, subagent calls etc) use agent, github charges premium requests only for user ones.

My guess (really a guess, have no evidence and would be nice someone confirm that), is that github is blocking requests when it notices that there was a lot of agent ones after a user and only a user will reset it.
WDYT?

[snoopier]

i'm not an expert but sounds valid for me. so we have to live with manual retry. thats ok i think.
thx for grabing deeper

BR Dennis

[ericdallo]

I'd love @zikajk opion on this, we can also research how copilot or other clients handle this