When CohereAPIError happens, error is not propagated to Jupyter AI, langchain is stuck in the loop

[andrii-i]

Description

When CohereAPIError happens, error is not propagated to Jupyter AI, langchain is stuck in the loop

Reproduce

1. Choose Cohere model, use bad API key
2. Attempt to send a chat message
3. Note that there is no response from Jupyternaut, note terminal error messages and server getting stuck in a loop

Current conversation:

Human: test

AI:
Retrying langchain.llms.cohere.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised CohereAPIError: invalid api token.
Retrying langchain.llms.cohere.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised CohereAPIError: invalid api token.
Retrying langchain.llms.cohere.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised CohereAPIError: invalid api token.

Expected behavior

Usual behavior: error is propagated to Jupyter AI, caught by ChatHandler, Jupyternaut sends error stack trace to user.

Tried implementing a workaround without success:

1. handling exception via try-except cohere.CohereAPIError block to CohereProvider._acall()
2. handling exception via try-except cohere.CohereAPIError in BaseProvider._generate_in_executor() and BaseProvider._call_in_executor()

[JasonWeill]

This is still an issue after recent revisions to the Cohere models in #584 and #594.