`handle_event` raises `IndexError` when handler raises `NotImplementedError`

[Alioth99]

Checked other resources

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
• I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Example Code

from langchain_core.callbacks.manager import handle_event
from unittest.mock import Mock

handler = Mock()
handler.on_chat_model_start = Mock(side_effect=NotImplementedError)
handlers = [handler]
event_name = "on_chat_model_start"
ignore_condition_name = None

handle_event(handlers, event_name, ignore_condition_name)

Error Message and Stack Trace (if applicable)

Traceback (most recent call last):
  ...
  File ".../langchain_core/callbacks/manager.py", line 276, in handle_event
    message_strings = [get_buffer_string(m) for m in args[1]]
IndexError: tuple index out of range

Description

The following code works fine and logs: Error in Mock.on_chat_model_start callback: KeyError()

from langchain_core.callbacks.manager import handle_event
from unittest.mock import Mock

handler = Mock()
handler.on_chat_model_start = Mock(side_effect=KeyError)
handlers = [handler]
event_name = "on_chat_model_start"
ignore_condition_name = None

handle_event(handlers, event_name, ignore_condition_name)

However, when I change the handler's raised exception to NotImplementedError, it raises IndexError and NotImplementedError is hidden.

System Info

System Information

OS: Linux
OS Version: #148-Ubuntu SMP Fri Mar 14 19:05:48 UTC 2025
Python Version: 3.10.16 (main, Dec 11 2024, 16:24:50) [GCC 11.2.0]

Package Information

langchain_core: 0.3.28
langchain: 0.3.13
langchain_community: 0.3.13
langsmith: 0.1.137
langchain_anthropic: 0.3.1
langchain_aws: 0.2.2
langchain_chroma: 0.2.0
langchain_experimental: 0.3.2
langchain_fireworks: 0.2.6
langchain_google_vertexai: 2.0.5
langchain_groq: 0.2.2
langchain_mistralai: 0.2.4
langchain_openai: 0.2.14
langchain_text_splitters: 0.3.4
langchain_together: 0.2.0
langchain_unstructured: 0.1.5
langgraph_sdk: 0.1.34

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.10.10
anthropic: 0.40.0
anthropic[vertexai]: Installed. No version info available.
async-timeout: 4.0.3
boto3: 1.35.42
chromadb: 0.5.15
dataclasses-json: 0.6.7
defusedxml: 0.7.1
fastapi: 0.115.3
fireworks-ai: 0.15.7
google-cloud-aiplatform: 1.70.0
google-cloud-storage: 2.18.2
groq: 0.11.0
httpx: 0.28.1
httpx-sse: 0.4.0
jsonpatch: 1.33
numpy: 1.26.4
openai: 1.57.4
orjson: 3.10.16
packaging: 24.1
pydantic: 2.11.1
pydantic-settings: 2.6.0
PyYAML: 6.0.2
requests: 2.32.3
requests-toolbelt: 1.0.0
SQLAlchemy: 2.0.40
tenacity: 9.0.0
tiktoken: 0.8.0
tokenizers: 0.21.0
typing-extensions: 4.13.0
unstructured-client: 0.25.9
unstructured[all-docs]: Installed. No version info available.

[x612skm]

Hi, I'm giving it a try and will be submitting a PR soon today. Thanks!

[x612skm]

This is what I understood from the stackTrace

vemvv/lib/python3.12/site-packages/langchain_core/callbacks/manager.py:272: in handle_event
    event = getattr(handler, event_name)(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/opt/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/mock.py:1137: in __call__
    return self._mock_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/opt/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/mock.py:1141: in _mock_call
    return self._execute_mock_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/opt/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/mock.py:1196: in _execute_mock_call
    raise effect
E   NotImplementedError

During handling of the above exception, another exception occurred:
libs/core/tests/unit_tests/callbacks/test_usage_callback.py:141: in <module>
    handle_event(handlers, event_name, ignore_condition_name)
vemvv/lib/python3.12/site-packages/langchain_core/callbacks/manager.py:278: in handle_event
    message_strings = [get_buffer_string(m) for m in args[1]]
                                                     ^^^^^^^
E   IndexError: tuple index out of range
======================================================================================== short test summary info =========================================================================================
ERROR libs/core/tests/unit_tests/callbacks/test_usage_callback.py - IndexError: tuple index out of range
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!

Also, when we only give an argument inside
handle_event(handlers, event_name, ignore_condition_name, {}) it fails with ValueError

When we give both arguments let's say
handle_event(handlers, event_name, ignore_condition_name, {}, []) it passes. [Maybe this is expected]

I'm not sure I tried to check it with condition if len(args) > 1 and isinstance(args[1], list): before the message_strings but seems to be of no help. Also I don't want to tweak anything inside for the message_strings, but I have a question

• Is it valid for args to have fewer than 2 elements? If so, we might want to skip fallback instead of crashing?

I wrapped the fallback logic with if len(args) > 1: as a guard, but this doesn’t fully clarify what structure of args is expected during a fallback from on_chat_model_start to on_llm_start.
so, could you please confirm what the expected shape of *args is when calling handle_event with on_chat_model_start?

Apologies if I get anything wrong here this is my first issue in this repo and new to open source as well.
Just want to clarify few of those things also any hints/feedbacks towards to solve this problem is highly appreciated. Thanks!

[Alioth99]

@x612skm Thanks for looking into this! Your fix sounds reasonable — mind sending a quick PR? It’d make the fallback logic more robust. If not, maybe a maintainer could take a look?

[x612skm]

@x612skm Thanks for looking into this! Your fix sounds reasonable — mind sending a quick PR? It’d make the fallback logic more robust. If not, maybe a maintainer could take a look?

@Alioth99 - Hey yeah sure! I'll be pushing my PR till the progress that I did today itself for further insights.. TY!

[x612skm]

@Alioth99 this is my approach above - would you mind to take a look and any feedback here would be appreciated this is my first PR for this repo so trying to understand the stuffs. Thanks! 🙇