Skip to content

[Bugfix] Use sys.modules.get() to avoid KeyError in modeling_utils#45043

Closed
Lidang-Jiang wants to merge 1 commit intohuggingface:mainfrom
Lidang-Jiang:fix/sys-modules-keyerror-45003
Closed

[Bugfix] Use sys.modules.get() to avoid KeyError in modeling_utils#45043
Lidang-Jiang wants to merge 1 commit intohuggingface:mainfrom
Lidang-Jiang:fix/sys-modules-keyerror-45003

Conversation

@Lidang-Jiang
Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes #45003. _can_set_attn_implementation and _can_set_experts_implementation in modeling_utils.py crash with KeyError when cls.__module__ is absent from sys.modules.

This happens in real-world scenarios where frameworks manipulate sys.modules (e.g., Griptape Nodes for Flux.1 ControlNets, vLLM CI with renamed image processor modules).

Fix

Replace sys.modules[cls.__module__] with sys.modules.get(cls.__module__) and add a None check to the existing guard. Returns False (safe default = "don't try to dynamically set implementation") instead of crashing. This matches the pattern already used in dynamic_module_utils.py:293 and import_utils.py:2373.

4 lines changed in production code, 2 regression tests added.

Coordination

Before / After

Before fix (KeyError on all three calls)
=== BEFORE FIX (using original code) ===

Python version: 3.12.12 (main, Jan 14 2026, 19:35:58) [Clang 21.1.4 ]

Loaded BertModel from: transformers.models.bert.modeling_bert
Module in sys.modules: True

After removing module:
  Module in sys.modules: False

Calling _can_set_attn_implementation()...
  ERROR: KeyError: 'transformers.models.bert.modeling_bert'

Calling _can_set_experts_implementation()...
  ERROR: KeyError: 'transformers.models.bert.modeling_bert'

Calling BertModel(BertConfig()) (full model init)...
  ERROR: KeyError: 'transformers.models.bert.modeling_bert'
After fix (no crash, returns False, model init succeeds)
=== AFTER FIX (using patched code) ===

Python version: 3.12.12 (main, Jan 14 2026, 19:35:58) [Clang 21.1.4 ]

Loaded BertModel from: transformers.models.bert.modeling_bert
Module in sys.modules: True

After removing module:
  Module in sys.modules: False

Calling _can_set_attn_implementation()...
  Result: False

Calling _can_set_experts_implementation()...
  Result: False

Calling BertModel(BertConfig()) (full model init)...
  Success! Model created: BertModel
  Parameters: 109482240

All operations completed without KeyError.
Regression tests
============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-8.4.2, pluggy-1.6.0 -- /ssd1/jianglidang/workspace/myenv-vllm/bin/python
cachedir: .pytest_cache
rootdir: /ssd1/jianglidang/workspace/transformers-fix-45003
configfile: pyproject.toml
plugins: rerunfailures-15.1, order-1.3.0, anyio-4.13.0, xdist-3.8.0, rich-0.2.0, hypothesis-6.151.9, asyncio-1.3.0, random-order-1.2.0, timeout-2.4.0, env-1.2.0
collecting ... collected 2 items

tests/utils/test_modeling_utils.py::TestSysModulesMissing::test_can_set_attn_impl_missing_module PASSED [ 50%]
tests/utils/test_modeling_utils.py::TestSysModulesMissing::test_can_set_experts_impl_missing_module PASSED [100%]

============================== 2 passed in 10.10s ==============================

AI disclosure

This PR was developed with Claude Code assistance. I have reviewed every changed line and can defend the approach end-to-end.

…erts_implementation (huggingface#45003)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Rocketknight1
Copy link
Copy Markdown
Member

No, sorry! The fix we need was still being discussed in the issue, so it is not helpful to open a full agent PR at this stage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

modeling_utils unsafely accesses sys.modules[]

2 participants