release 4.52.2: SentenceTransformer import fails with "NameError: name 'Replicate' is not defined"

[pxb5080]

System Info

21 May 2025 11:50:41 | polyaxon-main | from sentence_transformers import SentenceTransformer
21 May 2025 11:50:41 | polyaxon-main | File "/opt/conda/lib/python3.9/site-packages/sentence_transformers/init.py", line 14, in
21 May 2025 11:50:41 | polyaxon-main | from sentence_transformers.cross_encoder import (
21 May 2025 11:50:41 | polyaxon-main | File "/opt/conda/lib/python3.9/site-packages/sentence_transformers/cross_encoder/init.py", line 3, in
21 May 2025 11:50:41 | polyaxon-main | from .CrossEncoder import CrossEncoder
21 May 2025 11:50:41 | polyaxon-main | File "/opt/conda/lib/python3.9/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py", line 19, in
21 May 2025 11:50:41 | polyaxon-main | from transformers import (
21 May 2025 11:50:41 | polyaxon-main | File "", line 1055, in _handle_fromlist
21 May 2025 11:50:41 | polyaxon-main | File "/opt/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 2045, in getattr
21 May 2025 11:50:41 | polyaxon-main | module = self._get_module(self._class_to_module[name])
21 May 2025 11:50:41 | polyaxon-main | File "/opt/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 2075, in _get_module
21 May 2025 11:50:41 | polyaxon-main | raise e
21 May 2025 11:50:41 | polyaxon-main | File "/opt/conda/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 2073, in _get_module
21 May 2025 11:50:41 | polyaxon-main | return importlib.import_module("." + module_name, self.name)
21 May 2025 11:50:41 | polyaxon-main | File "/opt/conda/lib/python3.9/importlib/init.py", line 127, in import_module
21 May 2025 11:50:41 | polyaxon-main | return _bootstrap._gcd_import(name[level:], package, level)
21 May 2025 11:50:41 | polyaxon-main | File "/opt/conda/lib/python3.9/site-packages/transformers/modeling_utils.py", line 64, in
21 May 2025 11:50:41 | polyaxon-main | from .integrations.tensor_parallel import (
21 May 2025 11:50:41 | polyaxon-main | File "/opt/conda/lib/python3.9/site-packages/transformers/integrations/tensor_parallel.py", line 723, in
21 May 2025 11:50:41 | polyaxon-main | class ParallelInterface(MutableMapping):
21 May 2025 11:50:41 | polyaxon-main | File "/opt/conda/lib/python3.9/site-packages/transformers/integrations/tensor_parallel.py", line 733, in ParallelInterface
21 May 2025 11:50:41 | polyaxon-main | "colwise": ColwiseParallel(),
21 May 2025 11:50:41 | polyaxon-main | File "/opt/conda/lib/python3.9/site-packages/transformers/integrations/tensor_parallel.py", line 465, in init
21 May 2025 11:50:41 | polyaxon-main | self.input_layouts = (input_layouts or Replicate(),)
21 May 2025 11:50:41 | polyaxon-main | NameError: name 'Replicate' is not defined

The issue gets fixed when using the last stable release.

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

from sentence_transformers import SentenceTransformer

Expected behavior

No import error.

[d-kleine]

Wrong repo? -> https://github.com/UKPLab/sentence-transformers/

[pxb5080]

Wrong repo? -> https://github.com/UKPLab/sentence-transformers/

I use pip install sentence_transformers. This gets the correct repo.
The reported error started coming from this transformer version. I override the version to "transformers==4.52.1" and the error goes away. Everything else in the setup remains the same.

[Tree-Shu-Zhao]

Same here. I am working on a retriever based on the bert model. I almost only installed the transformers package, and it raised the following error:

File "/root/ret/scripts/retriever_server.py", line 40, in load_model
    model = AutoModel.from_pretrained(model_path, trust_remote_code=True)
  File "/root/anaconda3/envs/retriever/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 568, in from_pretrained
    model_class = _get_model_class(config, cls._model_mapping)
  File "/root/anaconda3/envs/retriever/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 388, in _get_model_class
    supported_models = model_mapping[type(config)]
  File "/root/anaconda3/envs/retriever/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 774, in __getitem__
    return self._load_attr_from_module(model_type, model_name)
  File "/root/anaconda3/envs/retriever/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 788, in _load_attr_from_module
    return getattribute_from_module(self._modules[module_name], attr)
  File "/root/anaconda3/envs/retriever/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 700, in getattribute_from_module
    if hasattr(module, attr):
  File "/root/anaconda3/envs/retriever/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 2045, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/root/anaconda3/envs/retriever/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 2075, in _get_module
    raise e
  File "/root/anaconda3/envs/retriever/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 2073, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/root/anaconda3/envs/retriever/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/root/anaconda3/envs/retriever/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 44, in <module>
    from ...modeling_utils import PreTrainedModel
  File "/root/anaconda3/envs/retriever/lib/python3.10/site-packages/transformers/modeling_utils.py", line 64, in <module>
    from .integrations.tensor_parallel import (
  File "/root/anaconda3/envs/retriever/lib/python3.10/site-packages/transformers/integrations/tensor_parallel.py", line 723, in <module>
    class ParallelInterface(MutableMapping):
  File "/root/anaconda3/envs/retriever/lib/python3.10/site-packages/transformers/integrations/tensor_parallel.py", line 733, in ParallelInterface
    "colwise": ColwiseParallel(),
  File "/root/anaconda3/envs/retriever/lib/python3.10/site-packages/transformers/integrations/tensor_parallel.py", line 465, in __init__
    self.input_layouts = (input_layouts or Replicate(),)
NameError: name 'Replicate' is not defined

The transformers version is 4.45.2, and the code works when I downgrade the version to 4.45.1 per @pxb5080's suggestion above.

[Qiuscraft]

Thanks for @pxb5080 's suggestion.

[harupy]

Ran into this error:

https://github.com/mlflow/mlflow/actions/runs/15177874339/job/42681570156?pr=15826

___________________ test_pyfunc_serve_and_score_transformers ___________________

    @pytest.mark.skipif(not _is_importable("transformers"), reason="This test requires transformers")
    def test_pyfunc_serve_and_score_transformers():
>       from transformers import BertConfig, BertModel


tests/pytorch/test_pytorch_model_export.py:869: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
<frozen importlib._bootstrap>:1075: in _handle_fromlist
    ???
        fromlist   = ('BertConfig', 'BertModel')
        import_    = <built-in function __import__>
        module     = <module 'transformers' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/__init__.py'>
        recursive  = False
        x          = 'BertModel'
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/utils/import_utils.py:2045: in __getattr__
    module = self._get_module(self._class_to_module[name])
        name       = 'BertModel'
        self       = <module 'transformers' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/__init__.py'>
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/utils/import_utils.py:2075: in _get_module
    raise e
        module_name = 'models.bert.modeling_bert'
        self       = <module 'transformers' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/__init__.py'>
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/utils/import_utils.py:2073: in _get_module
    return importlib.import_module("." + module_name, self.__name__)
        module_name = 'models.bert.modeling_bert'
        self       = <module 'transformers' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/__init__.py'>
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
        character  = 'm'
        level      = 1
        name       = '.models.bert.modeling_bert'
        package    = 'transformers'
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py:44: in <module>
    from ...modeling_utils import PreTrainedModel
        ACT2FN     = ClassInstantier([('gelu', <class 'transformers.activations.GELUActivation'>), ('gelu_10', (<class 'transformers.activa...SiLU'>), ('tanh', <class 'torch.nn.modules.activation.Tanh'>), ('prelu', <class 'torch.nn.modules.activation.PReLU'>)])
        BCEWithLogitsLoss = <class 'torch.nn.modules.loss.BCEWithLogitsLoss'>
        BaseModelOutputWithPastAndCrossAttentions = <class 'transformers.modeling_outputs.BaseModelOutputWithPastAndCrossAttentions'>
        BaseModelOutputWithPoolingAndCrossAttentions = <class 'transformers.modeling_outputs.BaseModelOutputWithPoolingAndCrossAttentions'>
        CausalLMOutputWithCrossAttentions = <class 'transformers.modeling_outputs.CausalLMOutputWithCrossAttentions'>
        CrossEntropyLoss = <class 'torch.nn.modules.loss.CrossEntropyLoss'>
        GenerationMixin = <class 'transformers.generation.utils.GenerationMixin'>
        List       = typing.List
        MSELoss    = <class 'torch.nn.modules.loss.MSELoss'>
        MaskedLMOutput = <class 'transformers.modeling_outputs.MaskedLMOutput'>
        MultipleChoiceModelOutput = <class 'transformers.modeling_outputs.MultipleChoiceModelOutput'>
        NextSentencePredictorOutput = <class 'transformers.modeling_outputs.NextSentencePredictorOutput'>
        Optional   = typing.Optional
        QuestionAnsweringModelOutput = <class 'transformers.modeling_outputs.QuestionAnsweringModelOutput'>
        SequenceClassifierOutput = <class 'transformers.modeling_outputs.SequenceClassifierOutput'>
        TokenClassifierOutput = <class 'transformers.modeling_outputs.TokenClassifierOutput'>
        Tuple      = typing.Tuple
        Union      = typing.Union
        __builtins__ = <builtins>
        __cached__ = '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/models/bert/__pycache__/modeling_bert.cpython-310.pyc'
        __doc__    = 'PyTorch BERT model.'
        __file__   = '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7fc8ed3db070>
        __name__   = 'transformers.models.bert.modeling_bert'
        __package__ = 'transformers.models.bert'
        __spec__   = ModuleSpec(name='transformers.models.bert.modeling_bert', loader=<_frozen_importlib_external.SourceFileLoader object a...rigin='/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py')
        _prepare_4d_attention_mask_for_sdpa = <function _prepare_4d_attention_mask_for_sdpa at 0x7fc8dc56b010>
        _prepare_4d_causal_attention_mask_for_sdpa = <function _prepare_4d_causal_attention_mask_for_sdpa at 0x7fc8dc56ae60>
        dataclass  = <function dataclass at 0x7fc93c0395a0>
        math       = <module 'math' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/lib-dynload/math.cpython-310-x86_64-linux-gnu.so'>
        nn         = <module 'torch.nn' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/nn/__init__.py'>
        os         = <module 'os' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/os.py'>
        torch      = <module 'torch' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/__init__.py'>
        version    = <module 'packaging.version' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/packaging/version.py'>
        warnings   = <module 'warnings' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/warnings.py'>
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/modeling_utils.py:64: in <module>
    from .integrations.tensor_parallel import (
        Any        = typing.Any
        Callable   = typing.Callable
        CompileConfig = <class 'transformers.generation.configuration_utils.CompileConfig'>
        CrossEntropyLoss = <class 'torch.nn.modules.loss.CrossEntropyLoss'>
        Dict       = typing.Dict
        Enum       = <enum 'Enum'>
        GenerationConfig = <class 'transformers.generation.configuration_utils.GenerationConfig'>
        Identity   = <class 'torch.nn.modules.linear.Identity'>
        List       = typing.List
        MutableMapping = <class 'collections.abc.MutableMapping'>
        Optional   = typing.Optional
        PeftAdapterMixin = <class 'transformers.integrations.peft.PeftAdapterMixin'>
        PretrainedConfig = <class 'transformers.configuration_utils.PretrainedConfig'>
        Set        = typing.Set
        Tensor     = <class 'torch.Tensor'>
        Thread     = <class 'threading.Thread'>
        Tuple      = typing.Tuple
        Type       = typing.Type
        TypeVar    = <class 'typing.TypeVar'>
        Union      = typing.Union
        __annotations__ = {}
        __builtins__ = <builtins>
        __cached__ = '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/__pycache__/modeling_utils.cpython-310.pyc'
        __doc__    = None
        __file__   = '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/modeling_utils.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7fc8dc556ad0>
        __name__   = 'transformers.modeling_utils'
        __package__ = 'transformers'
        __spec__   = ModuleSpec(name='transformers.modeling_utils', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7fc8dc556ad0>, origin='/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/modeling_utils.py')
        _load_state_dict_into_zero3_model = <function _load_state_dict_into_zero3_model at 0x7fc8dc69bf40>
        checkpoint = <function checkpoint at 0x7fc8ef895[120](https://github.com/mlflow/mlflow/actions/runs/15177874339/job/42681570156?pr=15826#step:12:121)>
        collections = <module 'collections' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/collections/__init__.py'>
        constraints = <module 'torch.distributions.constraints' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/distributions/constraints.py'>
        contextmanager = <function contextmanager at 0x7fc93c164940>
        copy       = <module 'copy' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/copy.py'>
        custom_object_save = <function custom_object_save at 0x7fc8ed45f5b0>
        dataclass  = <function dataclass at 0x7fc93c0395a0>
        deepspeed_config = <function deepspeed_config at 0x7fc8dc69beb0>
        defaultdict = <class 'collections.defaultdict'>
        find_tied_parameters = <function find_tied_parameters at 0x7fc8dc58be20>
        flash_attention_forward = <function flash_attention_forward at 0x7fc8dc58bf40>
        flex_attention_forward = <function flex_attention_forward at 0x7fc8dc699120>
        functools  = <module 'functools' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/functools.py'>
        gc         = <module 'gc' (built-in)>
        get_activation = <function get_activation at 0x7fc8dc61d120>
        importlib  = <module 'importlib' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/importlib/__init__.py'>
        init_empty_weights = <function init_empty_weights at 0x7fc8dc58bc70>
        inspect    = <module 'inspect' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/inspect.py'>
        is_deepspeed_zero3_enabled = <function is_deepspeed_zero3_enabled at 0x7fc8dc69be20>
        is_torchao_available = <function is_torchao_available at 0x7fc8f16a5240>
        is_zipfile = <function is_zipfile at 0x7fc93bb231c0>
        itertools  = <module 'itertools' (built-in)>
        json       = <module 'json' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/json/__init__.py'>
        nn         = <module 'torch.nn' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/nn/__init__.py'>
        os         = <module 'os' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/os.py'>
        partial    = <class 'functools.partial'>
        re         = <module 're' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/re.py'>
        sdpa_attention_forward = <function sdpa_attention_forward at 0x7fc8dc6991b0>
        shutil     = <module 'shutil' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/shutil.py'>
        split_torch_state_dict_into_shards = <function split_torch_state_dict_into_shards at 0x7fc8dc58a170>
        tempfile   = <module 'tempfile' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/tempfile.py'>
        torch      = <module 'torch' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/__init__.py'>
        version    = <module 'packaging.version' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/packaging/version.py'>
        warnings   = <module 'warnings' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/warnings.py'>
        wraps      = <function wraps at 0x7fc93c5e5630>
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/integrations/tensor_parallel.py:723: in <module>
    class ParallelInterface(MutableMapping):
        ALL_LAYERNORM_LAYERS = [<class 'torch.nn.modules.normalization.LayerNorm'>]
        Callable   = typing.Callable
        ColwiseParallel = <class 'transformers.integrations.tensor_parallel.ColwiseParallel'>
        GatherParallel = <class 'transformers.integrations.tensor_parallel.GatherParallel'>
        IsolatedParallel = <class 'transformers.integrations.tensor_parallel.IsolatedParallel'>
        List       = typing.List
        MutableMapping = <class 'collections.abc.MutableMapping'>
        Optional   = typing.Optional
        PackedColwiseParallel = <class 'transformers.integrations.tensor_parallel.PackedColwiseParallel'>
        PackedRowwiseParallel = <class 'transformers.integrations.tensor_parallel.PackedRowwiseParallel'>
        ReplicateParallel = <class 'transformers.integrations.tensor_parallel.ReplicateParallel'>
        RowwiseParallel = <class 'transformers.integrations.tensor_parallel.RowwiseParallel'>
        SequenceParallel = <class 'transformers.integrations.tensor_parallel.SequenceParallel'>
        TensorParallelLayer = <class 'transformers.integrations.tensor_parallel.TensorParallelLayer'>
        Tuple      = typing.Tuple
        Union      = typing.Union
        __annotations__ = {}
        __builtins__ = <builtins>
        __cached__ = '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/integrations/__pycache__/tensor_parallel.cpython-310.pyc'
        __doc__    = None
        __file__   = '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/integrations/tensor_parallel.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7fc8dc694f70>
        __name__   = 'transformers.integrations.tensor_parallel'
        __package__ = 'transformers.integrations'
        __spec__   = ModuleSpec(name='transformers.integrations.tensor_parallel', loader=<_frozen_importlib_external.SourceFileLoader objec...in='/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/integrations/tensor_parallel.py')
        _blocks_to_block_sizes = <function _blocks_to_block_sizes at 0x7fc8dc5b4af0>
        _get_parameter_tp_plan = <function _get_parameter_tp_plan at 0x7fc8dc5b4b80>
        _torch_distributed_available = True
        annotations = _Feature((3, 7, 0, 'beta', 1), (3, 11, 0, 'alpha', 0), [167](https://github.com/mlflow/mlflow/actions/runs/15177874339/job/42681570156?pr=15826#step:12:168)77216)
        dist       = <module 'torch.distributed' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/distributed/__init__.py'>
        distribute_module = <function distribute_module at 0x7fc8dc5b4dc0>
        get_packed_weights = <function get_packed_weights at 0x7fc8dc5b4c10>
        get_tensor_shard = <function get_tensor_shard at 0x7fc8dc5b4d30>
        initialize_tensor_parallelism = <function initialize_tensor_parallelism at 0x7fc8dc5b4a60>
        is_torch_greater_or_equal = <functools._lru_cache_wrapper object at 0x7fc8f16ac930>
        logger     = <Logger transformers.integrations.tensor_parallel (WARNING)>
        logging    = <module 'transformers.utils.logging' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/utils/logging.py'>
        nn         = <module 'torch.nn' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/nn/__init__.py'>
        operator   = <module 'operator' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/operator.py'>
        os         = <module 'os' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/os.py'>
        partial    = <class 'functools.partial'>
        re         = <module 're' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/re.py'>
        reduce     = <built-in function reduce>
        repack_weights = <function repack_weights at 0x7fc8dc5b4ca0>
        str_to_torch_dtype = {'BF16': torch.bfloat16, 'BOOL': torch.bool, 'F16': torch.float16, 'F32': torch.float32, ...}
        torch      = <module 'torch' from '/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/__init__.py'>
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/integrations/tensor_parallel.py:733: in ParallelInterface
    "colwise": ColwiseParallel(),
        __doc__    = '\n    Dict-like object keeping track of allowed attention functions. You can easily add a new attention function\n   ...needs to declare a new instance of this class inside the `modeling_<model>.py`, and declare it on that instance.\n    '
        __module__ = 'transformers.integrations.tensor_parallel'
        __qualname__ = 'ParallelInterface'
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <transformers.integrations.tensor_parallel.ColwiseParallel object at 0x7fc8dc5bcbe0>

    def __init__(
        self,
        *,
        input_layouts: Optional[Placement] = None,
        output_layouts: Optional[Placement] = None,
        use_local_output: bool = True,
        use_dtensor=True,
    ):
        super().__init__()
>       self.input_layouts = (input_layouts or Replicate(),)
E       NameError: name 'Replicate' is not defined

__class__  = <class 'transformers.integrations.tensor_parallel.ColwiseParallel'>
input_layouts = None
output_layouts = None
self       = <transformers.integrations.tensor_parallel.ColwiseParallel object at 0x7fc8dc5bcbe0>
use_dtensor = True
use_local_output = True

/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/transformers/integrations/tensor_parallel.py:465: NameError

[mudler]

same here - this is currently hitting all LocalAI's (https://github.com/mudler/LocalAI) PRs as well as we have a transformers backend since today.

[aristides86]

Installing Dreambooth extention into Web UI causes this error as well. For me it looks like it's the Dreambooth problem, because I have v 4.52.1 and I tried with earlier versions as well. Deleting Dreambooth folder from extensions is the only solution.

venv "c:\AI\SD-Web-UI\webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.10.1
Commit hash: 82a973c04367123ae98bd9abdf80d9eda9b910e2
Installing requirements
If submitting an issue on github, please provide the full startup log for debugging purposes.

Initializing Dreambooth
Dreambooth revision: bae1e87c9b0fe3bb0d261d6851ceffab6f99dfb6
Checking xformers...
Checking bitsandbytes...
Checking bitsandbytes (ALL!)
Installing bitsandbytes
Checking Dreambooth requirements...
[Installed version of bitsandbytes: 0.45.5
[Dreambooth] bitsandbytes v0.45.2 is already installed.
[Installed version of accelerate: 0.21.0
[Dreambooth] accelerate v0.21.0 is already installed.
[Installed version of dadaptation: 3.2
[Dreambooth] dadaptation v3.2 is already installed.
[Installed version of diffusers: 0.33.1
[Dreambooth] diffusers v0.32.2 is already installed.
[Installed version of discord-webhook: 1.3.1
[Dreambooth] discord-webhook v1.3.1 is already installed.
[Installed version of fastapi: 0.94.0
[Dreambooth] fastapi is already installed.
[Installed version of gitpython: 3.1.32
[Dreambooth] gitpython v3.1.32 is already installed.
[Installed version of pytorch_optimizer: 3.4.0
[Dreambooth] pytorch_optimizer v3.4.0 is already installed.
[Installed version of Pillow: 9.5.0
[Dreambooth] Pillow is already installed.
[Installed version of transformers: 4.30.2
[Dreambooth] transformers v4.48.3 is not installed.
Successfully installed safetensors-0.5.3 tokenizers-0.21.1 transformers-4.52.2

[Installed version of tqdm: 4.67.1
[Dreambooth] tqdm is already installed.
[Installed version of tomesd: 0.1.3
[Dreambooth] tomesd v0.1.3 is already installed.
[Installed version of xformers: 0.0.23.post1
[Dreambooth] xformers is already installed.
[Installed version of tensorboard: 2.19.0
[Dreambooth] tensorboard v2.18.0 is already installed.

Does your project take forever to startup?
Repetitive dependency installation may be the reason.
Automatic1111's base project sets strict requirements on outdated dependencies.
If an extension is using a newer version, the dependency is uninstalled and reinstalled twice every startup.

[+] accelerate version 0.21.0 installed.
[+] diffusers version 0.33.1 installed.
[+] bitsandbytes version 0.45.5 installed.
[+] xformers version 0.0.23.post1 installed.
Launching Web UI with arguments: --xformers
c:\AI\SD-Web-UI\webui\venv\lib\site-packages\timm\models\layers\__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
  warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
Traceback (most recent call last):
  File "c:\AI\SD-Web-UI\webui\launch.py", line 48, in <module>
    main()
  File "c:\AI\SD-Web-UI\webui\launch.py", line 44, in main
    start()
  File "c:\AI\SD-Web-UI\webui\modules\launch_utils.py", line 465, in start
    import webui
  File "c:\AI\SD-Web-UI\webui\webui.py", line 13, in <module>
    initialize.imports()
  File "c:\AI\SD-Web-UI\webui\modules\initialize.py", line 26, in imports
    from modules import paths, timer, import_hook, errors  # noqa: F401
  File "c:\AI\SD-Web-UI\webui\modules\paths.py", line 60, in <module>
    import sgm  # noqa: F401
  File "C:\AI\SD-Web-UI\webui\repositories\generative-models\sgm\__init__.py", line 1, in <module>
    from .models import AutoencodingEngine, DiffusionEngine
  File "C:\AI\SD-Web-UI\webui\repositories\generative-models\sgm\models\__init__.py", line 1, in <module>
    from .autoencoder import AutoencodingEngine
  File "C:\AI\SD-Web-UI\webui\repositories\generative-models\sgm\models\autoencoder.py", line 12, in <module>
    from ..modules.diffusionmodules.model import Decoder, Encoder
  File "C:\AI\SD-Web-UI\webui\repositories\generative-models\sgm\modules\__init__.py", line 1, in <module>
    from .encoders.modules import GeneralConditioner
  File "C:\AI\SD-Web-UI\webui\repositories\generative-models\sgm\modules\encoders\modules.py", line 13, in <module>
    from transformers import (
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "c:\AI\SD-Web-UI\webui\venv\lib\site-packages\transformers\utils\import_utils.py", line 2045, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "c:\AI\SD-Web-UI\webui\venv\lib\site-packages\transformers\utils\import_utils.py", line 2075, in _get_module
    raise e
  File "c:\AI\SD-Web-UI\webui\venv\lib\site-packages\transformers\utils\import_utils.py", line 2073, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "c:\AI\SD-Web-UI\webui\venv\lib\site-packages\transformers\models\clip\modeling_clip.py", line 27, in <module>
    from ...modeling_utils import ALL_ATTENTION_FUNCTIONS, PreTrainedModel
  File "c:\AI\SD-Web-UI\webui\venv\lib\site-packages\transformers\modeling_utils.py", line 64, in <module>
    from .integrations.tensor_parallel import (
  File "c:\AI\SD-Web-UI\webui\venv\lib\site-packages\transformers\integrations\tensor_parallel.py", line 723, in <module>
    class ParallelInterface(MutableMapping):
  File "c:\AI\SD-Web-UI\webui\venv\lib\site-packages\transformers\integrations\tensor_parallel.py", line 733, in ParallelInterface
    "colwise": ColwiseParallel(),
  File "c:\AI\SD-Web-UI\webui\venv\lib\site-packages\transformers\integrations\tensor_parallel.py", line 465, in __init__
    self.input_layouts = (input_layouts or Replicate(),)
NameError: name 'Replicate' is not defined
Press any key to continue . . .

[SunMarc]

Sorry for the inconvenience, we will release a patch shortly to fix this ! cc @ArthurZucker

EDIT: Should be fixed in this patch https://github.com/huggingface/transformers/releases/tag/v4.52.3