Skip to content

Comments

FIX TypeError: normal_() got an unexpected keyword argument 'generator' #723#791

Open
earlytobed wants to merge 1 commit intokarpathy:masterfrom
earlytobed:fix723
Open

FIX TypeError: normal_() got an unexpected keyword argument 'generator' #723#791
earlytobed wants to merge 1 commit intokarpathy:masterfrom
earlytobed:fix723

Conversation

@earlytobed
Copy link

This PR fixes #723, older PyTorch versions (pre-2.2, roughly, 7fc2929) don’t support arg generator, leading to the error:

Traceback (most recent call last):
    ...
    File "~/llm.c/train_gpt2.py", line 160, in _init_weights
    torch.nn.init.normal_(module.weight, mean=0.0, std=0.02, generator=self.init_rng)
TypeError: normal_() got an unexpected keyword argument 'generator'

after commit 86682af : torch.nn.init.normal_ called with generator, file: train_gpt2.py line146, line150

This PR replaces the torch.nn.init.normal_ with Tensor.normal_, ensuring compatibility with older PyTorch releases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TypeError: normal_() got an unexpected keyword argument 'generator'

1 participant