Fix device mismatch error in Whisper model during feature extraction

[thedebugger]

What does this PR do?

Fixes device mismatch error in Whisper. For instance, when torch default device is set (torch.set_default_device) to cuda, whisper fails with "stft input and window must be on the same device but got self on cpu and window on cuda:0". It is because torch functions (like hann) uses torch_default_device that can lead to device mismatch

[Rocketknight1]

cc @eustlb

[thedebugger]

I verified that this test fails with this addition & without my patch. Is adding this line okay? If not can someone recommend me what is better way to write/update the test case since it depends on cuda?

[eustlb]

To the best of my knowledge, we are not testing set_default_device in Transformers.
I would rather go with:

Suggested change

	torch.set_default_device("cuda")
	with torch.device("cuda"):

followed with indentation:

with torch.device("cuda"):
    input_speech = self._load_datasamples(3)
    feature_extractor = WhisperFeatureExtractor()
    input_features = feature_extractor(input_speech, return_tensors="pt").input_features

[eustlb]

Great catch, thanks! 🤗 Minor change to clean a bit further but otherwise LGTM

[eustlb]

Thanks for iterating! LGTM

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[thedebugger]

Hi @eustlb are you waiting on something to merge changes? It is holding my other PR, so want to get this merged at earliest if possible

[thedebugger]

Hi @eustlb what is the reason for the hold up?

[eustlb]

It got buried in my GitHub notifications, sorry about that!