Protect get_default_device for torch<2.3

[Cyrilvallez]

What does this PR do?

As per the title! It was reported in #38329!
set_default_device has been around for a long time, but not get_default_device interestingly!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ydshieh]

need push an empty commit. The circleci has some issue that I have to fix.

[peter-crist]

I'm on torch 2.2.2. In order to update transformers 0.42.4 to include this fix, I also have a requirement to update to torch 2.6.0 from this PR addressing the weights_only CVE:
#37785

Seems I can't upgrade transformers unless I upgrade torch (which given 2.6 is a breaking change, it would take more time to validate than a simple upgrade). And even when I do upgrade, I won't actually need this fix. Forgive me if I'm off the mark here.

[ydshieh]

I'm sorry, but if you don't need this fix (i.e. this PR), what's the motivation to update transformers? I must misunderstand something

[Cyrilvallez]

Hey @peter-crist! There are no hard requirements on torch 2.6 unless you try to use torch.load - so the fix is still valid as most model checkpoints are safetensors (which, as the name indicates, are intrinsically safe). You can still use torch 2.2 with safetensors checkpoints thanks to the fix