enable test_layerwise_casting_memory cases on XPU

[yao-matrix]

all 46 test_layerwise_casting_memory cases pased on XPU

[yao-matrix]

@a-r-r-o-w @DN6 , @yiyixuxu pls help review, thx.

[a-r-r-o-w]

Thank you, the changes LGTM! With the current changes, it implies that it is expected to pass on any torch accelerator. Can you add a note to the class saying all tests are intended to pass on CUDA, but may fail on other devices?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[yao-matrix]

Thank you, the changes LGTM! With the current changes, it implies that it is expected to pass on any torch accelerator. Can you add a note to the class saying all tests are intended to pass on CUDA, but may fail on other devices?

Glad to do so. Yet, could you help point me the place where you think is appropriate to add this note? Thx, I'm kind of not certain the right place to do this.