[tests] enable bnb tests on xpu #36233
Conversation
ydshieh
left a comment
ydshieh
left a comment
There was a problem hiding this comment.
Are those tests now passing on XPU?
| encoded_input = self.tokenizer(self.input_text, return_tensors="pt") | ||
| output_sequences = self.model_4bit.generate( | ||
| input_ids=encoded_input["input_ids"].to(torch_device), max_new_tokens=10 | ||
| input_ids=encoded_input["input_ids"].to(self.model_4bit.device), max_new_tokens=10 |
There was a problem hiding this comment.
I see several similar changes in this PR? Just wondering if such changes are necessary? (happy to keep them though)
There was a problem hiding this comment.
yes, they are necessary, because torch_device is "xpu", which defaults to "xpu:0", but the model is loaded with "device_map" which is not always "xpu:0". The same error also happens on CUDA. If users only have 1 GPU card, it will be fine, but for machines with multiple cards, this will raise an error.
yes, they are passing on XPU. |
|
Great, I will merge as @matthewdouglas also approved which gives me a lot more confidence. Thank you for taking care XPU with 🤗 ! |
4 participants
What does this PR do?
As the title suggests, this PR enables all bnb-related tests on XPU.
@ydshieh