Test case for larger (prompting-based) language models (+ LLM.int8())

[nfelnlp]

🚀 Feature Request

Following our discussion, it might be valuable to add a test case including larger language models that work with prompting such as T0 and its variants. Since availability of these kinds of models is becoming more common (see "Motivation"), we should show an example of feature attribution for them.

🔈 Motivation

The bitsandbytes integration and 8-bit precision (Dettmers et al., 2022) released in August enables the use of larger models on single GPU setups.

[gsarti]

From preliminary tests gradient-based attributions are broken when INT8 is used - likely to require a dedicated PR.

Edit: Opened an issue with minimal failure case to ask for clarifications in the bitsandbytes repo