Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable BNB multi-backend support #31098

Merged
merged 66 commits into from
Sep 24, 2024
Merged

Enable BNB multi-backend support #31098

merged 66 commits into from
Sep 24, 2024

Commits on Aug 27, 2024

  1. enable cpu bnb path

    @jiqing-feng @Titus-von-Koeller
    jiqing-feng authored and Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    846f853 View commit details
    Browse the repository at this point in the history

  2. fix style

    @jiqing-feng @Titus-von-Koeller
    jiqing-feng authored and Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    6c56703 View commit details
    Browse the repository at this point in the history

  3. fix code style

    @jiqing-feng @Titus-von-Koeller
    jiqing-feng authored and Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    3f02c9b View commit details
    Browse the repository at this point in the history

  4. fix 4 bit path

    @jiqing-feng @Titus-von-Koeller
    jiqing-feng authored and Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    9ccbf10 View commit details
    Browse the repository at this point in the history

  5. Update src/transformers/utils/import_utils.py 

    Co-authored-by: Aarni Koskela <akx@iki.fi>
    @jiqing-feng @akx @Titus-von-Koeller
    2 people authored and Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    89fa5ef View commit details
    Browse the repository at this point in the history

  6. add multi backend refactor tests

    @jiqing-feng @Titus-von-Koeller
    jiqing-feng authored and Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    a52d7af View commit details
    Browse the repository at this point in the history

  7. fix style

    @jiqing-feng @Titus-von-Koeller
    jiqing-feng authored and Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    6f67862 View commit details
    Browse the repository at this point in the history

  8. tweak 4bit quantizer + fix corresponding tests

    @Titus-von-Koeller
    Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    ee23eb0 View commit details
    Browse the repository at this point in the history

  9. tweak 8bit quantizer + *try* fixing corresponding tests

    @Titus-von-Koeller
    Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    678e673 View commit details
    Browse the repository at this point in the history

  10. fix dequant bnb 8bit

    @jiqing-feng @Titus-von-Koeller
    jiqing-feng authored and Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    0858b3e View commit details
    Browse the repository at this point in the history

  11. account for Intel CPU in variability of expected outputs

    @Titus-von-Koeller
    Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    c76d243 View commit details
    Browse the repository at this point in the history

  12. enable cpu and xpu device map

    @jiqing-feng @Titus-von-Koeller
    jiqing-feng authored and Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    5843f28 View commit details
    Browse the repository at this point in the history

  13. further tweaks to account for Intel CPU

    @Titus-von-Koeller
    Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    1a864a8 View commit details
    Browse the repository at this point in the history

  14. fix autocast to work with both cpu + cuda

    @Titus-von-Koeller
    Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    f3753fc View commit details
    Browse the repository at this point in the history

  15. fix comments

    @Titus-von-Koeller
    Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    0cc1b7e View commit details
    Browse the repository at this point in the history

  16. fix comments

    @Titus-von-Koeller
    Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    b611812 View commit details
    Browse the repository at this point in the history

  17. switch to testing_utils.torch_device

    @Titus-von-Koeller
    Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    ab4836e View commit details
    Browse the repository at this point in the history

  18. allow for xpu in multi-gpu tests

    @Titus-von-Koeller
    Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    7399500 View commit details
    Browse the repository at this point in the history

  19. fix tests 4bit for CPU NF4

    @jiqing-feng @Titus-von-Koeller
    jiqing-feng authored and Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    b41059c View commit details
    Browse the repository at this point in the history

  20. fix bug with is_torch_xpu_available needing to be called as func

    @Titus-von-Koeller
    Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    1a7a6fe View commit details
    Browse the repository at this point in the history

  21. avoid issue where test reports attr err due to other failure

    @Titus-von-Koeller
    Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    87983df View commit details
    Browse the repository at this point in the history

  22. fix formatting

    @Titus-von-Koeller
    Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    7f17188 View commit details
    Browse the repository at this point in the history

  23. fix typo from resolving of merge conflict

    @Titus-von-Koeller
    Titus-von-Koeller committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    bb3ba4a View commit details
    Browse the repository at this point in the history

  24. polish based on last PR review 

    Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
    @Titus-von-Koeller @SunMarc
    Titus-von-Koeller and SunMarc committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    463c211 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. fix CI

    @jiqing-feng
    jiqing-feng committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    6d89ee4 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. Update src/transformers/integrations/integration_utils.py 

    Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
    @jiqing-feng @ArthurZucker
    jiqing-feng and ArthurZucker authored Aug 29, 2024
    Configuration menu
    Copy the full SHA
    7e01cfb View commit details
    Browse the repository at this point in the history

  2. Update src/transformers/integrations/integration_utils.py 

    Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
    @jiqing-feng @ArthurZucker
    jiqing-feng and ArthurZucker authored Aug 29, 2024
    Configuration menu
    Copy the full SHA
    9bffc93 View commit details
    Browse the repository at this point in the history

  3. fix error log

    @jiqing-feng
    jiqing-feng committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    01b7587 View commit details
    Browse the repository at this point in the history

  4. fix error msg

    @jiqing-feng
    jiqing-feng committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    171b130 View commit details
    Browse the repository at this point in the history

  5. add \n in error log

    @jiqing-feng
    jiqing-feng committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    5e9bf9a View commit details
    Browse the repository at this point in the history

  6. make quality

    @jiqing-feng
    jiqing-feng committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    496c046 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. rm bnb cuda restriction in doc

    @jiqing-feng
    jiqing-feng committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    86d0016 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. cpu model don't need dispatch

    @jiqing-feng
    jiqing-feng committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    1c96ae9 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into bnb_cpu

    @jiqing-feng
    jiqing-feng authored Sep 3, 2024
    Configuration menu
    Copy the full SHA
    495354e View commit details
    Browse the repository at this point in the history

  3. fix doc

    @jiqing-feng
    jiqing-feng committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    3aec626 View commit details
    Browse the repository at this point in the history

  4. fix style

    @jiqing-feng
    jiqing-feng committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    daa1e27 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2024

  1. check cuda avaliable in testing

    @jiqing-feng
    jiqing-feng committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    d55db0e View commit details
    Browse the repository at this point in the history

  2. fix tests

    @jiqing-feng
    jiqing-feng committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    a21a916 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. Update docs/source/en/model_doc/chameleon.md 

    Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
    @jiqing-feng @SunMarc
    jiqing-feng and SunMarc authored Sep 11, 2024
    Configuration menu
    Copy the full SHA
    8ad17e8 View commit details
    Browse the repository at this point in the history

  2. Update docs/source/en/model_doc/llava_next.md 

    Co-authored-by: Aarni Koskela <akx@iki.fi>
    @jiqing-feng @akx
    jiqing-feng and akx authored Sep 11, 2024
    Configuration menu
    Copy the full SHA
    107e02b View commit details
    Browse the repository at this point in the history

  3. Update tests/quantization/bnb/test_4bit.py 

    Co-authored-by: Aarni Koskela <akx@iki.fi>
    @jiqing-feng @akx
    jiqing-feng and akx authored Sep 11, 2024
    Configuration menu
    Copy the full SHA
    20f6b5e View commit details
    Browse the repository at this point in the history

  4. Update tests/quantization/bnb/test_4bit.py 

    Co-authored-by: Aarni Koskela <akx@iki.fi>
    @jiqing-feng @akx
    jiqing-feng and akx authored Sep 11, 2024
    Configuration menu
    Copy the full SHA
    9ac038e View commit details
    Browse the repository at this point in the history

  5. fix doc

    @jiqing-feng
    jiqing-feng committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    3bab7d7 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'huggingface:main' into bnb_cpu

    @jiqing-feng
    jiqing-feng authored Sep 11, 2024
    Configuration menu
    Copy the full SHA
    968d9c5 View commit details
    Browse the repository at this point in the history

  7. fix check multibackends

    @jiqing-feng
    jiqing-feng committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    08f31f8 View commit details
    Browse the repository at this point in the history

  8. fix import sort

    @jiqing-feng
    jiqing-feng committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    9eb0970 View commit details
    Browse the repository at this point in the history

  9. remove check torch in bnb

    @jiqing-feng
    jiqing-feng committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    b506b98 View commit details
    Browse the repository at this point in the history

  10. docs: update bitsandbytes references with multi-backend info

    @Titus-von-Koeller
    Titus-von-Koeller committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    2be4169 View commit details
    Browse the repository at this point in the history

  11. docs: fix small mistakes in bnb paragraph

    @Titus-von-Koeller
    Titus-von-Koeller committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    e607b7c View commit details
    Browse the repository at this point in the history

  12. run formatting

    @Titus-von-Koeller
    Titus-von-Koeller committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    ac108c6 View commit details
    Browse the repository at this point in the history

  13. Merge remote-tracking branch 'origin/main' into bnb_cpu

    @Titus-von-Koeller
    Titus-von-Koeller committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    82dcb0d View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. reveret bnb check

    @jiqing-feng
    jiqing-feng committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    c66e7e7 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. move bnb multi-backend check to import_utils

    @jiqing-feng
    jiqing-feng committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    8f25ee2 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2024

  1. Update src/transformers/utils/import_utils.py 

    Co-authored-by: Aarni Koskela <akx@iki.fi>
    @jiqing-feng @akx
    jiqing-feng and akx authored Sep 14, 2024
    Configuration menu
    Copy the full SHA
    a4333cb View commit details
    Browse the repository at this point in the history

  2. fix bnb check

    @jiqing-feng
    jiqing-feng committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    32cbb8d View commit details
    Browse the repository at this point in the history

  3. minor fix for bnb

    @jiqing-feng
    jiqing-feng committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    4ce4b55 View commit details
    Browse the repository at this point in the history

  4. check lib first

    @jiqing-feng
    jiqing-feng committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    937ed3b View commit details
    Browse the repository at this point in the history

  5. fix code style

    @jiqing-feng
    jiqing-feng committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    e40f284 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'huggingface:main' into bnb_cpu

    @jiqing-feng
    jiqing-feng authored Sep 14, 2024
    Configuration menu
    Copy the full SHA
    03dd03b View commit details
    Browse the repository at this point in the history

  7. Revert "run formatting" 

    This reverts commit ac108c6.
    @jiqing-feng
    jiqing-feng committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    b8093ce View commit details
    Browse the repository at this point in the history

  8. fix format

    @jiqing-feng
    jiqing-feng committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    0551d23 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. give warning when bnb version is low and no cuda found]

    @jiqing-feng
    jiqing-feng committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    e33e43b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'huggingface:main' into bnb_cpu

    @jiqing-feng
    jiqing-feng authored Sep 18, 2024
    Configuration menu
    Copy the full SHA
    ced3c28 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2024

  1. fix device assignment check to be multi-device capable

    @Titus-von-Koeller
    Titus-von-Koeller committed Sep 22, 2024
    Configuration menu
    Copy the full SHA
    170dd58 View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. address akx feedback on get_avlbl_dev fn

    @Titus-von-Koeller
    Titus-von-Koeller committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    9ba4a5e View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2024

  1. we don't want the function tat publicc, as docs would be too much

    @Titus-von-Koeller
    Titus-von-Koeller committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    594f6f8 View commit details
    Browse the repository at this point in the history