Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quickfix: Accelerate YAML and LoRA Fused Ops #92

Merged
merged 11 commits into from
Oct 14, 2024
Merged

Quickfix: Accelerate YAML and LoRA Fused Ops #92

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
73f8a58
make accelerate.yaml 1.0 compatible
fabianlim Oct 11, 2024
87ad314
missed out biases in fused-lora
fabianlim Oct 11, 2024
2778eac
minor fix
fabianlim Oct 12, 2024
ebb166b
make compare with ref logic more robust
fabianlim Oct 12, 2024
7d196cb
again improve display bench logic
fabianlim Oct 12, 2024
df4ade7
relax mixed precision settings for full, regular-peft
fabianlim Oct 12, 2024
8178cd5
left out one bf16
fabianlim Oct 12, 2024
7e553e1
another improvement on compare_results
fabianlim Oct 12, 2024
6e7b7b8
update bench with full foak
fabianlim Oct 13, 2024
50e7e20
improve comments
fabianlim Oct 13, 2024
a50ff63
put back bf16 for baseline bnb
fabianlim Oct 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
improve comments 
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
  • Loading branch information
@fabianlim
fabianlim committed Oct 13, 2024
commit 50e7e20375a2b74e7083b0861898276ec724dbb4
2 changes: 1 addition & 1 deletion plugins/accelerated-peft/src/fms_acceleration_peft/framework_plugin_bnb.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ def augmentation(
train_args: TrainingArguments,
modifiable_args: Tuple[LoraConfig],
):
# assert that plugin requires mixed precision to be set
# - when using our prepare peft, we will enforce the mixed precision settings
assert (
train_args.bf16 is True or train_args.fp16 is True
), f"{self.__class__} requires mixed precision argument `--fp16` or `--bf16`"
Expand Down
3 changes: 2 additions & 1 deletion plugins/fused-ops-and-kernels/src/fms_acceleration_foak/framework_plugin_fast_kernels.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,8 @@ def augmentation(
has_quant = getattr(model, "quantization_method", None)

if has_quant:
# assert that plugin requires mixed precision to be set
# - only in the case where quant case, that we enforce the mixed precision settings
# - this is mostly for the fused-loras
assert (
train_args.bf16 is True or train_args.fp16 is True
), f"{self.__class__} requires mixed precision argument `--fp16` or `--bf16`"
Expand Down
Loading