-
Notifications
You must be signed in to change notification settings - Fork 8
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
6 changed files
with
49 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
31 changes: 31 additions & 0 deletions
31
sample-configurations/foak-fast-kernels-sample-configuration.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
# FMS Acceleration Plugin Configuration. | ||
# | ||
# Each stanza incorporates various configurations for | ||
# different fine-tuning / training tasks. | ||
plugins: | ||
# Configurations to accelerate data packing/padding in training | ||
training: | ||
|
||
fused_ops_and_kernels: | ||
|
||
# if under training stanza, then putting | ||
# base_layer and fused_lora will be a misnomer | ||
# - this should be in peft.quantized | ||
# However, if it is specified, it will still | ||
# be read. This is useful in use cases where | ||
# the yaml is system generated and not shown | ||
# to a user. | ||
|
||
# activate various unsloth optimizations | ||
# there are two versions of the plugin | ||
# - the FastKernel version supports individual kernels | ||
# - the FastQuantized version is all-or-nothing | ||
|
||
# fast loss triton kernels | ||
fast_loss: True | ||
|
||
# fast rms norm triton kernels | ||
fast_rsm_layernorm: True | ||
|
||
# fast RoPE embedding triton kernels | ||
fast_rope_embeddings: True |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters