Releases · OptimalScale/LMFlow

Major new features since v0.0.4

Support conversation templates
Support new optimization algorithms, e.g. LISA
Update requirements to support latest models
Fix bugs in qlora/lora scripts
Fix tokenization parallelism bug
Improve script interfaces

What's Changed

README refactor by @shizhediao in #607
Improve interface of finetuning scripts by @research4pan in #611
resize banner by @shizhediao in #612
Doc Reformat by @shizhediao in #614
Dev update transformers by @yaoguany in #616
Added QLoRA support for Decoder transformers with tune_strategy "Normal" by @TensorBlast in #613
announce long context support by @shizhediao in #621
fix deepspeed zero3 config bugs by @yaoguany in #622
Update version.py by @hendrydong in #624
update qr code by @shizhediao in #625
FIX BUG: trust_remote_code flag didn't take effect by @conght in #633
Add explanations about supported CUDA versions by @research4pan in #634
Added citation for RAFT by @shizhediao in #635
Update qrcode by @shizhediao in #636
[Features] Support multi_modal training by @lianqing11 in #628
Add scripts to convert raw file to text-only json by @research4pan in #638
speculative decoding by @wheresmyhair in #630
[Feature] Speculative Inference by @wheresmyhair in #640
add readme for speculative decoding by @wheresmyhair in #641
update news about speculative decoding by @shizhediao in #642
update llama flash attention by @yaoguany in #646
update qrcode by @shizhediao in #647
[FIX] Fix multi-modal training by @lianqing11 in #648
Fix: --disable_group_texts 1 keep short samples by @research4pan in #649
Support all types with --disable_group_texts 1 by @research4pan in #650
Fix model downloading for CPU-only servers by @research4pan in #651
add block size to fingerprint by @RolandMinrui in #653
update qrcode by @shizhediao in #654
Update QR code for wechat by @research4pan in #656
update qrcode by @shizhediao in #657
update qr code for wechat by @research4pan in #667
Update version of datasets dependency by @research4pan in #668
Add flash attention install for A6000 by @research4pan in #669
Update hf_decoder_model.py by @yaoguany in #670
fix bugs in llama flash attention by @yaoguany in #681
code exection class and test cases by @Bob17293729 in #674
Update README to reflect changes in v0.0.6 by @research4pan in #696
fix bugs in requirements.txt since previous one can cause errors by @xu1868 in #697
fix merge lora bug by @Dominic789654 in #698
Upgrade transformers deps to support mistral by @research4pan in #700
add lisa code and lisa args by @Dominic789654 in #701
add GPU memory check script by @Dominic789654 in #702
Support multi-gpu inference by @research4pan in #699
src/lmflow/args.py typo fix by @wheresmyhair in #703
add more info when fail to import flash attn by @wheresmyhair in #704
Add script to finetune llama-2 with lisa by @research4pan in #705
add LISA args introduction by @Dominic789654 in #706
add lisa bibtex by @Dominic789654 in #708
Rpan add news by @research4pan in #709
Fix typo in README.md by @research4pan in #710
Update lisa code by @Dominic789654 in #711
Fix recent bad commits, which lead to OOM in 7B by @research4pan in #712
Add more memory saving options for LISA script by @research4pan in #716
Add stable version 0.0.7 by @research4pan in #717
Export --lisa_interval_steps by @research4pan in #718
Update documentation.yaml to fix github check failure when submitting PRs by @wheresmyhair in #723
support more model by @Dominic789654 in #724
Update README.md to temporary allow latest fix by @research4pan in #725
Rpan dev lisa by @research4pan in #715
ShareGPT format support by @wheresmyhair in #720
Add latest support for flash-attention from hf by @research4pan in #727
Fix LISA init OOM caused by deepspeed by @research4pan in #728
Improve installation documentation in README.md by @research4pan in #733
Add temporary dataset document by @research4pan in #734
Improve README and requirement.txt by @research4pan in #735
documentation update and action fix by @wheresmyhair in #736
documentation page style update and readme dataset update by @wheresmyhair in #737
Add explanation for LISA with model-parallelism by @research4pan in #738
typo fix and disable html logo by @wheresmyhair in #739
Update dependency for transformers by @research4pan in #740
Rpan lisa more models by @research4pan in #742
Fix: typo in --lisa_layers_attribute by @research4pan in #743
Add inferencer by @research4pan in #744
Improve LISA layers_attribute mapping logics by @research4pan in #746
Change --disable_group_texts default to True by @research4pan in #747
readme (zh) update by @wheresmyhair in #748
readme (zh) typo fix by @wheresmyhair in #749
Change version to 0.0.7 in version.py by @research4pan in #750
readme (ko) update by @wheresmyhair in #751
Change lmflow.org to a temporary ip by @research4pan in #753
Add more clarification for data server problem by @research4pan in #755
Remove flash-attn in requirements.txt by @research4pan in #757
ko and zh readme update by @wheresmyhair in #758
ko readme update by @wheresmyhair in #761
[New feature] Integrate DPO by @gzliyu in #762
readme jp update by @wheresmyhair in #763
Recover data server by @research4pan in #764
remove data server address notice by @wheresmyhair in #765
readme update by @wheresmyhair in #766
hindi readme update by @wheresmyhair in #769
conversation dataset (sharegpt format) custom template dev by @wheresmyhair in #767
custom template document update by @wheresmyhair in #770
conversation template doc typo fix by @wheresmyhair in #771
template info update by @wheresmyhair in #772
add support for llama-3 template by @Wheres...