Skip to content

Tags: strangedove/mergekit

Tags

v0.1.2

Toggle v0.1.2's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fix plumbing of apply_chat_template/fewshot_as_multiturn in mergekit-…

…evolve (arcee-ai#535)

v0.1.1

Toggle v0.1.1's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
LoRA extraction fixes (arcee-ai#522)

Addresses arcee-ai#521.

Also adds:
* `--lora-merge-dtype` to specify dtype to use when applying LoRA adapters to models
* `--gpu-rich` alias for convenience
* Organize display of options in `--help`

v0.1.0

Toggle v0.1.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Compute-graph based `mergekit-extract-lora` (arcee-ai#505)

Now with better embedding handling, multi-gpu execution, and lazy
loading/saving of tensors.

When extracting a LoRA from an 8B model, execution time goes from ~6
minutes down to 40 seconds with `--cuda --multi-gpu` on an 8-GPU
machine.

Additionally, the `--sv-epsilon` flag can be used to set a tolerance for
singular values to opportunistically reduce rank when the fine tuned
difference is inherently lower rank.

Also reimplement a couple of merge methods using the `@easy_define`
decorator and add some missing tests.

v0.0.6

Toggle v0.0.6's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Update copyright (arcee-ai#497)

Also bump dependencies.

v0.0.4

Toggle v0.0.4's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Update README.md

v0.0.3.2

Toggle v0.0.3.2's commit message
Reorganize slightly

v0.0.3.1

Toggle v0.0.3.1's commit message
Bump copyright date to 2024

legacy

Toggle legacy's commit message
Copy tokenizer