Skip to content

Tags: VectorInstitute/vectorlm

Tags

v0.1.2

Toggle v0.1.2's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Add revised benchmarking logic and results (#9)

* Revised estimation of batch count, directly retrieving from len(train_dataloader).
Deleted unused timer_handle argument in Trainer.
Revised handling of "max_seq_len" override in benchmarking.
Added support for automatic switching between  lora and full-rank sharding scheme in benchmarking.

* Revised handling of unspecified max_seq_length.
Added llama-3 to benchmark model_list.

* Benchmarking: Revised benchmark script to ensure consistent per-device train batch size.

* Benchmarking: replaced trainer.step with trainer.train_step to avoid eval overhead in benchmarking.
Revised benchmark parsing logic; display optimal batch size for each context width value.

* Benchmarking: Updated reference throughput based on updated logic.

* Benchmarking: Updated reference throughput descriptions.

v0.1.1

Toggle v0.1.1's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Dist ckpt refactor (#8)

* refactored for distributed checkpointing

* addressing comments

* missed commit

v0.1.0

Toggle v0.1.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Create LICENSE