2.8.1

2.8.1 (2024-12-09)

Bug Fixes

• add 'clearMergeCache' (4f64377)

2.8.0

2.8.0 (2024-12-09)

Features

• add a LRU BPE merge cache (15d13b1), closes #68

Performance Improvements

• optimize token counting (c3e533c)

2.7.0

2.7.0 (2024-11-28)

Features

• implement 'countTokens' function (2d4146a), closes #67

2.6.2

2.6.2 (2024-11-13)

Bug Fixes

• correct special token matching & counting (3547826)
• unify property and variable names across the library (6030d91)

2.6.1

2.6.1 (2024-11-11)

Bug Fixes

• expose vocabulary size (402ff0b), closes #66
• use extensions in models.ts (78b803d), closes #65

2.6.0

2.6.0 (2024-11-04)

Bug Fixes

• initialize encodings array in parts (aa6c71d), closes #62

Features

• add new and update existing models (e832f9a)
• provide comprehensive data for all OpenAI models (ec2ad7e)

2.5.1

2.5.1 (2024-10-21)

(no changes, only benchmark update)

2.5.0

2.5.0 (2024-10-09)

Features

• added o1-preview and o1-mini chat completion models (#56) (41673af)

2.4.1

2.4.1 (2024-10-07)

Bug Fixes

• deps: update dependency gpt-tokenizer to ^2.4.0 (bf4b459)

2.4.0

2.4.0 (2024-09-23)

Features

• performance optimizations (661e283)