0.7.0

Added

• Add support o1 models (#18, #19)
• Add support chatgpt-4o models

0.6.0: Add support embedding models

Added

• Added support for models:
  • babbage-002
  • davinci-002
  • text-embedding-3-small
  • text-embedding-3-large

0.5.1

Fixed

• Fix invalidate cache of vocab when the hash is mismatch.

0.5.0

Added

• Added support for encoding o200k_base (GPT-4o) (#14)
• Invalidate cache of vocab by checksum

Changed

• [BC break] Changed signature of method VocabLoader::load

  - public function load(string $uri): Vocab;
  + public function load(string $uri, string|null $checksum = null): Vocab;

0.4.0: Improve performance

Changed

• Improved performance of converting text into tokens (PR #10)

Removed

• [BC break] Removed method EncodeUtil::fromBytes()

0.3.0

Added

• Added Encoder::encodeInChunks() method, which allows encode a text in chunks.

0.2.0: Use cache by default to avoid performance issues

Cache is enabled by default to avoid performance issues (see #2 and #5)