Skip to content

Finalize Transformers.js v4 migration#18

Merged
xenova merged 23 commits intomainfrom
migration-integration
Jan 20, 2026
Merged

Finalize Transformers.js v4 migration#18
xenova merged 23 commits intomainfrom
migration-integration

Conversation

@xenova
Copy link
Collaborator

@xenova xenova commented Jan 20, 2026

Changes I made:

  • Expose additional functionality needed by Transformers.js
  • Align closer to rust library API
  • Fix normalized vs unnormalized special tokens handling
  • Use unminified versions by default (users can implement their own minification, as this affects class names when console logging)
  • Update unit tests

@xenova xenova requested a review from nico-martin January 20, 2026 02:59
Copy link
Collaborator

@nico-martin nico-martin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great stuff!
Added some small suggestions

@xenova xenova merged commit 89532d9 into main Jan 20, 2026
5 checks passed
@xenova xenova deleted the migration-integration branch January 20, 2026 17:51
@xenova
Copy link
Collaborator Author

xenova commented Jan 20, 2026

Thanks for the review! I implemented your suggestions and cleaned things up a bit. Merging now, then will put out v0.1, and finalize Transformers.js integration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants