Skip to content

Update transformers backend name #2075

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 139 commits into from
Nov 23, 2025
Merged

Update transformers backend name #2075

merged 139 commits into from
Nov 23, 2025

Conversation

@3outeille
Copy link
Contributor

@3outeille 3outeille commented Nov 21, 2025 •
edited
Loading

following Huggingface efforts in VLLM (cf vllm-project/vllm#28725), we would like to uniformize the naming and make sure that people think we use the HF models only

3outeille added 30 commits August 28, 2025 08:05
@3outeille
create transformer_backend folder with debug run
7488385
@3outeille
add hf config
39a3b34
@3outeille
can now register train spec for hf model
ea7c594
@3outeille
can now switch with different flavors using HF Llama modeling
5f0adf5
@3outeille
it is now working up to apply_ac
7c3795c
@3outeille
now working up to init_weights
3fb2bf8
@3outeille
fix mapping when convert_to_hf_config + add breaking test to ensure p…
25daeca 
…roper mapping
@3outeille
define own apply_ac for transformer backend instead of reusing llama3
3e67f2c
@3outeille
HF model without any parallelism now train (but grad_norm is high)
8c5c0ae
@3outeille
a bit cleaner way to get passed args
4ae9560
@3outeille
now same number of params + same attention backend but noticed higher…
9be95f9 
… gradnorm and less tps with HF model
@3outeille
fix seed and deterministic
bf91447
@3outeille
fix torch deterministic for HF modeling that was producing Nans
4c2fc0b
@3outeille
HF model now numerically stable compared to TT (given a fixed attenti…
9bffa38 
…on backend)
@3outeille
handling the is_hf_initialized flag in patch
40d84cc
@3outeille
refactor HF transformer model args
bd3f332
@3outeille
wrapper model class to avoid transformers to be explicit in train.py
249be92
@3outeille
add better testing script with reference log for later sanity check
e2d4ada
@3outeille
no need to fill passed args
4b498a9
@3outeille
can now handle multiple HF modeling
eb403d5
@3outeille
handle pref logits accessing inside HF model wrapper
a0d67a7
@3outeille
isolate HF patch for llama in another file
ea05552
@3outeille
find hacky way to pass HF model.name through CLI
adefa2c
@3outeille
more granularity of logging when doing parameter breakdown
a235863
@3outeille
add __repr__ to HFTransformerModelArgs for better debugging logs
fc43dc8
@3outeille
HF deepseek v3 is now training
23ae378
@3outeille
refactor to make it clear which args comes from which parts
2573be4
@3outeille
fix refactor and simplify things
46ae0a3
@3outeille
hacky way to switch flavors for now
b33d575
@3outeille
hf deepseek train while matching same param counts as tt deepseek
007f005
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Nov 21, 2025
@3outeille 3outeille marked this pull request as ready for review November 21, 2025 13:14
wwwjn
wwwjn reviewed Nov 21, 2025
View reviewed changes
Copy link
Contributor

@wwwjn wwwjn left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

.ci/docker/requirements-transformers-modeling-backend.txt
Copy link
Contributor

@wwwjn wwwjn Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the name changed, you will also need to change the name here to make CI running:

torchtitan/.ci/docker/common/install_conda.sh

Line 46 in 58fa181

pip_install -r /opt/conda/requirements-transformers-backend.txt

@3outeille 3outeille force-pushed the update_transformers_backend_name branch from 387c632 to 1259bb2 Compare November 21, 2025 19:16
@3outeille 3outeille force-pushed the update_transformers_backend_name branch from 1259bb2 to 79e44c1 Compare November 21, 2025 19:18
wwwjn
wwwjn reviewed Nov 21, 2025
View reviewed changes
Copy link
Contributor

@wwwjn wwwjn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renaming is low risk - Will see @tianyu-l 's opinion on this

torchtitan/experiments/transformers_modeling_backend/README.md
# Huggingface Transformers backend

## Quick start

Copy link
Contributor

@wwwjn wwwjn Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe also change the title name in README, and add some description that we are only using huggingface model definition as backend

tianyu-l
tianyu-l approved these changes Nov 21, 2025
View reviewed changes
Copy link
Contributor

@tianyu-l tianyu-l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sgtm, please fix tests

@3outeille
Copy link
Contributor Author

3outeille commented Nov 23, 2025

seems like CI fail but outside of my control this time

@tianyu-l
Copy link
Contributor

tianyu-l commented Nov 23, 2025 •
edited
Loading

ci failure unrelated

@tianyu-l tianyu-l merged commit ad9f188 into pytorch:main Nov 23, 2025
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wwwjn wwwjn wwwjn left review comments

@tianyu-l tianyu-l tianyu-l approved these changes

@fegin fegin Awaiting requested review from fegin fegin is a code owner

@wconstab wconstab Awaiting requested review from wconstab wconstab is a code owner

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@3outeille @tianyu-l @wwwjn