-
Notifications
You must be signed in to change notification settings - Fork 3k
[hardware] feat: support qwen3vl ON ASCEND NPU #4734
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
ningmengliu seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request adds support for Qwen3-VL models on Ascend NPUs. The changes include updating the documentation to reflect this new support and adding new example and test scripts for Qwen3-VL-8B and Qwen3-VL-30B. While the changes are straightforward, the newly added shell scripts contain several critical syntax errors, such as missing line continuations and invalid characters in arguments, which will prevent them from running correctly. Additionally, the scripts rely on an undefined environment variable without proper checks. These issues need to be addressed to ensure the examples and tests are functional.
| actor_rollout_ref.rollout.max_num_batched_tokens=8192 | ||
| actor_rollout_ref.rollout.enable_chunked_prefill=True \ | ||
| actor_rollout_ref.actor.entropy_from_logits_with_chunking=True \ | ||
| actor_rollout_ref.ref.entropy_from_logits_with_chunking=True \ | ||
| +actor_rollout_ref.rollout.engine_kwargs.vllm.disable_mm_preprocessor_cache=True \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are a couple of syntax errors in the arguments passed to python3 that will cause the script to fail:
- Line 42 is missing a trailing
\to continue the command on the next line. This will cause the shell to terminate the python command prematurely and attempt to execute the following lines as separate commands. - Line 46 has a leading
+, which is an invalid character for a command-line argument here and will likely cause a parsing error in the python script.
| actor_rollout_ref.rollout.max_num_batched_tokens=8192 | |
| actor_rollout_ref.rollout.enable_chunked_prefill=True \ | |
| actor_rollout_ref.actor.entropy_from_logits_with_chunking=True \ | |
| actor_rollout_ref.ref.entropy_from_logits_with_chunking=True \ | |
| +actor_rollout_ref.rollout.engine_kwargs.vllm.disable_mm_preprocessor_cache=True \ | |
| actor_rollout_ref.rollout.max_num_batched_tokens=8192 \ | |
| actor_rollout_ref.rollout.enable_chunked_prefill=True \ | |
| actor_rollout_ref.actor.entropy_from_logits_with_chunking=True \ | |
| actor_rollout_ref.ref.entropy_from_logits_with_chunking=True \ | |
| actor_rollout_ref.rollout.engine_kwargs.vllm.disable_mm_preprocessor_cache=True \ |
| actor_rollout_ref.rollout.tensor_model_parallel_size=8 \ | ||
| actor_rollout_ref.rollout.name=$ENGINE \ | ||
| actor_rollout_ref.rollout.ignore_eos=False \ | ||
| +actor_rollout_ref.rollout.engine_kwargs.vllm.disable_mm_preprocessor_cache=True \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This line has a leading +, which is an invalid character for a command-line argument here and will likely cause a parsing error in the python script.
| +actor_rollout_ref.rollout.engine_kwargs.vllm.disable_mm_preprocessor_cache=True \ | |
| actor_rollout_ref.rollout.engine_kwargs.vllm.disable_mm_preprocessor_cache=True \ |
| actor_rollout_ref.rollout.max_num_batched_tokens=8192 | ||
| actor_rollout_ref.rollout.enable_chunked_prefill=True \ | ||
| actor_rollout_ref.actor.entropy_from_logits_with_chunking=True \ | ||
| actor_rollout_ref.ref.entropy_from_logits_with_chunking=True \ | ||
| +actor_rollout_ref.rollout.engine_kwargs.vllm.disable_mm_preprocessor_cache=True \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are a couple of syntax errors in the arguments passed to python3 that will cause the script to fail:
- Line 47 is missing a trailing
\to continue the command on the next line. This will cause the shell to terminate the python command prematurely and attempt to execute the following lines as separate commands. - Line 51 has a leading
+, which is an invalid character for a command-line argument here and will likely cause a parsing error in the python script.
| actor_rollout_ref.rollout.max_num_batched_tokens=8192 | |
| actor_rollout_ref.rollout.enable_chunked_prefill=True \ | |
| actor_rollout_ref.actor.entropy_from_logits_with_chunking=True \ | |
| actor_rollout_ref.ref.entropy_from_logits_with_chunking=True \ | |
| +actor_rollout_ref.rollout.engine_kwargs.vllm.disable_mm_preprocessor_cache=True \ | |
| actor_rollout_ref.rollout.max_num_batched_tokens=8192 \ | |
| actor_rollout_ref.rollout.enable_chunked_prefill=True \ | |
| actor_rollout_ref.actor.entropy_from_logits_with_chunking=True \ | |
| actor_rollout_ref.ref.entropy_from_logits_with_chunking=True \ | |
| actor_rollout_ref.rollout.engine_kwargs.vllm.disable_mm_preprocessor_cache=True \ |
| data.train_files=$data_path/train.parquet \ | ||
| data.val_files=$data_path/test.parquet \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The $data_path variable is used here but not defined in the script. This will cause the script to fail if the variable is not set in the environment. It's good practice to add a check to ensure it's set and exit with an error if it's not. For example, you could add this at the top of the script:
if [ -z "$data_path" ]; then
echo "Error: data_path environment variable is not set." >&2
exit 1
fi| data.train_files=$data_path/train.parquet \ | ||
| data.val_files=$data_path/test.parquet \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The $data_path variable is used here but not defined in the script. This will cause the script to fail if the variable is not set in the environment. It's good practice to add a check to ensure it's set and exit with an error if it's not. For example, you could add this at the top of the script:
if [ -z "$data_path" ]; then
echo "Error: data_path environment variable is not set." >&2
exit 1
fi| data.train_files=$data_path/train.parquet \ | ||
| data.val_files=$data_path/test.parquet \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The $data_path variable is used here but not defined in the script. This will cause the script to fail if the variable is not set in the environment. It's good practice to add a check to ensure it's set and exit with an error if it's not. For example, you could add this at the top of the script:
if [ -z "$data_path" ]; then
echo "Error: data_path environment variable is not set." >&2
exit 1
fi
What does this PR do?
Checklist Before Starting
[{modules}] {type}: {description}(This will be checked by the CI){modules}includefsdp,megatron,sglang,vllm,rollout,trainer,ci,training_utils,recipe,hardware,deployment,ray,worker,single_controller,misc,perf,model,algo,env,tool,ckpt,doc,data,cfg,reward,like[megatron, fsdp, doc]{type}is infeat,fix,refactor,chore,test[BREAKING]to the beginning of the title.[BREAKING][fsdp, megatron] feat: dynamic batchingTest
API and Usage Example
# Add code snippet or script demonstrating how to use thisDesign & Code Changes
Checklist Before Submitting
Important
Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.
pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=alwaysci-requestchannel in theverlSlack workspace. (If not accessible, please try the Feishu group (飞书群).)