support qwen3-next full_decode_only mode. #3949
Conversation
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds support for qwen3-next in full_decode_only mode by handling mixed attention types, specifically linear_attn. The changes in _build_dummy_attn_metadata correctly differentiate between attention builders to generate appropriate metadata for different layers. However, there is a block of redundant code that re-calculates attn_state, which should be removed to improve code clarity and maintainability.
| attn_state = AscendAttentionState.DecodeOnly | ||
| if self.speculative_config and \ | ||
| self.speculative_config.method == "deepseek_mtp": | ||
| attn_state = AscendAttentionState.SpecDecoding | ||
|
|
||
|
|
There was a problem hiding this comment.
This block of code is redundant as it re-calculates attn_state with the same logic as in lines 2720-2723. This can lead to confusion and potential maintenance issues. Please remove this duplicated block and use the attn_state variable that was already defined.
momo609
force-pushed
the
fulloptimze
branch
3 times, most recently
from
c69a4f5 to
63db91a
Compare
|
@momo609 Please elaborate on why we do not need to update linear attention params in |
momo609
force-pushed
the
fulloptimze
branch
5 times, most recently
from
97bc2b0 to
1cce890
Compare
weijinqian0
added
ready
Signed-off-by: wangxiaoxin-sherie <[email protected]>
### What this PR does / why we need it? support qwen3-next full_decode_only mode. bs=1, max_token=1024 | branch| tps| e2e time| | --- | --- | --- | |piecewise |3.06 | 8.15 | |fulldecodeonly | 7.2 | 3.47 | - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b Signed-off-by: wangxiaoxin-sherie <[email protected]> Co-authored-by: wangxiaoxin-sherie <[email protected]> Signed-off-by: Pz1116 <[email protected]>
### What this PR does / why we need it? support qwen3-next full_decode_only mode. bs=1, max_token=1024 | branch| tps| e2e time| | --- | --- | --- | |piecewise |3.06 | 8.15 | |fulldecodeonly | 7.2 | 3.47 | - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b Signed-off-by: wangxiaoxin-sherie <[email protected]> Co-authored-by: wangxiaoxin-sherie <[email protected]> Signed-off-by: luolun <[email protected]>
### What this PR does / why we need it? support qwen3-next full_decode_only mode. bs=1, max_token=1024 | branch| tps| e2e time| | --- | --- | --- | |piecewise |3.06 | 8.15 | |fulldecodeonly | 7.2 | 3.47 | - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b Signed-off-by: wangxiaoxin-sherie <[email protected]> Co-authored-by: wangxiaoxin-sherie <[email protected]> Signed-off-by: hwhaokun <[email protected]>
### What this PR does / why we need it? support qwen3-next full_decode_only mode. bs=1, max_token=1024 | branch| tps| e2e time| | --- | --- | --- | |piecewise |3.06 | 8.15 | |fulldecodeonly | 7.2 | 3.47 | - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b Signed-off-by: wangxiaoxin-sherie <[email protected]> Co-authored-by: wangxiaoxin-sherie <[email protected]> Signed-off-by: nsdie <[email protected]>
4 participants
What this PR does / why we need it?
support qwen3-next full_decode_only mode.
bs=1, max_token=1024
How was this patch tested?
Does this PR introduce any user-facing change?
How was this patch tested?