[Refactor] Remove redundant attention operator branches. #4531
+119
−470
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
November 28, 2025 13:15
Signed-off-by: weijinqian_v1 <[email protected]>
Signed-off-by: weijinqian_v1 <[email protected]>
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request refactors the attention mechanism by removing redundant operator branches and support for the 310P device. The changes unify different prefill attention paths into a single _forward_prefill method using npu_fused_infer_attention_score, which simplifies the codebase significantly. The _forward_v1_style method has been removed, and a new _forward_encode method has been introduced to handle encoder-only attention, making the main forward method cleaner and more readable. The logic for creating attention masks has also been simplified. Overall, the changes are well-executed, improve code maintainability, and appear to be correct. I have no major concerns.
added 4 commits
November 28, 2025 14:03
Signed-off-by: weijinqian_v1 <[email protected]>
Signed-off-by: weijinqian_v1 <[email protected]>
Signed-off-by: weijinqian_v1 <[email protected]>
Signed-off-by: weijinqian_v1 <[email protected]>
added 3 commits
November 28, 2025 16:44
Signed-off-by: weijinqian_v1 <[email protected]>
Signed-off-by: weijinqian_v1 <[email protected]>
Signed-off-by: weijinqian_v1 <[email protected]>
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
added 2 commits
November 29, 2025 09:25
Signed-off-by: weijinqian_v1 <[email protected]>
…pgrade. Signed-off-by: weijinqian_v1 <[email protected]>
added 2 commits
December 1, 2025 14:58
Signed-off-by: weijinqian_v1 <[email protected]>
Signed-off-by: weijinqian_v1 <[email protected]>
wangxiyuan
approved these changes
Dec 1, 2025
momo609
reviewed
Dec 1, 2025
| return None | ||
| return None | ||
|
|
||
| def _make_fia_attention_mask(self) -> torch.Tensor: |
Collaborator
Author
There was a problem hiding this comment.
It's used in pcp branch. It will be removed next PR.
added 5 commits
December 1, 2025 18:15
Signed-off-by: weijinqian_v1 <[email protected]>
Signed-off-by: weijinqian_v1 <[email protected]>
Signed-off-by: weijinqian_v1 <[email protected]>
Signed-off-by: weijinqian_v1 <[email protected]>
Signed-off-by: weijinqian_v1 <[email protected]>
weijinqian0
removed
ready
weijinqian0
added
ready
Signed-off-by: weijinqian_v1 <[email protected]>
This was referenced Dec 2, 2025
ChenCangtao
pushed a commit
to ChenCangtao/vllm-ascend
that referenced
this pull request
Dec 3, 2025
…t#4531) [Refactor] Remove redundant attention operator branches. Reason： We replace other attention ops with fused_infer_attention_score expect decode_only state. clean code and remove 310P support. vllm-project#4455 - vLLM version: v0.11.2 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.2 --------- Signed-off-by: weijinqian_v1 <[email protected]> Co-authored-by: weijinqian_v1 <[email protected]>
Mercykid-bash
pushed a commit
to Mercykid-bash/vllm-ascend
that referenced
this pull request
Dec 4, 2025
…t#4531) [Refactor] Remove redundant attention operator branches. Reason： We replace other attention ops with fused_infer_attention_score expect decode_only state. clean code and remove 310P support. vllm-project#4455 - vLLM version: v0.11.2 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.2 --------- Signed-off-by: weijinqian_v1 <[email protected]> Co-authored-by: weijinqian_v1 <[email protected]> Signed-off-by: Che Ruan <[email protected]>
Mercykid-bash
pushed a commit
to Mercykid-bash/vllm-ascend
that referenced
this pull request
Dec 4, 2025
…t#4531) [Refactor] Remove redundant attention operator branches. Reason： We replace other attention ops with fused_infer_attention_score expect decode_only state. clean code and remove 310P support. vllm-project#4455 - vLLM version: v0.11.2 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.2 --------- Signed-off-by: weijinqian_v1 <[email protected]> Co-authored-by: weijinqian_v1 <[email protected]> Signed-off-by: Che Ruan <[email protected]>
Contributor
|
@weijinqian0 #4713 gemma-2-9b-it & gemma-3-4b-it accuarcy test failed after the pr |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[Refactor] Remove redundant attention operator branches.
Reason：
We replace other attention ops with fused_infer_attention_score expect decode_only state.
clean code and remove 310P support.
#4455