Skip to content

Conversation

@swy20190
Copy link
Contributor

@swy20190 swy20190 commented Nov 28, 2025

What this PR does / why we need it?

Replace pyorch implement of sampling with triton kernels

Does this PR introduce any user-facing change?

No

@github-actions
Copy link

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request replaces several PyTorch-based sampling functions with Triton kernels, which should improve performance. The new kernels for greedy and random rejection sampling, batch expansion, and recovered token sampling appear to correctly replicate the logic of the original PyTorch functions. I've found a critical syntax error in the new Triton-related imports that will prevent the code from running. I've left a specific comment with a suggested fix. Additionally, it appears the unit tests in tests/ut/sample/test_rejection_sampler.py have not been updated to reflect the removal of the PyTorch functions and the introduction of the Triton kernels. These tests will fail and should be updated to validate the new implementations.

Comment on lines 12 to 13
from vllm.triton_utils import HAS_TRITON, triton
from vllm.triton_utils import triton.language as tl
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This import statement contains a syntax error. The from ... import ... syntax does not allow dot notation like triton.language in the imported names. To fix this and follow the common pattern in vLLM for handling the optional Triton dependency, you should import tl directly from vllm.triton_utils and combine the imports into a single line for conciseness.

from vllm.triton_utils import HAS_TRITON, triton, tl

@swy20190 swy20190 force-pushed the swy_rej_triton_kernels_1128 branch from 16cad8a to 7ff1172 Compare November 28, 2025 12:10
@whx-sjtu whx-sjtu added ready read for review ready-for-test start test by label for PR labels Nov 28, 2025
@whx-sjtu whx-sjtu force-pushed the swy_rej_triton_kernels_1128 branch from 6f4af8b to 6ffb171 Compare December 1, 2025 03:48
Lord_of_Ironhill and others added 2 commits December 1, 2025 11:48
@wangxiyuan wangxiyuan merged commit f4871c6 into vllm-project:main Dec 1, 2025
22 checks passed
ChenCangtao pushed a commit to ChenCangtao/vllm-ascend that referenced this pull request Dec 3, 2025
### What this PR does / why we need it?
Replace pyorch implement of sampling with triton kernels

### Does this PR introduce _any_ user-facing change?
No


- vLLM version: v0.11.2

---------

Signed-off-by: Lord_of_Ironhill <[email protected]>
Signed-off-by: whx-sjtu <[email protected]>
Co-authored-by: Lord_of_Ironhill <[email protected]>
Co-authored-by: whx-sjtu <[email protected]>
Mercykid-bash pushed a commit to Mercykid-bash/vllm-ascend that referenced this pull request Dec 4, 2025
### What this PR does / why we need it?
Replace pyorch implement of sampling with triton kernels

### Does this PR introduce _any_ user-facing change?
No

- vLLM version: v0.11.2

---------

Signed-off-by: Lord_of_Ironhill <[email protected]>
Signed-off-by: whx-sjtu <[email protected]>
Co-authored-by: Lord_of_Ironhill <[email protected]>
Co-authored-by: whx-sjtu <[email protected]>
Signed-off-by: Che Ruan <[email protected]>
Mercykid-bash pushed a commit to Mercykid-bash/vllm-ascend that referenced this pull request Dec 4, 2025
### What this PR does / why we need it?
Replace pyorch implement of sampling with triton kernels

### Does this PR introduce _any_ user-facing change?
No

- vLLM version: v0.11.2

---------

Signed-off-by: Lord_of_Ironhill <[email protected]>
Signed-off-by: whx-sjtu <[email protected]>
Co-authored-by: Lord_of_Ironhill <[email protected]>
Co-authored-by: whx-sjtu <[email protected]>
Signed-off-by: Che Ruan <[email protected]>
Meihan-chen pushed a commit to Meihan-chen/vllm-ascend that referenced this pull request Dec 5, 2025
### What this PR does / why we need it?
Replace pyorch implement of sampling with triton kernels

### Does this PR introduce _any_ user-facing change?
No


- vLLM version: v0.11.2

---------

Signed-off-by: Lord_of_Ironhill <[email protected]>
Signed-off-by: whx-sjtu <[email protected]>
Co-authored-by: Lord_of_Ironhill <[email protected]>
Co-authored-by: whx-sjtu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready read for review ready-for-test start test by label for PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants