[main][bugfix] bugfix for qwen3 moe quantization #4599
Conversation
Signed-off-by: Wang Kunpeng <[email protected]>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request addresses a bug in the Qwen3 MoE quantization by correctly initializing the base class for AscendFusedMoEMethod. The change ensures that the FusedMoEMethodBase constructor is called with the appropriate moe_config from the layer.
| quant_config: AscendQuantConfig, | ||
| prefix: str, | ||
| packed_modules_mapping: Dict[str, Any], | ||
| layer: torch.nn.Module = None): |
There was a problem hiding this comment.
The layer parameter in the __init__ method is defined with a default value of None, but layer.moe_config is accessed unconditionally on the next line. This creates a potential AttributeError if AscendFusedMoEMethod is ever instantiated without providing a layer argument. Given that layer is always expected to be a FusedMoE instance when this method is called (as seen in AscendQuantConfig.get_quant_method), it should be made a required argument to reflect its actual usage and prevent potential runtime errors.
| layer: torch.nn.Module = None): | |
| layer: torch.nn.Module): |
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
Signed-off-by: Wang Kunpeng <[email protected]>
Signed-off-by: Wang Kunpeng <[email protected]>
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
…to qwen3-moe-bugfix # Conflicts: # tests/e2e/multicard/test_data_parallel.py
### What this PR does / why we need it? Fix the issue where the qwen3 moe service cannot be started due to upgrading the vllm version Error info: AttributeError: 'AscendFusedMoE' object has no attribute 'use dp chunking' ### Does this PR introduce _any_ user-facing change? no - vLLM version: v0.11.2 --------- Signed-off-by: Wang Kunpeng <[email protected]>
| layer: torch.nn.Module = None): | ||
| def __init__(self, quant_config: AscendQuantConfig, prefix: str, | ||
| packed_modules_mapping: Dict[str, | ||
| Any], layer: torch.nn.Module): |
### What this PR does / why we need it? Fix the issue where the qwen3 moe service cannot be started due to upgrading the vllm version Error info: AttributeError: 'AscendFusedMoE' object has no attribute 'use dp chunking' ### Does this PR introduce _any_ user-facing change? no - vLLM version: v0.11.2 --------- Signed-off-by: Wang Kunpeng <[email protected]> Signed-off-by: Che Ruan <[email protected]>
### What this PR does / why we need it? Fix the issue where the qwen3 moe service cannot be started due to upgrading the vllm version Error info: AttributeError: 'AscendFusedMoE' object has no attribute 'use dp chunking' ### Does this PR introduce _any_ user-facing change? no - vLLM version: v0.11.2 --------- Signed-off-by: Wang Kunpeng <[email protected]> Signed-off-by: Che Ruan <[email protected]>
4 participants
What this PR does / why we need it?
Fix the issue where the qwen3 moe service cannot be started due to upgrading the vllm version
Error info:
AttributeError: 'AscendFusedMoE' object has no attribute 'use dp chunking'
Does this PR introduce any user-facing change?
no
How was this patch tested?