Skip to content

Commit b323be9

Browse files
authored
deepseek torchair adapt for torch_npu version (#3876)
### What this PR does / why we need it? To adapt the torch_npu version to avoid the precision problem of torchair deepseek. The torch_npu version may result in the different branches in the ops register, the rms_norm ops has two branches according to the verson_check, this pr unify the rms_norm in torchair by patch method. #3862 Signed-off-by: hust17yixuan <[email protected]>
1 parent 29bd923 commit b323be9

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

vllm_ascend/torchair/utils.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -210,7 +210,7 @@ def torchair_quant_method_register():
210210

211211
def torchair_ops_patch():
212212
from vllm_ascend.ops.activation import AscendSiluAndMul
213-
from vllm_ascend.ops.layernorm import AscendRMSNorm
213+
from vllm_ascend.ops.layernorm import AscendQuantRMSNorm, AscendRMSNorm
214214
from vllm_ascend.ops.rotary_embedding import (
215215
AscendDeepseekScalingRotaryEmbedding, AscendRotaryEmbedding)
216216
from vllm_ascend.ops.vocab_parallel_embedding import \
@@ -232,6 +232,9 @@ def torchair_ops_patch():
232232
AscendRMSNorm.__init__ = torchair_layernorm.torchair_rmsnorm_init_ # type: ignore[method-assign]
233233
AscendRMSNorm.forward_oot = torchair_layernorm.torchair_rmsnorm_forward_oot # type: ignore[method-assign]
234234

235+
AscendQuantRMSNorm.__init__ = torchair_layernorm.torchair_rmsnorm_init_ # type: ignore[method-assign]
236+
AscendQuantRMSNorm.forward_oot = torchair_layernorm.torchair_rmsnorm_forward_oot # type: ignore[method-assign]
237+
235238
AscendSiluAndMul.forward_oot = torchair_activation.torchair_silu_and_mul_forward_oot # type: ignore[method-assign]
236239
AscendVocabParallelEmbedding.forward = vocab_embedding_forward # type: ignore[method-assign]
237240

0 commit comments

Comments
 (0)