Skip to content

Commit 5ed3763

Browse files
menogreyLevi-JQLevi-JQzhaozx-cn
authored
[v0.11.0-dev][Bugfix][cherry-pick]bugfix for weight load of kimi-k2 (vllm-project#4190)
### What this PR does / why we need it? This is cherry-pick from vllm-project#3798 Fix kimi-k2 start bug, weight load ERROR:vllm-project#3785 ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? - vLLM version: v0.11.0rc3 - vLLM main: vllm-project/vllm@c9461e0 --------- Signed-off-by: Levi-JQ <[email protected]> Signed-off-by: menogrey <[email protected]> Co-authored-by: Levi <[email protected]> Co-authored-by: Levi-JQ <[email protected]> Co-authored-by: zhaozx-cn <[email protected]>
1 parent dded39f commit 5ed3763

File tree

2 files changed

+12
-1
lines changed

2 files changed

+12
-1
lines changed

.github/workflows/release_whl.yml

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,13 @@ jobs:
5757
- name: Print
5858
run: |
5959
lscpu
60-
60+
61+
- name: Free up disk space
62+
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1
63+
with:
64+
tool-cache: true
65+
docker-images: false
66+
6167
- name: Build wheel
6268
run: |
6369
ls

vllm_ascend/quantization/quant_config.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,11 @@ def get_scaled_act_names(self) -> List[str]:
193193
["experts.0.gate_proj", "experts.0.up_proj", "experts.0.down_proj"],
194194
"fused_qkv_a_proj": ["q_a_proj", "kv_a_proj_with_mqa"]
195195
},
196+
"kimi_k2": {
197+
"gate_up_proj": ["gate_proj", "up_proj"],
198+
"experts":
199+
["experts.0.gate_proj", "experts.0.up_proj", "experts.0.down_proj"]
200+
},
196201
"deepseek_v32": {
197202
"gate_up_proj": ["gate_proj", "up_proj"],
198203
"experts":

0 commit comments

Comments
 (0)