-
Notifications
You must be signed in to change notification settings - Fork 649
[v0.11.0-dev][Bugfix][cherry-pick]bugfix for weight load of kimi-k2 #4190
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[v0.11.0-dev][Bugfix][cherry-pick]bugfix for weight load of kimi-k2 #4190
Conversation
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request correctly adds the packed module mapping for the kimi-k2 model, which is a necessary bugfix for enabling weight loading in quantized scenarios. The change is straightforward and addresses the issue. I have included one suggestion to improve the code formatting for better readability and adherence to standard Python style conventions.
| "experts": | ||
| ["experts.0.gate_proj", "experts.0.up_proj", "experts.0.down_proj"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For better readability and adherence to Python's style guide, this list should be formatted more conventionally. The current layout, where the list value starts on a new line at the same indentation level as its key, is inconsistent with PEP 8.1
"experts": [
"experts.0.gate_proj",
"experts.0.up_proj",
"experts.0.down_proj",
]Style Guide References
Footnotes
-
PEP 8, the style guide for Python code, recommends consistent indentation for continuation lines to improve readability. The value associated with a dictionary key should either be on the same line or indented on the following lines. ↩
Signed-off-by: Levi-JQ <[email protected]> ### What this PR does / why we need it? Fix kimi-k2 start bug, weight load ERROR:vllm-project#3785 ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? - vLLM version: v0.11.0rc3 - vLLM main: vllm-project/vllm@c9461e0 Signed-off-by: Levi-JQ <[email protected]> Co-authored-by: Levi-JQ <[email protected]> Co-authored-by: zhaozx-cn <[email protected]> Signed-off-by: menogrey <[email protected]>
23489a9 to
ea76a68
Compare
Signed-off-by: menogrey <[email protected]>
…llm-project#4190) ### What this PR does / why we need it? This is cherry-pick from vllm-project#3798 Fix kimi-k2 start bug, weight load ERROR:vllm-project#3785 ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? - vLLM version: v0.11.0rc3 - vLLM main: vllm-project/vllm@c9461e0 --------- Signed-off-by: Levi-JQ <[email protected]> Signed-off-by: menogrey <[email protected]> Co-authored-by: Levi <[email protected]> Co-authored-by: Levi-JQ <[email protected]> Co-authored-by: zhaozx-cn <[email protected]>
What this PR does / why we need it?
This is cherry-pick from #3798
Fix kimi-k2 start bug, weight load
ERROR:#3785
Does this PR introduce any user-facing change?
How was this patch tested?
What this PR does / why we need it?
Does this PR introduce any user-facing change?
How was this patch tested?