-
Notifications
You must be signed in to change notification settings - Fork 624
add DeepSeek-R1 tutorial. #4666
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request adds a new tutorial for deploying the DeepSeek-R1 model. The documentation is well-structured and provides detailed instructions for both single-node and multi-node deployments. However, I found some critical inconsistencies in the model name within the provided commands, which will cause them to fail. Correcting these will significantly improve the user experience of following this tutorial.
| lm_eval \ | ||
| --model local-completions \ | ||
| --model_args model=path/DeepSeek-R1-w8a8,base_url=http://<node0_ip>:<port>/v1/completions,tokenized_requests=False,trust_remote_code=True \ | ||
| --tasks gsm8k \ | ||
| --output_path ./ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The model path path/DeepSeek-R1-w8a8 uses w8a8 in lowercase, which is inconsistent with the model name DeepSeek-R1-W8A8 used in the download link and the vllm serve commands. This will cause a 'file not found' error on case-sensitive filesystems.
Please correct this to DeepSeek-R1-W8A8 to ensure the command works as expected. Similar inconsistencies are present on lines 6, 18, and 291 and should also be corrected.
| lm_eval \ | |
| --model local-completions \ | |
| --model_args model=path/DeepSeek-R1-w8a8,base_url=http://<node0_ip>:<port>/v1/completions,tokenized_requests=False,trust_remote_code=True \ | |
| --tasks gsm8k \ | |
| --output_path ./ | |
| lm_eval \ | |
| --model local-completions \ | |
| --model_args model=path/DeepSeek-R1-W8A8,base_url=http://<node0_ip>:<port>/v1/completions,tokenized_requests=False,trust_remote_code=True \ | |
| --tasks gsm8k \ | |
| --output_path ./ |
|
|
||
| ```shell | ||
| export VLLM_USE_MODELSCOPE=true | ||
| vllm bench serve --model path/DeepSeek-R1-w8a8 --dataset-name random --random-input 200 --num-prompt 200 --request-rate 1 --save-result --result-dir ./ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The model path path/DeepSeek-R1-w8a8 is inconsistent with the official model name DeepSeek-R1-W8A8 from the download link. This will cause a 'file not found' error. Please correct it to DeepSeek-R1-W8A8.
| vllm bench serve --model path/DeepSeek-R1-w8a8 --dataset-name random --random-input 200 --num-prompt 200 --request-rate 1 --save-result --result-dir ./ | |
| vllm bench serve --model path/DeepSeek-R1-W8A8 --dataset-name random --random-input 200 --num-prompt 200 --request-rate 1 --save-result --result-dir ./ |
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
docs/source/tutorials/DeepSeek-R1.md
Outdated
| ::::{tab-item} A3 series | ||
| :sync: A3 | ||
|
|
||
| 1. Start the docker image on your node, refer to [using docker](../installation.md#set-up-using-docker). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
provide docker run command directly, like "#4399"
| :::: | ||
| ::::{tab-item} DeepSeek-R1-w8a8 A2 series | ||
| :sync: A2 | ||
|
|
||
| Run the following scripts on two nodes respectively. | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
provide docker run command by refering A3 series
Signed-off-by: Gongdayao <[email protected]>
Signed-off-by: Gongdayao <[email protected]>
|
/lgtm |
Signed-off-by: Gongdayao <[email protected]>
What this PR does / why we need it?
This PR adds tutorials for the DeepSeeK-R1 series models, including the A2 and A3 series, and provides accuracy validation results.
Does this PR introduce any user-facing change?
How was this patch tested?