@@ -25,25 +25,46 @@ If you want to deploy multi-node environment, you need to verify multi-node comm
2525
2626### Installation
2727
28- You can using our official docker image and install extra operator for supporting ` DeepSeek-R1-w8a8 ` .
29-
30- :::{note}
31- Only AArch64 architecture are supported currently due to extra operator's installation limitations.
32- :::
33-
34- :::::{tab-set}
35- :sync-group: install
36-
37- ::::{tab-item} A3 series
38- :sync: A3
39-
40- 1 . Start the docker image on your node, refer to [ using docker] ( ../installation.md#set-up-using-docker ) .
41-
42- In addition, if you don't want to use the docker image as above, you can also build all from source:
43-
44- - Install ` vllm-ascend ` from source, refer to [ installation] ( ../installation.md ) .
45-
46- - Install extra operator for supporting ` DeepSeek-R1-w8a8 ` , refer to the above tab.
28+ You can using our official docker image to run ` DeepSeek-R1-w8a8 ` directly.
29+
30+ Select an image based on your machine type and start the docker image on your node, refer to [ using docker] ( ../installation.md#set-up-using-docker ) .
31+
32+ ``` {code-block} bash
33+ :substitutions:
34+ # Update --device according to your device (Atlas A2: /dev/davinci[0-7] Atlas A3:/dev/davinci[0-15]).
35+ # Update the vllm-ascend image according to your environment.
36+ # Note you should download the weight to /root/.cache in advance.
37+ # Update the vllm-ascend image
38+ export IMAGE=m.daocloud.io/quay.io/ascend/vllm-ascend:|vllm_ascend_version|
39+ export NAME=vllm-ascend
40+
41+ # Run the container using the defined variables
42+ # Note: If you are running bridge network with docker, please expose available ports for multiple nodes communication in advance
43+ docker run --rm \
44+ --name $NAME \
45+ --net=host \
46+ --shm-size=500g \
47+ --device /dev/davinci0 \
48+ --device /dev/davinci1 \
49+ --device /dev/davinci2 \
50+ --device /dev/davinci3 \
51+ --device /dev/davinci4 \
52+ --device /dev/davinci5 \
53+ --device /dev/davinci6 \
54+ --device /dev/davinci7 \
55+ --device /dev/davinci_manager \
56+ --device /dev/devmm_svm \
57+ --device /dev/hisi_hdc \
58+ -v /usr/local/dcmi:/usr/local/dcmi \
59+ -v /etc/hccn.conf:/etc/hccn.conf \
60+ -v /usr/local/Ascend/driver/tools/hccn_tool:/usr/local/Ascend/driver/tools/hccn_tool \
61+ -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
62+ -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
63+ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
64+ -v /etc/ascend_install.info:/etc/ascend_install.info \
65+ -v /mnt/sfs_turbo/.cache:/root/.cache \
66+ -it $IMAGE bash
67+ ```
4768
4869If you want to deploy multi-node environment, you need to set up environment on each node.
4970
@@ -56,7 +77,6 @@ If you want to deploy multi-node environment, you need to set up environment on
5677:sync-group: install
5778
5879::::{tab-item} DeepSeek-R1-w8a8 A3 series
59- :sync: A3
6080
6181``` shell
6282#! /bin/sh
@@ -106,7 +126,6 @@ vllm serve path/DeepSeek-R1-W8A8 \
106126
107127::::
108128::::{tab-item} DeepSeek-R1-w8a8 A2 series
109- :sync: A2
110129
111130Run the following scripts on two nodes respectively.
112131
0 commit comments