Skip to content

Commit 8e2e8c5

Browse files
Pz1116hwhaokun
authored andcommitted
[main][doc][kv_pool]Add adxl timeout parameter in kv pool user guide (vllm-project#4012)
### What this PR does / why we need it? Add adxl timeout parameter in kv pool user guide, avoiding timeout error when initializing connections between devices. - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b Signed-off-by: Pz1116 <[email protected]> Signed-off-by: hwhaokun <[email protected]>
1 parent b1dc14a commit 8e2e8c5

File tree

1 file changed

+6
-0
lines changed

1 file changed

+6
-0
lines changed

docs/source/user_guide/feature_guide/kv_pool_mooncake.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,8 @@ export ASCEND_RT_VISIBLE_DEVICES=0,1,2,3
8787
export ACL_OP_INIT_MODE=1
8888
export ASCEND_BUFFER_POOL=4:8
8989
# ASCEND_BUFFER_POOL is the environment variable for configuring the number and size of buffer on NPU Device for aggregation and KV transfer,the value 4:8 means we allocate 4 buffers of size 8MB.
90+
export ASCEND_CONNECT_TIMEOUT=10000
91+
export ASCEND_TRANSFER_TIMEOUT=10000
9092
9193
python3 -m vllm.entrypoints.openai.api_server \
9294
--model /xxxxx/Qwen2.5-7B-Instruct \
@@ -147,6 +149,8 @@ export VLLM_USE_V1=1
147149
export ASCEND_RT_VISIBLE_DEVICES=4,5,6,7
148150
export ACL_OP_INIT_MODE=1
149151
export ASCEND_BUFFER_POOL=4:8
152+
export ASCEND_CONNECT_TIMEOUT=10000
153+
export ASCEND_TRANSFER_TIMEOUT=10000
150154
151155
python3 -m vllm.entrypoints.openai.api_server \
152156
--model /xxxxx/Qwen2.5-7B-Instruct \
@@ -246,6 +250,8 @@ export VLLM_USE_V1=1
246250
export ASCEND_RT_VISIBLE_DEVICES=0,1,2,3
247251
export ACL_OP_INIT_MODE=1
248252
export ASCEND_BUFFER_POOL=4:8
253+
export ASCEND_CONNECT_TIMEOUT=10000
254+
export ASCEND_TRANSFER_TIMEOUT=10000
249255
250256
python3 -m vllm.entrypoints.openai.api_server \
251257
--model /xxxxx/Qwen2.5-7B-Instruct \

0 commit comments

Comments
 (0)