Skip to content

Commit d8ca7fe

Browse files
authored
[bugfix][main]fix proxy decode bug (#3750)
### What this PR does / why we need it? fix proxy decode bug when parsing non-UTF-8 characters. - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@c9461e0 --------- Signed-off-by: CHEN <[email protected]>
1 parent b8796b0 commit d8ca7fe

File tree

2 files changed

+14
-2
lines changed

2 files changed

+14
-2
lines changed

examples/disaggregated_prefill_v1/load_balance_proxy_layerwise_server_example.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -556,7 +556,13 @@ async def generate_stream():
556556
instance_info.prefiller_idx,
557557
instance_info.prefiller_score)
558558
released_kv = True
559-
chunk_str = chunk.decode("utf-8").strip()
559+
try:
560+
chunk_str = chunk.decode("utf-8").strip()
561+
except UnicodeDecodeError:
562+
logger.debug(
563+
f"Skipping chunk: {chunk}")
564+
yield chunk
565+
continue
560566
if not chunk_str:
561567
continue
562568
if chunk_str.startswith("data: "):

examples/disaggregated_prefill_v1/load_balance_proxy_server_example.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -539,7 +539,13 @@ async def generate_stream():
539539
instance_info.prefiller_idx,
540540
instance_info.prefiller_score)
541541
released_kv = True
542-
chunk_str = chunk.decode("utf-8").strip()
542+
try:
543+
chunk_str = chunk.decode("utf-8").strip()
544+
except UnicodeDecodeError:
545+
logger.debug(
546+
f"Skipping chunk: {chunk}")
547+
yield chunk
548+
continue
543549
if not chunk_str:
544550
continue
545551
if chunk_str.startswith("data: "):

0 commit comments

Comments
 (0)