Skip to content

Commit 12e1500

Browse files
Shirley125Angazenn
authored andcommitted
[bugfix][0.11]fix proxy decode bug (vllm-project#3751)
### What this PR does / why we need it? fix proxy decode bug while parsing non-UTF-8 characters. --------- Signed-off-by: CHEN <[email protected]>
1 parent ee5ff17 commit 12e1500

File tree

2 files changed

+14
-2
lines changed

2 files changed

+14
-2
lines changed

examples/disaggregated_prefill_v1/load_balance_proxy_layerwise_server_example.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -556,7 +556,13 @@ async def generate_stream():
556556
instance_info.prefiller_idx,
557557
instance_info.prefiller_score)
558558
released_kv = True
559-
chunk_str = chunk.decode("utf-8").strip()
559+
try:
560+
chunk_str = chunk.decode("utf-8").strip()
561+
except UnicodeDecodeError:
562+
logger.debug(
563+
f"Skipping chunk: {chunk}")
564+
yield chunk
565+
continue
560566
if not chunk_str:
561567
continue
562568
if chunk_str.startswith("data: "):

examples/disaggregated_prefill_v1/load_balance_proxy_server_example.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -539,7 +539,13 @@ async def generate_stream():
539539
instance_info.prefiller_idx,
540540
instance_info.prefiller_score)
541541
released_kv = True
542-
chunk_str = chunk.decode("utf-8").strip()
542+
try:
543+
chunk_str = chunk.decode("utf-8").strip()
544+
except UnicodeDecodeError:
545+
logger.debug(
546+
f"Skipping chunk: {chunk}")
547+
yield chunk
548+
continue
543549
if not chunk_str:
544550
continue
545551
if chunk_str.startswith("data: "):

0 commit comments

Comments
 (0)