Skip to content

Commit 22d33ba

Browse files
authored
[FrontEnd][Perf] merge_async_iterators fast-path for single-prompt requests (vllm-project#15150)
Signed-off-by: Nick Hill <[email protected]>
1 parent b0e96aa commit 22d33ba

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

vllm/utils.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -411,6 +411,11 @@ async def merge_async_iterators(
411411
When it yields, it yields a tuple (i, item) where i is the index of the
412412
iterator that yields the item.
413413
"""
414+
if len(iterators) == 1:
415+
# Fast-path single iterator case.
416+
async for item in iterators[0]:
417+
yield 0, item
418+
return
414419

415420
loop = asyncio.get_running_loop()
416421

0 commit comments

Comments
 (0)