Skip to content

Commit 0d2ed94

Browse files
Mohammad OthmanMohammad Othman
authored andcommitted
perf: optimize LogprobsProcessor._update_prompt_logprobs iteration
This commit optimizes the loop iteration in LogprobsProcessor._update_prompt_logprobs() to reduce redundant operations and improve performance. **Problem:** The original implementation used range-based iteration with repeated list indexing: - token_ids[pos], prompt_logprobs[pos], prompt_token_ranks[pos] required 3 index operations per iteration - offset and offset_end were calculated even when decoded_tokens was None - Less Pythonic code structure **Solution:** 1. Use enumerate(zip(...)) to iterate through lists directly, eliminating repeated indexing 2. Only calculate offset when decoded_tokens is not None 3. Remove unnecessary offset_end variable **Performance Impact:** - Eliminates 3 list index operations per iteration - Avoids arithmetic when decoded_tokens is None - Micro-optimization that adds up for large num_prompt_tokens Signed-off-by: Mohammad Othman <[email protected]>
1 parent 6d17974 commit 0d2ed94

File tree

1 file changed

+12
-10
lines changed

1 file changed

+12
-10
lines changed

vllm/v1/engine/logprobs.py

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -138,21 +138,23 @@ def _update_prompt_logprobs(
138138
token_ids = token_ids.tolist()
139139

140140
# Make Logprob for each position.
141-
for pos in range(num_prompt_tokens):
141+
for pos, (token_ids_at_pos, logprobs_at_pos, ranks_at_pos) in enumerate(
142+
zip(token_ids, prompt_logprobs, prompt_token_ranks)
143+
):
142144
# Handle flattening.
143-
offset = pos * num_logprobs
144-
offset_end = offset + num_logprobs
145-
decoded_tokens_for_pos = (
146-
NONES if decoded_tokens is None else decoded_tokens[offset:offset_end]
147-
)
145+
if decoded_tokens is None:
146+
decoded_tokens_for_pos = NONES
147+
else:
148+
offset = pos * num_logprobs
149+
decoded_tokens_for_pos = decoded_tokens[offset : offset + num_logprobs]
148150

149151
# Update with the Logprob container for this pos.
150152
append_logprobs_for_next_position(
151153
self.prompt_logprobs,
152-
token_ids[pos],
153-
prompt_logprobs[pos],
154+
token_ids_at_pos,
155+
logprobs_at_pos,
154156
decoded_tokens_for_pos,
155-
prompt_token_ranks[pos],
157+
ranks_at_pos,
156158
self.num_prompt_logprobs,
157159
)
158160

@@ -179,4 +181,4 @@ def update_from_output(self, output: EngineCoreOutput) -> None:
179181
if output.new_logprobs is not None:
180182
self._update_sample_logprobs(output.new_logprobs)
181183
if output.new_prompt_logprobs_tensors is not None:
182-
self._update_prompt_logprobs(output.new_prompt_logprobs_tensors)
184+
self._update_prompt_logprobs(output.new_prompt_logprobs_tensors)

0 commit comments

Comments
 (0)