Skip to content

Commit 865f1f7

Browse files
Levi-JQLevi-JQ
andauthored
[Bugfix] Resolve the interface compatibility issue of get_input_embeddings in MM (#4638)
### What this PR does / why we need it? Resolve the interface compatibility issue of get_input_embeddings in MM, because the get_input_embeddings func of other model does not have the is_multimodal parameter --------- Signed-off-by: Levi-JQ <[email protected]> Co-authored-by: Levi-JQ <[email protected]>
1 parent 3b4cb23 commit 865f1f7

File tree

1 file changed

+13
-5
lines changed

1 file changed

+13
-5
lines changed

vllm_ascend/worker/model_runner_v1.py

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1394,11 +1394,19 @@ def _prepare_inputs(
13941394
# embeddings), we always use embeddings (rather than token ids)
13951395
# as input to the multimodal model, even when the input is text.
13961396
input_ids = self.input_ids[:total_num_scheduled_tokens]
1397-
inputs_embeds = self.model.get_input_embeddings(
1398-
input_ids,
1399-
multimodal_embeddings=mm_embeds,
1400-
is_multimodal=is_mm_embed,
1401-
)
1397+
model_type = self.vllm_config.model_config.hf_config.model_type
1398+
if model_type == "qwen2_5_vl":
1399+
inputs_embeds = self.model.get_input_embeddings(
1400+
input_ids,
1401+
multimodal_embeddings=mm_embeds,
1402+
is_multimodal=is_mm_embed,
1403+
)
1404+
else:
1405+
if mm_embeds:
1406+
inputs_embeds = self.model.get_input_embeddings(
1407+
input_ids, mm_embeds)
1408+
else:
1409+
inputs_embeds = self.model.get_input_embeddings(input_ids)
14021410
# TODO(woosuk): Avoid the copy. Optimize.
14031411
self.inputs_embeds[:total_num_scheduled_tokens].copy_(
14041412
inputs_embeds)

0 commit comments

Comments
 (0)