[Bugfix] Resolve the interface compatibility issue of get_input_embeddings in MM

[Levi-JQ]

What this PR does / why we need it?

Resolve the interface compatibility issue of get_input_embeddings in MM， because the get_input_embeddings func of other model does not have the is_multimodal parameter

Does this PR introduce any user-facing change?

How was this patch tested?

[gemini-code-assist]

Code Review

This pull request resolves an interface compatibility issue in multimodal models by checking if the get_input_embeddings function supports the is_multimodal parameter before calling it. This is a good approach for ensuring backward compatibility. However, the implementation uses inspect.signature within the _prepare_inputs method, which is a performance-critical path. My review includes a suggestion to cache the result of this check to avoid repeated overhead, improving the overall efficiency.

[gemini-code-assist]

Using inspect.signature inside _prepare_inputs can introduce a performance overhead, as this method is on a hot path and reflection is generally slow. This check should be performed only once and the result should be cached.

A better approach would be to perform this check once and store the result in an instance attribute. You can add a new attribute, for example _has_is_multimodal_in_get_input_embeddings, and compute it once within _prepare_inputs using lazy initialization.

Suggested change

	has_is_multimodal = 'is_multimodal' in inspect.signature(self.model.get_input_embeddings).parameters
	if has_is_multimodal:
	inputs_embeds = self.model.get_input_embeddings(
	input_ids,
	multimodal_embeddings=mm_embeds,
	is_multimodal=is_mm_embed,
	)
	else:
	if mm_embeds:
	inputs_embeds = self.model.get_input_embeddings(
	input_ids, mm_embeds)
	else:
	inputs_embeds = self.model.get_input_embeddings(input_ids)
	if not hasattr(self, "_has_is_multimodal_in_get_input_embeddings"):
	self._has_is_multimodal_in_get_input_embeddings = (
	'is_multimodal' in inspect.signature(
	self.model.get_input_embeddings).parameters)
	if self._has_is_multimodal_in_get_input_embeddings:
	inputs_embeds = self.model.get_input_embeddings(
	input_ids,
	multimodal_embeddings=mm_embeds,
	is_multimodal=is_mm_embed,
	)
	else:
	if mm_embeds:
	inputs_embeds = self.model.get_input_embeddings(
	input_ids, mm_embeds)
	else:
	inputs_embeds = self.model.get_input_embeddings(input_ids)

[MengqingCao]

I have checked that is_multimodal is not included in vLLM v0.11.0, thus only the custom Qwen2_5_vl in vllm-ascend have this arg. Maybe we can just check the model_type

[Levi-JQ]

ok

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[MengqingCao]

I think this pr is okay to merge as the broken test case tests/e2e/singlecard/test_vlm.py::test_multimodal_audio has already passed. https://github.com/vllm-project/vllm-ascend/actions/runs/19857925293/job/56900838462?pr=4638