Support DeepSeekV3.2 with MLAPO operator

[ZYang6263]

What this PR does / why we need it?

This PR adds support for the optimized MLAPO operator in DSV3.2 and this operator provides an optimized implementation that avoids redundant q_down recomputation.
The operator implementation and optimizations were introduced in PR #4707.

Does this PR introduce any user-facing change?

How was this patch tested?

• vLLM version: v0.12.0
• vLLM main: vllm-project/vllm@ad32e3e

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[gemini-code-assist]

Code Review

This pull request adds support for DeepSeekV3.2 with the MLAPO operator and introduces context parallelism for SFA (SFA CP). The changes are extensive, involving modifications to the attention mechanism, introduction of shared weight layers for memory optimization, and new distributed communication patterns.

My review has identified a few critical issues that will cause NameError exceptions at runtime due to variables being used before they are defined in all code paths. Specifically, cum_query_lens and seq_lens are used without being defined, and actual_seq_lengths_query and actual_seq_lengths_key are not initialized in all execution branches before being passed to a function. I have also found a logical error in a new utility function. Please address these issues to ensure the code is correct and robust.

[gemini-code-assist]

The variables cum_query_lens and seq_lens are used here but are not defined within this scope. They should be accessed from attn_metadata, e.g., attn_metadata.cum_query_lens. This also applies to their usages on lines 775, 780, and 792 within this if self.enable_sfa_cp: block. This will cause a NameError at runtime.

Suggested change

	actual_seq_lengths_query = torch.empty_like(cum_query_lens)
	actual_seq_lengths_key = torch.empty_like(seq_lens)
	actual_seq_lengths_query = torch.empty_like(attn_metadata.cum_query_lens)
	actual_seq_lengths_key = torch.empty_like(attn_metadata.seq_lens)

[gemini-code-assist]

The call to self.indexer_select requires actual_seq_lengths_query and actual_seq_lengths_key as arguments. However, these variables are only defined when self.enable_sfa_cp is true. They are not defined for the else case, nor for the if self.enable_mlapo and not forward_context.with_prefill: path. This will lead to a NameError.

You should define these variables in all code paths leading to this call. For the paths where they are not defined, they should likely be initialized to attn_metadata.cum_query_lens and attn_metadata.seq_lens respectively.

[gemini-code-assist]

The logic in this function seems inverted. If torch.equal(a, b) is true, it proceeds to calculate the difference (which will be zero) and returns (0.0, 0.0). If they are not equal, it returns False. A function named check_diff would typically return False or (0.0, 0.0) to indicate no difference, and return the difference metrics when the tensors are not equal.

Suggested change

	def check_diff(a: torch.Tensor, b: torch.Tensor) -> Any:
	if torch.equal(a, b):
	absolute = torch.abs(a - b)
	relative = torch.abs(a - b) / (torch.abs(a) + 1e-9)
	return (torch.max(absolute).item(), torch.max(relative).item())
	return False
	def check_diff(a: torch.Tensor, b: torch.Tensor) -> Any:
	if torch.equal(a, b):
	return False
	absolute = torch.abs(a - b)
	relative = torch.abs(a - b) / (torch.abs(a) + 1e-9)
	return (torch.max(absolute).item(), torch.max(relative).item())

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.