Commit 693234e
authored
[BugFix] Fix Qwen3-next break (vllm-project#3428)
### What this PR does / why we need it?
Fix Qwen3NextGatedDeltaNet, caused by
vllm-project/vllm#26437
### How was this patch tested?
```
def main():
prompts = [
"窗前明月光,",
"The president of the United States is Mr.",
"The capital of France is",
"The future of AI is",
"感时花溅泪,",
"家书抵万金啥意思?",
"plz tell me a story: ",
]
# Create a sampling params object.
sampling_params = SamplingParams(max_tokens=100, temperature=0.6, top_k=40, top_p=0.95)
# Create an LLM.
llm = LLM(
model="/root/.cache/modelscope/hub/models/Qwen/Qwen3-Next-80B-A3B-Instruct",
tensor_parallel_size=4,
enforce_eager=True,
trust_remote_code=True,
max_model_len=256,
gpu_memory_utilization=0.7,
block_size=64
)
# Generate texts from the prompts.
outputs = llm.generate(prompts, sampling_params)
for output in outputs:
prompt = output.prompt
generated_text = output.outputs[0].text
print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")
```
- vLLM version: v0.11.0rc3
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0
---------
Signed-off-by: Icey <[email protected]>
Signed-off-by: luolun <[email protected]>1 parent f3208e5 commit 693234e
1 file changed
+35
-11
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
| 54 | + | |
| 55 | + | |
54 | 56 | | |
55 | 57 | | |
56 | 58 | | |
| |||
201 | 203 | | |
202 | 204 | | |
203 | 205 | | |
204 | | - | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
205 | 211 | | |
206 | 212 | | |
207 | 213 | | |
| |||
216 | 222 | | |
217 | 223 | | |
218 | 224 | | |
219 | | - | |
220 | | - | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
221 | 228 | | |
222 | 229 | | |
223 | 230 | | |
| |||
242 | 249 | | |
243 | 250 | | |
244 | 251 | | |
245 | | - | |
246 | | - | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
247 | 259 | | |
248 | 260 | | |
249 | 261 | | |
| |||
293 | 305 | | |
294 | 306 | | |
295 | 307 | | |
296 | | - | |
297 | | - | |
298 | | - | |
299 | | - | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
300 | 318 | | |
301 | 319 | | |
302 | 320 | | |
| |||
404 | 422 | | |
405 | 423 | | |
406 | 424 | | |
407 | | - | |
408 | | - | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
409 | 433 | | |
410 | 434 | | |
411 | 435 | | |
| |||
0 commit comments