vllm部署失败 AttributeError: 'dict' object has no attribute 'model_type'

[shatang123]

启动命令：vllm serve "/home/user01/vllm/model/RUC-DataLab/DeepAnalyze-8B" --host 0.0.0.0 --port 18003

错误日志

(APIServer pid=2114544) INFO 11-25 17:08:47 [api_server.py:1977] vLLM API server version 0.11.2
(APIServer pid=2114544) INFO 11-25 17:08:47 [utils.py:253] non-default args: {'model_tag': '/home/user01/vllm/model/RUC-DataLab/DeepAnalyze-8B', 'host': '0.0.0.0', 'port': 18003, 'model': '/home/user01/vllm/model/RUC-DataLab/DeepAnalyze-8B'}
(APIServer pid=2114544) Unrecognized keys in `rope_scaling` for 'rope_type'='yarn': {'attn_factor'}
(APIServer pid=2114544) INFO 11-25 17:08:47 [model.py:631] Resolved architecture: Qwen3ForCausalLM
(APIServer pid=2114544) INFO 11-25 17:08:47 [model.py:1745] Using max model len 131072
(APIServer pid=2114544) INFO 11-25 17:08:47 [scheduler.py:216] Chunked prefill is enabled with max_num_batched_tokens=2048.
(APIServer pid=2114544) Traceback (most recent call last):
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/bin/vllm", line 10, in <module>
(APIServer pid=2114544)     sys.exit(main())
(APIServer pid=2114544)              ^^^^^^
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/cli/main.py", line 73, in main
(APIServer pid=2114544)     args.dispatch_function(args)
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/cli/serve.py", line 60, in cmd
(APIServer pid=2114544)     uvloop.run(run_server(args))
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/uvloop/__init__.py", line 96, in run
(APIServer pid=2114544)     return __asyncio.run(
(APIServer pid=2114544)            ^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/runners.py", line 195, in run
(APIServer pid=2114544)     return runner.run(main)
(APIServer pid=2114544)            ^^^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=2114544)     return self._loop.run_until_complete(task)
(APIServer pid=2114544)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=2114544)     return await main
(APIServer pid=2114544)            ^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 2024, in run_server
(APIServer pid=2114544)     await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 2043, in run_server_worker
(APIServer pid=2114544)     async with build_async_engine_client(
(APIServer pid=2114544)                ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=2114544)     return await anext(self.gen)
(APIServer pid=2114544)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 195, in build_async_engine_client
(APIServer pid=2114544)     async with build_async_engine_client_from_engine_args(
(APIServer pid=2114544)                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=2114544)     return await anext(self.gen)
(APIServer pid=2114544)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 236, in build_async_engine_client_from_engine_args
(APIServer pid=2114544)     async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=2114544)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/vllm/utils/func_utils.py", line 116, in inner
(APIServer pid=2114544)     return fn(*args, **kwargs)
(APIServer pid=2114544)            ^^^^^^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 203, in from_vllm_config
(APIServer pid=2114544)     return cls(
(APIServer pid=2114544)            ^^^^
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 114, in __init__
(APIServer pid=2114544)     tokenizer = init_tokenizer_from_configs(self.model_config)
(APIServer pid=2114544)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/vllm/transformers_utils/tokenizer.py", line 287, in init_tokenizer_from_configs
(APIServer pid=2114544)     return get_tokenizer(
(APIServer pid=2114544)            ^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/vllm/transformers_utils/tokenizer.py", line 213, in get_tokenizer
(APIServer pid=2114544)     tokenizer = AutoTokenizer.from_pretrained(
(APIServer pid=2114544)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 1156, in from_pretrained
(APIServer pid=2114544)     return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
(APIServer pid=2114544)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2112, in from_pretrained
(APIServer pid=2114544)     return cls._from_pretrained(
(APIServer pid=2114544)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=2114544)   File "/home/user01/vllm/.venv/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2419, in _from_pretrained
(APIServer pid=2114544)     if _is_local and _config.model_type not in [
(APIServer pid=2114544)                      ^^^^^^^^^^^^^^^^^^
(APIServer pid=2114544) AttributeError: 'dict' object has no attribute 'model_type'

transformers版本

Name: transformers
Version: 4.57.2
Location: /home/user01/vllm/.venv/lib/python3.12/site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by: compressed-tensors, vllm, xgrammar

[zhangshaolei1998]

您的vllm是0.8.5以上的版本么？

[shatang123]

您的vllm是0.8.5以上的版本么？

是的，用最新版，0.11.2

[shatang123]

您的vllm是0.8.5以上的版本么？

能告诉我你们使用的具体vllm版本号吗，因为仓库中只显示>=0.8.5就可以了

[shatang123]

您的vllm是0.8.5以上的版本么？

还有你们使用的python版本

[shatang123]

您的vllm是0.8.5以上的版本么？

下面是我环境安装的步骤

uv venv --python 3.12 --seed
source .venv/bin/activate

uv pip install vllm --torch-backend=auto

# vllm版本信息 vllm -v
INFO 11-26 09:21:29 [scheduler.py:216] Chunked prefill is enabled with max_num_batched_tokens=2048.
0.11.2

# 启动命令
CUDA_VISIBLE_DEVICES=4  vllm serve "/home/user01/vllm/model/RUC-DataLab/DeepAnalyze-8B"  --host 0.0.0.0  --port 18003

[lordk911]

vllm 0.9.1
torch 2.7.0
transformers 4.52.4

另外确认下模型目录下config.json 是否有 model_type ：

grep "model_type" ./*.json
./config.json: "model_type": "qwen3",

[shatang123]

vllm 0.9.1 torch 2.7.0 transformers 4.52.4

另外确认下模型目录下config.json 是否有 model_type ：

grep "model_type" ./*.json ./config.json: "model_type": "qwen3",

有的，下面是config文件的部分内容

"max_position_embeddings": 131072,
  "max_window_layers": 36,
  "model_type": "qwen3",
  "num_attention_heads": 32,
  "num_hidden_layers": 36,
  "num_key_value_heads": 8,
  "pad_token_id": 151645,
  "rms_norm_eps": 1e-06,
  "rope_scaling": {
    "attn_factor": 0.8782488562869419,
    "factor": 4.0,
    "original_max_position_embeddings": 32768,
    "rope_type": "yarn"
  },
  "rope_theta": 1000000,
  "sliding_window": null,
  "tie_word_embeddings": false,
  "torch_dtype": "bfloat16",
  "transformers_version": "4.51.0",
  "use_cache": false,
  "use_sliding_window": false,
  "vocab_size": 151936

下面是分词器tokenizer_config.json配置文件部分内容

  "clean_up_tokenization_spaces": false,
  "eos_token": "<｜end▁of▁sentence｜>",
  "extra_special_tokens": {},
  "legacy": true,
  "model_max_length": 131072,
  "pad_token": "<｜end▁of▁sentence｜>",
  "sp_model_kwargs": {},
  "tokenizer_class": "LlamaTokenizerFast",
  "unk_token": null,
  "use_default_system_prompt": false

这里的分词器使用LlamaTokenizerFast的而不是Qwen2的，这个没问题吗，我不太懂，AI说的这个地方应该使用Qwen系列的

[lordk911]

我这边两个环境 vllm 0.9.2\0.9.1 都没问题，
"tokenizer_class": "LlamaTokenizerFast", 我本地也是这个

[shatang123]

我这边两个环境 vllm 0.9.2\0.9.1 都没问题， "tokenizer_class": "LlamaTokenizerFast", 我本地也是这个

我降级vllm到0.9.2启动没问题了，可能是新版本vllm兼容性问题