[Bug]: can not pass token input to completions endpoint

[iWasOmen]

What happened?

A bug happened!

{ "model": "Qwen3-30B-A3B-Instruct-2507", "temperature": 0.6, "logprobs": true, "return_tokens_as_token_ids": true, "return_token_ids": true, "prompt": [ 151644, 8948, 198, 151645, 198, 151644, 872, 198, 14990, 151645, 198, 151644, 77091, 198 ] }
for /v1/completions endpoint, this request works in the origin vllm service
However, if I request to the litellm service which is just a gateway of the origin vllm service, it will return this:
{ "error": { "message": "litellm.APIConnectionError: APIConnectionError: OpenAIException - Unmapped prompt format. Your prompt is neither a list of strings nor a string. prompt=[151644, 8948, 198, 151645, 198, 151644, 872, 198, 14990, 151645, 198, 151644, 77091, 198]. File an issue - https://github.com/BerriAI/litellm/issues. Received Model Group=Qwen3-30B-A3B-Instruct-2507\nAvailable Model Group Fallbacks=None", "type": null, "param": null, "code": "500" } }
btw, if the input is long, the return message will change to this:
{ "error": { "message": "litellm.RateLimitError: RateLimitError: OpenAIException - Unmapped prompt format. Your prompt is neither a list of strings nor a string. prompt=[151644, 8948, 198, 2610, 525, 264...198]. File an issue - https://github.com/BerriAI/litellm/issues. Received Model Group=Qwen3-30B-A3B-Instruct-2507\nAvailable Model Group Fallbacks=None", "type": "throttling_error", "param": null, "code": "429" }

Can anyone help to fix it?

Relevant log output

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.80.0

Twitter / LinkedIn details

No response

[omkar806]

hi, can you share the code snippet that you have used ?

[iWasOmen]

hi, can you share the code snippet that you have used ?

first, I deploy a Qwen3-30B-A3B-Instruct-2507 model through vllm.
then in config.yaml:

model_list:
-model_name: Qwen3-30B-A3B-Instruct-2507
litellm_params:
model: openai/Qwen3-30B-A3B-Instruct-2507
api_base: http://xxx/v1
api_key: EMPTY

then, start the litelllm proxy service and request to the litellm service in postman or through curl:

curl --location 'http://xxx/v1/completions' \ --header 'Authorization: Bearer sk-xxx' \ --header 'Content-Type: application/json' \ --data '{ "model": "Qwen3-30B-A3B-Instruct-2507", "temperature": 0.6, "logprobs": true, "return_tokens_as_token_ids": true, "return_token_ids": true, "prompt": [ 151644, 8948, 198, 151645, 198, 151644, 872, 198, 14990, 151645, 198, 151644, 77091, 198 ] }'

I have tried both openai and vllm provider, they return the same error msg. If I request to the origin vllm service, no error.