Skip to content

Conversation

@huachenheli
Copy link
Contributor

@huachenheli huachenheli commented Aug 22, 2025

Summary:
This PR adds user-supplied uuids to the chat interface (#22044). Should only land after #23394 due to the changes needed in multimodal cache.

Test Plan:
pytest tests/entrypoints/test_chat_utils.py

e2e test:
HF_HUB_DISABLE_XET=1 CUDA_VISIBLE_DEVICES=2 pytest tests/entrypoints/openai/test_vision.py

python -m vllm.entrypoints.openai.api_server --model Qwen/Qwen2.5-VL-7B-Instruct --port 8001 --host 0.0.0.0 --dtype bfloat16 --limit-mm-per-prompt '{"video":1}'

chat_completion_from_url = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "video_url",
                    "video_url": {"url": video_url},
                    "uuid": "my_uuid_1234",
                },
                {
                    "type": "text",
                    "text": "Describe this video in detail",
                },
            ],
        }
    ],
    model=model,
)

Also manually verified provided UUIDs are correctly passed down to mm hasher.

Reviewers:

Subscribers:

Tasks:

Tags:

Purpose

Test Plan

Test Result

(Optional) Documentation Update


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

ywang96 and others added 4 commits August 22, 2025 02:17
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
Signed-off-by: Chenheli Hua <[email protected]>
@mergify mergify bot added frontend multi-modality Related to multi-modality (#4194) labels Aug 23, 2025
@huachenheli huachenheli changed the title [WIP] #22044 [WIP] User-provided uuids for medias (RFC #22044) Aug 23, 2025
@DarkLight1337
Copy link
Member

Are you working with @ywang96 on this? He has opened #23394

Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
@huachenheli
Copy link
Contributor Author

Are you working with @ywang96 on this? He has opened #23394

Yeah. I think this needs to rebased on top of that instead. I was originally thinking about tracking uuids as part of multi_modal_data but that would have compatibility issues.

Roger Wang and others added 7 commits August 24, 2025 23:56
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
Signed-off-by: Chenheli Hua <[email protected]>
@huachenheli huachenheli changed the title [WIP] User-provided uuids for medias (RFC #22044) [WIP] [Frontend] User-provided uuids for medias in chat. (RFC #22044) Aug 25, 2025
Signed-off-by: Chenheli Hua <[email protected]>
Signed-off-by: Chenheli Hua <[email protected]>
Signed-off-by: Chenheli Hua <[email protected]>
Signed-off-by: Chenheli Hua <[email protected]>
@huachenheli huachenheli force-pushed the mm_uuid branch 2 times, most recently from ae77156 to a26f41d Compare August 26, 2025 03:21
@ywang96 ywang96 enabled auto-merge (squash) September 4, 2025 21:01
@ywang96 ywang96 disabled auto-merge September 7, 2025 11:46
@ywang96 ywang96 enabled auto-merge (squash) September 7, 2025 11:46
@vllm-bot vllm-bot merged commit 01dfb5e into vllm-project:main Sep 8, 2025
41 of 43 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Multi-modality Core Sep 8, 2025
eicherseiji pushed a commit to eicherseiji/vllm that referenced this pull request Sep 9, 2025
…22044) (vllm-project#23449)

Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Chenheli Hua <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Cyrus Leung <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
skyloevil pushed a commit to skyloevil/vllm that referenced this pull request Sep 13, 2025
…22044) (vllm-project#23449)

Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Chenheli Hua <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Cyrus Leung <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025
…22044) (vllm-project#23449)

Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Chenheli Hua <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Cyrus Leung <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025
…22044) (vllm-project#23449)

Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Chenheli Hua <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Cyrus Leung <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
Signed-off-by: xuebwang-amd <[email protected]>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
…22044) (vllm-project#23449)

Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Chenheli Hua <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: Cyrus Leung <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
Signed-off-by: xuebwang-amd <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation frontend multi-modality Related to multi-modality (#4194) ready ONLY add when PR is ready to merge/full CI is needed v1

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants