Skip to content

Commit e5bdac5

Browse files
committed
Add Func: npugraph_batch_size auto-adjust to different model
1 parent 46e8acd commit e5bdac5

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm_ascend/worker/model_runner_v1.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1043,7 +1043,7 @@ def sample_from_list(self, sample_len) -> list[int]:
10431043
# we use this function to sample a new list from old list by given length, and aintain uniformity, for example:
10441044
# original: [1 8 16 24 32 40 48 56 64]
10451045
# --> sample length = 3: [1 32 64]
1046-
# --> sample length = 5: [1 16 32 48 56]
1046+
# --> sample length = 5: [1 16 32 48 64]
10471047
original_len = len(self.npugraph_batch_sizes)
10481048
step = (original_len - 1) / (sample_len - 1)
10491049
indices = [round(i * step) for i in range(sample_len)]

0 commit comments

Comments
 (0)