🚀 The feature, motivation and pitch

EPLB requires the number of experts to be chosen up front and there is a known minimum valid value that can be derived from the vllm startup configuration. Since extra EPLB experts trades kv cache memory for potential performance improvements, but that is not guaranteed to pay off, having the EPLB value default to the minimum valid value would reduce friction on enabling EPLB the first time until users are ready to tune.

As a consequence, it would also streamline templating the same config to work across multiple EP sizes for the default case.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.