dots1 expert bias routing still broken in v5.0.0rc0

[fjosw]

System Info

N/A

Who can help?

I just had a look at v5.0.0rc0 and saw that expert bias routing still seems to be broken for the dots1 model as a consequence of the refactoring in #40132.

I provided a potential fix in #41663 @ArthurZucker @Cyrilvallez

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

N/A

Expected behavior

N/A

[ArthurZucker]

We'll fix that! cc @vasqu if you have time or me hehe!

[vasqu]

Taking a look!

[vasqu]

I'm closing this since #41663 provides a proper fix. Thx a lot (and sorry for all the delays)! We potentially might move the function elsewhere (within the router) but I'll leave that to a separate PR.