-
-
Notifications
You must be signed in to change notification settings - Fork 12k
[Model] Automatic conversion of TokenClassification model #30666
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: wang.yuqi <[email protected]>
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces automatic conversion for TokenClassification models. This is achieved by adding a mapping for the ForTokenClassification model architecture suffix to treat it as a pooling/classification task, and enhancing the ModelForSequenceClassification adapter to dynamically handle score.bias from model checkpoints. A new test case for a Qwen3-based token classification model is also added to verify the changes. The implementation appears correct and is a good enhancement for robust model conversion. I have not found any issues of high or critical severity.
…ct#30666) Signed-off-by: wang.yuqi <[email protected]>
…ct#30666) Signed-off-by: wang.yuqi <[email protected]> Signed-off-by: Joachim Studnia <[email protected]>
…ct#30666) Signed-off-by: wang.yuqi <[email protected]>
…ct#30666) Signed-off-by: wang.yuqi <[email protected]> Signed-off-by: sihao.li <[email protected]>
…ct#30666) Signed-off-by: wang.yuqi <[email protected]>
Purpose
fix #30107
Test Plan
tests/models/registry.py
tests/models/language/pooling/test_token_classification.py
Test Result
pass
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.