-
Notifications
You must be signed in to change notification settings - Fork 31.3k
Closed
Labels
Description
System Info
transformers: 5.0.0.dev0 @5efd0d4aa5ba4455f229b10667718cc623ac56af
python: 3.12
torch: 2.8.0+cu129 and 2.9.1+cu128
platform: Ubuntu (WSL) and Windows
Who can help?
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
After merging PR #42455, loading mistralai/Ministral-3-14B-Instruct-2512 fails with a TypeError: 'NoneType' object is not subscriptable.
It seems that block_size is becoming None inside finegrained_fp8.py, causing the crash when trying to access block_size[0].
To Reproduce
-
Install
transformersfrom main:pip install git+https://github.com/huggingface/transformers.git@main
-
Run the following python script:
from transformers import Mistral3ForConditionalGeneration model = Mistral3ForConditionalGeneration.from_pretrained( "mistralai/Ministral-3-14B-Instruct-2512" )
Expected behavior
The model should load successfully without errors.
Traceback
Traceback (most recent call last):
File "test_ministral3.py", line 3, in <module>
model = Mistral3ForConditionalGeneration.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../site-packages/transformers/modeling_utils.py", line 250, in _wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File ".../site-packages/transformers/modeling_utils.py", line 3980, in from_pretrained
hf_quantizer.preprocess_model(
File ".../site-packages/transformers/quantizers/base.py", line 243, in preprocess_model
self._process_model_before_weight_loading(model, **kwargs)
File ".../site-packages/transformers/quantizers/quantizer_finegrained_fp8.py", line 172, in _process_model_before_weight_loading
model = replace_with_fp8_linear(
^^^^^^^^^^^^^^^^^^^^^^^^
File ".../site-packages/transformers/integrations/finegrained_fp8.py", line 514, in replace_with_fp8_linear
new_module = FP8Linear(
^^^^^^^^^^
File ".../site-packages/transformers/integrations/finegrained_fp8.py", line 328, in __init__
scale_out_features = (out_features + block_size[0] - 1) // block_size[0]
~~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable
As a workaround, installing the commit before the merge fixes the issue:
pip install git+https://github.com/huggingface/transformers.git@29e8522b85d9bf4287f63611aaa1a7db1fdc6489