Skip to content

[Bug] Ministral 3 model loading fails with TypeError after PR #42455 #42565

@Aratako

Description

@Aratako

System Info

transformers: 5.0.0.dev0 @5efd0d4aa5ba4455f229b10667718cc623ac56af
python: 3.12
torch: 2.8.0+cu129 and 2.9.1+cu128
platform: Ubuntu (WSL) and Windows

Who can help?

@SunMarc @MekkCyber

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

After merging PR #42455, loading mistralai/Ministral-3-14B-Instruct-2512 fails with a TypeError: 'NoneType' object is not subscriptable.

It seems that block_size is becoming None inside finegrained_fp8.py, causing the crash when trying to access block_size[0].

To Reproduce

  1. Install transformers from main:

    pip install git+https://github.com/huggingface/transformers.git@main
  2. Run the following python script:

    from transformers import Mistral3ForConditionalGeneration
    
    model = Mistral3ForConditionalGeneration.from_pretrained(
        "mistralai/Ministral-3-14B-Instruct-2512"
    )

Expected behavior

The model should load successfully without errors.

Traceback

Traceback (most recent call last):
  File "test_ministral3.py", line 3, in <module>
    model = Mistral3ForConditionalGeneration.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../site-packages/transformers/modeling_utils.py", line 250, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File ".../site-packages/transformers/modeling_utils.py", line 3980, in from_pretrained
    hf_quantizer.preprocess_model(
  File ".../site-packages/transformers/quantizers/base.py", line 243, in preprocess_model
    self._process_model_before_weight_loading(model, **kwargs)
  File ".../site-packages/transformers/quantizers/quantizer_finegrained_fp8.py", line 172, in _process_model_before_weight_loading
    model = replace_with_fp8_linear(
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../site-packages/transformers/integrations/finegrained_fp8.py", line 514, in replace_with_fp8_linear
    new_module = FP8Linear(
                 ^^^^^^^^^^
  File ".../site-packages/transformers/integrations/finegrained_fp8.py", line 328, in __init__
    scale_out_features = (out_features + block_size[0] - 1) // block_size[0]
                                         ~~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable

As a workaround, installing the commit before the merge fixes the issue:
pip install git+https://github.com/huggingface/transformers.git@29e8522b85d9bf4287f63611aaa1a7db1fdc6489

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions