Import warning when loading a local model from a path that ends with a slash

[kk-89]

System Info

• transformers version: 4.57.1
• Platform: Linux-6.8.0-87-generic-x86_64-with-glibc2.39
• Python version: 3.12.3
• Huggingface_hub version: 0.36.0
• Safetensors version: 0.7.0
• Accelerate version: 1.11.0
• Accelerate config:
  • compute_environment: LOCAL_MACHINE
  • distributed_type: NO
  • mixed_precision: fp16
  • use_cpu: False
  • debug: False
  • num_processes: 1
  • machine_rank: 0
  • num_machines: 1
  • gpu_ids: all
  • rdzv_backend: static
  • same_network: True
  • main_training_function: main
  • enable_cpu_affinity: False
  • downcast_bf16: no
  • tpu_use_cluster: False
  • tpu_use_sudo: False
  • tpu_env: []
• DeepSpeed version: not installed
• PyTorch version (accelerator?): 2.9.1+cu129 (CUDA)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using distributed or parallel set-up in script?: no
• Using GPU in script?: no

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Run code like this:

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained('/path/to/local_model/')

and see this in the output:

The module name  (originally ) is not a valid Python identifier. Please rename the original module to avoid import issues.

Expected behavior

The model should load without a warning.

Removing the trailing slash from the model path makes the warning go away. It also does not reproduce, with or without the slash, in an older venv I have at transformers 4.55.3

[Rocketknight1]

The fact that this is a recent issue is interesting - can someone do a git bisect to find out when it appeared?

[khushali9]

Sure @Rocketknight1 I can do git bisect.

[khushali9]

@Rocketknight1 I tried bisect and set good bad , but problem is my test script has few imports like this from transformers.models.auto.tokenization_auto import AutoTokenizer raise ImportError(
ImportError: tokenizers>=0.21,<0.22 is required for a normal functioning of this module, but found tokenizers==0.22.1.
Try: pip install transformers -U or pip install -e '.[dev]'.

Can you help how to go about this ?

[khushali9]

@Rocketknight1 Or shall I write .py file to test the bisect, and avoid import errors like how I did in my previous PR in transformers.

[Rocketknight1]

Hi @khushali9, yes, this can be annoying. Older versions of transformers need specific versions of tokenizers and huggingface_hub. To fix this, after each step of git bisect, do pip install -e .. This will install the right dependency versions and then you can run the test.

[khushali9]

@Rocketknight1 Thanks that info helped, so I did get this in bisect output as following :

This means the bug has been fixed between c54203a and [170b270].

But to test it with slash in the end, I tried to run my test for both commit and got same error for 'local_model/', shouldnt it work for good commit which is 170b270.

EXCEPTION: Repo id must use alphanumeric chars, '-', '_' or '.'. The name cannot start or end with '-' or '.' and the maximum length is 96: 'local_model/'.

attaching my test file.

bisect_test.py

[Rocketknight1]

Wait hang on, I just tested this myself on the latest version from main and it's working with a trailing slash! Is this actually still a live issue?

[kk-89]

It was still broken in 4.57.3 but I just tried in a test venv with 5.0.0.dev0 and there is no warning.

[khushali9]

Yeah I did not test this on current version , and tested now and it is working without warning. Should have done this first. Yaay! @Rocketknight1 @kk-89