git: Exit clearly when in a "dubious" directory

[ianw]

I hit a very hard to diagnose issue today after switching part of a CI environment. It turns out that due to non-privileged container magic the CI step was not running as the user that cloned the repo. This manifested as the sdist generated during this step just not having a few extra data files; but the build still passed. So several steps later, suddenly files have just vanished from the sdist, which of course is not something that is checked for and so testing blows up in very strange ways.

After I understood what was going on, there is a tiny little message hidden amongst the logs that gives a hint of what's going on.

...
writing requirements to src/proj.egg-info/requires.txt 
writing top-level names to src/proj.egg-info/top_level.txt 
listing git files failed - pretending there aren't any 
reading manifest file 'src/proj.egg-info/SOURCES.txt' 
writing manifest file 'src/proj.egg-info/SOURCES.txt' 
...

What is actually happening is that if you run git you get git status fatal: detected dubious ownership in repository at '/..proj'. This is the well-known CVE-2022-24765 issue where trusting a .git config dir from another user causes problems. In 6a3bb96 all the calls in setuptools_scm/git.py were updated to use --git-dir directly -- git will not complain if you have told it to explicitly trust the config dir like this.

However, there are calls in _file_finders/git.py to find the top-level that are not using this. It silently (modulo an easily missed log) skips adding files when this occurs.

I can not see that this would ever be the behaviour you would want. If it had of exploded telling me the git call failed, it would have short-cut all of the problem finding.

This adds an explicit match on the static part of this git failure message and raises a SystemExit if it hits. A test case that mocks such a situation is added.

Closes: #784

[RonnyPfannschmidt]

thank you for bringing this up and preparing the details

this is indeed a problematic oversight

unfortunately this direly needed correction is also something that may possibly need a opt-out, as bringing it about may break pipelines that "seemed to work"

the new behavior you added is correct, unfortunately we have to account for prior accidental miss-use that's unable to correct timely

so lets add a env var to allow the misbehavior to continue for people unable to currectly fix their pipelines in a timely fashion

however if a git config call can fix the issue as well then we may just document that in the error message instead of adding a opt out env var

i just fear that some ci envs are enough of a mess the env var will still be needed

[ianw]

unfortunately this direly needed correction is also something that may possibly need a opt-out, as bringing it about may break pipelines that "seemed to work"

Yes, I suspect you are right that people hit this and crafted unnecessary MANIFEST.in files. I've added and documented an env var.

[ianw]

however if a git config call can fix the issue as well then we may just document that in the error message instead of adding a opt out env var

The output I get in a failure case is

/tmp/proj/.tox/.pkg/lib/python3.13/site-packages/setuptools_scm/git.py:202: UserWarning: "/tmp/proj" is shallow and may cause errors
  warnings.warn(f'"{wd.path}" is shallow and may cause errors')
/tmp/proj/.tox/.pkg/lib/python3.13/site-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated
!!

        ********************************************************************************
        Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0).

        By 2026-Feb-18, you need to update your project and remove deprecated calls
        or your builds will no longer be supported.

        See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
        ********************************************************************************

!!
  corresp(dist, value, root_dir)
running egg_info
writing src/proj.egg-info/PKG-INFO
writing dependency_links to src/proj.egg-info/dependency_links.txt
writing entry points to src/proj.egg-info/entry_points.txt
writing requirements to src/proj.egg-info/requires.txt
writing top-level names to src/proj.egg-info/top_level.txt
fatal: detected dubious ownership in repository at '/tmp/proj'
To add an exception for this directory, call:

	git config --global --add safe.directory /tmp/proj
git introspection failed: fatal: detected dubious ownership in repository at '/tmp/proj'

ERROR Backend subprocess exited when trying to invoke get_requires_for_build_sdist

There's like 3 levels of warnings (the setuptools_scm warnings, deprecations from setuptools which are louder than problems, the log.warn level stuff :) But it stops and gives you a good clue what to do, so that's enough :)