Resource files are not correctly attached while building with Nuitka

[Bonaducci]

After recent update to 3.0 I noticed that winpty no longer works in standalone/onefile compiled version with Nuitka I found that some resource files (exe and/or dll) are not attached. Nuitka is supposed to handle data files used by dependencies, so it is most likely some misconfiguration of the package which doesn't communicate which files have to be included.

For others having the issue, the workaround is to manually add files. Example from using venv:

    --include-data-files=./.venv/Lib/site-packages/winpty/*.dll=winpty/* `
    --include-data-files=./.venv/Lib/site-packages/winpty/*.exe=winpty/* `

From my understanding, the package needs to specify files that have to be shipped with it to work and Nuitka just follows that. I guess @kayhayen can explain how that works a little better. I'm posting it as an issue with pywinpty because this most likely needs a fix in package configuration.

[kayhayen]

No, we configure this in Nuitka itself, in what we call Nuitka Package Configuration and don't support a way to do this by upstreams, since when they get it wrong, or we want to change format, that's a hassle with code living forever, and often us wanting to support old versions as well. It is detailed here how it can be done: https://nuitka.net/user-documentation/nuitka-package-config.html

These are not data files, but binaries. Merely copying those is not enough, which is why data file options are not the best for them. The easiest way is to open a project is to open an issue in Nuitka repo (if not a PR), describing what's needed for the project to work, with a small reproducer that shows it, and we will then do it for you.

Right now I have fetched from your projects docs the example and created config to do it, just trying it out and will push it with the next Nuitka release. With an issue in Nuitka repo, I would trace when that happens, but you can expect 2.8 to have it which is due for the coming week. Often I do these changes as part of hotfixes, but after 2.7.16 I won't have one anymore.

[kayhayen]

As an aside, why I don't like hunting for examples by package name. The one I took from the README of the repo is crashing with bytes cannot be converted to PyString in two places with pure Python, and it doesn't work at all when I launch it, it just leaves me with an unusable terminal prompt, where even reset doesn't help. But the DLLs and EXE are there and the behaviour is equally bad for me.

[kayhayen]

Thanks for your report. This is fixed on the factory branch, which is a development version that is under rapid development. You can try it out by going here: https://nuitka.net/doc/factory.html

Feedback on whether this is working is very welcome. Please do not share plans to do it; only confirm or deny that it is working.

I am out of here, from my point of view this can be closed.

[Bonaducci]

Thanks for details. I thought that Nuitka defaults to some information provided by packages. For me the workaround worked with:

    --include-data-files=./.venv/Lib/site-packages/winpty/*.dll=winpty/ `
    --include-data-files=./.venv/Lib/site-packages/winpty/*.exe=winpty/ `

Creating minimal example would be a bit more tricky as I have this deeply nested in large project running winpty to emulate TTY terminal to capture color codes produced by gdb during automated stack trace generation. It did not produce any errors but the output was empty in onefile.
I may try the factory version later but most likely I will not have time to verify that until there's official update available when I bump up project dependencies.

As a side question, is there any difference between standalone and the content of temporary directory of onefile build with no dll option? I'm building onefile apps for distribution as portable variants with option to extract them locally for faster startup, it just copies the temporary directory content. I was wondering if that has any disadvantages.