Developer tools to build for PEP-11 supported platforms

[cmaloney]

For PEP 11 there's a range of hardware + OSes required and goal is to support all tier-1 platforms with prebuilt-python.

Not all contributors have a setup which gives access to all those; wondering if there's ways / tools we could use to simplify development and testing across the platforms

Some options I've encountered:

1. Cross-builds -- build from a particular host to all supported targets using a toolchain which supports cross compiling. Used for some large projects but typically doesn't use platform-native tools (ex. compile with MSVC on Windows). Still need hosts to test.
2. Local VMs -- Provide tools to start local VMs (on at least some platforms) for all supported platforms; build with platform specific/native tools inside the VMs.
3. Access to cloud VMs / resources -- Removes need to figure out / maintain how to launch different systems; increases costs and adds potential onboard/setup for access.
4. Buildbot -- Currently supported; Lets devs test changes remotely but can be difficult to debug and isolation quality varies.

[geofft]

Tier 1 is Windows i686 and x86-64, Linux x86-64 and aarch64, and macOS aarch64.

Local macOS VMs on non-Apple hardware are illegal. Local Windows VMs are legal but (mostly?) require a license. Local Linux VMs are pretty easy.

For python-build-standalone, all of these platforms are built with native builds. We do cross-builds for other platforms, and we do a tiny bit of qemu-user-static testing in uv but not in python-build-standalone itself, which is a gap. On the other hand, nobody has reported major issues with those platforms (probably usage is very minimal).

The nice thing about this list is that GitHub Actions makes runners available for all of them, so it's possible to at least do an edit-compile-test cycle on your own, although true interactive use is cumbersome.

I am under the impression that while cross-building for Windows works with mingw, it's ABI-incompatible with MSVC builds, which is what the wheels on PyPI are.

So I would recommend doing builds and tests on GHA and making sure that unprivileged PRs work right.

(Personally, I own a Mac, and I do my Linux and Windows testing on VMs in AWS, which are billed by the second and so relatively cheap for quick testing, and I also occasionally use a local Linux VM. Unfortunately Mac VMs are billed with a 24-hour minimum which makes them less cheap.)

[brettcannon]

Not all contributors have a setup which gives access to all those

Correct, but that's not a new problem for Python either. This is in no way a showstopper for this project.

[cmaloney]

Ramping up for CPython development I built a number of local tools for things like "Make a FreeBSD VM and build a CPython branch in it", "Build two branches of CPython and run PyPerformance", etc. I suspect for this in CI will need to build some new tools as well as some new tools to developing locally. What I'd like to see is if we can get that sort of tooling for common paths to exist in-tree rather than as local development scripts / workflows.

That doesn't mean other tools can't be used, just that we make the standard ones used for release as accessible as possible. Rather than document "run these 6 commands" in a README or devguide have a script which runs those 6 commands possibly with comments.

[brettcannon]

Releases are driven by https://github.com/python/release-tools and documented in https://peps.python.org/pep-0101/ . Any changes to the process will require release manager sign-off.

Specifically for relocatable builds, I'm expecting there will be a Makefile target which may call code in Tools/, but I expect it to be self-contained in the CPython repo for making the binary.