Skip to content

Commit dcd0743

Browse files
freakboy3742freakboy3742
committed
Switch to using test-runtime.
1 parent 4051400 commit dcd0743

File tree

11 files changed

+64
-63
lines changed

11 files changed

+64
-63
lines changed

‎.github/workflows/test.yml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -80,12 +80,12 @@ jobs:
8080
python_version: '3.13'
8181
test_select: ios
8282
# Exercise iOS on a non-default simulator.
83-
test_execution: 'args: --simulator "iPhone 16e,OS=18.5"'
83+
test_runtime: 'args: --simulator "iPhone 16e,OS=18.5"'
8484
- os: macos-15-intel
8585
python_version: '3.13'
8686
test_select: android
8787
# Exercise Android on a non-default simulator
88-
test_execution: 'args: --managed minVersion'
88+
test_runtime: 'args: --managed minVersion'
8989
- os: macos-15
9090
python_version: '3.13'
9191
test_select: android
@@ -158,7 +158,7 @@ jobs:
158158
CIBW_ARCHS_MACOS: x86_64 universal2 arm64
159159
CIBW_BUILD_FRONTEND: ${{ matrix.test_select && 'build' || 'build[uv]' }}
160160
CIBW_PLATFORM: ${{ matrix.test_select }}
161-
CIBW_TEST_EXECUTION: ${{ matrix.test_execution }}
161+
CIBW_TEST_RUNTIME: ${{ matrix.test_runtime }}
162162

163163
- name: Run a sample build (GitHub Action, only)
164164
uses: ./
@@ -184,7 +184,7 @@ jobs:
184184
uses: ./
185185
env:
186186
CIBW_PLATFORM: ${{ matrix.test_select }}
187-
CIBW_TEST_EXECUTION: ${{ matrix.test_execution }}
187+
CIBW_TEST_RUNTIME: ${{ matrix.test_runtime }}
188188
with:
189189
package-dir: sample_proj
190190
output-dir: wheelhouse_config_file
@@ -209,7 +209,7 @@ jobs:
209209

210210
- name: Test cibuildwheel
211211
env:
212-
CIBW_TEST_EXECUTION: ${{ matrix.test_execution }}
212+
CIBW_TEST_RUNTIME: ${{ matrix.test_runtime }}
213213
run: |
214214
uv run --no-sync bin/run_tests.py --test-select=${{ matrix.test_select || 'native' }} ${{ (runner.os == 'Linux' && runner.arch == 'X64') && '--run-podman' || '' }}
215215

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -159,13 +159,13 @@ The following diagram summarises the steps that cibuildwheel takes on each platf
159159
| | [`test-groups`](https://cibuildwheel.pypa.io/en/stable/options/#test-groups) | Specify test dependencies from your project's `dependency-groups` |
160160
| | [`test-skip`](https://cibuildwheel.pypa.io/en/stable/options/#test-skip) | Skip running tests on some builds |
161161
| | [`test-environment`](https://cibuildwheel.pypa.io/en/stable/options/#test-environment) | Set environment variables for the test environment |
162-
| | [`test-execution`](https://cibuildwheel.pypa.io/en/stable/options/#test-execution) | Controls how the tests will be executed. |
162+
| | [`test-runtime`](https://cibuildwheel.pypa.io/en/stable/options/#test-runtime) | Controls how the tests will be executed. |
163163
| **Debugging** | [`debug-keep-container`](https://cibuildwheel.pypa.io/en/stable/options/#debug-keep-container) | Keep the container after running for debugging. |
164164
| | [`debug-traceback`](https://cibuildwheel.pypa.io/en/stable/options/#debug-traceback) | Print full traceback when errors occur. |
165165
| | [`build-verbosity`](https://cibuildwheel.pypa.io/en/stable/options/#build-verbosity) | Increase/decrease the output of the build |
166166

167167

168-
<!--[[[end]]] (sum: /aL2w4bcMB) -->
168+
<!--[[[end]]] (sum: dbfwOkj/k/) -->
169169

170170
These options can be specified in a pyproject.toml file, or as environment variables, see [configuration docs](https://cibuildwheel.pypa.io/en/latest/configuration/).
171171

‎bin/generate_schema.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -224,7 +224,7 @@
224224
test-environment:
225225
description: Set environment variables for the test environment
226226
type: string_table
227-
test-execution:
227+
test-runtime:
228228
description: Additional configuration for the test runner
229229
oneOf:
230230
- type: string
@@ -322,7 +322,7 @@
322322
test-sources: {"$ref": "#/$defs/inherit"}
323323
test-requires: {"$ref": "#/$defs/inherit"}
324324
test-environment: {"$ref": "#/$defs/inherit"}
325-
test-execution: {"$ref": "#/$defs/inherit"}
325+
test-runtime: {"$ref": "#/$defs/inherit"}
326326
"""
327327
)
328328

‎cibuildwheel/options.py‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,7 @@ class GlobalOptions:
9393

9494

9595
@dataclasses.dataclass(frozen=True)
96-
class TestExecutionConfig:
96+
class TestRuntimeConfig:
9797
args: Sequence[str] = ()
9898

9999
@classmethod
@@ -124,7 +124,7 @@ class BuildOptions:
124124
test_extras: str
125125
test_groups: list[str]
126126
test_environment: ParsedEnvironment
127-
test_execution: TestExecutionConfig
127+
test_runtime: TestRuntimeConfig
128128
build_verbosity: int
129129
build_frontend: BuildFrontendConfig
130130
config_settings: str
@@ -776,18 +776,18 @@ def _compute_build_options(self, identifier: str | None) -> BuildOptions:
776776
msg = f"Malformed environment option {test_environment_config!r}"
777777
raise errors.ConfigurationError(msg) from e
778778

779-
test_execution_str = self.reader.get(
780-
"test-execution",
779+
test_runtime_str = self.reader.get(
780+
"test-runtime",
781781
env_plat=False,
782782
option_format=ShlexTableFormat(sep="; ", pair_sep=":", allow_merge=False),
783783
)
784-
if not test_execution_str:
785-
test_execution = TestExecutionConfig()
784+
if not test_runtime_str:
785+
test_runtime = TestRuntimeConfig()
786786
else:
787787
try:
788-
test_execution = TestExecutionConfig.from_config_string(test_execution_str)
788+
test_runtime = TestRuntimeConfig.from_config_string(test_runtime_str)
789789
except ValueError as e:
790-
msg = f"Failed to parse test execution config. {e}"
790+
msg = f"Failed to parse test runtime config. {e}"
791791
raise errors.ConfigurationError(msg) from e
792792

793793
test_requires = self.reader.get(
@@ -897,7 +897,7 @@ def _compute_build_options(self, identifier: str | None) -> BuildOptions:
897897
test_command=test_command,
898898
test_sources=test_sources,
899899
test_environment=test_environment,
900-
test_execution=test_execution,
900+
test_runtime=test_runtime,
901901
test_requires=[*test_requires, *test_requirements_from_groups],
902902
test_extras=test_extras,
903903
test_groups=test_groups,

‎cibuildwheel/platforms/android.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -641,9 +641,9 @@ def test_wheel(state: BuildState, wheel: Path) -> None:
641641
# By default, run on a testbed managed emulator running the newest supported
642642
# Android version. However, if the user specifies a --managed or --connected
643643
# test execution argument, that argument takes precedence.
644-
test_execution_args = state.options.test_execution.args
644+
test_runtime_args = state.options.test_runtime.args
645645

646-
if any(arg.startswith(("--managed", "--connected")) for arg in test_execution_args):
646+
if any(arg.startswith(("--managed", "--connected")) for arg in test_runtime_args):
647647
emulator_args = []
648648
else:
649649
emulator_args = ["--managed", "maxVersion"]
@@ -658,7 +658,7 @@ def test_wheel(state: BuildState, wheel: Path) -> None:
658658
cwd_dir,
659659
*emulator_args,
660660
*(["-v"] if state.options.build_verbosity > 0 else []),
661-
*test_execution_args,
661+
*test_runtime_args,
662662
"--",
663663
*test_args,
664664
env=state.build_env,

‎cibuildwheel/platforms/ios.py‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -654,7 +654,7 @@ def build(options: Options, tmp_path: Path) -> None:
654654
)
655655
raise errors.FatalError(msg)
656656

657-
test_execution_args = build_options.test_execution.args
657+
test_runtime_args = build_options.test_runtime.args
658658

659659
# 2025-10: The GitHub Actions macos-15 runner has a known issue where
660660
# the default simulator won't start due to a disk performance issue;
@@ -668,21 +668,21 @@ def build(options: Options, tmp_path: Path) -> None:
668668
and os_version.startswith("15.")
669669
and arch == "arm64"
670670
and not any(
671-
arg.startswith("--simulator") for arg in test_execution_args
671+
arg.startswith("--simulator") for arg in test_runtime_args
672672
)
673673
):
674-
test_execution_args = [
674+
test_runtime_args = [
675675
"--simulator",
676676
"iPhone 16e,OS=18.5",
677-
*test_execution_args,
677+
*test_runtime_args,
678678
]
679679

680680
call(
681681
"python",
682682
testbed_path,
683683
"run",
684684
*(["--verbose"] if build_options.build_verbosity > 0 else []),
685-
*test_execution_args,
685+
*test_runtime_args,
686686
"--",
687687
*final_command,
688688
env=test_env,

‎cibuildwheel/resources/cibuildwheel.schema.json‎

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -569,13 +569,9 @@
569569
],
570570
"title": "CIBW_TEST_ENVIRONMENT"
571571
},
572-
"test-execution": {
572+
"test-runtime": {
573573
"description": "Additional configuration for the test runner",
574574
"oneOf": [
575-
{
576-
"type": "string",
577-
"pattern": "args:"
578-
},
579575
{
580576
"type": "string",
581577
"pattern": "^$"
@@ -584,6 +580,10 @@
584580
"type": "object",
585581
"additionalProperties": false
586582
},
583+
{
584+
"type": "string",
585+
"pattern": "args:"
586+
},
587587
{
588588
"type": "object",
589589
"additionalProperties": false,
@@ -600,7 +600,7 @@
600600
}
601601
}
602602
],
603-
"title": "CIBW_TEST_EXECUTION"
603+
"title": "CIBW_TEST_RUNTIME"
604604
},
605605
"overrides": {
606606
"type": "array",
@@ -672,7 +672,7 @@
672672
"test-environment": {
673673
"$ref": "#/$defs/inherit"
674674
},
675-
"test-execution": {
675+
"test-runtime": {
676676
"$ref": "#/$defs/inherit"
677677
}
678678
}
@@ -785,8 +785,8 @@
785785
"test-environment": {
786786
"$ref": "#/properties/test-environment"
787787
},
788-
"test-execution": {
789-
"$ref": "#/properties/test-execution"
788+
"test-runtime": {
789+
"$ref": "#/properties/test-runtime"
790790
}
791791
}
792792
}
@@ -916,8 +916,8 @@
916916
"test-environment": {
917917
"$ref": "#/properties/test-environment"
918918
},
919-
"test-execution": {
920-
"$ref": "#/properties/test-execution"
919+
"test-runtime": {
920+
"$ref": "#/properties/test-runtime"
921921
}
922922
}
923923
},
@@ -979,8 +979,8 @@
979979
"test-environment": {
980980
"$ref": "#/properties/test-environment"
981981
},
982-
"test-execution": {
983-
"$ref": "#/properties/test-execution"
982+
"test-runtime": {
983+
"$ref": "#/properties/test-runtime"
984984
}
985985
}
986986
},
@@ -1055,8 +1055,8 @@
10551055
"test-environment": {
10561056
"$ref": "#/properties/test-environment"
10571057
},
1058-
"test-execution": {
1059-
"$ref": "#/properties/test-execution"
1058+
"test-runtime": {
1059+
"$ref": "#/properties/test-runtime"
10601060
}
10611061
}
10621062
},
@@ -1118,8 +1118,8 @@
11181118
"test-environment": {
11191119
"$ref": "#/properties/test-environment"
11201120
},
1121-
"test-execution": {
1122-
"$ref": "#/properties/test-execution"
1121+
"test-runtime": {
1122+
"$ref": "#/properties/test-runtime"
11231123
}
11241124
}
11251125
},
@@ -1181,8 +1181,8 @@
11811181
"test-environment": {
11821182
"$ref": "#/properties/test-environment"
11831183
},
1184-
"test-execution": {
1185-
"$ref": "#/properties/test-execution"
1184+
"test-runtime": {
1185+
"$ref": "#/properties/test-runtime"
11861186
}
11871187
}
11881188
},
@@ -1244,8 +1244,8 @@
12441244
"test-environment": {
12451245
"$ref": "#/properties/test-environment"
12461246
},
1247-
"test-execution": {
1248-
"$ref": "#/properties/test-execution"
1247+
"test-runtime": {
1248+
"$ref": "#/properties/test-runtime"
12491249
}
12501250
}
12511251
}

‎cibuildwheel/resources/defaults.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ test-requires = []
2525
test-extras = []
2626
test-groups = []
2727
test-environment = {}
28-
test-execution = {}
28+
test-runtime = {}
2929

3030
container-engine = "docker"
3131

‎docs/options.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1672,16 +1672,16 @@ Platform-specific environment variables are also available:<br/>
16721672
CIBW_TEST_ENVIRONMENT: PYTHONSAFEPATH=1
16731673
```
16741674

1675-
### `test-execution` {: #test-execution toml env-var }
1675+
### `test-runtime` {: #test-runtime toml env-var }
16761676

16771677
> Controls how the tests will be executed.
16781678
16791679
On desktop environments, the tests are executed on the same machine/container as the wheel was built. However on Android and iOS, the tests are run inside a virtual machine – a simulator or emulator – representing the target.
16801680

1681-
For these embedded platforms, a testbed project is used to run the tests. The `test-execution` setting can define an `args` key that defines additional arguments that will be used when starting the testbed project.
1681+
For these embedded platforms, a testbed project is used to run the tests. The `test-runtime` setting can define an `args` key that defines additional arguments that will be used when starting the testbed project.
16821682

16831683
Platform-specific environment variables are also available:<br/>
1684-
`CIBW_TEST_EXECUTION_ANDROID` |`CIBW_TEST_EXECUTION_IOS`
1684+
`CIBW_TEST_RUNTIME_ANDROID` |`CIBW_TEST_RUNTIME_IOS`
16851685

16861686
#### Examples
16871687

@@ -1690,21 +1690,21 @@ Platform-specific environment variables are also available:<br/>
16901690
```toml
16911691
[tool.cibuildwheel.ios]
16921692
# Run the tests on an iPhone 16e simulator running iOS 18.5.
1693-
test-execution = { args = ["--simulator='iPhone 16e,OS=18.5'"] }
1693+
test-runtime = { args = ["--simulator='iPhone 16e,OS=18.5'"] }
16941694

16951695
[tool.cibuildwheel.android]
16961696
# Run the Android tests on the minimum supported Android version.
1697-
test-execution = { args = ["--managed", "minVersion"] }
1697+
test-runtime = { args = ["--managed", "minVersion"] }
16981698
```
16991699

17001700
!!! tab examples "Environment variables"
17011701

17021702
```yaml
17031703
# Run the tests on an iPhone 16e simulator running iOS 18.5.
1704-
CIBW_EXECUTION_IOS: "args: --simulator='iPhone 16e,OS=18.5'"
1704+
CIBW_TEST_RUNTIME_IOS: "args: --simulator='iPhone 16e,OS=18.5'"
17051705

17061706
# Run the Android tests on the minimum supported Android version.
1707-
CIBW_EXECUTION_ANDROID: "args: --managed minVersion"
1707+
CIBW_TEST_RUNTIME_ANDROID: "args: --managed minVersion"
17081708
```
17091709

17101710

‎docs/platforms.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -233,7 +233,7 @@ machine – for example, if you're building on an ARM64 machine, then you can te
233233
ARM64 wheel. Wheels of other architectures can still be built, but testing will
234234
automatically be skipped.
235235

236-
Any arguments specified using [`test-execution`](options.md#test-execution) will be passed as arguments to the Python script that starts the [testbed project](https://github.com/python/cpython/blob/main/Android/README.md#testing). cibuildwheel will automatically start the testbed project with `--site-packages` and `--cwd` arguments matching your test environment, as well as enabling verbose output with `-v` if [`build_verbosity`](options.md#build_verbosity) is enabled. The most common additional arguments to use will be `--managed minVersion` or `--managed maxVersion`, specifying the use of a managed Android emulator with the minimum or maximum supported Android version; or `--connected <serial>`, specifying the use of an existing booted Android emulator or device. By default, the testbed project will run with `--managed maxVersion`.
236+
Any arguments specified using [`test-runtime`](options.md#test-runtime) will be passed as arguments to the Python script that starts the [testbed project](https://github.com/python/cpython/blob/main/Android/README.md#testing). cibuildwheel will automatically start the testbed project with `--site-packages` and `--cwd` arguments matching your test environment, as well as enabling verbose output with `-v` if [`build-verbosity`](options.md#build-verbosity) is enabled. The most common additional arguments to use will be `--managed minVersion` or `--managed maxVersion`, specifying the use of a managed Android emulator with the minimum or maximum supported Android version; or `--connected <serial>`, specifying the use of an existing booted Android emulator or device. By default, the testbed project will run with `--managed maxVersion`.
237237

238238
Running an emulator requires the build machine to either be bare-metal or support
239239
nested virtualization. CI platforms known to meet this requirement are:
@@ -324,4 +324,4 @@ The iOS test environment can't support running shell scripts, so the [`test-comm
324324

325325
The test process uses the [same testbed used by CPython itself](https://github.com/python/cpython/tree/main/Apple/iOS#testing-python-on-ios) to run the CPython test suite. It is an Xcode project that has been configured to have a single Xcode "XCUnit" test - the result of which reports the success or failure of running `python -m <test-command>`.
326326

327-
Any arguments specified using [`test-execution`](options.md#test-execution) will be passed as arguments to the Python script that starts the testbed project. The testbed project will be started with `-v` enabling verbose output if [`build_verbosity`](options.md#build_verbosity) is enabled; the most common additional argument to use will be `--simulator`, which allows the specification of a specific device or iOS version for the test simulator. By default, the testbed project will attempt to find an "SE class" simulator (i.e., an iPhone SE, iPhone 16e, or similar), running the newest iOS version available.
327+
Any arguments specified using [`test-runtime`](options.md#test-runtime) will be passed as arguments to the Python script that starts the testbed project. The testbed project will be started with `-v` enabling verbose output if [`build-verbosity`](options.md#build-verbosity) is enabled; the most common additional argument to use will be `--simulator`, which allows the specification of a specific device or iOS version for the test simulator. By default, the testbed project will attempt to find an "SE class" simulator (i.e., an iPhone SE, iPhone 16e, or similar), running the newest iOS version available.

0 commit comments

Comments
 (0)