Integration testing updates #2302
Conversation
| runs-on: ${{ matrix.sys }} | ||
| env: | ||
| TEST_THREADS: ${{ contains(matrix.sys, 'macos') && '--test-threads=1' || '' }} # macOS has limited resources, so we run tests (that rely on `testcontainers`) with 1 thread | ||
| TEST_THREADS: ${{ contains(matrix.sys, 'macos') && '2' || '4' }} # macOS has limited resources |
There was a problem hiding this comment.
The macos-15 intel image has enough room to run multi-threaded, and this puts a cap on ubuntu images to help reproduce any errors due to too many tests running at once against a single test source
| # Only run Linux x64 on non-release PRs to save CI minutes | ||
| - sys: ${{ github.event_name != 'push' && !startsWith(github.ref_name, 'release/') && 'ubuntu-jammy-8-cores-arm64' }} | ||
| - sys: ${{ github.event_name != 'push' && !startsWith(github.ref_name, 'release/') && 'macos-15-large' }} |
There was a problem hiding this comment.
Thoughts on only running all 3 container types on release branches? It might not be necessary for main pushes / feature PRs
|
|
||
| #[tokio::test] | ||
| #[ignore] | ||
| async fn burn() { |
There was a problem hiding this comment.
this test didn't appear to cover any additional usage cases that aren't already covered by hello_world
There was a problem hiding this comment.
relocated to where the relevant code is: soroban-spec-tools
| #[tokio::test] | ||
| #[ignore] | ||
| async fn half_max_instructions() { | ||
| let sandbox = TestEnv::new(); | ||
| let wasm = HELLO_WORLD; | ||
| sandbox | ||
| .new_assert_cmd("contract") | ||
| .arg("deploy") | ||
| .arg("--fee") | ||
| .arg("1000000") | ||
| .arg("--instructions") | ||
| .arg((u32::MAX / 2).to_string()) | ||
| .arg("--wasm") | ||
| .arg(wasm.path()) | ||
| .arg("--ignore-checks") | ||
| .assert() | ||
| .stderr("") | ||
| .stdout_as_str(); | ||
| } |
There was a problem hiding this comment.
wasn't able to determine what this test was doing, opted to remove it. Looks like it had been marked #[ignore] since it got added.
mootz12
force-pushed
the
integration-testing-updates
branch
from
695ad2a to
1b6230c
Compare
mootz12
force-pushed
the
integration-testing-updates
branch
from
3edbddc to
cb488ff
Compare
mootz12
force-pushed
the
integration-testing-updates
branch
from
cb488ff to
ffebf0a
Compare
mootz12
force-pushed
the
integration-testing-updates
branch
from
ffebf0a to
cf08a22
Compare
What
Minor improvements to RPC tests, (hopefully) patching secure store intermittent CI failures, and allowing all container types to run on release PR updates.
Why
Failure rate of RPC tests is ~30% in CI pipelines, and a few tests have remained ignored without issues
Known limitations
Slightly increase CI minutes as now 3 container types will run RPC tests on release PRs as well