-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Open
Labels
flakesFlakes from Continuous IntegrationFlakes from Continuous Integration
Description
Proposal
- I found some tests often failed in CI.
- To mitigate this, we could implement a retry mechanism when reading the container logs. A similar approach is already used in logs_test.go and could be adapted for these tests: https://github.com/containers/podman/blob/main/test/e2e/logs_test.go#L62-L69
Observed flaky tests
- Some test often failed in CI
- The common thing in them is that they check the string from
podman log
# podman [options] logs testPod-testCtr
[FAILED] Expected
<string>:
to contain substring
<string>: ln: /noPodsShallPass: Operation not permitted
In [It] at: /var/tmp/go/src/github.com[/containers/podman/test/e2e/play_kube_test.go:3420](https://github.com/containers/podman/blob/f47f74cad63e4a1fcb3c71a59def30d86d20ff77/test/e2e/play_kube_test.go#L3420) @ 11/12/25 15:03:55.055
Full Stack Trace
github.com/containers/podman/v6/test/e2e.init.func56.54()
/var/tmp/go/src/github.com[/containers/podman/test/e2e/play_kube_test.go:3420](https://github.com/containers/podman/blob/f47f74cad63e4a1fcb3c71a59def30d86d20ff77/test/e2e/play_kube_test.go#L3420) +0x978
Hypothesis and solution
- I suspect this is a race condition.
- The podman log command in the test executes immediately, but the container's log output (which needs to be written to a file or journald) has not yet been fully flushed and made available to the client.
- This results in the test reading an empty or incomplete log, causing the assertion to fail.
- By wrapping the log check in an Eventually block, we can reliably wait for the logs to be flushed, making the test robust. e.g.: https://github.com/containers/podman/blob/main/test/e2e/logs_test.go#L62-L69
Metadata
Metadata
Assignees
Labels
flakesFlakes from Continuous IntegrationFlakes from Continuous Integration