Agenda 2025-08-26

[jcscottiii]

Notes from https://docs.google.com/document/d/1N9wdlGCOIlBo7_Dr7dcsgO4GIFPC8dnlb8qo_MaF97s/edit?tab=t.0#heading=h.za7rozepa87c

Aug 26, 2025 | Interop Mobile Testing

Attendees: @jcscottiii @jgraham @gsnedders

Notes

• Comment about score: Agenda 2025-06-10 #36 (comment)
  • Wpt.fyi PR to go out in next release: Update the score for the mobile testing investigation area wpt.fyi#4523
• Firefox Android: Completed successfully. But did not figure out the version.
• Chrome on iOS: Last run on Aug 20. Will reach out to the Eng Prod team
• Carryover] Have reliable iOS runs. Have stable Firefox (currently only experimental). Have stable Chrome (currently only experimental)
  • Reliable iOS runs:
    • No hard blockers on this.
    • Need to work on this
  • Firefox Stable Results
    • Supposed to trigger every day. May be related to the failures we are seeing. GeckoViewTest runner. Not full firefox.
  • Chrome Android stable
    • Still working.
  • Takeaway: keep the score the same (10%)
• Ensure testharness test results are meaningful.
  • James G: They are meaningful for some tests in Gecko. But not sure we have concerns around DPR
  • Sam: No concerns for testharness tests for DPR.
  • James G: Testharness tests could exclude problematic tests. But if we expected all the touch event tests to pass but they don't. That would be bad.
  • Sam: Need to check on one bug. May have broken interaction stuff on iOS.
  • James G: My feeling about the scoring for this one. Here are the hypothetical touch event tests. And we have the results for all browsers. We just need iOS results to score this completely. For now score this at 15% (half of 30%)
  • Takeaway: Increase to 15%
• Figure out how to make layout tests work on mobile. In particular, reftests
  • James G: Sam has done a lot of work on this.
  • Sam: In regards to DPR, this got left a month ago. Running Firefox and Chrome at different DPRs. The dynamic viewport sizing thing is the smaller problem here. Could do evil things like injecting meta name=viewport in the DOM.
  • James G: Isn't it around when the keyboard pops up. There are two problems. DPR and viewport size. Gecko has a special layout for reftests. But we are hoping that the tests were basically functional with different DPRs. tests that indicate it works on specific ratios we could skip or enforce. For viewport size, we talked about running tests in iframes.
  • Sam: Being able to switch between small and large browser UIs. Current Gecko approach does not not work because you don't have the dynamic viewport stuff
  • James G: Getting the existing ref tests to work on mobile. For certain tests on platforms that support changing the DPR. Viewport size: How to handle the ref tests required for the viewport size without custom internal APIs. With most ref tests they don't make the keyboard appear.
  • Sam: Might be reasonable to get the majority of the reftests to work without worrying about that
  • James G: The only thing we have done is work Sam has done for DPR. How close do you feel you are in the research
  • Sam: Need to kick off some of the runs and look through what actually changes. We expect an order of three or four digits of tests to change. Which is a lot.
  • James G: Not sure it's fair to split this into half. 5% understanding the problem, 5% fixing the tests, 5% hard code to run to specific pixel ratio. If we had the list, 5 out of 30. If we have done some work to fix it, 10 out of 30. Maybe 3 out of 30.
  • Takeaway: 3%
• Figure out the requirements for testing dynamic viewport sizing. Suggest a viable approach if possible.
  • James G: We have an idea of the plan. But not much more to justify more than a point or two.
  • Sam: We should create an issue on the repo and throw in the list of what we believe the problems are. Then we can sketch out.
  • James G: Makes sense. Keep this at 0%. Then if we have an issue on repo (ex. WebDriver repo), this would be done. Assuming we outlined the requirements in that issue
  • Takeaway: 2%
• Total score: 30%
• Non mobile note: Edge is missing. Microsoft is investigating this issue.
  • Sam: The old Windows Server images weren't meant to go away. But haven't
  • James G: The logs have a warning that mentions the deprecation notes.
  • Sam: Looks like the OS signals are failing to send.
  • James G: Let Microsoft continue their investigation. And we should sync back on this then.

Action items

1. Show the stable runs on the ecosystem dashboard for mobile - James S
2. Update the score on wpt.fyi with the score - James S
3. File an issue for dynamic viewport sizing - Sam