-
Notifications
You must be signed in to change notification settings - Fork 22
2025‐08‐14 Invalid Data in Production Database
Date: 2025-08-14
A test result saving issue was revealed by https://github.com/w3c/aria-at-app/issues/1489.
Consider the following assertion and scenario which describes an exception set to EXCLUDE the assertion for the down down command:
expand assertion
{
"id": "NDllMeyIzIjoiWVdRMk1leUl5SWpvaU1UWXpOalV4SW4wVGRtTTIifQ2RiMj",
"priority": "MAY",
"rawAssertionId": "tabPanelText",
"assertionPhrase": "convey text of tab panel, 'Peter Erasmus Lange-Müller (1 December 1850 – 26 February 1926) was a Danish composer and pianist. His compositional style was influenced by Danish folk music and by the work of Robert Schumann; Johannes Brahms; and his Danish countrymen, including J.P.E. Hartmann.'",
"assertionStatement": "Text of the tab panel, 'Peter Erasmus Lange-Müller (1 December 1850 – 26 February 1926) was a Danish composer and pianist. His compositional style was influenced by Danish folk music and by the work of Robert Schumann; Johannes Brahms; and his Danish countrymen, including J.P.E. Hartmann.', is conveyed",
"assertionExceptions": [
{
"priority": "EXCLUDE",
"settings": "virtualCursor",
"commandId": "down down"
}
],
"text": "Text of the tab panel, 'Peter Erasmus Lange-Müller (1 December 1850 – 26 February 1926) was a Danish composer and pianist. His compositional style was influenced by Danish folk music and by the work of Robert Schumann; Johannes Brahms; and his Danish countrymen, including J.P.E. Hartmann.', is conveyed",
"phrase": "convey text of tab panel, 'Peter Erasmus Lange-Müller (1 December 1850 – 26 February 1926) was a Danish composer and pianist. His compositional style was influenced by Danish folk music and by the work of Robert Schumann; Johannes Brahms; and his Danish countrymen, including J.P.E. Hartmann.'"
}expand scenario
{
"id": "NWQ2ZeyIzIjoiWVdRMk1leUl5SWpvaU1UWXpOalV4SW4wVGRtTTIifQDZhMj",
"atId": 1,
"settings": "virtualCursor",
"commandIds": [
"down",
"down"
],
"at": {
"id": 1,
"name": "JAWS",
"key": "jaws",
"vendorId": 1,
"defaultConfigurationInstructionsHTML": "Configure JAWS with default settings. For help, read <a href="https://github.com/w3c/aria-at/wiki/Configuring-Screen-Readers-for-Testing">Configuring Screen Readers for Testing</a>.",
"assertionTokens": {
"screenReader": "JAWS",
"interactionMode": "PC cursor active",
"readingMode": "virtual cursor active",
"readingCursor": "virtual cursor"
},
"settings": {
"virtualCursor": {
"screenText": "virtual cursor active",
"instructions": [
"Press <kbd>Alt</kbd>+<kbd>Delete</kbd> to determine which cursor is active.",
"If the PC cursor is active, press <kbd>Escape</kbd> to activate the virtual cursor."
]
},
"pcCursor": {
"screenText": "PC cursor active",
"instructions": [
"Press <kbd>Alt</kbd>+<kbd>Delete</kbd> to determine which cursor is active.",
"If the virtual cursor is active, press <kbd>Insert</kbd>+<kbd>z</kbd> to disable the virtual cursor."
]
}
}
},
"commands": [
{
"id": "down",
"text": "Down Arrow (virtual cursor active)",
"atOperatingMode": "virtualCursor"
},
{
"id": "down",
"text": "Down Arrow (virtual cursor active)",
"atOperatingMode": "virtualCursor"
}
]
}The hasExceptionWithPriority function would never work in this case because exception.commandId expects down down but the commands array would only ever produce down when being looped across. This would prevent a testResult including such a condition from being saved.
That function should operate similar to the assertionResultsResolver.
This was never caught because it would only happen in the case of a then-able command (eg. key1 key2 -> Key 1 then Key 2, down down -> Down Arrow then Down Arrow) having an exception set to EXCLUDE. After going through all the *-commands.csv's in w3c/aria-at, this pattern had the first instance of this occurrence. Other cases used key1 or key1+key2 which would have been valid.
#1491 ensures the then-able command is properly handled.
As of today (August 14, 2025), there are 2 known report submissions that were affected and have polluted to currently live data. These 2 items can be addressed in a one-off query without any further migration:
LouisDo10's 980 run, test 13 for 'Tabs with Automatic Activation V25.07.30 + JAWS + Chrome':
update "TestPlanRun"
set "testResults" = ( select jsonb_agg(case when test ->> 'testId' = 'YWQ2MeyIyIjoiMTYzNjUxIn0TdmM2' then jsonb_set(
test, array ['scenarioResults'], ( select jsonb_agg(jsonb_set(scenario, array ['assertionResults'],
( select coalesce(jsonb_agg(assertion), '[]'::jsonb)
from jsonb_array_elements(scenario -> 'assertionResults') as assertion
where assertion ->> 'id' !=
'MDExNeyIxNCI6Ik4yRTNaZXlJeE15STZJazU2Vm1oT1pYbEplRTFwU1RaUFZHZDNabEZYVVROT1ZDSjlHTmxNVCJ9DAzYT' )))
from jsonb_array_elements(test -> 'scenarioResults') as scenario ))
else test end)
from jsonb_array_elements("testResults") as test )
where id = 980;update "TestPlanRun"
set "testResults" = ( select jsonb_agg(case when test ->> 'testId' = 'OTg0OeyIyIjoiMTYzNjUxIn0TQzZj' then jsonb_set(
test, array ['scenarioResults'], ( select jsonb_agg(jsonb_set(scenario, array ['assertionResults'],
( select coalesce(jsonb_agg(assertion), '[]'::jsonb)
from jsonb_array_elements(scenario -> 'assertionResults') as assertion
where assertion ->> 'id' !=
'NzZlYeyIxNCI6IlpXRXhOZXlJeE15STZJbGt5U1RWUFpYbEplRTFwU1RaUFZHdDRabEZIU1RGWlZ5Sjl6UmpZeiJ9jFiNG' )))
from jsonb_array_elements(test -> 'scenarioResults') as scenario ))
else test end)
from jsonb_array_elements("testResults") as test )
where id = 991;