-
Notifications
You must be signed in to change notification settings - Fork 13
chore: fix a bug in flaky move test #720
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
FWIW I've been rerunning the tests and the latest failure was a different flaky test, not this one :) |
cpcallen
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch on the unwanted m—and thanks for educating me about the existence of these various fancy assertions. This is definitely an improvement.
test/webdriverio/test/move_test.ts
Outdated
| const bubble = this.browser.$('.blocklyMoveIndicatorBubble'); | ||
| await bubble.waitForExist(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If bubble isn't used again then why save it? (Unless line length?)
| const bubble = this.browser.$('.blocklyMoveIndicatorBubble'); | |
| await bubble.waitForExist(); | |
| this.browser.$('.blocklyMoveIndicatorBubble').waitForExist(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, I had remembered running into issues with doing this previously but it was when I needed to assert something about bubble which I'm not doing in this case. Fixed (had to add await so didn't commit your suggestion but fixed it locally and pushed)
|
Apologies for the delay--thanks for reviewing this @cpcallen! |
Fixes #719 hopefully?
There was a bug in this test, it both opened the context menu and pressed 'm' to start move mode. But that wasn't enough to fix the flaky test.
I changed some of the assertions to be more specific. using
chai.assert.isNullgives a better error message thanchai.assertas chai knows more about what you're intending to check. the old error didn't tell you what the value was, which could help with debugging. the new message will say something like "expected null, got 123". That did give better error messages but still didn't fix anything (it wasn't supposed to)Next I tried adding an assertion to make sure the problem wasn't with the context menu showing up. I added a check to make sure that the "Move" option was found, and that assertion never failed. Yet, the test flaked on.
I tried adding some pause time and still flaky, so removed it again.
I tried triggering the move with the 'm' key instead of the context menu and it was still flaky, so I reverted those commits.
Finally I added a "wait for exist" on the move icon, so that the test doesn't continue until move mode is actually activated. This required adding a class to the actual source code. All other changes are to the test code only.
At this point I don't know if it will solve the flaky test but even if it doesn't I think it's worth merging at this point as it does improve the tests.