Address flaky instrumentation tests#3754
Conversation
📝 WalkthroughWalkthroughThis change modifies Estimated code review effort🎯 1 (Trivial) | ⏱️ ~5 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/instrumentation-tests/src/org/commcare/androidTests/CaseClaimTest.java`:
- Around line 131-132: The current wait uses waitForView(withText(name)) via
onView(isRoot()) which can match the search EditText; change the matcher to
exclude EditText so it targets the result row (e.g., combine with
not(isAssignableFrom(EditText.class)) or equivalent) and use that same matcher
both in the waitForView(...) call and when performing the subsequent click so
you synchronize on and interact with the actual case result rather than the
search field.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 24419774-58eb-4cb6-8ff8-68fe080fdd5c
📒 Files selected for processing (1)
app/instrumentation-tests/src/org/commcare/androidTests/CaseClaimTest.java
| onView(isRoot()) | ||
| .perform(waitForView(withText(name), 10000, true)); |
There was a problem hiding this comment.
Wait condition is too broad and can pass before results load.
waitForView(withText(name), ...) can match the already-filled search EditText, so this may not actually wait for the result row. Use a matcher that excludes EditText (and reuse it for the click) so synchronization targets the case result itself.
Proposed fix
+import static org.hamcrest.Matchers.not;
@@
- onView(isRoot())
- .perform(waitForView(withText(name), 10000, true));
- onView(withText(name))
+ onView(isRoot())
+ .perform(waitForView(allOf(withText(name), not(withClassName(endsWith("EditText")))), 10000, true));
+ onView(allOf(withText(name), not(withClassName(endsWith("EditText")))))
.perform(click());🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@app/instrumentation-tests/src/org/commcare/androidTests/CaseClaimTest.java`
around lines 131 - 132, The current wait uses waitForView(withText(name)) via
onView(isRoot()) which can match the search EditText; change the matcher to
exclude EditText so it targets the result row (e.g., combine with
not(isAssignableFrom(EditText.class)) or equivalent) and use that same matcher
both in the waitForView(...) call and when performing the subsequent click so
you synchronize on and interact with the actual case result rather than the
search field.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3754 +/- ##
============================================
- Coverage 26.52% 26.51% -0.02%
+ Complexity 4526 4523 -3
============================================
Files 964 965 +1
Lines 57703 57724 +21
Branches 6871 6873 +2
============================================
- Hits 15307 15303 -4
- Misses 40522 40546 +24
- Partials 1874 1875 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
df088d8 to
d340d76
Compare
f957af2 to
21690c3
Compare
…re login offline" This reverts commit 216ac7e.
0ef9508 to
d56c6fc
Compare
fd291b1 to
38ec167
Compare
2601096 to
b412597
Compare
Technical Summary
Addresses recurring flakiness in the instrumentation (BrowserStack) test suite. Each fix targets state that leaked between tests or a flaky UI assertions:
InstrumentationUtility.changeWifi()now also toggles thewifi_wakeupglobal setting alongsidesvc wifi enable/disable. Without disabling Wi-Fi wakeup, the OS could silently re-enable Wi-Fi (auto-scan/reconnect) mid-test, breaking offline scenarios.DemoUserOfflineTestandManualQuarantineTest— add an@Afterteardown that re-enables Wi-Fi (changeWifi(true)), so a test that runs offline restores network state and doesn't poison subsequent tests.SessionExpirationTests— dismiss the notification drawer withpressBack()after verifying the login-expired notification, so the open drawer doesn't interfere with the next test.InstallFromListTest— make the "Install An App" matcher intearDowncase-insensitive (equalToIgnoringCase), since the label's casing varies and an exact match intermittently failed to find the button.Safety Assurance
Safety story
Test-only change — no production/application code is touched (all changes are under
app/instrumentation-tests/). There is no impact on app behavior or existing data; the blast radius is limited to the CI/instrumentation suite. Confidence comes from re-running the previously flaky tests and getting a successful build during the first attempt: https://app-automate.browserstack.com/projects/CommCare/builds/Build+app-commcare-release+apk/2559Automated test coverage
This PR stabilizes existing instrumentation coverage rather than adding new logic — the affected tests (
DemoUserOfflineTest,ManualQuarantineTest,SessionExpirationTests,InstallFromListTest) continue to assert the same behavior, now with deterministic setup/teardown of Wi-Fi and notification-drawer state.Labels and Review