Fixed on Android, still open on iOS: what retesting 13 medium findings showed
For product owners and CTOs who ship one app on two platforms.
Your developers say a finding is fixed. We retested 13 of them on both platforms, and several fixes had reached only one.
- Public safety
- SOS mobile app on Android and iOS
- Clients
- Reputation
- Money
What was at risk
Keys and data wiped without a word
The keystore failure that silently wiped keys and the database was fixed on Android and is still open on iOS, so an iOS user can still lose their keys and local data with no warning.
A fix that reached one platform
Two of the five findings graded fixed were fixed on Android only. A report that grades each finding for the platform it named would have called them closed while iOS users stayed exposed.
A fix that did not build
One iOS fix had correct logic, but the tag it relied on does not compile. It reads as done in review and fixes nothing until someone runs the compiler, so the hole looks shut while it is open.
Standards and rules in play
- Evidence gradesVerified: we read the line or ran the command. Inferred: one step follows from what we read. Unverified: we could not check.
- Grade against our findingA developer's note is an implementation note, not a closure. Where a note scoped a fix more narrowly than our finding, we graded against our finding.
Context: apps that hold emergency, health and location data are expected to meet GDPR security duties.
How we tested it
- Test level
- Static review of the second beta's source, plus three runs: a published build downloaded and inspected, a live file fetched, and the compiler's type check.
- Test types
- Security testing, and confirmation testing across Android and iOS.
- Test basis
- The 13 medium findings of the first beta, each graded against the finding as written.
- How we workedCode reviewWhere it showed upEvery finding was re-read in the Android source and in the iOS source, so a fix on one platform was not read as a fix on both.
- How we workedRunning the compilerWhere it showed upThe compiler's type check, a cheap command that a review of the diff would not run, showed that an iOS fix did not compile.
Terms as in the ISTQB CTFL 4.0.1 syllabus and glossary.
What we found, and what it costs
- FindingOpen on iOSThe keystore failure that silently wiped keys and the database: fixed on Android onlyWhat it costs youiOS users can lose keys and data without warning.
- FindingOpen on iOSAn expired cached certificate still unlocked rescue mode offline: fixed on Android onlyWhat it costs youOffline access that does not expire.
- FindingPartly fixedThe pairing presence oracle with no rate limit: partly fixed on Android, not fixed on iOSWhat it costs youSomeone can keep probing whether a device is present.
- FindingCannot verifyAn iOS fix for a malformed pairing message had correct logic, but its tag does not compileWhat it costs youA fix that does not build fixes nothing.
Totals for the 13: 5 fixed, 6 partly fixed, 1 not fixed, 1 cannot be verified from code. A debug certificate in a live verification file was confirmed fixed by fetching the file.
What changed
Each verdict names its platform, so a fix that reached only one shows as open on the other. Every verdict carries an evidence grade.
Shipping on Android and iOS?
We retest every platform and label what we verified. The first month is a trial.
