Street 05 › Devlog › The iPhone-only crash our tests couldn't see
The iPhone-only crash our tests couldn't see
In October, the home page showed an error screen on an iPhone 17 Pro Max: "Unexpected glitch caught. TypeError: Importing a module script failed." Resetting the site's data didn't help. On Android and on every desktop browser, the site worked.
Why every test passed
All our automated tests drove Chrome. They play every game, audit accessibility and check page speed, but always in one browser engine. An error that only Safari's engine shows will never appear in them, however many tests you write.
What actually happened
Every code file on the site has its version in its name, so we let browsers cache them for a year. During a deploy there's a short window when a page from the old version can ask for a code file the new version has removed. Before we had an edge worker answering those requests properly, the server replied with the home page's HTML instead, and that answer was cached for a year under the code file's name.
From then on, that phone asked for a code file and got HTML, every time. Safari reports it as "Importing a module script failed", and clearing the site's data doesn't touch that cache.
The fix
- Missing code files now get a proper "not found" that is never cached.
- When a code file fails, the page, the error screen and the service worker fetch it again with the cache bypassed, and the reset button clears cached files too.
- To prove it, we built a small test server that serves HTML in place of code files, reproduced the error screen, and confirmed the recovery.
Safari in every check
The real fix was to stop testing in one engine. We added Playwright's WebKit, the engine inside Safari, set up as an iPhone. It opens the main pages and every game, taps through a few seconds of play, and fails on any error screen, failed code file or security-policy block.
Against the bad test server it failed with the same error screen as the iPhone, which is the point: it would have caught the bug. Every push now runs it on the pages and a sample of games, and every deploy runs it on all 116 games on the live site: 123 checks.
One small surprise: WebKit refuses to connect to a few ports it considers unsafe, including the one our test server first used.
What we'd tell another team
- If your users have iPhones, test in WebKit. Chrome passing tells you nothing about Safari.
- Long cache lifetimes are right for versioned files, as long as a wrong answer can never be cached.
- When a bug only happens on someone else's phone, reproduce it locally before you fix it.