Thirteen Apps on Google Play, and What Actually Broke Getting There

In July I wrote that twelve GriswoldLabs apps were live on Google Play. As of this week it’s thirteen — AV Project Pro, a cable-and-rack tool for AV installers, reached production after two months on the internal track.

The number is not the interesting part. Every one of those apps goes through the same release pipeline before it can ship: typecheck, unit tests, a policy scan, an emulator smoke run with five hundred random events, rotation and low-memory tests, an accessibility pass, and a visual review. The pipeline refuses to build a release if any gate fails. It is, by any reasonable standard, thorough.

And in the last two weeks, four real bugs reached — or nearly reached — real users through a pipeline that was green the entire time. That’s the interesting part. Each one taught something specific about what an automated test can see, and what it structurally cannot.

1. The sound meter that couldn’t hear

dB Scout is a sound level meter. Its entire value is one number on the screen, and whether that number is right.

An AV professional with an acoustic calibrator — the kind of instrument that produces exactly 94 dB and exactly 114 dB at the push of a button — tested it properly. At 94 dB it read 94. Then he switched the calibrator to 114 dB, a true twenty-decibel step, and the meter rose about three.

Every gate had passed. Of course it had: an Android emulator has no microphone. The pipeline could verify that the app launched, rendered, survived rotation, and didn’t leak memory. It could not verify the one thing the app exists to do, because that thing requires air pressure.

The fix wasn’t a better test. It was admitting the category. Measurement apps — anything whose value is a physical reading that silicon can’t fake — now carry a bench sign-off file that a human with real instruments has to fill in for each specific version. The release tooling refuses to push one of these apps to production without it. A green pipeline is necessary. For this class of app it is nowhere near sufficient, and the tooling now says so instead of relying on someone remembering.

2. The bug that was our own screenshot robot

TracePals is a handwriting-practice app for kids. Our automated screenshot capture flagged a serious defect: on the word-tracing screen, the word “jump” rendered as six evenly spaced vertical bars instead of letters. For a children’s handwriting app, that’s close to the whole product failing. It was filed as a production bug.

It wasn’t one. The capture script draws demo strokes on the canvas so the screenshot doesn’t show an empty practice sheet, and for the word screen someone had written six lazy placeholder swipes — literally six vertical lines, a hundred pixels apart. The letter and number screens looked correct because those recipes had been given real letterforms to draw. The app rendered “jump” perfectly, in every guide row, underneath our own ink.

The lesson generalizes uncomfortably: once a test harness synthesizes input, its output is indistinguishable from the app’s behavior. A screenshot can’t tell you who drew a stroke. A log can’t tell you who sent a request. Before filing a captured frame as a bug, diff it against the script that produced it. It’s a text file sitting next to the PNG.

3. The fix that was never compiled

ThoughtWell, a private journal, had a genuine crash: start recording a voice entry, press Home, the app dies. The cause was in a dependency — an audio library’s background handler called a native pause method without checking whether the recorder was actually recording — and the fix was a three-line try/catch.

I patched the library source, rebuilt, and the build succeeded. The pipeline’s own patch-verification step confirmed the patch was applied. Everything said yes.

The patch did nothing. That library ships as a prebuilt binary; the source I’d edited was never compiled into the app. The build was green because it built something else. I only caught it by disassembling the shipped binary and finding no exception handler where I’d put one.

The real fix was a build-config flag that opts that one module out of the prebuilt path — and a rule I now hold to: a native patch is verified in the bytecode, never by a green build. A build that succeeds while ignoring your change is the most convincing possible way to ship the bug you think you fixed.

4. The leak that has no fix

Several of the apps show a banner ad. Change your phone’s font size while one is open and the ad SDK leaks the entire screen — the old one is never released. Do it enough times and the app runs out of memory. This was found by an overnight automated tester that deliberately flips system settings while apps are running, which is exactly the kind of thing a human tester never thinks to do.

Four different app-side fixes were tried and measured. All four changed nothing. The leak is inside the ad SDK itself and is now reported upstream. Until it’s fixed there, the honest position is: it’s real, it’s rare in practice (font-size changes are a set-once setting), and there’s no local fix to ship. Sometimes “what actually broke” is “something we can name but not repair,” and the useful work is knowing which apps carry it.

What changed

None of this argues against the pipeline. Every gate in it earns its place, and the overnight tester found bug four precisely because it does things people don’t.

What changed is how I read a green result. A passing test is a claim about what the test could observe. The emulator can’t hear. The screenshot can’t tell whose hand held the pen. The build can’t tell you it ignored your file. For each of those, the fix was the same shape: find the surface that can observe the thing — a person with a calibrator, a diff against the recipe, a disassembler — and make the tooling route to it instead of trusting the green.

Thirteen apps is a nice number. The four bugs are the part I’d want to read.

The always-current list of what’s live is at /apps, read from the Play Console on every deploy.