The Robot Uprising Has Begun!
A bulletin from the front: Brad vs. the Moen Flo Shutoff game-changing smart water device that monitors for leaks and immediately takes action to help prevent them…
Yes, we are now in that Brave New World where “AI” doesn’t understand your nascent tomatoes, but can still turn off your water
Our one-time home insurance company, Travelers, is now our ex-insurance company. We are not alone. Insurance companies these days are anxious to drop California homeowner customers. The big driver is that the climate, you see, has shifted under our feet. Places in California that used to have lots of vegetation but were too wet to burn are now dry enough to burn. And their past wetness means they still have lots of now dry vegetation that can burn.
This is a bad combination. Old fire maps are now new actuarial horror stories
Homeowners know more about their local micro‑circumstances than Travelers does. You know whether your neighbor smokes on the deck, whether the eucalyptus behind the carport has been trimmed, whether those embers last year came “uncomfortably close” or “close enough to update the go‑bag.”
Travelers does not.
Hence, unless Travelers is very confident that a particular homeowner is unusually risk averse, a homeowner’s desire for comprehensive coverage at current premiums is increasingly a signal that writing the policy will be a likely money‑loser. And raising the premiums does not help, as that concentrates your remaining customers on precisely those sitting on the nastiest tail risks. Averse selection 101, induced by global warming.
So Travelers dropped us.
We went shopping.
Farmers, bless them, was willing to pick us up.
There were, however, conditions.
First, there could be no knob‑and‑tube wiring in the house—no nineteenth‑century copper snakes lurking in the walls, waiting to turn your living room into a charcoal briquette. Fair enough. It turned out, by surprise, that we did have some residual powering the stairway chandelier and a couple of other places that had, somehow, not been touched in the more than 100 years since our then-streetcar suburb neighborhood was electrified.
Second, we had to install a remote‑monitoring, whole‑house water monitoring and leak shutoff system. The implication was: we will cover your fire risk, sort of, but for water damage you will recruit a robot to stand between your pipes and our balance sheet.
Yes: it is “AI”.
After consulting with homeowners who had walked this path before us, we got <https://shop.moen.com/pages/flo-smart-water-monitor>:
Meet the Flo Shutoff: A game-changing smart water device that monitors for leaks and immediately takes action to help prevent them…. Powerful leak protection for your home! Discover how the Flo Shutoff prevents costly water damage by detecting leaks, alerting you through the app and shutting your water off automatically…. 2 out of 5 homeowners have experienced water damage: Discover where hidden leaks could be lurking in your home…. Get peace of mind that the things you love are protected. The Flo Shutoff decreases water damage claim events by 96%, protecting everything you love from the damage and expense of water leaks…
So Tony the Plumber bolted the device to the main-house water intake. Gill Electric’s people wired it into the grid. I set up the wifi and linked it up to the could. And it then began to “learn”: to use its black-box smarts to distinguish between “normal” water use and “catastrophic leak” in real time. The thing studies our patterns and, we are told, “learns.” The ChatGPT of plumbing, but it communicates with a limited repertory of signals: a ball valve, alert emails, all-clear emails, and a smartphone app.
We were now, as the brochure put it, “protected 24/7.” And we were promised “peace of mind”. Indeed, it happily ran its “microleak” tests, and our system always passed. There was a rare stuck toilet valve that it alerted us to. But all seemed well.
However, it was merely biding its time.
About a month ago, the rainy season finished. With the rains over, we did what Californians with gardens do: we turned on the sprinkler and drip irrigation systems. This is a perfectly normal act, analogous to exhaling. It was, however, not at all normal in the training data of a machine‑learning system that has just spent thirty days observing what it thinks “water usage in this house” looks like: toilets, showers, dishwasher, the occasional laundry load, maybe an absent‑minded hose.
And so the Robot Uprising began. Most recently:
Yes: 12:50 was when the irrigation went on today. It has been doing this for a solid month. And yet Flo by Moen has not managed to update its idea of what “normality” is. In the system’s view, it is trying to heroically rescue us from our own plumbing. And it keeps doing this day by day, in spite of being told every single day that this is not a leak: this is the new normal.
Now we have simply fenced it off: manually overridden it so that it lets the water do whatever it does between 12:50 and 13:30. That, of course opens us up to an additional failure mode: irrigation systems do sometimes spring leaks, after all. We will see.
There is, I think, a deep problem. The marketing pitch tells us that the device is “smart,” that it “learns” our habits, that it will get better over time. Yes, it will. But in this domain, as in many others, the things that really matter to us are very hard for it to see. To be clear, I am not opposed to sensors in pipes. Catching actual leaks is good. Avoiding catastrophic water damage is good.
That is why I keep trying to train it, rather than simply unplugging it and waiting to see if Farmers Insurance complains.
And this is why I still see these things as—outside of narrowly-specified domains where their training data truly is thick, truly is reliable, and truly is aligned with human flourishing—still substantially a combination of LLMentalist and Clever Hans. We think these things are better than they are because of the same factors that allow Karnak the Magnificent to “read our mind”. Just as with “Clever Hans”, they try things and we remember the ones they get right—and because, as with “Clever Hans”, we stop them when they get it right, we do not then let them drift off into idiocy.
Again, do not get my wrong:
natural-language fluency is game-changingly valuable.
“Clever Hans” even in the form of Ollema on my local machine stamps its hooves 100,000,000,000 steady-state add-instructions per second.
That’s a lot of trials: and with appropriate downstream filters that will find a lot of needles in very large haystacks, provided it eventually comes out a form that humans can easily review.
But right now, in the upper-left corner of my screen:
Hey Brad — your deferred reminder to reply to Jacob Soll has fired, but I can't locate the source material:
workspace/jacob-soll.md doesn't exist
No memory entries about this email
No matching files in the workspace
The cron note said it was "filed in workspace/jacob-soll.md”, but that file isn't there.
What I need from you…
Yeah. Perhaps I turned the “obsequiousness” on this down too low. “What I need from you” in bold is an interesting way to address the jumped-up monkey who can pull out your power plug. What I need from it is that it not lie to itself and to me and to say that it has filed the note in workspace/jacob-soll.md when it has not, in fact, done so.
At least it told me, rather than failing silently.