What the Model Sees, What the User Feels, What You Can Prove

A note on the kinds of work that compound when the model underneath you is rented — written for anyone building on a frontier model they don't own.

The model underneath our product is rented. Same API, same pricing page, same rate limits as everyone else in the space. Whatever edge Waterr has, it is not in the pipe — there is one pipe, and a line of people renting it.

That one fact sorts every hour of work we do into two piles: work that evaporates the day the model gets swapped, and work that doesn't. After a year of putting AI personas into real interviews, sales roleplays, discovery calls, and onboarding sessions, I can tell you the second pile holds exactly three kinds of work.

What the model sees. What the user feels. What you can prove.

This post is the taxonomy — one honest example of each, and the insight each one cost us something to learn.

The swap test

Everyone in AI says "moat" and almost nobody means anything by it. Prompt libraries get called moats. Waitlists get called moats. Here is the definition we actually operate on:

A moat, for a company that rents its model, is anything that stays valuable the day the model underneath is swapped.

Fine-tuned prompt tricks fail the test — they're coupled to one model's quirks. A demo fails the test. What passes: the context architecture around the model, the ground truth about your users that you extracted by hand, and a public record of claims that survived being checked. Those are the three kinds of work.

Kind one: what the model sees

Watch a voice agent fail on a live call and the failure is almost never the voice. It's minute nine — the agent walks back a commitment it made two turns earlier, because a realtime model gets 200–500ms to start speaking and there is no room in that budget to re-read the conversation. A bigger realtime model doesn't fix this. The budget stays fixed; the conversation keeps growing.

Everything the agent needs is already sitting in the transcript. What's missing is a mechanism that puts it in front of the model at the moment it answers. That is a visibility problem — and visibility problems are harness problems.

So we built Monologue — a reasoning sidecar that runs beside the voice model instead of inside it. While the participant is still speaking, a text model reads the conversation and writes a short briefing in three typed slots — MEMORY (what was actually said, quoted), CONSTRAINT (what the agent already committed to), TRAJECTORY (what the next reply needs to do) — and injects it between turns, into an audio gap that already exists.

On AudioMC it scores 69.8% against 41.2% for the bare backbone. Inference memory up 16.3 points. Wall time per conversation: 22.5 seconds with it, 22.5 seconds without. The reasoning is free at the clock, because it was never on the clock. It runs at $0.03 per conversation-minute with the planner included.

The insight: the gap was already there. We didn't buy more compute or wait for a better model — we found dead time in the loop and put reasoning in it. Most of the harness work worth doing looks like this: moving what the model already knows into the moment it has to answer.

Kind two: what the user feels

The second kind of work has no code in it at all. We sat with a year of session recordings — real deployments, not benchmarks — and watched them the way a coach watches game tape. Two findings survived every domain we looked at.

• Self-coherence is the trust floor, not a feature. The moment the agent contradicts something it said two minutes earlier, the participant checks out — you can see it happen on the recording. People forgive an AI for a lot. They do not forgive it for arguing with its own earlier self.
• "Heard" and "understood" are different events, and the second one is visible. Heard is transcription. Understood is the agent adapting to what you meant — slowing down for a nervous candidate, taking a follow-up as a follow-up instead of restarting the topic. The tell, across every recording: the person's answers get longer right after it happens.

That second finding matches the oldest number we have — participants are roughly 10x more expressive answering an AI's questions than recording themselves cold. But the recordings told us when the expressiveness turns on. It's not at the start of the call. It's at the moment the participant stops performing for the agent and starts talking to it.

The insight: the metric was in the recordings, not the dashboard. Answer length over session time is a trust signal no benchmark hands you. If you build conversational AI and don't watch your own tape, you are optimizing numbers your users don't feel — and this work passes the swap test completely, because what earns a human's trust doesn't change when the model does.

Kind three: what you can prove

The third kind of work is the one I didn't expect to write about, because it started as marketing.

In July we rebuilt our public surface around the developer who actually calls the API, and we set two rules. First: docs.waterr.ai is the source of truth — if the docs page doesn't exist, the capability doesn't exist, for marketing purposes. Not "it's in the codebase." Not "next sprint." No page, no claim. Second: on comparison pages, every claim about a competitor gets verified against that competitor's own live documentation on the day it's written — including an honest "choose them if" section where they're genuinely the better fit.

Then the rule turned around and pointed at us. Running the same pass on our own pages flagged a line we had stopped even reading — that one API call returns the transcript, the recording, and the scores together. The actual response body disagrees: recordings come from their own endpoint, and they arrive minutes after the call, asynchronously. The line had spread the way copy spreads — written once, pasted fifteen-plus times, never checked against a real response again. The rewrite took an afternoon. The uncomfortable part was the pattern: nobody had lied, and the claim was still false everywhere it appeared.

The insight: verification is a two-way instrument. We built the pass to keep our competitor claims honest, and its first serious catch was us. That's exactly the property that makes a public record worth anything — the same check that qualifies you to publish a number is the one that stops you from publishing a wrong one. What survives the check compounds.

The hard parts nobody puts in the demo

The demo is ninety seconds of an AI holding a conversation. None of these three kinds of work is visible in it — and each one has a cost the demo hides.

• Harness work is invisible when it succeeds. Nobody watches a call and applauds the briefing that got injected between turns. The only visible version of this work is its absence — the contradiction in minute nine.
• Tape work doesn't scale gracefully. Someone has to actually watch the recordings. We looked for a shortcut and there isn't one yet; the signal lives in moments a transcript flattens.
• Truth work is slow and occasionally embarrassing. Verifying every claim against a live docs page on the day you write it feels like bureaucracy — until the day it catches a false claim of yours in fifteen-plus places, and you realize the alternative was shipping it.

Three questions, one test

The taxonomy folds down to three questions we now ask about any week of work:

Did we change what the model sees at the moment it has to answer? Did we learn something about what the user actually feels — from primary evidence, not a proxy? Did we publish something that survives being checked?

A week that answers yes to none of them was a week spent on the rented pipe — and the pipe doesn't remember.

The model will get swapped. Ours, yours, everyone's. What the model saw, what your users felt, and what you proved in public will still be there the morning after.

*This is part of the thinking series. Where the Work Went Next is the field report behind this taxonomy; Your Personal Agent Harness makes the harness argument for individuals. The Monologue numbers live at waterr.ai/research/monologue.*