The chatbot: two tiers, the tools, and the rules it must not break
· 8 min · ai, chatbot, tool-calling, openrouter
One component for three assistants, a free model for anonymous users and a better one for those who sign in, tools that pull from the same engine as the map. And the error that didn’t shout.
The chatbot was one of the first bricks I laid down, in February, for a practical reason: if I wanted to understand how a model behaves inside a real product (not in a notebook) I needed a real product to put it in. It’s the same component that today answers in three places: on the site, inside Seantral, inside Iterzen. Three assistants, one machine underneath.
Two tiers, depending on who you are
Anonymous, the chat runs on a free model (a cheap slot served via OpenRouter) with a message cap. Signed in, you move up to a better model. It isn’t a paywall in disguise: anonymous users get a real taste, and the quality difference is paid for by the fact that I at least know you signed in. The model choice isn’t made by the browser, it’s made by the edge based on role: the client asks, the function decides which slot to answer with. That way the good model is never exposed to anyone who didn’t go through login.
One component, three heads, a knowledge base for each
The interesting part is how three different assistants live in the same code. The knowledge base changes, not the machine. Each assistant has its own hand-curated base: what it can say, what it must keep quiet, how to answer when it doesn’t know. The rules are precise because they have to be. The site assistant talks about projects in generic terms, and the codenames of the ones not yet public it does not say, full stop. It isn’t a suggestion in the prompt the model can decide to ignore in a creative moment: it’s a rule I treat as one, with the list of names to withhold kept out of the context the model sees.
The tools, and the same engine as the map
The Seantral assistant doesn’t just chat: it has tools. If you ask “where should I swim this morning”, behind it calls a tool that ranks the spots, and that tool uses the same verdict engine that draws the map. It’s a choice I defend: if the AI used its own separate logic, sooner or later it would hand you a number different from the one you see on the spot, and at that point you trust neither. One engine, two ways of querying it.
The error that didn’t shout
And here I hurt myself. The tool’s rankings were slightly wrong. Not randomly, not glaringly: a spot that should have come second came out third, a score a touch off from the map’s for the same place at the same hour. No error, no crash, no alarm. Just an assistant answering, confidently, things that were a hair off.
The cause was subtle. The tool was reading a value frozen at the moment the data had been fetched, instead of projecting it to the current hour like the map does when you look at it. Same engine, different reference hour: that alone was enough to drift the ranking. The worst kind of bug, because the system doesn’t help you find it: it doesn’t break, it lies quietly. From it I carry a rule: distrust errors that don’t shout, and test tools with the hour pinned to a known value, not “now”.
Fallback chains, budget, and the 429
Every AI feature has a fallback chain. If the preferred model is down, or has gotten too pricey, or right now is answering slowly, it drops to the next one instead of slamming an error in the face of whoever’s typing. The user sees a slightly less brilliant answer, not a broken screen. On top there’s an AI spend budget per app (a daily cap) and a cap per single account. When an account blows its limit it gets a 429, the chat tells it in human words, and the count is done before calling the model: whoever’s already over threshold doesn’t spend more. It’s the same lesson as the brief (count before, not after) learned on a different machine.
What I tell whoever’s typing
Two things I keep explicit. First: the answers are generated by an AI and can be wrong: I write it where you chat, not in a terms page nobody opens. An assistant that seems sure of everything is more dangerous than one that now and then says “check this”. Second: I save conversations only with consent, and the consent is functional: it’s there to let you find the chat again, not to profile you. Without consent the conversation lives in the session and then vanishes. No silent archive of things people wrote thinking they were ephemeral.
What comes next
Where it’s heading. The Seantral assistant’s tools are few: it ranks spots, little else; I’d like to give it more that pull from the same engine, so that “how is Torre dell’Orso compared to yesterday” becomes a question it answers with real numbers rather than a generic sentence. On the model side I’d like the slot choice to account for the question too, not just the role: a trivial request doesn’t need the good model. And the knowledge base I curate by hand today: it works, but it doesn’t scale to many apps; the next step is keeping it close to the code it describes, so that when I change a feature the assistant doesn’t tell the version from two months ago. As always: directions, not dates.