My monitoring agent sent three weeks of noise
I built an agent that looks across all my systems and tells me what needs attention. For three weeks it mostly told me that things were happening.
Sentinel is an always-on agent on my own stack. It reads across Office 365, Close CRM, GitHub, Azure DevOps, PostHog, Notion and Billit. The idea was simple: one place where loose signals come together, so I don't have to walk through seven systems to know what's going on.
Technically it worked from day one. Every integration ran, the synthesis was readable, escalation over Telegram arrived cleanly. And it was still unusable.
The problem wasn't too little information
It was too much. Every morning I got an overview of what had changed since yesterday, and that was always plenty. A deal moved, a pull request opened, an invoice sent, a user active. All true, all correctly retrieved, and all presented as equally important.
An agent that reports everything reports nothing.
After a week I was skimming it. After two I barely opened it. That is precisely the failure mode of every monitoring system ever built, and I had walked into it with my eyes open because I assumed a language model would spot the difference.
What I changed
Three things, in order of impact.
One: memory of what has already been said. The first version was memoryless. Each run looked at the current state and described it. So if a deal sat still for three weeks, it reported that same stalled deal for three weeks. I added a persistent ledger of open loops: every finding gets an identity and a status, and the agent only reports what is new or what has deteriorated. That change alone removed about eighty percent of the volume.
Two: thresholds on change, not on state. Not "there are 14 open invoices" but "there are four more than last week, and three of them are over sixty days old". A state is a fact. A change is a signal. Only the second asks something of me.
Three: explicit silence. The agent may send nothing when there is nothing to report. That sounds obvious and it isn't, because the temptation to send a daily summary "for completeness" is strong. That completeness is exactly what makes it unreadable. A message that arrives now means something, which is why I read it.
What I underestimated
That deciding what matters is not a technical question. I had hoped the model would work it out from context. It doesn't, and in hindsight that's obvious: relevance depends on what I'm working on that month, where I see risk, and what I already know. None of that is in the data.
So I wrote it down. Literally, in a configuration file of thresholds: what counts as an amount that matters, how many days of standstill are suspicious, which customers are sensitive. I adjust that file roughly once a month, and that is exactly the kind of maintenance an agent needs and that most projects forget to plan for.
The architectural choice I did get right
Read broad, write narrow. Sentinel has read access to everything and write access to almost nothing. Every proposed action falls into three categories: execute it when it can irreversibly break nothing, put it forward when it changes something, and merely flag it when it's irreversible.
That's a boring choice and it has saved me every doubt about autonomy. The agent can annoy me, it cannot get me into trouble. That's the bar below which you shouldn't build one of these at all.
What to take from this
- Build memory before you build cleverness. An agent that doesn't know what it said yesterday repeats itself until you ignore it.
- Report change, not state. State belongs on a dashboard, change belongs in your attention.
- Write the thresholds down explicitly. The model doesn't know what matters to you, and it can't guess.
- Allow silence. A system that's allowed to stay quiet gets read when it speaks.
The useful part of Sentinel isn't the clever part. It's the filter.