What Is Your AI Agent Actually Doing? An Honest Audit

The short answerTo find out what your AI agent is actually doing, ignore the settings page and read three things: the action log, the autonomy setting on every duty, and where its output lands. In one audit of a company with 18 AI phone agents and 45 AI duties, every duty was set to watch or suggest, and the action log showed zero executed actions. Built is not wired.

On paper: 18 AI phone agents. Sixty-five phone numbers. Eleven AI "employees" with job titles and 45 responsibilities between them. A knowledge base, 58 automated flows, 22 skills.

Then I opened the action log. Actions the AI had ever executed in the business: zero.

18AI phone agents on the books

2,273calls queued, none placed

0actions executed, per the action log

This is the audit I'd make every company running AI agents, bought or built, sit through. Nobody lied. The parts were real and running on the live server. They were just switched off.

What did the client think its AI was doing?

That it was working. Reasonable assumption. Wrong, but reasonable.

The client runs its business on its own software, and that software had grown a whole AI layer: phone agents for leads, an AI for texts, a website chat bot, AI "staff" with job titles (the busiest one carried 23 duties). Every part existed and ran on the live server.

The phone agents had even worked hard once: tens of thousands of call records in the database. Then they went quiet, and nobody noticed.

I had a hunch going in: lots built, little of it wired into real workflows. Halfway through the audit, I told the agent running it how simple the write-up had to be:

i want a very simple dum-dum explanation how things are currently qired
me, to my agent, typo included

So the one-pager I made for the client draws the AI as a body: ears notice, a brain decides, hands act, and the work lands where staff actually live. Tap a part:

live and used built, off or barely used never fires

Ears: mostly amber. Brain: amber. Hands: red. Real work: no AI action ever landed.

How do you check what an AI agent is actually doing?

You stop reading the settings page and start reading the logs. Settings tell you what the AI could do. Logs tell you what it did.

I put three of my agents on it, read-only: one on the code and the live database, one on months of email and call transcripts (what people expected the AI to do), one on the features. About twelve minutes later the code agent came back with a big claim: the AI's skills (its reusable recipes) had never run, ever.

When an agent says "zero, ever", that's a rumor until somebody checks. So I re-ran the numbers on the live database, read-only, before anything went on the page:

The dial. Each of the 45 responsibilities had a setting: watch, suggest, draft or act. Count at "act": 0.
The action log. 122 attempts, every single one marked "suggested" or "watch". Executed: none.
The second log. In case skills logged elsewhere, I checked the general AI log. Nothing there either; its biggest category was 2,888 call summaries, and those had stopped months earlier.

One honest gap: an outside automation tool might keep its own log, and I didn't open it. So the claim is "zero in the system's own action log", not "zero in the universe". Write your audit at the resolution your evidence has.

Where were the wires cut?

In four places, and none of them was a bug.

The dial. With every responsibility on "watch" or "suggest", no action could ever fire. A brilliant intern, banned from touching anything.

The switches. AI text replies: off everywhere. AI calls: queued, never placed. Picture a lead texting at 10 pm. The reply engine is right there, built and deployed. Switched off. The lead waits until a person happens to see it.

Short hands. Only two of the system's nine possible actions could run from a skill, and neither touched a lead or a customer.

No owner. The knowledge base: untouched for over two months. The only scheduled routine: paused. Lots of people built pieces; nobody owned the workflow.

What the setup implied

18 phone agents calling leads
AI answering texts
11 AI staff doing their duties
A knowledge base feeding it all

What the logs said

Last real AI call: six weeks back
AI replies: switched off
0 duties allowed to act
Knowledge linked to nothing

This isn't rare. MIT's NANDA study found only about 5% of AI pilot programs reach rapid revenue growth (Fortune). Gartner expects over 40% of agentic AI projects to be scrapped (South China Morning Post). My guess: plenty of those look exactly like this one. Built, demoed, never plugged in.

How do you fix AI agents that never act?

You don't fix all eighteen. You pick one boring, everyday workflow and wire it end to end.

My recommendation: take "a new lead sends a text", wire ears to brain to hands to the real lead record, and move that one dial from "watch" to "draft". The AI prepares the reply; a person taps approve. That one move turns a museum into a machine. Measure it. Only then wire workflow number two.

"Draft" is the underrated setting. It's safe enough to switch on right away and real enough to produce a log you can count. Same instinct as my overnight agent rig: the thing doing the work is never the only one checking it. In draft, a person checks every move.

The five-question audit any owner can run

You don't need me for this. You need read access and the stubbornness to ask for logs when someone offers a demo.

When did it last do the real thing? Not "is it on". The timestamp of the last real call, reply or action.
What is it allowed to do? Find the autonomy setting on every duty. Count how many can act.
Show me the action log. Executed versus suggested. If nobody can produce one, you have your answer.
Where does the output land? In the system your staff work in, or in an inbox nobody reads?
Who owns the workflow? One name. Then check when the knowledge base was last touched and whether anything scheduled actually runs.

Then don't trust the answers either. Have someone who didn't build it re-run them (the whole idea behind giving agents a QA department).

Odds are your AI isn't lying to you either. It's sitting on "watch". Go check the dial.

Want AI agents that actually ship?

I build AI agents and the systems around them for businesses: memory, voice, overnight rigs, and the boring guards that make them safe to leave alone.

Talk to me on WhatsApp max@kilazi.com

Questions people ask

Why do AI agents get built but never used?

Usually because nobody owns the workflow end to end. Teams ship parts (a phone agent, a chat bot, a knowledge base) and each demo works, but the wires between them stay off: autonomy settings stuck on watch, replies disabled, output going to an inbox instead of the system staff use. Nothing is broken, so nothing gets fixed.

What is an AI autonomy dial?

A per-task setting that controls how far an AI agent may go: watch (observe only), suggest (propose an action), draft (prepare the action for a person to approve) and act (do it alone). It is the single most useful setting to check in an audit, because an agent on watch or suggest can never change anything in the business.

Is it safe to let an AI agent act on its own?

Start with draft, not act. In draft mode the AI prepares the reply, task or update and a person approves it with one tap. You get real logs and real outcomes with a human still in the loop, and you can raise autonomy duty by duty once the drafts prove themselves.

How long does an AI agent audit take?

With read access to the database and logs, the core numbers take hours, not weeks. In the audit described here, three read-only agents reported back in about 12 minutes, the key claims were re-checked against the live database with read-only queries, and the finished page was up roughly 35 minutes after the brief.

What logs should an AI system keep?

At minimum, one row per attempted action: which agent, which duty, what it tried, its autonomy level, and whether it was executed, suggested or blocked, with a timestamp. Without that log you cannot tell an AI that works from one that only looks busy.

Max Sergeev. Max builds AI agents and the systems around them: long-term memory, voice, overnight rigs, and the boring guards that make them safe to leave alone. He runs kilazi labs from Buenos Aires. kilazi.com

What Is Your AI Agent Actually Doing? The Action Log Said Zero