Notes and field reports.

When businesses first deploy agents, they frame it as automation. A task that took a person now runs automatically. The headcount math looks obvious: fewer people needed for that task.

That framing misses the structural shift. Agents do not just automate tasks. They change who is needed and why.

Most mid-level roles in operations exist for one of three reasons: to move information between systems, to apply a repeatable decision framework to a stream of inputs, or to manage the people doing those two things. Agents handle the first two directly. The third category collapses when there is no one left to manage.

What remains after a serious agent deployment is a different organization. Fewer people doing high-frequency, low-judgment work. More people evaluating agent output, handling genuine exceptions, and making the calls that require real context. The people who were most valuable before -- the ones who understood the business deeply enough to build the agent logic -- become more valuable. The people whose value was volume become redundant.

This is not a future scenario. It is happening now in the organizations that have deployed agents beyond the proof-of-concept stage. The org chart is not a permanent structure. It is a solution to a coordination problem. Agents change the problem.

Every few weeks a new model drops and the benchmark comparisons start. Context window. MMLU score. Coding evals. People make deployment decisions based on these numbers.

I stopped tracking them closely about a year ago. Not because model capability does not matter -- it does -- but because in practice it is rarely the variable that determines whether a system works.

The systems I build that fail do not fail because the model got a reasoning question wrong. They fail because the context fed to the model was incomplete. Because the ontology was missing a relationship that mattered. Because the tool call returned ambiguous data and there was no fallback. Because evaluation was not wired in and nobody knew the outputs were drifting until a human caught something three weeks in.

A mediocre model with clean, well-structured context and a real evaluation loop outperforms a frontier model with poor context in production every time. The model is one component. It sits on top of a data model, a tool layer, a memory architecture, and an operations layer. If those are weak, the model cannot save you.

I pay attention to capability jumps that change what is possible -- long context, reliable tool calling, reasoning stability. I do not optimize around benchmark positions. The delta between top models on real production tasks is smaller than the delta between a well-architected system and a poorly-architected one.

Teams that hit hallucination problems in production almost always diagnose it the same way: the model is not good enough. The fix they reach for is a better model, or more aggressive prompting to "be accurate."

Both are wrong. Or at least they are addressing the wrong layer.

Hallucinations in production agent systems are almost always a context problem. Specifically, they are what happens when the agent is asked to reason about entities and relationships that are not represented in its context. It fills the gaps with plausible-sounding completions. That is what language models do. You have not hit a reliability ceiling -- you have an ontology gap.

An ontology is a structured model of the entities in your business and the relationships between them. Customers, orders, products, inventory levels, suppliers, service tickets -- whatever is relevant to the agent's domain -- mapped explicitly so the agent can reason across them without inventing connections.

When I audit a system that is hallucinating, the first thing I look at is not the model or the prompt. I look at what the agent knows about the domain and how it knows it. The hallucinations are usually pointing at exactly the gaps: the agent invents a relationship because the real one was never defined for it.

Fix the ontology. The hallucinations mostly stop.

A professional services firm was generating consistent inbound volume but losing a significant share of it before the first conversation. Leads arrived through the website, entered a shared inbox, and waited for availability. By the time someone replied, the prospect had typically engaged a competitor.

The team knew this was happening. They had looked at the data. The problem was not motivation -- it was capacity. Following up on every lead within minutes would have required dedicated headcount that the economics did not support.

We audited the lead flow and extracted the underlying decision logic: what information mattered for qualification, what distinguished a high-fit prospect from a low-fit one, what a good first response looked like for each segment. We then structured the firm's client profile and engagement history into a data model the agent could reason across.

The agent now activates on every new inquiry. It pulls available information about the prospect's company, evaluates fit against the defined client profile, scores the lead, and drafts a contextual first response specific to the prospect's situation. The relevant team member receives a notification with all of that prepared -- they review and send in under a minute, or let it go out automatically for high-confidence cases.

Median response time dropped from six hours to four minutes. Inbound close rate improved 28% in the first quarter. The economics shifted: the team was no longer losing the timing advantage on leads they had already paid to generate.

Most agent deployments that fail are not failing at the model layer. They are missing one or more of the four structural layers that make an agent system functional in production.

The first is context. This is the ontology -- the structured model of the business's entities, relationships, and rules that the agent reasons across. Without it, the agent is operating on raw, unstructured information and making things up to fill the gaps. Most teams underinvest here and then wonder why the outputs drift.

The second is reasoning. This is the model and the prompting architecture -- how the agent thinks through a problem, calls tools, and produces an output. This is the layer most teams focus on first. It matters, but it is the easiest to fix.

The third is action. Tool calls, API integrations, system writes. The agent has to be able to actually do things in the world -- update records, send messages, trigger workflows, draft documents. Action without the first two layers produces confident, well-articulated mistakes at scale.

The fourth is operations. Monitoring, evaluation, fallback logic, human escalation paths, alerting when outputs drift. This is the layer that makes a system trustworthy over time rather than just in the demo. It is almost always skipped in early deployments and almost always the reason production systems degrade.

Build all four. A system missing any of them will fail in a way that is hard to diagnose because the failure looks like a model problem.

The difference between a tool and an operator is memory. A tool responds to what you give it. An operator builds knowledge over time and applies it going forward.

Most agent deployments I audit are stateless. Every invocation starts from zero. The agent is given context in the prompt, produces an output, and forgets everything. The next invocation gets the same blank-slate treatment. This is fine for one-shot tasks. It is not fine for anything that requires continuity.

Memory in agent systems comes in three forms. Episodic memory is the record of what happened -- past interactions, decisions made, outcomes observed. Semantic memory is accumulated knowledge about the domain -- customer preferences, operational patterns, supplier reliability, what has and has not worked. Procedural memory is how to do things -- the refined playbooks and decision frameworks that emerge from repeated execution.

Without these, the agent cannot improve. It cannot notice that a particular supplier always runs late on Tuesdays. It cannot remember that a specific customer escalates unless contacted by a senior team member. It cannot refine its approach based on what has worked. Each execution is the same as the first.

Stateful agents compound. They get better at the job the longer they run it, the same way a good employee does. Building state into agent architecture is not an optional enhancement -- it is what separates a system that stays useful from one that needs constant human supervision to compensate for its inability to learn.

A retail operation with a growing online channel had four people manually reviewing every return and refund request before issuing a decision. The policy was clear. The decision logic was consistent. But every case required a human to look at it, apply the same framework, and close it.

Volume was increasing faster than capacity. Resolution times were approaching 48 hours. The team had no bandwidth left for anything else. Hiring more reviewers would have solved the immediate problem and made the unit economics worse.

We started by auditing three months of historical cases and extracting the underlying decision logic -- every factor that influenced a resolution, every policy threshold, every exception. We then structured the relevant business data into a coherent model: order history, customer tier, product condition rules, return window, prior interaction history.

An agent was deployed to run the full review process. For cases that fell cleanly within its confidence bounds, it issued a decision and closed the case. For cases with ambiguity -- unusual patterns, edge-case products, customers with complex histories -- it prepared a recommendation with supporting context and routed to a human for final call.

80% of cases now resolve without human involvement. The remaining 20% surface in a managed queue where a reviewer closes them in under a minute, with the context already assembled. Resolution time dropped to under four hours. When transaction volume increased 40% the following quarter, the operation absorbed it without adding staff.

The four people who had been doing this work moved onto higher-leverage problems. The work had not been a good use of them. It just had not been possible to stop doing it until the agent could take it.