An operator I work with recently pulled up what their team's internal apps and agents were costing in tokens. The bill was bigger than anyone expected. Their reaction was: what am I doing?
Around the same time, I spoke with a team celebrating a company-wide agent rollout. They were proud to call themselves an AI-enabled organization.
In both companies, and most I talk to, nobody owns the decision about which agents should keep running as agents, what they should cost, or who answers for them when something goes wrong. Frontline developers build the agents but aren't handed AI rollout as a business function. Senior managers are focused elsewhere. It falls to whoever can see what's coming.
That responsibility is turning into a role, though not a formal one yet. Some companies call it AI ops, some call it AI director, and many call it nothing. It usually goes to a DevOps lead or a technical exec, and I'm starting to see DevOps leaders get AI ops added to their titles. There's no shared understanding yet of what the job covers, the way there is for a DevOps lead.
Most agent work today is still exploratory. Running a job through an agent first is how you learn which steps it needs, by watching it run and correcting it when it gets things wrong.
I wrote a while back about assistants and systems. Assistants spread bottom-up, one seat at a time. Systems get decided top-down, by someone who owns the outcome. Whoever does AI ops, with or without the title, becomes that owner, and most of the job is one handoff: deciding when work that started in an agent should move into a system.
Cloud computers and cloud functions
Software went through a version of this in the last wave. I'm not an engineer, but this is how I understand it. A common pattern was to start with one app that did everything, because it was simpler to build and change. As usage grew, pieces were split into separate services, and later into small functions that ran only when called. That split usually happened under pressure, often after something broke. Companies with a strong CTO or architecture team had someone who owned the call on what to split and when. Earlier-stage companies tended to wait until something was on fire. Most companies doing AI today are in that second group, whatever their size.
My co-founder uses that history to describe the agent handoff. Always-on agents are like cloud computers. Repeatable, deterministic tasks are like cloud functions.
A cloud computer is a full machine you rent and keep running. You can install anything, try things, and change your mind, and you pay the whole time it's on. An always-on agent works the same way. It can take a job nobody has fully specified, decide how to approach it, and adjust as it goes. That flexibility comes from the model reasoning through the job each time, and you pay for that reasoning on every run. Its scope also moves from run to run, which makes it hard to say what it's responsible for.
A cloud function is a small piece of code that does one defined thing when called, then stops. Inputs and outputs are fixed, and it runs the same way each time. You pay only while it runs. It's harder to experiment with and easier to test, monitor, and budget for. Its job is written down, so everyone knows what it's responsible for and who to call when it fails. A deterministic automation has the same properties: written steps, access scoped to those steps, and predictable output for a given input.
The parts teams split out were the ones that were stable, called often, and expensive to run on a full machine. Whoever made that decision had to understand the code, the load, and the bill well enough to know when rebuilding was worth it. Moving too early locked in a design before anyone understood the problem. Moving too late meant paying for idle capacity and debugging a system nobody could fully see into.
The AI ops role makes the same call with agents. In practice that comes down to a handful of decisions:
- Which agent work has proven itself and should become a fixed, repeatable task, and which should stay an agent
- Who can run each process, and with what access
- What each process costs to run, and whether that cost is worth it
- What happens when something misbehaves, and who gets told
Fixed tasks make roles clear
At some point, keeping track of all the agent work going on means drawing lines, the same way a company does with people. Each process needs a defined job, the access that job requires, and an owner who answers for it. An always-on agent makes those lines hard to draw.
Without those lines, the usual answer to agent sprawl is more people: someone to review outputs, someone to chase down what broke, someone to explain the bill. Once every task has a defined job and an owner, you can add more tasks without adding people to watch them.
Fixed tasks also make each run predictable, and the cost per run drops.
I rebuilt the same job three ways, and by the second version it had gone from roughly 50K tokens a run to roughly 5K. The output was the same, and the model was rethinking much less of the job on each run.
When something should move
I personally still build a lot of things as agents first. The question for the AI ops role is when a given piece of agent work has stopped being exploratory. There's no fixed threshold, but a few conditions tend to show up together.
The first is consistency. If the agent runs the job the same way most of the time, and you keep correcting the same mistakes, the job is already defined, and the remaining step is writing it down as a fixed task.
The second is volume. Usage is going up, and the job is heading toward a scale where the per-run cost adds up.
The third is dependence. Once someone outside the team relies on the output, a run that goes differently from the last one becomes their problem.
Work that doesn't meet those conditions can stay in the agent. Part of this role is keeping a short, honest list of what's still exploratory and what's now operational, and revisiting it as usage grows.
Where to start
My guess is the team that was celebrating will get to "what am I doing?" eventually. I'd want someone to own the answer before it shows up on an invoice.
If you're the person about to be handed this, start with an inventory. List what each of your agents does, who depends on it, and what it cost last month. If you can't fill in that last column, that's your first finding.
qbash Tasks take work you've already figured out in an agent and run it as a hosted, deterministic process with one defined job, scoped access, and a record you can audit. If you have one agent workflow you'd like to run the same way every time, bring it to us and we'll work through it with you.