Agentic AI Needs Guardrails, Not Hype

A clear work plan can reveal when an agent intends to use unnecessary tools, access irrelevant data, or overcomplicate a simple task.

Agentic AI does not have to replace an accountant to transform accounting. Its more  practical role may be far less dramatic — and far more useful.

In this episode of Accounting ARC, Donny Shimamoto, CPA.CITP, CGMA; Liz Mason, CPA; and Byron Patrick, CPA.CITP, explore how narrowly focused AI agents can help professionals retrieve information, analyze data, navigate systems, and make better-informed decisions.

Their conversation challenges one of the dominant narratives surrounding agentic AI: that firms should build autonomous digital employees capable of handling everything a staff accountant does. Instead, the hosts envision specialized agents that operate within structured systems, perform clearly defined tasks, and show users how they reach their conclusions.

“If you have agentic AI that’s built into an architecture that already exists, there’s a whole lot more that has to happen from a training perspective,” says Mason, CEO of High Rock Accounting.

That architecture, she explains, can include multiple agents working together rather than one agent attempting to handle an entire assignment.

The best agents may remain behind the scenes
Shimamoto, founder and managing director of IntrapriseTechKnowlogies LLC and founder and inspiration architect for the Center for Accounting Transformation, opens the discussion with an agentic AI application that catches his attention. A data analytics vendor uses the technology to let customers ask questions about information stored in a data lake. The user does not need to understand database structures or know how to write a technical query. Instead, the AI interprets the question, identifies the relevant data, and constructs the queries required to find the answer. Most importantly, it shows its work. Users can see how the system interprets the request, which data sources it consults, and how it develops the result. That transparency gives professionals an opportunity to evaluate whether the system is asking the right question before relying on its answer.

The distinction matters because a technically correct answer can still be wrong for the user’s intended purpose. A request for “sales,” for example, might refer to opportunities recorded in a customer relationship management system or revenue recognized in an accounting system. An effective agent needs enough context to identify which measure the user actually wants. Patrick, senior product manager for Karbon and founder and part-time educator for TB Academy, says the system should probe for that intent.

“What are you hoping to learn by asking for that?” he says, describing the type of follow-up an effective agent might provide.

One agent does not have to do everything
Mason describes an orchestrated approach in which several specialized agents contribute to the same workflow. One agent may interpret the user’s question and request clarification. Another may retrieve information and perform calculations. A third may translate the analysis into a concise answer the user can understand. That is still agentic AI, but it is significantly different from the “co-worker” agents often marketed as replacements for staff accountants.

“It is a thing, a robot that is trained to do a thing, and that’s it,” Mason says. “That’s all an AI agent is.”

Shimamoto places this type of agentic AI in the workflow automation category rather than treating it solely as a data analytics tool. The technology coordinates a series of tasks, calls on the appropriate resources, and returns the result through a simpler user experience.

Accounting professionals need to ask how a product’s AI is built, what information it accesses, how it is tested, whether it produces consistent results, and where hallucinations may occur.

The accountant may never interact directly with each agent. Instead, the agents operate inside software the professional already uses. That model also places responsibility on technology vendors. Most accounting professionals should not have to design, connect, supervise, and troubleshoot a complicated collection of agents. Vendors must build the structure, limitations, controls, and validation processes into their platforms.

More tools can create more trouble
Agentic AI becomes riskier when an agent has access to too many tools without sufficient boundaries. Patrick warns that agents can begin “spiraling,” turning a simple request into an unnecessarily complex project. An agent asked to summarize one Slack conversation might decide to search email, cloud storage, and other unrelated sources.

“The complexity comes out by adding more tools to their tool belt,” Patrick says.

He recommends asking an agent to present its work plan before allowing it to execute a task. Reviewing that plan can reveal whether the agent intends to access irrelevant information, use unnecessary tools, or take actions that exceed the original request. For individual users experimenting with general-purpose AI tools, that oversight remains essential. In professional applications, however, Mason says vendors should create those controls on the back end so users do not have to manage every part of the process. The goal is not to give an agent unlimited freedom. It is to give the agent the specific capabilities it needs to complete a defined task.

Reliable systems need creativity and consistency
The hosts also distinguish between generative and deterministic technology. Generative AI interprets language, works with ambiguity, and develops creative responses. Because it is probabilistic, it may respond differently when asked the same question more than once. Deterministic technology follows established rules. Given the same information and formula, it produces the same result. Business systems need both. An agent may use generative AI to interpret a user’s question and determine what information is needed. It can then use a deterministic process, such as a database query or calculation, to produce a consistent result. Generative AI can translate that result into a clear explanation for the user.

“You have generative on the front and the back, and then deterministic in the middle,” Mason says.

Patrick says that approach is also more efficient. Asking a generative AI model to ingest large quantities of raw data, create scripts, perform calculations, and explain the answer can consume significant computing resources. Allowing the AI to construct a database query — and letting the database perform the calculation — can be faster, less expensive, and more reliable.

Different jobs require different agents
There is no single architecture that works for every accounting application. An agent designed to help users navigate a practice management system has different requirements from one that interprets tax returns or researches complex tax questions. Mason says the agentic AI under development for tax research requires a different structure from technology designed primarily to improve navigation or workflow efficiency. Each use case requires its own data, controls, testing, and approach to accuracy. That makes vendor transparency increasingly important.

Accounting professionals need to ask how a product’s AI is built, what information it accesses, how it is tested, whether it produces consistent results, and where hallucinations may occur. Shimamoto urges accountants to push beyond marketing claims and request enough information to determine whether a product is appropriate for its intended use.

“What are the appropriate use cases?” he says. “How is this tested so that we can determine that it does make sense for the use that we’re intending for it?”

Professional standards are beginning to respond. Shimamoto says guidance is being developed to help vendors provide greater transparency and help accountants evaluate technologies that do not always produce deterministic results. The technical details may be difficult for many buyers to interpret. Even SOC 2 reports, which are already widely used to evaluate technology controls, can be challenging for professionals outside information technology and assurance specialties. Ironically, AI may also help accountants interpret those reports, identify potential concerns, and develop better follow-up questions for vendors. Human judgment, however, remains critical.

Accountants can influence what vendors build
The hosts’ message is not that every firm needs to race out and build its own collection of AI agents. For many firms, the wiser move is to watch for agentic capabilities embedded in the technology they already use — and tell vendors what they need those capabilities to accomplish. That may mean asking for natural-language access to firm data, greater visibility into how an answer is constructed, clearer explanations of AI controls, or better safeguards around the information an agent can access. Vendors prioritize features that affect adoption and sales. If accountants do not ask for transparency, reliability, and well-designed agentic workflows, those features may not move to the top of the development list.

Agentic AI’s future in accounting may not look like an autonomous digital accountant sitting beside every human employee. It may look like a collection of disciplined, narrowly trained specialists operating invisibly inside the systems accountants already trust. The technology does not need to do everything. It needs to do the right thing, within the right boundaries, and make it possible for a human to verify the result.

Listen to the full episode