A good doctor doesn't reach for an antibiotic every time you walk in with a fever. Most of our AI architecture reviews would be healthier if we behaved the same way.
A few months ago at Eteva, one of our insurance customers came to us wanting a plan recommendation capability. They had already sketched the architecture in their head: a local model, RAG over the plan documents, a model to be trained, and a fair amount of "what not" stacked on top.
We went and looked at the actual problem.
For any given candidate, the universe of plans they were eligible for was fixed. Not inferred, not retrieved — a known, discrete set. And we had the candidate's profile, complete and structured, sitting right there in the system.
Strip away the architecture diagram and the requirement was this: pick the right option from a small known list, and explain why.
I'll come back to what we actually shipped. First, the two questions I now ask before any of this starts.
Gate 1: Does this even qualify?
Not every workflow is a candidate. A model earns its place when:
- The input is unstructured free text, documents, images, voice
- The task needs judgement across ambiguous or incomplete cases
- The rules exist, but there are too many of them, or they change too often to maintain by hand
- Edge cases routinely defeat rule-based automation and RPA
And it does not, when:
- The input is already structured and a query or an API answers it
- The rule set is small, stable and cheap to maintain
- The output must be identical for identical input, every single time
- It's arithmetic, reconciliation, or any path an auditor will later re-perform
👉 Landing in the second column is a win, not a rejection. A deterministic solution costs less, runs faster, never hallucinates, and never needs an eval suite. Saying "this part doesn't need AI" is one of the most valuable things an architect can say in 2026, and one of the least rewarded.
Gate 2: If it qualifies, what shape does it take?
This is where "AI adoption" quietly turns into six or seven different design decisions that get treated as one.
The variable that separates them is simple: who decides the next step — your code, or the model?
Levels 0 to 4 are what Anthropic calls workflows. Levels 5 and 6 are agents. LangChain numbers the same axis and calls them cognitive architectures. Different vocabulary, same ladder.
Three rungs matter more than the seven labels:
- At level 3, the model's output starts steering control flow. You lose static traceability.
- At level 4, you introduce loops. Runtime and cost stop being bounded.
- At level 5, the action space is no longer fixed at design time. Your threat model changes completely.
Worth knowing where real systems actually sit: Databricks found that roughly 60% of LLM applications use some form of retrieval, and about 30% use multi-step chains. Levels 1 and 2. That is the production reality underneath the agent-shaped slide decks.
Back to the insurance problem
The recommendation genuinely passed Gate 1. Weighing a candidate's circumstances against a set of plans and articulating a defensible reason is judgement work, not a lookup. Something had to reason.
Gate 2 is where the proposed architecture fell apart.
What we shipped was a single light LLM call. The candidate's profile and the available plan options both go into the context. The model picks and explains. No RAG. No local model. Nothing trained.
Level 1 on the ladder. The lowest rung that still involves a model at all.
Here is the part worth taking away, because it generalises well beyond insurance. RAG, fine-tuning and a private local model are three different answers to the same question: how do I get the right context in front of the model? Retrieval finds it in a large corpus. Fine-tuning bakes it into weights. Both are machinery built to solve a problem that only exists when the context is too big, too scattered or too dynamic to simply hand over.
Our context was a profile and a short list. It fit in the prompt.
When the answer to "how do I get the context there?" is "type it", every layer above that is architecture you are going to own, debug, secure and pay for, in exchange for nothing.
👉 Everything they wanted to build was a way of getting the right information in front of the model. The information was already in their hands.
To be fair to the other side of this: if the plan universe ran to thousands of documents, or changed weekly, or varied by geography and regulator, RAG would earn its place immediately. The design was right for this problem. That is the whole point of running the gates rather than picking a favourite pattern.
And one more thing the ladder alone won't tell you. A recommendation is not a decision nobody's cover changes because a model suggested something. The moment that same level-1 call is wired to auto-enrol a candidate, the ladder position hasn't moved an inch, but the governance conversation has changed completely.
The axis mostly missed
That last point took me longest to see clearly.
Every published taxonomy Anthropic's, LangChain's, OpenAI's, the academic ones classifies capability. None of them classifies risk. So a read-only agent at level 5 looks terrifying on the chart, while a write-capable prompt chain at level 2 looks harmless. It is the second one that needs change control.
Capability and risk are independent axes. Review only one, and you will approve the wrong things.
Five questions
The whole framework compresses into five questions you can ask in a design review in under two minutes:
- Does your code or the model decide the next step?
- Is the loop bounded by turns, tokens or wall clock?
- Is the action space fixed at design time, or can it grow at runtime?
- Are the actions reversible, and who notices when they are wrong?
- How many independent decision-makers: one, or many with a protocol?
Answers 1 to 3 place the design on the ladder. Answer 4 sets the governance tier. Answer 5 tells you whether you have just inherited distributed-systems failure modes on top of the model ones.
The takeaway
There is a reason the antibiotic analogy keeps working. Reaching for the strongest option isn't caution, it's the opposite. It adds cost, side effects and resistance where a simpler intervention would have done the job. And the patient rarely knows enough to push back.
Our job as architects is to be the doctor who says "this will clear up on its own" when that's the truth and, when it won't, to prescribe at the right level rather than the most impressive one.
That conversation is harder than it sounds. Telling a customer their problem needs less technology than they walked in asking for is not what most vendors are incentivised to say. In my experience it is exactly what keeps them as customers.
👉 Not every fever needs an antibiotic. Not every workflow needs an agent.
💬 What's one AI use case in your organisation that you suspect is over-designed — sitting at level 5 when level 1 would have done the job?
Sources
- Anthropic Engineering, Building effective agents (2024) — workflows vs agents, the five workflow patterns
- Harrison Chase / LangChain, What is a "cognitive architecture"? (2024) — the six-level ladder
- OpenAI, A Practical Guide to Building Agents (2025) — single-agent loop, manager and handoff patterns
- Berkeley AI Research, The Shift from Models to Compound AI Systems (2024) — source of the 60% / 30% Databricks figures
- Sapkota, Roumeliotis & Karkee, AI Agents vs. Agentic AI: A Conceptual Taxonomy (arXiv 2505.10468, 2025)
First published on LinkedIn.