There is a moment in almost every manufacturing AI demo where somebody in the room asks the obvious question.
They have just watched the assistant pull a torque specification out of a 200-page equipment manual in four seconds. Genuinely useful. So they lean forward and ask it something operational: how many of part 4471-B do we have on hand right now. And the assistant answers. Confidently, in a full sentence, with a number. The room nods. Somebody says this changes everything. What nobody in that room does is walk out to the rack and count, which is unfortunate, because the number was wrong. Most conversations about RAG implementation services never get to this distinction, and it is the one that decides whether the system survives its first quarter.
The failure here is not a bug, and the vendor is probably not being dishonest. It is a category error, and it is the single most common reason manufacturing AI pilots quietly get switched off around month four.
Retrieval Is a Snapshot, and Inventory Is Not
Retrieval-augmented generation works by converting a body of text into vectors, storing them, and finding the passages closest in meaning to whatever the user asked. It then hands those passages to a language model and asks it to answer using them. When the underlying material is a maintenance manual, a work instruction, a material certificate, or a supplier agreement, this is close to ideal. Those documents are written once, revised rarely, and their meaning does not change between Tuesday and Wednesday.
Inventory is the opposite kind of fact. Quantity on hand is not a document. It is the current state of a ledger that receives hundreds or thousands of writes a day, from receiving, from putaway, from pick confirmations, from production consumption, from cycle count adjustments, from returns. It has a value at 9:14 a.m. and a different value at 9:16 a.m.
When a retrieval layer is pointed at inventory data, what it stores is the value at the moment of indexing. Ask the system a question three weeks later and it will return that indexed number, presented in exactly the same confident register it uses for the torque spec. There is no signal in the output distinguishing a stable fact from a stale one.
This is a recognized and well-mapped failure mode. Barnett and colleagues catalogued seven distinct RAG failure points across three real-world case studies, and stale or misaligned retrieval sits near the top. In legal applications, where the sources are far more stable than a warehouse ledger, studies have measured RAG hallucination rates between 17 and 33 percent. A survey of enterprise deployments found that around 80 percent of RAG projects hit critical failures, with roughly one in five reaching sustained production use.
The problem is not that these systems are unreliable in general. It is that they are being asked a class of question they were never designed to answer, and they do not have a mechanism for declining.
What RAG Is Genuinely Very Good At
It is worth being specific about the wins, because the category error runs in both directions and a plant that concludes RAG is overhyped will leave real value sitting on the table.
Mid-market manufacturers accumulate enormous unstructured archives. Engineering drawings going back decades. Equipment manuals from vendors who no longer exist. Material certifications, inspection reports, deviation records, customer specifications with clause-level requirements, supplier quality agreements, calibration histories. This material is genuinely knowledge, it is genuinely hard to search, and the person who knew where it all lived retired in 2019.
We worked with an industrial supplier whose engineering team spent up to eight hours locating specifications across a document library built over roughly forty years. A retrieval layer over that archive brought typical lookups down to under thirty minutes. Nothing about that result required the system to know a live quantity. Every question it answered was a question about a document, which is precisely the shape of problem retrieval solves.
The same is true for proposal support, onboarding, compliance evidence gathering, and answering “have we ever built something like this before.” These are high-value, document-shaped questions, and a well-built retrieval layer handles them with citations back to the source page, which is what makes the output auditable.
Why “Just Connect It to the Database” Does Not Fix It
The instinct at this point is to give the model direct database access and let it write its own queries. This is a better instinct than indexing the inventory table, and it is still harder than it sounds.
The BIRD benchmark was built specifically to test how well language models translate natural language into SQL against real, messy databases rather than tidy academic ones. On its test set, GPT-4 reached about 55 percent execution accuracy when it was handed curated domain hints explaining what the columns meant. Without those hints it dropped to roughly 35 percent. Human performance on the same task sits near 93 percent.
Now consider the gap between that benchmark and a real plant. BIRD’s databases average around seven tables. A production ERP schema runs to hundreds or thousands of tables, with column names inherited from a 1990s data model, custom fields added by three different consultants, and business logic encoded in conventions nobody documented. If a model gets it half right on seven tables, treating it as a free-form query engine against a live ERP inventory management schema is not an integration decision. It is a coin flip with a professional tone of voice.
And there is a second problem underneath the first, which is that a perfectly correct query against a wrong record still produces a wrong answer. CAPS Research found average inventory accuracy across surveyed businesses sitting at 83 percent, with only about 69 percent of companies tracking the metric at all. Other analyses put more than half of manufacturers and distributors below 80 percent accuracy. MRP and ERP planning logic generally needs 95 to 99 percent accuracy to function as designed.
So the honest version of the answer to “how many do we have” is frequently: the system says 412, the system has been wrong by more than five percent on this item class in each of the last three cycle counts, and the last physical verification of this bin was in March. That is a more useful answer than 412. It is also an answer no retrieval layer will ever produce on its own, because producing it requires the architecture to know what it does not know.
The Architecture That Actually Holds Up
The systems that survive contact with a plant floor separate the question types before answering, rather than routing everything through one pipeline. In practice that means four things.
A router that classifies the question before answering it. Document questions go to retrieval. State questions go to a live query. Questions that are actually requests for action go to a workflow with an approval step. This classification layer is unglamorous and it is the difference between a system that is trusted and one that is not.
Parameterized reads instead of free-form SQL. A defined set of read-only queries against agreed views, with the model selecting and filling parameters rather than composing arbitrary SQL. This gives up some flexibility and buys back determinism, which is the correct trade for anything a planner will act on.
Provenance and timestamp on every answer. A document answer cites the document and revision. A state answer states the source table, the time the value was read, and, where the plant tracks it, the accuracy history for that item class. An answer with no timestamp is not an answer, it is an assertion.
A human approval gate on anything that writes. Reading is low risk. Creating a purchase order, adjusting a quantity, or releasing a work order is not. The person who owns the outcome should own the click, with the reasoning and source data in front of them.
None of this is exotic architecture. It is mostly the discipline to admit that two different questions need two different mechanisms, at the point in the project where it is tempting to promise one interface that answers everything.
Four Questions Worth Asking a Vendor
If you are evaluating a manufacturing AI proposal, these four will tell you most of what you need to know within twenty minutes.
How does the system distinguish a document question from a live state question, and what happens at that boundary? If there is no boundary, everything downstream is guesswork.
When I ask about quantity on hand, what exactly gets read, and how old is it? Vague answers here are the whole ballgame.
What does the system do when it cannot answer reliably? A system that has no refusal behavior will hallucinate instead of declining, because declining was never built.
Show me an answer with its sources and timestamp attached. Not in a slide. In the working system.
The Test to Run This Week
If you already have an assistant deployed, run one experiment. Ask it a quantity question for an item you can physically verify. Then go verify it. Then ask it the same question again and see whether the answer changed after the count.
That takes about forty minutes and it tells you more about your AI investment than any dashboard will. A system that gets the document questions right and honestly declines the state questions is working correctly. A system that answers everything with equal confidence is not more capable. It just has fewer ways of telling you it is wrong.
About the Contributor
Nishkam Batta, Editor-in-Chief, HonestAI Magazine | AI Consultant, GrayCyan AI Solutions
Nish leads an applied AI company helping manufacturing and related companies automate operations with human-in-the-loop AI that integrates into ERPs, WMS, CRMs, and other enterprise tools, with an emphasis on explainable AI, clear audit trails, and measurable outcomes. His team builds agentic ERP systems that execute multi-step tasks inside approved guardrails so humans keep accountability, approvals, and override control.
