Your Model Is Fine. Your Data Plumbing Is Lying to You.

You swapped the model. You rewrote the prompt. You added "only answer from context" in bold. The wrong answers kept coming. That is because you were fixing the wrong layer. The model was reasoning fine. It was reasoning over bad evidence.

Six pieces this week land on the same spot from different angles: retrieval, freshness, intent, validation, data truth. They all point at the plumbing under your AI, not the brain on top of it. Let me catch you up.

The model does exactly what you told it

A support engineer watched a RAG chatbot explain the wrong refund policy in beautiful prose. The retriever had pulled a paragraph that said "refund" four times. Topically close. Also for a discontinued product line. The model answered with full confidence because nobody told it the context was wrong.

That is the pattern behind most RAG failures that get debugged. Teams spend two weeks tuning generation while the real damage sat upstream in chunking and retrieval. The hallucination rate barely moves because generation was never the problem.

So before you touch the model again, ask a plainer question: did the right evidence even reach it? If the answer is no, a smarter model just gives you a more convincing wrong answer.

High confidence, stale answer

Here is the failure that hides best. An enterprise RAG system returns outdated, flatly wrong info while logging a 0.90-plus cosine relevance score. To your dashboard, the system looks healthy. To the business, it is breaching policy.

Maya Chen calls this silent index drift. The words "refund policy" mean the same thing in 2024 and 2026, so the old chunk and the new chunk sit almost on top of each other in vector space. Vector search has no sense of time. Worse, when a source doc gets updated or moved, the old chunk often lingers as a "ghost embedding" nobody deleted.

The fix is not a better embedding model. It is metadata filters on version and status, hybrid search that catches exact version numbers, a cross-encoder reranker, and an event-driven sync that deletes the old chunk when the source changes. Cosine similarity measures language proximity, not truth.

When your data agrees with a person who no longer exists

Recommenders hit the same trap from a different door. A user shops for a winter coat for three weeks. Strong, sustained signal. Then the coat arrives, the goal is done, and the system keeps serving coats. "Complete the look." Seven more versions of the thing they already bought.

Zenefa Rahaman breaks intent change into three modes: switching goals, completing a goal, and slowly becoming a different person. The pipeline lags by construction. It trains on a fixed window of history and gets scored on held-out history, so a model that perfectly predicts last month scores great even when the user wants something new today.

Her one diagnostic worth stealing: compare your state-retention window against a category's intent half-life. If you keep six months of behavior to predict a two-week intent window, you are structurally misaligned by an order of magnitude. You can compute that from logs you already have, no live test needed.

Green pipelines, wrong revenue

Move up a level and the same crack runs through your whole data stack. Snowflake can prove your pipeline ran. Databricks can prove your jobs completed. Neither proves the business truth survived the trip from source to decision.

Khushbu Shah frames this as a trust boundary. Inside its own walls a platform has full context. The moment data crosses into another system, it runs on assumptions. Her example: Salesforce adds an optional field that changes how discounts get recorded, but an old dbt transform keeps using the previous logic. No query fails. No schema breaks. No freshness alert. Revenue is overstated across every report, and finance closes the month on it.

The cost is real. An analysis of over 1,000 pipelines found 72% of data quality issues surface only after they have already hit business decisions. Unity Technologies tied roughly $110 million in lost revenue to corrupted ad-model training data while every pipeline job ran clean. A healthy pipeline guarantees execution, not correctness.

The deep cut

The common fix across all of this is a boundary, not a smarter model. Alexandru Rotari puts it plainly: let the LLM turn ambiguity into structure, then validate that output before anything executes. Schema checks, constraint enforcement, confidence thresholds. If the output fails, it loops back for another try. It does not proceed. That validation layer is a first-class part of the design, not a patch you bolt on when something breaks.

This is also your hedge on model churn. The capability gap between top models is narrowing and switching costs are low, so anchoring your system to one model is a bet that ages badly. Build the clean boundary and the model becomes a swappable part. The workflows keep running. The validation still holds. On Monday, the move is not "which model," it is "where does unverified data get to run without a check," and go close that gap first. Sage Holloway's routing rule is the same instinct: when the needed fact is missing, the system should say "insufficient evidence," not invent one.

Three questions for your team

1. When our AI gives a wrong answer, can we tell whether retrieval pulled bad evidence or the model reasoned badly? If we can't split those, we're debugging blind.

2. What is the retention window on our recommender or RAG index, and how does it compare to how long that intent or policy actually stays true? Run the ratio before the next planning cycle.

3. Where does LLM output or transformed data reach a downstream system with no validation step in between? List those spots, then decide which one gets a schema and freshness check first.