How to reduce AI hallucinations in e-commerce: a practical guide to RAG

When an e-commerce chatbot quotes a discontinued product at last season's price, or confidently describes a return policy that changed three weeks ago, no amount of prompt engineering will fix it. The model followed its instructions correctly. It had the wrong context to work from. That is a retrieval problem, and retrieval is where the fix has to happen.

Retrieval-augmented generation (RAG) is the architectural answer. This guide covers why hallucinations happen at the model level, how they map to concrete e-commerce failure modes, and what a production RAG pipeline actually looks like to build and maintain.

We also draw on our experience building Chatguru, Netguru's open-source, white-label RAG chatbot, to show where theory meets production.

Why LLMs hallucinate by design

Large language models (LLMs) do not look things up. They generate text by predicting the most probable next token given everything before it. That mechanism is powerful for language tasks and largely indifferent to factual accuracy.

A few properties make hallucinations structurally likely without external grounding:

They were designed as pattern matchers. LLMs were built to detect statistical regularities in training data and generate text that fits those patterns. In practice, this means no built-in mechanism to verify claims, access ground truth, or know when training data is outdated. Whether modern models, particularly multimodal ones, develop reasoning capabilities beyond this is an active research question. What matters for production e-commerce systems is the practical implication: you cannot rely on the model to self-correct when it does not have the right context.

They have no concept of what they do not know. A model that has never encountered a specific product will not say "I have no data on this." It will generate a plausible description based on similar products in its training set, with full confidence.

They are designed to give you the output you expect. Training rewards responses that look correct in structure and tone, not responses that are factually grounded. A hallucinated product description reads exactly like a real one.

Telling a model to "be accurate" or "only use verified information" in a system prompt does not solve this. Those instructions are followed at the generation layer. The underlying issue is at the retrieval layer.

E-commerce hallucination taxonomy: the four main failure types

Hallucinations in e-commerce do not all fail the same way or cost the same amount. Separating them into a practical taxonomy helps teams prioritize what to fix first.

The most useful distinction is between intrinsic and extrinsic hallucinations. Intrinsic hallucinations occur when the model contradicts the context it was given: it had the right information and still got the answer wrong. Extrinsic hallucinations occur when the retrieved context contains no relevant information, forcing the model to generate from its training weights.

Both types produce wrong answers. They fail differently and require different fixes.

Price hallucinations

A product's price is updated in the PIM system, but the vector database index has not been refreshed. The model retrieves stale data and cites the old price. The shopper adds to cart, hits checkout, sees a different number, and abandons.

Business risk: Cart abandonment, price dispute tickets, potential compliance exposure in regulated categories.

Availability hallucinations

A discontinued or out-of-stock product is not purged from the index. The chatbot recommends it confidently. The shopper searches, finds nothing, contacts support.

Business risk: Support ticket volume, negative post-interaction sentiment, and eroded confidence in AI recommendations overall.

Attribute hallucinations

The retrieval system surfaces a similar but wrong product. The model describes materials, dimensions, or compatibility specs that belong to a different item.

Business risk: Returns for items that did not match the described specification (post-purchase trust damage).

Recommendation hallucinations

Retrieval returns nothing relevant, so the model fills the gap with a product that sounds plausible but does not exist in the catalog.

Business risk: Dead-end shopper journeys and loss of confidence in the AI layer entirely. This is the extrinsic failure mode: the fix is a coverage problem in the index, not a staleness problem.

What is RAG and why it matters for e-commerce

Retrieval-augmented generation is an architecture in which the LLM never relies solely on training knowledge. Before generating a response, the system retrieves relevant documents from an external knowledge base — your product catalog, return policies, inventory data — and injects them into the model's context window. The model responds based on that retrieved evidence, not on what it learned months ago.

The mechanism directly addresses the root cause of e-commerce hallucinations: training knowledge is frozen, while your catalog changes daily.

Three concepts underpin how it works:

Embeddings convert text — a product description or a user query — into a dense vector of numbers. Vectors that are close together represent semantically similar content. This is what allows "breathable running shoes under $100" to retrieve "Lightweight Mesh Trainers" even when no word overlaps.

Vector databases store embeddings at scale and enable fast similarity search. The query is embedded and compared against the index; the database returns the most relevant results from your catalog, not from model memory.

Semantic search is the result: a retrieval system that understands intent rather than matching keywords. A query for "orange shoes" should not retrieve fruit. Metadata pre-filtering prevents that category confusion, though it remains a real failure mode in poorly configured setups.

How a RAG pipeline works in practice: from chunking to generation

A production RAG pipeline has six stages. Each one introduces a tradeoff worth understanding before you tune anything.

Stage 1: Ingestion and chunking

Your product catalog is split into chunks: smaller units the embedding model represents as single vectors. Chunk size is an engineering decision with real consequences.

For e-commerce, chunk at attribute level, not page level. A single product page mixes a title, price, bullet attributes, long-form description, and availability. Embedding the whole page produces a vector that accurately captures none of them.

The approach that works: one chunk for structured attributes (price, availability, return policy), a separate chunk for the product description. Structured attribute chunks should have no overlap window — it adds noise without improving recall. Description chunks benefit from a small overlap (around 10% of chunk size) at sentence boundaries.

What a well-structured attribute chunk looks like in practice:

{
  "sku_id": "RUN-SHOE-001",
  "name": "Lightweight Mesh Trainer",
  "price": 89.99,
  "currency": "USD",
  "availability": "in_stock",
  "return_window_days": 30,
  "category": "footwear",
  "subcategory": "running",
  "indexed_at": "2025-06-01T08:00:00Z"
}

A separate description chunk for the same SKU:

{
  "sku_id": "RUN-SHOE-001",
  "chunk_type": "description",
  "content": "The Lightweight Mesh Trainer features a breathable upper designed for
              long-distance running. The responsive foam midsole absorbs impact on
              hard surfaces. Available in sizes 6 to 13.",
  "indexed_at": "2025-06-01T08:00:00Z"
}

Store SKU ID, category, and price tier as metadata on every chunk. Apply pre-filtering by category before the nearest-neighbor search runs. This narrows the candidate set from millions of SKUs to hundreds before semantic similarity is calculated, cutting both latency and false-retrieval rate.

Stage 2: Embedding

Each chunk is passed through a bi-encoder embedding model, such as text-embedding-3-large or a fine-tuned bge-large-en, which converts it to a dense vector. Bi-encoders are fast enough for real-time retrieval but trade some cross-attention accuracy for that speed. That tradeoff matters at stage four.

Stage 3: Vector indexing

Chunks are indexed in a vector database. For large catalogs, graph-based indexing (such as HNSW) can significantly reduce query latency compared to a flat index, at the cost of a small recall loss. The right choice depends on your catalog size, latency requirements, and infrastructure constraints. Benchmark both options early rather than retrofitting later.

Stage 4: Retrieval and reranking

The user query is embedded and compared against the index using approximate nearest-neighbor (ANN) search — a technique that finds semantically similar results quickly by searching a structured index rather than scanning every document. ANN is fast enough for real-time chat but may occasionally miss the single most relevant result.

The top ANN candidates then pass to a cross-encoder reranker, which scores each candidate against the query jointly. This is slower but more accurate: the document ranked third in the initial results may be more relevant than the one ranked first, and reranking corrects that before context is injected. For latency-sensitive interactions like autocomplete, route around the reranker and measure the precision tradeoff for your specific catalog.

Stage 5: Context injection

The top reranked results are assembled into the context window. Place the highest-confidence result first — some models are less reliable with information buried in the middle of a long context.

Stage 6: Response generation

The LLM generates a response constrained to the injected context, with guardrails instructing it to surface a fallback when evidence is missing and never to extrapolate prices or availability beyond what was retrieved. Lower temperature settings reduce creative divergence and tighten outputs around source material. Good retrieval does the heavy lifting; temperature is the final tightening pass.

Catalog staleness: the hidden hallucination trigger

Catalog staleness happens when your vector index falls out of sync with your actual product catalog. A product gets discontinued or repriced, the PIM update propagates correctly, but the index is refreshed on a weekly batch schedule. The chatbot quotes the old price because the stale data still surfaces as the top semantic match.

The reason this is so hard to catch: it will not surface in pre-launch testing, and it will not show up in standard faithfulness metrics either. Faithfulness only measures whether the answer reflects the retrieved content, not whether that content reflects the current world.

We observed this in an early Chatguru deployment, before enforcing a freshness SLA. The chatbot's faithfulness score stayed high throughout, but the data it was citing reflected a catalog state that no longer existed.

The fix requires two things: a freshness enforcement mechanism in the pipeline, and a test dataset that includes recently discontinued products and repriced items, run regularly in production rather than just at launch.

The three enforcement mechanisms we use in Chatguru's pipeline:

Event-driven re-indexing. PIM webhooks trigger chunk deletion and re-embedding within minutes of a status change: discontinued, repriced, or out-of-stock. More reliable than batch jobs because it responds to actual events rather than polling.

@app.route("/webhook/sku-updated", methods=["POST"])
def handle_sku_update():
    payload = request.json
    sku_id = payload["sku_id"]
    event_type = payload["event_type"]

    if event_type == "discontinued":
        vector_store.delete(filter={"sku_id": sku_id})
    else:
        updated_chunks = build_chunks(fetch_sku(sku_id))
        vector_store.delete(filter={"sku_id": sku_id})
        vector_store.upsert(updated_chunks)

    return {"status": "ok"}, 200

The key is delete before upsert: purge stale vectors before adding fresh ones, rather than relying on a version flag that similarity search might still surface.

TTL metadata on every chunk. Each document carries an indexed_at timestamp. The retrieval layer filters out anything older than your freshness SLA threshold, calibrated against how frequently your most volatile fields change.

results = vector_store.query(
    query_vector=query_embedding,
    filter={
        "category": {"$eq": user_category},
        "indexed_at": {"$gte": datetime.utcnow() - timedelta(hours=YOUR_SLA_HOURS)}
    },
    top_k=20
)

Hard-delete on discontinuation. Retiring a product triggers an immediate index purge. Soft-flagged documents still get queried; hard-deleted ones do not.

One note on semantic similarity: embedding models find vectors that are similar, not exact matches. A query for "orange shoes" in a poorly configured index may surface orange-colored products broadly because the embedding space treats "orange" as a color attribute without category context. Metadata pre-filtering — requiring category to match before similarity search runs — prevents this class of false retrieval.

RAG vs. fine-tuning vs. agentic AI

These three approaches are frequently confused. Choosing the wrong one is an expensive mistake.

RAG is the right choice for live knowledge retrieval. The model's weights stay fixed; only the context window changes per request. Use it when the information the chatbot needs changes frequently: catalog, pricing, inventory, promotions, return policies.

Fine-tuning adjusts model weights through additional training. It is effective for shaping behavior — tone, response format, brand vocabulary — but does not reliably encode factual knowledge that changes over time. A model fine-tuned on today's catalog will hallucinate about next quarter's inventory. Fine-tuning complements RAG rather than replacing it: use RAG to ground facts, fine-tuning to shape communication style.

Agentic AI goes beyond retrieval into action: calling APIs, executing workflows, making multi-step decisions. Reasoning models work well as planners that orchestrate complex decisions; faster models serve as doers that execute steps. Agentic systems introduce a different hallucination risk: agents take actions, not just generate text. A hallucinated refund amount triggers downstream processes. Guardrails for agents require human-in-the-loop review for high-stakes actions, with hard permission boundaries.

Diagnosing hallucinations in production

Knowing that your system hallucinated is less useful than knowing why. The root cause is almost always one of two things: retrieval failure or generation failure.

Retrieval failures happen when the system returns stale, semantically adjacent but incorrect, or simply absent content. The model then generates from bad inputs or training memory. This is the more common and more costly class in e-commerce systems.

Generation failures happen when the retrieval layer returns correct content but the model distorts it: paraphrasing a number incorrectly, blending two retrieved chunks, or extrapolating beyond what the context states. These point to prompt guardrail gaps or temperature settings that are too high.

How to separate them in practice:

Log retrieved chunks on every request. You cannot diagnose a retrieval failure without seeing what the model was working from. Langfuse provides trace-level logging that captures retrieved content alongside the generated response. Instrumenting a call takes around ten lines:

from langfuse import Langfuse
from langfuse.decorators import observe

langfuse = Langfuse()

@observe()
def rag_query(user_query: str) -> str:
    chunks = retrieve(user_query)
    context = format_chunks(chunks)
    response = generate(context, user_query)

    langfuse.trace(
        name="rag_query",
        input=user_query,
        output=response,
        metadata={"retrieved_chunks": [c.id for c in chunks],
                  "top_score": chunks[0].score}
    )
    return response

With this in place, any hallucination report maps back to a specific trace, and you can see exactly which chunks the model was working from when it gave the wrong answer.

Measure context precision separately from faithfulness. Faithfulness tells you whether the answer reflects retrieved content. Context precision tells you whether that content actually contained the answer. High faithfulness with low context precision means the model is accurately summarizing the wrong documents: a retrieval problem, not a generation problem.

Run LLM-as-judge evaluation on a sample of production traffic. A secondary model, isolated from the response-generation call, classifies responses as grounded or ungrounded, on-topic or off-topic. This distinguishes retrieval failures from generation failures at scale without manual review.

Monitor similarity score distributions. A drop in average confidence scores across a product category often precedes a rise in hallucination reports by one to two days, signaling an index freshness problem before customers surface it.

Maintain a hallucination dataset. Every confirmed hallucination from customer reports, support tickets, or automated flags should become a permanent test case. Run it on every pipeline change.

Lessons from building Chatguru: real-world implementation insights

Chatguru is Netguru's open-source, white-label RAG chatbot. White-label means the branding, conversation flows, and integrations are fully configurable per client, so the same core engine can power a multi-brand marketplace or a single DTC storefront without a rebuild. Being open-source and self-hosted also means the underlying retrieval logic can be adapted to whatever catalog structure a business already runs on.

That flexibility is what makes the architecture a fit for e-commerce. Product catalogs are large, change constantly, and often span multiple languages and markets. Retrieval needs to stay current as prices and stock shift, precise enough to isolate a single attribute like a return window, and aware of language boundaries.

Catalog freshness is treated as a first-class concern. Weekly re-indexing lets stale product data sit in the index for days. Chatguru supports event-driven re-indexing on high-churn fields like price and stock, so the index reflects catalog changes close to real time.

Reranking is built in as a second accuracy layer. A cross-encoder reranking pass catches relevant matches that retrieval alone can miss, at a latency cost that's worth paying on synchronous chat paths. Autocomplete-style interactions route around the reranker to stay fast.

Chunking happens at the attribute level, not the page level. A question about return eligibility needs to surface the return-policy chunk directly, not a full product page where the answer sits in paragraph three.

Multilingual retrieval treats language as a metadata dimension. A Polish query shouldn't retrieve chunks embedded from English descriptions, so language is filtered explicitly rather than left to embedding similarity alone.

Metadata governance is pushed upstream. When category taxonomy shifts mid-catalog, pre-filters break for affected products. Chatguru's design assumes the fix belongs in the PIM, not patched at the vector store layer.

Measuring hallucination rates in production

Hallucination monitoring is an ongoing production metric, not a pre-launch checklist item. The failure modes that matter most — catalog staleness, low-confidence retrieval, and generation distortion — all drift over time as your catalog evolves.

The four metrics that matter most:

Faithfulness score measures whether the answer contains only claims supported by retrieved content. Target above 0.85. Below 0.80 is a production incident.

Answer relevancy measures whether the answer addresses the actual query. A response can be faithful to its retrieved context and still fail to answer the question. Target above 0.80.

Context precision measures whether the top-ranked retrieved chunks actually contained the answer. Low context precision with high faithfulness is the signature of a retrieval configuration problem: the model is accurately summarizing the wrong content.

Retrieval recall measures whether the correct document was retrieved at all. Low recall on a specific product category usually signals a chunking or metadata filtering problem.

Grounded AI protects revenue

Many e-commerce AI hallucinations trace back to the same root cause: the model responded without verified, current context. A wrong price, a discontinued recommendation, a misquoted return policy — all are retrieval gaps, not model failures.

The fix is well understood: RAG grounds responses in retrieved catalog data, freshness SLAs keep that data current, and production metrics tell you when grounding breaks down. The challenge is running all three consistently as your catalog evolves.

Teams that treat hallucination reduction as a launch task rather than an ongoing operational concern are the ones surprised six weeks post-deployment, when a discontinued product is still being confidently recommended.

If you want to see how Chatguru handles these concerns in a production e-commerce environment, book a demo with our AI Consulting Lead.

FAQ

Does RAG completely eliminate AI hallucinations?

No. RAG eliminates hallucinations that stem from missing or outdated knowledge. It does not prevent faithfulness failures, where the model distorts retrieved context. Those require prompt guardrails, temperature tuning, and output validation. RAG eliminates the most common cause, not all causes.

What is the difference between RAG and agentic AI?

RAG retrieves documents and injects them into the context window before generation. The model reads and responds. Agentic AI takes actions: calling APIs, executing workflows, making multi-step decisions. A product advisor chatbot is a RAG use case. An order management system that initiates refunds is an agentic use case. Most production e-commerce AI uses both.

What is the difference between RAG and fine-tuning?

RAG changes what the model reads on each request. Fine-tuning changes how the model behaves by adjusting its weights. Use RAG when information changes frequently: catalog, pricing, policies. Use fine-tuning to shape tone and domain vocabulary. The two are complementary: fine-tune for behavior, use RAG for facts.

Why do e-commerce AI systems hallucinate so often?

Three reasons compound. Catalogs change frequently, creating staleness in the retrieval index. Queries are specific — exact prices, return windows, variant availability — where any deviation is immediately detectable. Without RAG, deployed chatbots rely on training weights for product knowledge that was never in their training data.