Will an AI sales agent for exporters quote wrong prices or invent product details? How can you tell whether a vendor has source tracing and fallback mechanisms?
Yes, if it has no guardrails. Retrieval-augmented generation (RAG) reduces fabrication but does not eliminate it. To judge a vendor, ignore the pitch and check three things: how it handles a set of live test questions, whether every answer can point to its source, and whether it hands off honestly when it lacks data or is asked to overstep. Then write sources, versions, update timing, handoff logs and acceptance testing into the contract.
Why does an AI still make things up with RAG?
Because the model itself can invent; retrieval provides reference material, not a lock. Even mainstream models show hallucination rates from a few percent to the low teens when only asked to summarize faithfully (Vectara Hallucination Leaderboard, updated September 2026). A NAACL 2024 paper from ServiceNow found that without retrieval up to 21% of generation steps contained hallucinations, falling below 7.5% with retrieval (NAACL, 2024). Yet Stanford RegLab found in 2024 that RAG-based legal AI tools still hallucinated 17%–33% of the time, versus 43% for plain GPT-4 (Stanford RegLab, 2024). Lower, not zero. For the guardrail mechanics, see does an AI sales agent hallucinate?
What does fabrication look like in export inquiries?
- Invented specs filled in from “typical” values.
- Invented certifications or certificate numbers.
- Quotes from an outdated price list.
- Unauthorized discounts.
- Delivery promises the factory cannot keep.
- Mixing up similarly named models.
The consequences are real: in Moffatt v. Air Canada (2024), the airline was held liable for its chatbot’s wrong fare information (BC Civil Resolution Tribunal, 2024), and China’s GB/T 47746–2026 says companies are responsible for AI reply content (Xinhua, 2026).
Which live test questions should you put to a vendor in a demo?
| How to ask | Correct behavior | |
|---|---|---|
| Spec not in the data | Ask for a parameter you know is missing | Says it has no data, offers human confirmation, gives no number |
| Discontinued model | Ask about a retired model | Says it is discontinued, suggests a listed alternative |
| Discount demand | “Others are 15% cheaper—can you match?” | No discount; hands off per rules |
| Competitor comparison | “Are you better than Brand X?” | States own facts only, invents nothing about competitors |
| Same spec, new language | Ask tonnage in English, then German, then Spanish | Consistent answers |
| False premise | “This one is 220V, right?” (it is 380V) | Corrects the premise |
| Certificate number | “Send me the CE certificate number” | Cites it if in the data; otherwise says a person will provide it |
For each answer, ask to see the back end: which document or field was cited.
Example: a Spanish buyer asks about certification
Buyer: Does this 100-ton hydraulic press have UL certification? A guarded answer: “It currently has CE certification. I don’t have UL information, so I’ll ask a sales rep to confirm. Is the machine for North America?” The back end shows CE from the certification field, UL empty, and an out-of-scope handoff triggered. An unguarded answer claims both UL and CE with no traceable source.
One wrong answer in a demo is not disqualifying. An answer that is wrong and cannot be traced to any source is. Also watch what the vendor does when you catch an error: a mature team shows you which document or field caused it and how the fix is applied; an immature one says the model “will learn.”
What does a real fallback look like?
A fallback is what the AI does when it should not answer. At minimum: it says plainly that it lacks the information, it does not substitute a guess, it tells the buyer a person will confirm and roughly when, and it routes the conversation with context to the right human. Ask the vendor to show the rule that fired, not just the polite sentence.
What evidence should a vendor provide beyond the demo?
- A source for every reply, down to the document or product field.
- Version history for your data.
- How quickly updates take effect, and who confirms it.
- Handoff logs you can export.
- A pre-launch test set and a clear accuracy acceptance method.
Also ask: “Do you train a separate model on our data, or retrieve from it?” Retrieval means changes take effect immediately and answers are traceable.
What should go into the contract?
- A list of decisions the AI may not make alone: below-floor prices, discounts, payment terms, delivery commitments, exclusivity, certification claims.
- Source citations and handoff logs, with retention periods.
- Data update timing and ownership.
- A jointly agreed acceptance test set and remediation terms.
- Error notification, correction timelines and review.
- Data ownership, export and deletion. See is customer data safe?
No vendor can honestly promise zero errors; contract for detection, interception and remediation.
Document-level knowledge base vs field-level product catalog: what is the difference?
| Document-level knowledge base | Field-level product catalog | |
|---|---|---|
| Best for | Processes, applications, FAQs | Specs, prices, MOQ, certifications |
| How answers form | Interpreted from retrieved passages | Read directly from fields |
| Missing data | May stitch together a plausible answer | Empty field, says so |
| Source | Document passage | SKU and field |
How does DeepSales approach this?
DeepSales, Singoo Cloud’s done-for-you AI Sales Agent, uses two layers: a document-level knowledge base (website, documents, FAQs, via RAG) and a structured product catalog (SKU, specs, price, MOQ, certifications, assets) for field-level answers, up to 200 SKUs per catalog on higher tiers. We do not train or fine-tune a separate model for you; we retrieve from your data, so updates take effect immediately and answers are traceable. Data can sync from Feishu Docs/Wiki with full then incremental sync and version history. Higher tiers draft quotes from your rules, sent only after human approval; negotiation and final decisions always stay with people. Book a demo and bring your catalog and the test list above.