The 5 Questions to Ask Before You Trust Any AI Tool With Numbers

Why This Matters Right Now

Sometime this year, someone in your company will demo an AI tool that produces numbers: a spend summary, a variance analysis, a risk score. The demo will look clean. The numbers will look confident. And in that meeting, almost nobody asks the only question that matters: where did that number come from?

Here is the uncomfortable fact behind every AI chat tool: a language model is a word predictor, not a calculator. When it produces "revenue grew 14%", it is generating plausible text — and plausible is not the same as correct. This is the same reason AI invents facts in ordinary reports; with numbers, the failure is just harder to catch by eye. A well-built AI tool knows this about itself and is engineered around it. A badly built one hides it.

I learned the difference by building a tool where a wrong number isn't an inconvenience — it's the end of the product. The five questions below are the ones that architecture forced me to answer. They work just as well from the other side of the table, when you're the one evaluating.

Question 1: Where Does the Math Happen?

The single most important design rule for any serious AI-and-numbers product: math lives in code, language lives in the model. Ratios, totals, growth rates, scores — all computed by ordinary, testable software. The AI's only job is to explain numbers it was handed, never to produce them.

Why it matters: code that computes a ratio gives the same answer every time and can be unit-tested against known values. A model that "computes" a ratio is guessing, and it will guess differently on Tuesday than it did on Monday.

Ask the vendor: "If I run the same report twice, do I get identical numbers? Which numbers in this screen were computed by code, and which were generated by the model?" A good vendor answers instantly, because the boundary is a design decision they made on day one. A vague answer means the boundary doesn't exist.

In your own prompts: never ask a chat AI to do arithmetic across a pasted table. Compute in Excel; ask the AI to explain, compare, or draft the commentary on numbers you supply.

Question 2: Can Every Number Be Traced to Its Source?

In a trustworthy tool, every displayed figure carries a citation that round-trips: click it, and you land on the exact line of the source document — the specific note, page, and line item it came from. Not "based on your uploaded files." The exact line.

This is checkable in a demo. Pick any number on the screen and ask to see its source. Then check that the figure on the screen matches the figure in the source, digit for digit.

The stricter version — and the mark of a genuinely serious product — is that this check runs automatically: the system verifies every cited figure against the source data, and a claim that fails verification is never shown to the user at all. Rejected, not repaired.

For every figure in your summary, quote the exact sentence from the document it came from. If you cannot produce the exact quote, do not include the figure.

Question 3: What Happens When the Data Isn't There?

Ask the tool a question its source data cannot answer, and watch. There are only two possible behaviors:

• The honest tool says so. "Insufficient evidence" is a first-class answer, not an error. Finding nothing to flag is displayed with the same confidence as finding something.
• The dishonest tool fills the gap with something plausible — and plausible fabrication inside a professional report is how careers end.

This is the cheapest test in this article and the most revealing. In the demo, deliberately ask for something you know is missing from the sample data. A tool that has never been designed to abstain will invent.

If a figure is not explicitly stated in the text below, reply only with: "Not in the document." Do not guess or estimate it.

Question 4: Is the Confidence Earned?

Many AI tools display confidence scores. Here is what most people don't know: a model's self-reported confidence is decoration. Research has repeatedly shown models are poorly calibrated about their own certainty — they sound equally sure when right and when wrong.

Earned confidence looks different, and you can recognize it by two features:

• Confidence is capped by evidence quality, not vibes. If a claim rests on a single source — one document, self-reported figures, no independent corroboration — a serious tool refuses to display high confidence for it, no matter how strong the claim sounds. The ceiling comes from where the evidence came from, applied by rules, not chosen by the model.
• The tool argues against itself. For every red flag or finding, it also shows the innocent explanations. A spike in write-offs can mean trouble — or a one-time cleanup after an acquisition. A tool that only ever prosecutes is a hype machine; a tool that shows you both readings is doing analysis.

Ask the vendor: "What caps a confidence score in your system? Show me a finding where the tool itself lists benign explanations."

Question 5: Does It Decide, or Help You Decide?

The final question is about framing, and it's the one with legal teeth. A well-designed analysis tool never outputs a verdict — "this is fraud," "this vendor is non-compliant," "reject this claim." It outputs areas to investigate, ranked and evidenced, and leaves the decision to a named human.

This isn't timidity. It's the correct division of labour: the tool is good at reading 300 pages and surfacing what deserves attention; you are accountable for the judgment. Any tool that presents its output as a decision is quietly transferring its errors onto whoever clicks "accept" — which, in your company, will be you.

Ask the vendor: "Show me the wording of a negative finding." If the language is accusatory rather than investigative, the tool's designers haven't thought about who carries the risk. You will be the one forwarding that language to your reporting manager. (The same discipline applies to picking the AI itself — see which paid AI subscription is actually worth it for the vendor side of this question.)

The 5-Minute Fix: Run This in Your Next Vendor Demo

Copy this checklist into your notes app before the meeting:

AI TOOL TRUST CHECK — 5 questions, ~5 minutes

1. MATH: "Which numbers here are computed by code vs generated
   by the model? Same report twice = identical numbers?"
2. TRACE: Pick any figure on screen. "Show me its exact source
   line." Verify digit-for-digit.
3. MISSING DATA: Ask for something the sample data doesn't
   contain. Does it abstain or invent?
4. CONFIDENCE: "What caps a confidence score? Show me a finding
   with its benign explanations."
5. VERDICTS: "Show me the wording of a negative finding." Is it
   "investigate this" or "this is X"?

Score: any invented number, untraceable figure, or confident
guess on missing data = do not trust it with real work yet.