Fair question, and worth answering properly instead of asking you to just trust us. Here's exactly what happens between a claim and a verdict, written so you can explain it to someone else, not just take our word for it.
When you ask an AI a question directly, it answers from memory — patterns it picked up during training. That's fast, but it means the AI is reciting something it half-remembers, and half-remembered things are exactly where AI confidently states things that aren't true. That's what "hallucination" means: not lying, just recalling wrong with full confidence.
For every fact in this dataset, we don't ask an AI what it remembers. We send it to go find the real answer, live, from the actual source, the way you'd send a research assistant to go pull the real document instead of answering from the top of their head.
Someone states a fact — a founding year, a court ruling, a lab result, a company's reported revenue.
We send Claude to the real source right then: a government database, an actual SEC filing, the real text of a court opinion. Nothing is answered from recall.
Its job is to find the specific fact on that real document and write down exactly what it says, not to answer from what it thinks it knows.
The claim is marked match, diverges, or unverifiable, and the exact source link ships with the row so anyone can check it themselves in under a minute.
Row MV02 in the Mathematical Claims Verification dataset. The claim: that 2³¹−1 (2,147,483,647) is prime, first proved by Leonhard Euler in 1772. Here's what the live lookup actually found, and, right below it, what happened when we separately asked Claude the identical question completely cold, no source, no lookup, just memory.
Same underlying fact, same model. Looked up live against a real source: correct, with a link you can click. Answered from memory alone: a confidently wrong name and a confidently wrong year. That gap is the entire reason this business exists.
There's still a step in the middle done by AI: something has to actually find the right figure on a real document and write down what it says. Documents are messy, so there's a small, real chance of grabbing the wrong line or misreading a table. We don't pretend that risk is zero.
That's exactly why every row ships the source link instead of asking for your trust, why several products deliberately check two independent sources instead of one so a mistake in one gets caught by the other, and why a claim that can't be confirmed is marked unverifiable rather than forced into a guess. Verify us. Don't just trust us.
We tested this directly: every claim in the catalog, asked to Claude completely cold, no source, no tools, exactly like a normal chat. The error rate depends heavily on the domain.
Across all 703 claims tested this way, the overall rate was 10.1% hallucinated, 73.8% correct even from memory, and 15.4% honestly hedged rather than guessing. Full numbers are in every product's README.
Sourcing standards, the null-over-guess rule, and exactly which primary sources back each product.