About

All True Fact is one engine with two fronts: this site with its MCP tool, and the All True Fact YouTube channel, where one claim per video is scored on screen as the evidence lands.

What it does

It takes a contested claim, gathers evidence for and against it, weights each document by source quality, directness and independence, adds the pushes in log-odds, and reports where the number landed and how much evidence supports landing there. It runs the same procedure on five different language models over the same evidence and publishes the divergence. "Insufficient evidence" is a verdict it gives often, and a claim that is a values question is refused rather than scored.

What it does not do

It does not remove the model's judgment. Source quality, directness, direction and what a document rests on are each a model's call, and a lean in any of them can run across many small calls. The engine makes every call narrow, blind where it can be, recorded with the model string and prompt version that made it, and cross-checked against four other models. That is compartmentalising bias and making it auditable. It is not eliminating it, and no page here says it is. The methodology lists what is still judgment; the version history retracts the claims the previous site made.

The channel

Politics is the lane, starting with historical claims and walking recency forward, because it is where the information environment is worst and an honest scored verdict has the most value. Every image in a video is a real document, real footage, or a chart of real numbers. The scoring is done by language models and the narration is synthesised. Every video's dataset is published here so that "show me the data" has an answer.

Cost and models

All model calls go through OpenRouter to pinned model versions. Published datasets use the public profile: five inexpensive models, roughly a dollar or two per claim. A reference set of frontier models is used for methodology episodes and spot checks, and switching profiles never changes a published verdict.

Open problems

• Detecting that two documents rest on the same source is hard, and getting it wrong wrecks the verdict. The clustering is one model's opinion and is shown in full.
• Residual model judgment can lean the same way across many small calls. Running five models exposes some of it; it does not remove it.
• Backtests leak: claims resolved before a model's training cutoff are already known to it. The clean measurement is rubric versus no-rubric on the same claims, and that is what the methodology reports.

Contact

support@alltruefact.com