The 2009 American Recovery and Reinvestment Act, the stimulus, reduced US unemployment below what it would otherwise have been.

Verdict per model

Every model scored the same evidence pool with the same rubric. Differences below are differences of judgment, not of retrieval. "Cold" is the model's own probability with no evidence shown, asked once; on a historical claim that is recall, not a forecast.

claude-haiku-4.5, cold rationale: The outcome is well-documented: job losses did peak in early 2009 and employment began recovering within about a year. The causal claim is more contested among economists, but mainstream analyses (CBO, most academic studies) estimate the Recovery Act added 1-3 million jobs net, supporting a moderate-to-high probability it was a major contributor, though some heterodox economists dispute this.

gpt-5.4-mini, cold rationale: The Recovery Act is widely associated with the sharp slowdown in job losses and the shift to employment growth within about a year, so the outcome part is likely true. But attributing a major share of the 2009-2012 employment recovery specifically to the Act, net of Fed policy, automatic stabilizers, and financial rescues, is much harder to support confidently and is more debated.

gemini-3.5-flash-lite, cold rationale: Economic data clearly confirms that US job losses shrank and employment stabilized within a year of the 2009 Recovery Act. Mainstream macroeconomic consensus and studies attribute a significant portion of this employment recovery to the stimulus, independent of monetary policy.

deepseek-v4.1-flash, cold rationale: The labor market clearly stabilized within about a year, with monthly losses shrinking from ~800k to near zero and payrolls bottoming around early 2010. The causal part is supported by CBO and most economists, but the exact counterfactual contribution beyond Fed policy, automatic stabilizers, and TARP remains uncertain.

grok-4.3, cold rationale: Employment stabilized and losses reversed by late 2009-early 2010 as described. Counterfactual estimates from models (CBO, private forecasters) attribute several million jobs to ARRA net of Fed, TARP, and stabilizers, though magnitudes and identification remain debated.

Decomposition

The claim as a tree of checkable parts. AND nodes multiply their children's probabilities, OR nodes combine them noisy-or. Parts tagged as values are excluded from scoring and shown as excluded.

Flags: 1s the pool did not test the other side (directional mass below the floor); ins evidence mass below the floor.

What moved it

Per model, the five documents with the largest push on the log-odds, so a reader can say "a real record, the move makes sense" or "three copies of the same story" from the table alone. Push is the item's log-likelihood ratio after the contested shrink; the listed items' share of the total push is stated per sub-claim.

Rubric versus the model's cold opinion

Each model was also asked, with no evidence and no search, for its probability that the claim is true. The difference between that and the rubric's number is what the structure changed. On a claim resolved before a model's training cutoff the cold number is recall, so this is not a forecasting score.

Divergence between models

Same pool, same rubric, different judgment. Spread is max − min log-odds across models on a sub-claim.

Documents the models rated most differently for source quality

Documents the models read in different directions

Value is direction × strength for the named sub-claim.

Evidence

One shared pool, retrieved once. Each row is a document; the cluster root is what the retrieval model judged it rests on. The first document in a cluster counts in full and each further one is discounted, so a hundred articles citing one study do not outweigh one independent record. Per-model columns give the signed push (llr) that model's judgments produced; the full per-model tables with quality, directness, weight and independence are below each sub-claim.

outcome US monthly job losses shrank sharply and employment stopped falling within about a year of the Recovery Act's enactment in February 2009

25 documents. One-sided pool for claude-haiku-4.5, gpt-5.4-mini, gemini-3.5-flash-lite, deepseek-v4.1-flash, grok-4.3.

cause The Recovery Act was a major cause of higher US employment in 2009-2012 than would have occurred without it, beyond what Federal Reserve policy, the automatic stabilizers and the financial-sector rescue would have produced on their own

19 documents. One-sided pool for claude-haiku-4.5, gpt-5.4-mini, gemini-3.5-flash-lite, deepseek-v4.1-flash, grok-4.3.

Retrieval and roots

Search backend ddg; queries by deepseek/deepseek-v4-pro-0813 (queries_v1); 67 unique URLs found; items by backend ddg 42. Fetch failures: HTTP 403 16, too little text 6, ReadTimeout 2, HTTP 401 1, HTTP 202 1.

outcome: 37 hits, 37 candidates, 25 fetched, 25 in pool.
for: “Bureau of Labor Statistics payroll employment monthly changes 2009 2010”; “Congressional Budget Office Recovery Act employment effects report 2010”
against: “census hiring masked private sector job losses 2010”; “unemployment remained high after 2009 Recovery Act”

cause: 34 hits, 33 candidates, 19 fetched, 19 in pool.
for: “CBO estimated Recovery Act employment effects 2010 2011”; “Blinder Zandi fiscal stimulus jobs multiplier 2009 study”
against: “Federal Reserve QE employment impact versus fiscal stimulus 2009”; “automatic stabilizers accounted for most employment recovery after recession”

Roots by deepseek/deepseek-v4-pro-0813 (roots_v1, aliases merged by roots_canon_v2): 63 distinct root ids, 6 alias groups.

Parameter snapshot

The dials this dataset was scored with, custom. Definitions on the rubric page.

What ran

• Models: anthropic/claude-haiku-4.5, openai/gpt-5.4-mini, google/gemini-3.5-flash-lite, deepseek/deepseek-v4.1-flash, x-ai/grok-4.3
• Schema 1.5.0, rubric 1.2.0, parameters custom, git unknown
• Topic phrase shown to the blind quality prompt: “US employment and federal fiscal stimulus, 2009-2012”
• Live spend on this run: $0.27. Judgments are cached by (model, prompt version, content), so a re-run with the same inputs makes zero calls.
• The dataset is the source of every number on this page.