The 2007 increase in US troop levels in Iraq, the surge, reduced violence in Iraq.

Verdict per model

Every model scored the same evidence pool with the same rubric. Differences below are differences of judgment, not of retrieval. "Cold" is the model's own probability with no evidence shown, asked once; on a historical claim that is recall, not a forecast.

claude-haiku-4.5, cold rationale: The outcome is well-documented: violence did decline substantially from 2007-2008 compared to 2006-2007 levels. However, the causal claim remains contested among scholars and analysts; while the surge likely contributed, isolating its effect from concurrent factors (Anbar Awakening, Sadrist ceasefire, sectarian sorting) is methodologically difficult, and many experts attribute significant weight to these alternative explanations.

gpt-5.4-mini, cold rationale: Civilian deaths and attacks in Iraq did fall sharply from 2006/early-2007 to mid-2008. The surge, together with new counterinsurgency tactics, was a major contributor to that decline, though it was not the only cause and other factors like the Anbar Awakening and the Sadrist ceasefire also mattered.

gemini-3.5-flash-lite, cold rationale: There is overwhelming consensus that violence fell dramatically in Iraq between 2007 and 2008, though historians and analysts vigorously debate the exact degree to which the US troop surge caused this decline versus concurrent local factors like the Anbar Awakening.

deepseek-v4.1-flash, cold rationale: The outcome is well-documented: violence in Iraq fell sharply from mid-2007 to mid-2008. The causal attribution is more contested, since the surge coincided with the Anbar Awakening, the Sadrist ceasefire, and sectarian displacement, though most analysts credit the surge and changed tactics with a substantial independent role.

grok-4.3, cold rationale: Violence indicators show a clear drop after mid-2007, but the surge's independent causal contribution beyond the Awakening, Sadrist ceasefire, and completed ethnic cleansing is contested in the historical literature.

Decomposition

The claim as a tree of checkable parts. AND nodes multiply their children's probabilities, OR nodes combine them noisy-or. Parts tagged as values are excluded from scoring and shown as excluded.

Flags: 1s the pool did not test the other side (directional mass below the floor); ins evidence mass below the floor.

What moved it

Per model, the five documents with the largest push on the log-odds, so a reader can say "a real record, the move makes sense" or "three copies of the same story" from the table alone. Push is the item's log-likelihood ratio after the contested shrink; the listed items' share of the total push is stated per sub-claim.

Rubric versus the model's cold opinion

Each model was also asked, with no evidence and no search, for its probability that the claim is true. The difference between that and the rubric's number is what the structure changed. On a claim resolved before a model's training cutoff the cold number is recall, so this is not a forecasting score.

Divergence between models

Same pool, same rubric, different judgment. Spread is max − min log-odds across models on a sub-claim.

Documents the models rated most differently for source quality

Documents the models read in different directions

Value is direction × strength for the named sub-claim.

Evidence

One shared pool, retrieved once. Each row is a document; the cluster root is what the retrieval model judged it rests on. The first document in a cluster counts in full and each further one is discounted, so a hundred articles citing one study do not outweigh one independent record. Per-model columns give the signed push (llr) that model's judgments produced; the full per-model tables with quality, directness, weight and independence are below each sub-claim.

outcome Civilian deaths and attacks in Iraq fell substantially between mid-2007 and mid-2008 from their 2006 and early-2007 levels

22 documents. One-sided pool for claude-haiku-4.5, gpt-5.4-mini, gemini-3.5-flash-lite, deepseek-v4.1-flash, grok-4.3.

cause The 2007 increase in US troop levels and the accompanying change in tactics was a major cause of the 2007-2008 decline in violence in Iraq, beyond what the Anbar Awakening, the Sadrist ceasefire and the completion of sectarian displacement in Baghdad would have produced on their own

14 documents. One-sided pool for claude-haiku-4.5, gpt-5.4-mini, grok-4.3.

Retrieval and roots

Search backend ddg; queries by deepseek/deepseek-v4-pro-0813 (queries_v1); 60 unique URLs found; items by backend ddg 35. Fetch failures: unsupported content type application/octet-stream 2, HTTP 403 16, HTTP 203 1, HTTP 404 1, too little text 4, HTTP 202 1.

outcome: 40 hits, 33 candidates, 22 fetched, 22 in pool.
for: “Iraq civilian deaths decline 2007 2008 official statistics”; “Iraq security incidents drop mid-2007 mid-2008 report”
against: “Iraq violence remained high 2008 civilian casualties data”; “criticism Iraq death toll statistics undercount 2007 2008”

cause: 34 hits, 28 candidates, 14 fetched, 14 in pool.
for: “Iraq surge 2007 troop increase counterinsurgency violence decline statistical analysis”; “Baghdad security plan 2007 US troops sectarian violence reduction report”
against: “Iraq violence decline 2007 Anbar Awakening Sadr ceasefire displacement study”; “surge Iraq 2007 not main cause violence decline scholarly”

Roots by deepseek/deepseek-v4-pro-0813 (roots_v1, aliases merged by roots_canon_v2): 65 distinct root ids, 6 alias groups.

Parameter snapshot

The dials this dataset was scored with, custom. Definitions on the rubric page.

What ran

• Models: anthropic/claude-haiku-4.5, openai/gpt-5.4-mini, google/gemini-3.5-flash-lite, deepseek/deepseek-v4.1-flash, x-ai/grok-4.3
• Schema 1.5.0, rubric 1.2.0, parameters custom, git unknown
• Topic phrase shown to the blind quality prompt: “Violence levels in Iraq and US force posture, 2006-2008”
• Live spend on this run: $0.01. Judgments are cached by (model, prompt version, content), so a re-run with the same inputs makes zero calls.
• The dataset is the source of every number on this page.