The rubric

Rubric 1.2.0, locked defaults locked-2026-09-17. The arithmetic below is a pure function of the cached judgments and this parameter file; it re-runs in milliseconds and changing a dial never calls a model. Every dataset records the rubric version, every prompt version and the full parameter snapshot, so any published number can be reproduced from its own JSON. The reasons behind every change are in the rubric log.

Arithmetic

weight_i       = quality_i ^ quality_exp × directness_i ^ directness_exp × basis_i
independence_i = independence_discount ^ (rank of i within its cluster − 1)
push_i         = direction_i × strength_i × scale × weight_i × independence_i
shrink         = 1 / (1 + contested_pull × mixed_mass / directional_mass)
L_leaf         = logit(prior) + shrink × Σ push_i
p_leaf         = sigmoid(L_leaf)
p_and          = Π p_children          p_or = 1 − Π (1 − p_children)
consensus      = median over models of L_leaf, re-aggregated through the tree

Evidence mass (the sum of weight × independence) below mass_min makes a leaf "insufficient evidence", which is a published verdict. A leaf whose weaker side has directional mass below side_mass_min is flagged one-sided and gets one bounded second retrieval pass for that side; the flag never changes the number.

What the model judges, and what it does not

Call	Version	What it returns	Blind?
source quality	`quality_v1`	0 to 1 for the source, given a topic phrase only	yes: neither the sub-claim nor which side the document helps is shown
directness	`directness_v1`	0 to 1: does the document bear on this exact sub-claim	no
direction, strength, basis	`direction_v2`	+1 / 0 / −1, 0 to 1, and finding / mixed / assertion; re-asked under the negated sub-claim on a sample	no; the polarity check flags inconsistency
roots	`roots_v1`, `roots_canon_v2`	what the document rests on (dataset, study, author group, funder, primary source); aliases merged once per pool	extracted once by one model so every scorer sees the same clusters
queries	`queries_v1`, `queries_side_v1`	search queries per sub-claim, for and against	–
decomposition	`decompose_v1`	the AND/OR tree; values parts tagged and excluded	hand-written for video claims, model-written for submitted ones
cold baseline	`baseline_v1`	the model's probability with no evidence shown; published next to the rubric's number	–

Everything in this table is judgment. Everything after it is arithmetic. That is the whole design: many narrow, recorded calls instead of one "what do you think", and the aggregation kept out of the model's hands.

Dials

Internal dials are all open and tunable against the backtest; the public endpoint exposes one, strictness. Videos and published datasets always use the locked values below.

dial	locked value	what it does
`prior`	0.5	Prior probability of each empirical leaf before any evidence (0.5 = no lean).
`scale`	2.0	Log-odds a single full-weight, fully-direct, strength-1 item can contribute.
`quality_exp`	1.0	Exponent on source quality; >1 punishes weak sources harder.
`directness_exp`	1.0	Exponent on directness; >1 punishes adjacent evidence harder.
`independence_discount`	0.5	Multiplier applied per rank inside a cluster: item k gets discount**k.
`basis_weights`	[1.0, 0.7, 0.4]	Weight factor by evidentiary basis (finding, mixed, assertion). An official estimate or allegation that presents no direct observation is testimony about the fact, not a record of it; it counts, but less. Unknown basis (direction_v1 judgments) gets 1.0. Rubric 1.1.0.
`contested_pull`	1.0	Rubric 1.2.0. Evidence that directly addresses a leaf and comes out mixed (direction 0, directness >= mixed_directness_min) is evidence that the question is contested. The leaf's summed log-odds are shrunk by 1 / (1 + contested_pull * mixed_mass / directional_mass). 0 disables (1.1.0 behaviour).
`mixed_directness_min`	0.5	Direction-0 items at or above this directness count as mixed; below it they are read as not addressing the leaf and carry no contested signal.
`side_mass_min`	0.5	A leaf whose weaker side (for or against) has less directional mass than this is flagged one-sided: the pool did not test the other side. A flag, not a penalty: settled facts legitimately have nothing against them.
`cluster_roots`	primary	Which roots merge items into one cluster: "primary" (each item's first, most load-bearing root) or "all" (any shared root, transitive). Default "primary" since 2026-09-16: once aliases merged, "all" put 39 of 40 bio items of the fixture in one cluster because nearly every article mentions the ISG, Powell or the NIE somewhere. "primary" models "rests on" rather than "mentions"; it under-clusters when the load-bearing root is listed second. Both are visible in the report.
`mass_min`	1.0	Minimum evidence mass (sum of \|weights\|) for a leaf verdict; below = insufficient.
`verdict_thresholds`	[0.15, 0.35, 0.65, 0.85]	Probability cut points: refuted \| leans_refuted \| unclear \| leans_supported \| supported.
`and_leak`	0.0	AND node: p = (1-leak)prod(children) + leakmin(children).
`or_leak`	0.0	OR node: p = (1-leak)noisy_or(children) + leakmax(children).
`max_pool_size`	25	Cap on evidence items per leaf sent for judgment. 40 until 2026-09-16; on the fixture every leaf was decided long before item 25 and cost scales linearly.
`polarity_sample`	0.2	Fraction of (item, leaf) pairs that also get the negated-framing direction call. The check is a diagnostic, not an input to the number; sampling it deterministically (by hash of item id and leaf id) cuts a third of the per-pair calls. 1.0 checks every pair, as Phase 2 did.
`queries_per_side`	2	Retrieval: search queries the model writes per leaf, for each of 'for' and 'against'.
`second_pass_items`	10	Phase 4. When a leaf comes out one-sided for a majority of the models that scored it, the retrieval model writes `queries_per_side` more queries aimed at the weaker side, the results are fetched, judged by every model and the leaf is re-scored. Once, bounded to this many new items per leaf. 0 disables the pass. The flag should clear itself where contrary material exists and stay where it does not; both outcomes are recorded in `dataset.second_pass`.
`results_per_query`	10	Retrieval: search results taken per query (Google CSE maximum is 10).
`excerpt_chars`	2000	Retrieval: characters of extracted main text shown to the judging prompts.
`strictness` public dial	0.5	The one public dial, 0..1; 0.5 reproduces the defaults. See with_strictness().

The public dial

strictness maps onto three internal dials. 0.5 is the default and reproduces the locked values; higher discounts repeated roots harder, demands more evidence mass before a verdict, and punishes weak sources more. It is applied as a re-score of the same cached judgments, shown next to the verdict of record, never in its place.

strictness	independence_discount	mass_min	quality_exp
0.0	0.70	0.50	0.50
0.25	0.60	0.75	0.75
0.5	0.50	1.00	1.00
0.75	0.40	1.25	1.25
1.0	0.30	1.50	1.50

Try it on any claim page with ?strictness=0.8.