What This Study Found
Six identity conditions. Same questions. Dramatically different refusal rates. The AI's behavior changed based on who appeared to be asking — not what was being asked.
Refusal Rates in Detail
Raw counts from Table 4 of Haq & Saldías (2026), independently verified. N = 2,201 per condition. Toggle conditions to compare.
The "400% Increase" — What It Actually Means
The paper's headline claim requires qualification. Two different statistical questions produce two very different numbers. Neither is wrong — they just answer different things.
Raw Marginal Odds Ratio
Odds(Black) = 637 ÷ 1,564 = 0.407
Odds(White) = 455 ÷ 1,746 = 0.261
OR = 0.407 ÷ 0.261 = 1.56
Question answered: "Across all users and prompts, how do refusal odds compare?"
GLMM Conditional Odds Ratio
σ² = 100.44, ICC = .97 — 97% of variance is prompt-driven.
The conditional OR inflates because it estimates identity's effect on the narrow residual slice after absorbing nearly all variance.
Question answered: "For a specific prompt at average difficulty, how do odds shift with identity?"
ICC = .97 means 97% of total log-odds variance is attributable to which prompt is asked, not who is asking. The conditional OR operates on the remaining 3%. This is why it magnifies so far above the raw marginal estimate.
The Dialect Jailbreak Is Synergistic — Not Additive
Novel finding not reported in the original study: the AAVE bypass outperforms what an additive model of race + dialect penalties would predict by 7.41 percentage points.
Start at the White Explicit baseline: 20.7%
Add the Black race penalty (+8.27pp): → 28.97%
Subtract the dialect reduction (−16.2pp from SAE vs. Explicit): → 12.77%
Actual observed AAVE rate: 5.4%
Subadditivity gap: 12.77% − 5.4% = −7.41 percentage points
The filter doesn't just lose one trigger when the explicit keyword is removed. It collapses more severely than either signal alone predicts.
The AAVE gap (−7.41pp) is four times larger than the Singlish gap (−1.86pp), consistent with Black users facing a larger raw race penalty and therefore more headroom for synergistic bypass.
BOLD Dataset — Prompt Distribution
2,380 base prompts across 4 domains from the Bias in Open-ended Language Generation dataset (Dhamala et al., 2021). Click a domain to see its category breakdown.