The same model fabricates twice as often writing about politics as about technology.
more likely that a politics article carries an unsourced claim than a technology one. The models and prompts are identical; only the subject changes.
Window 2026-05-03 → 2026-08-09
Source systems operated by weranai — method in Methodology
Our previous report measured how often an autonomous editorial pipeline writes claims its own sources do not support. Nearly one in three. This one splits that number by subject, and the split turns out to be the more useful finding.
Nothing changes between these groups except the topic. Same models, same prompts, same source-gathering step, same fact checker, same three-month window. 1,150 articles and 5,550 individual claims.
Article-level pass rate by subject
An article passes only when every claim in it traces back to the source material the writer was given. One unsupported figure is enough to hold the whole piece.
| Subject | Clean | Articles | Pass rate | |
|---|---|---|---|---|
| Technology | 99 | 322 | 30.7% | |
| Space | 27 | 99 | 27.3% | |
| AI | 52 | 208 | 25.0% | |
| Markets | 53 | 255 | 20.8% | |
| Politics | 40 | 266 | 15.0% |
Technology articles come through clean 30.7% of the time. Politics articles, 15.0%. A politics article is 2.04× more likely to carry a claim nobody can source.
The gap is smaller than it looks
Here is where a single number would mislead. The article is a harsh unit: it fails if any of its claims fails. Measured on the claim itself, the same ordering holds but the distance shrinks.
| Subject | Unsupported | Claims | Rate | Claims/article |
|---|---|---|---|---|
| Technology | 383 | 1,536 | 24.9% | 4.77 |
| Space | 140 | 481 | 29.1% | 4.86 |
| AI | 302 | 1,008 | 30.0% | 4.85 |
| Markets | 388 | 1,248 | 31.1% | 4.89 |
| Politics | 433 | 1,277 | 33.9% | 4.80 |
Per claim, politics fails at 33.9% against technology’s 24.9%. That is 1.36×, not 2.04×. The article-level gap is the same difference compounded across roughly five claims per piece.
Claims per article sit between 4.77 and 4.89 across all five subjects, so the compounding applies about equally to each. The ordering is not an artefact of some subjects being denser than others.
Where the unsupported claims are
Distribution of every unsupported claim in the sample, by subject.
Our reading
We cannot prove causation from this, so treat what follows as a hypothesis the data is consistent with rather than a finding.
Subjects with dense primary documentation give a writer something concrete to anchor to. A product launch has specifications, a filing, a release note, a price. The sentence a model wants to write is usually sitting in the source packet already.
Political coverage rewards synthesis: context, history, what a move means. Those sentences are rarely in any single document, and when a model reaches for context it reaches into its own weights. That is exactly the moment an unsupported specific appears — a date, a figure, a title — carrying the same confident tone as the sourced material around it.
If that holds beyond our pipeline, the practical implication is uncomfortable: the risk of generative systems in reporting is not uniform. It is highest where verification is hardest and where being wrong costs the most.
What this does and does not show
- “Unsupported” means absent from the source packet given to the writer, not false. Some flagged claims are true in the world; they simply could not be traced at write time, which is the standard that matters before publishing.
- Subjects are not equally represented. Space has 99 articles against technology’s 322, so its figure carries a wider margin than the others.
- The fact checker is itself a model with an unmeasured error rate. If that error is not uniform across subjects, part of this gap could belong to the checker rather than the writer. Measuring that is our next report.
- One pipeline, one set of prompts, one model family. This is what our system does, not a general law about language models.
Method, definitions and the status vocabulary are in Methodology. The overall rate this splits is in the fact-checking report.