Fatih Kansoy

Research · 19 November 2026

Can a Language Model Write Like an Economist?

Style rules barely hide AI prose in economics. In a new working paper and open replication repository, an open 31B model trained on pre-2023 paragraphs and adversarial feedback reaches about half the time or more under two frontier judges, with a clear trade-off on factual claims.

Posted
19 November 2026
Kind
Research
Topics
Large language models, Academic writing, Adversarial training, Economics
By
Fatih Kansoy
Two anonymous manuscript cards side by side on a desk with a judgement marker moving between them
Contents
  1. What the experiment finds
  2. The catch: content fidelity
  3. Read the paper and run the code

Today I posted a new working paper, Can a Language Model Write Like an Economist?, together with its project page and full replication repository on GitHub.

Why style rules fall short. When you ask a frontier model which of two economics paragraphs reads better, it usually picks an AI revision over what the authors wrote. Ask the same model which paragraph the economists wrote, and it spots the authors almost every time. Even when Claude Opus 5.5 receives a detailed writing guide, a cliché list and three published examples, an outside judge still identifies the authors in 119 of 120 blind comparisons.

  1. 1. Corpus2,120 pre-2023 papers, 5,267 screened draft-author pairs
  2. 2. Step 1Supervised fine-tuning on one 4-bit Gemma 4 31B GPU
  3. 3. Step 2Adversarial GRPO updates against a co-trained internal judge
  4. 4. AuditTwo outside frontier judges, blind content checks and 23 human readers

What the experiment finds

Tested on 137 introduction paragraphs from 2026 NBER working papers written after the base model's cut-off, two outside frontier judges (GPT-6.1 Sol and Claude Opus 5.5) mistake raw AI drafts for the authors only 6.9 and 8.4 per cent of the time. Giving Opus 5.5 a detailed writing guide moves its score to 21.2 per cent under GPT and 7.7 per cent under Opus. Supervised training on 5,267 pre-2023 pairs lifts an open 31B model to 40.1 and 40.5 per cent. Adding the adversarial stage raises it to 70.8 and 55.8 per cent in Run 1, and 68.2 and 54.4 per cent on a fresh seed in Run 2.

Blind authorship attribution across raw AI draft, guided Opus, supervised 31B and adversarial 31B on 2026 NBER working papers and 1999 to 2022 papers
Share of order-balanced blind comparisons where outside frontier judges take each AI version for the authors' paragraph (95 per cent paper-clustered bootstrap intervals).

Across all 191 test paragraphs and both judges, adversarial training adds 21.1 percentage points in Run 1 and 20.8 points in Run 2. Mean length shifts by at most 1.4 per cent, and paragraphs within 5 per cent of draft length keep almost the entire gain (+21.4 and +19.3 points). Both final writers keep every number in all 137 paragraphs from 2026. In a small blind reading test (23 readers, 95 answers), human readers also spot the authors 20.8 percentage points less often against our trained models than against raw drafts.

The catch: content fidelity

Sounding more like an economist comes with a real risk to factual accuracy. Two blind annotators checked every version across all 191 paragraphs against its input draft for dropped or added claims, changed hedges, entity errors and meaning shifts. Both annotators find that trained writers alter claims more often than the guided frontier model (28.3 and 6.8 per cent supervised against 8.4 and 0.5 per cent guided).

Content-fidelity audit showing major content change rates and fidelity-adjusted attribution across systems
Blind content-fidelity audit across all 191 test paragraphs. Even when every major flag counts as a failure, the net adversarial gain stays positive (+14.1 points under GPT, +19.1 under Opus).

The practical lesson is straightforward. Examples and adversarial feedback teach disciplinary prose far better than a style guide, even on a single GPU, yet research use still requires an author check on meaning before keeping any edit.

Read the paper and run the code

You can read the 17-page working paper at fatih.ai/ai_economist.pdf, explore the full AI Economist project page or the research summary, and reproduce every table, confidence interval and figure with one command from github.com/kansoy/ai-economist.

More news

All news