Fatih Kansoy

Research · November 2026

Can a Language Model Write Like an Economist?

Language models now help economists revise papers, yet outside judges still tell model prose apart from authors' text even after detailed style rules. This paper asks whether an open model can learn disciplinary prose directly from economists' paragraphs without explicit rules.

Fatih Kansoy

Senior Lecturer in Economics, University of Oxford

Published in
University of Oxford Working Paper
Year
2026
JEL codes
A14, C45, C88
Topics
AI and Data, Natural language processing, Academic writing, Reinforcement learning
Two manuscript stacks linked across a central judge block, with a revised sheet rising from the draft stack
Contents
  1. Research question
  2. Why it matters
  3. Data and method
  4. Findings
  5. Figures and tables
  6. Scope and limits
  7. How to cite
  8. Selected references

Language models now help economists revise papers, yet outside judges still tell model prose apart from authors' text even after detailed style rules. This paper asks whether an open model can learn disciplinary prose directly from economists' paragraphs without explicit writing rules.

We assemble 2,120 pre-LLM economics and finance papers (1995–2022), mostly open working papers before copy-editing, and build 5,267 screened training pairs matching an AI draft with the authors' paragraph. On a single rented A100 GPU, we train 4-bit Gemma 4 31B in two stages: supervised fine-tuning on paired examples followed by adversarial policy optimisation against a co-trained judge whose length-adjusted authorship probability supplies the sole stylistic reward.

On 137 introduction paragraphs from 2026 NBER working papers written after the base model's cut-off, two outside frontier judges (GPT-6.1 Sol and Claude Opus 5.5) take the trained model's revisions for the authors' text about half the time or more (70.8 and 55.8 per cent in run 1), far above raw drafts (6.9 and 8.4 per cent) and guided Claude Opus 5.5 (21.2 and 7.7 per cent). Across two independent seeds, adversarial training adds roughly 21 percentage points at nearly equal length, though blind content checks show trained writers alter claims more often than the guided model.

On 2026 NBER working papers, GPT-6.1 Sol and Claude Opus 5.5 take raw GPT drafts for the authors in 6.9 and 8.4 per cent of comparisons, and guided Opus reaches 21.2 and 7.7 per cent. Supervised training lifts both judges to 40.1 and 40.5 per cent before adversarial training raises them to 70.8 and 55.8 per cent in run 1.

Research question

Can an open language model learn the conventions of economists' prose directly from their paragraphs without explicit style rules, and how often does learning that style alter factual claims?

Why it matters

As language models enter everyday academic drafting, quality-focused judges often reward the polished smoothness that makes model prose sound un-economic and strips away technical precision. Understanding whether disciplinary writing conventions are better absorbed from examples and adversarial feedback than from explicit style checklists—and measuring the accompanying content-fidelity trade-off—clarifies both the promise and the safe workflow boundaries of AI writing assistants in research.

Data and method

Data

We build a pre-LLM corpus of 2,120 economics and finance papers (1995–2022) across five journal groups, mostly in open working-paper versions (754 from nber.org and at least 1,128 from working-paper archives) before publisher copy-editing. Screening yields 5,267 supervised training pairs and 158 validation pairs holding fixed facts across all sections, plus 1,225 clean judge-training pairs and 804 unseen adversarial drafts (204 corpus plus 600 from 51 Brookings Papers on Economic Activity articles). Evaluation combines 137 introduction paragraphs from 59 post-cut-off 2026 NBER working papers with 54 recall-screened records (53 distinct paragraphs) from 1999–2022.

Method

Both training stages run on one 4-bit Gemma 4 31B base model on a single A100 GPU with three LoRA adapters (writer, frozen Step 1 KL anchor, and pre-trained internal judge). Step 1 fine-tunes the writer on the 5,267 pairs; Step 2 draws four revisions per unseen draft at temperature 0.7, averages judge authorship probabilities across both display orders, removes within-group log-length slope from clipped log-odds rewards, and updates the writer via group-relative policy optimisation (GRPO) while updating the judge on replayed pairs. Outside evaluation scores all 191 test paragraphs in both orders under GPT-6.1 Sol and Claude Opus 5.5, supplemented by two blind content-fidelity annotators and an open human reading test.

Findings

  1. Training far outpaces explicit writing rules

    On 2026 NBER working papers, GPT-6.1 Sol and Claude Opus 5.5 take raw GPT drafts for the authors in 6.9 and 8.4 per cent of comparisons, and guided Opus reaches 21.2 and 7.7 per cent. Supervised training lifts both judges to 40.1 and 40.5 per cent before adversarial training raises them to 70.8 and 55.8 per cent in run 1.

  2. Replicated adversarial gains are not driven by length

    Across all 191 paragraphs and both judges, adversarial training adds +21.1 percentage points in run 1 (95% CI 15.4 to 26.6) and +20.8 points in run 2 (15.6 to 26.0). Mean length rises by at most 1.4 per cent, and paragraphs within 5 per cent of draft length keep gains of +21.4 and +19.3 points.

  3. Content fidelity shows a clear trade-off

    Blind annotators find that trained writers alter claims more often than guided Opus (28.3 and 6.8 per cent supervised vs. 8.4 and 0.5 per cent guided), and GPT's run-1 upper bound (+12.3 points) fails the pre-specified no-cost threshold. Counting every major flag as a failure still leaves run 1's gain clearly positive (+14.1 GPT, +19.1 Opus).

  4. Two operational rules govern adversarial stability

    Two failed pilots showed that the internal judge must already separate human from model prose before Step 2 (0.990 held-out accuracy after solo training on 1,225 pairs) and that training must sample at the judge's training temperature (0.7 rather than 1.0, where accuracy collapsed from 0.92–1.00 to 0.29–0.46). Human readers (23 readers, 95 answers) also spotted authors less often against trained writers than against drafts (-20.8 points pooled).

The paper shows that many conventions of economic prose are tacit and absorbed far more effectively through paired examples and adversarial feedback than through the style guide tested, all on a single GPU at modest API cost ($131 across pilots, drafts, judges, and content checks). It adapts reinforced adversarial learning to single-GPU 4-bit training with post-cut-off evaluation, within-group length debiasing, two-seed replication, pilot diagnostics, and an independent content-fidelity audit.

Figures and tables

Diagram of the two training stages on one 4-bit Gemma 4 31B base model: Step 1 supervised fine-tuning on 5,267 pre-2023 pairs initialising the Step 2 writer and frozen KL anchor, followed by Step 2 adversarial GRPO updates against a co-trained internal judge on 804 unseen drafts.
Figure 1: Two-stage training architecture combining supervised fine-tuning on 5,267 paired examples (Step 1) with length-adjusted adversarial policy optimisation against a co-trained internal judge (Step 2) on a single GPU.
Two-panel chart comparing outside-judge authorship attribution across GPT draft, Prompted Opus, Supervised writer, and Adversarial writer for 2026 working papers (0–80% scale) and 1999–2022 papers (zoomed 0–55% scale) with 95 per cent paper-clustered bootstrap confidence intervals.
Figure 2: Share of comparisons in which outside frontier judges (GPT-6.1 Sol and Claude Opus 5.5) take each version for the authors' paragraph across 137 post-cut-off 2026 introductions and 54 recall-screened 1999–2022 introductions.
Outside-judge attribution and robustness checks (% taken for authors; chance = 50%)
Evaluation / VersionGPT-6.1 Sol / Opus 5.5Key Takeaway
GPT draft (2026 working papers, N = 137)6.9% / 8.4%Raw AI drafts are spotted by both outside judges over 91% of the time.
Prompted Opus + writing guide (2026, N = 137)21.2% / 7.7%Explicit style rules gain only under GPT; Opus still spots its own prose.
Supervised writer — Step 0 (2026, N = 137)40.1% / 40.5%Learning from 5,267 pre-2023 pairs lifts both judges sharply.
Adversarial writer — Run 1 (2026, N = 137)70.8% / 55.8%+30.7 pp (GPT) and +15.3 pp (Opus) over Step 0; Opus 95% CI spans 50%.
Adversarial gain — All 191 paragraphs, both judges+21.1 pp (Run 1) / +20.8 pp (Run 2)Replicated across seeds; holds at +21.4 / +19.3 pp within 5% draft length.
Pre-2023 test set (1999–2022, N = 54) — Run 1 final31.5% / 25.9%Far below chance for every system, yet well ahead of drafts (0%) and guide (1.9% / 0.9%).
Joint outcome (major content flag = failure, Run 1)+14.1 pp (GPT) / +19.1 pp (Opus)Gains stay clearly positive, though trained writers still need an author content check.

Scope and limits

Several limits qualify these findings: main attribution results rely on two frontier judge families (and GPT-6.1 Sol wrote every test draft); the human reading sample is small (23 readers, 95 answers) with unverified identities; the task revises single introduction paragraphs with fixed content rather than drafting from notes; some 2026 working papers may carry light AI editing while older pre-2023 papers remain well below chance; the study compares two complete systems rather than isolating examples from rules; and both runs halted once the 31B internal judge reached its accuracy floor.

How to cite

APA

Kansoy, F. (2026). Can a Language Model Write Like an Economist? University of Oxford Working Paper. https://fatih.ai/research/ai_economist/

Chicago

Kansoy, Fatih. 2026. "Can a Language Model Write Like an Economist?" University of Oxford Working Paper, November 2026. https://fatih.ai/research/ai_economist/.

Harvard

Kansoy, F. 2026, 'Can a Language Model Write Like an Economist?', University of Oxford Working Paper, available at: https://fatih.ai/research/ai_economist/.

BibTeX
@unpublished{kansoy2026aieconomist,
  title  = {Can a Language Model Write Like an Economist?},
  author = {Fatih Kansoy},
  note   = {University of Oxford Working Paper},
  month  = nov,
  year   = {2026},
  url    = {https://fatih.ai/ai_economist.pdf}
}

Selected references

  1. Bou, A., Valdes-Aleman, J., Laurent, J. M., Rodrigues, S. G., White, A. D., Hinks, M., and Narayanan, S. 2026. Learning to reason from reference samples alone: Reinforced adversarial learning for modern LLM post-training. arXiv:2610.38851.
  2. Dubois, Y., Liang, P., and Hashimoto, T. B. 2024. Length-controlled AlpacaEval: A simple way to debias automatic evaluators. Conference on Language Modeling (COLM 2024).
  3. Korinek, A. 2023. Generative AI for economic research: Use cases and implications for economists. Journal of Economic Literature, 61(4), 1281–1317.
  4. Li, X., Li, M., Zhang, Y., Zhang, Y., Li, L., Yang, M., and Wu, Y. 2026. Beyond imitating the surface: Teaching LLMs to write like researchers via rubric-guided RL. arXiv:2601.14891.
  5. McCloskey, D. N. 1983. The rhetoric of economics. Journal of Economic Literature, 21(2), 481–517.
  6. Panickssery, A., Bowman, S. R., and Feng, S. 2024. LLM evaluators recognize and favor their own generations. Advances in Neural Information Processing Systems (NeurIPS), 37.
  7. Rivera Soto, R., Chen, Y., and Andrews, N. 2025. Few-shot paraphrase inversion for detecting paraphrased machine text. Proceedings of ACL 2025.

More research

All research