# Andy Hall

- Researcher: Andy Hall
- Role: Author of the 2020 vote-by-mail paper and of the Free Systems Substack; affiliation not stated on the page
- Field: political science
- Tools: Claude (AI coding assistant, Claude Code)
- Published: 2026-01-13
- Source: https://freesystems.substack.com/p/the-100x-research-institution
- Older account (before March 2026). Latest flagship models when written: Claude Opus 4.5 and GPT-5.2. Latest now: Claude Opus 5.5 and GPT-6.1 Sol. Agent tools have also changed since, so check whether a limitation reported here still applies.

## What they did

Hall took his own 2020 paper on vote-by-mail, handed it to an AI coding assistant and asked it to replicate the findings from scratch and extend the analysis with new data. He commissioned an independent audit: a UCLA PhD student, Graham Straus, updated the paper manually and then went through Claude's code line by line against his own ground truth. Hall reports that Claude's estimates were close to Straus's, that it coded 29 of 30 California counties correctly on treatment timing, and that its collected data correlated above .999 with manual figures. It did so in under an hour for about ten dollars.

## In their words

> "It made mistakes—missed some races, failed to run calculations it had the inputs for, documented poorly."

> "every new empirical paper should ship with proof that an AI agent successfully reproduced its results from the raw data."

## Lessons they draw

- Commission an independent manual audit rather than trusting the agent's own report (Hall).
- Errors he found: missed races, calculations not run despite available inputs, poor documentation (Hall).
- He proposes that agent-run replication from raw data could become a routine pre-publication check and that findings could be re-run as "living research" after each new election (Hall).

## Limits of this account

Author's own paper, audit commissioned by him; the post is partly a vision piece, and the audit write-up itself was not read.
