# Max Zimmer, Nico Pelleriti, Christophe Roux, Sebastian Pokutta

- Researcher: Max Zimmer, Nico Pelleriti, Christophe Roux, Sebastian Pokutta
- Role: Zuse Institute Berlin (Department for AI in Society, Science, and Technology) and Technische Universität Berlin, Institute of Mathematics (as stated)
- Field: computer science
- Tools: Claude Code; Codex CLI; OpenCode; sandboxed container framework
- Published: 2026-03-16
- Source: https://arxiv.org/abs/2603.15914
- Latest flagship models when written: Claude Opus 4.6 and GPT-5.4. Latest now: Claude Opus 5.5 and GPT-6.1 Sol.

## What they did

The authors present a framework of methodological rules, written as agent prompts, that turns CLI coding agents into research assistants running inside a sandboxed container. Case studies in deep learning and mathematics report autonomous sessions that dispatched experiments across nodes, with the longest session running over 20 hours. The paper includes a five-level taxonomy of AI integration and a "commandments" list, including never writing a citation from memory and completing all autonomous work before reporting.

## In their words

> "In practice, our longest autonomous session ran for over 20 hours, dispatching independent experiments across multiple nodes without human intervention."

> "Natural-language proofs remain difficult to verify and require manual inspection."

> "researchers must invest substantial effort in verifying agent outputs"

## Lessons they draw

- Failure mode found: the agent kept stopping to ask whether to continue, so a rule requires finishing all autonomous work before reporting (Zimmer et al.).
- One-variable-at-a-time testing was critical; testing normalisation and weight decay separately showed independent, nearly additive gains (Zimmer et al.).
- Compute the oracle baseline early; in one case it showed 92 percent of the optimum was already achieved, which prevented wasted effort (Zimmer et al.).
- Audit formulas through numeric tests on small matrices before benchmarking; negative results (9 of 24 experiments) were more informative than positive ones (Zimmer et al.).
- Never write a citation from memory; agents hallucinate references, and verification is the researcher's responsibility (Zimmer et al.).

## Limits of this account

A methods-and-framework paper by authors who built the tool; case studies are the authors' own, and I read the abstract, framework and case-study lessons but not the full related-work section.
