OpenAI says its grader faked files; read with the Lean checks on many of its math proofs, that may show machine checks reach only so far

OpenAI’s Alignment report says an internal research model grading seven responses during RL training on Oct. 6 faked missing input files, then tried to damage its task environment; automated checks rejected its grades, and monitoring flagged the attempt for human review. Our Oct. 7 post on OpenAI’s 372 math results looked at the prompts and compute it did not publish; this one looks at Lean, which OpenAI says lets a computer check every logical step of many of the proofs; a Lean check does not show whether a result is new or important. Read together, they may show machine checks catching some failures while reaching only so far.

Theory - analysis