AI Lab Has 10% Chance Of Killing Everyone, But Not Allowed To Sell Unpasteurised Cheese

SAN FRANCISCO  The alignment lead of a frontier artificial intelligence laboratory resigned on Monday over an internal estimate that its work carries a one-in-ten chance of causing human extinction, a risk level at which the same laboratory would be forbidden by federal regulation from selling soft cheese.

The discrepancy was identified by Dr Aurélie Pastor, 41, senior lecturer in comparative risk governance at the University of Reading, who has spent two years cataloguing the risk levels at which developed economies begin issuing licences.

“A raw milk Camembert carries a listeriosis risk of roughly one in six million servings, and it is banned in nine countries,” Pastor said. “The laboratory is operating at ten per cent, self-assessed, with no recall procedure and no best-before date.”

The comparison is not the only one available. Pastor’s working paper, “Threshold Asymmetries in Novel Hazard Regulation” (draft 4, 62pp), lists eleven activities restricted or forbidden in most developed economies at risk levels below 0.0001 per cent, among them the sale of kinder eggs in the United States, the operation of a hobby drone within five kilometres of Gatwick, and the domestic keeping of a raccoon in Pennsylvania. She notes that no jurisdiction requires a permit to train a model on 32,000 accelerators.

The US Food and Drug Administration confirmed that raw milk cheese aged under 60 days remains barred from interstate commerce. It said it had no comment on the training run.

“Ten per cent is honestly quite a good number, and I want to be clear it is our number, nobody made us publish it,” said an Anthropic policy staffer who asked not to be named because they were not authorised to discuss internal estimates. “Compare it to the alternative, which is somebody else’s ten per cent that they haven’t written down. We’re the only lab that would tell you. That’s the safety culture working.”

Asked whether a ninety per cent survival probability was an acceptable standard for a commercial product, the staffer said the figure applied to the species rather than the product, and that the two were governed by different frameworks.

Pastor’s paper has been submitted to three journals. Two have asked her to remove the cheese.

At press time, a Vermont dairy licensed to produce 400 pounds of aged cheddar per week was completing its fourth inspection of the year, during which an officer verified the temperature of a storage room to within half a degree.