In other Anthropic news today, I’ve been thinking about this bill that Anthropic proposed where it’s not a show-stopper if only 50 people might be killed or seriously injured:

The bill would let the government block a model’s release or use only if it poses a foreseeable, imminent risk of more than $1 billion in damages, or of killing or seriously injuring more than 50 people, according to the report.

It’s weird to even put a number on this. But also, if gun manufacturers had to meet a similar standard before putting semi-automatic rifles on the market, the world would be a better place.

@manton TBU (true, but useless) we’ve tried to move this discussion many times in the US but the destruction to human lives is, as they say, “priced in”. The same for cars and alcohol. The same is not true for whatever we’re calling OpenAI and Anthropic.

@manton Probably inspired by MIL-STD-882E or similar (see Table III, page 13), although $1B is 100x the “catastrophic” threshold in 882E, which also requires only 1 death, not 50.

Example: 20 deaths and $500M in damage is way beyond DOD’s definition of catastrophic, but below Anthropic’s threshold.

@manton The more important takeaway, at least from the reporting, is that in their proposal there’s no graduated response. It’s “nothing can be done” all the way up to $1B/50 deaths.

Meanwhile, we already have at least one existing, commonly used standard for mitigating risks with far more nuance.

@manton That is utterly depraved. Remind me why we are comparing computer systems with guns and considering military standards? How about full liability for anything more than $50 in damage? Death or injury to any human should not even be on the table. Unacceptable.