Excellent piece. Your point about people hiding behind classified information is especially salient. More often than not it comes from a good-faith inclination of career national security officials to try to plug every possible hole, address any possible threat. But it is also not infrequently used to discredit an opposing view, especially in any remotely partisan debate. In both instances, there is rarely a dominant consensus. I spent two decades in the intelligence community, and this happened all the time.
That perspective, even when offered in good faith, also relies on a narrow definition of national security. X bad actor uses Y tool, thus Y tool must be banned. What it ignores is the remaining scope of what constitutes national security, including economic competitiveness, innovation, and an overall bias toward growth and development.
This is correct, and stifling access to leading open weight models would harm defenders far more than attackers, especially given the challenges of refusals practitioners run into when trying to use the leading models for cyber-oriented defensive work.
The prosumer/hobbyist ecosystem around open weight models should have a positive affect on general AI safety. More people doing more testing in more ways always results in more findings. I’d feel safer in a world with 100+ models running on 100+ harnesses than 10 models running on 10 APIs owned by 10 people.
The structural feature here is the classified briefing. Evidence that cannot be shown cannot be contested, and it reliably argues in one direction. Notice who sits on both sides of that channel: the parties whose product becomes more valuable if the alternative is banned are also the parties briefing on the risk. That is not an accusation of bad faith. It is a description of a process with no adversarial step in it.
The air-gapped point is the part I wish got more attention. A defender running a model on a private network can log every prompt and tool call and audit what it actually touched, which is a kind of control you don't really get from a hosted API either. I'd be curious whether anyone has tried measuring defensive uplift in those deployments the way the offensive evals measure attacker uplift.
Working on a post, based on the data one might conclude that US labs are against Open-weights because they can't or won't compete in the space.
Excellent piece. Your point about people hiding behind classified information is especially salient. More often than not it comes from a good-faith inclination of career national security officials to try to plug every possible hole, address any possible threat. But it is also not infrequently used to discredit an opposing view, especially in any remotely partisan debate. In both instances, there is rarely a dominant consensus. I spent two decades in the intelligence community, and this happened all the time.
That perspective, even when offered in good faith, also relies on a narrow definition of national security. X bad actor uses Y tool, thus Y tool must be banned. What it ignores is the remaining scope of what constitutes national security, including economic competitiveness, innovation, and an overall bias toward growth and development.
This is correct, and stifling access to leading open weight models would harm defenders far more than attackers, especially given the challenges of refusals practitioners run into when trying to use the leading models for cyber-oriented defensive work.
Appreciate you chiming in, I feel like the comment section here could get interesting in a few hours...
The prosumer/hobbyist ecosystem around open weight models should have a positive affect on general AI safety. More people doing more testing in more ways always results in more findings. I’d feel safer in a world with 100+ models running on 100+ harnesses than 10 models running on 10 APIs owned by 10 people.
The structural feature here is the classified briefing. Evidence that cannot be shown cannot be contested, and it reliably argues in one direction. Notice who sits on both sides of that channel: the parties whose product becomes more valuable if the alternative is banned are also the parties briefing on the risk. That is not an accusation of bad faith. It is a description of a process with no adversarial step in it.
The air-gapped point is the part I wish got more attention. A defender running a model on a private network can log every prompt and tool call and audit what it actually touched, which is a kind of control you don't really get from a hosted API either. I'd be curious whether anyone has tried measuring defensive uplift in those deployments the way the offensive evals measure attacker uplift.
I feel like the discourse often misses the need for LLMs’ defensive capabilities to be wide spread as well
Do you you believe the prominence and persistence of this argument in the discourse has any relation to power concentration?
GLM 5.3 & 5.3-Flash are amazing for RP, great prose.