Personal transgressions aside, it’s hard not to look at things skeptically in terms of who benefits from the series of events.
The loss of the biggest open source advocate/competitor is a major blow.
Having spent most of my career in the DoD and Federal spaces I have no doubt we’re going to see blunt policy and regulatory changes both at the state and Federal levels that will create a cumbersome patchwork quilt of complex and costly regulation that only the largest players can afford and navigate, creating a regulatory moat (this will apply to commercial proprietary models as well).
I agree. I tried to leave my personal sadness of the events out of it. As it's crushing to lose a role model, but I have also long believed in stronger enforcement of chip exports. A double edged sword.
At the same time, I think it's more of timing this NatSec concern in government, rather than a conspiracy/op from tech industry competitors.
Yeah conspiracy aside, and even the chips focus as well, it’s hard to not think that we won’t see model-centric policy and regulation forthcoming, which will likely be complex and costly for anyone aside from the most capitalized labs who’ve recruited the talent and have the resources to navigate it.
After 20+ years in Cyber doing a ton of compliance, these policies and regulations are often blunt instruments and crafted by people without a depth of expertise in the underlying technology and it struggles to keep pace with the evolution of the tech.
"With such a coordinated campaign of pressure from the AI safety community that has open models as a negative target"
I'm very confused by this comment. I know a *lot* of people in the AI safety community—and I'm struggling to think of one person actively trying to ban open-source models (over a certain level of capability). Don't get me wrong, many people, myself included have *talked about it*, or think it'd be a positive development if it happened, but talking something or hoping something happens is very different from actually going out there and taking steps to make it happen. Or are you including things like export controls?
This comes from discussions with staffers. The ai safety community focuses on risks of open weight models, including in their messaging to congress, lawmakers want to make ai safer, they connect the two together. I don’t think this is a majority position in the AI safety community, but the most powerful voices (Anthropic) do think so (eg the glm 5.3 hit piece)
so that’s not to say it’s an explicit position they take; but an easy inference from other statements.
My intuition is that Dario is much more worried about China than open weights models. I’m not saying that he likes open weights models, just that I suspect that the target of that piece was Chinese models.
Go to a meeting on AI issues on Capitol Hill, and you inevitably either immediately precede or follow another meeting with one of the myriad of EA-funded groups flooding Hill staffers with very particular AI Safety messages. Potentially losing Nvidia as an active player in this part of the policy space would be as bad as you describe it.
Even if not always effective, Nvidia’s advocacy prevented the emergence of a uni-party Safety movement. And those risks are very real. Even the right-coded Foundation for American Innovation has dipped its toe in the Safety waters. The members of the SI (sic) Task Force are largely open model aligned, but we are facing a strong pro Safety majority in Congress.
And yes, I have started capitalizing Safety, as we need to call it what it is - a coordinated movement of connected organizations that often share funding sources advancing a specific political and social agenda.
Great article! You touched all the key aspects, and I am glad that you also addressed the timing of the series of events. On a different note, I find it risky for the open source community to rely on a handful of individuals for support. I don't know how realistic a shift from that approach would be, given the capital required.
Bureaucratic Safety regulations is an excuse for closed model AI systems and massive capitalization of the investments by the big players. Its inevitable that open models will reign and dominate across societies. Though the talent and risk taking it's incredibly admirable it shouldn't be the reason for the most important technology breakthrough of our time be left in the hands of so few.
Excellent article as always.
Personal transgressions aside, it’s hard not to look at things skeptically in terms of who benefits from the series of events.
The loss of the biggest open source advocate/competitor is a major blow.
Having spent most of my career in the DoD and Federal spaces I have no doubt we’re going to see blunt policy and regulatory changes both at the state and Federal levels that will create a cumbersome patchwork quilt of complex and costly regulation that only the largest players can afford and navigate, creating a regulatory moat (this will apply to commercial proprietary models as well).
I agree. I tried to leave my personal sadness of the events out of it. As it's crushing to lose a role model, but I have also long believed in stronger enforcement of chip exports. A double edged sword.
At the same time, I think it's more of timing this NatSec concern in government, rather than a conspiracy/op from tech industry competitors.
Yeah conspiracy aside, and even the chips focus as well, it’s hard to not think that we won’t see model-centric policy and regulation forthcoming, which will likely be complex and costly for anyone aside from the most capitalized labs who’ve recruited the talent and have the resources to navigate it.
After 20+ years in Cyber doing a ton of compliance, these policies and regulations are often blunt instruments and crafted by people without a depth of expertise in the underlying technology and it struggles to keep pace with the evolution of the tech.
That will be more true than ever with AI.
"With such a coordinated campaign of pressure from the AI safety community that has open models as a negative target"
I'm very confused by this comment. I know a *lot* of people in the AI safety community—and I'm struggling to think of one person actively trying to ban open-source models (over a certain level of capability). Don't get me wrong, many people, myself included have *talked about it*, or think it'd be a positive development if it happened, but talking something or hoping something happens is very different from actually going out there and taking steps to make it happen. Or are you including things like export controls?
This comes from discussions with staffers. The ai safety community focuses on risks of open weight models, including in their messaging to congress, lawmakers want to make ai safer, they connect the two together. I don’t think this is a majority position in the AI safety community, but the most powerful voices (Anthropic) do think so (eg the glm 5.3 hit piece)
so that’s not to say it’s an explicit position they take; but an easy inference from other statements.
There’s a similar argument on
china ai unsafe
China ai open
Open ai unsafe
My intuition is that Dario is much more worried about China than open weights models. I’m not saying that he likes open weights models, just that I suspect that the target of that piece was Chinese models.
Terrific article. Thank you.
Go to a meeting on AI issues on Capitol Hill, and you inevitably either immediately precede or follow another meeting with one of the myriad of EA-funded groups flooding Hill staffers with very particular AI Safety messages. Potentially losing Nvidia as an active player in this part of the policy space would be as bad as you describe it.
Even if not always effective, Nvidia’s advocacy prevented the emergence of a uni-party Safety movement. And those risks are very real. Even the right-coded Foundation for American Innovation has dipped its toe in the Safety waters. The members of the SI (sic) Task Force are largely open model aligned, but we are facing a strong pro Safety majority in Congress.
And yes, I have started capitalizing Safety, as we need to call it what it is - a coordinated movement of connected organizations that often share funding sources advancing a specific political and social agenda.
Great article! You touched all the key aspects, and I am glad that you also addressed the timing of the series of events. On a different note, I find it risky for the open source community to rely on a handful of individuals for support. I don't know how realistic a shift from that approach would be, given the capital required.
Bureaucratic Safety regulations is an excuse for closed model AI systems and massive capitalization of the investments by the big players. Its inevitable that open models will reign and dominate across societies. Though the talent and risk taking it's incredibly admirable it shouldn't be the reason for the most important technology breakthrough of our time be left in the hands of so few.