Frontier Model Safety
Summary
Alex Bores warns of a rapidly closing policy window for AI safety, outlines New York's frontier AI safety legislation, and urges direct constituent engagement with state legislators
SESSION Transcript
It's great to be here. My talk is on frontier AI safety policy. I'm going to start by saying that my wife and I are expecting our first child later this year. Thank you, thank you. I actually see a few friends in the crowd that I didn't get to tell personally, so I'm sorry you're finding out this way.
But the reason I start there is because, like many expecting parents, I have been reflective and thinking about that the world that my child is about to be born into is vastly different than the world that I was born into. But when we talk about AI, sometimes the world that you're born into is vastly different than the world at the start of the pregnancy. So I just want to talk about things that happened in the last nine months.
First, I want to start our story literally on Halloween. I like that Anthropic published this on Halloween, October 31st. But they said, this is in the wake of 1047 being vetoed, that government should urgently take action on AI policy in the next 18 months. Again, that was seven months ago. So they're saying by April 2026. In the US, this will ideally happen at the federal level, though urgency may demand it is instead developed by individual states.
Then in the wake, again, of 1047, Governor Newsom empaneled a commission to study what states should do in policy. And that commission, among many findings, said that policy windows do not remain open indefinitely. You have a limited time where you can have intervention in policy before path dependency sets in and making any changes with big technology that you're building upon actually becomes harder and maybe impossible.
And then of course, jumping ahead to just nine days ago, Palisade Research—anyone from Palisade Research here? No? Great, so you can't check me on this then. No, so Palisade Research put out a study where they found that ChatGPT, o3 in particular, had been asked to do a task and then allow itself to be shut down, and most of the time it did. Seven times it did not. One of the times when it was given the command to shut down, it simply said, "intercepted."
So I say all that to say, speed is of the essence. And when speed is of the essence, I encourage people to think about the states.
There's at least four reasons why AI policy at the state level is really important right now. The first is that we move a lot faster. Passing a bill at the federal level can take a year, three years, five years, 10 years. I've passed a bill at the state level in four weeks. I don't recommend that on AI policy. It should take more time, but the point is we have the capacity to move things very quickly.
The second is that we do a lot more bills. So in 2023, New York passed 774 laws all the way through the process signed by the governor. Congress as a whole in 2023 passed 27. So you have more shots when you're at the state level.
You also allow for unique coalitions. We saw with SB 1047 that you had academics and researchers and startups standing with actors and unions. At the state level, the politics are different than they are at the federal level. They may not be as entrenched. It can allow you to break things through that might not ever be possible at the federal level.
And the last, it's obvious we all know this, but it's worth repeating, is at the state level you get 50 bites at the apple. Not necessarily every year. Actually, some states, their legislature only meets once every two years, but minor detail, you get 50 bites at the apple to really make a change.
So I want to talk about eight that have been live this year. I'm going to start with the four where legislative session is done or wrapping up this weekend, so these bills have not passed this year, and then go to four more where they're still live. I will say I put this together very quickly. I am sure—I see people taking photos—I am sure I got one or two boxes wrong on this. So take this as a high-level overview and not necessarily absolute fact on each of these bills. But I want to talk about the different approaches that have been tried at the state level and then talk about what's live and where you can actually take action.
So I'm going to start, and again, they're ordered in terms of when the legislative session ends, which is sometimes a tricky question. In New York, our Senate ends on June 12th, our Assembly ends on June 17th—don't ask.
But starting with Colorado, May 7th, it's focused on whistleblower protections. This has been the most common aspect of any of the state bills, and I think the one that has the largest agreement on it.
Vermont, Monique Priestley, H341, focused on AI that generates text or is high risk, and talked about risk management plans and regular incident reporting.
Illinois—Illinois was really close this year. Dan Didech made a great push here, along with help from many people in this room, that focused on transparency around safety plans and making sure there was disclosure about what companies were actually doing without necessarily holding them to external standards, but getting that information out in the open, getting capability disclosure out in the open.
And then Texas, which is the one red state that's up here, started with a bill that included frontier models along with many other things, but it got edited down over session and so the final version didn't really cover that. But it was an attempt, it was a push forward, it's happening both blue states and red states, and a thing to keep in mind as we keep pushing going forward.
I'm going to jump to the ones that are live, and I'm actually going to do this backwards. So Massachusetts, which has the latest session ending around November 19th, Senator Barry Finegold has introduced basically an exact copy of 1047. There's, I think, a word change here or there, but you can model it very similar to SB 1047.
Conversely, in California, Senator Scott Wiener, the original author of 1047, now has a bill that is focused almost exclusively on whistleblower protections. It builds out a cloud compute cluster—to the comment earlier about state capacity and being able to do research, it aids on that side as well. California probably more than any other state frequently amends bills. So I think this one's already been amended three times and may evolve into something else in the future, but that's the current version of it.
Rhode Island has a pretty interesting proposal which doesn't really lay out specific standards for AI research, but does say that AI model developers will be strictly liable for harms that come from their models. So it basically says, government, maybe we are not the best to tell you how to keep your model safe, but we'll just make sure you internalize all of the risk. You are the experts. Figure out the best way to keep everyone safe, because if something goes wrong, it's all on your shoulders. I think it's an interesting proposal. I actually have a separate version of it I'm working on in New York, but a lot of credit to Senator Victoria Gu for pushing that forward.
And then we have the RAISE Act, which is what I'm working on in New York. Both because that's the one I know best, and also because it's the one with the most upcoming deadline, the most recent upcoming deadline, just less than two weeks away, June 12th, I want to spend some time talking about where this is and how we're thinking about AI frontier model regulation in New York.
So the RAISE Act asks for companies to do four things. They have to have an SSP. They have to have that SSP audited by a third party, not the government. They actually have to choose their own third party that does that. They have to disclose critical safety incidents, and we define that very specifically in the bill as to what would be something that would have to be disclosed. And they have to not terminate employees or contractors that raise critical risk. There's already very strong whistleblower protections in New York State. But this is just adding in a clause about something that isn't illegal but is up to a catastrophic risk. People should be protected for raising that up.
It applies only to large companies, so those that have spent $100 million or more in training frontier models. It exempts academia entirely. Obviously, that $100 million threshold, I think, exempts what any startup is currently building in terms of—again, that is just on compute and just on training. And it focuses on making sure that the companies are setting a baseline standard ahead of time through their SSP and not changing it after the fact.
I often talk about this as ensuring that the cigarette companies, the tobacco companies, don't know that their cigarettes cause cancer and still release their products, or the oil companies know that they are causing climate change and say, internally they know that, but externally they say, we're not causing climate change and still put it out. This is saying, you write up your safety plan, you design what the tests are, and if your own tests come back and say, hey, this is really scary and bad, well then you need to be taking action based on that. And it's what most companies are already doing, and in many ways what most companies have already committed to, but putting it into language to make sure that there is a baseline for everyone to take that action.
There's a lot more detail to be talked about here, especially in this room. I'm not going to get into all of it now, but I'll be here the whole day. Please come up to me, ask questions, happy to talk through any of the aspects of the RAISE Act going forward.
So where is that? You saw that June 12th deadline coming up. Well, it has passed both houses of the legislature. So in the Assembly, that means starting in the Science and Tech Committee, going through Codes, then going to the Rules Committee, then to the floor where it's either passed on consent or the opposition party says we want to debate this. I expect this one obviously to be debated. But I will point out that in both the Assembly and the Senate, we have bipartisan co-sponsors of this bill. And when it passed the Assembly Science and Tech Committee earlier this week, it did it unanimously. Everyone from both parties voted for it.
But like any good student, the New York State Legislature likes to leave all of our work for the last minute. So this looks like a long path, but I would say the majority of bills are probably in exactly this state. The majority of bills that will end up being passed in the next two weeks are in this state. And in the Senate, it's still sitting in the Internet and Tech Committee, and then it'll go to Codes. They don't have a rules committee. It'll then again go on a consent calendar or for a shorter debate.
And so what I want to focus on here is the fifth reason why you should look at states. I talked about four before, but the fifth reason is we're much more responsive. People email their congressmember all the time, they call their congressmember all the time. They're used to hearing a lot of things. Most state legislators answer our own phones and answer our own email. And hearing from a bunch of constituents or a bunch of industry experts actually makes a really big difference to us.
So here's at least two ways that I would invite anyone who's interested to get involved. The first is that if you are running an organization or at an organization that in any way is involved in AI, which I think basically means if you are at this conference—but if you are at this conference, you can scan that QR code, you can go to that Bitly, and it'll help prepare what's called a memo of support. You write in, I am this person, this is what I do in the field, this is why I would like to see some reasonable regulations put in at the state level.
I'll add in that no individual company has come out against the bill publicly. Some small startups have come out for the bill publicly. Many big companies are privately really pushing back against it. So a lot of trade associations have come out against the bill. And so I would add in separately that if you work at a large company or you work at a startup that is part of a trade association, I would ask internally, hey, how are we voting at the trade association? Are we supporting safety bills? Are we saying neutral? Are we saying that we should be opposing them? I think it's a question you all have the right to an answer for.
But the second one I would add in is that if you are from New York—anyone here from New York? Anyone here from the east side of New York? Do you have any constituents? Yeah, okay, we got to chat later, nice. But if you're from New York or even relevantly, if you have family or friends in New York, again, we don't get many emails, we don't get many calls. Scan that link, they just need to put in their address, they don't need to know who their assembly member or state senator are. Most don't, that's okay. But it'll tell who their assembly member and state senator is, it'll prepare an email for them about why they should support this bill.
And I want to point out that there are so many times where we have deep deliberation on the right path forward and deep academic discussions on the right path forward. And we're not always going to agree on every step. I think often when it comes to legislation, people say, well, I don't agree 100% with this. I don't agree 100% with the RAISE Act. It is a bunch of compromises to get there, as all legislation is. And you saw we had eight different states. They're all slightly different.
But read it, talk to me at this conference. If you think it's something that helps to advance the cause of safety, we have two weeks where your direct actions can make a huge difference in whether we get any of these laws passed in New York. And so I can assure you that there are many people who are fighting against this and want to ensure nothing goes forward. And I would invite you all to join the fight in making sure that something does that we can build upon.
So I just want to end this talk by thanking you for your time, thanking you for all that you have already done to make the world a safer place and to advance AI safety, and thank you for all that you are going to do going forward, both in your jobs and your professions already, and by directly taking action when you can on policy to make sure that the world that my child is born into is one that is safer than the world today. Thank you.
