Hi,
Last month I had the honor of moderating the Safety on the Frontier of AI panel for AI Safety Connect at the UN General Assembly. My big takeaway was that labs worldwide need shared technical standards to mitigate misuse risks – and I'm encouraged by how independent safety research can drive these standards forward. When we identify vulnerabilities through rigorous red teaming and evaluation, and labs respond by implementing patches before deployment, we see the collaborative safety ecosystem working as it should. Still, we’re only as safe as the most vulnerable model, so it’s crucial that every advanced system receives this level of scrutiny. Despite the challenges, I left cautiously optimistic: the risks are serious, but proactive vulnerability disclosure and iterative improvements are making each generation of AI systems more robust than the last.
I am also encouraged by the cooperation we've seen from model developers. For instance, our red teaming of GPT-5, and the disclosure of our Attempt to Persuade Eval benchmark led to OpenAI and Google implementing fixes helping improve the safety of the latest versions of Gemini 2.5 Pro and GPT-5.
However, there are still no technical solutions to many of the problems we find. I’m excited to announce we just raised over $30M to fix that. We’ll be doubling our team over the next 18 months – if you're excited by the opportunity to shape AI development, please apply to one of our open roles or recommend a talented peer.
