Join the FAR.AI Newsletter

In late May, I had the pleasure of speaking at our inaugural Technical Innovations for AI Policy Conference in Washington, D.C. It was wonderful to see technical experts, researchers, and policymakers come together to tackle challenges that seemed intractable not too long ago. This spirit of collaboration was equally evident at our Singapore Alignment Workshop. Through more than 40 presentations on critical vulnerabilities, it became clear that there is a growing, diverse, and committed community rising to meet the complex global challenges presented by advanced AI. In his keynote, Yoshua Bengio reminded everyone of the stakes involved by sharing his personal concerns for his children alongside a call for technical safeguards. As AI capabilities continue to progress rapidly, our research team has continued to investigate critical questions including how robust defense-in-depth pipelines really are, using lie detectors to teach AI honesty, and reverse-engineering neural networks to understand how AI agents plan. It is our hope that with rigorous research identifying problems, practical solutions to technical and governance issues, and the strategic partnerships we’re fostering, we can achieve a future where powerful AI systems benefit everyone. Thank you for being part of this journey with us.

This quarter has seen significant progress in our work on AI safety, furthering our mission to facilitate breakthrough AI safety research, foster coordinated global solutions, and advance global understanding of AI risks and solutions. In February, the Paris AI Security Forum brought together 200 researchers, engineers, and policymakers to address growing AI security challenges. This forum, along with our participation in the International Association for Safe and Ethical AI conference at the OECD, has contributed to a growing ecosystem of talented people working to secure and safeguard advanced AI systems. Our research teams have submitted four papers for review at academic conferences in areas like robustness in large language models, safety cases for open-weight models, and mechanistic interpretability for RL agent’s planning. We've also launched a controlled-access jailbreak-tuning demo showing vulnerabilities in frontier AI models — a valuable educational tool for researchers studying AI safety risks. I'm excited to share these developments with you and look forward to continuing our vital work together.

Welcome to the first edition of our newsletter! Last month I had the privilege of attending the Inaugural Convening of the International Network of AI Safety Institutes in San Francisco. It is exciting to see in the span of a year over ten countries establish AI safety institutes. I am pleased that FAR​.​AI has been at the forefront of encouraging global technical collaboration on AI safety. In particular, the Safe AI Forum team (fiscally sponsored by FAR​.​AI) organized two International Dialogues on AI Safety (IDAIS) this year, bringing together scientists from China and the West to address shared AI risks and explore solutions. At the most recent event in Venice, participants emphasized AI safety as a global public good, calling for emergency preparedness agreements, safety assurance frameworks, and independent global research. These events capped off a year of significant milestones for FAR​.​AI. Our researchers have been investigating compute requirements for attacking LLMs, bypassing guardrails in frontier OpenAI models, uncovering how AI agents form plans, and more. We also hosted our largest-ever Alignment Workshops yet in Vienna and the Bay Area, bringing together researchers and leaders to chart strategies for safe AI. There’s much more to come, and I’m excited to share the journey with you. As we wrap up 2024, I want to wish you and your loved ones a happy and peaceful holiday season! Your support this year has helped us achieve incredible milestones in advancing AI safety. As we look to 2025, your contributions can continue to make a lasting impact. Consider making a year-end donation to help us scale our research and programs.