Alignment Workshops: Vienna + Bay Area

This year, we welcomed two amazing new team members: Lindsay Murachver and Vael Gates. Together, they spearheaded two impactful Alignment Workshops—one in Vienna and the other in the Bay Area—bringing together global researchers and leaders to tackle the potential risks posed by advanced AI. Curious about the conversations that unfolded? Check out highlights below or explore the full playlist. Interested in attending a future workshop? Let us know!

In 2024, we released 12 papers, with highlights including:

GPT-4o Guardrails Gone!

Our new jailbreak-tuning data poisoning attack was conceived in a single morning and implemented in the afternoon. By evening GPT-4o was giving us detailed instructions to virtually any question we asked – like procuring ingredients and manufacturing meth. Read more.

Planning in Neural Networks

To better prepare for potential misalignment, we studied how neural networks learn to plan in Sokoban, trying to interpret and steer those plans. Read more.

Will Scaling Solve Robustness?

In recent years, increasing the size of language models and their datasets has unlocked a dazzling array of capabilities. Yet, these models are vulnerable to adversarial inputs that induce models to perform undesired behaviors. Will scaling compute improve robustness like it has capabilities? Read more.

Defending Go AIs from Attacks

Last year, we discovered that “superhuman” Go AIs can be beaten by human amateurs exploiting a blindspot that humans do not have. This year, we try to patch the problem using adversarial training and alternative architectures. Read more.

FAR.AI Seminars

Every Wednesday, we host a seminar at FAR.Labs, our collaborative AI safety research hub, where leading AI safety researchers and founders present their latest work for feedback and discussion. This year, we've featured fireside chats with Neel Nanda and Buck Shlegeris, presentations from Dan Hendrycks, and a series from frontier lab leaders including Anca Dragan, Rohin Shah, and Sam Bowman. Check out some of this year’s talks below or view the full playlist.

Growing Together

Over the past year, in conjunction with our dedicated members, we've put down roots and are excited to grow our programs and collaborations. We invite select Bay Area researchers and technologists to work part-time from FAR.Labs and attend events and programs at our offices. To apply, please complete our Community Collaborators form. This year, we’ve hosted many visitors—over 1,000 visitor-days—including participants in our residency and retreat programs. If you have a trip planned to the Bay Area and would like to visit our center, apply here.