News

Bay Area Alignment Workshop brought together researchers and leaders from academia, industry, government, and nonprofits convened to guide the future of AI toward safety and alignment with societal values. Over two packed days, participants engaged with pivotal themes such as evaluation, robustness, interpretability and governance.

A tiny dose of poisoned data can cause big problems for AI. Our jailbreak-tuning method causes models like GPT-4o to capably answer virtually any harmful question. And this may get worse: we find that larger LLMs are more vulnerable to poisoning after testing 23 LLMs from 8 model series.

Giving RNNs extra thinking time boosts their planning skills in Sokoban, helping them solve harder puzzles by pacing to buy more computation time. By training linear probes, we can predict and modify the agent’s internal plans, while model surgery allows the network to generalize to larger levels. Additionally, interpretability techniques like action channels and sparse autoencoders reveal how the RNN organizes its decision-making.

The Vienna Alignment Workshop gathered researchers to explore critical AI safety issues, including Robustness, Interpretability, Guaranteed Safe AI, and Governance, with a keynote by Jan Leike. It was followed by an informal Unconference, fostering further discussions and networking.

Frontier LLMs like ChatGPT are powerful but not always robust. Scale helps with many things. We wanted to see if scaling up the model size can ‘solve’ robustness issues.

Achieving robustness remains a significant challenge even in narrow domains like Go. We test three approaches to defend Go AIs from adversarial strategies. We find these defenses protect against previously discovered adversaries, but uncover qualitatively new adversaries that undermine these defenses.

We conducted 17 semi-structured interviews of AI safety experts about their big picture strategic view of the AI safety landscape: how will human-level AI play out, how things might go wrong, and what should the AI safety community be doing. While many respondents held “traditional” views (e.g. the main threat is misaligned AI takeover), there was more opposition to these standard views than we expected, and the field seems more split on many important questions than someone outside the field may infer.

We present LLMs with a series of moral choices and find that LLMs tend to align with human judgement in clear scenarios. In ambiguous scenarios most models exhibit uncertainty, but a few large proprietary models share a set of clear preferences.