All Posts

Frontier LLMs like ChatGPT are powerful but not always robust. Scale helps with many things. We wanted to see if scaling up the model size can ‘solve’ robustness issues.

Achieving robustness remains a significant challenge even in narrow domains like Go. We test three approaches to defend Go AIs from adversarial strategies. We find these defenses protect against previously discovered adversaries, but uncover qualitatively new adversaries that undermine these defenses.

We conducted 17 semi-structured interviews of AI safety experts about their big picture strategic view of the AI safety landscape: how will human-level AI play out, how things might go wrong, and what should the AI safety community be doing. While many respondents held “traditional” views (e.g. the main threat is misaligned AI takeover), there was more opposition to these standard views than we expected, and the field seems more split on many important questions than someone outside the field may infer.

FAR.AI is seeking applications for a Business Operations Manager, reporting directly to the Chief Operating Officer (COO). As Business Operations Manager, you will be responsible for day-to-day operational excellence at FAR.AI, ensuring our systems, processes and projects are thriving.
Deadline: Jun 7th, 2024.

FAR.AI is seeking a highly organized and detail-oriented Coworking Space Manager to oversee the daily operations of FAR.Labs, our AI safety co-working hub.

We present LLMs with a series of moral choices and find that LLMs tend to align with human judgement in clear scenarios. In ambiguous scenarios most models exhibit uncertainty, but a few large proprietary models share a set of clear preferences.

The New Orleans Alignment Workshop 2023 brought together leading ML researchers working to align advanced AI with human values and develop safe AI system. Presentations from industry, academia, and non-profits focused on topics spanning from oversight, interpretability, robustness, generalization and governance.

We red-team three new functionalities exposed in the GPT-4 APIs: fine-tuning, function calling and knowledge retrieval. We find that fine-tuning a model on as few as 15 harmful examples or 100 benign examples can remove core safeguards from GPT-4, enabling a range of harmful outputs. Furthermore, we find that GPT-4 Assistants readily divulge the function call schema and can be made to execute arbitrary function calls. Finally, we find that knowledge retrieval can be hijacked by injecting instructions into retrieval documents.

We are seeking research engineers looking to execute on safety research agendas and/or perform red-teaming.
Rolling Applications.