Research

FAR.Research delivers technical breakthroughs to improve the safety and security of frontier AI systems.

See all publications
Google Scholar

Complex problems demand complex solutions.

FAR.AI conducts research to address fundamental artificial intelligence (AI) safety challenges. We rapidly explore a diverse portfolio of technical research agendas, de-risking and scaling up only the most promising solutions. We share our research outputs through peer-reviewed publications, via partnerships with governmental AI safety institutes, and through red-teaming engagements for leading AI companies.

As AI systems become more powerful and integrated into society, it’s critical to ensure they remain trustworthy, secure, and aligned with human interests. The stakes are high, yet currently, there are no known techniques capable of providing high-confidence guarantees of safety for frontier AI systems.

FAR.Research is dedicated to delivering the novel technical breakthroughs needed to mitigate the potential risks posed by frontier AI. As a non-profit research institute, we leverage our unique flexibility to focus on critical research directions that may be too large or resource-intensive for academia and often overlooked by the commercial sector due to their lack of immediate profitability.

Our Impact

We drive change through incubating research, scaling safety solutions, and informing policy.

Incubating

We derisk and develop innovative solutions to trustworthy & secure AI. Through incubating research, we share key insights, research roadmaps, and tools needed for the broader research community to identify and make progress.

Scaling

We scale up the most promising safety solutions via in-house research, external collaborations, and targeted grantmaking. We facilitate rapid adoption of our findings by working with frontier model developers through red-teaming and other exercises.

Informing

Our research provides expert insights informing policy and public discussion. Our work has been cited in congressional testimony and mainstream media. In this way, we contribute to the establishment of technical standards that guide the development of AI.

Research highlights

Our research is focused on key insights to fundamental risks from advanced AI. This is an abridged selection of FAR.AI’s latest and most notable research.

Illusory Safety: Redteaming DeepSeek R1 and the Strongest Fine-Tunable Models of OpenAI, Anthropic, and Google

Model Evaluation

DeepSeek-R1 has recently made waves as a state-of-the-art open-weight model, with potentially substantial improvements in model efficiency and reasoning. But like other open-weight models and leading fine-tunable proprietary models such as OpenAI’s GPT-4o, Google’s Gemini 1.5 Pro, and Anthropic’s Claude 3 Haiku, R1’s guardrails are illusory and easily removed.

Pacing Outside the Box: RNNs Learn to Plan in Sokoban

Interpretability

Giving RNNs extra thinking time at the start boosts their planning skills in Sokoban. We explore how this planning ability develops during reinforcement learning. Intriguingly, we find that on harder levels the agent paces around to get enough computation to find a solution.

Beyond the Board: Exploring AI Robustness Through Go

Robustness

Achieving robustness remains a significant challenge even in narrow domains like Go. We test three approaches to defend Go AIs from adversarial strategies. We find these defenses protect against previously discovered adversaries, but uncover qualitatively new adversaries that undermine these defenses.

Our research portfolio covers three core agendas.

Robustness

Will superhuman AI systems be reliable and secure?

More on Robustness
Does Robustness Improve with Scale?
Frontier LLMs like ChatGPT are powerful but not always robust. Scale helps with many things. We wanted to see if scaling up the model size can ‘solve’ robustness issues.
Exploring Scaling Trends in LLM Robustness
While larger language models exhibit impressive capabilities, they remain vulnerable to adversarial prompts. Empirical findings show that robustness against such attacks significantly improves with adversarial training, but not with model scaling alone.
Even Superhuman Go AIs Have Surprising Failure Modes
Our adversarial testing algorithm uncovers a simple, human-interpretable strategy that consistently beats superhuman Go AIs.
Can Go AIs be adversarially robust?
We tested three approaches to defend Go AIs from adversarial strategies. While these defenses protect against previously discovered adversaries, we uncovered qualitatively new adversaries that undermine these defenses.
Beyond the Board: Exploring AI Robustness Through Go
Achieving robustness remains a significant challenge even in narrow domains like Go. We test three approaches to defend Go AIs from adversarial strategies. We find these defenses protect against previously discovered adversaries, but uncover qualitatively new adversaries that undermine these defenses.
Adversarial Policies Beat Superhuman Go AIs
We describe an attack on the state-of-the-art Go-playing AI system, KataGo. The adversaries do not win by learning to play Go better than KataGo but instead by tricking KataGo into making serious blunders, demonstrating that even superhuman AI systems may harbor surprising failure modes.

Model Evaluation

How can we tell if advanced AI systems are trustworthy and secure?

More on Model Evaluation
We Found Exploits in GPT-4’s Fine-tuning & Assistants APIs
We red-team three new functionalities exposed in the GPT-4 APIs: fine-tuning, function calling and knowledge retrieval. We find that fine-tuning a model on as few as 15 harmful examples or 100 benign examples can remove core safeguards from GPT-4, enabling a range of harmful outputs.
Exploiting Novel GPT-4 APIs
We red-team three new functionalities exposed in the GPT-4 APIs: fine-tuning, function calling and knowledge retrieval. We find that fine-tuning a model on as few as 15 harmful examples or 100 benign examples can remove core safeguards from GPT-4, enabling a range of harmful outputs. Furthermore, we find that GPT-4 Assistants readily divulge the function call schema and can be made to execute arbitrary function calls. Finally, we find that knowledge retrieval can be hijacked by injecting instructions into retrieval documents.
Inverse Scaling: When Bigger Isn't Better
We present 11 instances of inverse scaling: tasks where language models get worse with scale rather than better, selected from 99 submissions in an open competition, the Inverse Scaling Prize.

Interpretability

How can we understand an AI’s internals?

More on Interpretability
Planning behavior in a recurrent neural network that plays Sokoban
To understand how neural networks generalize, we studied an RNN trained to play Sokoban. The RNN learned to spend time planning ahead by "pacing" despite penalties for "taking longer", demonstrating that reinforcement learning can encourage strategic planning in neural networks.
Pacing Outside the Box: RNNs Learn to Plan in Sokoban
Giving RNNs extra thinking time at the start boosts their planning skills in Sokoban. We explore how this planning ability develops during reinforcement learning. Intriguingly, we find that on harder levels the agent paces around to get enough computation to find a solution.
InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniques
InterpBench is a collection of transformers with known circuits, trained using Strict Interchange Intervention Training (SIIT). These models exhibit realistic weights that reflect ground truth circuits, providing a benchmark for evaluating mechanistic interpretability techniques.
Codebook Features: Sparse and Discrete Interpretability for Neural Networks
We modified neural networks for greater interpretability and steerability with minimal performance loss. Each layer applies a quantization bottleneck, converting dense activation vectors into a discrete list of learned codes that are either on or off.
Codebook Features: Sparse and Discrete Interpretability for Neural Networks
We modified neural networks for greater interpretability and steerability with minimal performance loss. Each layer applies a quantization bottleneck, converting dense activation vectors into a discrete list of learned codes that are either on or off.
Subscribe to our newsletter
Organization
AboutTeamProgramsNewsSearch
Events
All EventsAlignment WorkshopsIDAISSpecialized WorkshopsAll Event Recordings
Research
All PublicationsResearch Overview
Robustness
Interpretability
Model Evaluation
Get involved
CareersContactDonateNewsletter
Financial Reports / 990sPrivacy PolicyTerms of Service

Cookies Notice: This website uses cookies to identify pages that are being used most frequently. This helps us analyze web page traffic and improve our website. We do not and will never sell user data. Read more about our cookie policy on our privacy policy. Please contact us if you have any questions.

© 2025 FAR AI, Inc.
Website by ODW
FAR.AI only uses cookies essential for website functionality and anonymous usage analytics.
I understand
View Privacy Policy to learn more.