Research

FAR.Research delivers technical breakthroughs to improve the safety and security of frontier AI systems.

See all publications

Complex problems demand complex solutions.

FAR.AI conducts research to address fundamental artificial intelligence (AI) safety challenges. We rapidly explore a diverse portfolio of technical research agendas, de-risking and scaling up only the most promising solutions. We share our research outputs through peer-reviewed publications, via partnerships with governmental AI safety institutes, and through red-teaming engagements for leading AI companies.

FAR.Research is dedicated to delivering the novel technical breakthroughs needed to mitigate the potential risks posed by frontier AI. As a non-profit research institute, we leverage our unique flexibility to focus on critical research directions that may be too large or resource-intensive for academia and often overlooked by the commercial sector due to their lack of immediate profitability.

Research

Research Agendas

View All Publications

Research agendas

Red-Teaming & Evaluation

How can we tell if advanced AI systems are trustworthy and secure?

More on Red-Teaming & Evaluation

Frontier LLMs Attempt to Persuade into Harmful Topics

Large language models (LLMs) are already more persuasive than humans in many domains. While this power can be used for good, like helping people quit smoking, it also presents significant risks, such as large-scale political manipulation, disinformation, or terrorism recruitment. But how easy is it to get frontier models to persuade into harmful beliefs or illegal actions? Really easy – just ask them.

It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics

In order to persuade users, LLMs must both be capable of persuading and willing to do so. Existing research explores the former, and we present the Attempt to Persuade Eval (APE) benchmark that tests how willing LLMs are to generate content aimed at shaping beliefs and behavior to flesh out the latter.

Layered AI Defenses Have Holes: Vulnerabilities and Key Recommendations

We tested the effectiveness of "defense-in-depth" AI safety strategies, where multiple layers of filters are used to prevent AI models from generating harmful content. Our a new attack method, STACK, bypasses defenses layer-by-layer and achieved a 71% success rate on catastrophic risk scenarios wher conventional attacks achieved 0% success against these multi-layered defenses. Our findings highlight that multi-layer AI defenses, while valuable, have significant vulnerabilities when facing attacks specifically designed to penetrate multiple defensive layers sequentially.

STACK: Adversarial Attacks on LLM Safeguard Pipelines

We tested the effectiveness of "defense-in-depth" AI safety strategies, where multiple layers of filters are used to prevent AI models from generating harmful content. Our a new attack method, STACK, bypasses defenses layer-by-layer and achieved a 71% success rate on catastrophic risk scenarios where conventional attacks achieved 0% success against these multi-layered defenses.

TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering

We built TamperBench, a unified framework for evaluating the tamper resistance of open-weight LLMs against weight-space and latent-space attacks, addressing the lack of standardized benchmarks in this area. It evaluates 21 models across nine attack types with systematic hyperparameter sweeps, covering both safety and utility metrics. Key findings include that jailbreak-tuning is generally the most severe attack and that Triplet is the strongest alignment-stage defense.

Deception

How can we detect and prevent AI deception?

More on Deception

The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes

Training against white-box deception detectors in a realistic coding environment reveals two obfuscation strategies models can develop: modifying internal representations to evade the detector, or producing deceptive text with justifications that bypass it. However, sufficiently strong KL regularization combined with a detector penalty can suppress both, validating deception detectors as viable training signals against reward hacking.

Auditing Games for Sandbagging

Using a red-team/blue-team auditing game, we find that black-box and naive model-internal methods fail to consistently distinguish sandbagging from benign underperformance. Training-based capability elicitation reliably restores full performance in sandbagging models but also risks false positives by boosting benign models.

Preference Learning with Lie Detectors can Induce Honesty or Evasion

Can training against lie detectors make AI more honest—or will they just become better at deceiving us? We find that under the right conditions—a high detector true positive rate, off-policy post-training methods, and high KL regularization—lie detectors reduce deception.

Avoiding AI Deception: Lie Detectors can either Induce Honesty or Evasion

Can training against lie detectors make AI more honest—or will they just become better at deceiving us? We find that under the right conditions—a high detector true positive rate, off-policy post-training methods, and high KL regularization—lie detectors reduce deception.

Can training against lie detectors make AI more honest—or will they just become better at deceiving us? We find that under the right conditions—a high detector true positive rate, off-policy post-training methods, and high KL regularization—lie detectors reduce deception.

Robustness & Security

Will superhuman AI systems be reliable and secure?

More on Robustness & Security

Does Robustness Improve with Scale?

Frontier LLMs like ChatGPT are powerful but not always robust. Scale helps with many things. We wanted to see if scaling up the model size can ‘solve’ robustness issues.

Exploring Scaling Trends in LLM Robustness

While larger language models exhibit impressive capabilities, they remain vulnerable to adversarial prompts. Empirical findings show that robustness against such attacks significantly improves with adversarial training, but not with model scaling alone.

Even Superhuman Go AIs Have Surprising Failure Modes

Our adversarial testing algorithm uncovers a simple, human-interpretable strategy that consistently beats superhuman Go AIs.

Can Go AIs be adversarially robust?

We tested three approaches to defend Go AIs from adversarial strategies. While these defenses protect against previously discovered adversaries, we uncovered qualitatively new adversaries that undermine these defenses.

Beyond the Board: Exploring AI Robustness Through Go

Achieving robustness remains a significant challenge even in narrow domains like Go. We test three approaches to defend Go AIs from adversarial strategies. We find these defenses protect against previously discovered adversaries, but uncover qualitatively new adversaries that undermine these defenses.

Adversarial Policies Beat Superhuman Go AIs

We describe an attack on the state-of-the-art Go-playing AI system, KataGo. The adversaries do not win by learning to play Go better than KataGo but instead by tricking KataGo into making serious blunders, demonstrating that even superhuman AI systems may harbor surprising failure modes.

Our Impact

We drive change through incubating research, scaling safety solutions, and informing policy.

Incubating

We derisk and develop innovative solutions to trustworthy & secure AI. Through incubating research, we share key insights, research roadmaps, and tools needed for the broader research community to identify and make progress.

Scaling

We scale up the most promising safety solutions via in-house research, external collaborations, and targeted grantmaking. We facilitate rapid adoption of our findings by working with frontier model developers through red-teaming and other exercises.

Informing

Our research provides expert insights informing policy and public discussion. Our work has been cited in congressional testimony and mainstream media. In this way, we contribute to the establishment of technical standards that guide the development of AI.

Subscribe to our newsletter

Organization

About Team Programs News Search

Events

All Events Alignment Workshops Specialized Workshops All Event Recordings

Research

All Publications Research Overview

Interpretability

Model Evaluation

Get involved

Careers Contact Donate Newsletter

Financial Reports / 990s Privacy Policy Terms of Service

Cookies Notice: This website uses cookies to identify pages that are being used most frequently. This helps us analyze web page traffic and improve our website. We do not and will never sell user data. Read more about our cookie policy on our privacy policy. Please contact us if you have any questions.

© 2025 FAR AI, Inc.

Website by ODW