Impact of Frontier AI in the Landscape of Cyber Security

Hi everyone. Thanks for being here. My name is Dawn Song. I'm a professor in computer science at UC Berkeley.
Today I'll talk about how Frontier AI will change the landscape of cybersecurity. This is condensed from an hour-long talk into five minutes, so I'll go through quickly. Please bear with me.
We are excited about the exponential growth of recent AI technology development. At the same time, there is a broad spectrum of AI risks, especially listed in the recent International AI Safety Report that I have been part of. There are many risks in this broad spectrum, and today I will focus particularly on one, which is in cybersecurity. I strongly believe cybersecurity, among all these AI risk domains, will be one of the biggest AI risk domains for us to pay attention to and be aware of for a number of reasons.
One, currently, there are diverse incentives for attackers to attack, and also we are already seeing everyday most attacks causing huge issues. With increased AI capabilities, it can significantly reduce attack cost and increase attack surface, especially as we have seen recently, with the fast increase in terms of coding and reasoning capabilities on the latest AI technologies. As we further develop these to help developers to develop software, find bugs, and automatically fix bugs and so on, these capabilities, of course, unfortunately will be misused by attackers as well.
Given that AI technology is a dual-use technology, one natural question is, and the key question is, how will frontier AI change the landscape of cybersecurity? In particular, here we have the traditional cybersecurity with attackers and defenders and traditional software systems, which are what we call symbolic programs written by humans. With cybersecurity with the frontier AI, we have attackers with frontier AI, defenders with frontier AI, and also even the software systems now are new. We call them hybrid, which contains both symbolic programs written by humans and also AI, and also non-symbolic programs with AI models and so on.
The key question is, who will frontier AI benefit more, attackers or defenders? In order to address these questions, actually there are many different aspects that we need to go through for systematic analysis. We need to know the enemy, we need to understand the impact of misused AI in attacks and know the defense, understand the impact of AI in defenses, and also there's a natural asymmetry between defense and offense. There are many lessons that we can learn from the past and the important predictions.
Currently, we are doing work, actually doing a systematic marginal risk assessment framework for analyzing this. It seems that my time is running really short, so I'll just skip through all these really quickly. So to understand how misuse of AI can make—can affect attacks, there's the two aspects, the AI can—attack machines and also humans. And for AI to attack machines, we actually need to pay attention to there is a complex like kill chain for—many steps needed to carry out a successful attack. So given an interest of time, I will skip many of these. So in summary, the current AI capabilities, they can help attacks to a certain extent, especially for social engineering, but for—but for other aspects, like attacking machines, it still has limited capabilities. And on defenses, also we show there's different steps how AI is helping defenses. So the challenge here is that the on one hand, as we improve AI capabilities for defenses and for software development, but also there's an equivalence class, the same capabilities can help on the attack side as well. So in the paper, we actually show this concrete, comprehensive analysis, and also there's a natural asymmetry between defense and offense. And also, given interest of time, I won't have time to go through. So to summarize, essentially, there's a huge consequence for [laughter] doing so there's no time to go through. So we're also doing some formal theorem proving, theorem proving using AI to help address these challenges. So in the end, I just want to show the QR code. So if you're interested—so we have a draft paper in this.

So if you're interested in this, you can please scan the QR code. We have a draft paper. We'd love to get feedback. You can fill in the form. And also we have, we are doing a survey, so you can join the survey as well. And then finally, I'm also teaching a massive, open, online course on large language models. You can go to this URL, and we actually cover some of the topics that I mentioned here, for example, using AI for theorem proving. Okay. Thank you.

‍