Governing AI Agents Under the EU AI Act
Summary
Robin Staes-Polet explains the dual requierements for AI agents posed by the EAU AI Act
SESSION Transcript
Okay, thanks for coming for this lighting talk, basically I’m going to present a report that we’ve done: "Governing AI agents under the EU AI Act". My name is Robin, I’m from The Future Society, an independent non-profit org, with the mission to align AI to better governance. So, yeah, the research question for this report is basically, how does the EU AI Act apply to AI agents, and what kind of governance measures should be implemented across the AI value chain?
And these are three takeaways I’ll like to share with you guys. So, first, is that the AI Act imposes requirements on both underlying models, the GPAI models and the agents’ systems themselves, and through this different chapters of the AI Act. And the second takeaway is that effective AI governance imposes requirements across the entire value chain, so this is for model providers, but also system providers and deployers, and these are done through four primary mechanisms listed here.
So, yeah, I think to start off, I think it would be good to define what an AI agent is, and that would be–there are two possible ways to do that. The first one would be through a more technical view of its components, that is basically that agents are GPAI models with scaffolding added on top. And that's chain of thought reasoning or external tool access, for instance. This helps us understand that we can look at them through the lens of both model regulation and system regulation, and requires the input and the obligations across different types of actors.
Secondly, I think we can look at it through an operational point of view, which is basically that two big buckets, one of which they can exhibit autonomy, and this is so they can produce actions in complex long-term planning. But secondly, also being able to interact with the world through this. So, this creates new pathways to harm, but also allows us to inform our risk assessments and mitigation strategies to be more targeted.
So yeah, zooming into the AI Act itself. So, as we've just taken away, AI agents are systems built on GPAI models. This means that regulation can apply at both levels, through chapter five and chapter three. Chapter five applies to model providers and talks about how model providers, in the case of both in-house models, in-house agents developed by themselves, such as if OpenAI develops its own model with agentic capabilities, but also how they have to think about downstream users and downstream cases. If you have AutoGPT for instance, using that, their models, they also need to be held liable.
So, chapter three on the other hand talks about the use cases, so the high-risk use cases, which are regulated under the AI Act, and it depends whether agent risk actually qualifies as a high-risk status. That's an open question, but we think there's definitely a lot of cases where there are high-risk use cases of GPAI models being used for agents.
Basically, this diagram shows that overlap, right? So, if chapter five and chapter three both apply, that's that circle in the middle. And if they're just non-high risk use case, that would be the purple bit.
So, yeah, based on a literature review of what we could find, and also some expert interviews, we tried to map what governance measures we could find to create a taxonomy based on the articles of the AI Act. And I think these are here listed, I don't have time to go into them, but what's important to note is that these mechanisms need to be applied across the value chain. Model providers, but also system providers and deploys have a role. And this role is basically across the value chain, and depends on the separate expertise, resources, and information asymmetries across these different actors.
So, for instance, for risk identification model providers have a more high-level risk pathways mapping, and it goes down to more specific risk scenarios and into just like filling in a form, which systems deployers need to do.
So, we want to emphasize though that model providers bear a particular responsibility, because of their resources and technical expertise. So, basically this is a sneak peek of our report that we have out, just mapping that kind of responsibilities and yeah, if you're interested, this is basically the report that we have, it's a draft almost being published, but this basically goes through all that we've covered. And yeah, you can subscribe to our newsletter if you want to stay engaged with The Future Society and our research. Thank you.
And these are three takeaways I’ll like to share with you guys. So, first, is that the AI Act imposes requirements on both underlying models, the GPAI models and the agents’ systems themselves, and through this different chapters of the AI Act. And the second takeaway is that effective AI governance imposes requirements across the entire value chain, so this is for model providers, but also system providers and deployers, and these are done through four primary mechanisms listed here.
So, yeah, I think to start off, I think it would be good to define what an AI agent is, and that would be–there are two possible ways to do that. The first one would be through a more technical view of its components, that is basically that agents are GPAI models with scaffolding added on top. And that's chain of thought reasoning or external tool access, for instance. This helps us understand that we can look at them through the lens of both model regulation and system regulation, and requires the input and the obligations across different types of actors.
Secondly, I think we can look at it through an operational point of view, which is basically that two big buckets, one of which they can exhibit autonomy, and this is so they can produce actions in complex long-term planning. But secondly, also being able to interact with the world through this. So, this creates new pathways to harm, but also allows us to inform our risk assessments and mitigation strategies to be more targeted.
So yeah, zooming into the AI Act itself. So, as we've just taken away, AI agents are systems built on GPAI models. This means that regulation can apply at both levels, through chapter five and chapter three. Chapter five applies to model providers and talks about how model providers, in the case of both in-house models, in-house agents developed by themselves, such as if OpenAI develops its own model with agentic capabilities, but also how they have to think about downstream users and downstream cases. If you have AutoGPT for instance, using that, their models, they also need to be held liable.
So, chapter three on the other hand talks about the use cases, so the high-risk use cases, which are regulated under the AI Act, and it depends whether agent risk actually qualifies as a high-risk status. That's an open question, but we think there's definitely a lot of cases where there are high-risk use cases of GPAI models being used for agents.
Basically, this diagram shows that overlap, right? So, if chapter five and chapter three both apply, that's that circle in the middle. And if they're just non-high risk use case, that would be the purple bit.
So, yeah, based on a literature review of what we could find, and also some expert interviews, we tried to map what governance measures we could find to create a taxonomy based on the articles of the AI Act. And I think these are here listed, I don't have time to go into them, but what's important to note is that these mechanisms need to be applied across the value chain. Model providers, but also system providers and deploys have a role. And this role is basically across the value chain, and depends on the separate expertise, resources, and information asymmetries across these different actors.
So, for instance, for risk identification model providers have a more high-level risk pathways mapping, and it goes down to more specific risk scenarios and into just like filling in a form, which systems deployers need to do.
So, we want to emphasize though that model providers bear a particular responsibility, because of their resources and technical expertise. So, basically this is a sneak peek of our report that we have out, just mapping that kind of responsibilities and yeah, if you're interested, this is basically the report that we have, it's a draft almost being published, but this basically goes through all that we've covered. And yeah, you can subscribe to our newsletter if you want to stay engaged with The Future Society and our research. Thank you.
