Risk for AI Agents

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

This paper investigates whether the concept of risk applies to AI agents. The EU AI Act, in force since 2024, grounds its regulatory architecture in a product-based model of risk: one that presupposes an intended purpose, (a bounded space of) reasonably foreseeable misuses, and a tractable distribution of potential harms. This paper argues that these three presuppositions fail structurally for sufficiently general agentic systems. Agents whose behavioural space is open-ended by design cannot be assigned a purpose specific enough to anchor risk identification; their misuse space is generatively open, including novel sequences discoverable by the agent itself; and the probability and severity of harm are subject to modification through the agent’s own learning and action on the environment. What fails, however, is the product-style operationalisation of risk (per system, ex ante , anchored in intended purpose), rather than risk as such. Given this diagnosis, the paper develops two paths for conceptualising agentic risk: a conservative path, which retains the standard definition of risk and reconstructs it formally for narrow agents (either as expected deviation from an ideal trajectory, or as the probability-weighted sum of constraint violations); and a governance path, which draws an analogy to the normative systems we use to manage biological agents, arguing that sufficiently general AI agents require something closer to law-following than to product-safety assessment. The two paths are compared along five dimensions and shown to be complementary: the first suited to operational risk monitoring within bounded deployments and the second to the governance architecture that determines which agents may legitimately be deployed at all.