The Rise of the Internet of Agents: A New Era of Cybersecurity
Authored by David Haber, Mateo Rojas-Carulla, and Matthias Kraft, co-founders of Lakera.ai.
The post was originally published here: https://www.lakera.ai/blog/the-rise-of-the-internet-agents-a-new-era-of-cybersecurity
As AI-powered agents go online, securing our digital infrastructure will require a fundamental shift in cybersecurity.
For the past few decades, AI has been mainly limited to data-driven technologies used to solve narrow tasks (e.g. classifying images into a predetermined set of categories). The emergence of Large Language Models (LLMs), and in particular the introduction of ChatGPT, brought about a step change in capabilities that massively accelerated AI adoption.
Today, more than ever before, AI applications are augmenting human output and productivity, and their projected impact has been compared to the harnessing of electricity. According to Gartner, 80% of organizations will have deployed GenAI technology in production by 2026.
Human-to-machine applications like ChatGPT are just the beginning. At Lakera, we believe that independently of whether we achieve Artificial General Intelligence (AGI), we will see the emergence of the Internet of Agents (IoA), a deeply integrated network of AI-to-AI applications. In the IoA, humans will gradually shift from their roles as “reviewers” and instead take on a supervisory role to ensure that agents in the network perform as expected.
AI agents will interact directly with each other to generate creative and productive outputs and execute tasks independently of humans. Over time, the IoA will become essential to many aspects of our lives, with the potential to lead to greater prosperity and human flourishing.
We can see the power and potential of the IoA by extending some of the most common practical applications of AI today:
- Human Assistants: Today, we might ask an AI to write an email to help us negotiate a lower cable bill. We would then review the email and send it to our cable company. If another round of negotiations was needed, we could send the response back to the AI and ask it to compose another email. In the future, we will trust the same AI to handle this negotiation entirely on its own, going back and forth with the company several times without human intervention. The company, in turn, is likely to move away from human customer service agents to its own AI-driven agents as a cost-saving measure, meaning that each of our AIs will negotiate directly with each other with little human oversight.
- Coding: Co-pilots have been rapidly adopted by software engineers to write large amounts of boilerplate code, increasing their productivity. As AI agents become more advanced, it is easy to imagine software writing agents executing larger and larger tasks entirely on their own, guided only by strategic directives from humans. Other agents, interacting with the outside world, could then be responsible for testing with early users and providing feedback to the software agents. We can also see many functions such as sales and marketing being taken over by agents, eventually leading us to companies that are almost entirely run by AI.
Just as the move to the internet (and later the cloud) introduced entirely new risks and gave rise to today’s cybersecurity, the IoA will once again transform the risk landscape in a more profound way.
AI will not simply add another element to existing cybersecurity tenets, it will transform them all. Today, it would be negligent for businesses to operate on the internet without cybersecurity; the risk would simply be too high.
Similar transformations in cybersecurity are needed to ensure that GenAI can be deployed at scale and that risks are managed appropriately. To ensure that society can reap the benefits of the IoA, we need to build a secure foundation for the IoA that empowers humans and mitigates risks introduced by a deeply interconnected network of agents.
What is different in the AI-driven cybersecurity landscape?
A skeptic might argue that we already deploy intelligent systems with wide agency and reach today, such as algorithmic trading agents, without a fundamental shift in cybersecurity. Two core features introduced by LLMs explain why things are now profoundly different.
- Universal interface: LLMs can now analyze and interpret any data thrown at them, most notably natural language. This ease of access explains why hundreds of millions of people have started using AI agents in just a few months. As a result of this versatility, the attack surface of such a model is not limited to an experienced hacker. Hacking has been democratized to anyone who can read and write, leading to a step change in the accessibility of cybersecurity attacks. You can try out Gandalf to get a feel for it yourself.
- Universal capabilities: AI agents can now perform new tasks out of the box, even when intended for narrow use cases (for example, helping users read their email). An AI agent can do anything from writing text to communicating with external APIs to executing code. It is able to generate multimodal data, from natural language to audio and video. An attacker or a simple malfunction can therefore lead to unpredictable behavior with unlimited consequences. Already today, as a result, AI assistants intended to summarize emails in your inbox can end up leaking your entire inbox to an attacker, and all they need to do is send you an email. Counter to traditional phishing attacks, you don’t even need to see it or open it. In this example, the ability to send an email is a capability that was not known to the developer of this agent and was maliciously exploited by the attacker.
As a result of these changes, AI agents of increasing intelligence will be vulnerable to manipulation and persuasion, with attackers able to exploit the flexibility and power of these applications at will.
This will be exacerbated as the data on the internet itself becomes part of the battlefield: attackers will focus on manipulating data that AI models are trained on in order to later influence and hijack the ensuing agents. It may be difficult to find evidence that this type of manipulation has ever occurred.
The IoA compounds security challenges
With the step change in capabilities and the deep interconnectedness of the IoA, there is an urgent need to ensure that this AI-powered ecosystem is safe and secure.
The interconnectedness of the network magnifies the impact of failure to unprecedented levels. Social networks have shown how misinformation can spread across deeply interconnected networks, threatening institutions and trust around the world. Similar properties inherent to networks will exacerbate the challenges of single-agent systems:
- Exploits affecting our foundation models will be inherited by the large number of downstream applications.
- Vulnerabilities will propagate through the network, allowing attackers to exploit and manipulate agents everywhere at scale.
Given the speed at which the IoA will operate, a compromised node in the agent network can very quickly propagate at scale. Today’s security challenges in human-to-machine applications will therefore dramatically expand as a result of extreme interconnectedness.
A new security paradigm needs to begin today
While risks like rogue AIs acting maliciously of their own volition are highly publicized and discussed at length in the public discourse, the increased magnitude of risks posed by the future of interconnected agents is discussed much less frequently, despite being much more likely to cause catastrophic damage.
As we race towards the interconnected future that will be the Internet of Agents, we must race just as quickly to build a new set of tools and infrastructure that will allow us to operate the IoA in a way that is just as secure as it is productive.
Building systems with containment, traceability, visibility, accountability, and actionability will be critical. Doing this requires a new way of thinking about cybersecurity design. Only by building with security and safety in mind from the ground up can we truly reap the benefits of the IoA and enter into a new era of human flourishing.
That building needs to start today.
