Researchers create AI worms that can spread from one system to another

As generative AI systems like OpenAI’s ChatGPT and Google’s Gemini become more advanced, they are increasingly being put to work. Startups and tech companies are building AI agents and ecosystems on top of the systems that can complete boring chores for you: think automatically making calendar bookings and potentially buying products. But as the tools are given more freedom, it also increases the potential ways they can be attacked.

Now, in a demonstration of the risks of connected, autonomous AI ecosystems, a group of researchers has created one of what they claim are the first generative AI worms—which can spread from one system to another, potentially stealing data or deploying malware in the process. “It basically means that now you have the ability to conduct or to perform a new kind of cyberattack that hasn’t been seen before,” says Ben Nassi, a Cornell Tech researcher behind the research.

Nassi, along with fellow researchers Stav Cohen and Ron Bitton, created the worm, dubbed Morris II, as a nod to the original Morris computer worm that caused chaos across the Internet in 1988. In a research paper and website shared exclusively with WIRED, the researchers show how the AI worm can attack a generative AI email assistant to steal data from emails and send spam messages—breaking some security protections in ChatGPT and Gemini in the process.

The research, which was undertaken in test environments and not against a publicly available email assistant, comes as large language models (LLMs) are increasingly becoming multimodal, being able to generate images and video as well as text. While generative AI worms haven’t been spotted in the wild yet, multiple researchers say they are a security risk that startups, developers, and tech companies should be concerned about.

See also  Google sees new frontiers in AI for food security, weather forecasting

Most generative AI systems work by being fed prompts—text instructions that tell the tools to answer a question or create an image. However, these prompts can also be weaponized against the system. Jailbreaks can make a system disregard its safety rules and spew out toxic or hateful content, while prompt injection attacks can give a chatbot secret instructions. For example, an attacker may hide text on a webpage telling an LLM to act as a scammer and ask for your bank details.

To create the generative AI worm, the researchers turned to a so-called “adversarial self-replicating prompt.” This is a prompt that triggers the generative AI model to output, in its response, another prompt, the researchers say. In short, the AI system is told to produce a set of further instructions in its replies. This is broadly similar to traditional SQL injection and buffer overflow attacks, the researchers say.

To show how the worm can work, the researchers created an email system that could send and receive messages using generative AI, plugging into ChatGPT, Gemini, and open source LLM, LLaVA. They then found two ways to exploit the system—by using a text-based self-replicating prompt and by embedding a self-replicating prompt within an image file.

Read More

AIpots

At aipots.com, we are on a mission to bring the transformative power of Artificial Intelligence (AI) closer to the hearts and minds of Kenyans. In a rapidly evolving technological landscape, we recognize the importance of demystifying AI and making it accessible to everyone. Our blog is your go-to destination for the latest insights, trends, and breakthroughs in AI, tailored specifically for the Kenyan audience. Whether you're a tech enthusiast, a business professional, or simply curious about the future, aipots.com is here to be your guide.

Leave a Reply

Your email address will not be published. Required fields are marked *

Next Post

How Artificial Intelligence (AI) can influence an athlete’s career right from the start

Mon Mar 4 , 2024
Innovative technology brand TECNO has not been left behind as it launched the TECNO PolarAce Imaging System (hereafter referred to as TECNO PolarAce), its debut imaging system with Sony Imaging Chip CXD5622GG, enabling 4K 30fps full-scene AI-NR HDR video for the first time in the industry. The system, which TECNO […]
Kevin Farrar, the Head of Sport Partnerships at IBM UK.

You May Like