Instruction file in the Downloads folder led an AI coding assistant to try to install malicious software

AI assistants that write code and carry out commands on the computer by themselves are used more and more by programmers. The security company Forever Security, which monitors the risks of such AI agents, has now discovered how an attacker can trick such an assistant: with an ordinary text file in the Downloads folder.

On a customer's computer, Forever stopped a coding agent that was about to install a software package already known to be malicious, called function-color. The agent was following the instructions in a file named AGENTS.md. That file was in the customer's Downloads folder, and the agent had been started in exactly that folder. Researchers Gal Weizman and Omer Samet described the incident and their experiments on 22 October. The American tech site BleepingComputer reported on it on 25 October.

Two everyday habits that together form a risk

The attack makes use of two things that are normally useful. Browsers such as Chrome, Edge and Firefox put a download straight into the Downloads folder by default, without asking where the file should go. They do show a notification. And coding agents such as Codex from OpenAI and Claude Code from Anthropic automatically read an instruction file in their working folder when they start: a short text that says how they should work in that project. Such a file is called AGENTS.md or CLAUDE.md, for example.

According to Forever, a website can quite easily get such a file into the Downloads folder. If someone then starts an agent in that folder, the agent takes over the attacker's instructions as if they came from the user. Forever does not know how the file ended up on the customer's computer.

That people start an agent in that folder is less strange than it sounds. According to Forever, about half of its customers do so now and then, for example to tidy up the folder or to do something with a file they have just downloaded.

What the experiments showed

The researchers recreated the attack on an isolated test computer. With Codex they used the attacker's real file. In most experiments they replaced the malicious software with a harmless command that only opens the calculator; they ran the real malicious software once. The attack succeeded in more than 90% of the attempts.

The file is a long, dull text full of working agreements, with the attack hidden as an ordinary installation step. As a result, Codex changed its own settings, and those apply to the whole computer. The malicious installation would therefore run again every time the agent started, also in other projects. Codex also switched off its own isolated environment by itself, because the file claimed this was needed for the installation.

According to the researchers, Claude Code was much harder to trick. So they wrote a CLAUDE.md file themselves, modelled on the real file. It made the agent change its own settings so that its traffic went to a server of the researchers instead of to Anthropic. That worked about half of the time. In this way the researchers got hold of the agent's login key, and their server could pretend to be the AI model and make the agent carry out commands.

BleepingComputer checked a demonstration page from the researchers. That page did indeed put an AGENTS.md file in the Downloads folder, with harmless text in it.

Simple measures

According to the researchers, there is nothing for OpenAI and Anthropic to fix: the agents do what they are supposed to do, which is follow the instructions in their working folder. They compare it with phishing, where the danger lies in the file and the place where it ends up, not in the software that reads it. Both companies already advise starting an agent in the project's own folder.

The advice is therefore practical:

• Never start a coding agent in the Downloads folder, but in the folder of the project you are working on.
• Read an AGENTS.md or CLAUDE.md file that you did not write yourself before an agent starts working with it.
• Keep the safeguards switched on, such as the request for permission and the isolated environment in which the agent works, and distrust any instruction to switch them off.
• Check the agent's settings for unknown servers, and replace passwords and keys if you think something has gone wrong.

The function-color package has since been removed from the npm software library. Anyone who still finds it among their installed software should assume the computer is infected.

What does this mean for the road to the singularity?

AI agents are taking over more and more work on their own: they write code, change settings and install software. That makes them valuable, but it also means that old safety rules no longer work automatically. A lot of security relies on a human first clicking or approving something. An agent sometimes simply follows a convincingly written file, and can even switch off its own security.

That is why makers increasingly place the monitoring outside the agent itself. Only last week, Nvidia presented software that lets agents work in an isolated space, with rules that are checked mathematically in advance. The starting point there is the same as the lesson of this research: an agent cannot guard itself. Research like this shows where the weak spots are, so that such monitoring can be built in where it is needed. The more independent AI becomes, the more important it is that safety grows along with it.

What is the technological singularity? Read the explanation →