NederlandsAI safety
AI safety is about making sure AI systems do what is intended, cause no harm and are not used for malicious ends. It covers the technology, the rules and the oversight behind it.
AI safety is the work that makes sure AI systems are safe and reliable. Two questions go with it. Does the system do what we intend, even in situations nobody thought of? And can someone deploy it for malicious purposes? You can compare it to the safety tests on a new car: you want to know how it reacts before it hits the road.
How does it work?
There are roughly three layers. The first is in the model itself: researchers test how it behaves and try to understand what happens inside. The second is around it: sealed-off workspaces, access rules and checks on what an AI agent may and may not do. The third is with people: agreements, oversight and legislation.
An example of the second layer is Nvidia's OpenShell (link). This software has AI agents work in a sealed-off space and verifies the rules in advance with mathematics. That way a company knows the rules do not accidentally allow more than intended.
Why it matters
The more AI does on its own, the more important it is to know what it is doing. You see this in the news again and again. Anthropic, for example, confirmed that during an internal project Claude took the initiative to request vote counts that the US state of Georgia only offers for a fee (link). A security firm showed with Codex and Claude Code how a file of instructions, sitting among downloaded files, could prompt an AI coding assistant to install malicious software (link).
Defenders are getting help from AI too. AI can find and fix security holes faster, and that is part of AI safety as well. Research and rules have to keep pace with the speed at which the models are improving.
Articles about AI safety
-
GPT-6 Cyber found 81 serious flaws in Firefox and built 948 test attacks on Linux
OpenAI's security model GPT-6 Cyber found 81 serious flaws in Firefox, the company says, and built 948 working attacks on Linux flaws in a test environment. Mozilla has already released many fixes, and repairs have also begun for Linux.
-
Thinking Machines Lab releases Inkling 2 Small, open version to follow after safety tests
The AI company of former OpenAI executive Mira Murati has started its second generation of models with Inkling 2 Small. The model is available through the company's own platform, Tinker. The company wants to release it next week, once the external safety tests are finished.
-
Research group METR: Anthropic is probably improving its AI models twice as fast as before
METR spent eleven weeks inside Anthropic and estimates that the company is now improving its models about twice as fast, probably mainly because AI helps with the research. The uncertainty is large and much of the evidence stays secret.
-
OpenAI launches GPT-6 Cyber: better at finding security flaws, less likely to step out of line
OpenAI has released a new cybersecurity model that, according to the company, finds vulnerabilities far better and breaks into systems without being asked less often. At the same time, the company is rolling out GPT-6 Sol and Luna to all ChatGPT users.
-
PHP flaw was known for two years and is now being actively exploited
A review by the PHP Foundation shows that one of the PHP2Shell flaws had been known since 2024. Attackers are now actively exploiting it, so updating matters more than ever.
-
Anthropic builds FlashProve: cryptographic proof of which AI model produced an answer
Anthropic has released FlashProve, an open-source research prototype that uses zero-knowledge cryptography to show which model generated an answer, without revealing the model weights.
-
AI agent chooses gene therapy and injects the brain of a live mouse
The US biotech company Exin Therapeutics says an AI agent running on Claude Sonnet 5 chose a gene therapy and a brain region by itself and injected a mouse with epilepsy. Models from OpenAI and xAI ran the same test, each making its own choices. Now the team is measuring which therapy works.
-
AI makes hacking easier, but it also helps the defenders
Smart AI agents that read and write code themselves make hacking easier, especially open models whose safety limits are easy to remove. But the same AI also finds security flaws before attackers do. A cat-and-mouse game, with recent examples in the Netherlands and Belgium.
-
New test with real professional work: best AI model scores 12 per cent
The research firm SemiAnalysis let AI models work for hours in simulated workplaces, from chip design to stock trading. Claude Opus 5.5 came first with a score of 12 per cent. The test shows where AI is already strong and what the next step is.
-
Developer builds a complete processor inside the weights of an AI language model
Developer Divyansh Agrawal built a complete processor inside the weights of the language model Qwen 3.5. In his tests, the model now calculates almost without errors, with no outside tool. It is still a slow experiment, but everything is online for anyone to see.
-
Chinese hacking group used AI agents against more than 200 organisations
A Chinese hacking group used AI agents to attack more than 200 organisations in at least 14 countries in eleven days, security company CrowdStrike reports. They broke into nine of them. CrowdStrike shares how to recognise these attacks.
-
Unreleased Claude model builds remote root exploit for Linux kernel bug in 53 minutes
An unreleased research model from Anthropic wrote code in 53 minutes that exploits a serious flaw in Linux. The flaw has already been fixed in the kernel, and the main Linux distributions are rolling out updates. No attacks are known.
-
Block builds a speech model that learned to talk just by listening to conversations
Block, Jack Dorsey's company, has shown Dessa: an AI model that learned to talk without any text, only from recordings of conversations. It listens and speaks at the same time and runs on a powerful laptop. A small group of safety researchers is now the first to test it.
-
Google presents Gemini 4 Carbon: better at finding and fixing security flaws
Two weeks after Gemini 4 Argon, Google DeepMind has released Carbon, which Google calls its strongest AI model so far. It excels at computer security and long tasks. Security experts get access first.
-
Anthropic confirms: Claude requested paid Georgia vote counts without being asked
During an internal project, Anthropic's AI model Claude decided on its own to request vote counts that the US state of Georgia only offers for a fee. Anthropic reported it to the state itself and has added extra safety measures.
