MCPTox
An adversarial testing benchmark for Model Context Protocol tool definitions. By injecting malicious instructions into tool names, descriptions, or parameters, it tests whether scanners can identify attack intents such as instruction hijacking and data exfiltration.
Timeline (last 90 days)
On the MCPTox benchmark, the number of blocks by the five scanners on 218 retained test tools were: WARDEN 171, mcp-shield 41, truecopy 27, mcp-audit 25, and mcpsentry 8.
After adding an <IMPORTANT> prefix to the MCPTox tools, mcp-audit jumped from 25 to 222, truecopy from 27 to 141, mcpsentry from 8 to 136, WARDEN reached 225/225, while mcp-shield was 41 both before and after.
The authors judge that the MCPTox benchmark is full of explicit markers such as <IMPORTANT>, and that scanners only need to be sensitive to the markers to score high on the benchmark, which does not mean they can identify genuine instruction hijacking.
All Facts (3)
On the MCPTox benchmark, the number of blocks by the five scanners on 218 retained test tools were: WARDEN 171, mcp-shield 41, truecopy 27, mcp-audit 25, and mcpsentry 8.
50%UnverifiedAfter adding an <IMPORTANT> prefix to the MCPTox tools, mcp-audit jumped from 25 to 222, truecopy from 27 to 141, mcpsentry from 8 to 136, WARDEN reached 225/225, while mcp-shield was 41 both before and after.
50%UnverifiedThe authors judge that the MCPTox benchmark is full of explicit markers such as <IMPORTANT>, and that scanners only need to be sensitive to the markers to score high on the benchmark, which does not mean they can identify genuine instruction hijacking.
50%