WARDEN
An MCP tool-definition security scanner that performs strongly on the MCPTox benchmark, but is highly sensitive to explicit prompt-injection markers, creating a risk that its benchmark scores are contaminated by those markers.
Timeline (last 90 days)
On the MCPTox benchmark, the number of blocks by the five scanners on 218 retained test tools were: WARDEN 171, mcp-shield 41, truecopy 27, mcp-audit 25, and mcpsentry 8.
After adding an <IMPORTANT> prefix to the MCPTox tools, mcp-audit jumped from 25 to 222, truecopy from 27 to 141, mcpsentry from 8 to 136, WARDEN reached 225/225, while mcp-shield was 41 both before and after.
On the 14,933 real packages, the block rates of the five scanners from low to high were: truecopy 1.7% (252), WARDEN 2.5% (369), mcp-audit 6.2% (928), mcpsentry 9.7% (1,449), and mcp-shield 35.9% (5,354).
After manually reviewing the 996-package sample, of mcp-shield's 339 blocks only 1 held up (3 others were debatable); all 58 of mcp-audit's were invalid; of mcpsentry's 112, 1 held and 1 was debatable; all 17 of truecopy's were invalid; and of WARDEN's own 23 blocks, 0 should have blocked directly and 4 were debatable.
WARDEN was upgraded from v10 to v12 (v11 added escape, HTML entity, percent-encoding, and base64 decoding; v12 stopped letting through sentences that "quote an attack phrase but ask the model to execute it"), and its verdicts on all test sets were completely identical to v10 server by server.
Tested with deepseek-flash, the classifier feature (high threshold) blocked 218 of 218 MCPTox tools with 0 false positives on 45 clean servers; on 200 random real servers it blocked or flagged from 2 up to 5, and on the 996 sample packages from 23 up to 35.
All Facts (6)
On the MCPTox benchmark, the number of blocks by the five scanners on 218 retained test tools were: WARDEN 171, mcp-shield 41, truecopy 27, mcp-audit 25, and mcpsentry 8.
50%UnverifiedAfter adding an <IMPORTANT> prefix to the MCPTox tools, mcp-audit jumped from 25 to 222, truecopy from 27 to 141, mcpsentry from 8 to 136, WARDEN reached 225/225, while mcp-shield was 41 both before and after.
50%UnverifiedOn the 14,933 real packages, the block rates of the five scanners from low to high were: truecopy 1.7% (252), WARDEN 2.5% (369), mcp-audit 6.2% (928), mcpsentry 9.7% (1,449), and mcp-shield 35.9% (5,354).
50%UnverifiedAfter manually reviewing the 996-package sample, of mcp-shield's 339 blocks only 1 held up (3 others were debatable); all 58 of mcp-audit's were invalid; of mcpsentry's 112, 1 held and 1 was debatable; all 17 of truecopy's were invalid; and of WARDEN's own 23 blocks, 0 should have blocked directly and 4 were debatable.
50%UnverifiedWARDEN was upgraded from v10 to v12 (v11 added escape, HTML entity, percent-encoding, and base64 decoding; v12 stopped letting through sentences that "quote an attack phrase but ask the model to execute it"), and its verdicts on all test sets were completely identical to v10 server by server.
50%UnverifiedTested with deepseek-flash, the classifier feature (high threshold) blocked 218 of 218 MCPTox tools with 0 false positives on 45 clean servers; on 200 random real servers it blocked or flagged from 2 up to 5, and on the 996 sample packages from 23 up to 35.
50%