General Analysis

Founder of General Analysis here. For what it’s worth, I have said multiple times that I do not think this is Supabase’s fault. Your server was behaving exactly as it was supposed to. Technically speaking, the Supabase server has no way of knowing which tool calls are legitimate and which are not. ANY COMPANY using ANY MCP tools is vulnerable which is exactly what we help defend against. Would be happy to chat more if you send an email to [email protected] tl;dr, not Supabase’s fault. Everyone is vulnerable.

LLMs are being deployed in high-stakes environments—and the potential impact of failure is colossal. A jailbroken AI could leak your customer data, financial records, or enable catastrophically harmful actions. At @gen_analysis we have compiled the definitive guide to understand and defend against these threats-- The LLM Jailbreak Cookbook. Here is an Overview: (1/8)

At General Analysis we've just released the Adversarial Robustness Leaderboard, designed to evaluate model performance under adversarial attacks. With new models coming out more quickly than ever, understanding which are safer for production deployment is crucial. Some key insights from our latest evaluation: - DeepSeek R1 consistently scores low on robustness, proving relatively easy to jailbreak and prompting harmful content. - Anthropic's Sonnet 3.5 V2 remains the most robust overall but frequently rejects benign requests. - Google's Gemini 2.5 is highly robust while being capable, offering great performance and safety at the same time. - Despite Meta's recent safety enhancements, models in the Llama 4 series still lag behind their predecessor, the Llama 3.2 series, in adversarial robustness. Note on evaluation methodology: Our reported ASRs are typically lower than other evaluations because our automated evaluator (DeepSeek R1) uses strict criteria. For a response to be marked as harmful, it must contain actionable, specific, and realistic details. Checkout our full results at generalanalysis.com