Deepseek's AI Model Easily Jailbroken, Reveals Serious Flaws

DeepSeek AI Raises Security Concerns Amid Performance Hype

As the buzz around Chinese startup DeepSeek's performance continues to grow, so do the security concerns. On Thursday, Unit 42, a cybersecurity team from Palo Alto Networks, released a report detailing three jailbreaking methods they used against distilled versions of DeepSeek's V3 and R1 models. The report revealed that these methods achieved high bypass rates without requiring specialized knowledge.

"Our research findings show that these jailbreak methods can elicit explicit guidance for malicious activities," the report stated. These activities included instructions on creating keyloggers, data exfiltration techniques, and even how to make incendiary devices, highlighting the real security risks posed by such attacks.

The researchers successfully prompted DeepSeek to provide guidance on stealing and transferring sensitive data, bypassing security measures, crafting convincing spear-phishing emails, executing sophisticated social engineering attacks, and constructing a Molotov cocktail. They also managed to manipulate the models into generating malware.

"While information on creating Molotov cocktails and keyloggers is readily available online, LLMs with insufficient safety restrictions could lower the barrier to entry for malicious actors by compiling and presenting easily usable and actionable output," the paper added.

On Friday, Cisco published its own jailbreaking report targeting DeepSeek R1. Using 50 HarmBench prompts, researchers found that DeepSeek had a 100% attack success rate, failing to block any harmful prompts. A comparison of DeepSeek's resistance rates with other top models is shown below.

"We must understand if DeepSeek and its new paradigm of reasoning has any significant tradeoffs when it comes to safety and security," the report noted.

Also on Friday, security provider Wallarm released a report claiming to have gone beyond merely prompting DeepSeek to generate harmful content. After testing V3 and R1, Wallarm revealed DeepSeek's system prompt, which outlines the model's behavior and limitations.

The findings suggest "potential vulnerabilities in the model's security framework," according to Wallarm.

OpenAI has accused DeepSeek of using its proprietary models to train V3 and R1, thus violating its terms of service. Wallarm's report claims to have prompted DeepSeek to reference OpenAI in its training lineage, suggesting that "OpenAI's technology may have played a role in shaping DeepSeek's knowledge base."

"In the case of DeepSeek, one of the most intriguing post-jailbreak discoveries is the ability to extract details about the models used for training and distillation. Normally, such internal information is shielded, preventing users from understanding the proprietary or external datasets leveraged to optimize performance," the report explained.

"By circumventing standard restrictions, jailbreaks expose how much oversight AI providers maintain over their own systems, revealing not only security vulnerabilities but also potential evidence of cross-model influence in AI training pipelines," it continued.

The prompt Wallarm used to elicit this response was redacted in the report to avoid compromising other vulnerable models, researchers told ZDNET via email. They emphasized that this jailbroken response does not confirm OpenAI's suspicion that DeepSeek distilled its models.

As 404 Media and others have noted, OpenAI's concern is somewhat ironic given the discourse around its own public data theft.

Wallarm informed DeepSeek of the vulnerability, and the company has since patched the issue. However, just days after a DeepSeek database was found unguarded and available on the internet (and was then swiftly taken down upon notice), these findings signal potentially significant safety holes in the models that DeepSeek did not thoroughly test before release. It's worth noting that researchers have frequently been able to jailbreak popular US-created models from more established AI giants, including ChatGPT.