In a study examining the effectiveness of phishing attacks generated automatically by large language models (LLMs), researchers concluded that artificial intelligence is as efficient as human-personalized phishing. Compared to random, non-targeted phishing, they observed a 3.5 times higher efficiency.
The study, available on arxiv.org, was conducted by a research group primarily from a Harvard University institute. The phishing attacks were tested on 101 participants to determine how well LLMs can execute personalized phishing attacks, known as spear-phishing.
The results were surprising. The 101 participants were divided into four groups. The control group received random, non-personalized phishing. One group saw phishing emails generated by LLMs. Another group received phishing emails specifically created by humans, and there was a final group that received spear-phishing, where humans refined the AI-generated emails.
Researchers measured how often links in these emails were clicked. Random phishing had a click rate of 12 percent. Links in fully automated AI-generated phishing emails were clicked 54 percent of the time, as frequently as in targeted, human-created phishing emails. When humans further refined the AI phishing, a slight increase in the click rate to 56 percent was observed.
To create targeted phishing, the group developed an AI-supported tool that analyzes the digital footprint of targeted individuals and creates personalized emails based on that, subsequently evaluating the success of the fraudulent strategy. In 88 percent of cases, the AI was able to find useful information about the target individuals. Their tool supports several LLMs, but for their study, the group mainly focused on using Claude 3.5 Sonnet and GPT-4o.
Additionally, the researchers tested how well five LLMs—Claude 3.5 Sonnet, GPT-4o, Mistral, LLama 3.1, and Gemini—could recognize phishing emails. Ultimately, Claude 3.5 Sonnet and GPT-4o emerged as the most promising candidates. On a larger dataset of 363 phishing emails and 18 legitimate messages, Claude 3.5 Sonnet delivered a detection rate of 97.25 percent with no false positives. The researchers believe they could improve these results with prompt-tuning. AI can thus clearly be used not only for creating but also for filtering phishing.
AI-supported phishing is not just a theoretical research case. Last week, the British insurance company Beazley and other firms warned of increasing “hyper-personalized” phishing emails written with the help of AI generators.