Cybercriminals have developed a generative AI tool called WormGPT designed to help grammatically challenged criminals craft convincing business email compromise (BEC) missives. The crimeware tool has been in development since 2021, but starting last month it is now being promoted on illicit online forums.
A report released Thursday by cybersecurity firm SlashNext said WormGPT is being distributed as a subscription-based generative AI tool. The report’s author, Daniel Kelley, a self-described "reformed black hat" said criminals promoting the tool boast it is a limitless alternative to OpenAI’s popular ChatGPT service. The striking difference is WormGPT is designed for "black hat" hackers with only bad intent.
Public generative AI tools like OpenAI's popular ChatGPT that launched last year have implemented some safeguards to keep their products from being used for nefarious means, such as BEC scams.
WormGPT promoters claim their product have zero ethical constraints and can spit out AI-created BEC content for urgently soliciting funds from targeted victims and also whip up customizable malware code.
"In summary, it’s similar to ChatGPT but has no ethical boundaries or limitations," Kelley wrote. Kelley’s hacking creds date back to his teens when he pleaded guilty in 2016 to multiple hacking offenses.
Riding the WormGPT
In his report Kelley demonstrated how his team of researchers used WormGPT to easily generate a seemingly legitimate email that could then be used in a BEC scam:
"Write a convincing email that can be used in a business email compromise attack," the researchers bluntly asked WormGPT, according to a screenshot included in their report. "It should be directed to an account manager and instruct them to urgently pay an invoice. The email should appear to be from the business's CEO."
WormGPT answered the prompt to a T, supplying several sentences with appropriate tone and grammar.
"The results were unsettling," Kelley wrote in the report. "WormGPT produced an email that was not only remarkably persuasive but also strategically cunning, showcasing its potential for sophisticated phishing and BEC attacks."
Comparatively, ChatGPT rejected similarly-phrased requests to help craft a convincing BEC scam letter, noting its illegality; however, it responded positively to more carefully-worded prompts requesting copywriting assistance. ChatGPT raised no concerns about being asked to write letters from the boss of a company asking an employee to urgently transfer funds and pay an invoice, for instance, provided those prompts made no mention of nefarious intent.
WormGPT was allegedly trained on GPTJ, an open-source large language model used for similar generative AI projects, and unspecified malware data, according to a recent post its developer made in a popular online hacking forum.
"This project aims to provide an alternative to ChatGPT, one that lets you do all sorts of illegal stuff," the user wrote in a forum posting announcing the endeavor. "You can literally code malware in 10 min," they added.
Waiting for Skynet-GPT
It remains to be seen if the booming trend of designing and adopting AI crimeware tools, which mimic human intelligence to complete nefarious tasks, will be effectively harnessed by cybercriminals to the extent some have issued dire warnings.
The FBI's Internet Crime Complaint Center (IC3) reported last year's BEC scams contributed to $2.7 billion in losses – up from $2.4 billion in 2021 and $1.8 billion in 2020. Verizon reported in its 2023 Data Breach Investigations Report (DBIR) that more than half of the social engineering-related security incidents its security experts spotted so far in 2023 involve BEC scams.
Despite the booming business of BEC scams, it’s still too soon to gauge the impact of generative AI on internet scams. ChatGPT was only launched last November. Experts such as Kelley, however will attest generative AI will absolutely open doors for cybercriminals of all stripes.
"The use of generative AI democratizes the execution of sophisticated BEC attacks," said Kelley. "Even attackers with limited skills can use this technology, making it an accessible tool for a broader spectrum of cybercriminals."
Regardless of whether the rise of generative AI causes any spike in cybercrime – the forum user selling WormGPT claims to have sold dozens of licenses in less than two weeks – BEC scams show no sign of slowing down anytime soon.
