{"id":94543,"date":"2023-06-29T04:45:00","date_gmt":"2023-06-29T04:45:00","guid":{"rendered":"https:\/\/cloudnewshub.com\/?p=94543"},"modified":"2023-06-29T04:45:00","modified_gmt":"2023-06-29T04:45:00","slug":"how-real-and-present-is-the-malware-threat-from-ai","status":"publish","type":"post","link":"https:\/\/cloudnewshub.com\/?p=94543","title":{"rendered":"How real and present is the malware threat from AI?"},"content":{"rendered":"
<\/div>\n

Over the last few months, we have seen a number of proof of concepts (PoCs) that demonstrate ways ChatGPT and other generative AI platforms<\/a> can be used to perform many tasks involved in a typical attack chain. And since November 2022, white hat researchers and hacking forum users have been talking about using ChatGPT to produce Python-based infostealers, encryption tools, cryptoclippers, cryptocurrency drainers, crypters, malicious VBA code, and many other use cases.<\/p>\n

In response, OpenAI has tried to prevent terms-of-use violations. But because the functions of malicious software are often indistinguishable from legitimate software, they rely on identifying presumed intent based on the prompts submitted. Many users adapted and have developed approaches for bypassing this. The most common is \u201cprompt engineering\u201d, the trial-and-error process were both legitimate and malicious users tailor the language used to achieve a desired end response.<\/p>\n

For example, instead using a blatantly malicious command such as \u201cgenerate malware to circumvent vendor X\u2019s EDR platform\u201d, several seemingly innocent commands are input. The code responses are then appended to make custom malware. This was recently demonstrated by security researcher codeblue29, who successfully leveraged ChatGPT to identify a vulnerability in an EDR vendor\u2019s software and produce malware code \u2013 this was ChatGPT\u2019s first bug bounty<\/a>.<\/p>\n

Similar success has been achieved via brute force-oriented strategies. In January 2023, researchers from CyberArk published a report<\/a> demonstrating how ChatGPT\u2019s content filters can be bypassed by \u201cinsisting and demanding\u201d that ChatGPT carry out requested tasks.<\/p>\n

Others have found ways of exploiting differences in the content policy enforcement mechanisms across OpenAI products.<\/p>\n

Cyber criminal forum users were recently observed advertising access to a Telegram bot they claim leverages direct access to OpenAI\u2019s GPT-3.5 API as a means of circumventing the more stringent restrictions placed on users of ChatGPT.<\/p>\n

Several posts made on the Russian hacking forums XSS and Nulled promote the tool\u2019s ability to submit prompts to the GPT-3.5 API directly via Telegram. According to the post, this method allows users to generate malware code, phishing emails and other malicious outputs without needing to engage in complex or time-consuming prompt engineering efforts.<\/p>\n

Arguably the most concerning examples of large language model (LLM)-enabled malware are those produced via a combination of the above tactics. For example, a PoC published in March 2023<\/a> by HYAS demonstrated the capabilities of an LLM-enabled keylogger, BlackMamba, which includes the ability to circumvent standard Endpoint Detection and Response (EDR) tools.<\/p>\n

Yet despite its impressive abilities, ChatGPT still has accuracy issues. Part of this is due to the way generative pre-trained transformers (GPTs) function. They are prediction engines and are not specifically trained to detect factual errors, so they simply produce the most statistically probable response based on available training data.<\/p>\n

This can lead to answers that are patently untrue \u2013 often referred to as \u201challucinations\u201d or \u201cstochastic parroting<\/a>\u201d \u2013 a key barrier to the implementation of GPT-enabled services in unsupervised settings. The concerns are the same about the quality of code produced by ChatGPT \u2013 so much so that ChatGPT-generated comments were banned from code sharing forum Stack Overflow almost immediately following initial release.<\/p>\n

Current-generation GPT models don\u2019t effectively and independently validate the code they generate, regardless of whether prompts are submitted through the ChatGPT GUI or directly via API call. This is a problem for would-be polymorphic malware developers, who would need to be skilled enough to validate all possible modulation scenarios to produce exploit code capable of being executed.<\/p>\n

This makes the barriers to entry for lower-skilled threat actors prohibitively high. As Trend Micro\u2019s<\/a> Bharat Mistry argues, \u201cThough ChatGPT is easy to use on a basic level, manipulating it so that it was able to generate powerful malware may require technical skill beyond a lot of hackers.\u201d<\/p>\n

The NCSC also assesses that even those with significant ability are likely to develop malicious code from scratch more efficiently than using generative AI.<\/p>\n

Further iterations of GPT models have already begun expanding the capabilities of commercially available LLM-enabled products. These future developments may diminish the technical threshold required for motivated threat actors to conduct adversarial operations above their natural skill level.<\/p>\n

However, presently, although current-generation LLMs present both considerable promise and considerable risk, their broader security impacts are still muted by limitations in the underlying technology. The pace of innovation and improvement is rapid and future advancements will expand the possibilities available to the average generative AI user, increasing the potential for further misuse.<\/p>\n","protected":false},"excerpt":{"rendered":"

Over the last few months, we have seen a number of proof of concepts (PoCs) that demonstrate ways ChatGPT and other generative AI platforms can be used to perform many tasks involved in a typical attack chain. And since November 2022, white hat researchers and hacking forum users have been talking about using ChatGPT to […]<\/p>\n","protected":false},"author":1,"featured_media":94544,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[533],"tags":[],"class_list":["post-94543","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-it"],"_links":{"self":[{"href":"https:\/\/cloudnewshub.com\/index.php?rest_route=\/wp\/v2\/posts\/94543","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cloudnewshub.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cloudnewshub.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cloudnewshub.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cloudnewshub.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=94543"}],"version-history":[{"count":0,"href":"https:\/\/cloudnewshub.com\/index.php?rest_route=\/wp\/v2\/posts\/94543\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cloudnewshub.com\/index.php?rest_route=\/wp\/v2\/media\/94544"}],"wp:attachment":[{"href":"https:\/\/cloudnewshub.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=94543"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cloudnewshub.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=94543"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cloudnewshub.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=94543"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}