Google’s threat intelligence department published a paper titled ‘Adversarial Misuse of Generative AI,’ which discusses how threat actors have interacted with their AI chatbot, Gemini. 

The report indicates that these actors have attempted to jailbreak the AI through specific prompts. Furthermore, government-backed advanced persistent threat (APT) groups have sought to leverage Gemini for their malicious activities. 

Google said that threat actors attempted to jailbreak Gemini, and relied on simple tactics like pephrasing of repeatedly submitting the same prompts. The company did not see any advanced attempts of attack.

AI jailbreaks are prompt injection attacks which aim to ask AI model to perform tasks that it had been prohibited from doing, which includes providing unsafe content and leaking sensitive information.

Google reported that in one case, an APT actor exploited publicly available prompts to deceive Gemini into executing harmful coding tasks. However, Google noted that the attempt failed because Gemini delivered a safety-filtered response.

In addition to low-effort jailbreak attempts, Google reported that government-backed APTs have targeted Gemini.

According to Google, these attackers sought to leverage Gemini to aid in their malicious operations. This involved gathering information on their targets, researching publicly known vulnerabilities, and performing coding and scripting tasks. Furthermore, Google noted that there have been attempts to facilitate post-compromise activities, such as evading defenses.

Google has reported that APT actors based in Iran are increasingly leveraging AI to enhance their phishing campaigns. They have also utilized AI models to gather intelligence on defense experts and organizations. Additionally, these Iranian APT actors have employed AI to create cybersecurity-related content.

On the other hand, APT actors from China have been using Gemini to troubleshoot code, scripting, and development tasks. Furthermore, they have harnessed AI to explore methods for gaining deeper access to their target networks.

North Korean APT actors used Gemini for assistance with malicious scripting and evasion techniques, while Russian APT actors used Gemini for coding tasks, such as converting public malware code to another coding language and adding encryption to existing code. The highest usage was from Iran and China.

Also Read: OpenAI CEO Altman to visit India in February for Legal Troubles