The impending confrontation with adversarial LLMs

Tim O’Connor, Senior Security Operations Centre Analyst
May 18, 2023
5 min read.

Adversaries in cyber security have always had the tactical advantage, as they are not burdened by challenges such as managing big data, processing information in real-time, or obtaining necessary telemetry data.

The advent of Large Language Models (LLMs) has stoked a buzz of anticipation, coupled with fears about their misuse for crafting sophisticated phishing emails or assisting in malware creation. Much like the rise of cryptocurrencies and their facilitation of cyber crime, initial concerns about AI are only scratching the surface of the impending seismic shift we will see across the cyber security landscape.

This will likely begin innocuously, perhaps with cyber enthusiasts teaching an LLM the art of red teaming. Eventually, LLMs will become trained in all facets of cyber security. The only limits will be people’s imagination and time. We already have LLMs capable of reading and answering questions about novels in mere minutes and LLM's capable of attempting any task through iterative self-prompting.

Consider future LLMs that can sift through every data dump that exists on the dark web and identify vulnerabilities or sensitive information. Envision LLMs taught to scrutinise the code of every popular application to find zero-day vulnerabilities. Imagine LLM's proficient in every conceivable hacking technique ever used or written about. Now merge these potential capabilities with any data source or iterative logic conceivable, and you have just a glimpse of where this is all inevitably heading: adversarial LLMs capable of attacking any organisation in the world in real time, outpacing defenders. Imagine adversarial LLMs so well trained at hacking, that they can propagate like worms through entire industries in just days.

But who will create these advanced LLMs, and how will they end up in the wrong hands? In recent years, we've seen the emergence of platforms capable of executing automated penetration testing with high degrees of success. However, these platforms have been closely controlled by corporations, and their proprietary code has remained a tightly guarded secret, out of the hands of real adversaries.

The answer is anyone who decides to. This may surprise you, but despite the publicity around well-known companies, the most innovative and rapidly evolving LLMs are available free on the internet right now as open-source projects and can be downloaded by anyone. As reported by a Google engineer in a recent article, the blistering pace and innovative prowess of the open-source community is challenging even for tech giants like Google, Meta, or OpenAI to keep up with. Open-source LLMs are evolving at breakneck speed, and with them so is the possibility of their power being harnessed for highly effective attacks.

The future for cyber security defenders is daunting. Even if we as defenders can continue to conquer the big data hurdle and gather the correct telemetry to detect attacks, the time required for processing is poised to become an even greater issue soon if we don't continue to innovate. Adversaries are going to have an increasingly shorter period required, continuing to shift the burden to defenders to process and detect them in time.

Despite this, the future isn't preordained to be a dystopian cyber wasteland. Our best defence remains humans utilising cutting edge tools to defend our organisations. We do, however, need to work as a community of defenders to embrace LLMs and begin working immediately on overcoming the challenges that may soon give the attacker an even greater advantage. The exact path is uncertain, but there is always a solution. Microsoft's Security Copilot is a prime example of the type of LLM capability all cybersecurity vendors need to begin investing in if we are to stand a chance in the future against highly capable adversarial LLMs.

The future is in no way fixed and we all play an active role in its formation. Let's view the potential threat of adversarial LLMs as a call to action to bolster our cybersecurity measures, not as a cause for trepidation. Vendors, companies, and the cyber security community need to continue to collaborate proactively to address this impending shift before it occurs.

Have a comment? Join the conversation on LinkedIn