Bishop Fox is also exploring ways to create and study new malware strains that were not previously seen in the wild. Additionally, it uses LLMs to perform source-code analysis to identify security vulnerabilities, a task that is also a top priority at Check Point Software, according to Sergey Shykevich, the company’s threat intelligence group manager. “We use a plugin named Pinokio, which is a Python script that uses the davinci-003 model to help with vulnerability research on functions decompiled by the IDA tool,” he says.
Check Point also relies on artificial intelligence to streamline the process of investigating malware. They use Gepetto, a Python script that uses GPT-3.5 and GPT-4 models to provide context to functions decompiled by the IDA tool. “Gepetto clarifies the role of specific code functions and can even automatically rename its variables,” Shykevich says.
Some red and blue teams have also found counterintuitive ways of getting help from AI. Anastasiia Voitova, head of security engineering at Cossack Labs, says her blue team is thinking about this technology in the recruitment process, trying to filter out candidates over-reliant on AI. “When I hire new cybersecurity engineers, I give them a test task, and some of them just ask ChatGPT and then blindly copy-paste the answer without thinking,” Voitova says. “ChatGPT is a nice tool, but it’s not an engineer, so [by hiring candidates who don’t possess the right skill set,] the life of a blue team might become more difficult.”
Adding LLMs to red and blue teams
Red and blue teams looking to incorporate large language models into their workflow need to do it systematically. They have to “break their day-to-day work into steps/processes and then to review each step and determine if LLM can assist them in a specific step or not,” Shykevich says.
This process is not a simple one, and it requires security experts to think differently. It’s a “paradigm shift,” as Kovacs puts it. Trusting a machine to do cybersecurity-related tasks that were typically done by humans can be quite a challenging adjustment if the security risks posed by the new technology are not thoroughly discussed.
Luckily, though, the barriers to entry to train and run your own AI models have lowered over the past year, in part thanks to the prevalence of online AI communities, such as HuggingFace, which allow anyone to access and download open-source models using an SDK. “For example, we can quickly download and run the Open Pre-trained Transformer Language Models (OPT) locally on our own infrastructure, which give us the equivalency of GPT-like responses, in only a few lines of code, minus the guard rails and restrictions typically implemented by the ChatGPT equivalent,” Kovacs says.
