In the arsenal of cybersecurity defenses is the exercise that goes by the name of red team/blue team simulated attack. These simulations are designed to closely mimic real-world conditions. For example, one red team member might take on the role of an employee clicking on a phishing link that deposits malware on the network. The defending team members must then find this malware before it spreads across their network and infects web servers and other applications. To make things more realistic, the simulation replays real network traffic to obscure the attacks, just like in the real world.
Let’s talk about the red and blue designations. Red team members usually play the role of attackers and try to overcome security protocols. They use the same tools and techniques that attackers use, similar to how penetration testers operate but on a much broader scale.
“Red teams don’t just test for vulnerabilities, but do so using the tools, tips and techniques of their likely threat actors, and in campaigns that run continuously for an extended period of time,” wrote Daniel Miessler, a security consultant who has witnessed numerous red/blue exercises, in a blog post. “A great red team can be an early warning system to find common origins of attacks and to track an adversary’s techniques.”
John, a retired IBM architect who has worked in large IT shops, tells CSO that “threats are going to emerge that red teams will never test for. There are threats that can overwhelm blue teams and possibly put companies out of business.”
According to Cris Thomas, global lead of strategy for IBM X-Force Red consulting organization, “Some companies just think about red teams in terms of a physical security break-in.”
The blue team is composed of the defenders, modeled after internal security teams that are now found in numerous IT shops. “What makes for a great blue team is their mental state, having a proactive mindset, endless curiosity and continuous improvement in terms of detection and response,” wrote Miessler.
The red/blue dichotomy is somewhat misleading. To really conduct one of these simulations, two more teams should be involved:
- A white team is composed of the network owners, the IT administrators who run the equipment and create the scripts for performing the simulations. Some exercises come with pre-built scripts while others build their own.
- A gold team has the subject matter experts who are consultants to the exercises and could involve security vendor representatives, legal advisors and perform specialized tasks such as digital forensics.
A note about purple teams
Let’s also talk about the color purple. This carries several different meanings, depending on how this team is constructed. The color gives you the idea that this is a combination of both red and blue teams, so that both can collaborate and improve their skills. This combination could mean that there are representatives from both sides working together on the exercise, or even as part of their jobs.
That may not be as effective as having the same people having both mindsets. Miessler likened this to waiters who don’t deliver food at restaurants because it isn’t their job. He has seen organizations where the red team thinks itself too elite to share information with the blue team, or they aren’t designed to interact with each other, or that IT doesn’t see both teams as part of the same effort.
Last summer, I attended the annual National Guard CyberShield event in Utah, where over the course of two weeks it conducts a simulated attack that is coordinated across 40 local Guard units. The units are split into red and blue teams with more than 800 members spread around the country. The Guard purposely schedules a “purple day” where both red and blue teams mingle with each other and collaborate to share tips and techniques.
“We know that the threat actors are collaborating way better than we are, and this gives us a chance for us to work closely with our partners and in realistic scenarios and build trust and deeper relationships,” says Lt. Col. Brad Rhodes, the officer in charge of the event. This building of trust is important because you want teams to learn from each other, rather than depend on a single analyst who may or may not be on duty or leave the Guard when an actual threat occurs. Rhodes has led six CyberShield exercises and works full time as head of IT security for Zvelo.
Walmart has both full-time internal blue and red team members. “We also periodically bring in outside blue and red team information security professionals to consult and we are starting to use a purple team approach to share our experiences. The two meet several times per month to help drive constant improvement for both teams, and we have seen fantastic collaboration between the two as they recognize they can drive more value to the organization,” says Jason O’Dell, vice president of security operations at Walmart.
Steps to design red team/blue team exercises
Here are some things to consider when designing your own exercise:
Decide what you will do in-house and what you will hire out. Do you need a specialized red team vendor? Do you already have full-time infosec staff that can act as a blue team? Can you use a pre-built cyber range that has everything set up a certain way?
Part of this decision is understanding the required skill sets for all team members. “A crucial skill for both teams is the desire to learn and be continuously curious,” says Walmart’s O’Dell.
Retired architect John agrees: “The ability to act quickly and effectively to any vulnerability is an absolute requirement these days.” He has never seen a company with a true red team. “Most of the time, this is outsourced to a consulting firm. Doing it in-house is hard, because of the difficulty in finding people with the exceptional skill levels needed for the job and then retaining them. If the red team is really effective, I can see them having a hard time growing their careers in the firm.”
Pick your simulation tools. Another way to phrase this is to decide on how realistic you want your exercise to be. Most of the time, these exercises won’t be done against production systems, so figure out what you will simulate or if you will use a cyber range (and usually not an exact replica of your running systems).
For the Guard’s CyberShield, they used the Persistent Cyber Training Exercise (PCTE) cloud-based simulation environment that was developed for the Defense Department. The CyberShield event is the largest operation conducted across this network, consuming more than 3,000 virtual machines and a petabyte of storage.
Formulate your goals. What are you trying to accomplish? Find weak spots? Shore up your defenses? Improve IT/end user collaboration? Identify working and failed security controls? The goal of these exercises is that more realism the better prepared everyone can be for the real attack, which gets back to the previous issue.
At the 2020 CyberShield, the red team built a piece of malware that eventually was posted on VirusTotal, according to one Guard participant I interviewed. “It was real enough when it then got picked up by Russian hackers which used it in the wild. Fortunately, its creator had placed a kill switch to neutralize it.”
Goals are critical, as Peter Kaloroumakis of MITRE, told me. “We see cases where red teams are able to successfully achieve their technical objectives but miss opportunities to have broader impact. Red or purple teams discover new information. It is essential they also engage infrastructure and architecture teams who develop strategic plans to improve security posture. It is easy to focus on specific configuration changes, but sometimes there are architectural changes which might address root cause issues.”
Decide how you will collect the data from the exercise and how you will conduct your post-mortem analysis. A big part of that is reporting on the level of communication amongst your teams. Architect John says, “The biggest problem I’ve seen here is language/communications and poor teamwork. In the era of outsourcing teams can be from different locations, speak different languages and so forth. If people cannot understand each other, that is a big problem during and after the exercise.”
Pick your time frame. The timing of your exercise varies tremendously. IBM’s Thomas says, “Some companies buy a subscription service from IBM and do constant retesting of a mobile app as they are developing it, through either nightly builds or a regular milestone.”
The Guard needs two weeks every year because it is also conducting training exercises, so that participants can take COMPTIA and other certification classes in addition to running the CyberShield simulations. “We conduct multiple tabletop and threat simulation exercises each year. In addition, our Red Team runs numerous full adversarial engagements every year. Sometimes these engagements will blend together,” says Walmart’s O’Dell. The ideal situation is to continuously probe your systems, but certainly stick to a schedule and just don’t react to a failed security audit.
Designing the most effective red/blue exercise means being clear on a lot of non-technical points, as you can see. Make sure you pay equal attention to both the technical and non-technical issues.
Copyright © 2022 IDG Communications, Inc.
