How to run successful Kubernetes attack simulations?

Learn how to run successful Kubernetes attack simulations with this straightforward guide.

Why run Kubernetes attack simulations?

Imagine a newly created Kubernetes cluster, exposed on the Internet. According to a 20 23 threat report, it takes only 22 minutes before it starts receiving attacks. This statistic emphasizes that it’s not a matter of “if” but of “when”, someone will target your cluster.

Attack simulation exercises allow organizations to take a proactive stance by simulating real-world threats to identify vulnerabilities, detect malicious activities, and work towards building a resilient defense.

These “purple team” exercises are not just about finding gaps. They’re an essential tool for security and operations teams to better understand real-world threats, as well as their own capabilities.

Step 0: Threat Modelling

Before carrying out an attack simulation against any target, it is essential that the threats posed against it are understood well. Kubernetes environments are no different, where the first step involves answering critical questions like:

What are the assets that need to be protected?
Where could attacks originate from?
What would motivate attackers to target a Kubernetes cluster?

Let’s start with the attack surfaces. From an offensive security perspective, this will also help us identify the levels we must execute in, in order to simulate these attacks.

The most common entry point for attackers is at the container level—where the application code runs, within pods. For example, think of a compromised web server or a backdoored image that found its way into the cluster. If the pod has access to a service account, the attacker may interact with the Kubernetes control plane, potentially expanding their reach.

Should they manage to break out of the pod, they gain access to the underlying node. From there, they could attempt to impersonate core components like the kubelet, which would effectively elevate their privileges. They could also try to access other Kubernetes resources running on the same node that maybe in a different namespace that was not previously accessible.

Attacks can also originate from external sources, such as the Internet, or any network the control plane listens to. Think of scenarios like leaked or stolen credentials, a compromised service, a malicious or coerced user and so on…

Incentives of the Kubernetes attacker

Once a threat actor is inside the cluster, their end goals may vary. An obvious objective is the workloads –– the applications running on Kubernetes – with an intention to access sensitive data, modify proprietary software, or perform supply chain attacks.

However, in many cases, the target is more direct: money. In the age of cryptocurrency, computer resources are a target in itself, due to its immediate potential for financial returns. And Kubernetes nodes are not immune to this pattern.

Finally, Kubernetes clusters –especially those deployed in cloud environments– present another attack opportunity: Instead of targeting applications, attackers might try to breach the overarching infrastructure –whether cloud or on-premises– and get a foothold into this other realm.

Kubernetes is also a less-monitored platform compared to traditional on-premise and Windows estates, which makes it an ideal place for attackers to establish persistence and hide their activities, allowing for long-term, undetected operations.

Step 1: Execution of a Kubernetes attack simulation

Once the threats to the particular environment in question are understood, one can proceed to carry out the actual simulation. Let’s assume there are two main approaches to go about this – depending on what the end goal is:

Objective-based adversary emulation involves replicating the exact attack chains of real-world threat actors, with specific goals (e.g., data theft). Threat intelligence like this can be found aggregated on platforms such as the Wiz Cloud Threat Landscape.
Breadth-first capability assessment focusing on greater coverage of TTPs, executed individually (de-chained) with the aim to evaluate – or baseline for the future – the overall defensive capability. For a map of the entire space of TTPs available, Microsoft’s Threat Matrix for Kubernetes comes in handy.

Tools, tools, tools

At this point we should have a clear idea and vision for the exercise, leaving only the “How” to answer. Of course, one can always default to “manual” methods, interacting with the cluster as usual (kubectl, HTTP clients, shelling on pods etc.) but it’s worth knowing that there are plenty of tools out there for this purpose, that could make our lives easier.

Let’s look at a few of them:

Leonidas, WithSecure’s cloud attack simulation framework, has recently been updated to support Kubernetes environments as well. Like its AWS predecessor, it’s built to be extensible, friendly to write attack test cases without being a developer, and easy to interact with programmatically, through an API – enabling use within e.g. CI/CD pipelines.

Atomic Red Team is the well-known ART framework that supports a few atomics for containers, and a couple for the Kubernetes control plane. However, it’s not really built for cloud environments.

Stratus Red Team on the other hand, was built specifically for the Cloud and for attack simulations, making it a great choice for our purpose – it even supports eight pure Kubernetes test cases!

Peirates is another option one might consider, if the attacks we chose previously happen to match those hard-coded in Peirates. It’s been used by threat actors themselves after all! Nevertheless, room for customization will be minimal.

Step 2: The defender’s perspective in attack simulations

Attack simulations aren’t just about running offensive exercises but improving defenses.

As stated earlier, the ultimate goal is to upskill defenders and build operational capability. So let’s take a moment to to talk about defenders, what they should expect and how they can prepare for an exercise like this.

Aspects of the traditional SOC are not irrelevant here: Log sources, blind spots, SIEM ingestion, detection rules and even EDR concepts. All of which can be applied to container orchestration environments.

Logs are the cornerstone of security monitoring, and in Kubernetes too. This can be challenging in practice, however, due to the lack of an official, built-in, centralized logging solution (“cluster-level” logging). Users are responsible for setting up their own logging architecture to suit their needs. At the same time, there are many types of logs In a Kubernetes cluster, which can be broadly categorized into:

Application/container logs capturing the output from the applications running in containers

Kubernetes component logs, which are logs from core components like the API server and kubelet

Logs from external sources, which are also crucial for the security of the ecosystem, such as cloud provider logs, if you’re using a cloud-managed Kubernetes service (e.g., IAM logs)

Naturally, not all these log sources are useful for detecting security threats. The rule of thumb is that those of security interest will be those answering the questions:

Who acted?
What did they do?
When and where from?

Kubernetes audit logs: The key to attack detection

From the various sources available, the API server’s Audit logs can be considered the most valuable source for security monitoring, as they record the interactions with the Kubernetes API server.

They can capture suspicious actions like unauthorized access or secret listings. They are not enabled by default, and administrators need to first define an audit policy, setting adjustable verbosity levels to each event or event class. Crafting the perfect policy for your cluster – one that provides the visibility necessary while balancing the noise generated– will be a continuous process requiring tuning and experimentation. By enabling audit logs and forwarding them to a centralized location, detection engineering teams can then proceed to write alert rules upon certain attack patterns in those logs. These building blocks enable creation and sharing of these “attack signatures” in a platform agnostic format such as Sigma rules.

As for the other attack surfaces in scope, visibility into container- and node-level activity can be achieved with tools like Falco, Hubble, and Tetragon. These solutions leverage the Linux kernel’s eBPF functionality, to monitor Kubernetes nodes, providing EDR-style alerting on platform events such as suspicious command execution or network connections etc.

Conclusion

In summary, attack simulations offer a proactive approach in building defenses, and this is no different in Kubernetes environments.

Once a good understanding of the threat model for the ecosystem in question is in place, blue and red teams can collaborate with the methods we described in this article, to build resilience and stay on top of new threats.

Now’s the time to get ahead of attacks!