Kubernetes Security Guide
12 Best Practices

What Is Kubernetes Security?

Kubernetes is today the de facto container orchestration platform to run scalable and reliable applications. With its easy-to-use API and developer-friendly characteristics, Kubernetes has become an indispensable part of the cloud ecosystem. Kubernetes security is a collection of best practices designed to keep the Kubernetes environment secure from cloud threats and vulnerabilities. 

The paradigm shift in software architecture and operations has inspired significant security changes. With microservice applications and containers deployed to cloud platforms such as Kubernetes, cloud-native security principles are required. This is because, unlike traditional data centers, the cloud has unclear security perimeters, leading to new challenges.

This guide will focus on the Kubernetes part of cloud security, discussing the attack surface, security lifecycle and best practices.

Most Common Kubernetes Security Vulnerabilities

Developers have to stay on top of a number of cloud vulnerabilities that might negatively impact Kubernetes security throughout the development lifecycle, which includes the build, deployment, and runtime parts of the process.

1. Build Vulnerabilities: Use of untrusted code with hidden malware that can grant access to threat actors without your knowledge. Additionally, containerized applications sometimes have excess packages or libraries that are not needed and could be compromised. Removing these excess packages is a good starting point. .
2. Deployment Vulnerabilities: Not having an identity access management solution might mean your teams are granting unnecessary privileges to users that don’t need the access and might be compromised or have malicious intent. This opens the door to a world of vulnerabilities, like insider threats or cyberattacks that can easily spread through lateral movement.
3. Runtime Vulnerabilities: During runtime, all elements within Kubernetes infrastructure contain their own attack surface, which can be a challenge to keep track of.

Kubernetes Attack Surface

A Kubernetes cluster consists of control plane components and nodes as diagrammed in Figure 1.

Figure 1: Kubernetes Components (Source: Kubernetes Docs)

The control plane is the brain of Kubernetes clusters, where definitions and the state of all Kubernetes resources are managed and stored. It is a critical vector for attackers. If a hacker gains access to your control plane components, they can deploy containers, read secrets or even delete the whole cluster.

The nodes are where the actual workload of the Kubernetes workload runs. All your applications and libraries are deployed as containers and run on the nodes, meaning they, plus the application layer, need protection as well. If the operating system of the node has vulnerabilities, it can lead to unwanted access. Similarly, if any dependency or library in your application containers has vulnerabilities, attackers can exploit these to access your application and data.

The third and last vector is where the Kubernetes clusters run: the cloud. Access to cloud services such as user management or a managed Kubernetes service could grant them privileges equal to a Kubernetes operator.

In the following section, we will discuss every step in the lifecycle of an application in a Kubernetes cluster to protect the attack surface.

What to Secure at Each Stage of the Kubernetes Lifecycle

The Kubernetes security lifecycle covers the complete journey of an application, starting from the source code to a running deployment in Kubernetes. Each step has its specific vulnerability issues and requires great care.

Before the Kubernetes Cluster

Kubernetes runs applications packaged as containers. This means you need to create container images with an application executable and dependencies. With the latest trends in software development, the build and release of applications and container images are done in pipelines like Jenkins, GitHub Actions or GitLab CI/CD to deliver faster with less effort.

In addition, image scanning is a container security process handled as part of CI/CD pipelines for the early detection of security issues. It should include the following steps:

• Check your operating system files and configuration, software packages, libraries and binaries
• Analyze Dockerfile for security flaws such as exposed ports or privileged access

There are popular open-source and commercially available tools for container image scanning, such as Anchore Engine, CoreOS/Clair, OpenSCAP and Falcon Image Assessment. These tools also have a rich set of integration points to be used as part of CI/CD pipelines.

In short, there are two things you need to protect before starting your Kubernetes journey:

• CI/CD pipelines
• Container images

Inside the Kubernetes Cluster

Kubernetes is a complex system with a layered architecture and well-defined APIs. It is designed as an extensible platform with multiple extension points. These allow you to develop new networking plugins to support custom networking infrastructure or create Kubernetes operators to transform human expertise into code. However, this extendibility makes the whole cluster vulnerable at multiple locations.

There are five important places that you need to ensure to achieve security inside a Kubernetes cluster: the Kubernetes control plane, access to the Kubernetes API, networking, nodes, and the container and runtime.

1. Kubernetes Control Plane

The Kubernetes control plane consists of the core services that keep the whole cluster up and running. The specification and state of the Kubernetes resources are managed by control plane components and stored in etcd, an open-source distributed key/value database. To protect the control plane, take the following actions:

• Make etcd accessible only via the Kubernetes API with correct permissions by putting restrictions on firewalls such as iptables and netfilters for etcd instances and authenticated access between API and etcd.
• Use and configure admission controllers  in upstream Kubernetes or write your own admission controllers and implement custom security rules.
• Configure an audit policy in the cluster to define which events are logged and store the logs in the filesystem or send them to external systems via webhooks.

2. Access to the Kubernetes API

The Kubernetes API is the interface of the control plane for external users, making authentication and authorization crucial parts of security.

• In order to authenticate real users, you need to connect with LDAP services or cloud identity services. Kubernetes provides OpenID Connect (OIDC) to use as an external authentication provider. This lets you use LDAP installation within your company and make everyone log in to Kubernetes with their work user accounts. It is highly suggested to secure your Kubernetes cluster access by authentication with an OIDC provider.
• Use  role-based access control (RBAC) to define access control specifications with actions such as read, update, and delete on every Kubernetes object and namespace. You need to design the hierarchy of users, user groups and access levels with great care and create corresponding RBAC definitions in Kubernetes clusters.

3. Networking

Pods running in Kubernetes clusters can easily connect to other pods with Kubernetes networking capabilities. To create secure communication within the cluster, you should:

• Configure network policies, Kubernetes-native specifications to create firewalls with pods and namespaces.
• Deploy third-party networking plugins like Calico or Weaveworks and a service mesh like Istio. If you install any networking plugin, it is vital to configure all security options and keep them up to date.

4. Nodes

Kubernetes nodes run the actual Kubernetes workload — your software as containerized applications. In every Kubernetes node, there is an agent called a kubelet to communicate between the control plane and the container engine on the node. Therefore, you need to secure the two sides of the kubelet:

• Communication between the Kubernetes API and the kubelet, which should connect to the Kubernetes API using certificates with a limited cluster role.
• Communication between the kubelet and the container engine. The kubelet and the container engine, such as Docker daemon, run on the node operating system, so it is highly suggested to use a minimal host operating system. It is also critical to have the latest system and security patches installed.

In cloud environments, nodes are assumed to be ephemeral, as they can be created and deleted on demand. This means it’s vital to not leave essential data on the node filesystem. In addition, it is suggested to use resource requests and limits to keep nodes healthy with enough capacity.

5. The Container and Runtime

The last step in a containerized application lifecycle is deploying it as a pod and running it as a container in a Kubernetes node. Even if you have done everything correctly, vulnerabilities such as misconfigurations, zero-day vulnerabilities, privilege escalations, and malware can come up during runtime.

To protect against active threats while containers are running, you can adopt open-source tools like seccomp, AppArmor or SELinux that focus on privilege escalations and limiting access to the binaries on Linux systems. While this offers some protection, it is not enough for runtime protection from malware.

For full protection, download  a cloud-native and modern solution like CrowdStrike Falcon Cloud Security, a security isolation solution that can work across multiple cloud providers and secures both nodes and containers running on the nodes.

12 Kubernetes Security Best Practices

Kubernetes is an extensible platform with various security vulnerabilities and unclear security perimeters. Given this, there are some widely accepted best practices you should apply to keep your clusters and access safe:

What are some Kubernetes Security Best Practices?

1. Image Scanning: Make sure to have an image scanning tool that will help you identify vulnerabilities present within an image throughout the CI/CD Pipeline. Containers created from compromised images will transmit vulnerabilities to the container as a whole.

2. Implement Principle of Least Privilege: Adopting a POLP approach will help in giving minimal privileges to containers on the host to help prevent privilege escalation attacks, where one compromised container can gain access to the whole system.

3. Reduce Attack Surface: Select an image from a container that has the minimal amount of software packages available. These software packages are not needed most of the time and could be compromised.

4. Integrate Security Resources With Kubernetes: Integrate your cybersecurity resources with your Kubernetes cluster, which allows perimeter security solutions like a next generation firewall (NGFW) to work alongside existing security toolsets to protect the Kubernetes cluster.

5. Third-party Authentication: Integrate an external authentication provider and use already-defined user groups for authorization to access your Kubernetes API.

6. Enable RBAC: Ensure that RBAC is enabled and configured correctly, as a slight change in RBAC rules can make your clusters available to the world.

7. Enable Audit logging: Ensure that audit logging is enabled and available, even if the cluster is deleted.

8. Isolate Nodes: Make sure you isolate the network of the nodes and monitor the traffic throughout the cluster.

9. Monitor Network Traffic: Because containerized applications make use of cluster networks, it is important to monitor all network traffic and compare it to the traffic allowed by Kubernetes cluster. This will help your team uncover further attack surface reduction opportunities by identifying unusual communications.

10. Process Whitelisting: Process whitelisting is the process of observing an application overtime and identifying normal application behaviors, helping you identify unexpected processes as a result.

11. Kubelet configuration: Ensure kubelet has only the minimum required access levels and configuration options.

12. Threat Defense: There are hundreds of types of cyberattacks that can compromise your Kubernetes clusters. Implement threat proactive defense mechanisms that focus on intrusion detection and prevention.

Conclusion

Kubernetes is a complex platform with an active community and an ever-changing environment, with new plugins and infrastructure extensions. Therefore, it comes with many risks in terms of creating a secure production-ready cluster.

Fortunately, Kubernetes is highly flexible and can work with other tools via extension points and the Kubernetes API. So, if you configure and use cloud-native modern tools, you can securely run your applications in Kubernetes in a reliable and scalable way.

Falcon Cloud Seucrity delivers containers, Kubernetes, and hosts from build to runtime in AWS, Azure, and Google Cloud while ensuring security in every step of the CI/CD pipeline.