How to use the Humio Operator to run Humio on Kubernetes

Part 2 of a series on the new Humio Operator

This blog was originally published Nov. 12, 2020 on Humio is a CrowdStrike Company.

Running distributed stateful systems on Kubernetes can be a challenge — but it doesn’t have to be. The Humio Operator facilitates the configuration and deployment, and eases the management of your Humio clusters running on Kubernetes. In our previous blog post, we introduced the Humio Operator. In this second of a two-part series, we walk you through how to use the Humio Operator to run Humio on Kubernetes.

To run Humio on Kubernetes, you need Apache ZooKeeper and Apache Kafka. For full end-to-end instructions on preparing and installing Humio on Kubernetes, see the Operator Quick Start Guide. For simplicity’s sake, we will create a Humio Cluster using Persistent Volumes, we assume we have Zookeeper and Kafka, and we will use ‘port-forward’ to connect to our cluster.

Install CRDs

Before the Humio Operator can be installed, we must create our Custom Resource Definitions. The CRDs define the resources that the Humio Operator will manage, which include HumioClusters, HumioExternalClusters, HumioIngestTokens, HumioParsers, and HumioRepositories.

Find the version of the Humio Operator that we want to install, and create the CRDs:

kubectl apply -f${HUMIO_OPERATOR_VERSION}/config/crd/bases/core.humio.com_humioclusters.yaml
kubectl apply -f${HUMIO_OPERATOR_VERSION}/config/crd/bases/core.humio.com_humioexternalclusters.yaml
kubectl apply -f${HUMIO_OPERATOR_VERSION}/config/crd/bases/core.humio.com_humioingesttokens.yaml
kubectl apply -f${HUMIO_OPERATOR_VERSION}/config/crd/bases/core.humio.com_humioparsers.yaml
kubectl apply -f${HUMIO_OPERATOR_VERSION}/config/crd/bases/core.humio.com_humiorepositories.yaml

Install the Humio Operator

The Humio Operator can be installed with Helm.

First add the helm repo:

helm repo add humio-operator

And install:

helm install humio-operator humio-operator/humio-operator --version="${HUMIO_OPERATOR_VERSION}"

A full list of Helm values can be found in the installation documentation.

Create a Humio Cluster

A HumioCluster resource can now be created. The HumioCluster will start up our Humio pods. We need to point this cluster to our existing ZooKeeper and Kafka clusters.

cat <<EOF | kubectl apply -f -
kind: HumioCluster
name: example-humiocluster
autoRebalancePartitions: true
enabled: false
storageClassName: standard
accessModes: [ReadWriteOnce]
storage: 10Gi
value: "develop3r"
- name: "HUMIO_JVM_ARGS"
value: "-Xss2m -Xms256m -Xmx1536m -server -XX:+UseParallelOldGC -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC"
value: "<zookeeper url>"
value: "<kafka brokers url>"
EOF created

Once the HumioCluster is created, it will go into the state of Bootstrapping until all Humio pods are up:

kubectl get humiocluster
NAME                   STATE           NODES   VERSION
example-humiocluster   Bootstrapping

Once all Humio nodes are up, the Operator will have access to the Humio cluster and update the status of the HumioCluster custom resource including node count and version. At this point it will automatically balance our digest and storage partitions and be ready to create other Humio resources.

kubectl get humiocluster
NAME                   STATE     NODES   VERSION
example-humiocluster   Running   3       1.16.0--build-293562267--sha-39bbd8579eba9d6527e8b1579cdb9504a426a411

We can access the Humio UI by using kubectl port-forward. For production installations, we would rely on Ingress.

kubectl port-forward svc/example-humiocluster 8080
Forwarding from -> 8080
Forwarding from [::1]:8080 -> 8080
Handling connection for 8080
Handling connection for 8080

Now go to and login using the username “developer” and password “develop3r”, which we set during the cluster creation. When we go to the Cluster Administration under the settings button in the top right corner, we see the list of our pods and that the digest and storage partitions have been balanced:

Create a Repository, Parser, and Ingest Token

Now that we have a running cluster, we can create some other resources that are managed by the Operator. Let’s start with a Repository.

cat <<EOF | kubectl apply -f -
kind: HumioRepository
  name: logs
  managedClusterName: example-humiocluster
  name: logs
  description: "Logs repository"
    timeInDays: 30
    ingestSizeInGB: 50
    storageSizeInGB: 10
EOF created

Now create a Parser and assign it to the Repository that we just created:

cat <<EOF | kubectl apply -f -
kind: HumioParser
  name: fluentbit-parser
  managedClusterName: example-humiocluster
  name: fluentbit-parser
  repositoryName: logs
  parserScript: |
    case {
        | /\[(?<@timestamp>[^\]]+)\]/
        | /^(?<@timestamp>.*)\[warn\].*/
        | parseTimestamp(format="yyyy/MM/dd' 'HH:mm:ss", field=@timestamp);
      * | kvParse()
EOF created

Finally, before we send logs to the cluster, we need to create an Ingest Token and assign it to the Repository and Parser.

cat <<EOF | kubectl apply -f -
kind: HumioIngestToken
  name: fluentbit-ingest-token
  managedClusterName: example-humiocluster
  name: fluentbit-ingest-token
  repositoryName: logs
  parserName: fluentbit-parser
  tokenSecretName: fluentbit-ingest-token
EOF created

Since we set “tokenSecretName” in the Ingest Token spec, the token content is stored as a secret in Kubernetes. We can then fetch the token:

export INGEST_TOKEN=$(kubectl get secret fluentbit-ingest-token -o template --template '{{.data.token}}' | base64 -D)

We can go to the Humio UI and verify that the Repository is configured with our Ingest Token and Parser:

Ingest Logs

Logs can now be sent to the cluster using the ingest token extracted from the previous step. To gather logs from the cluster, we install Fluent Bit using the Humio Helm Chart:

First, add the Helm repo:

helm repo add humio
helm repo update

Create a Fluent Bit values file, which points to our cluster:

cat >> humio-agent.yaml<<EOF
  enabled: true
  humioHostname: example-humiocluster
    inputConfig: |-
          Name             tail
          Path             /var/log/containers/*.log
          Parser           docker
          # The path to the DB file must be unique and not conflict with another fluentbit running on the same nodes.
          DB               /var/log/flb_kube.db
          Tag              kube.*
          Refresh_Interval 5
          Mem_Buf_Limit    512MB
          Skip_Long_Lines  On
        cpu: 100m
        memory: 1024Mi
        cpu: 100m
        memory: 512Mi

Now install it:

helm install humio humio/humio-helm-charts --set humio-fluentbit.token=$INGEST_TOKEN -f humio-agent.yaml

Once it’s installed, we can go back to the Humio UI and confirm that our Operator logs, along with logs from other containers running in the Kubernetes cluster, are being consumed by Humio:

We aren’t able to go over every capability of the Humio Operator in this blog as there is just too much to cover, but we hope you found this blog post a useful introduction to help you get started running Humio on your Kubernetes environment. More information is available in our Humio Operator installation documentation in addition to documentation that covers how to migrate an existing Humio deployment that utilized our previous Helm based deployment.

We also encourage you learn more by watching our on-demand webinar on Running Humio on Kubernetes with the Humio Operator. You’ll learn best practices for leveraging the Humio Operator to maintain Humio resources, tips for making the best use of your cloud environment’s storage options, and more.

Additional resources

Related Content