Amazon Web Services log data is an extremely valuable data source that comes in a variety of flavors depending on the services you are looking to learn more about. Some of the most common data sources being AWS VPC flow, Cloudtrail and CloudWatch. These logs can be directed to S3 buckets where they are often ingested by log management or SIEM solutions. More details on these logs can be found here. 

Prerequisites

To follow the steps, you will need access to AWS, basic knowledge of AWS architecture and access to a Falcon LogScale environment as well as an understanding of its architecture.

This article assumes you have full permissions and access to your S3 buckets. If not, you will need to configure permissions to allow S3 integration Falcon LogScale.

Writing Logs to S3

Your first step is to make sure that your AWS services are writing their logs to S3. This can be to a separate bucket or a directory within a bucket. These log files will then be pulled into Falcon LogScale for analysis and visualization, the format of the data can be line-delimited or AWS JSON events. AWS example events are referenced here.

Setting up a Repository in Falcon LogScale

The first step in the process is to configure a repository so that we can have a target for the logs we are ingesting and that they can be stored, searched and analyzed using Falcon LogScale.

Configure the Simple Queue Service 

A new SQS will need to be created and will notify us when new logs are available in the S3 bucket for ingest. The configuration of the SQS will require properties such as a name, retention period in days before they are deleted:

We will then need to setup an access policy under advanced settings:

This is where we retrieve the Amazon Resource Name and specify it in the configuration to give it access to the S3 bucket and allow it to send messages as follows:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": "s3.amazonaws.com"
            },
            "Action": "sqs:SendMessage",
            "Resource": "arn:aws:sqs:*",
            "Condition": {
                "ArnEquals": {
                    "aws:SourceArn": "S3_BUCKET_ARN"
                }
            }
        }
    ]
}

Then choose the option to create a queue.

The next step is to configure the S3 bucket to create event notifications: 

Ensure you select all object create events:

Ensure you select the SQS you created in the earlier step:

You should at this stage go to the SQS queue and check the monitoring tab to see the Number of messages being sent. This process may take a few minutes:

Configure IAM Role

The next step is to setup a role in IAM to allow reading of the data:

Select Roles and Create a Role with a custom trust policy:

Use the information retrieved from the Falcon LogScale configuration to populate the custom trust policy. To retrieve this information, go to Falcon LogScale > Settings > Ingest feeds > Add AWS ingest feed.

You have a variety of options to select including compression options such as Auto, Gzip or none. There are also preprocessing options to Split by AWS records or Split by new line. You then also have the ability to select the parser depending on the format or structure of your data.

When you select next you will be able to retrieve the IAM identity and External ID details so they can be specified in the custom trust policy:

The Trust policy should then look like:

{

    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowLogScaleAccess",
            "Effect": "Allow",
            "Principal": {
                "AWS": "LOGSCALE_AWS_IDENTITY_ARN"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "sts:ExternalId": "LOGSCALE_EXTERNAL_ID"
                }
            }
        }
    ]
}

You will then be able to provide a role name and then finish by choosing create role.

Once the role is created, you can search for it and select it:

The next step is to create an inline policy

The inline policy allows us to receive the notifications we configured and to read the logs from the S3 buckets, we will need information such as the S3 bucket ARN and the SQS ARN. The inline policy will be in the following format:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ReceiveS3Notifications",
            "Action": [
                "sqs:ReceiveMessage",
                "sqs:GetQueueAttributes",
                "sqs:DeleteMessage",
                "sqs:ChangeMessageVisibility"
            ],
            "Effect": "Allow",
            "Resource": "SQS_QUEUE_ARN"
        },
        {
            "Sid": "ReadS3Objects",
            "Action": "s3:GetObject",
            "Effect": "Allow",
            "Resource": "S3_BUCKET_ARN/*"
        }
    ]
}

Once you click next you will be asked to specify a policy name and complete the process by choosing the Create policy option.

For the final part of the process you will specify the role ARN, SQS URL and AWS region within the LogScale configuration:

Ingest feeds will look like the following:

You should now have data flowing into CrowdStrike Falcon LogScale from AWS S3 buckets.

Unbeatable Scale and Performance

Join leading organizations by augmenting or replacing your security information and event management (SIEM) solution with Falcon LogScale for unbeatable scale and performance. With its unique index-free architecture and advanced compression technology, Falcon LogScale empowers you to stop breaches by delivering high-speed search and sub-second latency for live dashboards and real-time alerting.

With Falcon LogScale, you can cut costs by up to 80% compared to legacy SIEM solutions. Its vast scale and affordable price let you avoid making tough tradeoffs between cost, how much data you can collect and how long you can store it. With Falcon LogScale, you can retain petabytes of data for years.

To learn more about Falcon LogScale integrations, visit the Integrations page. To find out if Falcon LogScale can help you fulfill your SIEM and logging requirements, contact a CrowdStrike expert today.

 Additional Resources

• View Zscaler and Broadcom ProxySG integration instructions.
• Download the CrowdStrike eBook, 8 Things Your Next SIEM Must Do, to understand the critical capabilities to look for when evaluating SIEM solutions.
• See Falcon LogScale in action by watching a fast-paced demo.
• Sharpen your threat hunting skills with Falcon LogScale by attending a hands-on lab.
• Find out how to log everything to answer anything in real time by visiting the Falcon LogScale product page.

Today, mobile devices are ubiquitous within enterprise environments. But with their proliferation, it provides adversaries with yet another attack surface with which they can target users and cause a breach.

From phishing attacks to malicious apps, mobile users tend to let their guard down and potentially click on obfuscated links to malicious sites. Falcon for Mobile protects users by preventing connections to malicious sites on both iOS and Android devices. Some examples of sources for these malicious connections are texts, emails, apps, or even QR codes.

Falcon for mobile can block navigating to malicious sites and notifying the user why it was blocked, educating the user and reducing the risk in the future.

Security administrators can also view reports within the Falcon Console to identify individuals within the organization that may require additional phishing training.

Video

Protecting Mobile Users

When the device attempts to connect to a phishing site, we see that it cannot be reached. The user is protected from the malicious site that could be attempting to harvest their credentials or install malicious software.

Falcon for Mobile also provides a notification as to why the site was blocked. The user can expand the notification to see additional details.

The mobile detection screen in the CrowdStrike Falcon Console is structured in a way that provides an easy way to view mobile threats identified within the environment.

Each line shows a high level overview of a detection, presenting information such as the threat name, mobile user, and device impacted.

An info pane on the right provides additional details about the threat, such as a description of the threat or additional host information.

The full details show related threat data such as the domain that was blocked as well as a timeline of activities related to this detection.

An analyst can adjust the case with options such as setting its status, assigning it to another analyst, adding a tag, or adding a comment.

After they’re done, the status will be updated and tracked to ensure resolution of the case.

Conclusion

So as we can see, Falcon for Mobile provides powerful protection that prevents users from unknowingly navigating to malicious sites and accidentally sharing data or credentials .

With phishing attacks being such a common vector for mobile attacks, blocking malicious network connections with Falcon for Mobile helps protect organizations from a breach.

More resources

• CrowdStrike 15-Day Free Trial
• Request a demo
• Guide to AV Replacement
• CrowdStrike Products
• CrowdStrike Cloud Security

Video

Bringing Proxy Logs into Falcon LogScale

You can use the HTTP API to bring your proxy logs into Falcon LogScale. When working with Zscaler, you can use Zscaler Nanolog Streaming Service (NSS), which comes in two variants:

1. Cloud NSS allows you to send logs directly to Falcon LogScale.
2. VM-based NSS allows you to collect logs on a VM, where they can be sent to Falcon LogScale via syslog.

Once data is streaming into Falcon LogScale, you can extract the relevant fields during the parsing process. These fields include:

The Zscaler package can be installed from the Falcon LogScale marketplace. It includes parsers for extracting fields from DNS, firewall, web and tunnel logs. It also provides saved queries and out-of-the-box dashboards, which show details such as:

Web: Threat Activity

This dashboard provides high-level threat activity showing a range of widgets, including IOCs, data loss prevention (DLP), vendor-defined threats and enrichment with CrowdStrike threat intelligence.

Web: Web Activity

This dashboard provides details about user activity and actions as well as blocked and allowed domain activity, application activity and even information about user agents being used.

Web: User Investigation

This feature allows you to drill down into a specific user’s activity and is a combination of the Threat Activity and Web Activity dashboards.

Threat Hunting Queries with Zscaler Proxy Data

Here are useful searches and queries to hunt for threats across Zscaler proxy data:

Proxy Traffic Summary by User

| groupBy([Vendor.deviceowner, Vendor.devicehostname, Vendor.cip, Vendor.department, event.action, Vendor.hostname] , function=[ collect([http.response.status.code,http.request.method,Vendor.proto,Vendor.contenttype,Vendor.appclass,Vendor.appname,Vendor.ereferer,Vendor.eurl,Vendor.urlcat,Vendor.urlclass,Vendor.urlsupercat,Vendor.agent.original]),
sum(Vendor.reqsize, as=totalRequestSize),
sum(Vendor.respsize, as=totalResponseSize),
count(Vendor.url, as=totalHits),
min(@timestamp, as=earliest),
max(@timestamp, as=latest)
])
| asn(Vendor.cip, as=asn)
| formatTime("%Y/%m/%d %H:%M:%S", as=earliest, field=earliest, locale=en_US, timezone=Z)
| formatTime("%Y/%m/%d %H:%M:%S", as=latest, field=latest, locale=en_US, timezone=Z)

Proxy Traffic Timeline for a User

| select([@timestamp, #Vendor.action, host.name, http.request.referrer, Vendor.urlcat,Vendor.urlclass,Vendor.urlsupercat,user_agent.original, http.response.status_code,http.request.method,Vendor.proto, Vendor.contenttype, Vendor.eurl])

Abnormal User Agent Strings

user_agent.original=/(?i)(?:bits|WebDAV|PowerShell|Curl|Microsoft)/

Open Redirect

Vendor.eurl=/s?\:\/\/(?:www\.)?t\.(?:[\w\-\.]+\/+)+(?:r|redirect)\/?\?/

Dynamic DNS

Vendor.urlcat = "Dynamic DNS Host"

Top User Agents

| event.action=Allowed | top(Vendor.eua, limit=100)

Top Threat Activity

| Vendor.threatname!="None" | timechart(Vendor.threatname, limit=10)

CloudFront Domain Connection

| event.action=Allowed Vendor.event.hostname=/cloudfront.net/i

Suspicious Web Categories

Vendor.urlcat=/(?i)(?:adware\/spyware\ssites|botnet\scallback|browser\sexploit|shost|malicious\scontent|phishing|remote\saccess\stools|spyware\scallback|spyware\/adware|suspicious\scontent)/ | table([Vendor.devicehostname ,Vendor.urlcat, Vendor.eurl, @timestamp])

Unbeatable Scale and Performance

Join leading organizations by augmenting or replacing your security information and event management (SIEM) solution with Falcon LogScale for unbeatable scale and performance. With its unique index-free architecture and advanced compression technology, Falcon LogScale empowers you to stop breaches by delivering high-speed search and sub-second latency for live dashboards and real-time alerting.

With Falcon LogScale, you can cut costs by up to 80% compared to legacy SIEM solutions. Its vast scale and affordable price let you avoid making tough tradeoffs between cost, how much data you can collect and how long you can store it. With Falcon LogScale, you can retain petabytes of data for years.

To learn more about Falcon LogScale integrations, visit the Integrations page. To find out if Falcon LogScale can help you fulfill your SIEM and logging requirements, contact a CrowdStrike expert today.

Additional Resources

• View Zscaler and Broadcom ProxySG integration instructions.
• Download the CrowdStrike eBook, 8 Things Your Next SIEM Must Do, to understand the critical capabilities to look for when evaluating SIEM solutions.
• See Falcon LogScale in action by watching a fast-paced demo.
• Sharpen your threat hunting skills with Falcon LogScale by attending a hands-on lab.
• Find out how to log everything to answer anything in real time by visiting the Falcon LogScale product page.

When Falcon LogScale and Grafana come together, they empower organizations to visualize their logs and traces from one user interface, thereby reducing complexity while giving you the scale and performance needed to store data for as long as you need it.

Tracing makes debugging and understanding distributed systems less daunting by breaking down what happens within a request as it flows through a distributed system. With traces, you can quickly pinpoint the source of the issue and identify which trace logs to focus on.

The LogScale data source plugin for Grafana allows you to easily ingest trace logs into LogScale, which can then be instrumented by popular open-source tracing solutions such as Zipkin, Jaeger and Tempo. The visualization of the traces are then available in Grafana, providing a complete picture of logs and traces.

Combine Logs and Traces using Falcon LogScale and Grafana

Before we show how to link up LogScale with Grafana, we’ll assume you’ve met the following two prerequisites:

1. You have a LogScale instance up and running, whether that’s a self-hosted deployment or a Cloud version, or an instance of Falcon LogScale Community Edition running on LogScale Cloud.
2. You have a repository in your LogScale instance with data in it.

For this demo, we’ll use:

• A Falcon LogScale Community Edition license
• The grafana/grafana source image from Dockerhub
• A LogScale repository setup with web server log data

Ready? Let’s roll.

Step 1: Install the Falcon LogScale Data Source Plugin

The key connector that facilitates this integration between LogScale and Grafana is the LogScale data source plugin for Grafana. We recommend running Grafana 8.4.7 or higher.

First, we’ll cover the steps for setting up the plugin as if you were running Grafana natively on an operating system. If you’re following along just to test out the plugin or want to get up and running sooner, jump ahead to the Containerized Quickstart section below.

Installing and configuring Grafana is outside of the scope of this demo. We’ve already installed Grafana using our package manager. Doing so gives us a grafana-cli utility which we can use to install the Falcon LogScale data source plugin.

$ sudo grafana-cli plugins install grafana-falconlogscale-datasource                                                                                                       

$ sudo systemctl restart grafana-server.service

That’s it!

Now, when you navigate in the Grafana interface to Administration > Plugins, you’ll see the data source plugin installed!

Containerized Quickstart

Clone Grafana Tempo. It’s a distributed tracing system which integrates Tempo traces with Grafana in just a few steps.

git clone https://github.com/grafana/tempo

If you want to have the LogScale plugin installed automatically during Grafana startup, edit the Docker compose file and add GF_INSTALL_PLUGINS=grafana-falconlogscale-datasource as an environment variable under the Grafana service.

The Docker compose file is located under tempo/example/docker-compose/local/

version: "3"
services:

  tempo:
    image: grafana/tempo:latest
    command: [ "-config.file=/etc/tempo.yaml" ]
    volumes:
      - ../shared/tempo.yaml:/etc/tempo.yaml
      - ./tempo-data:/tmp/tempo
    ports:
      - "14268:14268"  # jaeger ingest
      - "3200:3200"   # tempo
      - "9095:9095" # tempo grpc
      - "4317:4317"  # otlp grpc
      - "4318:4318"  # otlp http
      - "9411:9411"   # zipkin

  k6-tracing:
    image: ghcr.io/grafana/xk6-client-tracing:v0.0.2
    environment:
      - ENDPOINT=tempo:4317
    restart: always
    depends_on:
      - tempo

  prometheus:
    image: prom/prometheus:latest
    command:
      - --config.file=/etc/prometheus.yaml
      - --web.enable-remote-write-receiver
      - --enable-feature=exemplar-storage
    volumes:
      - ../shared/prometheus.yaml:/etc/prometheus.yaml
    ports:
      - "9090:9090"

  grafana:
    image: grafana/grafana:9.4.3
    volumes:
      - ../shared/grafana-datasources.yaml:/etc/grafana/provisioning/datasources/datasources.yaml
    environment:
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
      - GF_AUTH_DISABLE_LOGIN_FORM=true
      - GF_FEATURE_TOGGLES_ENABLE=traceqlEditor
      - GF_INSTALL_PLUGINS=grafana-falconlogscale-datasource
    ports:
      - "3000:3000"

Grafana also has a handy option that automatically installs the Falcon LogScale data source plugin when running a container.

Note: if you want to preserve your configuration during the container restarts, add ./grafana-data:/var/lib/grafana to the volumes section of the Grafana service.

Now, run the following command for a ready-to-use Grafana instance with the Falcon LogScale data source plugin installed.

$ cd tempo/example/docker-compose/local
$ docker-compose -f docker-compose.yaml up -d

This command automates the installation of a local Grafana stack. The Grafana interface is now available at http://localhost:3000 in your browser.

For a quick sanity check, let’s make sure our local container stack is running.

$ docker-compose -f docker-compose.yaml ps

Now that Grafana is up and running, navigate to http://localhost:3000 in your browser and log in. The container doesn’t need a specific authentication scheme; simply enter “admin” for username and password on the login page.

When you navigate to Administration > Plugins and search for Falcon LogScale, you’ll see the plugin installed.

Step 2: Configure the Falcon LogScale Data Source

Now we can review our data sources. The LogScale data source may require further edits for multiple clusters. For this demo, Tempo is already populated with some traces, which we’ll correlate with LogScale events later in the post.

From the menu, navigate to Connections > Your connections > Data sources to confirm that the Falcon LogScale plugin was installed by the Docker script. Edit the configurations as needed.

If the plugin is not installed, navigate to Administration > Plugins, search for Falcon LogScale and click it. There, you’ll see documentation about the plugin, but we’re interested in the button to Create a Falcon LogScale data source. Click it to proceed to the data source configuration.

Although there’s a lot of information on the configuration page, we’ll focus on the most important settings. First, the name of the data source is prepopulated with Falcon LogScale, but you can change this if you’d like. For our demo, we’ll continue with the default name.

Second, the connection URL will vary depending on your LogScale environment and license. We’re using Falcon LogScale Community Edition, so our URL is https://cloud.community.humio.com . For Falcon LogScale Cloud customers, change the URL to point to your LogScale instance, for example https://cloud.us.humio.com/. You can consult LogScale’s endpoints documentation for the URL you should use.

Finally, we need to configure authentication. Grafana needs some sort of credential or token to authenticate with LogScale to access repository data. There are several settings you can use to meet the needs of your environment. For ease of configuration, we’ll use a personal API token and add it to the last box shown here:

Note: If you’re actively using your personal API token for other use cases, resetting it will impact any other application using it!

To generate a personal access token, log in to LogScale and navigate to User Menu > Manage your account > Personal API Token. Then, set or reset your token. Once you have your token, you can copy its value and paste it into the Grafana “Token” box pictured above.

To correlate traces with the logs, configure the “Data Links” and set the “Field” values that match the field names extracted from your logs. Here, we set it to trace_ID and #type and add ${__value.raw} as the query that points to the internal Tempo integration.

Click Save & test to verify connectivity.

The last step is to configure the Tempo data source and link the traces with LogScale. Navigate to Connections > Your connections > Data sources and select Tempo.

Ensure these configurations in your environment:

• Select LogScale as a data source and set the time spans for your traces.
• Define a tag and ensure they match with the LogScale fields.
• Optionally, you can use a custom query to limit the events returned from LogScale.

Here’s an example screenshot with basic configurations.

Step 3: Run Queries and Configure Dashboards

Running queries on LogScale data in Grafana works like querying directly in LogScale. For example, taking our demo repository with web server data, we can run a time chart in Grafana the same way we would do it in LogScale. We navigate to the Explore section of Grafana, select our repository and run the query.

#type=kv | timeChart()

The Grafana Integrations page in the LogScale documentation has some tips on how certain outputs of LogScale map to Grafana.

Queries can get rather long, especially in complex use cases. Fortunately, LogScale lets you save queries, and the Falcon LogScale data source plugin lets you use those saved queries directly in Grafana!

Let’s start with our example query from above:

#type=kv | timeChart()

Save this query in the LogScale interface as “WebServer_timechart”.

Now you can go back to the query input in Grafana and reference the saved query as $WebServer_timechart()

When we use our saved query, everything looks the same, so we consider that a success!

Now that we have a baseline for what a Grafana panel looks like, we can put this together with other data sources into a single dashboard, effectively completing our observability strategy using LogScale as a key piece of the puzzle.

Here’s an example of viewing LogScale logs with other data sources.

The above dashboards provide an overview of the correlated LogScale events with the traces.

We copied a few trace IDs from the Tempo search output and replaced it with the <TRACE_ID> in the curl command.

curl https://cloud.community.humio.com/api/v1/ingest/json \
-X POST \
-H "Authorization: Bearer <INGEST_TOKEN>" \
-H "Content-Type: application/json" \
-d '[{"trace_ID": "<TRACE_ID>", "Trace name": "This is a track which can be correlated with Tempo", "user": "myuser", "app": "HelloWorld app"},{"trace_ID": "<TRACE_ID>", "Trace name": "This is another track which can be correlated with Tempo", "user": "myuser", "app": "HelloWorld app"}]'

Remember to set a valid ingest token. If you’re not using the Community cluster, specify the correct URL and direct the events to your LogScale environment. The events can be visualized in your LogScale instance, as seen below.

Go to your Grafana data sources and explore Tempo. Paste one of the copied trace IDs and search for it, as shown below. To explore more details, click on the log icon.

Conclusion

In this guide, we walked through how to connect Grafana with LogScale so that you can view and query LogScale data from within Grafana. We covered installing the Falcon LogScale data source plugin — both natively in an operating system and using the Grafana Docker image. We also covered configuration and authentication from Grafana to a LogScale repository with a personal API token. Finally, we demonstrated how to run LogScale queries in Grafana and leverage saved queries for portability.

For many organizations, LogScale may just be one piece of a wider observability strategy that also leverages Grafana for dashboards and visualizations. Integrating LogScale alongside other observability tools is straightforward. For technical details on other aspects of the LogScale data source plugin for Grafana, check out the Grafana Integration documentation.

Additional Resources

• Watch the CrowdCast, “Future Proof Your Observability Strategy with CrowdStrike and Cribl.” 
• Read the white paper, “Log Everything to Answer Anything in Real Time” to learn how Falcon LogScale revolutionized log management.
• See how CrowdStrike is driving the convergence of security and observability with Falcon LogScale and Falcon Complete LogScale.
• Get a free trial of Falcon LogScale and see for yourself how you can log everything to answer anything in real time.
• Contact us to learn more about Falcon LogScale.

AWS Verified Access (AVA), a feature of Amazon Virtual Private Cloud (VPC) networking, delivers secure, zero-trust access to private applications without a VPN by continuously evaluating each request in real time based on contextual security signals like identity, device security status and location. The service grants access based on the security policy configured for each application and then connects the users, thereby improving the security posture of the organization. CrowdStrike customers can leverage Falcon sensor’s deep inspection and CrowdStrike Threat Graph® analytics to provide highly accurate security posture scores for the service’s access decisions.

Video

Lab Environment

AWS Verified Access provides secure access to private applications (non-internet routable) hosted in an Amazon VPC by acting as a reverse proxy,  leveraging additional identity and device posture checks before routing the traffic to the application. Using CrowdStrike Falcon Insight XDR (XDR) and Zero Trust Assessment (CrowdStrike ZTA), we provide customers the ability to holistically assess their endpoint security posture, allowing AWS Verified Access to provide conditional access to resources that comply with your organization’s device posture policies.

AWS Verified Access relies on these primary components:

1. An AWS Verified Access instance with associated AWS Verified Access groups, policies, endpoint, and trust providers.
2. A Native Message Host deployed to each client endpoint which is responsible for reading the CrowdStrike ZTA score and securely communicating the payload to the browser extension.
3. A Browser Extension enabled on each client endpoints for device posture evaluation and communicating back to the AWS service.

In this demo, we have a sensitive application deployed in a private VPC. An AWS Verified Access Policy ensures that only the client endpoints with a CrowdStrike agent installed and a ZTA score above the configured threshold can access the application.

The CrowdStrike ZTA score is calculated by the locally-installed CrowdStrike Falcon Sensor based on the security posture of the device and written to the local filesystem on a periodic basis and in response to configuration changes. The AWS Native message host gets the CrowdStrike ZTA score from the local file, and then provides the ZTA score to the AWS Verified Access browser extension for evaluation against the AWS Verified Access policy for CrowdStrike.

Visit CrowdStrike AWS Verified Access GitHub for step by step instructions.

Install the CrowdStrike Falcon Sensor

Install the CrowdStrike Falcon Sensor on a Windows endpoint. This endpoint will also get the Native Messaging installer and browser extension installed.

Install the Native Message Host

Install the Native Message Host on your client endpoint. This will allow the AWS Verified Access browser extension to get the client endpoint’s CrowdStrike ZTA score.

Windows

1. Download the MSI via the following link
2. Install the MSI on your Windows client endpoint

Install the browser extension

In this step, you’ll install the AWS Verified Access browser extension on your client endpoint. In this example, we’ll be using the Chrome browser. However, AWS Verified Access supports Firefox, too and the instructions are nearly identical.

1. Navigate to the Chrome Extension Store
2. Search for AWS Verified Access and install the extension

CrowdStrike Zero Trust Assessment

Zero Trust Assessment (ZTA) monitors OS and Falcon Sensor settings of hosts within your organization. This granular assessment of eligible hosts is used to produce a score that uniquely represents the security posture of each host. You can use the Falcon Zero Trust Assessment dashboard to view a holistic overview as well as a detailed assessment of monitored hosts, to investigate and remediate insecure settings, and to improve the security posture of hosts.

Zero Trust Assessment calculates a security score from 1 to 100 for each host. A higher score indicates a better security posture for the host. A security score is specific to the unique configurations of your environment. Zero Trust Assessment does not define what constitutes a good score. Instead, the ZTA dashboard provides visibility into possible risks and insight into settings that can increase the security posture of hosts.

Security scores are derived from two distinct assessment sources:

When a change is detected in either the OS or sensor settings, security scores are updated.

A host’s security score is also dependent on the ZTA version used to assess the host. This version appears in the ZTA dashboard next to each host’s score. ZTA versions are updated by CrowdStrike to account for changes in how security scores are calculated. For example, if a new prevention policy becomes available in Falcon, ZTA calculations are updated to account for the new policy. Hosts are then assessed based on the new ZTA version and whether they meet the new requirement.

For this host, the security score is 26.

Key Components of Verified Access

AWS Verified Access evaluates each application request from your users and allows access based on:

When a user tries to access an application, Verified Access gets their data from the trust provider and evaluates it against the policies that you set for the application. Verified Access grants access to the requested application only if the user meets your specified security requirements.

In addition, Verified Access logs every access attempt, to help you respond quickly to security incidents and audit requests.

Trust Provider

We have configured 2 trust providers, CrowdStrike and Okta.

Access Policy

This Access Policy is configured to permit an action on a resource when the Overall CrowdStrike Assessment Score (ZTA) is more than 80.

Test connectivity to your application

Verify if your private application is properly protected by AWS Verified Access. Navigate to your Application Domain.

Since the client machine has a ZTA score of 26, it doesn’t meet the criteria set by the Access Policy. Hence, the client won’t be able to access the application.  You see 403 Unauthorized

CrowdStrike Prevention Policies

CrowdStrike Falcon uses overlapping methods to detect both known and unknown threats. This helps ensure detection and prevention of attacks at multiple stages, and is also why enabling all of our recommended prevention policies is critical.

Standard playbook tactics for malicious actors leverage privilege escalation and credential theft. These tactics enable lateral movement and exploitation or compromise of systems in your environment. Therefore, it’s vital that you have a view of activity across all potential attack phases. Enabling only 8 out of 10 policy toggles doesn’t mean you are 80% protected. If the one setting needed to detect a particular malicious attack in your environment is also the one that’s disabled, you’re still potentially 100% vulnerable.

You can test all policy changes in pre-production first, and then deploy the changes to production in stages. You can triage detections and adjust settings as needed to see fewer false positives, using IOC management and machine learning and IOA exclusions.

When new prevention policy options are made generally available, we recommend that you incorporate them into their production environments using your standard change control methodology.

Enabling ‘Strong‘ Prevention Policies will increase the security & ZTA score of the endpoint.

After Enabling stronger Prevention policies, the CrowdStrike Assessment score increases to 85.

The client machine now meets the criteria set by the Access policy (ZTA>80), so it will be granted access the private application.

The CrowdStrike AWS Verified Access integration is an open-source project and not a CrowdStrike product. As such, it carries no formal support, expressed, or implied. If you encounter any issues while deploying the integration, you can create an issue on our GitHub repository for bugs, enhancements, or other requests.

AWS Verified Access is an AWS product. As such, any questions or problems you experience with this service should be handled through a support ticket with AWS Support.

Register now to build skills at hands-on workshops and learn from skilled threat hunters.

Introduction

This document will outline how to use Falcon Device Control to create and enforce USB device policy in your environment to minimize the attack surface and risk of incidents.

Video 

This video provides a demonstration of the Falcon Device Control configuration as well as the USB device visibility available with Falcon Insight dashboards. Falcon Prevent customers can also access similar visibility options via the Activity app.

Device Control Policy Options

Falcon Device Control enables customers to use their existing management platform and lightweight agent to establish and enforce policies for USB devices on both Mac and Windows devices. The policies and options can be in the Endpoint Security section under USB device control.

From the policy list, you can choose to create a Windows or Mac policy, to edit an existing policy, or create a new policy. I’ll follow the path of creating a new policy.

Within each policy, you are presented with a list of different USB device classes. This gives you the granularity to define different policies for different types of devices. Device classes include imaging, printers and mass storage. For mass storage, you will see four options available. “Read, write and execute” gives users full access. For mass storage, “Read and write only” is often used to prevent the auto-execution of unwanted programs.

Within the policy for each class, you also have the option to define exceptions. These exceptions can be done for specific devices or in larger groups using vendor and product information. This allows you to provide necessary functionality while maintaining control over user access. It is also very helpful when enforcing specific corporate standards.

For Windows policies, CrowdStrike USB Device Control is able to provide enhanced file metadata collection. This allows the Falcon Agent to leverage machine learning to classify source code written to a USB device, provide the provenance of a file, identify data sensitivity labels, and view the contents of archived files.

This visibility enables you to investigate potential data exfiltration events.

After a policy is created assign that policy to a group for simpler management and automatic assignment. For CrowdStrike customers check out the full details in the USB Device Policy guide in the console.

Conclusion

Falcon Device Control provides the ability to establish, enforce and monitor policies around your organization’s usage of USB devices.

More resources

• CrowdStrike Free Trial
• Request a demo
• Guide to AV Replacement
• CrowdStrike Products
• CrowdStrike Cloud Security

Threat hunters and incident responders are under tremendous time pressure to investigate breaches and incidents. While they are collecting and sorting massive quantities of forensic data, fast response is critical to help limit any damage inflected by the adversary. This article and video will provide an overview of Falcon Forensics, and how it streamlines the collection and analysis of point-in-time and historic forensic data.

Video

Deploying Falcon Forensics

To facilitate the collection of Forensics data, CrowdStrike provides dissolvable agents for Windows, Mac and Linux that can be downloaded from the “Host setup and management -> Sensor Downloads” page in the Falcon UI with the “Tool Downloads” link. Those executables can then be deployed via Real Time Response, Falcon Fusion, or other software deployment tools. Once deployed and executed, CrowdStrike Falcon® Forensics collects a snapshot of both forensic and point-in-time triage data from the host.

The resulting data is then made available in the cloud. It can be exported via FDR or viewed through the Falcon user interface and a number of pre-configured dashboards. The Deployment Status dashboard highlights how many systems have the tool installed along with the number of collections in the past 24 hours. In addition to breakdowns by operating system and role, the hosts can be filtered by Agent ID and system name.

Analyzing Forensics Data

There are two main dashboards available to review the collected Forensics data. On the Host Info page, there are options to search by hostname and collection time. The dashboard provides a summary of events by source name as well as a count of Falcon Forensics Modules by source type. Integration with Falcon Intelligence allows for the viewing of IoC and threat types, as well as confidence levels and custom severity.

The Users information highlights potential areas of concern around account status, and admin privileges as well as failed login attempts. This page also includes details on process information, network processes, network interfaces and shim cache.

The Host Timeline report can be used to look at a specific host or a multi system view over a defined period of time. The event types are color coded with the option to focus on them individually. Highlighting a specific section of the timeline filters the support list to show only events during that window of time. Those events can also be further filtered by system name and source type as well as time and custom fields.

Windows Hunting Leads with Forensics

In addition to host based and custom searches, Falcon Forensics also includes a dashboard for hunting leads for Windows. By providing a list of panel groups, this feature helps organizations identify activity with a higher signal-to-noise ratio. It offers low-hanging fruit for analysts to quickly identify potential misconfigurations or hacker activity.

This example uses the registry to report a list of processes that could be victim to a specific MITRE technique related to Image File Execution Options. These quick win reports give analysts easy access to drill in on specific, potential misconfigurations or attacker driven activity.

Conclusion

Falcon Forensics streamlines the collection of point-in-time and historic forensic triage data for robust analysis of cybersecurity incidents. With predefined dashboards and flexible search options, responders can quickly identify relevant data and speed investigations.

More resources

• CrowdStrike 15-Day Free Trial
• Request a demo
• Guide to AV Replacement
• CrowdStrike Products
• CrowdStrike Cloud Security

Before you can write your ingest client, you must prepare a good foundation. That means preparing your MacOS instance via the following steps:

• Download Homebrew 
• Update your default MacOS Python
• Install Python Package Manager
• Download LogScale’s libraries

Ready? Let’s get started.

Prepare your MacOS instance

One of the methods of ingestion is to use LogScale’s software libraries that are available in a variety of languages. Today we’ll be working with Python and MacOS.

Step 1: Install Homebrew, a package manager for MacOS. Installing new packages with Homebrew is a simple command line in Terminal, similar to installing new packages in Linux. Follow the instructions on the Homebrew site.

Step 2:  Use Homebrew and update your default MacOS Python. As you may know, MacOS 10.15 (Catalina) is currently running on Python 2.7, even though newer releases are available. It’s essential to MacOS that the default Python remains at 2.7. We’ll need to update your Python to the latest version while retaining version 2.7 for essential MacOS functions.

Follow these instructions from Matthew Broberg: The right and wrong way to set Python 3 as default on your Mac.

Update for Ventura: For the latest version of MacOS 13.2.1 (Ventura) Python 3.9.x is available, but it isn’t installed by default. You will need to install XCode to install Python using the terminal with the following command:

xcode-select –install

You can find more information here: Python3 now included with Ventura

Step 3: Once we have the appropriate version of Python running in your MacOS, we’ll need to install Python Package Manager, pip, so that we can install LogScale’s Client Library. Usually pip comes packaged with Python and there’s no additional step to install.

To see if pip is installed, run the following command in your Terminal:

python -m pip --version

If pip is installed, you’ll see the following output:

Alternatively, you can manually install pip by opening Terminal and run the following command:

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py

Then run python -m pip --version again to verify pip is installed and you have the latest version.

Step 4: Install LogScale’s Python software library. The humiolib library is a wrapper for LogScale’s web API, supporting easy interaction with LogScale directly from Python. You can find more information in our humiolib github.

You can start the install by running the following this pip command in your Terminal:

pip install humiolib

This command will give you a series of outputs that tells you the files being installed. Once installation is complete, you’ve finished the prep work for your MacOS instance. Now we can move on to the fun stuff.

Build your ingest client

It’s time to start writing to an ingest client. Take a look at this example program:

Let’s break down some of the pieces of the code.

At a minimum, you’ll need to add humiolib to be able to run the codes required to send logs to LogScale.

from humiolib.HumioClient import HumioIngestClient

You’ll also need to create an ingest client with attributes that tells the client where to ship this log.

client = HumioIngestClient(
  base_url= "The url where LogScale resides",
  ingest_token="An API token from LogScale"
)

API Token can be retrieved from your LogScale instance.

Structured log messages

There are two types of messages you can send to LogScale: structured and unstructured.

In most of our use cases, LogScale receives structured data as a JSON object. There’s no strict format as to how the JSON object is structured, but you do need to ensure the JSON object is valid. You can check the structure of a JSON object using a tool like JSONLint.

Additionally, with structured data, you can send valid timestamps as part of the log entry, and LogScale will use the timestamp provided instead of inserting one of its own. Therefore, please ensure that the timestamp for the log entry is less than 24 hours from the time it is sent. Otherwise LogScale will assume it’s older data and drop the log entry without an error message.

Below is an example of structured data:

structured_data = [
 {
 		"tags": {
 			"host": "str(ip)",
 			"host_name": "str(host)",
            "filename": "str(caller.filename)",
 			"line": "str(caller.lineno)",
 			"error_level": "INFO"
 		},
 		"events": [
 			{
 				"timestamp": str(datetime.now(timezone("EST")).isoformat()), #.strftime("%Y-%m-%d %H:%M:%S %Z"),
 				"attributes": {
 					"message": "Structured message",
 				}
 			}
 		]
 	}
 ]

Once the structured data is validated, you can send it to LogScale using the following function where the variable structured_data is the object you created above to store your JSON:

client.ingest_json_data(structured_data)

Support for unstructured data

Alternatively, you can send unstructured data to LogScale. Unstructured data are timestamped at ingestion since it’s a long comma delimited string. Thus the timestamp that you may or may not provide in the log entry has no impact on the ingestion timestamp. Below is an example of unstructured data:

unstructured_data = ["Unstructured message","Hello Python World",str(datetime.now(timezone("EST")).isoformat())]

You can send it to LogScale using the following function where unstructured_data is the object that contains your message. Please note the differences in the syntax between ingesting structured and unstructured data.

client.ingest_messages(unstructured_data)

If you completed all the steps above, you should start seeing messages appearing in your LogScale instance. Happy logging!

This integration enables joint customers to detect and respond to email attacks more quickly, mitigating the risk of widespread damage. The integration drives more value for customers by enabling correlation across email and other log sources, and delivers more complete investigations that facilitate fast and targeted remediation.

What is the LogScale and Mimecast integration?

The integration enables joint customers to ingest their Mimecast email security logs into LogScale. Once ingested, customers can view summary dashboards to see trends and high-level information, as well as drill down with flexible searches of the Mimecast data. Complex correlation searches across Mimecast data and other log sources can also be created with LogScale’s query language. Customers can create live searches which trigger alerts when potential malicious activity is observed.

How does the integration work?

Mimecast has developed a LogScale connector which is available for free. The connector pulls logs from the Mimecast service and ingests them into LogScale. The connector works with LogScale SaaS or hybrid deployments.

By installing the accompanying package from the marketplace within the LogScale interface, customers get instant access to a comprehensive set of eight multi-panel dashboards correlating to the different log source types from Mimecast.

How customers benefit

With LogScale’s unlimited price plans and modern architecture that compresses data by up to 80x, many customers can afford to log everything and aren’t forced to make compromises that introduce blindspots and risk.

By ingesting Mimecast logs alongside other log sources, customers can obtain complete visibility across the environment. LogScale customers can get more value from their Mimecast service by taking Mimecast detections of suspicious URLs or attachments and searching for them across the rest of their estate.

Full fidelity cyber investigations enable rapid containment and targeted remediation

LogScale customers can afford to retain their data for longer. This is particularly important with email security logs, as cyber investigations often go back months. Being able to work back to the initial email attack allows you to perform complete, full-fidelity investigations and confidently uncover the full extent of the attack.

Without this access to the logs, many investigations are inconclusive, forcing customers to either adopt a broad remediation plan that is expensive and can impact productivity, or go with a narrower remediation plan that risks leaving the attacker with a presence in their systems.

Extend threat hunting to include email security logs

LogScale’s fast search capabilities and comprehensive query language enable threat hunters to quickly execute queries, including complex correlation searches across multiple data sources. Threat hunters can now include Mimecast email security logs in their analysis and get insight from endpoint and network logs, and correlate these with Mimecast email security logs to get the full picture.

Using the integration

There are numerous ways to get value from your Mimecast email security logs through LogScale. Say, for example, a security investigation is triggered from an endpoint detection that has seen suspicious process activity on a user machine. We’ll assume the endpoint tool can tell us the URL from which the malware was downloaded. If this isn’t the case, it may be necessary to search for network data to link the malicious file seen at the endpoint with a download URL.

By searching the Mimecast logs for that URL, customers can discover if that URL was contained as a link in a phishing email.

To search Mimecast for a certain URL, here’s the LogScale query for an example URL of http[:]//t.mitt.dn.se/r/?id=hda9764d9,6476bff9,6476c038

In this query, we’re limiting the search to the relevant log source (“ttp-url-logs“) for miniscule efficiency gains, but you could also just search for the URL and leave out the first line.

#source = "ttp-url-logs" 
| url = "http://t.mitt.dn.se/r/?id=hda9764d9,6476bff9,6476c038"

From the results, we can pick out key fields such as the @timestamp, subject, from UserEmailAddress, userEmailAddress, sendingIP and messageID too.

A next step may be to investigate whether that IP address has sent other emails that may be from different addresses and with different subject lines, but also contain URLs … and to list those URLs in a table with the below query:

sendingIp= 130.117.8.227 
| top(url)

As shown above, there are six other URLs that may be of concern and it’s probably wise to search for any evidence of connectivity to these URLs across other log sources in LogScale, including endpoint and network data.

Now that you have all the relevant emails, go to the Mimecast console and use the messageID field to find the relevant emails in Mimecast and delete them from the user’s inbox and archive.

Next steps

To get started, visit the marketplace from within the LogScale interface, install the Mimecast package and configure the connector per the instructions.

We’re always looking for feedback. If you have ideas and feedback for enhancements to the Mimecast package in the LogScale marketplace, let us know at humio_packages@crowdstrike.com. Learn more about LogScale at https://www.crowdstrike.com/products/observability/falcon-logscale/

FluentD is an open source log collector that provides unified log processing across multiple emitter systems, either on the same logical device or separate devices. For example, on a single k8s pod, logs are emitted from the pod itself, the container running in the pod and the microservice running in the container. FluentD can collect all of these logs, format them to a common schema, filter unnecessary data and ship this unified stream elsewhere for additional processing.

CrowdStrike Falcon^® LogScale (formerly known as Humio) is a centralized log management and observability solution used to bring sanity out of the log and events data emanating from disparate systems. Consider all the logs that come from different k8s clusters, databases, microservices, message brokers and other infrastructure running in your clouds. LogScale can ingest logs from multiple FluentD (and other sources) and further process them for a wider view of your cloud infrastructure.

In this guide, we’ll walk through how to set up FluentD for collecting logs on your system. We’ll also demonstrate how to configure FluentD to ship those logs to LogScale for organization and analysis.

Getting started with FluentD

FluentD is used to collect logs of a particular system into a single stream of data that can be filtered and shipped upstream. The final destination for these aggregated logs can be multiple places:

• A data lake
• An archive database
• Another instance of FluentD
• Another log aggregator, such as LogScale

FluentD allows for a unified logging experience in all your microservices. It allows for manipulating the log schema to add new fields onto logs dynamically while filtering out unnecessary logs.

FluentD is also highly extensible, with a robust set of plugins available for a variety of use cases. Plugins include the k8s metadata filter plugin, which allows for automatic annotation of k8s pod and cluster information for your logs. This is a very common pattern for k8s log aggregation, as it helps organize and filter the firehose of data hitting your log management system.

How to set up FluentD

Before installing FluentD, your system must meet certain criteria.

The FluentD documentation site lists several requirements that must be in place for your environment:

• Set up NTP
• Increase the maximum number of file descriptors
• Optimize the network kernel parameters

Please follow the instructions from the FluentD documentation before continuing. These are general specifications for Linux-based systems using the SystemD management tool. You may need to adjust these instructions to meet your environment.

After you’ve gone through the instructions above, you’re ready to install FluentD. Assume the following commands unless otherwise stated are run using superuser permissions, such as with sudo or logged into as root.

Installing FluentD

FluentD has packages for the major Linux distributions and also options for installing via a Ruby gem or directly from the source repo.

For OpenSuSE, there is a package available for installing the ruby gem named ruby3.1-rubygem-fluentd.

For Debian and Red Hat-based systems, the package installs a slightly different named tool td-agent. This is effectively the same thing as fluentd, but with a legacy name.

In this guide, td-agent may be used interchangeably with fluentd.

Setting up the FluentD configuration file

Some packages for FluentD will come with a default configuration file. However, some customization is usually necessary to capture the specific logs or files you want in your system.

The configuration is typically found in /etc/td-agent/td-agent.conf, but not all installation methods will supply this file. Therefore, it may be up to you to create a configuration file.

For a very simple configuration that will add a listener on HTTP port 8888, add this to the td-agent.conf file:

<source>
  @type http
  @id http_input

  port 8888
</source>

Running FluentD

If you’re using systemd to manage your running services, then it is likely that the installation process has already initialized and started fluentd (or td-agent) on your system. You can check by running:

systemctl status td-agent.service

Adjusting permissions

On Debian-based machines, the permissions for configuration files and directories needed to run FluentD may require some additional setup after installation. Doublecheck that the configuration files and any fluentd and td-agent folders underneath /var all have their group set to the td-agent group, and that group has read and execute permissions on those files and directories.

With Debian 11.3, for example, /var/fluentd  and /var/logs/td-agent were the two folders identified as needing permission changes.

To set these permissions, you may need to use the chgrp or chmod utilities. Your commands may look something like this:

chgrp td-agent /var/fluentd
chmod 775 /var/logs/td-agent

Startup

If you’re not using systemd to manage your FluentD installation, then you can start it up manually via this command:

fluentd -c <path/to/config/file> &

Of course, replace <path/to/config/file> with the path to the td-agent.conf file you created above.

If there are no error messages or if the status returned from systemd is active, then you can assume FluentD is up and running on your systems. It’s now waiting to meet conditions in your configuration file before taking action.

To test the listener configured for port 8888, you can use the following command:

curl -X POST -d 'json={"json":"test!"}' http://localhost:8888/debug.test

Next, check the FluentD logs to ensure the log was ingested:

tail -n 1 /var/log/td-agent/td-agent.log

You should see something like the following (with a current timestamp, of course):

2022-06-19 20:19:05.929908014 -0600 debug.test: {"json":"test!"}

FluentD is up and running!

Getting started with Falcon LogScale

Creating a LogScale account

The first step is to register an account with Falcon LogScale. There are tiers ranging from free trials to unlimited log ingestion. Choose the level that suits the needs of your organization. For this guide, you can get by with the Falcon LogScale Community Edition. Note: it may take up to two days to activate your new LogScale account.

Once you’ve created an account and logged in, navigate to the “Repository” list as shown in the image below:

Set up a LogScale repository

A “repository” in LogScale is the base layer for organizing data. You can add users, dashboards and logs to a repository. In the image above, we created a repository named “Tutorial” for this demo.

After you have created a new repository, you will need an ingest token to send logs from Logstash to your Falcon LogScale repository. The ingest token authenticates the log shipper, verifying that it is authorized to send logs to your Falcon LogScale repository.

When you navigate to your repository for the first time, you’ll be asked to create an ingest token. If you don’t create an ingest token at that time, you can always go to the Settings tab, then navigate to Ingest Tokens.

Integrating FluentD with Falcon LogScale

At this point in the guide, you should have:

• A FluentD instance working on a local machine or container
• A LogScale account
• A LogScale repository and ingest token

The next step is to start shipping logs from FluentD to LogScale. To do this, we need to add the Splunk Enterprise plugin to FluentD. To install a FluentD plugin, run one of the following:

• If you’re using a Debian-based system with td-agent, then run:

td-agent-gem install fluent-plugin-splunk-enterprise

• If you are using FluentD directly (a source install or non-Debian package), then run:

fluent-gem install fluent-plugin-splunk-enterprise

These two commands are equivalent, and only one needs to run. The right one for you depends on your FluentD installation type.

After the plugin installation completes, configure FluentD to use it. In the following example FluentD configuration file (td-agent.conf), the plugin is set to tail the Apache access logs and ship them to LogScale:

## read apache logs with tag=apache.access
<source>
  @type tail
  path /var/log/apache2/access_log
  pos_file /home/my-username/Documents/fluent/data.pos
  tag apache.access
  path_key filename
  <parse>
    @type none
  </parse>
</source>

## Mutating event filter
## Add a hostname field to apache.access tagged events
<filter apache.access>
  @type record_transformer
  <record>
    hostname "#{Socket.gethostname}"
  </record>
</filter>
# Ship logs to LogScale that are tagged apache.access
<match apache.access>
  @type                 splunk_hec
  host                  cloud.community.humio.com
  port                  443
  token                 <your-ingest-token>
  use_ssl               true
  ca_file               /etc/ssl/ca-bundle.pem
  <buffer>
    @type               memory
    flush_mode          interval
    flush_interval      2
    flush_thread_count  2
    overflow_action     block
    retry_forever       true
  </buffer>
</match>

For your use, you’ll need to specify the location for pos_file, which is the file on which FluentD will record the position it is read to. You’ll also need to set the token to the ingest token for your LogScale repository.

Also, you may need to change the ca_file field to use a different certificate. If you don’t have any certificates on your machine or image, then you may need to install a package like ca-certificates or create one using OpenSSL. The certificates should reside in the /etc folder but might be in /etc/ssl, /etc/pki, or some other custom location depending on your installation. Lastly, the host field may be different depending on your LogScale customer tier.

Now, assuming that Apache is running on your machine, use your browser to navigate to a web page on that server or use curl http://localhost/ to get the index. In your LogScale repository (for our demo, that’s the “Tutorial” repository), you’ll soon see how this action generated a log. It should look like this:

Here, we have a single log line highlighted in blue. Below it, we have a breakdown of the different fields that were present in this log entry as it was ingested by LogScale. We also see the hostname field that we added with the log transformer in our configuration. This field is not present in a standard Apache access log; FluentD added the field as it was processing the Apache logs.

Conclusion

In this how-to guide, we demonstrated how to configure FluentD to ingest logs from a single system and ship those logs to Falcon LogScale. 

In our example, FluentD tailed and transformed an Apache access log. We set up a LogScale repository and obtained an ingest token. Then, we provided the ingest token in our FluentD configuration, setting up FluentD to process our Apache logs and ship them to our LogScale repository.

This is where your journey of using FluentD begins. From here, you can use FluentD to shape and filter your system’s logs, and then ship them to LogScale for management and monitoring!