Corelight Demo Data Now in Falcon LogScale Community Edition
At CrowdStrike, we’re always looking for new ways to share the power of CrowdStrike Falcon® LogScale, our log management and observability solution. The latest advancement to Falcon LogScale, previously known as Humio, is adding Corelight demo data to the Falcon LogScale Community Edition.
What’s in the Corelight data?
Corelight demo data is a continuous loop of network data taken from a Corelight sensor. As a leader in open network detection and response (NDR), Corelight is a CrowdStrike technology partner. Their network sensors use the power of open-source Zeek to generate comprehensive network evidence logs, which are invaluable to cyber defense teams.
These logs cover a wide range of network protocols and activity, including DNS, HTTP, RDP, files, SSL, SSH, X509 and support Suricata IDS signatures, which can generate detection notices in Corelight logs.
Is this live data?
Corelight demo data was taken from a test lab environment over the course of about 74 minutes. It loops repeatedly, so users can see new data arrive, create live searches to trigger alerts and create a realistic experience when interacting with the data.
Does this data consume my Falcon LogScale Community Edition allowance?
No. This data doesn’t contribute to your Falcon LogScale Community Edition daily ingest allowance.
How do I access the data?
Corelight demo data is available on the main screen of Falcon LogScale Community Edition. After you log in to Falcon LogScale, look for “humio-organization-corelight-demo,” as seen below.
If you see the “Restricted Access” message above, follow these simple steps to adjust your account permissions.
Open the view “humio-organization-corelight-demo” and under Access Control in the left panel select Permissions, then choose Add and select your username.
Choose Select Role and on the next screen choose “Admin” and select “Confirm” to apply. This grants user Admin permissions to the view “humio-organization-corelight-demo.” Now if you click the Falcon logo in the top-left, you will return to the initial screen and the “Restricted Access” message will be gone.
If you don’t have permissions to make the above changes, please contact the “organization owner” for your Falcon LogScale Community Edition service, which is the person who initially registered for the service.
How to get started
Here’s a series of suggested actions to familiarize yourself with Falcon LogScale and the Corelight demo data.
Install Corelight Package From Falcon LogScale Marketplace
The Corelight data is already ingested and parsed in Falcon LogScale to ensure relevant fields are available (although in Falcon LogScale you can search without field names, as with Google).
A good way to start exploring the data is to install the Corelight Sensor package from the Falcon LogScale Marketplace. The package is named “corelight/sensor” and includes a comprehensive set of prebuilt dashboards that focus on different aspects of the network logs.
To install the Corelight Sensor package:
- Open the view “humio-organization-corelight-demo”
- Choose Settings from the top menu
- Select Marketplace from the left panel
- Select and install the “corelight/sensor” package
You can now use the dashboards, alerts and queries that have been installed by the package. As Falcon LogScale Community Edition comes pre-configured with a sample of Corelight data, you can disregard the “Installation” section of the package Readme, which guides you through how to ingest logs from your own Corelight sensor.
Check out the dashboards
Select Dashboards in the top menu and open the dashboard named “Corelight Files.” Select a time frame of 4 hours in the time picker in the top-right corner.
Notice all the text-based files and images, which you might expect. But we also have Shockwave Flash files, which may be unexpected, as few companies still use Flash. Let’s focus on those files to see what they’re doing in our environment.
Select the Mime Type parameter at the top of the dashboard in the center of screen, select the entry for “application/x-shockwave-flash” and apply the parameters. This recalculates some of the dashboard widgets to use only event data where: mime_type = “application/x-shockwave-flash”
If you scroll down to look at four of the dashboard widgets showing “Files Flow – Bytes Received,” “File Flow – Bytes Sent,” “Files Sent” and “Files Received,” it’s apparent these files are being received by, and not sent from, our network.
Let’s find out who’s receiving Flash files. This will take some searching, but don’t worry, it’s what Falcon LogScale does best!
Search for who/what is receiving Flash files
Click on Search in the top menu and type in the search bar: #path = “files” mime_type = "*flash"
In Falcon LogScale, there’s an implicit “AND” applied between two parts of a search, and * is a wildcard. This will find all events where the #path is “files” and the mime_type is anything ending in “flash.”
In the time picker, select “All Time” from the list of presets to find all instances of these Flash file downloads.
At the time of this search, we got 763 matches for “Flash.” (Note this number will depend on when you perform the search, as new events are constantly being added.)
Now let’s filter this data to focus on who or what is downloading these files. In Corelight, the Files logs have a field called rx-hosts[0]which tells us the IP address of the receiving host to which the file was transferred.
From the left panel showing the extracted field names, click on rx-hosts[0] and you’ll see something like the below, which displays all the different values of the field rx-hosts[0]
There are three different hosts receiving these Flash files. We now want to work out what protocol is used for these transfers and whether it’s from internal or external servers.
Find out how these Flash files are being received
Click on the field name “local_orig” and you’ll see for all events the value is always false. This means the origin of these files is not local and therefore they’re being downloaded by HTTP from a remote server.
We have three hosts downloading Shock Wave application files from the Internet via HTTP.
Visualize the connections
To visualize this data and see the relationship between the transmitting and receiving hosts — or to use the proper field names tx_host[0] and rx_host[0] — you can enter the following into the search bar of Falcon LogScale:
#path = “files” mime_type = "*flash" |sankey(target=tx_hosts[0],source=rx_hosts[0])
This runs the same query as before but pipes the result into the next part of the query. This creates a Sankey diagram of the data. You should get something like in the below screenshot.
Sankey diagrams are useful for showing the one-to-many and many-to-one relationships between different datasets (in this case transmitting and receiving hosts) where there’s a relatively small number of different values for each data (i.e., small number of hosts in this example).
If you think this is a useful diagram, save it as a dashboard widget to an existing or new dashboard by clicking “save as” then choose “Dashboard Widget.” You can then go back to the widget and change the time frame with the time picker from the top-left corner to match the time frame you’re interested in. You can also tick the Live Query option to have it update in real time.
Find where these files are downloaded from
Let’s see what we can find out about where these files are being downloaded from. Go back to the search without the Sankey diagram, as in the below screenshot. In the left panel if we click on the field name tx_host[0] you see the IP addresses that are transmitting these files.
IP addresses are useful, but we really want to know which URLs link to these IP addresses. This will tell us more about the site they’re being downloaded from.
Corelight logs have a category called HTTP, which is where we’ll find what URLs are resolved to the IP addresses. In this example we’ll take just one IP address: 74.125.232.124
Click in the search bar and type a new search of: #path = "http" id.resp_h = "74.125.232.124"
This searches the HTTP category of logs for the destination IP address 74.125.232.124 so we can find which HTTP sessions are connected to this IP address.
By clicking various field names in the left panel, we can explore more about the sites from which these files were downloaded, including:
- Referrer: this shows us the web page from which the actually GET request was made
- Uri: the full path to the downloaded file on the server
- User_agent: the user agent which is presented in the GET request and corresponds to which browser was used
As you can see, there are multiple sites that make the requests to transfer these files. And there appears to be three different types of user_agent used.
This helps understand the context behind these file downloads. We know which browsers, from which hosts were used to download Flash files from which websites, including the full uri path to the file. From here, we can determine whether this is something we deem acceptable or not.
How many different user agents are in use?
If you’re curious to see all the different user agents in use within this sample of logs, you can try the simple search: #path = "http" | groupBy("user_agent")
This pipes the results of the initial part of the search (all events in the HTTP category) into a groupBy function which groups by the user_agent and applies the default count function.
You’ll see a table of the count of each unique user_agent value. How many unique user_agent are there? It may depend on the time frame for your search. But if you set it to “all time” you should see 165.
Creating an alert
Let’s assume we don’t view this matter as a major problem, but we’d like to know if it continues to happen, especially on one of hosts, like 192.168.0.51. We can create an alert in Falcon LogScale that will trigger every time a new Flash file is downloaded by that host.
To do this, click on Alerts in the top menu and click New Alert. Give the alert a name and click continue (leave the Empty Alert option highlighted).
Now configure the alert per the below screen. Before you click Create Alert, add an Action to this alert by clicking Create One in the Action section.
You’ll notice in the search we used the full name of the potential Mime Type rather than the earlier approach of using *flash.” The asterisk is a wildcard and means that any result that ends in “flash” will match. For saved searches and alerts that are going to continually run, it’s good to be exact about these things. So we’ll specify the full value of application/x-shockwave-flash.
When you enter multiple field names in the Falcon LogScale search bar, there’s an implicit AND function applied. See the search query shown below. It will only match if the #path field equals “files” and the mime_type field is “application/x-shockwave-flash” AND the rx_hosts[0] field is ” 192.168.0.51“
In Falcon LogScale, an alert is the trigger. It looks for a match to the defined search. To make it useful, you need to add an Action and link it to the Alert. For this simple demo it’s easiest to define the Action within this Alert configuration by clicking the highlighted Create One.
Click on Create One in the Actions panel, select an email action from the drop-down and configure it as below with your email address. By clicking “Use Custom Email Subject” you can configure the subject line used in the email.
For simplicity, let’s stick with the default email template.
Click “Create Action” and it will confirm your action has been saved. Now click “Test Action” and click “Trigger Test Action”. The Action will run with the data configured under the Advanced Options. This tests the generation of email to you.
Now you’ll get an alert every time a new Flash file is downloaded by 192.168.0.51. And it will run continuously, looking at data in the last hour, to see if there’s a match. If there is, it will trigger the Action and you’ll receive an email alert. The throttle time set to one hour means the alert will only trigger once an hour and therefore you’ll only get one email an hour at most.
The email alert will look like this:
If you click the “Open in Humio” button and are logged into Falcon LogScale, a new search page will open with the search criteria of the alert, showing the matching events so you can investigate further if needed.
In summary
We covered a lot in this post. We:
- Installed a package and saw the value of pre-formatted dashboards.
- Used the dashboard to spot the presence of Flash files in the network.
- Searched and filtered results to find what hosts are downloading those files and from which websites and which browsers (user agents).
- Saw how a simple Sankey diagram can help represent relationships — between senders and receivers in this case.
- Created an alert and an email action to identify when our search gets new hits.
This is just a snippet of what you can do with Corelight demo data in Falcon LogScale. We hope you enjoyed this guide and found it useful. We look forward to sharing more exercises with you in the future to highlight more ways to interact with the data and fuel your curiosity.
