Start Logging Everything: Humio Community Edition Series

See how easy it is to find insights with Humio Community Edition in this step-by-step guide

This blog was originally published January 24, 2022 on humio.com. Humio is a CrowdStrike Company.

In this blog, we’ll show you, step by step, how to download stock data and then upload it to Humio. You can then search that data and build a dashboard for fast insights. Subsequent blog posts will expand on this dashboard and show you how to move from analyzing historical data to live data. To get started, you’ll need to access Humio Community Edition, which is available at no cost.

Step 1: Download stock price data

After you’ve created an account for Humio Community Edition, you’ll need a dataset to analyze. While the possibilities are endless, for this blog post, we’ll use the Python script below to download stock prices for the past week.

Copy the script below and save it where you can execute a Python script.

#!/usr/bin/Python3

from datetime import datetime
import yfinance as yf
import json
import sys

# Take ticker symbols from the command line arguments
tickers = sys.argv[1:]
stockData = []

for ticker in tickers:

data = yf.download(tickers=ticker,period="7d", interval = "1m", progress=False)
data.index = data.index.astype(str) #convert from DateTime to string
jdata = data.to_dict(orient='index')

for item in jdata:

#add ticker and timestamp to flatten data

jdata[item].update( {"ticker":ticker} )
jdata[item].update( {"MyTimestamp":item} )
stockData.append(jdata[item])

for item in stockData:

print(json.dumps(item))

If this is your first time working with the yfinance Python module, you’ll need to install it.

$ pip3 install yfinance

You can execute this script like this:

$ stocks.py CRWD > crwd.json

Essentially, this command takes stock ticker symbols as command line arguments and prints the price data for the past week JSON format. The command above gets the price data for CrowdStrike, and then saves it to a file called ‘crwd.json.’

Check the data from the command line to verify it worked:

$ tail crwd.json 

{"Open": 268.989990234375, "High": 269.17999267578125, "Low": 268.9599914550781, "Close": 269.1700134277344, "Adj Close": 269.1700134277344, "Volume": 81464, "ticker": "CRWD", "timestamp": "2021-11-01 15:59:00-04:00"}

{"Open": 269.1700134277344, "High": 269.1700134277344, "Low": 269.1700134277344, "Close": 269.1700134277344, "Adj Close": 269.1700134277344, "Volume": 0, "ticker": "CRWD", "timestamp": "2021-11-01 16:00:00-04:00"}

Since you’re running this at a different time, it won’t have the same values, but it should have the same data structure.

Now, to accommodate this data, we’re going to quickly assemble a parser.

Step 2: Parse the data

Humio does not require data to be parsed, and it can easily allow you to store and search unstructured data. But for the sake of the exercise, we want to be able to graph this data, so it’s best to start by parsing the various fields. Because the data is in JSON format, this should be fairly straightforward.

While in Humio Community Edition, click into your repository (you named this when you signed up) and then click Parsers at the very top menu and then the + New Parser button. Name this new one “stocks” and then click Create Parser.

For this parser, we’re copying the default JSON parser with minor modifications:

parseJson() | findTimestamp(field=MyTimestamp, timezone="America/New_York")

This tells the parser to read the field labels from JSON and where to find the timestamp.

Step 3: Create an ingest token

The next step is to prepare Humio to accept this data. We need to create an ingest token and assign it to the Parser we just created.

From the top menu, select Settings. Then, from the left-side menu, click Ingest Tokens. Click the + Add Token button. Set the Token Name to “StocksJSON” and then from the Assigned Parser drop-down, select the “stocks” parser you just created.

From the Ingest Tokens page, click the eye icon next to your newly created token to reveal the token key. Click the copy button.

Step 4: Send the data to Humio

Now that we have our ingest token, we can ship our stock data to Humio.

From the command line, run the following curl command, but paste in your ingest token from above.

curl https://cloud.community.humio.com/api/v1/ingest/hec/raw -X POST -H "Authorization: Bearer TOKEN" -T "crwd.json"

Step 5: Verify the data

You can quickly see if any data has arrived from your Settings page by selecting Data Sources from the left-side menu. It should look just like this.

We can see the data has been collected here. But let’s explore it directly and start working with it.

Step 6: Explore the data

From the top menu, select Search. You should see data from today, but click the time picker on the top and select “Last 7 days.”

(Click to enlarge)

Now, we can plainly see our data in the main pane but also the various fields that have been parsed on the left side panel. Let’s start by finding the week’s high. From that left side panel, select ticker. From the field pop-up box, you’ll see the CRWD row. From there, click the `=` button. The search bar at the top will now read:

ticker = CRWD

Since we only have one stock, this doesn’t change our returned dataset in any way. But assuming you may add additional ticker datasets, we’ll include it now. Similarly, if you are working with separate datasets, you may want to include Type, which will have the name of the parser used. You can select it from the menu just like we did with ticker or manually edit the Search to read as shown below. You can type the following query as one line, but for legibility, I’ve used two. You can use shift-enter while typing in the Search bar to move to the next line (and then press enter to execute it).

ticker = CRWD

| #type = stocks

If this is your first dataset imported, nothing will have changed with these search filters, but if you add additional data sources later, it will ensure you’re still focused on this one.

Step 7: High/low queries

Now, let’s start calculating some interesting stats from our data. To find the week’s high, run the following search:

ticker = CRWD

| #type = stocks

| max(High)

To stylize this result, use the drop-down in the upper left corner to select Gauge.

Now that we have a data point of interest, let’s save it to a dashboard. On the right side of your screen, find and click the Save as… button. Select Dashboard Widget.

Fill out the dialogue box to add this to the “Stock” dashboard you created previously, then add a Widget Title and click Save.

This will bring you to your new dashboard.

For the sake of symmetry, let’s round out this dashboard with the week’s low as well. Return to the Search page and enter this query.

ticker = CRWD

| #type = stocks

| min(Low)

Once again click Save as… and then Dashboard Widget and name this appropriately. It should default to selecting the Dashboard you already created.

Your dashboard should now look like this:

Step 8: Graph the stock price

Now let’s graph this stock’s price over the week. To do this, we will use the timeChart function, which allows us to bucket up the data we’re analyzing and apply a function to it. For simplicity’s sake, we will use one day as the bucket span. Since we’re interested in the close price at the end of the data, the function we’ll use is SelectLast to choose the latest value in that bucket. Enter the following query and ensure that you change the time picker to the Last (7d) seven days.

#type=stocks |ticker="CRWD"

| timeChart(series=ticker, function=SelectLast(Close), span=15m)

Our graph looks like this:

If you mouse over any data point you can view the data for that day.

Save this widget to your new dashboard!

We named this widget “Price Chart.” In this case, we unchecked Open dashboard after save and then clicked Save.

This leaves me at the same search screen. But given the setup here, quickly edit to extend our analysis here by simply changing “Close” to “Volume” in the query window.

Save this as a widget named “Volume Chart.”

Your dashboard should now look just like this:

Additionally, we can do some extra work to enhance the visibility and aesthetics. In the upper right corner, click the pencil icon. Now you can rearrange your widgets. Start by dragging the Price Chart widget to the top. Then drag the edge to the right to resize it across the page. Drag the Volume widget up to below the price one, and resize it to the right edge as well. The price is far more interesting, so drag the Price widget down to enlarge it, and resize the Volume widget to be shorter.

Finally, move the High and Low widgets to the bottom and resize them to be a bit smaller.

Looking at the dashboard now, it’s quite functional, but let’s bring more contrast between the price and volume data, aside from their relative size. In the upper right of the Price widget, click the three vertical dots and then Edit Style. From the right side panel, scroll to the bottom and expand the Series menu. In the Field text box, enter “CRWD.” Click the next box (which by default says AUTO) to open the color selector. I chose red.

Then, to highlight the value differences better, find the Min Value field and put something under the Low value we discovered earlier.

Finally, click the Save button. Here’s the final dashboard.

One final thing to check: When you save a query to a dashboard widget, it will keep the time frame you used in the original query. You can use this to have different widgets looking at different time frames. If you wish to have an overriding time frame, click the slider button next to Shared Time and choose the time window you want. It will then be applied to all widgets on the page.

Thank you for checking out Humio Community Edition! In our next post we’ll extend the dashboard and make it flexible enough to handle multiple stocks. See you then.

Additional Resources

  • Build your skills with Humio Community Edition by visiting The Nest
  • Join the Humio Community Edition quickstart workshop
  • Register for our six-part log management course to learn advanced observability skills
Related Content