Humio at FreeAgent: Transforming Fintech with Observability and Empowering Transition to Cloud Hosting

This blog was originally published on humio.com. Humio is a CrowdStrike Company.

“There are a lot of use cases — everything from support asking day-to-day questions to running reports, looking for trends, pattern matching, and alerting. It’s been quite transformative!” – Steve Williamson, Ops Team Lead, FreeAgent

The Company

FreeAgent provides accounting software for small businesses in the UK, to simplify accounting processes including payroll, taxes, and budgeting. Simply put, they demystify the financial side of running a small business. Their cloud-based accounting software helps more than 90,000 small businesses take care of their finances, and enables accountants and bookkeepers to manage their clients at a glance. The company was acquired by the Royal Bank of Scotland Group in early 2018, yet they remain operationally independent of the group.

FreeAgent leverages the power of software to simplify processes and organize streaming financial data. This helps inform the decisions of their users and liberates them from time spent on repetitive maintenance tasks.

When FreeAgent transitioned to Humio, their system was running on a unique self-hosted system based on Elasticsearch and Kibana, using containers for performance optimization.

The Challenge

Steve Williamson is the Ops Team Lead at FreeAgent. He describes what drove his company to look for a new log management solution.

“Elasticsearch was all right when looking for a very specific recent thing. What it was not suited for was analytics across all of the data. Breaking down all the data in Elasticsearch and seeing what’s generating the most logs per second – you just couldn’t do it.”

FreeAgent was running into performance issues on their self-hosted cluster. They couldn’t run the custom scripts they wanted to find answers to their questions without causing the cluster to fail. They wouldn’t even attempt to answer many other questions because they didn’t want to risk crashing it and spend significant time digging through Elasticsearch documentation to see what it would take to get their cluster up and running.

The Search

When FreeAgent went looking for a new solution to replace Elasticsearch, they wanted to be able to ask historical questions of their data. Humio was one of the only options that rose to the challenge. Humio’s advanced data compression of 10-20x reduces storage costs and makes it the most affordable option for users who need to store and search many months of logs. Steve explains how it caught his eye:

“Humio stood out because storing six months of data wasn’t an issue and it didn’t phase you. On some other solutions we looked at, if you go over two to three weeks of log data, they aren’t too keen on that.”

After becoming interested in Humio’s storage capacity, FreeAgent advanced to the next step, running a free trial of Humio in parallel to Elasticsearch. Initially, Steve was worried about users not wanting to even try Humio, but he found instead that employees were refusing to give it up.

In addition to meeting storage needs and boosting productivity immediately, Humio showed a responsiveness to FreeAgent’s needs and an exceptional level of support and communication that earned their trust. When notifying Humio of small bugs, they initially didn’t expect much of a response. Steve was pleased to discover that engineers at Humio would not only respond, but they would fix the bugs and have a new deployment out in an hour.

“It was nice to work with someone who offered more of a partnership than a buyer-seller relationship. One of the things that swung Humio was that transparency.”

The Solution

Migrating out of a performance necessity, Steve and his team at FreeAgent were pleasantly surprised by the transformative impact of Humio’s instant access to live and archived data.

“We weren’t interested in creating dashboards and metrics out of it initially. It’s not something we could do with Elasticsearch previously, but once we had that capability it was super useful. It’s been invaluable.”

Now about 100 people in a wide variety of departments at FreeAgent use Humio for a variety of purposes. By enabling users to ask questions of their databases, Humio has empowered a culture of curiosity at FreeAgent which fuels internal innovation and the ability to adapt to business needs.

Since moving to Humio, FreeAgent has also begun to migrate their mature self-hosted platform to AWS, which they might not have been able to do without Humio. They’ve entered a complex environment where it is near impossible to use the old solution of SSH into a box to detect root causes. Humio provides log-powered dashboards where users can detect root causes of problems in seconds, even in a container-rich environment full of microservices.

“If something goes wrong, you need to be able to observe. When you have hundreds and hundreds of containers, that process doesn’t scale anymore. Metrics and logs and building dashboards out of logs is critical.”

Request a Demo

Hear the full story of FreeAgent and their transition to Humio by listening to our podcast.

Want to see the difference observability can make for your system? Want to see if you can answer questions you’ve been asking for months in minutes? Sign up for a free Humio trial.

Related Content