In part one of this Node.js logging series, we learned that log files provide valuable information about system performance, events such as requests and data transfer, system and application errors, and more. In part two, we discussed advanced logging concepts and introduced logging libraries—such as winston, pino, and loglevel—for efficient logging.

However, for better understanding and visualization, logs should be gathered in a suitable, central location. This is when centralized logging becomes valuable. In this article, we’ll explore centralized logging and its use with default logging handlers in Node.js.

The Importance of Centralized Logging

Node.js applications serving thousands of users can generate a large number of logs. If multiple instances of the same application run on different servers, each instance will generate its own logs. Centralized logging helps gather logs from various systems, networks, and applications into one place for storage and analysis. It also standardizes logs from different microservices with different log formats for consistency and interoperability. This information helps uncover technical issues, improve performance, and strengthen security.

Several concepts related to centralized logging are worth discussing, including the methods for capturing logs, log transformation for consistent storage, and the logging backend.

Let’s look briefly at each of these concepts within the context of a Node.js application.

Capturing logs

The logs generated on specific machines are gathered by a service and then transported to the centralized location. For example, logs can be created in a web application using server-side logging libraries (such as winston or pino) and captured using HTTP requests. For logging to the console or to a file, use winston for various transports.

The logging configuration below sends application messages to the console and a log file specified in the config.

const winston = require('winston');
logger.add(new winston.transports.Console());
logger.add(new winston.transports.File({
  filename: "logs/app.log"
}));

Log transformation

Logs may have different formats and levels of detail when collected from different parts of an application. Therefore, to ensure consistent and effective storage, logs must be standardized by using a log parser or shipper. This shipper extracts relevant information from logs and then converts it to a standard format, such as JSON. This process also includes adding metadata to the logs (such as timestamps, hostnames, and severity levels) to make it easier for analysis and search.

{
  "message": "This is an info message",
  "level": "info",
  "timestamp": "2022-12-08T15:52:38.015Z",
  "version": "1.10",
  "hostname": "admin"
}

The Logging backend

The logging backend can be a file system, a database, or a dedicated logging platform, such as CrowdStrike Falcon LogScale. Organizations often choose their logging backend based on the  size and complexity of the application, the volume of logs generated, and the requirements for performance, reliability, and scalability.

The logging backend should provide the necessary capabilities for storing, indexing, querying, and visualizing logs. These capabilities make it easier to understand issues with the deployed systems, analyze trends, and identify threats.

Implementing Centralized Logging in Node.js

Node.js has a built-in logging facility with console.log(), but it has limited capabilities and reduces your system’s performance. To overcome this, libraries such as winston and pino can be used for logging within Node.js applications.

In Node.js, logging handlers process log messages and perform actions, such as printing them to the console, writing them to a file, or sending them to a remote server. Node.js provides several types of logging handlers. Let’s cover the main handlers, one at a time.

Console handler

The console handler logs messages to the console using the console.log() method, allowing the viewing and monitoring of log messages in real time without additional tools or processes. Below is an example of a console logging handler in Node.js using the winston logging library:

const winston = require('winston');

const logger = winston.createLogger({
  level: 'info',
  format: winston.format.json(),
  transports: [
    new winston.transports.Console()
  ]
});

Socket handler

The socket handler logs messages directly to the socket.io server. Below is an example of a socket logging handler in Node.js using the winston logging library:

const winston = require('winston');
const fs = require('fs');

let logger = winston.createLogger({
    level: "info",
    transports: [
        new winston.transports.SocketIO(
            {
                host: "myhost",
                port: 8080,
                secure: true,
                reconnect: true,
                namespace: "log",
                log_topic: "log"
            }
        )
    ]
});

Note that the socket handler is configured to stream log messages to myhost:8080, and assumes the host listens for messages via this port.

Stream handler

A stream handler logs messages to a stream, such as a writable file stream or a network socket stream, allowing the stream to log messages directly without intermediate buffering. Below is an example of using a stream logging handler in Node.js using the winston logging library:

const winston = require('winston');
const fs = require('fs');

const logger = winston.createLogger({
  level: 'info',
  format: winston.format.json(),
  transports: [
    new winston.transports.Stream({
      stream: fs.createWriteStream('logs/combined.log')
    })
  ]
});

File handler

A file handler logs messages to a local file system by sending persisted log messages to disk for later access and analysis. Below is an example of using a file logging handler in Node.js using the winston logging library:

const winston = require('winston');

const logger = winston.createLogger({
  level: 'info',
  format: winston.format.json(),
  transports: [
    new winston.transports.File({ filename: 'logs/combined.txt' })
  ]
});

HTTP handler

An HTTP handler logs messages to a specified host and path. This is beneficial when sending log messages to a remote logging service, such as a centralized logging platform. Below is an example of using an HTTP logging handler in Node.js using the winston logging library:

const winston = require('winston');

const logger = winston.createLogger({
  level: 'info',
  format: winston.format.json(),
  transports: [
    new winston.transports.Http({
      host: 'logs.example.com',
      port: 80,
      path: '/logs'
    })
  ]
});

After configuring the logger, use it to log messages with the logger.log() method, like this:

logger.log('info', 'Hello, world!');

CrowdStrike Falcon LogScale as a Centralized Logging Solution

The handlers discussed above transport the generated logs into centralized locations for further processing. CrowdStrike Falcon LogScale is a centralized logging solution that helps businesses collect, store, and analyze log data from various sources—such as servers, applications, and devices. LogScale offers a scalable, high-performance platform for managing large volumes of log data, including real-time searching, data archiving, and customizable alerting and reporting.

LogScale provides a JavaScript logging client for easy integration into a Node.js application. This logging library is distributed via npm.

Install the Humio JavaScript client with the following command:

npm install -S humio

Then, import the package and create a logging instance. We need an apiToken and ingestToken to authenticate with the Falcon LogScale server. To obtain these, first create a CrowdStrike Falcon LogScale account.

After creating an account, generate a new API token and an ingest token. Then, after generating the tokens, you can initialize the Humio JavaScript client in your Node.js application using the following code snippet:

const Humio = require("humio");
// create a new client
const logger = new Humio({
    apiToken: "<API_TOKEN>",
    ingestToken: "<INGEST_TOKEN>",
    host: "<HOST_NAME>",
    repository: "<REPOSITORY_NAME>",
});

After creating a logging instance, you can send logs to the server or run a query to find specific logs. For example, the code snippet below illustrates how to log a JSON message and run a query to search for info-level logs for the past hour.

logger.sendJson(logData);
logger
    .run({
        queryString: `level = "info"`,
        start: "1h",
        end: "now",
    })
    .then((res) => {
        console.log(res.data.events);
    })
    .catch((err) => {
        console.log(err);
    });

For further information on using the various logging queries and other features, consult the Humio library documentation.

Conclusion

A central system for gathering and storing log data enables businesses to monitor, analyze, and troubleshoot problems more efficiently. Systems with Node.js applications deployed produce a variety of logs that can be processed using different logging handlers and sent to a centralized logging system. A logging library such as winston enables centralized logging in a Node.js application. 

The logs generated using different logging handlers configured in a Node.js application can be pushed to CrowdStrike Falcon® LogScale as an efficient centralized logging solution.

In the next article of the Node.js logging series, we’ll learn about centralized logging in Express applications with the help of example applications, and we’ll discuss the challenges in logging with application frameworks. 

For further information on using the various logging queries and other features, consult the Humio library documentation.

Log your data with CrowdStrike Falcon Next-Gen SIEM

Elevate your cybersecurity with the CrowdStrike Falcon^® platform, the premier AI-native platform for SIEM and log management. Experience security logging at a petabyte scale, choosing between cloud-native or self-hosted deployment options. Log your data with a powerful, index-free architecture, without bottlenecks, allowing threat hunting with over 1 PB of data ingestion per day. Ensure real-time search capabilities to outpace adversaries, achieving sub-second latency for complex queries. Benefit from 360-degree visibility, consolidating data to break down silos and enabling security, IT, and DevOps teams to hunt threats, monitor performance, and ensure compliance seamlessly across 3 billion events in less than 1 second.

Schedule Falcon Next-Gen SIEM Demo