What is log analysis? Well, the simplest way to describe it would be monitoring. Logs can show a lot of information about your website and how your visitors are using your site. Some logs track visitors, their path through the website and their actions on the site. There are a number or tools that can help you to do this.
From log analysis to log storage, this week I take a look at six free tools that will help you manage logs within your organization. Log analysis is where the rubber meets the road for everything you’ve done in security so far, so these tools are essential for your tool belt!
Graylog
Graylog is a popular Elasticsearch-based open-source log management and analytics tool. It has a multi-threaded architecture, distributing your search across several nodes in a cluster. This architecture allows it to process a large volume of logs and quickly provide search results. You can perform full-text search queries, and there’s no need to learn a proprietary query language. Graylog also offers various “Content Packs” to simplify complex configurations. These packs are downloadable from the Graylog Marketplace. Furthermore, you can visualize your logs using various widgets and create stunning dashboards. In these visual dashboards, you can combine different data points in a single chart to efficiently analyze your logs. Graylog Open Source is sufficient for most purposes; however, you can also explore the paid version. The paid version offers several advanced features, including a correlation engine designed to prevent security breaches.
Sematext Logs
Sematext Logs is a cloud-based SaaS log management solution that offers you real-time actionable insights into your log analytics with hosted ELK as a service, in the cloud or on-premises. It’s compatible with a large number of log shippers, logging libraries, platforms, and frameworks, being able to aggregate logs from a wide variety of sources.
Sematext’s auto-discovery of logs and services lets you automatically start log monitoring from both log files and containers and forwarding them directly through the user interface.
Sematext Logs features sophisticated full-text searching, filtering, and tagging capabilities and allows you to correlate logs with infrastructure and application metrics within a single dashboard. You can set up alerts on both log data and metrics and use Live Tail to see new errors as they’re logged in real time. We offer both free and paid plans.
We offer both free and paid plans. Our commercial plan starts at $50 per GB per day for 7-day retention and you can try it free for 14 days.
You can also read a review of Sematext Logs on Network Admin Tools, amongst other log management solutions.
SolarWinds Loggly
Loggly is a cloud-based log management service that is appreciated for its logging analysis capabilities. It enables you to use traditional standards like HTTP or Syslog to collect and understand logs from a variety of data sources, whether server or client-side.
With its dynamic field explorer, you get a real-time overview of your logs categorized by structure or by customized view. Loggly has powerful full-text search capabilities featuring searches by individual fields, booleans, ranges, and more. Its interactive, ready-to-use dashboard provides performance indicators and metrics that allow you to spot trends as well as performance issues and compare data across a given timeline.
Loggly offers a free version and three paid plans starting with $79, $159, and $279 respectively. A 14-day trial is available for evaluation.
ELK Stack
The ELK stack (or the Elastic Stack) is a combination of three commonly used open-source tools: Elasticsearch, Logstash, and Kibana. The ELK stack has become highly popular because it offers a high level of deployment flexibility; you can choose to install it in the cloud or to use on-premises servers. Elasticsearch’s ability to search through large volumes of logs using its distributed architecture is well known. Combined with Logstash and Kibana, it’s a powerful solution capable of meeting log management and analysis needs for organizations of all sizes and domains. However, calling it a free solution is a little misleading. The costs of hosting, staffing, and managing the ELK Stack can be exorbitant for small businesses. Configuring the stack and extracting true value out of the solution isn’t easy.
Splunk
Splunk is one of the most well-known log monitoring and analysis platforms, offering both free and paid plans. It collects, stores, indexes, correlates, visualizes, analyzes, and reports on any type of machine-generated data, whether it’s structured, unstructured or sophisticated application logs, based on a multi-line approach.
When using the tool, you can search through both real-time and historical log data. Splunk allows you to set up real-time alerts where automatic trigger notifications can be sent through email or RSS. You can also create custom reports and dashboards to better view your data and detect and solve security issues faster.
Splunk is available free of charge and supports one user with up to 500 MB per day. If you need more complex features, they do offer two paid plans. Pricing is available upon request in that case, though.
Octopussy
Octopussy is another free and open-source log analyzer popular among IT professionals. It helps you analyze logs from different networking devices (routers, firewalls, load balancers, etc.) and all their applications and services supporting the syslog protocol. The tool sends alerts via email and some open-source instant messengers. With this tool, you can create maps to visualize your architecture and graph syslog activity. The tool can also generate and schedule reports with some plug-ins. Octopussy can be a useful tool for teams looking for a free solution to prevent system outages, security threats, and application errors.
Logentries (now Rapid7 InsightOps)
Rapid7 acquired Logentries and added it to its line of security- and automation-focused products. Renamed InsightOps, the tool is a cloud-based log management platform that also includes easy-to-use analytics tools so that you can monitor data trends and correlate events across your system.
InsightOps allows you to query data in real time with aggregated live-tail search to get deeper insights and spot events as they happen. it also features custom alerts that push instant notifications whenever anomalies are detected. You can then add custom tags to be easier to find in the future and analyze your logs via rich and nice-looking visualizations, whether pre-defined or custom.
InsightOps is available for both free and commercial use. The paid version starts at $48 per month, supporting 30 GB for 30-day retention.
Conclusion
Network security is the first thing that comes to mind when we say “log analysis”. However, log analysis is useful far beyond the cyber world. Logs are useful for detecting abnormal usage behavior, detecting network crashes/outages before your customers report them, detecting application issues like 500 errors or timeouts, detecting brute force attacks (DDoS), detecting suspicious events (like breach attempts, illegal access attempts etc.), identifying vulnerable hosts on your network and many other things.