What is and Why use Logstash?
At my place of employment we have recently set up Zabbix for our infrastructure monitoring. To complement that system and allow easier and faster diagnosis of problems detected we have looked into building a centralized logging system for our servers and applications. This has been an ongoing search but we never found something that truly fit our needs. Recently, I came across Logstash which uses a program written in JRuby which operates like many Unix style programs. It is described to be similar to
sed. Logstash is designed to chain a number of filters together to process an input source and output to many different places. One of them is to Elasticsearch which allows for easy searching, pattern matching and even correlation without needing to dump the entire system in a backend SQL database which is often slow and cumbersome to use on unstructured data like log files are.
This post will be a set of steps to install and configure a Centralized install of Logstash. It will be tested with Apache access logs.