Elasticsearch, Logstash, and Kibana – The ELK stack

Conclusion

The ELK stack is not just useful for web or mail servers, where you can expect high hit counts, but also for large server clusters with distributed logs. Elasticsearch, Logstash, and Kibana are team players that collaborate excellently, and they are capable of integrating more components (e.g., services like Filebeat) into the team. The current program versions impress across the board and certainly give administrators a powerful toolbox.

Although the installation was a painless affair, the ELK stack does lack state-of-the-art systemd units in part, and even an init script in the case of Kibana. We also missed meaningful Logstash default configurations for the services on a typical Linux server. Although the documentation is very exhaustive, with many examples by other users on the web, it is a pity that system administrators first need to compile the information they need painstakingly. The commented examples on the FTP site [6] should be of help here.

The ELK stack is unbelievably flexible, but you can expect a lengthy learning curve. Many paths lead to a process chain. If the developers were to provide a basic set of configuration examples, they would help many admins and could help achieve initial results quickly, provide better orientation, and even allow admins to develop their own style.

The Author

Christian Rohmann is part of the DevOps team at NetCologne, an Internet service provider for the Cologne, Bonn, and Aachen area of Germany. Christian implemented a complete ELK stack there, which he uses above all to analyze and evaluate the Postfix, Dovecot, Apache, Nginx, and Open-Xchange logfiles.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Logstash

    When something goes wrong on a system, the logfile is the first place to look for troubleshooting clues. Logstash, a log server with built-in analysis tools, consolidates logs from many servers and even makes the data searchable.

  • Perl: Elasticsearch

    The Elasticsearch full-text search engine quickly finds expressions even in huge text collections. With a few tricks, you can even locate photos that have been shot in the vicinity of a reference image.

  • Tutorials – Collectd

    The collectd tool harvests your system stats and stores them for plotting into colorful graphs.

  • Perl – Elasticsearch

    Websites often offer readers links to articles about similar topics. Using Elasticsearch, the free search engine, is one way to find related documents instantly and automatically.

  • FAQ

    Big data is like The Matrix – Better without the sequel

comments powered by Disqus

Direct Download

Read full article as PDF:

Price $2.95

News