SR Linux logging with ELK
Join the discussion: LinkedIn post · Twitter thread
In a not-so-distant past, manually extracting, parsing, and reading log files produced by network elements was standard practice for a sysadmin. With arcane piping of old-but-good grep, awk, and sed tools, one could swiftly identify a problem in a relatively large system. This was a viable approach for quite some time, but it became prey to a massive scale.
Today's network infrastructures often count thousands of elements, each emitting log messages. Getting through a log collection of this size with CLI tools designed decades ago might not be the best tactic. As well as correlating logs between network elements and application logs might be impossible without software solutions built with such use cases in mind.
The unprecedented growth in the application world boosted the development of multi-purposed centralized/cloud data collectors that make observability and discovery over huge data sets a reality. Elasticsearch / Logstash / Kibana (or ELK for short) is one of the most known open-source stacks tailored for the collection and processing of various documents, logs included.
To enable the processing of captured logs and deliver performant and robust search analytics log collectors rely on structured data. Unfortunately, the networking world is infamous for iterating slowly. For example, an outdated and informational Syslog interface still dominates the networking space when it comes to managing and transferring logs. Syslog RFC31644 was not designed to allow extensible structured payloads, which adds a fair share of problems with integrating such systems with modern log collectors.
This post explains how an SR Linux-powered DC fabric can be integrated with a modern logging infrastructure based on the Elasticsearch / Logstash / Kibana stack to collect, transform, handle, and view logs.