This post dives deeper into the asymmetric routing model1 for EVPN VXLAN fabrics on SR Linux. The topology in use is a 3-stage Clos fabric with BGP EVPN and VXLAN, with
server s1 single-homed to leaf1
s2 dual-homed to leaf2 and leaf3
and s3 single-homed to leaf4.
Servers s1 and s2 are in the same subnet, 172.16.10.0/24 while s3 is in a different subnet, 172.16.20.0/24. Thus, this post demonstrates Layer 2 extension over a routed fabric as well as how Layer 3 services are deployed over the same fabric, with an asymmetric routing model.
SR Linux (aka SRL), released back in 2021, is a new operating system from Nokia, designed to power data center fabrics, with network automation no longer being treated as a second-class citizen. SR Linux is built from the ground up using YANG, which is a modeling language describing how data is structured. As an operator, this enables you to view the entire structure as a schema tree (which we will see shortly).
Like any new operating system, there is a learning curve. In the past, I have had to learn several new operating systems (having originally started with Cisco IOS), including Cisco IOS-XE/NXOS, Arista EOS, Cumulus NCLU/NVUE, Juniper Junos and now, Nokia SR Linux. In general, I have always followed the same methodology in learning - learn by building something relatable. Since SR Linux focuses on data center fabrics, we're going to build something a little relatable to that. Let's dive in.
I still remember the day when we announced general availability of the SR Linux container image that everyone could just docker pull and start building their dream labs:
🚨This is not a drill🚨 🥳This day has come and I'm delighted it has happened on my birthday.
🚀Nokia makes its Data Center NOS - SR Linux - available to everybody without any regwall, paywall, licwall or any other wall typical for a vendor
The availability of a free, lightweight and fast-to-boot containerized NOS served as a catalyst for the community to start building labs as code and use the image in the CI pipelines as it was easy and quick to run it on the free runners. However, the container image was only available for x86_64 architecture, and as a result for a long time we were saying that running SR Linux on macOS, for instance, was a "no-go".
It was not only about macOS, though. The rise of ARM-based server systems also made it hard to say that SR Linux can run on any compute you might have in your possession. I would lie if I say that we had RaspberryPi in mind, but hey, people run all kinds of workloads on rPI, why not networking labs?
And, finally, the day has come! We are happy to announce that the SR Linux container image is now available as a preview for ARM64 architecture, and is ready to be used on any ARM64 system, including devices with Apple M chips.
The first preview release is distributed via the same ghcr.io/nokia registry, but as long as we are in the preview cycle, we will use a separate tag for it:
There is a lot to be said as to how SR Linux labs powered by Containerlab can be run on ARM64 systems, and to make it more interactive, I recorded a video about it:
Put those performance cores to work, and lfl (let's *ucking lab)! 🚀
A network digital twin can let engineers test changes before they get pushed into production. In this Video Byte, we discuss how to Nokia's Containerlab, in tandem with NetBox, to build a digital twin. Containerlab is an open-source tool that lets you run containerized versions of network operating systems and build network topologies. And by pairing with NetBox, a network source of truth, you can pull actual device configurations into your Containerlab environment to build your digital twin.
The Packet Pushers' Ethan Banks talks with Tim Raphael from Nokia and Mark Coleman from NetBox Labs on how make this happen.
Once in a while you need to take a closer look at the traffic that flows through your network. Be it for troubleshooting, monitoring, or security reasons, you need to be able to capture and analyze the packets.
Doing the packet capture in a virtual lab is a breeze - pick your favorite traffic dumping tool and point it to the virtual interface associated with your data port. But when you need to do the same in a physical network, things get a bit more complicated. Packets that are not destined to the management interface of your device are not visible to the CPU, and hence you can't capture it directly.
That is where the mirroring feature comes in. It allows you to copy the packets from a source interface to a mirror destination, where you would run your packet capture tool. By leveraging the ASIC capabilities, the mirroring feature is hardware-dependent, but luckily, SR Linux container image is built with mirroring support, so we can build a lab and play with mirroring in a close-to-real-world environment.
Since the inception of our Data Center Fabric program in 2019 we have been focusing on EVPN-based deployments as the preferred choice for data centers of all sizes. And historically, EVPN has been associated with Layer 2 services, such as VPLS, VPWS, E-LAN. However, network engineers know it all too well that BGP can take it all, and over time EVPN grew to support inter-subnet routing, and subsequently, layer 3 VPNs.
Now you can deploy L3 VPN services with EVPN, both in and outside of the data center. Yes, a single control plane EVPN umbrella can cover all your needs, or at least most of them.
It was important for us to start with L2 EVPN basics and cover the EVPN origins first, but now more and more workloads ditching the arcane requirement to have layer 2 connectivity, and more and more data centers can be built with pure layer 3 services.
But Layer 3 EVPN services have many flavors... Some, such as RT5-only EVPN, are quite simple, while others offer more advanced features and require symmetric IRBs, SBDs, Interfacefull mode of operation, and ESI support. To ease in the L3 EVPN introduction we chose to start with the simplest form of L3 EVPN - RT5-only EVPN.
To introduce you to the concept of L3 EVPN we prepared a comprehensive tutorial - RT5-only L3 EVPN Tutorial - that covers gets you through a fun lab exercise where you will configure a small but representative multitenant L3 EVPN network:
You'll get exposed to many interesting concepts, such as:
eBGP Unnumbered underlay to support the overlay services
iBGP overlay with EVPN address family
RT5-only EVPN service configuration for L3 workloads
EVPN service with BGP PE-CE routing protocol to support clients with routing on the host
So, have your favorite drink ready, and let's have our first dive into the world of L3 EVPN!
Pure L3 EVPN fabrics in the wild?
We shout out to the community to share their experiences with pure L3 EVPN fabrics. Have you deployed one? What were the challenges? What were the benefits?
The best labs are the labs that you can run anywhere, anytime, with a single click and preferrably for free.
The public SR Linux container image made labs easy to run on any machine with Docker. How about we also rule out the requirement to have a machine? Let us introduce you to GitHub Codespaces.
On today’s Tech Bytes, we explore SR Linux, the network operating system developed by today’s sponsor Nokia. Why should you care about the network OS running in your data center? Nokia designed SR Linux to support automation, orchestration, and customization. We’ll dig into SR Linux’s support for YANG and gNMI and how that ties into northbound orchestration platforms, how SR Linux provides streaming telemetry, and Nokia’s development kit that lets you customize this Linux-based network OS.
Our guest is Igor Giangrossi, Sr. Director, Consulting Engineering at Nokia
In the recent VLANs on SR Linux blog post we dived deep into the world of VLANs on SR Linux where we saw that VLAN handling in SR Linux is not quite like what we used to see on Cisco/Arista systems.
As a sequel to the original post we decided to mix SR Linux with another popular Network OS - Arista EOS. By mixing different vendor implementations we wanted to provide clear guidance on how to interop between distinct VLAN implementations and help new SR Linux to map existing VLAN concepts to the SR Linux model.
