Cumulus Linux on Dell Switches: Spine-Leaf Architecture & OSPF on Unnumbered Interfaces

In a prior blog, Cumulus Linux on Dell Networking Switches, I demonstrated how to install Cumulus Linux on a Dell S6000 switch and do some basic configuration. In this blog, I’ll demonstrate how to configure routing on Dell switches running Cumulus Linux. The example covered demonstrates a Spine-Leaf architecture with OSPF on unnumbered interfaces. In this example 2 x Dell S6000-ON and 2 x Dell S4810-ON switches [Cumulus Linux 2.1.1] are utilized. The lab setup is depicted by the network diagram below.

Cumulus Linux on Dell Switches: Spine-Leaf Architecture with OSPF

The management interface on the S6K-Cumulus-1 switch was configured in a prior blog; I can quickly check the status and IP address configuration with the ip addr show eth0 command.

S6K-Cumulus-1: Checking Status and IP Address Configuration of Management Interface

The status of all interfaces can be checked with the ip link show command; below I show a portion of the output. Note, eth0 represents the management interface and lo represents a loopback interface. The interfaces starting with swp represent the S6000 data ports; the numbers correspond directly to the numbering on the physical switch. For example, swp1 represents the 40GbE port 1 interface and the swp32 represents the 40GbE port 32 interface. It’s important to note that this is not the case with the Dell S4810 where swp1 represents the 10GbE port 0 on the physical switch; this means swp49 – swp52 represent the 4 x 40 GbE ports on the S4810.

S6K-Cumulus-1: Checking Status of All Interfaces on Cumulus Linux

The network diagram represents a small snapshot of what could be a much larger Spine-Leaf design. For this example, 4 x Dell switches are used with 2 x Dell S6000 switches at the Spine and 2 x Dell S4810 switches at the leaf. Since the configuration on all switches is pretty similar and to avoid redundancy and focus on value-added information, only the S6K-Cumulus-1 switch is configured in this blog; the other three switches have already been configured in an identical manner as represented by the network diagram.

From the network diagram, it’s clear the two interfaces on S6K-Cumulus-1 that need to be configured are the two 40GbE interfaces swp31 and swp32. Taking a look at the status, it’s confirmed both interfaces are currently in the DOWN status and not being utilized.

S6K-Cumulus-1: Checking Status of 'swp31' and 'swp32' Interfaces

As root user (sudo su), vi can be used to edit the /etc/network/interfaces file with the vi /etc/network/interfaces command. The edited /etc/network/interfaces file is shown below.

S6K-Cumulus-1: Configuring Loopback and 'swp31' and 'swp32' Ports

To have the configuration on all interfaces take effect, the ifup -a command can be utilized. (Note, be careful if using the ifdown -a command to bring down interfaces; if you are connected via the management port, you will swiftly lose connection and be locked out unless you have access to the console.) The ip addr show command can be used as shown below to confirm the configuration is in effect.

S6K-Cumulus-1: Enabling and Validating Configuration

Cumulus Linux supports OSPFv2 over point-to-point links with unnumbered interfaces. Unnumbered interfaces do not have their own IP address; they borrow the IP address from another interface on the router/L3 switch which has an IP address configured. Using unnumbered interfaces allows users to avoid configuring IP addresses on each router/L3 switch interface and also limits address space consumption.

With IPv6’s massive address space, one does not need to be concerned about reserving address space. It’s also important to note OSPFv3 intrinsically supports unnumbered interfaces. Forwarding to the next hop router is done using IPv6 link local addresses; therefore, it’s not required to configure any global IPv6 address to be used by interfaces between routers.

In this OSPFv2 example, point-to-point links avoid the need for DR election and type-2 LSAs. Also, the IP address assigned to the loopback interface is reused on the point-to-point links to neighboring nodes. The only time that OSPF can form an adjacency between neighbors that are not on the same subnet is when the neighbors are connected through an unnumbered point-to-point link.

Quagga, a fork of GNU Zebra, is the routing software suite used by Cumulus Linux. Quagga provides TCP/IP based routing services for Unix platforms and supports industry-standard routing protocols OSPFv2, OSPFv3, RIPv1, RIPv2, RIPng, and BGP-4. The Quagga architecture consists of a core daemon, zebra; zebra acts as an abstraction layer to the underlying Unix kernel and presents the Zserv API over a Unix or TCP stream to Quagga clients. In turn, the Zserv clients implement a routing protocol and communicate routing updates to the zebra daemon.

Since the Zebra and OSPF daemons are not enabled by default in Cumulus 2.x, they must be configured and enabled by editing the /etc/quagga/daemons file and restarting the Quagga daemons. The edited /etc/quagga/daemons file is shown below.

S6K-Cumulus-1: Enabling 'zebra' and 'ospf'

To have the changes take affect, the Quagga daemons must be restarted via the service quagga restart command as shown below.

S6K-Cumulus-1: Restarting Quagga Daemons

Each Quagga daemon is configurable via a network accessible CLI (vty). However, a tool called vtysh included with Quagga provides a single front-end for all the daemons, allowing for all configuration to be done in one place via vtysh. Below, the OSPF configuration is done via vtysh.

S6K-Cumulus-1: OSPF Configuration via vtysh

The standard OSPF commands are available for validating that OSPF is running, neighbors are formed, and the correct routes are learned. Below, it’s validated that the routes to the loopback interfaces on networks 10.11.0.1/24 and 10.21.0.1/24 on the leaf S4810 switches are learned.

S6K-Cumulus-1: 'show ip ospf' output

S6K-Cumulus-1: Validating OSPF Neighbors and Routes