Software-Defined Networking

Software-Defined Networking (SDN)

We have some well-established ways to start, monitor, and manage virtual servers and storage. The networking side has less in place for easily deploying and modifying Layer 2-4 connectivity through complex physical topologies. Software-Defined Networking or SDN aims to better manage physical and virtual network infrastructure.

In order to make sense of the following, you will need to understand IP addressing, subnets, VLANs, the distinction between switching and routing, TCP and UDP ports and how they're used by firewalls to enforce policy, and more. Start with my page on networking fundamentals and terminology if you need to get up to speed.

One striking feature of SDN is its use of generalized packet-forwarding devices. In the traditional OSI model we say that a repeater cleans up and relays signals at Layer 1, a switch forwards frames across a network link using Layer 2 hardware addresses, a router does hop-by-hop forwarding from network to network using Layer 3 IP addresses, while a firewall uses Layer 4 UDP and TCP port numbers (in addition to Layers 3 and 2) to make policy decisions on whether or not to forward a packet.

Then the marketing people got involved and started calling slightly faster routers "switching routers". (later, routing functionality was moved from a program running on a processor into hardware, making those routers a little closer to the current switch design, but it left it unclear just exactly what the devices were doing)

SDN uses devices that we call switches although they base forwarding decisions on Layers 2-4. This misleads some people into thinking that any Ethernet switch supporting VLANs will also support SDN, but that's just not true. We're stuck with the name switch, we can't go back in time and say "Hey, why don't we call these 'Fast Forwarders' to avoid confusion?"

Visualizing SDN

Network traffic is split into two classes: data traffic flowing from applications to other applications or storage (either of which can be local or "out in the cloud"), and control traffic flowing from network controller devices (physical or virtual) to switches and routers (again physical or virtual) altering flow between the growing number of physical and virtual machines in data centers. This is usually visualized as multiple layers, where higher-level management may be considered as a third layer or plane. Conceptually:

Filling in the details:

Applications can request traffic flows of desired connectivity and performance. They may do this directly through the API (or Application Programming Interface) of the controller through the northbound traffic. There may also (or instead) be an orchestration engine on the northbound side, which might be considered the management plane.

The controller sends configuration commands to the Layer 2-4 switching through the southbound traffic. This may be physical switches, or it may be a layer of virtualized switches on top of physical infrastructure.

Virtual switches running within the same hypervisor communicate directly with each other. In order to communicate across physical platforms, from one hypervisor to another, or possible between a virtual switch and a physical one, the traffic is tunneled or encapsulated.

Virtual switches include VMware Nicira and their proprietary vSwitch running in ESXi, Cisco's virtual Nexus 1000V, Microsoft's Hyper-V virtual switch, and the virtual switches within Linux KVM. This can get confusing, as Cisco's Nexus 1000V virtual switch can run on VMware, Hyper-V, and Linux KVM virtualization.

Open vSwitch

Open vSwitch defines an open-source virtual switch.

Open vSwitch OpenFlow Switch Specification RFC 7047: OVSDB, the Open vSwitch Database Management Protocol

We might call these "hosts" but that gets confusing: do we mean IP host in the networking sense of a device that can communicate by IP datagrams, or do we mean a virtualization host, a hypervisor on which virtual machines can run? Let's just say "devices connected to the network."

The devices are connected to the switch fabric. These are the servers, the clients, and other devices such as printers, phones, cameras, and more. Traffic between those devices is called east-west traffic.

That's the general model. Different implementations vary in their details — the controller itself, the API for northbound traffic, the protocol for southbound traffic, which components are physical and which are virtual, etc.

The following lists the specifics for some of the most popular SDN solutions, with the associated terms, acronyms, and protocols. It also has links to pages with much more background information. This page was meant to be a supplement to a software-defined networking course. I was the technical editor of the course, which was authored by Ola Sigurdson. Despite SDN being a hot topic at the time, the training company was unable to sell it to the right audience.

SDN Concepts and Background

SDN is not a new concept. QoS routing, ATM, MPLS, tag switching, IntServ, DiffServ, and other projects have been earlier work in this area.

Quality of Service (QoS)

MPLS (Multiprotocol Label Switching)

Network Function Virtualization

Network Function Virtualization or NFV is closely related to current SDN projects. VLAN or Virtual LAN technology and, of course, IP addresses and subnetting are fundamental supporting technologies you must understand before SDN can make any sense to you.

Webtorials 2016 SDN Guide

OpenFlow and OpenDaylight

The OpenDaylight project was founded in 2013 as a community-led industry-supported open-source framework. Cisco has been a reluctant and late addition, recommending their excellent but proprietary solutions. The OpenDaylight model is a descriptive one, documenting what has been found to work, as opposed to a prescriptive one specifying in advance what must be done.

The northbound interface is REST web programming. The applications can be written in Java, Python, .NET, and other languages.

OpenFlow

The OpenFlow switching specification is the southbound interface. It defines how a controller can change the operation of network devices. It was developed at Stanford University and is now overseen by the Open Network Foundation. The specification states "OpenFlow is based on an Ethernet switch, with an internal flow table, and a standardized interface to add and remove flow entries." OpenFlow uses TCP port 6653, preferably with TLS for data security. The connection is made from the an OpenFlow switch to one or more OpenFlow controller. Here's an example on an HP 2920-24G OpenFlow switch set up in a very liberal way — open SNMP, no TLS configured, two controllers.

HP-2920-24G# show running-config

Running configuration:

hostname "HP-2920-24G"
module 1 type j9726a
timesync sntp
sntp server priority 1 10.0.1.1 oobm
snmp-server community "public" unstricted
openflow
   controller-id 1 ip 10.0.0.1 controller-interface oobm
   controller-id 2 ip 10.0.0.2 controller-interface oobm
   instance aggregate
      listen-port oobm
      controller-id 1
      controller-id 2
      version 1.3
      enable
      exit
   enable
   exit
oobm
   ip address 10.0.0.254 255.255.255.0
   exit
vlan 1
   name "DEFAULT_VLAN"
   no untagged 10-20
   untagged 1-9,21-24,A1-A2,B1-B2
   no ip address
   exit
vlan 2
   name "DMZ"
   untagged 10-20
   no ip address
   exit

OpenFlow and OpenDaylight

OpenDaylight Project

OpenDaylight provides a controller which runs on a Linux host. The versions are named for the elements — Hydrogen, Helium, Lithium, Beryllium, etc. — as a reference to stellar nucleosynthesis I suppose. If so, that's ominous as it must blow up in a supernova to move beyond Iron...

Speaking of astrophysics, the Large Hadron Collider project uses OpenFlow to manage petabyte data flows.

Other OpenFlow Controllers

Big Switch Networks is a spin-off of Stanford University. They have released Project Floodlight, an Apache-licensed open-source OpenFlow controller.

See the Open Networking Foundation's list of OpenFlow products for more controllers.

Fabric-Based Model

The fabric-based model uses OpenFlow traffic directly to hardware. It needs a compatible physical network, so it likely requires upgrading much of the Ethernet infrastructure.

Mininet and Estinet are SDN simulators.

Here is an HP 2920-24G switch sitting in a classroom. It lists for about $1,300, surprisingly expensive for a 24-port gigabit switch until you realize that it can do SDN with OpenFlow. OpenFlow is an open protocol, and controllers like OpenDaylight and Floodlight are open-source software running on free operating systems. But it takes expensive hardware to support this technology. Just wait until you see the cost for the VMware NSX software licenses and the Cisco Nexus 9000 switch hardware...

The overlay model encapsulates and tunnels between virtual switches over your existing physical network. Encapsulation is done with NVGRE, VXLAN, or similar protocols. An orchestration engine (e.g., OpenStack, CloudStack, VMware vCloud suite) talks over the northbound API to applications and users, and over the southbound protocol to network virtualization (e.g., VMware NSX).

Cisco's Application Centric Infrastructure or ACI runs on Cisco Nexus 9000 series switches. The network controller is the Application Policy Infrastructure Controller or APIC.

Network Overlays

VMware, vSphere, NSX Controller, and VXLAN Overlay

This gets a little confusing. There used to be a company named Nicira which did SDN with their own proprietary versions of OpenFlow, Open vSwitch, and OpenStack. They had a controller named NSX.

Then VMware bought them for $1.26 billion in 2012. The Open* projects went away, and NSX became the name of the VMware vSphere SDN controller.

NSX software licensing is expensive, around $5000 per controller CPU or $50 per virtual machine per year.

In October 2016 VMware and Amazon announced VMware on AWS. It includes cloud-optimized versions of vSphere, VSAN and NSX. You can spin up a VMware cluster with an entire VMware Software-Defined Data Center stack in the AWS Cloud. You can manage it with vCenter, you only need vSphere in your data center.

Microsoft Hyper-V Controller and NVGRE

The Microsoft SC VMM or System Center Virtual Machine Manager is the SDN controller.

Overlay encapsulation is NVGRE or Network Virtualization GRE, a variant of the traditional Generic Router Encapsulation or GRE.

Cisco Application-Centric Infrastructure

ACI = Application-Centric Infrastructure

APIC = Application Policy Infrastructure Controller

This uses XML JSON API for the northbound traffic, and OpFlex for the southbound traffic.

Amazon Web Services (AWS) Virtual Private Clouds (VPC) and SDN

Orchestration

Cloud services like Google Cloud and AWS and Microsoft Azure and so on must use SDN. Here's what the AWS dashboard shows you of the orchestration parts of a multi-VM deployment with network orchestration. Amazon calls this "CloudFormation". Here we're starting multiple:

• Database instances
• Security groups (firewall rulesets)
• Load balancers

Many thanks to Carter Elmore for the screenshot!