KVM App Hosting on a Cisco Router

Run your own virtual machine directly on a Cisco router.

In 2013 we introduced a pretty cool new trick for Cisco routers. The ISR 4000 Series and ASR 1000 Series can host virtual applications directly in IOS XE operating system. Since then the ISR 4000 has supported a range of Cisco applications: starting with WAAS expanding to include Snort and now StealthWatch Learning Network. I introduced the architecture in a blog post here:

What the Heck is a Service Container?

That was more than 2 years ago, and things have progressed. The idea of hosting network services and applications within network devices is no longer extreme. Augmenting the capabilities of a network device through virtualization has become mainstream with other Cisco devices like the Nexus 9000 and the 800 Series Industrial Routers incorporating virtualization.

What’s New?

This summer, Cisco will officially introduce a capability that we quietly rolled out in a software release last November (IOS XE 3.17). You can now host your own custom or third party KVM lightweight applications directly on your ISR 4000 or ASR 1000. If you aren’t familiar with it, Kernel-based Virtual Machines are the standard virtualization technology in the embedded Linux world. If you’re writing applications to be hosted across a network, odds are you’re already using it. The ability to virtualize across the network is a key aspect of Cisco DNA so this capability dovetails very nicely with the larger solution for enterprise environments.

Built for Virtualization

Let’s re-visit the underlying architecture for the ISR 4000 and ASR 1000. Both of these platforms use a customized high-performance data plane for the actual business of forwarding and manipulating packets. The control plane is entirely Linux running on an x86
CPU from Intel. Since the control plane of most routers isn’t busy, that opens up the possibility for hosting Kernel Virtual Machines (KVM) directly on those spare CPU cycles.

The ISR 4000 Series specifically were overbuilt and include roughly 3 times more control plane processing power than necessary to run the router. They were designed from the onset to serve as a network function and application hosting platform.

What about performance? Since these virtual machines are not hosted in the data plane of the router, there is no impact on performance for packet forwarding and feature processing. Control plane functions (routing protocols and forwarding table management), run at a higher priority than hosted virtual machines. Again no impact on performance for the system. However, any time the router does not need all the resources reserved for control plane functions, that CPU time and memory can be used for virtual services. This is industry-standard Linux virtualization at its finest.

Designed for Quick Development

While we are supporting KVM applications with no restrictions, unfortunately, there is no real “standard” for a KVM package file. Most KVM deployments require a complex XML file that is version dependent and prone to human errors. To simplify things, we’ve cre

ated a human-readable YAML (Yet Another Markup Language – no kidding) format to describe the resources (CPU, memory, storage, networking, console, etc.) for the VM. If you have an existing KVM application, creating this YAML file takes about 10 minutes.

And that’s it. The next learning curve is learning to build KVM applications. We have a developer guide as well as a packaging script for you. We even take it one step further. KVM app hosting also works in our virtual router, the CSR 1000v. This makes it convenient to prototype your application and get all the kinks worked out before moving to a hardware deployment.

Use Cases

You’ll soon see third parties app offer for the ISR 4000 and ASR 1000 platforms. However, that doesn’t mean you can’t develop your own lightweight KVM application. There are also a few open source projects and even end customers that have developed and deployed their own applications. Here are a couple.

Validation and troubleshooting tools. Programs like Wireshark, iPerf, and other open source tools run just fine, and the enterprise router provides a unique hosting location for a troubleshooting VM.

Network Agents There is an entire industry of companies building network diagnostic tools that capture and analyze application traffic across a network. Many of these 3rd parties have agents that run in remote offices adding a unique perspective to discover what’s going on within the network. Many of these companies are building and testing agents to run directly on ISR4K and ASR1K routers. This is the technology they’re using when they announce their products in the coming months.

Virtualized Network Functions Plenty of network functions need to be run in small to medium offices but they might not justify a heavyweight server. Things like print servers, domain controllers, file storage, DNS and DHCP servers are just a few examples that can be hosted within the network.

Sandbox for education and research projects. Cisco Academy instructors, interested in creating a hosted NFV VM as an assignment for their students, had used the CSR 1000v as a development platform.

The only real limit to what you can do with a hosted function inside a router is what you can imagine. Because ISR4K and ASR1K routers are often found in remote branch offices with no IT staff and minimal infrastructure, finding a place to host a network function or application can often be a challenge. With the router already there providing other network functions, it makes an ideal place to host other functions within the branch office.

I’m Excited Now – Where Do I Start?

You need three things to host a virtual machine on a router: CPU, Memory and Storage. We’ve already covered performance in the “Optimized for Virtualization” section above. For memory and storage, just simply add more as needed. If your VM needs 2GB memory and 120GB of storage, you would upgrade an ISR 4000 to 8GB of memory and add in a 200GB SSD drive (NIM-SSD or SSD-MSATA-200G). Today we reserve the base 4GB of memory for the system itself. For example, with 8GB of memory, 4GB is preserved for the routing function; the remaining 4GB can be used for your virtual application.  

The main source for support building your own KVM application for an ISR4K or ASR1K is going to be through Cisco DevNet. DevNet is the premiere watering-hole for networking geeks looking to build cool things on top of the network. The Open Device Programmability area is where you’ll find information on IOS-XE support for 3^rd party KVM as well as some of the APIs your applications can make use of. Please join us there and let us know what you’re working on.

Shameless plug time. If you’re planning to visit Cisco Live next month I’d love to meet you. We’re going to have tons of exciting Enterprise Routing content in the World of Solutions as well as a few breakout sessions that will be covering this technology in more depth.

BRKARC-2014 – Branch Virtualization – The Evolving NFV Landscape

BRKARC-2091 – Emerging Trends in Branch Office Architectures

BRKARC-3001 – Cisco Integrated Services Router – Architectural Overview and Use Cases