Continuing the analysis of the data collected during Fosdem 2016.
This year, we replaced the router with a more powerful model: an ASR 1006 with RP2 and SIP20 modules. This allowed us to enable the NBAR 2 feature to analyse the traffic crossing the router.
NBAR 2 is a traffic analysis engine which allows the router to classify traffic into applications. It does this on the basis of ports, but also based on the content of the traffic itself. It recognizes, for example, the difference between general web traffic and traffic to youtube.
This feature is designed to be used as a classifier for use with QoS, so that one can give higher priority to important web traffic and less to some other traffic. Now that HTTP and HTTPS are the new “TCP sockets”, meaning that more and more applications use HTTP or HTTPS to interact with the world, we need deeper insight into the traffic and just using port numbers isn’t enough anymore to discriminate between different kinds of requests.
For Fosdem, we used this to learn what kind of traffic is being generated on the network and to extract interesting information from traffic flows (but more on that in a later article).
As part of the ‘discovering the usage of your networks’, enabling NBAR2 is a simple step beyond using access-lists and interfaces counters which we used in the previous article.
Over time protocols change, so one can download updated ‘protocol packs’ which include the description of newer protocols and applications. Simply upload them to the router and do:
asr1k#conf t asr1k(config)#ip nbar protocol-pack bootflash:/pp-adv-asr1k-155-3.S1-23-15.0.0.pack
This will teach NBAR2 about the new protocols and applications. Then to enable the discovery on the desired interface, configure:
interface Gig2/2/1 ip nbar protocol-discovery
From this moment on, the NBAR2 engine will start collecting information. Allow some traffic to flow across the interface, and then see what NBAR2 knows about it:
asr1k#sh ip nbar protocol-discovery
Last clearing of "show ip nbar protocol-discovery" counters 5d20h
Input Output
----- ------
Protocol Packet Count Packet Count
Byte Count Byte Count
5min Bit Rate (bps) 5min Bit Rate (bps)
5min Max Bit Rate (bps) 5min Max Bit Rate (bps)
------------------------ ------------------------ ------------------------
binary-over-http 50067876 109178677
5035136708 159890721603
0 0
4762000 116145000
...
This shows the number of packets, bytes and traffic rates for a large number of applications. Here, “Input” traffic is traffic arriving on the interface, so in this case from the wireless clients towards the internet or the other networks on the router. Similarly “Output” is traffic send out on the interface towards the clients.
The CLI allows you to interrogate this information in a liberal manner:
asr1k#show ip nbar protocol-discovery interface <interface> stats max-bit-rate top-n 3
<interface>
Last clearing of "show ip nbar protocol-discovery" counters 00:10:53
Input Output
----- ------
Protocol 5min Max Bit Rate (bps) 5min Max Bit Rate (bps)
------------------------ ------------------------ ------------------------
ssh 743408000 505670000
rtmp 93801000 1499000
binary-over-http 527000 45774000
Total 845853000 576473000
Here we asked for the top 3 application, sorted by maximum bit rate on a specific interface.
I collected this output for the different interfaces and, after some transformations, one can plot, for example, the total number of bytes send or received on the Internet interface for the top 20 applications:
As it is clear video-over-http, ssh and ssl dominate.
When we look at the maximum bitrate for the internet side we see that the highest rate is for ssh:
In the above graph ‘in’ means traffic arriving to the router from the internet.
This high rate for ssh (almost 1Gpbs) is due to the Video team who send quite a bit of rtmp traffic, but even more ssh traffic, as one can see from the statistics for the Video network interface traffic:
For the outgoing direction (from the point of view of the interface, as always) we see that the Video network itself received mostly ssh traffic:
Looking at the wireless networks we can see a clear difference between the traffic requested by the IPv6 only clients:
and the legacy dual stack clients:
It seems IPv6 clients watch video streams a lot more.
One can also see the traffic the wireless clients generated towards the internet:
One can conclude that bittorrent traffic seems to prefer legacy networks as bittorrent traffic disappears on the IPv6 only network.
The NBAR2 data also answers the question: which network is the most looking for love:
asr1k#show ip nbar protocol-discovery interface <Wifi> stats byte-count protocol okcupid
...
Input Output
----- ------
Protocol Byte Count Byte Count
------------------------ ------------------------ ------------------------
okcupid 7395809 26178355
Total 217481608124 1893527651702
...
asr1k#show ip nbar protocol-discovery interface <legacy> stats byte-count protocol okcupid
...
okcupid 4167154 54499457
Total 147878754133 950455266790
So about 32 Mbyte of OkCupid traffic on IPv6 but almost 56 Mbyte on the dualstack network. Clearly people looking for love are prepared to accept working with legacy protocols.
As one can see NBAR2 offers an easy and simple way to discover what users are doing on your network, both on the command line or when you slice’n dice the extracted information.
Great product to get detailed insight into network activity.