Real World DNS Abuse: Finding Common Ground

Prologue

The Domain Name System (DNS) is the protocol leveraged within the Internet´s distributed name and address database architecture. Originally implemented to make access to Internet-based resources human-friendly, DNS quickly became critical infrastructure in the intricate behind-the-scenes mechanics of the Internet, second only to routing in its importance. When DNS becomes inaccessible, the functionality of many common Internet-based applications such as e-mail, Web browsing, and e-commerce can be adversely affected—sometimes on a wide scale. This short blog post will explore some real-world examples of DNS abuse. I would like to welcome and thank Andrae Middleton for joining me as a co-author and presenting his expertise on this article.

There are a few different types of DNS attacks: cache poisoning, hijacking attacks, and denial of service (DoS) attacks (which primarily include reflection and amplification). In the news as of late are widespread and focused DoS attacks. Cisco Security Intelligence Operations (SIO), with its distributed sensors, is able observe and measure various aspects of the global DNS infrastructure. What follows are two vignettes detailing recent Internet DNS DoS attacks against the Internet’s DNS infrastructure. We will see that, though the attacks are different, the results are similar and the countermeasures and mitigations are the same.

Vignette One: Sodu.org

During the month of September 2012, Cisco SIO telemetry collection and analysis systems detected anomalous behavior in the form of traffic spikes from three of our sensors located in the Eastern United States (EUS), Western United States (WUS), and in Central Europe (CE). After isolating the traffic and analyzing it, it was determined we observed a DNS-based DoS attack. On two separate occasions an attacker sent a large number of spoofed A record queries for the “sodu.org” domain to two org Generic Top Level Domain (gTLD) nameservers.

Details of the attack are below:

Attack 1:

Approximate Start date: September 01, 2012 at 23:30 UTC
Approximate Stop date: September 02, 2012 at 05:00 UTC
Approximate Attack duration: 5.5 hours

Query Request packet type: A record
Query Request packet counts:

CE: 1,529,965,717

EUS: 4,299,977,251

WUS: 4,530,979,956

total: 10,360,922,924

Query Request size: 72 bytes
Total Traffic Generated: 745,986,450,528 bytes (695G)

Query Response packet type: referral response
Query Response packet counts:

CE: 293,930,730

EUS: 345,615,061

WUS: 354,119,645

total: 993,665,436

Query Response size: 159 bytes
Total Traffic Generated: 157,992,804,324 bytes (147G)

Attack 2:

Approximate Start date: September 05, 2012 at 14:00 UTC
Approximate Stop date: September 06, 2012 at 02:30 UTC
Approximate Attack duration: 5 hours

Query Request packet type: A record
Query Request packet counts:

CE: 1,280,172,566

EUS: 2,455,378,453

WUS: 2,350,329,545

total: 6,085,880,564

Query Request size: 72 bytes
Total Traffic Generated: 438,183,400,608 bytes (408G)

Query Response packet type: referral response
Query Response packet counts:

CE: 372,305,159

EUS: 326,794,446

WUS: 353,189,760

total: 1,052,289,365

Query Response size: 159 bytes
Total Traffic Generated: 167,314,009,035 bytes (155G)

Technical Discussion

The attacks were intermittent and each only lasted a few hours, but they caused a pretty large spike in org gTLD traffic. During the attacks, the spoofed queries were responsible for approximately 79% of all queries across all three sensors. Figure 1 clearly depicts the massive spike in A record query request traffic (click to enlarge):

Figure 1. DNS A Record Traffic

Given that the query packet was approximately 72 bytes and the response packet was about 159 bytes (yielding a 2x multiplier), this attack can probably be classified as a some sort of a low-rung DNS amplification attack. Figure 2 shows the referral response traffic spikes (click to enlarge):

Figure 2. DNS Referral Response Traffic

A true DNS reflection/amplification attack is much more devastating (as we will see below) and has a few key differences.

In a traditional DNS reflection/amplification attack, the attacker takes advantage of a few Internet truisms:

A single DNS query can result in a response eight or more times the original size (the amplification)

UDP packets are easily forged or spoofed (the reflection)

Over 10 million open resolvers exist on the Internet

In a typical DNS reflection/amplification attack, UDP datagrams are spoofed to purport their source address as the intended target and the DNS query request is sent to an open recursive resolver. Once the open recursive resolver has iterated through the DNS to resolve the domain, it will emit larger responses to the intended target, effectively flooding it with un-asked DNS response packets.

While the attack here shares some similarities in that it spoofs a query to a DNS server, it is not a true DNS reflection/amplification attack (in fact it appears to be more similar to the venerable ping flood attack of yesterday). Most notably, the attacker did not spoof the IP address of a specific target, but rather spoofed random IP addresses. In the first attack, SIO found around 2000 unique random source IP addresses, and in the second attack, a much larger amount, around 53 million. One theory is that the attacker was refining the tool being used to generate the spoofed DNS query requests and this explosion in the range of spoofed IP addresses might reflect the evolution of the tool.

Additionally, it appears as though most of the queries went unanswered, either through some sort of org gTLD sanity check or through some other countermeasure or happenstance.

Finally, it bears mentioning that as a second order effect, some number of ICMP destination unreachable messages (with varying codes) were also generated as the targeted nameservers attempted to respond to the spoofed IP addresses which would add to the total traffic count. A sampling of these messages, ICMP port unreachables, is shown below in Figure 3 (click to enlarge).

Figure 3. ICMP port unreachable messages

Attack Assessment

The SIO team had no visibility into the residuum of the attack and did not receive any report from the gTLD operators concerning overall impact. All things considered, the deleterious effects of the attack were probably not that severe—and it certainly didn’t last for very long (some DNS DoS attacks are known to go on for weeks). Perhaps these were just nascent tests of a new tool that is being erected with larger ambitions.

Vignette 2: DNS Amplification – Utilizing DNSSEC and large EDNS0 buffer size

DNS Amplification attacks are consistently prevalent at the DNS root level of the Internet. Throughout the month of October 2012 and consistently to-date, Cisco SIO has detected anomalous behavior from the EUS and CE sensors.

Starting on October 3rd, the CE sensor captured traffic that Cisco SIO identified as being used to attack a number of IPs.

As we look further and dig into the details we discover that 864,065,748 ‘ANY’ responses were directed at a specific IP, compared to 22,073,375 non-attack related ‘ANY’ responses.

Just as one would predict, if some is good, seemingly more is better…..

On October 5th, our EUS sensor was seeing an attack against another address:

Specifically, 793,426,549 ‘ANY’ responses were directed at this IP compared to 107,166,894 non-attack ‘ANY’ responses. This second attack rolled into the 5th and 6th of October, and the footprint of these two attacks is significantly more than any other DNS ‘ANY’ responses combined. Note, for those wondering who the sources are, as most often is the case the source addresses are quite random and allude to traffic being sent from spoofed sources as the addresses.

For further insight into these attacks, packet statistics divulging the DNS ‘ANY’ responses juxtaposed with EDNS payload size breakdown (with 9000 bytes being the typical size for the attacks) for the CE sensor is provided in Figure 4 below (click to enlarge):

Figure 4. UDP "ANY" Packet Counts and EDNS 9000 Payload Size Counts

Technical Discussion

Alright, enough with the numbers and quantitative nature, what does all of this actually mean for society? Well to understand that, one must first understand the nature of the attacks.

As we know, DNS amplification is based on the simple premise of sending voluminous amounts of data to specific sectors/links of the Internet, inherently reducing the ability for that link to send or receive legitimate data. DNS amplification is an evolution of DNS recursion attacks in which miscreants would leverage the inherent functionality of DNS queries for resource records to wage flooding attacks. DNS amplification attacks typically leverage the DNS query type ‘ANY’ by allowing a miscreant using a host (note, this can be spoofed or compromised hosts or simply one in which the miscreant has control of) to (mis)use an authoritative DNS server and instantiate a large text file on a record. This is exacerbated due to the evolution of DNSSEC, a solution developed to protect against various DNS threats by cryptographically signing aspects of the DNS infrastructure so others can be assured it is trustworthy and valid. DNSSEC requires support for Extension Mechanisms for DNS (EDNS0) which allows the DNS protocol to adopt changes and enhancements.

The inception of EDNS0 allows a DNS sender to advertise the largest UDP payload that it can reassemble, thus opening Pandora’s Box and increased nefarious behaviors—specifically amplification attacks. For further details regarding EDNS, refer to RFC2671. The miscreant will subsequently send queries (from the spoofed/compromised machines) to open recursive resolvers for a large resource record. This act will essentially “prime” the DNS servers as they look up the large record, answer the query, but more importantly cache the answer (the large record). From this point forward the miscreants send subsequent queries from spoofed/compromised hosts, further amplifying the attack surface.

Attack Assessment

Due to the nature of this traffic profile, the attack surface is vast and cascading. Certainly DNS root servers are the main proponents; however, all entities leveraging DNS are potentially impacted depending on the nature of a particular event.

Mitigation Solutions

While Cisco SIO was not directly involved with any remedial efforts for these incidents, there are a number of countermeasures and solutions that can be employed to mitigate DNS DoS (and similar attacks) as outlined below:

First and foremost, it is always a good practice to ensure that you do not place open DNS resolvers on the Internet. In fact, nowadays, organizations that do have open recursive resolvers in their infrastructure are greatly frowned upon. Removing the “openness” from these resolvers will limit who can access the recursive resolvers which will decrease the ability of an attacker to leverage them for malicious activities. For further details take a look at the ISC website regarding configuring DNS resolvers to NOT answer queries outside a specified range (typically your clients), unless authoritative for a zone and if that is the case it should be provisioned to only answer queries for names in that zone.

Note: This can be accomplished using filtering and abstraction techniques such as firewall rules, router IP access lists, or other filtering methods.

The DNS Response Rate Limiting (DNS RRL) is a relatively new initiative by Paul Vixie of ISC and Vernon Schryver of Rhyolite intended to give relief to victims of DNS reflection/amplification attacks. It works by offering a few techniques for limiting the number of responses a DNS server can emit. DNS RRL is a wonderful way to help prevent DNS nameservers from participating in DoS attacks by reducing the rate at which they can respond to incoming queries. If all DNS servers supported some sort of rate limiting, the effects/impacts of a reflection/amplification based attack would be substantially less effective, if not completely ineffective at saturating bandwidth (or spiking CPU utilization).

A succinct list of some mitigation solutions is provided below (largely based on Cisco security best practices):

Limit the effect of IP address spoofing by configuring Unicast Reverse Path Forwarding (URPF) on network routers. A router configured to use URPF (as defined in RFC3074) limits an attacker’s ability to spoof packets by comparing the packet’s source address with its internal routing tables to determine if the address is plausible. If not, the packet is discarded. For more details, see the DNS Best Practices, Network Protections, and Attack Identification whitepaper.

Leveraging URPF can help weed out certain types of invalid IPs:

Those with ‘0’ in the first octet

non-routable addresses as defined in RFC 1918, RFC 3927, and RFC 5735

Class E and possibly Class D addresses

Institute monitoring solutions to identify DNSSEC traffic (traffic with EDNS0 enabled). Potential indicators of such attacks include large numbers of outbound packets with the same destination address. Specifically those whose packet counts suddenly spike. Note: This is a first step as one would still need to analyze this traffic to disseminate and filter out permissible DNSSEC traffic from miscreant EDNS0-based traffic.

Some examples of these solutions are as follows:

Netflow

IOS Netflow

NSEL for Cisco Firewalls (Syslog over Netflow)

Syslog

Wire Level Packet Capture

IPS Solutions

Baselining tools

Security Information and Event Management (SIEM) tools

For those hardening enterprise environments, deploying filtering solutions such as firewalls, ACLs, and IPS devices to drop, limit, or delay any suspect incoming packets identified by the previously outlined tools/solutions is warranted. (Note: In high traffic load environments, (for example, DNS operations and provider levels) these filtering solutions are not recommended as they do not scale to such strenuous traffic loads). Leverage Cisco Firewall features (including DNS inspection, DNS guard, syslogs, and more) and IPS signatures to identify and mitigate attacks. See the Detecting and Preventing DNS Attacks using Cisco Products and Features section of the DNS Best Practices, Network Protections, and Attack Identification whitepaper for further details.

Employ Cisco’s Partner solution – Arbor Peakflow

Native Peakflow SP Product – http://www.arbornetworks.com/products/peakflow/sp

Cisco CRS Carrier-Grade Services Engine (CGSE) Module (Runs Arbor Peakflow SP) – http://www.cisco.com/en/US/prod/collateral/routers/ps5763/data_sheet_c78-614893.html

References and Additional Reading

How DNS Queries Work

DNS Best Practices, Network Protections, and Attack Identification

Preparing for DNSSEC

Understanding Denial of Service Attacks

DNS Amplification Attacks Explained

Ongoing Analysis by DNS Operators

Open Resolver Survey