Black Hat USA 2017: Cisco Umbrella Joins the NOC

Cisco Cloud Based Security in the SOC

Black Hat USA marked its 20th anniversary this year. The members of the NOC management showed me photos of the original NOC: a single router in a closet. The NOC has grown with the conference; into a well-managed team of experts from around the globe, from various vendors and backgrounds, all with the same mission: build and secure a robust network for one of the largest cybersecurity conferences in the world. The primary mission of the SOC team in the NOC was to protect the network from attacks: externally and by attendees; especially from DDOS and traffic floods.

New this conference was the addition of Cisco Umbrella for DNS visibility, at the request of the NOC manager, arising from our experiences at Black Hat Asia 2017 with Umbrella Investigate. The set up was very simple: log into the Black Hat Umbrella account created by Alejo Calaoagan, my counterpart in Business Development; add in the Network ranges provided by CenturyLink for the conference; verify the Network ranges were authorized with Umbrella support; and DNS protection and visibility began within an hour.

We also enabled the Cisco Threat Grid intelligence feeds integration, by adding in the API key.

This imports the ~15 curated feeds included in a Threat Grid premium subscription from DNS queries and network streams of malicious samples seen in the last day, including:

Autorun Registry Changes

Ransomware Communications

Banking Trojans

Remote Access Trojans

DGA Domains

Stolen Certificates

Documents with Network Connections

Modified Windows Host Files

Samples Downloading Executables

Checking for Public IP Address

By default, Umbrella will begin blocking DNS traffic to sites known to host Malware, to Command and Control Callback servers and Phishing Attacks. At Black Hat, we wanted to enable all traffic, so the trainers and briefers can show the latest exploits.

Many presenters set up new domains and simulated malware expressly for the training. Umbrella blocked over 6,800 DNS queries from this default protection, before we turned off the enforcement. Over the weekend, we saw about 4.6 million DNS requests.

As the conference attendance grew and more briefings & trainings were conducted, the DNS volume also expanded rapidly. The team knew there has always been a large about of DNS traffic, but the visibility really shocked everyone with the amount of DNS look ups the network was handling. Interestingly, as the conference moved from Training into the Business Hall and briefings, the DNS queries dropped off precipitously.

Malware quickly rose to one of the top Categories, peaking at Number 3, before settling in at Number 7.

The network saw over 17 million DNS requests during the week.

Key takeaways:

Over 300,000 DNS queries were observed to domains known to be malicious or host malware

Over 12,000 queries went to Dynamically Generated domains

Over 7,800 newly seen domains were queried from Black Hat USA, many likely created for the express purpose of training

Over 6,800 domains were queried that had seen prior communication from known malicious samples in Threat Grid

Umbrella Investigate

Digging into some of the malware traffic with Umbrella Investigate, we were able to examine the infrastructure.

This included global DNS queries, WHOIS information and links to Threat Grid malware analysis reports for samples that have DNS queries to the domain.

The Threat Grid analysis report gave a summary of the behavioral indicators observed by the sample, with the ability to pivot to the full Threat Grid report that includes integrated threat intelligence, the ability to download the report, download the sample and interact with the sample in the ‘Glovebox’ feature.

We could also scroll down to the DNS query.

Threat Grid

Providing the network forensics was RSA NetWitness Packets. The RSA team brought in the pre-release code for v11 and added in the Cisco Threat Grid API key for dynamic malware analysis. It was an excellent beta testing environment. The SOC team placed NetWitness Packets into Continuous Monitoring mode, where .exe, .dll, .pdf, .doc, .rtf and other potentially malicious payloads were carved out of the network stream and underwent Static analysis, Network intelligence and Community lookup; before sent to Threat Grid for dynamic malware analysis and additional static analysis. RSA customers can register for a no-cost Threat Grid account within NetWitness, for up to five samples per day for ad hoc analysis.

Many of the samples seen were for trainings, with names such as: evil.exe, bot.exe, payload.exe, my_build.exe, drop.exe and master.exe.

Some interesting behavior was seen in a series of samples injecting code into memory.

I also noticed Remote IP Address Connected and investigated the network stream. Threat Grid is not just a “sandbox” for dynamic analysis, it is a threat intelligence platform that correlates the samples it has analyzed historically and globally. Investigating the IP address destination, we found that a several executables were related in the hard coded command and control (CnC) callback.

The bot.exe sample had similar behavior, but an elevated Threat Score due to the persistence mechanism in the registry.

Likewise, it had code injection into memory and CnC callbacks. Again, we found a family of samples with a shared destination.

Working with the NetWitness team, we were able to investigate the source to the samples and identify the bot.exe upload from a training event, and not an external or lateral attack.

This is the first year the Black Hat conference was completely wireless. The NOC team detected some rouge access points broadcasting the BlackHat SSID and other attempted spoofing. We were successful in our mission to provide a robust, stable and secure network for the Black Hat conference attendees, trainers, presenters and sponsors. I’m looking forward to going to London in December, for Black Hat Europe 2017.