Cisco IT Streamlines Big Data Operations with Cisco Tidal Enterprise Scheduler

Like many IT organizations, Cisco’s internal IT department is deploying Big Data solutions to mine the ever-increasing data in-flow from a wide range of sources – and thus gain competitive advantage and insights.

The typical environment includes an ecosystem of different tools and data sources that looks something like this (image courtesy of @TorstenVolk):

Source: EMA Research

Cisco IT realized that as the demand for analysis of this data increased, the demands on their infrastructure and Day 2 operations management would likely grow exponentially. So they knew that they needed an enterprise-grade workload automation solution that could manage processes involving Hadoop, MapR, Cloudera, Informatica, Teradata, SAP HANA, BusinessObjects, Tableau as well as other analytics applications, data feeds and repositories.

Fortunately, we have a workload automation software solution – Cisco Tidal Enterprise Scheduler – that meets those requirements and more:

 

Cisco IT has standardized on Cisco Tidal Enterprise Scheduler and will use it for all future workload automation – including more than 10 big data analytics use cases.  As one example, Cisco IT developed a big data analytics solution that processes 1.5 billion customer records daily to identify partner sales opportunities. Anticipated incremental revenue from this solution, which is in production, is approximately $40 million just for this current fiscal year.  This internal application (called the 360 Data Foundation) uses Informatica to transform data from various data sources, Hadoop to store and process the data, and Teradata to store the data for analysis (see the figure below). Using built-in adapters, Cisco Tidal Enterprise Scheduler coordinates all workloads within this processing environment.

Cisco IT Big Data Architecture for “360 Data Foundation” internal application

Here are a few highlights from the implementation and use of Cisco Tidal Enterprise Scheduler (TES) within Cisco IT:

“We estimate that using Cisco Tidal Enterprise Scheduler for end-to-end data management reduced development time by 90 percent compared to using traditional scripting tools.”

Wow, imagine saving 90% on any data center operational task—all while reducing the risk of manual scripting and repetitive maintenance.

“Before, we had to deploy infrastructure for each big data application. Now, with Cisco Tidal Enterprise Scheduler, all applications access the same centralized server, which give us a high degree of scalability and reduces costs.”

As the build-out continues, the IT Engineer need only to make the new connections and implement the new workload—all from the same centralized fault-tolerant server.

“Using the Cisco Tidal Enterprise Scheduler client on our iPhones and iPads, we can receive alerts, log in to learn more about jobs, and respond by holding or restarting jobs. We actually get to go to lunch today without worrying about a job failure.”

So TES even adds to a positive “work/life” balance, with its mobile app. Cool.

You can read more about Cisco IT’s use of Cisco Tidal Enterprise Scheduler for their Big Data infrastructure here: Cisco IT Automates Workloads for Big Data Analytics Environments from Cisco Data Center

And for more on how Cisco IT uses Cisco Unified Computing System (UCS) as the platform for Big Data, you can refer to this case study: How Cisco IT Built Big Data Platform to Transform Data Management

I’d also encourage you to read the Big Data blog posts from my colleagues Raghunath Nambiar and Scott Ciccone.  Additional resources about Cisco’s solutions for Big Data can be found here: www.cisco.com/go/bigdata