Big Data: Not Just Big, but Fast and Smart, Too

I am attending South Korea’s Big Data Forum in Seoul, and one question here is, “How big is Big Data?” My friend and colleague Dave Evans has pointed out that by the end of this year, more data will be created every 10 minutes than in the entire history of the world up to 2008. Now, that’s big!

Much of this data is being created by billions of sensors that are embedded in everything from traffic lights and running shoes to medical devices and industrial machinery—the backbone of the Internet of Things (IoT). But the real value of all this data can be realized only when we look at it in the context of the Internet of Everything (IoE). While IoT enables automation through machine-to-machine (M2M) communication, IoE adds the elements of “people” and “process” to the “data” and “things” that make up IoT. Analytics is what brings intelligence to these connections, creating endless possibilities.

To understand why, let’s step back and take a look at the classic approach to Big Data and analytics. Traditionally, organizations have tended to store all the data they collect from various sources in centralized data centers. With this model, if a retailer wants to know something about the buying patterns of a certain store’s customers, it can create an analysis of loyalty card purchases based on data in the data warehouse. Collecting, cleansing, overlaying, and manipulating this data takes time. By the time the analysis is run, the customer has already left the store.

Big Data today is characterized by volume, variety, and velocity. This phenomenon is putting a tremendous strain on the centralized model, as it is no longer feasible to duplicate and store all that data in a centralized data warehouse. Decisions and actions need to take place at the edge, where and when the data is created; that is where the data and analysis need to be as well. That’s what Cisco calls “Data in Motion.” With sensors gaining more processing power and becoming more context-aware, it is now possible to bring intelligence and analytic algorithms close to the source of the data, at the edge of the network. Data in Motion stays where it is created, and presents insights in real time, prompting better, faster decisions.

So what does this have to do with the Internet of Everything? IoE is all about the value of the connections among people, process, data, and things. And Big Data and analytics are what help us mine that value. With data and analytics capabilities at both the edge and the center of the network, we now have real-time insight and intelligence to fuel immediate decisions and actions. For example:

Instead of just running a report about shoppers’ buying patterns, retailers can now sense when a shopper is in an aisle (where he or she is likely to need a product), and deliver an instant, personalized promotional offer to the shopper’s mobile phone.

“Smart” streetlights in central Nice, France conserve energy by adjusting their lighting level based on current time, weather, traffic conditions, and the number of pedestrians in the area.

Also in Nice, connected garbage dumpsters can sense when they are full and request a pickup. They can even request an early pickup if the temperature rises and odor becomes an issue.

These are a few examples of solutions we are working on with customers right now—and they are just the beginning. In my mind, what makes Big Data and analytics truly big is measured not in exabytes and petabytes, but in the value of the new processes, business models, and insightful, repeatable decisions they make possible in the Internet of Everything economy.