Data, Data Everywhere
Ours is a data rich world. Every five years, the amount of digital information increases tenfold. And data are the new coin of realm in business, an economic input almost on a par with capital and labor. Big data—large pools of data that can be captured, aggregated, stored, analyzed and communicated —is now part of every sector and function of the global economy. “It’s a revolution,” says Gary King, director of Harvard’s Institute for Quantitative Social Science. “We’re really just getting under way. But the march of quantification, made possible by enormous new sources of data, will sweep through academia, business and government. There is no area that is going to be untouched.”
It’s not just data from business transactions. Significantly, “information created by machines and used by other machines will probably grow faster than anything else…This is primarily ‘database to database’ information—people are only tangentially involved in most of it.”
But data is also being used in ever more ways within business and government. In the public sector, for example, data transparency is making relevant information more readily accessible across otherwise separated departments which in turn can sharply reduce search and processing time.
The presence of data can fundamentally enable the use of experiments in business and government. Using data to analyze variability in performance—that which either occurs naturally or is generated by controlled experiments—and to understand its root causes, can enable leaders to manage performance to higher levels. Internet companies like Google, Facebook and Amazon routinely use randomized experiments to conduct commerce.
Finding the Value of Big Data
Big data’s all well and good, but it’s the meaning that derives from algorithms applied to the data that creates value. That’s where analytics -- the use of data and related insights developed through applied quantitative disciplines (e.g. statistics, econometrics, biometrics, computer science, artificial intelligence, cognitive science and operations research) to drive evidence-based planning, decision-making, management, measurement, learning and product development – comes in.
Data and analytics allow organizations to create highly specific segmentations and to tailor products and services precisely to meet those needs. Long done in marketing and risk management, segmentation is now becoming a tool of government and health care.
Sophisticated analytics combined with data can substantially improve routine business decision-making, help minimize risks, and unearth valuable insights that would otherwise remain hidden. These analytics have applications for organizations -- from tax agencies that can use automated risk engines to flag candidates for further examination, to retailers that can use algorithms to optimize fine-tuning of inventories and pricing in response to real-time in-store and online sales.
Introducing Data Science
Big data enables companies to create new products and services, enhance existing products, and invent entirely new business models. The emergence of “data science” as a discipline that creates new data products drives business models at companies like Google, Facebook, Amazon, Harrah’s, Capital One and LinkedIn.
Those with data science skills – business domain knowledge, data/programming expertise, and data analysis/storytelling/statistical acumen – are in high demand. The U.S. alone faces a shortage of 140,000 to 190,000 analytical practitioners as well as 1.5 million managers to analyze big data and make decisions based on their findings.
And what indications do we have that data-driven business is effective? Surveys by IBM highlight a marked increase in the percentage of data-driven businesses from 2010 to 2011 that note a competitive advantage from the use of analytics. And research by Erik Brynjolfsson of MIT suggests that data-guided management is spreading across corporate America and starting to pay off. An investigation of 179 large companies found that those adopting “data-driven decision making” achieved productivity gains that were five to six percent higher than other factors could explain.
The use of big data and analytics in business is here to stay. The winners will be those companies that adopt an evidenced-based business culture and make the investments to both manage data as an asset and promote the effective use of analytics in their organizations. The successful companies might well adopt the mantra of Rollin Ford, the CIO of Wal-Mart: “Every day I wake up and ask, ‘how can I flow data better, manage data better, analyze data better?”