Skip to content

The Data Divide

2013 August 25
by Greg Satell

One of the great concerns of social scientists in the 21st century has been the digital divide, a stratification of society into those who have new economy skills and those who do not.

The mobile internet and cheap handsets have helped make enormous progress toward closing the digital divide, but have opened up a new rift, this time between firms who are able to use data to create value and those who are getting left behind.

To help bridge the gap, UC Berkeley’s School of Information has begun a Data Science Masters Program that will train a new breed of professionals to manage big data.  As the Web of Things becomes ever more pervasive, big data can no longer be considered something only tech companies do, but must be embraced by executives of all stripes.

A New Breed Of Businesses

We live in a disruptive age, in which the average lifespan of a Fortune 500 company is plummeting while new businesses seemingly spring out of nowhere and become multibillion dollar businesses almost overnight.

Two of the most salient examples, Google and Facebook, thought of their business as so different than what came before that they found it necessary to include letters in their prospectuses to explain their new approach, often referred to as the hacker way.  Here’s an excerpt from Mark Zuckerberg’s letter to investors:

The Hacker Way is an approach to building that involves continuous improvement and iteration. Hackers believe that something can always be better, and that nothing is ever complete. They just have to go fix it — often in the face of people who say it’s impossible or are content with the status quo.

He went on to explain that his company will eschew traditional business processes such as strategic planning and managing for shareholder return, but instead would continually experiment, shipping out new services before they were perfect and then improving them on the fly.

As recent Facebook financial results have shown, the approach has been enormously effective.

A Witches Brew Of Skills

Facebook is, of course, a company built on data from the start.  Unfortunately, most businesses are not and need to bring in new talent with new skills and that’s where things get dicey.  A report by McKinsey estimates a shortfall of 140,000 to 190,000 data scientists and 1.5 million managers who have the skills needed to use the insights to drive decisions.

AnnaLee Saxenian, the Dean of UC Berkeley’s School of Information notes that one of the difficulties in finding the right people is that big data demands a combination of skills, including basic programming, algorithmic design, visual communication and collaboration that heretofore had been separated into different silos.

While her program will help equip professionals to handle the work, Saxenian also notes that even if you have people with the right skills, senior executives in the C-Suite must also adopt a big data mindset for any effort to be ultimately successful.

And that, in fact, might be the greatest hurdle of all.

From Planning to Experimentation

Probably the biggest difference between enterprises that are native to data and others is how they approach strategy.  Legacy firms tend to undertake expensive research in order to gain a deep understanding of the marketplace.  Then highly paid strategists spend months interpreting the data, decide what it means and suggest a course of action.

Data driven firms like Facebook, Amazon and Google, on the other hand, simply run experiments—thousands upon thousands of them.  From how different shades of the same color or the placement of a button can affect actions to more complex, agent based models which simulate consumer behavior, these experiments determine what the firm will do.

A broad based study by researchers at MIT and Wharton found that firms who take a data driven approach to decisions get 5%-6% better results and McKinsey estimates that big data can increase profits in the retail sector by a whopping 60%.  This is largely because data allows firms to fail in the cheap world of bits, rather than in the expensive world of atoms.  Increasingly, innovation means simulation and data is what drives it.

However, UC Berkeley’s Saxenian notes that senior executives often have a hard time implementing big data insights because they show correlation, not causation.  In other words, they tell decision makers what, not why and that unnerves them.

Clearly, in the age of big data, incumbent firms will have to change the way they learn.  As Saxenian also points out, “when you’re trying to understand business behavior correlation is often enough.”  You can act first and then work to understand why later.

The Management Challenge

As the big data revolution gets underway, it’s becoming clear that the gap between firms will be not only of skills and investment, but of mindset.  James Manyika, a director at McKinsey’s technology practice, sees vast differences in practices even among companies in the same industry, with similar IT budgets, competing for the same consumer.

However, he also notes even “very small players, especially in health care are able to take advantage of data sources that have opened up.”  Manyika even sees great opportunities in the relatively sleepy world of components suppliers and distributors who, thanks to their unique position in the value chain, have access to huge swaths of valuable information.

Manyika urges managers to think in terms of the “3P’s” —proprietary, public and purchased data—to create entirely new business models, but warns managers not to go on a “5-year death march” in order to design their approach and advises that it if bringing in top flight data talent isn’t an option, “Go find a kid who’s a great statistician” and start looking for patterns.

Most of all, he emphasizes that it’s absolutely critical to have C-Level support.  “In almost every case, someone from the top drives it.”  And that is probably the most important insight of all.  The data divide exists not so much in skills or even technology investment, but in mentality.

– Greg

4 Responses leave one →
  1. August 25, 2013

    fascinating. lots of material to go through and think. tks

  2. August 25, 2013

    Thanks Armando!

    – Greg

  3. Sergei permalink
    August 27, 2013

    Good article, Greg. It is really a serious shift in using the DATA.

    Generally, in Management there is a shift from discrete (step-by-step) to continuous (overall) management (on-line data vs. data of former period, data flow vs. separate discrete data, creative vs. directive management, etc.).

    Sergei

  4. August 27, 2013

    Very true. Thanks Sergei.

    – Greg

Leave a Reply

Note: You can use basic XHTML in your comments. Your email address will never be published.

Subscribe to this comment feed via RSS