The New Science Of Effective Organizations

A recent survey reported that the consulting industry raked in $39.3 billion in 2012 renting out their advice to the world’s most powerful firms.  Yet still, a study by Innosight found that the average life on the S&P 500 has declined from 61 years in 1958 to only 18 today.

It makes you wonder what all those high-priced consultants are getting paid for.  With all of their fancy charts and two-by-two matrices, are they really anything more than modern day witch doctors dispensing a high tech version of folk wisdom?

We’re about to find out now that Sandy Pentland is on the case.  As one of the world’s top data scientists, he has developed a technique called reality mining that tracks human behavior through the use of a sociometer device he invented.  He’s recently published a book called Social Physics that explains his research into what really works in business.

What Get’s Done On A Coffee Break?

Many managers like to run a tight ship that is focused on the task at hand—all business and no chit chat.  Yet Pentland’s research has found that the most important predictor of success in a group is the amount—not the content—of social interaction.  It is exposure to peer activity that drives learning and changes in behavior.

Generally, this has not been controversial in high-level professional environments, but it is generally discounted in jobs with clear cut goals and easy to monitor procedures.  Yet, as Pentland describes an article published in Harvard Business Review, he found that it is just as important in even low-level jobs.

After equipping teams in a call center with sociometric badges, he found that interaction and engagement outside formal meetings accounted for one third of the variation of dollar productivity between groups.  Pentland suggested that instead of sending people on coffee breaks individually, they send whole teams together as a group.

This simple change resulted in a $15 million increase in productivity and it cost nothing to implement.  On other projects, seemingly innocuous changes like increasing the size of lunch tables to encourage interaction among more people also proved effective.  Chit chat, it turns out, is not only permissible, but profitable.

Cohesiveness and Diversity

Managers tend to look for people who are a “good fit” with the rest of their team, because cohesive groups tend to be able to come to an easy consensus and take action quickly. However, such teams also tend to fall into groupthink, where the wrong ideas get reinforced just as much as the right ones do.

In a study of currency traders, Pentland found that the most successful performers diversified their sources of information.  Some would be successful for a while if they hit on a successful strategy, but then would continue to follow it even after it became a loser.  Echo chambers are double-edged swords.

In truth, what really makes teams perform is a mix of both cohesiveness and diversity, with strong communication within the group, but most of the their time spent on far flung interactions outside of it.  Pentland is not alone in reaching this conclusion.  A study of star performers at Bell Labs and another of informal company networks found the same thing.

In my own experience, I have seen many teams that come to a strong consensus on seemingly innovative strategies because they all read the same two or three tech blogs, which are, in fact, merely parroting each other.  These ideas then get filtered down to 3-5 bullet points that reverberate through various industry conferences.

The Red Balloon Challenge

It used to be that successful organizations thrived because they knew how to do a few things really well.  They honed their skills and continuously improved at specific tasks.  Techniques like Six Sigma helped them achieve amazing precision and minimize errors.  The problem is that these techniques also limit your ability to take in new information.

Yet today, it’s crucially important to network your organization so that outside information gets in.  It was that idea that inspired Pentland’s strategy in the Darpa Red Balloon challenge in which teams competed to find ten red weather balloons distributed across the continental United States.  The prize was substantial: $40,000

While other teams focused on various search strategies, Pentland’s MIT group used a recursive method to build a network that built in both financial and social incentives. Volunteers were not only compensated for finding a balloon, but for referring someone else that found a balloon.  Pentland’s team won the challenge in less than nine hours.

What’s interesting is that in follow-up interviews, volunteers said that a primary motivation for recruiting others was not only to win a prize, but to strengthen social ties, similar to pooling office money to win a lottery.

What The Future Will Look Like

Historically, the study of management has been a mixture of folk wisdom and fads.  We looked at successful organizations, examined their lore and did our best to emulate them. Business schools and consultancies then perpetuated prominent ideas through case studies and PowerPoint presentations.

Pentland points the way to a new, more scientific approach, where we directly observe work practices and interactions in real time to create new learning organizations.  He foresees sociometers becoming small enough to be embedded in standard corporate badges and integrated into dashboards so that managers can monitor their networks.

Pentland is not alone.  Analytics firm GoodData has begun offering a Yammer analytics platform that tracks employee sentiment and identify influencers and topics of conversation. Icosystem uses similar data to build agent based models to predict effect of future actions. Undercurrent’s Aaron Dignan also recently wrote about new forms of organization such as holocracy, agile squads and self-organizing firms.

One thing is for sure, we need to change the software of our organizations.  But this time, the basis for the change won’t be a management fad or a case study derived from a handful of isolated success stories, it will be data, collected in real time and applied in the real world.

– Greg