by Rod Collins
With the rapid expansion of the digital technologies of mass collaboration, our capacity to tap into what has become known as the “wisdom of crowds” has become a common experience. On a regular basis, literally billions of us rely upon Wikipedia as our primary reference source and rarely a day goes by that we don’t do a Google search. Both of these applications leverage the collective intelligence of their users as the foundation for their operating systems.
In a 2012 Future Trends Report from the DaVinci Institute, futurist Thomas Frey argues that the recent popularity of the notion of the “wisdom of crowds” is nothing more than a passing fad, and he suggests that this vogue idea will be replaced by three emerging forces: a new generation of “super-influencers,” instant data-mining-based decision making, and what Frey calls “real human intelligence.” Frey seems to imply that a new breed of experts with new technological tools will restore our confidence in the supremacy of individual genius.
As evidence of the folly of the wisdom of crowds, Frey cites the shallowness of many of our past popularly held beliefs. For example, before 1929, it was common knowledge that stocks were a great place to put your money, and up until the recent financial collapse, no one questioned that real estate values would always go up.
While Frey correctly makes the point that crowds are not always wise, perhaps he paints with too broad a brush by totally discounting the crowd. Before dismissing the multitude, it may be important to understand the distinction between complicated problems and complex problems, and we may need to be careful not to confuse collective wisdom with conventional wisdom.
Whether the wisdom of crowds is reliable depends upon the circumstances. It depends upon whether the problem to be solved is complex or complicated. For most of the Industrial Age, our challenges have been mechanical problems, and mechanical problems are generally issues of complication. Solutions to mechanical puzzles often involve the search for the elusive fundamental cause – often referred to as the “silver bullet” – that is driving the undesired effect. If you need to find a “silver bullet,” rely on an expert. That’s why, in an airplane at 30,000 feet, we usually trust the pilot over the crowd. However, when the issue involves a complex problem, we are more likely to be dealing with a paradox than a puzzle. Thus, the solution is more likely to emerge from a holistic blending of several seemingly unrelated perspectives rather than from the search for a “silver bullet” by a trained expert. When this is the case, we often find that diversity trumps ability as we witnessed in the summer of 2011 when a group of online gamers on Foldit solved in ten days a difficult problem in AIDS research that had alluded the world’s best expert scientists for ten years.
The reliability of the wisdom of crowds also depends upon whether the conclusions are based on conventional wisdom or collective wisdom. The notion of the wisdom of the crowds was recently the subject of a book by James Surowiecki. He points out that crowds often outperform experts when four conditions are present: 1) diversity of opinion, 2) independent thinking, 3) local knowledge, and most importantly, 4) an aggregation mechanism. Without all four conditions, then the crowd can easily descend into the conventional wisdom of an unintelligent mob or myopic groupthink.
The debunked popular beliefs about the stock market and real estate values are examples of conventional wisdom. As Frey rightly points out, this type of wisdom is not always reliable because it is highly susceptible to bias from undue social influences. However, when Surowiecki’s four conditions are present, we have the capacity to tap into what is arguably the highest form of human intelligence, our collective intelligence. Perhaps that explains the incredible performance of the Foldit online gamers.
Until recently, tapping into our collective intelligence was not practical because we didn’t have the aggregation mechanisms to integrate our best thinking. Without these mechanisms, so-called super-influencers have been able to both dominate and color the thinking of the crowd. However, that’s all changing thanks to the digital revolution. Advances in digital technology have made wikis, Google and other forms of open sourcing possible. They have expanded our ability to aggregate human intelligence in the form of sophisticated data-mining technology. Thus, rather than displacing the wisdom of the crowd, this force is more likely to be an accelerator of our collective intelligence. When this happens, we will discover that the “real human intelligence” that Frey refers to is actually nothing more than our mastery of our emerging capacity for collective intelligence. The next generation of super-influencers will become a diminishing force as more of us gain access to our real and highest form of human intelligence.
While it is true that not all crowds are wise, sometimes crowds do outperform the experts. Given a complex problem and the presence of the right conditions, the wisdom of crowds is a reality and not a myth. There are circumstances where nobody really is smarter or faster than everybody.