Tapping the Wisdom of Crowds
from The Global Knowledge Review, November 2004
James Surowiecki’s book The
Wisdom of Crowds has provoked a great deal of controversy for
espousing, and providing compelling anecdotal evidence to support, a
blasphemous idea in a society with a cult of leadership and
almost-unlimited reverence for grey-haired cognoscenti:
Any large group of modestly
informed, independent, diverse individuals will consistently and
significantly outperform any expert (or small group of experts) in
solving problems or making decisions.
The book is so delightfully written that the implications of this
message take a back seat to the entertaining and astonishing stories of
how collective wisdom has triumphed over the greatest and most
experienced minds on the planet. But those implications, for business
managers in general and for those who work in the field of knowledge
management in particular, are profound:
- If there were an effective way to tap into the collective
intelligence of large numbers of people (and in a large organization,
‘all employees’ or ‘all customers’ would probably constitute a
more-than-adequate ‘crowd’ for this purpose), the value and need for
both senior management and outside consultants would be greatly
diminished, perhaps even eliminated entirely.
- In the absence of such collective intelligence, it is very
possible, maybe even likely, that sub-optimal decisions are being made
and sub-optimal solutions implemented every day in business
organizations, with serious or even catastrophic impact on the
business’ success. The ‘cost of not knowing’
is immense: Bad purchasing, hiring, promotion and new product
development decisions, incurring unnecessary litigation, loss of a key
customer or contract, entering into a bad deal and missing out on a
great one, are just a few examples.
- Knowledge Management professionals have the skills, focus,
and access to the resources needed to create an organizational ability
to tap into the Wisdom of Crowds. They are uniquely positioned to take
advantage of the incredible opportunity that Surowiecki’s book suggests
is there for the taking, and to significantly reduce the ‘cost of not
knowing’.
There is nothing remarkably new in the aspiration to gather collective
intelligence: Lew Platt, former CEO at HP coined the now-famous
expression “If only HP knew what HP knows” a decade ago. But most KM
practitioners took this to advocate the codification of everything that
everyone in the organization has learned and written down, just in case
that knowledge was useful again, and the design of search engines and
community of practice spaces to increase the likelihood that, if it
was, the people needing that knowledge might just be able to find it.
But Dave Snowden has often made the point that even if the needed
knowledge could be found, the loss of context that occurs in the
codification process often renders that knowledge unusable, dangerous,
or even unrecognizable. What Surowiecki is talking about is just in
time knowledge, not just in case knowledge. It is the result of a
knowledge process that I’ve coined knowledge canvassing
– the ubiquitous and intuitive process of, when you don’t know the
answer to something, picking up the phone or walking down the hall and
asking someone you think might know the answer. I’ve long been an
advocate of developing more formalized knowledge canvassing processes
that could identify the best people to call, and simultaneously canvass
a larger number of people to get additional perspectives.
But Surowiecki’s book has emboldened me to think about casting a much
broader net in the canvass. What if we were to create a new process
that would automatically canvass everyone in the company and every
current and potential customer of the company, whenever there was a
critical decision to make or a critical problem to solve? Here’s what I
think such a process, based on the classical decision-making process
model used by organizations like NASA, might look like:
Suppose, for example, the problem is the failure of a new product to
meet market expectations. The process identifies four points in the
decision-making process where ‘crowds’ could add value:
- Qualifying and ranking the issues, aspects or components of
the problem (in our example, is the new product failing in all or only
certain markets?, Were the expectations unreasonable?, etc.)
- Qualifying the root causes of the problem (in our example,
they could include poor pricing, bad timing, poor marketing,
competitive disadvantages etc.)
- Qualifying and ranking alternative solutions that address
the root causes (in our example, if poor pricing was the #1 rated root
cause, solutions might include lowering the price, changing to
distribution channels where the existing pricing is more acceptable
etc.)
- Critiquing and validating the proposal to implement the solution(s).
At each applicable stage in the process, employees, customers and
prospective customers, most of them novices at decision-making, would
be canvassed for their opinions: Are these the right alternatives to
consider, and if so, in what order of priority? The Wisdom of Crowds
answers could be benchmarked against the answers of both internal
experts (marketing managers in our example) and external experts
(marketing consultants in our example). My money’s on the crowd, and
would have been even if I hadn’t read Surowiecki’s book.
This raises all kinds of interesting questions and opportunities, of course. Some things to consider:
- The model above assumes that ‘crowds’ need some limits to
the alternatives they consider; that the assessments they make must be
selections from a finite list of alternatives. Surowiecki explains that
crowds are brilliant at guessing the number of jelly-beans in a jar
(the average guess is almost always very close), determining the best
retail price for a new product, or even pinpointing the location of a
missing submarine in the Pacific Ocean. But what about more open-ended
problems? The crowd may be smart, but are they also imaginative,
creative, capable of inductive reasoning and inference? Can the crowd
solve the problem of TiVo’s struggle with profitability, or the
inability of China to produce quality products, or the inability of
pharmaceutical companies to make as high a margin on drugs that cure
killer diseases as they make on Viagra, or the dearth of new products
and ideas in the banking, insurance, and residential construction
industries? Can they give us some ideas on the best ways to combat
global warming, or help the SEC predict which company will be the next
Enron?
- How do you reward or motivate the ‘crowd’ to participate in
the problem-solving and decision-making process? As much as we want to
help our employers and suppliers make good decisions, we are already
surveyed to death. How much should we pay employees and customers to
participate? Do we ‘game’ the system so that only the participants who
come closest to the crowd consensus get paid? Or is the recognition of
being acknowledged as the wisest in the crowd, the guy who always
guesses the right number of jellybeans, reward enough?
- Surowiecki shows what anyone who has worked in the
brokerage industry already knows: That the highly-paid stock market
investment analysts and economists don’t do any better than the average
Joe at predicting where markets are going. Could broad recognition of
this fact create a crisis of confidence in markets, and in business in
general? And what will all the displaced experts, consultants, gurus
and executives do when their competency proves to be overpriced and
unneeded?
- The opportunities for using collective wisdom to reduce the
‘cost of not knowing’ is not limited to the private sector: Could the
Wisdom of Crowds have told us that there were no WMD in Iraq, or warned
us that the 9/11 attacks were coming? Could it have predicted the Great
Blackout of 2003, SARS and Mad Cow outbreaks, or the precise route of
the 2004 hurricanes? While these may seem improbable tasks for amateur
crowds to solve, some of the successes in Surowiecki’s books are just
as amazing and incredible. What if we all knew what we all know?
I confess to being something of an evangelist on this subject: I’ve
written about it so often that when you Google “The Wisdom of Crowds”,
my weblog How to Save the World
ranks behind only the book’s publisher and Amazon in the results. But
creating the infrastructure to capture collective wisdom would be
inexpensive, and unless Surowiecki’s theories turn out to be
discredited when they’re put to more demanding tests (which I don’t
think will happen) the development of canvassing processes and
technologies would seem to present enormous opportunities for companies
large and small to reduce cost of failure and risk, and to innovate
more effectively. These opportunities might even be enough to spark a
resurgence in respect and demand for Knowledge Management.
|
