Competing Against Luck: The Story of Innovation and Customer Choice

Contents

Cover

Title Page

Contents

Section 1: An Introduction to Jobs Theory

Introduction: Why You Should Hire This Book

Chapter 1: The Milk Shake Dilemma

Chapter 2: Progress, Not Products

Chapter 3: Jobs in the Wild

Section 2: The Hard Work—and Payoff—of Applying Jobs Theory

Chapter 4: Job Hunting

Chapter 5: How to Hear What Your Customers Don’t Say

Chapter 6: Building Your Résumé

Section 3: The Jobs to Be Done Organization

Chapter 7: Integrating Around a Job

Chapter 8: Keeping Your Eye on the Job

Chapter 9: The Jobs-Focused Organization

Chapter 10: Final Observations About the Theory of Jobs

Acknowledgments

Index

About the Authors

Also by the Authors

Copyright

About the Publisher

Section 1

An Introduction to Jobs Theory

We’re lost, but we’re making good time!

—Yogi Berra

Introduction: Why You Should Hire This Book

This is a book about progress.

Yes, it’s a book about innovation—and how to get better at it. But at its core, this book is about the struggles we all face to make progress in our lives.

If you’re like many entrepreneurs and managers, the word progress might not spring to mind when you’re trying to innovate. Instead you obsess about creating the perfect product with just the right combination of features and benefits to appeal to customers. Or you try to continually fine-tune your existing products so they’re more profitable or differentiated from your competitors’. You think you know just what your customers would like, but in reality, it can feel pretty hit or miss. Place enough bets and—with a bit of luck—something will work out.

But that doesn’t have to be the case, not when you truly understand what causes consumers to make the choices they do. Innovation can be far more predictable—and far more profitable—but only if you think about it differently. It’s about progress, not products. So if you are tired of throwing yourself and your organization into well-intended innovation efforts that routinely underwhelm; if you want to create products and services that you know, in advance, customers will not only be eager to buy, but willing to pay a premium price for; if you want to compete—and win—against those relying on luck to successfully innovate, then read on. This book is about helping you make progress, too.

Getting Better and Better at the Wrong Things

For as long as I can remember, innovation has been a top priority—and a top frustration—for companies around the world. In a recent McKinsey poll, 84 percent of global executives acknowledged that innovation is extremely important to their growth strategies, yet a staggering 94 percent were unsatisfied with their own innovation performance. Most people would agree that the vast majority of innovations fall far short of ambitions, a fact that has remained unchanged for decades.

On paper, this makes no sense. Companies have never had more sophisticated tools and techniques at their disposal—and there are more resources than ever deployed in reaching innovation goals. In 2015, according to an article in strategy + business,¹ one thousand publicly held companies spent $680 billion on research and development alone, a 5.1 percent increase over the previous year.

And businesses have never known more about their customers. The big data revolution has greatly increased the variety, volume, and velocity of data collection, along with the sophistication of the analytical tools applied to it. Hopes for this data trove are higher than ever. Correlation is enough,² then-Wired editor in chief Chris Anderson famously declared in 2008. We can, he implied, solve innovation problems by the sheer brute force of the data deluge. Ever since Michael Lewis chronicled the Oakland A’s unlikely success in Moneyball (who knew on-base percentage was a better indicator of offensive success than batting averages?), organizations have been trying to find the Moneyball equivalent of customer data that will lead to innovation success. Yet few have.

Innovation processes in many companies are structured and disciplined, and the talent applying them is highly skilled. There are careful stage-gates, rapid iterations, and checks and balances built into most organizations’ innovation processes. Risks are carefully calculated and mitigated. Principles like six-sigma have pervaded innovation process design so we now have precise measurements and strict requirements for new products to meet at each stage of their development. From the outside, it looks like companies have mastered an awfully precise, scientific process.

But for most of them, innovation is still painfully hit or miss. And worst of all, all this activity gives the illusion of progress, without actually causing it. Companies are spending exponentially more to achieve only modest incremental innovations while completely missing the mark on the breakthrough innovations critical to long-term, sustainable growth. As Yogi Berra famously observed: We’re lost, but we’re making good time!

What’s gone so wrong?

Here is the fundamental problem: the masses and masses of data that companies accumulate are not organized in a way that enables them to reliably predict which ideas will succeed. Instead the data is along the lines of this customer looks like that one, this product has similar performance attributes as that one, and these people behaved the same way in the past, or 68 percent of customers say they prefer version A over version B. None of that data, however, actually tells you why customers make the choices that they do.

Let me illustrate. Here I am, Clayton Christensen. I’m sixty-four years old. I’m six feet eight inches tall. My shoe size is sixteen. My wife and I have sent all our children off to college. I live in a suburb of Boston and drive a Honda minivan to work. I have a lot of other characteristics and attributes. But these characteristics have not yet caused me to go out and buy the New York Times today. There might be a correlation between some of these characteristics and the propensity of customers to purchase the Times. But those attributes don’t cause me to buy that paper—or any other product.

If a company doesn’t understand why I might choose to hire its product in certain circumstances—and why I might choose something else in others—its data³ about me or people like me⁴ is unlikely to help it create any new innovations for me. It’s seductive to believe that we can see important patterns and cross-references in our data sets, but that doesn’t mean one thing actually caused the other. As Nate Silver, author of The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t, points out, ice cream sales and forest fires are correlated because both occur more often in the summer heat. But there is no causation; you don’t light a patch of the Montana brush on fire when you buy a pint of Häagen-Dazs.

Of course, it’s no surprise that correlation isn’t the same as causality. But although most organizations know that, I don’t think they act as if there is a difference. They’re comfortable with correlation. It allows managers to sleep at night.

But correlation does not reveal the one thing that matters most in innovation—the causality behind why I might purchase a particular solution. Yet few innovators frame their primary challenge around the discovery of a cause. Instead, they focus on how they can make their products better, more profitable, or differentiated from the competition.

As W. Edwards Deming, the father of the quality movement that transformed manufacturing, once said: If you do not know how to ask the right question, you discover nothing. After decades of watching great companies fail over and over again, I’ve come to the conclusion that there is, indeed, a better question to ask: What job did you hire that product to do?

For me, this is a neat idea. When we buy a product, we essentially hire something to get a job done. If it does the job well, when we are confronted with the same job, we hire that same product again. And if the product does a crummy job, we fire it and look around for something else we might hire to solve the problem.

Every day stuff happens to us. Jobs arise in our lives that we need to get done. Some jobs are little (pass the time while waiting in line), some are big (find a more fulfilling career). Some surface unpredictably (dress for an out-of-town business meeting after the airline lost my suitcase), some regularly (pack a healthy, tasty lunch for my daughter to take to school). Other times we know they’re coming. When we realize we have a job to do, we reach out and pull something into our lives to get the job done. I might, for example, choose to buy the New York Times because I have a job to fill my time while waiting for a doctor’s appointment and I don’t want to read the boring magazines available in the lobby. Or perhaps because I’m a basketball fan and it’s March Madness time. It’s only when a job arises in my life that the Times can solve for me that I’ll choose to hire the paper to do it. Or perhaps I have it delivered to my door so that my neighbors think I’m informed—and nothing about their ZIP code or median household income will tell the Times that either.

This core insight emerged in the course I teach at Harvard Business School, but has subsequently been refined and shaped over the past two decades by numerous conversations with my coauthors, trusted colleagues, collaborators, and thought-leaders. It’s been validated and proven in the work of some of the world’s most respected business leaders and innovators—Amazon’s Jeff Bezos and Intuit’s Scott Cook, for example—as well as in the founding of highly successful entrepreneurial ventures in recent years. Who would have imagined that a service that makes travelers pay to stay in a stranger’s spare bedroom would be valued at more than Marriott, Starwood, or Wyndham Worldwide? Airbnb did it. The videos that Sal Khan made to teach math to his young cousin were, by his description, cheaper and crappier than many other educational videos already online, but they now enable millions of students all over the world to learn at their own pace.

These innovations weren’t aimed at jumping on the latest trends or rolling out another new flavor to boost sales. They weren’t created to add more bells and whistles to an existing product so the company could charge customers more. They were conceived, developed, and launched into the market with a clear understanding of how these products would help consumers make the progress they were struggling to achieve. When you have a job to be done and there isn’t a good solution, cheaper and crappier is better than nothing. Imagine the potential of something truly great.

This book is not focused on celebrating past innovation successes, however. It’s about something much more important to you: creating and predicting new ones.

The foundation of our thinking is the Theory of Jobs to Be Done, which focuses on deeply understanding your customers’ struggle for progress and then creating the right solution and attendant set of experiences to ensure you solve your customers’ jobs well, every time. Theory may conjure up images of ivory tower musings, but I assure you that it is the most practical and useful business tool we can offer you. Good theory helps us understand how and why. It helps us make sense of how the world works and predict the consequences of our decisions and our actions. Jobs Theory⁵, we believe, can move companies beyond hoping that correlation is enough to the causal mechanism of successful innovation.

Innovation may never be a perfect science, but that’s not the point. We have the ability to make innovation a reliable engine for growth, an engine based on a clear understanding of causality, rather than simply casting seeds in the hopes of one day harvesting some fruit.

The Theory of Jobs to Be Done is the product of some very real-world insights and experiences. I’ve asked my coauthors to work with me on this book in part because they’ve been using Jobs Theory in their everyday work for years and have much experience bringing the theory into the practical realm of innovation. Together we have shaped, refined, and polished the theory, along with the thoughts and contributions of many trusted colleagues and business leaders, whose work and insights we’ll feature throughout this book.

My coauthor Taddy Hall was in my first class at Harvard Business School and he and I have collaborated on projects throughout the years, including coauthoring with Intuit founder Scott Cook the Harvard Business Review (HBR) article Marketing Malpractice that first debuted the Jobs to Be Done theory in the pages of HBR. He’s currently a principal at the Cambridge Group (part of the Nielsen Company) and leader of the Nielsen Breakthrough Innovation Project. As such, he has worked closely with some of the world’s leading companies, including many of those mentioned throughout this book. More important, he’s used Jobs Theory in his innovation advisory work for years.

Karen Dillon is the former editor of Harvard Business Review and my coauthor on How Will You Measure Your Life? You’ll see her perspective as a longtime senior manager in media organizations struggling to get innovation right reflected in this book. Throughout our collaboration, she has seen her role as that of a proxy for you, the reader. She is also one of my most trusted allies in helping bridge the worlds of academia and practitioners.

David S. Duncan is a senior partner at Innosight, a consulting firm I cofounded in 2000. He’s a leading thinker and adviser to senior executives on innovation strategy and growth, helping them to navigate disruptive change, create sustainable growth, and transform their organizations to thrive for the long term. The clients he’s worked with tell me they’ve completely changed the way they think about their business and transformed their culture to be truly focused on customer jobs. (One client even named a conference room after him.) Over the past decade, his work in helping to develop and implement Jobs Theory has made him one of its most knowledgeable and innovative practitioners.

Throughout the book, we’ve primarily chosen to use the first-person I simply to make it more accessible for readers. But we have written this book as true partners; it’s very much the product of a collaborative we and our collective expertise.

Finally, a quick roadmap of the book: Section 1 provides an introduction to Jobs Theory as the causal mechanism fueling successful innovation. Section 2 shifts from theory to practice and describes the hard work of applying Jobs Theory in the messy tumult of the real world. Section 3 outlines the organizational and leadership implications, challenges, and payoffs posed by focusing on Jobs to Be Done. To facilitate your journey through each of these sections of the book and to maximize its value to you, at the outset of each chapter we’ve included The Big Idea as well as a brief recap of Takeaways. At the end of chapters 2 to 9, we’ve included a list of questions for leaders to ask their organizations, with the aim of helping executives start to put these ideas into practice.

Our preference is to show through examples more than to tell in the form of assertion or opinion. As is true in discovering Jobs to Be Done, we find that stories are a more powerful mechanism for teaching you how to think, rather than just telling you what to think—stories that we’ll weave throughout the book. Our hope is that in the process of reading this book, you will come away with a new understanding of how to improve your own innovation success.

What Job Did You Hire That Product to Do?

Organizations around the world have devoted countless resources—including time, energy, and mindshare of top executives—to the challenge of innovation. And they have, naturally, optimized what they do for efficiency. But if all this effort is aimed at answering the wrong questions, it’s sitting on a very tenuous foundation.

As W. Edwards Deming is also credited with observing, every process is perfectly designed to deliver the results it gets. If we believe that innovation is messy and imperfect and unknowable, we build processes that operationalize those beliefs. And that’s what many companies have done: unwittingly designed innovation processes that perfectly churn out mediocrity. They spend time and money compiling data-rich models that make them masters of description but failures at prediction.

We don’t have to settle for that. There is a better question to ask—one that can help us understand the causality underlying a customer’s decision to pull a new product into his or her life. What job did you hire that product to do? The good news is that if you build your foundation on the pursuit of understanding your customers’ jobs, your strategy will no longer need to rely on luck. In fact, you’ll be competing against luck when others are still counting on it. You’ll see the world with new eyes. Different competitors, different priorities, and most important, different results. You can leave hit-or-miss innovation behind.

Endnotes

1. Jaruzelski, Barry, Kevin Schwartz, and Volker Staack. Innovation’s New World Order. strategy+business, October 2015.

2. Anderson, Chris. The End of Theory: The Data Deluge Makes the Scientific Method Obsolete. Wired, June 23, 2008.

3. My son Spencer was a really good pitcher in our town’s Little League. I can still see his big hands wrapped around the ball, his composure when a tough batter was at the plate, the way he’d regroup after each pitch with renewed focus. He was unflappable in some very big moments. Someplace there is data that will tell you the number of games he won and lost, how many balls and strikes he threw, and so on. But none of that will ever tell you why. Data is not the phenomenon. It represents the phenomenon, but not very well.

4. During the 1950s, the US Air Force realized that pilots were having trouble controlling their planes. As recounted by Todd Rose, director of the Mind, Brain, and Education program at the Harvard Graduate School of Education, in The End of Average, the Air Force first assumed the problem was poor training or pilot error. But it turned out that wasn’t the problem at all. The cockpits had a design flaw: they had been built around the average pilot in the 1920s. Since it was obvious that Americans had gotten bigger since then, the Air Force decided to update their measurements of the average pilot. That involved measuring more than four thousand pilots of nearly a dozen dimensions of size related to how they’d fit into a cockpit. If those cockpits could be redesigned to fit the average pilot in the 1950s, the problem should be solved, the Air Force concluded. So how many pilots actually fell into the definition of average after this enormous undertaking? None, Rose reports. Every single pilot had what Rose called a jagged profile. Some had long legs, while others had long arms. The height never corresponded with the same chest or head size. And so on. The revised cockpits designed for everyone actually fit no one. When the Air Force finally swept aside the baseline assumptions, the adjustable seat was born. There’s no such thing as average in the real world. And innovating toward average is doomed to fail. Rose, Todd. The End of Average: How We Succeed in a World That Values Sameness. New York: HarperCollins, 2015.

5. Throughout the book, we use the Theory of Jobs to Be Done and Jobs Theory interchangeably. They mean the same thing.

Chapter 1

The Milk Shake Dilemma

The Big Idea

Why is innovation so hard to predict—and sustain? Because we haven’t been asking the right questions. Despite the success and enduring utility of disruption as a model of competitive response, it does not tell you where to look for new opportunities. It doesn’t provide a road map for where or how a company should innovate to undermine established leaders or create new markets. But the Theory of Jobs to Be Done does.

Why is success so hard to sustain?

That question nagged at me for years. In the early years of my career, I had the opportunity to work closely with many companies that were in trouble, first as a consultant for Boston Consulting Group and then as the CEO of my own company, CPS Technologies, a company I founded with several MIT professors to make products out of a set of advanced materials they had developed. And I witnessed firsthand how a lot of smart people were unable to fix the problems of once-great companies. At that same time, I watched the rise of a local Boston company, Digital Equipment Corporation (DEC), as it became one of the most admired in the world. Whenever you read explanations about why it was so successful, inevitably its success was attributed to the brilliance of the company’s management team. Then about 1988 Digital Equipment fell off the cliff and began to unravel very quickly. When you then read explanations about why it had stumbled so badly, it was always attributed to the ineptitude of the management team, the same folks running the company who had earned unfettered praise for so long.

For a while, the way I framed it was, Gee, how could smart people get so stupid so fast? And that is the way most people accepted the demise of DEC: somehow the same management team that had its act together at one point was out of its league at another. But the stupid manager hypothesis really didn’t hold up when you considered that almost every minicomputer company in the world collapsed in unison.

So when I returned to Harvard Business School (HBS) for my doctorate, I brought with me a set of puzzles to try to answer as an academic. Was there something other than bad management that played a key role in the demise of these great companies? Were they only successful in the first place because they’d gotten lucky in some way? Had these incumbents fallen behind the times, relied on antiquated products, and just lost their step as more nimble competitors appeared? Was the creation of new successful products and businesses intrinsically a crapshoot?

But after diving into my research, I realized that my initial assumptions were wrong. What I found was that even the best professional managers—doing all the right things and following all the best advice—could lead their companies all the way to the top of their markets and then fall straight off a cliff after arriving there. Nearly all the incumbents in the industry I studied—disk drive manufacturers—were eventually beaten by new entrants with cheaper and initially far inferior offerings—what I called disruptive innovations.

That work led to my theory of disruptive innovation,¹ which explains the phenomenon by which an innovation transforms an existing market or sector by introducing simplicity, convenience, accessibility, and affordability where complication and high cost have become the status quo—eventually completely redefining the industry.

At its core, it’s a theory of competitive response to an innovation. It explains and predicts the behavior of companies in danger of being disrupted, providing insight into the mistakes incumbent leaders make in response to what initially seem to be minuscule threats. It also provides a way for incumbents to predict what innovations on the horizon are likely to be the greatest disruptive threats. But over the past two decades, the theory of disruption has been interpreted and misapplied so broadly as to mean anything that’s clever, new, and ambitious.

But the theory of disruptive innovation does not tell you where to look for new opportunities. It doesn’t predict or explain how, specifically, a company should innovate to undermine the established leaders or where to create