How To Be A Lean Product Ninja

+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+
How to be a
Lean Product
Ninja
Dan Olsen
Olsen Solutions
October 3, 2013
+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+
My Background
n
Educa7on
n
n
n
n
BS, Electrical Engineering, Northwestern

MS, Industrial Engineering, Virginia Tech
MBA, Stanford
Web development and UI design
20 years of Product Management Experience

n
n
n
n
n
Managed submarine design for 5 years

5 years at Intuit, led Quicken Product Management
Led Product Management at Friendster
CEO & Cofounder of YourVersion, Pandora for your news
PM consultant to startups: Box.net, YouSendIt, Epocrates

Copyright 2013 Olsen Solu7ons
What does Lean mean?

n
Lean Startup
n Achieving product-market t
n Valida7ng product with users
n Improving & itera7ng your
product quickly
n Tes7ng hypotheses & learning
n
Product-market t = product
managements responsibility

Copyright 2011 YourVersion
Why Ninja?
n High level of exper7se
n Self-sucient
n Uses wide array of skills
to get the job done
Whats the Formula for

Product-Market Fit?
n
A product that:
n Meets customers needs
n Is beaer than other alterna7ves
n Is easy to use
n Has a good value/price
Focus of my
talk today
Will cover basics. See
my talk How to be a
UX Design Army of
One
The Lean Product Ninja Way:

Steps to Achieving Product-Market Fit
1.
2.
3.
4.
5.
Understand underserved customer needs

Dene your products value proposi7on
Build wireframe/mockup
Get user feedback
Iterate
Understanding
Customer
Needs
Problem Space vs. Solu7on Space

n Problem Space
n A customer problem,
need, or benet that the

product should address
n A product requirement
Solu7on Space
n A specic
implementa7on to
address the need or
product requirement

Example:
n Ability to write in space n
NASA: space pen
(zero gravity)
M R&D cost)
($1
n Russians: pencil
Problem Space vs. Solu7on Space:

Product Level
Problem Space
(user benet)
Solu7on Space
(product)
Prepare
my taxes
Pen and
paper
Check my
taxes
File my
taxes
TurboTax
Maximize
deductions
Reduce
audit risk
TaxCut
Problem vs. Solu7on Space: Feature Level

Problem Space

Help me
Empowerement/
Confidence
Save Time
Save Money
Solu7on Space
prepare taxes
Tax Interview
Wizard
Reduce my
audit risk
Audit Risk
Analyzer
Check my
return
Tax Return
Error Checker
Save time
preparing taxes
Tax Data
Downloader
Save time filing

taxes
Electronic Tax
Return Filing
Maximize my
tax deductions
Tax Deduction
Finder
Priori7zing Benets & Features based

on Customer Value
n How do you priori7ze:
n Which user benets should you address?
n Which product features to build (or improve)?
n Priori7za7on should be based on customer
value
n Importance vs. Sa7sfac7on Framework
n Importance of user need (problem space)
n Sa7sfac7on with how well a product meets the
users need (solu7on space)
Importance of User Need
High Importance + Low Sa7sfac7on =

Opportunity to Add Customer Value
High
Opportunity
Compe77ve
Market
Not Worth Going Aler

Low
Low
High
User Sa7sfac7on with Current Alterna7ves
Kano Model: User Needs & Sa7sfac7on

User Sa7sfac7on
Delighter (wow)
Performance
(more is beaer)
Need
not met
Need
fully met
Must Have
Needs & features
migrate over 7me
User Dissa7sfac7on
Olsens Hierarchy of Web User Needs

(adapted from Maslow)
What does it mean to us?
How easy to use is it?
Usability & Design
Does the functionality

meet my needs?
Increasing
Satisfaction
Customers Perspective
Does the functionality work?
Is the site up when I want to use it?
Absence of Bugs
Decreasing
Dissatisfaction
Is the site fast enough?
Feature Set
Page Load Time
Uptime
Importance vs. Sa7sfac7on

Ask Users to Rate for Each Feature
100
Great
95
84 87
90
Bad
85
Importance
98
70
55
80
79
75
80
72
70
86
84
80
75
65
60
55
41
50
40
50
60
70
80
90
100
Satisfaction
Recommended reading: What

Customers Want by Anthony Ulwick
What is Your Value Proposi7on?

n Which user benets are you providing?
n How are you beaer than compe7tors?
Compe&tor A Compe&tor B
Must Have Benet 1
You
Performance Benet 1
High
Low
Med
Performance Benet 2
Low
High
Low
Performance Benet 3
Med
Med
High
Delighter Benet 1
Delighter Benet 2
Return (Value Created)
Priori7zing Product Ideas by ROI

4
?
Idea D
3
Idea A Idea B
2
Idea C
1
Idea F
1
2
3
4
Investment (developer-weeks)
Building
Wireframes/
Mockups
Dans UX Design Iceberg

What most
people see
and react to
Visual
Design
Interaction
Design
What good
product
people
think about
Information
Architecture
Conceptual
Design
The Elements of User Experience

by Jesse James Garrea
Diagram available
at www.jjg.net
Elements of User Experience Design

Consists of Three Dis7nct Elements:
n Informa7on Architecture
n
n
n
n
Interac7on Design
n
n
n
Structure and layout at both site and page level

How site is structured (sitemap)
How site informa7on is organized (site layout)
How each page is organized (page layout)
How user and product interact with one another
User ows (e.g., naviga7on across mul7ple pages)
User input (e.g., controls and form design)
Visual Design
n
n
How it looks vs. What it is, olen called chrome

Fonts, colors, graphical elements
Geqng Customer Feedback:

Problem Space vs. Solu7on Space
n Customers CANT ar7culate problem space
n Customers CAN react to solu7on space
n Thats why you need:
n Wireframes: low to medium delity visual
representa7on of your product

n Mockups: high delity visual representa7on of
your product
n Prototype: interac7ve version of your product
that emulates its func7onality (throw-away)
Design Ar7facts:
Interac7vity vs. Fidelity
Interactivity
Alpha
Prototype
Hand
sketch
Clickable
Wireframe*
Clickable
Mockup**
Static
Wireframe*
Mockup
Fidelity
* Balsamiq: balsamiq.com
** Invision: invisionapp.com
Design Tools:
Fidelity vs. Eort
Design Fidelity
Photoshop
Non-designer
Product Person
2
1
Balsamiq
Visual
Designer
Hand
sketch
Effort to Create Artifact

You Should Wireframe

n In case you dont have UX designer (olen)
n To clarify and rene your thinking:
n Informa7on Architecture, layout, naviga7on
n To communicate your product ideas to
others
n Modern tools make it easy and fast
Why Everyone Should Use Balsamiq

n Super easy to learn and use
n Widget library has most UI elements
n Can annotate with s7cky notes
n Can add click naviga7on between pages to
illustrate a user story

n Can export as PDF (so others dont need to
have Balsamiq), mul7ple pages in 1 PDF
n Best $79 youll spend
n hap://www.balsamiq.com
Learning from
Customers
What Are You Going to Get Feedback &

Learnings About?
Problem Space
(your mental model)
Solu7on Space
(what users can react to)

Feature Set
Customer
Understanding
(needs &
preferences)
UI Design
Messaging
Ramen User Feedback for Startups

n Anyone can do it!
n Ingredients:
n Solu7on-space product/mockup to test
n 1 customer (with laptop if tes7ng code)
n 1 desk
n 1 person to conduct the session
n Pen and paper
n Op7onal note-taker and observers
Typical Format for Customer Session

5 - 10 min: Ask ques7ons to understand user
needs and solu7ons they currently use
n 30 - 50 min: User feedback
n
n Show user product/mockup
n Non-directed as much as possible
n When necessary, direct user to aaempt to
perform a specic task
5 - 10 min: Wrap-up
n Answer any user ques7ons that came up
n Point out/explain features you want to highlight

n Ask them if they would use the product
Dos & Donts of User Feedback Sessions

n
Do
n
Explain to the user:
Their feedback will help improve the product

n Not to worry about hur7ng your feelings
n Think Aloud Protocol
n
n
n
n
Ask user to aaempt the task, then be a y on the wall

Ask non-leading, open-ended ques7ons
Take notes and review them alerwards for take-aways
Dont
n
n
n
n
n
Ask leading ques7ons

Help the user or explain the UI (e.g., click over here)
Respond to user frustra7on or ques7ons (un7l test is over)
Get defensive
Blame the user
Iterate Your
Wireframes Based
on Feedback
Itera7ng Your Product Vector Based on

User Feedback in Solu7on Space
Problem Space
(your mental model)

Help user
book travel
Solu7on Space
(what users can react to)
Help user
plan travel
Mockups / Code
Customer Feedback
Geqng Feedback on Your V1 Product

How others see your v1
How you see your v1
How much do you like the

elephant?
Progress = absence of
issues & concerns that
customers raised in
prior versions
Persevere or Pivot?
Increasing
Product-Market
Fit
Pivot
Product-Market Fit =
Geqng enough data to
validate that youre
climbing up the
right mountain
Pivot
Case Study on Product-Market Fit:

Marke7ngReport.com
Product-Market Fit Case Study:

Marke7ngReport.com
n My consul7ng client, CEO of TrustedID,
had an idea for a new product

n Team: me, CEO, head of marke7ng,
UI design consultant
n Goal:
n Validate product-market t quickly, cheaply
without wri7ng a single line of code

n Determine if their was a business
opportunity here

Developing Product Concept
Product Concept was marke7ng report that
let consumers control the direct mail that they
receive
n Concept was fuzzy with various components,
so I broke it into 2 dierent avors:
n
n #1 Marke7ng Shield: Service to reduce/stop
junk mail
n #2 Marke7ng Saver: Opt in & receive money-
saving oers
Clustering Poten7al User Benets to

Create Product Concepts
Shield Concept
Saver Concept
Reduce
Junk Mail
Find out what

they know
about you
Save
Trees
Marketing
Report
Marketing
Score
Money
Saving
Offers
Compare
Yourself
to Others
Social
Networking
Marketing
Profile

Recrui7ng People
n
Telephone recruit of prospec7ve customers

n Wrote phone-screen ques7onnaire to create rough
target customer segmenta7on
n Wanted users who work full-7me & use internet

n Fit for opt-in oers: use coupons, Costco membership
n Fit for an7-junk mail: use paper shredder, block caller ID
Paid each person $75

n Scheduled 3 groups of 2 or 3 people to discuss
each product concept for 90 minutes
n Moderated each group through the paper
mockups to hear their feedback
n

Findings on Concepts &User Benets
Shield Concept
Saver Concept
Reduce
Junk Mail
Find out what

they know
about you
Save
Trees
Marketing
Report
Marketing
Score
Marketing
Profile
Money
Saving
Offers
Compare
Yourself
to Others
Social
Networking
Legend
Strong appeal
Some appeal
Low appeal

Learnings from Research
n
n
n
n
Learned that Shield (an7-junk mail) concept was

stronger than Saver
People didnt like many of the Saver concept
components
Learned users concerns / ques7ons about Shield
concept
Rened Shield concept:
n
n
Removed irrelevant components

Improved messaging to address user concerns / ques7ons
Validated revised Shield concept with quick 2nd

round of tests
n
n
No customer concerns
Clear willingness to pay

Summary
n 4 weeks from 1st mee7ng to validated
product concept with zero coding

n Reasonable cost
n 1 round of itera7on on product concept
n Iden7ed compelling concept that users
are willing to pay $10/month for
n Trimmed away non-valuable pieces
n You can achieve similar results
Metrics to Validate Product-Market Fit

n
Survey results
n
Importance & Sa7sfac7on

Net Promoter Score
Survey.io/Qualaroo
How would you feel if you could no longer use Product X?

n
Very disappointed, Somewhat disappointed, Not disappointed
User behavior
n
n
n
Prospects sign up (high conversion rate)

They keep using it (high reten7on rate)
They use it olen (high frequency of use)
Op7miza7on with Metrics:

Dene the Equa7on of your Business

Adver7sing Business Model:
Prot = Revenue - Cost

Unique Visitors x Ad Revenue per Visitor

Impressions/Visitor x Eec7ve CPM / 1000

Visits/Visitor x Pageviews/Visit x Impressions/PV

New Visitors + Returning Visitors

Invited Visitors + Uninvited Visitors

# of Users Sending Invites x Invites Sent/User x Invite Conversion Rate
Sample Signup Page Yield Data

Daily Signup Page Yield vs. Time
New Registered Users divided by Unique Visitors to Signup Page
100%
Daily Signup Page Yield
90%
80%
70%
60%
50%
40%
Started requiring
registration
30%
20%
10%
Changed
messaging
Added questions
to signup page
0%
1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/1
0
Iden7fying the Cri7cal Few Metrics

n
n
What are the metrics for your business?

Where is current value for each metric?
How many resources to move each metric?
n
Developer-hours, 7me, money
Which metrics have highest ROI opportuni7es?

Metric B
Bad ROI
Return
Return
Metric A
Good ROI
Investment
Metric C
Great ROI
Return
Investment
Investment
Case Study:
Op7mizing Friendsters Viral Loop
% of users
sending = 15%
invites
Active
Users
% of users
who are
active
Invites per
sender = 2.3
Invite
Prospective
Users
Invite
click-through rate
Click
Registration
Process
Fail
Succeed
Dont
Click
Conversion
= 85%
rate
Users
Mul7plied together, these metrics determine your viral ra7o

Which metric has highest ROI opportunity?
The Upside Poten7al of a Metric

?
100%
100%
85%
15%
0
Registra7on
Process Yield
Max possible
improvement
0.15 / 0.85 = 18%
2.3
0
% of users sending
invita7ons
0.85 / 0.15 = 570%
0
Avg # of invites
sent per sender
? / 2.3 = ?%
Okay, so how can we improve the metric?

How do we increase the average number of
invites being sent out per sender?
n For each idea:
n
n Whats the expected benet? (how much will it
improve the metric?)

n Whats the expected cost? (how many engineer-
hours will it take?)
n
You want to iden7fy highest ROI idea
Aler Launching Address Book Importer
Lean Product Ninja

Cheat Sheet
Understand the problem space to iden7fy
underserved customer needs
n Dene your value proposi7on
n Create wireframes/mockups
n Get user feedback
n Revise your feature set, UI design, and
messaging to improve product-market t
n Launch, learn, and iterate
n
Building Great Products the Lean

Startup Way
n 5-hour product workshop in SF
n Thu Oct 10th 1 to 6 pm
n hap://bit.ly/greatprod
n Use promo code google for 30% o
n Part of 3-day Product Bootcamp
n Product Summit Fri Oct 11th
n Product Camp Sat Oct 12th
n Ques7ons?
dan@olsensolu7ons.com
@danolsen

How To Be A Lean Product Ninja

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

How To Be A Lean Product Ninja

Uploaded by

Copyright:

Available Formats

+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+

BS, Electrical Engineering, Northwestern

20 years of Product Management Experience

Managed submarine design for 5 years

What does Lean mean?

to get the job done

Copyright 2011 YourVersion

Whats the Formula for

Copyright 2013 Olsen Solu7ons

The Lean Product Ninja Way:

Understand underserved customer needs

Copyright 2013 Olsen Solu7ons

Problem Space vs. Solu7on Space

need, or benet that the

Copyright 2013 Olsen Solu7ons

Problem Space vs. Solu7on Space:

Problem vs. Solu7on Space: Feature Level

Save time filing

Priori7zing Benets & Features based

n Priori7za7on should be based on customer

n Importance of user need (problem space)

n Sa7sfac7on with how well a product meets the

users need (solu7on space)

Copyright 2013 Olsen Solu7ons

Importance of User Need

High Importance + Low Sa7sfac7on =

Not Worth Going Aler

User Sa7sfac7on with Current Alterna7ves

Copyright 2013 Olsen Solu7ons

Kano Model: User Needs & Sa7sfac7on

Copyright 2013 Olsen Solu7ons

Olsens Hierarchy of Web User Needs

How easy to use is it?

Usability & Design

Does the functionality

Does the functionality work?

Is the site up when I want to use it?

Is the site fast enough?

Page Load Time

Importance vs. Sa7sfac7on

Recommended reading: What

Copyright 2013 Olsen Solu7ons

What is Your Value Proposi7on?

Copyright 2013 Olsen Solu7ons

Return (Value Created)

Priori7zing Product Ideas by ROI

Dans UX Design Iceberg

Copyright 2013 Olsen Solu7ons

The Elements of User Experience

Elements of User Experience Design

Structure and layout at both site and page level

How it looks vs. What it is, olen called chrome

Geqng Customer Feedback:

representa7on of your product

Copyright 2013 Olsen Solu7ons

Effort to Create Artifact

You Should Wireframe

n To communicate your product ideas to

Copyright 2013 Olsen Solu7ons

Why Everyone Should Use Balsamiq

illustrate a user story

Copyright 2013 Olsen Solu7ons

What Are You Going to Get Feedback &

+++++++++++++++++