1
Research Techniques for
Uncovering End-User Needs
London, October 20th
Frankfurt, October 22nd
Who we are
Beacon Technology Partners LLC is a full-service primary market research firm
dedicated to B-to-B marketers (primarily technology)
– Founded in 1996
– Offers a full-range of primary market research services, including:
• Survey research (including telephone, Web-based, mail and hybrid
methodologies)
• Statistical modeling (including correspondence analysis or perceptual
mapping, regression modeling, conjoint analysis, market segmentation, and
other statistical tools and methods)
• Qualitative research (focus groups – both in-person and on-line - in-depth
interviews and small group studies)
2
Why this topic?
• Information managers are asking (or being asked)
– Who are their users/customers/prospects?
– What these users/customers/prospects need or want?
– How satisfied are these users/customers/prospects with the
tools and services they currently use?
• Information managers are having to answer these
questions
–
–
–
–
Anecdotally, or
Via do-it-yourself (DIY) techniques, or
Collaboratively with others in cross-functional projects, or
Thru partnering with 3rd party firms and service providers
3
Strategic Research Process
Project
Development
Questionnaire
Design
Consultation
Consultation
Interviews &
Focus Groups
Data Collection
Programming and Fielding
Statistical Modeling
Reporting
Reporting
Analysis
Tabulation
4
Presentations
Webinars
In-person
Whitepapers
Strategic Research – Planning and Consultation
Project Definition
Establishing the critical objectives – determining the upfront
strategy – is a critical step in any strategic research initiative.
It’s here where the problems to be resolved are isolated and
examined and the appropriate strategies are formulated.
During the planning and consultation period, you should ascertain:
•
•
•
•
•
Insights to be gleaned
Scope – geographic, linguistic, budgetary, timeline
Suitable methodologies and experimental design
Sample architecture – who needs to be heard from, how
easily can they be reached, how much data is required for
sound analysis
Best analytic tools and models employ
5
Strategic Planning – Data Collection
Questionnaire Design/Development, Programming and Tabulation
Industry standard
data collection and tabulation
Analysts craft questionnaires tailored
specifically to their organization’s needs.
Project managers program and field the
questionnaire, compile and tabulate
responses in a comprehensive report that
clients study to answer their questions.
Survey research packages differ markedly
with regard to functional capabilities.
Rather than employing cookie cutter
survey platforms, Beacon uses
programmers skilled in Java and Perl to
customize its online questionnaires. This
allows analysts to deploy innovative
questionnaire designs and randomization
algorithms to provide the best data for its
modeling.
6
Strategic Research – Data Collection
Qualitative Methodologies
Qualitative techniques are often best
for answering “why?” questions
Qualitative research techniques - such as focus
groups, “mini” groups or one-on-one in-depth
interviews – can be employed quickly and cost
effectively for projects that face geographical or time
constraints.
In Depth Interviews (IDIs)
Diads, Triads, Mini-Groups
Beacon uses qualitative techniques for:
•
•
•
•
•
Product concept evaluation
Customer needs
Pricing
Customer loyalty
Segmentation and persona development
and many other topics.
Face to face Focus Groups
Webcam Focus Groups
7
Strategic Research - Questions
Field and Tab
What do you need to know?
Who should we target?
What are our competitive
advantages?
How loyal are our
customers?
How should we communicate
our advantages?
8
What do our customers want?
How effective
is our marketing?
What does the future
hold?
Strategic Research - Analysis
Who should we target?
• Problem to be solved
– Need to differentiate users/customers/prospects/patrons
•
•
•
•
Who are your “best” customers?
What do they want or need?
What drives or motivates them?
How can they be reached effectively?
– Audience subgroups can be hard to identify and target
9
Strategic Research - Analysis
Who should we target?
• Case #1 (June 2012)
– Online publishing platform provider
– Wanted to understand customer motivations across multiple job
functions/responsibilities
• Case #2 (October 2010)
– Software supplier of auction software (similar to eBay) for non-profits
– Needed to understand user motivations and psychographics
– Had basic contact information (basic) linked to click stream bidding
behavior
10
Strategic Research - Analysis
Field
and
Tab
Who
should
we target?
Answer:
Persona and segmentation development
Personas (case #1)
•Depicts “at-a-glance” archetypes
•Delves into demography,
motivations, skills, experiences,
behaviors, attitudes, goals and
aspirations.
•Employs qualitative and
ethnographic techniques.
Segmentation (case #2)
•Utilizes multiple quantitative
research methods (e.g. survey
research, data mining, cluster
analysis etc.)
•Forms rigorous and identifiable
depictions of distinct audiences
that can be found and targeted with
specific products, promotions and
pricing.
11
Strategic Research - Analysis
How loyal are our customers?
• Case #3 – Voice of the Customer
– Reprints Desk (July 2010)
– Needed to measure
• The efficiency/effectiveness of its operations
• Its competitive advantages/disadvantages
• Customer loyalty – and isolate which customers were at risk
– Had contact and other information (e.g. tenure, transaction data, sales
contacts, etc.)
12
Strategic Research - Analysis
Field
Tab
Howandloyal
are our customers?
Voice of the customer projects are unique
Loyalty matrix
• Sample architecture need to include
current (both new and long term)
and past customers
• If possible, include data already in
hand about each customer
Expectation Analysis
• Attributes need to be actionable
• Scales need to be re-balanced to
correct for positive skew in ratings
• Need to employ derived rather than
stated importance
13
Strategic Research - Analysis
Field
Tab
Howandloyal
are our customers?
Reprints Desk performance metrics – expectation analysis – July 2010
Falls short of
expectations
Keeps us informed about
changes to their products
and solutions
Has an easy-to-navigate
2%
website
Provides complete
information about order
status, availability, or pricing
via its website
59%
38%
16%
61%
23%
14
Based on 10-pt. scale
Ranked in descending order of derived importance
7.69
8.56
84%
14%
Website provides ample
3%
resources
8.45
64%
26%
MEAN
(10-pt. scale)
8.35
80%
20%
10%
Exceeds
expectations
84%
16%
Clear and easy to read
documentation
Provides clear information
about order status,
availability, or pricing via its
website
Meets
expectations
7.70
7.35
Strategic Research - Analysis
Field
Tab
Howandloyal
are our customers?
Reprints Desk performance metrics – loyalty matrix – July 2010
Greater Concern:
Focus improvements –
Competitive vulnerability (relative)
•Competitive pricing
•Reducing total cost of ownership
•Complete solution
•Effectively balancing workloads
Less Concern:
Reduced urgency
•Completeness of Web-based order
status, availability or pricing
Competitive Advantage:
Maintain or improve performance –
competitive advantage
•Customer service, support,
communication, empathetic
understanding and response
•Utility of web site (navigation, product
updates, ease of use)
•Add capacity
•Captures all bibliographic data
Maintain:
Maintain or reduce investment
•Delivery service
•Services are easy to use, easy to
manage, easy to configure
15
What this shows:
The loyalty matrix is a
critical tool which
leads to corrective
action measures. It
separates performance
attributes into those
which merit attention,
can be downplayed or
upgraded, or safely
ignored.
Strategic Research - Analysis
What do our customers want…
• Problem to be solved
– Customers/prospects don’t know what they want in the way of new
products….
….but they do know about their pain points and challenges
– Customers/prospects can’t adequately articulate how to solve their
problems….
….but they can react to suggested solutions
16
Strategic Research - Analysis
What do our customers want…
Problem of “stated importance” - scaled questions can be overly artificial or inadequate
15. When you consider [TOPIC], how important are each of the following features or benefits? (Please select ONE best response for each feature/benefit.)
Not at all
important
Somewhat
unimportant
Neither
important
nor
unimportant
Somewhat
important
Extremely
important
Protection from loss with upside earning potential
1
2
3
4
5
Tax deferral
1
2
3
4
5
Higher potential returns than CDs
1
2
3
4
5
Lifetime income option
1
2
3
4
5
Flexible fixed indexed-linking options
1
2
3
4
5
Why fixed indexed annuities may be appropriate in an IRA or qualified plan
1
2
3
4
5
Solution: require respondents to make choices – just like they do in real life
17
Strategic Research - Analysis
What do our customers want…
•
•
•
Case #4 (June 2011)
– Reprints Desk
– Needed to test new service concept (“Bibliogo”) for STM lit acquisition
– Wanted reaction among four segments (scientists/researchers, corporate librarians,
brand/product managers, compliance)
Case #5 (Sept 2013)
– Digital signage software supplier
– Wanted to develop “next generation” features/functionality that would be desired by its
current and potential users
Case #6 (June 2014)
– Supplier of semiconductor IP cores
– Wanted to prioritize customer needs/challenges
18
Strategic Research - Analysis
Field
and do
Tab
What
our customers want….
Answer #4: Focus groups w/concept test – June 2011
Bibliogo is a web application for secure workgroup collaboration around journals, citations, and bibliographies (lists of citations).
•Users create their own content by adding RSS feeds (via blogs, news websites or literature search engines or by journal title or product brand name), entering
citations one at a time, or through bulk citation import.
•This content can be tagged, searched, shared, approved and annotated within the Bibliogo application; and it is all “alertable” by authenticated RSS feed or by
email.
•This enables users to easily follow updates by search terms, journal, product brand name, company name, therapeutic area, and more.
•All alerting is managed within Bibliogo rather than via multiple publisher websites or platforms.
•Regardless of how alerts are received, each alert item arrives embedded with article tools. Users can quickly obtain full-text copies of articles in any quantity, legally
send citations or articles internally or externally, request copyright permissions, request price quotes for bulk article reprints, and much more.
•Any user can become a bibliography owner simply by sharing a personal bibliography with one or more colleagues or other collaborators. The bibliography then
becomes, in effect, a bibliographic workgroup.
•Bibliography owners can set permissions for each citation in the bibliography. Admin rights and permissions can be assigned at the individual and workgroup level.
Sharing can be made open or more narrowly controlled by limiting access to users with emails from approved domains (e.g., name@yourcompany.com) and/or by IP
address.
•
Preferred words/phrases (underlined and italicized in paragraph above)
 All alerting is managed within Bibliogo
 Secure workgroup collaboration
 Create own content & manage RSS feeds
 Quickly obtain full-text copies
 All “alertable”
 Users can easily follow updates
 Legally send citations or articles internally or externally
•
Problematic words/phrases
 The bibliography then becomes, in effect, a bibliographic work group – (unclear)
 Any user can become a bibliography owner simply by sharing a personal bibliography with one or more
colleagues or other collaborators – (unclear)
 Entering citations one at a time – (appears manual, fights automation benefit)
 Through bulk citation import – (unclear)
19
Strategic Research - Analysis
Field
and do
Tab
What
our customers want….
More important
Benefit laddering – June 2011
Eliminate the need to
use multiple systems
Simplify bibliographic
information
management
Simplify creation and
distribution of SDIs
(Selective
Dissemination of
Information)
Simplify regulatorycompliant article
usage
Improve your typical
or routine activities
discovering and
accessing citations
and articles
Simplify copyrightcompliant article
usage
Simplify literature
monitoring
Save time accessing
full-text literature
Simplify the ability to
share what I am
working on
Less important
20
Discover what
colleagues are
working on
Strategic Research - Analysis
Field
and do
Tab
What
our customers want….
Answer for Case #5: Kano modeling
Customer
Satisfaction
Performance
Excitement
How to read this chart:
Indifference
Kano analysis classifies
attributes into 4 categories:
•Basic attributes are “must
haves” – absence dissatisfies
but presence doesn’t add to
satisfaction
Needs Fulfilled
Needs Not Fulfilled
Basic
•Performance attributes
increase satisfaction and their
absence decreases
satisfaction
•Excitement attributes are
“extras” that delight or add to
satisfaction but are not
expected
•Indifferent attributes make
little impact on satisfaction
Customer
Dissatisfaction
21
Strategic Research - Analysis
What
What do
do our
our customers
customers want….
want….
Basic
22
Performance
Excitement
Strategic Research - Analysis
Field
and do
Tab
What
our customers want….
Answer for Case #6: Maximum difference (MaxDiff) scaling/modeling
–
–
–
Each respondent presented with x tasks (depending on sample size and attributes), which consisted
of a random combination of any four of the attributes being tested
Respondent chooses between each set of attributes (most important and least important) that is
more descriptive or appropriate (e.g. the type of vendor with whom they prefer to do business)
Respondent choices are summarized as utilities
1
Offers IP that is easy to use
7
Offers IP that is easy to integrate into your
designs
2
Has high performance IP
8
Has a track record as a technology leader
3
Has a flexible pricing or licensing model for their
interconnect IP
9
Is a trusted supplier of IP
4
Offers responsive support
10
Is an innovative supplier
5
Produces IP that is of highest quality and
reliability
11
Enthusiastically seeks solutions on behalf of its
customers
6
Offers the widest breadth of interconnect IP
12
Offers a credible future technology roadmap for
its next generation IP
23
Strategic Research - Analysis
Field
and do
Tab
What
our customers want….
Represents over
40% of overall
utility
Average
Utility
*Attribute utilities scores are relative rather
than absolute; even the lowest-rated
attribute may have a positive impact (albeit
less important) on supplier selection.
24
Strategic Research - Analysis
What do our customers want…
… and what should we charge?
• Problem to be solved
– Customers have difficulty assigning value to goods and services
– “Price” has multiple meanings and connotations
• Case #7 (November 2013)
– Major American newspaper
– Wanted to test market acceptance (including features and pricing) of new mobile app
• Case #8 (March 2012)
– Digital library/database provider
– Wanted to test new e-book concept and subscription pricing model
25
Strategic Research - Analysis
Field
and do
Tab
What
our customers want and what should we charge?
Answer: Choice based conjoint (CBC)
 Each respondent presented with 12 tasks which consists of mobile app subscription package
options or attributes compared one to another in various combinations
 Respondent choose between three sets of attribute groupings and a “None” option that
describe the type of subscription options they prefer to have
 Respondent choices summarized as part-worths or utilities
26
Strategic Research - Analysis
our customers want and what should we charge?
Potential Monthly Revenue
Field
and do
Tab
What
What this chart shows:
This estimates total monthly revenue
for the mobile app by examining likely
demand at various price levels
between $.99/month to $6.99/month.
The upper estimate assumes that all
likely app purchase will actual buy a
subscription, while the lower estimate
uses a rule of thumb that only half of
those who say they are likely
purchasers will in fact actually follow
through and purchase.
Subscription Price (per month)
Share of Preference
PRICE POINT
App purchasers
App
Conservative
Price
HH App
purchasers rule of thum b elasticity purchasers
$0.99
97.2%
48.6%
$2.99
93.5%
46.8%
$4.99
85.1%
42.6%
$6.99
73.5%
36.7%
Potential
m onthly
revenue App
purchasers
Conservative rule of
thumb
Potential
m onthly
HH
revenue Conservative Conservative
rule of thum b rule of thum b
54,268
$53,700
27,134
-0.02
52,188
$156,000
26,094
$78,000
-0.13
47,518
$237,100
23,759
$118,600
-0.34
41,008
$286,700
20,504
$143,300
27
$26,900
Notice that as prices rise, revenue
also increases (evidence of demand
inelasticity, previously noted). But
also note that revenue growth tends
to slow and begin to flatten as prices
are increased, evidence of a demand
“elbow” and greater price sensitivity.
Strategic Research - Analysis
Field
and do
Tab
What
our customers want and what should we charge?
+ $1.24: One e-book plus three
chapters - 90-day expiration
Library: + $1.25
Bookshelf: + $1.23
Cancelled: + $1.08
+ $.81: One e-book plus five chapters 30-day expiration
Library: + $.81
Bookshelf: + $.70
Cancelled: + $1.01
$42.99: Base product: One e-book plus
three chapters - 30-day
expiration
Current subscription prices
28
With the addition of two
chapters and with a 90-day
expiration to the base product –
subscribers value the added
content downloading options at
roughly $2 per month
regardless of subscription
status
Strategic Research - Analysis
Field
and do
Tab
What
70
our customers want and what should we charge?
Demand for Concept 1 (one e-book, 3 additional chapters, no accrual - 30 day expiration)
at Varying Price Points
60
Elasticity @ increase of $3
= 14.6
Share of Preference
50
40
30
Elasticity @ increase of $1.60
= 7.3
Current Library
Subscribers
20
Bookshelf / Cancelled
Subscribers
10
0
$42.99
$44.59
$45.99
29
$47.49
Price per month
$48.99
$50.49
$51.99
So where do you go from here?
• Do primary research
• If you do it yourself
– Follow the correct process (including stablishing
objectives and designing survey architecture)
– Be careful with questionnaire wording, scales and models
– Know what responses you want or are looking for before
you ask questions
• Partner with an independent research consultancy
30
We’re here to help you
Contact:
Email:
Phone:
Web site:
Jim Warrick
jim@beacontech.com
(978) 248-0797
www.beacontech.com