Why Stated Preferences Fail: The Say/Do Gap in Market Research
- People's stated preferences often differ from their actual behavior due to psychological factors like limited self-awareness and a desire to meet the imagined expectations of others. This is known as the "say/do gap," and these discrepancies are usually unconscious, rather than resulting from purposeful dissimulation.
- Traditional market research methods like surveys and focus groups rely heavily on verbal self-reports, which can be unreliable guides to consumer behavior.
- Emerging techniques like behavioral experiments, and implicit measures help uncover non-conscious attitudes and revealed preferences.
- Comparing results from multiple methods provides a more complete view of motivations and helps narrow the intention-behavior gap.
- By using new techniques that capture both stated and revealed preferences, marketers can gain insights that better predict real-world consumer actions.
Try a simple experiment: Pick up any packaged product near you and examine it closely for a few moments. Take your time observing details you may have never noticed before. I guarantee you'll gain new insights that subtly shift your perspective.
For example, I tried this exercise with a bottle of mineral water on my desk. At first glance, I registered the usual details - the blue and white label, the narrow shape, the thin plastic that crinkles when squeezed. But peering more intently, I spotted additional label elements that didn't immediately make sense. This spurred me to generate theories about why those particular graphic choices were made.
In just a minute or two, a whole new set of associations came to mind when I looked at a previously mundane object. Simply taking the time to observe something mindfully, beyond my habitual glance, unlocked new observations and opinions. These novel impressions changed my relationship to this everyday item.
When we ask people to stare closely at an object and verbalize associations, we are capturing an atypical response. Their perceptions have been intentionally altered by the mindset prompted through this attentive looking, as well as by the knowledge that their responses will be observed and evaluated.
Much of legacy market research is like this. For example, in focus groups and surveys, participants will look at products and packaging designs with a focus and for a length of time that is unusual, and doesn’t replicate the way we naturally respond to such things: rapidly, taking in the gist of them. Therefore can we really trust consumers’ utterances in such situations as representative of how they will act in real life?
Equally, Marketers are all too familiar with products that fell flat in focus groups or surveys, only to succeed with consumers. The Sony Walkman, Red Bull energy drinks and the Dyson Vacuum cleaner are just some of the most successful innovations that faced skepticism in traditional market research.
This reflects a common phenomenon - frequently there is a gap between what people say they think, feel, and intend to do and their actual attitudes, emotions and behaviors. In psychology, this disconnect between stated preferences and revealed preferences is known as the “say/do gap” or “intention-behavior gap” . For marketers, ignoring this gap can be perilous. However, by understanding the psychological mechanisms behind it, steps can be taken to narrow this divide and make customer insights more actionable. And by using research methods that reveal preferences rather than ask people to give their opinions we can help avoid this gap.
In our own research we have seen examples where more Implicit and System 1 measures have revealed that people’s reports of their own behavior and preferences are not accurate, including:
* Shoppers claiming that they chose a particular retail shelf-hanger display as their most preferred because it was more ‘eye-catching’, but their eye-tracking data revealed they didn’t look at it any quicker or for longer than any of the alternatives
* Respondents claiming that they had seen a particular brand advertised before in the mass media, but it was a fictitious brand we had invented purely for our research
* Consumers claiming that they are open to eating a more plant-based diet, but their timed reactions reveal that they are actually much more uncertain or ambivalent about switching to plant-based foods than they claimed to be.
Then there are many questions which our experience as researchers would suggest are just inadequately answered by direct survey style questions, such as, for example, matters of design and aesthetics. If only asked to select their favourite of several alternative different new designs for a product’s packaging, it's often simply the one that is most different - and therefore most likely to grab their gaze - that is selected. But the unusual context that made them select it, will obviously never exist in the real world.
In our research we often include direct, ‘System 2’ questions, but we use them selectively when they make sense (e.g. when asking participants to give us factual information, such as demographics or purchase history, rather than opinions or hypotheses about their motivations).
There are multiple reasons why people’s own statements and responses are a poor guide to how they will act. Psychologists have identified at least eight:
1. Limitations of Conscious Awareness
A fundamental challenge is that people have limited conscious access to the true drivers of their emotions and actions. We often logically justify our behaviors after the fact, a phenomenon psychologists call confabulation . The influential “split-brain” experiments revealed our left brain hemisphere will even make up explanations for behaviors initiated in the right hemisphere  (it's beyond the scope of this article, but if you aren’t familiar with these experiments, I encourage you to read about them). When consumers verbally report why they like or dislike an ad, package design, or product, they are trying to introspect on subconscious processes that evade easy understanding. We suffer from an “introspection illusion” that we have more insight into our mental processes than is actually the case . The roots of our preferences often remain hidden from our conscious minds.
2. Not Only Unaware of Origins, But Even Outcomes
Even more troubling, we seem to lack full awareness of our actual preferences and choices, as evidenced by past behavior. In studies of “choice blindness,” people fail to notice when experimenters switch their selection of products - such as different flavors of tea or jam - and then happily justify the randomly assigned preference . Our conscious minds do not always track each subtle judgment and behavior. People are readily willing to offer explanations for choices they did not even intentionally make.
3. Intention-Behavior Gap
Nowhere is the say/do discrepancy more evident than in the frequent failures to act on stated intentions. For example, the majority of ambitious New Year’s resolutions to change habits soon fall by the wayside . Though people sincerely intend to eat healthier, exercise more, or save money, various psychological obstacles impede following through. Intentions fade as practical concerns and daily frustrations crowd out aspirational plans. The path from intent to action is filled with obstacles.
4. Affective Forecasting Failures
Not only do people mispredict their future actions, but they also misremember past experiences. Research on the “experiencing self” versus the “remembering self” reveals we often look back inaccurately when reconstructing how we felt in the past . People also integrate misleading information into memories without awareness, known as the “misinformation effect” . We are unreliable narrators of our own emotional lives over time.
5. Unintended Influences of Market Research
Ironically, the act of doing customer research itself can shift later attitudes and behaviors through priming effects (i.e. simply suggesting the possibility of buying something makes people think they are more likely to). In studies, when simply asked about purchase intent, people became up to 25% more likely to buy the product, an effect called “mere measurement” . The questioning primes related thoughts and feelings, swaying downstream preferences. Surveys designed to be neutral can still alter outcomes.
6. Power of Environmental Cues
Subtly changing situational cues induces people to shift behaviors in ways they do not consciously register. Altering music tempo in a wine shop changes purchase volume. Putting product images on credit card bills sparks more buying . When reporting attitudes, consumers do not realize how context unconsciously shapes them. Our beliefs are fluid, not fixed.
7. Social Desirability Bias
Self-reported data also suffers from many people’s desire to present themselves favorably. Customers exaggerate socially esteemed behaviors like engaging in healthy habits or buying green products. This reflects a “social desirability bias” rather than honest self-assessment . People want to be seen as ethical and consistent, even if they are not.
8. Moral Licensing
After displaying ethical behavior, people can feel licensed to make more self-interested choices, a phenomenon called “moral licensing” . A consumer may intend to consistently buy fair trade coffee but feel justified splurging on sweatshop-made luxury goods after a virtuous initial purchase. Morality is malleable, not absolute.
Limited introspective access to our true drives, the tendency to confabulate logical explanations, distortions of memory, blind spots around situational factors, and unconscious influences from the questioning itself—all undermine purely verbal self-reports. To supplement what people say, innovative market research techniques are needed to capture emotional, non-conscious, and behavioral data. People's stated beliefs often reveal just the tip of the motivational iceberg.
Techniques to Narrow the Say/Do Gap
Fortunately, emerging methodologies better align insights with consumer actions:
Big data analytics of actual purchase transactions, browsing history, social media activity, and geo-location data reveals what people do, not just say .
Behavioral experiments and interventions test responses to real-world stimuli and scenarios, not hypothetical intentions . For example, online retailers can usually set up A/B tests where they change the look or price of an offer between visitors to test which works best.
Implicit techniques like the Implicit Association Test uncover non-conscious attitudes and associations people are unwilling or unable to verbally report . Subtle but significant differences in reaction times can reveal the unconscious relative strengths of associations and preferences.
These tests can be used together with eye-tracking and carefully applied direct questions to gain a more holistic view that helps us avoid the pitfalls of the say/do gap. By comparing and contrasting the results of different methods we end up with a more robust and confident picture of the actual thoughts, feelings and preferences that determine consumer choices.
Leveraging these methods helps narrow the reported intention-behavior gap that hamstrings many market research projects. While verbal reports provide one view into consumer psychology, implicit, behavioral, observational, and environmental data give a more complete perspective. Combining multiple lenses brings predicted preferences and actual behavior into closer alignment.
 Sheeran, P. and Webb, T.L. (2016) 'The intention–behavior gap', Social and Personality Psychology Compass, 10(9), pp. 503-518.
 Johansson, P., Hall, L., Sikström, S. and Olsson, A. (2005) 'Failure to detect mismatches between intention and outcome in a simple decision task', Science, 310(5745), pp. 116-119.
 Gazzaniga, M.S. (1967) 'The split brain in man', Scientific American, 217(2), pp. 24-29.
 Pronin, E. (2009) 'The introspection illusion', in Advances in Experimental Social Psychology, Vol. 41, pp. 1-67. Academic Press.
 Hall, L., Johansson, P. and Strandberg, T. (2012) 'Lifting the veil of morality: Choice blindness and attitude reversals on a self-transforming survey', PLoS ONE, 7(9), p.e45457.
 Norcross, J.C., Mrykalo, M.S. and Blagys, M.D. (2002) 'Auld lang syne: Success predictors, change processes, and self-reported outcomes of New Year's resolvers and nonresolvers', Journal of Clinical Psychology, 58(4), pp. 397-405.
 Kahneman, D. and Riis, J. (2005) 'Living, and thinking about it: Two perspectives on life', in The Science of Well-Being, pp. 285-306.
 Loftus, E.F. (2005) 'Planting misinformation in the human mind: A 30-year investigation of the malleability of memory', Learning & Memory, 12(4), pp. 361-366.
 Dholakia, U.M. (2010) 'A critical review of question–behavior effect research', Review of Marketing Research, 7, pp. 147-199.
 North, A.C., Hargreaves, D.J. and McKendrick, J. (1999) 'The influence of in-store music on wine selections', Journal of Applied Psychology, 84(2), p. 271.
 Grimm, P. (2010) 'Social desirability bias', Wiley International Encyclopedia of Marketing.
 Merritt, A.C., Effron, D.A. and Monin, B. (2010) 'Moral self‐licensing: When being good frees us to be bad', Social and Personality Psychology Compass, 4(5), pp. 344-357.
 Edelman, D. and Singer, M. (2015) 'Competing on customer journeys', Harvard Business Review, 93(11), pp. 88-100.
 Levav, J. and Fitzsimons, G.J. (2006) 'When questions change behavior: The role of ease of representation', Psychological Science, 17(3), pp. 207-213.
 Greenwald, A.G., McGhee, D.E. and Schwartz, J.L. (1998) 'Measuring individual differences in implicit cognition: the implicit association test', Journal of Personality and Social Psychology, 74(6), p. 1464.