Is Your Research Strong Enough to Guide a Real Decision?
A version of this post was included in my super-fun weekly email for marketers with tips, tricks, and stories.
Sign up for emails and never miss out on the fun!
I’m sure you’ve seen it…a garish color combination on a website, a complex banner design, a cluttered interface, or pop-ups that make you want to shut the whole thing down.
Bad design is easy to spot.
Bad market research?
Much harder to detect.
And that’s dangerous — because you might build strategy on it.
GIF from giphy.com
One of my past clients, a health food brand, initially told me they were targeting women over 65.
Their data told them that.
But when I heard how they got their data, I instantly knew the insight wasn’t valid.
They got their data by surveying their email list of subscribers. Now normally, that could be ok. But the problem here was that everyone on their email list:
Bought the product in store (they could only sign up with in-store QR codes)
Felt strongly enough to figure out how to join the email list during a grocery shopping trip
Took the time to read the emails
Took the time to respond to a survey that (as I found out) was over 20 minutes long!
That’s not “the market.”
That’s your most engaged fans.
And if the audience skewed older, it’s because it was their retired audience who took the time to follow all of these steps.
Younger customers weren’t less interested — they were simply less likely to join the email list and respond to a survey in this way.
Basing a marketing strategy off this data would have been disastrous for this company.
They literally could have gone out of business.
So today’s post is about how to tell if the data you’re working with is reliable enough that you can confidently base your business decisions around it. Because I want to squeeze the most out of your market research— boosting customer engagement, sales figures, and other forms of ROI from your market research.
So how do you tell if your research is actually solid?
Here are five filters we use.
#1: Strong Research Uses Both Qual + Quant
Photo By The Matter of Food
Strong research:
Uses qualitative and quantitative in a deliberate “sandwich” method. It starts with qualitative — in-depth interviews, observation, or ethnography — to uncover motivations, tensions, language, and decision drivers that would never emerge from a survey alone. Those insights shape focused hypotheses. Quantitative research then measures how widespread those patterns are across a defined population. Finally, the process often returns to qualitative to dig deeper into unexpected findings revealed in the data.
Qual → Quant → Qual. Exploration, validation, refinement.
Each layer builds intentionally on the last.
Red flag:
Jumping straight into a large survey without grounding it in human conversation — and realizing after the fact that the wrong questions were asked.
#2: Strong Research Recruits the Right Audience
Strong research:
Defines the audience based on who actually drives the buying decision — not who is easiest to access. Recruitment screeners qualify behavior, usage patterns, and decision influence, not just demographics. In qualitative work, this may include proof of purchase, usage verification, or contextual evidence to confirm fit. In quantitative studies, quotas are intentionally structured to ensure meaningful representation of key segments rather than relying on a broad “nationally representative” sample that dilutes insight.
The goal is alignment between participants and the real-world decision-makers the strategy depends on.
Red flag:
Surveying an internal email list, social followers, friends and family, or a loosely qualified panel — and assuming the results represent the broader growth audience.
#3: Strong Research is Right-Sized for the Decision
Strong research:
Aligns sample size with both the overall decision and the level of segmentation required. If cross-tab analysis is needed — by persona, age group, geography, customer type, or behavioral segment — each subgroup must have enough respondents to produce stable, interpretable results.
A total sample of 400 may sound robust. But if that sample is divided across five segments, and one subgroup only has 35 respondents, conclusions about that segment become fragile and unreliable.
Sample size isn’t just about the total number. It’s about whether each comparison being made is adequately powered to support a decision.
At the same time, more is not automatically better. Oversampling can make minor differences statistically significant without being strategically meaningful.
Strong research balances statistical power with practical relevance.
Red flag:
Reporting insights on subgroups that are too small to interpret confidently — or inflating sample size without considering how the data will actually be analyzed.
#4: Strong Research is Designed to Reduce Bias
Strong research:
Is structured to test assumptions, not protect them. Questions are written neutrally, without embedding a conclusion in the wording or answer choices.
For example, if there is an internal belief that a tool is confusing, a biased question might ask:
How confusing is this tool to use?
a.Very confusing
b. Confusing
c. A little confusing
d. Not at all confusing
The framing assumes confusion is present and forces respondents into that narrative.
A neutral version would ask:
How would you describe the interface on this tool?
a.Very simple
b. Somewhat simple
c. Neither simple nor confusing
c. Somewhat confusing
d. Not at all confusing
OR
List 3 adjectives that describe the interface of this tool: ________
(if you don’t mind coding a qual response!)
The difference is subtle but critical. The first question primes confusion. The second allows both positive and negative experiences to surface naturally.
Strong research eliminates leading language, separates assumptions from inquiry, and creates space for contradiction.
Red flag:
Research structured to validate internal beliefs rather than surface objective reality.
#5: Strong Research Reflects the Current Decision Environment
Strong research:
Is grounded in the realities shaping today’s buying behavior — not the conditions that existed when the last study was conducted. Consumer expectations evolve alongside technology, competitive landscapes shift, pricing sensitivity fluctuates with economic pressure, and new generations enter the market with different assumptions and norms.
A study conducted three or four years ago may have been rigorous at the time — but if it predates major shifts in digital behavior, platform usage, purchasing channels, regulatory environments, or category saturation, its conclusions may no longer hold.
Strong research evaluates whether prior findings are still valid before relying on them. It treats timeliness as a strategic variable, not an afterthought.
Red flag:
Making pricing, positioning, expansion, or product decisions based on data gathered under fundamentally different market conditions — assuming customer behavior is static when it is not.
******
Most weak research doesn’t look weak.
It looks impressive. It has charts. It has percentages. It has statistical significance.
The question isn’t whether the data exists.
The question is whether it was designed to move a real decision forward.
Strong research isn’t accidental. It’s intentional — built around the decisions at stake from the very beginning.
Not sure if it’s time for new research — or how to fix research that isn’t pulling its weight?
Let’s talk it through so you can get clarity before moving forward:
Sarah Weise is the CEO of award-winning marketing research agency Bixa and the bestselling author of InstaBrain: The New Rules for Marketing to Generation Z. For over 20 years, Sarah has been a guide to hundreds of leading brands, including Google, IBM, Capital One, Mikimoto, PBS, and U.S. Army, to name a few. Sarah helps brands achieve a laser focus on their customers and build experiences that are downright addictive. She lectures at Georgetown University’s McDonough School of Business and speaks at conferences and corporate events worldwide.