So you’ve conducted surveys and followed all the rules. The questions were good, and the responses better than expected. The results are in, and you have lots of information to sort through. Others have told you only a team member with a master’s degree in statistics can make sense of it. Forget it! Analyzing survey data is a lot easier than you think, and you can do it in-house without going to extra lengths.
The best way to analyze survey data is with a structured approach, so insights become meaningful. Conversely, a haphazard approach can result in your research going nowhere. Follow the criteria outlined below to make the most of your hard-earned data.
Make sure the sample size is reliable.
Suppose 20,000 visitors landed on your “Products and Services” page over eighteen months. Your survey centered itself on one big question, “Would you recommend our brand to your friends?” — “Yes” or “No.” As a start, calculate the response rate:
- If a hundred visitors responded, it’s hardly a fair representation of the majority opinion. In this case, 100 divided by 20,000 = 0.5% — not very encouraging.
- On the other hand, if 1000 (i.e., 5%) answered the survey questions, you may well have something to go on.
- The higher your response rate, the more confidence there is in the results. It’s wise to establish a minimum reliability percentage as a standard.
Attach percentages to penetratively analyze survey data
Percentages make the most sense, but you can’t get to them until you aggregate real numbers. Once you’ve decided that the sample is representative, view the results through a percentage viewfinder. It’s the best way to analyze survey data — transcending you to the crux of the matter quickly. In many survey software packages or outsourcing hosts, counting numbers and calculating ratios is automatic. The point is this — aggregating is there for a reason, so don’t ignore it or let the numbers bypass you.
Follow traditional data analysis directions.
Using the survey analysis example above, we should appreciate that adding a few other questions powers you to cross-tabulate and expand your overview of the customer experience (CX). Here are some other possibilities:
Demographic and Geographic splitting on the “Yes” or “No” responses is viable by inserting unobtrusive check-boxes that uncover the following about the respondents:
- Gender
- Age
- Religion
- Marital status
- Education level
- Language
- Race
- Income level
- City, town, or zip code
Behavioral Segmentation, splitting down similarly by:
- Brand loyalists
- Early customers
With added quizzing, the depth and breadth of the survey results expand dramatically. However, there’s a fine line between getting enough information and no information. It rests on your audience’s motivation to take the survey (which is quite another discussion). Visitors frequently regard too many probes as too time-consuming or even intimidating.
Respect the benchmarks.
“I get the percentage thing, but how do I know whether they’re good or bad?” is a common question that business managers ask all the time when trying to find the best way to analyze survey data. The answer is to create benchmarks (i.e., critical yardsticks) to build a sense of confidence in the numbers while simultaneously reflecting trends.
For example, a colossal benchmark emerges from looking at the same survey over time to compare the results (e.g., one year, six months, three months). In this case, it must be the same survey so that it’s an “apples to apples” comparison.
Let’s go back to our “20,000 visitors” case study above to see that it’s genuinely a straightforward exercise. As a reminder, the company surveyed over eighteen months. Here is a typical configuration of compiling numbers first, then percentages to establish yardsticks and trends. Of the 1000 respondents (i.e., 5% of the total):
- 580 (or 58%) answered “Yes.”
- 420 (or 42%) answered “No.”
The result reflected in the:
- The last 6 months was – 500 (or 50%).
- 400 “Yes” (or 80%) and “No” 100 (or 20%).
- The last year – 700 (or 70%).
- 490 “Yes (or 70%) and “No” 210 (or 30%).
- The last 18 months – 1000 (or 100%).
- 580 “Yes” (or 58%) and “No” 420 (or 42%).
From the above information, it’s easy to observe an ascending positive brand image trend and increasing momentum in visitors taking the survey over time.
There are many ways you can slice and dice data to pick up patterns, and a look back over time is a common practice. In the research industry, we call this longitudinal analysis.
There are naturally-forming benchmarks — like the Total Average = 58% “Yes” and 43% “No.” Compare these to, say, how women under age 40 respond (over or under the average). How far you can take this boils down to the information depth (as demonstrated above).
The more you segment the data, the more meaningful the observations become:
- You can pinpoint outperformers, underperformers, and trends over time across all the verticals.
- When values are higher than the average, we rate the category as over-indexed, and when the standards beat you – under-indexed.
- It gives you some perspective of the power of well-managed statistics and the range of potential insights that they provide.
Maximize the value of “Why?” in surveys data.
A word of caution
Qualitative data emerges when you couple the “Yes” or “No” question with “Why?” In other words, “Why would you recommend or not recommend our product to a friend?” While this opens up a vast oversight arena, which can be mind-blowing, don’t let it divert your attention or disrupt your quantitative survey results.
- Begin by squeezing as much out of the numbers aggregated and converted to percentages as you can.
- It’s a crucial first step before assigning opinions and emotions to targeted market segments.
- The percentages applied across the board show you the relevance of different groupings before connecting them to thoughts and feelings about your brand. It’s the essence of market segmentation and a powerful force it indeed is, giving a weighting to the CX drivers.
The Power of “Why?”
A response like “yes” or “no” is one thing. Digging down with open-ended questions creates the opportunity to draw up your targeted customers’ psychographic and behavioral profiles. Feedback can tell us who influences buyer decisions, the emotions driving them (e.g., love, ego, the release of frustration), and top-of-mind thoughts:
- It’s the key to connecting to core problems that obstruct the customer experience and threaten customer retention — a bedrock of a company’s ROI.
- Many surveys with well-thought-out questions can prove or disprove assumptions C-suite and marketing executives outline before launching them.
- There’s a broad division between accepting an assumption as real and going forward with the confidence that it’s an undoubted driving force.
- If applied in a measured way, surveys are the “open sesame” to lighting the path ahead.
Pay attention to survey analysis examples—particularly understanding “correlation” and “causation.”
There’s nothing like looking at a survey analysis example or examples to understand the difference between correlation and causation.
Correlation
When two variables move simultaneously in sync, there may be a degree of correlation. For example, in the summer, you may observe a closeness between trends in the buying of swimsuits, surfboards, roller-blades, and skateboards. Graphing the percentage direction changes (with the oncoming season) for the four product categories will probably reflect similarities. Pure observation is often enough for verification.
Causation
Causation is another kettle of fish. In the example above, seasons are at the root of market changes in these markets. Watersports and swimming are activities that gain momentum as the weather warms up – a cause, not a correlation. This may seem obvious, but similar shifts may emerge from more oblique causations. For example, as the pandemic abates over months, one may see restaurants, gyms, and traditional retailing come back to life. Don’t make the mistake of interpreting correlations as causations and visa versa (e.g., believing that opening restaurants helped gyms open or that swimsuit upticks boosted surfboard activity). Looking at things with common sense and an objective eye is essential to avoid common analysis pitfalls.
Use your survey data most effectively.
Fixating on numbers and percentages is generally unengaging in its naked format or tables. If you want them to motivate your people to act on them, it’s all about communicating the bottom line creatively. Why?
- You want them to develop ideas to accelerate trends (if they’re working in your favor) and reverse them (if going against you).
- By all means, use the percentages to motivate your results, but develop a narrative that captures the team’s attention.
- Use comparisons, benchmarks, and analogies to convey a central message.
Companies like Sogolytics are consummate professionals at all stages in the process and explaining how to analyze survey data. They have the statistical techniques to assess the reliability of surveys data and separate causation from correlation. Moreover, Sogolytics knows everything about customer segmentation and how to get the cross-tabulation working for you most effectively. Remember that the goal is to structure marketing and promotional programs that hit directly at the heart of CX as it pertains to your products and services. The survey analysis is a vital decider, so don’t think twice about getting an expert resource like Sogolytics in your corner.
