Our norms aim to provide a self-serve, expert-led analysis, flexible enough to address your specific needs. This article compiles guidance on where to find and how to understand norms when using Zappi, including:
- What is a norm?
- Which measures in my results have norms?
- What assets are in norms?
- How does Zappi calculate norms?
- I can't find a relevant norm, what should I do?
- More norm options
What is a norm?
A norm is a statistical concept. It refers to the aggregate response of a certain population, against which a similar subject or answer set is compared. Simply put, comparing your results to a norm allows you to determine "what good looks like" for key metrics.
Which measures in my results have norms?
In general, norms are generated for measures that are asked the same way in every survey. Norms availability differs by Zappi product, so the quickest way to see what measures have norms is to open your results, select a chart, and view which norm is automatically applied and what others may be available.
To see which norm is automatically applied, look for a blue box above the chart.
To select a different norm, select the Norms tab next to the chart and scroll to compare your results to other populations. Often these will be either less specific (i.e., a global or country norm with no further filtering) or more specific (i.e., a country norm filtered by your category, or stage of development of the asset (Animatic, Final cut, etc.)).
What assets are in norms?
In the image above, you can easily see the number of assets each norm is based on next to the label. Those that are greyed out have not yet reached the minimum number of assets (30) to make comparison statistically relevant. For transparency, the number of tested assets in the norm you select is also included in the footnote of any slide you share or export from the platform.
How does Zappi calculate norms?
Norms are created by aggregating the results of a measure across all of the assets tested in Zappi that meet the criteria you've selected. Most often you will see norms based on the average (mean) answer to a rating scale question asked in your survey.
In most norms there is no relationship between category and sampling. This means that a norm for "food and drink" in the UK can be composed of assets tested with a "nationally representative" sample, but also "older females" and "men under 35" sample, as long as the asset was tested in the UK with food and drink consumers.
Some notable exceptions to the above:
- The “Lovers” norm is based on the top three box scores (i.e., scoring 8-10 on a 0-10 scale), rather than the mean.
- Key message norms are created by averaging the proportion of respondents who chose a given message.
- Prioritize It norms are based on category audiences.
- You can also create norms by tag, combined categories, and more.
- MARC norms are based on category audiences.
- Most are aggregates of top boxes, and only the top 25% of best-performing concepts in a category are included in the norm.
I can't find a relevant norm, what should I do?
While our overall database includes hundreds of thousands of stimuli, there are instances in which there isn't a norm available for a particular market, category, or industry.
- When running your test, include a benchmark cell (asset) so you can compare the results.
- Create a norm using a broader geographic region, or adjacent category using our advanced norm functionality.
More norm options
Flexibility on norm dates:
To provide more flexibility to your analysis, you have the ability to select the date on which the norm will be calculated for the majority of our products, providing the best possible results. Norm customization applies to most of our products.
This option enables you to define a custom selection of countries and industries to be included in a norm, to suit your needs.
For example, you may not have conducted sufficient studies in Colombia, for instance, to be able to build a country norm, so you might roll up those that you have done with any others you have done in neighboring Central American countries to create a custom norm that is more relevant to your products or advertisements than a global norm would be.
User- or customer-defined norms:
These are norms made up of all the assets you've tested within your platform domain. This enables you to compare your results to your own historical performance. For example, you might compare your ads or concepts with the best you ever tested, or you might compare them with any of your other tests that have the same tags, so you can analyze your performance on particular themes. In most Zappi tools with norms, the minimum number of stimuli required to view a user-defined norm is 2. The exceptions to this are:
- Amplify suite and Prioritize It = 30 minimum for percentages, 2 minimum for mean scores
- MARC = 20 minimum