- What is a norm?
- What norm or benchmark should I use?
- Which measures in my results have general norms?
- What assets are in norms?
- I can't find a relevant norm, what should I do?
- More norm options
What is a norm?
A norm is a statistical concept. It refers to the aggregate response of a certain population, against which a similar subject or answer set is compared. Simply put, comparing your results to a norm allows you to determine "what good looks like" for key metrics.
What norm or benchmark should I use?
Which measures in my results have norms?
To select a different norm, select the Norms tab next to the chart and scroll to compare your results to other populations. Often these will be either less specific (i.e., a global or country norm with no further filtering) or more specific (i.e., a country norm filtered by your category, or stage of development of the asset (Animatic, Final cut, etc.)).
What assets are in norms?
In the image above, you can easily see the number of assets each norm is based on next to the label. Those that are greyed out have not yet reached the minimum number of assets (see next section) to make comparison statistically relevant. For transparency, the number of tested assets in the norm you select is also included in the footnote of any slide you share or export from the platform.
What is the minimum number of assets for a norm to populate?
From the 26th June 2023, the minimum is 20 stimuli for a Zappi country/category Norm where it used to be 30. This will help you benefit from a relevant comparison sooner, particularly for new categories or countries. As databases grow and diversify, they will become more stable. While the norm includes less than 50 cases, we encourage using your own internal or competitive benchmarks alongside these smaller norms.
- Many solutions, categories, countries already had 30 or more cases in which case you will experience no difference.
How does Zappi calculate norms?
Norms are created by aggregating the results of a measure across all of the assets tested in Zappi that meet the criteria you've selected. Most often you will see norms based on the average (mean) answer to a rating scale question asked in your survey.
In most norms there is no relationship between category and sampling. This means that a norm for "food and drink" in the UK can be composed of assets tested with a "nationally representative" sample, but also "older females" and "men under 35" sample, as long as the asset was tested in the UK with food and drink consumers.
Some notable exceptions to the above:
- The “Lovers” norm is based on the top three box scores (i.e., scoring 8-10 on a 0-10 scale), rather than the mean.
- Key message norms are created by averaging the proportion of respondents who chose a given message.
- Prioritize It norms are based on category audiences.
- You can also create norms by tag, combined categories, and more.
- MARC norms are based on category audiences.
- Most are aggregates of top boxes, and only the top 25% of best-performing concepts in a category are included in the norm.
I can't find a relevant norm, what should I do?
While our overall database includes hundreds of thousands of stimuli, there are instances in which there isn't a norm available for a particular market, category, or industry.
Use your own benchmark made up of fewer, but highly relevant, cases including for example your previous ideas/products/ads and those of your competitorsIf you have no cases, include a benchmark cell (asset) so you can compare the results. Ideally this could be an ad/idea/product that you know has been successful in market. Or maybe a competitor you want to beat. Or a previous ad from within a campaign.
- Create a norm using a broader geographic region, or adjacent category using our advanced norm functionality.
More norm options
Flexibility on norm dates:
To provide more flexibility to your analysis, you have the ability to select the date on which the norm will be calculated for the majority of our products, providing the best possible results. Norm customization applies to most of our products.
This option enables you to define a custom selection of countries and industries to be included in a norm, to suit your needs.
For example, you may not have conducted sufficient studies in Colombia, for instance, to be able to build a country norm, so you might roll up those that you have done with any others you have done in neighboring Central American countries to create a custom norm that is more relevant to your products or advertisements than a global norm would be.
User- or customer-defined norms:
These are norms made up of all the assets you've tested within your platform domain. This enables you to compare your results to your own historical performance. For example, you might compare your ads or concepts with the best you ever tested, or you might compare them with any of your other tests that have the same tags, so you can analyze your performance on particular themes. In most Zappi tools with norms, the minimum number of stimuli required to view a user-defined norm is 2. The exceptions to this are:
- Amplify suite and Prioritize It = 30 minimum for percentiles, 2 minimum for mean scores
- MARC = 20 minimum