Zappi Amplify TV

Summary

Zappi has taken the very best of existing legacy approaches and worked with leading global brands to develop a game-changing insight tool for TV-advertising effectiveness measurement. Zappi Amplify TV predicts how well your ads will deliver ROI, both via short-term sales and long-term brand equity. 

Insights are framed and communicated to ensure they are easily understood by stakeholders at all levels of the business.

Consumer engagement is captured across a range of rational, emotional, and behavioral measurement, and communicated in a way that is easily digested by non-insights professionals.

Stimuli: 1-5 video ads per test

Evaluation: Monadic

Norms: Yes; market-wide norms are activated when total stimuli tested reaches N=20 and your own customer norm per market activates when you've tested 20 stimuli in each market.

Contents

Getting started

• Key measures & Inputs
  • Amplify 1.0: the 5 Rs framework
    • Amplify 1.0: Configuration checklist
  • Amplify 2.0: the 3 Rs framework
    • Amplify 2.0: Configuration checklist
• Testing process
• Testing animatics

Analysis topics

• Understanding potential with Profiles
• Interpreting data and taking decisions
• Interpreting cultural sensitivity
• Get one-click automated analysis and reporting with AI Quick Reports, now in beta

Key measures

Depending on which version of Amplify TV reporting you are using, you'll see either 5 or 3 key measures make up the framework. There are 3 key measures in our updated Amplify TV version based on customer feedback of what's most important to internal stakeholders and easiest to understand. They are otherwise quite similar, as Risk and Return were simply absorbed by Resonance and Response respectively.

Amplify 1.0: the 5 Rs framework

The Zappi Amplify TV framework is comprised of measures grouped into five key ROI-driver areas:

• Reach: cut through the clutter, link to the brand, and build distinct memory structures
• Resonance: engage and trigger an emotional response, and communicate key messages and category drivers
• Response: increase brand purchase and appeal
• Risk: avoid insensitivity and damaging virality
• Return: deliver sales uplift and build brand equity

The reporting output focuses on a blend of behavioral and survey data, “Metrics that matter”, which ladder into two important KPIs:

1. Creative Sales Impact (CSI): how likely is this ad to deliver ROI via short-term sales uplift?
2. Creative Brand Impact (CBI): how likely is this ad to deliver ROI via long-term brand equity building?

Amplify 1.0: Configuration checklist

• Videos: you can upload up to 5 videos in an order, within the same country and category, each of which will generate a separate survey.
• Logos: we would recommend presenting the image files in a square format to provide consistency when viewing.
• Ensure you add the following information about your stimuli: 
  • Ad name
  • Brand information 
  • Stage of development
  • Presence of music and celebrities
  • Target Audience Profile
  • 2-20 brand/category attributes that you are looking to test that people might associate with your stimuli
    • 255 character limit, per attribute
  • 1-4 key messages for your stimuli
  • Brand competitive set images
  • Tags
    • Tagging your stimuli allows you to categorize your content efficiently, as well as unlocking additional analytic capabilities
      • Standardized tags - taxonomy defined by your organization
      • Custom and smart tags

Amplify 2.0: the 3 Rs framework

• Reach: cut through the clutter, link to the brand, and build distinct memory structures
• Resonance: engage and trigger an emotional response, and communicate key messages and category drivers (includes metrics from the previous Risk chapter)
• Response: increase brand purchase and appeal

The reporting output (the previous Return chapter) focuses on a blend of behavioral and survey data, which ladder into two comprehensive indicators for sales impact and brand impact: 

1. Sales Impact Score: The sales impact score measures the potential of the creative to drive short term sales. If you are familiar with the former Creative Effectiveness Score, it is simply that score renamed.
2. Brand Impact Score: The brand impact score measures the potential of the creative to build the brand and drive sales into the future.

Amplify 2.0: Configuration checklist

• Videos: you can upload up to 5 videos in an order, within the same country and category, each of which will generate a separate survey.
• Logos: we would recommend presenting the image files in a square format to provide consistency when viewing.
• Ensure you add the following information about your stimuli: 
  • Ad name
  • Brand information 
  • Stage of development
  • Presence of music and celebrities
  • Target Audience Profile
  • 2-15 brand/category attributes that you are looking to test that people might associate with your stimuli
    • 255 character limit, per attribute
  • 1-4 key messages for your stimuli
  • Brand competitive set images
  • Tags
    • Tagging your stimuli allows you to categorize your content efficiently, as well as unlocking additional analytic capabilities
      • Standardized tags - taxonomy defined by your organization
      • Custom and smart tags

Testing process

1. Upload between 1-5 video ads per test
2. A default of 400 category-consumer respondents are exposed to each ad within the selected in-context environment, with the flexibility to increase or decrease the sample size to between 200-800 respondents depending on the specification of the project.
3. Key questionnaire components:
   • Pre-exposure shopping exercise
   • In-context forced exposure
   • Unaided brand recall and brand cut-through
   • Post-exposure shopping exercise
   • Unaided message communication, questions metrics, and second-by-second emotion capture (with emojis)
4. Results are provided in the context of the available norm (e.g. country-level, category-level)

Testing animatics

Zappi Amplify TV supports animatic and finished film format ads in the same testing methodology. By testing animatics, you can assess an ad’s potential early on in the development process and guide improvement before commissioning a finished film. Read the guidelines to test animatic stimuli. 

Understanding potential with Profiles

What are ‘profiles’?

Profiles offer unique functionality for you to quickly and easily diagnose which sub groups your ad is resonating particularly well with. Unlike with filtering, when you select a profile, it not only selects data from that sub group for the ad, but it compares it to data for only that sub group across all other ads in the norm you’ve selected.  This is particularly useful given that some sub groups tend to be more positive/negative in general. It basically means you are, for example, seeing how 16-34 year olds relate to your ad versus other ads in that country (assuming you have selected a country norm). 

How can profiles help me with understanding my ad?

These are ideal to use diagnostically to understand who your ad is working particularly well for, whether the core target is responding well and why certain groups react as they do so you can optimize the ad. 

The data should be used diagnostically for an individual ad rather than for comparing effectiveness among a profile group between two ads. We don’t weight data at the sub sample level (just total sample) and hence any differences in the makeup of the subgroup sample between ads could impact how valid this direct sub group comparison is.

How do I create and select profiles during analysis?

You can choose from an available list of profiles in a drop down list and then see your charts automatically updated to reflect that sub group rather than total audience in just a click. 

You can also create new profiles based on any profiling/screener question used in the survey (i.e. brand buyers, shoppers for a specific retailer, heavy/lapsed buyers). If your profile doesn’t exist, speak to your Customer Success rep to get it built (but make sure your requested profile will have enough data to meet the guardrails below)

What are the guardrails around using profiles?

As mentioned above, profiles should be used diagnostically to understand which subgroups the ad works particularly well for.

There are guardrails in place to ensure we maintain research quality and prevent incorrect decisions being made off a profile subset with a small base size. 

• On an individual measure, if you select a profile that has a subset sample size of less than 30, no data will appear for the subsample.  Where the sub sample size is between 30 and 99, you will get a warning that the base size is low and to use the data with caution. 
• For summary metrics using percentiles, such as brand or sales impact score, if the sample size for the sub group is less than 50, no data will display.  If it’s between 50 and 149, a warning will appear to use data with caution. The functionality is designed to be used more diagnostically so do use this with caution.
• If you select a profile (e.g. women) which includes fewer than 20 projects with a minimum profile sample size of n=20 (i.e. 20 projects minimum with 20 women minimum in each), data is hidden and an alert message displays. In other words: if you select  “weekly brand users” as the profile and “country” as the norm, data will not appear unless 20 studies have been run in that country that captured data from n=20+ respondents who are weekly brand users.
• Norms will be available for any profile created (assuming at least 20 adverts available with a minimum base of n=20 per concept).

Interpreting data and taking decisions

There are a number of different comparisons and decisions people want to make when pre-testing ads.  Most brands need at least some media support, so ‘using nothing’ is often not an option.  Therefore, common decisions people need to make are:

1. Is this ad/are any of my ads strong enough for good ROI? 
2. Is/are the ad(s) good or great?
3. Which execution(s)/creative route(s) is/are strongest?
4. Is this new ad stronger than the most recent advertising I’ve invested in? Is this new ad stronger than my competitor’s ad?
5. Which iteration of the ad is strongest? (recommended for meaningful differences between creative, not small iterations)

Within the platform there are two different analysis routes that enable you to do all of the above and there is a simple toggle available on the right hand side of the page called ‘significance testing’ so you can switch between them. You should toggle to ‘Norm’ for ‘Questions 1 and 2 (comparing to a norm). You should toggle to ‘stimuli’ to sig test between the chosen ads and answer questions 3-5.

How does the ad to ad comparison work?

The ad to ad comparison sig tests between all the stimuli on each measure. 

• Where a measure for a specific ad is significantly above another ad, the colour will be bolded to draw attention to this strength
• And then in place of showing the norm under the achieved score, there is a letter which denotes which column (ad) this ad is stronger than (is for example it says B, it means the ad in this column is stronger than the ad in column B). 
• For sales/brand impact, you will still see the percentile score for each ad but we are using the absolute sales/brand impact score (from which the percentile is calculated) to inform whether one ad is significantly above another on the summary metric

To learn more about taking decisions with Amplify, read here.

Interpreting cultural sensitivity

Why measure it

It's crucial to approach the interpretation of sensitive or potentially offensive content with empathy and cultural awareness. This type of feedback is largely subjective, but by actively engaging with the feedback and striving to stay attuned to cultural nuances, you can refine your advertising to better align with the values and expectations of your audience, thus building trust and equity for your brand.

How to find it in your report

In Amplify, we use the Cultural Sensitivity prompt from the questionnaire to collect a numeric value for the portion of individuals who found something offensive, unpleasant, or disturbing about the ad. We then collect verbatim from those who did report something sensitive about the content.

Original Amplify:

• You will find the “Cultural Sensitivity Check” numeric measure included on summary charts contained within the Return chapter.
• You will find the “Cultural Sensitivity Verbatim” auto-coded chart within the Risk chapter.

Amplify 2.0:

• You will find the “Cultural Sensitivity Check” numeric measure included on summary charts contained within the Creative Effectiveness Summary chapter.
• You will find the “Cultural Sensitivity Verbatim” auto-coded chart within the Resonance chapter.

How to analyze the score

Zappi’s Cultural Sensitivity measure is designed to ensure that the content of your creative does not inadvertently offend any group of consumers. It does this by first capturing how many respondents feel that there may be room for any people to find the stimuli offensive, unpleasant or disturbing. We then ask those respondents what it is that they believe people may find offensive, unpleasant or disturbing.

No norm or benchmark is associated with this measure. Instead our advice is that any number greater than zero should be evaluated, and that all of the comments from respondents should always be read for this measure. Ideally these comments should be read by a cross-section of the communities represented in the population for which the stimuli was tested. We advise this for a few reasons:

• The measure is itself a projective exercise. We do not ask respondents what they find offensive - instead we ask them if there is anything that they think others may find offensive. The data that we collect for this measure can therefore be somewhat speculative. For example, if an advert uses dry or close-to-the bone humor, then it is possible that while no respondents do find the ad offensive themselves, they may speculate that others will. This may result in ads that pursue a certain strategy receiving higher scores for this measure purely as a result of that creative strategy. This potential makes this measure unsuitable for benchmarking.
• Groups that may find something offensive vary in size. Our experience with this measure tells us that different groups are likely to react to this measure in different ways, depending on the specifics of the stimuli being tested. Zappi uses stratified samples via weights and quotas to represent the populations of each of the markets that fieldwork is conducted in. It is very possible that the specific communities that may find a reason to find an ad offensive vary in size. As an example, roughly 6% of the US population is Asian, and 14% are Black or African American. If the offensive caused is only picked up on by a given community, the proportion of respondents reporting that an ad has potential for offense can vary in line with the relative size of that community.

Therefore, we recommend that any Cultural Sensitivity Check score greater than zero trigger a closer look at the verbatim.

How to analyze the verbatim

Cultural sensitivity is of course subjective, so we recommend drawing on these best practices when sensitive content has been flagged by respondents:

• Be sure to assess the severity of the perceived offense, as well as the breadth. Some feedback may highlight minor cultural misunderstandings or preferences, while others may point to genuinely harmful or disrespectful content. Some may be mentioned only once or twice, or some may be mentioned by most who responded. Take the time to understand the specific aspects of the ad that triggered the offense and whether they align with widely accepted cultural norms.
• Negate the risk of unconscious bias or the lack of cultural context of a single user. If you are analyzing on behalf of a market that you do not personally live in, or analyzing an advertisement geared toward an audience of which you don’t belong or identify, we recommend supplementing your analysis with that of a colleague or consultant living within the local market or target audience.
• Prioritize transparency and accountability in your decision-making process. By communicating openly with stakeholders about the feedback received, it could be that an open and thorough review of the content in question leads to only minor adjustments to mitigate offense, without compromising the overall message.

AI-Generated Reports

AI Quick Report: Automated reporting is available upon the completion of a survey which provides a concise summary of results and diagnosis of why.  It is produced by combining human expertise in how to interpret and diagnose consumer responses from your survey with AI.

If you are looking for any additional information, please check out the links below:

• Zappi Amplify TV 1.0 FAQ
• Animatic stimuli guidelines
• Demo Report