Zappi Optimize Rank

Zappi Optimize Rank uses an expert-led ranking technique to detect the most powerful items that will make your concept take off.

Contents

Key Measures

  • Flexibility to rank a wide variety of stimuli including packshots, ideas, claims, flavors, ingredients, taglines, etc.
  • Simple and intuitive output, which can be manipulated for deeper analysis. 
  • Make the best choices running complex analysis and gleaning respondent-level scoring. Cut your data by sample or custom question measures for deeper granularity.
  • Expert-led configuration, with recommended sample sizes and robust MaxDiff survey design algorithm

Flexible configuration on the Optimize Suite

With the Zappi Optimize Suite you tailor your projects by choosing the metric with which you want to run your rankings against (rank by appeal, behavior change, believability, relevance, meet needs, impact to appeal, impact to behavior change, fit with brand impressions). Jump to Choosing the Metric You Need.

back to top

Utility Score: how we calculate it

Utility scores tell you the value of each item relative to the top performing item based on your rank measure (appeal, behavior change, etc.).  
How it's calculated
  • For each item, we calculate an average "max diff" score based on the number of times each respondent selects the item as most [appealing], least [appealing] and not selected.  
  • We sort the max diff average scores from highest to lowest and, rather than reporting each item’s max diff score, we normalize each item as a % of the top scoring item.
  • The top performer will always have a utility score of 100%.  
Use the utility scores to understand
  • Tradeoff between any 2 items - The difference between any 2 items’ utility scores (percentage values) represents the expected tradeoff in [appeal] when choosing between them.
  • Statistical significance - Color coding indicates whether an item’s max diff score is statistically different from the #1 item.  The top ranked item will always be in green. Yellow means no statistical difference; red means the item is statistically lower than the #1 ranked item.
  • Differentiation - The spread between the top and bottom items is an indicator of how differentiated the set of items are from each other.  A bottom score of 10% suggests greater differentiation than a bottom score of 50%

back to top

Example Analysis

In the above example, the researcher tested 6 claims, ranking on Appeal. 

  • Top ranked claim is Stimuli 1
  • Second ranked claim (Stimuli 2) has a utility score of 97% --  its max diff score is 96% of the #1 claim, and you can expect 3% less appeal if you go with #2 instead of #1
  • But this difference between #1 and #2 is not statistically significant
  • Claims ranked 3-6 all have max diff scores statistically lower than #1 
  • With only a 22 point swing between #1 and #6 (100 - 78), this set of claims are not dramatically distinct from each other

back to top

Testing Process:

  1. A user uploads between 5 and 50 stimuli, with an optional survey ‘primer’ (e.g. a video, an image, and/or a statement).
  2. From this information, the Zappi platform uses an algorithm to calculate the optimal sample size and survey design to enable an optimal analysis.
  3. Respondents see up to 11 exercises (screens); they are asked to select the most and least favorite from a selection of five stimuli, as determined by the algorithm (Best-Worst Scaling).

Because we believe in keeping surveys short, we don’t expose any respondent to more than 11 exercises. Making respondents do more than that would be a significant cognitive load for a discrete choice. Thus, to meet the requirements of serving all 5-stim combinations in order, we need more respondents. Because of this reason and also to keep our fielding time reasonable, we limit the max number of stims per survey to 50 (resulting into n=1.500).

back to top

Zappi Optimize Rank Configuration Checklist:

  • Audience: Custom Audience or Zappi Audience (list of available Zappi Audiences can be found on the platform)
  • Metric to rank against: Choose the metric you'd like to run your rankings against and select it in configuration:
    • Appeal
    • Impact to appeal
    • Behavior change
    • Impact to behavior change
    • Believability
    • Relevance
    • Meets needs
    • Fits with brand impressions
  • Stimuli Type, Offering, and Category will get piped into the introduction of the survey
    • Stimuli Type (text or image-based): packshots, ideas, claims, flavors, ingredients, taglines, etc.
    • Type of Offering: Service or Product
    • Category: Select from available categories on the Zappi platform
    • Stimulus/stimuli:
      • 5-50 text or image based stimuli
      • Text-based stim can have up to 255 characters per variety
      • Sample size, cost, and fieldwork time will increment as you add more varieties
  • (Optional) Survey introduction statement: You can choose to provide a custom introduction of what the survey is about (with Video & Text, Image & Text, Text Only). 
    • For non-English markets, your custom statement needs to be in the local language.
    • If you opt for the Templated Text Introduction option, it will pipe your selections for stimuli type and product/service into a simple statement (Example: "This study is about different [FLAVORS] for a [CIDER] [PRODUCT]").
  • Tagging: Tagging your stimuli allows for better organization of your stimuli and meta-analysis of your concepts later on. Don’t worry if you can’t think of your tags now, they can always be added later.
  • Custom questions (optional for an additional cost): You will need to provide translations for non-English markets. Questions can take the following forms:
    • Grid
    • Scale
    • Open-end
    • Multiple choice
    • Single choice

back to top

Choosing the metric you need

Optimize Rank will let you choose the metric you need to run your rankings against:

Metric Question
Appeal Which of the [type of stimuli] below do you find most appealing and least appealing?
Impact to appeal Which of the [type of stimuli] below would make the [item] most appealing and least appealing?
Behaviour Change Which of the [type of stimuli] below would you be most and least likely to [action]?
Impact to Behaviour Change Which of the [type of stimuli]  below would make you most and least likely to [action] the product?
Believability Which of the [type of stimuli] below do you find most believable and least believable?
Relevance Which of the [type of stimuli] below do you find most relevant and least relevant to you personally?
Meets needs Which of the [type of stimuli] below meet your needs the most and least?
Fits with brand impressions

Which of the [type of stimuli] below fits best and fits worst with your impressions of the product?

back to top

If you are looking for any additional information, please check out the links below: