Zappi Optimize Rank

Zappi Optimize Rank uses an expert-led ranking technique to detect the most powerful items that will make your concept take off.

Key Measures
Utility Score: How We Calculate It
Example Analysis
Testing Process
Configuration Checklist
Choosing the Metric You Need
Additional Resources

Key Measures

Flexibility to rank a wide variety of stimuli including packshots, ideas, claims, flavors, ingredients, taglines, etc.
Simple and intuitive output, which can be manipulated for deeper analysis.
Make the best choices running complex analysis and gleaning respondent-level scoring. Cut your data by sample or custom question measures for deeper granularity.
Expert-led configuration, with recommended sample sizes and robust MaxDiff survey design algorithm

Flexible configuration on the Optimize Suite

With the Zappi Optimize Suite you tailor your projects by choosing the metric with which you want to run your rankings against (rank by appeal, behavior change, believability, relevance, meet needs, impact to appeal, impact to behavior change, fit with brand impressions). Jump to Choosing the Metric You Need.

Utility Score: how we calculate it

Utility scores tell you the value of each item relative to the top performing item based on your rank measure (appeal, behavior change, etc.).

How it's calculated
For each item, we calculate an average "max diff" score based on the number of times each respondent selects the item as most [appealing], least [appealing] and not selected.
We sort the max diff average scores from highest to lowest and, rather than reporting each item’s max diff score, we normalize each item as a % of the top scoring item.
The top performer will always have a utility score of 100%.

Use the utility scores to understand
Tradeoff between any 2 items - The difference between any 2 items’ utility scores (percentage values) represents the expected tradeoff in [appeal] when choosing between them.
Statistical significance - Color coding indicates whether an item’s max diff score is statistically different from the #1 item. The top ranked item will always be in green. Yellow means no statistical difference; red means the item is statistically lower than the #1 ranked item.
Differentiation - The spread between the top and bottom items is an indicator of how differentiated the set of items are from each other. A bottom score of 10% suggests greater differentiation than a bottom score of 50%

Example Analysis

In the above example, the researcher tested 6 claims, ranking on Appeal.

Top ranked claim is Stimuli 1
Second ranked claim (Stimuli 2) has a utility score of 97% -- its max diff score is 96% of the #1 claim, and you can expect 3% less appeal if you go with #2 instead of #1
But this difference between #1 and #2 is not statistically significant
Claims ranked 3-6 all have max diff scores statistically lower than #1
With only a 22 point swing between #1 and #6 (100 - 78), this set of claims are not dramatically distinct from each other

Testing Process:

A user uploads between 5 and 50 stimuli, with an optional survey ‘primer’ (e.g. a video, an image, and/or a statement).
From this information, the Zappi platform uses an algorithm to calculate the optimal sample size and survey design to enable an optimal analysis.
Respondents see up to 11 exercises (screens); they are asked to select the most and least favorite from a selection of five stimuli, as determined by the algorithm (Best-Worst Scaling).

Because we believe in keeping surveys short, we don’t expose any respondent to more than 11 exercises. Making respondents do more than that would be a significant cognitive load for a discrete choice. Thus, to meet the requirements of serving all 5-stim combinations in order, we need more respondents. Because of this reason and also to keep our fielding time reasonable, we limit the max number of stims per survey to 50 (resulting into n=1.500).