How to Compare Video Edits & Stimuli Side-by-Side
When testing multiple creative executions or comparing an optimized edit against an original version, you must use the correct dashboard configurations to identify meaningful differences.
Follow these steps and platform rules to accurately compare your stimuli.
⚠️ The Golden Rule of Creative Comparison
Never use percentile scores to compare your edits. Percentile scores are best used only for an "at-a-glance" sense of how an ad has performed globally (Great, Good, OK, or Poor). A change in a percentile score does not necessarily reflect a statistically significant difference.
To validate if there is a meaningful difference between two executions, always look at the absolute scores, as the platform calculates significance testing (ad-to-ad) using absolute sample numbers.
Step 1: Configure Your View
Open your project reporting dashboard.
Select the Stimuli-to-Stimuli Comparison View.
Do NOT use the Ad-to-Norm comparison view when evaluating specific edits against each other.
Step 2: Identify Significant Differences
Begin your analysis by checking the absolute scores for your primary, high-level metrics.
Scan the metrics that feed your core sales and brand impact scores.
Look for designated significant differences across individual measures.
The Decision Rule: If one specific ad edit is significantly better across a number of these key measures, this is the definitive version to progress or optimize further.
Step 3: Diagnose the "Why" (Tie-Breaking)
If your absolute scores show no significant differences—meaning the edits are performing very similarly on your high-level metrics—you must look deeper into the diagnostic charts to choose which version to progress.
Navigate to your open responses and moment-by-moment charts, and evaluate the following four diagnostic areas:
1. Overall & Moment-by-Moment Emotion
Overall Emotion: Identify which edit conveys your desired emotion more effectively overall.
Emotional Moments: Look closely at the moment-by-moment emotional response curve. Determine which edit best represents your intended story arc and generates stronger or more frequent "peaks" in key emotions.
End Emotion: Check the final moments of the curve to see which edit leaves respondents feeling the most positive as the ad closes.
2. Story Playback & Comprehension
Review the Story Playback data. Check if one ad edit is more clearly understood by respondents than the other.
Look at Dislikes and story data to see if your revised version successfully resolved a misunderstanding or narrative issue identified in previous tests.
3. Unaided Messaging & Brand Associations
Review the Unaided Messaging data to see which edit more clearly and successfully conveys your intended brand associations.
If you tested a highly functional ad, check if the delivery of key messages or Reasons to Believe (RTBs) improved compared to versions that lacked message take-out or credibility.
4. Open-Ended Responses (Likes & Dislikes)
Look for specific moments, characters, or creative themes that respondents frequently mention in their positive open-ended text. Note if these elements are present in one edit but missing in the other.
Check the Dislikes section. Review whether your adjustments (such as restructuring the story, changing character appearances, or integrating distinctive assets) successfully reduced negative feedback or improved brand presence during the most memorable aspects of the ad.