Confidence Levels

What are Confidence Levels?

Confidence Levels indicate how high we set the bar for statistical significance.

A simple way to think about confidence intervals is that they refer to how often we can expect a difference that is observed in results to be repeated. If a difference is flagged as statistically significant at 95% confidence, and we ran the same survey 100 times, then we would expect to see the same difference in 95 of them.

This means that if you really want to be sure that a differences is ‘real’ when comparing stimuli to a norm, or stimuli to one another, then a higher degree of statistical significance is desirable. When applied to charts, confidence levels dictate what we consider to be significant difference between stimulus.

Why is this Feature Useful?

When Confidence Levels are Adjusted:

  • Lower Confidence Levels (90%)
    • Smaller Intervals: Assumes more precision in the statistical measurements, but accepts higher risk of a false positive (a significant difference where there may not be one).
    • Useful if wanting to tease apart smaller differences in performance of the things being evaluated. Often used in research where acquiring larger sample sizes might be problematic, due to a small population, or difficulties in sampling.
  • Standard Confidence Level (95%) - Zappi Default
    • Balanced Intervals: Provides a middle ground, offering a solid balance between precision and reliability.
    • Ideal for most use cases and widely accepted in professional settings. Often used in social sciences.
  • Higher Confidence Levels (99%)
    • Wider Intervals: Shows more conservative estimates with higher certainty, but the range may obscure smaller differences in data.
    • Best for critical decision-making where minimizing error is crucial. Often used in medical research.

When None is Selected:

  • No Confidence Intervals: Focuses purely on raw data visualization, removing uncertainty indicators for a simplified view.
  • Useful for presentations or cases where confidence intervals might clutter the chart or confuse the audience.

How the Confidence Levels Works

With the Confidence Level Selector, users can:

  1. Select a Confidence Level: Options include 90%, 95%, or 99%, affecting the size of the intervals displayed.
  2. Select None: Disable confidence intervals entirely for a cleaner visualization.

The feature is located in the chart settings panel and dynamically updates the chart in real-time as confidence levels are changed.

How can I edit Confidence Level?

Setting on chart is easy; simply navigate to the chart of your choice (and see the list of charts below to find out where this feature is available), navigate to the Statistics sidebar and select your confidence level. The chart will update to show you the chart with your selected confidence level.

You may also set a default confidence level for your organisation. To do this, please contact Zappi Platform Support.

On what charts can I edit Confidence Level?

You can find a full list of charts where confidence level is available to be set below. Please note, the name of the chart type listed below and the title of the chart in your report may differ.

  • Unaided Brand Recall
  • Brand Purchase Page
  • Individual Scene Diagnostic
  • Second-by-Second Emotional Response
  • Overall Emotions
  • Percentile Metrics Overview
  • Percentile Metrics Summary
  • Scene Comparison
  • Measure vs Average Summary Page
  • Norms Breakdown
  • Key Message Clarity
  • Overall Performance
  • Choice measure summary
  • Flexible grid question bar chart
  • KPI Executive Summary
  • Individual Multi Measure Summary
  • Multiple measure summary page
  • Measure Summary with Norm
  • Question bar chart
  • Grid question bar chart
  • Per survey grid question bar chart
  • Grid multiple choice question bar chart
  • Grid scale question bar chart
  • Scatter Plot
  • Flexible Chart
  • Repeated Choice
  • Additional Question