Open the experiment in Cooee to see live metrics broken down by arm. Every metric is calculated per unique visitor, and results refresh daily.
What each metric means
Metric | What it measures |
Impressions | Unique visitors assigned to this arm who started a tracked session |
Add to Cart Rate | Share of visitors who added at least one product to cart |
Conversion Rate (CVR) | Share of visitors who completed a purchase |
Revenue per Visitor (RPV) | Total revenue divided by total visitors — the most complete single measure |
Average Order Value (AOV) | Average order size among visitors who converted |
Session Duration | Average time spent on site per session |
Bounce Rate | Share of visitors who left without interacting |
Pure Revenue | Revenue from visitors who converted within the same session in which they were first assigned |
Pure Revenue vs Revenue per Visitor. Pure Revenue is deliberately conservative — it ignores purchases that happened on a later visit. RPV counts all revenue from assigned visitors whenever it arrived. If the two disagree sharply, your buying cycle probably spans multiple sessions, and RPV is the fairer measure.
Check the guardrails, not just your primary metric
A theme change moves several things at once. Always read Conversion Rate, Revenue per Visitor, and Bounce Rate alongside whatever you nominated as the primary metric.
A variant that lifts Add-to-Cart Rate while conversion drops is not a win — it is usually a sign that the new layout pushes more people into the cart who were never going to buy. Revenue per Visitor is the hardest metric to fool, so treat it as the tie-breaker.
Check mobile and desktop separately as well. A theme that wins on desktop and loses on mobile is common, and it points to a device-specific rollout rather than a straight switch.
How long to run
Run for at least two full business cycles — 14 days for most stores. Shopping behaviour varies strongly by day of week, and a test that covers only part of a week measures the week as much as the theme.
Stores with fewer than 500 daily unique visitors should plan for four to six weeks before conversion-level metrics settle.
Resist checking daily and stopping the moment a result looks good. Early numbers swing widely, and stopping at the first favourable moment is the most reliable way to ship a change that does nothing.
Why the variant may show slightly fewer sessions
Variant visitors are served the challenger theme through one additional page load. Visitors who leave during that brief load are counted as assigned but never complete a tracked session, so the variant arm typically shows around 3–5% fewer tracked sessions than control even with a perfectly even split.
This is inherent to redirect-based testing across the industry, not a fault in your setup. Two things follow from it:
The gap stays roughly constant. It does not widen over time, and a widening gap is worth reporting to us.
The visitors lost are disproportionately fast bouncers, so the variant's engagement metrics read very slightly better than reality.
For most experiments the effect is too small to change the decision. If your result is close, compare using total assigned visitors rather than tracked sessions for the more conservative read.