The overview of a theme, template, Split URL or price experiment compares every variant with your control. Three columns answer the question you care about most: did the variant win, and by how much?
What the columns mean
Column | What it tells you |
CR Lift | How much higher or lower the variant's conversion rate (orders per visitor) is than control's, as a percentage of control's. +10% means the variant converts 10% better than control, not 10 percentage points better. |
RPV Lift | The same comparison for revenue per visitor. |
CR Significant | Yes when the difference in conversion rate is unlikely to be chance, No when it could still be noise. Hover over it to see the p-value; Yes means it is below 0.05. |
The control row is always blank in these columns, since it is what the variants are measured against. A variant is also blank until both it and control have had visitors.
How to read them
Call a winner only when CR Significant says Yes. A lift shown with No is a direction, not a result.
Small lifts on few visitors are usually noise. Early in a test, a variant can swing from +20% to −10% in a few days without anything real changing.
Treat RPV Lift as directional. There is no significance test for revenue per visitor: a handful of large orders can move it a long way. Use it to check that a conversion win is not coming at the cost of smaller baskets, and as the tie-breaker described in Reading theme experiment results.
A worked example
Control and the variant each get 3,000 visitors. Control takes 60 orders (a 2.0% conversion rate) and the variant 66 (2.2%).
CR Lift is +10%: 2.2% is 10% more than 2.0%.
CR Significant is No. Six extra orders out of 3,000 visitors is well within what chance alone produces.
Even with ten times the traffic, 30,000 visitors per arm converting at the same 2.0% and 2.2%, the difference is still not significant. A 10% lift on a 2% conversion rate is a small effect, and it takes a lot of visitors to tell it apart from noise. Bigger differences show up sooner.
Where the numbers come from
Lift and significance are calculated from each arm's totals over the experiment's whole run, refreshed every hour.
Lift is the variant's rate minus control's rate, divided by control's rate.
Significance comes from a standard statistical test (a two-proportion z-test) on orders per unique visitor, which asks how likely a difference this large would be if the two versions really performed the same.
Conversion counts orders, so a visitor who orders twice counts twice. The test treats each order as a separate visitor converting, which makes it slightly optimistic when many customers place repeat orders during a test.
Cooee calculates these on its servers, so you see the same numbers on the dashboard and when you ask Claude about an experiment through the Cooee MCP server.
Before you stop an experiment
Let it run its planned duration. Checking every day and stopping the first time CR Significant says Yes greatly raises the chance of picking a variant that does nothing. A test at p < 0.05 is wrong about one time in twenty when checked once; checked daily, far more often.
With very few orders per arm, the test is not reliable yet. Wait until each arm has at least a few dozen orders before reading anything into CR Significant.
With more than one variant, each is compared with control on its own. The more variants you run, the more likely one of them looks significant by chance, so be more cautious with a single Yes among several.
Not sure whether a result is real? Message us from the chat and we will look at it with you.