AdWords Experiments Equations
Hi guys, before the update to the 'Understanding Experiments' there was a page that listed the equations used to figure out confidence interval. I don't see it anymore. Can someone point me to it?
Re: AdWords Experiments Equations
It's changed slightly with arrows being used instead. You can read more in the Help Center here: http://support.google.com/adwords/bin/answer.py?hl=en-AU&answer=2375390&from=168307&rd=1
Copying the most relevant part:
If your experimental data is statistically significant, meaning that it's likely that any differences in performance aren't due to chance, we'll display an up arrow or down arrow next to that data depending on whether your performance has increased or decreased. As many as three arrows can appear in the same direction, and the more arrows in the same direction, the more statistically significant the results are. One arrow means that we're 95 percent certain that the change is not due to chance, two arrows means we're 99 percent certain, and three arrows means we're 99.9 percent certain. Two gray arrows in opposite directions mean the results are not statistically significant.
The more statistically significant the results, the more likely that the results you see in your experiment will continue if you apply these experimental changes. The more traffic you have on a keyword, ad, or ad group, the faster you're likely to get statistically significant results.
Hope this helps!
Re: AdWords Experiments Equations[ Edited ]
May 2012 - last edited May 2012
In addition to Karl's excellent advice, I'd note that relevant calculations and formulas are based on so called confidence intervals that any one might be familiar with who's ever dived in advanced theories of probabilities. Yes, that's math! When I was still at school we had to know the formula and calculate on our own. However, the world has changed a lot and now you have online tools that do the hard (and boring) work for you.
The Split Tester linked to above is a very simple free tool of this kind. Playing with it for a few hours you may develop a feeling about how much data you may need to accrue so you can be e.g. 95% certain about a hypothesis. (Of course it means that you will err 5 times out of 100 times.)