AdWords
1.8K members online now
1.8K members online now
Use AdWords conversion tracking and reporting to measure your results. Have a question about Google Analytics? Post it here, on the Google Analytics Community!
Guide Me
star_border
Reply

AdWords Experiments Equations

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Hi guys, before the update to the 'Understanding Experiments' there was a page that listed the equations used to figure out confidence interval.  I don't see it anymore.  Can someone point me to it?

1 Expert replyverified_user
Marked as Best Answer.
Solution
Accepted by Karl (Explorer ✭ ✭ ☆)
September 2015

Re: AdWords Experiments Equations

Explorer ✭ ✭ ☆
# 2
Explorer ✭ ✭ ☆

Hi Ryan5231

 

It's changed slightly with arrows being used instead. You can read more in the Help Center here: http://support.google.com/adwords/bin/answer.py?hl=en-AU&answer=2375390&from=168307&rd=1

 

Copying the most relevant part:

If your experimental data is statistically significant, meaning that it's likely that any differences in performance aren't due to chance, we'll display an Arrow up up arrow or Arrow down down arrow next to that data depending on whether your performance has increased or decreased. As many as three arrows can appear in the same direction, and the more arrows in the same direction, the more statistically significant the results are. One arrow means that we're 95 percent certain that the change is not due to chance, two arrows means we're 99 percent certain, and three arrows means we're 99.9 percent certain. Two gray arrows Arrows in opposite directions mean the results are not statistically significant.

The more statistically significant the results, the more likely that the results you see in your experiment will continue if you apply these experimental changes. The more traffic you have on a keyword, ad, or ad group, the faster you're likely to get statistically significant results.

 

Hope this helps!
Karl 

Marked as Best Answer.
Solution
Accepted by Karl (Explorer ✭ ✭ ☆)
September 2015

Re: AdWords Experiments Equations

[ Edited ]
Top Contributor
# 3
Top Contributor

Hi Ryan,

 

In addition to Karl's excellent advice, I'd note that relevant calculations and formulas are based on so called confidence intervals that any one might be familiar with who's ever dived in advanced theories of probabilities. Yes, that's math! When I was still at school we had to know the formula and calculate on our own. However, the world has changed a lot and now you have online tools that do the hard (and boring) work for you.

 

     http://www.splittester.com/index.php

 

The Split Tester linked to above is a very simple free tool of this kind. Playing with it for a few hours you may develop a feeling about how much data you may need to accrue so you can be e.g. 95% certain about a hypothesis. (Of course it means that you will err 5 times out of 100 times.)

 

Best,

Lakatos