AdWords
4.9K members online now
4.9K members online now
Improve your AdWords performance and boost your ROI, CTR, and Quality Score
Guide Me
star_border
Reply

A/B Testing of Text Ads - Scan tool for Statistical Significance

[ Edited ]
Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Hi AdWords Community -

 

So we are running several text ad A/B tests across many ad groups.  These ad group have different daily impressions, and different CTR's, so they need to run for different lengths of time to reach significance.

 

Is there a tool which can scan our whole account, look for active ad pairs, and identify when the differences in CTR become statistically significant?  We do this now by downloading into Excel and looking at the ad pairs individually which takes forever.

 

Some paid PPC management software has this built in as part of their software suite, but I do not want to buy the rest of their services?  Thanks in advance for your help!

1 Expert replyverified_user

Re: A/B Testing of Text Ads - Scan tool for Statistical Significance

Google Employee
# 2
Google Employee

Hi Paul,

 

Thanks for your question. If you are using Experiments, here's how you can monitor your experiment but this may not have the in depth analysis and automation capabilties that you are looking for. 

 

Has anyone else found a work around for this?

Re: A/B Testing of Text Ads - Scan tool for Statistical Significance

Visitor ✭ ✭ ✭
# 3
Visitor ✭ ✭ ✭
Thanks Shanea. We do not have these setup as Experiments due to the difficultly of setting up and tracking text ad A/B tests. Plus, we are just looking for a simple solution to do the following basic tasks:

Download the Account.
Look for Ad Groups running more than one Ad
Highlight any Ad Pairs which have a statistically different CTR.

Google recommends that all advertisers do A/B testing of ads, and it is simple to set them up. However, the analysis of those tests is way harder than it needs to be.
Marked as Best Answer.
Solution
Accepted by topic author Paul S
September 2015

Re: A/B Testing of Text Ads - Scan tool for Statistical Significance

Visitor ✭ ✭ ✭
# 4
Visitor ✭ ✭ ✭

We wrote our own in Access / Excel.  Not easy to use, but a lot better than manual.  If someone solves this with a commercial product, please let us know.  Thanks!

Re: A/B Testing of Text Ads - Scan tool for Statistical Significance

Follower ✭ ✭ ✭
# 5
Follower ✭ ✭ ✭
Brad Geddes built a great tool for ad testing called Adalysis. It is a very powerful tool that works great for testing hundreds or thousands of ads... it shows you the statistical significance of every ad test you are running. I use it and it's really helpful.

Re: A/B Testing of Text Ads - Scan tool for Statistical Significance

Visitor ✭ ✭ ✭
# 6
Visitor ✭ ✭ ✭

Here's a free, friendly statistical significance testing tool that I just used:  http://www.usereffect.com/split-test-calculator.  

Marked as Best Answer.
Solution
Accepted by topic author Paul S
September 2015

Re: A/B Testing of Text Ads - Scan tool for Statistical Significance

Visitor ✭ ✭ ✭
# 7
Visitor ✭ ✭ ✭
There is a new tool which is great for this application: AdAlysis - www.adalysis.com - full disclosure: I have nothing to do with this company except using their software and I really works well. Thanks for all of the help!