AdWords
2.6K members online now
2.6K members online now
Improve your AdWords performance and boost your ROI, CTR, and Quality Score
Guide Me
star_border
Reply

How to cross test text ads with any degree of statistical significance?

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Hi. Has anyone got any ideas how the likes of Perry Marshall supposedly cross test ads with true statistical significance? We have difficulty doing cross tests where ads are 'rotating more evenly' (but sometimes still heavily favoured after even a day) and ads can have different positions and be paying slightly different avg CPCs and can also be heavily skewed by the CTR of search partners. Also, it's impossible to serve ads evenly in a funnel of multiple ad clicks and attribute any significance to conversion rate, although it is CTR which we are primarily concerned with cross testing. Does anyone know a formula which removes all the variables and leaves you with a statistical significant cross test? Thanks a lot. James

3 Expert replyverified_user
2 ACCEPTED SOLUTIONS

Accepted Solutions
Marked as Best Answer.
Solution
Accepted by Mini-CM (Community Manager)
September 2015

Re: How to cross test text ads with any degree of statistical significance?

Top Contributor
# 2
Top Contributor

Hi James,

 

Welcome to the forum Smiley Happy

 

Althought I don't use some 3rd party tools for analyzing Adwords related metrics, but I find this particular tool very helpful in determining the performance gauge of my ads. 

 

You might be interested in taking a look at it. Here it is.

 

Addition to this, you might give it a try to Adwords Campaign Experiments, which provides statistical data pertaining to the experiments you are running and whether they are successful or not. You can find more about them here.

 

Just a thought!

Pankaj

View solution in original post

Marked as Best Answer.
Solution
Accepted by Mini-CM (Community Manager)
September 2015

Re: How to cross test text ads with any degree of statistical significance?

Top Contributor
# 3
Top Contributor

Hi James,

 

While testing ads, there are 2 things I do:

 

- Change only 1 element of an ad. This helps you know what performed better. That 1 element could be as small as shifting the CTA from the headline to the description line.

 

- The other thing that I do is wait for a significant sample size. this may mean any number that is enough to convince you that ad1 is better than ad2. Personally, more than the number of click,conversions of a single ad to consider as sample size.. I prefer looking at the difference. If the difference is big enough, it convinces me! Smiley Happy

 

Hope this helps!

 

Cheers,

Shweta

Cheers,
Shweta

AdWords Top Contributor
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Accepted Solution’!

View solution in original post

Marked as Best Answer.
Solution
Accepted by Mini-CM (Community Manager)
September 2015

Re: How to cross test text ads with any degree of statistical significance?

Top Contributor
# 2
Top Contributor

Hi James,

 

Welcome to the forum Smiley Happy

 

Althought I don't use some 3rd party tools for analyzing Adwords related metrics, but I find this particular tool very helpful in determining the performance gauge of my ads. 

 

You might be interested in taking a look at it. Here it is.

 

Addition to this, you might give it a try to Adwords Campaign Experiments, which provides statistical data pertaining to the experiments you are running and whether they are successful or not. You can find more about them here.

 

Just a thought!

Pankaj

Marked as Best Answer.
Solution
Accepted by Mini-CM (Community Manager)
September 2015

Re: How to cross test text ads with any degree of statistical significance?

Top Contributor
# 3
Top Contributor

Hi James,

 

While testing ads, there are 2 things I do:

 

- Change only 1 element of an ad. This helps you know what performed better. That 1 element could be as small as shifting the CTA from the headline to the description line.

 

- The other thing that I do is wait for a significant sample size. this may mean any number that is enough to convince you that ad1 is better than ad2. Personally, more than the number of click,conversions of a single ad to consider as sample size.. I prefer looking at the difference. If the difference is big enough, it convinces me! Smiley Happy

 

Hope this helps!

 

Cheers,

Shweta

Cheers,
Shweta

AdWords Top Contributor
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Accepted Solution’!

Re: How to cross test text ads with any degree of statistical significance?

Top Contributor
# 4
Top Contributor

Hello jimmy_c;

 

 

Any statistical tool, that compares performances of campaigns, to decide which one is statistically significant performing better, and does not quote the level of confidence, - is useless.

 

 

-Moshe

 

jimmy_c wrote:

Hi. Has anyone got any ideas how the likes of Perry Marshall supposedly cross test ads with true statistical significance?

Does anyone know a formula which removes all the variables and leaves you with a statistical significant cross test? Thanks a lot. James




 

 

 

Moshe, AdWords Top Contributor , Twitter | Linkedin | Community Profile | Ad-Globe
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Best Answer’