AdWords is now Google Ads. Our new name reflects the full range of advertising options we offer across Search, Display, YouTube, and more. Learn more

Ads
1.4K members online now
1.4K members online now
Improve your Google Ads performance and boost your ROI, CTR, and Quality Score
Guide Me
star_border
Reply

How do you calculate minimum sample size for Adwords test with cost/conversion as goal?

Follower ✭ ✭ ☆
# 1
Follower ✭ ✭ ☆

Hi everyone,

 

For A/B testing there is the element of statistical significance on one hand, and the element of minimum sample size for statistical significance to be "real", on the other hand. However, for determinating the minimum sample size, most online sample size calculators take as starting point the baseline conversion rate, and then one just has to type the amount of desired difference in CR for the variation(s). My question is: What happens when my goal is not to improve the Conversion Rate, but to improve the Cost per Conversion? This might happen through the Conversion rate, but also through other elements such as CTR or CPC. Therefore I believe there is no point in determining a base and a target CR, since that is not my goal.

 

How would you determine then the sample size for a case of improving the cost per conversion?

Do you know any tools for it like there are when the target is to improve the conversion rate?

 

Thanks a lot!

1 Expert replyverified_user

How do you calculate minimum sample size for Adwords test with cost/conversion as goal?

[ Edited ]
Participant ✭ ✭ ✭
# 2
Participant ✭ ✭ ✭

It's important to keep in mind, when you are measuring conversion rate, you are not measuring a static phenomena in nature, or a physical object which remains constant.  Conversion rate often fluctuates day to day, week to week and it influenced by the ever-changing psychology of the market (buyers). So attempting to quantify it very precisely can reach the point of diminishing returns very quickly. 

 

If you assume CTR is 1% or better, then 100 impressions is a minimum sample size to get you close to the real number. But keep in mind, CTR is also a moving target.

 

If you have a 1% CTR and want to measure the conversion rate, then 10,000 impressions will give you about 100 clicks which should produce 1 to 5 conversions if it falls into the 1-5% zone.  But a larger sample size is better. 

 

Cost per conversion (CPA) is a calculated number: cost per click divided by conversion rate. To scientifically calculate the number of samples in those two parameters then you will need to know their historical standard deviation. So you will have to define the length of your sampling window. Practically speaking, changes in CPA should become fairly obvious when adjustments are made to the campaign, just by observing shifts in those variables within a few sampling periods. 

 

How do you calculate minimum sample size for Adwords test with cost/conversion as goal?

Follower ✭ ✭ ☆
# 3
Follower ✭ ✭ ☆

Hi Steve B,

thanks for your reply. Sure, historical data is there, and the volume of conversions is enough for a scientifical test.

 

Regards,

Arturo.

 

How do you calculate minimum sample size for Adwords test with cost/conversion as goal?

Participant ✭ ☆ ☆
# 4
Participant ✭ ☆ ☆

Great question first of all. 

 

However, increasing your conversion rate within the same budget will in turn improve your cost per conversion. So, in that respect there should be no difference in the way you work out your sample sizes?

 

In my mind, in order to test this accurately you'd have to test a single keyword against a single ad at a time in its own ad group. Because that would be the only way to use a sample size which was the same across all tests. 

 

You'd also need to make sure you were using Manual CPC bidding and the same max CPC. 

 

I think testing in that way would probably be the easiest. So just work out your sample size as you normally would.

 

 

How do you calculate minimum sample size for Adwords test with cost/conversion as goal?

Top Contributor
# 5
Top Contributor

This is a very good question;

The CPA targeting is a machine learning algorithm  which factors in  our  metrics relevant (CTR, impressions, CPC,)  as mentioned.

As already also mentioned this is a statistical  dynamic algorithm. Google recommends at least 30 conversations during the last 30 days. I assume the standard deviation of this sample is quite broad. My recommendation: triple this sampling to at leas 90 conversions. Further, if you have the data statistically, It would be the best to have 6 times this figure (e.g. About 200 conversations)

 

Moshe, AdWords Top Contributor , Twitter | Linkedin | Community Profile | Ad-Globe
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Best Answer’

How do you calculate minimum sample size for Adwords test with cost/conversion as goal?

[ Edited ]
Participant ✭ ✭ ✭
# 6
Participant ✭ ✭ ✭

Yep Moshe is spot on there.  I have seen Target CPA fail with 30-50 conversions, but once you give it a couple hundred conversions it works. From what I understand it is a linear algorithm, basically pattern matching, so it needs a lot of training in order to work.  And it does not always work, even with a high conversion volume; depends upon whether the algorithm can make sense of the patterns.  It is worth trying though, i have also seen some sensational results from Target CPA. 

 

How do you calculate minimum sample size for Adwords test with cost/conversion as goal?

[ Edited ]
Follower ✭ ✭ ☆
# 7
Follower ✭ ✭ ☆

Hi Moshe,

thanks for your reply. Indeed the amount of conversions is key, I am aware, and at best it is to go beyond the needed sample size in conversion volume. But thats precisely the issue: 2 or 4 or 6 times the volume of a minimum sample size that I cannot calculate, since it has to be calculated based not on a conversion rate, but based on a conversion cost, which is what I want to improve. All sample size calculators allow to calculate your sample size based on a conversion rate, not on cost per conversion.

 

Thanks!

 

How do you calculate minimum sample size for Adwords test with cost/conversion as goal?

Follower ✭ ✭ ☆
# 8
Follower ✭ ✭ ☆

Thanks for your reply Abbie,

you are right that a better CR will improve my cost per conversion, but what I think is that if I focus on it from the point of view of CR, then I am closing the door to other options that might help me too, since I dont know how that cost reduction will take place (whether its via CR or via CPC or CTR), thats also why i do the experiment or test.

So the problem is not so much having the same sample size for all tests, but knowing what is the needed size in order to know when I can stop the test (if it is significant).

Thanks again!