AdWords is now Google Ads. Our new name reflects the full range of advertising options we offer across Search, Display, YouTube, and more. Learn more

Ads
4.3K members online now
4.3K members online now
Improve your Google Ads performance and boost your ROI, CTR, and Quality Score
Guide Me
star_border
Reply

A/B Testing different ad copy

[ Edited ]
Follower ✭ ✭ ☆
# 1
Follower ✭ ✭ ☆

Hi everyone,

 

I'm wondering if there is a rule of thumb about how many impressions I need before my data is accurate? I'm running 4 ads and I get about 80 impressions total a day (20 per ad).  My guess was after an ad has about 500 impressions I will have a decently accurate CTR.

 

 

Starting slow to get the hang of things.

 

Thanks

 

2 Expert replyverified_user

Re: A/B Testing different ad copy

Follower ✭ ✭ ✭
# 2
Follower ✭ ✭ ✭

1000 impressions should be fairly accurate.

 

then again, i'm just pulling this out from a hat, for all practical purposes.

 

you might get there way before, especially if your CTRs are high, or get there way after, if they're low

 

rule of thumb: higher the CTR, sooner you get an accurate idea. the lower the CTR, the more impressions you need

 

off the top of my head, I would say around 4% CTR should be accurately measurable after 200-600 impressions

 

for an accurate answer to this question, though, a statistical analysis would be needed.

 

 

Re: A/B Testing different ad copy

Top Contributor
# 3
Top Contributor

Hello and welcome,

 

To get the statistical analysis for anything new you want to know, there is something called Adwords Experiments. You might consider running them inside your Adwords account to make proper business decisions.

 

http://support.google.com/adwords/answer/2385204/?hl=en&

 

Just a thought!

Pankaj

Re: A/B Testing different ad copy

Top Contributor
# 4
Top Contributor

Hi birchlore,

 

I don't think I would use an experiment just to test ad copy. Place both ads in the same ad group and set the rotation to even "forever".

 

It does depend a bit on the CTR how many impressions you will need to get repeatable performance, but I don't think that's where your focus should be. You really need to look at the conversion rates, too. You'll need to get enough impressions to establish that conversion rate. Google waits for 1000 impressions on a new keyword before that keyword in that campaign and ad group begin to influence the QS. That's probably a good number.

 

Best of Luck!

 

Pete

 

petebardo -- Deadhead doing AdWords

Re: A/B Testing different ad copy

Follower ✭ ✭ ☆
# 5
Follower ✭ ✭ ☆

Thanks guys!

 

Followup question:

 

I am currently a/b/c/d testing 4 ads. Is that too many since I only get about 80 impressions total per day (and about 2 clicks and .2 conversions)? Do you think it's more optimal to only run 2 at a time in this scenario?

Re: A/B Testing different ad copy

Top Contributor
# 6
Top Contributor
Hello again,

Yes I prefer running 2 ad copies with rotation mode settings as mentioned by Pete above. Running too much ad copies within a single ad group will not do much justice for you because of the low impression volume per day which you received.

My 2 cents!
Pankaj

Re: A/B Testing different ad copy

Follower ✭ ✭ ☆
# 7
Follower ✭ ✭ ☆

I hope this topic isn't getting too diluted but I'm finding this very helpful. Too be honest I'm a little overwhelmed by all the different testing possibilities.

 

When starting my a/b testing... should I test two ads that are completely different, or only slightly different?

 

For example, which would be a better starting a/b set?

 

1)

a)

Toronto Mortgage Broker

Purchases, Refinances, and Renewals.

Contact me for professional advice.

 

b)

Toronto Mortgage Broker

Great Service and Excellent Rates.

Contact me for professional advice

 

 

2)

 

a)

Toronto Mortgage Broker

Purchases, Refinances, and Renewals.

Contact me for professional advice.

 

b)

Jason Williams

Local Toronto Mortgage Broker

Get pre-approved in 5 minutes.

 

I think if I was getting a ton of impressions it would be really useful to use pairs like the first set. But since each test is going to take me month(s) before getting enough data, wouldn't it be more useful to start with major differences and then work it until it becomes more and more specific... (although if I use the second set, I won't know if it's the headline, the service, or the CTA that made the difference).

 

thanks

 

Re: A/B Testing different ad copy

Top Contributor
# 8
Top Contributor

Hi birchlore,

 

You are probably right about running only 2 ads in a group that takes that long to accrue impressions and CTR data.

 

The first set is good in that it only tests your service description. The second set test both headlline and CTA. You are correct it will make it difficult to tell which of those made a difference.

 

Best of Luck!

 

Pete

 

petebardo -- Deadhead doing AdWords

Re: A/B Testing different ad copy

Explorer ✭ ✭ ✭
# 9
Explorer ✭ ✭ ✭

First, you should always run two - and only two - ads at a time. Running one ad is not a test as there's nothing to compare it to during the time it was running. Unfortunately, I've seen many run one ad, stop it and test with another. This tells you nothing.

 

The other extreme I've seen is running many ads. This really doesn't tell you much either as you're never sure when each ad did run. There can be differences in performance depending on the time of day, day of week that ads run and you need to know this. Having a dozen ads active at once will only muddle the test. So run two ads. Three at the most but I do this rarely and only when there is a high volume of searches, high meaning hundreds.

 

Now, how many impressions are needed? The answer is in mathematics.

 

What you are measuring and comparing is click rate. If your click rate is 1%, waiting for 80 impressions is not enough. Even 500 is not enough. What every one here has said is basically waiting for a certain number of impressions which is the denominator (the number that tells us how many parts something is divided in).

 

What you need to do is wait until you get a certain number of clicks, not impressions. That should be for each ad you are testing, not the total of your test.

 

So how many clicks? I use 20 clicks for each ad minimum and that gives me a fairly high confidence level that the winner will actually be the winner something like 95% of the time. Of course, more clicks is better to get even higher confidence. Note that this has nothing to do with time as some people say like "run your test for a week". If it takes a week or a month or a year to get those 20 clicks per ad, that's what you have to do.

 

As for differences in ads, yes, use ads that are different. Again, I've seen others using the same ads for their test, save for maybe the headline. Doing this tells you next to nothing. You have the opportunity here to test different things, use different ads, different description lines. You can test switching the description lines but I would not do that at first. So I'd use your second example as they are totally different. Mix up your descriptions, try variations. What often happens is that you notice one line in particular that seems to work and you tend to stick with it. In that case, vary the other line, mix them up.

Re: A/B Testing different ad copy

Follower ✭ ✭ ☆
# 10
Follower ✭ ✭ ☆

Thanks for this response. I really like the concept of waiting for a certain number of clicks instead of impressions.