AdWords
2.2K members online now
2.2K members online now
Get started with AdWords - learn the basics to get set up for success
Guide Me
star_border
Reply

Testing same A/B Ad split in different "Match type" Ad groups

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Hi, I've been trying to figure out my particular problem for a few weeks, but can't figure it out. I've looked on line and haven't found any information specific to my particular issue.  I'm hoping to get some good advice, so thanks in advance for any help you can offer.  This is driving me crazy!

 

Here's the scenario: 

 

I am trying to test ads. I have 4 ad groups in total.  Each ad group has a particular theme and each theme has 2 ad groups - one for phrase match keywords and one modified broad match keywords.  I put the same exact ad for each theme into both the phrase and the broad ad group.  I've broken it down below, including the stats.  

 

"Blue - broad"

Ad A - impressions: 142, CTR 11.72%

Ad B - impressions: 1065, CTR 7.98%

 

"Blue - phrase"

Ad A - impressions: 188, CTR 6.91%

Ad B - impressions: 1088, CTR 6.71%


"Yellow - broad"

Ad C - impressions: 919, CTR 6.31%

Ad D - impressions 687, CTR 3.93%

 

"Yellow - phrase"

Ad C - impressions: 963, CTR 5.82%

Ad D - impressions: 756, CTR 7.67%

 

FYI,  

1. the rotation is set on "rotate indefinitely"

2. the time frame for these stats is since the new ads have been created.

3. I am testing simple word changes in the headlines

 

So, here are my questions:

 

1. in the "Yellow" Broad ad group, Ad C is out performing Ad D in terms of CTR (6.31 vs 3.93), but in the "Yellow" Phrase group Ad D is outperforming Ad C (7.67 vs 5.82)

 

Question: How do I know which is the better performing ad when the results are the opposite in the two Ad groups?

 

2. In the "Blue" Broad ad group, Ad A has 142 impressions with a 11.72 CTR and Ad B has 1065 impressions with a 7.98 CTR.  In the "Blue" phrase Ad group,  Ad A has 188 impressions with a 6.91 CTR and Ad B has 1088 impressions with a 6.71 CTR.  The rotation seems very uneven to me.

 

Question: How the heck do I know which is the better performing ad when the distribution is so uneven?

 

I thought I was doing everything right by separating key words into different ad groups by match type, but now I'm really confused. I have looked into this thoroughly and can't find any answers....

 

Can somebody please help me with some advice as to how to interpret these stats and some best practices moving forward?

 

Thank you so much!

 

 

 

 

 

1 Expert replyverified_user

Re: Testing same A/B Ad split in different "Match type" Ad groups

Top Contributor
# 2
Top Contributor

@Kate ;

The issue here is Statistics, not AdWords. You set a multi variable experiment hence the results are  not conclusive.

The test should have been set differently:  a few steps, each of a single variable test, using the drafts and the experiments tool in AdWords
I encourage you to consult a statistician  about the  correct method to  properly set your test.

https://support.google.com/adwords/answer/6318732?hl=en

Moshe, AdWords Top Contributor , Twitter | Linkedin | Community Profile | Ad-Globe
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Best Answer’

Testing same A/B Ad split in different "Match type" Ad groups

Visitor ✭ ✭ ✭
# 3
Visitor ✭ ✭ ✭

True, and thank you for the response.  There are too many variables.  But that doesn't explain the dramatic difference in impressions in the "blue" ad groups - especially considering that it is on "rotate indefinitely".  I'm not sure what that means, why it is happening, and what I can do about it......

Testing same A/B Ad split in different "Match type" Ad groups

Badged Google Partner
# 4
Badged Google Partner

You would hope/think that you should be getting more even impressions between these ads in the same adgroup, over the same period of time, when both ads were active. The only thing that would seem to make sense is that the QS attributes of ad B are so much better that your adrank is higher, allowing you to qualify for more impressions. In terms of clicks alone, the best performing ad would seem to be ad A. If you paused ad B long enough to accrue similar impression volume, then you would have a more accurate data set.

 

I would say ad D is the better performing ad for the Yellow adgroups, simply because it's the best performing ad, in the best performing adgroup. You have so many more variables with broader match types, if they both received equal impressions, then the CTR might be more closely aligned with the phrase match adgroup.

 

The fewer variables the better. Pour over your search terms reports and find the top 5-10 best performing exact match terms, and put them into their own campaign. Then run your A/B ad tests at the campaign level, and you will eliminate some of the variables which could be skewing your data. 

 

Tom

Testing same A/B Ad split in different "Match type" Ad groups

Visitor ✭ ✭ ✭
# 5
Visitor ✭ ✭ ✭

Thank you so much Tom, that was very helpful.  The thing that is most interesting and infuriating about Adwords is that there is always something new to learn!  Thanks again for taking the time to respond.  

Testing same A/B Ad split in different "Match type" Ad groups

Visitor ✭ ✭ ✭
# 6
Visitor ✭ ✭ ✭

I just wanted to share that, after months of trying to figure this out, and lots of online research, I finally figured it out!  Although I had my ad delivery setting as "rotate indefinitely", in my Bid Strategy I had checked the Enhanced CPC option.  That seemed to be affecting the ad rotation.  Since unchecking the box, my ads have been rotating much more evenly.  Hopefully this will help anybody else who may be having a similar issue.