AdWords
5K members online now
5K members online now
Get started with AdWords - learn the basics to get set up for success
Guide Me
star_border
Reply

Complex A/B test in Google AdWords

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Hello

I have a complex test to run. I have 1 product with 1 set of keywords.

I‘d like to try different approaches to see what advert / landing page works better.

In addition to that, I have 2 domains (it was set up like that for technical reasons and this is not changeable at the moment).

 

I have 6 different combinations to set up: (1 product with same set of keywords):

 

Ad 1 – domain A – landingpage A

Ad 1 – domain B – landingpage B

Ad 1 – domain B – landingpage C

 

Ad 2 – domain A – landingpage D

Ad 2 – domain A – landingpage A

Ad 2 – domain B – landingpage C

 

Ideally I should have kept all these combinations in 1 ad group. However, it is impossible because of the 2 domains used. So I split them into 2 ad groups – with domain A and B. But now I understand these ad groups might be competing against each other.

 

What should I do to set it up correctly?

 

Many thanks

2 Expert replyverified_user

Complex A/B test in Google AdWords

Eva Rising Star
Rising Star
# 2
Eva Rising Star
Rising Star

Hi @Jo D

 

Adwords has a feature which is great for A/B testing purposes. It's called campaign drafts and experiments. You may read more on how to set up the experiment here https://support.google.com/adwords/answer/6318732?hl=en&visit_id=1-636228606340380320-1087771207&rd=...

 

 

Regards,
Eva

Complex A/B test in Google AdWords

Visitor ✭ ✭ ✭
# 3
Visitor ✭ ✭ ✭

Thanks for the link, Eva, but i'm still not sure how to set it up. 

 

If I do it via the Campaign Experiments and Drafts, it means i set up one of these combinations as my main campaign (control), and the rest 5 combinations as drafts. However, I need all 6 of these combinations to run at the same time, whereas it clearly states that: "Note that while you can have multiple drafts for a given campaign, only one of those drafts can run as an experiment at a time."

 

Any more advice you could give me, please?

 

Complex A/B test in Google AdWords

Eva Rising Star
Rising Star
# 4
Eva Rising Star
Rising Star

@Jo D

You won't need to create more than one draft, 
So basically, you may have Domain A's landing pages at your original campaign and Domain's B landing pages as a draft/experiment.

For example, your main campaign's ads 

Ad 1 – domain A – landingpage A

Ad 2 – domain A – landingpage D

Ad 2 – domain A – landingpage A

 

And Draft:

Ad 1 – domain B – landingpage B

Ad 1 – domain B – landingpage C

Ad 2 – domain B – landingpage C

 

Then you will know which landing page performs better with the same set of keywords. 

Let me know if this solution works for your purposes.

Regards,
Eva

Complex A/B test in Google AdWords

Collaborator ✭ ☆ ☆
# 5
Collaborator ✭ ☆ ☆

@Jo D @Eva

 

I don't meant to be disparaging, but I think that the new Experiments and Drafts makes this kind of scenario impractical if not impossible.

In the past, we could create an experiment in one campaign on the ad level, and set all all the necessary experiment and control variables there. 

 

Now that we have to do it on the campaign level, it makes tests like this really hard.  Especially if you have multiple ad groups and keywords in the campaign being tested.  The split is happening on the campaign level, which means that the potential for variance on the ad level is huge! There was potential for variance the old way, but it was very small. You could count on that your ads were being split evenly 50/50, or whatever other ratio you set.   With the 50/50 split now occurring on the campaign level, you can no longer count on that.

Another thing to consider is the amount of traffic you have. If you're only targeting one city, you potentially have to wait months if not years before you obtain anything significant.  That length of time is just not practical for testing landing pages.

I do think that this new version does offer many powerful testing options, but specific landing pages is not one of them. I don't know if this is just oversight, or intentional. It could be they made it a priority for testing specific Adwords related items like bids, and targeting methods.

I have actually been looking for 3rd party software that will allow me to do proper landing page tests for Adwords traffic. 

Complex A/B test in Google AdWords

Eva Rising Star
Rising Star
# 6
Eva Rising Star
Rising Star

Hi @David K

Thanks for your reply, and I agree with you that updated experiment is not the best solution for huge accounts.

 

I see the potential problem with updated experiments for campaigns with multiple ad groups, but in this case, we are talking about a single ad group campaign with one set of keywords. The issue with landing pages, as I understand, is a temporary thing (technical issues). So probably, 3rd party software is too advanced for these purposes.

 

Basically, the purpose of an experiment would be to separate domains without creating additional ad groups ( as @Jo D mentioned, he doesn't want to create any more ad groups) and reverse everything back to one of the domains in the future. Which is exactly what drafts and experiments are good for. 

 

 

 

 

Regards,
Eva

Complex A/B test in Google AdWords

Collaborator ✭ ☆ ☆
# 7
Collaborator ✭ ☆ ☆

@Eva

 

sorry,  I missed the part about 1 ad group.  Even with 1 ad group we essentially still have multiple splits going on. You've got the split between campaign and draft, and then the rotation of the 3 ads.  So depending on how many keywords are in the adgroup, I still think there is a potential for things to get skewed. Just not as bad as in the scenario I described above.

 

What are your thoughts on doing these tests one at a time?  I'm thinking that would make for obtaining an answer much quicker than trying to run all three at the same time, and make it easier to ensure a proper split.   

 

 

Re: Complex A/B test in Google AdWords

Top Contributor
# 8
Top Contributor

multi variable statistical experiment is not something I would recommend unless you have an experienced  statistician  by your side... Smiley Surprised  Take one variable per experiment (i.e. one landing page vs another aka:  single variable  test.)  The best would be to use Analytics experiments.

https://support.google.com/analytics/answer/1745152?hl=en

Moshe, AdWords Top Contributor , Twitter | Linkedin | Community Profile | Ad-Globe
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Best Answer’

Complex A/B test in Google AdWords

Eva Rising Star
Rising Star
# 9
Eva Rising Star
Rising Star

@David K

 

I agree with you, it is always a best practice to test one thing at a time. 

 

However, my advice was given based on the information provided: one product/keywords set, 2 ads variations (each of ads has 3 different landing pages on 2 different domains (so 6 in total), one ad group. 

Based on information provided, we have to test all these things together. he specified that:" However, I need all 6 of these combinations to run at the same time".

To make more advanced recommendations I would prefer to take a look at the campaign's structure to see keywords, ads, landing pages,to know what are his objectives behind having this structure and why he prefers to have all these landing pages within one one ad group (which is already not beneficial taking into account ads rotation). 

 

 

 

Regards,
Eva

Complex A/B test in Google AdWords

Collaborator ✭ ☆ ☆
# 10
Collaborator ✭ ☆ ☆

@Eva

 

For the record I was never questioning your advice. Most of my accounts are city specific and a service that has multiple descriptions. Therefore I have at least 5 to 6 ad groups in each campaign. This has been a sore spot for me, and I've been meaning to discuss it here.  That's why I read my situation into the OPs first post and missed the part about 1 ad group.  I just like hearing different perspectives from people in different situations.

@MosheTLV

 

That's what I was thinking. Multi-variate stuff is tough even if you can get the split perfect. Smiley Very Happy

Quick question on the Analytics side of experiments. I looked into that. Am I correct in understanding that you can't apply it to a certain type of traffic, i.e. Adwords?  For a certain market I'm in, it just wouldn't work for organic because most of the traffic is not commercial in nature.  Only our focused Adwords campaigns lets us drive traffic that is 100% commercial intent.