AdWords
1.6K members online now
1.6K members online now
Improve your AdWords performance and boost your ROI, CTR, and Quality Score
Guide Me
star_border
Reply

Campaign Draft Experiment - Not Ideal For Large Account?

[ Edited ]
Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Hi Everyone,

I have a question regarding utilizing Campaign Experiment to conduct testing of new ad copies, KW bid changes, and adding new KWs.

Our account is fairly big, with each campaign encompassing over 5 ad groups, sometimes as many as 15.

With so many ad groups, we'd like to use Campaign Experiment to test new ad copies across all of these ad groups simultaneously and launch them in the shortest time possible.

When we optimize, we optimize on an ad group to ad group basis -- meaning we optimize everything within an ad group before moving on to the next ad group (ie. making new ad copies, KW bids, adding new KWs, etc) rather than focusing on specific "category" (ie. setting up an experiment for test ad copies across all ad groups).

But Google's best practice is to test one variable at a time to determine its effect on the ad group. If we want to test different ad copies, KW changes, even geo bid modifiers at the same time, what would be the best method for us while doing a 50/50 traffic split? With how big our campaigns are, it would take forever for us to wait for ad copy test to finish before conducting KW change experiments.

Thank You

1 Expert replyverified_user

Re: Campaign Draft Experiment - Not Ideal For Large Account?

Top Contributor
# 2
Top Contributor

@Peter H;

>>"But Google's best practice is to test one variable at a time to determine its effect on the ad group"

Can you provide a reference?

Indeed, a single variable test is easier to set, monitor and conclude if  the effect of a single change is statistically significant.

But, technically, in both  the old setting of campaign experiments and the in  new version of campaign drafts you can test several variables in a given experiment.

>>"The experiment shares your original campaign’s traffic (and budget) and lets you test changes to your campaign so you can make more informed decisions on which tactics give you a better return on investment.

https://support.google.com/adwords/answer/6261395

Moshe, AdWords Top Contributor , Twitter | Linkedin | Community Profile | Ad-Globe
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Best Answer’

Campaign Draft Experiment - Not Ideal For Large Account?

Visitor ✭ ✭ ✭
# 3
Visitor ✭ ✭ ✭

Hi @MosheTLV,

Let's say you'd like to optimize a campaign with multiple ad groups and ad copies. You launch the experiment on 1/9/2017 with new ads for ad group #1. A few days later, you add test ads for additional ad groups. Then a few days later, you add KW bid changes.

 

Since the variables being changed didn't start on the initial experiment date 1/9/2017, wouldn't the data in the Experiment group in the scoreboard be inaccurate?

 

Meaning... Google might dictate there's a statistical significance as a result of the FIRST ad copy test, but not the last KW bid change. How might this happen? It could be that the ad group with the KW bid changes don't get as much impression to make an impact on the resulting data.

 

 

 

Campaign Draft Experiment - Not Ideal For Large Account?

Top Contributor
# 4
Top Contributor

Hello again @Peter H

Google does not offer this option as a test because statistically, this is not the approach to take.  Changes in variables must be kept constant during the test (this is  statistics - nothing to do with Google)

Moshe, AdWords Top Contributor , Twitter | Linkedin | Community Profile | Ad-Globe
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Best Answer’

Campaign Draft Experiment - Not Ideal For Large Account?

Visitor ✭ ✭ ✭
# 5
Visitor ✭ ✭ ✭

How would you approach this case then if you had a campaign with multiple ad groups, and 3 ad copies within individual ad groups?

 

It'd take forever for each experiment to end to start another one, if we were to keep the variables constant.

 

Moreover, during the ad copy test, everything within that ad group cannot be changed. So we can't optimize KWs, change geo bids, RLSA bids, etc. 

 

Thank You Smiley Happy 

Campaign Draft Experiment - Not Ideal For Large Account?

Top Contributor
# 6
Top Contributor

Well... Statistical testing could be  a routine  long process. This is an intrinsic part of the statistics.  A simple (but not a "pure" statistical) approach would be to rotate  the ads evenly  and check which one gets a better CTR. In a short time, you will be able to see which ad-copy performs better. (Though this  would not necessarily be a  statistically significant conclusion).

Moshe, AdWords Top Contributor , Twitter | Linkedin | Community Profile | Ad-Globe
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Best Answer’

Campaign Draft Experiment - Not Ideal For Large Account?

Visitor ✭ ✭ ✭
# 7
Visitor ✭ ✭ ✭

Unfortunately, it wouldn't as the rotation setting is on the campaign level, which would affect other ad groups not yet in the ad copy test.

 

However, I did think about this longer, and perhaps we can conduct "mini experiments" within the main Campaign Experiment. And for experiment duration of Campaign Experiment, we would basically set this for however long it'd take for us to conduct a round of ad copy tests within a specific campaign. That way, we can apply the changes to the existing campaign.

 

"Mini experiments" can be achieved because the control campaign and the experiment campaign are mutually exclusive, meaning as long as the changes we made in the control campaign are also mirrored in the experiment campaign immediately, then these changes would not be an issue in the experiment -- we would be treating both groups with the same changes.

 

For measuring, we don't have to rely on the campaign experiment scoreboard to tell us how the experiment performed. We can take the metrics from the ad copies themselves and put them in a statistically significance calculator to determine the winner. 

 

What are your thoughts on this?

Campaign Draft Experiment - Not Ideal For Large Account?

Top Contributor
# 8
Top Contributor

You are right. You want to test at the ad-group level.

Let me think on that "mini experiment". I'll get back to you.

Moshe, AdWords Top Contributor , Twitter | Linkedin | Community Profile | Ad-Globe
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Best Answer’

Campaign Draft Experiment - Not Ideal For Large Account?

Top Contributor
# 9
Top Contributor

Not sure how exactly you would set the mini experiment. Once an experiment starts  you can't make any changes in the test group without a statistical  bias in /to  the experiment itself.

Moshe, AdWords Top Contributor , Twitter | Linkedin | Community Profile | Ad-Globe
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Best Answer’

Campaign Draft Experiment - Not Ideal For Large Account?

[ Edited ]
Visitor ✭ ✭ ✭
# 10
Visitor ✭ ✭ ✭

Hi MosheTLV, 

 

We would ignore the statistical data gathered in the campaign scoreboard during the experiment; that data would only be valid if we set up the experiment and not change anything within the experiment campaign. As you mentioned, any changes done after the experiment starts would skew the data. We would use external statistical significance calculators to find winner/loser.

 

The way mini-experiments are set up is actually a lot simpler than it sounds. And we can start the mini-experiment any time regardless of whether the experiment campaign has been running for one day or 200 days. 

 

Let's say we have a Desktop Monitors campaign set up (e-commerce company). Within this campaign, we have multiple ad groups for different products (LED monitors, CRT monitors, etc).

 

For the Experiment Campaign, I want to test new ad copies within LED Monitors ad group. I create Campaign Draft and write new ad copies here. Since I am only testing new ads within LED Monitors ad group, I would pause everything else I am not testing. I then launch it as an experiment with a test duration of one month. 

 

Let's say a few days later, I want to test new ads for CRT Monitors ad group. I would go into the Experiment Campaign and enable CRT Monitors ad group, pause existing ad copies within that group, and write test ad copies there.

 

Another important thing to note is that both Original Campaign and Experiment Campaign needs to have the same changes applied if you make changes that are not part of the experiment. For example, during this ad copy experiment, you might harvest more KWs from your Search Query Report and add them to your campaign. When you do this in the Original Campaign, make sure to also add the KW to your Experiment Campaign. If you change KW bids, make sure to also change it in the Experiment Campaign as well.

 

By doing this, even if KW changes did affect the CTR in the Original Campaign, at least we know that this treatment is also being applied to our ad copy in Experiment Campaign. 

 

This may not be 100% accurate because we wouldn't know the effects of KW changes on our control and test ad copies. In an ideal scenario, or for small accounts, we would just create ad copies and not touch/change anything while the experiment is running. However, for big accounts (imagine Best Buy), time is money, and they would despise Campaign Experiment. But then again, I'm certain they have internal tools to do testing.

 

Let me know what your thoughts are on this approach. Not the best, but probably second to best.