AdWords
1.8K members online now
1.8K members online now
Use AdWords conversion tracking and reporting to measure your results. Have a question about Google Analytics? Post it here, on the Google Analytics Community!
Guide Me
star_border
Reply

Split Testing with different average positions

Follower ✭ ☆ ☆
# 1
Follower ✭ ☆ ☆

I am split testing two ads.  Both are rotating evenly.  The problem is, one has an avg. position of 5.0, while the other has 3.4.  The ad with the 3.4 avg position has a CTR of 3.51%, the ad with the 5.0 position has a CTR of 3.05%.  That makes sense, but doesn't exactly make it easy to split test.

I'm assuming the 5.0 position ad is more relevant to my keywords, and gets a higher quality score.  I can't up the bid of a single ad, so how do I do this?

 

Thanks!

3 Expert replyverified_user
1 ACCEPTED SOLUTION

Accepted Solutions
Marked as Best Answer.
Solution
Accepted by topic author Tommy D
September 2015

Re: Split Testing with different average positions

Top Contributor
# 3
Top Contributor

Hi Tommy,

 

I'm assuming that these two Ads are running in the same Group and that by "split testing" that's all you mean, not that you're actually running a Campaign experiment with one as a control and the other as an experiment; if I'm wrong, please correct me.

 

I'm also assuming that the problem you're describing is that the lower position may itself be the cause of the lower CTR for the Ad in position #5.0, not the copy of the Ad itself.  Again, please correct me if I'm wrong.

 

If the above is true then there's only really one way to do a direct "position to position" comparison and that's to put each Ad into its own Group.  In this way you can effectively increase the bid for just one of the two Ads and use scripts/automated Rules, to adjust the bids so that it matches the average position (over a reasonable time) of the other Ad being tested.  However, I'm not sure you need to do this.  If one Ad has a markedly lower position than another - with a good representative sample of data - I'd argue that this in itself indicates that Ad is a lower "quality" and should be improved.  Since both Ads have the same Max CPC from their Keywords, if one is consistently appearing below the other, this is more than likely because that combination of Ad and Keyword(s) causes a lower quality score.

 

You must, of course, make sure that you have set the Ad Rotation settings to "Rotate evenly" or the Ads won't be entered into the same number of auctions.  You must also ensure that you're looking at a time period when both Ads were active, and that there's no obvious time related issues - for example, reasons why one Ad might appear more relevant or interesting at a certain time of day or day of the week than the other.

 

Jon

AdWords Top Contributor Google+ Profile | Partner Profile | AdWords Audits

View solution in original post

Re: Split Testing with different average positions

Top Contributor
# 2
Top Contributor
Hi Tommy,

Welcome to the Adwords community.

The ad positions will be different for two different ads as the components of their quality score (expected click through rate, ad relevance, and landing page experience) will be different (which influences ad rank and position).

https://support.google.com/adwords/answer/1722122

May be you can use Adwords automated rules to increase your ad positions when it is low but you need to change your bid to do that. So again you are influencing/changing one component in ad rank (position).

A better approch can be finding the most suitable ad to achieve your marketing goals. Find the ad which bring more visits (CTR) or more sales/leads (conversions). Once you find the most profitable ad, you can test different positions by adjusting the bid.

Quality score and average positions are few indicators that can give you some direction in campaign optimization.

Deepak
Marked as Best Answer.
Solution
Accepted by topic author Tommy D
September 2015

Re: Split Testing with different average positions

Top Contributor
# 3
Top Contributor

Hi Tommy,

 

I'm assuming that these two Ads are running in the same Group and that by "split testing" that's all you mean, not that you're actually running a Campaign experiment with one as a control and the other as an experiment; if I'm wrong, please correct me.

 

I'm also assuming that the problem you're describing is that the lower position may itself be the cause of the lower CTR for the Ad in position #5.0, not the copy of the Ad itself.  Again, please correct me if I'm wrong.

 

If the above is true then there's only really one way to do a direct "position to position" comparison and that's to put each Ad into its own Group.  In this way you can effectively increase the bid for just one of the two Ads and use scripts/automated Rules, to adjust the bids so that it matches the average position (over a reasonable time) of the other Ad being tested.  However, I'm not sure you need to do this.  If one Ad has a markedly lower position than another - with a good representative sample of data - I'd argue that this in itself indicates that Ad is a lower "quality" and should be improved.  Since both Ads have the same Max CPC from their Keywords, if one is consistently appearing below the other, this is more than likely because that combination of Ad and Keyword(s) causes a lower quality score.

 

You must, of course, make sure that you have set the Ad Rotation settings to "Rotate evenly" or the Ads won't be entered into the same number of auctions.  You must also ensure that you're looking at a time period when both Ads were active, and that there's no obvious time related issues - for example, reasons why one Ad might appear more relevant or interesting at a certain time of day or day of the week than the other.

 

Jon

AdWords Top Contributor Google+ Profile | Partner Profile | AdWords Audits

Re: Split Testing with different average positions

Top Contributor
# 4
Top Contributor

If this is the case (of significant gaps in the ad positions) -  I would run a campaign experiment and let the system do the complex statistics and decide whether one ad-copy is statistically significant performing  better over another ad-copy.

All other methods / techniques  could be  statistically biased.

Moshe, AdWords Top Contributor , Twitter | Linkedin | Community Profile | Ad-Globe
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Best Answer’

Re: Split Testing with different average positions

Follower ✭ ☆ ☆
# 5
Follower ✭ ☆ ☆
Thanks Cobnut. All your assumptions were correct, and your solution sounds like a great idea.

I also thought about the difference in positions, as you said, and thought the same thing. But, if I get two ads that are markedly different in position, but still close in CTR, wouldn't it stand to reason that the one that Google scores as lower quality is actually a better ad? For it to be so much further down in position yet still maintain a CTR close to the other ad. A bummer that Google algorithm gets to determine that.

Re: Split Testing with different average positions

Top Contributor
# 6
Top Contributor

Hi Tommy, I did wonder what your basis was for thinking the Ad at #5.0 was better Smiley Wink

 

Yes, maybe, no (not necessarily in that order).  Generally speaking higher positions lead to higher CTR, but there aren't really any hard and fast rules that say this is always true, or if it is, by what percentage each position "jumps".  One of the biggest problems in trying to analyse this in detail is that CTR, position and Quality Score are all averages (Quality Score is actually more of an "indication" than an average, but the difference is semantic for this explanation) and depending upon your exact circumstances, could be averages including quite extreme individual values and then there's location and time variations to consider.  If Ad A has an average position of 3 and Ad B one of 5, that doesn't mean Ad A is never in position #6 and Ad B never in position #2, it's just their averages and it may be that Ad B is frequently in position #2 in Nevada and Ad A frequently in position #6 in Florida.  You get the idea.

 

Positions #3.4 and #5.0 are really quite close when you consider them as averages in a range that can go from 1.0 right down to 12.0 so maybe a difference in CTR of just 0.46% is about right - especially if both Ads tend to appear on the right frequently - it's virtually impossible to say.  (BTW, you may want to look at your Top vs Other stats to see if that sheds any light - Ads tab, Segment, Top vs Other.)

 

What sort of data sample are you looking at?   100s or 1000s of clicks?  Do you target a large region?

 

Jon

AdWords Top Contributor Google+ Profile | Partner Profile | AdWords Audits

Re: Split Testing with different average positions

Top Contributor
# 7
Top Contributor
Hi Jon,

Just asking this to have bit more clarity on the suggestion "put each Ad into its own Group".

We can put the ads in to different ad groups, but what about the keywords? If both the ad groups have same keywords then the keyword/ad/landing page combination with the highest quality score/ad rank will always win the ad auction. And if the keywords are different then the position comparison is meaningless.

Re: Split Testing with different average positions

Top Contributor
# 8
Top Contributor

Hi Deepak, putting the Ads into their own Group removes any rotation or optimisation "interference" in the delivery of the Ads.  If both Groups have the same Keywords, same CPC, then the only difference will be the "quality" of the Ad itself, so if there continues to be a difference in the stats, that can only really be down to the Ad, not any optimisation or rotation (provided the experiment runs long enough to gather a reasonable data sample).


Alternatively, if the CPCs are increased for the Ad that was in the lower position, such that it attains the same position as the other Ad, the stats might then tell us something else, for example, the CTR might increase, which might increase QS and allow a lower actual CPC.

 

The primary idea though is simply to remove the somewhat unknown element of rotation, optimisation and random choice of Ad from the experiment.

 

J

AdWords Top Contributor Google+ Profile | Partner Profile | AdWords Audits

Re: Split Testing with different average positions

Top Contributor
# 9
Top Contributor