AdWords is now Google Ads. Our new name reflects the full range of advertising options we offer across Search, Display, YouTube, and more. Learn more

Ads
1.5K members online now
1.5K members online now
Improve your Google Ads performance and boost your ROI, CTR, and Quality Score
Guide Me
star_border
Reply

When to make changes? (statistical significance vs. gut feeling)

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Hello, 

I have been searching a lot for this answer without much success, so I hope you guys can help. 

 

Basically I'm often not sure exactly when to make changes to my account. For instance how long should i test two ads to know which one to pause? In theory i probably should have like a thousand impressions to conclude anything, but we all know that this doesn't fly in the real world - especially if you are running campaigns with smaller budgets and where the focus is on the less competitive keywords. It simply takes too long for them to acquire enough data. My campaigns are profitable, so what Im doing is definitely working, but I feel that its a lot based on a hunch. I probably should mention that my goal is always conversions - never branding. 

 

The answer I always get is "it depends". But I'm sure some of you have some kind of rule of thumb that ses the frame for your work?

 

Im interested in knowing what you do for ads and keywords. And when I say changes I mean both minor changes like splittesting ads, to the larger onces - for instance when I decide to give up on a keyword because I assume that its never going to be profitable. The questions is always opportunity cost. In theory i should let a test run as long as possible, but sometimes it seems to be better to pause a keyword and invest the money on another keyword even though I have only accrued a fourth of the data I need to have something conclusive to say. 

 

I know theres significance calculators out there, but Im interested in knowing what you guys do?

 

Thanks a lot in advance!

Mads

2 Expert replyverified_user
1 ACCEPTED SOLUTION

Accepted Solutions
Marked as Best Answer.
Solution
Accepted by topic author Mads E
June 2016

Re: When to make changes? (statistical significance vs. gut feeling)

[ Edited ]
Top Contributor
# 7
Top Contributor

I don't have any secrets except where I am under NDA which is quite often the case. Experience accrued during many years and some good eyesight combined with reading a lot from the right sources may of course help you develop techniques and a holistic "feeling" of how you should progress and go ahead. The particular techniques are usually termed and discussed as Best Practices in this forum and most publications.

Please let me make a personal note that when I was a student I used to be national finalist in math and much later on I ,of course, expected Adwords at the advertisers' end to be manageable based on mathematical stats. It isn't, as it was said above. Now as for going thru excels: that might be a daily routine when I smell money or rats. Less important when it is business as usual. Averages are mostly okay in the AdWords interface, therefore when I need excel or pivots in particular, I typically look for outliers. Understanding them often sheds light on issues. (Another good use of excel is to look for standard deviation when need be. That's something really important completely missed by AdWords tables.)

Let me also give you a sort of example, this time in relation to experiments: Google algos are based on math therefore lots of times you simply don't have the time and funds to let it run till they reach a conclusion. Your best and most profitable decisions will be made when you can already see a tendency in the stats displayed and simultaneously you also find a sort of logical explanation for what you see. Every one makes mistakes and bad decisions, therefore I'd advise you this way: don't get frightened by erring, it's part of the learning curve. What you should care for is the balance.

Best,
Lakatos

View solution in original post

Re: When to make changes? (statistical significance vs. gut feeling)

Top Contributor
# 2
Top Contributor
Hi Mads,

I have been here in the forum since 2007 and met a lot of people who supposed hat a scientific approach to mastering AdWords could be appropriate. The underlying math knowledge is of course associated with confidence intervals and the theory of probability. In a nutshell: based on my experience that is the wrong approach, firstly because of the reasons that you also mentioned above. Hence for us, advertisers, AdWords is more of an art than science and it will remain so for many years to come. Hence an combination of statistical significance and your educated gut feelings seems to be the best attitude.

However, Google algos like experiments are based on math.

Best,
Lakatos

Re: When to make changes? (statistical significance vs. gut feeling)

Top Contributor
# 3
Top Contributor

Mads,
Well... If you want a data driven decision is AdWords use AdWords experiments, or any 3rd party tools. In AdWords experiments that system would not end the test until the result is statistically significant (but as you mentioned that could take a while... )
There are holistic rules, that every advertiser has adapted over times. But, though "gut feelings" / instincts do play a role, I, personally, prefer a decision based on data analysis.
Moshe, AdWords Top Contributor , Twitter | Linkedin | Community Profile | Ad-Globe
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Best Answer’

Re: When to make changes? (statistical significance vs. gut feeling)

Top Contributor
# 4
Top Contributor
Yeah, "holistic rules" sounds much better than "educated gut feelings".

Re: When to make changes? (statistical significance vs. gut feeling)

Visitor ✭ ✭ ✭
# 5
Visitor ✭ ✭ ✭
@MosheTLV @Lakatos Without giving too much of of your secrets away, can you tell a little about how you exactly do? Like do you run through the numbers in a excel sheet or use a script? Or do you just look at the numbers and say act upon what you can tell is fairly statistical significant? Also, do you have any specific rules that guide your actions? For instance "when two ads have had at least 100 impressions then I will pause the loosing one".

Re: When to make changes? (statistical significance vs. gut feeling)

Top Contributor
# 6
Top Contributor
>>>""when two ads have had at least 100 impressions then I will pause the loosing one".
100 impressions is NOT a big enough statistical sampling to make a (even an estimated) decision.
Moshe, AdWords Top Contributor , Twitter | Linkedin | Community Profile | Ad-Globe
Did you find any helpful responses or answers to your query? If yes, please mark it as the ‘Best Answer’
Marked as Best Answer.
Solution
Accepted by topic author Mads E
June 2016

Re: When to make changes? (statistical significance vs. gut feeling)

[ Edited ]
Top Contributor
# 7
Top Contributor

I don't have any secrets except where I am under NDA which is quite often the case. Experience accrued during many years and some good eyesight combined with reading a lot from the right sources may of course help you develop techniques and a holistic "feeling" of how you should progress and go ahead. The particular techniques are usually termed and discussed as Best Practices in this forum and most publications.

Please let me make a personal note that when I was a student I used to be national finalist in math and much later on I ,of course, expected Adwords at the advertisers' end to be manageable based on mathematical stats. It isn't, as it was said above. Now as for going thru excels: that might be a daily routine when I smell money or rats. Less important when it is business as usual. Averages are mostly okay in the AdWords interface, therefore when I need excel or pivots in particular, I typically look for outliers. Understanding them often sheds light on issues. (Another good use of excel is to look for standard deviation when need be. That's something really important completely missed by AdWords tables.)

Let me also give you a sort of example, this time in relation to experiments: Google algos are based on math therefore lots of times you simply don't have the time and funds to let it run till they reach a conclusion. Your best and most profitable decisions will be made when you can already see a tendency in the stats displayed and simultaneously you also find a sort of logical explanation for what you see. Every one makes mistakes and bad decisions, therefore I'd advise you this way: don't get frightened by erring, it's part of the learning curve. What you should care for is the balance.

Best,
Lakatos

Re: When to make changes? (statistical significance vs. gut feeling)

Visitor ✭ ✭ ✭
# 8
Visitor ✭ ✭ ✭
OK. So what Im hearing both of you say is that I have to make my own experiences and test what works for me. I was hoping for some firmer advices that I could get inspiration from - like "if y happens then generally i do x". But I understand that its hard to say something on a general level as it depends. Thanks for helping outSmiley Happy!

Re: When to make changes? (statistical significance vs. gut feeling)

Top Contributor
# 9
Top Contributor
If you make a search for the term Best Practices in this forum, you'll find a pretty good number of techniques with a more specific approach to really specific problems.

Re: When to make changes? (statistical significance vs. gut feeling)

Visitor ✭ ✭ ✭
# 10
Visitor ✭ ✭ ✭
Thanks. I did try that before, but I will try again.