Analytics
5.2K members online now
Understand information in your reports and troubleshoot reporting issues such as self-referrals, (not set) data, and inaccurate information
 
Guide Me
star_border
Reply

Contrudictive results on experiment results

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Hi there,

 

We are using Google Experiments heavily and recently seen an experiment that ended with the winner having lower score than other variations -- see the attached screenshot. Anyone else seen that? What does this mean?

 

Thanks for the help!

Mike.

 

1 Expert replyverified_user

Re: Contrudictive results on experiment results

Visitor ✭ ✭ ✭
# 2
Visitor ✭ ✭ ✭
Anyone knows why this is happening?

Re: Contrudictive results on experiment results

[ Edited ]
Rising Star
# 3
Rising Star

jFrost I wish I did but I get the same results in my experiments the numbers show that one or more variations work better but the final analysis by GA says the Origional is the best solution to stick with. This is one of many features inside of GA that I use other methods to determine outside of GA.

Re: Contrudictive results on experiment results

Visitor ✭ ✭ ✭
# 4
Visitor ✭ ✭ ✭
This is a small test, it happens with large scale tests as well (500K+ page views). It makes me feel analytics data is unreliable, at least in terms of the tests...

I would be more than grateful to hear and official google reply to this.

Thanks for the help!
Mickey.

Re: Contrudictive results on experiment results

Visitor ✭ ✭ ✭
# 5
Visitor ✭ ✭ ✭
Maybe if you can star my question it will be more noticeable?

Re: Contrudictive results on experiment results

[ Edited ]
Visitor ✭ ✭ ✭
# 6
Visitor ✭ ✭ ✭

This is extremely frustrating :-(
We just got another such result with a 250K samples data set.

Does anyone knows what does it mean? Can it be related to higher variance of one of the variations? If this is the case, why does the test stop?

 

Re: Contrudictive results on experiment results

[ Edited ]
Visitor ✭ ✭ ✭
# 7
Visitor ✭ ✭ ✭

Did anyone find an answer for this one?