Optimize
2.4K members online now
2.4K members online now
Understand the information in your Optimize reports and in your Google Analytics reports.
Guide Me
star_border
Reply

Why does my A/A Test have a winner?

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

I'm currently running a Test with a Website and A/A/A Testing (3 times the same website) Goals like Session Duration or Bounces since there are no conversions on this Site.

 

For some reason with every goal the original has much higher probability to win than the other two exact same variations. What's the reason for this Result? Experiment ist still running but I have over 1500 Samples so it should be somewhat similar?

Why does my A/A Test have a winner?

Visitor ✭ ✭ ✭
# 2
Visitor ✭ ✭ ✭

I see the same problems with my test, I set up an A/A/B test, I expected the var A would be very close to the original. For some reason, the variant A shows about 10% lower in session duration and 10% worse in bounces than Orginal. I understand that there is chance small variance between tests but we have over 700,000 sessions collected over 20 days, and almost all 20 days shows a similar pattern (i.e. Var A worse than Original by about 10%). it is a sizable sample that should eliminate any random variance. Any idea why?

Re: Why does my A/A Test have a winner?

Follower ✭ ☆ ☆
# 3
Follower ✭ ☆ ☆

Seeing similarly confusing results. 

 

Set up essentially an A/A test (change only noticeable if you're looking for it) to confirm the optimization implementation was working correctly. 

 

Results would imply that heading text alignment on a interior content page massively influences purchasing decision? I'm skeptical..
RevenueRevenue

 

 

Session durationSession duration

Anyone can lend some insight into these results? 

Why does my A/A Test have a winner?

Visitor ✭ ✭ ✭
# 4
Visitor ✭ ✭ ✭

Can you tell us the number conversions you actually had? If it's low the result could be because of random variance. However Google shouldn't point out a winner then.

 

Did you have some special offers lately or maybe a couple that spent an extraordinary sum?