Optimize
5.1K members online now
5.1K members online now
Understand the information in your Optimize reports and in your Google Analytics reports.
Guide Me
star_border
Reply

Changing Weight of Variants While a Test is Running

Follower ✭ ✭ ✭
# 1
Follower ✭ ✭ ✭

While running an A/B/C experiment, the weight of the variants were changed. Originally, we started at 90%/5%/5% then we changed this the next day to 70%/15%/15%. When we compare the data in Google Analytics on a click event goal to the number of conversions for that goal in Google Optimize, there is a difference of a few hundred between Analytics and Optimize.

 

What could possibly be causing this difference? We've tested the click event itself, and it is definitely firing into our dataLayer, and we see it as a real time goal. We also don't expect a simple copy change to drop CTR by 30%, seems like something in reporting is way off.

Changing Weight of Variants While a Test is Running

Follower ✭ ✭ ✭
# 2
Follower ✭ ✭ ✭

Addressing my own post. The reason there is a difference in reporting between Optimize and Analytics is because all metrics in Optimize are first processed by Analytics then pushed to Optimize. The push to Optimize can take up to 12 hours which would eliminate the approach to reconcile by filtering for earlier dates since there will always be this difference in reporting. When the experiment ends, data accrued between the last push to Optimize and us ending the experiment will not be pushed to Optimize, but will be available in Analytics. Bottom line is that there will always be a difference between the data found in Optimize versus data found in Analytics. Google recommends to use Optimize results to decide when to end an experiment and which variant is the leader.
Reference: http://bit.ly/2rovFOZ

 

Still haven't found any information on changing variant weight while an experiment is running so would appreciate any insight on that.