Optimize
4.5K members online now
4.5K members online now
Learn how to create your first experiment and get tips on creating A/B, redirect, and multivariate tests.
Guide Me
star_border
Reply

Google Optimize A/B Tests Flip-Flopping?

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Hi there-

 

I've been working on implementing Google Optimize on our site for some time now, and while there wasn't any trouble getting it to present both A and B variants on the site, I'm having trouble getting it to stick to just one for a given user.  This may be "as intended" but our analytics guy tells me it should be one view to a given user forever.

 

To clarify: I can load up the target page and have it appear as Variant A, then press ctrl+f5 a couple times and have it appear as variant B.  Then wait a couple minutes and refresh, you'll be back on Variant A.

 

Is it possible we're doing something wrong?  A Consultant told us the speed at which the site is loading could be playing a role, but the decision makers have been reluctant to take action on that.  In case this is important - our site does load through third party services such ad Hybris and DDX.  I'm including the code snippet for both Google Analytics and Google Optimize through DDX, though it's not been an issue for Google Analytics.

 

Any insights would be appreciated - thanks!

1 Expert replyverified_user
Marked as Best Answer.
Solution
Accepted by Jason O (Community Manager)
May 2017

Google Optimize A/B Tests Flip-Flopping?

Google Employee
# 2
Google Employee

Hi

 

I would be surprised if a user flips between 2 real different variants of the experiment - what is more likely is that the user maybe sometimes excluded from the experiment.

 

For example if you have in an experiment:

- The original variant

- Variant B

- Variant C

 

I would be very surprised if a user would see either B or C by reloading (your analytics consultant is right on that). If that happens then there must be a problem with cookies on your page (somehow cookies are removed or you run out of cookie space - i.e. you have some other very large cookies).

You may troubleshoot by using the Application tab in Chrome dev tools and inspect the gaexp cookie value - it shouldn't change.

 

The speed of loading Optimize may matter a lot more that Google Analytics though - your consultant is also right on that.

 

The reason is that if you make visual changes on static elements in the page (which is the common case), you don't want your users to see the original version of your page before Optimize has a chance to load and change it (flickering). This is controlled by the page hiding snippet.

 

So you may consider increasing the timeout there, this may solve your problems. have a look at:

https://developers.google.com/optimize/

 

What maybe happening is that because of loading through DDX, Optimize may take longer to load  and your users may be sometimes excluded from the experiment.

This is different than viewing the original variant  since in this case your users will not be considered as included in experiment traffic.

 

In general it would be great if you could find a solution to load Optimize earlier (i.e. inline in the top of head).

 

Another option - if for some reason you don't care about flickering (perhaps the effects of your experiments are taking place a lot later), is to not use the page hiding snippet.

 

 

 

 

Google Optimize A/B Tests Flip-Flopping?

Visitor ✭ ✭ ✭
# 3
Visitor ✭ ✭ ✭

Hi there-

 

I appreciate the in-depth reply.  Fortunately - or perhaps unfortunately - things are working as intended now, but it was by way of one of those "it just started working" situations.  A minor change was made to the code that should have had little or no impact on functionality, but here we stand.  Should things go awry I'm glad I'll have this to reference, so thank you!

Google Optimize A/B Tests Flip-Flopping?

Visitor ✭ ✭ ✭
# 4
Visitor ✭ ✭ ✭

Hi there,

 

Is the page redirected before it loads, while redirecting test with google optimize?