Optimize
1.6K members online now
1.6K members online now
Learn how to create your first experiment and get tips on creating A/B, redirect, and multivariate tests.
Guide Me
star_border
Reply

Simultaneous A/B testing on the same page

Follower ✭ ☆ ☆
# 1
Follower ✭ ☆ ☆

What happens when you run two different A/B tests on the same page at the same time?

 

I see three possible outcomes:

A. Google Optimize knows how to measure the conversions properly (treating it as a multivariate test)

B. A different experiment will be chosen randomly for each visitor.

C. Both experiments will run at the same time, and the results will be effected, which means they're invalid.

 

Any reference to the proper documentation will be appreciated.

1 Expert replyverified_user

Simultaneous A/B testing on the same page

Follower ✭ ✭ ✭
# 2
Follower ✭ ✭ ✭

Just one question:  Why do you want to run two on the same page?

Simultaneous A/B testing on the same page

Follower ✭ ☆ ☆
# 3
Follower ✭ ☆ ☆

I already have an A/B test running for a couple of weeks, and I would like to test another change without losing the data already collected by the first one.

Simultaneous A/B testing on the same page

Visitor ✭ ✭ ✭
# 4
Visitor ✭ ✭ ✭

I have the same question. 

Simultaneous A/B testing on the same page

[ Edited ]
Visitor ✭ ✭ ✭
# 5
Visitor ✭ ✭ ✭

I have the same question for how Google Optimize handles this - does it cookie users so that they are only bucketed into ONE test at a time? No matter what page the test is on? Can users have a cookie per test so that they are seeing the same variation of any test that they encounter, but they might be in more than one test? This is important because I want to know if running tests simultaneously is actually going to require MORE traffic than running one at a time.

 

I found an Optimizely Q/A that is on the same/similar topic here if anyone finds it helpful.

https://community.optimizely.com/t5/Using-Optimizely/Simultaneous-Testing-Running-two-different-test...

Simultaneous A/B testing on the same page

Visitor ✭ ✭ ✭
# 6
Visitor ✭ ✭ ✭

I do Server-side experiments and in my experience Google Optimize will only receive hits for the last experiment field set.

 

If I do:

 

ga("set", "exp", "exp-osXAFQQ6zMKDI4guivw.1")

ga("set", "exp", "exp-mxjSjm2e-FZqxIOzrA.1")

ga("send", "pageview")

 

I can see with Analytics Debugger Chrome plugin the "pageview" event wraps the last experiment only:

 

...

exp              (&exp)  exp-mxjSjm2e-FZqxIOzrA.1

hitType          (&t)    pageview

...

 

This is consistent with the results I'm getting in the reporting page and could be evidence of Google Optimize not supporting funnel tests atm: https://www.en.advertisercommunity.com/t5/forums/v3_1/forumtopicpage/board-id/optimize_experiments/m...

Simultaneous A/B testing on the same page

Google Employee
# 7
Google Employee

Hi,

 

You need to combine the two experiments into a single "exp" parameter separated with '!' character like:

 

ga("set", "exp", "exp-osXAFQQ6zMKDI4guivw.1!exp-mxjSjm2e-FZqxIOzrA.1")