Optimize
1.6K members online now
1.6K members online now
Discover how to select who is eligible to be in your experiment, and when they’re served experiment variations.
Guide Me
star_border
Reply

Parallel Experiments: Does Optimize exclude Users between different Experiements?

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Hi!

 

We are currently preparing several A/B-test experiments. During preparation somebody asked, if Optimize excludes users of experiment A from experiment B, which we could not answer even after some research. We considered, that there might be some effect on the outcome of experiment B if the user was included in experiment A during the same time. Especially when both experiments run in similar areas of the website and users are more likely to come across both experiments.

 

If Optimize dosen't handle this issue: Creating audiences in GA for each experiment, publishing it to Optimize, and applying a negative targeting between experiments is probably the way to go. Can someone confirm this method?

 

Any help is appreciated!

 

Best,

Christian

 

 

1 Expert replyverified_user

Parallel Experiments: Does Optimize exclude Users between different Experiements?

Visitor ✭ ✭ ✭
# 2
Visitor ✭ ✭ ✭

Hi Christian

 

I have the exact same question... strange to see no one is answering you here. Have you figured it out since your post in June and if so, can you please shareSmiley Happy?

 

Best regards

Christian Evensen

Fjellsport.no

Parallel Experiments: Does Optimize exclude Users between different Experiements?

Visitor ✭ ✭ ✭
# 3
Visitor ✭ ✭ ✭

Hi,

 

I have the same question too! Other A/B tests competitors provide a way to achieve this (exclude visitors of a running campaign from another and vice versa) but I couldn't find a direct way to do it with Optimize.

Thanks,
Jeet

Parallel Experiments: Does Optimize exclude Users between different Experiements?

Visitor ✭ ✭ ✭
# 4
Visitor ✭ ✭ ✭

Hi, 

I have asked the Google Support today because I had the same problem. Here is their answer:

 

I wanted to share with you that, a user can be a part of only one experiment at a time. If a user is a part of X experiment, the he will be the part of same experiment, until he clears the cache and cookies of his machine and a become a part of new experiment.

 

Bests,

Denise

Parallel Experiments: Does Optimize exclude Users between different Experiements?

Google Employee
# 5
Google Employee

Hi

 

Optimize experiments are independent so a user in experiment A may also be included in experiment B.

 

While there is no support yet for "reserving" traffic for an experiment, there is an easy way to implement this using a "custom javascript" variable based on the Google Analytics client ID.

 

This article has more details:

 

https://www.en.advertisercommunity.com/t5/Google-Optimize-Target/Prevent-overlapping-between-2-Exper...

Parallel Experiments: Does Optimize exclude Users between different Experiements?

Visitor ✭ ✭ ✭
# 6
Visitor ✭ ✭ ✭

Hi Dimitri,

 

thank you very much for your reply. I think I will talk to the support member who gave me the incorrect anwer.

 

Bests,

Denise

Parallel Experiments: Does Optimize exclude Users between different Experiements?

Google Employee
# 7
Google Employee

Hi,

 

Just to make sure that there is no misunderstanding with the exact behavior though:

 

If a user enters experiment A and is assigned say to Variant 1, he/she will remain as part of the experiment and will keep seeing the same Variant 1, each time that visits the page under test and the rules are met (will never see "Original" or "Variant 2").

 

This will be the case until the experiment ends (or the user clears cookies etc).

 

There is nothing stoping that user though from entering a second experiment B in another page or even the same page. That user may be assigned to any variant or the original of the second experiment.

 

 

Parallel Experiments: Does Optimize exclude Users between different Experiements?

Visitor ✭ ✭ ✭
# 8
Visitor ✭ ✭ ✭

Hi Dimitris,

 

thank you very much, but I was aware that a user only sees one variant.

 

Because of this I asked for the experiment. And sorry but I am a little bit disturbed that I got a wrong answer on friday and started a experiment today that should have had a different targeting.

 

By the way: Do you have an idea how to exclude the traffic like in your example above with the dataLayer variable in the situation when there is already another experiment runnign that is not targeted this way? Thank you very much for your competend support. I whish I such a good feedback back on friday.

 

Best regards,

Denise

Parallel Experiments: Does Optimize exclude Users between different Experiements?

Google Employee
# 9
Google Employee

Hi

 

I am not sure that I have understood exactly the situation:

 

A) If you have an existing experiment A with a targeting rule

"dataLayer var A" equals "value",

then you could add a rule to all your new experiments like  

"dataLayer var A" "not equals" "value"

 

In addition you could use the "bucketing" custom javascript variable describe in the article above, to reserve parts of your remaining traffic for each new experiment.


Keep in mind though that if your variable for a user was matched in the past (i.e. "dataLayer var A" was "value"), and the user was included in the experiment, then he will stay in that even if later "dataLayer var A" does not equal to "value".

 

This doesn't mean that he/she will see the variant though; the rules in order to display the variant are evaluated each time. But any conversions will be still attributed to the experiment.

 

For example:

- You have a variant that is shown for logged in users.

- A user is logged in and sees a variant but then logs out

- Next time the user comes to the site, he is not logged in and doesn't see the variant

- Still the user converts. This conversion will be attributed to the fact that he has seen the variant in the past (when was logged on) and will be measured in the experiment.

 

B) If what you mean is not related to existing targeting, there are some options but not sure they will work well:

 

- You have an experiment A on page page1.html that has been running for say a month

- You want new visitors on a different page page2.html where a second experiment B runs to enter only if they haven't been on page1 and seen experiment A,

 

There is a way to do it: you can make another custom javascript variable that searches to the experiment ID (you can find the ID in the experiment details page) of A in the experiment cookie, i.e:

 

hasNotSeenA:

function() {

  return document.cookie.indexOf('<experiment A ID>') == -1;

}

 

Then you can add a rule "hasNotSeenA equals true".

 

The problem with this approach though, is that there is nothing stopping visitors from entering into experiment A later when later then visit page1.html (after they have been included in B).

 

You could change the traffic allocation of experiment A to 0 (so no new users will get into the experiment) but of course this will stop it from collecting more data - so its probably not what you want.