I launched 3 A/B tests with a free Optimize account (2 at the same time and still running)
- Test 1 from 15/12/2016 to 02/01/2017 with only 1 variant tested on 100% of our trafic (50/50 with the original), on desktop. All was good, we reached almost the same experiment sessions as our daily session on desktop. After that, we decided to launch 2 other tests at the same time.
- Test 2 from 18/01/2017 to (still running) with 4 variants tested on 100% of our trafic (20/20/20/20/20) on desktop.
- Test 3 from 18/01/2017 to (still running) with 4 variants tested on 100% of our trafic (20/20/20/20/20) on desktop.
On the two last tests (Test 2 & 3) I witnessed a huge drop of experiment sessions after the 1st day. And the experiment sessions don't look like our daily session on desktop.
What can explain this drop?
Thanks for your help!