Analytics
2K members online now
2K members online now
Learn how to set up goals and Ecommerce tracking, verify accuracy of attribution traffic sources, and discuss funnel visualization, Multi-Channel Funnels, the Goal Flow Report, and Enhanced Ecommerce
 
Guide Me
star_border
Reply

Using Custom Dimension for onsite AB tests - attribution problem

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Hi GA community,

 

I'm hoping somebody out there can help with a custom dimension (CD) attribution challenge.

 

I'm attempting to use custom dimensions to report on A/B test experiments from a 3rd party testing tool. The experiments that are occurring on my client's website. This 3rd party tool passes two pieces of data to two custom dimensions working together to create the reports. The first CD captures the test name and the second CD captures the experience within each test. Using this we can then understand how many visitors got into each test and the experience they saw. We can also attribute metrics and goals to the appropriate test and experience. This seems to be working fine when a visitor only comes across a single experiment.

 

Both CDs are set to User scope.

 

------------------------------

Here's my challenge

------------------------------

I'm not sure how this kind of reporting works when one visitors sees more than one experiment in a single session. For example say I have three different experiments set up on a series of pages that are part of the purchase funnel:

  • Experiment #1: Home page test
  • Experiment #2: Product page test
  • Experiment #3: Cart page test

 

In this scenario, it appears that only the CD with the Cart Test value is getting credit for all the Goals that were generated. Probably because it's the last one set and the CD scope is set to User.

 

Is there a way to make it so that all CDs that were seen by person get credit when that person reaches our goal?

 

Given the the example I provided, I'd like to see something like the following: 

2016-02-17_15-04-03.jpg

 

Instead, i'm just seeing credit being given to the last CD:

2016-02-17_15-05-46.jpg

 

Thanks!

Re: Using Custom Dimension for onsite AB tests - attribution problem

Visitor ✭ ✭ ✭
# 2
Visitor ✭ ✭ ✭

Hi Scott,

 

First, a clarification

Would you mind clarifying on the reasons for this?  I'm attempting to use custom dimensions to report on A/B test experiments from a 3rd party testing tool.

 

A simpler solution would probably be to use GA's own content experiment framework for both the multivariate testing and the reporting, or the 3rd party for both. e.g. GA's content experiments https://support.google.com/tagmanager/answer/6003007

Those give you experiment test results like this for multivariate testing - https://support.google.com/analytics/answer/2365329

 

 

If your third party is something like Optimizely for example, they are also capable enough to run the tests and report on them in a very sophisticated manner in their own app and can also send to GA.

 

Is there a specific reason you're trying to do it this way?   It may be simpler to keep it all in one system.

 

Second, a word of caution

I would suggest testing only one page at a time so that you can be sure of which changes actually caused any lift/drop in conversion rate. Running multiple test at the same time is not the best way to go, unless you can segment traffic so there is no overlaps

 

Third, something maybe helpful

If you could you paste in your code snippets that would confirm, but GA's own documentation does state this limitation

https://support.google.com/analytics/answer/2709828?hl=en

 

User-level scope

Lastly, if two user-scoped custom dimension values are set within the same session, the last value set gets precedence for the current session, and is applied to future sessions for that user.

 

 

 

Make sure that there aren’t multiple tests that use the same Custom Dimension (or Custom Variable for Classic) slot in GA – they will overwrite each others data. Use one test per slot and you should be good to go.

 

Hope this helps.

Re: Using Custom Dimension for onsite AB tests - attribution problem

Visitor ✭ ✭ ✭
# 3
Visitor ✭ ✭ ✭
Hi Matthew,

Thanks for all the details. All great points you make. Unfortunately we cannot use GA's content experiments or even a tool like Optimizely's. The client was required to build their own experimentation solution and it has it's own reporting. However, the reporting pretty simple (only reports which experience increases a KPI) and I'd like to something more robust that allows us to dig into and explore visitor behavior by experience seen. Although you can't dig much into the data, the homegrown solution does actually report how every test affects every business KPI, which is pretty cool and has gotten the organization used to that type of view.

I'm with you on the fact that the tests can influence each other and we should only run one test at a time. Unfortunately it's difficult to govern, especially given the established culture of testing everything. Therefore I was trying to see if this is even possible within GA. It sounds like out of the box it's not, but I'm hoping for a creative solution. Smiley Happy