AdWords is now Google Ads. Our new name reflects the full range of advertising options we offer across Search, Display, YouTube, and more. Learn more

Analytics
2.7K members online now
2.7K members online now
Dive into multiple domain (Cross/Sub) tracking, implementing Ecommerce and Enhanced Ecommerce, setting up Event tracking, and Universal Analytics code.
 
Guide Me
star_border
Reply

How to add analytics for A/B testing, to be apart from other Analytics categories?

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

 Hello all,

I'm new here. We developed a very large Android app that already has Analytics.
Now we wish to add A/B testing, and also add analytics to it.
As I've found, it is mentioned that Google has a solution that combines both A/B testing and Analytics together, for apps:
"The Google Analytics Content Experiments Framework enables you to test almost any change or variation to a website or app"
(source: https://developers.google.com/analytics/solutions/experiments-feature-reference )

However, I can't find any sample or even code to show how to use it.

How can I find more information about it?

In addition, since we've found about A/B testing of Google only recently, we already developed our own solution for A/B testing (including a server to handle it), and only need the Analytics part. Using Analytics, how can I track experiments, their variant, and actions that were done? I can't see this kind of hierarchy of being able to use "experiments" -> "experiment A" -> "variant 1" -> "user clicked on button B".

I want to have the "experiments" level to differentiate it from already used categories from Analytics

The only hierarchy I see is of 3 levels, not 4:

Tracker tracker=AnalyticsTracker.getInstance(context).getTracker();
tracker.send(new HitBuilders.EventBuilder().setCategory(category).setAction(action).setLabel(label).setValue(value).build());



Can anyone help?

Re: How to add analytics for A/B testing, to be apart from other Analytics categories?

Explorer ✭ ✭ ☆
# 2
Explorer ✭ ✭ ☆
Hi,

You won't find the experiment code anywhere, because this is something generated dynamically while configuring the experiment through GA console.

Sam

Re: How to add analytics for A/B testing, to be apart from other Analytics categories?

Visitor ✭ ✭ ✭
# 3
Visitor ✭ ✭ ✭
But it needs to be reported somehow via code, no?
If I set an experiment with 2 variants, shouldn't the app know which experiment and which variant it has, to change itself (example: variant 1 changes toolbar color to blue, and variant 2 changes it to red) , and then report about specific actions that belong to this specific experiment&variant (example of an action: "user clicked on toolbar button") ?

How does the integration of A/B testing together with Analytics work with apps ? Does it get into the normal screen of Analytics (with the events), or does it have its own screen?

Also, is there a way for Analytics to have an addition layer for the A/B testing ? One for experiments only ?

Re: How to add analytics for A/B testing, to be apart from other Analytics categories?

Explorer ✭ ✭ ☆
# 4
Explorer ✭ ✭ ☆
Above mentioned points you need to mention while configuring the experiment, along with that you need to specify what % of traffic should split across the variants.

Also while reporting, you want to combine both(or all) the screen into one or you want to report as it is.

Experiment is independent of analytics code, you can can run any number of experiments after adding the GA tracking code. Each experiment will have it's own experiment code which is associated with GA property view at the console and this association will be used at the time of reporting, and you no need to worry about the integration while implementing.

At the moment only Experiments is available.

Sam


Re: How to add analytics for A/B testing, to be apart from other Analytics categories?

[ Edited ]
Visitor ✭ ✭ ✭
# 5
Visitor ✭ ✭ ✭

Not sure I understand.
Code should also have some reference to the experiments.
Suppose I have the example I wrote:
Experiment name : "ToolbarColorExperiment" , experiment id: 1 .
Variant: version 1 : half with toolbar of color blue, and rest with red
available events: user clicked on FAB, user clicked on toolbar button

So, in code, there should be a call when the user causes the events I wrote about, so that the data is sent to the server. Something like:

experimentTracker.trackExperimentEvent( /*experimentId : */ 1, /*variantVersion : */ 1, "clickOnFab") ;

 

There should also be code for the changing of the toolbar color, base on the experiment's variant. 

 
How is this working in Google's A/B testing service?

Is there maybe a step-by-step tutorial/manual for A/B testing using what Google provides, for Android apps?

 

EDIT: I've found this:
https://www.youtube.com/watch?v=EVUmo4sSzgg

It looks nice. Hoping to find all answers to my questions, including how to create the variants and how to spread them.

 

EDIT: watched the video, and tried this website:
https://www.google.co.il/analytics/tag-manager/

I don't see how to create experiments, containers, variants. It looks very different than the website of the video.

 

Re: How to add analytics for A/B testing, to be apart from other Analytics categories?

Explorer ✭ ✭ ☆
# 6
Explorer ✭ ✭ ☆
Not sure of any such step by step video that will helps you....

Had done experiments on the web site and not for APP, But the above Youtube video looks good to go ahead.

Sam

Re: How to add analytics for A/B testing, to be apart from other Analytics categories?

[ Edited ]
Visitor ✭ ✭ ✭
# 7
Visitor ✭ ✭ ✭

OK, I still have a question about A/B testing experiments, whether it's for web or apps:
Can I control how many of the users will have each varient ? Say, 40% for varient 1 and 60% for varient 2 ?
Can I choose how it works in various countries, for example?

For example, in the US a specific in-app billing purchase might be X dollars, and for the others, have it Y dollars. The purpose would be to know which of them is paying more.


Or is it all completely automatic?

Re: How to add analytics for A/B testing, to be apart from other Analytics categories?

Explorer ✭ ✭ ☆
# 8
Explorer ✭ ✭ ☆
Hi,

Partrial Yes for first one but NO for 2nd one.

For 1st one you are able to specify what % of traffic to distribute among the variants, For example you mentioned 30%, then this 30% wil be distributed among the variants equally and the remaining will be to the Original page.

2nd one is not feasible at the moment.

Sam

Re: How to add analytics for A/B testing, to be apart from other Analytics categories?

[ Edited ]
Visitor ✭ ✭ ✭
# 9
Visitor ✭ ✭ ✭

1. I actually mentioned 40 and 60 . I don't understand though what you mean.

2. How come?
Maybe I can somehow get all variants of the experiment into the app, and then choose which variant should be selected for it, and report it back to Google?

Do you work at Google? Can you please tell them to make it more flexible? I also find it quite hard to get started with it (so many steps are needed in the manual ).


3. Is this possible : Variant 1 for users who used the app for over a year, variant 2 for the rest ?

4. We use Analytics in IOS too, and not just Android (though they are separated there). Is it possible to combine them both, and get combined results, handle exact same experiments on both, having variant 1 for Android and variant 2 on IOS ?

5. I've noticed that there is a "custom variable", but when I choose it, I can't customize anything. How come?

Re: How to add analytics for A/B testing, to be apart from other Analytics categories?

Explorer ✭ ✭ ☆
# 10
Explorer ✭ ✭ ☆
Hi,

Experiments in GA is not so customisable/flexible like Test & Target in Adobe Suite. Only available option here is Portion of the Traffic you want to split across the Variants for a given page.

Expecting some improvements from Google in near future and Hope GA is listening to us and our struggle, by the way I don't work at Google.

Sam