AdWords is now Google Ads. Our new name reflects the full range of advertising options we offer across Search, Display, YouTube, and more. Learn more

Analytics
2.8K members online now
2.8K members online now
Dive into multiple domain (Cross/Sub) tracking, implementing Ecommerce and Enhanced Ecommerce, setting up Event tracking, and Universal Analytics code.
 
Guide Me
star_border
Reply

Get all fullReferrers for the whole timeframe

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Hello,

we have a big client who connected his analytics profile to our project. we want to receive the following data:

 

1. A list of all Referrers including the amount of sessions from the last 30 days => This is not a problem and is working

2. A list of all Referrers including the amount of sessions from the beginning (alltime) => The request is working, but the data which are returned are strange. For example there are Refferers which are included inside the last 30 days, but missing inside the resultset of alltime. This is not possible Smiley Happy. In know the timeframe is pretty long ... but it does not explain why data are totally missing.

 

This is the API request we make:

 

https://www.googleapis.com/analytics/v3/data/ga?ids=xxx&start-date=2005-01-01&end-date=today&metrics...

 

Can anybody help us?

 

Thank you,

Bastian

1 Expert replyverified_user
1 ACCEPTED SOLUTION

Accepted Solutions
Marked as Best Answer.
Solution
Accepted by topic author Bastian B
July 2016

Re: Get all fullReferrers for the whole timeframe

Rising Star
# 6
Rising Star
Hi Bastian,

You really have no choice, you'll have to break it down into chunks that have less than 500k sessions -- weeks or months or years, whichever works - then stitch the data together. If you need to do this on a regular basis, consider warehousing the historical data and just adding in the new weeks or days; then use this as your data source (if that's practical and suits your needs/requirements).

Best,

Theo Bennett

Analytics Evangelist at MoreVisibility | Contact Me
Connect on LinkedIn

View solution in original post

Re: Get all fullReferrers for the whole timeframe

Rising Star
# 2
Rising Star
Hey Bastian,

You are probably getting sampled data with that long of a request. It's better to break down the request into smaller chunks and then put them back together. Avoid asking for averages etc. like bounce rate (use bounces)

Best,

Theo Bennett
Analytics Evangelist at MoreVisibility | Contact Me
Connect on LinkedIn

Re: Get all fullReferrers for the whole timeframe

Visitor ✭ ✭ ✭
# 3
Visitor ✭ ✭ ✭
Hi Theo, Thank you. How would you do this? The Timeframe is pretty long, when starting from 2005-01-01. When i break down for each year, will that help? I dont want to use that much quota credits, what i would do when i make a request per calender week.

Re: Get all fullReferrers for the whole timeframe

Rising Star
# 4
Rising Star
Hi Bastian,

First confirm that the metrics and dimensions set that you are requesting is returning sampled data. If it is then you'll need to chunk it out into sessions of less than 500k.

The time range really depends on how much data is in your account.

https://developers.google.com/analytics/devguides/reporting/core/v3/reference#sampling

Hope that helps.

Best,

Theo Bennett
Analytics Evangelist at MoreVisibility | Contact Me
Connect on LinkedIn

Re: Get all fullReferrers for the whole timeframe

Visitor ✭ ✭ ✭
# 5
Visitor ✭ ✭ ✭
Hi Theo,

This are some insights about sampling:

getSampleSize() => string(6) "689833"
getSampleSpace() => string(8) "30007290"
getContainsSampledData() => bool(true)

What we already do is using pagination. We only get 5000 results per page. The dimension i cannot reduce, because we need the given dimensions to get the data. The only thing we could optimize is the datarange ... but when i split the datarange into weeks for example ... we will do a lot of APi requests which will take a while but the single request should be faster as the while daterange.

what would you recommend?

thank you very much!
Marked as Best Answer.
Solution
Accepted by topic author Bastian B
July 2016

Re: Get all fullReferrers for the whole timeframe

Rising Star
# 6
Rising Star
Hi Bastian,

You really have no choice, you'll have to break it down into chunks that have less than 500k sessions -- weeks or months or years, whichever works - then stitch the data together. If you need to do this on a regular basis, consider warehousing the historical data and just adding in the new weeks or days; then use this as your data source (if that's practical and suits your needs/requirements).

Best,

Theo Bennett

Analytics Evangelist at MoreVisibility | Contact Me
Connect on LinkedIn

Re: Get all fullReferrers for the whole timeframe

Visitor ✭ ✭ ✭
# 7
Visitor ✭ ✭ ✭
Yes i know ... thank you very much Smiley Happy

Re: Get all fullReferrers for the whole timeframe

Visitor ✭ ✭ ✭
# 8
Visitor ✭ ✭ ✭
I also checked if i can pass the URLs inside the filters where i want to have the session count. but this query is also not good, because they go through each row and use the filters ....

the problem is that the website has a massive amount of referals .... even when i set ga:medium == referral and i only select 6 months ... i get over 500k different referral urls.

when i want to do this now for alltime (starting from 2005) it is endless Smiley Sad.

maybe it helps you when i tell you what i need:

i have 10000 referral urls and i want to have the sessions.

how would you do this?

Re: Get all fullReferrers for the whole timeframe

Rising Star
# 9
Rising Star
Hi Bastian,

This should be pretty straight forward. I would use the dimension of source (along with date so that you can put it back together), segment id for referral (gaid:-8); metric of sessions.

As for the quantity... well if there is that much data the other option is to go to GA Premium/360 and request an un-sampled report. Smiley Happy

Best,

Theo Bennett
Analytics Evangelist at MoreVisibility | Contact Me
Connect on LinkedIn

Re: Get all fullReferrers for the whole timeframe

Visitor ✭ ✭ ✭
# 10
Visitor ✭ ✭ ✭
Hi Theo,

So in the end you would use this:

metrics:ga-sessions
dimensions:ga:fullReferrer
segment:gaid::-8

and then split the api calls regarding the timeframe:

1. Call: startdate: 2005-01-01 | enddate: 2005-05-31
2. Call: startdate: 2005-06-01 | enddate: 2005-12-31

and merging the result arrays together.

am i right?