Analytics
5.2K members online now
Understand information in your reports and troubleshoot reporting issues such as self-referrals, (not set) data, and inaccurate information
 
Guide Me
star_border
Reply

Disallow utm parameter urls in robots.txt

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Hi all,

 

We currently use utm parameter for traffic tracking.

For SEO purpose I would like to disallow googlebots crawling urls like /product/item-a?utm_source=? in robots.txt

I assume that it won't affect utm tracking in google analytics at all but I just wanted to be definitely sure before going on Smiley Happy

Thank you !

E.

 

 

2 Expert replyverified_user

Re: Disallow utm parameter urls in robots.txt

Rising Star
# 2
Rising Star
Hey Emmanuel, how are things?

Since we're talking about UTMs, I'll move your post to the Analytics section, ok? Any doubts let me know.

Now, one thing caught my attention: You should NOT in any case use UTM parameters to track internal data of your website, if that's what you're doing.

Those parameters are used to track links OUTSIDE of your page, like a link on facebook, twitter and such. Those parameters are for you to know where the user came from, where they clicked to get to your site (a post, an image, a tweet...) and this kind of stuff.

If you use UTMs for internal data, you'll replace the original information from the user and you'll never get to know where the user came from!

For internal data, you should map your website at least with Analytics Events: https://developers.google.com/analytics/devguides/collection/analyticsjs/events

Hope this helps.
_


Leandro Martinez | Basta1Click

Re: Disallow utm parameter urls in robots.txt

Visitor ✭ ✭ ✭
# 3
Visitor ✭ ✭ ✭
Hello Leandro

Thank you for your reply!

We do not use UTM for internal data, we use UTM only for traffic coming from outside as you said, no worry Smiley Happy

As I analyzed googlebots logs on our website I noticed a lot of UTM tracked urls (from google shopping for example) and I would like to disallow all these urls to be crawled by googlebot.

Why? Because the same url with an without UTM becomes two different urls, one onsite and another which is not onsite. It's very annoying as I do analyze googlebots logs on a regular basis.

So the best solution for me is to disallow UTM urls inside robots.txt. I think that probably it won't affect analytics tracking at all, but I would like that someone else tells me "go on, it's safe"

Smiley Happy


Re: Disallow utm parameter urls in robots.txt

Rising Star
# 4
Rising Star
Emmanuel if these UTM parameters are not on your site how is Google Bot crawling them. If they are showing up from other sites referencing your site then the robots.txt file will do you no good since only bots crawling your site will view the robots.txt file. It is also important to note that not all bots honor the robots.txt file. If you do not want the UTM parameters to be seen in GA you could add them to the exclusion list but since I have not tried that approach I dont know if your campaigns would still work. If you do want to try the robots.txt file I assume an entry such as Disallow: *utm_* might work. Either way I dont see how the robots.txt file would have an effect on GA only on what would show up in search engines.

Re: Disallow utm parameter urls in robots.txt

Visitor ✭ ✭ ✭
# 5
Visitor ✭ ✭ ✭
Thank you Brian for your answer.

"If they are showing up from other sites referencing your site then the robots.txt file will do you no good since only bots crawling your site will view the robots.txt file." => yes, you're right...

Emmanuel