5.9K members online now
5.9K members online now
Learn how to set up goals and Ecommerce tracking, verify accuracy of attribution traffic sources, and discuss funnel visualization, Multi-Channel Funnels, the Goal Flow Report, and Enhanced Ecommerce
 
Guide Me
star_border
Reply

Spike in activity from Ashburn, VA? Webcrawler? How to stop it?

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

We've been hit with an unusually large spike today and something is crawling our website with 100+ visitors at the same time from Ashburn, VA.  Can someone tell me what/who this is and how do I block them?  Thanks - L.

2 Expert replyverified_user

Re: Spike in activity from Ashburn, VA? Webcrawler? How to stop it?

Top Contributor
# 2
Top Contributor
Hi Nicole,
This is a frequent occurrence. I'd recommend first making sure that it's not legitimate traffic. Don't mean to sounds condescending, that's just an important first step.

Next, you need to either filter this out using something unique about the "traffic" using a filter or ultimately, you'll have to set up a custom "clean traffic" custom segment. As your site grows, you'll need to become adept at learning to filter out all this crap as Google doesn't do much to prevent it.

In particular, I'd pay attention to your events, goals and conversions where it's possible to enter some custom string. That's where the spam occurs.

I'll assume that you have the "block bots and crawlers" option ticked and you've got a robots.txt directive and this is a bad actor in that respect.

One of our amazing TCs here on this very forum created the best guide I know to combating this and it can be read here:
http://help.analyticsedge.com/spam-filter/definitive-guide-to-removing-google-analytics-spam/

Yes, it's a MASSIVE pain and yes, this should NOT be the way GA works but we're stuck with it. Print out that article and read it over and over again, you'll be referring to it frequently and I promise you, this is going to get worse. Best become adept at cleaning out your unwanted traffic through filters now so that when the really big ones hit, you'll know what to do, and do it fast.

Once you get the hang of this, the biggest challenge is actually on the client end. Getting them to use segments by default is surprisingly challenging. Many clients and site owners like to think "Hey! What did you do? You broke our analytics! We were receiving a million 'hits' per day and now we're only getting 3000. You're awful!". Convincing them that what they believed to date was a lie is surprisingly difficult.

So, integrating proactivity into your daily routine will pay dividends and you'll eventually get a flow and a custom filter list. Welcome to intermediate Google Analytics. See this as a good thing. You're in the trenches now and it's going to get a lot worse. If you're lucky.

Understand the problem, communicate it to the client (if indeed this is for a client) and get into a habit of proactively blocking it (the intelligence reports are a super valuable asset here so you can catch them before they destroy your data) and you'll become a lot more valuable as a web analyst.

Finally, yes, this is something Google should be doing more about but we've been shouting for years and it doesn't seem to be high on the priority list.
There's MANY useful threads on this very community that will give you tips, tricks and ways to keep ahead of these absolute guttersnipes ruining our data. One of our team is actually thinking of writing a book on the topic because it has taken Google so long to take action here.

Sorry there's nothing more concret that I can help you with right now but if you'd like to share some more detail, some troubleshooting data (after reading the post I linked above), I'll be happy to work with you on designing your filters and segments so that you get the most value from them , Just remember, communicating this problem to stakeholders is JUST AS VITAL as actually stopping it. Trust me on that.

Sorry I can't fix the problem in one reply and sorry it's not the reply you wanted/thought you needed but it's the best i've got right now.
Again, feel free to share some details and we can work through some pretty robust filters and segments. Just remember, segments are useless if nobody uses them and is dead set on believing that the default segment is an accurate view of the state of the site,

Hope that helps.

Dave

Re: Spike in activity from Ashburn, VA? Webcrawler? How to stop it?

Visitor ✭ ✭ ✭
# 3
Visitor ✭ ✭ ✭
Dave, many thanks for the thoughtful reply. I'm going to noodle on this. Smiley Happy

Re: Spike in activity from Ashburn, VA? Webcrawler? How to stop it?

Top Contributor
# 4
Top Contributor
Just as a follow up to this post -- I have clients with this problems as well. Ashburn is a big data hub in North America, and there are many possible causes for traffic to come from there. Here are some recent discoveries:

- if you use HubSpot, check for browser's labelled 'HubSpot ContentPagesUI' or similar. Seems when you edit your pages in their interface, they register a GA session from their server.
- if you use Call Rails call tracking service, check to see if the sessions have a lot of 'Phone Calls' events. They log non-website calls from their server (this can also create sessions with no pageviews, which does some bizarre things to averages).
- check the browser versions, specifically old Chrome versions like 27.0.1453.116, and old safari versions like 7.0.3. There seems to be a bot running on Windows XP and Mac 10.9 logging a lot of direct and social visits. It might be one of those social booster services pumping up your stats with fake clicks.
Mike Sullivan, Google Analytics Top Contributor
Was my response helpful? If yes, please mark it as the ‘Best Answer.’ Learn how here.

Spike in activity from Ashburn, VA? Webcrawler? How to stop it?

Follower ✭ ☆ ☆
# 5

Spike in activity from Ashburn, VA? Webcrawler? How to stop it?

Visitor ✭ ✭ ✭
# 6
Visitor ✭ ✭ ✭

This happened to us when we activated Pingdom (web monitoring service) Make sure no one in your company added a web monitoring service first. In Google Analytics these visits were from Ashburn (service provider was Amazon) and time on site was a fraction of a second.

Spike in activity from Ashburn, VA? Webcrawler? How to stop it?

Visitor ✭ ✭ ✭
# 7
Visitor ✭ ✭ ✭

Personally, I wouldn't care about the large spike in traffic, regardless of where it originates. The problem that we're seeing is that the traffic from this area is taking an order of magnitude longer to complete the pageload. For example, a page taking 5 seconds elsewhere is taking 100 seconds coming from the Ashburn, VA. This is playing havoc with our analytic services.

Spike in activity from Ashburn, VA? Webcrawler? How to stop it?

Visitor ✭ ✭ ✭
# 8
Visitor ✭ ✭ ✭

Had the same thing happen to me just now. 40+

Spike in activity from Ashburn, VA? Webcrawler? How to stop it?

N/A
# 9
N/A

There appears to be an IP Address: 54.83.82.185 coming from Ashburn, VA where someone is attempting to compromise Google Accounts.

Experts in the Topic
Dave_Davis
Mike S