AdWords is now Google Ads. Our new name reflects the full range of advertising options we offer across Search, Display, YouTube, and more. Learn more

Ads
3.7K members online now
3.7K members online now
For questions related to Google Shopping and Merchant Center. Learn to optimize your Shopping ads
Guide Me
star_border
Reply

Leakage or dodgy anonymous scraping by Google

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

We maintain 1000s of records in Google Merchant Centre so that the products we sell on our ecommerce store can be advertised as Shopping Ads in Adwords.

 

For weeks now, we have had our website scraped by anonymous visitor(s) landing on our product landing pages. No other pages are visited, only the urls provided to Merchant Centre are being hit, thousands upon thousands each day.These visits are not coming from Googlebots of any kind, the user agents and referer strings are clearly being faked or blanked. Also, the IP addresses are not in Google's ranges but are either Amazon EC2 or TOR IPs. 

 

As a test we added some impossible-to-guess coding to the URLs sent to Merchant Centre and within hours these visits started to include that coding. 

 

Either Google are doing this scraping themselves (above and beyond their normal bot activity) or are selling this Merchant Centre data to 3rd parties who are then scraping using the URLs.

 

We are blocking these visits, but they don't stop! (still in the logs). Is this something to worry about or should this be reported somehow (if so, how?). As these are not Googlebot visits, presumably it is safe to continue to block them?

 

Many thanks.

 

1 Expert replyverified_user

Re: Leakage or dodgy anonymous scraping by Google

[ Edited ]
Top Contributor
# 2
Top Contributor


for merchant-center related crawl issues google may be contacted directly --
https://support.google.com/merchants/contactflow

also, the webmaster-forums may be consulted for crawling related issues --
https://productforums.google.com/forum/#!forum/webmasters

 

see also

https://support.google.com/webmasters/answer/1061943

 

Re: Leakage or dodgy anonymous scraping by Google

Visitor ✭ ✭ ✭
# 3
Visitor ✭ ✭ ✭
Thanks.