Prevent Google from Crawling Certain Products
I have an e-commerce store with many, many items. Some of them are being denied by Google returning issues. Obviously I can fix the problems. However, in the meantime I want Google to stop parsing the products it finds. Is there anyway I can stop Google from trying to look at the product and find issues with it?
I know there's the robots.txt file, but then that'll stop the whole parsing altogether. I'm looking for a way to throw if-statements in between the products.
Re: Prevent Google from Crawling Certain Products[ Edited ]
November 2015 - last edited November 2015
currently, about the only method is to prevent
items from being submitted in the first place --
for example, if a feed is used, deleting all such
items from the feed and resubmitting the feed.
google may take 72-hours or so to fully delete an item.
otherwise, if google cannot crawl a link landing-page page --
such as being restricted via robots.txt, .htaccess, or similar --
the item can be flagged; if google cannot match all submitted
data to the displayed site data then, the item can be flagged;
if the automatic-item-updates feature is enabled in the account
then, the microdata on the site must match the submitted and
displayed data or, the item can be flagged.
google may also crawl the entire site for any potential policy related issues --
e.g. secure checkout-flow, proper contact information, spam techniques, etc.
generally, googlebot, adsbot-google, and googlebot-image,
must be able to crawl all images and the entire site that
can be seen by users.
note that if items are removed, be certain their id
values are never reused for any other item and that
the exact same id is used for the exact same item
when that item is resubmitted.
otherwise, a merchant-center product-listing-ads
support-specialist may be contacted directly.