AdWords is now Google Ads. Our new name reflects the full range of advertising options we offer across Search, Display, YouTube, and more. Learn more

Ads
2.5K members online now
2.5K members online now
For questions related to Google Shopping and Merchant Center. Learn to optimize your Shopping ads
Guide Me
star_border
Reply

robots.txt-excluded image error

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

Could anyone shed some light on this error that I'm receiving on my uploaded product feed in the merchant center? 

Issue: Images cannot be crawled because of robots.txt restriction

 

I have 35 products uploaded, but this error only applied to 3 of the products. From what I understand, robot.txt is applied from domain level. Is that how it works? If so, why is that the error only happened to 3 of the products, but not all of them? 

2 Expert replyverified_user
1 ACCEPTED SOLUTION

Accepted Solutions
Marked as Best Answer.
Solution
Accepted by James_Clemens (Top Contributor)
January

robots.txt-excluded image error

[ Edited ]
Top Contributor
# 3
Top Contributor

google may only show a random sample of items/issues --
primarily to use as examples, to find a root-cause, and
apply a long-term solution, so that all items are fixed,
not just those items that may have been flagged, so

that the same similar issue never, ever, reoccurs.

the number of actual items effected may be far greater than the sample reported.

 

in this particular case, the robots.txt file

may be placed under any directory within a

website and may exclude all files or some

files, depending on the details.

 

google must be able to crawl all images, landing-pages,

and all publicly accessible web-pages, from any location,

regardless of any user-specific detail, such as browser

user-agent, ip-address, type of device, etc., and return

identical information, at all times.

a best-practice is to review all rules and policies
so that no future issues occur -- too many flagged
issues, issues flagged too frequently, or any single
serious policy flag, may result in a more permanent
suspension from the program, at any time.

 

View solution in original post

robots.txt-excluded image error

Top Contributor
# 2
Top Contributor

Hi there,

 

Robots.txt can be applied to directories or individual pages as well. 

 

You can see yours at yoursite.com/robots.txt

 

Marked as Best Answer.
Solution
Accepted by James_Clemens (Top Contributor)
January

robots.txt-excluded image error

[ Edited ]
Top Contributor
# 3
Top Contributor

google may only show a random sample of items/issues --
primarily to use as examples, to find a root-cause, and
apply a long-term solution, so that all items are fixed,
not just those items that may have been flagged, so

that the same similar issue never, ever, reoccurs.

the number of actual items effected may be far greater than the sample reported.

 

in this particular case, the robots.txt file

may be placed under any directory within a

website and may exclude all files or some

files, depending on the details.

 

google must be able to crawl all images, landing-pages,

and all publicly accessible web-pages, from any location,

regardless of any user-specific detail, such as browser

user-agent, ip-address, type of device, etc., and return

identical information, at all times.

a best-practice is to review all rules and policies
so that no future issues occur -- too many flagged
issues, issues flagged too frequently, or any single
serious policy flag, may result in a more permanent
suspension from the program, at any time.