AdWords
6.2K members online now
6.2K members online now
For questions related to Google Shopping and Merchant Center. Learn to optimize your Shopping ads
Guide Me
star_border
Reply

Issue with feed uploads and https

Visitor ✭ ✭ ✭
# 1
Visitor ✭ ✭ ✭

I thought I would post this as we believe we have found a potential bug/limitation in Googles feed fetching engine when working over SSL which may save others a lot of wasted time.

 

We have just spent 3 hours trying to solve an issue with a feed continually returning the following error:

"Failed to obtain the file information. Please make sure the file exists AND that your feed has permission to access it"

 

In the end we worked out that if we submit the feed using http it works perfectly but if we submit the exact same feed on exactly the same URL using https it fails with the above message. Looking at the Apache logs on the server we don't even see a request from Google for the file when it fails.

 

Testing shows the link works fine using both http and https in Chrome, Firefox and IE and also with command line utilities such as wget and curl. And obviously we know the feed content is OK as it uploads and populates our products when using http or the file upload method. Just to make sure I even downloaded the file over both http and https and checksummed them.

 

The requests go direct to Apache, we are not using SNI and we have A rated SSL:

https://www.ssllabs.com/ssltest/analyze.html?d=www.newtontrailers.com

 

A more verbose error message would be useful to diagnose this. The generic error message we received gives no indication that this is even an SSL issue which had us stumped for quite some time.

1 Expert replyverified_user

Re: Issue with feed uploads and https

[ Edited ]
Top Contributor
# 2
Top Contributor

feedback about the issue may be sent directly to google:
https://support.google.com/merchants/answer/2533365
https://support.google.com/adwords/contact/aw_complaint

including the exact (feed file) url and the exact method and
steps used to submit the file would likely help; simply having
a site's url is typically not enough information for such cases.

note that being able to access the feed file within a browser or
via the command-line does not guarantee that google's agents
will be able to access the file with respect to the protocol or the
file's location on the site, for example during a scheduled-fetch.

as an aside, our afternoon feed fetch, using https, succeeded without error.

a merchant-center support-specialist at google may also be
contacted directly for issues related to a specific account --
https://support.google.com/merchants/contactflow
https://support.google.com/merchants/answer/4581834

Re: Issue with feed uploads and https

[ Edited ]
Visitor ✭ ✭ ✭
# 3
Visitor ✭ ✭ ✭

You stated that...
"being able to access the feed file within a browser or via the command-line does not guarantee that google's agents will be able to access the file with respect to the protocol or the file's location on the site"

I agree, that's why we tested the following:
Access from multiple physical locations using different internet connections, even from different countries.
Multipl clients including both CLI and browsers.

We have also checked the following for restrictions and there are none:
robots.txt
Firewall rules
Application code

The following information is also relevant:
Google currently indexes the site in https so its crawlers have access
The feed works over http and manual upload so the feed is good
Because it works over http we know this isn't a DNS or routing issue

I will contact Google and see what they have to say as this is indeed an intriguing problem. My current "best guess" is they have some issue coping with the SSL configuration which is why a useful error message would be nice Smiley Happy

Re: Issue with feed uploads and https

Top Contributor
# 4
Top Contributor
note that specific referrers, user-agents,
and multiple methods are also involved.

google typically sends rather general error messages,
given all the possible outliers -- sometimes pointing to
somewhat more useful information in the documentation.

posting the exact url here in the public forums may allow for more
specific suggestions; otherwise, the issue could be as simple as
a typo or something more involved with the tls/ssl configuration --
contacting google directly is the best likely course.

Re: Issue with feed uploads and https

[ Edited ]
Visitor ✭ ✭ ✭
# 5
Visitor ✭ ✭ ✭

I'd rather not post the actual feeds as they are rather large but I've set up one of Google's example feeds which suffers from the exact same scheduled fetching issue here...

http://www.newtontrailers.com/feeds/test/test.xml
(This works and we get item errors which is as you would expect because its just a dummy feed)

https://www.newtontrailers.com/feeds/test/test.xml
(Fails with "Failed to obtain the file information. Please make sure the file exists AND that your feed has permission to access it")

 

You can reproduce this by adding a feed using the "+ FEED" button under the "Feeds" section of https://merchants.google.com/ then clicking "Fetch Now"

The options used are mostly irrelevant as it doesn't actually fetch the file.

Re: Issue with feed uploads and https

Top Contributor
# 6
Top Contributor
the certificate looks to be domain-control-validated, together with the
data-transfer mechanism, which looks to be chunked rather than using
content-length -- are at least two differences from working https-feeds.

waiting for google to respond is likely best.

Re: Issue with feed uploads and https

Visitor ✭ ✭ ✭
# 7
Visitor ✭ ✭ ✭
Yes but neither of those are reasons it shouldn't work.

Re: Issue with feed uploads and https

[ Edited ]
Top Contributor
# 8
Top Contributor

if most any file on the site is accessed using
http or a non-www version, the server returns
a 301-moved to https/www.

however, the test-feed returns a 301-moved
for the non-www version but 200-ok for both
http and https for the www-version.

check if anything was changed to accommodate that

specific behavior and may have impacted this issue.

also, anything related from the server's logs may also help.

 

Re: Issue with feed uploads and https

[ Edited ]
Visitor ✭ ✭ ✭
# 9
Visitor ✭ ✭ ✭

Yes this is because we had to disable the https redirect for the feed specifically because Google were unable to fetch it over https.

Unfortunately there's nothing in the logs our side. Its as if Google never actually makes a connection, this is why I believe it's an SSL issue their side.