URL function encoding help would be appreciated
#1
I am working on an HD-Trailers scraper, but cannot get it to work due to an URL encoding issue.

Currently I am calling the scraper function with this line:

Code:
<url function="GetHDTrailer">http://www.hd-trailers.net/blog/?s=\2 (Theatrical Trailer)</url>

...and I am getting the following error in the debug log:

Code:
20:11:23 T:2800 M:1062498304   DEBUG: scraper: GetDetails returned <?xml version="1.0" encoding="iso-8859-2" standalone="yes"?><details><title>Terminátor Salvation</title><year>2009</year><director>Joseph McGinty  Nichol (McG)</director><url function="GetHDTrailer">http://www.hd-trailers.net/blog/?s=Terminator Salvation (Theatrical Trailer)</url></details>
20:11:23 T:2800 M:1062494208   DEBUG: FileCurl::Open(0012D838) http://www.hd-trailers.net/blog/?s=Terminator Salvation (Theatrical Trailer)
20:11:23 T:2800 M:1062494208   DEBUG: XFILE::CFileCurl::CReadState::FillBuffer: curl failed with code 22
20:11:23 T:2800 M:1062494208   ERROR: CFileCurl::CReadState::Open, didn't get any data from stream.
20:11:23 T:2800 M:1062494208   DEBUG: FileCurl::Close(0012D838) http://www.hd-trailers.net/blog/?s=Terminator Salvation (Theatrical Trailer)

The URL which CURL tries to open is correct. You can verify if you copy it to a browser.

It seems, that it has troubles with the "?" (question mark in the URL), because it seems the URL could be opened withoutit, but unfortunatelly it is needed for the search.

I've already tried to encode the URL like:

Code:
<url function="GetHDTrailer">http://www.hd-trailers.net/blog/?s=\2%20(Theatrical%20Trailer)</url>
<url function="GetHDTrailer">http://www.hd-trailers.net/blog/?s=\2%20%28Theatrical%20Trailer%29
<url function="GetHDTrailer">http://www.hd-trailers.net/blog/%3Fs%3D%5C2%20(Theatrical%20Trailer)</url>
<url function="GetHDTrailer">http://www.hd-trailers.net/blog/%3Fs%3D%5C2%20%28Theatrical%20Trailer%29</url>

and some other various way, but without success.
The interesting piece is that it is working perfectly in XML Scraper editor/tester. I know it's different, but still makes me more confuse... Smile

Could you please point me into the right direction about that?

Thank you in advance.
Reply

Logout Mark Read Team Forum Stats Members Help
URL function encoding help would be appreciated0